Document of The World Bank Report No.: 78781 PROJECT PERFORMANCE ASSESSMENT REPORT ROMANIA AVIAN INFLUENZA CONTROL AND HUMAN PANDEMIC PREPAREDNESS AND RESPONSE PROJECT UNDER THE GLOBAL PROGRAM FOR AVIAN INFLUENZA (GPAI) (IBRD-48390) June 24, 2013 IEG Public Sector Evaluation Independent Evaluation Group ii Currency Equivalents (annual averages) Currency Unit = New Romanian Lei (RON) 2006 1 EUR (Euro) = 3.53 Ron 2011 1 EUR = 4.13 Ron Abbreviations and Acronyms EU European Union FAO Food and Agriculture Organization GPAI Global Program on Avian Influenza HPAI Highly Pathogenic Avian Influenza ICR Implementation Completion and Results Report IEG Independent Evaluation Group M&E Monitoring and Evaluation OIE World Organization for Animal Health (Office International des Epizooties) PPAR Project Performance Assessment Report UNICEF United Nations International Children‘s Fund USAID United States Agency for International Development WHO World Health Organization Fiscal Year Government: January 1 – December 31 Director-General, Independent Evaluation : Ms. Caroline Heider Director, IEG Public Sector Evaluation : Mr. Emmanuel Jimenez Manager, IEG Public Sector Evaluation : Ms. Martha Ainsworth (Acting) Task Manager : Mr. Stephen Hutton/Ms. Nalini Kumar iii Contents Principal Ratings ................................................................................................................. v Key Staff Responsible......................................................................................................... v Preface............................................................................................................................... vii Summary .............................................................................................................................. i 1. Country and Project Context ........................................................................................... 1 Country Background ....................................................................................................... 1 The World Bank and Romania ....................................................................................... 2 Disease and Project Context ........................................................................................... 3 2. Objectives, Design, and their Relevance ........................................................................ 7 Objectives ....................................................................................................................... 7 Relevance of the Objectives............................................................................................ 7 Relevance of the Original Objectives ......................................................................... 7 Relevance of the Revised Objectives .......................................................................... 9 Design ............................................................................................................................. 9 Components ................................................................................................................ 9 Instrument and Financing ......................................................................................... 10 Monitoring and Evaluation Design ........................................................................... 11 Implementation Arrangements.................................................................................. 11 Relevance of Design ..................................................................................................... 11 Relevance of Design to the Original Objectives ....................................................... 11 Relevance of Design to the Revised Objectives ....................................................... 12 3. Implementation ............................................................................................................. 12 Implementation Experience ...................................................................................... 13 Safeguards Compliance ............................................................................................ 15 Fiduciary ................................................................................................................... 15 Implementation of Monitoring and Evaluation......................................................... 15 4. Achievement of the Objectives ..................................................................................... 16 Reduce the threat posed to humans by HPAI and other zoonoses (Original objective) 16 Reduce the threat to the poultry sector by HPAI (Original objective) ......................... 18 Prepare for, control, and respond to influenza pandemics and other infectious disease emergencies (Original and revised objective)............................................................... 18 This report was prepared by Nalini Kumar, who traveled to Romania in September 2011, and finalized by Stephen Hutton. The report was peer reviewed by Stephen Hutton and panel reviewed by Ridley Nelson. Yezena Z. Yimer provided administrative support. iv Build capacity to respond to potential future infections of humans by HPAI and other zoonoses (Revised objective) ........................................................................................ 20 5. Efficiency ...................................................................................................................... 21 6. Ratings .......................................................................................................................... 22 Outcome ........................................................................................................................ 22 Risk to Development Outcome ..................................................................................... 23 Monitoring and Evaluation ........................................................................................... 24 Bank Performance ......................................................................................................... 25 Quality at Entry ......................................................................................................... 25 Quality of Supervision .............................................................................................. 26 Borrower Performance .................................................................................................. 27 Government Performance ......................................................................................... 27 Implementing Agency Performance ......................................................................... 28 7. Lessons .......................................................................................................................... 29 References ......................................................................................................................... 31 Annex A. Basic Data Sheet ............................................................................................... 35 Annex B. Additional Documentation ............................................................................... 37 Annex C. Assessing the Level of Preparedness in Romania to respond to Avian Influenza and other infectious diseases ............................................................................................. 45 Annex D. Borrower‘s Comments ..................................................................................... 51 Boxes Box 1. The Global Program for Avian Influenza (GPAI) control ...................................... 6 Tables Table 1. Fiscal impact and economic losses in the Romanian economy due to the avian influenza outbreak, October 2005-May 2006 (Euros '000) ................................................ 4 Table 2: Planned vs. actual costs by component............................................................... 13 Table 3: Original and Revised Project Objectives ............................................................ 16 Figures Figure 1: Absolute Poverty rates and GDP per capita in Romania, 1995-2006 ................. 1 Figure 2: Influenza Vaccine Production at the Cantacuzino Institute 2002-2011 ............ 21 v Principal Ratings ICR* ICR Review* PPAR Outcome Moderately n.a. Unsatisfactory Unsatisfactory Risk to Moderate n.a. Significant Development Outcome Bank Performance Moderately Satisfactory n.a. Moderately Unsatisfactory Borrower Moderately n.a. Moderately Performance Unsatisfactory Unsatisfactory * The Implementation Completion and Results Report (ICR) is a self-evaluation by the responsible Bank department. The ICR Review is an intermediate IEG product that seeks to independently verify the findings of the ICR. Key Staff Responsible Division Chief/ Project Task Manager/Leader Sector Director Country Director Appraisal Matthias Grueninger Benoit Paul Blarel Anand K. Seth Completion Wezi Marianne Msisha Daniele Dulitzky Peter Harrold vi IEG Mission: Improving World Bank Group development results through excellence in evaluation. About this Report The Independent Evaluation Group assesses the programs and activities of the World Bank for two purposes: first, to ensure the integrity of the Bank’s self-evaluation process and to verify that the Bank’s work is producing the expected results, and second, to help develop improved directions, policies, and procedures through the dissemination of lessons drawn from experience. As part of this work, IEG annually assesses 20-25 percent of the Bank’s lending operations through field work. In selecting operations for assessment, preference is given to those that are innovative, large, or complex; those that are relevant to upcoming studies or country evaluations; those for which Executive Directors or Bank management have requested assessments; and those that are likely to generate important lessons. To prepare a Project Performance Assessment Report (PPAR), IEG staff examine project files and other documents, visit the borrowing country to discuss the operation with the government, and other in-country stakeholders, and interview Bank staff and other donor agency staff both at headquarters and in local offices as appropriate. Each PPAR is subject to internal IEG peer review, Panel review, and management approval. Once cleared internally, the PPAR is commented on by the responsible Bank department. The PPAR is also sent to the borrower for review. IEG incorporates both Bank and borrower comments as appropriate, and the borrowers' comments are attached to the document that is sent to the Bank's Board of Executive Directors. After an assessment report has been sent to the Board, it is disclosed to the public. About the IEG Rating System for Public Sector Evaluations IEG’s use of multiple evaluation methods offers both rigor and a necessary level of flexibility to ad apt to lending instrument, project design, or sectoral approach. IEG evaluators all apply the same basic method to arrive at their project ratings. Following is the definition and rating scale used for each evaluation criterion (additional information is available on the IEG website: http://worldbank.org/ieg). Outcome: The extent to which the operation’s major relevant objectives were achieved, or are expected to be achieved, efficiently. The rating has three dimensions: relevance, efficacy, and efficiency. Relevance includes relevance of objectives and relevance of design. Relevance of objectives is the extent to which the project’s objectives are consistent with the country’s current development priorities and with current Bank country and sectoral assistance strategies and corporate goals (expressed in Poverty Reduction Strategy Papers, Country Assistance Strategies, Sector Strategy Papers, Operational Policies). Relevance of design is the extent to which the project’s design is consistent with the stated objectives. Efficacy is the extent to which the project’s objectives were achieved, or are expected to be achieved, taking into account their relative importance. Efficiency is the extent to which the project achieved, or is expected to achieve, a return higher than the opportunity cost of capital and benefits at least cost compared to alternatives. The efficiency dimension generally is not applied to adjustment operations. Possible ratings for Outcome: Highly Satisfactory, Satisfactory, Moderately Satisfactory, Moderately Unsatisfactory, Unsatisfactory, Highly Unsatisfactory. Risk to Development Outcome: The risk, at the time of evaluation, that development outcomes (or expected outcomes) will not be maintained (or realized). Possible ratings for Risk to Development Outcome: High, Significant, Moderate, Negligible to Low, Not Evaluable. Bank Performance: The extent to which services provided by the Bank ensured quality at entry of the operation and supported effective implementation through appropriate supervision (including ensuring adequate transition arrangements for regular operation of supported activities after loan/credit closing, toward the achievement of development outcomes. The rating has two dimensions: quality at entry and quality of supervision. Possible ratings for Bank Performance: Highly Satisfactory, Satisfactory, Moderately Satisfactory, Moderately Unsatisfactory, Unsatisfactory, Highly Unsatisfactory. Borrower Performance: The extent to which the borrower (including the government and implementing agency or agencies) ensured quality of preparation and implementation, and complied with covenants and agreements, toward the achievement of development outcomes. The rating has two dimensions: government performance and implementing agency(ies) performance. Possible ratings for Borrower Performance: Highly Satisfactory, Satisfactory, Moderately Satisfactory, Moderately Unsatisfactory, Unsatisfactory, Highly Unsatisfactory. vii Preface This is the Project Performance Assessment Report (PPAR) for the Romania Avian Influenza Control and Human Pandemic Preparedness and Response Project (IBRD- 48390) under the Global Program for Avian Influenza (GPAI). The project was approved on September 8, 2006 and became effective on March 19, 2007. A total of EUR 29.6 million was originally committed for the project. At project closure EUR 17.9 million was disbursed. The project closed on December 31, 2010, one year behind schedule. The report presents findings based on a review of the project‘s implementation completion report, program paper, legal documents, sector reports, and other relevant material. In addition, an IEG mission to Romania in September 2011 held discussions in Bucharest with World Bank country office staff, government officials and agencies, project staff, parliamentarians, beneficiaries, key donors, hospital staff, doctors and academia, and visited the Cantacuzino Institute, the Victor Babes Clinic, and the Bals Institute. The mission also visited organizations such as the Poultry Breeder‘s Association to get stakeholder views on poultry rearing conditions in Romania. The mission also made a field visit to Brasov County, where it visited the Clinic Hospital for Infectious Diseases and held discussions with doctors and other hospital staff. A second field visit was to Tulcea County where the mission visited the local veterinary laboratory and held discussions with veterinary staff. The mission also visited the village and backyard poultry farm where the first case of the H5N1 virus in Romania was detected in early October 2005 and held discussions with village officials and the village vet. In Brasov and Tulcea counties the IEG mission carried out an informal survey of about 50 people to assess the awareness amongst the general population about avian influenza and the Bank project. The contributions of all stakeholders, including World Bank staff in Tirana, are gratefully acknowledged. Following standard IEG procedures, copies of the draft PPAR were sent to the Government of Romania for comments. All comments received are included as Annex D. i Summary Romania‘s transition from communism was difficult and the country faced economic and financial instability that led to a major financial crisis in 1998-99. The crisis, pressure from international organizations, and prospects of European Union (EU) accession all generated momentum for structural reforms in the country. As a consequence, good progress was made on stabilization and private sector development, and the Romanian economy grew rapidly between the years 2003-08. Over time, Romania‘s national development policies have become increasingly connected to the EU, although in areas such as consumer health protection, food safety, veterinary and phyto-sanitary standards, the country still has to undertake major reforms to comply with EU standards. EU membership, which came in January 2007, has put considerable pressure for application of EU standards in implementation of laws and strategies as well as in responding to or demonstrating preparedness for emergencies. One such emergency arose in 2005 in the form of avian influenza. The first case of the H5N1 virus in Romania was detected on a poultry farm in the Danube Delta in early October 2005. Additional outbreaks occurred across much of the country, for a total of of 183 outbreaks identified by June 2006. The Romanian government adopted national legislation for implementing EU directives on measures for prophylaxis, surveillance, and control of avian influenza and took immediate steps to contain the outbreaks. It put in place a Contingency Plan for both animals and humans, established emergency response structures at the central and local levels, and established a National Center for the Coordination of the Fight against Avian Influenza as an advisory body. Romania also received support from the United States Agency for International Development (USAID), Sweden, the Netherlands, and Germany to tackle the infection. In March 2006, the Government requested support from the World Bank and an emergency project under the Global Program for Avian Influenza Control (GPAI) was approved in the same year. The Romania project was part of a horizontal Adaptable Program Loan to help countries around the world prepare for and contribute to prevention of a global avian influenza pandemic. With a loan of EUR 29.6 million, the project development objective was ―to assist the Borrower in reducing the threat posed to humans and the poultry sector by HPAI [Highly Pathogenic Avian Influenza] and other zoonoses, and preparing for, controlling and responding to influenza pandemics and other infectious disease emergencies.‖ The original objective had two distinct parts, one related to upgrading capacity to manage avian influenza and other zoonoses, and a second related to preparedness for influenza pandemics and infectious disease emergencies in humans. There was already substantial capacity in Romania to respond to avian influenza emergencies, as demonstrated in the 2005- 2006 outbreaks, but there was some scope to improve capacity in the animal health surveillance, diagnostic, and other systems. Another motivation for the project was to demonstrate to the EU that Romania, which was about to become an EU member, was giving importance to responding to future emergencies. Other zoonotic diseases were included in the objectives to ensure consistency with the overall GPAI program goal and to help the ii country cope with diseases such as swine flu and bovine tuberculosis, in compliance with EU standards for trade in animals and animal origin products. The emergency project was approved by the Bank‘s Board in September 2006 and did not become effective until March 2007 because the Romanian parliament was not in session. During the six month gap between project approval and effectiveness, in January 2007, Romania acceded to the European Union. The pace and interest in reforms in the country, including on demonstrating preparedness for dealing with emergencies, slowed considerably after EU accession. The Animal Health and Food Safety Agency also became reluctant to implement the project as originally designed and wanted to use the project resources for the construction of a new building. Ultimately, the lack of performance on the part of the agency forced project restructuring and on July 7, 2009 the animal health component was eliminated from the objectives in a formal restructuring. The project‘s objective was changed: ―to assist the Borrower in building its capacity to respond to potential future infections of humans by HPAI and other zoonoses, and preparing for, controlling and responding to influenza pandemics and other infectious disease emergencies in humans.‖ While this made the project less ambitious, it also made it less relevant for reducing the threat posed from avian influenza, since controlling infections in poultry and improving biosecurity are critical for reducing conditions conducive to the mutation of the H5N1 virus and its spread to and among humans. The project ultimately contributed to building intensive care capacity in seven regional and two central infectious disease hospitals, although inadequate trained staff for running the facilities remains an issue in several hospitals. It helped increase vaccine production capacity at the Cantacuzino Institute, a national institute for research and development in microbiology, and provided support for upgrading of the National Influenza Laboratory. However, critical rehabilitation work at the Institute was not completed and the biosecurity level 3 (BSL3) laboratory was not operational as of project closure. The project did not make any contribution to improving biosecurity in backyard poultry farms, which remains low because of traditional animal management practices. Project efficiency suffered from serious implementation delays, partially completed activities, and facilities that were not operational. Overall, taking into account both original and revised project objectives, project outcome is rated unsatisfactory. The difficult financial conditions in Europe have made budgets and resources tighter in Romania. Under these circumstances, resources for hiring staff, training, and operation and maintenance remain limited. The intensive care units were constructed and equipped but most of the hospitals are finding it difficult to effectively utilize them because of lack of adequate trained personnel and doctors. Vaccine production at the Cantacuzino Institute shut down in early 2012, and has not yet been restored. Overall, the assessment rates the risk to development outcome as significant. Bank performance is rated moderately unsatisfactory and Borrower performance is rated moderately unsatisfactory. The Bank did not take the tremendous ongoing changes in Romania into account when agreeing to the project design. Had it done so, it would have become clear that Romania did not actually need support to tackle the immediate outbreaks iii of avian influenza. The Government asked for support for building capacity for the future; the appropriateness of an emergency loan as a part of a horizontal adaptable program loan for this purpose is questionable. The Borrower used the project to help its EU accession process and did not actually need it to tackle the avian influenza emergency. Ultimately the project that was designed was very ambitious and had too many sub-components for an emergency operation. Inadequate attention was given to the quality of monitoring and evaluation and its use for decision-making. On the Government side, once the accession was achieved there was significant waning of interest in project activities. There were also severe weaknesses in planning, programming, budgeting, and execution of project activities. The performance of the Animal Health and Food Safety Agency was a serious constraint on implementation, leading to the animal health component being dropped. The main lesson identified by this project is that countries that have already effectively controlled avian influenza outbreaks in poultry may not need an emergency project, but rather long-term capacity building. Following this project, avian influenza is still a threat in Romania, as gaps in animal health sector capacity remain and biosecurity on backyard farms is still weak. Caroline Heider Director-General Evaluation 1 1. Country and Project Context Country Background 1.1 Romania, situated in the Balkan Peninsula in South-Eastern Europe, faced a difficult transition from communism that began in 1989. Market distortions and structural problems were issues in all transition countries. In addition, Romania was saddled with the legacy of former president Ceausescu‘s highly personalized rule that left the country with a bureaucracy that was insecure, politicized, and prone to corruption (IEG 2005). The Romanian economy was also plagued by economic and financial instability that led to a major financial crisis in 1998-99 (ibid). That crisis, the pressure from international financial institutions, and prospects of European Union (EU) accession generated a momentum for structural reforms in the country. As a consequence good progress was made on stabilization and private sector development. The economy grew rapidly, at more than 6 percent a year between 2003-08, and absolute poverty declined from 35.9 percent in 2000 to 13.8 percent in 2006 (Figure 1). Figure 1: Absolute Poverty rates and GDP per capita in Romania, 1995-2006 Source: World Bank 2007a. 1.2 Romania joined the EU on January 1, 2007. The reform momentum slowed after the accession. Political differences escalated and reform fatigue set in, affecting the consistency and coherence of public policies, and complicating the adoption and implementation of longer-term policies and programs (World Bank 2009c). 1.3 In late 2008 the global economic and financial crisis hit the country and real gross domestic product (GDP) plummeted, dropping by 13 percent in the fourth quarter of 2008 (IMF 2010). In early 2009 the authorities put together a comprehensive policy reform package supported by an International Monetary Fund (IMF) Stand-by Agreement and funds from the EU and other multilateral institutions (IMF 2010). This contributed to stabilizing the 2 economy, although economic activity remained weak throughout 2009. Since then, growth has returned to Romania although the recovery remains fragile (IMF 2012). The World Bank and Romania 1.4 The World Bank has been supporting the government of Romania in its efforts to promote growth and development since 1991. The Bank‘s portfolio in the post 2004 period was closely connected to Romania‘s EU accession and integration priorities (Joint Portfolio Review 2008). However, the Bank‘s financial involvement and leverage declined when Romania acceded to the EU in 2007. The financial crisis led to re-engagement of the World Bank, with demand for technical advice in particular.1 1.5 The agriculture sector, which accounts for 7 percent of the country‘s gross value added, is critical to Romania‘s European integration and social cohesion goals (World Bank 2010a).2 Romania is the second largest agricultural producer in Central and Eastern Europe after Poland, with an agricultural area of about 14.8 million hectares (Ilie and Zaharia 2007). In the period prior to the accession, the World Bank provided considerable support for reform of the agriculture sector, including livestock. In addition to providing loans, the Bank also undertook analytical work (see for example, World Bank 2005b) to help the country‘s policymakers identify the key challenges in agriculture and options to prepare Romania for the accession. 1.6 Over time, Romania‘s national development policies, including in agriculture, have become increasingly connected to the European Community policies and the acquis.3 In several areas, such as consumer health protection, food safety, veterinary, and phyto-sanitary standards, however, the country has yet to undertake major reforms to meet EU standards. 1 For example, the Bank signed contracts for EUR 3.2 million with the government for a Fee-based Service for the functional reviews of selected ministries and government agencies. The government asked for these reviews to support the structural changes required by the EC under its Memorandum of Understanding with the government for EUR 5 billion in financial support (World Bank 2011c). The functional review for the Ministry of Public Health (World Bank 2011b) has been quoted in this report. 2 About 45 percent of Romania‘s population lives in the rural areas and animal products represented about 33 percent of the value of agricultural production in 2009. As an EU member state, Romania implements the European Union‘s Common Agricultural Policy. Despite the substantial support from donors, including the EU, in the past decade there remains considerable discrepancy between Romania‘s agriculture sector and the Common Agricultural Policy‘s reform principles. Romania and other countries that joined the EU later were given time to catch up with EU laws. For example, EU legislation granted Romania the right to defer meeting certain sanitary requirements for raw milk and certain structural requirements for dairy establishments until December 2011 (EC 2011b). 3 ―The acquis is the body of common rights and obligations that is binding on all the Member States of the European Union. It is constantly evolving and comprises: the content, principles and political objectives of the Treaties; legislation adopted pursuant to the Treaties and the case law of the Court of Justice; declarations and resolutions adopted by the Union; instruments under the Common Foreign and Security Policy; instruments under Justice and Home Affairs; international agreements concluded by the Community and those entered into by the Member States among themselves within the sphere of the Union's activities. Adoption and implementation of the acquis are the basis of the accession negotiations.‖ http://ec.europa.eu/enlargement/glossary/terms/acquis_en.htm 3 Public health indicators also lag behind those of other EU member countries.4 Accession to EU membership was a strong motivation for Romania to adopt EU standards in implementation of laws and strategies and in responding to or demonstrating preparedness for emergencies.5 One such emergency arose in 2005 in the form of avian influenza, a couple of years before Romania became an EU member. Disease and Project Context 1.7 Between 2003 and 2011, the highly pathogenic H5N1 virus that causes avian influenza affected 61 countries, including Romania, mainly in poultry, although several countries also reported human infections with the virus. The first case of the H5N1 virus in Romania was detected on a backyard poultry farm6 in Tulcea County in the Danube Delta in early October 2005. Subsequently, cases were identified in backyard poultry systems in 52 towns across nine other counties in the Danube Delta and Black Sea area. This was the first wave of avian influenza in Romania that lasted until late April 2006. Three weeks later a second wave of infections started in Brasov County, a considerable distance from the Danube Delta and on a commercial farm. On May 21, 2006, the presence of avian influenza was confirmed in Bucharest and as of June 2006 the second wave had led to 121 confirmed cases in 19 of Romania‘s 42 counties. In the second wave of outbreaks about one million birds either died or were culled and the total cost of disease control and economic losses was estimated to exceed EUR 90 million (Table 1). 1.8 Avian influenza, as the name suggests, is generally a disease of birds. However, it can also infect humans if they come in contact with infected birds. The H5N1 virus is deadly and has led to very high mortality in poultry and to 371 recorded deaths in humans worldwide.7 To date, no cases of avian influenza have been identified among humans in Romania. Avian influenza virus types generally do not readily replicate and transmit between humans; the H5N1 virus has so far largely ravaged the poultry industry. But avian influenza viruses, including H5N1, could mutate to a form that is able to replicate and spread easily among humans. If this happens a pandemic can occur and lead to massive death tolls as happened in the 1918-19 ―Spanish Flu‖ pandemic.8 4 The World Bank‘s 2011 functional review of the health sector in Romania compared health outcomes for the initial 15 members of the EU (EU15), those of the twelve more recent accession countries (EU12), and those of Romania (World Bank 2011b). The study showed Romania lagging behind both groups. In terms of life expectancy, infant mortality, maternal mortality, ischemic heart disease, and cancer of the cervix, the EU15 countries rank higher, the EU12 countries rank second, and Romania lags behind. 5 For example, the government had to submit an Avian Influenza Contingency Plan for Romania to the EU as a requirement for accession on January 1, 2007. 6 In these poultry production systems that are common in developing countries around the world, farmers rear several animals such as chickens, ducks, pigs, cows in their backyards, and in close proximity with human populations. 7 The case-fatality ratio globally among reported human cases of the disease through contact with infected poultry has been high (60 percent), but many non-fatal cases have probably not been reported. 8 Influenza in humans can be caused by type A, B, or C viruses, with the former two causing most human infections. Influenza outbreaks in humans occur annually with changes in surface proteins of the virus and the severity of the disease varies from year to year. The changes in the viruses are occurring continuously hence the 4 Table 1. Fiscal impact and economic losses in the Romanian economy due to the avian influenza outbreak, October 2005-May 2006 (Euros '000) Cost Items Fiscal costs Economic losses (financial prices) LOSS OF PRODUCTIVE ASSETS Value of poultry culled (meat and eggs, at average 2005 12,908 prices) Compensation to poultry holders 3,900 OUTBREAK CONTROL AND RELATED EXPENDITURES Outbreak operations (disinfection, culling, incineration of 5,720 5,720 poultry, etc.) Prophylactic treatment with anti-viral medication (Tamiflu) 200 200 Vaccination of humans 1,041 1,041 Sanitary & protection equipment, information materials for 278 278 protecting public health Road disinfection ("filters") 45,183 45,183 LOSSES TO THE POULTRY INDUSTRY DUE TO INTERRUPTION IN PRODUCTION AND COLLAPSING MARKETS Production loss during the outbreak control period 1,489 (disinfection and surveillance) Drop in poultry demand (less sales) 15,400 Depressed prices (sales at lower prices) 28,00 Export ban 9,800 Extra storage costs 12,600 Extra disinfection costs for commercial farms 9,800 Delays in production flows 700 LOSSES TO OTHER INDUSTRIES Tourism 50,000 Total 56,322 193,119 Note: Certain items of public expenditure that were related to the control of avian influenza were not reported, such as some expenditures by security forces deployed by the Ministry of Administration and Interior to the outbreak areas, or the dedicated education classes required by the Ministry of Education and Research. The table, therefore, presents very conservative estimates, the actual costs were likely much higher Source: Technical Annex (World Bank 2006a). 1.9 While it is not possible to anticipate the timing of an avian influenza pandemic, the risk remains.9 One of the biggest worries for the international community and for countries need to continuously update influenza vaccines. By contrast influenza pandemics occur infrequently when viruses such as those that cause AI to which humans have not been exposed, mutate and acquire the ability to transmit from person to person. On average, three pandemics per century have been documented since the 16th century, occurring at intervals of 10-50 years. Details on the disease and different virus types and their characteristics, including information on past pandemics, are in Annex B. 9 Highly pathogenic avian influenza H5N1 viruses continue to circulate in poultry and cause disease, and remain a threat to human and animal health (FAO-OIE-WHO 2011a). From January to March 2011, 826 H5N1 outbreaks in poultry were reported from 12 countries around the world (FAO 2011). 5 such as Romania is that the conditions for mutation of the virus abound among birds living in close contact with humans in backyard poultry production systems common throughout the developing world.10 About 40 percent of Romania‘s poultry production is carried out by smallholders, for which prevention and control measures are difficult to enforce.11 Increased globalization and trade and the natural reservoirs for the virus in migratory birds have increased the risk that the H5N1 virus could spread rapidly from wild birds to poultry and from infected poultry to humans. The outbreak in Tulcea County in the Danube delta in 2005 is thought to have occurred because of the contact between poultry and wild birds. 1.10 Following the outbreaks, Romania adopted legislation for implementing European Council Directive 92/40/EC on measures for prophylaxis, surveillance, and control of avian influenza into national legislation and took immediate measures to contain the disease. It also put in place a Contingency Plan for avian influenza for both animals and humans and established emergency response structures both at the central and local levels. In May 2006 the Government established a National Center for the Coordination of the Fight against Avian Influenza as an advisory body. After the first confirmed outbreak, Romania received immediate support from the United States Agency for International Development (USAID), Sweden, the Netherlands, and Germany to tackle the infection. EU grants through the Phare program12 and the Special pre-Accession Programme for Agriculture and Rural Development were already available for upgrading of diagnostic and institutional capacities. In March 2006 the Government also requested support from the World Bank under the Global Program for Avian Influenza (GPAI) (Box 1). 1.11 Building on the disease context discussed above, the potential threats from avian influenza to Romania arise from four scenarios:  An outbreak among poultry, with economic costs for the agriculture sector.  Human contact with infected animals, leading to human avian influenza cases, but not of a form that could be easily transmitted between humans. This would lead to a modest number of influenza cases and possibly deaths.  A strain of avian influenza transmissible between humans that develops within Romania and leads to an influenza epidemic or pandemic if not contained. 10 The virus could change either through mutation or re-assortment. Mutation occurs when a virus changes its genetic makeup, whereas re-assortment occurs when two different viruses exchange genetic material. The re- assortment could happen if animals that are susceptible to both human and avian flu, such as pigs, which are also found in backyard systems, become concurrently infected with both kinds of flu. Either mutation or re- assortment could lead to a strain that is transmissible between humans. 11 Backyard poultry declined in importance from 72 percent to about 61 percent of total production in 2010, according to the Press Release dated June 30, 2011, on Preliminary Results of the General Agricultural Census 2010 in Romania. The national veterinary agency reported that as of 2013, the proportion of birds reared on backyard farms had declined to 40 percent. 12 The Phare program is one of three pre-accession instruments financed by the EU to assist the applicant countries of Central and Eastern Europe in their preparations efforts for joining the EU. It was originally created in 1989 to assist Poland and Hungary, but later covered other countries such as Bulgaria and Romania. http://ec.europa.eu/enlargement/how-does-it-work/financial-assistance/phare/index_en.htm 6  A strain of avian influenza transmissible between humans that develops abroad, causing a pandemic that spreads to Romania. 1.12 Both poultry outbreaks and cases of transmission from poultry to humans have already happened in several countries. However, Romania has had only outbreaks among poultry to date. A strain that is transmissible between humans has not yet been documented anywhere in the world but the risk remains, both within countries and as transmitted from other countries. Box 1. The Global Program for Avian Influenza (GPAI) control The threat of a severe global human pandemic arising from mutation of the H5N1 virus has caused great concern to the international community. Billions of dollars have been pledged for efforts to control avian influenza. The Food and Agriculture Organization (FAO), World Organization for Animal Health (OIE) and the World Health Organization (WHO) have committed to work together. The international technical agencies realize that both human and animal health sectors have a stake in tackling pathogens that can threaten animal and human health and that responding to these diseases requires a multi-sectoral and multi-institutional response. The World Bank agreed to provide financial support to the effort being undertaken through these international organizations and has helped more than 50 countries deal with AI. The Bank has two main mechanisms to support client countries in this area: the Global Program for Avian Influenza (GPAI) and the multi-donor Avian and Human Influenza Facility . The Bank's Board of Executive Directors endorsed the GPAI in January 2006, and extended it in June 2009. The Avian and Human Influenza Facility was also created in 2006 to assist developing countries in meeting financing gaps in their integrated country programs to minimize the risk and socioeconomic impact of avian and possible human pandemic influenza. In many cases, the facility co-finances projects under GPAI. GPAI is a global horizontal Adaptable Program Loan that allows for the use of up to US$ 1 billion (an increase from the original amount of US$500 million) under which individual countries can obtain separate loans/credits/grants (depending on country case) to finance their own national projects. The overall program development objective for the GPAI is "to minimize the threat posed to humans by HPAI infection and other zoonoses and to prepare for, control, and respond to influenza pandemics and other infectious disease emergencies in humans." (World Bank 2005, p. 16). The GPAI draws on an integrated approach developed in conjunction with FAO, OIE, and WHO. Countries can access funding to strengthen their veterinary and health services to deal with avian influenza outbreaks among animals, minimize the threat to people, and prepare for and respond to any potential human influenza pandemic. GPAI operations are processed using emergency procedures, which allow quick preparation and approval. A country qualifies for support for an emergency project under the Program when it demonstrates its commitment and readiness to implement early detection and rapid response measures appropriate to the specific country conditions. Different criteria were established for country participation on the basis of the gravity of the avian influenza situation in the country. Romania was considered a ―newly infected country‖, with active avian influenza among poultry but no human infection, and the existence of an appropriate program of rapid response, detection and containment measures. Note: OIE is an independent intergovernmental organization founded in 1924 with 178 members. OIE‘s mandate is to improve animal health worldwide. OIE‘s headquarters are in Paris and there are 11 regional offices. In 2004 OIE member countries approved the creation of a single list of diseases to be notified to OIE within 24 hours. Source: World Bank 2005; World Bank website, OIE website and FAO 2007b. 7 2. Objectives, Design, and their Relevance Objectives 2.1 The project development objective as noted in the Technical Annex (World Bank 2006a) and the Loan Agreement was ―to assist the Borrower in reducing the threat posed to humans and the poultry sector by HPAI [Highly Pathogenic Avian Influenza] and other zoonoses, and preparing for, controlling and responding to influenza pandemics and other infectious disease emergencies.‖13 The project was restructured on July 7, 2009 and the objective was changed: ―to assist the Borrower in building its capacity to respond to potential future infections of humans by HPAI and other zoonoses, and preparing for, controlling and responding to influenza pandemics and other infectious disease emergencies in humans.‖ Essentially, the poultry sector was removed from the objectives and the emphasis was put on infections in humans. In addition, ―reducing the threat posed by HPAI and other zoonoses‖ was replaced by building the ―capacity to respond.‖ 2.2 Two main reasons were given for the restructuring. First, the government agency responsible for the animal health component had not effectively managed the project making the achievement of the related part of the objective unlikely. That part of the project had been under suspension since November 18, 2008; corrective action that would allow the Bank to lift the suspension had not been taken by the March 31, 2009 deadline. Second, the Ministry responsible for the implementation of the human health component had requested additional resources for a cost overrun and for expenditures not foreseen at appraisal to complete agreed activities. Relevance of the Objectives RELEVANCE OF THE ORIGINAL OBJECTIVES 2.3 There were two distinct parts to the original objectives—one related to avian influenza and the second to other zoonoses and infectious disease emergencies. The objective of reducing the threat posed to humans and the poultry sector from avian influenza and preparing for the influenza pandemic was relevant to the country context. There had been several outbreaks in poultry in Romania in backyard systems and in commercial farms. Romania was the first European country to confirm an avian influenza outbreak with the H5N1 virus in 2005 and spread of the virus among poultry significantly affected the country‘s agriculture sector and economy. It also presented a risk for spread of infection to poultry in other countries of the region. The Romanian Danube Delta is on the path of migratory birds from Europe and Asia to Africa. Further, conditions for the mutation of the virus were also present in the backyard systems where about 60 percent of poultry in 13 Zoonoses are infectious diseases in animals that can be transmitted to people. The natural reservoir for the infectious pathogen is the animal. Rabies and anthrax are examples of zoonoses. Infection via zoonoses can be acquired either by direct contact with infected animals, or by consuming contaminated food products. Zoonotic agents could be viruses, bacteria, fungi, parasites or other biological entities. 8 Romania are reared. Resources and technical support for control of avian influenza were available from other donors. The Bank‘s resources, in keeping with the GPAI goals, were to complement these efforts. The global public good nature of the project was also recognized by the 2006 Country Partnership Strategy (World Bank 2006b). The project was broadly in keeping with the approach in the 2009 Country Partnership Strategy (World Bank 2009b) that emphasized flexibility and responsiveness to evolving circumstances and Government priorities, but was not specifically mentioned in any of the pillars of the strategy. 2.4 By the time of the project concept review in June 2006, the Government of Romania had already controlled two waves of avian influenza outbreaks among poultry in October 2005 and April 2006, demonstrating substantial capacity to respond to outbreaks.14 Romania already had disease monitoring in high risk districts, a passive surveillance system, and significant diagnostic capacity at a central biosecurity level 3 (BSL 3) animal health laboratory. 15 However, the Bank reported that there were still a number of weaknesses in capacity in the veterinary services agency: monitoring was not done at a national level, there was no active surveillance program, there was limited diagnostic capacity in county laboratories, there was an insufficient level of epidemiological analysis, there was no integrated animal disease information system, and carcass disposal was still done through open burning (World Bank 2006a). 2.5 Avian influenza remained a relevant threat to humans and the poultry sector as of project closure, with a continued threat of outbreaks from infection by wild birds. An outbreak occurred among backyard poultry in Tulcea County in March 2010, and was controlled successfully though culling operations, without support from the project. 2.6 An important factor in the client demand for the project was to demonstrate to the EU that the country was prioritizing capacity building for emergency response. IEG mission conversations with government officials revealed that adequately dealing with outbreaks and being ready for future emergencies were ‗tests‘ for Romania as it prepared for accession to the EU. In fact, the Government had to submit an avian influenza contingency plan to the EU as a requirement for accession on January 1, 2007. The 2009 Country Partnership Strategy clearly recognized that the EU and related institutions had become Romania‘s main external partner (World Bank 2009b). Hence, supporting the Government in its accession priorities was clearly in keeping with the Bank‘s strategic approach. 2.7 With respect to other zoonoses and infectious disease emergencies, IEG discussions with government officials and project staff reveal that there was another reason which is not explicitly mentioned in project documents. Unless Romania could demonstrate that it had eliminated (or at least was able to control) several zoonotic diseases, such as swine flu and bovine tuberculosis, it could not comply with EU market standards for trade in animals and animal-origin products. Part of the motivation to include other zoonoses in the objectives was 14 The government undertook a set of specific measures to contain the outbreaks. These included among others: culling of poultry, specific laboratory exams on a large number of samples, expenditure on disinfectants and drugs, distribution of diagnosis kits, incineration and euthanasia of thousands of poultry, compensations to owners of deaf and destroyed poultry. 15 For differences in levels of BSL-2, BSL-3, and BSL-4 laboratories, see Annex B 9 coming from the goal to increase the country‘s capacity to monitor and track diseases of animal origin so that they could be eliminated. However, these other zoonotic diseases were not an emergency in the same sense as avian influenza. Bovine tuberculosis, swine fever, and rabies are prevalent in Romania, but they do not have the potential capacity to mutate in ways that would allow sustained human-to-human transmission, as is the case for avian influenza. Unlike other zoonoses, influenza pandemics can kill millions of people once launched. 2.8 On balance relevance of the original objectives is assessed as substantial. RELEVANCE OF THE REVISED OBJECTIVES 2.9 After the restructuring, reducing the threat to the poultry sector was removed from the objectives, narrowing the focus and reducing the ambition of the project. However, preventing avian influenza in poultry is critical to preventing infections in humans. The revised objectives were only designed to build capacity to respond to the possibility of human infections through contact with infected animals, and to respond to an influenza pandemic emerging within Romania or from abroad. Thus, the base of the pyramid was missing. As long as the virus continues its circulation in birds, the danger of a mutation that can spread between humans remains. The Joint United States Centers for Disease Control and Prevention (CDC)/World Health Organization (WHO) mission to Romania, in response to a request from the Ministry of Public Health in the context of increasing infections in wild birds and poultry in the Danube area noted: ―Overall, the prevention of human cases in Romania requires the control of avian influenza in poultry, particularly backyard poultry, and a proactive social mobilization targeted to households and children in rural setting.‖ 16 (Joint CDC/WHO Mission to Romania 13-17 February 2006). It is no wonder that the three technical organizations—WHO, the Food and Agriculture Organization (FAO), and the World Organization for Animal Health (OIE)—have emphasized containment at the source and considered reducing the load and circulation of the virus in poultry a critical part of the strategy to reduce the risk of human infections or the development of a pandemic (FAO 2007b). 2.10 The relevance of the revised objectives is rated as modest. Design COMPONENTS 2.11 The project had three components—Animal Health, Human Health and Public Awareness, and Communications and Implementation Support. Each of the components had several sub-components, summarized below: 2.12 Component I: Animal Health (Appraisal estimate EUR 14.25 million, Actual EUR 0.22 million) This was by far the largest component, with four sub-components: (a) 16 As the mission document noted, backyard poultry and children coexist in poor rural areas. 10 Strengthening the Institutional Framework, through technical assistance for evaluation of national veterinary services, technical support for realigning the structures and resources of veterinary and food safety institutions with identified priorities and tasks, and technical assistance for HPAI prevention and control contingency planning. (b) Strengthening Disease Surveillance, Diagnostic Capacity, and Research, through technical assistance and training for the staff of veterinary service institutions at national and local levels and for epidemiological studies and surveillance programs; technical assistance and finance of priority laboratory equipment and consumables and increasing the diagnostic capacity of the Institute for Control of Biological Products and Veterinary Drugs; setup of a comprehensive integrated national animal disease information system, and preparatory support for zoonotic disease research and improved facilities for live animal research. (c) Strengthening HPAI Control Programs, through purchase of equipment for animal culling and disposal, and technical assistance for planning and implementation of poultry vaccination campaigns. (d) Improving Bio-security in poultry production and trade, on small-scale poultry farms through awareness campaigns, knowledge assessment and transfer, and investment incentives for infrastructure upgrades. 2.13 Component II: Human Health (Appraisal estimate EUR 19.73 million, Actual EUR 17.1 million) This component had three subcomponents: (a) Enhancing Public Health Program Planning and Coordination, through financial and technical support for research, conferences, and workshops. (b) Strengthening of National Public Health Surveillance Systems, through upgrading the National Influenza Laboratory to BSL 3 and strengthening regional laboratories by providing equipment and training.(c) Strengthening Health System Response Capacity, through strengthening isolation and case management capacity in national and regional hospitals, and upgrading an influenza vaccine production unit at the Cantacuzino Institute. 2.14 .Component III: Public Awareness, Communications, and Implementation Support (Appraisal estimate EUR 3.47 million Actual EUR 0.7 million) (a) Public Awareness and Communication, through strengthened institutional capacity for the National Sanitary, Veterinary, and Food Safety Authority (henceforth the national veterinary agency), support for development an integrated communications strategy and action plan for both human and animal health based on research, and implementation of that plan. (b) Implementation Support, Monitoring, and Evaluation, through financing staff and operating costs of the existing project management unit in the Ministry of Public Health and the unit to be established in the national veterinary agency, and studies and surveys for monitoring and evaluation (M&E). INSTRUMENT AND FINANCING 2.15 The Romania project was part of a horizontal multi-country adaptable program loan and, similar to avian influenza projects in other countries, was processed as an emergency investment operation using procedures under Operational Policy 8.50 on Emergency Recovery Assistance (see also Box 1). Total project cost at approval was EUR 37.5 million, financed by a loan of EUR 29.6 million. The Borrower was to provide EUR 7.9 million. The project was not co-financed. However, the Borrower sought parallel financing from the EU for upgrading 16 country laboratories in all of Romania‘s eight regions. The European 11 Union‘s Phare project also allocated EUR 500,000 for consumables to upgrade diagnostic capacity at the regional and national levels, as well as culling and disposal equipment and disinfectant points on roads. MONITORING AND EVALUATION DESIGN 2.16 As originally designed, monitoring and evaluation was the responsibility of both the public health ministry and the national veterinary agency. At design three outcome indicators were identified (Annex B, Table 6). While they were relevant they were not specific enough to appropriately capture project outcomes. For example, it was not clear what would be considered an ―appropriate‖ contingency plan with respect to the first indicator. Further, the indicator related to risk of human infections was too broad and it is difficult to see how gains in it could be attributed to the project as several factors, including improved nutrition, can lead to decline in the risk of human infections. 2.17 There were also intermediate results and corresponding indicators identified for each component. It would have been helpful if the task team had also linked the 33 intermediate result indicators to the three outcome indicators instead of just the components. IMPLEMENTATION ARRANGEMENTS 2.18 With animal and human health components, both the Ministry of Public Health and the National Agency for Sanitary, Veterinary and Food Safety were project implementation agencies. A Project Management Unit was responsible for project management in each implementing agency. A Steering Committee consisting of representatives from the Prime Minister‘s Office, the Ministry of Economy and Finance, Ministry of Public Health, the national veterinary agency, and the two project management units was responsible for project coordination. Relevance of Design RELEVANCE OF DESIGN TO THE ORIGINAL OBJECTIVES 2.19 In keeping with the GPAI Program, the Romania project had separate animal and human health components and the causal framework emphasized the strengthening of institutional capacity, including capacity in surveillance, diagnostic, and response systems as a means of increasing the country‘s preparedness for avian influenza. The inclusion of an animal health component was important not only for achieving the objective of reducing the threat to the poultry sector, but also for reducing the threat posed by avian influenza in the poultry sector to human health. The initial design was largely appropriate to the original objectives. 2.20 The use of an emergency instrument was less relevant. Unlike many other countries supported under the GPAI, Romania already had significant capacity to control avian influenza outbreaks in the short term, and its main gaps in capacity were for longer term issues that could potentially be addressed using a standard investment instrument. Activities such as support for strengthening applied veterinary research capacity did not justify an 12 emergency intervention. Further, civil works were included as part of the project and could not begin without government permits. Increasing vaccine production capacity relies on a licensing process that takes time. These factors made for a complex and complicated project design, unlikely to be completed in a three-year emergency intervention. 2.21 Financing through the horizontal Adaptable Program Loan set up for the GPAI contributed to this shortcoming, because it meant that operations in all countries facing a threat from avian influenza would be processed as an emergency, even in cases like Romania where effective emergency response capacity had already been demonstrated.17 2.22 On balance, however, relevance of design to the original objectives is rated as substantial. RELEVANCE OF DESIGN TO THE REVISED OBJECTIVES 2.23 Because all activities related to the animal health component were eliminated from the project, it led to a major disconnect with the revised objective to build capacity to prepare for, control and respond to future infections of HPAI and other zoonoses. Neither of these could be tackled without some specific activities on the animal health side. Further, the change from ―reducing the threat posed …‖ to “building capacity to respond …..‖ on the human health side made the objective even more focused on longer-term capacity building and less appropriate for an emergency operation. Operational Policy 8.50 notes the importance of considering urgency when deciding on support with an emergency project. The ‗building of capacity‘ does not happen immediately and was not in keeping with an emergency project with a focus on rapid preparation and appraisal. 2.24 Relevance of revised design to the revised objectives is rated as modest. 3. Implementation 3.1 The project was appraised in July 2006 and approved within two months on September 8. However, project effectiveness was delayed until March 19, 2007 because the Romanian Parliament was not in session.18 Even though it was an emergency operation, it was affected by the overall delayed effectiveness that characterized the rest of the Bank 17 The GPAI document noted: ―Since this Program is designed to provide emergency finance to participating countries to contain an outbreak of avian flu through early detection and rapid response measures, the operations under the Program would be processed under the Banks‘ emergency policy, OP/BP 8.50. The Program would treat specific emergency response projects in each country as ―phases‖ of a horizontal APL [adaptable program loan]….‖ (World Bank 2005 p..13) 18 The loan was approved by the Bank‘s Board on September 8, 2006, signed on October 5 and passed by Romania‘s Chamber of Deputies on December 13, 2006. Since the Romanian Parliament was not in session during the entire month of January the project effectiveness was delayed. The original deadline for effectiveness was February 2, 2007. The Government of Romania sent a request for a 45 day extension of the effectiveness deadline on January 12 which was granted and the new deadline of March 19, 2007 was established. 13 portfolio in Romania.19 The government and the World Bank office in Bucharest organized a workshop to launch the project in May 2007. Significantly, the project did not become effective until after Romania‘s accession to the EU in January 2007. 3.2 Project restructuring was approved by the Board on July 28, 2009 and primarily consisted of canceling the animal health component which was not performing and reallocation of EUR 3.9 million to human health to support the upgrading of the vaccine production capacity at the Cantacuzino Institute. The reason for the restructuring was the poor performance of the national veterinary agency in implementing the project according to the original objectives and design. At the time of restructuring, animal health-related management and communication activities were also canceled from the public awareness, communication and implementation support component. 3.3 The project closing date was extended by a year to December 2010. In September 2010, the Government requested a second extension to allow for the completion of the civil works at the Cantacuzino Institute, which was denied. Actual total project cost was EUR 20.5 million with the Bank loan being for EUR 17.9 million and the Borrower providing EUR 2.6 million. EUR 11.6 million of the Bank loan was canceled. The human health component was nearly completely financed, however only a fifth of the public awareness component was ultimately financed and virtually none of the component on animal health (Table 2.) Table 2: Planned vs. actual costs by component Actual as a Appraisal Estimate Actual Expenditure Component Percentage of (US$ million) (US$ million) Appraisal (%) Animal Health 14.25 0.22 1.5 Human Health 19.73 19.06 96.6 Public Awareness, 3.47 0.71 20.5 Information, and implementation support Total Project Cost 37.44 20.53 54.8 Source: World Bank (2011). IMPLEMENTATION EXPERIENCE 3.4 Disbursement initially was very slow. IEG mission discussions with government officials revealed that project implementation started at a time when Romania was dealing with a range of issues associated with the EU accession and capacity in general was stretched. This was partly responsible for slow implementation. The pace and interest in reforms in the country had also slowed considerably after the January 2007 accession. 3.5 On the animal health side changes in the top management of the national veterinary agency after the accession led to changes in priorities from preparedness for avian influenza 19 According to a World Bank portfolio review (World Bank 2007b) the average time lag between Board approval and effectiveness for Bank projects in Romania was six months, significantly more than the Bank average of 90 days. 14 to supporting the construction and equipment of a new animal disease control center. This had an adverse impact on project implementation. As late as March 31, 2009, when 70 percent of the implementation period since effectiveness had elapsed, only EUR 1.7 million – less than 6 percent of the original loan – had been disbursed. With the national veterinary agency‘s continued reluctance to implement the project as designed and ineffective decision making in the agency, the Bank threatened partial suspension of the loan in July 2008.20 On March 31, 2009, the Borrower requested that Bank reallocate and partially cancel the loan, which made the restructuring necessary. 3.6 The human health component, to which part of the resources were reallocated at restructuring, also had its share of problems both before and after the restructuring that led to delays in implementation. Several challenges arose: the contract with the company for the consultancy services for designing the BSL 3 laboratory within the Cantacuzino Institute was delayed; in August 2008 the Cantacuzino Institute had to shut down its vaccine production because its Good Manufacturing Practice certificate expired; the design of the vaccine production line was not completed for a while due to an unresolved issue with the design firm; and in December 2009 civil works for the upgrading of the National Influenza Laboratory at the Cantacuzino Institute were suspended because no building permit had been issued by the Government. The situation was finally rectified with the efforts of the project management unit, but valuable time was lost; Cantacuzino Institute‘s accreditation for vaccine production was suspended by the National Drug Agency in April 2010 further delaying the planned civil works.21 Unforeseen expenditures also emerged—for example additional equipment for the filling line had to be procured. The existing equipment could only wash the ampoules but not the vials for the vaccines and had to be upgraded. 3.7 Serious budget constraints at the Ministry of Public Health also impacted project activities. Finally, some of the activities that could not be completed by project closing—the rehabilitation of the vaccine production unit and rehabilitation of the vaccine filling station – were transferred to the ongoing Bank-financed Health Sector Reform (Adaptable Program Loan 2) project. At the time of the IEG mission in September 2011, 9 months after closing, these activities remained incomplete. 20 Several operational and institutional noncompliance issues justified the threat of suspension: the Steering Committee did not provide effective guidance and oversight, the National Agency for Sanitary, Veterinary, and Food Safety project management unit was not maintained in a manner satisfactory to the Bank, the Agency‘s Project Coordinator had not effectively managed the implementation of the Project and had not prepared annual work programs and budgets, and the Agency had not implemented the project in accordance with the provisions of the Project Implementation Manual. The implementation of most of the agreed activities had not started. In addition, against the background of these significant delays, the needed and requested update of the procurement plan for activities under the animal health component had not been provided to the Bank. 21 An inspection of the Cantacuzino Institute by the National Drug Agency in February 2010 found some conditions necessary for the production of BCG and influenza vaccines lacking. To get temporary permits for vaccine production the Cantacuzino Institute was required to rectify this situation, in particular through the installation of water purification equipment. Following the inspection carried out by the National Drug Agency on May 12, 2010 the accreditation was renewed and vaccine production resumed. 15 SAFEGUARDS COMPLIANCE 3.8 The project was an Environment safeguard category B. Project activities were not expected to have significant adverse environmental effects, though a moderate risk from inadvertent spread of the avian influenza virus and improper waste management were identified. The investments in facilities, equipment, and training for veterinary and public health staff and laboratories were expected to improve the effectiveness and safety of existing procedures and bring them in line with World Organization for Animal Health (OIE) and WHO standards. Overall, the national legislation in Romania was found by the project‘s Environmental Management Plan to be comprehensive and in line with EU directives (World Bank 2006b). As an EU member state, Romania implements and manages the European Union‘s Common Agricultural Policy that puts considerable emphasis on environmental standards and good agricultural practices, including animal welfare standards (Otiman and others 2010). The Joint Portfolio Review in 2008 also found that the local capacity in environment management had increased considerably in the last few years and noted that this could be one factor why two Romanian projects out of eight Bank-wide were chosen in 2006 for piloting the use of national environmental systems. FIDUCIARY 3.9 Given the low level of disbursement by December 31, 2007, the Bank agreed to a temporary audit waiver and allowed for a combined first audit for fiscal years 2007 and 2008. There was some delay in submitting the audit report from the independent auditors. Overall the report was clean, although one internal control issue was mentioned for the National veterinary agency arising primarily because of the lack of adequate financial management staff. In May 2010, the Bank‘s financial management specialist carried out a comprehensive assessment of project accounting and reporting arrangements, staffing, internal control procedures, planning, and budgeting, counterpart funding, financial manual and external audits. The assessment found that the Ministry of Public Health project management unit had addressed all the recommendation from the 2009 financial management supervision mission and its financial management arrangements were satisfactory. However, the arrangements at the national veterinary agency project management unit remained unsatisfactory, mainly because of insufficient financial management capacity but also because of the several layers of control for the approval of each contract. Both these factors led to heavy delays in approval of contracts and payments and in reconciliation of accounting records with bank account statements. IMPLEMENTATION OF MONITORING AND EVALUATION 3.10 As late as July 2007, work on an appropriate monitoring and evaluation (M&E) system had not started and the relevant Implementation Status and Results Reports (ISRs) rated M&E as unsatisfactory. Progress continued to be rated moderately or fully unsatisfactory for a large part of the project period, rising to moderately satisfactory only in December 2009 and then declining again. 16 3.11 With the restructuring, the first two outcome indicators were dropped (Annex B, Table 2). All intermediate indicators related to Animal Health and Strategic Communication were also dropped. The reporting in the ISRs on intermediate results and indicators is problematic. First, the ISRs mix up the intermediate results and their indicators as laid out in the Technical Annex and report on a mixture of ten without indicating why the specific ones were picked. Second, M&E reporting was focused mainly on activity completion or initiation and inputs rather than results. The opportunity to make the M&E framework more useful for capturing project achievements was missed at the restructuring. 4. Achievement of the Objectives 4.1 Over the life of the project there were four objectives, of which two were original objectives that were later dropped, one objective was introduced at restructuring, and one objective was maintained throughout the project (Table 3). The evidence of achievement of each of the four objectives is discussed below. Table 3: Original and Revised Project Objectives Objective Original Revised Reduce the threat posed to humans by HPAI and other zoonoses √ Reduce the threat to the poultry sector by HPAI √ Prepare for, control, and respond to influenza pandemics and other infectious disease √ √ emergencies Build the Borrower‘s capacity to respond to potential future infections of humans by HPAI √ and other zoonoses Reduce the threat posed to humans by HPAI and other zoonoses (Original objective) 4.2 Animal products management practices: Since the animal health component was canceled with minimal disbursement, the project did not make any contribution to improving the biosecurity in backyard poultry farms, where traditional animal management practices prevail.22 The commercial poultry houses generally have good levels of biosecurity, have their own slaughterhouses and have made investments to get in line with EU requirements (Moldovan 2010). Investments in machinery and equipment for slaughterhouses and in buildings and means of transport for the pork and beef meat industry have increased, reflecting the priority to bring technology in line with EU norms and standards in the country 22 Biosecurity refers to measures taken to prevent incursion and spread of disease. It includes hygiene and management measures to minimize the risk of introducing pathogens on farms or poultry establishments (―bio- exclusion‖) and measures to minimize the risk of transmission of infection to other farms in case a disease outbreak occurs (―bio-containment.‖) These two sets of measures would require among other things, appropriate construction of poultry houses to prevent entry of wild birds and rodents, proper ventilation, limiting entry of non-essential visitors, use of appropriate disinfectants in foot and wheel dips, hand hygiene, aerial and litter disinfection, protective clothing. These measures are often supplemented with vaccination. 17 (ibid). A recent European Commission audit found that while progress has been made, there are still major deficiencies in compliance with European Commission regulations in the disposal of unprocessed animal by–products originating in backyard farms and in hygiene conditions in some animal by-product processing plants (EC2011a). The audit also found weaknesses in official controls at such plants. An OIE assessment in 2008 noted that the existence of a large number of small farms makes it difficult to maintain quality standards (OIE 2008). 4.3 Capacity to test for zoonotic bacteria and viruses to a quality standard: The capacity to test for bacteria and viruses in both animals and humans exists in Romania. Each county has a state veterinary laboratory and private laboratories and commercial farms often have their own laboratories. The project, as designed, was to help strengthen diagnostic capacity in Romania‘s national reference laboratory. However, as the animal health component was canceled, the project did not make a contribution in this area. On the human health side, however, the project supported upgrading the National Influenza Laboratory at the Cantacuzino Institute. The civil works at the biosecurity level 3 (BSL3) laboratory were completed, but the laboratory was not operational as of September 2011, as it lacked some equipment and had not received certification. 4.4 Surveillance and monitoring for zoonoses: The 2008 OIE evaluation found that border security and quarantine followed EU norms and were generally good (OIE 2008). The project was to provide technical assistance and training for staff of veterinary service institutions at national and local levels on routine collection of samples and intensive follow- up around outbreak areas. It was to also provide technical assistance and training for epidemiological studies and surveillance programs. However, none of these activities were carried out. 4.5 Farmers’ incentives to report incidence of diseases: Romania has a system that pre-dates the project to compensate famers for culled birds. Compensation payments are subject to provisions of the national ―Compensation for Animal Diseases‖23 allocation under the Ministry of Agriculture budget (World Bank 2006c). A framework and procedures were effectively utilized during the 2005 and 2006 outbreaks for backyard poultry farmers. Compensation was made at locally determined market prices. The project did not make any contribution to the establishment of the compensation strategy, which was updated in 2009. 4.6 Measures such as culling taken in time to prevent infection from spreading among birds and other animals: Romania was able to effectively deal with the avian influenza outbreaks in poultry in 2005 and 2006. However, project documents at appraisal identified weaknesses in disinfection stations, in the culling practices for infected and at-risk poultry, and the safe disposal of poultry carcasses. The project did not contribute to improvements in these areas because animal sector related activities, including the planned procurement of CO2 containers and incinerators, were not implemented. The actions that were undertaken by the Government to prevent spread of infection in 2005 and 2006, before the project became effective, included quarantine of villages and restriction of movement of humans, mass culling of poultry, removal of poultry meat from the supermarket, and 23 Examples of animal diseases for which the fund can be used are avian influenza and swine fever. 18 restricting movement of poultry. These measures created panic and caused a huge loss to the poultry industry.24 Data obtained from the Union of Poultry Breeders of Romania show a decline in poultry meat prices after the 2005 outbreak. The enforcement of restrictions by the authorities also led to resentment against the government (Matei and others 2007). This hostility was clearly visible in the reaction that was seen amongst respondents to an IEG quick survey of about 50 people undertaken in two counties. 4.7 Overall the project made a negligible contribution to the achievement of this objective. Reduce the threat to the poultry sector by HPAI (Original objective) 4.8 The project contribution to this objective was also minimal, as the animal health component barely disbursed before it was canceled. The threat to poultry, particularly in backyard systems, remains. In fact, further outbreaks in backyard poultry in Romania were reported in 2007 and 2010. However, the population is aware of avian influenza. The IEG mission carried out an informal survey of about 50 people in two counties, Brasov and Tulcea. More than 95 percent of respondents knew about avian influenza, although several did not see it as a real danger and thought that the events in 2005 and 2006 were a scare spread by the government. It is not possible, however, to attribute the awareness created to the Bank project. The government took active measures to communicate information on the disease and communications campaigns in the country were supported through UNICEF and donors such as USAID. 4.9 Overall the project made a negligible contribution to the achievement of this objective. Prepare for, control, and respond to influenza pandemics and other infectious disease emergencies (Original and revised objective) 4.10 A human influenza pandemic arising from a mutation of the H5N1 virus did not materialize over the life of the project or since it closed, so it isn‘t possible to assess the ability to control an infectious disease epidemic. The government was successful in controlling the outbreak of H5N1 in birds in 2005 and 2006, before the project was launched, by enforcing quarantine of infected villages, school closure, and mass culling of birds to prevent spread of infection. A national program for surveillance, prevention, and eradication of animal diseases is in place in conformity with EU legislation. The project, however, did not contribute to the surveillance activities. A 2008 OIE evaluation found that border security and quarantine follow EU norms and are generally good (OIE 2008). 4.11 Biosecurity continues to be weak in backyard poultry systems. Over the years there have been recurrent outbreaks among poultry in Romania—the most recent in March 2010.25 24 It was estimated that avian influenza led to a decrease by 85 percent in poultry sales (Matei and others 2007). 25 ―An outbreak of avian influenza or bird flu has been confirmed in Romania, the European Union website said in a media statement on March 16, 2010. The European Commission will assess risk areas declared by the Romanian authorities in relation to the highly pathogenic avian influenza in a backyard poultry farm in the 19 Data from OIE show that countries around the world continue reporting occasional outbreaks of H5N1 in poultry and birds, and the risk of spread through migratory birds, remains.26 Because of the cancelation of the animal health component, the project did not make a contribution in helping reduce the risk of recurrent avian influenza outbreaks in birds. The risk of a human pandemic remains as long as conditions for the multiplication of the virus remain in poultry. 4.12 The project did help contribute towards building capacity in the country to prepare for and respond to influenza pandemics and other infectious disease emergencies in humans, however, by contributing to building intensive care capacity in seven regional and two central infectious disease hospitals. Equipment provided through the project included ventilation machines, monitors, furniture, intensive therapy accessories, clinical examination and treatment equipment, and laboratory and mobile examination equipment. IEG mission discussions with the Ministry of Public Health and hospital staff revealed that there were technical problems with some of the hospital beds that were imported and that considerable time and resources were lost in attempting to get them fixed by the supplier. The IEG mission visited the two central hospitals in Bucharest and one of the regional hospitals. The facility at the Matei Bals central hospital appeared to be effectively utilized. The other central hospital had not begun utilizing the intensive care facility established under the project, reportedly due to inadequate trained staff resources for running the facilities. The fully equipped intensive care unit at the regional hospital was not being used. 4.13 Annex C presents broad criteria for determining the level of preparedness and responsiveness, as gleaned from various WHO/FAO/OIE documents and discussions with FAO and OIE staff. The extent to which the project helped Romania prepare for, control and respond to influenza pandemics is assessed in the last column of the table against the broad criteria. Preparing for, controlling, and responding to influenza pandemics requires actions both on the human and animal health sides. It also requires recognizing and responding to the two different diseases that the virus can cause in birds and in humans. The initial project design had put emphasis on combining upgrading of the urgently needed capacities in the country for controlling avian influenza in poultry and preventing human infections with an increase in Romania‘s preparedness for a potential influenza pandemic. The project actually only supported actions on the human health side with minimal contributions on the animal health front. 4.14 Overall, achievement of this objective is assessed as modest. commune of Letea, in Tulcea county, at the Danube's delta near the border with Ukraine. Authorities said it was the H5N1 strain of avian influenza. To thwart the spread of the virus, all poultry has been culled in the infected farm and a protection zone set up within a three-km radius and a surveillance zone within a 10-km radius around the farm. According to the report, anywhere inside the 10km zone around the outbreak will be a critical or "high risk area", surrounded by a low risk area in the periphery. Authorities also said that certain restrictions on movements and additional bio-safety measures for poultry farms will apply.‖ The Sofia Echo March 16, 2010. 26 For an update of avian influenza outbreaks in countries around the globe see the OIE website at http://www.oie.int/index.php?id=1228 20 Build capacity to respond to potential future infections of humans by HPAI and other zoonoses (Revised objective) 4.15 Through building increased capacity for vaccine production, which could be used effectively during times of greater need (as it was during the H1N1 pandemic in 2009), the project helped build the capacity to respond to potential future infections in humans. The staff at the Cantacuzino Institute also received training on good manufacturing practice standards in vaccine production. Contingency plans for pandemic influenza were prepared and tested during the 2009 H1N1 outbreak. 4.16 The vaccine production unit at the Cantacuzino Institute was upgraded and the trivalent influenza vaccine production capacity is reported to have increased from 30 doses per week in 2008 to 79 doses per week in 2009. During the 2009 H1N1 pandemic, the Institute was able to significantly increase pandemic vaccine production (Figure 2).27 As of September 2011 the Institute could produce 8,000 vaccine doses per month. However, some rehabilitation work for the bulk vaccine production area and the ampoule sealing and filling station could not be completed during the project and was transferred to the then ongoing Health Sector Reform 2 project. 4.17 The project also helped build a BSL-3 laboratory at the Cantacuzino Institute. The IEG mission visited the laboratory and found that most of the equipment had been installed. However, some pieces of equipment, such as the glove box (a specialized bio-safety cabinet) and the decontamination system, were still to be procured and the laboratory had to be certified before it could be effectively utilized. 4.18 The equipping of intensive care facilities in seven regional and two central infectious disease hospitals has also helped build capacity in the country to deal with infections of humans by HPAI and other zoonoses, although being able to utilize the facilities effectively with adequate medical personnel remains a concern. 4.19 Overall, achievement of this objective is assessed as substantial. 27 The first case with the H1N1 virus in Romania was detected on 27 May 2009 and the Cantacuzino Institute was able to undertake such diagnosis. Reagents for molecular detection of pandemic H1N1 virus from the United States Centers for Disease Control and Prevention in Atlanta arrived at Cantacuzino Institute on May 5, 2009. Pandemic vaccine preparation activities commenced soon thereafter at the Institute. 21 Figure 2: Influenza Vaccine Production at the Cantacuzino Institute 2002-2011 2011 2010 2009 2008 2007 Pandemic vaccine production 2006 Seasonal vaccine 2005 production 2004 2003 2002 0 2,000,000 4,000,000 6,000,000 Source: Cantacuzino Institute 5. Efficiency 5.1 Assessment of efficiency involves considering the benefits in relation to the costs, cost-effectiveness, and efficient use of resources. It is very difficult to conduct a meaningful economic analysis of avian influenza control projects, because the likelihood and severity of avian influenza outbreaks among poultry, the likelihood of pandemics among humans, and the quantitative efficacy of mitigation efforts are largely unobservable and unknown. An economic analysis at project closure (World Bank 2011d) estimated an economic rate of return of 28-86 percent in a base case scenario, but this is based on a methodology with a number of weaknesses. The analysis uses real historic data on the costs of 2005-6 avian influenza outbreaks among poultry, but then relies on arbitrary and arguably implausible assumptions about the probability of further outbreaks (severe outbreaks occurring every 5 years) and the degree to which the project investments would reduce the risk of outbreak among birds or would reduce the impacts of an influenza pandemic (halving the infection rate among humans).28 The assumed impacts on infection rates among poultry (from 3 or 5 percent to 2 percent are particularly implausible, given the cancellation of the animal health component; no project activities were completed that could plausibly reduce the probability or severity of poultry sector outbreaks. 28 It is not clear why the ex post analysis used a 3 percent gross attack rate for the without project scenario when the Technical Annex (World Bank 2006a, page 86) confirms that the actual gross attack rate was 1.41 percent in 2005-2006. There is also a problem with the assumption of a severe outbreak with a mutated virus occurring every five years. In the 20th century there were three human influenza pandemics (1918, 1957, 1968) and there has been one so far in the 21st century (H1N1 in 2009) (see Annex B of this report, Table 1). 22 5.2 While the project implemented far less than was planned, it also expended only 55 percent of planned resources. Yet there is ample evidence that, as implemented, it was inefficient in design and was not implemented at least cost. In terms of design, both before and after the restructuring the project addressed only coping with the spread of avian influenza among humans once it occurred and preparation for future human outbreaks; it neglected to address preventing transmission to humans in the first place by reducing infections among poultry. In terms of least-cost implementation, there were serious delays in procurement, disbursement, and implementation of the ―emergency‖ operation, due in part to limited coordination between the human and animal health sides of the project (not facilitated by the Coordination Committee), frequent changes in the national veterinary agency‘s management and support for the project, frequent turnover of staff in the implementing agencies and their project management units. This left a lot of investments unfinished or not yet operational as of the end of the project, among them the rehabilitation of vaccine production at the Cantacuzino Institute, the procurement of the remaining equipment to make the BSL3 laboratory functional, the certification of the laboratory, and adequate technical and other staff capacity to actually effectively utilize the intensive care units at the communicable disease hospitals, among others. There is also the issue of quality of some of the beds procured for the regional hospitals and the two central hospitals, which has implications for efficiency of resource use. 5.3 Overall efficiency is assessed as modest. 6. Ratings Outcome 6.1 According to the harmonized evaluation criteria used by the World Bank and IEG, when a project‘s objectives are formally revised by a restructuring with the Board, the project outcome is assessed under the original objectives and the revised objectives, and the two outcome ratings are weighted by the share of the operation disbursed at the time of restructuring. The restructuring of this project took place in July 2009, at which time about 9 percent of the actual project cost had been disbursed the remainder being disbursed after the restructuring. 6.2 The original objectives and project design were substantially relevant to Romania, but achievement of two of the objectives – reducing the threat posed to humans by HPAI and other zoonoses and reducing the threat to the poultry sector – was negligible and of the third objective – to prepare for, control, and respond to influenza pandemics and other infectious disease emergencies – was modest. Efficiency of the project as implemented was modest. Under the original objectives, the project‘s outcome was unsatisfactory. 6.3 The restructuring that revised the objectives and components introduced weaknesses in the relevance of the objectives and design, reducing the relevance of both to modest. Achievement of the objective of building capacity to respond to potential future infections of humans by HPAI and other zoonoses was substantial, but modest for the other objective, to prepare for, control, and respond to influenza pandemics and other disease emergencies. 23 Efficiency was also modest. Thus, also under the revised objectives, the project‘s outcome is assessed as unsatisfactory. 6.4 As the outcome rating under both the original and revised objectives is the same, when weighted by the share of the operation that had been disbursed, the overall outcome rating is also unsatisfactory, denoting major shortcomings in relevance, efficacy, or efficiency. Risk to Development Outcome 6.5 The difficult economic conditions in Europe have tightened budget and resource constraints in Romania. Project documents show that the share of the Ministry of Public Health budget earmarked for projects funded by international organizations was already lower in 2010, the last year of project implementation, than the amount initially requested by the Ministry‘s project management unit. The government has begun a comprehensive but difficult medium-term reform program for the public sector to balance revenue and spending and enhance efficiency of service delivery but the fiscal situation remains tight (World Bank 2010a). Under these circumstances, resources for hiring staff, training, and operation and maintenance remain very limited, with consequences for the sustainability of services from the project‘s investments. 6.6 The intensive care units were constructed and equipped in the seven regional and two central communicable disease hospitals. However, the hospitals are finding it difficult to utilize them because of lack of adequate trained personnel and doctors. There has been a recruitment freeze in public sector employment and cuts in personnel costs as part of the measures to contain the financial crisis (World Bank 2010b). Following the recent decentralization of hospitals, managers of the regional hospitals, including those on communicable diseases, report to the district councils and city halls and this has introduced a new complexity that would influence the overall contingency plan for dealing with emergencies and pandemics. 6.7 Several budget-related challenges are also being faced by the Cantacuzino Institute. The institute does not receive any budget transfer from the Ministry of Public Health. It is dependent on its own income primarily from sale of vaccines and payments for tests performed in its laboratories, but also to a lesser extent research and graduate and postgraduate teaching in microbiology. While the project contributed to the building and equipping of the BSL3 laboratory at the institute, the laboratory was not completed, as the decontamination system and the glove box (a specialized bio-safety cabinet) had not been built. The World Bank reported that the laboratory was completed and certified in late 2011 and that the facility is fully capable, but has not been used, as there have been no further outbreaks of avian influenza. However, the BSL-3 laboratory provides additional capacity to undertake molecular diagnosis should avian influenza or other serious infectious diseases arise. It is unclear whether there are sufficient resources to sustain the facility. BSL3 laboratories are more expensive to operate and maintain (among other things the need for maintaining negative pressure, generators to ensure continuous electricity supply, and regular decontamination). Resources for kits, reagents, and other basic consumables will be an additional requirement. In addition, there is need for continuous training for taking adequate 24 precautions when dealing with highly dangerous pathogens in high containment biological research laboratories (BSL3 laboratories for example).29 6.8 As regards vaccine production, IEG mission conversations with technical staff at the Cantacuzino Institute revealed that the work related to the building of the shell and the upgrading of the machine at the filling line was essential, as without it future vaccine production at the Institute would be jeopardized. While this was not completed by project closure, it was transferred to the Health Sector Reform (Adaptable Program Loan 2) project. This work had not been completed by the time of the IEG mission in September 2011, but the World Bank reports that this work was completed in 2012. As of June 2013 final tests and assessments for certification of good manufacturing practice were still in progress, so vaccine production had not yet restarted. 6.9 Overall risk to development outcome is assessed as significant. Monitoring and Evaluation 6.10 Design and implementation. As noted in earlier sections, the M&E design before the restructuring incorporated three appropriate outcome indicators, though not specific enough to distinguish the contribution of the project. After restructuring, the outcome indicator of reduced ―risk‖ of human infections remained, though difficult to quantify. Implementation of the M&E system was delayed and rated in the unsatisfactory range for most of the project‘s life. Most of what was collected was in terms of outputs. 6.11 Utilization of the data. Overall the usefulness of the project M&E system to project management was limited. First, because of the restructuring, the project objectives and most of the outcome indicators changed mid-stream and therefore there were only limited areas where results could be monitored from start to finish. Second, there was limited reporting on M&E in the early project years. Finally, the opportunity to make the M&E more relevant to the project was missed at the restructuring. 6.12 Overall, project M&E is rated negligible. 29 ―At the workshop, participants discussed similarities and differences in infrastructure, oversight and personnel training between bio-safety level 3 (BSL-3) and bio-safety level 4 (BSL-4) laboratories. There was clear consensus among the participants that hands-on proficiency training, mentorship, and didactic training are critical for establishing and evaluating the researcher‘s ability to work in a high-containment laboratory; a variety of individuals, from researchers to administrators and support staff to equipment service personnel, require some level of training before gaining access to high-containment laboratories; bio-safety training programs have to be flexible to account for the research (one size does not fit all), model systems, facilities, and job function; and the average cost of personnel training varies from $4000-$7000 for researchers to hundreds of dollars to $4000 for training non-scientists, like administrators.‖ (AAAS 2009). For differences in levels of BSL2, BSL3 and BSL4 laboratories see Annex B. 25 Bank Performance QUALITY AT ENTRY 6.13 To meet the challenges of containing avian influenza and the threat of a pandemic, the project as originally designed had three features that promised a coordinated response:  First, it provided for the animal health and food safety agency and the human health ministry to come together in one intervention and allowed for a holistic approach;  Second, recognizing the difficulty in getting staff from different sectors to work together, it realistically provided for a multi-sector, multi-agency Steering Committee that also included the Ministry of Economy and Finance;  Finally, through a horizontal adaptable program loan, the country-level effort was to be coordinated with the global effort to control avian influenza and involved cooperation between several donors and the World Bank. 6.14 The Bank responded with alacrity to the Borrower‘s request for support, moving from concept review to approval in 10 weeks. However, effectiveness took another 6 months (a significant delay beyond the World Bank standard for an emergency operation) because the Romanian Parliament was not in session.30 6.15 The Bank did not take the tremendous ongoing accession-related changes in Romania into account when advocating the emergency intervention. Such consideration would have made it clear that Romania did not actually need support to tackle the immediate outbreaks of avian influenza, which had already been addressed. What the government asked for was support for building capacity for the future. An emergency loan – even if related to the themes of the HPAI horizontal adaptable program loan – was not an appropriate instrument for building long-term capacity.31 Further, the project as designed included civil works that needed special permits, vaccine production that needed licensing, and purchase of equipment for hospitals that is generally time-consuming; the amount of time needed for these approvals and clearances made it unsuitable for an emergency operation. Ultimately, the project that was designed was complex and ambitious, with a longer time horizon and too many sub- components for an emergency operation. 6.16 The implementing arrangement for the project with two separate implementing agencies, two project management units, and a Steering Committee did not work effectively. The Bank reports that it had strong commitment from the national veterinary agency at entry, 30 In Romania a World Bank loan can only become effective after the passage by the Parliament of the ratification law, the promulgation by the President, and the publication in the Official Gazette. Endorsements by the Ministry of Public Finance, the Ministry of Justice, and the Ministry of Foreign Affairs are a prerequisite for submission to the Parliament (Joint Portfolio Review 2005). The Romanian government did not choose to use emergency parliamentary procedures to approve the project loan and hasten project effectiveness, suggesting they may also have recognized that the project was not addressing an emergency. 31 In fact, project files reveal that the Ministry of Economy and Finance had actually approached the Bank informally for a ―research‖ project. 26 and that commitment declined only after approval with a change of management and staff in the agency. But this risk was not entirely unforeseeable: Romania has historically had a highly politicized bureaucracy, and this contributed to several changes in top leadership at the national veterinary agency, which then led to waning interest and lack of implementation. 6.17 Preparation paid too little attention to coordination of project activities with other sector interventions. For example, the World Bank was supporting the Modernization of Agricultural Knowledge and Information System project, approved in FY05, whose objective was to improve the competitiveness of farmers and agro-processors while helping them meet EU standards. Similarly, in the health sector the Bank was supporting the second phase of a health adaptable program loan, also approved in FY05, which, among other things, provided for purchase of medical equipment in hospital emergency health rooms. 6.18 It is also not clear whether the project design was well coordinated with other government and donor programs on zoonotic diseases or food safety improvements, such as those supported by the European Union Phare program.32 Other serious shortcomings at entry included weak design of monitoring and evaluation. 6.19 Overall quality at entry is rated moderately unsatisfactory. QUALITY OF SUPERVISION 6.20 Project implementation experienced significant delays because of serious problems in institutional decision-making in the implementing agencies. The project became effective on March 19, 2007 and as of November 30, 2007, EUR 0.27 million of the Bank loan (i.e. 0.9 percent) had been disbursed. Early supervision missions spent considerable time discussing with project counterparts, measures to increase efficiency of decision-making in implementing agencies and speed of project implementation. However, they were not very successful in increasing the disbursement rate until after the restructuring. The IEG mission found that there was concern in the national veterinary agency that the Bank was too rigid in not agreeing to changing priorities. Project files reveal that the task team tried to be flexible but also had to keep the emergency intervention on track. The Bank tried to use suspension of disbursements to encourage the veterinary agency to act, but this was unsuccessful. The Bank was proactive in helping restructure the project and cancel the animal health component that was not performing. However, inadequate attention was given to the quality of monitoring and evaluation and its use for decision-making during project implementation. 6.21 As initially designed, the project involved Bank staff from both the agriculture and health sectors, with the task management being with agriculture. After the restructuring, with the elimination of the animal health component, task management was transferred to health. Interviews with task team members suggest that task management of the project with the agriculture sector during the initial project period posed a challenge in terms of incentives for 32 The functional review of the health sector in Romania (World Bank 2011a) found that special health programs have grown much faster than any other item in the health budget and now account for about one-fifth of the total health expenditures and 2.3 times larger than the total budget for primary care. The program for communicable diseases such as HIV/AIDS and tuberculosis accounts for about 10 percent of the budget for national programs. 27 the Bank health sector staff and there was learning in getting the two sectors to work together. 6.22 Quality of supervision is rated moderately satisfactory. 6.23 Overall Bank performance is rated moderately unsatisfactory. Borrower Performance GOVERNMENT PERFORMANCE 6.24 The Government, with the support of the international community, mounted a rapid response to contain the outbreaks and, among other things, requested Bank support to increase Romania‘s avian influenza preparedness and the response capacity in its human and animal health institutions. Part of the motivation came from Romania‘s aspirations to be part of the EU; following accession in January 2007, interest in the project and in undertaking other reforms waned. In fact, in the national veterinary agency‘s priorities changed completely with a new President after the accession. 6.25 Interviews with government officials and other stakeholders in the field reveal that as it approached accession, Romania also went through a process of tremendous policy change and reform that stretched the capacity of government agencies and significantly affected the country‘s capacity to absorb resources and implement development projects. The Joint Portfolio Review in 2008 found that the performance of development projects, deteriorated after 2005. The Bank also gradually declined in importance as a player; with the accession Romania acquired access to a wide array of financial resources.33 To some extent the project was caught in these changing times. 6.26 There were some country-specific factors that existed across the lending portfolio that adversely influenced the performance of the project. One of these was the weakness in the planning, programming, budgeting, and execution of project activities arising from inadequate public financial management, slow approval of projects and high turn-over of staff in the country (Joint Portfolio Review 2008). Another generic issue was related to the implementation arrangements for World Bank projects in Romania. Bank-supported projects have been implemented through project management units because of in-country capacity constraints. There are salary differences between management unit and Ministry/agency staff. The project management units are generally staffed by experts who move from one project to the other and typically do not belong to a particular implementation agency. Hence, if project implementation is not proceeding well, the project management unit Director and staff may move on to a management unit for another project. The 2005 Joint Portfolio Review had already noted the need for streamlining project management unit implementation arrangements. In the avian influenza project, particularly on the animal health side, the 33 ―Romania successfully joined the European Union on January 1st 2007 and now enjoys access to a wide array of financial resources either through the financial market—domestic and international thanks to the investment grade status achieved in 2004, through potentially significant fiscal transfers from the European Union in the form of structural and cohesion funds as well as from the Common Agricultural Policy, or through international financial institutions of which the World Bank is only one.‖ Joint Portfolio Review 2008. 28 management unit staff found it difficult to work in the highly centralized decision making structure that characterized the national veterinary agency and consequently staff turnover was a major constraint. 6.27 There was limited coordination between the human and animal health project management units and the fact that the Steering Committee did not actively support coordination of project activities was also a serious concern. In fact, the first meeting of the Steering Committee did not happen until December 2007, more than a year after the project became effective. 6.28 IEG mission discussions with government officials revealed that relations between the Ministry of Economy and Finance and the national veterinary agency were also difficult and communication between them was weak. Project files and discussion with staff revealed that the national veterinary agency did not move ahead with implementation of the original project design for a while because it was expecting the Ministry of Economy and Finance to agree to their changed priorities. However, this did not happen. 6.29 Government performance is rated moderately unsatisfactory. IMPLEMENTING AGENCY PERFORMANCE 6.30 The national veterinary agency was a serious constraint to project implementation. It had been established in 2004 as the main body in charge of food safety and animal health in the country. It reported directly to the Prime Minister‘s office, yet the precise distribution of responsibilities and the chain of command within the Agency were not clear. There were also uncertainties about its ministerial affiliation, as its responsibilities were covered partly by the Ministry of Agriculture and Rural Development and partly by the Ministry of Public health. These organizational issues contributed to a lack of clarity in decision making processes and chains of command that reduced project performance. 6.31 Several other aspects of the National veterinary agency‘s performance severely constrained project implementation and made restructuring essential: there were frequent changes in the top management at the agency that led to shifts in priorities and lack of support for the project as designed; the highly centralized decision-making process, hierarchical structure and managerial weaknesses within the agency made it difficult for the project management unit to get contracts approved; there was limited connectivity of the project management unit to the national veterinary agency institution; disagreements between the agency‘s legal and technical departments and the Bank made it difficult for procurement packages to be prepared; relations between the agency and the Ministry of Economy and Finance were difficult and the agency could not get support from the latter for its changed priorities. The frequent turn-over of staff in the management unit made things worse. In fact, the implementation of both the Modernization of Agriculture and Rural Development and the avian influenza projects were affected by the Agency‘s institutional weaknesses and lack of familiarity with Bank projects.34 Staff turnover was also an issue at 34 Implementation of activities related to the National Agency were discontinued in the Modernization of Agricultural Knowledge project as well (World Bank 2010a) 29 the Ministry of Public Health and its project management unit, adversely affecting implementation. 6.32 Ministry of Public Health commitment and performance was overall strong, although the frequent change in leadership in the Ministry negatively affected implementation. There were also budget-related challenges that arose because of the financial crisis that made it difficult for the government to provide counterpart funding in time. The ministry‘s project management unit carried out regular reconciliation of accounts and used internal control procedures in accordance with Romanian legislation. However, procurement activities were significantly behind schedule and delays had led to several unfinished tasks in the Cantacuzino Institute before project closure. 6.33 Implementing agency performance is rated moderately unsatisfactory. 6.34 Borrower performance is rated moderately unsatisfactory. 7. Lessons 7.1 Building on the project experience this assessment identifies the following lesson: 7.2 Countries that have already effectively controlled avian influenza outbreaks in poultry many not need an emergency project, but rather long term capacity building. In this case, Romania controlled avian influenza outbreaks in 2005, 2006, and 2010, and already had a passive surveillance system, centralized diagnostic capacity, and a functioning veterinary service agency. The real emergency issue was EU accession, but after accession was achieved the government lost interest in the project. 7.3 The evaluation also notes that following this project, avian influenza is still a threat to Romania. Avian influenza risks cannot be minimized without ensuring adequate capacity in the animal health sector and adequate biosecurity in the poultry sector. Interventions that target only the human health sector will miss the most effective way of reducing the risk to humans – by controlling the disease among poultry. Project designs that do not improve biosecurity on commercial farms or backyard systems will miss an opportunity to reduce the likelihood of avian influenza outbreaks. 31 References American Association for the Advancement of Science. 2009. ―Biological Safety Training Programs as a Component of Personnel Reliability‖. Workshop Report, American Association for the Advancement of Science. Blackwell, Mark. 2010. ―The Benefits of biosecurity in poultry farming.‖ International Poultry Production 12(6), pp.7-9. Centers for Disease Control and Prevention 2010. ―Key Facts About Avian Influenza (Bird Flu) and Highly Pathogenic Avian Influenza A (H5N1) Virus – Fact Sheet.‖ www.cdc.gov/flu/avian/gen- info/facts.htm accessed August 2011. Atlanta, Georgia, USA. Central Intelligence Agency. 2011. World Fact Book Romania Economy, https://www.cia.gov/library/publications/the-world-factbook/geos/ro.html European Commission 2009a. ―Final Country Profile of Romania on Food and Feed Safety, Animal health, Animal Welfare and Plant Health.‖ European Commission Health and Consumers Directorate General. —―. 2010. ―Final Report of a Specific Audit Carried out in Romania From 17 to 26 May 2010 In order to Evaluate the Import/transit Control System and Border Inspection Posts in the Context of a General Audit ― European Commission Health and Consumers Directorate-General DG (SANCO) 2010-8560. —―. 2011a. ―Final Report of an Audit Carried out in Romania From 11 to 20 May 2011 In order to Evaluate the Food Safety Control Systems in Place Governing the Production and Placing on the Market of poultry Meat and Poultry Meat Products.‖ European Commission Health and Consumers Directorate- General DG (SANCO) 2011-8857. —―. 2011b. ―Final Report of an Audit Carried out in Romania From 07 to 18 February 2011 In order to evaluate the Implementation of health Rules on Animal By-Products.‖ European Commission Health and Consumers Directorate-General DG (SANCO) 2011-8941. —―. 2011c. ―Summary Record of the meeting of the Standing Committee on the Food Chain and Animal Health Held in Brussels on !6 March 2011.‖ Brussels. Embassy of Netherlands. 2009. ―The Romanian Food Sector and the use of EU Funds for Investments.‖ FAO 2006. ―Strengthening Veterinary and Public Health Surveillance Systems.‖ Mission Report —―. 2007a. ―Risks Associated with Poultry Production Systems.‖ Background paper for the FAO sponsored International Conference entitled ―Poultry in the 21st Century – Avian Influenza and Beyond‖. Bangkok 5-7 November. —―. 2007b. ―Animal Welfare in Poultry Production Systems: Impact of European Union Standards on World Trade.‖ Background paper for the FAO sponsored International Conference entitled ―Poultry in the 21st Century – Avian Influenza and Beyond‖. Bangkok 5-7 November. —―. 2007c. ―OIE Standards and guidelines related to trade and poultry diseases. ‖ Background paper for FAO-sponsored International Conference entitled ―Poultry in the 21st Century – Avian Influenza and Beyond‖. Bangkok 5-7 November. —―. 2007d. ―Veterinary Services for Poultry Production.‖ Background paper for the FAO-sponsored International Conference entitled ―Poultry in the 21st Century – Avian Influenza and Beyond‖. Bangkok 5- 7 November. —―. 2010. ―Smallholder Poultry Production—Livelihoods, Food Security and Sociocultural Significance.‖ FAO Smallholder Poultry Production. Paper No. 4. Rome —―. 2011. ―H5N1 Global Overview January/March 2011.‖ FAO Emergency Prevention System FAO-OIE-WHO 2010. ―Sharing Responsibilities and Coordinating global activities to address health risks at the animal-human-ecosystems interfaces: A Tripartite Concept Note.‖ April. —―. 2011a. ―Current evolution of avian influenza H5N1 viruses.‖ Technical Update. 32 —―. 2011b. ―Influenza and other Emerging Zoonotic Diseases at the Human-Animal Interface.‖ FAO/OIE/WHO Joint Scientific Consultation 27-29 April 2010 Verona (Italy) Proceedings Government of Romania 2005. National Development Plan 2007-2013, Bucharest —―. 2006. ―National Plan for Agriculture and Rural Development over the 2000-2006 period under the EU Special Accession Program for Agriculture and Rural Development (SAPARD)‖ Bucharest Independent Evaluation Group 2005. Romania Country Assistance Evaluation. Report No. 32452. Washington, DC: World Bank. Ilie, Anca and Rodica Zaharia. 2007. ―Common Agricultural Policy. The implementation of the Acquis Communiautarie in Romanian Agriculture‖ Romanian Economic Journal Year X(23). June. IMF 2010. Country Report No. 10/227. Romania --Staff Report for the 2010 Article IV Consultation, Fourth Review Under the Stand-By Arrangement, and requests for Modification and Waiver of Nonobservance of Performance Criteria —―. 2012. IMF Country Report No. 12/290. Romania: Staff Report for the 2012 Article IV Consultation, Sixth Review Under the Stand-By Arrangement, and Requests for Waiver of Nonobservance of Performance Criterion and Modification of Performance Criteria Joint Portfolio Review 2005. Romania Government of Romania and the World Bank. Joint Portfolio Review 2008. Romania Government of Romania and the World Bank. Lazzari, Stefano, and Klaus Stohr 2004. ―Avian influenza and influenza pandemics editorials‖ Bulletin of the World Health Organization 82 (4), pp. 242-242 Lee, Vernon J., Jonathan Yap, Alex R. Cook, Mark I. Chen, Joshua K. Tay, Ian Barr, Anne Kelso, Boon Huan Tan, Jin Phang Loh, Raymond Lin, Lin Cui, Paul M. Kelly, Yee Sin Leo, Kee Seng Chia, Wee Lee Kang, Paul A. Tambyah, and Benjamin Seet. 2010. ―Effectiveness of Public Health Measures in Mitigating Pandemic Influenza Spread: A Prospective Sero-Epidemiological Cohort Study.‖ Journal of Infectious Diseases 202(9):1319–1326. Matei , Gabriela, M. Decun, and G. Ontanu. 2007. ―A Comparative Risk Consequences Assessment for Avian Influenza Outbreaks Occurred in Romania.‖ Lucrari Stiintifice Medicina Veterinara XL Timisoara. McCrea, Brigid A. and Francine A. Bradley, 2008. ―Biosecurity for Poultry at Community Farms‖, Publication 8280, Division of Agriculture and Natural Resources, University of California Davis. McKibbin, Warwick, and Andrew Stoeckel. 2006. ―The Financial Effects of a Bird Flu Triggered Pandemic: Economic Scenarios.‖ Centre for International Economics and McKibbin Software Group Pty Limited. Moldovan, Minodora. 2010. ―The Romanian Meat Sector—Characteristics and Post-Accession Assessments.‖ Agricultural Economics and Rural Development New Series VII(1): 141-159 Monto, Arnold S., Lorraine Comanor, David K. Shay, and William W. Thompson. 2006. ―Epidemiology of Pandemic Influenza: Use of Surveillance and Modeling for Pandemic Preparedness.‖ Journal of Infectious Diseases 194:S92–7. OIE 2008. ―Performance of Veterinary Services Evaluation Report on the Veterinary Services of Romania, World Organization for Animal Health‖, Paris Otiman Paun Ion, Filon Toderoiu, Violeta Florian, Cecilia Alexandri, Camelia Gavrilescu, Iuliana Ionel, Mariana Grodea, Cornelia Alboiu, Minodora Moldovan, Vasile Gosa, Andrew Nagy. 2010. Agriculture and Rural Development in Romania—The Mian Modalities to Attenuate the Crisis Effects and to Resume Economic Growth. Agricultural Economics and Rural Development New Series. Year VII (2):163-211. Petric, Martin, Lorraine Comanor, and Cathy A. Petti 2006. Role of the Laboratory in Diagnosis of Influenza during Seasonal Epidemics and Potential Pandemics. Journal of Infectious Diseases 194:S98– 110. Union of Poultry Breeders in Romania 2011. Buletin Informativ August. 33 United Nations and the World Bank 2010. ―Animal and Pandemic Influenza A framework for Sustaining Momentum.‖ Fifth Global Progress Report. UN System Influenza Coordination and the World Bank UNICEF. 2008. ―A Specialist for the left kidney and a Specialist for the right kidney: A preliminary assessment of health promotion and communication capacity in eight CEE/CIS countries .‖ Whitney, Richard J. and Arnold S. Monto. 2006. ―Seasonal and Pandemic Influenza Preparedness: A Global Threat.‖ Journal of Infectious Diseases 194:S65–9. World Bank 2005a. ―Program Framework Document for Proposed Loans/Credits/grants in the Amount of US$500 million equivalent for a Global Program for Avian Influenza Control and Human Pandemic Preparedness and Response.‖ Report Number 34386,Washington, DC. —―. 2005b. ―Romanian Food and Agriculture from a European Perspective.‖ ECSSD Environmentally and Socially Sustainable Development Working Paper No. 39. Washington, DC. —―. 2006a. ―Technical Annex on a Proposed Loan in the Amount of EUR 29.6 million (US$37.7 million equivalent) to Romania for an Avian Influenza Control and Human Pandemic Preparedness and Response Project Under the Global Program for Avian Influenza and Human Pandemic Preparedness and Response. ‖ Report No. T7679-RO. Washington, DC. —―. 2006b. ―Country Partnership Strategy for Romania for the Period FY06-FY09.‖ Report No. 36147- RO. Washington, DC. —―. 2006c. Romania - Avian Influenza and Human Pandemic Preparedness and Response Project : environmental assessment, E1558, Bucharest —―. 2006d. Enhancing Control of Highly Pathogenic Avian Influenza in Developing Countries through Compensation Issues and Good Practice. World Bank, FAO, IFPRI and OIE. —―. 2007a. ―Romania Poverty Monitoring Analytical and Advisory Assistance Program.‖ Report No. 40120-RO. Washington, DC. —―. 2007b. ―Romania Quarterly Progress Review.‖ Portfolio Note. Washington, DC. —―. 2007c. ―Status of Projects in Execution –FY07.‖ Europe and Central Asia Region, Romania, Operation Policy and Country Services, Report No. 50728. Washington, DC. —―. 2008. ―Status of Projects in Execution –FY08.‖ Europe and Central Asia Region Country: Romania Operation Policy and Country Services, Report No. 49723. Washington, DC. —―. 2009a. Status of Projects in Execution –FY09 Europe and Central Asia Region Country: Romania Operation Policy and Country Services. Washington, DC. —―. 2009b. ―Romania - Avian Influenza Control and Human Pandemic Preparedness and Response Project : Restructuring Project paper.‖ Report Number 49458. Washington, DC. —―. 2009c. ―Country Partnership Strategy for Romania for the Period July 2009-June 2013.‖ Report No. 48665 – RO. Washington, DC. —―. 2010a. ―Status of Projects in Execution –FY10.‖ Europe and Central Asia Region Country: Romania Operation Policy and Country Services, Report No. 57433. Washington, DC. —―. 2010b. Romania Public Expenditure and Institutional Review, Volume I Main Report Washington, DC. —―. 2010c. Romania Public Expenditure and Institutional Review, Volume II Background Papers. Washington, DC. —―. 2010d Agricultural and Rural Development Administration Romania Functional Review Final Report. Europe and Central Asia Region, Washington, DC.. —―. 2011a. World Bank—Romania Partnership Program Snapshot. Washington, DC. —―. 2011b. Romania Functional Review of the Health Sector —―. 2011c. Quarterly Portfolio Review Q3-FY2011. 34 —―. 2011d.‖ Implementation Completion and Results Report on a loan to Romania for an Avian Influenza Control and Human Pandemic Preparedness and Response Project under the Global Program for an Avian Influenza and Human Pandemic Preparedness and Response (GPAI). ‖ Report No: ICR00001911, Washington, DC. WHO 2005. ―Pandemic Influenza Preparedness and Response.‖ Global Influenza Programme. Geneva. —―. 2006. ―Influenza research at the human and animal interface‖. Report of a WHO working group. Geneva, Switzerland 21–22 September 2006 —―. 2008. Pandemic influenza preparedness and mitigation in refugee and displaced populations. WHO/HSE/EPR/DCE/2008.3. Geneva. 35 ANNEX A Annex A. Basic Data Sheet ROMANIA: AVIAN INFLUENZA CONTROL AND HUMAN PANDEMIC PREPAREDNESS AND RESPONSE PROJECT (LOAN IBRD -48390) Key Project Data (amounts in EUR million) Appraisal Actual or Actual as % of estimate current estimate appraisal estimate Total project costs 37.45 20.53 54.82 Loan amount 29.60 17.98 60.74 Cancellation - 11.63 - Cumulative Estimated and Actual Disbursements FY07 FY08 FY09 FY10 FY11 Appraisal estimate (EUR 12.48 19.77 29.60 29.60 29.60 million) Actual (EUR million) 0.07 0.49 1.85 5.49 17.98 Actual as % of appraisal 0.56 2.47 6.25 18.54 60.74 Date of final disbursement: May 2011 Project Dates Original Actual Concept Note 06/01/2006 06/29/2006 Negotiations 06/01/2006 06/29/2006 Board approval 09/21/2006 09/08/2006 Signing - 10/05/2006 Effectiveness 02/02/2007 03/19/2007 Closing date 12/31/2009 12/31/2010 Staff Inputs USD Thousands (including travel and Lending consultant costs) FY06 81.46 FY07 55.08 Total: 136.54 Supervision/ICR FY07 81.23 FY08 106.78 Total: 188.01 ANNEX A 36 Task Team Members Names Title Unit Mathias Grueninger Senior Agriculture Economist ECSS1 Vladimir Savic Consultant ECSSD Andriy Rozstalnyy AI Coordinator FAO Francois Decaillet Lead Public Health Specialist ECSHD Victor Olsavszky Public Health Specialist WHO Frans Stobbelaar Consultant ECSHD Codruta Hedesiu Communication Officer UNICEF Holger Kray Senior Sector Economist ECSSD Goetz Schreiber Lead Economist ECSSD Irena Ramniceanu Economist ECSSD Stefan Nicolau Consultant ECSSD Violeta Visan Consultant ECSSD Radhika Srinivasan Senior Social Scientist ECSSD Nicholay Chistyakov Senior Finance Officer LOAG1 Jean-Charles de Daruvar Senior Counsel LEGEC Hans-Juergen Gruss Chief Counsel LEGEC Rohan Selvaratnam Senior Program Assistant ECSSD Nurul Alam Senior Procurement Specialist ECSO2 Nadia Badea Operations Analyst ECSS1 Brian G. Bedard Sr Livestock Spec. ECSS1 Irene Bomani Senior Program Assistant ECSSD Bogdan Constantin Constantinescu Sr Financial Management Specialist ECSO3 Ruxandra Costache Counsel LEGEM Agnes Couffinahl Senior Health Economist ECSH1 Amy Evans Consultant ECSS1 Richard Florescu Senior Operations Officer ECSH3 Camelia Gusescu Program Assistant ECCRO Ana Maria Ihora Program Assistant ECCRO Vladislav Krasikov Senior Procurement Specialist EAPPR Carmen Laurente Program Assistant ECSHD Gabriela Doina Manea Resource Management Analyst HRSRM Wezi Marianne Msisha Health Specialist ECSH1 Cesar Niculescu Environmental Specialist Doina Petrescu Senior Rural Development Speci ECSS1 Dan Ioan Sava Consultant ECSPE Barbara Ziolkowska Procurement Analyst ECSO2 Rob Coenraad de Rooij Consultant ECSSD Anneliese Viorela Financial Management Specialist ECCRO 37 ANNEX B Annex B. Additional Documentation Annex B1: Influenza and Avian Influenza (AI).35 Influenza viruses: Influenza in humans can be caused by type A, B or C viruses, with the former two causing most human infections. Influenza A viruses naturally infect humans, as well as such animals as birds, pigs, and horses, and they generally cause yearly epidemics and, potentially, pandemics. Infections with influenza B virus are generally restricted to humans and cause epidemics more rarely. Avian Influenza in birds: Avian influenza is caused by A type viruses that occur naturally among wild birds that usually do not exhibit symptoms. It is very contagious among birds. Infected birds shed the virus in their saliva, nasal secretions, and feces. Domestic birds such as chickens, ducks and turkeys can become infected through contact with infected wild birds, from other infected domestic birds, or through contact with dirt in cages, water or feed that has been contaminated. Infection in domestic birds can result in either a low pathogenic form of the disease which may go undetected because of mild, not very obvious symptoms (such as ruffled feathers or a drop in egg production) or a highly pathogenic form (where multiple organs get affected) that spread rapidly with mortality rates that can reach 90 - 100 percent within 48 hours. An example of the latter is infection with the H5N1 virus. The subtypes differ based on differences in two main proteins on the surface of the virus (hemagglutinin [HA] and neuraminidase [NA]). There are 16 known HA subtypes and 9 known NA subtypes of influenza A viruses. Many different combinations of HA and NA proteins are possible. Each combination represents a different subtype. Human infections with avian influenza viruses: The virus does not normally infect humans but can infect them if they come in contact with infected poultry or contaminated surfaces. Symptoms in humans range from usual human influenza-like symptoms (example, fever, cough, sore throat, and muscle aches) to eye infections, pneumonia, severe respiratory diseases (such as acute respiratory distress), and other life-threatening complications. The symptoms and severity vary depending on the virus type. The highly pathogenic H5N1 virus that has recently been circulating in poultry does not as yet readily circulate in humans. However, where human infections have occurred in contact with infected poultry the virus has been deadly. As of March 16, 2013, the laboratory-confirmed human toll stood at 622 cases including 371 fatalities, but the reported human instances of the disease from contact with infected birds understate the true number of infected people. Although disease awareness has increased, cases are still likely to be underreported. Pandemic in humans with the avian influenza virus: Annual or seasonal influenza epidemics are caused by the previous seasons‘ viruses or by ones with slight antigenic changes. By contrast a pandemic is caused by an A virus that contains hemagglutinin (HA) for which there is no preexisting immunity, facilitating the virus‘s rapid spread throughout the world. Through re-assortment or mutation the virus could become capable of replicating 35 Source for this information: Source: CDC 2010, McKibbin and Stoeckel (2006), United Nations and the World Bank 2010, World Health Organization 2005, Lazzari and Stohr 2004, Whitney and Monto 2006, Monto and others 2006, Petric and others 2006 ANNEX B 38 in humans and spreading easily among them. Mutation occurs when a virus changes its genetic makeup allowing it to transmit from person to person, whereas re-assortment occurs when two different viruses exchange genetic material. The re-assortment could happen if animals that are susceptible to both human and avian flu, such as pigs, become concurrently infected with both kinds of flu. The 2009 H1N1 pandemic was caused by a virus that emerged from pigs. H3N2, H1N1, and H2N2 are examples of avian influenza virus types that have, in the past, circulated among humans. Studies confirm that the 1918-19 Spanish flu likely originated from the re-assortment of avian and human viruses. Since the 1918 pandemic, the population of the world has grown 3-fold; therefore, if a new influenza A virus is as lethal as the 1918 virus, as many as 180–360 million people could die. The 1957 H2N2 and 1968 H3N2 pandemics also involved new influenza viruses generated by re-assortment events, and, although they were less virulent than the 1918 H1N1 virus, they still caused significant morbidity and mortality. Both of these viruses resulted from re-assortment of a Eurasian wild waterfowl virus with a previously circulating human H1N1 virus. Annex B Table 1: The three pandemics of the last century Pandemic date & Area of Influenza Viral Change Estimate Estimated Age groups common name Emerge A Virus d case attributable excess most affected nce Subtype fatality mortality rate worldwide 1918-19 ―Spanish Unclear H1N1 Mutation from 2-3% 20-50 million Young adults Flu‖ avian virus, in HA gene 1957-58 ―Asian Flu‖ Souther H2N2 Reassortment of 3 <0.2% 1-4 million Children n China segments from avian virus (HA, NA, and polymerase protein) in combination with 5 genetic segments of the H1N1 virus 1968-69 ―Hong Kong Souther H3N2 Reassortment of 2 <0.2% 1-4 million All age Flu‖ n China segments from groups H2N2 virus replaced by H3 and polymerase protein Source: CDC 2010, McKibbin and Stoeckel (2006), United Nations and the World Bank 2010, World Health Organization 2005, Lazzari and Stohr 2004, Whitney and Monto 2006, Monty and others 2006, Petric and others 2006. For a pandemic to occur from an avian influenza virus, three conditions would need to be met: (i) a new subtype would have to emerge for which there is little human immunity; (ii) it would have to infect humans and cause illness; (iii) it would have to spread easily and sustainably among humans. To date the third condition has not been met because the virus has not improved its transmissibility among humans. The concern is that, with some re- assortment of genes (through co-infection with two viruses) or adaptive mutation (that could 39 ANNEX B increase the capability of the virus to bind to human cells), this could occur. Resistance in current virus strains to one of the two classes of available antiviral drugs as demonstrated in vitro has added to anxiety about Annex B Figure 1. Human Cases of the Disease controlling a pandemic if it does occur. Around the World Between 2003 and 2008, 47 countries reported the highly pathogenic H5N1 form of avian influenza in their domestic poultry (IFPRI 2008). H5N1 was first seen in a farmed goose in China in 1996 (IMCAPI Hanoi 2010). The first outbreak was recorded in Korea in December 2003 (World Bank 2008). By 2004 the virus had spread to several East Asian countries and by 2006 had reached several Asian, European and Middle Eastern and African countries. Unchecked trade and movement of infected poultry was one of the main triggers behind the spread of the lethal virus (FAO 2006). In the first three months of 2011, Bangladesh, Cambodia, Source: http://web.worldbank.org/WBSITE/EXTERNAL/NEWS Hong Kong SAR, China, India, Japan, /0,,contentMDK:20649058~menuPK:34480~pagePK:64 Korea, Myanmar, and Vietnam reported 257043~piPK:437376~theSitePK:4607,00.html outbreaks. To date the virus has affected 61 countries (Figure 1). Annex B2: High Containment Laboratories There are four levels of bio-safety laboratories (AAAS 2009 and Petric and others 2006). The lowest level, bio-safety level 1 (BSL-1), includes research ―involving well-characterized agents not known to consistently cause disease in immune competent adult humans, and present minimal potential hazard to laboratory personnel and the environment.‖ Bio-safety level 2 (BSL-2) includes ―work involving agents that pose moderate hazards to personnel and the environment.‖ Bio-safety level 3 (BSL-3) ―is applicable to clinical, diagnostic, teaching, research, or production facilities where work is performed with indigenous or exotic agents that may cause serious or potentially lethal disease through inhalation route exposure,‖ and requires that laboratory personnel receive specific training in handling pathogenic and potentially lethal agents, and be supervised by scientists competent in handling infectious agents and associated procedures. Bio-safety level 4 (BSL-4), the highest level, ―is required for work with dangerous and exotic agents that pose a high individual risk of life-threatening disease, aerosol transmission, or related agent with unknown risk of transmission.‖ ANNEX B 40 Annex B Table 2. Project Development Objective and Outcome Indicators Before and after Restructuring Table 2a Objective and Outcome Indicators Table 2b Objective and Outcome Indicators before Restructuring after Restructuring Original Objective Original Outcome Revised Objective Revised Indicators Indicators To assist the Appropriate contingency To assist the Borrower in -dropped- Borrower in reducing plans prepared and effective building its capacity to the threat posed to capacity to implement respond to potential future humans and the contingency plans built infections of humans by poultry sector by HPAI outbreaks in poultry HPAI and other zoonoses, -dropped- HPAI and other effectively contained and preparing for, zoonoses, and Risk of human infections controlling and Risk of human preparing for, reduced and treatment responding to influenza infections reduced and controlling and improved pandemics and other case management for responding to infectious disease infectious diseases influenza pandemics emergencies in humans. improved and other infectious disease emergencies 41 ANNEX B Annex B Table 3: Intermediate Results Indicators at Appraisal As reported in Technical Annex As reported in the ISRs Intermediate Results Results Indicators for Each Component Intermediate outcome Indicator (s) Component 1 - Animal Health A - Strengthening National HPIA Preparedness and Prevention Capability A1: Institutional framework for  Evaluation of veterinary services completed 1. Enhanced effective disease control according to OIE guidelines. effectiveness of strengthened.  Enhanced effectiveness of disease control at county disease control at level. county level. A.2: HPAI prevention and  Contingency plans revised and consolidated, and control planning improved and implementation guidelines and manuals available. 2. County laboratories coordinated across sectors. provide timely (according to EC B - Strengthening Disease Surveillance, Diagnostic Capacity and Research legislation and OIE B.1: Monitoring and surveillance  Veterinary staff at central and country levels have guidelines) disease systems strengthened. improved knowledge and skills to design and perform diagnoses. epidemiological surveys, and to analyze animal disease data. B.2: Diagnostic capacity at  County laboratories provide timely (according to EC 3. Efficient animal national and regional levels and legislation and OIE guidelines) disease diagnoses. disease information veterinary medicinal product  National reference laboratory provides timely quality control strengthened. confirmation of the local diagnoses. system operational in  National Institute for Control of Biological Products compliance with EC and Veterinary Drugs effectively applies molecular- legislation and OIE biological methods for quality control of veterinary guidelines. medicinal products. B.3: Integrated national animal  Efficient animal disease information system disease information system operational in compliance with EC legislation and 4. Quarantine, culling established. OIE guidelines. and disposal measures B.4: Applied veterinary research  National veterinary agency has appropriate designs are improved, in program defined and isolators for applied veterinary research program and institute accordance with operational. to fulfill its new mandate. environmental and  The National Institute for Control of Biological animal welfare Products and Veterinary Drugs can operate facilities guidelines. for administering highly infectious pathogens on live animals. C - Strengthening HPAI control programs 5. Behavioral & C.1: Virus eradication at the  Quarantine, culling and disposal measures are managerial changes to source improved through more improved, in accordance with environmental and higher bio-security in effective outbreak control. animal welfare guidelines. small-holder poultry C.2: Vaccination policy and  Poultry vaccination policy, contingency plan, and systems, after develop. implementation manuals implementation manual developed. of & consultations for developed. bio-security investments & D - Bio-Security in small-scale  Evidence of behavioral and managerial changes implementation of poultry production improved. towards higher bio-security in small-holder poultry awareness campaigns. systems, following the successful implementation of awareness campaigns and knowledge transfer activities.  Options for bio-security investments developed and 6. Strengthened known to poultry producers, and bio-security surveillance and manuals available. laboratory capacity. Component 2 - Human Health A - Enhancing Public Health  Procedures and standards upgraded and staff ANNEX B 42 As reported in Technical Annex As reported in the ISRs Intermediate Results Results Indicators for Each Component Intermediate outcome Indicator (s) Program Planning and trained in their use 7. Strengthened health Coordination – effective  Technical guidelines and manuals updated care response capacity. regulatory framework for  Social distancing plan prepared National Pandemic Influenza  Clean out plans for hospitals prepared Action Plan (NPIAP) as well as contingency plans that detail 8. Political and civic required activities on all levels leadership organized including implementation around a national guidelines and manuals for all strategic risk relevant agencies and services communication plan.. B - Strengthening of National  Increased surveillance and outbreak Public Health Surveillance investigation capacity; System  NIL at biosafety level 3 9. Strengthened health  Regional laboratory capacity strengthened (with care response capacity. at least 2-3 labs able to adequately performed avian influenza tests) C - Strengthening Healthcare System Response Capacity 10. Awareness-raising C.1: Strengthening isolation and  Infectious disease hospitals at the central level fully and behavior change case management capacity of equipped interventions adapted hospitals - Critical medical care  Isolation and intensive care units in 7 regional to population at risk networks strengthened and hospitals established using appropriate readied to cope with increased  Health professionals and support personnel trained communications demand for services, and to for active surveillance, case finding, and proper channels. prevent the spread of infection diagnosis, treatment and care. among high-risk populations and  Case fatality rate below the international average health care workers level during both pre-epidemic and pan-endemic phases ; C.2: Upgrading the influenza  Vaccine production facility at "Cantacuzino Institute" vaccine production unit strengthened upgraded.  Capacity to produce up to 2.5 million doses of trivalent vaccines and 5 million doses of monovalent vaccine established Component 3 - Strategic Communication Coordinated communications  Political and civic leadership organized around a process involving national national strategic risk communication plan; veterinary agency and Ministry  Research-based risk communication strategies and of Public Health, addressing products developed, responding to the needs of information /communication priority audiences; needs of priority audiences,  Awareness-raising and behavior change interventions during pre-pandemic and with population at risk using appropriate pandemic A-I phases; communications channels consistent;  Communications strategies and products developed and used highlighting the actions and investments of Informed at-risk communities participating ministries and the mobilization of group adopting safe health practices, resources to contain the epidemic, generating social reporting, and contributing to trust and credibility; outbreak control actions;  Evidence existent of consistent communication and information technologies, to promote reporting of outbreaks, fast response and an uninterrupted social Educated citizenry, aware of the dialogue; impact and social containment  Informational products developed and disseminated measures needed if avian that educate priority audiences about possible influenza escalates across scenarios and mitigation and control actions to be pandemic phases. undertaken. 43 ANNEX B 45 ANNEX C Annex C. Assessing the Level of Preparedness in Romania to respond to Avian Influenza and other infectious diseases Determinants of the Level of preparedness Contribution AREAS ACTIVITIES STATUS IN ROMANIA BANK PROJECT CONTRIBUTION IN EACH AREA Communication Dissemination Romania was able to communicate As part of component 3 technical strategy material, information on avian influenza to the assistance and training was to be working with public at the time of the 2005 and 2006 provided for communication staff media, etc. outbreaks but the Government at the national veterinary agency. containment strategy created panic and Support was also to be provided led to significant loss for the poultry for the delivery of a number of industry. As a part of the process of information campaigns. These developing a communication strategy communication activities were to inform the public about avian canceled from component 3 when influenza conditions, UNICEF and the project was restructured. USAID, in consultation with the communications working group, prepared, pre-tested and disseminated a range of prevention materials including TV, radio and newspaper, brochures and leaflets and through inter-personal channels such as puppet theatres for The Steering committee which children. USAID withdrew from was to provide effective Romania when it became a part of the communication across various EU in 2007, although UNICEF government departments and continues to be present and can agencies did not function continue to take an active role on effectively. communications if needed. UNICEF also supported a symposium on crisis communication that was convened for government and media participants in November 2007. However, coordination between different government departments and agencies at the national level remains weak. When there is a crisis, a fire fighting mode comes into play to handle it, as was the case in 2005-2006 with outbreaks. Surveillance Surveillance A national program for surveillance, The project was to support the in animals of disease in prevention and eradication of animal establishment of a comprehensive poultry and diseases is in place in conformity with national animal disease other animals EU legislation. The program is information system but this approved annually by the national activity was dropped when the veterinary agency and the strategy for animal health component was sampling is prepared at the county canceled. level. However, the poultry sector The project did not make a remains polarized with some large contribution in this area commercial farms and many small- ANNEX C 46 Determinants of the Level of preparedness Contribution AREAS ACTIVITIES STATUS IN ROMANIA BANK PROJECT CONTRIBUTION IN EACH AREA scale backyard producers. The latter produce mainly for home consumption and sell only a little surplus on the local market. Biosecurity and surveillance remains weak on the backyard systems. The risk from migratory birds and water fowl continues. But efforts are being made to reduce the risk, for example, through placement of sentinel birds with wild birds in the Danube delta and follow up with serial testing. Surveillance There is a National Avian Influenza The project did not make a in humans Task Force, chaired by the Ministry of contribution in this area Agriculture, under the jurisdiction of the National Committee for Emergencies, and is coordinated by the Prime Minister‘s office. Appropriate contingency plans for human pandemic influenza were prepared, coordination mechanisms were defined and legislation was revised. However, coordination between different government departments and agencies at the national level remains weak. When there is a crisis, a fire fighting mode comes into play to handle it, as was the case in 2005-2006 with the avian influenza outbreaks. A center for communicable diseases is set up in the Institute of Public Health to be a direct counterpart of the European Center for Disease Prevention and Control (ECDC) in Stockholm Sweden.* Romania is part of a European clinical surveillance system that is generally based on reports made by sentinel general practitioners The practitioners usually represent 1-5 percent of physicians working in the country or region. Most sentinel surveillance systems report data on the number of new cases of influenza-like illness or acute respiratory infection.** During the influenza season (October - April), the Sentinel System collects data on a weekly basis Diagnosis Laboratory The Institute for Diagnosis and Animal The project did not make a capacity for capacity to Health is the national reference contribution in this area 47 ANNEX C Determinants of the Level of preparedness Contribution AREAS ACTIVITIES STATUS IN ROMANIA BANK PROJECT CONTRIBUTION IN EACH AREA identification of test for laboratory for animal diseases and is disease in virus/bacteria under the national veterinary agency, birds/animals 41 county veterinary state laboratories and the Bucharest municipality laboratory perform analyses in animal health. There are also several private laboratories. Good network of public and private vets exists in the country. Overall the technical capacity is good though managerial capacity is weak. Diagnosis hospital and Diagnostic capacity for avian influenza The National Influenza capacity and laboratory in humans exists in the country. The Laboratory at the Cantacuzino treatment in capacity National Influenza Laboratory exists in Institute was mostly upgraded to humans the Cantacuzino Institute in Bucharest BSL3 although some equipment and is notified as a National Centre by is still lacking. The certification the Ministry of Public Health and also to be done by an internationally recognized by WHO. The Cantacuzino accredited laboratory is still Institute provides support and technical pending and will be undertaken guidance for public health districts, and only after the equipment that is epidemiological surveillance of lacking is bought. communicable diseases. It also serves as a reference laboratory for other The purchase of new equipment countries in the region. has boosted the vaccine production capacity at the Institute which can now produce up to 8,000 vaccine doses per month. However the rehabilitation of the ampoule sealing and filling station remains incomplete. The Bank provided equipment for the ICUs of two Bucharest infectious disease hospitals and Two Bucharest hospitals and 7 regional the establishment of isolation and centers for infectious diseases have intensive care units in the 7 intensive care units (ICUs) for isolation regional centers for infectious of patients. However, there is lack of diseases (Brasov, Craiova, adequate capacity at the regional Constanta, Cluj, Iasi, Timisoara infectious disease hospitals and ICUs and Targu Mures). There was a to manage patients and utilize the problem with the quality of some equipment in the event of a pandemic of the beds that were procured. Cantacuzino Institute representative was trained in the US in 2010 for laboratory supervision. Institute staff also received training on Good Manufacturing Practice standards in vaccine production ANNEX C 48 Determinants of the Level of preparedness Contribution AREAS ACTIVITIES STATUS IN ROMANIA BANK PROJECT CONTRIBUTION IN EACH AREA Control Stamping out, Measures to respond immediately The project did not make a strategy bio-security generally exist although transportation contribution in this area and weaknesses might delay responses in movement remote areas. Practices for culling etc. control when are in place. Vaccination of animals is infections given importance. occur EU controls on third country imports require a system of border inspection posts to be completed to EU standards at external borders. Other projects helped with construction of such posts in Romania. Health care Ability to The Cantacuzino Institute produces The project helped increase response acquire seasonal influenza vaccine and is vaccine production capacity capacity with medicines and capable of expanding supply for stockpiles of vaccines at pandemics if needed anti- short notice virals/vaccines Emergency procurement of drugs Legal and Veterinary The government has a compensation The project did not make a regulatory law strategy in place which was updated in contribution in this area framework Compensation 2009. Earlier compensation was paid for farmers from the Ministry of Agriculture budget although since 2009 payment of compensation is from budget allowed to the national veterinary agency. Simulation Desk top Contingency plan for avian influenza The Bank project was to support exercises simulation established in 2006 was updated and a a consultant to help update the human influenza pandemic plan was contingency plan. Since the prepared. Coordination mechanisms Ministry of Public Health went HINI were defined and legislation was ahead and updated the plans in pandemic revised. The Contingency Plan outlined May 2009 in response to the actual event an inter-sectoral plan and actions to growing H1N1 pandemic threat that required respond to a pandemic situation. the Bank support was canceled. all parties to come and act The first case of H1N1 was detected in A Public Awareness Campaign to together Romania in October 2009 and until be implemented by the national January 2010 the country had a total of veterinary agency was dropped at 7,000 cases. 1.7 million people were the time of the restructuring vaccinated against the pandemic flu and anti-viral drugs were provided to all communicable disease hospitals for treatment of confirmed cases. The government also supported a large scale public awareness campaign through its own resources. All medical units were obligated to report H1N1 suspected cases and the flow of patients accessing emergency rooms in the District Hospitals was monitored 49 ANNEX C Determinants of the Level of preparedness Contribution AREAS ACTIVITIES STATUS IN ROMANIA BANK PROJECT CONTRIBUTION IN EACH AREA with the most severe cases being isolated. Surveillance at border points with neighboring countries was strengthened and measures were introduced to allow for temporary closure and disinfection of schools where cases were identified. Vaccine production at Cantacuzino Institute was increased. Coordination Between The urgency to deal with avian and rapid donors influenza brought together response Between international technical organizations in Ministries a way that has not happened before. AFO, UNICEF, UNDP, WHO , OIE and the World Bank have come together to assist the country in dealing The Steering Committee with the disease. established under the project did National Task Force that met not perform adequately. frequently during the 2005 and 2006 outbreaks. Coordination cited as constraint. Notes: *ECDC is an independent agency of the EU established in 2005 with the aim of strengthening Europe's defense against infectious diseases. With the EU economic integration cooperation on public health issues also became important. The outbreak of SARS in 2003 and its rapid spread across countries created some urgency for the creation of an EU-wide institution for public health. In 2005 H5N1 avian influenza emerged in several European countries. The ECDC is responsible for the surveillance of infectious diseases in the EU and maintains databases for epidemiological surveillance. ** for more on the European system of surveillance see http://www.ecdc.europa.eu/en/activities/surveillance/EISN/surveillance/Pages/sentinel_surveillance.aspx 51 ANNEX D Annex D. Borrower’s Comments ANNEX D 52 53 ANNEX D ANNEX D 54 55 ANNEX D ANNEX D 56 57 ANNEX D