Document of The World Bank FOR OFFICIAL USE ONLY Report No: ICR00005124 IMPLEMENTATION COMPLETION AND RESULTS REPORT (IBRD 84490) ON A LOAN IN THE AMOUNT OF EUR 227.48 MILLION (US$300 MILLION EQUIVALENT) TO THE REPUBLIC OF SERBIA FOR THE FLOODS EMERGENCY RECOVERY PROJECT April 30, 2020 Water Global Practice Europe and Central Asia Region This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed without World Bank authorization. CURRENCY EQUIVALENTS (Exchange Rate Effective October 31, 2019) Currency Unit = EUR EUR 1= US$1.12 EUR 1.24 = SDR 1 FISCAL YEAR January 1 – December 31 ABBREVIATIONS AND ACRONYMS ARAP Abbreviated Resettlement Action Plan B/C Benefit-to-Cost Ratio CERC Contingent Emergency Response Component CPF Country Partnership Framework DAP Directorate for Agrarian Payments DRM Disaster Risk Management DWM Directorate of Water Management EBITDA Earnings before Interest, Taxes, Depreciation, and Amortization EIRR Economic Internal Rate of Return EMP Environmental Management Plan EPS Elektroprivreda Srbije EU European Union FERP Floods Emergency Recovery Project GDP Gross Domestic Product ICR Implementation Completion and Results Report ISR Implementation Status and Results Report M&E Monitoring and Evaluation MAEP Ministry of Agriculture and Environmental Protection MAFWM Ministry of Agriculture, Forestry, and Water Management MCTI Ministry of Construction, Transport and Infrastructure NPV Net Present Value PAD Project Appraisal Document PDO Project Development Objective PIMO Public Investment Management Office PIU Project Implementation Unit POM Project Operations Manual RNA Recovery Needs Assessment RPF Resettlement Policy Framework TOR Terms of Reference Regional Vice President: Cyril E. Muller Country Director: Linda Van Gelder Regional Director: Steven N. Schonberger Practice Manager: David Michaud Task Team Leader(s): Berina Uwimbabazi, Darko Milutin ICR Main Contributor: Nooshafarin Tayebi TABLE OF CONTENTS DATA SHEET .......................................................................................................................... 1 I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES ....................................................... 6 A. CONTEXT AT APPRAISAL .........................................................................................................6 B. SIGNIFICANT CHANGES DURING IMPLEMENTATION (IF APPLICABLE) ..................................... 11 II. OUTCOME .................................................................................................................... 14 A. RELEVANCE OF PDOs ............................................................................................................ 14 B. ACHIEVEMENT OF PDOs (EFFICACY) ...................................................................................... 16 C. EFFICIENCY ........................................................................................................................... 20 D. JUSTIFICATION OF OVERALL OUTCOME RATING .................................................................... 21 E. OTHER OUTCOMES AND IMPACTS (IF ANY) ............................................................................ 21 III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME ................................ 22 A. KEY FACTORS DURING PREPARATION ................................................................................... 22 B. KEY FACTORS DURING IMPLEMENTATION ............................................................................. 22 IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME .. 24 A. QUALITY OF MONITORING AND EVALUATION (M&E) ............................................................ 24 B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE ..................................................... 25 C. BANK PERFORMANCE ........................................................................................................... 27 D. RISK TO DEVELOPMENT OUTCOME ....................................................................................... 29 V. LESSONS AND RECOMMENDATIONS ............................................................................. 29 ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS ........................................................... 32 ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION ......................... 40 ANNEX 3. PROJECT COST BY COMPONENT ........................................................................... 43 ANNEX 4. EFFICIENCY ANALYSIS ........................................................................................... 44 ANNEX 5. BORROWER, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS ... 51 ANNEX 6. SUPPORTING DOCUMENTS (IF ANY) ..................................................................... 52 ANNEX 7. MAP .................................................................................................................... 53 The World Bank Floods Emergency Recovery Project (P152018) DATA SHEET BASIC INFORMATION Product Information Project ID Project Name P152018 Floods Emergency Recovery Project Country Financing Instrument Serbia Investment Project Financing Original EA Category Revised EA Category Partial Assessment (B) Partial Assessment (B) Organizations Borrower Implementing Agency Elektroprivreda Srbije, Directorate of Agrarian Payments Republic of Serbia (DAP), Directorate of Water Management Project Development Objective (PDO) Original PDO The Project Development Objective (PDO) is to: (i) help restore power system capability to reliably meet domestic demand; (ii) protect livelihoods of farmers in the flood affected areas; (iii) protect people and assets from floods; and (iv) improve the Borrower’s capacity to respond effectively to disasters. Page 1 of 53 The World Bank Floods Emergency Recovery Project (P152018) FINANCING Original Amount (US$) Revised Amount (US$) Actual Disbursed (US$) World Bank Financing 300,000,000 295,779,849 238,929,636 IBRD-84490 Total 300,000,000 295,779,849 238,929,636 Non-World Bank Financing 0 0 0 Borrower/Recipient 0 0 0 Total 0 0 0 Total Project Cost 300,000,000 295,779,849 238,929,636 KEY DATES Approval Effectiveness MTR Review Original Closing Actual Closing 03-Oct-2014 06-Mar-2015 12-May-2017 31-Dec-2017 31-Oct-2019 RESTRUCTURING AND/OR ADDITIONAL FINANCING Date(s) Amount Disbursed (US$M) Key Revisions 28-Mar-2017 195.07 Change in Results Framework Change in Components and Cost Change in Loan Closing Date(s) Reallocation between Disbursement Categories Change in Implementation Schedule KEY RATINGS Outcome Bank Performance M&E Quality Satisfactory Moderately Satisfactory Substantial Page 2 of 53 The World Bank Floods Emergency Recovery Project (P152018) RATINGS OF PROJECT PERFORMANCE IN ISRs Actual No. Date ISR Archived DO Rating IP Rating Disbursements (US$M) 01 20-Apr-2015 Satisfactory Satisfactory 96.11 02 08-Jan-2016 Satisfactory Satisfactory 181.49 03 27-Sep-2016 Moderately Satisfactory Moderately Satisfactory 192.29 04 11-May-2017 Moderately Satisfactory Moderately Satisfactory 195.07 05 29-Dec-2017 Moderately Satisfactory Moderately Satisfactory 205.79 06 29-Jun-2018 Moderately Satisfactory Moderately Satisfactory 211.99 07 28-Dec-2018 Satisfactory Moderately Satisfactory 223.08 08 24-Jun-2019 Satisfactory Satisfactory 235.07 SECTORS AND THEMES Sectors Major Sector/Sector (%) Agriculture, Fishing and Forestry 23 Other Agriculture, Fishing and Forestry 23 Public Administration 1 Other Public Administration 1 Energy and Extractives 70 Other Energy and Extractives 70 Transportation 1 Other Transportation 1 Page 3 of 53 The World Bank Floods Emergency Recovery Project (P152018) Water, Sanitation and Waste Management 5 Other Water Supply, Sanitation and Waste 5 Management Themes Major Theme/ Theme (Level 2)/ Theme (Level 3) (%) Private Sector Development 100 Jobs 100 Finance 17 Finance for Development 17 Disaster Risk Finance 17 Urban and Rural Development 51 Disaster Risk Management 51 Disaster Response and Recovery 17 Disaster Risk Reduction 17 Disaster Preparedness 17 Environment and Natural Resource Management 33 Environmental Health and Pollution Management 33 Air quality management 11 Water Pollution 11 Soil Pollution 11 ADM STAFF Role At Approval At ICR Regional Vice President: Laura Tuck Cyril E Muller Country Director: Ellen A. Goldstein Linda Van Gelder Director: Anita Marangoly George Steven N. Schonberger Practice Manager: Ranjit J. Lamech David Michaud Page 4 of 53 The World Bank Floods Emergency Recovery Project (P152018) Berina Uwimbabazi, Darko Task Team Leader(s): Claudia Ines Vasquez Suarez Milutin ICR Contributing Author: Nooshafarin Tayebi Page 5 of 53 The World Bank Floods Emergency Recovery Project (P152018) I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES A. CONTEXT AT APPRAISAL Context 1. In early May 2014, unexpected rainfall caused massive floods in Serbia and neighboring countries, which had a severe impact on economic growth. In Serbia, the heavy rainfall led to an increase of water levels in eight of the main rivers across the country (annex 7 - Map of flood-affected municipalities) causing flash floods and affecting both urban and rural areas. This flooding resulted in the declaration of national state of emergency on May 15, 2014. In physical terms, 51 people were killed, more than 1.6 million people were affected (one-fifth of the total population), approximately 32,000 people were evacuated from their homes, and about 110,000 households did not have access to electricity supply. Economically, the floods were estimated at appraisal to have a gross impact on growth of −0.9 percent, erasing the year’s 0.5 percent expected growth and inducing a 0.4 percent contraction. The poor and the vulnerable were the hardest hit. Almost 52,000 people suffered at least temporary unemployment because of disruptions in production and incomes declined accordingly. At appraisal, it was estimated that 125,000 people fell below the poverty line due to this disaster, with a resultant 7 percent increase in the national poverty rate over the previous year. Significantly, Serbia’s Human Development Index decreased in 2014, reversing approximately two years of progress. 2. The most affected sector was mining and energy as the biggest open pit coal mines, Tamnava West and Veliki Crljeni, were flooded resulting in a protracted shortage of electricity which affected households and industry. Transmission and distribution transformer substations were also flooded, and their equipment were severely damaged. The two flooded mines are the primary sources of fuel supplying the Nikola Tesla Thermal Power Plant which accounts for 40 percent of the total installed capacity in the country. As a result, the flooding of the mines created a shortage of power supply of approximately 15 percent of demand over the winter season. The dewatering and rehabilitation of the mines were estimated at appraisal to cost EUR 23 million. 3. The Government of Serbia conducted a Recovery Needs Assessment (RNA) between June and July of 2014. The RNA focused on the 24 most affected municipalities and revealed that the total effects of the disaster amounted to EUR 1.525 billion (Table 1). When all 49 municipalities affected by the floods are taken into account, the total value of disaster effects increases to EUR 1.7 billion, or over 4 percent of gross domestic product (GDP). Agriculture was also affected significantly by the floods as about 12,000 ha of cropland were made unsuitable for cultivation. The RNA estimated damages and losses in the amount of EUR 228 million, to both farmers’ income from crops (EUR 90.6 million) and to livestock (EUR 5.3 million). This flooding also damaged the flood protection infrastructure including drainage, which left the country more vulnerable to future floods. The total cost of repairing this infrastructure was estimated at EUR 24 million. Table 1. Estimated Damages and Losses Due to the 2014 Floods (EUR, millions) Sectors Damages Losses Total Social 234.6 7.1 241.7 Housing 227.3 3.7 230.9 Page 6 of 53 The World Bank Floods Emergency Recovery Project (P152018) Sectors Damages Losses Total Education, health, and culture 7.4 3.4 10.8 Productive 516.1 547.6 1063.6 Mining and energy 181.9 305.8 487.7 Agriculture 107.9 120.1 228.0 Trade 169.6 55.2 224.8 Manufacturing and tourism 56.7 66.5 123.2 Infrastructure 117.3 74.8 192.1 Transport and communications 104.9 71.5 176.5 Water and sanitation 12.4 3.2 15.7 Cross-cutting (environment and 17.2 10.6 27.9 governance) Total 885.2 640.1 1,525.3 4. The RNA was presented at a donor’s conference convened in Brussels on July 16, 2014, in the aftermath of the natural disaster. The donor’s conference mobilized pledges of EUR 995.2 million from 60 countries and 23 international organizations, of which EUR 107 million are grant funds. In addition, EUR 41.4 million were pledged for cross-border activities. It was agreed between the Government and the donors that the housing sector, which had recorded the second-highest total damage and loss, would be addressed through funding from the European Commission while the World Bank would concentrate on the energy and agriculture sectors. 5. The World Bank supported the Government of Serbia in the aftermath of the flood. The World Bank aided the preparation of the RNA by providing expertise from the Disaster Risk Management (DRM) Department, led the diagnostic efforts in the energy and transport sectors, and provided technical inputs to several other sectors. It also assisted Serbia in promoting donor coordination and conducted a portfolio review to restructure two projects to help with the recovery efforts. However, the centerpiece of the World Bank’s assistance to Serbia after these floods was an emergency loan for the ‘Floods Emergency Recovery Project’ (FERP) in the amount of EUR 227.8 million. FERP supported the energy and agriculture sectors and flood protection. The World Bank coordinated with other donors such as the European Union (EU) and the United Nations which focused their support in the reconstruction of housing since this was also one of the worst hit sectors. The World Bank-financed FERP assisted the Government to (a) Help close the financing gap and ensure continued provision of electricity services, particularly during the first winter following the floods; (b) Provide direct financial support to farmers in affected areas at a time when the fiscal accounts are under severe stress and may be unable to deliver the needed support; (c) Help improve resilience to disasters by financing investments in critical flood prevention infrastructure; and (d) Manage the risk of future disasters by including a contingent emergency response component (CERC), which would allow project resources to be quickly diverted to disaster response in case of a future emergency. This was a new instrument in the World Bank arsenal at the time but was appealing due to its contingent nature and its ability to be deployed in case of a recurrence of similar disasters. Page 7 of 53 The World Bank Floods Emergency Recovery Project (P152018) 6. FERP was initially designed as a disaster recovery operation but was later restructured to focus more on reconstruction as discussed in the following paragraphs. 7. The institutional arrangements shared responsibility for the implementation of the four project components between three agencies of the Serbian Government, as shown in Figure 1: Figure 1. Institutional Arrangements 8. Oversight of project implementation rested with the Office for Reconstruction and Flood Relief and was transferred to the Public Investment Management Office (PIMO) once the Office for Reconstruction and Flood Relief reached the end of its legislated mandate. Day-to-day project management was divided by component. The public power utility, Elektroprivreda Srbije (EPS), was responsible for implementing Component 1 through its existing structures which had experience in implementing World Bank-financed projects. Components 2 and 3 were the responsibility of the Ministry of Agriculture and Environmental Protection (MAEP) through two of its directorates. The Directorate for Agrarian Payments (DAP) managed Component 2, which was consistent with its mandate of making payments to farmers. Finally, a Project Implementation Unit (PIU) established within the Directorate of Water Management (DWM) implemented Component 3 as it had also recently implemented an irrigation and drainage project financed by the World Bank. Theory of Change (Results Chain) 9. With regard to the context described earlier, the project aimed at addressing the emergency needs of the country across multiple sectors. The higher objective of this project was to contribute to restoring Serbia on a growth path following the devasting floods of 2014 and later in 2016 by contributing to the recovery and reconstruction efforts in the critical sectors of energy, agriculture, and flood defenses. Due to the floods, Serbia entered a recession in 2014;1 GDP contracted by 1.8 percent from 2.6 percent growth in 2013. The 2014 recession was widespread. Energy output, for example, was 17 percent lower in 2014 than in 2013. 1Country Partnership Framework (CPF) Serbia 2015. Report No. 94687. World Bank. http://pubdocs.worldbank.org/en/111121446462343202/Serbia-CPF-eng-web.pdf. Page 8 of 53 The World Bank Floods Emergency Recovery Project (P152018) 10. This disaster had wider economic impacts, including on livelihoods and employment of many but more significantly on vulnerable groups such as elderly farmers, which account for Serbia’s poorest 40 percent. To address these challenges, this emergency investment focused on the sectors which were the most affected and contributed substantially to economic growth, specifically, the energy and agriculture sectors, while improving the medium- and longer-term resilience to climate events through flood protection investments. By ensuring continued energy supplies to meet demand, compensating for lost income of farmers, improving protection for the population’s livelihood and assets from future floods, and strengthening the capacity of the borrower, this project improved the conditions for economic recovery. In 2015, growth bounced back to 0.8 percent. 11. The Implementation Completion and Results Report (ICR) developed the Theory of Change in Figure 2 based on the original Results Framework in the Project Appraisal Document (PAD) because a Theory of Change was not required at the time of appraisal. Figure 2. Theory of Change Project Development Objectives (PDOs) 12. The Project Development Objective (PDO) is to (i) help restore power system capability to reliably meet domestic demand; (ii) protect livelihoods of farmers in flood affected areas; (iii) protect people and assets from floods; and (iv) improve the Borrower’s capacity to respond effectively to disasters. 13. The PDO was not changed during project implementation. Page 9 of 53 The World Bank Floods Emergency Recovery Project (P152018) Key Expected Outcomes and Outcome Indicators 14. The project’s key expected outcomes, as stated in the PDO, consisted of four parts: ensure continued access to energy supplies, increase resilience and improve protection for population against future floods, protect livelihood of farmers by compensating for their lost income, and provide additional disaster response capacity by the use of a CERC. These objectives were to be achieved by the following outcome indicators: (a) Availability of power supplies restored to pre-flood levels (b) Farm income in the flood affected areas recovered to pre-flood level (c) Agricultural land receiving improved flood protection (d) Number of households receiving improved flood protection Components 15. At appraisal, the project was strongly focused on disaster recovery. It had four components: (a) Energy Sector Support, (b) Agricultural Sector Support, (c) Flood Protection, and (d) Contingent Emergency Response. These are described below. Component 1: Energy Sector Support (EUR 157.11 million) 16. The objective of this component was help restore power system capability to reliably meet domestic demand through power purchases, improved reliability of the distribution system, and energy conservation measures and help the restoration of strategic energy assets. 17. Subcomponent 1A: Support for electricity purchases (EUR 119.82 million). Subcomponent 1A aimed to support electricity purchases by EPS to improve power availability and avert an impending energy crisis particularly over the first winter heating season following the floods. The project supported power purchases made under commercial practices under multi-monthly, monthly, and weekly contracts. Retroactive financing for power import contracts from June 2014 to the signing of the Loan Agreement was included, carrying ex post facto review for eligibility. 18. Subcomponent 1B: Urgent restoration of the distribution network and load management (EUR 14.29 million). Subcomponent 1B intended to support the provision of (a) metering devices for the flood- affected areas; (b) mobile substations; and (c) energy-efficient light bulbs, technical assistance, and other goods to support load management activities. 19. Subcomponent 1C: Dewatering of the Tamnava West Field Mine (EUR 23 million). Subcomponent 1C intended to support the first critical step to put the Tamnava West Field Mine in the Kolubara mining basin back into operation and help avoid significant health hazards associated with the flooded mine, including the potential pollution of groundwater, an increase in waterborne diseases, as well as threats to surrounding flora and fauna that would emerge as a result of stagnant water in the mine pit. Subsequent activities related to mine recovery and coal mining equipment rehabilitation were undertaken by EPS and outside the scope of this project. Given the urgent need to begin this health- and Page 10 of 53 The World Bank Floods Emergency Recovery Project (P152018) safety-related activity, the procurement of the eight overall dewatering service contracts was carried out in accordance with the Public Procurement Law of Serbia. The contract was signed on August 19, 2014, and it was considered under retroactive financing carrying ex post facto review for eligibility. Component 2: Agricultural Sector Support (EUR 53.08 million) 20. Component 2 aimed to support the ongoing Farm Incentives Program to protect the livelihood of farmers affected by the floods and offset their income losses. This component covered up to EUR 53.08 million in costs of the program for payments made to farmers in the 49 affected municipalities from May 15, 2014 to December 31, 2015. Component 3: Flood Protection (EUR 16.72 million) 21. The objective of this component was to support urgent rehabilitation of flood protection and drainage control infrastructure and strengthen the technical capacity of government agencies for improved flood prevention and management. 22. Subcomponent 3A: Investments in Flood Protection (EUR 14.72 million). Subcomponent 3A finance rehabilitation and reconstruction of priority flood protection and drainage control infrastructure through the provision of goods, civil works, and consultants’ services in the flood-affected areas and areas vulnerable to flooding. To the extent possible, rehabilitation and reconstruction intended to follow the principle of ‘building back better’ after a natural disaster (for example, through heightening and improving design and construction standards). 23. Subcomponent 3B: Project Implementation Support for Flood Management (EUR 2 million). This subcomponent aimed to support project management, including operating costs. It also supported strengthening technical capacity for improved flood prevention and management, including technical studies to identify priority flood prevention infrastructure works, preparation of technical designs and bidding documents, terms of reference (TOR) for such activities, and options for strengthening institutional arrangements and DRM in the water sector. Component 4: Contingent Emergency Response (EUR 0) 24. The objective of this component was to improve Serbia’s capacity to better respond to disasters. Following an adverse natural or man-made event that causes a major disaster, the Government of Serbia could have requested the World Bank to reallocate project funds to this component to partially cover emergency response and recovery costs. This component could also have been used to channel additional funds should they become available as a result of the emergency. B. SIGNIFICANT CHANGES DURING IMPLEMENTATION (IF APPLICABLE) Revised PDOs and Outcome Targets 25. The PDO was not changed during project implementation. Page 11 of 53 The World Bank Floods Emergency Recovery Project (P152018) 26. As described in the following section, the Results Framework was updated when the project was restructured in March 2017 to reflect the implementation schedule (extended by 22 months) and the proposed activities financed under the project. The restructuring marked a transition from disaster recovery to reconstruction with a stronger focus on preparedness and institutional aspects. In addition, due to the emergency nature of this activity, during implementation, the indicators were refined to reflect information that became available later on through early stages of implementation. Revised PDO Indicators 27. PDO indicators were revised during the 2017 restructuring. One PDO-level indicator ‘Farm income recovered to preflood level (measured on the basis of standard output (%))’ was removed during the restructuring since statistics on Standard Output were no longer available from the National Statistics Office. However, this indicator was replaced by a new indicator to measure the agricultural production in flood affected municipalities recovered to pre-flood levels (Yes/No). The end target was revised for the PDO-level indicator on availability of power supplies restored to preflood levels through imports/domestic generation (GWh) to reflect a lower need for imports in view of the faster recovery of mine operations. The remaining PDO-level indicators were revised to take into account additional flood protection investments financed through the restructuring. Revised Components 28. The components were not substantially revised by the restructuring in March 2017. Changes made were linked to the reallocation between disbursement categories. Substantial savings were realized under Component 1 (Energy Sector Support) and US$17.36 million was reallocated to Component 3 (Flood Protection). This reallocation (from Component 1 for power imports and dewatering of the Tamnava mine) financed works for flood protection infrastructure works, consulting and non-consulting services, and goods for the project. 29. The post-restructuring configuration and achievements were as follows: Component 1: Energy Sector Support (actual allocation EUR 135.25 million) 30. Under this component, the project helped restore power system capability to reliably meet demand during the 2014–15 winter season by financing power imports to address the supply and demand shortages. It also helped restore critical energy assets as the dewatering of the Tamnava Mine was successfully completed in the first trimester of 2015. The restoration of the largest mine in the country helped bring the output of the electricity sector back up to preflood levels, thereby contributing to restoring economic activity in 2015 and beyond. Under this component, the procurement of 10 mobile substations to support fast response in case of a climate event or emergency was completed. In addition, all meters for public lighting and residential houses in areas hit by the floods were installed in May 2018, as planned. However, the refurbishment of five fixed substations remains incomplete as equipment were purchased but the installation was not completed by project closing due to long delays in the obtention of construction permits. Page 12 of 53 The World Bank Floods Emergency Recovery Project (P152018) Component 2: Agricultural Sector Support (estimated allocation EUR 53.08 million, actual allocation EUR 51.36 million) 31. This component aimed to provide support and improve agricultural production in flood-affected municipalities and support recovery of their income to preflood level. Grant support through the Farm Incentives Program provided critical support to farmers, thus helping protect their livelihood and offset income losses in the aftermath of the floods. Component 3: Flood Protection (estimated allocation EUR 16.72 million, actual allocation EUR 33 million) 32. This component aimed at investments to rehabilitate critical flood protection infrastructure in the flood-prone areas. This component received EUR 16.28 million reallocated from Component 1 to finance the purchase of meteorological radars, rehabilitation of flood protection infrastructure and technical assistance to improve flood management. Under this component, support was provided for urgent rehabilitation of the flood protection and drainage control infrastructure and to enhance the function and effectiveness of existing facilities in six subproject areas. Component 4: Contingent Emergency Response (estimated allocation EUR 0, actual allocation EUR 0) 33. This component was not triggered during the project implementation. Other Changes 34. In addition to changes to the PDO indicators, several of the targets of intermediate outcome indicators were changed during the restructuring. These changes are summarized in Table 2: Table 2. Summary of Revisions of Intermediate Indicators during 2017 Restructuring Intermediate Results Indicators Action Comment Dewatering of the Tamnava Revised End target date revised since the activity was completed West Field mine finalized earlier in the first half of 2015. Number of meters replaced in Revised This indicator was revised according to the new affected areas implementation schedule; extending the end target date. Number of energy efficient light Deleted This intermediate outcome indicator was removed since bulbs installed/energy efficiency this activity was not necessary given that power supplies measures implemented through imports/domestic generation ensured reliable supply and consumption reduction was not needed. This activity was therefore not financed under the project. Farms benefiting from Revised End target date was revised since the activity was fully Government support in the completed by December 2015. affected areas Target value was revised to take into account legal changes to eligibility criteria for beneficiaries implemented after the project was approved. Number of public lighting meters Added during Medium-term energy efficiency/load management activity installed restructuring was financed under the project. Page 13 of 53 The World Bank Floods Emergency Recovery Project (P152018) Intermediate Results Indicators Action Comment Number of fixed substations Added during Activity supporting the strengthening of the distribution rehabilitated in the flood restructuring network was financed under the project. affected areas Length of flood protection Revised End target date and values were revised to take into reconstruction/rehabilitation account definitive list of subprojects financed under the works project (50 km) and additional flood protection works financed under the restructuring (15 km) due to reallocation of funds. Rationale for Changes and Their Implication on the Original Theory of Change 35. The theory of change was not significantly altered by the changes during implementation. During the restructuring of the project, EUR 17.36 million was reallocated from Component 1 to Component 3 as requested by the Government in its letter (October 13, 2016). By the time of the restructuring, the ratio of commitments and disbursements reached 79 percent, and key activities to facilitate the country’s transition to its preflood growth path had been completed. The request for the restructuring was triggered by a flood event in March of 2016 which indicated the need for improving flood protection planning and management. It was decided that the operation, which was initially focused on implementing emergency activities, could support Serbia in strengthening its climate disaster preparedness. The reallocation allowed the use of the project’s savings from dewatering of the Tamnava mine and power imports to finance works, goods, and services to improve flood protection infrastructure and management which were achieved during the project. In addition, investments improved the demand-side management activities related to the energy sector. 36. During restructuring the implementation timeline was extended by 22 months to allow necessary time to complete all activities and the Results Framework was updated to monitor the implementation of all the activities. In addition, the indicators were revised based on additional information which was not available at project approval. 37. The changes in the implementation during the restructuring were for practical reasons to allow more time and efficient use of project resources for the implementation of climate resilience activities. The changes did not affect the scope or the PDO of the project; neither did they trigger any new safeguard policy nor did they change an existing safeguard category. II. OUTCOME A. RELEVANCE OF PDOs 38. The PDO supported both focus areas under Serbia’s CPF for FY16–202: (a) economic governance and role of the state and (b) private sector growth and economic inclusion. 39. The development objective to “help restore power system capability to reliably meet domestic demand” remained highly relevant to the CPF, specifically to Objective 1c “… more efficient and 2 Country Partnership Framework (CPF) Serbia 2015. Report No. 94687. World Bank. Page 14 of 53 The World Bank Floods Emergency Recovery Project (P152018) sustainable power utility (In addition reform of utilities should help to increase access to utility services to marginalized groups (Roma in particular) and to those in rural areas (where poverty is much higher).).” Aligned with this objective in support to the energy sector, Component 1 helped restore power system capabilities and it became essential to ensure reliable electricity supply to households and firms to preflood level, in addition to improving reliability of the distribution system and restoration of energy assets. Further, this was critical to protect the vulnerable population affected by the flood during the winter season since an important share of heating needs is covered by electricity. Ensuring reliable electricity supplies and electricity output remains one of the key government priorities as laid out in its ‘Energy Sector Development Strategy of the Republic of Serbia for the period by 2020 with projections by 2030.’3 40. Though support to the agriculture sector is explicitly excluded in the FY16–20 CPF, the Farm Incentives Program was conceived before FY16 and included in the project to protect the livelihood of farmers affected by the flood and offset their income losses to cope with short-term income shortfall. These farmers are identified as some of the poorest, falling in the bottom 40 percent of the income distribution, and the CPF notes specifically that the rural population has been “particularly affected by the economic crisis and natural disasters.” The support to the Farm Incentives Program was therefore not an agricultural measure per se but rather an emergency income support program to prevent the flood disaster from driving more Serbians into poverty. At the same time, it supported farmers in the flood- affected areas to procure key agricultural inputs (for example, seeds and fertilizer) and therefore ensure that agriculture output and income could return to their preflood levels during the next season. In this sense, it is consistent with supporting the twin goals as expressed in the Serbia CPF. 41. The development objective on flood protection which was to support urgent rehabilitation of flood protection and drainage control infrastructure was aligned with Objective 2d “enhanced infrastructure networks” of the CPF. This objective ensured proper investment in rehabilitation and reconstruction of priority flood protection infrastructure while supporting the technical capacity of the government agencies in achieving the overall objectives and maintaining sustainability in the long term. Substantial capacity building financed under this component supported key government policies to improve the country’s resilience to climate risks and disasters given the high vulnerability of Serbia to these events. In late 2014, Serbia established the National Disaster Risk Management Program and raised more than US$70 million to build capacity for the country’s disaster risk reduction and crisis management systems. 42. In addition to the two focus areas, this project, through all of its developmental objectives, supports the cross-cutting theme of the CPF for FY16–20. Responding to climate change and natural disasters is essential to lower the risks that they pose to economic development and the impact on the poor, especially the disproportionate impact on rural areas which includes the most vulnerable segment of the Serbian population. 3 https://www.mre.gov.rs/doc/efikasnost- izvori/23.06.02016%20ENERGY%20SECTOR%20DEVELOPMENT%20STRATEGY%20OF%20THE%20REPUBLIC%20OF%20SERBIA.pd f. Page 15 of 53 The World Bank Floods Emergency Recovery Project (P152018) Assessment of Relevance of PDOs and Rating 43. With the alignment of the development objectives to both focus areas and the cross-cutting theme of the CPF, the relevance of the PDO is rated ‘High’. B. ACHIEVEMENT OF PDOs (EFFICACY) 44. The PDO consists of four objectives: (a) help restore power system capability to reliably meet domestic demand; (b) protect livelihoods of farmers in the flood-affected areas; (c) protect people and assets from floods; and (d) improve the Borrower’s capacity to respond effectively to disasters. Evidence for the achievement of the PDO indicators that were linked to these objectives is presented in Table , complemented by the results of intermediate outcome indicators and additional supporting information. Table 2. Achievement of PDO Indicators Outcome PDO Indicator Achievement Help restore power system Availability of power supplies 21,635 GWh of power restored meeting the capability to reliably meet restored to pre-flood levels revised target domestic demand through additional domestic generation/imports Protect people and assets Number of households receiving 24,944 households versus target value of from flood improved flood protection 9,000 Protect livelihood of farmers Agriculture production in flood All agricultural production in flood-affected in the flood-affected areas affected municipalities recovered areas recovered to preflood levels. to pre-floods levels Agricultural land receiving 25,041 ha of land received improved flood improved flood protection protection versus target value of 20,000 ha. Improve Serbia’s capacity to NA Establishing the Office for Reconstruction and better respond to disasters Flood Relief and increasing Serbia’s capacity to respond more effectively to disasters Assessment of Achievement of Each Objective/Outcome Objective 1 - Help restore power system capability to reliably meet domestic demand 45. The project achieved this objective through its activities under Component 1: Energy Sector Support, broken down in three sub-components: (a) Support for electricity purchases, (b) Urgent restoration of the distribution network and load management, and (c) Dewatering of the Tamnava West Field Mine. These are described as follows: (a) Support for electricity purchases. The project helped restore power system capability to reliably meet demand during the 2014–15 winter season by financing power imports to secure sufficient supply to meet demand shortages. The power supply was restored to preflood levels through additional domestic generation and imports as the power utility EPS Page 16 of 53 The World Bank Floods Emergency Recovery Project (P152018) managed to supply 21,635 GWh, which was in line with the revised target.4 The original target was 30,845 GWh, which proved to be higher than actually needed to meet the demand. This was one of reasons for the cost savings that were transferred to Component 3 during restructuring. There were no power shortages recorded during the project period, a fact that is mainly attributed to the project intervention. (b) Dewatering of the Tamnava West Field Mine. The project also helped restore critical energy assets as the dewatering of the Tamnava mine was successfully completed in the first quarter of 2015. That secured the regular production of coal and therefore regular production of electricity, reducing the need for imports as domestically produced electricity supply was reliable. (c) Urgent restoration of the distribution network and load management. This subcomponent was slightly modified, and implementation time of its operations was extended. The intermediate outcome indicators include (i) Number of meters replaced in affected area, which according to results were 20,000 and met the target; (ii) Number of public lighting meters installed, which the results show 1,162 which was on target; and (iii) five fixed substations to be rehabilitated in the flood-affected areas. While the client successfully procured the five substations, the project closed without their installation. The ostensible reason was lack of permits from the Ministry of Construction, Transport and Infrastructure (MCTI) and though the team and the CMU intervened severally to urge expedite measures from both MCTI and the Ministry of Mining and Energy, the government was unable to adopt and execute steps necessary to install the equipment. All of these measures contribute to ensuring the power system’s capability to reliably meet domestic demand. The installed meters would allow demand responses through smart meter functions, which would then contribute to a more secure electricity supply under emergency situations in the future. Moreover, mobile substations were procured and stored for emergency use, to help enhance EPS’s capability in prompt restoration of any disruption in its distribution networks. 46. In the case of Component 1, the Government maintains the reliability and security of power supplies as one of its key priorities in the energy sector as laid out in its ‘Energy Sector Development Strategy of the Republic of Serbia for the period by 2020 with projections by 2030’. Specifically, in the power sector, the country seeks to ensure a sustainable exploitation of coal resources, including from the Tamnava mine, for its power generation needs while upgrading its thermal power plants to improve their conversion efficiency and reduce the environmental footprint of the power sector. 47. After achieving the objectives of the first component, there remained available resources, which were later on reallocated to finance activities related to the flood protection component. Efficacy shortcomings under this component included incompletion of the substations. Therefore, the efficacy of this objective is considered Substantial. 4The original target was 30,845 GWh. Since the target was reduced, a split evaluation was considered, but it will not have an impact on the outcome rating. Page 17 of 53 The World Bank Floods Emergency Recovery Project (P152018) Objective 2 - Protect livelihoods of farmers in the flood affected areas 48. The project achieved this objective through its activities under Component 2: Agricultural Sector Support. This component on protecting the livelihood of farmers in the flood-affected areas was successful in covering all the farmers in the affected areas. Component 2 was not divided into subcomponents. The indicators of this objective are described below: (a) Agricultural land receiving improved flood protection. Evidence shows that 25,041 ha5 out of the targeted 20,000 ha received improved protection from floods, which exceeded the original target. (b) Agriculture production in flood affected municipalities recovered to pre-flood levels. A study was conducted to measure the flood’s effect and the recovery of production to preflood levels. It showed that agricultural production measured by total output in crops and crops production recovered in 2016, by 9.84 percent higher compared to 2013 base year, whereas total output of livestock and livestock products recovered in 2017 by 1.54 percent higher than in 2013.6 Therefore, it was estimated that the full recovery to preflood levels was achieved in 2017.7 However, it is difficult to attribute all the recovery to the project as agricultural production is volatile and depends on many factors. In 2017, due to a severe drought, agriculture production declined sharply in Serbia by about 20 percent, suggesting high sensitivity to frequent climatic disruptions. (c) Farms benefiting from Government support in the affected areas. Grant support through the Farm Incentives Program provided critical support to farmers, helping protect their livelihood and offset their income losses in the aftermath of the floods. A total of EUR 53.1 million was provided to 101,839 farmers living in the flood-affected areas between May and December 2016. This result exceeds, by about 2,000 farmers, the revised targeted number of beneficiaries. The main reason was that the Farm Incentives Program allocation was slightly modified during implementation, by reducing the subsidy to large farms. As a result, resources for small- and medium-size farms increased, resulting in more farms benefiting from the program. However, the baseline number seems to have been overstated at 120,000, suggesting incorrect information at appraisal. 49. This component exceeded the target within the allotted time frame. The efficacy rating is therefore High. Objective 3 - Protect people and assets from floods 50. The project achieved this objective through its activities under Component 3: Flood Protection, divided into two subcomponents: (a) Investments in Flood Protection; (b) Project Implementation Support for Flood Management. This objective was successfully achieved after receiving EUR 17.3 million 5 Based on data received from the borrower’s project completion report. 6 Source: A Study on Project Development Objective Indicators. Levels of Agriculture Production Recovery in the Municipalities Affected by the 2014 Floods. Natalia Bogdanov, Belgrade, June 2018. Table 7. 7 Source: Project Completion Report. FERP, Component 2: Agriculture Sector Support, Component 3: Flood Protection. DWM, Ministry of Agriculture, Forestry and Water Management, Republic of Serbia. PIU of FERP, October 2019. Page 42, Table 13. Page 18 of 53 The World Bank Floods Emergency Recovery Project (P152018) reallocated from support to the energy sector. The reallocation of funds also required extension of implementation by 22 months. After some initial delays in implementation, all activities related to this objective were executed successfully by the project closing date. 51. Investments in Flood Protection. This subcomponent financed rehabilitation and reconstruction of priority flood protection and drainage control infrastructure through the provision of goods, civil works, and consultants’ services in the flood-affected areas and areas vulnerable to flooding. To the extent possible, rehabilitation and reconstruction followed the principle of ‘building back better’ after a natural disaster. This was accomplished through the application of improved design and construction standards to both reconstruct damaged works and strengthen existing ones. The number of households receiving improved flood protection was 24,944, which exceeded the original target of 9,000 by a factor of more than 2.5 as a result of the additional resources allocated during the project restructuring. The number of direct beneficiaries was 362,244, which also exceeded the original estimate of 360,000, of which 30 percent were female. Length of flood protection reconstruction/rehabilitation works was another indicator used. The evidence shows that 71 km instead of the targeted 65 km were completed by October 2019. In addition, there was a case of mis-procurement associated with the acquisition of two meteorological radars to improve weather monitoring and preparedness. As a result, the portion of the activities related to these radars could not be completed, which had a moderate impact on the achievement of this objective. 52. The results exceeded the targets, so the efficacy ranking is considered High. Objective 4 - Improve the Borrower’s capacity to respond effectively to disasters 53. A CERC, with an allocation of EUR 0, is defined in the preparation for a future adverse natural or man-made event that causes a major disaster when the Government of Serbia may request the project’s funds to be reallocated from other components to this component to partially cover emergency response and recovery costs. This component could also be used to channel additional funds should they become available because of the emergency. 54. Though there were no results indicators associated with this component, the capacity of Serbia to respond to disasters was improved through other components of the project. The World Bank supported the government in establishing the Office for Reconstruction and Flood Relief as the implementing unit in charge of coordinating all financial assistance for the flood recovery. In addition, there were US$10 million in trust fund financing towards long-term mitigation and resilience building outside the project. 55. The CERC was never triggered. However, at the time of flooding in 2016, the borrower sought to trigger it. The Loan Agreement assigned responsibility to prepare the Project Operations Manual (POM) for CERC to the borrower but not a specific agency. As a result, the POM was never prepared and in 2016 the CERC was not able to be deployed during implementation of the project. Therefore, its efficacy is rated Not Applicable. Page 19 of 53 The World Bank Floods Emergency Recovery Project (P152018) Justification of Overall Efficacy Rating 56. Given the full achievement of the targets set out in the PDO indicators, and the almost full achievement of the intermediate indicators, the overall efficacy is considered Substantial. The project almost fully achieved its objectives. C. EFFICIENCY Assessment of Efficiency and Rating 57. At the time of project preparation, simplified economic and financial analyses were conducted. Due to data and time constraints, a simplified economic analysis and a financial analysis for Component 1: Energy Sector Support using financial data for EPS, an economic analysis for Component 2: Agriculture Sector Support, and an economic analysis for Flood Protection Investments were done. This ICR follows the same structure. 58. The benefit-to-cost ratio (B/C) is estimated at 12.7. The proposed rating for the energy sector support efficiency is High. The economic internal rate of return (EIRR) for Agriculture Support is 24.4 percent compared to EIRR of 15–30 percent benchmark, and the B/C is 3.5. The proposed rating for Agriculture Support efficiency is Substantial. The Flood Protection Support EIRR is estimated at 77.3 percent, which is much higher than the approximate 20 percent benchmark estimated at the appraisal stage. In addition, the B/C is strong at 8.2.8 59. Overall results are as shown in Table . Table 3 - FERP Cumulative Results from Economic Analysis Net Present Value EIRR (%) B/C (NPV) (EUR, millions) FERP cumulative 2,338.6 n.a. 12.7 Energy Sector Support 2,012.3 n.a. 17.1 Agriculture Sector Support 161.8 24.4 3.5 Flood Protection Support 163.8 77.3 8.2 60. For a detailed economic analysis, please refer to annex 4. 61. Analysis of administrative and operational efficiency of the project suggests that the largest components (import of electricity and agriculture sector support) disbursed as planned and were highly efficiency. There were some savings in energy sector support, and the funds were reallocated to flood protection support, suggesting realized gains due to efficiency. The reallocation of funds resulted in the extension of the project duration, slightly increasing management supervision costs. It should be noted 8At the appraisal stage, only one economic model was developed, for Component 1: Energy Sector Support, and this model was not available for comparison at the completion stage. No economic models were developed for either Agriculture Support or Floods Protection Support at the appraisal stage, but some benchmarks were used based on other similar projects. Therefore, the results from the efficiency analysis might not be fully comparable despite the use of the same assumptions in both cases. Nonetheless, the results show a high efficiency of investments in all three components, suggesting a High ranking. Page 20 of 53 The World Bank Floods Emergency Recovery Project (P152018) that the extension of 22 months enabled the project to achieve results beyond the targets and therefore improve the overall efficiency. 62. Based on the results shown above, the proposed overall rating for efficiency is Substantial. D. JUSTIFICATION OF OVERALL OUTCOME RATING 63. Based on the abovementioned assessment and ratings—the relevance of the PDO is rated High, the efficacy of the PDO achievement is rated Substantial, and the efficiency of the project is rated Substantial—the overall outcome of the project is rated Satisfactory. E. OTHER OUTCOMES AND IMPACTS (IF ANY) Gender 64. At the time this project was prepared, gender tagging was not yet an established practice within the World Bank. As a result, this project does not have the now-familiar triplex of analysis, activities, and indicators linked to gender aspects. The project’s gender dimension was concentrated on the core indicator of proportion of beneficiaries who are female. Initially, this was set at 51 percent, but at the restructuring, it was revised to reflect the data that became available after appraisal. This number was measured and reported continuously, and the target of 30 percent was achieved. Institutional Strengthening 65. Substantial capacity was built during the implementation of emergency activities at EPS. Specifically, global best practices were deployed to support the dewatering of the largest flooded open pit coal mine in the world. External technical support was made available to ensure that the mine would not collapse during the dewatering, threatening the sustainability of the power sector. Capacity was also built at EPS to identify and address potential environmental and social impacts of dewatering the mine. The company and the contractor implemented a detailed environmental and social monitoring program during this activity. EPS’s capacity in procuring large amounts of imported power also improved through the implementation of standardized international procurement of these (on a multi-monthly, monthly, and weekly basis) in international tenders. The company had experience in power trading but not at the scale required in the post-flood period. 66. Component 3 of the project financed measures to strengthen technical capacity for improved flood prevention and management. These included technical studies to identify priority flood prevention infrastructure works, preparation of technical designs and bidding documents, TOR for such activities, and options for strengthening institutional arrangements and DRM in the water sector. This included equipment for improved response in the event of flooding, for example, mobile flood defenses and protection and rescue team equipment for national and regional water management institutions and the sector for emergency management. The Office for Reconstruction and Flood Relief was established as the implementing unit in charge of coordinating all financial assistance for the flood recovery. This was one of the successful outcomes of the project and the World Bank’s engagement in supporting the government during recovery period. In addition, longer-term risk mitigation and resilience building were Page 21 of 53 The World Bank Floods Emergency Recovery Project (P152018) addressed outside the project through a series of World Bank-executed and recipient-executed trust fund grants totaling more than US$10 million. Poverty Reduction and Shared Prosperity 67. The floods of 2014 and 2016 resulted in fatalities and caused significant economic hardship for much of the population of Serbia, disproportionally affecting poor and vulnerable groups. In addition, these floods had a significant impact on growth, employment, and income and, through these channels, on poverty and shared prosperity. The negative impact on livelihoods and employment is also more acute in vulnerable groups such as rural elderly farmers, which account for a sizable share of Serbia’s poorest 40 percent. By ensuring continued energy supplies, improving protection for the population against future floods, and compensating for lost income for farmers, the project had a positive impact on the poor and helped restore income growth of the poorest 40 percent, thereby reducing poverty and promoting shared prosperity. III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME A. KEY FACTORS DURING PREPARATION 68. Realistic objectives and design. In line with the high relevance of the PDO with Serbia’s development priorities, and emergency mode preparation of the project, FERP led to realistic expectations being set. 69. Readiness for implementation. This project undertook appropriate measures to ensure readiness for implementation in spite of the emergency nature of the project. The activities chosen, for instance under Component 1, were ready to implement due to the existence of detailed studies allowing tenders to be launched immediately after effectiveness. All energy sector activities in Component 1, except for the mine dewatering, were based on commodities or readily available goods and technologies. The use of retroactive financing also helped the client launch key procurement packages before Board approval to avoid delays in the implementation of key activities. Before effectiveness, activities under Component 3 were at an advanced stage of readiness, with final estimates, specifications, and designs. 70. Institutional arrangements. EPS and the PIU in DWM were selected based on their readiness and experience working with the World Bank which allowed the team to minimize the risk of delays in the setup phase. However, EPS did not have a designated implementing unit and the project team was comprised of staff of EPS which throughout implementation created challenges regarding responsiveness. B. KEY FACTORS DURING IMPLEMENTATION 71. Floods of 2016. A recurrence of flooding in March of 2016 led to substantial disruption of the country and of the project. While these floods were smaller and caused less displacement of people and economic activity than the floods of 2014, they were of sufficient magnitude that the Government of Serbia sought to invoke the CERC under FERP to use its funds to respond. In the event, the POM associated with the CERC was not ready at the time, prompting the World Bank to restructure FERP instead to incorporate elements of disaster response. Page 22 of 53 The World Bank Floods Emergency Recovery Project (P152018) 72. Human resources and organizational capacity. The overall risk for this operation was substantial at appraisal due to the nature of the project having investments in several sectors with diverse implementation arrangements. The dewatering of the Tamnava mine and power imports were the riskiest activities due to their technical, fiduciary, and environmental impacts. Once their implementation was completed, the overall risk ratings were lowered. EPS was selected for the majority of the investment based on its adequate technical, fiduciary, and monitoring capacities, including experience in implementing World Bank projects. After the emergency period, the unit responsible faced a high turnover rate of staff and managing directors with minimum or no handover procedure in place. After the restructuring, EPS was not able to maintain continuity of staff, and the communication with the World Bank team suffered as a result. 73. PIMO was also an important contributor to project success. Reporting to the Government Office, it took over the role of project coordination once the Office for Reconstruction and Flood Relief ceased to operate at the end of 2015 as mandated in its enabling legislation. For the remainder of project implementation, PIMO played a strong coordinating role and its placement in a central agency helped it to have the visibility and resources needed to maintain its effectiveness. 74. Fiduciary arrangements. The project team made an effort to comply with existing Government and World Bank standards by carefully studying all project documents, guidelines, and regulations relevant to the management units. There were two incidents regarding procurement, as indicated in the procurement section. During implementation, one PIU did not maintain a procurement specialist for over a period of one year, which does not comply with the World Bank procurement policy. However, this was resolved with guidance from the World Bank team and efforts from the PIU. In another instance related to purchasing of meteorological radars, there was a declaration of mis-procurement due to lack of supervision of the contracting process and lack of communication from the PIU informing the World Bank team of changes. As a result, a total amount of EUR 3,200,000 was cancelled from the project. 75. Restructuring. Since this project was prepared in response to an emergency situation, up-to-date information was not available at the design phase, though the project was prepared using the best data available to the team. During the restructuring, PDO indicators were revised to reflect the new information which had become available over the first phase of implementation. Several intermediate results indicators were also updated. Early achievement of the targets and the cost savings under Component 1 made possible the reallocation of funds to Component 3. 76. In addition, the restructuring reallocated funds between disbursement categories, moving US 17.36 million from Component 1 to Component 3 to finance (a) works to rehabilitate flood protection infrastructure, (b) feasibility studies and detailed designs for flood protection infrastructure, and (c) procurement of two meteorological radars to improve weather monitoring and preparedness. In light of that reallocation, it was also deemed necessary to extend the project closing date by 22 months to allow time for these new activities to complete over two full construction seasons. In the end, this arrangement allowed implementation to be almost fully completed on time using nearly full loan resources and with almost full achievement of its objectives by the new closing date. 77. The restructuring also marked a significant change in focus for the operation. In its earliest stages, the project was concentrated in the energy sector, with power purchases to meet shortages and emergency dewatering of the mines supplying fuel to Serbia’s coal-fired generating stations. A second Page 23 of 53 The World Bank Floods Emergency Recovery Project (P152018) early focus was the delivery of income support to affected farmers. However, these activities were mostly completed by the end of 2015 to mid-2016. As mentioned earlier, 79 percent of project funds were either disbursed or committed at that time and there was an opportunity to restructure the project to concentrate more on flood defenses and preparedness. This shift was reflected within the World Bank by transferring the responsibility for the project from the Energy Global Practice to the Water Global Practice. IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME A. QUALITY OF MONITORING AND EVALUATION (M&E) M&E Design 78. The project followed a clearly designed structure, outlining how project activities of each component would contribute to achieving the PDO. This is despite the fact that the project was an emergency loan distributed across various sectors. The PDO was disaggregated succinctly into four main PDO indicators, which covered all the areas affected by the 2014 flood events and resulted in a better level of preparedness for future events. 79. The emergency nature of the operation did make it necessary to rely upon preliminary information for some of the targets and baselines in the Results Framework. Some of those numbers were later proven to be inaccurate and in need of adjustment. These were revised during the restructuring to make use of more reliable information gained during implementation up to that point. 80. The PIU for Component 1, EPS, was already equipped with monitoring and evaluation (M&E) capacity, as it had extensive experience with World Bank projects. Responsibility for M&E for Components 2 and 3 was assigned to the new PIU within the DWM under the MAEP. 81. There was a dated covenant in the Legal Agreement that required the DWM PIU to maintain staff to the satisfaction of the World Bank which also included an M&E specialist. This ensured that the new PIU had the capacity to conduct M&E throughout the project. M&E Implementation 82. The project M&E work was conducted by qualified staff as agreed with the World Bank, and these staff undertook continuous monitoring, analysis, and reporting on outcome and intermediate indicators regarding the project’s progress. This is specifically noted for Components 2 and 3. The M&E system was implemented using existing structures to which the PIU already had access, in existing regional branches of the ministry, for example. This arrangement was able to maintain satisfactory M&E performance throughout implementation. M&E Utilization 83. The results of the M&E process were compiled and utilized by project management and decision makers to monitor progress closely and make changes to the project’s design during the restructuring. The M&E process allowed the project team to keep close watch over implementation progress. This was Page 24 of 53 The World Bank Floods Emergency Recovery Project (P152018) instrumental in the decision, during the restructuring, to transfer funds from Component 1 to Component 3, which allowed timely restructuring and, by extension, the achievement of better results. Justification of Overall Rating of Quality of M&E 84. The M&E functioned well throughout implementation. Considering the comprehensively designed, implemented, and utilized M&E framework, the overall rating of quality of M&E is Substantial. B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE 85. Fiduciary. Financial management performance was assessed throughout the implementation as acceptable, and no major issues were identified by the auditors or the World Bank’s supervision. The public enterprise EPS oversaw the fiduciary responsibilities for Component 1 while the MAFWM through the DWM and the DAP did the same for Components 2 and 3. These responsibilities included planning and budgeting, accounting, financial reporting, internal controls, withdrawals and flow of funds, and external audit. These were assessed as comprehensive and reliable throughout implementation. 86. A full set of interim unaudited financial reports was prepared for each calendar quarter throughout the life of the project for all project components and all sources of funding. They were due within 45 days after the end of each quarter. They were submitted on time and assessed to be reliable. 87. The audit reports were received on time in most instances. The latest audit reports received were for the period January 1 to December 31, 2018. The audit for Component 1 and EPS was conducted by PWC, a private audit firm acceptable to the World Bank. The auditor issued a modified opinion on the basis that the company did not recognize the cleanup provisions for depots in its thermal power plants in Kolubara, Kostolac, Morava, Nikola Tesla A, and Nikola Tesla B. The same issue had been identified in the previous year’s report; however, it is not considered to have had a direct impact on project implementation. The audit for Components 2 and 3 was conducted by Deloitte, a private auditing firm acceptable to the World Bank. The auditor issued a clean (unmodified) opinion with emphasis of matter regarding the part of the loan cancelled in the amount of EUR 3.2 million due to mis-procurement. The same auditor issued a clean opinion for 2017. The audit reports for the final year 2019 are due in June 2020 and therefore not available as of this writing. 88. Internal controls and procedures defined in the POM were applied throughout implementation. 89. Procurement. The departure of the PIU procurement specialist in the DWM in early 2016 was not reported to the World Bank team, and the position remained unfilled for more than a year. The World Bank team was not informed immediately of the departure of this staff, and no action was taken by the PIU. Due to the lack of communication and guidance between the World Bank and the PIU, some activities were delayed. However, a new PIU procurement specialist was selected in December 2017 through Single- Source Selection. The pending procurement activities were then resumed. 90. In October 2018, the World Bank declared mis-procurement for an activity under Component 3 for the procurement of two meteorological doppler radar systems in the amount of EUR 3,200,000 and informed the borrower that this amount would be cancelled. Before the declaration of the mis- procurement, the World Bank was notified of changes/amendments to the bidding documents (extension Page 25 of 53 The World Bank Floods Emergency Recovery Project (P152018) of the bid submission deadline and changes to the location of the radars after the start of the bidding process). This appears to be a case in which the PIU did not comply with the World Bank procurement guidelines as required for a prior review contract. However, the issue was only discovered when a bidder directly contacted the World Bank team to complain about late changes in the bidding procedure. Better communication and closer supervision could have substantially improved the chances of detecting and resolving these issues before they reached a crisis point, which eventually led to declaration of mis- procurement – the first such in Serbia in over 10 years. 91. Social impacts and implementation. Land acquisition and clearance of public land have been conducted in compliance with the World Bank social safeguards requirements, the project Resettlement Policy Framework (RPF), and applicable Abbreviated Resettlement Action Plans (ARAPs) and Serbian expropriation law. The World Bank Operational Policy on Involuntary Resettlement (OP 4.12) was triggered since the infrastructure rehabilitation and reconstruction work under Component 3 (investments in flood protection) in some cases led to temporary acquisition of private land for securing the right-of-way. Since the size, scale, and location of activities could not be determined during project preparation, an RPF and Environment and Social Management Framework were prepared to mitigate potential environmental and resettlement impacts. 92. Site-specific ARAPs were prepared for six construction sites. These were published on the website of the DWM and publicly available at local municipalities. Monthly Social Safeguards Reports were regularly prepared and delivered to the World Bank during the project. 93. No land take occurred without prior compensation. Out of a total of 12 flood protection subprojects under Component 3 (including 4 which were added during the restructuring), land or asset loss was identified for 6 sub-projects. Three of these sub-projects had impacts affecting private land and required land acquisition by eminent domain, while the remaining three subprojects required clearance of private assets attached to public land. No livelihood impacts nor physical displacement occurred on any of the subprojects. 94. It is noted that the PIU submitted monthly social monitoring reports regularly. 95. Environmental. Project activities under Component 1 (Dewatering of the Tamnava West Field mine) triggered OP 7.50 for Projects on International Waterways. On September 11, 2014, the International Commission for the Protection of the Danube River confirmed that riparian member states were notified, and no issues were raised. 96. The policy on Pest Management (OP 4.09) was also triggered because of Component 2: Agricultural Sector Support. In agreement with the World Bank in 2015, the PIU prepared a Pest Management Study on best practices and subsidies, which was reviewed and approved by the World Bank in April 2016. 97. The policy on Environmental and Social Assessment (OP 4.01) was triggered. Under Component 3, site-specific Environmental Management Plans (EMPs) were prepared for all 12 subprojects/construction sites for rehabilitation and improvement of flood protection. As the proposed activities under Component 3 relate only to rehabilitation and/or repair of already existing infrastructure, Page 26 of 53 The World Bank Floods Emergency Recovery Project (P152018) which will not change their existing footprint, the Environmental and Social Impact Assessments in most cases were not prepared. 98. EPS was responsible for the implementation of Component 1 and maintained an environmental safeguards specialist on staff through the completion of dewatering the mine. However, later in implementation, the rehabilitation of substations stalled, and though EPS assigned a staff for this position the role was not dedicated, and quality suffered (the last 12–18 months of the project before closing). No action was taken on this deficiency due to other issues related to the completion of this subcomponent such as permitting and ownership issues of the substation which eventually resulted in incompletion of the task. 99. Regarding Components 2 and 3, the PIU responsible complied with all environmental safeguard policies. Site-specific EMPs were published on the website of the DWM and made publicly available in affected local communities (municipalities). Semiannual Environmental Monitoring Reports for Component 3 were prepared and delivered to the World Bank, throughout project implementation. Even though there were some delays in submission of these reports, all supervision of EMP implementation on the ground was satisfactory. C. BANK PERFORMANCE Quality at Entry 100. Considering the emergency nature of the project, the task team was under substantial time pressure to prepare the project. Under those circumstances, the team opted to finance activities for which either implementation was straightforward (for example, financing electricity purchases and farm income support payments) or the implementation readiness was high, especially in the case of flood protection works for which detailed technical studies were already available. 101. This project focused on a set of activities that were most urgently needed to recover rapidly from the disaster. This is natural and logical in the context of the events which prompted the project. It also resulted in relatively complicated institutional arrangements which grouped three disparate sectors together, however necessary this may have been under the circumstances. Having two implementing agencies and a complicated institutional structure is often highly detrimental to implementation, so the team took a risk by opting for this arrangement during preparation. As discussed elsewhere, this risk proved to be justified and yielded good results. 102. As mentioned earlier, the Results Framework designed at appraisal made use of the best information available at the time in the context and pressure of emergency project preparation. This means that some indicators were chosen (for example, Number of energy efficient light bulbs installed) without fully determining their feasibility. As a result, some indicators needed to be revised during the restructuring. After revision of the Results Framework, implementation moved forward and resulted in most indicators being exceeded. 103. The assessment of the risks at entry differs greatly from the assessments during implementation, implying that risks may have been overestimated at appraisal. For example, in the PAD, the overall implementation risk was rated Substantial. Early in implementation, in the space of one Implementation Page 27 of 53 The World Bank Floods Emergency Recovery Project (P152018) Status and Results Report (ISR) less than one year after effectiveness (January 2016), the stakeholder risk went from High to Moderate due to the completion of dewatering of the Tamnava mine. This may well have been justified once risks associated with dewatering were past but should have been better documented. In another ISR (September 2016), all the risks were modified from Moderate to Low without clear justification. However, beyond the restructuring, the risk ratings remained mostly steady with proper clarification. 104. The PAD refers in several places to applying the principle of ‘building back better’ in the disaster recovery activities of Components 1 and 3. However, this term is never clearly defined or quantified beyond generic reference to improved design and construction standards, nor does it figure in the Results Framework. This leads to a lingering ambiguity that makes it difficult to assess whether this principle was observed and what effect it might have had. Capacity building and increasing resilience were mentioned to have similar impact but were similarly difficult to measure. Quality of Supervision 105. Formal supervision occurred at longer intervals than the recommended World Bank practice. The ISR records show that there were only six implementation support missions from effectiveness to closing. There were co-TTLs based in Belgrade throughout the implementation so there was almost certainly more regular contact between the World Bank and the implementing agencies outside of formal missions. However, there are indications that implementation may have suffered as a result of insufficient formal supervision. In the instance of the declaration of mis-procurement, better communication and closer supervision could have substantially improved the chances of detecting and resolving the issues before they reached a crisis point. 106. There were also problems with essential documentation such as Aide Memoires and the Midterm Review, which played a role during the flood event of 2016 when the borrower requested to trigger the CERC. Critical discussions with the PIU on deploying the CERC to respond to the later floods were not documented which led to some tension during the time a decision had to be made. The CERC was never triggered as the POM was not ready. As a result, the CERC was not ready to be deployed, and this made the restructuring necessary. 107. The project team provided ongoing support on fiduciary and safeguard issues during project implementation. In particular, it provided support on safeguards and ensured compliance with World Bank policies on financial management. 108. The achievement of the PDO was consistently rated ‘Satisfactory’ throughout implementation indicating both project team and implementing agencies remained focused on development impact and achieved targets despite shortcomings. Justification of Overall Rating of Bank Performance 109. Based on the quality of World Bank performance at entry and at supervision, the overall rating of World Bank performance is Moderately Satisfactory. Page 28 of 53 The World Bank Floods Emergency Recovery Project (P152018) D. RISK TO DEVELOPMENT OUTCOME 110. This project was an emergency loan to address urgent needs following severe floods. As such, some of the interventions are for one-time effect such as dewatering the mine and supporting farmer incomes in a crisis. 111. Because these are disaster recovery interventions, it is possible they could reoccur especially under the uncertainties of climate change. These possible changes could affect the development outcome of this project. On the other hand, the remaining interventions such as flood prevention infrastructure should help mitigate such scenario of future changes. These infrastructures were designed and completed, are durable if properly maintained, and should help in case of another flood event. 112. Under Component 1, as noted elsewhere, the client successfully procured equipment for rehabilitation of five substations. However, due to separate contracting of equipment and installation, at project closing installation was not complete at any of the five sites due to permitting problems. At project closing the team was not aware of a credible follow up plan to assure installation without further delay. This is indicative of the capacity and coordination challenges that continue and remain a concern even after project closure. 113. There exist two programs or policies that will ensure the sustainability of the development results achieved by Component 2. In the medium term, the sustainability of a reliable support system for farmers will be assured through the implementation of Serbia’s Strategy for Agriculture and Rural Development, which outlines the path for sector development during 2014–24. Serbia’s EU accession path and its convergence with EU farm support programs under the EU Common Agricultural Policy will ensure the sustainability of farm support schemes in the long term. 114. The rehabilitation and restoration of infrastructure under Component 3 were of an emergency nature, aimed at meeting immediate needs, and enabling a return to normal economic and social activities in the affected areas. At closing, these components have shown their ability to withstand similar future floods. 115. The sustainability of this component will depend on the Government’s ability to preserve the rehabilitated assets through proper maintenance. This has a financial implication which will need government support, though it is possible that some of the institutional structures created due to this project, such as PIMO, may help in this respect. V. LESSONS AND RECOMMENDATIONS 116. Technical designs for protective works are crucial to rapid response in an emergency. When the additional resources from Component 1 became available, they could be quickly absorbed into Component 3 because a significant set of technical designs for protective works already existed and could be implemented right away. Otherwise, they can take months or years to prepare, taking up scarce time in an emergency situation. Emergency operations in disaster recovery offer a rare opportunity to finance and implement protective measures, either ‘hard’ infrastructure or ‘soft’ institutions or procedures, that can then build resilience against future disasters. The increased attention focused on protective measures Page 29 of 53 The World Bank Floods Emergency Recovery Project (P152018) and disaster management that naturally follows in the aftermath of a disaster creates sudden momentum in favor of taking action, which can be harnessed, particularly if technical studies already exist as they did in the case of this project. For this reason, it would be worth considering ways in which the World Bank- funded investment projects could finance technical studies for more infrastructure than each can actually build. In this way, the borrower would gradually accumulate a library of studies for as-yet-unfunded measures that could be taken off the shelf and deployed quickly if a disaster occurs. This kind of rapid deployment would contribute to the country’s overall disaster resilience. 117. Flexible procurement approaches and retroactive financing are important tools to help countries respond to emergencies. The project used national procurement approaches for the dewatering of the Tamnava mine to allow EPS to launch the international procurement shortly after the floods happened and much before project effectiveness. Likewise, the World Bank allowed the procurement of power imports following industry practices before the project was approved by the World Bank. As a result, over EUR 80 million was disbursed under retroactive financing. A waiver from the Managing Director related to the World Bank’s inspection and audit right was granted to allow the retroactive financing for power imports. Flexibility in the procurement approaches and retroactive financing supported a timely response by the client to the disaster. In a situation of limited fiscal resources, retroactive financing that was provided allowed EPS to finance these emergency expenditures at affordable financing costs to the company and the country. 118. Preparation of CERC POMs must be treated as a priority. In this project, the precise responsibility for preparing the CERC POM was not clearly assigned and no deadline was fixed for its completion. Though the CERC was a new instrument at the time of appraisal, this project offers a concrete example of why teams should put priority on producing a POM for CERC promptly. This project included a CERC, which implied it could be deployed in case of need when in fact there was a missing element. In effect, the absence of the CERC POM prevented the World Bank from delivering on one of its obligations. This could have been avoided if the CERC POM had been produced early in implementation or even during preparation. This project illustrates how clear responsibility and timelines for CERC POM preparation are essential to proper functioning of the instrument in case of need. 119. Strong political championship can be decisive. In any operation, it can be risky to have two PIUs from different sectors and different entities. This project had two different implementing agencies in different sectors, but the risk was mitigated by choosing a strong entity to coordinate between them. For FERP, that was the Office for Reconstruction and Flood Relief and later the responsibility was transferred to PIMO. That political championship and executive authority allowed the project to overcome what could have been a very problematic institutional arrangement under different circumstances. 120. Supply and installation of equipment should not be separated in contracting. The rehabilitation of fixed substations was an important project activity, and the intention had been to procure the needed goods and installation services together in integrated contracts as provided in the Procurement Plan. However, at the request of EPS, the procurement of equipment was separated from the installation services with the justification provided by EPS that there are local firms qualified to execute installation. In addition, EPS did not facilitate obtaining the required construction/land permits though this was requested by the World Bank team. The result was that the equipment was purchased and delivered, but by project closing, there was still no clear solution apparent to have them installed. Had the contracts not been separated in this manner, this problem could have been avoided. Page 30 of 53 The World Bank Floods Emergency Recovery Project (P152018) . Page 31 of 53 The World Bank Floods Emergency Recovery Project (P152018) ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS A. RESULTS INDICATORS A.1 PDO Indicators Objective/Outcome: Help restore power system capability to reliably meet domestic demand Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Availability of power supplies Gigawatt 0.00 30845.00 21635.00 21635.00 restored to pre-flood levels through additional domestic 01-Jun-2014 29-Dec-2017 30-Dec-2016 31-Oct-2019 generation/imports. Comments (achievements against targets): 100% target achieved. Objective/Outcome: Protect livelihoods of farmers in the flood affected areas Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Agricultural land receiving Hectare(Ha) 0.00 18950.00 20000.00 25041.00 improved flood protection 01-Dec-2014 29-Dec-2017 31-Dec-2019 31-Oct-2019 Page 32 of 53 The World Bank Floods Emergency Recovery Project (P152018) Comments (achievements against targets): 125% target achieved. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Agriculture production in Yes/No N Y Y Y flood affected municipalities recovered to pre-floods levels 01-Dec-2014 29-Dec-2017 29-Dec-2017 31-Oct-2019 Comments (achievements against targets): 100% target achieved. Objective/Outcome: Protect people and assets from floods Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of households Number 0.00 7000.00 9000.00 24944.00 receiving improved flood protection 01-Dec-2014 29-Dec-2017 31-Dec-2019 31-Oct-2019 Comments (achievements against targets): Page 33 of 53 The World Bank Floods Emergency Recovery Project (P152018) 277% target achieved. A.2 Intermediate Results Indicators Component: Component 1: Energy Sector Support Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Dewatering of the Tamnava Yes/No N Y Y Y West Field mine finalized 01-Jun-2014 31-Dec-2015 31-Dec-2015 31-Oct-2019 Comments (achievements against targets): 100% target achieved. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of meters replaced Number 0.00 20000.00 20000.00 20000.00 in affected areas 01-Jun-2014 29-Dec-2017 31-Dec-2018 31-Oct-2019 Comments (achievements against targets): Page 34 of 53 The World Bank Floods Emergency Recovery Project (P152018) 105.81% target achieved. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of fixed substations Number 0.00 5.00 5.00 0.00 rehabilitated in the flood affected areas 01-Jun-2014 31-Jul-2019 31-Jul-2019 31-Oct-2019 Comments (achievements against targets): The target was not achieved. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of public lighting Number 0.00 1192.00 1192.00 1162.00 meters installed 01-Jun-2014 31-Dec-2018 31-Dec-2018 31-Oct-2019 Comments (achievements against targets): 97.48% target achieved. Page 35 of 53 The World Bank Floods Emergency Recovery Project (P152018) Component: Component 2: Agricultural Sector Support Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Farms benefiting from Text 120,000 120000 100000 101839 Government support in the affected areas 01-Jun-2014 29-Dec-2017 31-Dec-2015 31-Oct-2019 Comments (achievements against targets): 101.8% target achieved. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Direct project beneficiaries Number 360000.00 441000.00 350000.00 362244.00 01-Jun-2014 29-Dec-2017 31-Dec-2019 31-Oct-2019 Female beneficiaries Percentage 30.00 51.00 30.00 30.00 Comments (achievements against targets): 103.5% target achieved. Page 36 of 53 The World Bank Floods Emergency Recovery Project (P152018) Component: Component 3: Flood Protection Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Length of flood protection Kilometers 0.00 71.50 65.00 71.00 reconstruction / rehabilitation works 01-Jun-2014 29-Dec-2017 31-Dec-2019 31-Oct-2019 Comments (achievements against targets): 109% target achieved. Page 37 of 53 The World Bank Floods Emergency Recovery Project (P152018) B. KEY OUTPUTS BY COMPONENT Objective/Outcome 1: Help restore power system capability to reliably meet domestic demand 1. Availability of power supplies restored to pre-flood levels through additional domestic Outcome Indicators generation/imports (Gigawatt) 1. Dewatering of the Tamnava West Field mine finalized (Yes/No) 2. Number of meters replaced in affected areas (Number) Intermediate Results Indicators 3. Number of fixed substations rehabilitated in the flood affected areas (Number) 4. Number of public lighting meters installed (Number) 1. The Tamnava West Field mine was dewatered. Key Outputs by Component 2. Total 20,000 meters were replaced in the affected area, meeting the target. (linked to the achievement of the Objective/Outcome 1) 3. No substations have been rehabilitated, compared to target of 5 substations9. 4. 1,162 public lighting meters were installed, compared to target of 1,192. Objective/Outcome 2: Protect livelihoods of farmers in the flood affected areas 1. Agricultural land receiving improved flood protection (Hectare(Ha)) Outcome Indicators 2. Agriculture production in flood affected municipalities recovered to pre-floods levels (Yes/No) 1. Farms benefiting from Government support in the affected areas Intermediate Results Indicators 2. Direct project beneficiaries (Number) 3. Female beneficiaries (Percentage) 1. 101,839 farmers benefited from government support, compared to target of 120,000. Key Outputs by Component 2. 362,244 direct beneficiaries, compared to target of 441,000. (linked to the achievement of the Objective/Outcome 2) 3. 30% female beneficiaries, compared to 51% target. Objective/Outcome 3: Protect people and assets from floods Outcome Indicators 1. Number of households receiving improved flood protection (Number) Intermediate Results Indicators 1. Length of flood protection reconstruction/rehabilitation works (Kilometers) 9 As of March 2020, 65% of installation was completed for Gornji Milanovac substation. It is expected that this substation will be completed by June 2020. Page 38 of 53 The World Bank Floods Emergency Recovery Project (P152018) Key Outputs by Component 1. 71 km of flood protection work, compared to 71.50 km target (linked to the achievement of the Objective/Outcome 2) Page 39 of 53 The World Bank Floods Emergency Recovery Project (P152018) ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION A. TASK TEAM MEMBERS Name Role Preparation Claudia Ines Vasquez Suarez Team Leader(s) Olivera Jordanovic Team Leader David Malcolm Lord Team Member Aleksandar Crnomarkovic Financial Management Specialist Naima A Hasci Social Specialist Bekim Imeri Social Specialist Nikola Ille Environmental Specialist Benedicta T. Oliveros Procurement Specialist Ifeta Smajic Social Specialist Jorge E. Villegas Social Specialist Nebojsa Arsenijevic Operations Officer Ramiro Ignacio Jauregui-Zabalaga Counsel Raymond Bourdeaux Program Leader Donor Coordination Kashmira Daruwalla Team Member Naima A Hasci Social Scientist Toshiaki Keicho Senior Urban Development Specialist Agnes I. Kiss Regional Environmental and Safeguards Advisor Vesna Kostic Senior Communications Officer Dung Kim Le Team Member Luiz T. A. Maurer Principal Industry Specialist, Energy Karuna Elisabeth Phillips Consultant Energy, Project Coordination Gregory Scopelitis E T Consultant Economic and Financial Lazar Sestovic Senior Economist Economic Adam Shayne Lead Counsel Mirjana Simic Bowen Consultant Procurement Efstratios Tavoulareas Senior Operations Officer, Energy Page 40 of 53 The World Bank Floods Emergency Recovery Project (P152018) Jose C. Joaquin Toro Landivar Senior Disaster Risk Management Specialist Heather B. Worley Senior Communications Officer Desanka Stanic Team Member Supervision/ICR Berina Uwimbabazi Team Leader(s) Darko Milutin Team Leader Javier Zuleta Team Member Benedicta T. Oliveros Procurement Specialist Aleksandar Crnomarkovic Financial Management Specialist Satoshi Ishihara Social Specialist Ifeta Smajic Social Specialist Nikola Ille Environmental Specialist Desanka Stanic Procurement Team Olivera Jordanovic Procurement Team Dung Kim Le Procurement Team Koji Nishida Team Member Manjola Malo Procurement Team Bisera Nurkovic Team Member Zhimin Mao Team Member Page 41 of 53 The World Bank Floods Emergency Recovery Project (P152018) B. STAFF TIME AND COST Staff Time and Cost Stage of Project Cycle No. of staff weeks US$ (including travel and consultant costs) Preparation FY15 59.007 340,422.46 FY16 1.930 2,991.26 Total 60.94 343,413.72 Supervision/ICR FY15 7.075 85,642.60 FY16 16.524 95,443.31 FY17 36.773 171,738.77 FY18 33.553 207,019.60 FY19 32.817 178,723.18 FY20 18.350 106,174.84 Total 145.09 844,742.30 Page 42 of 53 The World Bank Floods Emergency Recovery Project (P152018) ANNEX 3. PROJECT COST BY COMPONENT Components Amount at Approval Actual at Project Percentage of (US$, millions) Closing (US$, millions) Approval (%) Component 1: Energy Sector Support 207.25 151.03 72.9 Component 2: Agricultural Sector Support 70.00 59.21 84.6 Component 3: Flood Protection 22.00 34.39 156.3 Component 4: Contingent Emergency 0.00 0.00 0.0 Response Total 299.25 244.63 82.0 Page 43 of 53 The World Bank Floods Emergency Recovery Project (P152018) ANNEX 4. EFFICIENCY ANALYSIS Efficiency Assessment Rating: Substantial 1. At the time of project preparation, the simplified economic and financial analysis of FERP was done. Due to data and time limitations, a simplified economic analysis and a financial analysis for Component 1: Energy Sector Support using financial data for EPS, an economic analysis for Component 2: Agriculture Sector Support, and an economic analysis for Component 3: Flood Protection investments were done. This ICR will try to follow the same structure. 2. Economic models were developed for all three components, unlike in the appraisal stage when it was done only for Component 1: Energy Sector Support was done. The cumulative results of economic analysis for all three components show that the project was very efficient, with an overall rating of Substantial The cumulative NPV is EUR 2,338 million, and the B/C is 12.7. The NPV for Component 1: Energy Sector Support is estimated at EUR 2,012 million compared to EUR 1,936 million projected at appraisal. The overall EIRR could not be calculated for the first component. The B/C is estimated at 14.1. The proposed rating for the energy sector support efficiency is ‘High’. The EIRR for Agriculture Support is 24.4 percent compared to EIRR of 15–30 percent benchmark, and the B/C is 3.5. The proposed rating for Agriculture Support efficiency is ‘Substantial’. The Flood Projection Support EIRR is estimated at 77.3 percent, which is much higher than the approximate 20 percent benchmark at the appraisal stage. Also, the B/C is strong at 8.2. These results suggest a rating of ‘High’.10 The overall results are as shown in table 4.1. Table 4.1. FERP Cumulative Results from Economic Analysis NPV (EUR, millions) EIRR (%) B/C FERP cumulative 2,338.0 n.a. 12.7 Energy Sector Support 2,012.3 n.a. 17.1 Agriculture Sector Support 161.8 24.4 3.5 Flood Protection Support 163.8 77.3 8.2 Source: World Bank staff calculations. 3. Analysis of the administrative and operation efficiency of the project suggest that the largest components (import of electricity and agriculture sector support) disbursed as planned and were highly efficient as the targets were met and exceeded. There were some savings in Component 1: Energy Sector Support, and the funds were reallocated to Flood Protection Support, suggesting increase in efficiency. The reallocation of funds resulted in extension of project life, slightly increasing management supervision costs. 10At the appraisal stage, only one economic model was developed for Component 1: Energy Sector Support, which was not available when asked to conduct an analysis at the completion stage. There were no economic models developed for Agriculture Support nor Floods Protection Support at the appraisal stage, but some benchmarks were used based on other similar projects. Therefore, the results from efficiency analysis might not be fully comparable despite the use of the same assumptions in both cases. However, the results show a high efficiency of investments in all three components, suggesting a High ranking. Page 44 of 53 The World Bank Floods Emergency Recovery Project (P152018) Economic and Financial Analysis of Energy Sector Support Rating: High 4. The ICR will first discuss briefly on methodology and findings from economic analysis to be then followed by financial analysis. Methodology for Economic Analysis 5. The economic appraisal was carried out for 2014–2025 with a cost-benefit analysis comparing scenarios with and without the project. The medium-term horizon of the project implementation (five years after the extension of original period by 22 months), justifies the relatively short period considered in analysis. Project Costs 6. The cost of the Energy Sector Component was EUR 139.743 million (original design was 157.1, but revised after savings) and includes the following elements: • Support for electricity purchases (EUR 103.637 million). The project helped purchase electricity equivalent to 2,263,672 MWh at an average price of EUR 45.8 per MWh for the 2014–15 winter season. • Loss reduction and reliability improvement in the distribution network and load management (EUR 15.1 million). • Dewatering of the Tamnava West Field mine (EUR 21 million). Project Benefits 7. The project benefits of Component 1: Energy Sector Support stem from avoided economic costs associated with the reduction in expected load-shedding in Serbia. On average, Serbia has been an electricity net exporter in the recent past, depending on hydrological conditions. Serbia has no structural unserved demand, and the project helped maintain an unbroken record during the time of crisis. To estimate the avoided cost of load-shedding, the electricity intensity of GDP is used as a proxy and was estimated at US$1.34 per kWh in 2014 as in the appraisal stage. 8. A simple analysis was conducted to measure NPV and B/C, whereas an EIRR could not be calculated. Results of Economic Analysis for Energy Sector Support 9. On the basis of the ICR, the project was assessed as economically efficient. At appraisal, the project’s economic analysis of Component 1: Energy Sector Support estimated an NPV of EUR 1,935 million and an EIRR of 99.7 percent. The original model is not available, but a new model was developed using the same assumptions and using actual data, when known. This might mean that the results are not comparable in magnitude, but they are comparable when it is evaluated if the project was efficient or not. 10. The ICR argued that there was no reason to expect that the economic benefit of the project would be significantly different from what was calculated at appraisal. This is because of the conclusion that Page 45 of 53 The World Bank Floods Emergency Recovery Project (P152018) successful implementation of Component 1: Energy Sector Support for supply of demand with energy from import resulted in no electricity shortages during the high season of 2014–15, and implementation at the time of the dewatering of mine was crucial to support an efficient project. There were some savings made that could have increased project efficiency. There was however a restructuring of the project, which resulted in some delays in implementation and extension of the implementation period, that might have canceled out the additional benefits from savings. That suggests a very similar economic impact of this project component, as the ICR reported an economic NPV of EUR 2,012 million and a B/C of 17.1. The NPV discount rate is 10 percent. The EIRR could not be computed. 11. It also might have had some positive impact on the financial performance which led to improvement of the financial NPV of US$167.9 million, which is much better than what was anticipated at appraisal. The financial results are discussed in the next section. Financial Analysis 12. The financial analysis is based on data and information presented by EPS’s audited financial reports. It reflects the actual financial data between 2013 and 2018. For the purpose of building a cash flow up to 2025, it is assumed that operation revenues and expenditures remain constant to the last year available (2018) whereas it can be assumed that cash flows from investments declined by 1 percent every consecutive year. Cash inflows from financing activities are constant to 2017 (in 2018, there were no cash inflows), and cash outflow from financing activities is half of what it was in 2018 but at the same time it is 3.1 times higher than the average from 2013 to 2017. That is because 2018 was an outlier, therefore using half of it for the rest of the period is a very conservative assumption. 13. Ex post financial data show that EPS performance did not meet the expectations in some parameters but performed better on other parameters compared to the expected results in the appraisal financial analysis. The earnings before interest, taxes, depreciation, and amortization (EBITDA) margin (calculated as EBITDA over total revenues) was fairly stable at 22 percent through years with a slight decline to 19 percent in 2018. It also had a solid and improving EBITDA interest coverage. These financial ratios are presented in table 4.2. Table 4.2. Key Financial Ratios for EPS Performance, 2013–2018 2013 2014 2015 2016 2017 2018 EBITDA margin (%) 27.50 14.20 21.60 21.90 22.50 19.20 EBITDA/operation revenue (%) 30.70 15.70 23.60 26.10 25.00 20.80 Interest/EBITDA 0.09 0.13 0.06 0.04 0.03 0.03 Source: EPS consolidated financial statements, World Bank staff calculations. 14. As stated at the appraisal stage, based on the EPS management assessment, the EPS’s operations and financial situation were not sustainable over the medium to long term. EBITDA did not turn negative in 2016 as foreseen in the appraisal stage, but EBITDA was negative in 2014 due to flood effects. The debt- to-EBITDA ratio was large, between 1.2 times in 2013 and about 3 times in 2014 and 2018, as anticipated in the appraisal stage. Liquidity ratios, such as current ratio and quick ratio, were calculated for 2013 to 2018. The current ratio moves from 1.3 in 2014 to 1.7 in 2016 to decline back to 1.2 in 2018. The quick ratio follows the similar path but is below 1 in 2014 and again in 2018. This suggests that the indebtedness level is quite sensitive and can increase over time. The results are presented in table 4.3. Page 46 of 53 The World Bank Floods Emergency Recovery Project (P152018) Table 4.3. Liquidity Ratios of EPS 2013–2018 2013 2014 2015 2016 2017 2018 Debt/EBITDA 1.19 3.02 2.43 2.36 2.43 2.96 Current ratio 1.22 1.26 1.50 1.68 1.34 1.19 Quick ratio 1.02 0.95 1.15 1.33 1.00 0.86 Source: EPS consolidated financial statements, World Bank staff calculations. 15. Unlike what was expected at appraisal, these investments should bring an additional 5 percent to the EBITDA margin by 2018. The actual financial data suggest that the EBITDA margin, after its decline in 2014 to 14.2 percent, recovered only partially to 22.5 percent in 2017 but did not reach the 2013 level, and then declined again to 19.2 percent in 2018. Despite increase in tariffs by 12 percent in 2015, followed by another increase of 3.8 percent in 2016, the growth in revenues from sales was only 8 percent in 2015, followed by 1.1 percent growth in 2016, both years being less than tariff increases. This suggests efficiency loses by EPS. Sales revenues plummeted by 2.6 percent in 2017, to recover with a growth of 4.3 percent in 2018. Economic Analysis of Agriculture Sector Support Rating: Substantial 16. The project addressed the immediate needs to prevent a further decrease of farm household income following the natural disaster which resulted in degradation of the environment, land, and physical resources; a precipitous drop in production; and broken links in the agricultural supply chain. The NPV discount rate is 10 percent. 17. An economic analysis was not undertaken for this specific program at the appraisal stage to be able to compare ex post. However, at completion a model to evaluate direct economic benefits from the incentive programs was developed, which includes (a) ensured food self-sufficiency; (b) additional cash income; (c) local jobs maintained; and (d) benefits to the upstream and downstream sectors (that is, input supplying industries, food processing and trade sector, where only 0.11 percent is used as contribution to GDP, which is very conservative). 18. Assumptions for analysis are the same as in the appraisal stage, but actual data are used on results achieved such as number of farms that benefitted, average wage for agriculture sector by years available (2014–2019), and so on. The same assumption of EUR 510 per household income was used and, keeping in mind that the farms in this area generate 8–10 percent of their income from the sale of agricultural products, it is estimated that the project will contribute to generating additional cash income derived from sales of agricultural products of about US$500 per household. About 103,000 farms have benefitted from the program. 19. As the affected areas are dominated by labor-intensive sectors of agriculture, such as horticulture and viticulture, the project had a significant impact on the preservation of jobs of permanent and seasonal workers. A high percentage of rural labor is employed in agriculture, suggesting that maintaining agricultural production at preflood levels (the actual data show that it was exceeded in 2016) in the flood- affected municipalities will keep over 20,000 individuals fully employed in agriculture (nine months per year is used as a more conservative approach). It is estimated that about 100 farm holdings and over 10,000 family farms will be able to hire seasonal workers (four months of employment was used as seasonality, multiplied by three seasonal workers per farm—again a very conservative approach). Page 47 of 53 The World Bank Floods Emergency Recovery Project (P152018) 20. Based on the assumption used and the actual data mentioned earlier, the economic impact of this project component was positive and strong. The ICR reported an economic NPV of EUR 161.8 million, an EIRR of 24.4 percent, and a B/C of 3.5. Economic Analysis of Flood Protection Investment Rating: High 21. An economic analysis of the proposed Flood Protection Investments was not carried out at appraisal. Therefore, any results from the economic analysis of Flood Protection Investment, done ex post, are not comparable to the results used at appraisal. The main hypothesis is that flood protection and drainage investments reduce expected losses and generate substantial economic benefits justifying the investment costs. This includes a direct positive impact on the livelihoods of project beneficiaries, as losses—including public infrastructure, agricultural and livestock production, as well as family household assets—are significantly reduced. 22. Some of the expected benefits from the flood control and drainage structures, which will be used as main basis for calculation of economic impact, include • Avoided damages of agriculture production on land likely to suffer frequent floods; • Avoided damages of assets (private houses, apartment houses, industrial facilities, roads, electricity, schools, kindergartens, health facilities, and other public buildings); • Avoided losses of business due to uninterrupted production, provision of services, interrupted communications, traffic disruptions, and so on; • Avoided human health costs due to reduction of pollution in the water supply systems, reduced waterborne diseases, and reduced risks of human life losses; • Increased recreation benefits including tourism development; and • Indirect economic development effects. 23. Analysis results show that Floods Protection Investment was highly efficient and economically very viable. Even with a very conservative scenario of only 15 percent probability of recurrence and using minimum impact (much lower than the damages assessed for the 2014 floods) on infrastructure, agriculture, businesses, and other sectors, the NPV is 163.8 million, using a discount rate of 10 percent. The EIRR is 77.3 percent and B/C is 8.2. These results justify many times this investment. Table 4.4. Cumulative Results of Economic Analysis FERP Year Cost Energy Support Agriculture Flood Protection Cash Flow Support 2014 82,863,430 928,410,000 183,661,892 145,416,400 1,174,624,863 2015 99,919,609 1,223,700,000 (20,923,229) (3,414,750) 1,099,442,412 2016 13,915,109 78,790,000 25,457,859 7,620,173 97,952,922 2017 8,733,259 53,257,748 8,042,400 10,347,698 62,914,587 2018 7,977,944 59,097,424 14,837,839 17,119,483 83,076,802 Page 48 of 53 The World Bank Floods Emergency Recovery Project (P152018) Year Cost Energy Support Agriculture Flood Protection Cash Flow Support 2019 7,977,944 57,495,941 13,534,893 9,598,216 72,651,106 2020 — 49,889,659 6,308,175 9,827,039 66,024,874 2021 — 47,430,100 6,522,997 10,064,793 64,017,890 2022 — 45,103,963 6,746,937 10,311,899 62,162,799 2023 — 42,904,398 6,979,857 10,568,798 60,453,053 2024 — 40,824,917 7,222,595 10,835,956 58,883,467 2025 — 38,859,379 7,475,158 11,113,858 57,448,395 83,785,158 NPV 2,337,989,187 EIRR Not Computed B/C 12.7 Source: World Bank staff calculations. Table 4.5. Results of Economic Analysis for Energy Sector Support Year Cost Energy import Dewatering Improvement Cash Flow Benefits Tamnava Network Benefits Benefits 2014 53,465,430 928,410,000 — — 874,944,570 2015 70,521,609 1,207,700,000 8,000,000 8,000,000 1,153,178,391 2016 8,203,109 49,790,000 22,000,000 7,000,000 70,586,891 2017 3,021,259 48,607,748 900,000 3,750,000 50,236,488 2018 2,265,944 53,482,924 927,000 4,687,500 56,831,479 2019 2,265,944 50,681,756 954,810 5,859,375 55,229,996 2020 — 48,027,299 983,454 878,906 49,889,659 2021 — 45,511,869 1,012,958 905,273 47,430,100 2022 — 43,128,185 1,043,347 932,432 45,103,963 2023 — 40,869,346 1,074,647 960,405 42,904,398 2024 — 38,728,814 1,106,886 989,217 40,824,917 2025 — 36,700,393 1,140,093 1,018,893 38,859,379 117,799,975 NPV 2,012,341,533 EIRR Not Computed B/C 17.1 Source: World Bank staff calculations. Table 4.6. Results from Economic Analysis for Agriculture Sector Support (EUR) Year Cost Food Self- Food Preserving Other Sector Cash Flow sufficiency Surplus for Jobs Benefits Benefit Market 2014 26,542,000.0 52,530,000 3,769,981 83,151,911 44,210,000 157,119,892 2015 26,542,000.0 1,050,600 135,161 (22,490,990) 382,000 (47,465,229) 2016 — 1,071,612 139,657 22,878,590 1,368,000 25,457,859 2017 — 1,093,044 141,238 7,195,119 (387,000) 8,042,400 2018 — 1,114,905 138,828 3,596,106 9,988,000 14,837,839 2019 — 1,137,203 150,760 9,468,880 2,778,050 13,534,893 2020 — 1,159,947 155,283 2,075,992 2,916,953 6,308,175 2021 — 1,183,146 159,538 2,117,512 3,062,800 6,522,997 2022 — 1,206,809 164,325 2,159,862 3,215,940 6,746,937 2023 — 1,230,945 169,115 2,203,060 3,376,737 6,979,857 Page 49 of 53 The World Bank Floods Emergency Recovery Project (P152018) Year Cost Food Self- Food Preserving Other Sector Cash Flow sufficiency Surplus for Jobs Benefits Benefit Market 2024 — 1,255,564 174,335 2,247,121 3,545,574 7,222,595 2025 — 1,280,676 179,567 2,292,063 3,722,853 7,475,158 46,064,628 NPV 161,827,768 EIRR 24.4% B/C 3.5 Source: World Bank staff calculations. Table 4.7. Results of Economic Analysis for Flood Protection Support (EUR) Year Cost Agriculture Assets Businesses Recreation Other Cash Flow Tourism Indirect Benefits 2014 2,856,000.0 9,947,250 86,175,000 20,557,650 6,631,500 22,105,000 142,560,400 2015 2,856,000.0 (1,586,250) 6,032,250 (3,278,250) (1,057,500) (3,525,000) (6,270,750) 2016 5,712,000.0 256,500 6,092,573 530,100 171,000 570,000 1,908,173 2017 5,712,000.0 704,250 6,153,498 1,455,450 469,500 1,565,000 4,635,698 2018 5,712,000.0 2,043,000 6,215,033 4,222,200 99,250 4,540,000 11,407,483 2019 5,712,000.0 568,238 6,277,184 1,174,358 315,688 1,262,750 3,886,216 2020 — 596,649 6,339,955 1,233,075 331,472 1,325,888 9,827,039 2021 — 626,482 6,403,355 1,294,729 348,045 1,392,182 10,064,793 2022 — 657,806 6,467,388 1,359,466 365,448 1,461,791 10,311,899 2023 — 690,696 6,532,062 1,427,439 383,720 1,534,881 10,568,798 2024 — 725,231 6,597,383 1,498,811 402,906 1,611,625 10,835,956 2025 — 761,493 6,663,357 1,573,751 423,051 1,692,206 11,113,858 19,920,555 NPV 163,819,886 EIRR 77.3% B/C 8.2 Source: World Bank staff calculations. Page 50 of 53 The World Bank Floods Emergency Recovery Project (P152018) ANNEX 5. BORROWER, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS On April 22nd, 2020, the Ministry of Agriculture, Forestry and Water Management responded that they have no objection and no comments to the ICR. Comments were received from EPS on April 23rd, 2020. These comments were reflected and incorporated in this document. Page 51 of 53 The World Bank Floods Emergency Recovery Project (P152018) ANNEX 6. SUPPORTING DOCUMENTS (IF ANY) • Project Appraisal Document (PAD), Report No. PAD1129, dated September 22, 2014 • Loan Agreement for Loan Number 8449-YF • Project Agreement for Loan Number 8449-YF • Implementation Supervision Reports, numbers 1–8 • Serbia’s Project Completion Report (both PIU) • Aid Memoires of Implementation Support Missions • Restructuring Paper of March 20, 2017, Report Number RES22941 • Recovery Needs Assessment of the Republic of Serbia, 2014 • Country Partnership Framework FY16–20 for Serbia, Report Number 94687 Page 52 of 53 The World Bank Floods Emergency Recovery Project (P152018) ANNEX 7. MAP Map of flood-affected municipalities Republic of Serbia, Floods Emergency Recovery Project (P152018) Page 53 of 53