Document of The World Bank FOR OFFICIAL USE ONLY Report No: ICR00004437 IMPLEMENTATION COMPLETION AND RESULTS REPORT (IBRD-74600, TF-90657) ON A LOAN IN THE AMOUNT OF US$230 MILLION TO THE REPUBLIC OF AZERBAIJAN FOR A NATIONAL WATER SUPPLY AND SANITATION PROJECT June 28, 2018 Water Global Practice Europe and Central Asia Region CURRENCY EQUIVALENTS (Exchange Rate Effective May 04, 2018) Currency Unit = New Azerbaijanian Manat (AZN) AZN 1.7025 = US$1 US$0.5874 = SDR 1 FISCAL YEAR January 1 – December 31 Regional Vice President: Cyril E Muller Country Director: Mercy Miyang Tembon Senior Global Practice Director: Guang Zhe Chen Practice Manager: David Michaud Task Team Leader(s): Hadji Huseynov ICR Main Contributor: Toma Alexandrov Yanakiev ABBREVIATIONS AND ACRONYMS ADB Asian Development Bank AF Additional Financing BSPI Baku State Project Institute CBA Cost-benefit Analysis CPF Country Partnership Framework CPS Country Partnership Strategy EBIT Earnings Before Interest and Taxes EBITDA Earnings Before Interest, Taxes, Depreciation, And Amortization EIA Environmental Impact Assessment EU European Union FSU Former Soviet Union GEF Global Environment Facility GIS Geographic Information System GoA Government of Azerbaijan GBRDP Greater Baku Regional Development Plan GBWSP Greater Baku Water Supply Project IBRD International Bank for Reconstruction and Development ICR Implementation Completion and Results Report IEG Independent Evaluation Group (World Bank) IFI International Financial Institution IFRS International Financial Reporting Standards INT The World Bank Integrity Department IPF Investment Project Financing ISR Implementation Status and Results Report KfW German Government Owned Development Bank LA Legal Agreement M&E Monitoring and Evaluation NPV Net Present Value NWSSP National Water Supply and Sanitation Project O&M Operations and Maintenance PAD Project Appraisal Document PDO Project Development Objective PE Persons Equivalent PMU Project Management Unit SAWMA State Amelioration and Water Management Agency SCD Systematic Country Diagnostic SCUPA State Committee for Urban Planning and Architecture SNWSSP Second National Water Supply and Sanitation Project SPPRED State Program on Poverty Reduction and Economic Development TF Trust Fund TTL Task Team Leader WSS Water Supply and Sanitation WTP Willingness to Pay WWTP Wastewater Treatment Plant TABLE OF CONTENTS DATA SHEET ....................................................................... ERROR! BOOKMARK NOT DEFINED. I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES ....................................................... 6 A. CONTEXT AT APPRAISAL .........................................................................................................6 B. SIGNIFICANT CHANGES DURING IMPLEMENTATION (IF APPLICABLE) ..................................... 10 II. OUTCOME .................................................................................................................... 15 A. RELEVANCE OF PDOs ............................................................................................................ 15 B. ACHIEVEMENT OF PDOs (EFFICACY) ...................................................................................... 16 C. EFFICIENCY ........................................................................................................................... 18 D. JUSTIFICATION OF OVERALL OUTCOME RATING .................................................................... 21 E. OTHER OUTCOMES AND IMPACTS (IF ANY) ............................................................................ 22 III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME ................................ 26 A. KEY FACTORS DURING PREPARATION ................................................................................... 26 B. KEY FACTORS DURING IMPLEMENTATION ............................................................................. 26 IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME .. 28 A. QUALITY OF MONITORING AND EVALUATION (M&E) ............................................................ 28 B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE ..................................................... 29 C. BANK PERFORMANCE ........................................................................................................... 30 D. RISK TO DEVELOPMENT OUTCOME ....................................................................................... 32 V. LESSONS AND RECOMMENDATIONS ............................................................................. 33 ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS ........................................................... 34 ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION ......................... 42 ANNEX 3. PROJECT COST BY COMPONENT ........................................................................... 44 ANNEX 4. EFFICIENCY ANALYSIS ........................................................................................... 45 ANNEX 5. BORROWER COMMENTS ...................................................................................... 58 ANNEX 6. SUPPORTING DOCUMENTS .................................................................................. 59 The World Bank NATIONAL WATER SUPPLY & SANITATION (P096213) DATA SHEET BASIC INFORMATION Product Information Project ID Project Name P096213 NATIONAL WATER SUPPLY & SANITATION Country Financing Instrument Azerbaijan Investment Project Financing Original EA Category Revised EA Category Full Assessment (A) Full Assessment (A) Organizations Borrower Implementing Agency Azersu Open Joint Stock Company of Republic of Ministry of Finance Azerbaijan Project Development Objective (PDO) Original PDO To improve the availability, quality, reliability, and sustainability of water supply and sanitation services in twenty of Azerbaijan?s regional (rayon) centers/ To provide quality and reliable water supply and sanitation services in selected regional (rayon) centers of Azerbaijan. PDO as stated in the legal agreement To improve the availability, quality, reliability, and sustainability of water supply and sanitation services of the Borrower. Page 1 of 59 The World Bank NATIONAL WATER SUPPLY & SANITATION (P096213) FINANCING Original Amount (US$) Revised Amount (US$) Actual Disbursed (US$) World Bank Financing 230,000,000 207,445,473 207,445,473 IBRD-74600 1,951,706 1,948,917 1,948,917 TF-90657 Total 231,951,706 209,394,390 209,394,390 Non-World Bank Financing Borrower 80,000,000 0 0 Total 80,000,000 0 0 Total Project Cost 311,951,706 209,394,390 209,394,390 KEY DATES Approval Effectiveness MTR Review Original Closing Actual Closing 14-Jun-2007 14-Mar-2008 29-Oct-2012 31-Jan-2012 31-Dec-2016 RESTRUCTURING AND/OR ADDITIONAL FINANCING Date(s) Amount Disbursed (US$M) Key Revisions 16-Mar-2009 2.19 Change in Implementing Agency Reallocation between Disbursement Categories 22-Nov-2011 24.85 Change in Implementing Agency Change in Project Development Objectives Change in Results Framework Change in Components and Cost Change in Loan Closing Date(s) Cancellation of Financing Reallocation between Disbursement Categories Change in Legal Covenants Change in Institutional Arrangements 28-Feb-2013 56.75 Change in Loan Closing Date(s) 12-Dec-2014 141.02 Change in Results Framework Change in Loan Closing Date(s) Reallocation between Disbursement Categories 21-Jul-2016 189.72 Page 2 of 59 The World Bank NATIONAL WATER SUPPLY & SANITATION (P096213) KEY RATINGS Outcome Bank Performance M&E Quality Moderately Unsatisfactory Moderately Unsatisfactory Modest RATINGS OF PROJECT PERFORMANCE IN ISRs Actual No. Date ISR Archived DO Rating IP Rating Disbursements (US$M) 01 14-Aug-2007 Satisfactory Satisfactory 0 02 09-Nov-2007 Satisfactory Satisfactory 0 03 26-Feb-2008 Satisfactory Moderately Satisfactory 0 04 26-Jun-2008 Satisfactory Satisfactory 1.00 05 07-Dec-2008 Satisfactory Moderately Satisfactory 1.63 06 12-Jul-2009 Unsatisfactory Unsatisfactory 2.55 07 13-Feb-2010 Unsatisfactory Unsatisfactory 5.37 08 10-May-2010 Unsatisfactory Unsatisfactory 6.40 09 01-Jan-2011 Unsatisfactory Unsatisfactory 10.75 Moderately 10 05-Jul-2011 Moderately Unsatisfactory 17.93 Unsatisfactory Moderately 11 05-Feb-2012 Moderately Unsatisfactory 26.98 Unsatisfactory Moderately 12 06-May-2012 Moderately Unsatisfactory 29.79 Unsatisfactory Moderately 13 26-Dec-2012 Moderately Unsatisfactory 51.44 Unsatisfactory 14 29-May-2013 Moderately Satisfactory Moderately Satisfactory 57.80 15 27-Dec-2013 Moderately Satisfactory Moderately Satisfactory 73.38 16 28-Jun-2014 Moderately Satisfactory Moderately Satisfactory 98.80 17 09-Dec-2014 Satisfactory Moderately Satisfactory 141.02 18 20-May-2015 Moderately Satisfactory Moderately Satisfactory 160.27 Page 3 of 59 The World Bank NATIONAL WATER SUPPLY & SANITATION (P096213) 19 11-Nov-2015 Moderately Satisfactory Moderately Satisfactory 168.54 20 10-May-2016 Moderately Satisfactory Moderately Satisfactory 181.02 21 09-Nov-2016 Moderately Satisfactory Satisfactory 198.34 SECTORS AND THEMES Sectors Major Sector/Sector (%) Public Administration 2 Central Government (Central Agencies) 2 Water, Sanitation and Waste Management 98 Sanitation 39 Water Supply 59 Themes Major Theme/ Theme (Level 2)/ Theme (Level 3) (%) Private Sector Development 20 Business Enabling Environment 20 Investment and Business Climate 20 Urban and Rural Development 60 Urban Development 60 Urban Infrastructure and Service Delivery 20 Services and Housing for the Poor 40 Environment and Natural Resource Management 21 Environmental Health and Pollution Management 21 Air quality management 7 Water Pollution 7 Soil Pollution 7 Page 4 of 59 The World Bank NATIONAL WATER SUPPLY & SANITATION (P096213) ADM STAFF Role At Approval At ICR Regional Vice President: Shigeo Katsu Cyril E Muller Country Director: D-M Dowsett-Coirolo Mercy Miyang Tembon Senior Global Practice Director: Jose Luis Irigoyen Guang Zhe Chen Practice Manager: Sumter Lee Travers David Michaud Task Team Leader(s): Andreas Rohde Hadji Huseynov ICR Contributing Author: Toma Alexandrov Yanakiev Page 5 of 59 The World Bank NATIONAL WATER SUPPLY & SANITATION (P096213) I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES A. CONTEXT AT APPRAISAL Context 1. Azerbaijan was one of the fastest growing economies in the Commonwealth of Independent States at the time of Project appraisal (May 2007). Its gross national product per capita rose to US$1,190 in 2005 from a post-independence low in 1995 of US$390 per capita. This rapid growth was mainly attributed to positive developments in the oil industry, which accounted for almost one-third of Azerbaijan’s gross domestic product. Growth in non-oil output had also risen since 1999, with an average annual growth rate of more than 10 percent. The Government was seeking to build upon the opportunities provided by its oil revenue, as well as to put into place measures that would promote growth in the non-oil economy to ensure that the full benefits of economic growth reach all sectors and segments of society to attain a sustainable reduction in poverty. 2. Azerbaijan had one of the world’s leading economic growth rates due to oil revenue and subsequently infrastructure has been modernized, non-oil resource economy developed, and social welfare and state-owned assets improved. Substantial reforms intended to support a market-based economy have been instrumental in facilitating growth. However, the reliance on petroleum resources placed the country at risk of volatility and raised concerns about long-term sustainability and macroeconomic stability. Given the long time which it takes to establish a diversified, modern economy, Azerbaijan needed to make greater progress in the non-oil sectors. 3. Azerbaijan inherited a relatively extensive water supply system from the Former Soviet Union (FSU). At the time of the appraisal, about 95 percent of the population in Baku and about 83 percent of those living in secondary cities and small towns were connected to piped water supply. While coverage appeared generally high in comparison to most of the other countries with similar income, the quality of infrastructure and services had deteriorated severely over the years due to the lack of investment and deferred maintenance. In many secondary and small towns, water treatment facilities were largely dysfunctional or lacking completely, so that the population in these towns did not have access to safe water supply. In addition, almost everywhere in the country piped water supply was unreliable and often only available for less than 12 hours a day. Centralized piped water supply systems were rare in rural areas and less than 33 percent of the rural population had access to piped water supplies. 4. About 55 percent of Azerbaijan’s population had access to improved sanitation facilities. Access to sanitation facilities in urban areas was higher than in rural areas, with coverage estimated at 73 percent and 36 percent, respectively.1 The sewerage network in Baku served about 78 percent of the population,2 although only about 50 percent of wastewater in the area was treated.3 Sewerage coverage in other urban areas was only about 32 percent and a minor proportion of the sewage underwent any treatment before disposal. Rural areas primarily depended on on-site sanitation. There was insufficient control by the public health department on the location and condition of on-site sanitation facilities, and there was no effective regulation of the emptying trucks which periodically removed the sludge, increasing the risk of clandestine 1 World Development Indicators data, 2004. 2 The United Nations Environmental Performance Review, 2004 gives sewerage coverage in Baku as 72 percent. 3 Data are taken from the United Nations Environmental Performance Review, 2004. Page 6 of 59 The World Bank NATIONAL WATER SUPPLY & SANITATION (P096213) and unsafe discharge of this matter into the environment. It was imperative that fecal sludge from these facilities be treated before it is discharged into the environment both for public health and environmental reasons. 5. Azerbaijan water sector and its regulation faced different transformations. In the 1990s, the utilities were smaller and there was a ministry of utility services. Structural changes, including decentralization and tariffs, were opportunities to be discussed at that time. Basic infrastructure services started to be decentralized to local authorities, based on the Water Supply and Wastewater Law and a series of laws on municipalities. There were separate tariffs for different cities and the regions were in-charge of making decisions. A State Owned Enterprise (SOE) – Absersonsu was established in 2000 as the Regional Water Company for Greater Baku and the Absheron peninsula areas with a population of 3.5 million persons. There was expansion of residential construction (mainly for refugees) and very high expectations of what could be done. Sustainability of the system was affected and the Government of Azerbaijan (GoA) subsidy for user fees was needed. In 2004, the Azersu Joint Market company was established to centrally manage water supply and wastewater services for all urban areas nationwide. Thus, Azersu stopped the decentralization and took control over decision making and budgets. 6. At the time of the appraisal of the National Water Supply and Sanitation Project (NWSSP), the World Bank involvement in supporting the water supply and sanitation (WSS) sector had been modest in comparison to the country’s huge investment needs in the sector, particularly outside Baku. The World Bank had earlier financed the Greater Baku Water Supply Project (GBWSP) (P008288), which closed in January 2006, and supported substantial improvements to the area’s water supply. Given the significant needs in the Greater Baku area and the rest of the country, the GBWSP was designed as a first step in addressing the country’s WSS needs. The Government’s Poverty Reduction Strategy (2003–05), the State Program on Poverty Reduction and Economic Development (SPPRED), included strengthening of the utility sectors as one of its strategic goals. The Government decided to focus on, among other things, creating the infrastructure needed for regional development and improvement of public utilities. It adopted a nationwide approach aimed at covering as much of the sector’s needs as possible through its internal resources and support from various donors and International Financial Institutions (IFIs). Thus, in parallel with the NWSSP preparation, systems in several rayons were being funded by the Asian Development Bank, German Government Owned Development Bank (KFW), and State Secretary for Economic Affairs, Switzerland, while the World Bank, Japan International Cooperation Agency, and Japan Bank for International Cooperation were preparing additional projects. Together, the combined efforts of the Government and these donors were covering the needs of about 80 percent of the urban population in rayons outside Baku and it was expected that the rest would be addressed in the near future. By contributing to the nationwide approach, the Project provided an opportunity to broaden the World Bank’s support to many of Azerbaijan’s rayons, thus contributing to the overall goals of economic development and poverty reduction. 7. At the time of the appraisal, the project contributed to higher-level objectives outlined in the following key strategic documents. 8. Country Partnership Strategy 4 (CPS) FY07–FY10. The CPS referred to the Project as part of the FY07 deliverables. Improving infrastructure service delivery, including utilities such as water supply and sanitation was consistent with the CPS which sought to support four main pillars: (i) improving the quality and transparency in public sector governance; (ii) supporting sustainable and balanced growth of the non-oil economy; (iii) increasing the quality of and access to social services; and (iv) improving environmental management. The Project helped achieve the following four-year CPS development outcomes: (i) develop rural infrastructure, and (ii) improve coverage of water supply systems. In addition, the Project assisted in achieving one of the CPS governance outcomes, the establishment of accountable public financial 4World Bank, November 8, 2006. CPF FY07–10 for Republic pf Azerbaijan, Report No. 37812-AZ, South Caucasus Country Unit, Europe and Central Asia Region. Page 7 of 59 The World Bank NATIONAL WATER SUPPLY & SANITATION (P096213) management and procurement systems, through introduction of International Financial Reporting Standards (IFRS) and strengthening of procurement capacities in the water sector. 9. The Water Sector Strategy for Azerbaijan: at the time of appraisal, for the period 2006-2015 was prepared by the Government in collaboration with the World Bank, provided a vision for the sector and guidance on the management and allocation of water resources in Azerbaijan for the short- to medium-term goals (2006-2015). It addressed both Water Supply and Sanitation (WSS) issues and Water Resources Management (WRM) issues, including Irrigation and Drainage, drawing on international experience and the specific conditions of Azerbaijan. The strategy reviewed key issues that were faced by the water economy in the country, and articulated an Action Plan to improve the operational, financial, and commercial performance of the sector and its institutional effectiveness through a combination of: (i) investments for rehabilitation and reconstruction of infrastructure; and (ii) institutional modernization, through institutional, financial and governance reforms to ensure accountability, improvement of operational efficiency, and sustainability of the sector. The Project would substantially contribute to the implementation of the Water Sector Strategy. 10. Transformation from Planned Economy to Market Economy: In parallel to the project activities, transformation to a market economy was taking place. The institutional changes were to replace the administrative system that was inherited from the Soviet Union period, with a new system that enhanced free competitive relations, measures so that a market economy system could be formed, changes regarding privatization of state-owned companies and the formation of new ones. These substantial reforms supported the development of a market-based economy which is instrumental in facilitating growth. All these transformations also influenced the WSS sector, its operation, effectiveness and capability to adjust to the supply and demand of clients and subcontractors. Theory of Change (Results Chain) 11. A combination of an inherited and relatively extensive water supply system from the Former Soviet Union (FSU), lack of investment, deferred maintenance and damage due to natural calamities such as an earthquake in 2000 with an epicenter just offshore Baku that was the strongest in almost 160 years, resulted in severely deteriorated quality of infrastructure and services over the years. The compounded impact of these developments negatively affected water supply and sanitation users. In many secondary and small towns, water treatment facilities were largely dysfunctional or lacking completely, to the point that the population in these towns did not have access to safe water supply. Furthermore, most rayons did not have a functioning sewerage collection system, and the few facilities that were available were in a poor state of disrepair. This was a serious threat to public health and had severe negative impacts on the environment. In addition, almost everywhere in the country piped water supply was unreliable, and was often available for less than 12 hours a day. 12. To mitigate these impacts and improve the availability, quality, reliability, and sustainability of water supply and sanitation services, a two-track approach was followed: (a) Investments in rehabilitation and reconstruction of water supply and sanitation infrastructure. There was investment in the whole water cycle—starting from water intake and storage, water supply and sewage network, and wastewater treatment. This was a unique approach as it not only brought complete processes with all related benefits for the population, but also provided higher standards for environmental protection and for future Operations and Maintenance (O&M). This is visible in the first two actions of figure 1, which also shows the respective outputs and Project Development Objectives (PDOs)/long-term outputs. Page 8 of 59 The World Bank NATIONAL WATER SUPPLY & SANITATION (P096213) (b) Implementation of a comprehensive Institutional Modernization Component to strengthen the WSS sector’s capacity to manage WSS services in an efficient, effective, and sustainable manner. It was focused on National and Greater Baku levels with regards to planning, financial management, and effectiveness. However, it also focused at the local utility level for collection improvement, transparency, accountability, and automation. This is illustrated in the last two actions with respective institutional development which has the long-term sustainability outcome. Figure 1. Graphical Representation of Theory of Change Project Development Objectives (PDOs) 13. Project Development Objectives: The original PDO defined in the Legal Agreement (LA) is to improve the availability, quality, reliability, and sustainability of water supply and sanitation services of the Borrower. 14. There is a difference between the LA and PAD PDO: The PDO in the Project Appraisal Document (PAD) specifies that the project improvement is “in twenty of Azerbaijan’s regional (rayon) centers� instead only “of the Borrower.� Page 9 of 59 The World Bank NATIONAL WATER SUPPLY & SANITATION (P096213) Key Expected Outcomes and Outcome Indicators 15. Progress towards achieving the abovementioned development objectives were to be monitored through the following key performance indicators in the project area: (a) Availability: (i) Number of people served by piped water supply; (ii) number of people connected to a sewerage network with an operating wastewater treatment plant. (b) Quality: Percentage of drinking water samples in project area meeting Azeri water quality standards. (c) Reliability: Daily number of hours of water supply service. (d) Sustainability: Working ratio (operating expenditures divided by collected revenues) of the water utilities. Components Table 1. Components Amount at Actual at Project Approval Closing (US$M) (US$M) Component A: Rayon Investment 268.75 207.00 Rehabilitation and extension of water supply and sewerage systems as well as facilities for water, wastewater, and sludge treatment in the project area. Component B: Regional Development Plan 5.00 5.00 Preparation of a regional development plan for the Greater Baku area. Update of the Water and Wastewater Master Plan. Component C: Institutional Modernization 12.67 12.67 Development and implementation of key reforms needed for the modernization of the water supply and sanitation sector to improve the efficiency and sustainability of its water supply and sanitation services. Component D: Project Management 1.95 1.95 Strengthening the management capacity of the utilities to monitor and administer implementation of the Project, including audit. Contingencies 21.63 Total 310.00 226.62 B. SIGNIFICANT CHANGES DURING IMPLEMENTATION (IF APPLICABLE) 16. The project had five restructurings. Table 2 below summarizes the key changes made to the project. Page 10 of 59 The World Bank NATIONAL WATER SUPPLY & SANITATION (P096213) Table 2. Restructuring Changes Restructuring Change PDO Closing No. Date Scope PDO PDO Targets Other Indicators date Improve the availability, 22 Initially quality, Rayons reliability, and sustainability Change in 22 1 March 2009 implementing Rayons agency ‘sustainability’ Changed to Revised with 4 and reflect the 2 October 2011 the new PDO Rayons ‘availability’ revised PDO indicators were removed and activities Extended February 4 3 to June 2013 Rayons 2015 Revised to Extended 4 reflect the to 4 Decembr2014 Rayons new closing December date 2016 Revised end targets to 4 account for 5 July 2016 Rayons actual number of beneficiaries 17. The key changes made to the project were during the second restructuring in October 2011. These included the following: (i) Subcomponent A.1: Rayon Investment (Azersu). Due to cost increases and additional investments not anticipated in the original investment plan used at project appraisal i.e. house connections, metering, repaving--Azersu would rehabilitate and extend water supply and sewerage systems in four rayons (Shamaki, Gabala, Saatli and Sabirabad) rather than in 22 rayons as initially planned. These four rayons were selected given their advanced preparation status, proximity to Baku and to each other, as well as their confirmed commitment to the project. More details on these changes are available in the key factors during implementation section. (ii) Subcomponent A.2: Rayon Investment State Amelioration and Water Management Agency (SAWMA). This subcomponent, originally planned to rehabilitate and extend the water supply and sewerage systems in rayons in Nakhchivan, was cancelled due to misprocurement by SAWMA. Page 11 of 59 The World Bank NATIONAL WATER SUPPLY & SANITATION (P096213) 18. More details on the specific areas of changes are available in the subsequent paragraphs. Revised PDOs and Outcome Targets 19. The PDO changes were introduced during the second restructuring of October 2011 and the PDO was revised as follows: The original PDO was “to improve the availability, quality, reliability, and sustainability of water supply and sanitation services in 20 of Azerbaijan’s regional (rayon) centers.� (according to the PAD dated May 2007, the LA PDO was different—see paragraph 9). The revision was to drop the words ‘Sustainability’ 5 and ‘Availability’ and to replace ‘20 of Azerbaijan’s regional (rayon) centers’ with ‘selected regional (rayon) centers of Azerbaijan’. The revised PDO is thus “to provide quality and reliable water supply and sanitation services in selected regional (rayon) centers of Azerbaijan,� whereby quality and reliable water supply and sanitation services is defined as 24-hour continuous service that complies with water supply quality standards, including treatment in conformity with the European Union (EU) Wastewater Treatment Directive for secondary treatment. Revised PDO Indicators 20. The PDO indicators were revised as detailed in the following paragraphs. 21. During the second restructuring of October 2011 the Results Framework was revised to reflect the revised PDO and activities. In addition, the project indicators were simplified and reduced in number to strengthen the overall monitoring framework. 22. New PDO indicators introduced. (a) People in project area receiving improved water supply and sanitation services from the project (improved water supply is defined as 24-hour continuous service that complies with quality standards) and (b) pollution load measured in persons equivalent (PE) eliminated through adequate wastewater treatment (adequate treatment is defined as conformity with the EU Wastewater Treatment Directive for secondary treatment). 23. Dropped PDO indicators. (a) Number of people served with piped water supply, (b) number of people connected to sewerage network, (c) percentage of drinking water samples in project area meeting Azeri water quality standards, (d) daily number of hours of water supply service, and (e) working ratio (operating expenditures divided by collected revenues) of the 20 rayon utilities in which investment is completed. 24. During the fourth restructuring in December 2014, the Results Framework was revised to reflect the new closing date of December 31, 2016 and revised targets as shown in table 3 below. 25. During the fifth restructuring, in July 2016, specific changes were made to the Results Framework: (a) updating the end targets for two PDO and four intermediate indicators, to reflect the Governments’ decision not to provide additional financing to cover the financing gap. Overall, the end targets for 6 out of 15 indicators were revised to account for the actual number of beneficiaries based on approved final amendments to the ongoing contracts. 5The World Bank team had determined that the original time frame allotted by the project was insufficient to achieve the sustainability objective, which should be implemented over a longer time horizon. Page 12 of 59 The World Bank NATIONAL WATER SUPPLY & SANITATION (P096213) 26. The end targets in the revised Results Framework for both restructurings are summarized in table 3. Table 3. PDO Targets Changes End Target as End Target as Unit of End Target as Indicator Name of October of December Measure of July 2016 2011 2014 People in project area receiving improved water supply and Number 200,000 145,000 93,633 sanitation services from the project Pollution load measured in persons equivalent (PE) Number 105,000 89,000 46,667 eliminated through adequate wastewater treatment Revised Components 27. During the second restructuring in October 2011, the following changes were introduced in addition to the key changes listed in paragraph 17. • Component C: Institutional Modernization. (i) Subcomponent C.1 continued to support Azersu’s new management to restructure services and lay the foundations for achieving financial sustainability in a later phase. The following originally planned activities were financed: (a) developing a financial restructuring and recovery plan; (b) implementing a metering and billing and collection system; and (c) introducing International Financial Reporting Standards (IFRS). (ii) Subcomponent C.2, providing support to SAWMA, was dropped due to the elimination of SAWMA as the implementing agency. (iii) Subcomponent C.3, Strengthening the Tariff Council, was dropped due to alternative support provided by the European Bank for Reconstruction and Development; and (iv) Subcomponent C.4, Review and Improvement of Selected Technical Standards, was also dropped. • Component D.2, Project Management, supporting the strengthening of SAWMA’s management capacity, was dropped. Other Changes 28. In the first restructuring of March 2009, following the GoA’s request, the State Committee for Urban Planning and Architecture (SCUPA) was charged with the implementation of Component B of the project. 29. During the second restructuring in October 2011, the following additional changes were made: (a) cancelling US$22.42 million due to the misprocurement of two contracts for water supply systems and sewerage collectors for the Sharur and Babek Rayons in Nakhchivan, (b) eliminating financial covenants on collected revenues and operating expenses for Azersu, (c) extending the closing date to February 28, 2013, and (d) reallocating project costs to reflect the cancellation and other changes as shown in table 4. Page 13 of 59 The World Bank NATIONAL WATER SUPPLY & SANITATION (P096213) Table 4. Revised Project Costs Project Costs (World Bank-financed portion) Components/Activities Original Revised in October 2011 Component A: Rayon Investment 200.00 174.58 Component B: Regional Development Plan 3.75 4.32 Component C: Institutional Modernization 9.50 12.36 Component D: Project Implementation and Management 1.50 2.62 Price Contingencies 15.25 13.70 Total 230.00 207.58 30. The third restructuring in February 2013, extended the project closing date from February 28, 2013, to December 31, 2014, to enable completion of the ongoing contracts and fully meet the PDOs. 31. The fourth restructuring in December 2014, extended the project closing date to December 31, 2016, to provide the implementing agency sufficient time to complete the existing contracts and support satisfactory achievement of the PDOs. The expenditures categories were reallocated to reflect additional implementation costs and align with committed funds. 32. There were various changes in scope and time of the NWSSP. In addition to the reduced number of rayons from 22 to 4, the networks design in the 4 rayons were scaled up and the number of households that needed to be connected doubled and tripled. The scope of the contracts was extended. This was done within the available budget resources. As a result, there was shortage of funds to finalize the wastewater treatment plants (WWTPs) in two rayons. Rationale for Changes and Their Implication on the Original Theory of Change 33. As explained in the restructuring, ‘Sustainability’ and ‘Availability’ were removed from the PDO. While ‘Reliability’ and ‘Quality’, which are left in the PDO, well-described the WSS service outcomes, and they were not sufficiently representative of the expected outcome of water made available for the newly connected population. The team did not provide sufficient justification for removing ‘Availability’ in the restructuring paper for this change. For the Implementation Completion and Results Report (ICR) evaluation after the second restructuring, the PDO indicator for people with improved service was subsumed under the ‘reliability’ objective. 34. The original time frame allocated for project implementation was considered insufficient at the restructuring to achieve the ‘Sustainability’ objective, which should be achieved over a longer time horizon. This implied shifting the PDO objective, ‘Sustainability’, measured with utility working ratio to the long-term objective influenced by other projects and interventions. This change reduced the development objective and outcomes ambition, but was considered to align the PDO with the realistic expectation from the project. 35. The restructurings revised not only the PDO, but also the scale of the project’s impact. The scope of the project was substantially reduced from what was originally envisaged, given the significant increases in the cost of the investments due primarily to price increases and additional elements incorporated in the investments (particularly house connections, metering, and repaving). Initially, investments were planned for 20 rayons, which were increased to 22 when the project started, but in the end the investments were made only for 4 rayons. 36. All these changes affected the original theory of change, especially shifting sustainability (measured with operational cost recovery) to an indefinite time in the future. In fact, all the other three elements of the PDO (Availability, Page 14 of 59 The World Bank NATIONAL WATER SUPPLY & SANITATION (P096213) Reliability, and Quality) are part of sustainable service provision. Nevertheless, the project managed to have targeted infrastructure investments (although at a significantly smaller scale) combined with institutional development which had a development impact on the outcomes at a national and Greater Baku level. II. OUTCOME A. RELEVANCE OF PDOs Assessment of Relevance of PDOs and Rating 37. Alignment with Country Partnership Framework. The project is aligned with the focus areas of the Country Partnership Framework (CPF) for Azerbaijan covering FY16–FY20.6 Focus Area 1 of the CPF (Public sector management and service delivery), Objective 1.1: Strengthen capacity for public resource management—includes preparing annual financial reports under international standards which are aligned with the introduction of IFRS under the project. Under Focus Areas 1 and 2 (Economic competitiveness), the CPF promotes investments in infrastructure for better access to public services and infrastructure for growth. These are two elements for improved connectivity and productivity growth defined by the Systematic Country Diagnostic (SCD)7 as a key constraint to the country’s inclusive growth. Assuming the long-term successful engagement in the water sector, the World Bank Group program will help improve access to water and sanitation services, through the completed and ongoing operations. Following the SCD recommendations, the CPF puts a special focus on rural development (Focus Area 2) through investments in quality infrastructure that will contribute to enhanced competitiveness. The CPF also focuses on improved human development outcomes and increased prosperity through better access to water and sanitation and improved quality of the environment that have had a proven impact on health and social welfare outcomes where the project has contributed. 38. The current CPF considers the Government’s intention to optimize the external borrowing over the medium term as part of the ongoing effort to maintain fiscal sustainability. The sustainability of the significant public investment in construction and rehabilitation of infrastructure is undermined by ineffective and underfunded maintenance systems. In that context, the fact that project restructuring dropped the ‘Sustainability’ objective from the PDO can be seen as an effort to move away from the World Bank’s overall engagement strategy. However, the Government is committed, despite the ongoing fiscal consolidation, to continue expanding physical infrastructure at the local and community levels to complete the agenda of supplying water and sanitation and improved environmental assets to underserved populations. Improved access to clean water and sanitation are critical for improving health outcomes. Improved utilities will positively impact the quality of life and economic opportunities, especially, outside the capital city, where the majority of the bottom 40 reside. The project is supporting the Government’s commitment in this respect. 39. Alignment with national policies. The Government is implementing the third State Program of Social and Economic Development of Regions for 2014–18, which aims at reducing regional inequalities and providing households, in all rayon centers, with reliable basic services, such as water and sanitation and other communal services. 40. The PDO is rated to be of Substantial relevance to the current World Bank CPF and GoA priorities. 6 Report No. 95860-AZ 7 Report No. 97113, June 3, 2015, South Caucasus Country Management Unit (ECCU3), Europe and Central Asia. Page 15 of 59 The World Bank NATIONAL WATER SUPPLY & SANITATION (P096213) B. ACHIEVEMENT OF PDOs (EFFICACY) 41. The project had five restructurings, including narrowed scope, revised PDO, changed PDO indicators and targets, reallocated funds, cancelled funds, and a few closing date extensions. Due to the narrowed scope, a split rating is applied. Some of the restructurings are related only to time extension or implementation entity change. They are not included in the weighted split rating as they do not have a direct impact on the rating changes. Table 5 summarizes the key changes during the restructurings. Table 5. Key Changes During Project Restructurings Restructuring Change In Included N Date Scope PDO PDO PDO Closing Other in the Indicators Targets Date Split Rating 1 March Change in 2009 implementing No agency 2 October Reduced Changed Changed Changed Yes 2011 3 February Extended No 2013 4 December Changed Extended Yes 2014 5 July 2016 Changed Yes Assessment of Achievement of Each Objective/Outcome 42. Pre-restructuring. For the assessment of the efficacy of the PDO, the original PDO as in the LA (To improve the availability, quality, reliability, and sustainability of water supply and sanitation services of the Borrower) has been split into the following four subobjectives/outcomes: (a) Availability (b) Quality (c) Reliability (d) Sustainability 43. The following sections summarize the ratings according to the different restructurings included in the split rating (table 3 on page 14 summarized the PDO indicator changes during the project). 44. Improved availability of water supply and sanitation services. The ‘improved availability’ aspect of the PDO is assessed by the following two indicators: (a) number of people served by piped water supply and (b) number of people connected to a sewerage network with an operating WWTP. ‘Availability’ was removed from the PDO in the October 2011 restructuring. At that time the relevant PDO indicators defined targets at 20,000/10,000 respectively. By the end of the project these targets were exceeded. Thus, ‘Availability’ is rated Substantial. However, it should be noted that these Page 16 of 59 The World Bank NATIONAL WATER SUPPLY & SANITATION (P096213) targets are inappropriately low in comparison with the project scope at that time (originally 20 rayons with a target of 20,000 were to be serviced versus 4 rayons at the end of the project with almost 100,000 people being serviced). 45. Improved quality of water supply and sanitation services. The ‘Quality’ outcome is assessed by the following two indicators: (a) percentage of drinking water samples in project area meeting Azeri water quality standards—the project achieved the target at 100 percent. This indicator was moved from PDO to an intermediate result indicator, but because this objective is sustained in the PDO and the target of 100 percent was not changed, it is assessed, for the entire project, as High; (b) pollution load measured in persons equivalent—this was introduced, during the October 2011 restructuring, as a PDO indicator. As such, it reflects different and more targeted measures of an additional (connected with sanitation) aspect of the ‘Quality’ PDO—both indicators are considered for this part of the objective. However, because the subsequent restructuring significantly reduced the PDO target and project ambition, it triggers a split rating. Thus, the second PDO indicator on ‘Quality’ is assessed as Negligible after the second restructuring as the target was more than twice higher than what was achieved by the end of the project. The second ‘Quality’ indicator, pollution load measured in persons equivalent, is assessed as Modest as it had a revised target, which was significantly higher than what was achieved. It is expected to achieve the expected target once the Gabala WWTP becomes operational. It is dependent on the GoA-financed secondary treatment to be installed. Thus, ‘Quality’ measured with ‘pollution load’ had not reached the respective target, but one of the two WWTPs is operational and it is rated Modest. 46. Reliability. The reliability dimension reflects on the achievement against the indicator ‘Daily number of hours of water supply service’. ‘Reliability’, which was kept in the PDO through the entire project, is assessed for the entire project life as High as it increased the reliability from 4 to 24 hours water supply per day. The ‘Availability’ aspect of the PDO was dropped. The PDO indicator ‘People in project area receiving improved water supply and sanitation services resulting from the project’ previously considered under ‘Availability’ is considered under the objective ‘Reliability’ after the change of PDO. Thus, the indicator used to measure ‘Availability’ had multiple revisions to PDO indicator targets with respective reduction in project ambition. At the second and fourth restructuring ‘Availability’ targets of 200,000 and 145,000 were significantly higher compared to the achievement at the end of the project (98,597) and it is rated Modest. After the fifth restructuring, the PDO target reduction to 93,633 made it possible to have the project result exceed the PDO target. However, it should be noted that the project managed to complete one of the two WWTPs. The one in Gabala is going to have a biological treatment stage financed with the GoA’s resources. More than a year after the project was closed, it is still not operational. In addition, there are still a few customers ready to be connected to sanitation, but still not due to their willingness to continue using septic tanks. Thus, it is rated Substantial. 47. Sustainability. The sustainability dimension reflects the achievements against the working ratio (operating expenditures divided by collected revenues) of the water utilities. The cost recovery of even the O&M costs was not reached neither at the Azersu Group level nor in the specific utilities where project investment took place. It was estimated to be between 0.70 and 0.80 implying that the operational costs were fully covered by the revenue, while the actually achieved operational ratio was above 1 with the exception of Gabala for 2017. This is the reason ‘Sustainability’ is rated Negligible. Justification of Overall Efficacy Rating 48. Given the achievement of the targets set out in the PDO indicators and weighted split rating, the overall efficacy can be rated from Modest to Substantial in the different project period splits as shown in table6. Page 17 of 59 The World Bank NATIONAL WATER SUPPLY & SANITATION (P096213) Table 6. Efficacy Rating PDO (May 2007) Before After After After Restructuring 2 Restructuring 2 Restructuring 4 Restructuring 5 (October 2011) (December (July 2016)/End 2014) of Project To provide quality and reliable water supply and To improve the availability, quality, reliability, and sustainability of sanitation services in selected regional (rayon) centers water supply and sanitation (WSS) services of the Borrower. of Azerbaijan. Efficacy (PDO) Modest Modest Substantial Substantial Availability: Number of people: Target 20,000/10,000 (a) served with piped water supply Progress (b) connected to sewerage network Efficacy Substantial (Removed PDO indicator) Quality: Percentage of drinking water Target 100% 100% 100% samples in project area meeting Azeri Progress 70% 100% 100% water quality standards Efficacy High (Moved from PDO to Intermediate Result) Quality: Pollution load measured in Target 105,000 89,000 46,667 persons equivalent (PE) eliminated Progress — — — through adequate wastewater Efficacy Negligible Modest Modest treatment (New PDO indicator) Reliability: Daily number of hours of Target 24 24 24 24 water supply service Progress 4 10 12 24 (Moved from PDO to Intermediate Efficacy High Result) People in project area receiving Target 200,000 145,000 93,633 improved water supply and sanitation Progress 50,000 87,800 98,597 services resulting from the project Efficacy Modest Modest Substantial a (New PDO indicator) Sustainability: Working ratio Target <1 (operating expenditures divided by Progress 1.2 collected revenues) of the 20 rayon Efficacy Negligible water utilities. (Dropped PDO indicator) Note: a. The WWTP in Gabala is still not operational—this is the reason why the second ‘Quality’ rating is not high. C. EFFICIENCY 49. Economic analysis. As part of project preparation, a financial model was developed, which shows the financial projections of Azersu on a consolidated basis, and the financials of the 20 rayons where investment was supposed to take place, covering the 20-year period of the term loan to the Government. The only benefits that were measured in monetary terms are (a) user benefits measured through stated willingness to pay (WTP) and reduced coping costs and (b) system benefits through energy efficiency gains. Page 18 of 59 The World Bank NATIONAL WATER SUPPLY & SANITATION (P096213) 50. For this ICR and according to the appraisal methodology, a quantitative version of the economic analysis was developed. In addition to the two types of benefits that were quantitatively analyzed at appraisal, the benefits from sanitation improvement were also quantified. The lack of data, methodological issues, and comparability prevent a quantitative evaluation of other impacts discussed in this section. The project’s main impacts and benefits include (i) Economic gains, resulting from sustaining or increasing current water consumption levels; (ii) Reduction of coping strategies undertaken by households to mitigate water shortages and/or lack of access to water supplies and lack of adequate sanitation facilities; (iii) Positive impacts on public health, from sustaining or increasing current water consumption levels, and improving the quality of water supplies; (iv) External effects (for instance, on the environment); and (v) Macroeconomic impacts. Economic Analysis of the Four Rayons Where Investment Took Place 51. The economic cost-benefit analysis (CBA) relies on the following three main sources of quantitative information: (i) For user benefits and respective coping costs for water supply (ii) For user benefits and respective coping costs for sanitation (iii) For system benefits such as energy efficiency gains 52. The project investment was US$207 million and the assessed quantified annual benefits amount to AZN 17,700 million (US$10.4 million) for a 20-year period. Table 7 shows the estimated net present values (NPVs) and internal rates of return of the project. The economic analysis followed a conventional approach in which the financial cash flows have been translated into economic cash flows by using standard conversion factors, while adding externalities where appropriate (see detailed assumptions and calculations in annex 4). Table 7. Summary of the NWSSP Economic Results Present Value of Flows At Appraisal At Completion NPV (AZN, million) 6.76 ‒100.4 Economic Internal Rate of Return 10.4% n.a. a Note: a. The NPV is highly negative and it is not possible to calculate the internal rate of return. 53. The negative NPV of the project is also a result of the highly reduced scope (12 times less population with new connections and 8 times less population with improved service) and at the same time almost the same amount of investment. Table 8 shows the reduced scope and investment per capita which is now US$2,322 instead of US$387 at appraisal. This estimate is based on the total project cost, which includes institutional component and other services not directly related to the WSS investment sites. This would to some extend reduce the actual price per-capita. However, for users with new connections the investment per capita is even higher. The reasons for this increase are as follows: (a) Azersu had requested changes to the technical designs, including treatment in conformity with the EU Wastewater Treatment Directive for secondary treatment; these changes proposed by Azersu generally implied higher cost for lump- Page 19 of 59 The World Bank NATIONAL WATER SUPPLY & SANITATION (P096213) sum contracts and respective investment per capita increased and; (b) the feasibility studies used were not with the highest quality and underestimated the investment cost. 54. Following consultation with other IFIs which support the WSS sector in Azerbaijan, it was noted that the changes of specifications also led to an increase of investment per capita from a range of US$1,200 to US$3,288 per capita. It was found out that the large difference of investment per capita depends on the inclusion of sanitation and WWTPs. Projects which are focused only on water supply have significantly lower costs than the others which have sanitation and WWTPs according to EU standards. In comparison with these projects, the NWSSP has a per capita investment which is even lower than the other IFIs. Also, the Second National Water Supply and Sanitation Project (SNWSSP) has average investment per capita ranging from US$1700–US$2400. 55. This scale of investment per capita compared with an even more extensive quantitative assessment of the benefits will not realistically bring a positive NPV within the infrastructure lifespan. Table 8. Project’s 20 Rayons at Appraisal versus Actual 4 Rayons At Appraisal (20 rayons) Actual Project (4 rayons) 2007 2013 2008 2017 Total production (m3 per year, thousands) 52,676 42,869 4,297 4,722 Total consumption (m3 per year, thousands) 21,070 40,726 3,529 2,714 Population 484,783 695,010 103,315 145,642 Service of population coverage ratio (%) 56% 100% 52% 61% Population served 271,967 695,010 53,429 89,130 Metering - Coverage of total population (%) 24% 100% 0% 54% Table 9. Investment per capita At Appraisal Actual Project Ratio (%) Other IFIs Population served 695,010 89,130 13% including new connections 423,043 35,701 8% Investment cost $269,000,000 $207,000,000 77% Investment per capita $387 $2,322 600% $1,200 to $3,288 56. Azersu Group - financial assessment. Despite notable improvements in 2005, Azersu was unable to meet its operating costs and cash flow requirements at appraisal time, let alone provide a sufficient return on invested capital to allow for future reinvestments, expansion of systems, and improvement of service quality. Revenues were forecasted to increase in the period 2006–11 and the working ratio was estimated to be between 0.70 and 0.80 implying that the operational costs were fully covered by the revenue and available resources for capital expenditures. However, analyzing the actual (adjusted) data received from the audited financial statement, the group continues to fall short of producing sustainable working ratios and is hardly able to cover its O&M cost (for more information see annex 4). 57. According to the independent auditor’s reports from 2012–16, the going concern basis assumes that the Azersu Group will continue its operations for the foreseeable future. The group's incurred net losses give rise to a material uncertainty which may cast significant doubt about the group's ability to continue as a going concern. However, the management believes that appropriate measures are being taken for the group to continue its operational existence in the foreseeable future. Accordingly, the group applied the going concern basis in preparing its consolidated financial statements. Page 20 of 59 The World Bank NATIONAL WATER SUPPLY & SANITATION (P096213) 58. Water tariff. The extent to which the water utilities and the Tariff Council are pursuing a cost recovery policy depends on various factors like purchasing power and affordability. The proposed rehabilitation of infrastructure under the project helped in this respect because it is designed to minimize operation cost. The utilities were also provided with tools for increasing collection rates with the assistance of the computerized billing and collection system installed under the project. Other than increased collection rates, significant tariff increases were among the single most important factors contributing to reforming the financials of Azersu. A more or less doubling of tariffs for water and wastewater services since January 8, 2007, was a significant step in the right direction. Tariff increase was an important step toward achieving financial viability in the rayon water utilities. On May 13, 2016, the Tariff Council of the Republic of Azerbaijan approved the increase of water prices and wastewater treatment tariffs charged to customers. Eventually this will enhance the group's financial position upon completion of the capital projects, which are currently under construction and connection of new customers to the WSS system. 59. The financial analysis detailed in annex 4 includes the following elements: (a) brief financial assessment of Azersu as of December 31, 2016, (b) financial analysis and key assumptions for the Azersu Group before and after the project, (c) water tariff analysis, (d) financial analysis for the four rayons covered under the project, and (e) economic CBA of project investments. Assessment of Efficiency and Rating 60. The economic efficiency based on the quantified and monetized benefits is negative, even after additional benefits were quantified at the ICR stage. However, the project-related non-monetized benefits have significant social, environmental, and economic benefits (presented in table 4.3 in annex 4) represent significant outcomes from the project. Furthermore, the investment per capita for WSS projects financed by other IFIs is at a comparable level. There were also numerous extensions of project closing date. The latter generated an increased cost of supervision and implementation. Thus, the efficiency is rated Modest. D. JUSTIFICATION OF OVERALL OUTCOME RATING 61. Based on the abovementioned assessment and split ratings provided in table 10, the overall outcome rating of the project is rated ‘Moderately Unsatisfactory’. Notwithstanding this rating, the project was the first in the water sector at the national level and is considered a breakthrough project, considering the challenging environment of having just come out of the Soviet Union era in which the project was prepared and implemented. 62. The Theory of Change is significantly affected both by (a) outcome ambition—where sustainability was removed from the PDO and the WSS service operation is not even close to the O&M cost recovery as initially envisaged and (b) the scale of infrastructure investment and respective number of beneficiaries were significantly reduced and investment per capita went to levels where economic justification for the project is doubtful, leading to a Moderately Unsatisfactory ICR rating in disconnect from the project’s last Implementation Status and Results Report (ISR) rating. Nevertheless, the project managed to have targeted infrastructure investments combined with other outcomes related to the institutional component, which have longer-term influence on sector development, transparency, accountability, and sustainability. Page 21 of 59 The World Bank NATIONAL WATER SUPPLY & SANITATION (P096213) Table 10. Overall Outcome Rating PDO (May 2007) Before After After After Restructuring Restructuring Restructuring Restructuring 5 2 2 (October 4 (December (July 2011) 2014) 2016)/End of Project To improve the availability, quality, reliability, and To provide quality and reliable water supply and sustainability of water supply and sanitation (WSS) services sanitation services in selected regional (rayon) of the Borrower. centers of Azerbaijan. Disbursement (US$, millions at 24.85 141.02 181.02 189.72 the end of the period) Disbursement (% of budget at 12.00% 68.13% 87.45% 91.65% appraisal) Disbursement (% of total 13.10% 74.33% 95.41% 100.00% disbursement at completion) Relevance of objective Substantial Efficacy (PDO) Modest Modest Substantial Substantial Efficiency Modest Outcome ratings Moderately Moderately Moderately Moderately Unsatisfactory Unsatisfactory Satisfactory Satisfactory Numerical value of the outcome 3 3 4 4 ratings Share of disbursement (%) 13.1% 61.2% 21.1% 4.6% Weighted value of the outcome 0.39 1.84 0.84 0.18 rating Rounded value 3.3 Final Outcome Rating Moderately Unsatisfactory E. OTHER OUTCOMES AND IMPACTS (IF ANY) Gender 63. The CPF program FY16–FY20 is underpinned by two cross-cutting themes and one of them is gender. It is recognized by the SCD as critical for social inclusion and accumulation of diversified assets. In addition, the project team was also encouraged to monitor the gender impact of this large project to help document better the positive development impacts of such projects. The project has been prepared in 2006 when Bank did not have strict requirement to gender disaggregate proposed indicators. This explains why the project did not measure gender related outcomes. However, the Environmental and Social Impact Assessments that have been prepared for each rayon financed under the project included gender-sensitive consultations and analysis. Institutional Strengthening Component A: Institutional Strengthening 64. All institutional targets and indicators set by the project are fully met. Overall, institutional strengthening was significant and the World Bank expects to continue supporting substantial institutional reforms/improvements in the sector under ongoing dialogue. Page 22 of 59 The World Bank NATIONAL WATER SUPPLY & SANITATION (P096213) 65. Training was provided as part of the construction contracts. Initially, a transition period was arranged when specialized contractor’s staff remained on-site in the WWTP to carry out the important training of the operations staff. The timing/length of the training depended on the contract conditions/requirements. Component B: Rayon Investment 66. These processes are accompanied by the introduction of an IFRS-based accounting system and implementation of audit recommendations submitted by entity auditors contracted under the project. The IFRS-based entity audit for 2013 and complete asset revaluation have been concluded. 67. The Greater Baku Regional Development Plan (GBRDP) Component master plan preparation and update for different cities in Azerbaijan is a lengthy process that started during the Soviet Union era. It was the responsibility of different committees and ministries over the years. Since the establishment of the State Committee for Urban Planning and Architecture (SCUPA) in 2007, the planning work was intensified with 45 plans ordered and 35 prepared and approved. Among all other city master plans, Baku is the most significant with a population of 2 million, which is 6–7 times larger than the second/third largest cities. In 2007, the World Bank agreed to proceed with the necessary arrangements regarding development of the GBRDP which is a comprehensive strategic planning document that defines an integrated economic, environmental, transport, and social framework for the development of the city. The World Bank’s support started with development of terms of references for the assignment, continued with reviewing consultant outputs, and providing, as needed, advice to the Government on international best practices. This was very timely support assuming the weak capacity in urban planning of both Azersu and SCUPA as newly created institutions. In addition to the international practices that were brought, a strategic environment assessment was piloted for the first time. Upon finalization of the draft GBRDP, it has been publicly discussed and disclosed. SCUPA and Baku State Design Institute have informed that draft land use and zoning plans have been prepared covering the entire administrative territory of Greater Baku completing the work already done for the central part of Baku under the draft GBRDP. Followed by support and hands-on training from the World Bank, now the counterparts in Baku are ready to continue on their own. The investment program also benefitted from an Integrated Urban Water Resources Management Study for Greater Baku (IUWRM) study, financed through grant funds leveraged by the World Bank Water Partnership Program. Overall, this component has been regarded as a success as outlined by its impact and the various achievements underscored. 68. The Baku State Project Institute was playing the leading role in the GBRDP assignment, handling necessary coordination among key consultants and preparing economic, environmental, and spatial inputs. The institute has substantially improved its skills and staffing through the process and managed to retain most of its local and international staff that worked on the GBRDP. The same team is now actively engaged in the planning process and continues updating the GBRDP in line with changing planning environment. That explains the high level of government commitment and sustainability of the GBRDP component, which is a rare case for similar types of planning activities financed under World Bank loans where the leading preparation role is very often assigned to international planning firms. 69. On December 22, 2016, SCUPA organized the final public consultation of the final report. The consultation workshops were attended by broad representatives of the Government, academia, nongovernmental organizations, donor organizations, and mass media. Overall, the work has been highly recognized widely by the public. The land use plan of all settlements in the Greater Baku area have been prepared for the first time ever in Azerbaijan history. The current Government practice is to annually provide AZN 1.6 million to SCUPA for urban planning, which provides sufficient resources for proper continuation of the process. Page 23 of 59 The World Bank NATIONAL WATER SUPPLY & SANITATION (P096213) Component C: Institutional Modernization 70. The NWSSP was one of the first large investment projects in the sector that led to the preparation and adoption of sets of technical designs, construction, and supervision standards that are being currently applied under the entire Azersu Water Program financed by multiple donors and the state budget. Component D: Project Management 71. The team was pleased to notice the progress under the key institutional modernization components, which were financed from Azersu’s internal resources. In particular, Azersu updated Baku’s water and sanitation master plan produced in 1998 with the support of the World Bank. There was substantial improvement of the existing billing and collection system, increased metering, including installation of smart meters, removal of illegal collections, and reduction of the volume of unpaid water. The systems were installed in local utility branches and are fully operational. This was further supported by a needs assessment for Azersu financed under the World Bank Public Investment Capacity Building Project (P115396). Azersu also established a separate geographic information system (GIS) department aiming to use the most advanced GIS technologies in the design and operational activities implemented by the company, as well as the automated supervisory control and data acquisition (SCADA) management systems that are available in the regional utilities. 72. The financing and restructuring plan was adopted by the Azersu management. There were different reshufflings of administrative structures of the company with new technical expertise and quality control departments introduced in Azersu. However, the specifically mentioned financing and restructuring plan, which was part of the project Results Framework, was not made available to the team at the ICR stage. 73. While the GBRDP as a strategic document is available to authorities, the Baku proper planning, which is the more detailed plan, was finalized in December 2014, but as of March 2018 it was still not approved. Without proper plans and zoning ordinance requirements, the business corporations can construct any type of buildings anywhere in Baku regardless of the consequences. Supported by market economy, major commercial and residential projects have developed at an extraordinary pace and often exceeded the limits defined in the GBRDP. While this is not a major threat for the GBRDP’s sustainability, it will imply revisions to various levels of urban planning and establishment of clear process for these updates. Mobilizing Private Sector Financing Not applicable. Poverty Reduction and Shared Prosperity 74. The quality of environmental assets is a critical contributing factor to the well-being of people, affecting their health and economic opportunities, especially for the bottom 40. If the depletion of resources continues at the rate of the last decade, and water contamination is not properly addressed, Azerbaijan will be at substantial risk of significant deterioration of its environmental assets with gloomy prospects for environmental sustainability. In particular, landscape degradation, desertification, limited wastewater treatment leading to significant pollution of river waters and the Caspian Sea, and poor quality of drinking water are major concerns. The project contributed to improved health outcomes and Page 24 of 59 The World Bank NATIONAL WATER SUPPLY & SANITATION (P096213) increased prosperity through targeting water and sanitation services, quality of environmental assets, and reforms in the delivery of WSS services. Other Unintended Outcomes and Impacts 75. The Integrated Urban Water Management Study for Greater Baku, was an unintended benefit from the project. The study analyzed conditions and trends in water resources management in the Greater Baku Area, and its findings and recommendations informed extensively the update of the Greater Baku Water and Sanitation Master Plan, and the preparation of the GBRDP. 76. Support to storm water management in the Greater Baku area was also provided in response to a request of the Cabinet of Ministers of Azerbaijan. The World Bank’s water team mobilized resources to review the new Water and Sanitation Master Plan for Greater Baku area and advice regarding priority actions toward improved storm water management system in downtown part of Baku city. The World Bank team produced a Storm Water Management Note for the Greater Baku area in 2017 and continues the dialogue between the GoA and the World Bank intended to enhance policy and operational efficiency in the sector as part of pipeline operations in the sector 77. The WSS industry was not developed at the time of project appraisal. Pipes and other WSS material were imported from abroad. During the project implementation, many WSS industries were established in Azerbaijan. Initially there were two or three companies in this sector with turnover around US$5 million–US$15 million equivalent. Because of the Government’s WSS program supported by the project, the local market has developed large companies in the WSS sector, with turnover more than 1 billion. Outside the domestic market Azeri companies are now implementing projects in Uzbekistan, Tajikistan, Ukraine, and Georgia. 78. The project also influenced the development of local expertise in implementation of İnternational contracts, procurement, construction supervisions, and financial management. After the end of the project, more than 10 companies have worked in local and foreign WSS consulting markets. 79. During construction supervision the World Bank team developed an innovative GIS web solution to supervise construction under multiple contracts in an online mode. The system was later expanded by Azersu’s IT department and transformed into countrywide GIS asset management tool that contributes to the overall sustainability of project investments. 80. The project preparation and implementation has been conducted in close consultation with citizens. As such the borrower had a series of public consultations during preparation of Environmental Impact Assessments (EIAs) and feasibility studies. All contractor’s camps and local offices of construction supervision firms had well-established grievance mechanisms that were used during the construction period as confirmed by regular inspections of the World Bank’s safeguards specialists. The World Bank did not receive a single complaint during the construction period, but rather received multiple letters of appreciation addressed to Azersu. Page 25 of 59 The World Bank NATIONAL WATER SUPPLY & SANITATION (P096213) III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME A. KEY FACTORS DURING PREPARATION 81. The Government’s approach was to cover WSS improvements in all of Azerbaijan’s rayons. At this time the Government, with support from IFIs, had projects under implementation or preparation in about 80 percent of the rayons and intended to cover the remaining rayons in the near future. Nevertheless, the selection of the rayons included in this project was based on a needs assessment, focusing on factors such as population size, the state of deterioration of facilities, and service quality. 82. Studies carried out as part of project preparation revealed that most water supply production, treatment, transmission, and distribution infrastructure exceeded their economic life span and were severely deteriorated, affecting the quality and reliability of services. 83. The feasibility studies, mostly prepared before project effectiveness, were developed using key design criteria (that is, daily consumption per capita, planning horizon, trench standard, and so on) based on best international practices and agreed between the Azersu and the World Bank. However, during project implementation several technical revisions to the key design criteria were introduced by Azersu. These changes generally implied higher cost for the lump-sum contracts and respective investment per capita increased. It should also be noted that the significantly reduced scope, from 22 to 4 rayons, is to a large extent the result of initially underestimated investment costs and substantial scale-up of the WSS networks in the 4 rayons. Results Framework Design 84. The initial PDO indicator targets describing ‘Availability’ outcomes were (a) number of people served with piped water supply and (b) number of people connected to sewerage network—target of (a) 20,000 and (b) 10,000, while the key assumptions for the 20 project rayons in the economic analysis were referring to the population served going up from 271,967 to 695,010. Moreover, after the second restructuring, when the project investment component was significantly reduced (from 22 to 4 rayons) the revised PDO indicator ‘people in project area receiving improved water supply and sanitation services resulting from the project’ was targeted at 200,000. This shows that the initial PDO indicator for 22 rayons with 20,000 and 10,000 is not in accordance with the planned appraisal outcome. Readiness for Implementation 85. Effectiveness was declared in March 2008 and shortly after that the construction management firm was mobilized and the Procurement Plan was submitted to the World Bank. Feasibility studies for 10 rayons in the main land were completed. The Project Management Unit (PMU) within Azersu was fully operational and the funds were transferred to the Designated Account. B. KEY FACTORS DURING IMPLEMENTATION 86. Following the NWSSP, an IBRD loan in the amount of US$230 million and an IDA credit in the amount of US$30 million equivalent were approved by the World Bank for the SNWSSP on May 27, 2008. These two projects are very similar in design and both finance rehabilitation and expansion of WSS systems in rayon capitals, four in the case of the NWSSP and eight under the SNWSSP. Although these projects are implemented by two different agencies (Azersu, national water Page 26 of 59 The World Bank NATIONAL WATER SUPPLY & SANITATION (P096213) supply company, in the case of NWSSP, and Amelioration and Water Management, in the case of SNWSSP), all rehabilitated water systems are operated by Azersu. Therefore, these two projects need to be seen as complementary operations, as well as part of a broader engagement and program in the country. The SNWSSP heavily built on lessons of the NWSSP. 87. In October 2009, The Deputy Prime Minister and Azersu President expressed concerns about the very limited number of local contractors pre-qualified for the construction works to be carried out in the six rayons under the NWSSP. To address this issue, the World Bank recommended the following actions: (a) project implementation units to organize meetings with interested local companies to inform them about the WSS program and advise on participation in the bidding process (mainly through association), (b) include in the future pre-qualification and bidding documents a statement encouraging international companies to associate with Azeri firms, and (c) consider relaxing, to the extent possible, the experience criteria. 88. On January 19, 2011, the World Bank declared misprocurement of the two contracts for water supply and sewerage systems for the Sharur and Babek Rayons of the Nakhchivan Autonomous Republic with the cancellation of the loan proceeds accordingly. 89. Based on the noticeable progress in the implementation of main investments and institutional modernization, the World Bank upgraded the project’s implementation rating from Moderately Unsatisfactory to Moderately Satisfactory in early 2013. The World Bank team was informed, by Azersu, that the top administration of all four rayons requested a larger scope for contracted civil works such as connecting additional villages and households and associated additional metering, rehabilitation of administrative buildings for Azersu’s subsidiaries, and installing storm water drainage systems. This resulted in amendments to the scope and amounts of the existing network contracts in all rayons under the project. To cover the larger scope than originally envisaged, the World Bank urgently requested the Government’s decision on the source of additional financing. 90. Following the new institutional structure of Azersu the newly created technical and quality review departments started reviewing the detailed designs prepared by the contractors. The changes proposed by Azersu, following the review, generally implied higher costs for the current lump-sum contracts. Also, due to additional investments not anticipated in the original investment plan—that is, house connections, metering, and pavement reinstatement—Azersu would rehabilitate and extend the water supply and sewerage systems in four rayons (Shamaki, Gabala, Saatli, and Sabirabad) rather than in 22 rayons as initially planned. These reviews also resulted in some delays with the approval of detailed designs and even suspension of the ongoing civil works (that is, construction of the WWTP and networks in Shamakhi Rayon). In fact, the bulk of Azersu's comments discussed with the World Bank’s team were considered technically sound and acceptable to the World Bank, leading to an increased efficiency of new systems, but also to increased cost of contracts. 91. With a clear financing gap of between US$1,020 million and US$1,150 million to complete the existing investment program by 2013, the World Bank's project team encouraged the Government to urgently prepare a financing plan that could include further state funding and mobilize resources from different IFIs. Given the size of the proposed investment program, the project team proposed these investments to be phased so that there was adequate implementation and appropriate supervision of works. 92. In November 2014, the GoA requested additional financing from the World Bank to cover the increased number of house connections, and so on. At the same time, Azersu amended (for example, scaled up the water supply networks and connections) all the ongoing contracts based on the Government’s decision for additional financing and the World Page 27 of 59 The World Bank NATIONAL WATER SUPPLY & SANITATION (P096213) Bank did the restructuring to reflect this scale up as these contracts were financed under the NWSSP. After the Government’s decision to drop the additional financing, Azersu scaled down the contracts back to a realistic scope and the team did the final restructuring to align the project Results Framework to the final scope. 93. The contract for construction of the Gabala WWTP was signed on January 9, 2013, and finalized at project’s closure only for mechanical treatment. The WWTP was still not operational at the time of the ICR. To successfully implement the project on time, project planning should adequately schedule implementation stages, including design approval. Significant factors which also affected successful completion were (a) slow connection rate of households to sewer systems and (b) Azersu’s limited capacity to take over the facilities financially and technically. 94. The design modifications mostly did not affect the original employer requirements in the contracts and the contracts have been modified within their original lump-sum price. At the same time the Bill of Quantities-based civil work contracts worked well for civil work-intensive contracts such as construction of water intakes, transmission lines, and water and wastewater networks given the design modifications and scale-up of contract boundaries. The contractor and borrower have simply agreed to use the original unit rates while modifying the original BoQ or scaling up the scope. This approach provided a considerable economic benefit (based on economy of scale) and reduced the per capita cost of capex. 95. Delayed ICR. The NWSSP closed on December 31, 2016. Given the programmatic nature of two water projects (see paragraph 86), the close similarity of the PDOs and project scope, and the short period between the closing dates, the possibility to prepare one single ICR for the NWSSP and SNWSSP was considered. This was very important from a program and strategic engagement perspective in the context of the CPF and sector dialogue in relation to the Government’s focus on the water sector and the World Bank’s value added, especially with respect to institutional support for the sector and Azersu. Given that the SNWSSP’s closing date has now been extended until December 31, 2018, it did not seem practical to deliver a single ICR on the due date and so it was decided to have separate ICRs. IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME A. QUALITY OF MONITORING AND EVALUATION (M&E) M&E Design 96. The PDO outcomes were significantly revised during implementation. The PDO was disaggregated succinctly into the four main PDO indicators. Two of them, ‘Availability’ and ‘Sustainability’ were removed. Moreover, the PDO indicator connected with ‘Availability’; was ‘number of people served with piped water supply’. This indicator had a target of 20,000, while the economic analysis refers to population served going up from 271,967 to 695,010. This is significant discrepancy of the most important PDO indicator and raises concerns about the quality of the M&E design. 97. The Project Implementation Team and the PMU implemented the data collection and monitoring systems from the beginning of the project. Effective systems were put in place and indicators were systematically monitored by the PMU. It was diligent in data collection and reporting on performance data to the World Bank. The M&E results were appropriately reported in the NWSSP ISRs. Page 28 of 59 The World Bank NATIONAL WATER SUPPLY & SANITATION (P096213) M&E Implementation 98. The M&E teams proceeded with continuous data collection and field investigation throughout project implementation and timely monitoring of the project’s overall progress. Effective systems were put in place and indicators were systematically monitored by the PMU. It was diligent in data collection and reporting on performance data to the World Bank. 99. There were many revisions to the M&E structure and targets. While this is considered as an improvement of the Results Framework, there were some revised targets which were overambitious regarding the expected outcomes. As can be seen in table 2 and explained in paragraphs 75–77, the PDO indicator ‘people in project area receiving improved water supply and sanitation services from the project’ was set at 200,000 in 2011 and reduced to 145,000 in 2013. The latter target was set when the GoA requested an increase in the project scope and beneficiaries within the four rayons. However, this increased scope had to involve additional financing, which was not processed and, therefore, the respective PDO target was reduced to 93,633 in 2016. These changes show that not only was the initial target of 20,000 not realistic, but the 2011 revised target was supposed to be set at around 100,000 in accordance with the respective scope. This discrepancy was to a large extent due to the feasibility study which was not of decent quality. M&E Utilization 100. The results of the M&E process were compiled and used by project management and decision makers to monitor progress closely and make changes to the project’s structure during restructurings. Project management could identify major issues regarding the scope and status of the implementation progress and align the targets of the Results Framework indicators accordingly so that almost all of them were achieved upon completion. Justification of Overall Rating of Quality of M&E 101. Considering the comprehensively designed, implemented, and used M&E framework, but also considering a few targets which were not set according to the project ambition at the time, the overall rating of quality of M&E is ‘Modest’. B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE 102. Fiduciary. General financial management issues included slow implementation, lack of counterpart funds during the early stages of the project, and contract payments. However, there were no major issues related to financial management of the project and counterpart funding was contributed in full. Azersu’s PMU was Satisfactory to the World Bank’s financial management arrangements. The interim financial reports of the project were submitted to the World Bank and were found to be satisfactory. 103. The project envisioned to improve the Azersu Group’s financial management as a whole. However, a significant part of this financial management agenda did not get implemented. Annual audits according to IFRS were introduced in 2012. This was a significant improvement as Azersu did not have corporate standards regarding financial management and reporting. The audit reports brought out many issues and found that Azersu is not a financially viable institution. The Page 29 of 59 The World Bank NATIONAL WATER SUPPLY & SANITATION (P096213) audits had limited impact on the modus operandi of Azersu and recommendations by the auditor were not implemented. Limited progress was made in this area. The audits were done at the project level and for the entire company. The project audit was postponed several times at the beginning of the project. 104. Procurement. At the beginning of the project, the Government had no history or track record of bidding and preparing feasibility studies. An international supervision company was introduced to supervise the construction on-site as the Government did not have the experience. The supervision company was supportive. 105. The first supervision company originated the misprocurement case under the project. On January 19, 2011, the World Bank declared the misprocurement of the two contracts for water supply systems and sewerage collectors for the Sharur and Babek Rayons of the Nakhchivan Autonomous Republic, with the subsequent cancellation of the loan proceeds allocated to Subcomponents A.2, C.2, and D.2 of the project, accordingly. 106. Out of the four rayons, procurement for two rayons were done on the basis of design and build contracts and for the other two rayons were done as civil works contracts. All contracts were done through international competitive bidding and all bids were through joint ventures (German and Turkish). There were no Azeri companies that were able to bid at the start. There were various changes in scope and time of the NWSSP. The networks designed were scaled up and the number of households that needed to be connected doubled and tripled. The scope of the contracts was extended. This was done within the available budget resources. As a result, there was shortage of funds to finalize the WWTPs in two rayons. There were issues without precise bill of quantities and difficult adjustment of the lump-sum contracts. Contract management by the client was difficult with designs being approved very late. The scaling up of the volume of works prevented the contractors from complaining about these delays. The contractors were engaged for six to seven years, instead of two years according to the initial contracts. 107. Environmental Assessment (OP/BP 4.01). The World Bank’s OP/BP 4.01 on Environmental Assessment was triggered and the category required was ‘A’ for the wastewater part of the project. The Environmental Impact Assessment Framework Report was prepared on March 22, 2007, for the NWSSP. Detailed EIAs were conducted for 11 rayons in accordance with the procedures for the environmental category ‘A’ assigned for the NWSSP. By November 2011, all the EIAs were completed and disclosed. There were no major issues during implementation. All site-specific documents were disclosed. Monitoring was in place. The Project Implementing Unit’s capacity to handle the EIA assignments and ensure proper safeguards management during construction works was weak and with high staff turnover. 108. International Waters (OP/BP 7.50). The project triggered the World Bank’s Operational Policy on International Waters OP 7.50. Because it was decided to remove the Nakhcivan (which was the only one location for which OP 7.50 was triggered) component from the project there were no investments which trigger the Operational Policy on international Waterways. 109. Involuntary Resettlement (OP/BP 4.12). Based on results of all updated detailed designs, there was no resettlement in any of four rayons covered under the NWSSP. C. BANK PERFORMANCE Quality at Entry 110. The World Bank teams incorporated relevant lessons from the GBWSP and from several other WSS projects in the region. The design of the project ensured that the objectives were fully aligned with the issues highlighted in strategic Page 30 of 59 The World Bank NATIONAL WATER SUPPLY & SANITATION (P096213) documents, namely the CPS and respective FY07 deliverables, and the draft Water Sector Strategy of the GoA. The project’s design was further aligned with key Government priorities at the national and provincial levels. The World Bank ensured that a team of specialists was mobilized to address all relevant project aspects, including technical aspects, safeguards, and M&E. 111. In addition, the feasibility studies used for signed civil work contracts in Shamakhi, Gabala, Saatli, and Sabirabad Rayons were developed using the key design criteria (that is, daily consumption per capita, planning horizon, trench standard, and so on) based on best international practices and agreed between the Azersu and the World Bank. Feasibility studies subsequently appeared to be underestimated and the investment price was significantly higher than envisaged. 112. There were many revisions to the M&E targets. There were many revised targets which were overambitious regarding the expected outcomes. These changes show that not only was the initial PDO target of 20,000 not realistic, but the revised 2011 target of 200,000 was also twice more than what it should be in accordance with the respective scope. This discrepancy was to a large extent due to the feasibility study which was not of decent quality. Quality of Supervision 113. Given the nature of the NWSSP design and the numerous changes that took place during the implementation period (scope, technical requirements), intense supervision and follow-up by the World Bank was required. The World Bank undertook regular supervision (and more frequent supervision when needed), including field visits and physical checks of investments. The project task team leaders were mostly based in the Azerbaijan country office during the entire project life. This resulted in well-facilitated day-to-day client engagement, stronger field presence, and strengthened project supervision. The rest of the key team members such as procurement, financial management, and environmental specialist were based in the country office from the beginning and until project closing. Critical problems and bottlenecks were identified as early as possible and solutions sought in collaboration with the PMU. Although the midterm review, dated October 29, 2012, is recorded in the portal, there are no Aide Memoires dedicated to the midterm review and the economic benefits assessment was not updated. 114. The strengths of the supervision performance included a proactive supervision team, well-balanced in skills between headquarters and in the field and with substantive experience in WSS infrastructure and institutional strengthening. 115. During implementation, the project team was working on a project with different issues at the design stage. Moreover, the different changes in client requirements during deep transition in Government priorities and sector- specific objectives were introduced. The team had to overcome a lot of profound challenges in adjusting the project to the specific client needs and still align with the World Bank’s CPF. The World Bank supervision failed to address the growing challenges around Azersu, leading to a Moderately Unsatisfactory ICR rating in disconnect from the last project’s ISR rating. Justification of Overall Rating of Bank Performance 116. The World Bank’s overall performance is rated ‘Moderately Unsatisfactory’, largely based on the Quality at Entry shortcomings. In spite of this, it is important to note that the collaboration between the World Bank and the borrower, particularly Azersu, SCUPA, and the PMU, has been valuable and critical for the project’s final achievements. As reported by the clients, the World Bank team was continuously engaged throughout the project preparation and implementation Page 31 of 59 The World Bank NATIONAL WATER SUPPLY & SANITATION (P096213) and demonstrated sufficient professionalism. However, there were some significant shortcomings in the preparation phase (scale of the project scope, Results Framework, quality of data, failed feasibility studies, underestimation of investment costs, and so on), Quality at Entry, and during implementation, which led to significant delays at the beginning of implementation. D. RISK TO DEVELOPMENT OUTCOME 117. The authorities and all different stakeholders continue to be committed to the project development outcomes and continuous improvement of the WSS service. Since project closure and during the additional one year given to deliver the ICR, the Government has made investments to complement the work that the World Bank has done and, to a certain extent, addressed the issues of sustainability, thus clearly reflecting commitment and ownership. These works include the following: (a) water supply networks - 100.3 km, (b) three water reservoirs with the total capacity of 6,000 m3, (c) water sources - one piece, (d) administration building - one piece in Shamakhi, (e) 8,616 water metering pieces, (f) water spring - 234 pieces, and (g) sewer collector - 18.4 km. 118. The Government also undertook the following work that is under construction: (a) water network – 79.4 km, (b) sewer networks - 46 km, and (c) a WWTP in Gabala. The operation of the plant is expected to commence no later than November 2018 and includes biological treatment. 119. However, there are several challenges for the authorities in the post-completion phase of the project. The following risks need to be addressed to maintain the project achievements: (a) Review of cost recovery for water supply and wastewater services provision. The financial sustainability of Azersu is still precarious and heavily dependent on the GoA subsidies, as tariffs have remained low. It is not ensured that Azersu can fully cover its operating costs, especially for the WWTP, whose operation is expensive, creating a risk for project investment sustainability. There are also some issues with customers willingness to get connected. (b) If specific care is not taken to continue institutional development, in particular, GBRDP, and its regular update and alignment with the more detailed proper plans, the development outcome of this work is at risk. (c) Operational efficiency improvements, including metering, billing and transparency (IFRS audited financial reports) need to be continued and expanded to not put the project outcome at risk. (d) In addition, the limited technical capacity of the Azersu staff to manage the wastewater assets is an important risk as well. 120. The contract for construction of the Gabala WWTP was finalized at the project’s closure only for mechanical treatment. The biological treatment stage is financed with the GoA resources. More than a year after the project has closed, it is still not operational at the time of the ICR. Moreover, the treatment facility in Shamakhi is not getting sufficient wastewater load as compared to its design capacity, as less than half of the residents have connected to the sewer system. The project financed the construction of the sewer network and treatment plant but not the household connections. Many households continue to use the septic tanks that they have been using for a long time. This is a significant risk to the development outcome of the project. Page 32 of 59 The World Bank NATIONAL WATER SUPPLY & SANITATION (P096213) V. LESSONS AND RECOMMENDATIONS 121. Importance of ‘Quality at Entry’, project readiness, and undertaking due diligence at the preparation stage. The number of restructurings, including changes to the Results Framework undertaken during the life of the project, reflected the inadequacy of the Results Framework. A well-designed Results Framework that includes indicators aligned with operational objectives and having appropriate targets is essential. Ensuring project readiness would help avoid unforeseen lags during implementation and costly implementation support efforts taken to correct design-related issues as was encountered during implementation of the project. 122. Massive construction while necessary does not always translate to sustainable services. There is a need for sustainable services delivery and build-up toward sector financial sustainability. The team needs to engage in a dialogue with the Government on how to provide service sustainability. The potential for crowding in new financing—enhanced cost recovery through tariffs and potential for private-public partnerships should be explored. 123. Capacity building takes time. The institutional and local industrial capacity is being gradually built. Staged implementation meant early visible results. Future projects should acknowledge these institutional complexities and consider appropriate measures to adress sector institutional capacity development. 124. Use of turnkey design and built contracts for the WWTPs proved to be the right choice given frequent modifications of the original design. The design modifications did not affect original employer requirements and contracts have been modified within their original lump-sum price. 125. Low connectivity to a sewer network is not just a technical issue, but also a social and citizen engagement issue that needs to be properly addressed. The project teams should be aware and the design of project activities should address the unwillingness to change with suitable communication campaigns and citizen engagement, alongside possible legal or financial incentives for people to connect. 126. Traditional Investment Project Financing has its own constraints when the objective is to support a national- level reform in a dynamic environment. Given various changes in design norms, standards, GoA priorities, capacity- building needs, and objective and development plans over the life of the Government Program, a programmatic approach could have been more suitable than a standard Investment Project Financing, which had to be restructured five times with changes to almost all aspects (scope, PDO, indicator targets, and so on). . Page 33 of 59 The World Bank NATIONAL WATER SUPPLY & SANITATION (P096213) ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS A. RESULTS INDICATORS A.1 PDO Indicators Objective/Outcome: Availability Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion People in project area Number 0.00 200000.00 93633.00 98597.00 receiving improved water supply and sanitation services 21-Nov-2012 27-Jan-2012 31-Dec-2016 19-Oct-2016 from the project Comments (achievements against targets): The indicator target was introduced at the second restructuring in January 2012. It was reduced twice, but it is achieved at the end of the project. 105% achieved based on the revised target. Objective/Outcome: Quality Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Pollution load measured in Number 0.00 105000.00 46667.00 0.00 persons equivalent (PE) eliminated through adequate 21-Nov-2012 27-Jan-2012 31-Dec-2016 19-Oct-2016 wastewater treatment Comments (achievements against targets): The indicator target was introduced at the second restructuring in January 2012. This indicator is expected to Page 34 of 59 The World Bank NATIONAL WATER SUPPLY & SANITATION (P096213) be fully met when both WWTPs are fully operational. The WWTP in Gabala is still not operational due to biological treatment financed by GoA. Objective/Outcome: Reliability Objective/Outcome: Sustainability A.2 Intermediate Results Indicators Component: Component A: Rayon Investment Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of rayons in which Number 0.00 20.00 2.00 1.00 water supply and wastewater systems in the 21-Nov-2012 31-Mar-2008 31-Dec-2016 19-Oct-2016 project area are rehabilitated and operational Comments (achievements against targets): The indicator target was revised two times from 20 to 4 at the second restructuring in January 2012. Subsequently revised to 2. One of the two WWTP is operational. However, the WWTP in Gabala is going to have biological treatment stage financed with GoA resources. More than a year after project was closed – it is still not operational. 50% achieved based on the revised target. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion New piped household water Number 0.00 28152.00 19922.00 20978.00 Page 35 of 59 The World Bank NATIONAL WATER SUPPLY & SANITATION (P096213) connections that are 21-Nov-2013 25-Jan-2010 31-Dec-2016 19-Oct-2016 resulting from the project intervention Comments (achievements against targets): The target was reduced At completion, the project had achieved 105% of its revised target. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion New household sewer Number 0.00 8440.00 9222.00 9611.00 connections constructed under the project 21-Nov-2012 27-Jan-2012 31-Dec-2016 19-Oct-2016 Comments (achievements against targets): The target was initially 8,440, it was revised to 17183.00, subsequently reduced in the last project year to 9,222 and respectively achieved at 104% of its last target. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Water network rehabilitated Kilometers 0.00 450.00 537.00 575.00 21-Nov-2012 27-Jan-2012 31-Dec-2016 19-Oct-2016 Comments (achievements against targets): The indicator was introduced in 2012 and at completion, the project had achieved 107% of its revised target. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Sewerage network Kilometers 0.00 380.00 201.00 202.00 Page 36 of 59 The World Bank NATIONAL WATER SUPPLY & SANITATION (P096213) rehabilitated 21-Nov-2012 27-Jan-2012 31-Dec-2016 19-Oct-2016 Comments (achievements against targets): The indicator was introduced in 2012 and at completion, the project had achieved 107% of its revised target. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion New reservoir capacity Cubic 0.00 17000.00 34400.00 34400.00 provided Meter(m3) 21-Nov-2012 27-Jan-2012 31-Dec-2016 19-Oct-2016 Comments (achievements against targets): 100% achieved. The indicator was introduced in 2012 and at completion has been achieved. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Daily number of hours of Number 4.00 0.00 24.00 24.00 water supply service (average per connection) 21-Nov-2012 31-Mar-2008 31-Dec-2016 10-Apr-2018 Comments (achievements against targets): This target has been 100% achieved. Component: Component B: Regional Development Plan Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Greater Baku Regional Text No Yes Yes Yes Page 37 of 59 The World Bank NATIONAL WATER SUPPLY & SANITATION (P096213) Development Plan 21-Nov-2012 31-Mar-2008 31-Dec-2016 19-Oct-2016 completed Comments (achievements against targets): This target has been achieved. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Updated Absheron Peninsula Text No Yes Yes Yes Water Supply Masterplan completed 21-Nov-2012 25-Jan-2010 31-Dec-2016 19-Oct-2016 Comments (achievements against targets): This target has been achieved. Component: Component C: Institutional Modernization Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Financing and restructuring Text No Yes Yes Yes plan completed and adopted by Azersu 21-Nov-2012 25-Jan-2010 31-Dec-2016 19-Oct-2016 Comments (achievements against targets): This target has been achieved, however the Financing and restructuring plan was not made available to the World Bank team. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Page 38 of 59 The World Bank NATIONAL WATER SUPPLY & SANITATION (P096213) IFRS adopted by Azersu Text No Yes Yes Yes 21-Nov-2012 27-Jan-2012 31-Dec-2016 19-Oct-2016 Comments (achievements against targets): This target has been achieved. Unlinked Indicators Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Percentage of drinking water Percentage 40.00 100.00 100.00 100.00 samples in project area meeting Azeri water quality 21-Nov-2012 31-Mar-2008 31-Dec-2016 19-Oct-2016 standards Comments (achievements against targets): This target has been 100% achieved. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Effluent water samples Percentage 0.00 100.00 50.00 0.00 passing water quality tests 21-Nov-2012 27-Jan-2012 31-Dec-2016 19-Oct-2016 Comments (achievements against targets): The indicator was introduced in 2012 and subsequently revised to 50% in 2014. At completion. This target has not been achieved due to delayed construction of the WWTPs. Page 39 of 59 The World Bank NATIONAL WATER SUPPLY & SANITATION (P096213) B. KEY OUTPUTS BY COMPONENT Objective/Outcome 1 To provide quality and reliable water supply and sanitation services. 1. People in project area receiving improved water supply and sanitation services from the project Outcome Indicators 2. Pollution load measured in persons equivalent (PE) eliminated through adequate wastewater treatment 1. Number of rayons in which water supply and wastewater systems in the project area are rehabilitated and operational 2. New piped household water connections that are resulting from the project intervention 3. New household sewer connections constructed under the project 4. Water network rehabilitated Intermediate Results 5. Sewerage network rehabilitated Indicators 6. New reservoir capacity provided 7. Percentage of drinking water samples in project area meeting Azeri water quality standards 8. Effluent water samples passing water quality tests 9. Daily number of hours of water supply service 10. Working Ratio (operating expenditures divided by collected revenues) 1. 98,597 people receiving improved water supply and sanitation service. Out of them 20,978 are newly piped household with water connections and 9,611 with sewer connections (Shamakhi WWTP is fully operational and Gabala still needs to have Key Outputs by Component secondary treatment completed with GoA funding) A: Rayon Investment 2. 575 km water network rehabilitated and 202 km. sewerage network rehabilitated Component 3. 34,400 m3 new reservoir capacity provided 4. 100% of drinking water samples in project area meeting Azeri water quality standards. 5. Increased daily number of hours of water supply service from 4 to 24. Objective/Outcome 2 Institutional modernization Outcome Indicators n.a. 1. Draft Greater Baku Regional Development Plan completed Intermediate Results 2. Updated Absheron Peninsula Water Supply Masterplan completed Indicators 3. Financing and restructuring plan completed and adopted by Azersu 4. IFRS adopted by Azersu Page 40 of 59 The World Bank NATIONAL WATER SUPPLY & SANITATION (P096213) Key Outputs by Component 1. The GBRDP has been publicly discussed and disclosed. It also supported land use and zoning plans that were prepared B and C: GBRDP and 2. Updated Absheron Peninsula Water Supply Master Plan completed Institutional Modernization 3. Training was provided to Azersu staff Component 4. Azersu IFRS-based accounting system Page 41 of 59 The World Bank NATIONAL WATER SUPPLY & SANITATION (P096213) ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION A. TASK TEAM MEMBERS Name Role Preparation Andreas Rohde Task Team Leader Sanyu Lutalo Operations Analyst Christophe Bosch Sr. Water and Sanitation Economist Agnes Kiss Lead Ecologist Hermine de Soto Social Specialist / Consultant Ellen Hamilton Sr. Urban Planner Nijat Valiyev Infrastructure Specialist Karl Skansing Procurement Specialist / Consultant Ida Muhoho Sr. Financial Management Specialist Junko Funahashi Sr. Counsel Hannah Koilpillai Sr. Finance Officer Klas Ringskog Lead Water and Sanitation Specialist/Consultant/Peer Reviewer Stefan Rattensperger Consultant Hadji Huseynov Infrastructure Specialist Manuel Marino Lead Water and Sanitation Specialist/Peer Reviewer Milane Reyes Program Assistant Teresa Lim Program Assistant Supervision/ICR Hadji Huseynov Task Team Leader(s) Salih Bugra Erdurmus, Andres Eduardo Mac Gaul Procurement Specialist(s) Tural Jamalov Financial Management Specialist Anne N. Ranasinghe Team Member Guy Tchakounte Tchabo Team Member Jose C. Janeiro Team Member Gulana Enar Hajiyeva Social Safeguards Specialist Ahmed A. R. Eiweida Team Member Vusala Mamed Asadova Team Member Ida Srbec Team Member Angela Nyawira Khaminwa Social Safeguards Specialist Aleksandre Bibileishvili Team Member Jimena Garrote Counsel Lela Shatirishvili Social Safeguards Specialist Sabina Guliyeva Team Member Lidija Kvrgic Team Member Agayeva Nigar Cingiz Environmental Safeguards Specialist Page 42 of 59 The World Bank NATIONAL WATER SUPPLY & SANITATION (P096213) B. STAFF TIME AND COST Staff Time and Cost Stage of Project Cycle No. of staff weeks US$ (including travel and consultant costs) Preparation FY06 34.873 123,318.28 FY07 62.959 589,646.85 FY08 0 2,182.82 FY16 0 325.45 Total 97.83 715,473.40 Supervision/ICR FY06 0 0.00 FY08 44.207 213,905.87 FY09 40.544 132,967.34 FY10 46.817 213,715.28 FY11 31.547 128,907.85 FY12 24.202 133,421.55 FY13 25.175 187,883.55 FY14 24.142 168,587.66 FY15 14.565 77,107.97 FY16 19.508 89,801.69 FY17 19.649 76,461.64 Total 290.36 1,422,760.40 Page 43 of 59 The World Bank NATIONAL WATER SUPPLY and SANITATION (P096213) ANNEX 3. PROJECT COST BY COMPONENT Amount at Approval Actual at Project Closing Percentage of Approval Components (US$, Millions) (US$, Millions) (%) Component A: Rayon Investment 200.00 174.58 87% Component Component B: Regional Development Plan for Greater Baku 3.75 4.32 115% Component Component C: Institutional 9.50 12.36 130% Modernization Component Component D: Project Implementation and Management 1.50 2.62 175% Component Price Contingencies 15.25 13.70 90% Total 230.00 207.68 90% Page 44 of 59 The World Bank NATIONAL WATER SUPPLY and SANITATION (P096213) ANNEX 4. EFFICIENCY ANALYSIS Introduction 1. The PDO was to provide quality and reliable water supply and sanitation services in selected regional (rayon) centers of Azerbaijan. The primary benefits expected from the project were (a) economic gains, resulting from sustaining or increasing water consumption levels, (b) reduction of coping strategies undertaken by households to mitigate water shortages and/or lack of access to water supplies and lack of adequate sanitation facilities, (c) positive impacts on public health, from sustaining or increasing current water consumption levels, and improving the quality of water supplies, (d) external effects (for instance, on the environment), and (e) macroeconomic impacts. 2. The purpose of this annex is essentially to (a) provide an assessment of Azersu’s financial situation during the project, examining the company’s financial capacity and whether it reached financial viability and (b) assess the financial impact associated with the investments in the 4 rayons covered under the project and compared with the initially planned 20 rayons. Methodology 3. As part of project preparation, a financial model was developed, which shows the financial projections of Azersu on a consolidated basis and the financials of the 20 rayons where investment was supposed to take place, covering the 20-year period of the term loan to the Government. For this ICR the same approach was followed, including the steps detailed in the following paragraphs. 4. A cost-benefit methodology was used to calculate the economic rate of return and NPV of the project’s water supply investments. The economic analysis is linked to the financial analysis calculating the incremental financial flows to Azersu from the project. With and without project scenarios are defined to identify the incremental costs and benefits of the project. Azersu Group: Summary Financial Assessment as of December 31, 2016 5. The following section analyses Azersu’s recent financial performance and key challenges, its ability to operate as a solvent and a financially viable utility group during and following the project period, and its ability to service any financing obligations related to the project. 6. The Azersu Group is actively involved in investing activities covering the construction of new water pipelines and sewerage systems in the Republic of Azerbaijan and receives subsidies from the Government on a regular basis. Subsides are the Ministry of Finance’s decision, however, the calculation is coordinated with the Ministry of Economy. New investments are fully covered by the GoA. There is a specific commission of five ministries (led by Deputy Prime Minister) that analyze the cost and revenue of 14 institutions (including Azersu), which decides, on a regular basis, what the subsidy should be. There is also an option for ad hoc requests in extraordinary situations. Historically the group has had operating losses, negative cash flows from operations, and working capital deficit. However, the subsidization process gives, to some extent, certainty that this company will continue its operation. Page 45 of 59 The World Bank NATIONAL WATER SUPPLY and SANITATION (P096213) Table 4.1. Azersu Group - Profit and Loss Account (AZN) 2012 2013 2014 2015 2016 Sales revenue 122,896,791 134,857,363 140,979,000 139,086,000 159,837,000 Sale of water 112,939,128 110,303,563 119,162,000 117,542,000 115,545,000 Sewerage service 3,146,051 17,154,754 17,730,000 19,645,000 41,183,000 Other sales revenue 6,811,612 7,399,046 4,087,000 1,899,000 3,109,000 Cost of sales ‒105,753,543 ‒93,253,890 ‒87,991,000 ‒77,220,000 ‒84,578,000 Gross Revenues 17,143,248 41,603,473 52,988,000 61,866,000 75,259,000 Administrative expenses ‒31,652,611 ‒30,655,935 ‒29,875,000 ‒31,169,000 ‒19,883,000 Selling and distribution ‒91,854,266 ‒74,547,050 ‒94,912,000 ‒89,420,000 ‒89,938,000 expenses Other operating expenses ‒88,314,161 ‒987,521,327 ‒837,190,000 ‒357,646,000 ‒543,282,000 Foreign exchange losses ‒140,440,000 ‒39,800,000 Other income 0 17,323,325 23,338,000 9,761,000 5,840,000 Operating expenses ‒511,821,038 ‒1,075,400,987 ‒938,639,000 ‒608,914,000 ‒687,063,000 Operating result (EBIT) ‒494,677,790 ‒1,033,797,514 ‒885,651,000 ‒547,048,000 ‒611,804,000 Finance cost ‒309,009 ‒725,956 ‒370,000 ‒2,815,000 ‒840,000 Loss before tax ‒494,986,799 ‒1,034,523,470 ‒886,021,000 ‒549,863,000 ‒612,644,000 Income tax ‒282,737 61,770 ‒11,000 ‒797,000 ‒277,000 benefit/(expense) Loss for the year ‒495,269,536 ‒1,034,461,700 ‒886,032,000 ‒550,660,000 ‒612,921,000 Net result/sales ‒403% ‒680% ‒539% ‒370% ‒370% (including other income) Note: EBIT = Earnings before interest and taxes. 7. According to the independent auditor’s reports from 2012–16. The going concern basis assumes that the Azersu Group will continue its operations for the foreseeable future. However, the group incurred a net loss of AZN 612,921 thousand during the year ended December 31, 2016 (December 31, 2015 restated: AZN 550,660 thousand). Moreover, the group incurred a net loss of AZN 1,034,461,700 during the year ended December 31, 2013. This is almost six times larger than the revenue. These conditions give rise to a material uncertainty which may cast significant doubt about the group's ability to continue as a going concern. However, the management believes that appropriate measures are being taken for the group to continue its operational existence in the foreseeable future. Accordingly, the group applied the going concern basis in preparing its consolidated financial statements. Azersu Group: Financial Assessment Before and After The Project 8. The following section analyses Azersu’s recent financial performance compared to before the project and forecasted during the project financial performance. A set of terms, conditions, and agreed project activities were used to construct a financial model and forecast for a 20-year horizon during project appraisal. The financial model was built as a tool to evaluate the options for financial recovery of Azersu. Following the same approach—recent financial data were adjusted to be compatible with the Page 46 of 59 The World Bank NATIONAL WATER SUPPLY and SANITATION (P096213) original assessment. Given the large number of assumptions underlying the financial forecasts, it is important to note that the financial analysis is speculative. So far as there are a number of data gaps, the financial audits were heavily qualified and only in 2015–16 were the audit reports unqualified. 9. The following section outlines critical financial assumptions regarding the revenue and cost structure of Azersu based on the projected capital investments. Azersu’s financial position is compared, based on consultations with the Azersu management, under a set of assumptions about the parameters pertaining to income and expenses. There is a significant portion of impairment of nonfinancial assets, which in some years, like 2013, was AZN 988 million, while the total revenue is only AZN 123 million. As this was part of the capital assets revaluation during introduction of the IFRS and was not connected with the actual operational expenditures in the specific year, it is excluded from the assessment. Similarly, the foreign exchange losses are excluded as they are an external factor to Azersu and related mostly to the ability to service foreign debt and were not directly connected with operational cost recovery, which is the important performance improvement target. Similar to the analysis at appraisal, all projections are in real terms (constant prices based at 2016). 10. According to the actual data 2003–05, tariff levels were insufficient to cover operating expenses. Figure 4.1. Working Ratio 1.5 1.4 1.3 1.2 1.1 1 0.9 0.8 0.7 0.6 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 Actual Forecast Actual revised OPEX covered by the revenues 11. Despite notable improvements since 2005, Azersu was unable to meet its operating costs and cash flow requirements, let alone provide a sufficient return on invested capital to allow for future reinvestments, expansion of systems, and improvement of service quality. Revenues were forecasted to increase in the period 2006–11 and the working ratio was estimated to be between 0.70 and 0.80 implying that the operational cost would be fully covered by the revenue and that resources would be available for capital expenditures. Page 47 of 59 The World Bank NATIONAL WATER SUPPLY and SANITATION (P096213) Figure 4.2. Earnings Before Interest, Taxes, Depreciation, and Amortization (EBITDA 30 Millions AZN 20 10 0 -10 -20 -30 -40 -50 -60 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 Actual Forecast Actual revised OPEX covered by the revenues Figure 4.3. Net Result/Sales 0% -10% -20% -30% -40% -50% -60% -70% -80% -90% -100% 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 Actual Forecast Actual revised 12. However, analyzing the actual (adjusted) data received from the audited financial statement, the group continues to fall short of producing sustainable working ratios and is hardly able to cover its O&M cost. The EBITDA has been negative over the last years to briefly turn out a positive result in 2013. However, accounting for amortization/depreciation the group incurred a negative operating result. The EBITDA is again negative in 2014–16 with an operating ration above 1.0. This implies significant capital cash shortages and precludes critical future investments. The operating margin stands at a problematic – 33.0 percent in 2015 and –17.4 percent in 2016. 13. In addition, at the current levels of depreciation of existing assets and new investments planned, the net result, remains hugely negative even at the end of the project. The key financial indicators are shown in table 4.2 (current prices). Page 48 of 59 The World Bank NATIONAL WATER SUPPLY and SANITATION (P096213) Table 4.2. Azersu Group - Preliminary Profit and Loss Account (AZN) 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 Actual Actual Actual Forecast Forecast Forecast Forecast Forecast Forecast Actual Revised Actual Revised Actual Revised Actual Revised Actual Revised Sales revenue 28,000,678 31,006,102 41,374,158 44,595,586 82,425,662 85,692,702 91,740,117 97,038,919 102,717,794 122,896,791 152,180,688 164,317,000 148,847,000 165,677,000 Sales of water 112,939,128 110,303,563 119,162,000 117,542,000 115,545,000 Sewerage service 3,146,051 17,154,754 17,730,000 19,645,000 41,183,000 Other sales revenue 6,811,612 7,399,046 4,087,000 1,899,000 3,109,000 Cost of sales ‒6,088,983 ‒5,958,644 ‒5,958,644 ‒7,812,568 ‒9,377,463 ‒9,377,463 ‒10,878,463 ‒10,878,463 ‒10,878,463 ‒82,064,131 ‒64,908,092 ‒81,698,000 ‒74,232,000 ‒73,925,000 Other income 549,661 995,085 995,085 995,085 995,085 995,085 995,085 995,085 995,085 0 17,323,325 23,338,000 9,761,000 5,840,000 Gross Revenues 21,911,695 25,047,458 35,415,514 36,783,018 73,048,199 76,315,239 80,861,654 86,160,456 91,839,331 40,832,660 87,272,596 82,619,000 74,615,000 91,752,000 Administrative expenses ‒22,443,505 ‒26,336,200 ‒27,822,000 ‒30,314,000 ‒17,978,000 Selling and distribution ‒77,681,873 ‒39,546,529 ‒91,229,000 ‒81,071,000 ‒83,596,000 expenses Other operating expenses ‒629,821 0 ‒1,019,000 ‒119,000 ‒70,000 Foreign exchange losses 0 0 0 0 0 Operating expenses ‒27,175,254 ‒30,545,763 ‒32,666,341 ‒38,223,620 ‒52,156,061 ‒54,160,533 ‒62,708,712 ‒64,258,918 ‒66,411,279 ‒100,755,199 ‒65,882,729 ‒120,070,000 ‒111,504,000 ‒101,644,000 EBITDA ‒5,263,559 ‒5,498,305 2,749,173 ‒1,440,602 20,892,138 22,154,706 18,152,942 21,901,538 25,428,052 ‒59,922,539 21,389,867 ‒37,451,000 ‒36,889,000 ‒9,892,000 Depreciation 6,145,000 6,969,000 10,357,060 10,822,693 ‒47,070,911 ‒29,506,082 ‒12,029,000 ‒12,192,000 ‒18,900,000 Operating result (EBIT) ‒11,408,559 ‒12,467,305 ‒7,607,887 ‒12,263,295 4,465,888 459,748 ‒9,546,039 ‒10,390,061 ‒9,253,087 ‒106,993,450 ‒8,116,215 ‒49,480,000 ‒49,081,000 ‒28,792,000 Finance cost ‒4,560,000 ‒6,312,400 ‒6,312,400 ‒6,048,794 ‒309,009 ‒725,956 ‒370,000 ‒2,815,000 ‒840,000 Income tax benefit/(expense) 17,600 197,200 223,400 223,400 ‒282,737 61,770 ‒11,000 ‒797,000 ‒277,000 Net Result ‒15,950,959 ‒18,582,505 ‒13,696,887 ‒18,088,689 ‒2,250,814 ‒4,989,595 ‒14,823,823 ‒15,521,883 ‒14,290,841 ‒107,585,196 ‒8,780,401 ‒49,861,000 ‒52,693,000 ‒29,909,000 PROFITABILITY EBITDA/Sales ‒18.80% ‒17.73 6.64 ‒3.23 25.35 25.85 19.79 22.57 24.76 ‒48.76 14.06 ‒22.79 ‒24.78 ‒5.97 Operating Margin (%) ‒40.74 ‒40.21 ‒18.39 ‒27.50 5.42 0.54 ‒10.41 ‒10.71 ‒9.01 ‒87.06 ‒5.33 ‒30.11 ‒32.97 ‒17.38 Net Result/Sales (%) ‒6.97 ‒59.93 ‒33.10 ‒40.56 ‒2.73 ‒5.82 ‒16.16 ‒16.00 ‒13.91 ‒7.5 ‒5.77 -30.34 ‒35.40 ‒18.05 Working ratio 1.19 1.18 0.93 1.03 0.75 0.74 0.80 0.77 0.75 1.49 0.86 1.23 1.25 1.06 Page 49 of 59 The World Bank NATIONAL WATER SUPPLY and SANITATION (P096213) Water Tariff 14. The extent to which the water utilities and the Tariff Council are pursuing a cost recovery policy depends on various factors like purchase power and affordability. The proposed rehabilitation of infrastructure under the project helped in this respect because it is designed to minimize operation costs. The utilities were also provided with tools for increasing collection rates with the assistance of the computerized billing and collection system installed under the project. Other than increased collection rates, significant tariff increases were among the most important factors contributing to reforming the financials of Azersu. A more or less doubling of tariffs for water and wastewater services since January 8, 2007, was a significant step in the right direction. Tariff increase was an important step toward achieving financial viability in the rayon water utilities. On May 13, 2016, the Tariff Council of the Republic of Azerbaijan approved the increase of water prices and wastewater treatment tariffs charged to customers. Eventually this will lead to enhance the group's financial position upon completion of the capital projects, which are currently under construction. Table 4.3 summarizes the 2008 tariff per cubic meter applied to each group of customers outside Greater Baku and tariff increases during the project and the real increase of constant prices as of 2017 (adjusted with gross domestic product deflator). Table 4.3. Water Tariffs Current Prices 2017 Constant Prices Increase 2008 2011 2016 Increase 2008 2011 2016 2017 (AZN/m3) (%) Tariff for Water supply Residential 0.14 0.25 0.30 214% 0.19 0.31 0.34 0.30 154% Nonresidential 0.70 1.00 1.00 143% 0.97 1.23 1.15 1.00 103% Tariff for sanitation Residential 0.03 0.06 0.15 500% 0.04 0.07 0.17 0.15 360% Nonresidential 0.20 0.30 1.00 500% 0.28 0.37 1.15 1.00 360% 15. The tariff change shows real increase in residential water supply and even higher for the sanitation. Only the nonresidential water tariff is practically not increased. An additional advantage is the different pricing between Greater Baku, where the affordability is significantly higher, and the other areas which are mostly rural with lower purchase power of the population. Although, with increased tariffs, the group continues to remain dependent on its ability to obtain sufficient funding from the Government to sustain operations and complete its current investment projects. There were initial plans for technical assistance to the Tariff Council. However, there was no follow-up support or contacts, which is important for the sustainability of the WSS sector institution. 16. The GoA and, particularly, the Ministry of Economy are working on a social program for the poor under the Roadmap 2020. It is considered to be developed for vulnerable people when tariffs will be further increased. By the end of 2018, the program is expected to be ready. Studies to research appropriate mechanisms have been done and the request for a technical assistance grant was sent to the World Bank. On January 31, 2018, the World Bank confirmed support for this process and shared a preliminary report on Albania and Bulgaria in the WSS system. This social protection mechanism will provide an opportunity for further increase of tariff and the level of cost recovery without affecting vulnerable people. Page 50 of 59 The World Bank NATIONAL WATER SUPPLY and SANITATION (P096213) World Bank-financed Project (Four Rayons) Summary of Benefits and Costs 17. According to the appraisal methodology, a simple quantitative version of the economic analysis was developed, and the only benefits that were measured in monetary terms were (a) user benefits with reduced coping costs and (b) system benefits through energy-efficiency gains. In addition to the appraisal quantitative analysis, the benefits from sanitation improvement were also quantified. The lack of data, methodological issues, and comparability prevent a quantitative evaluation of other impacts discussed in this section. The project’s main impacts and benefits include (i) Economic gains, resulting from sustaining or increasing current water consumption levels; (ii) Reduction of coping strategies undertaken by households to mitigate water shortages and/or lack of access to water supplies, and lack of adequate sanitation facilities; (iii) Positive impacts on public health, from sustaining or increasing current water consumption levels, and improving the quality of water supplies; (iv) External effects (for instance, on the environment); and (v) Macroeconomic impacts. 18. Table 4.4 describes in a summary presentation the key benefits of the project, in economic and financial terms. Table 4.4. Expected Project Outcomes and Benefits Project Outcomes Benefits Economic Financial Reduce impact of poor WSS In general, low-income households would suffer disproportionately from service on low-income increasing water shortages and unreliable supplies without the project. They households. cannot afford coping arrangements that richer households can pay for (storage tanks, pumps, and tanker supply). Sustained and improved service will have a greater impact on health and the quality of life of low-income households. Reduce water losses by 1. Inputs savings: energy, chemicals, and other variable (short-run) costs rehabilitating deteriorated 2. Avoided costs of expanding or rehabilitating water production and networks as well as improving conveyance facilities (long-run costs) network design and operation. 3. Load reduction of wastewater collection and treatment systems (but organic Reduce indoor leakage and load per capita expected to remain constant) wastage through demand 4. Avoided damages to private and public property (damages to apartments management (metering, due to water leaks, damages to roads and foundations). Increase in property pricing). values. Improve water service 5. Avoided mitigating/coping expenditures (fetching water, purchasing water reliability (continuity of tanks or booster pumps, and so on) service). 6. Increased willingness-to-pay for water Page 51 of 59 The World Bank NATIONAL WATER SUPPLY and SANITATION (P096213) Project Outcomes Benefits Economic Financial Improve water quality by 7. Reduction of risk of water-related diseases rehabilitating key elements of 8. Avoided private mitigating expenditures (for example, filtering, boiling the existing water treatment water, purchasing bottled water) installations. 9. Increased willingness-to-pay for water Network optimization Energy savings, avoided damages to system (for example, water hammer) Prevention of further Savings in asset renewal deterioration of systems Improvement of environmental Protection of surface water functions and services (recreation, fish production, conditions, particularly ambient ecosystems, and water supplies) water quality Tariff adjustments, in line with Increased financial revenues and financial cash flow equilibrium objective autonomy Increase of billing and Increased financial revenues and financial collection rates, reduction of autonomy arrears Skills upgrading and increasing Reduced financial costs, including productivity efficiency gains Improved managerial, Private sector involvement (through twinning or service contracts) is expected commercial, and financial to induce a higher level of commitment and private financing could be situation of the utility leveraged. 1. The earlier analysis means that substantial investments in WSS would be difficult to justify in economic terms if only private benefits and efficiency are taken into account. There are public benefits (for the community and the environment) which need to be carefully assessed from a qualitative standpoint given the challenges associated with quantitative analysis of the WSS benefits. Summary Financial Assessment of the Four Rayons Before and After the Project 2. The current situation at the four regions—Saatli, Sabirabad, Shamakhi, and Gabala—has continued to generate annual cash shortages. Despite notable improvements in Gabala and Saatly in 2017, the group is still unable to meet its operating costs, let alone provide a sufficient return on invested capital to allow for future reinvestments, expansion of systems, and improvement of service quality. In total, the four rayons continue to fall short of producing sustainable working ratios and are hardly able to cover their O&M costs. The EBITDA has been negative over the last year and the negative EBITDA ratio versus sales is even larger than before the project. Table 4.5. Four Project Regions—Saatli, Sabirabad, Shamakhi, and Gabala AZN 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 Total 690,462 712,749 662,272 710,091 665,244 750,406 783,485 799,318 924,153 1,190,069 revenues O&M 1,049,489 1,185,072 1,552,395 1,797,161 1,894,940 1,722,041 1,793,100 3,484,215 4,067,613 3,084,843 ‒ ‒ ‒ ‒ ‒ ‒ EBITDA ‒359,027 ‒472,323 ‒890,123 ‒971,635 1,087,071 1,229,696 1,009,615 2,684,896 3,143,460 1,894,774 EBITDA/ Sales 52% ‒66% ‒134% ‒153% ‒185% ‒129% ‒129% ‒336% ‒340% ‒159% Page 52 of 59 The World Bank NATIONAL WATER SUPPLY and SANITATION (P096213) Figure 4.4. Working Ratio - Gabala 2.00 1.50 1.00 0.50 0.00 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2026 Actual At appraisal OPEX covered by the revenues Figure 4.5. Working Ratio - Saatli 4.00 3.00 2.00 1.00 0.00 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2026 Actual At appraisal OPEX covered by the revenues Figure 4.6. Working Ratio - Sabirabad 4.00 3.00 2.00 1.00 0.00 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2026 Actual At appraisal OPEX covered by the revenues Page 53 of 59 The World Bank NATIONAL WATER SUPPLY and SANITATION (P096213) Figure 4.7. Working Ratio - Shamakhi 5.00 4.00 3.00 2.00 1.00 0.00 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2026 Actual OPEX covered by the revenues 3. The above working ratios show the actual status from 2008 up to and including year 2017 compared with the estimate at appraisal which goes up to 2026. Economic CBA for Water Supply Investments 4. The economic CBA relies on two main sources of quantitative information: (i) For user benefits (ii) For system benefits User Benefits: Coping Costs for Water Supply 5. Both residential and nonresidential users incur a variety of expenditures to cope with poor service. Virtually everyone suffers from the lack of continuity of water services and insufficient pressure in the rayons and, in several cases, from a complete breakdown of systems. A significant share of the population though connected to the network, hardly receive any water. The coping expenditures are regressive and disproportionately affect the poor. Indeed, they are often equally expensive in absolute terms regardless of income and are not subsidized. 6. According to theory, there should be a significant overlap between WTP and avoided coping costs, as households when stating their WTP for improved service would anticipate a corresponding reduction in costs they are forced to incur to cope with poor service. However, given that only a fraction of the coping costs is actually quantified (health costs in particular are not monetized), the potential double- counting is limited. Based on this reasoning and data from appraisal, the total coping costs that would be avoided because of the project are estimated at AZN 2.5 per household per month. The coping cost is adjusted accordingly with household average income increase by the end of the project. User Benefits: Coping Costs for Sanitation 7. A few baseline indicators on sanitation were estimated from the household survey done at project appraisal. Benefits from sanitation improvement were not quantified at appraisal, but due to the Page 54 of 59 The World Bank NATIONAL WATER SUPPLY and SANITATION (P096213) importance of this benefit it was included in the cost benefit at the end of the project. In particular, households are willing to pay a premium for a sewer connection in relation to the cost they are incurring for on-site sanitation systems. The estimated benefits for emptying the cesspit were adjusted according to the income increase and used to quantify the sanitation benefits. There may be public benefits (for the community and the environment), which are assessed from a qualitative standpoint given the challenges associated with quantitative analysis of sanitation benefits. System Benefits: Energy Efficiency Data from Technical Studies 8. During appraisal, it has been estimated that in rayons the annual pumping costs can be reduced by about 45–50 percent over the original system, based on greater economies in pumping gained from (a) lower quantities of water to be pumped and (b) lower transmission losses. In addition, due to poor operation and maintenance, the efficiency of most pumps is very low. 9. In summary, the energy efficiency of pumped systems would be increased about 45 percent due to: • More accurate gauging of system water demand (less raw water extraction and production) • Lower pumping requirements 10. Given that not all systems are pumped, it is conservatively assumed that the overall increase in energy efficiency will be about 30 percent, equivalent to a reduction of 0.30 kWh per m3 of water produced. The economic cost of electricity in the rayons is estimated at AZN 0.20 per kWh. Note that the economic analysis is using the 2017 nonresidential electricity price of AZN 0.09 per kWh. Results of the Analysis 11. Based on the above assumptions and other baseline indicators already used in the financial analysis, a project NPV have been estimated and compared with the analysis as appraisal. Table 4.6. Summary of CBA Present Value of Flows At appraisal At completion NPV (AZN, million) 6.76 −100.4 Economic Internal Rate of Return 10.4% n.a. a Note: a. The NPV is highly negative and it is not possible to calculate the internal rate of return. 12. The economic flows (incremental costs and benefits) over the period of the economic analysis are shown in figure 4.8. Page 55 of 59 The World Bank NATIONAL WATER SUPPLY and SANITATION (P096213) Figure 4. 8. Economic Flows from the Project’s Water Supply Investments (AZN constant price 2016) 10 Millions AZN 0 -10 -20 -30 -40 -50 -60 -70 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 2021 2022 2023 2024 2025 2026 WSSCs result (EBITDA) Incremental Project cost Incremental Project benefits Financial Assessment and Required Terms, Conditions, and Assumptions 13. The assumptions used to forecast the financial performance in the 4 rayons covered under the investment component of the World Bank-financed project are compared with the initially planned 20 rayon’s investment. Table 4.7. Project’s 20 Rayons at Appraisal versus Actual 4 Rayons At Appraisal (20 rayons) Actual Project (4 rayons) 2007 2013 2008 2017 Total Production (m3 per year, thousands) 52,676 42,869 4,297 4,722 Total Consumption (m3 per year, thousands) 21,070 40,726 3,529 2,714 Population 484,783 695,010 103,315 145,642 Service of population coverage ratio (%) 56% 100% 52% 61% Population served 271,967 695,010 53,429 89,130 Metering - Coverage of total population (%) 24% 100% 0% 54% Page 56 of 59 The World Bank NATIONAL WATER SUPPLY and SANITATION (P096213) Table 4.8.Investment per capita At appraisal Actual project Ratio Other IFIs Population served 695,010 89,130 13% Including new connections 423,043 35,701 8% Investment cost US$269,000,000 US$207,000,000 77% Investment per capita US$387 US$2,322 600% US$1,200 to $3,288 14. The negative NPV of the project is also a result of the highly reduced scope (12 times less population with new connections and 8 times less population with improved service) and at the same time almost the same amount of investment. This is visible from the investment per capita which is now US$2,322 instead of US$387 at appraisal. This estimate is based on the total project cost, which includes institutional component and other services not directly related to the WSS investment sites. This would to some extend reduce the actual price per-capita. However, for users with new connections the investment per capita is even higher. This scale of investment per capita compared with even more extensive quantitative assessment of the benefits is not realistic to bring positive NPV within infrastructure lifespan. 15. Azersu has also requested changes to the technical designs, including treatment in conformity with the EU Wastewater Treatment Directive for secondary treatment. The feasibility studies used were not of the highest quality and underestimated the investment cost. The changes proposed by Azersu generally imply higher cost for current lump-sum contracts and increase in the respective investment per capita. 16. Following consultation with other IFIs which support the WSS sector in Azerbaijan, it was noted that the changes of specifications also led to increase of investment per capita to a range of US$1,200 to US$3,288 per capita. It was found that the large difference of investment per capita depends on inclusion of sanitation and WWTPs. The projects which are focused only on water supply have significantly lower costs than the others which have sanitation and WWTP according to EU standards. In comparison with these projects, the NWSSP has per capita investment which is even lower than the other IFIs. Also, the SNWSSP has an average investment per capita ranging from US$1700–US$2400. Page 57 of 59 The World Bank NATIONAL WATER SUPPLY and SANITATION (P096213) ANNEX 5. BORROWER COMMENTS 1. The Borrower had the opportunity to review and comment on the draft ICR. Overall, the Borrower agreed in their assessment that the World Bank was very supportive of the Project implementation. 2. The other specific points raised are related to: i) deviation between the number of household connections estimated at appraisal and actual number achieved; and ii) the actual infrastructure investment per capita should be less than the total project cost divided by the number of beneficiaries – due to the institutional component. 3. Bank’s response: The above comments have minor effects on the assessment presented in the ICR and doesn’t affect comparability between estimates at appraisal and at the end of the project. This has been clarified in the ICR. Page 58 of 59 The World Bank NATIONAL WATER SUPPLY and SANITATION (P096213) ANNEX 6. SUPPORTING DOCUMENTS 1. Project Appraisal Document (PAD), dated May 16, 2007. 2. Legal Agreement, dated August 17, 2007. 3. Project Agreements 4. Implementation Supervision Reports (numbers 1 – 21) 5. World Bank Supervision Aide Memoires 6. Mach 2009 Restructuring Paper 7. October 2011 Restructuring Paper 8. February 2013 Restructuring Paper 9. December 2014 Restructuring Paper 10. July 2016 Restructuring Paper 11. Azerbaijan - Country partnership strategy FY07-10 12. Country Partnership Framework for Azerbaijan for the period FY16-FY20 13. Azerbaijan Water Sector Strategy for 2006–2015 14. Azersu Financial Statements with Independent Auditors’ Report (2012-2016) 15. Borrowers Project Completion Report, 2016 16. Implementation Completion and Results Report (ICR) for Investment Project Financing (IPF) Operations, July 5, 2017 Page 59 of 59