Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Sarajevo Waste Water (for. Mun. Dev.) (P090675) Report Number : ICRR0021017 1. Project Data Project ID Project Name P090675 Sarajevo Waste Water (for. Mun. Dev.) Country Practice Area(Lead) Bosnia and Herzegovina Water L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IBRD-78420,TF-12937,TF-A1998,TF_1- 30-Nov-2015 37,000,000.00 12937,TF_2-A1998 Bank Approval Date Closing Date (Actual) 22-Dec-2009 31-May-2017 IBRD/IDA (USD) Grants (USD) Original Commitment 35,000,000.00 11,191,363.02 Revised Commitment 30,609,537.96 11,191,344.73 Actual 24,793,430.27 14,148,367.62 Prepared by Reviewed by ICR Review Coordinator Group Katharina Ferl Christopher David Christopher David Nelson IEGSD (Unit 4) Nelson 2. Project Objectives and Components a. Objectives According to the Project Appraisal Document (PAD) (p.3) and the Loan Agreement of March 17, 2010 (p. 8) the objective of the project was to “improve the living conditions of populations in the areas covered by the Sarajevo Water and Waste Water Company and in downstream riverside communities by: a) reducing the population’s exposure to, and reliance on highly polluted water from Milijacka and Bosna rivers; and b) improving the efficiency of the waste water collection network in the Sarajevo Canton.” Page 1 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Sarajevo Waste Water (for. Mun. Dev.) (P090675) b. Were the project objectives/key associated outcome targets revised during implementation? No PHEVALUNDERTAKENLBL c. Will a split evaluation be undertaken? No d. Components The project included three components: Component A: High Priority Infrastructure Rehabilitation (appraisal estimate US$36 million, of which Bank financing US$34 million and the Sarajevo Canton Government US$2 million; additional financing by the European Commission Instrument for Pre-Accession Assistance (IPA) of US$2 million, actual US$38 million): This component was to finance priority investments in the rehabilitation of waste water infrastructure in the Sarajevo Canton including repair and replacement of primary and secondary sewers in Sarajevo, and rehabilitation of the Sarajevo waste water treatment plant (WWTP). Component B: Institutional Development Support (appraisal estimate US$0.5 million, all financed by the Bank, actual US$0.5 million): This component was to finance technical assistance to build capacity and to strengthen the Sarajevo Water and Waste Water Company (ViK). Activities included were institutional strengthening for improving financial management capacity and operational efficiency to enhance long-term financial viability and supporting the preparation of feasibility studies, technical designs, and other technical studies for follow-up investments in water and waste rehabilitation. Component C: Project Management (appraisal estimate US$0.5 million, all financed by the Bank, actual US$0.5 million): This component was to finance the support of the Project Management Team (PMT) and the Project Implementation Team (PIT) through training of staff on financial management and procurement, project monitoring and evaluation, operating costs, office equipment, financial audits, and other incremental costs. e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project Cost: The project was estimated to cost US$37 million. Actual cost was US$39 million. Financing: The project was financed by a US$35 million IBRD loan, and European Commission Instrument for Pre-Accession Assistance Grants of US$11.7 million. On September 30, 2017 2.9 million Euros was cancelled due to savings of project funds and the substitution of the loan amounts with a grant by the European Commission to upgrade and refurbish the Sarajevo WWTP. Borrower Contribution: The borrower was to contribute US$2 million to the project which materialized. Dates: • On November 4, 2015 the project was restructured to: i) extend the closing date by 18 months from November 30, 2015 to May 31, 2017 in order to allow for the completion of the main contract for the Sarajevo WWTP rehabilitation and ii) reallocate funds within Component A from Category 1 (goods and works) to Category 2 (consultant services) to cover increased costs for the supervision of construction works. This did not necessitate a split evaluation. Page 2 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Sarajevo Waste Water (for. Mun. Dev.) (P090675) 3. Relevance of Objectives Rationale The objective of the project was highly relevant given Bosnia and Herzegovina’s challenges in basic service delivery due to the 1992-1995 conflict. The Sarajevo Waste Water Treatment Plant (WWTP) was severely damaged. The lack of proper waste water treatment resulted in the polluting of the Milijacka and Bosna rivers and had a negative impact on the living conditions of almost 200,000 people living downstream of the Sarajevo WWTP. By 2008, almost 45,000 households lacked a connection to a wastewater collection network and instead used pit-latrines, cesspools and other waste water solutions resulting in the pollution of potable water. In addition, Sarajevo ViK had not increased its tariffs since 1997 and its tariff system was based on population occupancy and consumption without metering-based charges or wastewater treatment fees, resulting in a financial deficit of the company. Given the situation of the wastewater sector, the government aimed to rehabilitate damaged wastewater infrastructure. At the time of appraisal the project’s objective was aligned with the second pillar of the Bank’s Country Partnership Strategy (2008-2011) which aimed to improve the quality of government spending and public service delivery for the vulnerable by enhancing the delivery of municipal services and strengthening municipal finance. At project closing the objective of the project remained in line with the Bank’s most recent Country Partnership Strategy (FY2016-FY2020). In particular, with the third pillar which focuses on environmental sustainability and aims to ensure a sustainable use of natural resources, such as water and forestry. In addition, the objective of the project was in line with the environmental requirements of the European Union including the availability of wastewater treatment facilities for all settlements with a population equivalent of 10,000 or above. Rating High 4. Achievement of Objectives (Efficacy) PHEFFICACYTBL Objective 1 Objective Improve the living conditions of populations in the areas covered by the Sarajevo Water and Waste Water Company and in downstream riverside communities by reducing the population’s exposure to, and reliance on highly polluted water from the Milijacka and Bosna rivers: Rationale The project’s Theory of Change linked an increase in connections to wastewater mains and improvements to the WWTP as the basis for how lives will be improved in the targeted communities. This aligns with the Page 3 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Sarajevo Waste Water (for. Mun. Dev.) (P090675) project’s achievement in the following way: Outputs: • 48,000 sewer connections benefited from rehabilitation works surpassing the target of 32,000 sewer connections. • 26.5 kilometers of sewage collectors were repaired and/or rehabilitated, surpassing the target of 16 kilometers. • All WWTP rehabilitation works such as the installation of the nutrient removal process, rehabilitation of a sludge digester and the installation of a combined heat and power unit, were completed and the plant was transferred to the client in October 2017, meeting the target. The EC IPA Trust Fund financed the following outputs: • A preliminary gravitation settlement tank with associated electromechanical equipment was constructed. • The following was reconstructed: raw water pump station, coarse and fine grid with associated equipment, aerated grit chamber with grease trap and associated equipment, primary settling facilities, primary sludge pump station, primary sludge thickeners, primary thickened sludge pump station facilities, electrical substation facilities, and blowing station facilities. The main laboratory was refurbished and equipped. Outcomes: • River pollution was reduced and BOD5 7,066 tons per year through processing at the WWTP. All wastewater collected is treated to a tertiary level at the Sarajevo WWTP, achieving the designated target. • 300,000 people were provided with access to improved sanitation services and improved sanitation facilities, achieving the target. However, it is not clear how this figure was calculated since the theory of change on page 7 of the ICR linked this figure to 48,000 beneficiary households. The ICR (p. 14) stated that in focus group meetings beneficiaries in Butlia, a typical settlement within the Sarajevo ViK operation mandate, stated that convenience and water consumption has increased in addition to an increase in the properties’ values and influx of small businesses. There has not been any flooding from filled latrines and the project helped to reduce the threat of pollution of the water table for the Sarajevo ViK. Rating Substantial PHREVDELTBL PHEFFICACYTBL Objective 2 Objective Improve the efficiency of the waste water collection network in the Sarajevo Canton: Rationale The project’s Theory of Change linked an improvement in efficiency to the waste water collection network in the Sarajevo Canton to improving the living conditions of populations in the areas covered by the Sarajevo ViK. The original activities that were to be implemented under this objective were replaced with procurement of instruments needed for an efficient operation of the water network and other instruments. However, since a loan by the European Bank for Reconstruction and Development (EBRD) also aimed at the financial and institutional strengthening of the Sarajevo Vik, the Bank agreed to drop its original activities. However, the Page 4 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Sarajevo Waste Water (for. Mun. Dev.) (P090675) implementation of the EBRD’s activities only started in 2017, a few months before the project closed. Therefore, it is not clear to what extent the outcomes can be attributed to the project’s activities. Outputs: • Information technology infrastructure in the Sarajevo ViK was reconstructed including developing a wide area network, and an integrated information system and internet application server. In addition, consulting services for the development of the information system on the oracle platform was provided. • A geographic information system (GIS) was developed and maintained. • Leak detection instruments were purchased. • Two studies on sludge utilization were completed. Outcomes: • The amount of sewer blockages decreased from 3,300 in 2009 to 2,800 in 2017, surpassing the target of 2,900. • Revenues of the Sarajevo ViK increased from 100% in 2009 to 117% in 2017, surpassing the target of 110%. • The Volume of waste water collected treated at primary level was 160,000 m3/day. The target was set at 200,000 m3/day, however, the reserved capacity is needed for future city development and in case of emergency flooding within Sarajevo ViK areas of operations. Rating Substantial PHREVDELTBL PHEFFICACYTBL Objective 3 Objective Improve the living conditions of populations in downstream riverside communities Rationale The Results Framework did not include any indicators to measure the achievement of this objective. However, the ICR (p. 41) provides a summary about a meeting with beneficiaries from Butila village which is located in the Sarajevo municipality. Before the village was connected to the Sarajevo Vik wastewater network through this project, all residents used pit latrines for domestic sanitation. This limited their water consumption as the volume of the pit latrines was limited. In addition, at different times throughout the year (after snow melted or heavy rains), the pit latrines flooded and their content spilled over properties, irrigation channels, and roads resulting in a bad smell and having negative health impacts to the local population. Some families, especially those with young children, intended to move away from Butila for that reason. In 2014, Butila was connected to the Sarajevo ViK system. The project financed the construction of 750 meters of secondary main wastewater network to connect the village with the Sarajevo ViK wastewater system. The project’s contractor provided extensions to every house and residents built their own in-house plumbing under the supervision and advise of the contractor. The project’s beneficiaries stated that in addition to convenience and increase in water consumption, property values increased and the settlement experienced an influx of small businesses. Page 5 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Sarajevo Waste Water (for. Mun. Dev.) (P090675) However, the Results Framework did not include any specific targets for this objective. Therefore, it is not clear to what extent this objective was achieved. Rating Modest PHREVDELTBL PHOVRLEFFRATTBL Rationale Achievement of the first two objectives was Substantial and achievement of the third objective was Modest, resulting in an overall Substantial rating for the project’s efficacy. Overall Efficacy Rating Substantial 5. Efficiency Economic efficiency: The PAD (p. 45) conducts an Economic analysis based on the results of the 1999 willingness-to-pay survey which estimated the value household and non-household sewage generators placed on an improved environment due to waste water treatment at the Sarajevo WWTP. The willingness-to-pay data was adjusted assuming real growth of per capita Gross Domestic Product (GDP) from 1998 to 2008. The analysis estimated a Net Present Value (NPV) of US$54.9 million and an Economic Internal Rate of Return of 32 percent indicating that the project was a worthwhile investment. The ICR (p. 34) conducts a before- and after- cost-benefit analysis. The project was to generate benefits on two levels. On the local level the project was to have a positive impact on the property value of households newly connected to the Sarajevo WWTP and public health benefits from a decrease in waterborne and water- washed diseases. On the global level the project was to have a positive impact on the reduction of nutrient pollution of the Danube basin and a reduction of Green House Gas emissions. Costs included the project’s investments and local costs such as the payment for the wastewater services by consumers and additional electricity consumption for nutrient removal. The analysis estimated the local benefits with a NPV of US$38 million and an Internal Rate of Return (IRR) of 80 percent. The global benefits were estimated for a reduction of 1kg of nutrients (at US$5) resulting in a negative NPV of US$6.5 million. The Bank team stated that the reason for the negative NPV is the fact that the actual cost of nutrient removal is higher than US$5 per kg, indicating that the nutrient accounting unit (used for calculating global environmental benefits) is off-mark and might need to be revised by the international community, as it was already done for the carbon market. The benefit from a reduction of 1 ton of carbon equivalent emissions (estimated at US$15 until 2025 and afterwards US$30) was estimated. The calculation of the economic returns was consistent, used reasonable Page 6 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Sarajevo Waste Water (for. Mun. Dev.) (P090675) assumptions and correctly identified the return on the project’s investment. This led to returns of a NPV of US$8.4 million and an IRR of 34 percent indicating that the project was a worthwhile investment. However, the project was not able to increase the efficiency of the Sarajevo ViK whose tariffs (as of 2010) remained below its operating costs. Also, tariff collection was poor and staff costs and turnover were high. Even though Sarajevo ViK was able to increase revenue flows and reduce its Operation and Maintenance costs, the collection rate remains low at 85 percent. Operational efficiency: It was possible to reduce the construction costs of wastewater mains through competitive bidding and through rebuilding SWWTO rather than through new construction. This made the entire construction process quicker since no permits and land allocations were necessary. However, the project’s closing date was extended by 18 months allowing for the completion of the reconstruction, its testing and commissioning to the client. The project extension might be indicative of an inefficient use of project resources. In addition, several activities of component 2 had to be replaced with other activities to avoid duplication of activities implemented by the European Bank for Reconstruction and Development together. This also indicates inefficiencies in the design phase of the project. Taking everything together the project’s efficiency is rated Modest. Efficiency Rating Modest a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 0 Appraisal 0 Not Applicable 0 ICR Estimate 0 Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome Relevance of objective was High given the impact of poor waste water treatment on the Milijacka and Bosna rivers and the living conditions of people living along these two rivers. Efficacy was rated Substantial and Efficiency was rated Modest. Taking everything together, the overall outcome rating is Moderately Satisfactory. a. Outcome Rating Moderately Satisfactory Page 7 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Sarajevo Waste Water (for. Mun. Dev.) (P090675) 7. Risk to Development Outcome The government and all entities that were engaged in the project continue to be committed to improving wastewater treatment. Furthermore, the government of the Sarajevo canton has demonstrated its commitment by its financial contribution to the project. However, there are several challenges that the government needs to address. First, the water and wastewater sector requires additional financing given the condition of its infrastructure. Improvements are necessary in order to be able to implement any tariff reforms. Second, financial sustainability of the Sarajevo ViK is still not ensured without its ability to cover its operating costs entirely when it is operating the WWTP at full capacity. And third, the Sarajevo WWTP experiences a shortage of qualified staff which might have a negative impact on the maintenance the WWTP requirements. Given these risks there is a need for additional support for the rehabilitation of water supply and wastewater systems of the Sarajevo ViK, conducting a review of cost recovery for water supply and wastewater service provision, and hiring new staff at the Sarajevo WWTP. 8. Assessment of Bank Performance a. Quality-at-Entry The project benefited from the experience of previous Bank projects in the waste management sector in Bosnia and Herzegovina such as the Urban Infrastructure and Service Delivery project (P083353) and the first and second phase of the Solid Waste Management project (P057950). Also, lessons learned from other water and sanitation projects in the Europe and Central Asia region were taken into account when designing the project. The Bank team identified several risk factors as substantial such as the fragmented oversight responsibilities in the municipal sector adding complexity to the already complicated institutional framework and actual damages to specific components of the WWTP being more severe than anticipated. An additional risk that was identified included the lack of capacity on the Project Management Team (PMT) to oversee large-scale infrastructure rehabilitation, and perform the financial management and procurement functions. The Bank team’s mitigation measures were adequate and the identified risks did not materialize. However, project preparation was slow and took over three years. The project design did not account for the burden of contributions from the Sarajevo Canton Government. In addition, the Results Framework had a major shortcoming since it did not include any indicators to measure a critical aspect of the PDO “improve the living conditions of populations”. Quality-at-Entry Rating Moderately Satisfactory b. Quality of supervision The Bank team had the necessary expertise and skills to implement the project and conducted regular supervision missions including field visits and physical checks of investments. Throughout the project’s implementation there were four changes in the authorities. In order to ensure that the project remained on Page 8 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Sarajevo Waste Water (for. Mun. Dev.) (P090675) track, the Bank intensified its engagement with the counterpart. Implementation bottlenecks were identified in a timely manner and the mid-term review focused on speeding up the implementation of the largest contract for the WWTP rehabilitation. Furthermore, the Bank team supervised the project’s fiduciary and safeguard aspects adequately. Disappointingly, the Bank did not modify the Results Framework to include indicators for measuring the project’s impact on improving living conditions. Quality of Supervision Rating Moderately Satisfactory Overall Bank Performance Rating Moderately Satisfactory 9. M&E Design, Implementation, & Utilization a. M&E Design The project’s objective was clearly specified and most outcome and output indicators were aligned with the PDO statement. Also, the theory of change was sound and reflected in the Results Framework. However, the Results Framework did not include any indicators to measure a critical aspect of the objective “improved living conditions”. The Results Framework included six PDO indicators and three intermediate outcome indicators. The indicators were specific, measurable, relevant, and time bound. However, since only one of the PDO indicators and only one of the intermediate outcome indicators had a baseline it is difficult to say whether the targets were realistic. The PMT was responsible for the monitoring and evaluating of the project and for submitting quarterly progress reports to the Bank. b. M&E Implementation The PIT and PMU implemented the M&E system at the beginning of project preparation. Indicators in the Results Framework were measured by the PMU effectively and reported in the Implementation Status Reports. A shortcoming is that the Results Framework was not adapted during project implementation to add indicators measuring “improved living conditions”. In 2012, the project added a new indicator “number of people provided with improved sanitation”. M&E data was reliable and of good quality. The M&E functions and processes are part of the Sarajevo ViK reporting system and therefore likely to be sustainable. The M&E results were reported by the project directly to the Sarajevo Canton government and the Bank team. c. M&E Utilization M&E was used to measure implementation progress. It is not clear if M&E was used to inform decision making. Page 9 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Sarajevo Waste Water (for. Mun. Dev.) (P090675) M&E Quality Rating Substantial 10. Other Issues a. Safeguards The project was classified as Category B and triggered the Bank’s policy OP/BP 4.01 (Environmental Assessment) and OP/BP 7.50 (International Waterways). An Environmental Management Plan was prepared and implemented and a review of potential environmental impacts associated with the project was conducted. No significant negative environmental impacts were anticipated but instead it was anticipated that the project would have positive environmental impacts such as improving the quality of drinking water, wastewater management, reducing the use of chemicals and better handling of water treatment residuals. The project was also assessed against OP/BP 7.50 but it was found that the project would not adversely change the quality and quantity of water flows to riparian countries and would not be adversely affected by other riparian countries’ possible water use nor did the project increase the water use. The project received an exception by the Europe and Central Asia Vice Presidency to the notification requirement under OP/BP 7.50. The project complied with safeguards relevant to this project. b. Fiduciary Compliance Financial Management The project’s institutional financial management arrangements and internal control procedures were adequate. The project implementation units were well staffed and managed. Quarterly unaudited interim financial reports were submitted on a regular and timely basis and were found acceptable by the World Bank. Also external audits were conducted in a timely manner and had unqualified opinions. One issue relating to financial management was a delay in the submission of counterpart funding by the Sarajevo Canton Government in 2013. However, the PMT and PIT solved the issue adequately. The project fulfilled all financial covenants. The ICR (p. 21) states that the project was constantly in compliance with fiduciary and procurement requirements. Procurement The project followed the Bank’s procurement guidelines and did not experience any procurement related delays. The ICR (p. 21) states that procurement was mostly satisfactory during project implementation. c. Unintended impacts (Positive or Negative) N/A Page 10 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Sarajevo Waste Water (for. Mun. Dev.) (P090675) d. Other --- 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment Moderately Moderately Outcome --- Satisfactory Satisfactory Moderately Moderately Bank Performance --- Satisfactory Satisfactory Quality of M&E Substantial Substantial --- Quality of ICR Substantial --- 12. Lessons • Addressing the different aspects of wastewater management is critical in order to obtain sustainable results. In this project only one aspect of wastewater management was addressed. While all the wastewater is being collected and treated and the number of wastewater blockages continues to decrease, the sector still struggles with weak institutional and financial capacity, growing water losses, and inefficient operation of the Sarajevo ViK. Therefore, the project would have benefited from a more system wide approach. • Strong cooperation between different donors is critical to ensure good results. In this project, the Bank worked with other development partners who were responsible for the implementation of complementary activities. However, several activities of component 2 had to be replaced with other activities in order to avoid duplication of activities to be implemented under the loan by the European Bank of Reconstruction and Development (EBRD). Added by IEG: • Addressing the financial sustainability of a major actor in the sector is critical for ensuring long-term sustainability of project outcomes. In this project the Sarajevo ViK’s cost recovery is still not sufficient to cover its Operation and Maintenance costs since it has not been able to increase its tariffs since 2010. 13. Assessment Recommended? No 14. Comments on Quality of ICR The ICR provides a good overview of project preparation and implementation. The ICR is concise and internally consistent. The report’s candid coverage of the issues related to cost recovery for water supply and Page 11 of 12 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Sarajevo Waste Water (for. Mun. Dev.) (P090675) waste water service provision allowed for a better understanding of sustainability issues. Also, the ICR provides an adequate traditional economic analysis and includes useful lessons learned. However, the ICR provides limited information on critical areas such as M&E which makes it challenging to assess to what extent M&E was used to inform decision making and guide the implementation direction of the project. Also, the ICR does not describe the linkage of outputs produced and outcomes achieved under the second objective sufficiently. This makes the attribution of project activities to the achievement of the objective challenging. Overall, the ICR is rated Substantial. a. Quality of ICR Rating Substantial Page 12 of 12