Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Naghlu Hydropower Rehabilitation Project (P132944) Report Number: ICRR0023515 1. Project Data Project ID Project Name P132944 Naghlu Hydropower Rehabilitation Project Country Practice Area(Lead) Afghanistan Energy & Extractives L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) TF-A1691 30-Sep-2022 21,011,924.83 Bank Approval Date Closing Date (Actual) 14-Dec-2015 31-Mar-2024 IBRD/IDA (USD) Grants (USD) Original Commitment 83,000,000.00 83,000,000.00 Revised Commitment 22,592,422.68 22,592,422.68 Actual 21,011,924.83 21,011,924.83 Prepared by Reviewed by ICR Review Coordinator Group Ihsan Kaler Hurcan Kishore Laxmikant Ramachandra Jammi IEGSD (Unit 4) Nadkarni 2. Project Objectives and Components DEVOBJ_TBL a. Objectives According to the Afghanistan Reconstruction Trust Fund (ARTF) Grant Agreement (p.4) dated January 19, 2016, the project objective was “to improve dam safety and sustainability of hydropower, and to increase the supply of domestically generated hydroelectricity at the Naghlu Hydropower Plant.” According to the Project Appraisal Document (PAD, p.8) dated November 22, 2015, the project objective was “to improve dam safety and to increase the supply of electricity at the Naghlu Hydropower Plant.” The Page 1 of 28 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Naghlu Hydropower Rehabilitation Project (P132944) objective in the PAD did not include the sustainability of hydropower, and its formulation was slightly different without a major material difference. At the only project restructuring on June 30,2021, the project objective was revised to read similar to the project objective in the PAD, and the Darunta Hydropower Plant was added: “to improve dam safety, and to increase the supply of electricity at the Naghlu and Darunta Hydropower Plants.” As agreed between the Independent Evaluation Group (IEG) and the World Bank’s South Asia Region, this review will conduct a split assessment of the project performance for the periods before and after the Taliban’s takeover of power in the country in August 2021. Furthermore, the revision of the project objectives requires a split assessment of the project performance before and after the revision. Since there was no disbursement after the revision of the project objectives between the project restructuring in June 2021 and the Taliban takeover in August 2021, this review will assess the project’s performance based on the parsing of the project objective as follows: Before the Taliban takeover in August 2021: Objective 1: to improve dam safety and sustainability of hydropower at the Naghlu Hydropower Plant. Objective 2: to increase the supply of domestically generated hydroelectricity at the Naghlu Hydropower Plant. After the Taliban takeover in August 2021: Objective 1: to improve dam safety at the Naghlu Hydropower Plant. Objective 2: to increase the supply of electricity at the Naghlu Hydropower Plant. Objective 3: to improve dam safety at the Naghlu Hydropower Plant. Objective 4: to increase the supply of electricity at the Darunta Hydropower Plant. b. Were the project objectives/key associated outcome targets revised during implementation? Yes Did the Board approve the revised objectives/key associated outcome targets? No c. Will a split evaluation be undertaken? Yes d. Components The project consisted of three components: A. Mechanical, Electrical, and Electromechanical Works. (Appraisal cost: US$30.00 million including physical and price contingencies of 25 percent; revised cost at project restructuring: US$35.00 million Page 2 of 28 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Naghlu Hydropower Rehabilitation Project (P132944) including the addition of the Darunta Hydropower Plant dam rehabilitation to the project scope at the project restructuring in June 2021 (see Revised Components below); actual cost as of August 15, 2021: US$12.71 million; the cancelled amount was US$22.29 million) This component consisted of two subcomponents for the rehabilitation of the electrical and electromechanical parts of the power plant that could not be completed under the Emergency Power Rehabilitation Project (EPRP, P083908), which closed in March 2013. A.1. Rehabilitation of Unit 1 and Balance of Plant. The Naghlu Hydropower Plant has four Francis turbines each with a generation capacity of 23.5 megawatt (MW) totaling 94 MW. The EPRP financed the rehabilitation of three units but could not complete the rehabilitation of Unit 1. Under this subcomponent, the project was to finance the rehabilitation of the electrical and electromechanical parts of Unit 1 (such as repair or replacement of the bent rotor shaft) and the associated supporting and/or auxiliary components (balance of plant). A.2. Enhancing Maintenance of the Powerhouse. This subcomponent was to finance the provision of spare parts and consumables, operations and maintenance equipment, and the construction of related warehouses to enhance the operation and maintenance of other three units (2, 3, and 4) of the powerplant, which had not been maintained for an extended period. B. Dam Safety and Power Generation Improvement. (Appraisal cost: US$33.00 million including physical and price contingencies of 25 percent; revised cost at project restructuring: US$27.00 million; actual cost as of August 15, 2021: US$0.57 million, the cancelled amount was US$26.43 million) This component consisted of two subcomponents: B.1. Dam Safety Audit and Safety improvement Measures. This subcomponent was to finance activities to improve the safety of the Naghlu Hydropower Plant dam, such as the audit of the dam’s structural and operational safety, reactivation of the bottom outlet, improvement of head gates closing systems, clearance of unexploded ordnance from the dam structure, studies on hydrological and seismic data, flood routing including spilling arrangements, implementation of dam safety plans, and identification of sustainable sediment management. B.2. Optimization of Power Generation. This subcomponent was to finance the feasibility of various options to optimize power generation, such as appropriate dam operation and dam management, additional storage upstream of the dam (the current dam was insufficient; therefore, the power plant was generating electricity most like a run-of-river powerplant), additional mechanisms for controlled flushing of sediments that had been preventing some units to generate electricity, raising the dam crest, and catchment area treatment. This subcomponent, the cost of which was estimated at US$8.00 million, was cancelled at the project restructuring in June 2021, and a new subcomponent was added to the project scope related to the optimization of power generation at the Darunta Hydropower Plant (see Revised Components below). C. Environmental and Social Sustainability, Project Management Support, and Future Project Preparation. (Appraisal cost: US$20.00 million including physical and price contingencies of 25 percent; revised cost at project restructuring: US$21.00 million; actual cost as of August 15, 2021: US$7.67 million, the cancelled amount was US$13.33 million) This component consisted of two subcomponents: Page 3 of 28 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Naghlu Hydropower Rehabilitation Project (P132944) C.1. Environmental and Social Sustainability. This subcomponent’s goal was to ensure the environmental and social sustainability of the Naghlu Hydropower Plant dam through the provision of electricity to eight villages and development assistance to the communities around the dam (such as road development and vocational training including specific skills development for women) and support to the implementation of environmental and social safeguards. C.2. Project Management Support and Future Project Preparation. This subcomponent was to support Da Afghanistan Breshna Sherkat (DABS – public electricity utility and the project implementing entity) in project implementation, the work of the Environmental and Social Advisory Panel and the Project Technical Advisory Panel, and the identification of potential future projects for hydropower development by DABS. Revised Components At the project restructuring in June 2021 (in the sixth year of project implementation), raising the crest of the Naghlu Hydropower Plant dam under subcomponent “B.2. Optimization of Power Generation” was cancelled because the Project Technical Advisory Panel concluded after their first visit to the dam that it was not feasible to raise the dam crest as this would have create dam safety, environmental, and social risks (ICR, p.8). Two new subcomponents were added to the project scope to utilize grant savings from Component A and the cancellation of subcomponent B.2, and activities under Component C were adjusted. A.3. Darunta Powerhouse and Switchyard Rehabilitation. (Cost estimated at the time of project restructuring: US$12.00 million; actual cost as of August 15, 2021: US$1.30 million; the cancelled amount was US$10.70 million) This new subcomponent was to finance the rehabilitation of three generation units each with 3.85 MW generation capacity (a total of 11.55 MW), spillway gates and switchyard, and a geotechnical investigation of the power plant. Darunta Hydropower Plant is located 80 km downstream of the Naghlu Hydropower Plant close to the Pakistani border and generates electricity for Jalalabad city. B.2. Dam Safety Measures. (Cost estimated at the time of project restructuring: US$21.50 million; actual cost as of August 15, 2021: US$0.50 million; the cancelled amount was US$21.00 million) This new subcomponent, which replaced “B.2. Optimization of Power Generation” at the project restructuring, was to finance implementation of dam safety measures including instrumentation, rehabilitation of gates and hoisting mechanism controls of the Darunta Hydropower Plant dam along with seepage control, and construction of fences and security posts around the Naghlu Hydropower Plant dam. The dam safety works on Darunta Hydropower Plant were transferred from the World Bank-financed Irrigation Restoration and Development Project (IRDP – P122235) following the identification of safety issues with the dam, which was considered to have a high risk of dam failure. The IRDP closed in December 2020 in the process of country portfolio adjustment in response to COVID-19 pandemic (ICR, p.8). Under Component C, the following adjustments were made at the project restructuring: C.1. Environmental and Social Sustainability. (Cost estimated at appraisal: US$8.00 million; cost estimated at the time of project restructuring: US$12.00 million; actual cost as of August 15, 2021: US$5.0 million; the cancelled amount was US$7.00 million) Following the consultation with the local communities, it was agreed to provide vocational training to the Sorobi region under the ongoing National Horticulture and Livestock Program (NHLP – P157997), and the vocational training activities under this subcomponent were cancelled. The subcomponent’s scope was extended to include the electrification of 93 additional villages in the Sorobi and Tagab districts. Page 4 of 28 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Naghlu Hydropower Rehabilitation Project (P132944) C.2. Project Management Support and Future Project Preparation. (Cost estimated at appraisal: US$12.00 million; cost estimated at the time of project restructuring: US$9.00 million; actual cost as of August 15, 2021: US$2.67 million; the cancelled amount was US$6.33 million) The project management support for the Darunta Hydropower Plant related works was added to the scope. The scope of the subcomponent was expanded to support DABS in the collection of baseline geographic, hydrological, environmental, social, and disaster risk data from Panji River for future hydropower project identification and preparation. Lastly, two contracts from ARTF-funded DABS Planning and Capacity Support Project (P131228) that could not be completed before the project closing date of July 31, 2020 because of the onset of the COVID-19 pandemic were transferred to this subcomponent. The activities covered by these two contracts were the provision of training aid equipment, supply of furniture for the training center, and staff capacity building training totaling around US$1.00 million. e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project Cost: The project cost was originally estimated at US$83.00 million. At the project restructuring in June 2021, the estimated project cost did not change. The project’s actual coast was US$22.60 million, which was significantly lower than the cost estimated at appraisal because of the incomplete project activities following the Taliban takeover in August 2021. Financing: At appraisal, the IDA grant amount was estimated at US$83.00 million that would fully finance the project cost. At the time of project restructuring in June 2021 (two months before Taliban takeover), the project had disbursed US$22.60 million (including US$2.66 million eligible expenditures). The undisbursed amount of US$60.40 million was cancelled on April 8, 2022 as part of the ARTF portfolio-wide cancellation exercise (ICR, p.1). Borrower’s contribution: At appraisal, no borrower’s contribution was estimated, and none materialized at project closing. Project Restructuring (Level 2 – June 20, 2021): The project was restructured late in project implementation—five years and five months after project effectiveness. It was restructured to cancel subcomponent B.2 and add two new subcomponents to the project scope (A.3) for the rehabilitation of the Darunta Hydropower Plant and extension of the grids in the Sorobi and Tagab districts and (B.2) dam safety at Naghlu and Darunta Hydropower Plants. Consequently, the project objective was revised to include the improvement of dam safety and increase in electricity generation at the Darunta Hydropower Plant. The World Bank’s South Asia Region Vice President approved this project objective revision. Additionally, two contracts of the ARTF-funded DABS Planning and Capacity Support Project (P131228) that could not be completed were transferred to the Naghlu Hydropower Rehabilitation Project (see Revised Components above). Consequently, the project closing date was extended by 18 months from September 30, 2022 to March 31, 2024 to allow time for the completion of the newly added project activities and the activities related to the dam safety works at Naghlu Hydropower Plant that could not be implemented because of the delays in the procurement of the consultant for dam safety and hydromechanical audits due to the limited availability of experts and contractors in this very specific and technical field and reluctance of consultants to work in a fragile and conflict environment in Afghanistan. The onset of COVID-19 in March 2020 also contributed to the delays in project implementation. Dates: The project was approved on December 14, 2015. The ARTF Grant Agreement was signed on January 19, 2016, and the project became effective on the same day as there were no conditions defined Page 5 of 28 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Naghlu Hydropower Rehabilitation Project (P132944) for effectiveness. The Mid-Term Review was conducted in June 2019. The original project closing date was September 30, 2022. The project closing date was extended by 18 months, and the project will officially close on March 31, 2024 (The reasons for project closing date extension are given in the restructuring entry above). Since the undisbursed grant was cancelled on April 8, 2022 and the project could not practically be implemented, the World Bank project team prepared the Implementation Completion and Results Report of the project before the revised project closing date of March 31, 2024. Disbursement Percentages The following disbursement percentages will be used in deriving the weights to be applied to split Outcome rating. Disbursed and Undisbursed Amounts Percentage of Project Objective before and after Taliban Takeover the Total Grant Amount ($ million) (%) Before Taliban Takeover 22.60 (disbursed) 27.23 After Taliban Takeover 60.40 (undisbursed) 72.77 Total 83.00 100.00 3. Relevance of Objectives Rationale The original project objective was highly aligned with the World Bank’s strategy as defined in the Country Partnership Framework for FY2017-20 (CPF) for Afghanistan, which was extended through fiscal year 2022. The project sought to address the development problem of unreliable and insufficient electricity supply to Kabul, the capital of Afghanistan, because of neglect and insufficient maintenance of the Naghlu Hydropower Plant and its dam, which provides more than half of Kabul’s electricity, and inadequate institutional capacity in the electricity sector. The project also sought to address the underutilization of the domestically available cheap hydropower potential that would replace the more expensive electricity imported from the neighboring countries improving the affordability of electricity. As reliable and sufficient supply of electricity has a direct impact on improved socioeconomic welfare of the people in the medium and long-terms, the project was expected to contribute to the reduction of poverty, which was one the main drivers of fragility in Afghanistan. These development problems fit under the second pillar of the CPF, i.e., Supporting Inclusive Growth, and contributes to the achievement of “Objective 2.3: Increased power generation and access to electricity.” The CPF (p.23) defines the intended outcomes at the end of the CPF period as “increased access to both on-grid and off-grid electricity, reduction in distribution losses as well as improved regional energy cooperation” to be achieved through increased local power generation and regional cooperation initiatives, such as CASA 1000 that supported sustainable electricity trade between Tajikistan, Kyrgyz Republic, Afghanistan, and Pakistan through the construction of a transmission line. The project was to support the achievement of these outcomes through the rehabilitation and overhauling of the Naghlu Hydropower Plant, establishment of improved operation and maintenance (O&M) practices (improved institutional capacity), and improvement of the safety of the Naghlu Hydropower Plant dam. These were expected to lead to the improvement of the availability, reliability, affordability, and sustainability of electricity supplied to Kabul from domestic hydropower resources. The project objective also corresponded to the first pilar of the CPF, i.e., Building Strong and Accountable Institutions, which aims to improve service delivery by building the capacity and self-reliance of government institutions, and the Page 6 of 28 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Naghlu Hydropower Rehabilitation Project (P132944) third pillar, i.e., Social Inclusion, the goal of which is to reduce differences in access to services in the country and address vulnerability shocks (CPF, pp.17 and 26). The revised project objective, which aimed also to increase power generation from the Darunta Hydropower Plant and improve its dam safety, was aligned with the CPF because of the reasons explained above with one difference that Darunta Hydropower Plant supplies electricity to Jalalabad, the fifth largest city in Afghanistan in terms of population. The project objective was also highly relevant to the country context. The project objective supported the implementation of the Afghanistan National Development Strategy (ANDS). The ANDS identified poor and unreliable access to electricity as a barrier to economic development and poverty alleviation and aimed at investing in transmission network and power generation to increase the electrification rate and improve the electricity service quality in the country. Given the fragile and conflict-affected situation (FCS) in Afghanistan, the project objective was appropriately pitched for the development status in the country as described in the CPF. As advised in the World Bank’s Systematic Country Diagnostic (SCD, 2016), the project’s intervention was carefully selected and tailored to the fragile circumstances in the country. The Naghlu Hydropower Plant was of high strategic importance as the power plant was the main source of electricity supplied to Kabul. The previous World Bank-financed EPRP (P083908) could only rehabilitate three out of four units at the Naghlu Hydropower Plant and was moderately successful in establishing sustainable O&M practices. The Naghlu Hydropower Rehabilitation Project was to rehabilitate the remaining unit and overhaul the previously rehabilitated ones, which deteriorated because of insufficient O&M. Additionally, the project was to improve the safety of the dam including the clearance of the sedimentation and unexploded ordnance in the reservoir. The rehabilitation of the Darunta Hydropower Plant, which was included to the project scope at the project restructuring in June 2021, was also highly relevant to the country context as its dam needed urgent repairs to ensure sustained electricity supply to Jalalabad. Following the reengagement in 2002, the World Bank extensively supported Afghanistan through interventions in power generation, transmission and distribution, and technical assistance and policy advice. Some World Bank-financed electricity projects in Afghanistan were as follows: The Emergency Infrastructure Reconstruction Project (P077779), the Emergency Power Rehabilitation Project (P083908), the Kabul, Aybak, Mazar-e-Sharif Power Project (P106654), and the Power System Development Project (P111943). These projects provided support to the rehabilitation of hydropower generation assets, rehabilitation and construction of transmission and distribution assets with the goal of increasing access to electricity in the country and improving the availability, reliability, and quality of electricity supply. Therefore, the objective of the project under this review, excluding its focus on “sustainability of hydropower,” was not fully pitched at the outcome level given the World Bank’s extensive experience in the country and the sector. However, given the fragile, conflict, and violence (FCV) status of the country and rapidly challenging security situation, this review assesses the project objective as sufficiently challenging and relevant to the FCV status of Afghanistan. Overall, the relevance of objective is rated High. Rating Relevance TBL Rating High 4. Achievement of Objectives (Efficacy) Page 7 of 28 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Naghlu Hydropower Rehabilitation Project (P132944) EFFICACY_TBL OBJECTIVE 1 Objective To improve dam safety and sustainability of hydropower at the Naghlu Hydropower Plant. Rationale Theory of Change The main causes of Afghanistan’s fragility were defined as weak state institutions, poor governance, and dysfunctional societal relations that frequently led to violent conflict between groups (ICR, p.1). To address the drivers of fragility, the World Bank strategy (CPF 2016) was designed to build strong and accountable institutions, support inclusive growth, and expand and deepen social inclusion. The project was to contribute to the achievement of the World Bank strategy by improving the availability, reliability, and sustainability of electricity supply to the capital city of Kabul through the rehabilitation of the Naghlu Hydropower Plant and the improvement of its dam safety. By providing reliable electricity to households, institutions, and businesses, the project was to contribute to the improvement of economic and social activity and consequently to the reduction of poverty in the project area and Kabul (ICR, pp.5-6). The project was also to address the fragility driver of weak state institutions through technical assistance to build DABS capacity in the sustained operation and maintenance of the Naghlu Hydropower Plant. The project was to use the project’s inputs—IDA grants—to finance the rehabilitation and overhauling of the Naghlu Hydropower Plant, the provision of spare parts for O&M, the inspection of the dam and the hydromechanical works, sediment assessment and sediment removal including unexploded ordnance, rehabilitation of the dam instrumentation for safety, rehabilitation of the hydromechanical equipment, and the reactivation of the flushing system. If decided, the project was to increase the height of the dam to optimize power generation by the hydropower plant. Additionally, the project was to provide training to the DABS personnel to increase the entity’s capacity in O&M. The expected outputs of these activities would have been all four units of the Naghlu Hydropower Plant generating electricity, improved safety of the dam, increased reservoir area (because of sediment cleaning and increased height of the dam) to optimize power generation, and increased institutional capacity in O&M. These outputs would have been expected to lead to the project outcomes of increased availability of electricity supply, reduction in load shedding and unplanned outages (i.e., increased reliability of electricity supply), and sustainability of electricity supply because of improved O&M practices and dam safety. The project was also to finance the electrification of villages around the Naghlu Hydropower Plant by extending the distribution grid, but the project did not finance the cost of the line- drops required to connect the households to the grids. The project objective formulation did not cover all the outcomes expected from the project’s intervention except improved dam safety and sustainability of hydropower and increased power generation. Therefore, project objective was more output-oriented rather than outcome-oriented, but taking the country’s FCV status into account, the formulation of the project objective was realistic and appropriate to country conditions. More electricity supplied to the Kabul grid in a more sustainable way because of improved dam safety and O&M practices was to be expected to contribute to the improvement of the overall socioeconomic welfare of the residents in the project area and address drivers of fragility. Overall, the causal pathways in the theory of change from inputs to outcomes were valid and direct (except the weakness in covering the cost of line-drops for households’ connection to the distribution grid), and the achievement of the outcomes and project Page 8 of 28 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Naghlu Hydropower Rehabilitation Project (P132944) objective could be attributed to the project’s intervention. However, the indicators in the results framework were insufficient to capture the outcomes that were expected from the implementation of dam safety activities and achievement of related outputs. The M&E system as designed and implemented was insufficient to assess the performance of the project in achieving the project objectives (see section 9. M&E Design, Implementation, and Utilization below). The addition of the rehabilitation of the Darunta Hydropower Plant and the improvement of its dam safety to the project scope at the project restructuring in June 2021 did not change the theory of change of the project. The rehabilitation of the three generation units and the installation of a switchyard at the Darunta Hydropower Plant would have been expected to ensure sustained and reliable power generation at the power plant, which supplied electricity to Jalalabad. The rehabilitation of the spillway gates and the investigation of the dam would have been expected to improve the safety of the dam that would contribute to the sustainability of power generation. Outputs Dam Safety Procedures Developed and Maintained. The project could not develop dam safety procedures. However, by the time of the Taliban takeover, consultants had already started working on dam safety and hydromechanical audit and the environmental and social impact assessment of the dam safety measures. According to the project implementation schedule, the project would have completed the development of dam safety procedures by September 30, 2022, but could not do so because of the Taliban takeover. Dam Safety Audit in place. The project could not conduct the dam safety audit because of delays caused by the experts’ reluctance to visit the project site due to security concerns and the onset of the COVID-19 pandemic. However, the bidding documents were about to be completed, and the experts were scheduled to visit the dam in August 2021 when the Taliban took power in the country. Therefore, the visit did not happen. Number of staff trained in hydropower operation and maintenance under the project. The project was to provide training to 130 members of DABS staff in hydropower operation and maintenance. The actual number of members of staff trained under the project was 25 because of delays in hiring consultants to give these trainings. However, the remaining members of staff were expected to be trained by the revised target date in August 2022, but the trainings could not be delivered because of the Taliban takeover. The project financed the installation of pumps in the drainage and grouting galleries of the dam. The Project Technical Advisory Panel recommended the installation of these pumps. The project financed the rehabilitation of the gates in the flushing system. This work was completed in mid-2022. After the Taliban takeover, the hydropower plant staff, including a resident engineer, completed the cleaning of the flushing tunnel, and the repair of the sliding gate on the face of the dam and the radial gate at the end of the flushing tunnel. These improvements were needed to flush the sediment in front of the lower outlet. The project also financed the construction of a security wall around the Naghlu Hydropower Plant’s periphery to secure the hydropower plant, the dam, and the reservoir. The project did not finance works to increase the height of the dam as it was found to be not feasible during project implementation because of risks related to the dam safety. Outcomes Page 9 of 28 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Naghlu Hydropower Rehabilitation Project (P132944) As a result of the completion of some project activities, the project was able to partially achieve expected outcomes in dam safety and sustainability of hydropower at the Naghlu Hydropower Plant. The pumps installed by the project emptied the water in the galleries that had been putting significant pressure on the dam safety. The elimination of this pressure improved the stability of the dam (ICR, p.13). As of May 2022, the hydropower plant staff had been using the pumps daily, and 50 percent of the drainage holes and pipes had been operational. Spare parts were available and safely stored in the warehouses built under the project for regular O&M of the power plant. The improvements in the flushing system resulted in the successful flushing of the sediment in front of the lower outlet, which was required for safe and sustainable operation of the hydropower plant. The training given to a limited number of member of staff is expected to increase the O&M quality. The Third-Party Monitoring Agent hired under the project reported that all four turbines had been inspected and maintained annually, and there had been no efficiency loss in the turbines. Despite significant delays in the procurement of project activities and hiring of consultants related to dam safety and sustainability of hydropower at the Naghlu Hydropower Plant, the project activities were on track to achieve the project outputs and outcomes by the revised target dates in August 2022. Despite the Taliban takeover in August 2021, the hydropower plant staff continued with the works started under the project to improve the safety of the dam and improve the sustainability of hydropower with some success. Overall, the project’s efficacy in achieving the project objective to improve dam safety and sustainability of hydropower at the Naghlu Hydropower Plant is rated Substantial with moderate shortcomings. Rating Substantial OBJECTIVE 1 REVISION 1 Revised Objective To improve dam safety at the Naghlu Hydropower Plant. Revised Rationale For theory of change and project outputs and outcomes please see Objective 1 above. The objective was revised in June 2021 only two months prior to the Taliban takeover in August 2021. However, the project could not disburse any grant during the two-month period. Therefore, the project’s efficacy in achieving the revised Objective 1 is assessed based on the project’s achievements prior to the Taliban takeover, and the additional works completed by the hydropower plant as a result of the capacity created and equipment provided by the project. According to the report of the Third-Party Monitoring Agent in May 2022, the hydropower plant had continued to implement regular inspection and maintenance of the hydropower plant. The flushing system that cleans the sediment in front of the lower outlet is functional. However, the project could not remove the sediment in front of the intake gates. It is not expected that DABS will continue with the work to remove the sediment in front of the intake gates, and further accumulation of the sediment will most likely block the intake gates, after which the hydropower plant could be not operational and cease to generate electricity. The removal of the sediment in front of the intake gates stands out to be the major barrier to the sustainability of the hydropower plant. However, as the World Bank project team commented, if the flushing system, which was rehabilitated under the project and after the Taliban takeover, continued to work properly, it could flush the sediment that Page 10 of 28 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Naghlu Hydropower Rehabilitation Project (P132944) would otherwise block the intake gates. Yet, without further investigation it cannot be known for sure how efficient the flushing system would be in clearing the sediment. Therefore, the risk is high that further accumulation of sediment could adversely affect the operation of the power plant. Overall, the project’s efficacy in achieving the project objective to improve the dam safety at the Naghlu Hydropower Plant is rated Modest for the post-Taliban takeover period because the risks to the sustainability of the outcomes achieved related to the dam safety is significantly high; the hydropower plant could be out of operation within two to four years unless urgent measures are taken to address the sediment problem and maintain regular inspection and maintenance practices. Revised Rating Modest OBJECTIVE 2 Objective To increase supply of domestically generated hydroelectricity at the Naghlu Hydropower Plant. Rationale Outputs Installed capacity of Unit 1 of the Naghlu hydropower Plant ready for utilization. The project completed the rehabilitation of the 25 MW Unit 1 of the hydropower plant in March 2018 as targeted. Overhaul of Unit 3 of the Naghlu Hydropower Plant. The project completed the overhaul of the 25 MW Unit 3 of the hydropower plant in October 2018 as targeted. The international technical experts hired under the project supported DABS personnel in the rehabilitation of Unit 4, which was not included in the project scope. The rehabilitation of Unit 4 was completed within two months between September and November 2019. The project financed the installation of distribution transformers and expansion of the distribution grid in the Sorobi district for the electrification of villages. There was no indicator in the results framework capturing the number of transformers installed and the length of the distribution grid expanded. The project was to finance the overhauling of Unit 2 because of the operational issues at this unit that required dismantling of the rotor shaft. The project had signed a contract for the overhauling of this unit, but the works could not be implemented because of the Taliban takeover. Outcomes As a result of the rehabilitation of Unit 1 and the overhauling of Unit 3, the Naghlu Hydropower Plan was able to generate electricity at its 100 percent capacity for the first time in March 2019. The power plant generated 436 Gigawatt-hour (GWh) electricity in 2019, and 464 GWh in 2020, which correspond to 49.77 percent and 52.97 percent capacity factors, respectively. These capacity factors are assessed to be reasonable given that hydroelectrical components of the power plant were designed in the 1960s during the Soviet era (PAD, pp.66-67). Although the results framework did not include any indicator capturing the increase in the electricity generation at the Naghlu Hydropower Plant following the rehabilitation works, the PAD (Figure 5.2. Page 11 of 28 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Naghlu Hydropower Rehabilitation Project (P132944) Generation at Naghlu 1968-2010 on p.66) gives the average amount of power generated annually by the power plant in the first decade of 2000s at 250 GWh, occasionally increasing above 300 GWh, but with significant deviations from the average. Therefore, the achievement of annual power generation of 436 GWh and 464 GWh in 2019 and 2020, respectively, are assessed to be significant. The project team confirmed that there was no generation data for the period between 2011 and 2018, and a comparison of the project’s achievement before and after the project was not possible. Total number of unplanned power outages due to deficiency of the Naghlu Hydropower Plant. The baseline in 2015 was 100 outages per year attributable to the power plant. The target was to reduce the number of outages caused by the hydropower plant to 20. The achievement between June 2020 and August 2021, when project activities stopped because of the Taliban takeover, was one outage. Number of villages in the immediate vicinity of the dam provided with access to electricity by the project. Because of the installation of distribution transformers and expansion of the distribution grid, the project expanded the electricity distribution infrastructure to 15 villages. However, not all 15 villages, nor households in those villages, had access to electricity. According to the Third-Party Monitoring Agent’s report in May 2022 (ICR, p.13), only in four villages all households had access to electricity. Additionally, electricity would be accessible to households in three villages if households could afford the cost of line-drops (the project did not finance the cost of line-drops). Yet, electricity was not accessible to households in the remaining eight villages—DABS has not electrified the distribution lines for four villages in the downstream of the Naghlu Hydropower Plant, and households in the four villages in the upstream of the hydropower plant are yet to purchase meters for line-drops. The target was to electrify eight villages. The achievement was seven villages where not every household is not connected to the distribution grid. The project provided electricity to 534 households, which have electricity between 18 and 24 hours a day. The grid is accessible for immediate connection of 1,334 households if they pay the cost of drop- lines. An additional 370 households could have access to the grid if grid extension works were completed. The results framework did not include an indicator capturing the achievement of access to electricity at the household of population levels. The project was successful in increasing the amount of electricity generated by the Naghlu Hydropower Plant through the rehabilitation works. The amount of electricity generated by the power plant increased from an average of 250 GWH per year to above 400 GWh depending on the availability of water in its reservoir. Since the power plant supplies electricity to Kabul, this indicates an increased availability of electricity for the consumers in the capital. The unplanned power outages caused by deficiency at the hydropower plant decreased from a high of 100 per year to only one. This is a significant increase in the reliability of electricity supply, but information is insufficient to assess the impact of this improvement on the customers. If outages because of the dilapidated transmission and distribution networks are still frequent, the reduction in the number of unplanned outages caused by the Naghlu Hydropower Plant might not have improved the overall reliability of electricity supply in the system. Additionally, there is no evidence that increased electricity supply from the Naghlu Hydropower Plant had an impact on the affordability of electricity or not. Lastly, the project was successful in extending the electricity distribution grid to the villages in the project area, but the cost of line-drops (which was not covered by the project) was a barrier to increase households access to electricity. Overall, the efficacy of the project in achieving the project objective to increase supply of domestically generated hydroelectricity at the Naghlu Hydropower Plant is rated Substantial. Page 12 of 28 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Naghlu Hydropower Rehabilitation Project (P132944) Rating Substantial OBJECTIVE 2 REVISION 1 Revised Objective To increase the supply of electricity at the Naghlu Hydropower Plant. Revised Rationale For project outputs and outcomes please see Objective 2 above. The formulation of this objective was revised in June 2021 only two months prior to the Taliban takeover in August 2021. There was no material change in the project objective before and after the revision. Since the project could not disburse any grant during the two-month period, the project’s efficacy in achieving the revised Objective 2 is assessed based on the achievements prior to the Taliban takeover. By the time the project was restructured in June 2021, the project had already achieved the rehabilitation of Unit 1 and overhauling of Unit 3 totaling 50 MW. At the project restructuring, the target was revised to 86.50 MW including rehabilitation of 11.50 MW of the Darunta Hydropower Plant (see Revised Objective 4 below) and overhaul of 25 MW Unit 2 of the Naghlu Hydropower Plant. The project could not achieve the revised target because project activities were halted following the Taliban takeover in August 2021. At the same project restructuring, the number of villages to be electrified by the project was increased from eight villages to 111 villages (29 villages in Sorobi, 39 villages in Tagab phase 1, and 43 villages in Tagab phase 2). Because of the Taliban takeover, the project could not achieve this target except the seven villages that had already been electrified prior to the project restructuring. According to the report of the Third-Party Monitoring Agent in May 2022, the project outcomes of increased power generation at the Naghlu Hydropower Plant has been sustained after the Taliban takeover in August 2021. The personnel at the powerplant have been conducting regular O&M activities. For example, the powerplant staff identified around 200 issues in the generation units in 240 inspections they carried in 2021, and they addressed all these O&M issues (ICR, p.13). Currently, the power plant continues to generate electricity to Kabul. Overall, the efficacy of the project in achieving the revised project objective to increase the supply of electricity at the Naghlu Hydropower Plant is rated Substantial for the post-Taliban takeover period. Revised Rating Substantial OBJECTIVE 3 Objective Page 13 of 28 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Naghlu Hydropower Rehabilitation Project (P132944) The project did not have a third objective before the Taliban takeover. This entry is inserted here to assess the new two objectives related to the Darunta Hydropower Plant in the post-Taliban takeover period as revised objectives. Rationale Please see above. Rating Not Rated/Not Applicable OBJECTIVE 3 REVISION 1 Revised Objective To improve the dam safety at the Darunta Hydropower Plant. Revised Rationale Outputs Rehabilitation of spillway gates in Darunta Dam. Because of the Taliban takeover, the project could not implement this activity. Geotechnical investigation of Darunta Dam. Similar as above, the project could not implement this activity because of the Taliban takeover. Outcomes Because the project could not implement the project activities related to the safety of the Darunta Hydropower Plant dam, the expected outcomes did not materialize. Therefore, the project’s efficacy in achieving the project objective to improve dam safety at the Darunta Hydropower Plant is rated Negligible. Revised Rating Negligible OBJECTIVE 4 Objective The project did not have a third objective before the Taliban takeover. This entry is inserted here to assess the new two objectives related to the Darunta Hydropower Plant in the post-Taliban takeover period as revised objectives. Rationale Please see above. Page 14 of 28 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Naghlu Hydropower Rehabilitation Project (P132944) Rating Not Rated/Not Applicable OBJECTIVE 4 REVISION 1 Revised Objective To increase the supply of electricity at the Darunta Hydropower Plant. Revised Rationale Outputs Hydropower generation capacity rehabilitated under the project. At the time of the project restructuring in June 2021, the project had already completed the rehabilitation of Unit 1 and the overhaul of Unit 3 of the Naghlu Hydropower Plant (see Objective 2 and revised Objective 2 above). This new indicator had a target of 86.5 MW generation capacity to be rehabilitated consisting of Unit 2 of the Naghlu Hydropower Plant and all three units of the Darunta Hydropower Plant totaling 11.5 MW. Because of the Taliban takeover in August 2021, the rehabilitation of the Darunta Hydropower Plant could not start although the contractor was ready to mobilize. The target was not achieved. Rehabilitation of Darunta switchyard. The project was to rehabilitate the switchyard at the Darunta Hydropower Plant but could not complete this activity because of the Taliban takeover. After the restructuring, the project targeted electrifying 111 villages in the project area (see Revised Objective 2 above). At the time of the Taliban takeover in August 2021, two bids for the electrification of 12 additional villages in the Sorobi district and 39 villages in the Tagab district as Phase 1 were in the final stage of bid evaluation, and the third bidding document for Tagab district phase 2 for the electrification of 43 villages was about to be finalized. Because of the Taliban takeover, the project could not implement these activities. Outcomes As the project could not implement any project activity related to the rehabilitation of the Darunta Hydropower Plant and the electrification of the additional villages, the project could not achieve the outputs expected from these activities, and the expected outcomes did not materialize. Therefore, the efficacy of the project in achieving the project objective to increase the supply of electricity at the Darunta Hydropower Plant is rated Negligible. Revised Rating Negligible OVERALL EFF TBL OBJ_TBL OVERALL EFFICACY Rationale Because of significant delays in the procurement of civil works and hiring of consultants related to the dam safety and sustainability of hydropower at the Naghlu Hydropower Plant, the project could only partially Page 15 of 28 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Naghlu Hydropower Rehabilitation Project (P132944) achieve its outputs and outcomes related to dam safety. However, the bidding document preparation and bidding evaluation for such works and hiring of consultants were on track, and project activities would have been expected to be completed by the revised target date of August 15, 2022 if the Taliban takeover had not happened. Therefore, the project’s efficacy in achieving the project objective to improve dam safety and sustainability of hydropower at the Naghlu Hydropower Plant is rated Substantial with moderate shortcomings for the pre-Taliban takeover period. The project was successful in completing the project activities to improve the generation capacity at the Naghlu Hydropower Plant by 2019. This resulted in an increase in the electricity generated at the power plant and a significant reduction in unplanned power outages caused by deficiency at the hydropower plant from 100 per year to only one by August 2021. However, evidence is insufficient to adequately assess the impact of increased availability and improved reliability of electricity supplied by the Naghlu Hydropower Plant on the customers. The evidence is also insufficient regarding the impact of the achievements on the affordability of electricity supply. The project was also partially successful in expanding the access of the villages in the project area to electricity. Therefore, the project’s efficacy to increase supply of domestically generated hydroelectricity at the Naghlu Hydropower Plant is rated Substantial. Overall, the project’s efficacy to achieve the project activities before the Taliban takeover in August 2021 is rated Substantial with moderate shortcomings. Overall Efficacy Rating Substantial OBJR1_TBL OVERALL EFFICACY REVISION 1 Overall Efficacy Revision 1 Rationale After the Taliban takeover in August 2021, DABS did partially continue with activities to improve the safety of the Naghlu Hydropower Plant dam, and major dam safety activities were not implemented. The parts that are rehabilitated under the project still work as reported by the Third-Party Monitoring Agent in May 2022, and the hydropower plant personnel continue with regular inspection and maintenance of the power plant and the dam. However, the sediment in front of the intake gates could not be removed, which poses a serious risk to operation of the hydropower plant and the security of the dam. It is highly likely that the hydropower plant will be out of operation within the next two to four years because of the further accumulation of the sediment if the flushing system that was cleared under the project fails to flush the sediment. Therefore, the project efficacy in achieving the revised project objective to improve dam safety at the Naghlu Hydropower Plant is rated Modest for the post-Taliban takeover period. According to the Third-Party Monitoring Agent report, the hydropower plant continues to generate electricity using its all four units, and the hydropower plant is regularly maintained by the DABS personnel. While the project could not achieve its revised target of 111 villages electrified under the project, the efficacy of the project in achieving the revised project objective to increase supply of electricity at the Naghlu Hydropower Plant is rated Substantial for the post-Taliban takeover period. The project was restructured in June 2021 to add project activities for the improvement of the dam safety and the increase in the supply of electricity at the Darunta Hydropower Plant. The project was at the bidding Page 16 of 28 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Naghlu Hydropower Rehabilitation Project (P132944) documents preparation stage for the contracts related to the works at the Darunta Hydropower Plant when the Taliban took power in the country; the project could not implement any activity related to the dam safety and power generation at the Darunta Hydropower Plant. Therefore, the project’s efficacy in achieving these two objectives is rated Negligible. Overall, the project’s efficacy in achieving the project objectives in the post-Talban takeover period is rated Modest. Overall Efficacy Revision 1 Rating Primary Reason Modest External shock 5. Efficiency Economic Analysis A “with the project” and “without the project” economic analysis was conducted at appraisal. Through Monte Carlo simulation, it was shown that the Naghlu Hydropower Plant would have been out of operation by 2021 if rehabilitation and dam safety works had not been implemented (PAD, p.64). The main economic benefit of the project was assumed to be the displacement of imported electricity and diesel self-generation during peak hours by the incremental increase in the electricity generated at the Naghlu Power Plant. The increased electricity supply from the hydropower plant would have avoided the construction of a new diesel-fired power plant to supply electricity to Kabul or replace the electricity generated by gas-fired power plants and imported from Uzbekistan to meet the growing demand in Kabul. When the future imports of hydropower through the CASA- 1000 included in the calculations, including an overall three precent differential transmission losses, the avoided cost of imported electricity because of the incremental electricity supply increase at the Naghlu Hydropower Plant was estimated to be at US$0.066 per kWh. The avoided cost of diesel self-generation was taken at US$1.09, the variable cost of diesel fuel in Kabul at the time of appraisal, corresponding to an avoided cost of US$0.327 per kWh. These were conservative estimates as the cost of imported electricity and diesel-generated electricity were likely to increase because of increases in the prices of natural gas and diesel, and increased loss of electricity during transmission of the imported electricity. After the rehabilitation, the hydropower plant was expected to gradually increase its electricity generation reaching at 438 GWh in 2020 and gradually declining at the absence of another life-extension rehabilitation and abandonment of the hydropower plant in 2028 (PAD, p.67). The reduction in greenhouse gas emissions was valued at US$30 per ton because of the displacement of gas and diesel-fired power generation. The costs were taken as the total rehabilitation cost of US$118 million, including US$32.6 million, the cost of rehabilitation implemented under the ERPR. Overall, the assumptions used in the economic analysis were realistic and sufficient to assess the economic viability of the project. Excluding the benefits from a reduction in the GHG emission, the calculations resulted in an Economic Rate of Return (ERR) of 42.6 percent with a Net Present Value (NPV) US$103 million at a discount rate of 10 percent with a project lifetime of 15 years. When benefits from a reduction in GHG emissions are included, the ERR increases to 52.6 percent with a NPV of US$140 million. These results show that while the benefits from the project were estimated conservatively—the socio-economic benefits of increased availability and improved reliability of electricity supply were not included in the economic analysis—the ERRs and NPVs show that the project was economically viable. Page 17 of 28 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Naghlu Hydropower Rehabilitation Project (P132944) At project closing, an economic analysis was conducted that was limited with the benefits achieved from the activities that were substantially completed before the Taliban takeover in August 2021. The analysis included benefits achieved from the rehabilitation of Unit 1 and overhauling of Unit 3, and different from the analysis conducted at appraisal, it also included benefits from the electrification of the villages in the project area. It is estimated that the rehabilitation and overhauling works, including those implanted at Unit 4 that were out of the scope of the project, extended the economic life of the Naghlu Hydropower Plant by 15 years to 2034. However, the analysis does not take into consideration the accumulation of sediment in the reservoir basis that is expected to block the intake gates within two to four years putting the hydropower plant out of operation unless the flushing system works sufficiently to flush the sediment. The project could not implement the sediment removal activity because of delays in hiring consultants for the dam and reservoir inspection and the abrupt ending of project activities after Taliban takeover. The economic analysis valued the incremental power produced by the power plant at the marginal cost of electricity imports from Uzbekistan. The replacement of diesel-fired self-generation at peaks hours was not included in the analysis. Regarding the electrification of the villages, the difference between the consumers’ average willingness to pay for electricity and the incremental cost of supply as additional imports was taken as the project’s benefits to the consumers based on a household energy use survey conducted between 2016 and 2019. The costs were taken as the total rehabilitation cost of the Naghlu Hydropower Plant and expansion of the distribution grid. Additionally, O&M cost was included in the economic analysis. The after-project economic analysis resulted in an ERR of 42.9 percent and a NPV of US$59.6 million at a 10 percent discount rate for 15 years. When the reduction in greenhouse gases expected from the replacement of gas-generated electricity from Uzbekistan by hydropower from the Naghlu Hydropower Plant is included in the analysis with different carbon prices, the ERR ranges between 58 and 70 percent. Overall, the post-project economic analysis shows that the project was viable, but the economic analysis is built on the assumption that the hydropower plant will continue generating electricity for another 15 years. However, the accumulation of sediment in the reservoir poses a significant risk to the operation of the project. The project could not implement the sediment removal activity and other dam safety activities. Therefore, despite the significant technical improvements at the power plant that increased the amount of electricity generation, it is highly unlikely that the power plant will be able to continue generating electricity in the medium-term because of sediment accumulation in front of the intake gates; the benefits expected from the rehabilitation of the hydropower plant might not materialize as expected. Financial Analysis At appraisal, the project’s financial benefits to DABS were assumed to come from the cost of avoided electricity imports and the additional sales revenue from electricity that could be supplied during off-peak hours to the consumers who had to use diesel self-generation. Since the grant was on-lent to DABS by the Ministry of Finance under regular IDA terms, DABS’ debt service obligations of the on-lent funds were included in the financial analysis. The calculations showed that DABS significantly benefits from the project because of incremental cash flows and achieves a consistent surplus with an NPV of US$83 million. A Financial Rate of Return was not calculated at appraisal. A financial analysis was not conducted after project closing. Administrative and Operational Efficiency The implementation of the project activities related to the rehabilitation of the units at the Naghlu Hydropower Plant was efficient. This is reflected in a high ERR because of the benefits achieved as a result of the Page 18 of 28 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Naghlu Hydropower Rehabilitation Project (P132944) completion of the rehabilitation activities. However, the project could not efficiently implement the project activities related to the dam safety including the removal of sediment because of significant delays in procurement caused by lengthy time required to form the bid evaluation committees and prepare the shortlists and bid evaluation reports. The gaps in the reports also delayed the procurement process along with the delays in the decisions of the National Procurement Authority. The sediment issue poses a high risk to the sustainability of electricity generation at the power plant as explained above under the Economic Analysis section. The worsening security situation in the country adversely affected the project implementation efficiency. This prevented site visits by experts for dam safety audits and eventually resulted in the cancellation of project activities related to major dam safety activities at the Naghlu Hydropower Plant. The project could not even start the implementation of the rehabilitation and dam safety activities at the Darunta Hydropower Plant because of the Taliban takeover in August 2021, nor the electrification of 111 villages in the project area. Although the economic analysis resulted in a high post-project ERR and an NPV, the likelihood that the project benefits will be sustained is low because of dam safety issues (i.e., accumulation of sediment in front of the intake valves that is expected to put the hydropower plant out of operation within two to four years). The significant delays in procurement of dam safety contracts under the second component resulted in incomplete project activities. Lastly, the project’s implementation efficiency was adversely affected by the changing political situation in the country as a result of the Taliban takeover in August 2021. Overall, the project’s efficiency in achieving the project objectives is rated Modest. Efficiency Rating Modest a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 100.00 Appraisal 42.60 Not Applicable 100.00 ICR Estimate 42.90 Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome The project objective was highly aligned with the World Bank strategy and relevant to the country context at the time of project closing. Therefore, the relevance of project objective is rated High. A split assessment was undertaken to evaluate the project’s efficacy in achieving the project objective before and after the Taliban takeover in August 2021 and with respect to the revision in the project objective at the project restructuring in June 2021. Despite some delays related to the implementation of the dam safety activities, the project substantially completed the rehabilitation and overhauling of Units 1 and 3 of the Naghlu Hydropower Plant and Page 19 of 28 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Naghlu Hydropower Rehabilitation Project (P132944) was on track to achieve the project outputs and outcomes related to dam safety by the revised project closing date prior to Taliban takeover. Therefore, the project’s overall efficacy in achieving the project objective is rated Substantial for pre-Taliban period. However, after the abrupt cessation of the project activities following the Taliban takeover, the project could not complete remaining activities related to the Naghlu Hydropower Plant, nor start the rehabilitation and dam safety activities at the Darunta Hydropower Plant, which were added to the project scope at the project restructuring in June 2021. Therefore, the expected outcomes did not fully materialize for the Naghlu Hydropower Plant and non for the Darunta Hydropower Plant. Hence, the project’s revised overall efficacy in achieving the project objective is rated Modest for post-Taliban takeover period. Because of the high risk to the sustainability of the project’s benefits due to the incomplete dam safety activities at the Naghlu Hydropower Plant and the significant delays in procurement of project activities under the second component, the project’s efficiency in achieving the project objective is rated Modest. These ratings result in an Outcome rating of Moderately Satisfactory for pre-Taliban period and a Moderately Unsatisfactory rating for post-Taliban period in accordance with the World Bank guidance (p.37). According to the split rating methodology adapted specifically for the Afghanistan portfolio, the project’s overall outcome is rated Moderately Unsatisfactory. The calculation of the split rating is given in Table 1 below. Table 1: Split Rating Calculation Before Taliban Takeover After Taliban Takeover Relevance of Objectives High Efficacy Substantial Modest Efficiency Modest Outcome Rating Moderately Satisfactory Moderately Unsatisfactory Outcome Rating Value 4 3 Amount Disbursed/Undisbursed 22.60 (disbursed) 60.40 (undisbursed) (US$ million) Disbursement (%) 27.23% 72.77% Weight Value 1.0892 2.1831 Total Weights 3.2723 (rounds down to 3) Overall Outcome Rating Moderately Unsatisfactory (3) a. Outcome Rating Moderately Unsatisfactory 7. Risk to Development Outcome Technical: The project resulted in a significant increase in the generation capacity of the Naghlu Hydropower Plant, but the sustainability of hydropower depends on the completion of the overhauling of Unit 2 of the power plant and the removal or sustained flushing of the sediment in the reservoir. If the sediment is not removed or flushed regularly, it is expected that it will accumulate and block the intake gates within two to four years, after which the hydropower plant will be out of operation. Furthermore, the hydropower plant dam has not been fully assessed for safety, and because of its old Soviet design, flood discharge system does not meet the current international dam safety standards. The collapse of the dam during floods cannot be ruled out, which would endanger the safety of the people living in the downstream communities. All these technical issues pose an extremely high risk for the sustainability of dam and the development outcomes. Page 20 of 28 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Naghlu Hydropower Rehabilitation Project (P132944) Financial: It is not clear whether DABS will have sufficient funds to adequately operate and maintain the Naghlu Hydropower Plant under the new regime in the country. According to the Third-Party Monitoring Agent’s report in May 2022, DABS continued to adequately inspect and maintain the hydropower plant. The hydropower plant continues to supply electricity to Kabul subject to the availability of water. However, the risk is high that DABS’ financial situation may be insufficient to sustain the operation and maintenance of the hydropower plant. Political: The current political situation poses a high risk for the sustainable of electricity generation at the Naghlu and Darunta Hydropower Plants. A total collapse of authority could result in the total neglect of these important hydropower plants. Although the power plants survived decades of armed conflict because warring parties intentionally avoided causing damage to large infrastructure such as the Naghlu Hydropower Plant, a resumption of armed conflict may cause damage to the power plants or result in improper inspection and maintenance of the power plants shortening their expected operation lives. 8. Assessment of Bank Performance a. Quality-at-Entry At the time of project preparation, the improvement of the energy sector was a high priority for the Government of Afghanistan and a critical part of its national development program. As the Naghlu Hydropower Plant supplied more than half of the electricity consumed in Kabul, the rehabilitation of the power plant was of high strategic importance to ensure the sustainability of electricity supply. The project design benefited from the extensive experience gained during the rehabilitation of the power plant under the EPRP. It was to complement the work done under the EPRP, establish sustainable O&M practices, and improve the safety of the dam. The technical aspects of the project were adequate. The inspections and assessments conducted during appraisal, such as dam safety assessment and sediment investigation, provided sufficient information about the general scope of work, which would be further improved through detailed inspection of the power plant and full safety audit of the dam during implementation. The design of the project components was sufficient to achieve the project objectives; however, the exclusion of the financing of the line-drops to the households that were to be electrified under the project was a moderate shortcoming in the project design. At appraisal, the project was already at the bid evaluation stage for the rehabilitation of Unit 1 and the overhauling of Unit 3. The economic analysis was robust; the calculations showed that the project was economically viable although the benefits and costs were defined conservatively. Implementation arrangements of the project were adequate. DABS had the overall responsibility for project implementation and appointed a project manager for day-to-day project management. The project had already started providing training to DABS personnel to strengthen the utility’s project implementation capacity. The fiduciary aspects of the project were also adequate. The safeguards policies that were triggered by the project were adequately identified. The project was designed to ensure that the communities in the project area economically and socially benefited from the project through the expansion of the electricity distribution grid and vocational training. The lessons learned during the implementation of previous projects in Afghanistan were adequately included in the project design such as early start of procurement, allocation for contingencies to cover cost increases, ensuring the support of the local community, and establishment of panel of experts for dam safety and environment impact management. The risks were identified in detail, and mitigation measures were in place such as the hiring of Third-Party Monitoring Agent for project Page 21 of 28 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Naghlu Hydropower Rehabilitation Project (P132944) implementation supervision, provision of training for capacity development in project implementation, financial management, procurement, and safeguards policy implementation. The political and governance risk was rated high, and it was beyond the project’s control to mitigate that risk. The monitoring and evaluation (M&E) system as designed was broadly sufficient to test the causal links in the project’s theory of change, but indicators were insufficient to capture the outcomes expected from the project’s intervention. Overall, the World Bank’s performance in identifying, facilitating the preparation of, and appraising the project at entry is rated Substantial. Quality-at-Entry Rating Satisfactory b. Quality of supervision The World Bank’s project team held in-person missions twice a year until the onset of the COVID-19 pandemic in March 2020, after which the project team held virtual supervision missions. The presence of some members of the project team in Kabul facilitated frequent contacts with the Ministry of Energy and DABS; the Kabul-based project team held monthly meetings with DABS to review progress in project implementation and provide support on technical, fiduciary and safeguards issues. A review of the Aide Memoirs and Implementation Status and Results Reports (ISRs) showed that the project team closely monitored the project implementation, which was adequately reflected in these documents. The candor and quality of performance reporting was high. The project team’s focus on the development impact of the project was sufficient. The project had successfully completed most of the rehabilitation works at the Naghlu Hydropower Plant. However, there were some moderate shortcomings in the supervision of the implementation of project activities related to the dam safety. These activities were delayed because of procurement issues that were partly beyond the control of the project team (see section 10.b. Fiduciary Compliance below) and led to an extension of the project closing date. The project team was proactive in revising the project scope to include the rehabilitation of the Darunta Hydropower Pant and improve the safety of its dam, but the project was restructured substantially late in the project implementation (in its sixth year) when disbursement rate was at around 27 percent of the total commitment. Although the project activities related to the Naghlu Hydropower Plant dam safety and those related to the rehabilitation and dam safety of the Darunta Hydropower Plant were on track for completion by the revised project closing date, they could not be implemented because of the Taliban takeover in August 2021, only two months after project restructuring. The project team’s supervision of fiduciary and safeguards aspects of the project was adequate. However, the project team did not use the only project restructuring as an opportunity to improve the quality of the results framework to better capture the project outputs and outcomes (see Section 9.b M&E Implementation). Overall, the quality of the project team’s supervision of the project is rated Moderately Satisfactory. Quality of Supervision Rating Moderately Satisfactory Page 22 of 28 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Naghlu Hydropower Rehabilitation Project (P132944) Overall Bank Performance Rating Moderately Satisfactory 9. M&E Design, Implementation, & Utilization a. M&E Design While the project objectives were output-oriented, they were clearly specified and sufficiently challenging for an intervention in a FCV status country. The causal links from project activities to the achievement of project outputs and outcomes were direct and valid, but the indicators did not sufficiently capture the project outputs and outcomes. For example, the results framework did not include an indicator measuring the increase in the amount of electricity generated at the Naghlu Hydropower Plant before and after the project’s intervention related to the second objective. There was no specific indicator in the results framework to capture the achievement related to the improvement in dam safety because the scope of these activities was to be determined during project implementation following a full dam safety inspection. Some indicators recorded the achievement of the targets as “Yes” or “No,” such as “Overhaul of Unit 3 completed” and “Dam safety procedures developed and maintained.” These indicators were not measurable. The indicators related to the achievement of project outputs and outcomes for the second objective were measurable, time-bound, relevant, achievable, and specific, such as “Installed capacity of Unit 1 ready for utilization” and “Total number of unplanned power outages due to deficiency of the Naghlu Hydropower Plant.” The indicators were insufficient to capture the project’s impact on the electrification of the households in the project area. The M&E arrangements were in place. DABS was responsible for overall M&E implementation. A Third-Party Monitoring Agency (TPMA) to be hired under the project was to verify project implementation progress and the M&E data. b. M&E Implementation DABS collected M&E data and reported them bi-weekly. At the time of project restructuring in June 2021, the results framework was revised to include new indicators related to the increased project scope (i.e., addition of the rehabilitation and dam safety of the Darunta Hydropower Plant), but the weaknesses in adequately capturing the project outcomes, especially those related to the improvement in dam safety and the sustainability of O&M practices, were not addressed (Because of these shortcomings, the ICR introduced two new indicators to assess the achievements of the project: (a) Improved O&M of the Naghlu Hydropower Plant; and (b) Hydromechanical Rehabilitation and Selective Sediment Removal). The work of the Third-Party Monitoring Agent was critical in implementing M&E during project implementation and after the Taliban takeover in August 2021. The Third-Party Monitoring Agent was able to provide detailed information about the status of the Naghlu Hydropower Plant and the sustainability of the power generation at the power plant at the time of project implementation in May 2022. Given the partial improvement in the O&M capacity of the hydropower plant personnel, it is expected that the M&E functions and process will be sustained as long as DABS has sufficient funds to inspect and maintain the power plant. c. M&E Utilization Page 23 of 28 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Naghlu Hydropower Rehabilitation Project (P132944) DABS communicated the M&E findings to the Ministry of Energy and the World Bank project team regularly. The M&E findings led to the cancellation of project activities, such as the civil works to increase the height of the Naghlu Hydropower Plant dam. Following the partial assessment of the dam safety, the M&E findings led to a project restructuring to add the rehabilitation and dam safety of the Darunta Hydropower to the project scope. However, the M&E findings were insufficient to provide evidence of the achievement of project outcomes, as the weaknesses in the results framework were not addressed during project implementation. As explained in the previous section, the ICR adopted two new indicators to adequately assess the project achievements. There were some significant weaknesses in the M&E design and implementation making it difficult to test the links in the causal chain and assess the achievement of the stated objectives, which were addressed by the ICR through the introduction of two new indicators and additional evidence. Overall, the M&E quality is rated Modest. M&E Quality Rating Modest 10. Other Issues a. Safeguards At appraisal, the project was classified as Category A under Environmental Assessment (OP/BP 4.01) and triggered Natural Habitats (OP/BP 4.04), Physical Cultural Resources (OP/BP 4.11), Involuntary Resettlement (OP/BP 4.12), Safety of Dams (OP/BP 4.37), and Project on International Waterways (OP/BP 7.50) safeguard policies. The project did not trigger any additional safeguard policy at the time of project restructuring in June 2021 when rehabilitation and dam safety works related to the Darunta Hydropower Plant were added to the project scope. Environmental Assessment (OP/BP 4.01): The project was classified as Category A because of the potential severe environmental impact of the dam safety activities to be financed under the second component of the project, such as removal and disposal of sediment material, handling and disposal of unexploded ordnance, and the increase in dam height and storage that could result in flooding of more land requiring enhanced water quality control measures in the reservoir. The environmental impact of the hydropower plant activities to be financed under the first component were expected to be mostly noise, fust, storage, and handling and disposal of used oils and removes parts. The impact of the project activities under the third component was negligible as these activities consisted of extending the electricity grid to the limited number of villages in the project area. A framework approach was adopted “to ensure that the environmental and social issues associated with each component are fully analyzed and understood and that the requisite mitigation and other management measures were agree and put in place” (PAD, p.23). Consequently, DABS prepared an Environmental and Social Management Framework (ESMF) and Environmental and Environmental and Social Management Plans (ESMPs) for activities under Components A and C. A separate ESMP was also prepared for the Darunta Hydropower Power Plant dam safety works. The ESMF and the ESMPs were disclosed in the Afghanistan in the local language and on the World Bank’s InfoShop in English. The project could not mobilize consultants to complete the Environmental and Page 24 of 28 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Naghlu Hydropower Rehabilitation Project (P132944) Social Impact Assessment and the related ESMP for the sediment removal and dam safety works related to the Naghlu Hydropower Plant because of challenging security environment in the project area. Natural Habitats (OP/BP 4.04): The project triggered this policy because of the potential impact of the project activities on natural habitats around the reservoir because of an increase in the dam crest planned under the second component that would have resulted in flooding of additional land beyond the then present boundaries of the reservoir. As the project did not finance works to increase the height of the dam, the surface area of the reservoir did not change. Therefore, this safeguard policy was not triggered during project implementation. Physical Cultural Resources (OP/BP 4.11): The project triggered this safeguard policy because of the presence of graves and shrines in the project area, which would have been required to be displaced if the project had increased the height of the Naghlu Hydropower Plant dam. As the project did not finance works to increase the height of the dam, the surface area of the reservoir did not change. Therefore, this safeguard policy was not triggered during project implementation. Involuntary Resettlement (OP/BP 4.12): The project triggered this safeguard policy because of the possibility of land or other asset acquisition that could have been required to create temporary or permanent roads for sediment removal and the potential impact of the project on the livelihoods of people living downstream of the dam. A Resettlement Policy Framework was prepared, which would be applicable to project activities that required land acquisition, asset loss, or resettlement. The ICR does not report the implementation of this safeguard policy. The project team confirmed that the project activities did not require any involuntary resettlement of households or businesses and did not have an impact on the livelihoods of the people in the project area. Safety of Dams (OP/BP 4.37): The project triggered this safeguard policy because the safety management of the Naghlu Hydropower Plant dam was unsatisfactory, and the dam required urgent repairs. An independent Project Technical Advisory Panel was formed to advise on dam safety risks and revise the dam safety plan that was prepared during project implementation. However, an Emergency Preparedness Plan, a requirement under this safeguard policy, could not be completed because of the challenges faced in hiring experts following the end of the contract of the first group of experts. The adverse security situation in the country and the restrictions imposed following the onset of the COVID-19 pandemic were the main challenges in hiring qualified experts to reconstitute the Project Technical Advisory Panel and implement the safeguard policy at both Naghlu Hydropower Plant and Darunta Hydropower Plant dams. The project was non-compliant with the requirements of the Safety of Dams safeguard policy. Projects on International Waterways (OP/BP 7.50): The project triggered this safeguard policy as it was located on the Kabul River, which flows into Pakistan. In accordance with the requirement of this safeguard policy, the World Bank, on behalf of the Government of Afghanistan, notified the Government of Pakistan on April 14, 2015 about the project. The World Bank responded to the additional request of the Government of Pakistan about the project on September 28, 2105. Since the Government of Pakistan did not request any additional information nor corresponded with the World Bank regarding the project, the Regional Vice- President approved the start of the project negotiations. It was confirmed that the rehabilitation of the Naghlu Hydropower Plant and its dam would not adversely affect the quality or quantity of water flows to other riparian, and the project would not be adversely affected by the other riparian’s use of water (PAD, p.24). The project was complaint with the requirements of the Projects on International Waterways safeguards policy. Page 25 of 28 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Naghlu Hydropower Rehabilitation Project (P132944) b. Fiduciary Compliance Financial Management DABS had the overall responsibility for the project’s financial management. A senior financial management specialist hired under project headed the financial management department of the Project Implementation Unit (PIU) under DABS. The World Bank provided training on financial management to the staff of the PIU. A financial management manual was in place for the project. The PIU submitted interim financial reports on time and in line with the World Bank requirements. There was no overdue financial statement when project activities ceased following the Taliban takeover in August 2021. The Supreme Audit Office (SAO) carried out the annual audit of the project with the support of consultants hired under the project. The project team confirmed that the audit opinions of the SAO were unqualified, and the reports were acceptable to the World Bank. There were no known issues of corruption or misuse of funds associated with the project. Procurement Procurement was conducted in accordance with the World Bank procurement guidelines. Although the procurement of some contracts started even during project preparation, the project faced significant procurement delays related to the contracts under Component B. The causes of delays were as follows (ICR, p.23): (a) Delays in the formation of bid evaluation committees by DABS; (b) lengthy time required for the preparation of shortlists and bid evaluation reports; (c) gaps in the reports prepared by bid evaluation committees; and (d) delays in the decisions of the National Procurement Authority. Because of procurement delays, consultants and contractors could not start their work. These led to a project closing date extension, and when project activities abruptly ended because of the Taliban takeover in August 2021, most of the activities under the second component had not even started. Although there were some complaints received by the World Bank regarding how procurement was conducted, the ICR (p.23) reports that the investigation of these complaints did not discover non-compliance with the World Bank’s procurement policies and guidelines. c. Unintended impacts (Positive or Negative) None. d. Other None. 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment Page 26 of 28 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Naghlu Hydropower Rehabilitation Project (P132944) Moderately Moderately Outcome Unsatisfactory Unsatisfactory Moderately Bank Performance Moderately Satisfactory Satisfactory The M&E design had weaknesses in adequately capturing the project outcomes, and these weaknesses were not Quality of M&E Substantial Modest addressed during project implementation. To provide evidence, the ICR introduced two new indicators during project evaluation. Quality of ICR --- Substantial 12. Lessons This review has drawn three lessons based on the information in the ICR. In an FCV status country, prioritizing the rehabilitation and/or overhauling of large generation assets can deliver high economic results in a relatively short period. It is observed that despite decades long armed conflict, major generation assets in Afghanistan were significantly in good shape without any irreversible damage. Such assets do generally continue generating electricity even during heightened armed conflict periods. Therefore, Naghlu Hydropower Plant did not have any major damage to its dam or hydroelectric parts before the start of the project. The project completed the rehabilitation and overhauling the power plants units within the planned period, and the hydropower plant was able to use its all four units at the same time. Even after the Taliban takeover, DABS was able to continue with the rehabilitation of the flushing gates, which partially contributed to the improvement of dam safety and the sustainability of power generation. Therefore, in FCV status countries, following an initial adequate assessment, prioritizing large generation assets that have survived armed conflict can yield high economic results. In high security risk environments, hiring resident engineers familiar with legacy systems and equipment of the power plants can be more effective to deliver results in hydropower plant projects rather than trying to hire consultancy firms that might be reluctant to work in such environments. One of the main reasons for the delay in the implementation of the dam safety works was the reluctance of international engineering companies to visit the project site. The hiring of two resident engineers who were familiar with the legacy Russian systems and equipment at the Naghlu hydropower Plant was effective in delivering results both in the rehabilitation and overhauling of the units and the partial improvement of the dam safety through the rehabilitation of the flushing gates. These resident engineers were embedded in the hydropower plant staff and provided technical assistance to the DABS’ staff in hydropower plant operation and maintenance. Their technical expertise was also crucial in scoping the rehabilitation works both at Naghlu and Darunta Hydropower Plants and the preparation of the tender documents. Hiring of resident engineers was also significantly less expensive that hiring of engineering firms. Page 27 of 28 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Naghlu Hydropower Rehabilitation Project (P132944) The cost of line-drops to connect households to the distribution grid can be a major barrier to increasing access to electricity in rural areas. The project financed the expansion of the distribution grid to a limited number of villages in the project area, but the cost of the line-drops was excluded from the project financing. This proved to be a major bottleneck because the households did not have sufficient financial means to cover the cost of connecting to the grid. Therefore, although the project brought electricity to the villages, fewer households than expected were connected to the grid. This was a significant weakness in the project design that restricted the impact of the project on addressing the drivers of fragility through inclusive growth and social inclusion. 13. Assessment Recommended? No 14. Comments on Quality of ICR The ICR provides a candid, accurate, and substantiated set of observations that are aligned to the project objective. The narrative is supported by new evidence in addition to those in the results framework. The report follows most of the Bank Guidance and its Appendix L, Preparation of ICR in FCV Setting for IPF Operations. The ICR provides a complete critique of the project and is sufficiently evaluative; the interrogation of evidence was sufficient. The theory of change is well-articulated and informs the reader as how the project achievements were assessed. There is a logical linking and integration of the various parts of the report, and the results are mutually reinforcing. The entries in section V. Lessons and Recommendations respond to the specific experiences of the project, but they are mostly in the form of findings rather than lessons learned. Lastly, the section IV.B. Environmental, Social, and Fiduciary Compliance could have benefited from a more detailed discussion in accordance with the Bank guidance. a. Quality of ICR Rating Substantial Page 28 of 28