Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Ethiopia Geothermal Development Project (P133613) Report Number: ICRR0024299 1. Project Data Project ID Project Name P133613 Ethiopia Geothermal Development Project Country Practice Area(Lead) Ethiopia Energy & Extractives L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IDA-54680,TF-17206 30-Jun-2020 109,865,161.74 Bank Approval Date Closing Date (Actual) 29-May-2014 31-Dec-2023 IBRD/IDA (USD) Grants (USD) Original Commitment 178,500,000.00 24,500,000.00 Revised Commitment 130,370,000.01 24,500,000.00 Actual 109,983,120.47 24,313,750.05 Prepared by Reviewed by ICR Review Coordinator Group Maria Shkaratan Fernando Manibog Avjeet Singh IEGSD (Unit 4) 2. Project Objectives and Components DEVOBJ_TBL a. Objectives The Original Project Development Objective (PDO) was “to develop geothermal resources in the territory of the Recipient” (Project Agreement, page 6). The PDO was substantively the same but phrased slightly differently in the Project Appraisal Document (PAD): “to develop geothermal resources in Ethiopia” (PAD, page 5). The PDO was not revised. Page 1 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Ethiopia Geothermal Development Project (P133613) b. Were the project objectives/key associated outcome targets revised during implementation? Yes Did the Board approve the revised objectives/key associated outcome targets? Yes Date of Board Approval 27-Jul-2020 c. Will a split evaluation be undertaken? Yes d. Components 1. Original components Component 1 Aluto Geothermal Site Development (cost at appraisal: US$126.2 million; actual cost: please see section “Revised Components” below) was to finance goods and labor for drilling and testing of approximately 22 wells to establish the economic viability of the geothermal resources of the Aluto geothermal site. The site already had a small pilot geothermal plant, operational since 1998. Component 2 Alalobad Geothermal Site Development (cost at appraisal: US$25.8 million; actual cost: please see section “Revised Components” below) was to finance goods and labor for drilling and testing of approximately four wells to establish the economic viability of the geothermal resources and finalize a feasibility study of the Alalobad geothermal site. The site was in the area that was expected to contain multiple distinct geothermal reservoirs. Component 3 Drilling Rigs, Associated Accessories, and Spare Parts (cost at appraisal: US$63.5 million; actual cost: please see section “Revised Components” below) was to finance the acquisition of two modern diesel-electric full-size drilling rigs, with all needed tools, accessories, and spare parts. The rigs would enhance operational efficiency and significantly upgrade drilling capabilities. Component 4 Legal, Institutional, and Regulatory Framework Development (cost at appraisal: US$3.0 million; actual cost: please see section “Revised Components” below) was to finance the preparation of the Geothermal Development Policy and related legislation, technical assistance, advisory services, and capacity building on geothermal development to Ethiopian Electric Power (EEP) (the country’s generation and transmission utility), the Geological Survey of Ethiopia (GSE) and the Ministry of Water, Irrigation and Energy (MoWIE). The component would also finance Project management. Revised Components: The components were revised during Restructuring 1 of June 2020, reducing Project scope, as follows:  Component 2 (Alalobad Geothermal Site Development) was cancelled.  New Component 1 Aluto Geothermal Site Development (cost at restructuring: US$126.8 million; actual cost at closure: US$108.2 million) combined original Components 1 and 3 (Aluto Page 2 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Ethiopia Geothermal Development Project (P133613) Geothermal Site Development and Drilling Rigs, Associated Accessories, and Spare Parts), as both were to focus on Aluto site development. Financing was lowered due to a reduced scope.  New Component 2 Legal, Institutional and Regulatory Framework Development (cost at restructuring: US$5 million, actual cost at closure: US$1 million), comprised the original Component 4 activities and new activities, including the preparation of the geothermal sector strategy, strengthening the geothermal development institutions, training, and purchasing software for drilling. e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project Cost: The appraisal estimate was US$218.5 million, and the actual disbursement was US$109.2 million. The difference is explained by the cancellation of US$50 million of IDA funding at Restructuring 1 in June 2020 and of US22.6 million of IDA funding at Restructuring 4 in December 2023; revoked contributions from the Borrower and the Icelandic International Development Authority (ICEIDA); and the cancellation of the original Component 2 and the scale-down of original Components 1 and 3 at Restructuring 1. Project Financing: The Project was financed by the International Development Association (IDA) credit (appraisal estimate: US$178.5 million; actual amount at closure: US$85.6 million) and the World Bank- administered Strategic Climate Fund (SCF)/Scaling-up Renewable Energy Program (SREP) grant (appraisal estimate: US$24.5 million; actual amount at closure: US$24.4 million). Parallel financing from the ICEIDA was expected, but there was no contribution (appraisal estimate: US$3.5 million; actual amount at closure: zero). Borrower/Recipient contribution: The Borrower’s contribution was estimated at US$12.0 million at appraisal, but there was no Borrower contribution. Project Dates: The Project was approved on May 29, 2014, and became effective on August 5, 2014. The midterm (MTR) review was on February 28, 2017. The Project was restructured four times: (i) on June 5, 2020; (ii) on July 27, 2020; (iii) on December 21, 2021; and (iv) on December 13, 2023. The original closing date was on June 30, 2020. The Project was extended twice, for the total of three years and six months (or 42 months), to December 31, 2023, which was the date of the Project’s actual closure. Restructurings: The Project had four restructurings: Restructuring 1 (June 5, 2020) involved the following changes (in addition to the modification of components, as described above), reducing the Project scope:  Partial cancellation of IDA/SREP financing in the amount of US$50.0 million, to finance supplementary Budget Support to second Ethiopia Growth and Competitiveness Programmatic Development Policy Financing to mitigate the fiscal impact of COVID-19.  Reduction in Borrower contribution from US$12.0 million to US$8.0 million.  Reallocation of the remaining financing across components.  Changes to the results framework (RF), as follows: o Three PDO outcome targets were significantly reduced due to the encountered delays and insufficient time remaining until Project closure:  For PDO1 indicator “Geothermal wells drilled and tested”: from 26 wells to 14 wells; Page 3 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Ethiopia Geothermal Development Project (P133613)  For indicator “Geothermal wells with potential to generate electricity”: from 13 wells to eight wells;  For indicator “Geothermal sites confirmed”: from two sites to one site. o One PDO indicator was added: “Geothermal Sector Strategy Developed (yes/no)”. o Eight intermediate results indicators (IRIs) were marked for deletion and six new IRIs were added, in relation to the merging of original Components 1 and 3 and to the cancellation of original Component 2.  Extension of the closing date by 18 months, from June 30, 2020 to December 31, 2021, to allow time for Project implementation. Restructuring 2 (July 27, 2020) included adding a SREP indicator “Geothermal capacity confirmed for electricity generation, MW” to the Project’s RF, and reducing its target from 70 MW to 35 MW because after Restructuring 1 of June 2020, only Aluto site remained under the Project and, based on the newly available data, generation capacity estimate for Aluto was 45 MW. Restructuring 3 (December 21, 2021) was to extend the closing date by 24 months, from December 31, 2021, to December 31, 2023, to finalize wells’ drilling and testing and complete the geothermal resource assessment and the feasibility study for the geothermal plant. Restructuring 4 (December 13, 2023) was to cancel US$22.63 million from IDA commitment, at government’s request (US$18.63 million from Component 1 and US$4 million from Component 2). Split evaluation. Restructuring 1 of June 2020 and Restructuring 2 of July 2020 resulted in a reduction in the Project’s scope. While the financing was also decreased, it is unclear whether the PDO of developing geothermal resources was achieved, given the significant reductions in targets and incomplete activities, thus necessitating a split evaluation. The split will be applied as of Restructuring 2. 3. Relevance of Objectives Rationale Country and Sector Context. At Project appraisal, energy sector was suffering from insufficient electricity supply and a fast-growing demand. Considering the government’s goal to reach an 11 percent GDP growth over the next five years, achieve universal electricity access, and establish Ethiopia as a regional power hub, electricity needs were expected to grow by over 25 percent annually, requiring investments in supply infrastructure. At the same time, the country’s reliance on the rainfall-dependent hydropower, in the context of climate change, was calling for a diversification of electricity generation sources, including the development of large untapped geothermal resources. This was challenging considering limited geothermal resources exploration and development; insufficient institutional and regulatory frameworks, technical capacity, and financial resources; and limited private sector participation. The Project was to address the upstream challenges and support exploration, absorbing the high investment risk typical for this phase and enabling private sector participation in the following phases. Relevance to Government Strategies during implementation and at closure. The Project was aligned with government strategies, including the 2019 National Electrification Program 2.0 (NEP 2.0), which aimed at universal electricity access by 2025. NEP 2.0 stated that the government had “focused on aggressively Page 4 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Ethiopia Geothermal Development Project (P133613) diversifying its energy mix with wind, solar, and geothermal sources to complement the large base of hydro resources” (NEP 2.0, page 3), and discussed the ongoing geothermal exploration as high priority projects (including those supported by the World Bank) (NEP 2.0, page 4); it also emphasized the importance of geothermal development for climate mitigation and referred to financing from the Climate Investment Funds (NEP 2.0, page 124). Relevance to the WBG’s Assistance Strategies at closure. The Project was aligned with the objectives of the WBG’s Country Partnership Framework (CPF) for Ethiopia for FY2018-22, extended to FY2023-24, through Focus Area 1.2: “Increased access to reliably energy supply”. The Project was also aligned with the WBG’s Regional Integration and Cooperation Assistance Strategy for FY2021-23, applicable at Project’s closure, through Focus Area 1.1 “Supporting Energy Access and Regional Power Pools” and Pillar 4 “Reinforcing resilience”, which emphasized the development of low carbon energy systems. Previous sector experience and related projects. In 2010, the Bank facilitated geothermal development through the Energy Access Project (P120172 and P049395), which included a geothermal resources exploration component. The reviewed Project continued this task, and the results were to support the design of a power plant in Aluto, to be financed by private sector. To scale up the outcomes of the reviewed Project, a new multi-phase Bank project, Power Sector Reform, Investment and Modernization in Ethiopia (PRIME), was approved on March 26, 2024. The Project was in alignment with the WBG's CPF at closure and with the government objectives of geothermal energy development, and there was a continuity with the previous Bank operation, as well as a planned next phase operation. However, Project’s design was overambitious considering the country’s Project implementation capacity. On balance, the Relevance of Objectives rating is Substantial. Rating Relevance TBL Rating Substantial 4. Achievement of Objectives (Efficacy) EFFICACY_TBL OBJECTIVE 1 Objective PDO 1: to develop geothermal resources in the territory of the Recipient. It is worth noting that the formulation of the PDO was too broad and beyond the reach of the Project. Whereas the PDO aimed at developing geothermal resources, the planned (and later implemented) activities focused on testing to determine economic viability, not full development. There was therefore a mismatch between the formulation of the PDO and what the project intended to do on the ground. However, the Project Paper for Restructuring 1 (June 5, 2020) specified that the Project was to “be limited to the confirmation of geothermal resources for electricity generation” (Project Paper, page 10). This definition is consistent with the Project’s PDO outcome indicators and activities; therefore, efficacy analysis will be conducted using this definition. Page 5 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Ethiopia Geothermal Development Project (P133613) Rationale The theory of change (ToC) for the Project was not included in the PAD; it was prepared for the ICR (ICR, page 3). According to the ToC, the Project supported the following activities: (i) development of the Aluto and Alalobad geothermal sites; (ii) the acquisition of drilling rigs, accessories, and spare parts; and (iii) the development of legal, institutional, and regulatory framework. The expected outputs of these activities were: (i) two sites tested, four wells drilled, feasibility study completed; and (ii) four workshops held and 94 staff trained/retrained. The achievement of these outputs would ultimately result in the following outcomes: (i) 26 wells drilled and tested, two geothermal sites confirmed, 13 wells have potential to produce electricity, and 70 MW of geothermal capacity confirmed; (ii) geothermal strategy developed. This would enable private investment in geothermal sector. Critical assumptions were listed, as follows: technical assistance addresses public sector staff capacity; panel of experts helps to minimize the risk of unproductive wells; fiduciary shortfalls adequately addressed; and environmental and social safeguards adequately implemented. The ICR’s ToC had some shortcomings, as follows. Instead of depicting the logic of Project implementation (result chains) by explaining the links from activities to outputs, then to intermediate results, and to Project outcomes, it provided lists of Project components (column “Activities), IRI targets (column “Outputs/intermediate outcomes”), and PDO indicator targets (“PDO outcomes”). This made the chain of direct causality vague. For example, it is unclear how the development of the legal, institutional, and regulatory framework (listed as an activity) would produce the output of staff training, to which it is directly linked in the ToC chart; or how training and workshops would directly lead to the development of the geothermal strategy (as per the ToC chart). The overall objective ("the confirmation of geothermal resources for electricity generation", which is the clarified PDO used in the efficacy analysis (ICR, page 8-9)) is missing from the chart, and the ToC creates an impression that the Project-level overall objective was to enable private sector investment in geothermal energy, which is not how the Project and the RF were designed. In addition, the listed critical assumptions were selected on the basis of accomplished tasks, while missed opportunities were not included. For example, as the ICR stated, the implementation was negatively affected by a weak institutional setup: the implementing agency EEP, whose expertise was in hydropower sector, was not yet ready to oversee a geothermal project at approval (page 10); this observation could have been reflected in the TOC’s critical assumptions. The achievement of the Original Project’s results will be assessed using the RF indicators, as specified in the Project’s PAD: Outputs/Intermediate Outcomes: 1. The following eight Original Project IRIs were replaced at Restructuring 1 in June 2020, in relation to the revision of the geothermal development activities, and their achievement is unknown: “Drilling Contractor Contract signed, yes/no”; “Drilling consumables contract signed, yes/no”; “Supervision Engineer contract signed, yes/no”; “Number of sites tested”, “Number of test wells drilled”, “Tender floated (Rig and accessories), yes/no”; “Contract signed (Rig and accessories), yes/no”; “Rig delivered onsite, yes/no”. 2. The following five Original Project IRIs, all measuring capacity development, were monitored until Project closure: - “Training/Workshops held, number” (baseline: zero events, target: four events). The achievement at closure was two surveys; the target was 50 percent (partially) reached. Page 6 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Ethiopia Geothermal Development Project (P133613) - “EEP Staff trained, number” (baseline: 0 staff, target: 45 staff). The achievement at closure was 28 staff; the target was 62 percent (partially) reached. - “GSE/MoWIE/MoM staff trained, number” (baseline: 0 staff, target: 45 staff). The achievement at closure was zero staff; the target was not reached. - “EEP retrained procurement staff, number” (baseline: 0 staff, target: 4 staff). The achievement at closure was two staff; the target was 50 percent (partially) reached. - “Feasibility study completed, yes/no”. The study (for the development of geothermal power plant after Project closure) was outstanding at closure, therefore, the target was not reached. It was to be completed once drilling and testing of all targeted wells is done (by the follow-up PRIME project). It is worth noting that while the preparation of the feasibility study was measured by an IRI, it was a main Project outcome, a final outcome in the results chain and a direct input into the next step of the geothermal sector development: design and construction of a geothermal plant (which, however, was outside of the Project scope). Capacity building and institutional development activities were not limited to the ones measured by the RF. For example, a rules and regulations working group reviewed and developed rules for licenses for exploration, use and development, and competitive bidding and the drilling of deep geothermal wells; and a Geothermal Roundtable involving key government entities, development partners, and private entities was held to discuss possible roadmaps for developing the geothermal sector in Ethiopia. PDO outcomes: 1. "Geothermal wells drilled and tested, number” (baseline: 0 wells, target: 26 wells). The achievement at closure was 12 wells; the target was 46 percent (partially) reached. 2. “Geothermal wells with potential to generate electricity, number” (baseline: 0 wells, target: 13 wells). The achievement at closure was nine wells; the target was 69 percent (partially) reached. 3. “Geothermal sites confirmed, number” (baseline: 0 sites, target: 2 sites). The achievement at closure was 1 site; the target was 50 percent (partially) reached. Rating. The Original Project partially achieved its intended results, and its efficacy rating is Modest. Specifically, all three PDO targets (on geothermal development) and most of the IRI targets (on training) were achieved partially, the important IRI on the completion of the feasibility study was not accomplished, while other IRIs were dropped, and their achievement at closure is unknown. Rating Modest OBJECTIVE 1 REVISION 1 Revised Objective PDO 1: to develop geothermal resources in the territory of the Recipient (same as in the Original Project). Page 7 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Ethiopia Geothermal Development Project (P133613) As noted under Original Project, the PDO was formulated too broadly, however, it was clarified at Restructuring 1 (June 2020) that the Project was to “be limited to the confirmation of geothermal resources for electricity generation” (Project Paper, page 10). Efficacy analysis is conducted using this definition. Revised Rationale Please see the discussion of the ToC under Objective 1, Original Project. The achievement of the Revised Project’s results will be assessed using the indicators and indicator targets as specified during Restructuring 2 of July 2020, as follows: Outputs/Intermediate Outcomes: 1. Aluto Geothermal Site Development activities: - “Civil work and water supply construction finalized, yes/no”. The target was reached. - “First rig installed in Aluto site, yes/no”. The target was reached. - “Second rig arrival in Aluto site, yes/no”. The target was reached. - “First rig drilling operation starts, yes/no”. The target was reached. - “Second rig drilling operation starts, yes/no”. The target was reached. - “Geothermal capacity confirmed for electricity generation, MW” (baseline: 0 MW, target: 35 MW). The achievement at closure was 26 MW; the target was 74 percent (substantially) reached. Note: This indicator was added at Restructuring 2 of July 2020, to reflect the main SCF/SREP target. 2. Capacity development activities: - “Training/Workshops held, number” (baseline: zero events, target: four events). The achievement at closure was two surveys; the target was 50 percent (partially) reached. - “EEP Staff trained, number” (baseline: 0 staff, target: 45 staff). The achievement at closure was 28 staff; the target was 62 percent (partially) reached. - “GSE/MoWIE/MoM staff trained, number” (baseline: 0 staff, target: 45 staff). The achievement at closure was zero staff; the target was not reached. The ICR noted that the reason was that training activities were focused on capacity building at EEP (ICR, page 24). - “EEP retrained procurement staff, number” (baseline: 0 staff, target: 4 staff). The achievement at closure was two staff; the target was 50 percent (partially) reached. - “Feasibility study completed, yes/no”. The study (for the development of geothermal power plant after Project closure) was an important Project outcome, and it was outstanding at closure, therefore, the target was not reached. PDO outcomes: Page 8 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Ethiopia Geothermal Development Project (P133613) 1. "Geothermal wells drilled and tested, number” (baseline: 0 wells, target: 14 wells). The achievement at closure was 12 wells; the target was 86 percent (substantially) achieved. 2. “Geothermal wells with potential to generate electricity, number” (baseline: 0 wells, target: eight wells). The achievement at closure was nine wells; the target was exceeded. 3. “Geothermal sites confirmed, number” (baseline: 0 sites, target: 1 site). The achievement at closure was 1 site; the target was reached. 4. “Geothermal Sector Strategy Developed, yes/no”. The strategy was developed but not yet adopted by the government or the EEP; therefore, the target was partially reached. Rating. The Revision 1 reached or substantially reached all of its geothermal site development targets (for both PDO indicators and IRIs). At the same time, the PDO indicator target of geothermal strategy development was partially reached, the IRI target of feasibility study completion was not achieved, and the indicator targets for training were partially reached. On balance, the rating is Substantial, with moderate shortcomings. Revised Rating Substantial OVERALL EFF TBL OBJ_TBL OVERALL EFFICACY Rationale Original Project: For the Original Project, the rating for efficacy is Modest because the PDO outcomes on geothermal sites’ development, as well as the outputs on training, were achieved partially, while the important output of completing the feasibility study was not accomplished, and the achievement of several IRIs is unknown because they were dropped. Overall Efficacy Rating Primary Reason Modest Low achievement OBJR1_TBL OVERALL EFFICACY REVISION 1 Overall Efficacy Revision 1 Rationale For the Revision 1 Project, the rating for efficacy is Substantial, with moderate shortcomings. All geothermal site development targets were reached or substantially reached. At the same time, the geothermal strategy was not developed, the feasibility study was not completed, and the training targets were partially reached. Page 9 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Ethiopia Geothermal Development Project (P133613) Overall Efficacy Revision 1 Rating Substantial 5. Efficiency a. Economic Analysis: At appraisal, economic analysis produced the following outcomes: at ten percent discount rate, the economic internal rate of return (EIRR) was 30 percent, and the net present value (NPV) was US$167 million; and at 12 percent discount rate, the EIRR was 14 percent, and the NPV was US$73 million. The following inputs were used: discount rate of 10 percent and 12 percent; actual project costs; power plant’s and transmission lines’ construction cost, based on plant’s capacity; tariffs as per the Willingness to Pay (WTP) study, US$0.30/kWh; system losses from 22 percent to 18 percent; Project-related incremental electricity sales; average economic cost of supply at US$0.06/kWh; and operation and maintenance (O&M) cost at 2 percent of the capital cost. The benefits were: avoided cost of alternative fuels and the value of having access to electric appliances. Both were measured based on the Willingness To Pay (WTP) study. At closure, economic analysis produced the following outcomes: EIRR was eight percent, and the NPV ranged from a negative US$23 million (at ten percent discount rate) to a negative US$38 million (at 12 percent discount rate). The inputs used were as at appraisal, except for the updated plant’s and transmission lines’ construction cost and Project-related incremental electricity sales. The EIRR at closure was below the opportunity cost (discount rate), making the Project economically inefficient. The negative NPV supports this conclusion. Also, the NPV and EIRR at closure were lower than at appraisal. The ICR stated that this was due to delays in drilling; and also because of the lower than projected geothermal resource capacity (as explained in section 2.e, under “Restructuring 2”). b. Administrative Efficiency: Project implementation was expected to take 72 months (or six years and one month), but it was delayed by 42 months (or three years and six months). At the same time, the activities were significantly scaled down. It was estimated that an additional nine-months extension would be needed to fully reach the targets. In the first six years of implementation, prior to Restructuring 1 of June 2020, very little progress was achieved, mainly because of procurement delays due to inadequate staffing of the Project Implementation Unit (PIU) and a lengthy process of addressing this issue. Also, the original procurement strategy of separating rig ownership from drilling soon proved to be inefficient, but it took two years for the EEP and the government to agree to the Bank’s advice to combine the two. In addition, a setback occurred when one of the drilling rigs became unavailable due to inadequate maintenance by the implementing agency. Efficiency improved after Restructuring 1, but three months before closing, the Project was suspended due to a security incident at the Project site, effectively reducing implementation time. COVID-19 also led to delays. Also, the design of water supply infrastructure had to be modified due to a heavy rainfall in 2020, causing a delay with the drilling operation. (ICR, pages 12-13, 16-17, and 19) Page 10 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Ethiopia Geothermal Development Project (P133613) Considering the EIRRs below the opportunity cost of capital and the negative NPV at closure, as well as a reduction of the EIRR estimates from approval to closure, and also taking into account delays and cancellations in the context of reduced scope, Project’s efficiency is rated as Modest. Efficiency Rating Modest a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 98.60 Appraisal  14.00  Not Applicable 99.00 ICR Estimate  8.00  Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome Original Project: Revised Project: Relevance of objectives Substantial Efficacy Modest Substantial Efficiency Modest Outcome Moderately Unsatisfactory Moderately Satisfactory Outcome value 3 4 Amount disbursed, US$ million 42.07 67.1 Disbursement percentage 38.5% 61.5% Weight value 1.16 2.46 Total weight 3.61 Overall outcome rating Moderately Satisfactory The Relevance of Objectives is rated as Substantial. Original Project’s efficacy is Modest, and Revised Project’s efficacy is Substantial, with moderate shortcomings. Efficiency is Modest. Thus, the Outcome is rated as Moderately Unsatisfactory for the Original Project and Moderately Satisfactory for the Revised Project. Based on the shares of the disbursed funds before and after the second restructuring of July 2020 (US$42.07 million or 38.5 percent; and US$67.1 million or 61.5 percent), the overall Project outcome rating is Moderately Satisfactory* (0.385 * 3 + 0.615 * 4 = 3.61). Page 11 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Ethiopia Geothermal Development Project (P133613) *Based on a six-point scale, where: 1 = Highly Unsatisfactory, 2 = Unsatisfactory, 3 = Moderately Unsatisfactory, 4 = Moderately Satisfactory, 5 = Satisfactory, and 6 = Highly Satisfactory. a. Outcome Rating Moderately Satisfactory 7. Risk to Development Outcome Political. The Project was affected by a security incident at the Project site, and it was suspended as a result. Civil unrest might affect the scale-up and the following phases of geothermal sector development in the future. Economic. The ICR noted that the macroeconomic situation the country faces, especially the unstable local currency, poses a risk to private sector participation (ICR, page 20), which is expected at post-exploration stages of geothermal sector development. In this case, additional concessional finance will be needed. Government ownership or commitment. The Project was designed to support the upstream activities, specifically, geothermal exploration, absorbing the high risk typical for this phase. A scale-up of these activities was still needed at Project closing, to be followed by geothermal plants’ construction. There is a risk that the government's and EEP's commitment is not sufficient to continue their support of these activities. This risk was mitigated by the preparation and approval of the Bank’s PRIME project. Institutional. The Project was negatively affected by constrained institutional capacity for its implementation. This risk was partially mitigated by the Project through the capacity building activities; however, it still remains. The ICR noted that a proper institutional setup to lead geothermal development could mitigate such risk and that the establishment of a separate entity or a dedicated EEP department for geothermal development would be instrumental in increasing public sector knowledge and proving leadership to the program (ICR, page 20). 8. Assessment of Bank Performance a. Quality-at-Entry The Project’s design was generally adequate and accounted for risks. The Project supported the government objective of geothermal sector development by investing in the exploration stage, which has high upfront costs and high risk and therefore is not attractive for private sector. The design of the components was based on sectoral expertise and was part of the overall sector development strategy. The Project continued geothermal exploration efforts made under a previous Bank operation and was to provide inputs into the next phase of geothermal development (geothermal plant construction, to be financed by the Government of Japan). However, as the ICR mentioned, the Project scale at design was too ambitious considering capacity limitations in the country, specifically, the plan to drill 22 wells in Aluto and 4 wells in Alalobad was too Page 12 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Ethiopia Geothermal Development Project (P133613) demanding, and focusing on one site and a smaller number of wells at start could have been more efficient (ICR, pages 13, 15). Also, the Project commenced without a proper institutional setup for implementation: the implementing agency EEP, whose expertise was in hydropower sector, was facing a steep learning curve in geothermal development (page 10). At approval, the Project was not sufficiently ready for implementation due to inadequate capacity and systems in the PIU and implementing agency at effectiveness. (Page 19) Quality-at-Entry Rating Moderately Unsatisfactory b. Quality of supervision While in the first six years of implementation, until Restructurings 1 and 2 (June 2020 and July 2020), the progress with Project implementation was minimal, the situation significantly improved afterwards, and the Project was on a path to achieving its objectives by the (extended) closing date. This was due to a reduction in the Project scope, aligning its ambition with institutional capacity and expertise base. It was also a result of the adjustment of the implementation strategy and provided capacity building. However, In October 2023, there was a security incident due to civil conflict at the Project site, requiring evacuation of the personnel. The implementation was suspended three months before the closing date, effectively reducing the implementation period by three months, and not all expected results were reached. At the same time, implementation in the first six years was very inefficient: only two out of the targeted 26 wells had been drilled, no geological sites had been confirmed, and it had not been established if any of the two wells had potential to generate electricity. The main reasons, as mentioned in section 5.b. Administrative Efficiency, were internal: procurement shortcomings due to inadequate PIU staffing and the deficiencies of the implementation strategy. The ICR stated that the Bank should have been more assertive to ensure a timely adjustment of the implementation strategy. There also were external factors, such as a deteriorated security situation, COVID-19 impact, and a climatic event: exceptionally heavy rain that caused a delay in the drilling operation due to the need to modify the design of the water supply infrastructure (ICR, pages 15-17). Quality of Supervision Rating Moderately Satisfactory Overall Bank Performance Rating Moderately Satisfactory 9. M&E Design, Implementation, & Utilization a. M&E Design The RF reflected the logic of Project interventions as planned at appraisal and was sound, technical, and manageable. All RF indicators were time-bound and attributable to the Project. All PDO indicators and Page 13 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Ethiopia Geothermal Development Project (P133613) several IRIs were quantitative, while other IRIs were non-quantitative (binary, yes/no), which was appropriate. The PDO indicators were aligned with the Project objective as clarified in the Restructuring 1 Project Paper (see the explanation in section 4: the Project was to “be limited to the confirmation of geothermal resources for electricity generation”) and were specific, measurable, and relevant. While there was no PDO indicator to measure the institutional development aspect of the Project at design, it was added at Restructuring 1 in June 2020. Intermediate-level indicators were adequately selected to measure the contribution of activities and outputs toward the PDO outcomes. The RF was designed in consultations with the stakeholders and adjusted based on their feedback. M&E reporting was specified at appraisal: there would be annual reports on outcomes more detailed bi-annual reports. Due to the ToC deficiencies, the alignment between the ToC and the RF was weak, as described in section 4: instead of depicting the logic of Project implementation (result chains) by explaining the links from activities to outputs, to intermediate results, and to outcomes, the ToC listed Project components, IRI targets, and PDO indicator targets, making the chain of direct causality between successive results vague. b. M&E Implementation The RF was re-designed at Restructuring 1 in June 2020, to reflect the changes in the project scope and components. The revised RF was aligned with the restructured project, reflected all expected outcomes though PDO indicators, and the logic of achieving them through IRIs. The targets were adequately adjusted to balance Project ambition and existing institutional capacity, as well as the timeline of implementation. The ICR reported that annual and bi-annual reports contained the required data, although sometimes they were submitted late or needed clarifications. There was a case when unacceptable EEP financial audits were submitted, and it took four years to resolve this issue. Also, during the first year of implementation, progress reports were not in compliance with the covenant; however, they were in full compliance afterwards. At least two supervision missions were held per year, except for the first two years and the last year when annual missions took place. Aide memoires and ISRs were filed as expected. Apart from missions, the country-based Bank team members provided ongoing support. (ICR, page 18) c. M&E Utilization The ICR noted that there were shortcomings in M&E utilization that were responsible for the implementation delays and the non-achievement of the original outcome targets, specifically, a disagreement between the government and the Bank on whether the original strategy that separated procurement of drilling rigs and drilling services should be kept or replaced with the approach of combining rig ownership and drilling. Despite the data and analyses showing that the latter strategy would be more efficient, the Bank team was unable to ensure a timely adjustment of the strategy, and the change was made only two years later, a year before the original closing date. On the positive side, practical experience in resettlement and access restrictions resulted in timely adjustments to the Grievance Redress Mechanism. (ICR, page 18) Page 14 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Ethiopia Geothermal Development Project (P133613) Considering the sound RF design, both at approval and at Restructuring 1, but also the shortcomings in reporting during M&E implementation and in M&E utilization (the latter being responsible for implementation delays and the non-achievement of the original outcome targets), the M&E quality is rated as Modest. M&E Quality Rating Modest 10. Other Issues a. Safeguards Environmental and Social Safeguards. At appraisal, the Project was classified as Environmental Category A (Full Assessment). The following safeguards policies were triggered: Environmental Assessment (OP/BP 4.01), Natural Habitats (OP/BP 4.04), Physical Cultural Resources (OP/BP 4.11), Indigenous Peoples (OP/BP 4.10), Involuntary Resettlement (OP/BP 4.12). The ICR reported that the Environmental and Social Impact Assessment (ESIA) was prepared and disclosed on time, on March 13, 2013, and updated in 2020, when site-specific management plans were prepared. A Resettlement Action Plan (RAP) was prepared and disclosed in August 2019. An independent audit of May-July 2023 found that the implementation of the RAP was in reasonably good compliance but noted the need to improve trust- building, awareness of communities, and inclusion of women. The Overall Safeguards Rating was Satisfactory for the first five years, until drilling was due to commence. It was then reduced to Moderately Satisfactory due to a delayed completion of Occupational and Community Health, Safety and Security (OCHSS) requirements, which caused a delay in the commencement of drilling, and stayed at this level until Project closing. (ICR, page 18-19) b. Fiduciary Compliance Financial management (FM). The ICR noted that until Restructuring 1 of June 2020, FM was reported as either Unsatisfactory or Moderately Unsatisfactory. The main shortcomings were a delayed opening of the Designated Account, a delayed revision of the FM manual, the lack of counterpart budget at the beginning, and low disbursement. The issues faced in the initial stages of the Project were mainly the result of unbundling of the power sector, during which the separation of accounts was not done properly. (ICR, page 19) After Restructuring 1 and until the closing date, the ratings were Satisfactory or Moderately Satisfactory. Procurement. The ICR reported that there were serious problems with procurement performance. Delays in procurement were the main contributor to delayed implementation and to the non-achievement of Project outcomes. The issues arose due to poor staffing of the PIU at the beginning and delays in addressing this issue. The original procurement strategy of solely relying on drilling rigs to be supplied by EEP and separating procurement of drilling rigs and drilling services, significantly contributed to the delays. Procurement rating ranged from Highly Unsatisfactory to Moderately Unsatisfactory from the second to the Page 15 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Ethiopia Geothermal Development Project (P133613) fifth year of implementation; it then improved to Moderately Satisfactory and stayed at this level until Project closing. (ICR, page 19) c. Unintended impacts (Positive or Negative) --- d. Other --- 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment Moderately Outcome Moderately Satisfactory Satisfactory Moderately Bank Performance Moderately Satisfactory Satisfactory Quality of M&E Modest Modest Quality of ICR --- Substantial 12. Lessons The following lessons were derived from the ICR (ICR, pages 20-21): 1. Limiting project ambition and scope with consideration of the country’s institutional capacity and resource limitations can support the achievement of outcomes better. In the reviewed Project, the scope was overambitious, and the Project had to be restructured as a result, with a significantly reduced scope. The limited capacity of the EEP and PIU, and the design that separated procurement of drilling rigs and drilling services, added to implementation risk. An adequate assessment of this risk at the design stage was needed but not conducted, negatively affecting Project performance. 2. Proper analysis, strategy formulation and implementation planning ahead of project commencement provides an important basis for effective and efficient project implementation. In the reviewed Project, implementation plan had deficiencies, and the process of negotiating a new plan was lengthy and complicated, affecting Project outcomes. If an adequate assessment were done during Project preparation, the eventually negotiated (by the government, EEP, and the Bank) efficient implementation strategy could have been pursued from start and delays avoided. 3. Absorption of the geothermal exploration risk by public sector is a win-win strategy because it supports de-risking of the geothermal development overall and enables private sector participation at the next (less risky) phases of geothermal plants’ construction and operation. Comparison between the reviewed Project and parallel private sector concessions Page 16 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Ethiopia Geothermal Development Project (P133613) demonstrated that the upfront investment required for the exploration stage is too large and risky for the private sector, and when the upstream and downstream activities are combined in a private sector project, the expected return might be even lower. 4. Ensuring adequate implementation capacity and systems at the project preparation stage proved critical for successful implementation. In the reviewed Project, the PIU was established and staffed, but the related due diligence had shortfalls, and not all systems were ready, including those for procurement and financial management (FM). Procurement capacity needed strengthening, but this process was delayed. The FM system was not fully in place. Project performance would have significantly benefited if all systems were in place at the start of implementation and retained throughout implementation. 13. Assessment Recommended? No 14. Comments on Quality of ICR The ICR provides sufficient technical details to understand the value-added of the Project; the context of Project implementation, the factors of Project performance; main outcomes; and various aspects of Project implementation, including efficacy, administrative efficiency, M&E quality, and safeguard and fiduciary compliance. The ICR is analytical and internally consistent, and the lessons are thoughtful and applicable to similar projects across client countries. However, there were minor shortcomings. The efficacy assessment of the Original Project was based on the outcomes at the end of the pre-restructuring period (ICR, paragraphs 25 and 37), whereas, as per the OPCS guidelines, it should have been conducted for the entire project life, assessing the achievement of the original targets against the actual outcomes at closure. There were also shortcomings in the ToC as described in sections 4 and 9. On balance, considering the sufficient information and analysis, but also the above noted minor shortcomings, the rating is Substantial. a. Quality of ICR Rating Substantial Page 17 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Ethiopia Geothermal Development Project (P133613) Page 18 of 18