Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review LR-Electricity System Enhancement (FY11) (P120660) Report Number : ICRR0021233 1. Project Data Project ID Project Name LR-Electricity System Enhancement P120660 (FY11) Country Practice Area(Lead) Additional Financing Liberia Energy & Extractives P129097 L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IDA-48420,IDA-50550 16-Jun-2014 40,326,314.20 Bank Approval Date Closing Date (Actual) 30-Nov-2010 31-May-2017 IBRD/IDA (USD) Grants (USD) Original Commitment 10,000,000.00 0.00 Revised Commitment 31,917,548.34 0.00 Actual 30,342,571.20 0.00 Prepared by Reviewed by ICR Review Coordinator Group Natsuko Toba John R. Eriksson Christopher David Nelson IEGSD (Unit 4) PHPROJECTDATATBL Project ID Project Name LR: Lighting Lives in Liberia ( P124014 P124014 ) L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) TF-11962 11,417,182.96 Page 1 of 20 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review LR-Electricity System Enhancement (FY11) (P120660) Bank Approval Date Closing Date (Actual) 26-Jan-2012 IBRD/IDA (USD) Grants (USD) Original Commitment 0.00 1,454,540.00 Revised Commitment 0.00 1,433,439.96 Actual 0.00 1,433,439.96 PHPROJECTDATATBL Project ID Project Name GPOBA W3: Liberia Electricity Access P110723 ( P110723 ) L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) TF-11962 11,417,182.96 Bank Approval Date Closing Date (Actual) 21-Dec-2011 IBRD/IDA (USD) Grants (USD) Original Commitment 0.00 10,000,000.00 Revised Commitment 0.00 9,983,743.00 Actual 0.00 9,983,743.00 2. Project Objectives and Components a. Objectives According to the parent IDA Credit Financing Agreements (FAs), the Project Development Objective (PDO) was to improve and increase access to electricity in Liberia (IDA 4842 FA, 12/02/2010, p.5; IDA 5055 FA, 03/20/2012). Page 2 of 20 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review LR-Electricity System Enhancement (FY11) (P120660) In addition to the above PDO that covers all project components, two parallel co-financing grant agreements had two PDOs for specific activities of the project: (1) Sub Component A2 was financed by a grant as described below. Since this project was an emergency project, an Emergency Project Paper (EPP) served as the PAD. Also, although not explicit in the GEF Grant Agreement (GA) (03/20/2012. p.4), the Global Environmental Objective (GEO) was stated in the project document as below. The objective of the Sub Component A2, grant-funded by Global Partnership on Output-based Aid (GPOBA), was to help ensure broad-based and inclusive access to electricity and significantly improved living conditions among the poor (GPOBA, GA, 12/21/2011, p.5). The objective of Component C, grant-funded by Africa Renewable Energy Access (AFREA), was to provide modern renewable energy services to off-grid users in Liberia (AFREA Trust Fund Grant Agreement Letter, GA, 03/15/2015. p.4). There was also a Global Environmental Facility (GEF) grant with a Global Environment Objective (GEO) to reduce greenhouse gas emissions when compared with Liberia’s emissions growth baseline (Project Paper, 01/03/2012, pp. iv & 31). This ICR review evaluates this project against the overall objective of the parent IDA credits (IDA 48420 and 50550) and assesses the other objectives and GEO in terms of their contribution to the overall objective. The ICR review uses the definition articulated in the PAD of "electricity access" as improvement to the power supply in Monrovia, higher power capacity, and access to modern energy services. b. Were the project objectives/key associated outcome targets revised during implementation? Yes Did the Board approve the revised objectives/key associated outcome targets? Yes Date of Board Approval 26-Jan-2012 PHEVALUNDERTAKENLBL c. Will a split evaluation be undertaken? No d. Components The project included the following four components. A vertically integrated public power company, called the Liberia Electricity Corporation (LEC), implemented all components except Component C, which a government agency, called the Rural and Renewable Energy Agency (RREA), implemented. At appraisal of the original project, IDA held unallocated amounts of US$0.7 million. Except for GEF, all co-financing Page 3 of 20 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review LR-Electricity System Enhancement (FY11) (P120660) grants were parallel co-financing (Additional Financing Project Paper [AF PP] 2012, pages 1, 2 and 6 and 28; PAD pages 1, 7 and 19). Component A: Distribution Services (Appraisal US$42.80 million, Actual US$60.95 million). Sub-component A.1: Distribution network reinforcement and extension (Appraisal US$3.80 million IDA, Actual US$10.98 million IDA). Expansion and upgrading of the distribution network in urban Monrovia. Sub-Component A.2: Connection of new low-income customers (Appraisal US$10 million GPOBA, Actual US$9.98 million GPOBA). Performance based subsidies to LEC for new electricity connections to qualifying low income households. If GPOBA targets were not achieved, the Management Contractor would be penalized as specified in the Management Contract. As the connections would be pre-financed from Norwegian grants, LEC would use GPOBA’s subsidies to finance other investments. Sub-Component A.3 (Appraisal and Actual US$29 million Government of Norway): Same as Sub- Component A.1. Component B: Enhancing Options for Power Generation (Appraisal US$4.7 million IDA, Actual US$17.77 million IDA and Government of Norway). Rehabilitation of the oil jetty, pipelines and storage facilities for future heavy fuel oil (HFO) power plants; and the first-phase overhaul of the diesel generator units in Monrovia. Component C (Appraisal US$2 million AFREA; Actual US$2.97 million AFREA and GEF): Providing Modern Renewable Energy Services to Off-Grid Users. It was the phase two of an ongoing AFREA’s support to the RREA. Sub-component C.1: Renewable energy pilot activities in rural areas. (i) Sustainable Solar Market Packages (SSMP) and “Lighting Africa” approach to deploy solar lanterns and systems in targeted rural areas to render their sustainable usage; and (ii) rehabilitation of a pre-war micro-hydropower plant in Lofa County. Sub-component C.2: Technical Assistance (TA). Support to the institutional framework to promote modern renewable off-grid options, including ongoing support to the RREA. It would also provide assistance in developing policy foundations and strategy work to underpin this sub-component. Component D (Appraisal US$0.8 million IDA, Actual US$2.10 million IDA): Technical Assistance (TA). Support to the project implementation, including training the staff of the LEC, the Department of Energy of the Ministry of Lands, Mines and Energy (MLME) and other energy agencies, implementation of the Environmental and Social Management Framework (ESMF) and Resettlement Policy Framework (RPF), and carrying out LEC’s annual financial audits. e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Page 4 of 20 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review LR-Electricity System Enhancement (FY11) (P120660) Project Cost Total project cost at appraisal was US$51 million. The actual total project cost was US$83.78 million. This 64 percent increase of the actual project cost from the appraised cost was due to the scaling up of the distribution network under Component A, the additional solar lanterns exchange program in Component C, the additional 10 MW power plant in Component B and the increase in TA. Financing For the original parent project (P120660), an IDA Emergency Recovery Loan (ERL) credit (IDA 48240) of US$ 10 million was approved. For the additional financing, another IDA ERL credit (IDA 50550) of US$22 million was approved. Actual disbursement of the total of two IDA credits was US$30.34 million or 95 percent of the approved amount. The undisbursed amount was cancelled after the disbursement application deadline date. GPOBA approved a parallel co-financing grant of US$10 million (P110723). Actual disbursement was US$9.98 million or 99.8 percent. The undisbursed amount was cancelled after the disbursement. For the original project, the Government of Norway (GoN) approved a parallel co-financing grant of US$29 million. For the additional financing, the GoN approved US$0.5 million. The total disbursement of the GoN was US$29.5 million. AFREA approved a parallel co-financing grant of US$2 million (P118439). Actual disbursement was US$1.54 million or 77 percent of the approved amount. GEF approved a co- financing grant of US$1.45 million (P124014). Actual disbursement was US$1.43 million or 99 percent of the approved amount. Borrower Contribution At appraisal of the GEF financing to Component C, the Government of Liberia (GoL) was expected to contribute US$50,000 of in kind staff time of the implementing agency RREA on monitoring and evaluation (M&E). (Project Paper 2012, pages 40 and 43). The actual GoL contribution was not reported in the ICR because per the Bank team, the budget from the GoL to RREA was not allocated to a specific project. Dates The original IDA 48420 parent project was approved on November 10, 2010. It became effective on July 7, 2011 about eight months after the IDA’s approval. The additional financing (AF) of IDA 50550 was approved on January 26, 2012. The AF was to scale up the distribution network of Component A, add the 10 MW thermal power plant in Component B and increase TA. It became effective on October 29, 2012. The GPOBA grant was approved on December 21, 2011 and was declared effective on March 19, 2012. The AFREA grant was approved and was declared effective on March 15, 2011. The GEF grant was approved and was declared effective on March 20, 2012. IDA had four project extensions and five project restructurings (RPs), including additional financing (AF) as summarized in a table below. Actual closing date of 05/31/2017 was almost 3 years after the original closing date of 06/16/2014. IDA Key Changes Restructurings Page 5 of 20 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review LR-Electricity System Enhancement (FY11) (P120660) 01/23/2014 (AF • AF to scale up, additional or revise activities due to the failed bidding of SMMP. PP 01/03/2012) • Revision of Result Framework including PDO level target indicators. • Closing date changes from June 16, 2014 to December 31, 2014 to implement additional activities. 06/24/2014 (RP • (i) Contingency funds and (ii) Component B’s savings from the cancellation of 06/23/2014) the HFO facilities construction, due to the failed bidding and procurement cost saving of HFO plants, were reallocated to Component A for the material to connect new customers and Component D for project preparation. • Modification of the results framework including PDO level target indicators to reflect the reduced scope of Component B and the scale up of Component A. • Closing date changes from December 31, 2014 to June 30, 2015 to commission the power plant and implement the scale-up distribution network. 03/11/2015 (RP • Closing date changes from June 30, 2015 to May 31, 2016 due to Ebola crisis in 03/06/2015) Liberia that practically halted the activities for eight months. 12/11/2015 (RP • Cancellation of US$2 million for Supervisory Control and Data Acquisition 12/10/2015) (SCADA) and load dispatch system (LDS) in Component A because they should have been installed after the new power plants become operational. Per the Bank team, a reallocation of US$2 million from IDA 50550 was needed for the Liberia Renewable Energy Access Project (LIRENAP, P149683). But a cancellation was not needed as the LIRENAP found alternative funding • Cost saving of HFO power plant of Component B reallocated to Component D to prepare safeguard instruments of LIRENAP. • Closing date changes from May 31, 2016 to May 31, 2017 to complete (i) new connections; (ii) a second transformer for the power plant and (iii) safeguard instruments for the LIRENAP. 05/19/2017 • Utilization of unallocated US$2 million to additional distribution network Amendment expansion under Component A and additional power transformer for the 10 MW Letter (RP HFO power plant under Component B. undated) The GEF grant closing date was extended twice to achieve the sales target of 100,000 solar products. The actual closing date was 05/31/2017, which was about 2.5 years from the AF IDA 50550 original closing date of 12/31/2014. The GPOBA grant closing date was extended twice under two project restructurings. The AFREA grant closing date was extended twice under two project restructurings. Closing date was extended in 2012 due to the nonresponsive bidding results from the SSMP pilot package. The actual closing date was 02/28/2014, resulting in a total extension of 17 months beyond the original closing date on 09/30/2012. 3. Relevance of Objectives Page 6 of 20 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review LR-Electricity System Enhancement (FY11) (P120660) Rationale The PDO was relevant to, and consistent with, the World Bank Liberia Country Partnership Strategy (CPS) for FY2013-2017 when the project was closing. This project was included in the CPS. The project and PDO were relevant as per the World Bank’s latest available data, the percentage of the population living below the national poverty lines was 54 percent in 2014 and the access to electricity in percentage of population was 19.8 percent in 2016. • Alignment with strategy: The project sought to address the development problem of the electricity access gap. The CPS’s economic transformation pillar supported the efforts of the Government of Liberia (GoL) to reduce the infrastructure gap and increase access and connectivity. It was expected to contribute to increasing the generation of, and access to, affordable and reliable electricity, and to improving rural access to alternative renewable electricity generation. The project addressed the pillar with a combination of grid and off-grid electricity supply infrastructure investment and capacity building TA. • Country context: The objectives were outcome-oriented and appropriately pitched for development status and capacity in the country as described in the CPS. The CPS support to reducing the infrastructure gap included energy sector as vital to the economic development and state- and peace-building agendas. The CPS noted (i) the limited government capacity and fragile post-conflict situation, (ii) virtually the entire electricity supply infrastructure was destroyed, (iii) only 1.6 percent of the population was connected to electricity nationwide, (iv) the cost of electricity was high due to high fuel cost, and (v) limited staff skills and systems. Hence, expected PDO outcomes were focused on emergency recovery of the electricity infrastructure, with a view to restoring the medium term power supply, off-grid renewable energy pilots and capacity building. • Relevance to the sector experience: After the approval of this project, the following two projects approved in Liberia were especially relevant (i) Liberia Accelerated Electricity Expansion Project (LACEEP, P133445, FY2013, Additional Financing, P153124, FY2015) and (ii) Liberia Renewable Energy Access Project (LIRENAP P149683, FY2016). At appraisal, the PDO was relevant to and consistent with the Liberia’s Country Assistance Strategy for FY 2009-2011, which included this project and targeted rural and urban energy. Rating High 4. Achievement of Objectives (Efficacy) PHEFFICACYTBL Objective 1 Objective Improve access to electricity in Liberia Page 7 of 20 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review LR-Electricity System Enhancement (FY11) (P120660) Rationale Outputs • Rehabilitated four 66/22 kilovolt (kV) substations and synchronized the isolated grids to improve the reliability of power supply. Eight hundred twenty (820) transformers installed. Exceeded the target of 375 transformers by 119 percent. The baseline was zero supply. • Overhauled the four high speed diesel generators (HSDG) totaling 9.64 megawatt (MW) installed in 2006 to operate until the commissioning of the 10MW HSDG funded by the United States Agency for International Development (USAID) in 2011. The target was “first phase of main generating units overhaul completed” and the baseline was zero. Supplied maintenance parts for the USAID HSDG until the commissioning of the first 10MW heavy fuel oil (HFO) (dual fuel) power plant funded by this project. • HFO pipeline and storage facility not operationalized but designed, construction of which were financed by the LACEEP • Included prior actions for electricity in the Third Poverty Reduction Support Development Policy Operation (PRSDPO-III) in this project to support GoL’s policy to shift from diesel fuel to HFO to reduce the cost of electricity generation in Liberia: (i) License to LEC to import HFO for power generation and (ii) Policy Resolution of LEC’s Board on competitive HFO procurement (Aide Memoire, April 2016). • SCADA and LDS not installed but designed, installation of which were financed by the LACEEP follow-on project. • Yandohun micro hydropower system commissioned in 2013, and handed over to the Yandohun community cooperative for its operation and maintenance in February 2014 (Grant Reporting and Monitoring [GRM] Report, 2015) • Sustainable Solar Market Packages (SSMP) not delivered but revised with different set of targets. • Least Cost Power Development Plan (Implementation Status and Results [ISRs]). Per the Bank team, it was finalized and approved by the MLME. • Nine solar system retail partners were created and sold 76,637 or 77 percent of the targeted 100,000 solar products, providing improved lighting for over 59,800 households in Liberia. Thirty six percent of the local retail partners for solar products were made up of women's groups. The remaining 23,363 are being distributed under LIRENAP. Outcomes • The System Average Interruption Frequency Index (SAIFI) remained at 20 per month in May 2017 (the project closing month), which missed the target of lowering the interruptions to 15. The baseline was 20 and thus it remained unchanged. • The GoL approved for LEC to procure HFO competitively, to make electricity services more affordable to the Liberians under PRSDPO-III: the pertinent policy resolution was issued and the HFO import license granted to LEC (Aide Memoire, July 2016,). Page 8 of 20 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review LR-Electricity System Enhancement (FY11) (P120660) • Two hundred (200) homes for 1,000 people were connected with electricity from less polluting energy sources, exceeding the target of 600 people from the baseline of zero. • Least Cost Power Development Plan (ISRs). Per the Bank team, the plan was disseminated to other donors and MLME used this plan to develop the power system. • A 10MW capacity gap and load shedding was reduced with the 10 MW HFO power plant. The project did not set a baseline or target but a Bank study in 2010 estimated a power supply gap of at least 10 MW by 2013. Unfortunately, after commissioning of the plant, serious defects were revealed and these were not repaired as of May 2018 per the Bank team. • GHG emissions avoided by replacing the kerosene and battery-powered lanterns with solar photovoltaic (PV) lanterns were estimated at 211.1 tons of carbon dioxides equivalent (tCO2e), which was 0.1 percent of the 335,000 tCO2 estimated at appraisal from the target of 100,000 solar lantern sales. However, the methodology of this tCO2e calculation was not clear because solar lantern sales were in fact 77 percent (76,637) of the targeted 100,000 solar products. • Over 59,800 households in Liberia used less polluting lighting. Progress has clearly been made in improving access to electricity. The capacity gap has been closed, relevant policies have been prepared and there has been a distribution of solar products to households. However, there are moderate shortcomings. The SAIFI indicator was not met and thus the system is still ostensibly unreliable. There is some debate as to the cause of this reliability issue, but in essence nothing has moved on this indicator. Likewise, there were issues in the HFO power plant. Though they are being dealt with and do not prevent it from operating, it is a disappointing aspect of the project's performance. Rating Substantial PHREVDELTBL PHEFFICACYTBL Objective 2 Objective Increase access to electricity in Liberia Rationale Outputs • Electricity connection of about 42,000 households in Monrovia, exceeded the target of 26,100 by 61 percent from the baseline of 2,162. Within this output, 17,165 low-income households were connected with Page 9 of 20 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review LR-Electricity System Enhancement (FY11) (P120660) the GPOBA subsidies, which exceeded the GPOBA target of 16,806 by 2 percent. While attribution of this achievement is somewhat opaque due to the range of projects taking place in the sector (per Bank team), and given the connection numbers for Monrovia illustrate LEC’s total connections, there is a presumption that the project played an important part in progress. Outcomes • Increased number of people with advanced electricity access to the Monrovia grid increased to about 210,000 in 2017, exceeding the target of 112,700 by 86 percent from a baseline of 4,992 in 2010. In this total, 73,810 people were GPOBA targeted low-income households. • 1,000 people in Yandohun community have access to advanced electricity. The target was 600 people and the baseline was zero. • Public institutions, households and commercial establishments in Bong and Lofa counties continued without access to advanced electricity, which were expected to be available by SSMP. Rating Substantial PHREVDELTBL PHOVRLEFFRATTBL Rationale While there were moderate shortcomings with regards to the project's attribution and the quality of the investments, progress against the two objectives was commendable and thus the overall rating for Efficacy is Substantial. Overall Efficacy Rating Substantial PHREVISEDTBL 5. Efficiency Ex-Ante Economic Analysis An ex-ante cost benefit analysis covered component A, which was 84 percent of the project cost at appraisal in 2010. The analysis’s main benefit was newly grid-connected households’ electricity cost saving compared to the counterfactual costlier diesel power generation (US$3.96/kWh). The electricity tariff was set to recover operation and maintenance (O&M) cost of LEC. Thus, in the actual scenario, the cost of electricity was assumed to be the tariff at US$0.43/kWh. The resulting net present values at 12 percent discount rate and Page 10 of 20 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review LR-Electricity System Enhancement (FY11) (P120660) economic internal rate of return (NPV and EIRR) were case (1) connection rate of US$1,621, US$117 million and 77 percent and case (2) connection rate of US$700, US$245 million and 130 percent. However, the number of connections assumed in case (1) 10,000 and case (2) 18,528 were inconsistent with the target number of connections of 4,066 in the EPP (p.26) ) or 20,000 in in the Additional Financing Project Paper (AF PP 2012. p.15). The ex-ante analysis’ investment cost was underestimated because it only included additional distribution network costs without including the additional generation capacity investment costs. The EPP (page 80) noted the capacity shortage increases from about 4 MW in 2012 to about 13 MW in 2014 in Monrovia, which could be filled by new HFO fired power plants with an HFO pipeline and storage facility. At the AF appraisal, the GEF grant cost effectiveness to meet the GEO was assessed. It was assumed that 10 percent of the lamps replaced were kerosene lamps and the remainder were battery-powered lamps because in 2011,10 percent of Liberians depended on kerosene for lighting since battery powered light emitting diodes (LED) were more available, cheaper, better quality and safer than kerosene lamps and candles (PP, 2012, page 29). An avoided emission was estimated to be 335,484 tons of CO2 at an average GEF grant cost of US$3.4/ton of CO2, considering only the direct grants for 100,000 lanterns of US$ 1,124,540. Ex-Post Economic Analysis Ex-post analysis covered Components A, B and C, covering 90 percent of the total actual project cost (but EIRR was calculated for Components A and B, which was 89 percent of total actual project cost). For Component A, the results of the analysis were NPV of US$1.2 million at 12 percent discount rate and EIRR of 12 percent. But given the ex-ante analysis’ inconsistent assumptions regarding the counterfactual (households’ kerosene use) compared to the actual survey results by GPOBA and GEF survey data (very few households used kerosene), the ICR Review presumes that these forecasts may be ambitious. Ex-post analysis of the Component B compared the actual scenario of HFO medium speed plant with a counterfactual of the existing HSDG for operating years of 2016-2030. HSDG The results were an EIRR of 12 percent with NPV of US$0.12 million at 12 percent discount rate. There are efficiency shortcomings with regards to the analysis. First, the HFO power plants were running on higher cost diesel fuel from October 2015 to May 2018 . Therefore, the results in the analysis overestimate benefits. This project rehabilitated HSDG, which could not be the counterfactual, as it is the actual scenario. If the counterfactual scenario assumed more intensive rehabilitation, such additional costs would need to be included in the calculation. Since load shedding started around 2015 March until the Bank-financed HFO plant started operation in 2015 and HFO had not been available, a potential counterfactual would be the purchase of emergency HSDG because continuing intensive rehabilitation of existing HSDG would be more expensive. The analysis did not include the avoided outages costs (value of lost load) by the project-financed HFO power plant’s additional capacity of power supply and contribution to reducing load shedding in 2015. Page 11 of 20 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review LR-Electricity System Enhancement (FY11) (P120660) For the GEF-financed part of Component C, a cost-effectiveness approach was adopted, largely consistent with the ex-ante assumption. The benefits of the PV products yielded an NPV of US$1.36 million compared to a cost of around US$484,000, hence a benefit cost ratio of 2.8. The ICR did not include the ex-post assessment of the GEO’s cost effectiveness. IEG examined the project's efficiency based on the GEF grant for 76,637 solar lanterns distributed, representing 211.1 tons of CO2e avoided (page 24, ICR) with a cost of roughly US$ 484,000. The result was an average GEF grant cost of US$2,293/ton of CO2 avoided, which was much more expensive than the appraisal cost of US$3.4/ton CO2. Ex-Ante Financial Analysis The EPP reported the results of an ex-ante analysis conducted for the base capital costs of about US$15 million. It was not clear whose perspective was assumed and what capital costs that the analysis covered (e.g., Component A or/and B). Service life of equipment and project life were considered to be 20 years, which was inconsistent with the economic analysis above that used 10 years for the life of the infrastructure investment. The analysis resulted in a financial internal rate of return (FIRR) of 15.5 percent and an NPV of US$2.102 million at a financial discount rate of 12 percent. At the AF, an ex-ante financial analysis of an HFO plant investment was conducted (73 percent of the IDA AF of US$22 million). The main conclusions of the assessment were the following. If HFO was available from 2014, either a 10MW dual fuel plant or a HFO plant would be financially viable (FIRR respectively of 22.7 percent and 13.5 percent). If the dual fuel plant has to run with more expensive fossil fuel for more than four years, the plant would not be financially viable. Ex-Post Financial Analysis An ex-post financial analysis was from LEC’s point of view. The LEC costs for providing service to the connected consumers of the project include (a) the O&M costs of the network connections in Monrovia (b) the variable generation costs associated with the Bushrod power plant units, and (c) the share of fixed O&M costs of the generation plant including Mount Coffee (ICR, page 54). There was no cost given corresponding to “(c) the share of fixed O&M costs of the generation plant including Mount Coffee” in the Table A4-3. Moreover, there were no transmission costs given in Table 4-2. The analysis did not include capital investment costs because from the LEC’s perspective they were free grants or IDA-type loans to LEC. There were shortcomings in the analysis as presented in Table A4-3, page 54 of the ICR, in which the net benefits included only fixed costs but did not include the variable costs. Hence, by including the variable costs, the result was NPV of US$21.37 million at a financial discount rate of 6 percent with a FIRR of 104 percent at the electricity tariff of US$340/MWh. This assumed tariff of US$340/MWh was not consistent with the LEC’s O&M cost recovery tariff policy because the analysis’ O&M costs were US$200/MWh in 2016 declining further in later years. As discussed in Section 4, the incremental new customers are not fully attributable to this project and thus, generation, sales and revenue are overestimated. Aspects of design and implementation efficiency Page 12 of 20 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review LR-Electricity System Enhancement (FY11) (P120660) The project designs of SSMP and HFO facilities that would not attract the market resulted in repeated bidding failures and eventual cancellation and restructuring, which led to implementation inefficiency. All project analyses did not report the base year of the analysis and used nominal and/or constant prices or did not report prices, which prevents comparable assessment of ex-ante and ex-post efficiency. Also, no analyses explained how the discount rate was estimated. Efficiency Rating Substantial a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 84.00 Appraisal  130.45 Not Applicable 89.00 ICR Estimate  12.00 Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome Relevance of objectives was high. Efficacy was Substantial. While there were some shortcomings, efficiency is still rated Substantial. Thus the overall outcome rating is therefore Satisfactory. a. Outcome Rating Satisfactory 7. Risk to Development Outcome The risks to the PDO were mitigated by the two follow up projects LACEEP and LIRENAP. The technical risk may be substantial as the ICR reported the limited increase of local LEC staff capacity. Financial risk is high. LEC’s website news dated on March 8, 3018 reported that they could recover only half of generation costs, power theft accounted for 49 percent of LEC’s commercial losses in which some LEC employees were involved, and several commercial customers were indebted to the LEC in the amount of over US$5 million. As such, social risk is high in terms of the strength of stakeholder support to reduce non-technical loss of the LEC’s power Page 13 of 20 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review LR-Electricity System Enhancement (FY11) (P120660) supply. Institutional and governance risks are uncertain for the same reasons for the financial risk. The ICR reported continuing poor commercial performance and management capacity (page 34). In 2017, the United Nations Mission in Liberia left but sustained peace is anticipated. The sustainability of development outcomes is exposed to economic risks due to the high dependency on the fuel imports for power generation, electricity imports and the mining sector led growth, which were vulnerable to the external shocks. Environmental risks are substantial due to the continuing use of fossil fuel for power generation, which however, is offset by the promotion of renewable energy based electricity. Government ownership risk is moderate. In his first State of the Nation Address on January 29, 2018, the President stated the key government priorities included the provision of affordable and adequate electricity for all people. 8. Assessment of Bank Performance a. Quality-at-Entry The strategic relevance and approach were satisfactory at entry. The project was prepared under OP/BP 8.00, due to the urgency in restoring the power system to develop other sectors and to address the significant LEC capacity gap that required intensive supervision support. Technical, financial, economic, poverty, gender, and social development aspects were well integrated in the design. IDA funds under the project were used to extend the backbone medium voltage network to the GPOBA targeted communities. This allowed LEC to connect more affluent customers within the catchment. They provided an important revenue base that incentivized LEC to roll-out the GPOBA program. In Monrovia, with electricity, some women were able to set up small businesses. The street lights improved the security particularly for women. The proposed distribution network re-design was intended to reduce technical and commercial losses and minimize supply interruptions, to improve the reliability and quality of the electricity services. Output-based grant for GPOBA household connections had been already pre-grant-financed. As the management contractor normally does not undertake financial and commercial risks, this pre-grant financing could reduce the incentive of the contractor to meet the GPOBA target. Hence, performance risks by the Management Contractor in this project was addressed in the contract that included the non-compliance penalties set if the agreed results are not achieved as well as incentives for satisfactory performance. The Emergency Power Programs I and II helped identify the priorities for the reconstruction of the power system in Monrovia. Policy and institutional aspects were well studied with a sector study. The safeguards issues were properly addressed. Borrower’s implementation arrangements and responsibility were clear between LEC and RREA. Borrower’s monitoring and evaluation (M&E) arrangements were adequate. The project was part of the management contract of the LEC, which gave a strong incentive for the contractor to meet the PDO. Fiduciary aspects were addressed properly. Most of the risks were adequately identified. But there were moderate shortcomings. The technical design overlooked the risk that the transmission network that might not be updated in a timely manner to match the project work on the distribution network expansion and later new power plants, resulted in increasing SAIFI. While there was consideration for alternative funding for these works, the level of ambition was a stretch and shouldhave been better mitigated in the project design. The Bank team did not anticipate the HFO fuel supply risk Page 14 of 20 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review LR-Electricity System Enhancement (FY11) (P120660) and the policy change requiring fuel import license. This issue was brought up late in Aide Memoire, April 2016 when the project’s HFO plant (dual fuel) was running on diesel fuel. The capacity risk of LEC was not properly assessed or addressed. The risk of not attracting potential private sectors to bid the SSMP and HFO facilities were not identified. Quality-at-Entry Rating Satisfactory b. Quality of supervision Focus on development impact of the Bank team was demonstrated by their frequent changes to the project designs and activities within the project and among the projects to adjust to changing circumstances and priorities to improve alignment with the PDO. The Bank team was comprised of experienced staff members. Sufficient resources were allocated for supervision activities. A highly competent resident Energy Specialist in project supervision and management joined the team just at the onset of the Ebola Epidemic in July 2014. This offered the Bank the opportunity to provide hands-on supervision support despite the trying conditions at the time. The mid-term review (MTR) took place opportunely, soon after the crisis had passed, in September – October 2015, to keep the project on track and focused on achieving the PDO. The Bank energy and macroeconomic teams collaborated in a timely manner through the PRSDPO-III to support LEC at a policy level so that they would be given a license to import HFO through competitive procurement for generation of electricity to reduce the cost of electricity generation. Bank reporting was candid. The Bank team persistently followed up on the RREA’s legal status in every ISR up to ISR number 8 when a Law establishing RREA as a permanent public institution was approved by Parliament. Through AFREA, the Bank had helped establish RREA since mid-2009. However, ratings for Development Objectives (DO) and Implementation Progress (IP) were optimistic as they were all Satisfactory or Moderately Satisfactory despite delays in GPOBA electrification and the eventual cancelation of HFO storage and facilities and SCADA. The Bank coordinated well with donors to maximize impacts, essential for the network nature of the power system. In addition to pre-grant- financing the new connections, Norway also pre-financed the distribution network materials due to the delayed effectiveness of the original IDA 48420. The EU funds for the rehabilitation of the substations and the 66kV transmission lines ensured adequate power for customers in the GPOBA areas. Supervision of fiduciary and safeguard aspects were adequate. The Bank team prepared the follow-on LACEEP and LIRENAP, incorporating lessons from the LESEP to continue supporting the power sector. Quality of Supervision Rating Satisfactory Overall Bank Performance Rating Satisfactory Page 15 of 20 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review LR-Electricity System Enhancement (FY11) (P120660) 9. M&E Design, Implementation, & Utilization a. M&E Design All indicators were specific, measurable, achievable, relevant, and time-bound and had baselines and targets. The GPOBA subsidized-connections had specific third-party verification methods. Given the weak capacity within the GoL for monitoring and verification of outputs, these functions were outsourced to an Independent Verification Agent (IVA). Given Monrovia’s prevailing poverty conditions and clearly identifiable slum or low-income areas, GPOBA’s beneficiaries were selected through geographic targeting combined with some household screening. However, during the MTR, the criterion of excluding households with economic activities was found to be irrelevant because many poor households had economic activities on their premises (e.g., small shops, meals services, etc.). In addition, the PDO indicators did not sufficiently capture the outcomes outside the Monrovia and other urban areas. They included only the number of grid connections and the number of urban households connected to electricity attributed to “improve access to electricity in Monrovia after post-conflict” (EPP, page 26). The theory of change was sound following a reasonable causal chain. The key activities included (i) urban areas: restoration of the distribution network and connections to consumers, rehabilitation of the emergency HSDGs to secure electricity supply before the new ones arrive; and (ii) rural areas: rehabilitation of micro hydropower and distribution of solar power systems. The planned outputs were stated as: (i) functioning urban distribution networks that deliver power to increased consumers with less interruptions and system losses, (ii) rural isolated micro hydropower supply to local communities and (iii) use of solar powered electricity by rural establishments and households. The envisaged outcomes of improved access to electricity were stated as: (i) higher power capacity, more continuous and less expensive power supply to the Monrovia grid, isolated rural micro hydropower and solar systems, with the attributes of “advanced electricity access” and (ii) less polluting lighting from solar lanterns that have the attributes of “basic electricity” access. The outcomes of the increased access to electricity would be the increased number of people with using advanced electricity from the Monrovia grid, micro-hydropower and solar systems. The PDOs were at first glance simple. However, given various definitions of access, the term should have been defined. In particular, “improved” and “increased,” required clear definitions because these two words could have the same meaning. b. M&E Implementation In M&E implementation, the input, output, outcome, and impact evidence was collected but not all were analyzed in a methodologically sound manner in terms of attribution. During the additional financing in 2012 and the project restructuring in 2014, the results framework was revised, adding an attribute of improved access of SAIFI and number of people with electricity supply from off-grid or mini-grid and solar lanterns. The major issue was attribution of connections and the electrification rate. As discussed under Objective 2, Section 4, the number of connections and electrification rate pertain to LEC’s total customers. Thus, these indicator values are not fully attributable to the project. Page 16 of 20 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review LR-Electricity System Enhancement (FY11) (P120660) The management contractor of LEC ensured the LEC’s M&E implementation. The IVA verified the quantity and technical quality of connections on representative samples and delivered all verification reports in a timely manner except during the Ebola period. RREA needed to rely on its local retail partners (LRP) for data on products sold. To mitigate inadequate reporting, RREA trained LRPs on managing a retail business, which proved effective in obtaining better quality information in the later years of the project. The reliability and quality of indicators of the number of people were uncertain because they were derived from the connections and solar lanterns in use. The ICR noted different calculation methods (e.g. five or 4.3 family numbers per connection; different customer types; etc.). It was not clear how the methods were calculated or verified. The solar lanterns indicator included the number of solar systems as the number of lanterns equivalent (ISR number 8). It was not clear in the ICR or ISRs how the number of solar systems were related to the total number of solar lanterns. c. M&E Utilization M&E findings were shared with USAID, the Norwegian Government and EU for their comments that helped correct the M&E data. M&E contributed to making decision and designing of project restructurings. The M&E data were used to provide evidence on outputs and inputs. M&E data informed the structure of components of LIRENAP. Lanterns sales data of the project were incorporated into LIRENAP to include larger solar system capacity and change the dissemination approach, such as helping local retailers procure materials. M&E Quality Rating Substantial 10. Other Issues a. Safeguards Environmental and Social Safeguards: The project was classified as Category B under OP/BP 4.01 (Environmental Assessment) since no significant and/or irreversible adverse environmental and social issues were expected from the project. The World Bank policy for Involuntary Resettlement (OP/BP 4.12) was nevertheless triggered without specific reason given in the EPP. The project complied with the safeguards policies. b. Fiduciary Compliance Financial Management: LEC complied with the financial covenant and submitted annual audited financial statements. The ICR noted instances of delayed, qualified, and corrected audits but did not clarify the nature of the qualifications, the measures taken to address them or whether all audit recommendations had been addressed. Major disbursement delays originated in an excessive number of steps in the approval process Page 17 of 20 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review LR-Electricity System Enhancement (FY11) (P120660) within LEC and management turnover. Disbursement improved towards the end of the project after a series of interventions by the Bank’s FM Specialist. No corruption with the project was reported in the ICR. All IDA and trust fund resources were accounted for. Procurement: The project followed Bank procurement guidelines. The timely flow of information from procurement to the disbursement unit was poor, worsening the disbursement ratio of the project. A procurement specialist was hired and procurement processing times and quality were improved. Unresponsive bidding processes resulted in design modifications and restructurings such as of the SSMP and HFO tanks. Procurement delays and quality failings resulted in a sharp reduction in the connection rate for the GPOBA (18 connections in the second quarter of 2013, compared to 2,600 connections in the previous six months during September 2012 – March 2013, ISR number 4). In addition to the lack of material for the first semester of 2013 due to delays in ordering and delivery, around 40 percent of the installed meters installed were malfunctioning, which was subsequently solved. Disbursement: Per an Aide Memoire of September-October 2015, the fiscal year (FY) 2014 audit opinion was modified with the auditors highlighting an unauthorized disbursement of US$48,800 from the project accounts. LEC later reimbursed the project with the amount. c. Unintended impacts (Positive or Negative) Not Applicable. d. Other Not Applicable. 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment Outcome Satisfactory Satisfactory --- There were shortcomings in Bank Performance Highly Satisfactory Satisfactory Quality at Entry There were methodological shortcomings in dealing with Quality of M&E High Substantial attribution of the project acheivements. Quality of ICR Substantial --- 12. Lessons Page 18 of 20 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review LR-Electricity System Enhancement (FY11) (P120660) Following are lessons learned are largely drawn from the ICR, with some modification of languages. 1 . Potential perverse incentives from pre-grant-financing an output-based grant can be mitigated through use of positive and negative financial incentives. The management contractor normally does not undertake financial and commercial risks. Hence, performance risks by the Management Contractor in this project was addressed by the non-compliance penalties set by the Management Contract if the agreed results are not achieved. Failing to reach the GPOBA target would expose the Management Contractor to the risk of forgoing success fees and incurring significant non-compliance penalties. As a result, the Contractor had met the GPOBA target. This illustrates the need for a clear and enforceable contract clause to ensure performance. 2 . Low-income customers need to be connected strategically to optimize the benefits. IDA financed the extension of the backbone medium voltage network to the GPOBA targeted communities. This allowed LEC to connect the more affluent customers within the catchment area, who provided a revenue base and incentives for LEC to roll-out low-income customer connections under the GPOBA program. This highlights the importance of a holistic electrification strategy. 3 . Tight and timely coordination and cooperation among development partners and within the Bank are critical in the power sector in fragile contexts. IDA and other development partners were well-coordinated to support the energy sector reconstruction. The project did not achieve the reliability target (interruption rate) in Monrovia partly due to the network nature of the electricity system requiring very tight coordination among the segments of the system. The project covered only part of the distribution and power generation segments of the system. The transmission network was outside the project’s scope and did not keep up with the project’s expansion of distribution network and generation capacity. On the other hand, the Bank energy team timely collaborated with the Bank macroeconomic team to resolve the LEC’s HFO licensing and competitive HFO procurement through policy operations (PRSDPO-III). 13. Assessment Recommended? No 14. Comments on Quality of ICR The ICR covers a wide range of issues, follows most of the guidelines and is focused on results. The ICR made efforts to establish a theory of change and is comprehensive in providing an overview of the project's performance. However, there were shortcomings. The ICR did not explain how the HFO and duel fuel power plants were running on diesel fuel or competitively priced HFO, despite the fact that the latter was a key to achieving the PDO and that the project dropped HFO facilities construction. Additionally, the ICR did not report the Bank energy and macroeconomic teams’ collaboration to resolve this HFO issue through policy operations (PRSDPO-III). The ICR assessed only one of the two main causes of the project missing the SAIFI target, i.e., increased generating power later during the project. There were also issues in how the Efficiency analysis was undertaken and inconsistencies in how the approach was explained. Page 19 of 20 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review LR-Electricity System Enhancement (FY11) (P120660) a. Quality of ICR Rating Substantial Page 20 of 20