Document of The World Bank FOR OFFICIAL USE ONLY Report No: ICR00004851 IMPLEMENTATION COMPLETION AND RESULTS REPORT TF-016279 ON A SMALL GRANT IN THE AMOUNT OF US$4.85 MILLION TO THE REPUBLIC OF VANUATU MINISTRY OF FINANCE AND ECONOMIC MANAGEMENT FOR GPOBA IMPROVED ELECTRICITY ACCESS (P133701) February 2, 2020 Energy and Extractives Global Practice East Asia and Pacific Region ABBREVIATIONS AND ACRONYMS DEMMR Department of Energy, Mines, and Mineral Resources DoE Department of Energy ESMF Environmental and Social Management Framework ESDP Energy Sector Development Project GoV Government of Vanuatu GPOBA Global Partnership on Output-Based Aid GPOBA IEA GPOBA Improved Electricity Access GRM Grievance Redress Mechanism ICR Implementation Completion and Results Report ISR Implementation Status and Results Report IVA Independent Verification Agent MoFEM Ministry of Finance and Economic Management MoU Memorandum of Understanding MTR Midterm Review M&E Monitoring and Evaluation NERM National Energy Roadmap OBA Output-Based Aid OVR Output Verification Report PAD Project Appraisal Document POM Project Operations Manual PMU Project Management Unit SIA Subsidy Implementation Agreements UNELCO Union Electrique du Vanuatu (Electrical Union of Vanuatu) URA Utilities Regulatory Authority of Vanuatu VUI Vanuatu Utilities and Infrastructure WTP Willingness to Pay Regional Vice President: Victoria Kwakwa Country Director: Michel Kerf Senior Global Practice Director: Riccardo Puliti Practice Manager: Jie Tang Task Team Leader(s): Kamleshwar Prasad Khelawan ICR Main Contributor: Maria Ayuso Olmedo Contents DATA SHEET .......................................................................................................................... 1 I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES ....................................................... 4 A. CONTEXT AT APPRAISAL ......................................................................................................4 B. SIGNIFICANT CHANGES DURING IMPLEMENTATION .............................................................9 II. OUTCOME .................................................................................................................... 11 A. ACHIEVEMENT OF PDOs (EFFICACY) ................................................................................... 11 B. JUSTIFICATION OF OVERALL OUTCOME RATING ................................................................. 16 C. OTHER OUTCOMES AND IMPACTS (IF ANY) ........................................................................ 16 III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME ................................ 17 A. KEY FACTORS DURING PREPARATION ................................................................................ 17 B. KEY FACTORS DURING IMPLEMENTATION.......................................................................... 19 IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME .. 23 A. QUALITY OF MONITORING AND EVALUATION (M&E) ......................................................... 23 B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE.................................................. 25 C. BANK PERFORMANCE ........................................................................................................ 26 D. RISK TO DEVELOPMENT OUTCOME .................................................................................... 28 V. LESSONS LEARNED AND RECOMMENDATIONS .............................................................. 28 ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS ........................................................... 32 ANNEX 2. PROJECT COST BY COMPONENT ........................................................................... 42 ANNEX 3. RECIPIENT, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS ...... 43 ANNEX 4. SUPPORTING DOCUMENTS (IF ANY) ..................................................................... 46 ANNEX 5. ELIGIBILITY CRITERIA, SUBSIDY DESIGN, AND IMPLEMENTATION APPROACH ........ 48 ANNEX 6. DETAILED RESULTS: SERVICE CONNECTIONS AND HOUSEHOLD WIRING................ 51 The World Bank GPOBA Improved Electricity Access (P133701) DATA SHEET BASIC INFORMATION Product Information Project ID Project Name P133701 GPOBA Improved Electricity Access Country Financing Instrument Vanuatu Investment Project Financing Original EA Category Revised EA Category Organizations Borrower Implementing Agency Ministry of Finance and Economic Management Department of Energy Project Development Objective (PDO) Original PDO The Project Development Objective for this operation is to increase sustainable access to formal grid-based electricity services within Vanuatu’s electricity concession service areas for low income consumers through targeted subsidies. FINANCING FINANCE_TBL Original Amount (US$) Revised Amount (US$) Actual Disbursed (US$) Donor Financing TF-16279 4,850,000 1,433,504 1,433,504 Total 4,850,000 1,433,504 1,433,504 Total Project Cost 4,850,000 1,433,504 1,433,504 Page 1 of 51 The World Bank GPOBA Improved Electricity Access (P133701) KEY DATES Approval Effectiveness Original Closing Actual Closing 16-May-2014 23-Sep-2014 30-Jun-2018 31-Dec-2018 RESTRUCTURING AND/OR ADDITIONAL FINANCING Date(s) Amount Disbursed (US$M) Key Revisions 11-Jun-2018 1.14 Change in Loan Closing Date(s) 30-Oct-2018 1.30 Change in Components and Cost Other Change(s) KEY RATINGS Outcome Bank Performance M&E Quality Unsatisfactory Unsatisfactory Modest RATINGS OF PROJECT PERFORMANCE IN ISRs Actual No. Date ISR Archived DO Rating IP Rating Disbursements (US$M) 01 13-Dec-2014 Satisfactory Satisfactory 0.10 02 25-Jun-2015 Satisfactory Satisfactory 0.15 03 10-Dec-2015 Satisfactory Satisfactory 0.25 04 23-Jun-2016 Satisfactory Moderately Satisfactory 0.44 05 30-Jun-2017 Moderately Satisfactory Moderately Satisfactory 0.76 Moderately 06 04-Dec-2017 Moderately Unsatisfactory 0.98 Unsatisfactory Moderately 07 03-Oct-2018 Moderately Unsatisfactory 1.30 Unsatisfactory Page 2 of 51 The World Bank GPOBA Improved Electricity Access (P133701) ADM STAFF Role At Approval At ICR Regional Vice President: Axel van Trotsenburg Victoria Kwakwa Country Director: Franz R. Drees-Gross Michel Kerf Director: John A. Roome Ranjit J. Lamech Practice Manager: Michel Kerf Jie Tang Task Team Leader(s): Kamleshwar Prasad Khelawan Kamleshwar Prasad Khelawan ICR Contributing Author: Maria Ayuso Olmedo Page 3 of 51 The World Bank GPOBA Improved Electricity Access (P133701) I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES A. CONTEXT AT APPRAISAL Context Country Background 1. At the time of project approval, Vanuatu had become one of the fastest growing economies of the Pacific region and experienced a period of strong and sustained economic growth driven by tourism, construction, and aid inflows. With a population of approximately 285,000 people, in 2014, Vanuatu’s average household monthly income was US$892 (VUV 83,800) and between US$204 (VUV 18,400) and US$504 (VUV 45,400) for low-income households. For these households, the cost of basic infrastructure services was high and access to finance remained difficult despite the economic growth of the country. In fact, only 27 percent of the overall population, and 20 percent in rural areas, had formal access to electricity. Sector Background 2. At the time of project preparation, the grid electricity supply was restricted to the main urban centers of Port Vila (Efate Island), Luganville (Espiritu Santo Island), and small areas in the islands of Malekula and Tanna. About 85 percent of Vanuatu’s total installed capacity (30.7 MW) was located in Port Vila, where peak demand reached 11.3 MW in 2014. The remaining capacity was installed in Luganville (4.1 MW), Malekula (0.34 MW), and Tana (0.27 MW), with peak demands of 1.5 MW, 0.12 MW, and 0.12 MW, respectively. Around 20 percent of the electricity was produced from renewable energy sources, mainly hydro and wind, and coconut oil. The remaining 80 percent of the electricity was generated from imported diesel. 3. The private sector was responsible for generating and supplying electricity under concession agreements and management contract with the Government of Vanuatu (GoV). There were two service providers, the Electrical Union of Vanuatu (Union Electrique du Vanuatu Ltd., UNELCO) and Vanuatu Utilities and Infrastructure Ltd. (VUI). UNELCO supplied the Port Vila, Malekula, and Tanna concession areas and VUI supplied the Luganville concession area under a management contract. The Utilities Regulatory Authority (URA) was tasked to apply the provisions of the Electricity Supply Act and the concession agreements and regulate consumption tariffs. Tariffs were set across five consumer categories with lower tariffs of 34 percent of the base tariff for low-consumption domestic consumers. The electricity base tariff was around US$0.59 (VUV 54.20) per KWh. 4. The Department of Energy (DoE), within the Ministry of Climate Change Adaption, Meteorology, Geo-hazards, Energy, and Environment and the National Disaster Management Office, was responsible for the development of energy policies, legislations, and regulations to guide the development of energy services in Vanuatu and improve service delivery. The DoE was also responsible for identifying, Page 4 of 51 The World Bank GPOBA Improved Electricity Access (P133701) implementing, managing, and evaluating energy projects; monitoring and facilitating energy activities; and providing awareness and training.1 National Priorities 5. There was no Country Partnership Strategy for Vanuatu at the time of project approval. The project was designed to be aligned with: (a) the GoV’s Priority and Action Agenda for 2006–2015, which aimed at reducing the cost of services, extending the coverage of rural electrification, and promoting the use of renewable energy; and (b) the GoV’s National Energy Roadmap (NERM), which sets targets to increase household electricity access from 27 percent to 90 percent by 2020 and 100 percent by 2030. Beyond providing access to secure, reliable, and affordable electricity to all citizens by 2030, the NERM identified four other priority areas. These areas included affordability of energy services in Vanuatu, energy security in the country at all times, climate change mitigation through renewable energy and energy efficiency, and reliability of petroleum supply throughout the country. The project was therefore designed to contribute to increased access to grid-connected electricity in Vanuatu. Rationale for World Bank Assistance 6. At the time of project preparation, the costs of electricity connections were a significant barrier for low-income consumers to connect to the grid. The estimated cost of new connections was up to US$700 (VUV 66,393), excluding household wiring. It was also estimated that approximately 2,000 households shared connections through informal arrangements with their neighbors, compromising safety and increasing the unit consumption costs of electricity, as consumers fell into higher tariff brackets due to group consumption. In addition, there were no wiring standards or licensing of electricians in Vanuatu, though the concession holders worked to either Australian and New Zealand or French standards. The introduction of an output-based-aid (OBA) financing approach to provide a subsidy for service connections and basic household wiring was therefore considered. 7. The Global Partnership on Output-Based Aid (GPOBA) was established in 2003 as a Multi-Donor Trust Fund administered by the World Bank with the mandate to explore and apply innovative results- based financing solutions for enabling access to basic services.2 In addition, the GoV and the World Bank had experienced a collaborative relationship throughout the preparation and implementation of the Energy Sector Development Project (ESDP)3, the establishment of the URA and the development of the NERM. Particularly, the NERM set the foundation for the Vanuatu energy sector policy and paved the way to rationalize investments aimed at achieving priorities for increasing access to secure and affordable 1 Department of Energy, 2019. https://doe.gov.vu/index.php?r=site/about. 2 At the time of project approval, the GPOBA had implemented 14 energy access projects and accounted for 44 percent of the total GPOBA portfolio. The GPOBA annual report for 2014 is available at http://www.gprba.org/knowledge/publications/gpoba- annual-report-2014. 3 The ESDP was approved a year before the GPOBA IEA Project in 2013. Its development objective was to provide technical assistance to the Department of Energy, Mines, and Mineral Resources (DEMMR) of Vanuatu and the URA in the development and execution of their work programs. The project had two key components: (a) policy support to the DEMMR, which was geared toward planning and the implementation of policy, legislative, and regulatory reforms and appropriate institutional arrangements; and (b) regulatory support to the URA, which would finance consultant services for the work program, and provide on-the-job training, mentoring, and formal training for the URA. This component also comprised a program of advisory assistance and capacity building for the URA to address continuing as well as emerging regulatory challenges. (http://documents.worldbank.org/curated/en/608251468134372741/Vanuatu-Energy-Sector-Development-Project) Page 5 of 51 The World Bank GPOBA Improved Electricity Access (P133701) electricity for all citizens in the country. The ESDP provided technical assistance and supported the work programs of the DoE and the URA. Thus, the World Bank was positioned as a trusted donor to support the GoV to meet its long-term energy aspirations and address the investment gaps through the GPOBA. In this landscape, the GPOBA Improved Electricity Access (GPOBA IEA) Project was designed to disburse subsidies to the service providers upon achievement and verification of household connections for small domestic consumers, limited to 5 A, and to provide household wiring, where it was not up to standards or not in place, including a Ready Board.4 Project Development Objectives (PDOs) 8. The PDO was to increase sustainable access to formal grid-based electricity services within the Recipient’s electricity concession service areas for low income customers through targeted subsidies. 9. Both the Project Appraisal Document (PAD) and the Grant Agreement state the same PDO, specifying the former “Vanuatu� in place of “the Recipient’s�. Key Expected Outcomes and Outcome Indicators 10. The key expected outcome of this project was to increase access to electricity for low-income customers in Vanuatu. Four outcome indicators were defined to measure the achievement of this objective, as shown in Table 1. Table 1. Key Project Outcome Indicators and Targets Unit of Baseline Value End Project Key Project Outcome Indicator Measure (2014) Target (2018) People provided with access to electricity under the Number 0 21,437 project by household connections (PDO1)c People continuing to utilize electricity connections three Number 0 21,437 months after connection (PDO2) c Community contributions (customer co-contribution) in US$ 0 511,875 the total project cost (PDO3) c Wiring rules (standards) adopted and a traininga regime Standards 0 Standards in place (PDO4)c adopted; training regime in placeb Note: a. PDO indicator as defined in the PAD Results Framework and Implementation Status and Results Reports (ISRs). In the Project Operations Manual (POM), this PDO was defined as “Wiring rules (Standards) adopted and a certification regime in place: a) Process established, b) Consultations commenced, c) Regulations and standards determined/Training Plan, d) Implementation commenced .� b. This target was not defined in the PAD nor reflected in the Grant Agreement or the POM. The ISRs defined it as ‘Standards in place’. c. The terminology ‘PDO1’, ‘PDO2’, ‘PDO3’ and ‘PDO4’ will be used in this report to refer to PDO -level indicators as defined in this table. Components 4The Ready Board would include the necessary protection, two light sockets with switches, two power outlets, and two energy- saving light bulbs (see section I.A. under the Components subsection for more details). Page 6 of 51 The World Bank GPOBA Improved Electricity Access (P133701) 11. The project consisted of the following four components, as described in the project documents: • Component 1: OBA subsidies for new electricity connections for low-income households. This component would finance up to 80 percent of the cost of connecting 4,3755 households to the national grid through postpaid and, where available, prepaid metered connections. Figure 1 illustrates this connection between the power pole and the utility meter box. This component would be implemented through the existing utilities, UNELCO and VUI, and verified by the DoE.6 Annex 5 includes further details of the eligibility criteria, subsidy design, and approaches adopted during implementation. • Component 2: OBA subsidies for household wiring for low-income households accessing electricity. This component would finance household wiring (up to a predefined limit of VUV 40,0007 per household) for low-income households accessing grid-based electricity services under the project, where the wiring was not standard or in place. The households would be wired according to a standard design for the project, in accordance with Australian and New Zealand standards (AS/NZS 3000) on wiring rules. The wiring service provided under this component would include cabling and backfilling from the utility meter box to the Ready Board in the consumer’s premises. Consumers would be responsible for organizing the digging of cable trenches and, if required, a hole for a service connection pole.8 The Ready Board would include two light sockets with switches, two power outlets, and two energy- saving light bulbs. For safety reasons, the wiring would be carried out and/or certified by qualified electrical contractors9 contracted or authorized by UNELCO or VUI. The description of components in the PAD and POM states that the utilities/service providers would supervise the electrical contractors and on completion, provide verification that the internal wiring had been completed to standard.10 Annex 5 includes further details of the eligibility criteria, subsidy design, and approaches adopted during implementation. • Component 3: Implementation support for project management, communications, outreach, and training. This component would support the DoE in overseeing the overall implementation of the project and developing longer-term institutional capacity, specifically financing: (a) the development and adoption of technical standards and licensing for household wiring in technical project implementation, monitoring and evaluation (M&E), and financial management of OBA subsidies; (b) outreach to raise beneficiaries’ awareness of the project and consumers’ awareness on electrical safety; (c) DoE technical training, if 5 According to the feasibility study carried out for the project by an independent consultancy firm, there were approximately 6,000 households that lacked access to electricity entirely and a further 2,000 that had informal connections within the concession areas of VUI and UNELCO. 6 Through an IVA, according to the description of Component 4. 7 The conversion rate used in the PAD indicates a converted limit of US$444. It has been observed that page 20 makes references to a converted amount of US$421 for the VUV 40,000 limit. Across the POM, the limit is set in local currency consistently at VUV 40,000. 8 This would be at the consumers’ cost or completed as works in kind. 9 Grade A, Australian electrical/mechanical fitters’ license or equivalent. 10 It must be noted that physical inspection and verification of household wiring was one of the IVA services (see description of Component 4). Page 7 of 51 The World Bank GPOBA Improved Electricity Access (P133701) needed; (d) the services of an independent auditor,11 if needed; (e) operating costs incurred by the DoE on outreach activities and project implementation; and (f) goods to be purchased by the DoE required to implement the project. • Component 4: Independent verification of outputs. This component would cover the cost of services of an independent verification agent (IVA) to undertake the verification of outputs under the project. Specifically, the IVA services would include12: (a) desk-based verification of all connections installed by the utilities under the project; (b) physical inspection and verification of a random sample of 20 percent of the service connections and household wiring in the first year and 10 percent of the service connections in the subsequent years of the project; and (c) desk and physical verification of no connection upgrades within 12 months after connection,13 according to subsidy eligibility criteria.14 Figure 1. Schematic Diagram of Service Connection and Household Wiring (Model Without Service Connection Pole) 12. The estimated and actual funds allocated to each component are presented in table 2. The table shows that the actual disbursement per component does not add up to the estimated components costs because US$3.36 million was undisbursed by project closing. Section I.B discusses the cancellation of undisbursed funds and section III presents key challenges that hindered disbursements. Table 2. Actual and Estimated Cost per Component 11 This independent auditor would be hired to prepare the financial audits of the project in the event that the Auditor General Office had insufficient capacity to undertake the audits. 12 According to paragraph 67 of the PAD. 13 Except in the final year of the project, when the restriction to upgrade of the physical connection would apply from the date of connection until the project closing date. 14 Refer to the POM for a detailed presentation of the eligibility criteria set out for this project. Page 8 of 51 The World Bank GPOBA Improved Electricity Access (P133701) Actual Percentage of Estimated Cost Disbursement Approval (%) Project Components at Appraisal at Closing (US$) (US$)a 1. OBA subsidies for new electricity connections for 2,200,000 701,836 31.9 low-income households 2. OBA subsidies for household wiring for low- income households accessing electricity services 2,100,000 597,038 28.4 under the project 3. Implementation support for project management, 350,000 100,944 28.8 communications, outreach, and training 4. Independent verification of outputs 200,000 94,029 47.0 Total project cost 4,850,000 1,493,847 30.8 Undisbursed 3,356,153 69.2 Note: a. Exchange rate: US$1 = VUV 103.988. B. SIGNIFICANT CHANGES DURING IMPLEMENTATION Revised PDOs and Outcome Targets 13. The PDOs and outcome targets were not revised. Revised PDO Indicators 14. A new indicator, ‘people provided with new or improved electricity service’, was added in ISR Seq. 5 (June 2017) to comply with World Bank Guidance on Corporate Results Indicators (issued in April 2017), as shown in Table 3. Revised Components 15. The components were not revised. Other Changes 16. First restructuring: extension of the project closing date to address the safety concerns related to household wiring. In May 2018, four years after project approval and a month before closing, a restructuring was approved to extend the closing dates of the original grant from June 30, 2018 to December 31, 2018. The reason was to facilitate the completion of household wiring to address safety concerns in UNELCO concession areas relating to whether households that received service connections but had not chosen the option of household wiring under the GPOBA IEA Project met minimum international standards. The Subsidy Implementation Agreements (SIAs) between the utilities and the Ministry of Finance and Economic Management (MoFEM) were amended to give effect to the project closing date extension in accordance with the Restructuring Paper. 17. The restructuring was conducted promptly to respond to the Moderately Unsatisfactory progress toward the achievement of the PDO. By the time the restructuring took place, roughly 40 percent of the targeted household connections and 20 percent of the targeted household wiring had been achieved and projections estimated that US$3.5 million funds would remain undisbursed by the original project closing. Page 9 of 51 The World Bank GPOBA Improved Electricity Access (P133701) At the time of this first restructuring, VUI had performed up to expectations in terms of accomplishing new connections and household wiring as had been planned. By then, UNELCO reported underperformance achieving 34 percent of its targeted service connections and 10 percent of the household wiring target set at project design. This underperformance was mainly associated to the following two factors: (a) in parallel to the GPOBA IEA Project, UNELCO offered more flexible and financially beneficial options for the customers; and (b) not all households that obtained service connections under the project opted for the wiring offer, but many found alternative wiring options, which resulted in wiring not meeting international standards in some cases. These factors, as well as the actions taken by the borrower and the World Bank team, are detailed in section III. 18. Second restructuring: changes in Component 2. A second restructuring was approved in October 2018 to amend Component 2 to meet household wiring costs in UNELCO’s concession areas by: (a) removing the limit on the amount of subsidy for household wiring set out in the POM; and (b) including the costs for preparation of cable trenches to facilitate completion of household wiring to address safety concerns. The SIA and the POM were amended to give effect to the approved changes. Given the low disbursement rates, there was sufficient budget to adopt the changes proposed in this second restructuring.15 19. The second Restructuring Paper explains that, in UNELCO concession areas, up to 716 households that had acquired service connections under the project and household wiring aside from UNELCO’s wiring contractor services were found to be in need of rectification works to bring wiring up to standards. At that time, only 137 of these had been rectified. The consumer’s reluctance to pay costs above the subsidy amount or to prepare the necessary trench delayed the rectification of works. For a small fraction of households, the household wiring had not yet been completed because they were still under construction.16 The second restructuring was therefore approved in an attempt to improve the progress toward achievement of the PDO, which remained Moderately Unsatisfactory with overall numbers of service connections and household wiring estimated respectively at roughly 46 percent and 39 percent of their targeted values. 20. Cancellation of funds. Despite the restructuring carried out, US$3.36 million remained undisbursed by the revised project closing date, December 31, 2018. The main reasons for this underspend were: (a) low rates of service connections and household wiring in UNELCO’s concession area; and (b) lower average costs of service connections than estimated at project appraisal. This amount is therefore being cancelled. The arguments for cancellation without opportunities for further extension of the closing date as stated in ISR Seq. 7, dated October 3, 2018, indicate that: (a) VUI’s concession area had been exhausted by then and presented low prospects for new service connections; and (b) UNELCO was unlikely to extend the network in its concession area soon and the utility would continue with its own options for electricity access, which would keep impacting the results of the GPOBA IEA Project. 15 With a balance of over US$3.0 million undisbursed funds, the budget required to complete households wiring and address safety concerns was estimated at US$909,000 (based on an average of US$700 per household for wiring costs, excluding trenching, and an average cost of US$1,320 per household to prepare cable trenches, and estimated 450 households requiring these services). 16 In this case, the project would not fund any household wiring works for household constructions finalized after the project completion date. Page 10 of 51 The World Bank GPOBA Improved Electricity Access (P133701) II. OUTCOME A. ACHIEVEMENT OF PDOs (EFFICACY) Assessment of Achievement of Each Objective/Outcome 21. The project partly achieved its objective and intended outcome to increase sustainable access to formal grid-based electricity services within the recipient’s electricity concession service areas for low- income customers through targeted subsidies. The PDO-level indicators and the achievement rates attained for each of the indicators support this conclusion and are presented in table 3. The concept of sustainable access is not only considered the physical connection, but also its affordability, reliability or durability, among other sustainability aspects. 22. Achievement rates of 50 percent and 30 percent can be attributed to almost all PDO indicators. The indicators PDO1, PDO2, and PDO317 are valid and measurable to show the extent of achievement of objectives. 23. In general, the PDO-level indicators inform on the progress toward the achievement of the PDO. Indicator PDO1 captures the physical service connections established under the GPOBA IEA Project. For more accuracy, indicator PDO1 should have been defined as ‘People provided with service connections under the project’. To clearly reflect ‘access to electricity’ and ‘household connections’, indicator PDO1 could have reflected households or people receiving service connections together with household wiring. Indicators PDO2 and PDO3 reflect the access affordability and reliability by measuring the customer’s contribution and continued use of electricity, months after connection. Some redundancy on the indicators is observed as indicators PDO2 and PDO3 directly depend on PDO1. Failure to succeed on achieving the target of indicator PDO1 intrinsically implies the impossibility to achieve PDO2 and PDO3 targets. In retrospect, it is observed that some directly dependent indicators could have been intermediate indicators. 24. People provided with access to electricity under the project by household connections (PDO1) (and ‘people provided with new or improved electricity service’). In this indicator, ‘access to electricity’ and ‘electricity service’ refer to ‘service connections’. The actual numbers achieved for this indicator are well below the end target. Overall, 4,375 household connections were targeted: 3,570 and 805 in UNELCO and VUI areas, respectively. UNELCO connected 1,407 households (39.4 percent) and VUI connected 780 households (96.9 percent), achieving together 2,187 connections (50.0 percent) of the 4,375 connections targeted.18 Annex 6 includes further details on the service connections and household wiring accomplished under the GPOBA IEA Project throughout implementation. The household connections targeted were estimated based on historical trends of households lacking electricity access in both areas. The target expected to be achieved at closing, in terms of people provided with new or improved electricity service, was estimated assuming an average of 4.9 inhabitants per household. This same ratio of inhabitants per household was used to calculate the actual number of people provided with electricity under the project and applied to the actual number of service connections according to the verified services on Output Verification Reports (OVRs). The reason for the low achievement rate is mainly 17 ‘PDO1’, ‘PDO2’, ‘PDO3’ and ‘PDO4’ is terminology used in this report to refer to PDO-level indicators as defined in Table 1. 18 According to the last Project Progress Report for July 2018–December 2018 dated February 2019. Page 11 of 51 The World Bank GPOBA Improved Electricity Access (P133701) attributed to customers in UNELCO concession areas having similar options for service connections to the GPOBA IEA Project as discussed in the recipient’s comments on Annex 3. This point is also further explained in section III. 25. People continuing to utilize electricity connections three months after connection (PDO2). This indicator directly correlates to ‘people provided with access to electricity under the project by household connections’ (or ‘people provided with new or improved electricity service’), which explains the low actual values achieved. The expected target for this indicator was estimated assuming that 100 percent of all households receiving service connections under the project would maintain the connection three months after the connection date and customers connected through prepaid meters would maintain the connection unless physically disconnected. The actual value achieved for this indicator was verified by the IVA based on billing data (customer electricity bill records) for all customers and a sample-based physical verification and reported in the OVRs submitted to the DoE19. Both the target and the actual value achieved apply the same ratio of inhabitants per household as in indicator PDO120. 26. Community contributions (customer co-contribution) in the total project cost (PDO3). The actual customer co-contributions are lower than the estimated amount, given the low number of customers receiving new or improved electricity service under the project. The target, set at US$511,875 by the end of 2018, was estimated at 11 percent of the total cost of electricity connection (service connection and household wiring). This contribution was based on the willingness to pay (WTP) estimated in feasibility reports.21 The actual numbers of community contributions were tracked based on the service provider’s claims issued to the DoE, which reflected customer payments. Table 3 shows that customers contributed with US$151,406 according to the last Project Progress Report22, less than the US$155,972 reported at project closing. In any case, the average community contributions achieved are lower than expected, mainly due to: (a) a lower number of service connections than estimated; (b) a lower share of contributions by customers than estimated; and (c) exchange rate differences at appraisal and project closing date. Assuming the ratio of beneficiaries per household and the overall number of beneficiaries reported in PDO1, 2,187 connections were made under the project. Using the same estimations to calculate the target (this is, user co-contribution of US$511,875 for 4,375 connections), the reported community contribution in indicator PDO3 (US$151,406) is lower than proportionally estimated (US$255,879). 27. Wiring rules (standards) adopted and a training regime in place (PDO4). The project did not achieve this indicator by the closing date. The formulation of this PDO indicator and its target, ‘standards in place,’ is ambiguous. The following observations are made regarding the directness of connections between the activities of the project and the achievement of these objectives. First, the POM defined the 19 Three months after connection, the IVA inspected a sample of 10 percent of the houses connected in each claim of the service provider to verify that households were still connected and the connection respected the 5 A cap to be maintained for a year. This verification would only apply to connections with postpaid meters. 20 The results achieved under indicator PDO1, as well as other PDO indicators are presented in table 3. 21 The feasibility reports, conducted by an independent consultancy firm, estimated that consumers could afford between US$110 (VUV 10,466) and US$170 (VUV 16,175) for service connections. This represented approximately 31 percent to 48 percent of the average monthly income of low-income households. Annex 5 includes further details of the eligibility criteria, subsidy design, and approaches adopted during implementation. 22 The Project Progress Report for July 2018–December 2018 dated February 2019 offers further information, in local currency, of the proportions from this amount attributed to the two utilities. The document reflects that VUV 6,045,970 is attributed to VUI and 11,064,211 is attributed to UNELCO. Page 12 of 51 The World Bank GPOBA Improved Electricity Access (P133701) indicator ‘Wiring rules (Standards) adopted and a certification regime in place’ as ‘a. Process Established; b. Consultations Commenced; c. Regulations and standards determined/Training Plan; d. Implementation commenced’, but the project did not define specific activities to fully accomplish the target of this PDO indicator. Second, the ‘standards and enabling legislative and regulatory framework’ drafted to achieve this indicator were funded under the Bank-supported ESDP and thus, they are not attributable to the GPOBA IEA Project. According to the team, there was continued follow up with DoE on steps required to achieve this indicator during the supervision of the GPOBA IEA Project. According to the Borrower’s ICR, the non-adoption of the wiring standards is due to issues that arose from combining the wiring rules with the technical regulations for the network under the same legislative instrument, an URA act (see section III.B). Finally, the project documents do not present the definition of the training regime, and subsequently, the achievement is hard to measure. Table 3. PDO Indicators’ Results, Targets, and Achieved Performance Rate Attributable to the Project Achievement Achievement End Latest Baseline Actual Value Rate as per Rate as per Key Project Outcome Project Actual Value Value at Closing Actual Value Latest Actual Indicators Target Available (2014) (2018) at Closing Value (2018) (2019)a (%) Available (%) People provided with access to electricity under the project by 0 21,437 9,844 45.9 10,716 50.0 household connections (PDO1) (Number) People continuing to utilize electricity connections three 0 21,437 9,687c 45.2 10,716 50.0 months after connection (PDO2) (Number) Community contributions (customer co- 0 511,875 155,972 30.5 151,406 29.6 contribution) in the total project cost (PDO3) (US dollars) Standards Standards Wiring rules developed. developed. (standards) adopted Standards Standards Standards and a training regime 0 not approved — not — in place in place (PDO4) by the approved by (Standards) Council of the Council Ministers. of Ministers. People provided with new or improved 0 21,437 9,844 45.9 10,716 50.0 electricity service (Number)b Note: Page 13 of 51 The World Bank GPOBA Improved Electricity Access (P133701) a. The numbers in this column represent the values of the indicators achieved by the project closing, in December 2018, but reported after closing in the last Project Progress Report for July 2018 –December 2018 submitted in May 2019. b. This indicator was added in ISR Seq. 5 (June 2017) to comply with World Bank Guidance on Corporate Results Indicators as described in section I.B. c. The Project Progress Report for January 2018–June 2018, dated August 2018, explains that this value does not include the verification of 32 households connected to the date as the reporting fell within the three-month validation period. This explains the difference toward the achievement of indicator PDO1 at closing. 28. Project outputs and outcomes against intermediate indicators. The achievement of outcomes arises from the activities and outputs of the project. The outputs are measured through intermediate indicators that, in general, succeed to inform on progress of the project’s activities toward achieving the PDO. The intermediate indicators defined according to the original grant include: (a) people qualifying for a new household connection to the utility network under the project; (b) annual volume of electricity sold to all low income households connected under the project; (c) annual revenue generated from electricity sales to low income households connected under the project; (d) low income households receive household wiring to standard design under the project; (e) number of electricians employed and receive training to install project design standard household wiring; (f) low income households received an information bulletin on household electricity safety under the project; (g) community groups who received an information bulletin about the GPOBA subsidy available under the project; and (h) women’s groups who received an information bulletin about the GPOBA subsidy available under the project. Based on these intermediate indicators, the achievement of outcomes is attributable to the project’s activities as follows: • OBA subsidies for new electricity connections for low-income households. To accomplish this activity, households qualifying for new connections under the project were identified. During the project, the electricity sold to all low-income households connected and the annual revenue that accrued from the sales were also monitored. These intermediate indicators directly inform the PDO on the deployment of new physical electricity connections and track the customer’s affordability of electricity access. Potential for new physical connections and the customers’ affordability are both key indications of the project’s impact to increase sustainable access. All three intermediate indicators are lower than the expected targets for the same reasons that hindered the achievement of PDO1 and PDO2 targets. It must be noted that the electricity sales achieved by project closing exceed the target (127 percent). The average consumption per household is higher than expected due to an underestimation of the demand for low-income households at project preparation and the amperage limit increase from 5 A to 10 A (see section III). However, the revenue achieved is about 73 percent of the target (US$596,996). This has been attributed to: (a) fluctuation of the exchange rate between 97.7VUV/US$ at project appraisal and 103.988VUV/US$ at closing; and (b) an up to 25 percent reduction in the actual base tariff since project appraisal (base tariff was set at 66 percent of the VUV54.2/kWh tariff) due to regulatory reviews and a fall in diesel prices over the project period23. 23 Utilities Regulatory Authority. December 2018. Electricity Factsheet 2012-2017. Port Vila, Vanuatu. http://www.ura.gov.vu/attachments/article/97/Electricity%20Fact%20Sheet%202018.pdf Page 14 of 51 The World Bank GPOBA Improved Electricity Access (P133701) • OBA subsidies for household wiring for low-income households accessing electricity services under the project. The project measured that 39.3 percent (specifically 1,720) of the 4,375 targeted low-income households received wiring under the project. UNELCO wired 942 of the 1,407 connected households (67.0 percent) and VUI wired 778 of the 780 connected households (99.7 percent) targeted in their respective concession areas.24 This intermediate indicator directly indicates progress toward the PDO. In VUI’s concession area, the utility provided service connections and household wiring with in-house staff and local technicians as a combined service with one consumer fee. VUI trained its staff and local technicians to undertake the work and achieved 96.6 percent of the targeted household connections and provided household wiring to 99.7 percent of the connected households. VUI trained at least 28 electricians to wire households up to standards. The number of electricians trained exceeds the target initially set. The training is not attributable to the project because it was entirely conducted, organized, and financed by VUI as the project did not contemplate a structure training of electricians among its activities. This is further discussed in section II.C. Since service connections is UNELCO’s obligation under its concession agreement and household wiring (including preparation of the cable trench) is a consumer’s responsibility, UNELCO contracted out the provision of household wiring (unregulated services) to a private company with trained electricians, as a separate service to the provision of service connections (regulated service). The customers’ reluctance to pay wiring costs above the subsidy level and prepare trenches resulted in lower household wiring. As a result, only 67 percent of the connected households chose to complete the household wiring under the project and this component only disbursed about 28 percent of the funds allocated. • Implementation support for project management, communications, outreach, and training. By the closing date of the project, roughly over 18 percent (800 households) of all targeted low-income households under the project received an electricity safety bulletin25 and 54 percent (19 groups) of all community groups initially targeted, received information on the GPOBA IEA Project subsidies26. The project’s documents indicate that 300 women groups (56 more than initially targeted) received information on the GPOBA IEA Project subsidies. This intermediate indicator was misreported as the actual value registered represents the estimated number of individual women who received information bulletins in awareness campaigns instead of the number of women groups targeted. The distribution of information bulletins to households and groups does not necessarily imply that users read, assimilate, and understand the information and attributing bulletin dissemination to effective communication and outreach is difficult to prove. The DoE also used other dissemination channels, such as radio advertisements about the GPOBA IEA Project in both, English and Bislama languages, according to the Bank team. No intermediate indicators were designed to track project management implementation support and project management training. 24 According to the last Project Progress Report for July 2018–December 2018, dated February 2019. 25 This number represents the safety bulletins delivered by the DoE. 26 This number represents the number of community campaigns conducted by the DoE for different groups. Page 15 of 51 The World Bank GPOBA Improved Electricity Access (P133701) • Independent verification of outputs. No intermediate indicators were defined to track progress towards this activity. 29. A more detailed presentation of the intermediate indicators is presented in annex 1. B. JUSTIFICATION OF OVERALL OUTCOME RATING Rating: Unsatisfactory 30. Based on the assessment of objectives’ achievement discussed in the preceding paragraphs, the Implementation Completion and Results Report (ICR) concludes that the overall outcome efficacy rating of the project is Unsatisfactory. The project has not achieved its respective targets for indicators PDO1, PDO2, and PDO3 by the project closing date. The respective target values of indicators PDO1 and PDO2 have been attained by 50 percent. The targeted value of indicator PDO3 has been attained by approximately 30 percent. PDO4 is unfulfilled and not adequately defined to measure the achievement of expected outcomes. It can be concluded that there were major shortcomings in the operation’s achievement of its objectives.27 C. OTHER OUTCOMES AND IMPACTS (IF ANY) Gender 31. The project included a gender analysis and targeted to inform women’s groups of the GPOBA subsidy available under the project (see section IV.A). The aim was facilitating women’s access to electricity, given that 24 percent of households in the country are female headed and women earn less income than men.28 According to the IVA’s reports, the project provided electricity access to 219 female- headed households (49 and 170, respectively, in VUI’s and UNELCO’s concession areas). Some female beneficiaries of the GPOBA IEA Project have reported experiencing a change on the labor household activities traditionally undertaken by women at home (for example, faster cooking or sewing through the use of rice cookers and/or sewing machines, respectively, because of the electricity access). Such change might facilitate freeing valuable time for women, hence widening their employment opportunities. In addition, access to electricity enables access to modern cooking and lighting solutions, which has potential to displace the use of kerosene and other fossil fuels. While the indoor air quality improvement resulting from this displacement benefits both sexes, it represents a larger impact for women, who spend longer times at home due to the traditional division of household labor. 27 The ICR also concluded that the restructuring approved during the project did not trigger split rating because it did not have an impact on the project objective or the outcome targets and the scope of the project remained the same. The restructuring extended the closing deadline and amended Component 2 to remove the household wiring subsidy cap and include the cost for preparation of cable trenches in alignment with its goal and without modifying the original activities contemplated in the POM. Finally, the PDO indicators added since preparation of the original loan ( ‘people provided with new or improved electricity service’) did not reflect different or additional measures of project achievement and did not lower or raise the project’s ambition. 28 According to the PAD, rural women earn 61.5 percent of rural men’s income (VUV 83,600), while in urban areas, they earn 89.6 percent of men’s income (VUV 99,200). Average monthly household expenditure for female -headed households is estimated as VUV 64,600, which represents 81.5 percent of the male-headed households’ average monthly expenditure. This means that on average, rural female-headed households do not have sufficient income to meet expenditure. Page 16 of 51 The World Bank GPOBA Improved Electricity Access (P133701) Institutional Strengthening 32. The project contributed to strengthening the DoE capacity by financing competent consultants on fiduciary matters, who was later absorbed as DoE staff to carry out DoE fiduciary operations beyond the GPOBA IEA Project. The IVA recruited under the project also joined the DoE as the on-grid officer. A program assistant was also recruited and absorbed as part of DoE current staff. Together with other Bank- supported projects, the GPOBA IEA project provided DoE a work program for rural electrification and enabled DoE to hire a project manager for oversight of various project activities. The project did not directly result in training of DoE staff despite the undisbursed funds allocated to Component 3 (dedicated to implementation support and training among other activities). However, the project’s requirements to offer an option to customers to get household wiring up to standards through the utility, contributed to motivate VUI’s own training of local electricians on standardized household wiring. Mobilizing Private Sector Financing 33. The project was designed relying on two private electricity service providers for its implementation. At the time of appraisal, grid-connection costs were a barrier for the access to electricity of low-income and potentially new consumers. Thus, the rationale for the GPOBA involvement was turning access to electricity into a financially viable service to those who did not have regular or sufficient income, own land, and could not demonstrate a credit history. By doing this, the project design intended to close the gap between the private utilities’ business and new low-income customers by which, once the upfront payment needed to get the service connection is completed, could entail additional monthly revenues for the utilities and boost the interest of private sector investors. Poverty Reduction and Shared Prosperity 34. Some beneficiaries of the GPOBA IEA Project have experienced income-generating opportunities with access to electrification through, for example, the establishment of small businesses (such as refrigeration of fish for selling and domestic production of kava or ice creams) and the acquisition of electric tools (such as sewing machines) to substitute traditional manual tasks. Electrification has also facilitated education and learning activities and the access to media and communication devices (phone charging). The project has helped some beneficiaries by alleviating the monthly energy expenses made on kerosene, candles, solar kits, diesel, and diesel gensets for lighting and cooking. III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME A. KEY FACTORS DURING PREPARATION 35. In general, the project set realistic objectives and had a simple design despite some weaknesses. The project’s level of ambitiousness was adequate for the country context. Most PDO indicators were Page 17 of 51 The World Bank GPOBA Improved Electricity Access (P133701) aligned with the operational objective of the project and had appropriate baselines and targets. While most project components had an operational logic, Component 3 could have been more clearly structured by strengthening the definition of activities included and providing an appropriate timing and sequencing of tasks, particularly for training. PDO4 however was not clearly defined and the project did not include straightforward provisions to achieve this output. 36. The DoE capacity to adequately meet the project implementation requirements was limited since appraisal. The project was designed to count on an on-grid officer at the DoE, who would undertake the GPOBA IEA Project management duties. During preparation, there was no designated project manager. The position remained vacant intermittently during implementation and when available, the on-grid officer’s time and responsibilities were shared among a portfolio of other DoE projects with limited time to supervise the GPOBA IEA Project. The project was designed for the DoE to obtain support from the Project Management Unit (PMU) on fiduciary issues. At appraisal, the PMU was already leading four other World Bank-funded projects in various sectors.29 In view of the limited PMU capacity to support the GPOBA IEA Project implementation duties, the DoE recruited one staff member to undertake fiduciary issues for DoE projects after the project approval. Neither the DoE nor the PMU had social and environmental safeguards specialists during project preparation and implementation. In retrospect, the DoE’s readiness for implementation at appraisal was considered low and to have affected the implementation progress. 37. The analytics and intelligence gathered from customers to inform the project design presented inaccuracies in UNELCO concession areas. According to the PAD, the estimated cost of service connections was validated against UNELCO’s actual average costs of connections using actual historical costs with an upward adjustment. These costs were used to fix the subsidy levels for service connections of VUI, which opted for conducting the connection services ‘in house’30. The subsidy levels for UNELCO were to be based on actual costs achieved from the competitive bidding processes for the connection services. The household wiring costs were estimated based on the distance from the utility meter box to a location inside the house and the requirement to backfill the cable trench to international standards. Under the approach adopted by VUI, the utility averaged the estimated costs for service connections and household wiring and covered the differences to the actual costs. Under the approach adopted by UNELCO, consumers were responsible for offsetting the difference between the actual costs and the 80 percent subsidy for service connections, as well as the difference between the actual costs and the subsidy limit for household wiring on an individual household basis. Annex 5 includes further details of the eligibility criteria, subsidy design, and approaches adopted during implementation. In the UNELCO concession area, the actual service connections turned out to be less expensive than estimated at appraisal. The number of prepaid meters installed was larger than estimated and, because prepaid meters were cheaper than postpaid meters, this contributed to lower the actual costs for service connections. 31 Conversely, household wiring works were pricier in this concession area. The household wiring design 29 At project appraisal, the PMU was already established within the Department of Meteorology under the Ministry of Climate Change Adaption, Geo-Hazards, Meteorology, Energy, and Environment. 30 According to the PAD, Luganville did not have a sufficiently competitive market for such services. 31 The use of prepaid meters was approved for Port Vila by the URA. Final Report: Pre-payment meters - Final Decision. Utilities Regulatory Authority. August 2013. According to the PAD, the economic analysis conducted for this project includes the cost of the service connection and the cost of basis household wiring estimated at US$1,034 for a postpaid meter. The analysis considers also incremental costs of fuel and operating costs of the utility. The economic analysis considered that half of the meters installed would be prepaid with an average cost per service connection of US$552 for prepaid meters and US$613 for postpaid meters. Page 18 of 51 The World Bank GPOBA Improved Electricity Access (P133701) based on Australian and New Zealand standards turned out being a more expensive arrangement than estimated at appraisal and contributed to higher household wiring costs. Other issues that negatively affected the project during implementation in UNELCO areas, such as an arguably low amperage limit, the requirement to remain on the low-income consumer tariff and the reluctance of customers to arrange digging works for cable trenches, have emerged from a misinformed project design for these areas. Finally, it appears that the subsidies designed in UNELCO concession areas failed to be an attractive option for the utility to promote service connection amongst its customers under the GPOBA IEA project. The following section III.B discusses these issues, as well as the consequences that weaknesses in the analytics at project design entailed during implementation. B. KEY FACTORS DURING IMPLEMENTATION Natural Disasters 38. Category 5 Cyclone Pam struck Vanuatu on March 13, 2015, severely affecting Efate and Tanna Islands in UNELCO concession area. Consequently, according to ISR Seq. 2, UNELCO had to undertake extensive repair works across the concession areas during the same year. Similarly, the DoE participated in the emergency relief tasks led by the GoV. This slowed down project implementation for several months, resulting in delays on regular implementation tasks, such as submitting subsidy claims. The first subsidy claim was registered and processed in May 2015. Household Wiring Challenges 39. UNELCO’s subcontractor for wiring works faced schedule delays and observed higher quotes than estimated for these works. Cyclone Pam also affected UNELCO subcontractor’s schedule for wiring works. This contributed to the disbursement delays that downgraded progress of Component 2 to Moderately Satisfactory by December 2015, a year after effectiveness. By that date, the IVA had verified 151 households receiving service connections under the project, but only 118 households receiving wiring. In addition, the subcontractor wiring quotes resulted into higher costs than estimated at appraisal. This was attributed to longer wiring distances considered than those estimated at project preparation. Seeking to reduce costs, a POM addendum had been approved in May 2015 to allow household wiring to be provided in accordance to either the French standards (NF C 12-100) or Australian/New Zealand Standards (AS/NZS3000). According to the Aide Memoire of February 2018, UNELCO’s basic household wiring was also redesigned with protection only and left light bulbs and power points optional at the cost of the consumer. The exclusion of bulbs coverage under Component 2 was discussed and agreed with the bank team, but associated provisions in the legal documents were not modified. These adjustments helped speed up the house wiring in UNELCO concession area but were not able to help achieve the targeted results of Component 2, which was affected by additional factors (see section II.A). 40. The delays experienced in providing household wiring services resulted in some householders developing their own wiring solutions, which were not up to the standards in some cases. This observation was made during a field trip to Siviri and Tanoliu in Efate Island in April 2016. To address this issue, the World Bank team advised the DoE to: (a) inform households of the wiring completion time lines through the service providers; and (b) request the IVA to verify and report in OVRs whether households were aware of such a time line. By then, the project had disbursed over 7 percent of total grant funds (US$0.4 million). Furthermore, the Mid-Term Review (MTR), prepared in November 2017, flagged that in Page 19 of 51 The World Bank GPOBA Improved Electricity Access (P133701) some cases, householders were installing their own substandard wiring when they were uninterested in waiting for the household wiring services to be provided by UNELCO subcontractors. 41. Other challenges to deliver household wiring under the project revolved around the reluctance of some householders to pay for the GPOBA household wiring offer and to arrange the wiring trenches in UNELCO concession areas. Different results were achieved in VUI and UNELCO concession areas, although the project offered the same level of subsidy for household wiring in both areas. The number of household wiring offers taken in VUI’s concession areas was consistent with the number of the service connections. The effectiveness to achieve household wiring targets is mainly attributed to VUI providing connections to customers as a bundled service with household wiring and charging a flat rate for household wiring. However, in UNELCO concession areas, the provision of household wiring was unbundled from service connections. Household wiring remained the responsibility of customers, who could opt for the provision of household wiring through UNELCO´s subcontractor or other parties. In this area, roughly only half of all households that obtained service connections under the project opted for the GPOBA IEA Project’s wiring offer. This was attributed to the following two factors: first, some customers implemented alternative wiring solutions possibly to save cost and labor work when costs exceeded the amount covered by the subsidy under the wiring offer, or because provisions of the wiring works were delayed. According to the team, data from the OVRs indicates that the majority of the households whose actual costs were below the subsidy threshold took up the household wiring option under the project. However, those households whose actual costs exceeded the subsidy threshold chose other solutions. In some cases, this resulted in wiring that was not up to the standards. Following the concerns raised by the World Bank team, the DoE and UNELCO agreed to identify and revisit the households that had not chosen the wiring services under the GPOBA IEA Project, estimated at 716 customers, to verify compliance with the General Conditions of Public Electricity Supply in Vanuatu.32 For those households that did not have wiring up to the General Conditions of the Public Electricity Supply in Vanuatu, UNELCO issued a defect notice and offered wiring rectification options33 given the safety concerns related to this issue as recorded in the ISR Seq. 5, dated June 2017. As found in the MTR, some consumers were not keen on arranging the digging of trenches required for cabling. Since this activity was initially excluded from the services offered by the project, the World Bank restructured the project in October 2018 to include services of digging the trenches in the project’s household wiring offer (see section I). The restructuring timing allowed only two months before closing for the subcontractors and utility to facilitate the services of digging the trenches for customers. Service Connection Challenges 42. Customers in UNELCO concession areas had other options for service connections aside of the GPOBA IEA Project. The MTR report indicates by November 2017, that UNELCO had connected 2,668 32 Article 5of the General Conditions of Public Electricity Supply in Vanuatu states that (a) internal installations must be protected by one or more circuit breakers at the control board selected from those which conform with French standards; (b) a certificate of conformity should be supplied by the private contractor who installed the household wiring; and (c) if the installation is considered defective, UNELCO may discontinue the supply until defects are corrected. According to the Aide Memoire prepared by the World Bank team in May 2018, about 8,000 UNELCO’s customers (about 50 percent of its entire customer base) were unlikely to have circuit breakers on the control board. Among them, there were one-third of the households who had received service connections under the project and had not taken up the household wiring offer. 33 According to the team, UNELCO installed differential circuit breakers at the meter box to provide earth leakage protection and protection against overcurrent and short circuit to its assets and the households. Page 20 of 51 The World Bank GPOBA Improved Electricity Access (P133701) households, of which 913 were connected through the GPOBA IEA Project. The remaining were undertaken under UNELCO’s options for service connections, financed through its own funds. Like the GPOBA IEA Project, UNELCO’s options for service connection presumably covered 80 percent of the actual cost of service connection but offered a 10 A amperage cap, which was higher than that allowed under the project. In addition, UNELCO options did not have the project’s restriction for customers to upgrade the limit after connection34 or remain on the low-income consumer tariff for 12 months. The Aide Memoire of April 2016 and ISR Seq. 4 of June 2016 identified this issue, indicating that only 17 percent of UNELCO’s small domestic consumers’ requests would fall within the 5 A category while 83 percent of the service connection demand would require up to 10 A connection. To promptly address this issue, the DoE and the World Bank prepared and approved an addendum to the POM, dated April 2016, to increase the amperage cap to 10 A for eligible households of the GPOBA IEA Project in UNELCO’s concession areas. The overlap between UNELCO options for service connections and GPOBA IEA subsidy scheme is considered to have slowed down the progress toward achievement of outcomes of the GPOBA IEA Project. It could be therefore argued that the measures taken to parallel the GPOBA IEA Project to UNELCO’s options for service were not optimal as this made both programs overlap in the same market segment, interfering in the operations financed through private funds. In either case, keeping flexible designs to allow changing certain design parameters with limited formal complexities is considered a good practice which, if implemented, could have given the project more room for maneuver to address this issue and others that arose during implementation. This is further discussed in the section V. Apart from the GPOBA IEA Project, there is no knowledge of other options for service connections available to customers in the VUI concession area, where the utility accomplished 96.9 percent of the targeted service connections. 43. The field trip conducted in April 2016 to Siviri and Tanoliu in Efate Island revealed deficiencies in project dissemination and safety community awareness activities. During the field trip conducted in April 2016 to Siviri and Tanoliu in Efate Island, the Bank team found that some householders were not aware of the services that could be subsidized by the GPOBA IEA Project despite the Communications Plan developed by the DoE and the mass media advertisements conducted.35 In addition, some households connected in these areas had not received the information bulletin on electrical safety, as planned under Component 3 of the project. The World Bank team requested the DoE to intensify the efforts on community awareness and include verification of whether households had received the information bulletin, as part of the IVA’s regular reporting in OVRs. There is, however, no evidence that this was accomplished. Standardization Gaps 44. Till date, Vanuatu has not yet adopted household wiring standards and the project adapted to the most cost-efficient standardized wiring layout in UNELCO concession areas. There are no household wiring standards in Vanuatu. The general conditions of electricity supply in Vanuatu set out the requirements for internal installations, which must follow French standards or equivalent. The project 34 According to the Borrower’s ICR. Also, according to this document, there were no subsidized connection programs in Vanuatu at project preparation. 35 According to the Communications Plan developed by the DoE on 2015, various types of formal communication would be used to disseminate the project features, including broadcast (radio, television, and so on); digital (Internet and mobile); print (newspapers, brochures, and so on); and outreach (campaigns, public speaking, and so on). In practice, the DoE used radio advertisement, printed brochures, and outreach campaigns to communicate the GPOBA IEA Project subsidies to potential beneficiaries. Page 21 of 51 The World Bank GPOBA Improved Electricity Access (P133701) therefore adopted a basic household wiring design for funding under the GPOBA IEA Project to offer households an interim solution until national household wiring standards came into force. The basic household wiring design was based on Australian and New Zealand standards with agreement of both service providers. In UNELCO’s concession areas, the networks are designed to French standards. This resulted in the design based on Australian and New Zealand standards being costlier (see section III.A) and delays in household wiring following the basic household wiring design under the GPOBA IEA project. To address this issue, the POM and the Environmental and Social Management Framework (ESMF) were amended to include French standards (NF C 15-100) as the basis for basic household wiring design in UNELCO’s concession area. Household wiring standards were nevertheless drafted during the GPOBA IEA Project implementation.36 By project closing, the standards were not yet officially adopted. According to the Borrower’s ICR, the standards37 were prepared in combination with the technical regulations of the network under the same URA act, resulting in complications from combining regulated parts of the business (network) with unregulated parts (household wiring) within the same act. The non-adoption of standards could result in households acquiring sub-standardized wiring solutions and posing safety concerns. Fiduciary Improvements 45. Misunderstandings in UNELCO Memorandum of Understanding (MoU) with the GoV led to corrections on accounting of reimbursements. The MTR and the Aide Memoire of February 2018 indicate that misunderstandings in the MoU led the utility to include the subsidy payments towards service connections funded through GPOBA in its regulated asset base. The regulated asset base is used for tariff setting purposes. The GoV and UNELCO agreed to remove the subsidy payments towards service connections from the regulated asset base and to include UNELCO’s investments instead. 46. The auditing reports of this project were unqualified and timely submitted through the Auditor General Office. The Restructuring Paper of May 2018 required a policy waiver to proceed. This was triggered by an overdue revision of audited financial statements of another project38 implemented by the DoE. This did not affect the GPOBA Project, or the DoE’s capacity to handle auditing. Shortly after obtaining the waiver, the revised audited financial statements were submitted, and the audit opinion was modified. Overall Supervision and Monitoring 47. A moderately high IVA turnover affected implementation progress. The project’s IVA resigned twice: first, in October 2016 and then again in June 2018, six months before the revised date of completion in December 2018. According to the MTR, after the first resignation, it took until April 2017 for the post to be filled. From October 2016 until April 2017, DoE staff performed the IVA responsibilities. 48. Semester progress reports were adequately submitted over the course of the project, but the MTR was delayed for more than a year and the project did not disburse the available funds to reinforce the DoE capacity needs for project management. With the exception of the first project progress report, 36 It mustbe noted that section II.A. discusses the attribution of these standards to the GPOBA IEA Project since the development of the standards was financed with ESDP funds. 37 The Borrower’s ICR refers to the standards as enabling legislations and regulations for household wiring. 38 Energy Sector Development Project. Page 22 of 51 The World Bank GPOBA Improved Electricity Access (P133701) the DoE submitted adequate semester progress reports on time. However, the MTR was significantly delayed and left insufficient time to implement corrective measures before the project closing date. This was attributed to delays in hiring consultancy support to undertake this task. The World Bank’s rapid intervention to timely conduct the MTR in-house would have been appropriate to find out the key issues affecting implementation and initiate corrective measures earlier. By the time the MTR was published, in September 2017, seven months were left till the original project closing date and only US$0.85 million of project funds were disbursed, with the remaining US$4 million undisbursed. At the actual closing date, December 31, the project disbursed 31 percent of the total funds of the project and only 29 percent of the funds dedicated to support project implementation under Component 3 despite the need to reinforce the DoE’s capacity for project management. Summary of Impacts 49. The factors presented in this section significantly affected the project implementation progress and outcomes achievement. VUI performed to expectations and met the service connections and household wiring targets on time before the original closing date. Despite VUI’s performance and results, the issues exposed above downgraded the progress toward achievement of the PDO to Moderately Satisfactory by end of the calendar year 2016. Most issues continued to affect the project throughout 2017 without sufficient correction measures in place. By the end of 2017, both the progress toward achievement of the PDO rating and the overall implementation progress ratings were downgraded to Moderately Unsatisfactory. Finally, the World Bank concluded that the PDO would not be achieved, even over an extended period, and the project closure took place in December 2018. IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME A. QUALITY OF MONITORING AND EVALUATION (M&E) M&E Design 50. The PDO indicators are mostly considered to inform the achievement of the PDO statement; be adequate to capture the contribution of the project Components 1 and 2; and be specific, measurable, achievable, and time-bound. PDO1, PDO2, and PDO3 indicators are directly dependent on each other. The Results Framework could have included some of them as intermediate indicators provided that other PDO-level indicators had been added to capture access affordability and durability. The assumptions and methodology followed to monitor the actual value achieved of indicator PDO2 in relation to prepaid meters could have been explicitly detailed in the project documents. PDO4 indicator was adequate to track progress towards the achievement of the project’s overall objective by informing on access reliability. However, the achievement of PDO4 target was not supported by specific activities defined under the project components. In addition, PDO4 was not clearly defined and the target was ambiguous. 51. The project captured the impact on gender through the utility connection reports. These reports tracked applicant’s contact details and gender, as well as information on connection history (see section III.C under the Gender subsection for further details). Alternatively, the Results Framework could have segregated PDO1 and PDO2 by gender to help quantify the project’s impact on women for World Bank tracking purposes and use. Page 23 of 51 The World Bank GPOBA Improved Electricity Access (P133701) 52. The intermediate indicators designed to track progress against Component 3 of the project on implementation support (that is, low income households received an information bulletin on household electricity safety under the project, community groups who received an information bulletin about the GPOBA subsidy available under the project, and women’s groups who received an information bulletin about the GPOBA subsidy available under the project) could have been strengthened as disseminating bulletins does not necessarily indicate an effective communication with beneficiaries and outreach. All other intermediate indicators are considered to succeed in informing on progress toward achieving the PDO. See section II.A under subsection Assessment of Achievement of Each Objective/Outcome for further discussion on the design of the Results Framework. 53. The M&E strategy did not include indicators to monitor the progress of Component 4. However, most verification activities were reflected in the OVRs submitted to the PMU. M&E Implementation 54. The M&E strategy for implementation included the following elements: (a) POM setting, reporting, and verification requirements, subsidy structure, and implementation arrangements including fiduciary matters and disbursements and general processes; (b) the DoE’s semiannual performance reports for the GPOBA and the World Bank; (c) interim financial reports prepared quarterly by the PMU for the World Bank; (d) annually audited financial statements provided to the World Bank by an independent auditor hired by the Auditor’s General Office; (e) UNELCO and VUI connections reports for the DoE prepared quarterly before subsidies’ payment; (f) IVA OVRs for the DoE to pay subsidies to the utilities; (g) one MTR of implementation and outcome progress; (h) World Bank team supervision visits to sites and corresponding Aide Memoires; and (i) ISRs. These M&E elements were officially formalized in the POM and accepted by all parties after consultations among the DoE, utilities, PMU, and the URA. 55. Throughout implementation, these M&E activities were timely fulfilled by the responsible parties except for the MTR. The DoE was responsible for the overall coordinating and implementing the M&E activities during the implementation of the project. The World Bank team regularly updated the progress in achieving results and reporting the monitored indicators in the ISRs accordingly. Overall, it is considered that data were collected methodologically with some irregularities in the monitoring of PDO3 and few intermediate indicators: (a) the ‘number of electricians employed and received training to install project design standard household wiring’ was monitored and accounted under the project, but it is not attributable to it and (b) the ‘women’s groups who received an information bulletin about the GPOBA subsidy available under the project’ is misreported as it reflects female individual instead of women groups. M&E Utilization 56. The implementing agency and the World Bank team used the indicators to monitor physical progress and make appropriate adjustments to plans. Particularly, tracking the low-income households receiving household wiring under the project was key to adjust the project closing date and the POM. Both factors triggered issuance of two Restructuring Papers, as discussed in previous sections of this report. Overall Rating Quality of M&E Page 24 of 51 The World Bank GPOBA Improved Electricity Access (P133701) Rating: Modest B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE Environmental and Social Safeguards Compliance 57. Environmental and Social safeguards. The project was classified as Category B and triggered OP/BP 4.01 (Environmental Assessment). In alignment with this policy, an ESMF was prepared.39 Oversight of ESMF compliance was the responsibility of the DoE,40 while the utilities were responsible for overseeing contractors and implementing mitigation measures. Due to the nature of its operations, the social and environmental risks of the GPOBA IEA Project were perceived to be low. Main impacts from the project were identified as removal of obstacles, such as trees or shrubs, minor traffic, or earth disturbance from digging holes and trenches for the installation of poles and wires. These impacts were triggered upon voluntary service request from applicants and their informed consent. The project did not result in any major impact. 58. The project also triggered OP 4.10 (Indigenous Peoples). Adverse impacts on indigenous peoples’ resources, culture, or livelihoods were not foreseen and the project did not result in adverse impacts of this nature. 59. Utility policies aligned with World Bank policies on involuntary impacts, compensations, and resettlement. The project triggered OP 4.12 (Involuntary Resettlement). Utility policies required compliance with the requirement of no involuntary resettlement impacts on land or assets of applicants and other affected parties such as landlords or neighbors. In addition, according to the utility policies, any compensation claim upheld should be calculated at current market value. Compensations for clearance of obstructions, as required to make the connections, were not contemplated under this project in view of the voluntary nature of participation. To comply with World Bank policies for management of involuntary resettlement impacts, a Resettlement Policy Framework was included in the ESMF of the project. However, the project involved only voluntary donation of affected land and assets, which was planned to be made in accordance with the voluntary land access guidance in the framework and operating rules of the two utilities. 60. The project defined a grievance redress mechanism (GRM), backed by the URA. According to the Communications Plan developed by the DoE for the GPOBA IEA Project, the URA was responsible for recording and managing any project-related grievances and report to the World Bank through the DoE. The reporting had to include information on the nature and number of project-related complaints resolved, time, and costs to arrive at a resolution. GRM channels included telephone numbers published on the URA’s website. In practice, there is no tracked evidence of the grievances received and the resolutions made to address them. 39 The ESMF included a community participation and consultation plan to be conducted by the DoE, involving the utilities. Following the consultation plan, a public information bulletin was released, and stakeholder consultation meetings were held with printed copies of the ESMF. 40 At the time of ESMF preparation, the DoE was formally called the Department of Energy, Mines and Mineral Resources (DEMMR). Page 25 of 51 The World Bank GPOBA Improved Electricity Access (P133701) Fiduciary Compliance 61. Financial management. The assessment of project compliance with the World Bank policy OP/BP 10.00 was positive. The PMU financial management arrangements already in place for other four World Bank projects were used in this project as well. Recognizing that the last financial order is due by June 2019, the financial management of the project is considered to have been Satisfactory throughout project implementation. All interim financial reports and financial orders were submitted on time with unmodified opinions. The DoE regularly reviewed the actual project budget and expenditures. The DoE also managed, maintained, and reconciled all activities related to the project Designated Account. Overall, it is considered that the project has complied with the national and World Bank financial management requirements. 62. According to the Aide Memoire of February 2014, the financial risk of the project was assessed as Substantial due to the Moderately Unsatisfactory performance of the PMU in managing topics such as budgeting, accounting, internal controls and audits, flow of funds, financial reporting, and external audit. The project was designed for the DoE to receive fiduciary support (on both financial and procurement issues) from the PMU. The consultant providing support in this capacity transitioned to become DoE staff in the early implementation years and continued assuming this role until project closing. With very limited support from the PMU, the DoE has fulfilled financial management duties satisfactorily through its staff members. This is further discussed in section III. 63. Procurement. The project procurement processes followed the World Bank Guidelines: Procurement under IBRD Loans and IDA Credits and the Guidelines: Selection and Employment of Consultants by World Bank Borrowers. Procurement practices followed by UNELCO and VUI were acceptable to the World Bank and in agreement with the World Bank Procurement Guidelines relating to Public-Private Partnership Arrangements, which were valid at the time of project approval. 64. Given the limited capacity and experience of the DoE to carry out procurement processes as an implementing agency, the available procurement capacity of the DoE was regularly monitored to ensure sufficient support to the DoE in this domain. Throughout implementation, the DoE’s capacity was considered sufficient to handle procurement issues given the small number and size of contracts to carry out under the project.41 The World Bank team carried out the necessary post review of procurement. C. BANK PERFORMANCE 65. Quality at entry. The World Bank identified, prepared, and appraised the operation in such a manner that it was likely to achieve the planned development objective. The project objective was relevant for Vanuatu’s long-term goals, and the project approach proposed by the World Bank was realistic for the country context. The indicators identified were mostly appropriate to inform on the project objectives’ achievements. The technical design of the project was relatively sound with weaknesses related to: (a) the implementation approach leaving options for customers to acquire 41 The contracts carried out under the project consisted on contracts for operating goods, individual consultants, and the independent auditor (firm), procured through the General Auditor’s Office. The contracts for operating goods (one vehicle, two laptops, and a camera) signed under the project reached an overall amount of about US$26,200. The overall value of contracts for individual consultants (IVA, financial management consultant, information management and communications consultant, MTR consultant, and project administration assistant) reached US$350,000. Page 26 of 51 The World Bank GPOBA Improved Electricity Access (P133701) household wiring through the project or other parties in a country context that lacks wiring safety standards; (b) unclear links in the theory of change between the activity outputs or intermediate results and the targeted outcome of standards and certification regime in place and training of electricians; and (c) exclusion of the trenches digging and preparation works from the services provided under the project. Safeguards and fiduciary policy compliance and policy institutional arrangements were strong. Overall, implementation arrangements were sound. 66. Quality of supervision. Supervision was underpinned by experienced team members, their low turnover, and the continuity of the task team leader. Supervision inputs included periodical missions conducted at least once a year, as well as regular reporting of operations and threats to development outcomes. The Aide Memoires and ISRs offer pertinent information to inform on project progress. These documents capture little information on: (a) project management and verification in absence of an “on- grid� officer and an IVA; (b) progress toward the adoption of standards and the certification regime; and (c) the exclusion of light bulbs coverage in UNELCO concession areas. Supervision on the field was limited with two technical field trips conducted by the Bank team during project implementation according to the Aide Memoires and ISRs. 67. As part of the supervision inputs, an MTR was produced. The MTR is a thorough, clear, and concise document that identified threats to the achievement of the development objective. The preparation of the MTR was delayed and finalized seven months before the initial project closing date. Closer World Bank supervision and actions taken for the timely preparation of the MTR together with a requirement for its accomplishment in the legal agreement would have helped avoid the delay in the actions taken to reduce and mitigate the various emerging issues during implementation. Once the MTR identified these issues, the World Bank took prompt action to restructure the project twice to resolve main shortcomings affecting implementation. The restructurings, particularly the second, were not conducted timely to allow sufficient time to significantly rectify implementation progress. Particularly before the MTR took place, the World Bank could have: (a) downgraded the project performance ratings to ‘moderately unsatisfactory’ earlier during the implementation as the actual project disbursements significantly lagged behind projections from the project initiation and throughout its lifetime; and (b) intensified its supervision efforts to effect the necessary changes in project design on time and address the slow project progress in the UNECLO concession area. For example, this could have accelerated the identification of issues related to the amperage limit, as well as the amendment in the POM to this effect earlier than it happened in 2016. Finally, based on the insufficient results achieved in training and capacity building through the project and given the low disbursements rate achieved in Components 2 and 3 (respectively, 28 percent and 29 percent), the World Bank could have enhanced its supervision on the consultancy support and capacity building delivered through the project to: (a) strengthen the DoE’s shortage of human resources, particularly on project management, and reinforce project implementation; and (b) support the utilities in training electricians. 68. Fiduciary matters were supervised satisfactorily on a regular basis. The close supervision was underpinned by the experienced team fiduciary specialists. Safeguards supervision was less intense due to: (a) the low environmental and social risks identified; (b) unavailability of staff, resulting in short-term consultants covering safeguards supervision for much of the duration of the project; and (c) a relatively low engagement of available safeguards’ specialists during supervision, as suggested by the low disbursement rate achieved in Component 3 (29 percent). On this point, it must be noted that safeguards supervision ought to be covered under the project budget given that individual budget lines for safeguards Page 27 of 51 The World Bank GPOBA Improved Electricity Access (P133701) were unavailable as the project was financed through a trust fund. When unavailable, human resources aimed at covering safeguards supervision could have been reinforced with appropriate consultancy support. The low environmental and social risks identified and little involvement of safeguards specialists in this project resulted in limited safeguards missions accomplished throughout implementation, with the specialists invited to join the missions on an ad hoc basis and only at the MTR and afterwards. Overall Rating of Bank Performance Rating: Unsatisfactory. D. RISK TO DEVELOPMENT OUTCOME 69. The utilities’ economic losses in operations and maintenance to supply electricity to small domestic consumers could pose a risk to guaranteeing the sustainability of the outcomes over time. Cross- subsidies from larger consumers, which are considered in the tariff review process, may offset the impact of such losses in the utilities. In the absence of such cross-subsidies or an offset mechanism, recurrent losses incurred in the utilities might negatively affect their financial health. V. LESSONS LEARNED AND RECOMMENDATIONS 70. The preparation and implementation of the GPOBA IEA Project have been relevant learning experiences for the World Bank and the borrower. These lessons will be valuable for the future implementation of other investment operations in the country in general and in the fields of access to electricity and OBA schemes in particular. The following lessons can be derived from these experiences. 71. Energy access is highly contextual. The application of best practices globally does not guarantee the success of electrification programs in local contexts. The flexibility and ability to change parameters effectively and efficiently according to the local context is key for successful implementation. For example, changes in project design, such as subsidy increases for household wiring or subsidy coverage extension for the preparation of cable trenches, could have been accomplished through a disciplined track of POM amendments while relieving the project from the additional burden of formal restructurings. This flexibility and ability come hand in hand with thorough monitoring and sound market intelligence. Conducting good analytics up front and monitoring their veracity and relevance during project implementation is essential to understand customers’ needs and demand and how it may fluctuate over the lifetime of the project. This can help inform the project on key parameters for electrification projects, such as WTP, the reluctance to arrange the digging work of the trenches, or the amperage needs. When not performing up to realistic expectations, thorough and timely monitoring and analytics can detect the project design parameters that are not fitting to the local context and the project should allow their correction in a flexible manner before they lead to major implementation bottlenecks. The borrower and the World Bank should therefore avoid rigid project designs, leaving opportunities to promptly restructure projects and conduct pertinent changes in the project documents and framework as local context changes during implementation. To address changes on project design promptly, the World Bank shall maintain a rigorous monitoring of the quality in analytics at design and overall implementation afterward. 72. The GPOBA IEA Project proposed an open design for implementation of household wiring to customers and utilities. Within the same project, two approaches were observed during project Page 28 of 51 The World Bank GPOBA Improved Electricity Access (P133701) implementation. Under one approach, prevalent within VUI concession areas, service connections and household wiring were provided through the utility. Under the second approach, observed mainly in UNELCO concession areas, the acquisition of household wiring was unbundled from the installation of service connections. While service connections would remain the responsibility of the utility under this approach, various customers decided to acquire household wiring through a third party. The second approach is commonly adopted as utilities, which tend to manage risks related to defects in consumers’ installations and appliances or unsafe practices leading to incidents within the customers’ private premises. However, in a context lacking enforcement of safety standards for electrical connections, the second approach can also result in household wiring not meeting international standards and an unsuccessful achievement of outcomes. In such contexts, implementing this second approach requires a thorough consideration of due diligence obligations that must be foreseen at project inception and accomplished throughout project implementation. Due diligence obligations could include the IVA verification of safe household wiring in place prior to electrifying households regardless of the wiring services supplier. This could avoid the exposure of householders to wiring safety hazards when not provided through the project, utility or not following safety standards. This is particularly relevant in the absence of a regulation that enforces the application of such standards. In the absence of a regulatory framework on electrical wiring standards and certification valid in the country, end-to-end solutions provided through the utility (such as those adopted in the first approach) have proven to achieve outcomes, deliver outputs, and guarantee actual access up to international standards to beneficiaries. It must be noted that, either bundled with service connections or as a separate service, providing household wiring or any other service beyond the meters requires paying special attention to legal liabilities in case of accidents within the private premises. Hence, for these services, best practices suggest a clear demarcation of roles and responsibilities up front. Going forward, the borrower is encouraged to adopt an appropriate regulatory framework on electrical wiring safety when continuing its efforts on providing electricity access. The borrower and the World Bank need to assess whether the appropriate regulatory framework on electrical wiring safety is in place when adopting the second approach to ensure that all households are wired and energized up to international standards. 73. The GPOBA IEA Project relied on two private utilities to implement the subsidies scheme. One of the utilities assumed uniform rates for the service connections and household wiring implemented under the GPOBA IEA Project, regardless of the distances from the power poles and meter readers to the houses and any extra material or work required to accomplish the connections and wiring beyond the estimates. In those cases where the real costs exceeded the estimates, the utility absorbed the differential cost. The second utility assumed a case-by-case pricing model for both services—connections and wiring. Under this modality, customers absorbed the differential amounts when costs exceeded the estimates. In both cases, the utilities were responsible for ordering and purchasing the materials up front, effecting the initial expenses until receiving the subsidy reimbursement. In this respect, three observations can be made. First, while the first modality met the connections target, both utilities felt the administrative burden of up-front investments and reimbursement periods of up to a year from purchase, which slowed down their activities. In return, the revenues obtained from connecting low-income consumers were insufficient to complement the utilities’ cash flow. Under the second modality, the utility handled its own options for service connections, possibly observing less administrative costs or more benefits in doing so than in pursuing the GPOBA IEA Project. Thus, the second observation indicates the need to include working capital support to cash-constrained utilities, for which the OBA scheme of reimbursement of funds upon verification might be unaffordable. Finally, it also must be observed that when there are no legally Page 29 of 51 The World Bank GPOBA Improved Electricity Access (P133701) contractual reasons to split the service connections and household wiring subsidies, one subsidy scheme encompassing both can simplify the overall process, increasing the potential for lower transaction costs and accelerating implementation. Simplifying the subsidy schemes, reducing the administrative burden for private intermediaries, and observing their incentives to meet targets or supporting them with working capital can significantly contribute to motivate private utilities and investors to participate in OBA subsidy schemes. OBA financing schemes need to ensure agile administrative processes and consider the incentives of private intermediaries to guarantee their sustainability. This is particularly relevant when the private intermediaries are small and medium enterprises participating in large OBA programs. 74. The PMU did not have sufficient allocated human resources to adequately implement the project and the project disbursed about 29 percent of the funds allocated to project management support. From the early implementation years, the DoE absorbed competent staff that carried out the GPOBA IEA fiduciary matters and a communications officer was hired under the project to support the PMU. However, the GPOBA IEA Project manager’s responsibilities fell on the on-grid officer role at the DoE and this position remained vacant for significant periods throughout implementation. When available, the on- grid officer’s time and responsibilities were shared among a portfolio of projects, leaving the officer with limited time to supervise the GPOBA IEA Project. In addition, the PMU did not engage qualified staff or consultants in social and environmental safeguards for consultations, oversight, monitoring, and support. Insufficient human resources dedicated to the PMUs can critically affect project supervision, performance, and outcomes. The borrower needs to ensure that PMUs are well resourced and that key project management positions are filled. In parallel, the World Bank needs to ensure that enough funds are not only dedicated to support the PMU in meeting its resources’ needs but also effectively disbursed with this purpose. In countries with multiple operations under preparation and implementation, a centralized management unit can be an efficient strategy to guarantee support across the portfolio on common project management areas such as fiduciary, safeguards, or communications. 75. The project indicators included the adoption of wiring standards and a training or certification regime. The definition of such indicator remains unclear as it does not specify what ‘adoption’ implies nor does it indicate the local authorities to take action in this respect to meet the target. In addition, the project did not define activities to achieve such outcomes and there was no budget allocation to the development of standards, thus a document on Electrical Safety Standards was prepared with ESDP funds. A results chain with weak links between the operation’s interventions, the outputs, and the desired outcomes and unclear definition of targets and indicators can jeopardize the attribution of results and compromise the project performance. In the same line, another example refers to the indicators defined to track progress on communication and dissemination project interventions, ‘Low Income Households received an information bulletin on household electricity safety under the project’ and ‘Community groups who received an information bulleting about the GPOBA subsidy available under the project’. The distribution of bulletins to households and groups cannot easily prove the effectiveness of communicating and disseminating information to householders and citizens as such distribution does not necessarily imply that users read the bulletins and assimilate the information. Teams need to get a deeper understanding of efficient ways to communicate and disseminate project interventions and effectively track progress on these interventions. Conducting customer surveys to get further understanding on this point can be helpful in some contexts. In any case, the definition of appropriate indicators, based on the project’s activities, is key to set realistic expectations and effectively track progress towards outputs and outcomes. Similarly, budget provisions need to be planned to support the achievement of each PDO indicator. Page 30 of 51 The World Bank GPOBA Improved Electricity Access (P133701) 76. Vanuatu is in an area prone to natural disasters such as cyclones or tsunamis. Because of the adverse effects of Category 5 Cyclone Pam, there were significant delays in project implementation in parts of the UNELCO concessionary area. High exposure to natural disasters can have a negative impact on the project’s progress and affect the implementation schedule. In cases of high risk to natural and climate change hazards, borrowers are encouraged to define mitigation plans and prepare emergency response mechanisms that help ensure prompt access to sufficient human and financial resources put at the disposal of the governmental actors participating in reconstruction works. The World Bank helps ensure this through mechanisms such as the Contingency Emergency Response Component, which provides swift response in the event of crisis or emergency, including adverse impacts associated with natural disasters. 77. The World Bank supervision on social and environmental safeguards was financed through trust fund budget. Limited involvement and under-resourcing of the specialists available to cover regular safeguards supervision resulted in loose supervision of the GPOBA IEA Project in social and environmental issues. Low risks were nevertheless identified in relation to these issues in the specific case of the GPOBA IEA Project. However, incorporating enough economic resources at project design and sufficiently involving human resources as required for a sound World Bank safeguards supervision is key to ensure that projects financed through trust funds received adequate World Bank support. On this point, the coordination among trust funds, borrowers, and the World Bank is crucial from the project preparation stage. Page 31 of 51 The World Bank GPOBA Improved Electricity Access (P133701) ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS A. RESULTS INDICATORS A.1 PDO Indicators Objective/Outcome: The Project Development Objective is to increase sustainable access to formal grid-based electricity services within Vanuatu's electricity concession service areas for low income customers through tar Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion People provided with new or Number 0.00 21437.00 21437.00 9844.00 improved electricity service 02-Jun-2014 30-Jun-2018 31-Dec-2018 31-Dec-2018 People provided with access Number 0.00 21437.00 21437.00 9844.00 to electricity by hhold connections-Grid 02-Jun-2014 30-Jun-2018 31-Dec-2018 31-Dec-2018 Comments (achievements against targets): Partially achieved. This indicator was originally phrased as "People provided with access to electricity under the project by household connections" in the Results Framework. The indicator started to be phrased as "People provided with access to electricity by hhold connections-Grid" in ISR. Seq. 2, dated June 2015. 15. A new indicator, ‘people provided with new or improved electricity service’, was added in ISR Seq. 5 (June 2017) to comply with World Bank Guidance on Corporate Results Indicators (issued in April 2017). The latest actual value available according to the last Project Progress Report for July 2018– December 2018, dated February 2019, indicates that this indicator reached 10,716 (50.0%). Page 32 of 51 The World Bank GPOBA Improved Electricity Access (P133701) Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion People continuing to utilize Number 0.00 21437.00 21437.00 9687.00 electricity connections three months after connection 02-Jun-2014 30-Jun-2018 31-Dec-2018 31-Dec-2018 Comments (achievements against targets): Partially achieved. The Project Progress Report for January 2018–June 2018, dated August 2018, explains that the value achieved at closing does not include the verification of 32 households connected to that date, as the reporting fell within the three-month validation period. This explains the difference towards the achievement of indicator PDO1 at closing. The latest actual value available according to the last Project Progress Report for July 2018 – December 2018, dated February 2019, indicates that this indicator reached 10,716 (50.0%). Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Community contributions Amount(USD) 0.00 511875.00 511875.00 155972.00 (customer co-contribution) in the total project cost 02-Jun-2014 30-Jun-2018 31-Dec-2018 31-Dec-2018 Comments (achievements against targets): Partially achieved. The latest actual value available according to the last Project Progress Report for July 2018 –December 2018, dated February 2019, indicates that this indicator reached US$151,406 (29.6%). This indicator was misreported at closing, as the community contributions reported then (US$155,972) is higher than the latest available value achieved (US$151,406), which was confirmed in the latest Project Progress Report and the Borrower's ICR. Page 33 of 51 The World Bank GPOBA Improved Electricity Access (P133701) Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Wiring rules (standards) Text 0 Standards in place Standards in place Draft standards have adopted and a training regime been developed. The in place Government will submit the draft standards for Council of Ministers approval. 02-Jun-2014 30-Jun-2018 31-Dec-2018 31-Dec-2018 Comments (achievements against targets): Not achieved. Draft standards were developed. The development of the standards was financed under another World Bank-financed project (ESDP). Wiring rules were not adopted and a training regime was not in place by the project closing date. According to the Borrower’s ICR, t his is due to issues that arose from combining household wiring (unregulated part of the business) with the technical regulations for the network (regulated part of the business) within the same legislative instrument. A.2 Intermediate Results Indicators Component: Subsidy funding for new connections Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion People qualifying for a new Number 0.00 21437.00 21437.00 9844.00 household connection to the utility network under the 02-Jun-2014 30-Jun-2018 31-Dec-2018 31-Dec-2018 Page 34 of 51 The World Bank GPOBA Improved Electricity Access (P133701) project Comments (achievements against targets): Partially achieved. The latest actual value available according to the last Project Progress Report for July 2018-December 2018, dated February 2019, indicates that this indicator reached 10,716 (50.0%). Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Annual volume of electricity Megawatt 0.00 3024.00 3024.00 2848.58 sold to all Low Income hour(MWh) Households connected under the project 02-Jun-2014 30-Jun-2018 31-Dec-2018 31-Dec-2018 Comments (achievements against targets): Almost fully achieved. The latest actual value available according to the last Project Progress Report for July 2018 –December 2018, dated February 2019, indicates that this indicator reached 3,832.98 MWh (126.7%). This result exceeds the intended target. Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Annual revenue generated Amount(USD) 0.00 596996.00 596996.00 284522.00 from electricity sales to Low Income Households connected 02-Jun-2014 30-Jun-2018 31-Dec-2018 31-Dec-2018 under the project Page 35 of 51 The World Bank GPOBA Improved Electricity Access (P133701) Comments (achievements against targets): Partially achieved. The latest actual value available according to the last Project Progress Report for July 2018 –December 2018, dated February 2019, indicates that this indicator reached US$434,207 (72.7%). Component: Household wiring - compliance assessment and wiring where required Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Low Income Households Number 0.00 4375.00 4375.00 1288.00 receive household wiring to standard design under the 02-Jun-2014 30-Jun-2018 31-Dec-2018 31-Dec-2018 project Comments (achievements against targets): Partially achieved. The latest actual value available according to the last Project Progress Report for July 2018 –December 2018, dated February 2019, indicates that this indicator reached 1,720 (39.3%). Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Number of electricians Number 0.00 15.00 15.00 28.00 employed and receive training to install project design 02-Jun-2014 30-Jun-2018 31-Dec-2018 31-Dec-2018 standard household wiring Page 36 of 51 The World Bank GPOBA Improved Electricity Access (P133701) Comments (achievements against targets): Exceeded. However, the actual value achieved at completion represents electricians trained by VUI under its own financing. The project did not directly financed training. Component: Capacity building, training, outreach and independent verification Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Low Income Households Number 0.00 4375.00 4375.00 800.00 received an information bulletin on household 02-Jun-2014 30-Jun-2018 31-Dec-2018 31-Dec-2018 electricity safety under the project Comments (achievements against targets): Partially achieved. Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Community groups who Number 0.00 35.00 35.00 19.00 received an information bulletin about the GPOBA 02-Jun-2014 30-Jun-2018 31-Dec-2018 31-Dec-2018 subsidy available under the project Page 37 of 51 The World Bank GPOBA Improved Electricity Access (P133701) Comments (achievements against targets): Partially achieved. Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Women’s groups who received Number 0.00 244.00 244.00 300.00 an information bulletin about the GPOBA subsidy available 02-Jun-2014 30-Jun-2018 31-Dec-2018 31-Dec-2018 under the project Comments (achievements against targets): Exceeded. However, the actual value achieved represents number of women who received the information bulletin about the GPOBA subsidy available under the project. The latest actual value available according to the last Project Progress Report for July 2018-December 2018, dated February 2019, indicates that this indicator reached the value of 19. The large disparity between this result and the actual value reported at completion or the target set at appraisal indicates that this indicator was misreported. Page 38 of 51 The World Bank GPOBA Improved Electricity Access (P133701) B. ORGANIZATION OF THE ASSESSMENT OF THE PDO Objective/Outcome 1: The development objective was to increase sustainable access to formal grid-based electricity services within the Recipient’s electricity concession service areas for low income customers through targeted subsidies. As per the original loan and project closing date: 1. People provided with access to electricity under the project by household connections (Number) 2. People continuing to utilize electricity connections three months after connection (Number) Outcome Indicators 3. Community contributions (customer co-contribution) in the total project cost (US$) 4. Wiring rules (standards) adopted and a training regime in place (Standards) Additional outcome indicators as per project closing date: 5. People provided with new or improved electricity service (Number) 1. People qualifying for a new household connection to the utility network under the project (number) 2. Annual volume of electricity sold to all low-income households connected under the project (kWh) 3. Annual revenue generated from electricity sales to low income households connected under the project (US$) 4. Low Income Households received household wiring to standard design under the project (number) 5. Number of electricians employed and receive training to install project design standard household wiring Intermediate Results (number) Indicators 6. Low income households received an information bulletin on household electricity safety under the project (number) 7. Community groups who received an information bulletin about the GPOBA subsidy available under the project (number) 8. Women’s groups who received an information bulletin about the GPOBA subsidy available under the project (number) Component 1: OBA subsidies for new electricity connections for low-income households Key Outputs by Component • Intermediate indicator 1: People qualifying for a new household connection to the utility network under the project Page 39 of 51 The World Bank GPOBA Improved Electricity Access (P133701) (linked to the achievement End target: 21,437 people of the Objective/Outcome Output (actual value at project closing): 9,844 people gained household connections to the utility 1) network (45.9 percent of its intended target). The latest actual value available according to the last Project Progress Report for July 2018–December 2018, dated February 2019, indicates that this indicator reached a value of 10,716 (50.0 percent of its intended outcome). • Intermediate indicator 2: Annual volume of electricity sold to all low-income households connected under the project End target: 3,024 MWh Output (actual value at project closing): 2,848.58 MWh sold to all low-income households connected under the project (94.2 percent of its intended target). The latest actual value available according to the last Project Progress Report for July 2018–December 2018, dated February 2019, indicates that this indicator reached 3,832.98 MWh (126.8 percent of its intended target). • Intermediate indicator 3: Annual revenue generated from electricity sales to low income households connected under the project End target: US$596,996 Output (actual value at project closing): US$284,522 revenues were generated from electricity sales to low-income households connected under the project (47.7 percent of its intended target). The latest actual value available according to the last Project Progress Report for July 2018–December 2018, dated February 2019, indicates that this indicator reached US$434,207 (72.7 percent of its intended target). Component 2: OBA subsidies for household wiring for low-income households accessing electricity services under the project • Intermediate indicator 4: Low income households receive household wiring to standard design under the project End target: 4,375 households Output (actual value at project closing): 1,288 low-income household received household wiring (29.4 percent of its intended target). The latest actual value available according to the last Project Progress Report for July 2018–December 2018, dated February 2019, indicates that this indicator reached a value of 1,720 (39.3 percent of its intended target). • Intermediate indicator 5: Number of electricians employed and receive training to install project design standard household wiring Page 40 of 51 The World Bank GPOBA Improved Electricity Access (P133701) End target: 15 electricians Output (actual value at project closing): 28 electricians received training. While this indicator value meets and surpasses the target, there is no evidence that these electricians were employed, and they were all trained by VUI utility on its own costs. This result can therefore not be attributed to the project. Component 3: Implementation support for project management, communications and outreach, and training • Intermediate indicator 6: Low income households received an information bulletin on household electricity safety under the project End target: 4,375 households Output (actual value at project closing): 800 low-income households received an information bulletin on household electricity safety under the project. This value represents 18.3 percent of its intended target. • Intermediate indicator 7: Community groups who received an information bulletin about the GPOBA subsidy available under the project End target: 35 community groups Output (actual value at project closing): 19 community groups received an information bulletin about the GPOBA subsidy available under the project. This indicator met 54.3 percent of its intended target. • Intermediate indicator 8: Women’s groups who received an information bulletin about the GPOBA subsidy available under the project End target: 244 women’s groups Output (actual value at project closing): 300 women received an information bulleting about the GPOBA subsidy available under the project. It must be noted that the number tracked corresponds to female individuals and not groups. The latest actual value available according to the last Project Progress Report for July 2018-December 2018, dated February 2019, indicates that this indicator reached a value of 19 (7.8 percent). The large disparity between this result and the actual value reported at closing or the target set at appraisal seems to indicate that this indicator was misreported. Page 41 of 51 The World Bank GPOBA Improved Electricity Access (P133701) . ANNEX 2. PROJECT COST BY COMPONENT Amount at Actual at Percentage Components Approval Project Closing of Approval (US$) (US$) (%) OBA subsidies for new electricity connections for low income 2,200,000 701,836 31.9 households OBA subsidies for household wiring for low income 2,100,000 597,038 28.4 household accessing electricity services under this project Implementation support for project management, 350,000 100,944 28.8 communications, outreach, and training Independent verification 200,000 94,029 47.0 Total 4,850,000 1,493,847 30.8 Page 42 of 51 The World Bank GPOBA Improved Electricity Access (P133701) ANNEX 3. RECIPIENT, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS First of all, the Department of Energy (DoE) would like to thank you for this opportunity to comment on the World Bank's (WB) Implementation Completion Report (ICR). The WB's ICR is comprehensive and captures a lot of lessons learnt during the implementation of the project that will be beneficial for other similar projects in Vanuatu and elsewhere. Generally, the overall connection figures were achieved but not under the GPOBA project. Approximately only 50% of the connections were undertaken under the GPOBA project and as a result not all funds were disbursed. The Government is disappointed that the Project Development Objective’s (PDO) 1, 2 & 3 were not achieved, and is also disappointed that a significant amount of the grant funds could not be utilized under the project. The main reason for the results not being realized is that in one of the service providers concession, there was an identical alternative program that resulted in customers not being connected under the GPOBA project because of certain constraints in the project (5 A limitation and the requirement of 12 months not to upgrade). The Government was aware of the alternative program at the early stages of the GPOBA project and to ensure that the projects connection targets were achieved, it had entered to a Memorandum of Understanding (MoU) with the service providers to connect all eligible customers under the GPOBA project. The other service provider which did not operate an identical alternative program connected the number of households expected under the project and disbursed funds as expected under the project. We believe that if the customers had not choose the identical alternative program, PDO’s 1, 2 & 3 would have been achieved. Other than that, there were 2 issues identified in the early stages of the program. This led to the load limit on the connection and the cost of household wiring under Australian standards. However, this was resolved in a timely manner. PDO 4 was not achieved during the implementation period due to lack of policy clarity in terms of the design of the enabling legislation and partly due to lack of a strong determined personnel as a result of staff turnover at the DoE. In additional to the issues identified above, the project was also affected by the category 5 cyclone Pam which severely affected the UNELCO concession areas which further resulted in rehabilitation and construction works which took up to almost 2 years to complete. Management changes had UNELCO and staff turnover at the DoE during this period which had an effect on the implementation of the program. We note the overall outcome rating as unsatisfactory and from the GPOBA project's perspective in terms of results and disbursements and we agree with the rating. However, from the perspective of the number of low-income consumers being connected to the electricity grid, we are satisfied that that outcome was achieved. Prior to the GPOBA project, there were no subsidized connection programs in Vanuatu and Page 43 of 51 The World Bank GPOBA Improved Electricity Access (P133701) low-income earners were not connected to the grid or shared connections with their neighbors. The GPOBA project led to subsidized program being adopted and which are expected to continue has helped low income consumers access electricity. There has been an increase in connections by around 10% over the project period which can be largely attributed to the initiative regardless of whether this was achieved under the GPOBA project or under the service providers’ identical program. GPOBA had identified safety concerns arising from the design of the project. Our investigations revealed that household wiring for over 50% of all households in Vanuatu do not meet international standards. We had requested and hoped that the Bank would direct the remaining funds towards addressing the household wiring of all households. Unfortunately, we were advised that use of funds for this purpose was not in accordance with the GPOBA operating principles. With respect to key factors that affected the implementation and outcome during project preparation, we generally agree with the World Bank’s assessment of key design factors affecting implementation; • We note that the design was sound as demonstrated by the successful implementation of the project in accordance with the design by one of the service providers; • We were of capacity constraints as the implementation agency and that was the reason for entering into an arrangement with the service providers who are experts in this area to support the implementation of the project. The project worked aside from the loss of connections to an alternative program for various reasons; • The design of the project was based on a study by consultants Castalia prior to the preparation of the project. Castalia had identified the number of low-income consumers that required assistance and had undertaken the capacity to pay assessments which were used in the design; • The load limits and requirement to remain on the low-income tariff for 12 months were implemented to meet the “targeting� requirements by GPOBA to ensure only low -income households benefited from the program. This was the main reason that consumers chose the alternative program that did not have these constraints. • We do not have any specific comments on the matters affecting the project during implementation. We note the suggestion that at the time of the 2016 restructuring, the GPOBA and the UNELCO program should have been merged. We were aware of the UNELCO program prior to that restructuring and the approach taken to address that was through a Memorandum of Understanding whereby UNELCO would connect all households eligible under the GPOBA program under that program and would use money the saved to extend the grid. UNELCO wished to continue to offer its program to households who did not qualify to ensure consistency of connection offers across its concessions and avoid consumer complaints. However, as a result of confusion between programs and in some cases preference (there were other constraints under the GPOBA offer) there were a number of eligible households that chose the UNELCO offer. Page 44 of 51 The World Bank GPOBA Improved Electricity Access (P133701) • We note the lessons and recommendations which are useful for future such projects in Vanuatu or elsewhere. o During design phase there was emphasis on some consumer contribution that was affordable. The consumer contribution limits took into account the estimated costs of the services, the consumers’ capacity to pay with a preference towards “in kind’ contribution such as preparing the trenches, and the subsidy provided by GPOBA for such services in other countries. The preparation of the trenches by the consumers required the householders to consult with their neighbors and landlords to agree on the route in the village where title is not clear thereby reducing potential disputes and complaints. This arrangement worked out well as expected for VUI and also where household wiring was installed in UNELCO’s concessions. o The initial design during the early concept stage was for the service connection and the household wiring to be a combined service. During consultations the services providers pointed out that the service connection was a regulated service and that the service provider was obliged to provide the householders with service connections and that household wiring was not regulated. The householders are responsible for household wiring and carry any risks associated with that part of the service. The service providers agreed to facilitate household wiring and supervise the process but did not want to be held “responsible� for providing the service. The service providers also indicated that it was their practice to only connect households where the household wiring was considered to be safe and they will disconnect any, where it was found to be not safe. Since there were no known safety incidents and based on the service providers practice, the household wiring was to be provided as an option based on householders’ request. o In relation to administrative burden, we note that the project required a verification process that took time to complete and caused some delays and complaints from service providers. This was a requirement from GPOBA that could not be avoided. In relation the other design, verification, and payments of the scheme was well understood and easy. Page 45 of 51 The World Bank GPOBA Improved Electricity Access (P133701) ANNEX 4. SUPPORTING DOCUMENTS (IF ANY) Bibliography: 1. Department of Energy. 2019. https://doe.gov.vu/index.php?r=site/about. 2. Global Partnership for Results-based Approaches. 2014. GPOBA Annual Progress Report 2014. http://www.gprba.org/knowledge/publications/gpoba-annual-report-2014. 3. The World Bank and The United Nations. 2018. Knowledge into Action Notes. 4. Article 5 of the General Conditions of Public Electricity Supply in Vanuatu. 5. Pre-payment meters - Final Decision. Utilities Regulatory Authority. August 2013. 6. The World Bank. 2013. Project Information Document. Vanuatu Energy Sector Development Project. http://documents.worldbank.org/curated/en/608251468134372741/Vanuatu-Energy- Sector-Development-Project. 7. Utilities Regulatory Authority. December 2018. Electricity Factsheet 2012-2017. Port Vila, Vanuatu.http://www.ura.gov.vu/attachments/article/97/Electricity%20Fact%20Sheet%202018. pdf Working documents: 8. Borrower’s Implementation Completion and Results Report (2019); available in Annex 3 9. GPOBA Improved Electricity Access: P133701 Implementation Status and Results Report, Sequences 1 through 7 (from 2014 to 2018) 10. GPOBA Improved Electricity Access: P133701 Aide Memoires (from 2012 to 2018) 11. GPOBA Improved Electricity Access: P133701 Project Appraisal Document (2014) 12. GPOBA Improved Electricity Access: P133701 Global Partnership on Output-based Aid Grant Agreement (2014) 13. GPOBA Improved Electricity Access: P133701 Restructuring Paper (May 2018) 14. GPOBA Improved Electricity Access: P133701 Restructuring Paper (October 2018) 15. GPOBA Improved Electricity Access: P133701 Mid-term Review (2017) Page 46 of 51 The World Bank GPOBA Improved Electricity Access (P133701) 16. GPOBA Improved Electricity Access: P133701 Environmental and Social Management Framework and Resettlement Policy Framework (2013) 17. GPOBA Improved Electricity Access: P133701 Communications Plan (2015) 18. GPOBA Improved Electricity Access: P133701 Project Operations Manual (2014) 19. GPOBA Improved Electricity Access: P133701 Project Progress Report, reporting period: July 2014–December 2014 (February 2015) 20. GPOBA Improved Electricity Access: P133701 Project Progress Report, reporting period: January 2015–June 2015 (August 2015) 21. GPOBA Improved Electricity Access: P133701 Project Progress Report, reporting period: January 2016–June 2016 (August 2016) 22. GPOBA Improved Electricity Access: P133701 Project Progress Report, reporting period: July 2016–December 2016 (March 2017) 23. GPOBA Improved Electricity Access: P133701 Project Progress Report, reporting period: January 2017–June 2017 (June 2017) 24. GPOBA Improved Electricity Access: P133701 Project Progress Report, reporting period: July 2017–December 2017 (January 2018) 25. GPOBA Improved Electricity Access: P133701 Project Progress Report, reporting period: January 2018–June 2018 (August 2018) 26. GPOBA Improved Electricity Access: P133701 Project Progress Report, reporting period: July 2018–December 2018 (February 2019) 27. GPOBA Improved Electricity Access: P133701 Project Progress Report, reporting period: July 2018–December 2018 (May 2019) 28. Borrower’s Implementation Completion Report (June 2019) Page 47 of 51 The World Bank GPOBA Improved Electricity Access (P133701) ANNEX 5. ELIGIBILITY CRITERIA, SUBSIDY DESIGN, AND IMPLEMENTATION APPROACH 1. Eligibility for subsidies under the project. The project was design for consumers to be eligible to receive an OBA subsidy if they had no formal or individual connection to the electricity grid and were eligible to connect under the utilities’ category of ‘small domestic consumers. Under the utility practices valid at project design, customers under this category were charged subsidized tariffs equivalent to 34 percent of the base tariff for consumption of up to 120 kWh per month. Customers already connected to the grid would not be eligible for the subsidies, neither for service connections nor for household wiring. Both prepaid and postpaid meters were eligible under the GPOBA IEA Project. 2. Limiting the potential for non-eligible customers to seek subsidized connections under the GPOBA IEA Project. The connection would be technically constrained to 5 A for 12 months to avoid higher- consumption users, such as higher-income households and businesses to apply to the subsidy because the technical constraint was not supposed to be sufficient to meet their electricity requirements. The project targeted connections for low-income consumers with estimated consumption of 30 kWh per month. Non-eligible customers seeking the subsidized connection and upgrading the connection within the first 12 months would be required to refund the full amount of the subsidy to the utility (except in the final year of the project, when the restriction to upgrade the connection would be from the date of connection until the closing date of the project). Ineligible upgrades would be recorded by the utilities and adjusted in the utilities’ refund claims. 3. Subsidy design. The subsidy and the consumers’ contribution, as estimated at project design, are presented in table 5.1. The subsidies were estimated based on the average cost of inputs needed to connect and provide household wiring to low-income households. The affordability of the users’ contribution was determined on the basis of the WTP analysis, conducted by an independent consultancy firm. This analysis estimated that consumers could afford between US$110 (VUV 10,466) and US$170 (VUV 16,175), which represented approximately 31 percent to 48 percent of the average monthly income of these customers. The WTP analysis showed that the benefit to the average low-income consumer was around US$19 and US$23 per month, validated against the amount that consumers paid when using a shared connection. According to the PAD, the subsidy payments would vary by (a) whether a pole was required, (b) the meter was postpaid or prepaid, (c) the distance from the distribution line, and (d) whether the service connection was above the ground or underground. 4. Implementation approach. In practice, two approaches were adopted during implementation: (a) In VUI concession areas, service connections and household wiring were provided through the utility. The utility trained its own electricians to conduct the household wiring up to international standards. VUI assumed flat rates for the service connections and household wiring implemented under the GPOBA IEA Project, regardless of the distances from the power poles and meter readers to the houses and any extra material or work required to accomplish the connections and wiring beyond the estimates. In those cases where the real costs exceeded the estimates, the utility absorbed the differential cost. (b) In the UNELCO concession areas, the acquisition of household wiring was unbundled from the installation of service connections. Service connections remained the responsibility of Page 48 of 51 The World Bank GPOBA Improved Electricity Access (P133701) the utility and household wiring was carried out through a subcontractor. Some customers acquired household wiring through the subcontractor’s utility, while others did it through third parties. UNELCO assumed a case-by-case pricing model for both services—connections and wiring. Under this modality, customers absorbed the differential amounts when costs exceeded the estimates. Page 49 of 51 The World Bank GPOBA Improved Electricity Access (P133701) Table 5.1. Subsidy Design and Customer’s Contribution in UNELCO and VUI Concession Areas UNELCO VUI Costs Estimation of service For UNELCO, the estimation was based on actual historical costs with an upward adjustment. UNELCO’s estimation was connections costs also used for VUI estimated service connection costs. In both cases, it was US$590. Based on an assessment of costs carried out by the independent consultancy firm and adjusted to the distance from the Estimation of household utility meter box to a location inside the housea along with the costs to backfill the cable trench to international standards. wiring costs Validated by local electricians, it was US$444. Total estimated cost of service connections and US$1,034 household wiring Subsidy and Consumer Payment Scheme Service 80 percent of the actual cost of service connection, On average, US$476 for connection with postpaid meter and connection US$472. US$413 for prepaid meter. Subsidy Household The actual cost of wiring up to US$444. The actual cost of wiring up to US$444. wiring Total subsidy US$916 US$920 for postpaid meters and US$857 for prepaid meters. Service 20 percent of the actual cost of service connection, The amount in excess of a US$476 subsidy toward postpaid meters connection US$117. and the amount in excess of US$413 toward prepaid meters. Customer’s Household The actual cost of household wiring in excess of The actual cost of household wiring in excess of US$444. contribution wiring US$444. Total consumer US$117 if wiring does not exceed the subsidy No charge if the service connection and household wiring does not co-contribution coverage. exceed the subsidy. Note: a. Initially estimated at roughly 37 m average. Page 50 of 51 The World Bank GPOBA Improved Electricity Access (P133701) ANNEX 6. DETAILED RESULTS: SERVICE CONNECTIONS AND HOUSEHOLD WIRING UNELCO VUI TOTAL Service Service Service Household Wiring Household Wiring Household Wiring Period Connection Connection Connection (Number of (Number of (Number of (Number of (Number of (Number of households) households) households) households) households) households) July-December 0 0 19 19 19 19 2014 January-June 0 33 19 118 19 151 2015 January-June 139 339 361 364 500 703 2016 July-December 139 339 361 364 500 703 2016 January-June 312 573 472 474 784 1047 2017 July-December 344 1212 541 543 885 1755 2017 January-June 642 1361 646 648 1288 2009 2018 July-December 2018 642 1361 646 648 1388 2009 (value at closing) July-December 2018 942 1407 778 780 1720 2187 (latest value available)a Note: a. The numbers in this column represent the values of the indicators achieved by the project closing, in December 2018, but reported after closing in the last Project Progress Report for July 2018–December 2018 submitted in May 2019. Page 51 of 51