ARMENIA
Energy Efficiency Project
Report No. 135626
MARCH 28, 2019
© 2019 International Bank for Reconstruction This work is a product of the staff of The World RIGHTS AND PERMISSIONS
and Development / The World Bank Bank with external contributions. The findings, The material in this work is subject to copyright.
1818 H Street NW interpretations, and conclusions expressed in Because The World Bank encourages
Washington DC 20433 this work do not necessarily reflect the views of dissemination of its knowledge, this work may be
Telephone: 202-473-1000 The World Bank, its Board of Executive reproduced, in whole or in part, for
Internet: www.worldbank.org Directors, or the governments they represent. noncommercial purposes as long as full
attribution to this work is given.
Attribution—Please cite the work as follows: The World Bank does not guarantee the
World Bank. 2019. Armenia—Energy Efficiency accuracy of the data included in this work. The Any queries on rights and licenses, including
Project. Independent Evaluation Group, Project boundaries, colors, denominations, and other subsidiary rights, should be addressed to
Performance Assessment Report 135626. information shown on any map in this work do World Bank Publications, The World Bank
Washington, DC: World Bank. not imply any judgment on the part of The Group, 1818 H Street NW, Washington, DC
World Bank concerning the legal status of any 20433, USA; fax: 202-522-2625; e-mail:
territory or the endorsement or acceptance of pubrights@worldbank.org.
such boundaries.
Report No.: 135626
PROJECT PERFORMANCE ASSESSMENT REPORT
ARMENIA
ENERGY EFFICIENCY PROJECT
(TF-12163)
March 28, 2019
Financial, Private Sector, and Sustainable Development
Independent Evaluation Group
i
Abbreviations
ESCO energy service company
EBRD European Bank for Reconstruction and Development
ESA energy service agreement
GEF Global Environment Facility
GHG greenhouse gas
ICR Implementation Completion and Results Report
IEG Independent Evaluation Group
KWh kilowatt‐hour
NEEAP National Energy Efficiency Action Plan
NPV net present value
PPAR Project Performance Assessment Report
R2E2 Fund Renewable Resources and Energy Efficiency Fund
UNDP United Nations Development Programme
Director‐General, Independent Evaluation : Ms. Alison Evans
Director, IEG Financial, Private Sector and Sustainable Development : Mr. José Candido Carbajo Martinez
Manager, IEG Sustainable Development : Ms. Midori Makino
Task Manager : Ms. Victoria Alexeeva
ii
Contents
Preface .......................................................................................................................................................................... v
Summary .................................................................................................................................................................... vi
1. Project Background and Context .................................................................................................................1
Sectoral and Institutional Context .................................................................................................................. 1
Energy Efficiency Policy, Legal and Regulatory Framework ............................................................................... 1
Energy Efficiency Financing ........................................................................................................................................ 2
World Bank Group and Donor Support in Energy Efficiency ........................................................................... 2
2. Results.....................................................................................................................................................................6
3. What Worked, and Why?.................................................................................................................................9
Design and Preparation ....................................................................................................................................9
Implementation and Supervision ................................................................................................................. 10
4. What Didn’t Work, and Why? ..................................................................................................................... 11
Design and Preparation ...................................................................................................................................11
Implementation and Supervision ................................................................................................................. 12
5. Lessons ................................................................................................................................................................ 14
Appendix A. Project Ratings.............................................................................................................................. 19
Appendix B. Project Data and Other Issues ................................................................................................ 28
Appendix C. R2E2 Fund Energy Efficiency Investments, Financial Data, and Estimated Energy
Savings................................................................................................................................................................. 30
Appendix D. Overview of Main Donors and International Financial Institution Funds Available for
Energy Efficiency in Armenia ....................................................................................................................... 32
Appendix E. List of Existing Energy Efficiency Legal, Regulatory, and Normative-Technical
Documents in Armenia.................................................................................................................................. 33
Appendix F. Project Performance Assessment Report Overview ........................................................ 38
Appendix G. Methods and Evidence.............................................................................................................. 39
Appendix H. List of Persons Interviewed ...................................................................................................... 40
Appendix I. Bibliography .................................................................................................................................... 44
Appendix J. Borrower Comments ................................................................................................................... 45
iii
Tables
Table 1.1. Energy Efficiency Project Financing by Source and Component ($, millions) ................... 4
Table 2.1. Energy Efficiency Onlending of R2E2 Fund and Estimated Energy Savings ....................... 7
Figure
Figure 1.1. Energy Efficiency Project Theory of Change ................................................................................5
This report was prepared by Victoria Alexeeva who assessed the project in November 2018. Vahe
Odabashian and Artak Eloghlyan provided technical assistance as local consultants. The report was peer
reviewed by Istvan Dobozi and panel reviewed by Christopher David Nelson. Romayne Pereira provided
administrative support.
iv
v
Preface
This Project Performance Assessment Report (PPAR) of the Independent Evaluation Group (IEG) of the
World Bank Group evaluates the development effectiveness of the Energy Efficiency Project in Armenia
(P116680). The project was selected for a PPAR to learn from an innovative pilot project that influenced
the design and experience of other energy efficiency projects and interventions. Energy efficiency is of
strategic importance for the World Bank given its role in supporting climate change mitigation, which is a
major corporate priority.
The project development objective was to reduce energy consumption of social and other public facilities
in Armenia and decrease greenhouse gas emissions through the removal of barriers to the
implementation of energy efficiency investments in the public sector. The project was financed through a
Global Environment Facility (GEF) grant and government funds totaling $10.7 million. The project was
implemented in 2012–16.
This PPAR presents its findings and conclusions based on a review of the World Bank’s project
documentation, combined with a field mission to Yerevan, Armenia conducted jointly with the GEF’s
Independent Evaluation Office (IEO) in November 2018. IEG and IEO conducted interviews with a range
of different stakeholders linked to the project, including government officials, the implementing agency,
World Bank staff, other development partners, academia, beneficiaries, and the private sector.
This report is a pilot effort to reform the structure of the PPAR. The pilot simplifies the main text by
focusing on what worked and what did not, by covering other issues in the main text only if they were
significant in this project, and by emphasizing the learning function of the report rather than project
ratings. However, accountability elements are retained: project ratings and their justification are still
present in appendix A, and these ratings are still based on the IEG‐Operations Policy and Country
Services harmonized project evaluation guidelines, based on assessment against project objectives.
The PPAR mission was conducted together with Anna Birgitta Viggh, GEF IEO senior evaluation officer.
Artak Eloghlyan and Vahe Odabashian were local consultants who supported the mission in Armenia
and provided technical assistance. The PPAR team is grateful to Sylvie K. Bossoutrot, country manager,
and the World Bank Country Management Unit in Armenia for the IEG mission support. Emil Zalinyan
provided invaluable technical inputs, and Irina Tevosyan assisted with the most efficient mission
organization and logistics. The PPAR team appreciates the generous time and attention given by the
government institutions and all concerned parties (see appendix H for a complete list of stakeholders
who provided inputs for this project assessment).
Following standard IEG procedure, copies of the draft PPAR was shared with relevant government
officials for their review and comment. All comments are included in appendix J of this report.
v
Summary
Project Background and Description
The project development objective was “To reduce energy consumption of social and other
public facilities, and its global environmental objective was “To decrease greenhouse gas
emissions through the removal of barriers to the implementation of energy efficiency
investments in the public sector.” The project was appraised for $10.7 million financed
through a Global Environment Facility (GEF) grant of $1.82 million, $8.3 million from
government funding, and cofinancing from the implementing agency, the Renewable
Resources and Energy Efficiency (R2E2) Fund, estimated at $0.54 million. The GEF‐financed
project was carried out in 2012–16.
This was a pilot project designed to demonstrate a replicable and sustainable model for
energy efficiency investments in Armenia’s public sector. The project was to finance energy
efficiency upgrades and retrofits in public and social facilities and remove barriers that
hampered the wide penetration of energy efficiency investments in Armenia’s public sector.
Through public sector energy efficiency investments, the project intended to demonstrate
the viability of energy efficiency investments and their economic benefits. Technical
assistance was geared to help remove the existing barriers to the realization of energy
efficiency potential and create an enabling environment for energy efficiency in Armenia’s
public sector. These interventions were to lead to reduced energy consumption in public
and social facilities and a decrease in greenhouse gas (GHG) emissions in Armenia.
Results
The energy consumption of selected public and social facilities whose improvements
were funded by the project was reduced significantly. The project financed energy
efficiency retrofits and upgrades in eligible public and social buildings in Armenia. Sixty‐
three subprojects were completed at project closure, estimated to result in lifetime energy
savings of about 540.2 million kilowatt‐hours and CO2 emission reduction of about 145,700
tons. The results were almost three times the original targets set at appraisal: 216 million
kilowatt‐hours and 50,500 tons equivalent of CO2. The project invested in 124 public
buildings, exceeding its target, though this represented only a small fraction of the overall
needs. This was about 2 percent of more than 5,800 public buildings in Armenia, and
according to the government’s National Energy Efficiency Action Plan (NEEAP‐2), the
vi
commercial and public service sector has recorded a steady increase of energy consumption
since 2010.
The project contributed substantially to removing energy efficiency policy,
informational, and knowledge barriers. The government adopted the second National
Energy Efficiency Action Plan (NEEAP‐2) after technical support under the project.
Extensive support geared toward raising awareness helped address informational and
knowledge barriers to energy efficiency development, in addition to crowding in other
donor funds for energy efficiency investments. The GEF project served as a reference point
and source of knowledge to initiate energy efficiency activities in the public and private
sectors. All stakeholders interviewed during the evaluation mission acknowledged the R2E2
Fund’s role in energy efficiency information initiatives and in raising awareness of potential
energy efficiency economic and environmental benefits. The government adopted energy
service agreements (ESAs) developed under the project as sample agreements for energy
efficiency procurement (Decree No. 728N, dated 25 June 2015). This was an important legal
milestone to enable public entities to implement energy efficiency retrofits under the project.
However, ESAs cannot be applied after project closure as they were adopted only for the
project’s duration.
The project was not successful in addressing financing barriers that constrain the
implementation of energy efficiency investments at scale in the public sector. The project
introduced a new financing model for energy efficiency retrofits in public and social
facilities based on a repayment scheme from energy savings. An energy saving agreement
was developed to facilitate financing of energy efficiency investments in public and social
buildings without public budget support. However, the model has not become sustainable
because the barriers to financing were not removed. Under the project, the R2E2 Fund
provided turnkey services (energy audit, procurement, detailed design, financing,
construction, and monitoring) for energy efficiency upgrades in public buildings. It put
significant effort into the capacity building of construction firms, introducing them to
performance‐based contracting, a new procurement concept, and measurement and
verification of energy savings. Despite these efforts, the acquired capacity is largely
contained within the fund, which cannot make large investments in energy efficiency in the
public sector due to limited funds. Most important, the R2E2 Fund did not partner with a
sizable number of commercial banks in a cofinancing mode. There was no spillover effect
and continuity in the project’s results and no development impact beyond the project
interventions.
vii
What Worked, and Why?
Several features in the project design and implementation facilitated successful
operationalization of the new concept for energy efficiency financing in the public sector.
The project was equipped from the start with necessary technical and financial data to target
public buildings with the highest energy‐saving potential. The R2E2 Fund did an extensive
stocktaking exercise collecting the information on key technical parameters of buildings,
energy consumption, and location to correctly identify the specific needs. The project
effectively targeted public facilities with the highest potential for energy savings factoring
both investment cost and energy savings. Unlike traditional procurement, which is based on
the lowest cost, the R2E2 Fund used a net present value (NPV)–based procurement that
specified minimum energy savings, but allowed bidders to propose their best technical
solutions to maximize energy savings. Bidder selection was based on the highest NPV
rather than the lowest cost.
In addition, project implementation benefited greatly from a semiautonomous, highly
professional project implementation unit, the R2E2 Fund, which had considerable
experience, capacity, and effective management. Many stakeholders identified the R2E2
Fund’s performance as a major driver of the successful implementation of subprojects
because of its effective leadership and staff performance.
What Did Not Work, and Why?
The project design focused on facilitating the implementation of subprojects in the public
sector and building energy efficiency capacity for a range of stakeholders, but it failed to
identify the essential characteristics of a sustainable energy efficiency investment model.
Energy specialists note that it is important for an energy efficiency revolving fund to model
partnership agreements for financial institutions to involve them in cofinancing or partial
credit guarantee arrangements. Because the model introduced by the project is not being
replicated and taken up at scale, it represents a missed opportunity to leverage cofinancing
from local commercial banks and other private financiers in support of a learning‐by‐doing
approach to build sustained, market‐based capacity to develop and finance energy
efficiency projects.
To increase the demonstration effects and development impact of similar projects, the R2E2
Fund should have focused on promoting the energy efficiency agenda on a sustainable basis
by partnering with different energy efficiency players. There was no concerted effort to
viii
engage and build the capacity of a wider network of energy efficiency players outsourcing
various functions to government entities, municipalities, financial institutions, and
commercial entities. Besides subcontracting private companies for design and construction,
no other entities had a role in the market, for example, identifying energy efficiency projects,
preparing economic and financial appraisals, and conducting procurement. Greater efforts
might have been necessary to partner with donor organizations. Given that there are
numerous donors in Armenia with significant flows of grants and loans in energy efficiency,
the R2E2 Fund might have been more active in securing additional financing to build a
stronger capital base and ensure the flow of energy efficiency benefits in the public sector.
This might have required a coordinated, donor‐supported effort that the government should
lead.
IEG project ratings are described in appendix A.
Lessons
Energy efficiency revolving funds can be market enablers by partnering with
commercial and financial institutions, but there are few prospects for scale up and
energy efficiency market transformation without the commitment of private
businesses. Ultimately, the domestic financial sector is the key source of sustainable
energy efficiency finance. Piloting new models for energy efficiency service
provision needs to create suitable local conditions and develop the capacity of
commercial financiers through initial cofinancing and risk mitigation. The R2E2
Fund in Armenia did not make systematic efforts to work and partner with financial
institutions and other commercial players in the implementation of energy efficiency
subprojects. The approach of creating a model of energy services provision for which
the R2E2 Fund is the main agent and construction companies are largely passive
implementing subcontractors limits longer‐term market impacts. In such a setting,
the risks of continuity of energy‐saving services become the risks of the continuity of
the revolving fund itself.
Practical demonstration of the technical and financial feasibility of an innovative
energy efficiency transaction program can only influence positive systemic change
in the legal and regulatory framework if there is government commitment to the
approach and long‐term funding. A new financing agreement was piloted under
this GEF project to enable public entities to implement energy efficiency retrofits
without affecting the budget of the subproject entities. As a result, public
ix
organizations were able to enter into financing agreements that paid for the
investment through future monetized energy savings. However, without sufficient
government oversight and commitment to these approaches, the sustainability of the
model is impossible. The government adopted ESAs as sample agreements for
energy efficiency procurement only for the project’s duration, and ESAs cannot be
applied after the project closure. The project demonstrated the actual energy
efficiency improvements have the capacity to repay the investment, but this requires
legal support, access to capital and a system for the application of savings to future
investment.
Appropriate legislation and regulation can provide incentives to undertake energy
efficiency measures, but they are not sufficient without a strong government
energy efficiency agency in place that is responsible for monitoring and
enforcement. The energy efficiency regulatory and legal framework has advanced
significantly (attributable to the project in part), but enforcement and actual
implementation are still lagging in several dimensions. Although the government
enacted a new modality of energy efficiency delivery in the public sector that was
operationalized successfully under the project, it has not followed up with measures
to ensure a wider replication and scale‐up. A market transformative policy function
cannot be farmed out to a revolving fund or a public energy service company
lacking authority. The government of Armenia has no effective institutional focal
point for energy efficiency implementation support, though this is critical for the
wider diffusion of the successful energy efficiency delivery mechanisms in the public
sector. As a result, international donors largely set and implement Armenia’s energy
efficiency agenda.
The design of a pilot project needs to go beyond demonstration effects and lay the
foundation for sustainable operations over time. Although this energy efficiency
project effectively targeted public facilities with the highest potential for energy
savings, essentially harvesting low‐hanging fruit, the broader project experience
revealed that many public buildings in Armenia did not comply with the eligibility
criteria. They are in a poor state of maintenance and repair that would require
substantial investments in structural rehabilitation, not just energy efficiency
retrofits. Energy cost savings alone supported by the project will not be sufficient to
cover the required investments in these buildings unless supplemented by
concessionary budgetary funds or grants. Instead of being developed as a direct
x
payment agent to finance the retrofits, the R2E2 Fund could have focused on
enabling commercial financing by acting as an intermediary guarantee fund. As a
guarantee fund, it could have fully de‐risked principal payments to commercial
banks on a rolling basis, thereby attracting commercial financing. However, because
the R2E2 Fund made direct payments, it largely decapitalized itself without
leveraging its initial funding to attract commercial financing.
José Carbajo Martínez
Director, Financial, Private Sector, and
Sustainable Development
Independent Evaluation Group
xi
1. Project Background and Context
Sectoral and Institutional Context
1.1 Three key challenges were identified in the energy sector at the time of project
appraisal in 2011: emerging power supply gap, threatened energy security, and
increasingly unaffordable energy tariffs (World Bank 2012). Energy efficiency
represented a large untapped energy resource to Armenia. The Readiness for Investment
in Sustainable Energy survey conducted by the World Bank in 2013–14 scored Armenia’s
energy efficiency at only 37 out of 100. A World Bank study found that Armenia could
save about 5 percent of its gross domestic product by investing in energy efficiency and
estimated that public sector investments had the highest return on investment overall
(World Bank 2008). The report noted that energy efficiency potential was not being
realized because of informational, financing, regulatory, and institutional barriers.
Energy Efficiency Policy, Legal and Regulatory Framework
1.2 The Armenian government prioritized the realization of economically viable
energy efficiency potential as one way to solve key challenges in the energy sector. In
2004, the National Parliament passed the Law on Energy Savings and Renewable
Energy, creating the legal basis for energy efficiency in Armenia. However, the law was
largely declarative and had no provisions for mandatory energy efficiency measures or
enforcement. In 2011, the law was amended to add the concept of energy labeling for
equipment and clarifications about energy audits. In 2016, amendments and addendums
were made to the law, which defined a number of mandatory requirements, such as
energy efficiency requirements for buildings; energy balance requirement; labeling
requirements for energy consuming devices; disposal requirement for life‐threatening
and environmental hazardous devices; and energy expertise requirement. The Law on
Energy Saving and Renewable Energy was an important milestone that defined many
concepts necessary for energy efficiency, serving as the basis to kick‐start energy
efficiency regulatory measures and as the genesis for an energy framework.
1.3 At the policy making and implementation levels, the most important documents
at the time were the National Program on Energy Efficiency and Renewable Energy of the
Republic of Armenia (2007) and the Action Plan of the Government of Republic of Armenia
Aimed at the Implementation of the National Program on Energy Efficiency and Renewable
Energy of Republic of Armenia (that is, the first phase of the National Energy Efficiency
Action Plan (NEEAP‐1), adopted in 2010). The system of codes and standards related to
energy efficiency of buildings was already well developed at project appraisal.1
However, the construction norms (though mandatory) were neither checked for
compliance by the licensed entities nor enforced by the relevant government bodies,
1
while application of the standards was completely voluntary. There was also no definite
policy in place for public procurement of energy efficient products. Article 13 of 2011
Amendment of the Republic of Armenia Law on Energy Saving and Renewable Energy
referred to energy assessment of procured products and services to be implemented “in
case if necessary.”
1.4 At the institutional level, there were capacity gaps in policy‐making and
regulatory bodies, and there was no dedicated energy efficiency agency in Armenia that
could anchor and provide leadership on the implementation of energy programs by
coordinating government policies in energy efficiency, serving as a clearinghouse for
information on energy efficiency, and coordinating various donor‐funded programs.
Energy Efficiency Financing
1.5 The government lacked sufficient resources for budget financing of energy
efficiency investments in the public sector. Although the banking sector and capital
markets were relatively developed in the country, local financial institutions were not
lending for energy efficiency projects because of perceived high credit risks, high
transaction costs, lack of experience with energy efficiency project finance, and lack of
skills to prepare bankable energy efficiency investment projects. Awareness among
financial institutions, private entities, public institutions, and energy consumers about
the potential financial gains from energy efficiency investments was limited. Most
importantly, there was limited borrowing capacity of public sector entities.
1.6 In the local energy efficiency service market, about 30 private firms provided
engineering or consulting services for energy efficiency investments in Armenia.
However, these firms did not participate as performance contractors or energy service
companies (ESCOs) that specialize in energy efficiency project development and
implementation and are considered an important market‐based mechanism involved in
the delivery of energy efficiency investments (World Bank 2008). ESCOs are broadly
defined to include any company using energy performance contracting as part of energy
efficiency investment transactions.2 There are many benefits in using ESCOs to help
scale up energy efficiency projects through offering specialized technical and financial
services for project design and implementation (Hofer, Limaye, and Singh 2016); they
are also dependent on access to finance from financial institutions because of their small
capital base.
World Bank Group and Donor Support in Energy Efficiency
1.7 An extensive number of donor activities support energy efficiency in Armenia,
and that number has increased over time. At project preparation, the World Bank Group
had several operations in the energy sector,3 including a development policy loan, under
2
which the government adopted the Energy Efficiency Action Plan for 2010–13 that
prioritized energy efficiency measures for various sectors. The International Finance
Corporation supported the banking sector in developing energy efficiency lending and
raising awareness on sustainable energy finance. The European Bank for Reconstruction
and Development and the U.S. Agency for International Development (USAID) also
supported the banking sector to increase the availability of bank financing for energy
efficiency projects. The Global Environment Facility (GEF) administered by the United
Nations Development Programme (UNDP) led the external support for policy
improvements in the energy efficiency area, including energy efficiency in buildings,
appliances, and equipment. It had a parallel project that was complementary in many
aspects to the World Bank–implemented GEF energy efficiency project in technical
support.
Project Design and Financing
1.8 The project development objective was “To reduce energy consumption of social
and other public facilities” (GEF grant agreement dated April 20, 2012, page 7). The
global environmental objective was “To decrease greenhouse gas emissions through the
removal of barriers to the implementation of energy efficiency investments in the public
sector” (World Bank 2012, 4).
1.9 Project preparation was originally initiated in April 2009 under the World Bank’s
Electricity Supply Reliability and Energy Efficiency Project. In 2011, the project’s
transmission component required more financing than originally budgeted; resources
were reallocated from the energy efficiency component to make up for the shortfall, so
the energy efficiency component was dropped. In late 2011, the government and the
World Bank decided to implement the energy efficiency component as a stand‐alone
GEF project, with cofinancing from reflows from earlier World Bank investment projects
with the implementing agency, the Renewable Resources and Energy Efficiency Fund
(R2E2). The government established the R2E2 Fund in 2005 as a nonprofit organization
with a mandate to promote the development of renewable energy and energy efficiency
markets in Armenia, and to facilitate investments in these sectors.4
1.10 The project was financed through a GEF grant of $1.82 million, $8.3 million from
government funding that consisted of repayments of the revolving funds under the two
closed World Bank projects (Urban Heating Project and Renewable Energy Project), and
cofinancing from the R2E2 Fund originating from service fees and interest under the
proposed energy service agreements (ESAs) estimated at $0.54 million.5 The project was
appraised for $10.66 million (table 1.1). The project implementation period was planned
for about three years, from March 27, 2012 to June 30, 2015; the closing date was
3
subsequently extended for another year to June 30, 2016 because of a slower‐than‐
expected start.
Table 1.1. Energy Efficiency Project Financing by Source and Component ($, millions)
Actual
Costb
Appraisal Percentage of
Components Estimatea (2012) (2016) Appraisal
Component 1: Energy efficiency 8.7 9.83 113
investments
Component 2: Technical assistance 1.96 1.49 76
Total 10.66 11.32 106
Source: a. From World Bank 2012.b. World Bank 2016.
1.11 The project’s theory of change was premised on demonstrating replicable and
sustainable models for energy efficiency service provision and related cobenefits.
Through energy efficiency investments in public and social facilities, the project
intended to demonstrate the viability of energy efficiency investments, thus building the
demand for energy efficiency financing. Technical assistance was geared to help remove
the existing barriers to the realization of energy efficiency potential and create an
enabling environment for energy efficiency in the public sector. These interventions
were to lead to reduced energy consumption by public and social facilities and
decreased greenhouse gas (GHG) emissions (figure 1.1).
4
Figure 1.1. Energy Efficiency Project Theory of Change
Note: EE = energy efficiency; GHG = greenhouse gas emissions.
The project included two components:
Component 1: Energy efficiency investments in public facilities—supported
energy efficiency investments in social and other public facilities, for example,
schools, kindergartens, hospitals, administrative buildings, and street lighting.
The target facilities were selected from more than 5,000 public and social facilities
in Armenia, for which the R2E2 Fund had collected information on key technical
parameters of buildings, energy consumption, and location.
Component 2: Technical assistance—supported (i) training and technical
assistance to build the technical and financial capacity of the R2E2 Fund; (ii)
program marketing and capacity building to target public sector entities to
address energy efficiency information and knowledge gaps, build demand for
financing, and improve the sustainability of energy savings; (iii) energy
efficiency policy development; (iv) capacity building for energy service providers
and other market actors to screen, design, appraise, finance, implement, and
measure energy efficiency investments in the public sector; and (v) project
management.
5
2. Results
2.1 The energy consumption of selected public and social facilities whose
improvements were funded by the project was reduced significantly. The project
financed energy efficiency retrofits and upgrades in eligible public buildings. Sixty‐three
subprojects were completed at project closure, estimated to result in lifetime energy
savings of about 540.2 million kilowatt‐hours and CO2 emission reduction of about
145,700 tons. The results were almost three times the original targets set at appraisal: 216
million kilowatt hours and 50,500 tons equivalent of CO2.6
2.2 The project invested in 124 public buildings, exceeding its target, though this
represented only a small fraction of the overall needs. This was about 2 percent of the
more than 5,800 public buildings in Armenia. According to NEEAP‐2, the commercial
and public service sector has recorded a steady increase in energy consumption since
2010. Project‐financed facilities included schools and universities, hospitals and medical
centers, penitentiary institutions, street lighting, kindergartens, and theaters. The project
primarily financed insulation of walls, basements, and attics; repair or replacement of
external doors and windows; window optimization; reflective surfacing of walls behind
radiators; improvements or replacement of boilers and heating systems; and
replacement of mercury vapor lamps with high‐pressure sodium vapor lamps (or light‐
emitting diodes) and incandescent bulbs with compact fluorescent lamps.
2.3 No energy efficiency subprojects were implemented after project closure,
despite a vast potential for energy efficiency investments in buildings. The R2E2 Fund
did not make investments in energy efficiency retrofits in the public sector because of
limited funds. Even though repayments from energy savings under subprojects were
received in a timely manner, these were not sufficient to continue the same model of
investments without additional flow of financial resources. The average payback period
for subprojects was eight years,7 and the fund had about $1 million of funds. No
additional financing was secured from other sources, including the government or
donors.
2.4 The R2E2 Fund resorted to onlending to deal with the challenge to its own
sustainability. Toward the project closure, the fund started to explore opportunities to
onlend its remaining capital through financial institutions at a higher interest rate. It
reached agreements with one bank and two credit organizations to onlend to non‐
gasified communities and individuals to support energy efficiency solutions, including
street lighting, insulation works for kindergartens and other community buildings,
acquisition and installation of solar panels for water heaters, and photovoltaic stations in
households (table 2.1).8 The disbursement of funds started at the end of 2017, 18 months
6
after the project. However, this is a temporary solution for the fund because its
onlending operations raise questions in the context of the existing legislation, which
allows lending operations only to licensed financial organizations.
Table 2.1. Energy Efficiency Onlending of R2E2 Fund and Estimated Energy Savings
Annual Energy
Financial Institutions Type of Projects Beneficiaries
Savings (kWh)
Energy efficiency for
ACBA-Credit Agricole Bank CJSC 6 communities 549,170
communities
Energy efficiency for non- 394 households
ACBA Leasing CJSC n.a.
gasified communities and 1 LLC
Energy efficiency for non-
Global Credit UCO CJSC 509 households 2,612,200
gasified communities
Source: Renewable Resources and Energy Efficiency Fund, Armenia.
Note: kWh = kilowatt-hour; LLC = limited liability company; R2E2 Fund = Renewable Resources and Energy Efficiency
Fund.
2.5 The project contribution to energy efficiency policy development in Armenia
is notable. The R2E2 Fund completed an assessment of the implementation progress
under the first National Energy Efficiency Action Plan (NEEAP‐1), which described a set
of programmatic and policy measures for energy efficiency improvement for all
economic sectors of the country. It became the basis for developing and approving the
second phase of the Republic of Armenia NEEAP‐2 on February 2, 2017 (2017–18) that
assessed the effectiveness of ongoing efforts by the government, donors, international
financial institutions, and the private sector to improve the efficiency of energy use since
2010, when Armenia’s first NEEAP was developed.9 NEEAP‐2 identified important
barriers to accelerating the implementation of energy efficiency measures and proposed
measures to help overcome them. It is one of Armenia’s most important documents in
the energy efficiency field that sets the policy environment.
2.6 Extensive support geared toward raising awareness helped address
informational and knowledge barriers to energy efficiency development. The GEF
project served as a reference point and source of knowledge to initiate energy efficiency
activities in the public and private sectors. All stakeholders interviewed during the
evaluation mission acknowledged the R2E2 Fund’s role in energy efficiency information
initiatives and raising awareness of energy efficiency cobenefits. Local officials,
particularly the Municipality of Yerevan, confirmed their knowledge of the R2E2 Fund
model of energy efficiency investments in the public sector that increased awareness of
the economic potential for improving energy efficiency in municipal public buildings.
Yerevan Municipality is initiating an energy efficiency project with financing from other
donors (a loan from the European Investment Bank, grants from the Eastern Europe
Energy Efficiency and Environment Partnership‐E5P, and the Green Climate Fund
7
administered by the UNDP and GEF). This project will implement both seismic
resilience strengthening (48 buildings) and energy efficiency measures (147 buildings) in
public facilities of Yerevan, including kindergartens, and medical, cultural, and athletic
buildings. The R2E2 Fund is also known for organizing workshops and an annual event
(Energy Week) that brings together a wide range of stakeholders in the energy sector,
including the government, the financial and private sectors, nongovernmental
organizations, academia, and donors. The R2E2 Fund was incorporated in a Covenant of
Mayors as a Supporter and is assisting the communities in preparation of Sustainable
Energy Plan, and development of energy efficiency measures for community facilities.
2.7 The project attribution is unclear regarding the removal of regulatory barriers
for the implementation of energy efficiency investments in the public sector. The
government made changes to 13 provisions of the Law on Energy Savings and
Renewable Energy during the project implementation period,10 but these resulted from
the concerted effort of other energy efficiency players in Armenia, particularly the
UNDP‐GEF that provided support in equipment labeling, certification, and
establishment of building codes, among others. The UNDP office in Armenia confirmed
close collaboration with the R2E2 Fund in the technical area during project
implementation. The government adopted ESAs developed under the project as sample
agreements for energy efficiency procurement (Decree No. 728N, dated 25 June 2015).
This was an important legal milestone to enable public entities to implement energy
efficiency retrofits under the project. The legal adoption of ESAs opened the potential to
be used in contracts that would pay for investment through future savings, however,
ESAs could not be applied after the project closure.
2.8 The net present value (NPV)–based procurement approach developed under
the project has not been adopted formally. The current Law of the Republic of Armenia
on Procurement (adopted December 16, 2016, last amended March 23, 2018) includes
provisions for “non‐price” conditions and criteria (for example, stipulating minimum
savings and maximum consumption) for the procured object (article 13), bidding (article
30), and selection of the winner (article 34). This means there are no legal barriers to use
NPV (or any other method, for that matter) in public procurement. The Procurement
Law specifies that a public facility itself should proceed with tender to select a company
to perform energy service in its building. The R2E2 Fund can be selected by a public
entity via tender to proceed with an NPV‐based procurement, however it needs to be
registered in the Anti‐Monopole Committee as a unique fund to proceed with energy
saving measures in the facilities. Alternatively, a new draft degree could facilitate this,
and the fund prepared a draft decree in 2017, but no progress has been made. At the
time, the R2E2 Fund is negotiating with the government to accelerate the approval
process.
8
2.9 Capacity and training activities had a positive effect, but there was no scale‐up
in application of the acquired knowledge and skills for energy efficiency solutions in
the public sector. The R2E2 Fund put significant effort into capacity building of
construction firms, introducing them to performance‐based contracting, a new
procurement concept, and measurement and verification of energy savings. The R2E2
Fund developed its own internal capacity and technical skills to identify, assess, and
implement energy efficiency projects. Under the project, the R2E2 Fund provided
turnkey services (energy audit, procurement, detailed design, financing, construction
and monitoring) for energy efficiency upgrades in public buildings. Project stakeholders
praised its technical expertise, particularly energy auditing, as confirmed through the
interviews. Despite these efforts and positive feedback, the acquired capacity is largely
contained within the R2E2 Fund, which no longer invests in energy efficiency in the
public sector because of lack of funds. The evaluation team found that none of the
construction firms supported under the project developed specialized ESCO‐type
services similar to those carried out by the R2E2 Fund. Construction firms still search for
tenders and opportunities to continue business as usual, though some expressed interest
in this business model provided the availability of concessional financing and long‐term
funds.
2.10 The project piloted, tested, and demonstrated an innovative concept for
financing of energy efficiency investments in the public sector, but these have not
been replicated to date. The project built a demand for energy efficiency financing in the
public sector,11 but the model has not become sustainable. In practice, the R2E2 Fund
acted as a public ESCO.12 It was a self‐contained entity that performed an ESCO‐type
function providing ESAs. Besides the subcontracting of private firms for design and
construction, no other entities had a role in the market, for example, identifying projects,
preparing financial appraisals, and conducting procurement. There was no concerted
effort to engage and build the capacity of a wider net of energy efficiency players
outsourcing various functions to government entities, municipalities, financial
institutions, and commercial entities. Most important, the R2E2 Fund did not partner
with a sizable number of commercial banks in a cofinancing mode. Therefore, there was
no spillover effects and continuity, and no development impact beyond the project
interventions.
3. What Worked, and Why?
Design and Preparation
3.1 This pilot initiative was built on extensive analytical work and experience in
the sector. A technical analysis was performed for Energy Efficiency Armenia (2008),
9
which estimated that about 5 percent of the country’s gross domestic product could be
saved through energy efficiency investments, and investments in public facilities had the
highest returns with paybacks of two to 10 years. A subproject pipeline was identified
during project preparation. The R2E2 Fund did an extensive stocktaking exercise
collecting the information on key technical parameters of buildings, energy
consumption, and location (World Bank 2012). The fund built on the experience of
working with the World Bank and the information and data collected under earlier
operations in the public sector, including energy savings from the implementation of
energy efficiency measures and financial viability of energy efficiency subprojects.13 This
facilitated a solid basis for the project initiation.
3.2 The effective analysis correctly identified the specific needs. The project was
well prepared and equipped from the start with necessary technical and financial data to
target public buildings with the highest energy‐saving potential. Financing of facilities
was demand‐driven, but subject to some basic screening criteria and secondary technical
and financial eligibility criteria. The project put an effective monitoring and evaluation
system in place to assess the impact of energy efficiency improvements. An energy‐
saving agreement was developed to facilitate financing of energy efficiency investments
in public and social buildings without budget support to demonstrate a new financing
model and the economic benefits of energy savings.14
Implementation and Supervision
3.3 Implementation benefited greatly from a semiautonomous, highly
professional project implementation unit, the R2E2 Fund. The R2E2 Fund had
considerable experience, capacity, and effective management. Many stakeholders
identified the R2E2 Fund’s performance as a major driver of the successful
implementation of subprojects because of its effective leadership and staff performance.
From its functions as a project implementation unit under the World Bank projects, the
R2E2 Fund evolved into an established commercial service provider and is marketing its
services, taking on repayment and performance risks, and charging fees during the
project implementation. The dedicated leadership and strong staff performance helped
the project to carry out energy efficiency investments effectively in the public sector.
3.4 Marketing efforts were essential for supporting effective implementation. The
R2E2 Fund had to work around barriers that included a lack of incentives to implement
energy efficiency projects, and a lack of capacity of public sector entities and design and
construction firms. Potential clients were not interested in energy efficiency investments
at the beginning of the project, or they preferred grant or budget support for energy
efficiency. Implementation progress was very slow during the first 18 months, with
delays in the finalization and signing of ESAs and setbacks on procurement—the first
10
five tenders failed, with one or fewer responsive bids (World Bank 2016). The R2E2
Fund intensified its public awareness campaign, including in focused marketing to
target clients, lobbying through line ministries, and advertisements on television, radio,
and other media outlets. With awareness increasing about energy efficiency and
increased energy tariffs, the demand for energy efficiency investments grew once the
initial subprojects were disseminated, accelerating the pace of implementation under the
project.
3.5 Accurate targeting and the applied financial model facilitated the project
achievement of its development objective of energy consumption reduction in public
facilities. The selection criteria for public buildings and a choice of procurement method
helped ensure the highest level of energy savings, thus maximizing financial returns
under the project.15 The project effectively targeted buildings with the highest potential
for energy savings, factoring both investment cost and energy savings. The application
of an NPV‐based procurement resulted in more savings per dollar invested and
significant overachievement of the project targets. Unlike traditional procurement,
which is based on the lowest cost, the R2E2 Fund used an NPV‐based procurement that
specified minimum energy savings but allowed bidders to propose their best technical
solutions to maximize energy savings. Selection was based on the highest NPV rather
than the lowest cost. All bids were required to meet or exceed the minimum energy
savings level, which was about 30 percent. Overall, the project experience revealed that
many public and social buildings in Armenia did not comply with the selection criteria
and are in a poor state of maintenance and repair that would require substantial
investments in rehabilitation and structural retrofits. Unless some budgetary funds or
grants are offered, energy cost savings alone will not be sufficient to cover the required
investments in these buildings. For example, of the 326 applications the R2E2 Fund
received, 64 percent were rejected because they did not meet the eligibility criteria,
mostly in relation to low‐baseline energy use (that is, underheating).
4. What Didn’t Work, and Why?
Design and Preparation
4.1 A shortcoming at project design was not factoring in cofinancing from
financial or commercial entities. The project design focused on facilitating the
implementation of subprojects in the public sector and building energy efficiency
capacity for a range of stakeholders, but it failed to identify the essential characteristics
of a sustainable energy efficiency investment model. Energy specialists note that it is
important for an energy efficiency revolving fund to model partnership agreements for
financial institutions to involve them in cofinancing or partial credit guarantee
11
arrangements, indicating that a minimum level (5–10 percent) of own financing by
project sponsors is a good practice in energy efficiency funding. Given the legal
constraints imposed on budget entities, the fund could have acted as a commercial
banks’ co‐financing/guarantee partner. Financing could have been outsourced to a bank
in a co‐financing mode or a guarantee form if co‐financing partners were not
comfortable with the risk profile of the sub‐projects or sub‐project entities. Evidence
from other countries suggests that ultimately, the local banking sector is the key source
of sustainable and sizable flows of finance for energy efficiency in most countries.
Because the model introduced by the project is not being replicated and taken up at
scale, it becomes apparent that this was a missed opportunity to leverage cofinancing
from local commercial banks and other private financiers in support of a learning‐by‐
doing approach and for building a sustained market‐based capacity to develop and
finance energy efficiency projects. The project has not addressed a financing gap
constraining the implementation of energy efficiency investments in the public sector at
scale. The energy efficiency public sector niche remains the least developed in Armenia.
Public buildings accounted for only 0.3 percent of energy efficiency investments
channeled by international financial institutions for energy efficiency financing through
local financial institutions in 2010–15.16
Implementation and Supervision
4.2 The project had only modest spillover effects beyond the works under the
project, despite intensive engagement and capacity building of construction firms.
The NPV‐based procurement used under the project prompted construction companies
to seek more modern and energy‐effective design solutions and technologies, bringing
the business‐like mindset and sense of ownership to both beneficiaries and contractors.
Many construction companies learned about opportunities in the field of energy
efficiency, received trainings from the R2E2 Fund, and developed their capacities in
procurement and monitoring of energy savings, but have remained classical
construction companies. None of the companies that dealt with the R2E2 Fund’s energy
efficiency initiatives assumed ESCO‐type activities similar to those performed by the
fund. Based on the insights gained from interviews with the construction companies,17
the fund has not propagated the establishment of ESCO companies with relevant
guidance and technical help on how to become ESCOs. The fund assumed all of the
business risks, leaving bidders with only the task of proposing energy‐saving solutions,
completing the construction works, and ensuring energy savings.18 For increasing the
demonstration effects and development impact of similar projects, the fund should have
focused on promoting the energy efficiency agenda on a sustainable basis by partnering
with different energy efficiency players instead of focusing on the fund’s sustainability.
Preparation of feasibility studies and implementation supervision could have been
12
outsourced to qualified engineering firms. Energy specialists agree that partnering on
the identification, financial and economic appraisal, and implementation of the energy
efficiency subprojects could have generated a broader market demonstration effect and
emergence of independent players that could ensure sustainable and continuous energy
efficiency services in the market.
4.3 Activities that sought to maximize energy savings did little to address
building structural norms. The project applied the financial model using an NPV‐based
procurement program that maximized energy savings with minimum investment in
public and social buildings. The emphasis was on energy efficiency retrofits and
upgrades that could ensure high energy savings and returns on investment. The project
design did not allow to include measures that would not ensure cost recovery. The
NPV‐based procurement developed and used during the project was widely acclaimed
as a method that contributed to factoring in the future benefits of energy efficiency
measures, prompting market players to seek and use more modern and effective energy
efficiency solutions and technologies. Although NPV‐based procurement could be more
effective and efficient than the least‐cost criteria traditionally used for public
procurement that generally specify design, technical experts on the building codes
raised concerns that this model that emphasized financial performance might make
contractors attribute lower priority to meeting technical requirements of the energy
efficiency‐related codes and standards in the country. This particularly applies to the
relatively new codes: the Republic of Armenia Construction Norms (building codes)
RACN 24‐01‐2016 “Thermal Protection of Buildings” (Ministry of Urban Development
Order No. 120‐N of June 16, 2016) and RACN 22‐03‐2017 “Artificial and Natural
Lighting” (Urban Development Committee Order No. 56‐N of April 13, 2017). In the
design, bidding, and implementation processes, construction firms might tend to focus
on NPV and have little incentive to follow the energy efficiency codes strictly. The major
issue that adds complexity is that the codes (norms) in Armenia, though mandatory, are
not always enforced vigorously, especially relative to energy efficiency refurbishing of
buildings versus new construction.
4.4 Greater efforts might have been needed to partner with financial institutions
and donor organizations. Failure to include relevant activities and to plan for
cofinancing from financial institutions in design does not preclude the project team from
being flexible and proactive to explore these opportunities during the project
implementation to ensure continuity of energy efficiency investments in the public
sector. In addition, because there are numerous donors in Armenia with significant
flows of grants and loans in energy efficiency, the R2E2 Fund might have been more
active in securing additional financing to build a stronger capital base.19 Based on the
feedback from various stakeholders and the mission’s own analysis, energy efficiency
13
donor efforts are not coordinated in Armenia, and there is competition for donor
funding among academia, nongovernmental organizations, and financial institutions in
Armenia—each is trying to highlight its own efforts and visibility in the energy field.
The government of Armenia is supportive of developing the energy efficiency–enabling
environment, but in the implementation of energy efficiency, it relies on significant
funding from donors and international financial institutions.20 There is no strong
institutional focal point in Armenia to coordinate various donor‐sponsored programs on
energy efficiency to ensure a broader systemic impact. This might have required a
coordinated, donor‐supported effort that the government should lead.
5. Lessons
5.1 Energy efficiency revolving funds can be market enablers by partnering with
commercial and financial institutions, but there are few prospects for scale up and
energy efficiency market transformation without the commitment of private
businesses. Ultimately, the domestic financial sector is the key source of sustainable
energy efficiency finance. Piloting new models for energy efficiency service provision
needs to create suitable local conditions and develop the capacity of commercial
financiers through initial cofinancing and risk mitigation. The R2E2 Fund in Armenia
did not make systematic efforts to work and partner with financial institutions and other
commercial players in the implementation of energy efficiency subprojects. The
approach of creating a model of energy services provision for which the R2E2 Fund is
the main agent and construction companies are largely passive implementing
subcontractors limits longer‐term market impacts. In such a setting, the risks of
continuity of energy‐saving services become the risks of the continuity of the revolving
fund itself.
5.2 Practical demonstration of the technical and financial feasibility of an
innovative energy efficiency transaction program can only influence positive systemic
change in the legal and regulatory framework if there is government commitment to
the approach and long‐term funding. A new financing agreement was piloted under
this GEF project to enable public entities to implement energy efficiency retrofits
without affecting the budget of the subproject entities. As a result, public organizations
were able to enter into financing agreements that paid for the investment through future
monetized energy savings. However, without sufficient government oversight and
commitment to these approaches, the sustainability of the model is impossible. The
government adopted ESAs as sample agreements for energy efficiency procurement
only for the project’s duration, and ESAs cannot be applied after project closure. The
project demonstrated the actual energy efficiency improvements have the capacity to
14
repay the investment, but this requires legal support, access to capital and a system for
the application of savings to future investment.
5.3 Appropriate legislation and regulation can provide incentives to undertake
energy efficiency measures, but they are not sufficient without a strong government
energy efficiency agency in place that is responsible for monitoring and enforcement.
The energy efficiency regulatory and legal framework has advanced significantly
(attributable to the project in part), but enforcement and actual implementation are still
lagging in several dimensions. Although the government enacted a new modality of
energy efficiency delivery in the public sector that was operationalized successfully
under the project, it has not followed up with measures to ensure a wider replication
and scale‐up. A market transformative policy function cannot be farmed out to a
revolving fund or a public energy service company lacking authority. The government
of Armenia has no effective institutional focal point for energy efficiency
implementation support, though this is critical for the wider diffusion of the successful
energy efficiency delivery mechanisms in the public sector. As a result, international
donors largely set and implement Armenia’s energy efficiency agenda.
5.4 The design of a pilot project needs to go beyond demonstration effects and lay
the foundation for sustainable operations over time. Although this energy efficiency
project effectively targeted public facilities with the highest potential for energy savings,
essentially harvesting low‐hanging fruit, the broader project experience revealed that
many public buildings in Armenia did not comply with the eligibility criteria. They are
in a poor state of maintenance and repair that would require substantial investments in
structural rehabilitation, not just energy efficiency retrofits. Energy cost savings alone
supported by the project will not be sufficient to cover the required investments in these
buildings unless supplemented by concessionary budgetary funds or grants. Instead of
being developed as a direct payment agent to finance the retrofits, the R2E2 Fund could
have focused on enabling commercial financing by acting as an intermediary guarantee
fund. As a guarantee fund, it could have fully de‐risked principal payments to
commercial banks on a rolling basis, thereby attracting commercial financing. However,
because the R2E2 Fund made direct payments, it largely decapitalized itself without
leveraging its initial funding to attract commercial financing.
1 The following construction norms (codes) were in place: RACN II‐7.02‐95 Construction
Thermophysics of the Building Envelopes: Design Standards; CNM II‐7.102‐98 Construction
Thermophysics of the Building Envelopes; RACN II‐7.01‐2011 Construction Climatology, CNM
Construction of Settlements, Buildings, and Structures under Climatic Conditions of the Republic
of Armenia; RACN IV‐12.02.01‐04 Heating, Ventilation, and Air Conditioning. There are many
15
energy efficiency standards relating to thermal insulation materials and elements in construction,
including some that were approximated or harmonized with the European and ISO standards.
2 Energy service companies (ESCOs) normally sign performance‐based energy contracts, design
the project, install the required equipment, track savings, and receive compensation in the form
of materialized energy savings and respective financial savings. In some cases, ESCOs might
structure their projects in a way that might transfer the burden of initial investment in required
equipment and components to clients, and in other cases, they might finance the whole project. In
any case, energy savings represent the main source of income for an ESCO (Taylor et al. 2008).
Operations in the sector included the following: Urban Heating Project, International
3
Development Association–Global Environment Facility (GEF) Renewable Energy Project, GEF
GeoFund 2: Geothermal Project, and Electricity Supply Reliability Project.
4 The Renewable Resources and Energy Efficiency Fund (R2E2 Fund) is governed by a board of
trustees chaired by the Minister of Energy Infrastructures and Natural Resources, and includes
Ministry of Finance, Ministry of Nature Protection, Urban Development Committee, Territorial
Administration and Development, Central Bank of Armenia, and two nongovernmental
organizations.
5 Under energy service agreements (ESAs), an energy efficiency revolving fund, ESCO, or other
energy efficiency service provider offers a full package of services to identify, finance, implement,
and monitor energy efficiency projects for clients. The client is usually required to pay all of their
baseline energy bill or a portion of it to cover the investment cost and associated fees until the
contract period ends. ESA payments can also be bundled with a client’s energy bills (World Bank
2014).
6 This significant overachievement of the targets was a result of the application of a procurement
based on net present value (NPV) under the project that maximized energy savings with
minimum investments. The targets were based on conventional estimates at appraisal using the
least‐cost criteria traditionally applied in Armenia. The NPV‐based procurement resulted in more
savings per dollar invested. As the ICR (page 17) specifies, the focus was on repayable
investments, and less funding was allocated to structural improvements.
7 R2E2 Fund project data.
The R2E2 Fund reached agreements with ACBA‐Credit Agricole Bank CJSC, ACBA Leasing
8
CJSC, and Global Credit Universal Credit Organization CJSC.
9 The progress analysis of the first National Energy Efficiency Action Plan (NEEAP‐1) and
development of the second plan (NEEAP‐2) were delayed because of difficulties related to
availability, completeness, consistency, and reliability of energy efficiency and energy‐related data
in the country. The lack of statistical data limiting the ability to properly assess the energy efficiency
potential and set targets is cited in the implementation progress reports on NEEAP‐1 and NEEAP‐
2. Despite the progress made in 2015–16 in enabling legislative amendments and practical
arrangements made between the Ministry of Energy Infrastructures and Natural Resources and
16
the National Statistical Service of Armenia, there were still hindrances for the development of
NEEAP‐2 caused by lack of data or its reliability.
The changes included development of energy efficiency and renewable energy standardization
10
documents, labeling of energy efficiency equipment, energy audit regulatory framework,
renewable energy net metering, tariffs, and so on.
The project received applications for project funding from 327 buildings, though 64 percent of
11
the applications received in the first two years were deemed ineligible, mostly because of low
baseline comfort or heating levels, thus requiring additional structural investments beyond
energy efficiency interventions (World Bank 2016).
A public ESCO is usually a government‐owned company that provides financing and technical
12
assistance for energy efficiency projects in the public sector, with repayments based on energy
cost savings (Hofer, Limaye, and Singh 2016).
Energy audits conducted for eight public facilities and energy efficiency investments in 11
13
schools under the World Bank–financed Urban Heating Project confirmed the financial viability
of energy efficiency investments and indicated the reduction of energy consumption by 30–40
percent.
For public clients, ESAs were not classified as debt, but rather long‐term service contracts,
14
thereby allowing financing of government entities that are typically not allowed to borrow and
municipalities that may have already reached their debt limits or otherwise have borrowing
restrictions.
15 The screening criteria included (i) confirmation of public ownership of facility; (ii) structural
soundness of the facility (absence of major structural damages that may jeopardize integral
stability of the building); (iii) absence of plans for closure, downsizing, or privatization of the
facility; and (iv) comfort level of more than 50 percent (the comfort level ratio is defined as the
actual energy consumption over the estimated energy consumption required to meet all heating
and lighting national norms).
Most energy efficiency loans in 2010–15 totaling $87.96 million were provided to the industry
16
(more than 35 percent), about 22 percent to the electricity sector and power generation, and 18
percent to small and medium‐size enterprises. Approximately 14 percent was invested in
municipal infrastructure projects.
IEG met with 12 construction firms, or about half of the energy efficiency and renewable energy
17
private companies that dealt with the R2E2 Fund.
The project introduced a performance‐based approach, performing a commissioning test that
18
was linked to the contractor payment. Payments were then made based on both milestones and
performance. An indicative payment schedule was as follows: 10 percent advance payment, 10
percent after approved final design, 50 percent approved after delivery of project per design, 20
percent after a commissioning test to verify the actual energy savings (against the promised
savings and NPV in the contractor bid), and 10 percent after a 12‐month defects and liability
period (to allow for performance monitoring over one full heating season).
17
These donors include the World Bank Group, the European Bank for Reconstruction and
19
Development, United Nations Development Programme, European Union, European Investment
Bank, AFD, U.S. Agency for International Development, KfW, Green Climate Fun, GEF, Eastern
Europe Energy Efficiency and Environment Partnership.
20 This is also evidenced in NEEAP‐2 stating that significant additional funding necessary for the
implementation of the energy efficiency investments will be required from donors and
international financial institutions.
18
Appendix A. Project Ratings
ICR ICR Review PPAR
Outcome Highly satisfactory Highly satisfactory Moderately satisfactory
Risk to development Negligible Modest Substantial
outcome
Bank performance Satisfactory Satisfactory Moderately satisfactory
Borrower performance Satisfactory Satisfactory Moderately satisfactory
Note: The Implementation Completion and Results Report (ICR) is a self-evaluation by the responsible Global Practice. The
ICR Review is an intermediate Independent Evaluation Group product that seeks to independently validate the findings of
the ICR. PPAR = Project Performance Assessment Report.
Relevance
Based on the overview provided in section 1, the relevance of objectives is rated high,
and the design is rated modest.
Relevance of Objectives
The project development objectives were highly relevant to the challenges in Armenia’s
energy sector and the government priorities (discussed in sections 2.1–2.4). The World
Bank Group’s 2014–17 Country Partnership Strategy specifically mentioned the
continuation of support to energy efficiency measures and reduction in the energy
consumption of public buildings (World Bank 2013). The Bank Group’s Systematic
Country Diagnostic discusses the priority to scale up the development of energy
efficiency measures and states that the government could consider scaling up existing
successful models for financing public energy efficiency investments to include other
social, public, and residential buildings (World Bank 2017).
The global environment objective was consistent with the Global Environment Facility’s
(GEF) climate change focal area, particularly GEF Operational Program 5 (Energy
Efficiency), and strategic programs under GEF‐4: SP1 “Promoting Energy Efficiency
Technologies and Practices in Appliances and Buildings.”
Relevance of Design
The links between outputs, outcomes, and the project development objectives were
clear.
The project aimed to reduce energy consumption and decrease greenhouse gas (GHG)
emissions through energy efficiency retrofits in public and social buildings. Technical
assistance and capacity building were geared to remove barriers to the implementation
of energy efficiency investments in the public sector, thus further decreasing GHG
19
emissions. However, the project design failed to factor in cofinancing from financial
institutions or commercial entities. In retrospect, this reduced the chances for the project
energy efficiency financing model to be replicated at scale. According to energy experts,
a minimum level of own financing by project sponsors (5–10 percent) is a good practice
in energy efficiency funding.
The relevance of design is rated modest.
Efficacy
The project development objective was “To reduce energy consumption of social and
other public facilities” (GEF grant agreement dated April 20, 2012, page 7). The global
environmental objective was “To decrease greenhouse gas emissions through the
removal of barriers to the implementation of energy efficiency investments in the public
sector” (World Bank 2012, 4).
Objective 1: Reduction in energy consumption of social and other public facilities
Outputs
Cumulative investments in public facilities: target $8.7 million, achieved $10.2
million
Number of public sector buildings commissioned: target 121,1 achieved 124
Outcome
The project achieved 250 percent of the original target for energy savings in
retrofitted social and other public facilities. The original target was 216 million
kilowatt hours saved, and the actual achievement was 540 million kilowatt hours
saved. Original estimates were based on previous projects that used conventional
procurement. Because the project used a net present value (NPV)–based
procurement that maximized energy savings with a minimum level of
investments, the results were significantly overachieved.
The project achieved a reduction of 145,739 tons CO2 equivalent, almost three
times its target of 50,549 tons.
The achievement of objective 1 is rated substantial. Although the outcomes were
significantly overachieved by project closure, the downgrade from high (as assessed in
the ICR and IEG ICR Review) reflects the lack of sustainability of the project outcomes.
The longer‐term outcome failed to illustrate the ongoing impact on reduction of energy
consumption and related CO2 emissions in the public sector. There was no continuity in
20
the project’s outcomes, and no development impact beyond the project interventions
(see section 2, Results).
Objective 2: Removal of barriers to the implementation of energy efficiency
investments in the public sector
Outputs
The analysis was conducted for the implementation progress of NEEAP 1 that
became the basis for the Second National Program on Energy Savings and
Renewable Energy (NEEAP 2), which the government adopted.
The government adopted an energy services agreement (ESA) in 2015 that the
Renewable Resources and Energy Efficiency Fund (R2E2 Fund) developed to
facilitate the implementation of energy efficiency investments in the public
sector.
The project introduced an NPV‐based procurement that was not mainstreamed
in the national procurement.
The R2E2 Fund organized a range of energy efficiency public awareness events,
including workshops, conferences, seminars, and training for potential bidders
and construction firms on procurement and the measurement and verification
process. The most widely known is an annual Energy Week conference led by
the R2E2 Fund that brings together a wide range of stakeholders in the energy
sector and energy efficiency.
Outcome
The R2E2 Fund contributed substantially to removing policy, informational, and
knowledge barriers to energy efficiency implementation. The government adopted a
sample agreement for energy efficiency investments, but it has not yet been used in
practice. The legal adoption of ESAs opened the potential to be used in contracts that
pay for investment through future savings; however, this measure on its own has not
been able to institute an environment in which investment is under way in Armenia’s
public sector. The project failed to address financing barriers that constrain the
implementation of energy efficiency investments at scale in the public sector. Piloting
new models for energy efficiency service provision needed to create local conditions and
develop the capacity of financial entities through cofinancing and risk mitigation that
could ensure their sustainability. The project did not sufficiently address the barriers to
the implementation of energy efficiency investments in Armenia’s public sector that
could facilitate the continued flow of energy savings (see section 2, Results).
21
The achievement of objective 2 is rated modest.
Efficiency
Economic and Financial Analyses
At appraisal, a cost‐benefit analysis was conducted for each type of public facility
(hospital, school, kindergarten, municipality building, and street lighting) that might be
financed under the project. Ex ante estimated economic internal rates of return (EIRRs)
ranged from 31 percent for kindergartens to 77 percent for street lighting. The average
payback period ranged from 2.5 years for municipality buildings to 3.8 years for
hospitals. The main quantifiable economic benefit from energy efficiency investments in
public facilities was the economic value of saved energy. Energy savings were valued at
the estimated long‐run marginal cost of electricity or gas supply, depending on the
facility and the heating option used before implementation of energy efficiency
measures. Other benefits, such as emission reduction, were not quantified.
The ex ante financial internal rates of return (FIRRs) ranged from 14 percent for hospitals
and schools to 32 percent of municipality buildings. Energy bill reduction was the main
estimated benefit, valued at then‐current effective electricity and gas tariffs, depending
on the type of facility and the heating option the facility used before energy efficiency
investments. The estimated financial payback periods ranged from 4.2 to 7.7 years.
At closure, ex post EIRRs were estimated for 47 completed and commissioned
subprojects. The payback period for all of these subprojects ranged from 2.6 to 8.8 years,
and energy savings ranged from 27 percent to 80 percent, with an average of 50.9
percent. Because of the requirement for full repayment and NPV‐based procurement,
the investment cost required to achieve these savings was low at only about $24.40 per
square meter (in the buildings sector, about one‐half the investment required for World
Bank projects in other countries) and at an impressively low cost of only $0.19 per
kilowatt (for all subprojects). However, because of the relatively low grid emission factor
in Armenia (0.234 kilograms CO2 per kilowatt‐hour) and baseline heating fuel (natural
gas), the cost per ton of CO2 emissions reduction was higher than in other projects
($72.20 per ton CO2), though the cost per ton of CO2 for the GEF grant was lower at only
$29.80 per ton CO2.
The ex post FIRRs ranged from 10 percent for schools to 38 percent for street lighting.
Tariff increases during the project period and local currency devaluation by 26 percent
affected FIRRs. The realized financial payback periods ranged from three to seven years.
22
Administrative and Operational Efficiency
In the early stage of project implementation, the uptake of subprojects was very slow
because of a low demand for energy efficiency investments. This required a one‐year
extension of the project closing date to complete the planned activities.
Efficiency of the project is rated substantial.
Outcome
Relevance of objectives is rated high, and relevance of design is modest. The first
objective of reducing energy consumption of social and other public facilities was
substantially achieved. The outcome targets were significantly overachieved by project
closure—the stated energy and GHG savings appear significant only in relation to the
very modest original targets. However, the longer‐term outcome failed to illustrate the
ongoing impact on reduction of energy consumption and related CO2 emissions in the
public sector. The second objective of reducing barriers to the implementation of energy
efficiency investments in the public sector was modestly achieved. A financing barrier to
the implementation of energy efficiency investments in the public sector remains. There
was no continuity in the project’s outcomes and no development impact beyond the
project interventions. Efficiency is rated substantial.
Overall, the outcome rating is moderately satisfactory.
Risk to Development Outcome
The energy savings from the investments in building retrofits are likely be maintained.
The fund receives 100 percent of financial returns from these investments. At the
institutional level, at project closure the R2E2 Fund was expected to be sustainable at
least over the following three to five years, even without additional capital because the
R2E2 Fund could support investments of $1.3 million to 1.5 million per year (according
to the ICR). In fact, the fund is in a financial survival mode—additional donor support
has not been secured, and there is no government support for energy efficiency
financing. Onlending of the remaining capital of $1 million was a survival response and
could not be continued under the fund’s legal status.
In addition, the government is restructuring the sector by liquidating project
implementation units of various international organizations and transferring their
functions to ministries. Therefore, the fund’s future is highly uncertain because the
government (which is completing the implementation of the World Bank projects)
considers it a project implementation unit (based on the IEG interviews).
Risk to development outcome is rated substantial.
23
Bank Performance
Quality at Entry
The project was based on extensive analytical work and the World Bank’s long‐standing
engagement in Armenia’s energy sector. The implementing agency, the R2E2 Fund, had
sufficient experience with implementation of World Bank projects.
This project was originally designed as an energy efficiency component of the 2009
Electricity Supply Reliability and Energy Efficiency Project. It was dropped from this
project because of insufficient funds and later resurrected as a stand‐alone GEF project
with the concept of using resources in the R2E2 Fund that had accumulated as
International Development Association loan repayments of earlier World Bank projects.
Overall, a significant shortcoming in the project design was not factoring in cofinancing
from financial institutions, which lowered the project’s chances for replication and scale
up of energy efficiency investments in Armenia’s public sector. In addition, the
monitoring and evaluation (M&E) design had moderate weaknesses.
Quality at entry is rated moderately satisfactory.
Supervision
The World Bank team worked closely with the government and the R2E2 Fund in
support of the energy efficiency project implementation and in training the staff in NPV
evaluations and World Bank procurement methodology. It conducted periodic
implementation support missions and provided operational advice and technical
support when needed. The project complied with fiduciary and safeguard policies (ICR,
page 23), and no related issues were raised during the IEG evaluation mission after
project closure.
In retrospect, because the pilot for energy efficiency financing has not been replicated,
the World Bank might have provided direction and support to the R2E2 Fund to ensure
a broader energy efficiency impact by, for example, engaging and partnering with
financial institutions and commercial entities to ensure sustainability of the energy
efficiency financing agenda (see section 4).
Quality of supervision is rated moderately satisfactory.
The overall Bank performance rating is moderately satisfactory.
24
Borrower Performance
Government
The government of Armenia initially demonstrated strong commitment to the project
during implementation by (i) pledging reflows from the investments of two closed
World Bank projects as cofinancing to the GEF grant; (ii) supporting line ministries and
other public facilities that participated in the project; and (iii) strengthening the energy
efficiency regulatory and legal framework.
On balance, although the government supports the energy efficiency enabling
environment, enforcement is lagging, and there is a lack of coordination of significant
donor fund flows into energy efficiency without much development impact in energy
efficiency in the public sector. The government has not provided support to the R2E2
Fund to ensure scale up and continuation of the energy efficiency financing model in the
public sector, which is the only existing program currently in Armenia for these types of
investments. Thus, there is a lack of government commitment in investing in energy
efficiency in the public sector and supporting the mechanisms to facilitate such
investments. In addition, the fund’s director under the project was replaced, and there
was no further continuity in the appointment of directors, who changed three times
within two years after project closure.
The government performance rating is moderately satisfactory.
Implementing Agency
The R2E2 Fund has motivated and professionally competent personnel under strong
and effective leadership who implemented the project and introduced several
innovations to facilitate the energy efficiency implementation in the public sector. These
innovations include (i) the introduction and implementation of ESAs, (ii) the use of
NPV‐based procurement in an industry accustomed to least‐cost procurement, (iii)
introduction of repayment obligations among clients accustomed to grant‐based
financing and ensuring full and timely repayment, and (iv) establishment of
performance‐based payments for contractors who were used to output‐based payments.
It also provided support to contractors to strengthen their capacity to implement their
subprojects, maximize the project’s NPV, and act in accordance with the World Bank’s
procurement guidelines. It provided support to the government for upgrading its
institutional and regulatory framework. The R2E2 Fund also implemented aggressive
marketing campaigns and an accelerated implementation timeframe, which enabled it to
exceed the targeted indicators.
25
The R2E2 Fund has resorted to on‐lending for energy efficiency projects after project
closure. It is actively seeking to secure additional financing to continue its operation
model and replenish the revolving fund.
Implementing agency performance is rated satisfactory.
Borrower performance rating is moderately satisfactory.
Quality of Monitoring and Evaluation
Design
Two outcome indicators were identified to measure the achievement of the project
development objectives: (i) energy savings in selected buildings, and (ii) related CO2
emission reduction. A third indicator, the introduction of regulations, legislative
amendments, and guidelines to further promote energy efficiency, was considered
intermediate and should have been upgraded to a primary outcome indicator because it
would affect all expected energy efficiency investments after the project finishes.
The M&E design was highly simplistic and did not reflect the broad nature of activities
planned under the technical assistance component. The types of barriers were not
specified, and no clear outputs or targets were identified for the institutional outcomes.
In addition, the targets of the outcome indicators were based on conventional
procurement while the project intended to introduce a new NPV‐based procurement.
Implementation
During project preparation and throughout implementation, the R2E2 Fund established
its internal M&E capacity and systems for tracking subproject screening results, the
pipeline of accepted subprojects, and the commitment and invested amounts for
accepted projects.
In addition, the fund developed a measurement and verification system for proper
baseline determination of energy consumption, investment repayments (under ESA),
NPV assessments, and payments to contractors. The R2E2 Fund, beneficiary, and
contractor all monitored subproject performance indicators (indoor temperature,
lighting intensity, energy bills, and so on), which helped minimize disputes among the
stakeholders. The R2E2 Fund also conducted a beneficiary survey at the end of the
project to document social impacts and client satisfaction with the investments.2
Utilization
The R2E2 Fund used the M&E tracking system for reporting project progress and
outcomes to the World Bank and the government, including the identification of
26
subproject delays, repayment progress, and defaults. At the completion of each
subproject, the M&E tracking system was used to evaluate energy savings and CO2
reductions.
The M&E quality is rated modest because of moderate weaknesses in the project M&E
design.
1 The Implementation Completion and Results Report and ICR Review indicate the target as 85,
considering that several subprojects included multiple buildings.
2 The focus of the Independent Evaluation Group (IEG) assessment was on transformational
impact of the pilot project on the energy efficiency enabling environment and market. The
assessment methodology did not include follow‐up surveys with beneficiaries. During the IEG
mission, there were two meetings held with the project’s beneficiaries for the overall insights. The
findings are not reported because they are not representative.
27
Appendix B. Project Data and Other Issues
Fiduciary and Safeguard Aspects
Financial Management
The Renewable Resources and Energy Efficiency Fund’s semiannual interim financial
reports were generally submitted on time and approved by the World Bank. Auditors
issued unmodified (clean) opinions on the project’s annual financial reports. The fund
also complied with public disclosure requirement of the audited financial statements.
Procurement
There were no major procurement issues during implementation. Procurement was
rated satisfactory in all of the Implementation Status Reports (ISRs) throughout the
project. Reviews were conducted during the life of the project.
Environmental and Social Safeguards
The project was classified as an environmental category B at appraisal and triggered
OP/BP 4.01 Environmental Assessment. An environmental management plan was
developed during project preparation, as required, and site‐specific plans were shared
with the World Bank for approval and publicly disclosed for each individual
investment. Stakeholders were given sufficient time and opportunity to share their
comments on these plans.
The project retained a satisfactory rating on safeguards performance throughout its life.
All agreements and permits required for specific works were obtained from the national
authorities. Mitigation measures were confined to the proper handling of construction
waste and adherence to workplace safety rules. Field supervision of works was
performed regularly, and a good record on environmental monitoring outcomes was
kept. No damage to the natural environment has been recorded. Streetlight bulbs
containing mercury hat were replaced with energy‐efficient and nontoxic bulbs were
sent to adequate municipal storage until relevant facilities for safe destruction or
disposal of such waste becomes available in the country. Based on the information
obtained during the Project Performance Assessment Report mission, they remain in the
municipal storage.
28
Basic Project Information
Energy Efficiency Project (Global Environment Facility Grant TF12163)
Table B.1. Key Project Data
Appraisal Estimate Actual Estimate Actual as Percent of
Financing ($, millions) ($, millions) Appraisal Estimate
Total project costs 10.66 11.32 106
Loan amount 1.82 1.81 99.45
Table B.2. Cumulative Estimated and Actual Disbursements
Disbursements FY12 FY13 FY14 FY15 FY16
Appraisal estimate ($, millions) 0.30 0.70 1.20 1.70 1.82
Actual ($, millions) 0 0.15 0.75 1.28 1.81
Actual as percent of appraisal 0 21.42 62.5 75.29 99.45
Table B.3. Project Dates
Event Original Actual
Concept review n.a 02/18/2010
Board approval n.a. 03/27/2012
Signing n.a. 04/20/2012
Effectiveness n.a. 08/10/2012
Closing date 06/30/2015 06/30/2016
29
Appendix C. R2E2 Fund Energy Efficiency
Investments, Financial Data, and Estimated Energy
Savings
Indicators 03/2012–06/2016 07/2016– 08/2017–
07/2017 10/2018
Process Cumulative energy 10,197,863
efficiency investments in
social and other public
buildings ($)
Number of public 5,800
buildings in Armenia
Public facilities retrofitted 124
(number)
Applications for 326
retrofitting of public
buildings (number)
Number of construction 20
companies contracted
under the project
Results Cumulative energy 540,240,000
savings over 20‐year
useful life of energy
efficiency investments
(kilowatt‐hours)
Cumulative CO2 emission 145,739
reduction over 20‐year
useful life of energy
efficiency investments
(tons of CO2 equivalent)
Revolving Cash flow from energy 550,147 855,028 1,473,770
fund efficiency investments
Repayments 98,495 113,158 126,280
Complaints (number)a 0 0 0
30
Energy Total energy efficiency– 1,098,324
efficiency related onlending through
commercial commercial banks ($)
financiers and
ACBA‐Credit Agricole 222,880
financial
Bank CJSC ($)
institutions
engaged after ACBA Leasing CJSC ($) 347,033
project closure
Global Credit UCO CJSC 528,411
($)
Cash flow from onlending 31,713
to banks
Types of ACBA‐Credit Agricole Energy
projects Bank CJSC efficiency for
(description) communities
ACBA Leasing CJSC Energy
efficiency for
non‐gasified
communities
Global Credit UCO CJSC Energy
efficiency for
non‐gasified
communities
Results ACBA‐Credit Agricole 6
reported by Bank CJSC communities
banks (energy
Annual energy savings 549,170
savings,
(kilowatt‐hours)
number, and
types of ACBA Leasing CJSC 394
beneficiaries) households
and 1 LLC
Global Credit UCO CJSC 509
households
Annual energy savings 2,612,200
(kilowatt
‐hours)
Note:
a. All issues arising during the bidding or construction process are resolved immediately.
31
Appendix D. Overview of Main Donors and
International Financial Institution Funds Available
for Energy Efficiency in Armenia
Source: International Energy Charter 2017.
32
Appendix E. List of Existing Energy Efficiency
Legal, Regulatory, and Normative-Technical
Documents in Armenia
Table E.1. List of the legal, regulatory, and normative technical documents in the Republic
of Armenia related to energy saving and energy efficiency of buildings (2018)
I. Legal Acts
AL‐122‐N, 09.11.2014 The Law of the RA on Energy Saving and Renewable Energy
1.
Resolution of the Government of the Republic of Armenia on
2. 25.12.2014, 1504‐N Implementation of Energy Saving and Energy Efficiency Improvement
Measures in Facilities Constructed (Reconstructed, Renovated) under the
State Funding
A resolution on “Establishing technical regulations on energy saving and
energy efficiency in new residential apartment buildings, as well as in
3.
facilities under construction (reconstructed) at the expense of state
funds”, to come into force in October 2019.
II. Construction Norms (mandatory building codes)
1. RACN II‐7.01‐2011 Construction Climatology
2. RACN 24‐01‐2016 Thermal Protection of the Buildings
3. RACN 22‐03‐2017 Artificial and Natural Lighting
III. National Standards (voluntary standards, unless referred to in government‐adopted technical regulations)
1. AST 240‐2005 Energy saving; terms and definitions
Energy conservation: Regulatory and procedural guidelines: Basic
2. concepts
AST 246‐2006
AST 247‐2006 Energy conservation, energy efficiency, composition of indicators, basic
3.
concepts
Energy conservation. Energy consuming general industrial equipment.
4. Types, groups, and indicators of energy efficiency; identification
AST 248‐2006
33
Energy conservation; assurance for energy consuming products energy
5. efficiency indexes to their normative values. methods of verification.
AST 249‐2006
general requirements
Energy conservation; informing the consumers about the energy
6. efficiency of appliances and residential utility equipment; general
AST 250‐2006
requirements
AST 254‐2006 Energy conservation; energy certificate of the consumer of fuel and
7.
energy resources; basic rules; standard forms
AST 255‐2006 Energy conservation; methods of determining the economic effectiveness
8.
of the energy efficiency measures
AST 256‐2006 Energy conservation; secondary energy resources; terms and definitions
9.
10. AST ISO 16818‐2008 Building environment design; energy efficiency; terminology
11. AST 1434‐1‐2010 Heat meters, part 1: general requirements
Heat meters, part 6: installation, commissioning, operational monitoring
12. and maintenance
AST 1434‐6‐2010
GOST EN 822‐2011 Thermal insulating products for building applications: determination of
13.
length and width
GOST EN 823‐2011 Thermal insulating products for building applications: Determination of
thickness
14.
GOST EN 824‐2011 Thermal insulating products for building applications: determination of
15.
squareness
GOST EN 825‐2011 Thermal insulating products for building applications: determination of
16.
flatness
GOST EN 826‐2011 Thermal insulating products for building applications: determination of
17.
compression behavior
GOST EN 1602‐2011 Thermal insulating products for building applications: determination of
18.
the apparent density
GOST EN 1604‐2011 Thermal insulating products for building applications: determination of
19. dimensional stability under specified temperature and humidity
conditions
34
GOST EN 1605‐2011 Thermal insulating products for building applications: determination of
20. deformation under specified compressive load and temperature
conditions
GOST EN 1606‐2011 Thermal insulating products in building applications: determination of
21.
compressive creep
GOST EN 1607‐2011 Thermal insulating products for building applications: determination of
22.
tensile strength perpendicular to faces
GOST EN 1608‐2011 Thermal insulating products for building applications: determination of
23.
tensile strength parallel to faces
GOST EN 1609‐2011 Thermal insulating products for building applications: determination of
24.
short term water absorption by partial immersion
GOST EN 12085‐2011 Thermal insulating products for applications in building: determination
25.
of linear dimensions of test specimens
GOST EN 12086‐2011 Thermal insulating products in building applications: determination of
26. water vapor transmission properties
GOST EN 12087‐2011 Thermal insulating products in building applications: determination of
27. long term water absorption by immersion
GOST EN 12088‐2011 Thermal insulating products in building applications: determination of
28.
long‐term moisture absorption by diffusion
GOST EN 12089‐2011 Thermal insulating products in building applications: determination of
29. bending behavior
GOST EN 12090‐2011 Thermal insulating products in building applications: determination of
30.
shear behavior
GOST EN 12091‐2011 Thermal insulating products in building applications: determination of
31.
freeze‐thaw resistance
GOST EN 12430‐2011 Thermal insulating products for building applications: determination of
32.
behavior under point load
GOST EN 12431‐2011 Thermal insulating products used in building for floating floors:
33.
determination of thickness
35
GOST 31924‐2011 Thick building materials and products of high and medium thermal
34. resistance: determination of thermal resistance by means of guarded hot
plate and heat flow meter
AST EN 15217‐2012 Energy performance of buildings: methods for expressing energy
35.
performance and for energy certification of buildings
AST EN 15316‐1‐2012 Heating systems in buildings: method for calculation of system energy
36. requirements and system efficiencies, part 1: general provisions
AST EN 15603‐2012 Energy performance of buildings: overall energy use and definition of
37.
energy ratings
AST ISO 23045‐2012 Building environment design: guidelines to assess energy efficiency of
38.
new buildings
GOST 32025‐2012 Thermal insulation: determination of steady‐state thermal transmission
39. properties of thermal insulation for circular pipes
(EN ISO 8497:1996)
AST 362‐2013 Energy conservation: energy passport of the building. Basic rules.
40.
Standard form
AST EN 15242‐2014 Ventilation for buildings ‐ Calculation methods for the determination of
41.
air flow rates in buildings including infiltration
AST ISO 9251‐2014 Thermal insulation ‐ Heat transfer conditions and properties of materials
42.
– Vocabulary
AST ISO 10211‐2014 Thermal bridges in building construction ‐ Heat flows and surface
43.
temperatures ‐ Detailed calculations
AST ISO 13789‐2014 Thermal performance of buildings ‐ Transmission and ventilation heat
44.
transfer coefficients ‐ Calculation method
AST ISO 13790‐2014 Energy performance of buildings ‐ Calculation of energy use for space
45.
heating and cooling
AST ISO 14683‐2014 Thermal bridges in building construction ‐ Linear thermal transmittance ‐
46.
Simplified methods and default values
AST EN 15316‐3‐3‐2015 Heating systems in buildings ‐ Method for calculation of system energy
47. requirements and system efficiencies ‐ part 3‐3: Domestic hot water
systems, generation
AST 371‐2016 Methodology for performing energy audit in residential and public
48.
buildings
36
IV. Advisory Handbooks
06.11.2013 N343 Technical solutions for thermal insulation of envelopes of residential,
1. public and industrial buildings in construction and reconstruction in the
Order of the Minister of Urban
RA
Development of the RA
23.12.2013 N394
2. Order of the Minister of Urban Catalogue of Replicable Energy Efficient Individual Residential Houses in
Development of the RA Communities of the RA
37
Appendix F. Project Performance Assessment
Report Overview
About This Report
The Independent Evaluation Group (IEG) assesses the programs and activities of the World Bank for two purposes: first, to ensure
the integrity of the World Bank’s self-evaluation process and to verify that the World Bank’s work is producing the expected
results, and second, to help develop improved directions, policies, and procedures through the dissemination of lessons drawn
from experience. As part of this work, IEG annually assesses 20–25 percent of the World Bank’s lending operations through
fieldwork. In selecting operations for assessment, preference is given to those that are innovative, large, or complex; those that
are relevant to upcoming studies or country evaluations; those for which Executive Directors or World Bank management have
requested assessments; and those that are likely to generate important lessons.
To prepare a Project Performance Assessment Report (PPAR), IEG staff examine project files and other documents, visit the
borrowing country to discuss the operation with the government, and other in-country stakeholders, interview World Bank staff
and other donor agency staff both at headquarters and in local offices as appropriate, and apply other evaluative methods as
needed.
Each PPAR is subject to technical peer review, internal IEG panel review, and management approval. Once cleared internally,
the PPAR is commented on by the responsible World Bank Country Management Unit. The PPAR is also sent to the borrower for
review. IEG incorporates both World Bank and borrower comments as appropriate, and the borrowers’ comments are attached to
the document that is sent to the World Bank’s Board of Executive Directors. After an assessment report has been sent to the
Board, it is disclosed to the public.
About the IEG Rating System for Public Sector Evaluations
IEG’s use of multiple evaluation methods offers both rigor and a necessary level of flexibility to adapt to lending instrument,
project design, or sectoral approach. IEG evaluators all apply the same basic method to arrive at their project ratings. Following is
the definition and rating scale used for each evaluation criterion (additional information is available on the IEG website:
http://ieg.worldbankgroup.org).
Outcome: The extent to which the operation’s major relevant objectives were achieved, or are expected to be achieved,
efficiently. The rating has three dimensions: relevance, efficacy, and efficiency. Relevance includes relevance of objectives and
relevance of design. Relevance of objectives is the extent to which the project’s objectives are consistent with the country’s
current development priorities and with current World Bank country and sectoral assistance strategies and corporate goals
(expressed in Poverty Reduction Strategy Papers, country assistance strategies, sector strategy papers, and operational policies).
Relevance of design is the extent to which the project’s design is consistent with the stated objectives. Efficacy is the extent to
which the project’s objectives were achieved, or are expected to be achieved, taking into account their relative importance.
Efficiency is the extent to which the project achieved, or is expected to achieve, a return higher than the opportunity cost of
capital and benefits at least cost compared with alternatives. The efficiency dimension is not applied to development policy
operations, which provide general budget support. Possible ratings for outcome: highly satisfactory, satisfactory, moderately
satisfactory, moderately unsatisfactory, unsatisfactory, highly unsatisfactory.
Risk to development outcome: The risk, at the time of evaluation, that development outcomes (or expected outcomes)
will not be maintained (or realized). Possible ratings for risk to development outcome: high, significant, moderate, negligible to low,
and not evaluable.
Bank performance: The extent to which services provided by the World Bank ensured quality at entry of the operation and
supported effective implementation through appropriate supervision (including ensuring adequate transition arrangements for
regular operation of supported activities after loan or credit closing, toward the achievement of development outcomes). The
rating has two dimensions: quality at entry and quality of supervision. Possible ratings for Bank performance: highly satisfactory,
satisfactory, moderately satisfactory, moderately unsatisfactory, unsatisfactory, and highly unsatisfactory.
Borrower performance: The extent to which the borrower (including the government and implementing agency or
agencies) ensured quality of preparation and implementation, and complied with covenants and agreements, toward the
achievement of development outcomes. The rating has two dimensions: government performance and implementing agency(ies)
performance. Possible ratings for borrower performance: highly satisfactory, satisfactory, moderately satisfactory, moderately
unsatisfactory, unsatisfactory, and highly unsatisfactory.
38
Appendix G. Methods and Evidence
This evaluation is based on (i) interviews with project stakeholders in Yerevan, Armenia in
November 2018, (ii) interviews with World Bank staff, (iii) a review of project documents,
and (iv) a review of reports published by the World Bank or other development partners.
The Project Performance Assessment Report (PPAR) evaluation questions focus on the
project’s catalytic effect in addressing barriers to energy efficiency (energy efficiency)
development in Armenia’s public sector. This includes looking at demonstration effects and
engagement of financial institutions and private companies, market transformation, and the
effects on policy and legal and regulatory frameworks. These goals are implicit in the
project, based on the project development objective used in the project appraisal document
and the Implementation Completion and Results Report. The PPAR aims to determine the
effectiveness and causes of the project in affecting the energy efficiency sector development.
Stakeholder interviews focused on generating lessons from the pilot project and on specific
questions regarding the following: (i) the project implementation model, (ii) the reason for
not continuing the project model, (iii) the project’s effects on energy efficiency market
development, (iv) the effects of capacity‐building activities, (v) effects on energy efficiency
awareness, and (vi) effects on energy efficiency policy development.
Project stakeholders included (i) national and local government representatives, (ii)
Renewable Resources and Energy Efficiency Fund, (iii) nongovernmental organizations, (iv)
financial institutions, (v) construction companies, (vi) academia, (vii) project beneficiaries
(directors of public facilities), (viii) development partners in Armenia, and (ix) World Bank
staff (see appendix H).
39
Appendix H. List of Persons Interviewed
Armenia Renewable Resources and Energy Efficiency Fund (R2E2)
Mr. Karen Asatryan, Director
Ms. Zaruhi Gharagyozyan, head of procurement group
Ms. Siranush Gorgyan, Financial Manager
Mr. Hrant Ter‐Gabrielyan, Head of Engineering, technical group
Former Directors of R2E2 Fund
Ms. Tamara Babayan, Sustainable Energy Practice Lead, Energy Saving Foundation
Mr. Armen Melkikyan, Country Director for Armenia, Eurasian Development Bank
Ministry of Energy Infrastructures and Natural Resources
Mr. Vahagn Atayan, Head of Energy Efficiency and Technical Normative Division
Urban Development Committee
Ms. Tanya Arzumanyan, Head of the Housing Stock Management and Communal
Infrastructures Department
Ms. Yevgenya Atayan, Advisor to the Chairman
Yerevan Municipality
Mr. Tigran Sargsyan, Head of Development and Investment Programs Department
Energy Saving Foundation NGO
Ms. Astghine Pasoyan, Executive Director
Scientific Research Institute of Energy
Mr. Ruben Yeghoyan, General Director
Financial Institutions
ACBA‐Credit Agricole Bank
Mr. Hakob Andreasyan, CEO
Mr. Norik Nazaryan, Deputy CEO, Sales Promotion and Regional Management
Mr. Styopa Zakinyan, Deputy CEO, Chief Financial Officer
Mr. Mikhail Amirkhanyan, SME and Corporate Business Department
ACBA Leasing CJSC
40
Mr. Arsen Bazikyan, General Manager
Mr. Vahagn Shmavonyan, Consumer Attraction and Marketing Division
Global Credit UCO CJSC
Mr. Aram Ghukasyan, Deputy Executive Director
AMERIABANK CJSC
Mr. Karen Hakobjanyan, Head, SME Lending
Private Sector
Renewable Energy Producers Association
Mr. Roman Melikyan, Executive Director
Redinet CJSC
Mr. Andranik Karamyan, CEO
Gevorgyan & Nersisyan Ltd.
Mr. E. Gevorgyan
Zet‐Profil Ltd.
Mr. Robert Zakaryan, Director
Gevorg Zaqaryan
Arpi Solar
Mr. Ruben Gevorgyan, CEO
Freenergy
Mr. Hayk Chobanyan, CEO
AR & AR Design Construction
Mr. Ara Petrosyan, President
Arjermek LLC
Mr. Hrant Arabyan, Director
SolarOn
41
Mr. Arthur Alaverdyan, Founder
Contour Global Hydro Cascade
Mr. Ara Hovsepyan, General Manager
Amber Capital
Mr. Karen Arabyan, COO
Solar Power Station Investor
Mr. Karen Kazaryan
Academia
American University of Armenia
Mr. Alen Amirkhanian, Director, Acopian Center for the Environment
Mr. Artak Hambarian, Associate Director of the Engineering Research Center
Institute of Molecular Biology of the National Academy of Sciences (project beneficiary)
Mr. Arsen Arakelyan, Director, Head of Bioinformatics Group
Kanaker‐Zeytun Medical Center (project beneficiary)
Mr. Hayk Petrosyan, Director
Development Partners
United Nations Development Programme/Global Environment Facility
Ms. Diana Harutyunyan, Programme Coordinator
Mr. Vahram Jalalyan, Project Task Leader
United Nations Industrial Development Program
Ms. Anahit Simonyan, Country Representative
Green Economy Financing Facility
Mr. Mikhail Gevorgyan, PFI Relationship Manager
European Bank for Reconstruction and Development
Ms. Anna Khachatryan, Associate Director
KfW
42
Ms. Bella Andreasyan, Senior Project Coordinator
Habitat for Humanity Armenia
Ms. Luisa Vardanyan, Executive Director
World Bank Group
Ms. Sylvie Bossoutrot, Country Manager for Armenia, Europe and Central Asia Region
Mr. Emil Zalinyan, Energy Specialist
Ms. Ani Balabanyan (Lead Energy Specialist, project Task Team Leader)
Mr. Jasneet Singh (Lead Energy Specialist, project co‐Task Team Leader)
.
43
44
Appendix I. Bibliography
Energy Charter Secretariat. 2015. In‐Depth Review of the Energy Efficiency Policy of Armenia. Brussels:
Energy Charter Secretariat.
Hofer, Kathrin, Dilip Limaye, and Jas Singh. 2016. “Fostering the Development of ESCO Markets for
Energy Efficiency.” Live Wire: A Knowledge Note Series for the Energy and Extractives Global
Practice, 2016/54, World Bank, Washington, DC.
Singh, Jas, Dilip R. Limaye, Brian Henderson, and Xiaoyu Shi. 2010. Public Procurement of Energy
Efficiency Services: Lessons from International Experience. Washington, DC: World Bank.
Taylor, Robert P., Chandrasekar Govindarajalu, Jeremy Levin, Anke S. Meyer, and William A. Ward.
2008. Financing Energy Efficiency: Lessons from Brazil, China, India, and Beyond. Washington,
DC: World Bank.
World Bank. 2008. “The Other Renewable Resource: The Potential for Improving Energy Efficiency in
Armenia.”
———. 2012. “Armenia—Energy Efficiency Project.” Project Appraisal Document 67035‐AM, World
Bank, Washington, DC.
———. 2013. Armenia—Country Partnership Strategy, FY14–17. Washington, DC: World Bank.
———. 2014. “Western Balkans: Scaling Up Energy Efficiency in Buildings.” World Bank,
Washington, DC.
———. 2016. “Armenia—Energy Efficiency Project.” Implementation Completion and Results
Report, World Bank, Washington, DC.
———. 2017. Armenia—Future Armenia: Connect, Compete, Prosper. Systematic Country
Diagnostic. Washington, DC: World Bank.
44
Appendix J. Borrower Comments
I. Comments from the Government:
№ ____________________________
____________20 թ.г.y.
Mrs. MIDORI MAKINO
MANAGER, SUSTAINABLE DEVELOPMENT DEPARTMENT
INDEPENDENT EVALUATION GROUP
Dear Mrs. Makino
Taking the opportunity I would like to thank you for elaboration of the Draft Performance
Assessment Report of Energy Efficiency Project. The Ministry of Energy Infrastructures and
Natural Resources have the following
comments on this matter: in the Report.
1. Add in the 1.2 of Project Background and Context;
“Amendments and addendums made to the 2016 Law on Energy Saving and Renewable
Energy which define a number of mandatory requirements, such as:
‐ energy efficiency requirements for buildings;
‐ energy balance requirement;
‐ labeling requirements for energy consuming devices;
‐ disposal requirement for life‐threatening and environmental hazardous devices;
‐ energy expertising requirement”.
45
2. Add in Legal Acts of Appendix E: List of EE existing legal, regulatory and
normative‐technical documents in Armenia:
II. Comments from the implementing agency, the R2E2 Fund:
Unfortunately, the Government adopted ESAs as a sample agreement for energy efficiency
procurement for the WB‐ Financed Energy Efficiency Project only, thus after the project
closing it was impossible to apply it.
The R2E2 Fund tried to continue applying the same model of procurement. It revised the
“Open Bidding” Public Procurement document to adjust to NPV‐ based procurement, but it
faced barriers, i.e. the Procurement Law says that a public facility itself should proceed with
a tender to select a company to perform energy service in its building. The R2E2 Fund
should be selected by the relevant public entity via a tender process to proceed with NPV
Procurement. In fact, the R2E2 Fund might be approached directly by a public facility, but
only in cases where it is registered with the Anti‐Monopoly Committee as a unique fund to
proceed with energy service in their facilities, or where there is a new Government Decree
adopted. Registration in the Anti ‐Monopoly Committee is expensive and there is no
guarantee of success. With regard to the Government Decree, the R2E2 Fund prepared the
Draft Decree with all enclosed documents in the end of 2017, but still, perhaps because of
the government changes, the progress is difficult to see. Currently, the R2E2 Fund is
negotiating with the new government to accelerate the approval process.
The R2E2 Fund approached Banks for co‐financing but given the fact that the majority of
public entities have no right for taking out a loan, the Banks preferred working with the
private sector.
It is important to note that the R2E2 Fund has not de‐capitalized itself and continues making
investments in Energy Efficiency and Renewable Energy and is actively seeking financing
for replenishment of the revolving fund.
After the project closure, the R2E2 Fund has been working actively in creating the EE
environment in Armenia: the Fund organized a series of workshops and Energy Weeks,
where priority was given to energy efficiency; it initiated the implementation of energy
efficiency projects in cooperation with the financial institutions (i.e., ACBA Credit, ACBA
Leasing, Global Credit); it conducted an energy audit of Yerevan Metropolitan; and it
cooperated and assisted with the Health PIU and Ministry of Defense of the Republic of
Armenia in finding energy efficiency solutions.
46
Another important achievement is that the R2E2 Fund was incorporated in a Covenant of
Mayors as a supporter and is assisting the communities in preparation of a Sustainable
Energy Plan, and the development of energy saving measures for community facilities.
Also, the R2E2 Fund has successfully passed the first stage of accreditation to Green Climate
Fund (GCF) and has already issued documents for the second stage. As soon as the R2E2
Fund becomes eligible for GCF financing, it may get considerable (about 10 million USD)
financing for replenishment of the revolving fund.
In the case of a grant, like the UNDP/GEF project, it is easier to meet EE‐related codes and
standards, but the scheme of our project did not allow us to include measures that would
not ensure cost recovery, i.e., insultation of tuff‐stone walls.
47
48