ICRR 14441
Report Number : ICRR14441
IEG ICR Review
Independent Evaluation Group
1. Project Data: Date Posted : 12/29/2014
Country : Poland
Project ID : P070246 Appraisal Actual
Project Name : Energy Efficiency Gef US$M ):
Project Costs (US$M): 53.5 46.8
Project
L/C Number : Loan/ US$M ):
Loan /Credit (US$M): 11.0 11.0
Sector Board : Energy and Mining US$M):
Cofinancing (US$M ): 42.5 35.8
Cofinanciers : Financial Board Approval Date : 10/14/2004
Intermediaries and Closing Date : 06/30/2011 12/31/2012
Private Sector
investment in Thermal
Modernization
projects.
Sector (s): Energy efficiency in power sector (60%); Banking (20%); Health (10%); Primary education
(10%)
Theme (s): Climate change (67% - P); Other financial and private sector development (33% - S)
Prepared by : Reviewed by : ICR Review Group :
Coordinator :
Richard L. Berney Alain A. Barbu Christopher David IEGPS1
Nelson
2. Project Objectives and Components:
a. Objectives:
The Global Environmental objective of the proposed project was to increase public and private sector investments
in energy efficiency in buildings in the territory of the recipient (Trust Fund Grant Agreement page 17). The same
wording is used in the Appraisal Document (page 3)
b.Were the project objectives/key associated outcome targets revised during implementation?
No
c. Components:
Component 1 - Partial Guarantee Facility (PGF) for Energy Efficiency (EE) project financing - (Appraisal US$5.7
million; Actual US$0.050 million): It was to provide commercial banks partial coverage of risk exposures against
loans made for EE projects in buildings throughout Poland .
Component 2 - Capital Grant Facility (CGF) - (US$2.0 million; Actual US$2.05 million): It provided 30 percent grants
for EE projects larger than US$250,000 and was expected to leverage US$ 6.67 million in EE financing, including
US$ 4 million in commercial loans and Krakow Energy Service Company (KESCO) financing and US$ 670,000
co-financing by clients.
Component 3 – Technical Assistance - (Appraisal US$3.3 million; Actual US$2.25 million) The focus was on
supporting EE through activities to increase awareness and demand for EE investments among both private and
public building-owners: building capacity in the Bank Gospodarstwa Krajowego (BGK) for administering the
guarantee facility and collecting project monitoring data and disseminating information on the results; support to
Energy Service Company (KESCO), which the earlier Krakow Energy Efficiency Project had helped to establish, for
the development of an energy performance contracting model; and provision of training to local banks .
Component 4: Thermo-Modernization Investments (TM ) Component - Added in the 2011 restructuring- (Initially
US$5.8 million, Actual US$6.65 million -reallocated from the PGF (US$5.65 million and the TA Component (US$0.15
million): ). This component used the existing delivery mechanism of the national TM Fund, which provided up to 20
percent subsidy for commercial EE loans for TM in residential buildings . It was expected to leverage US$ 58.5 million
investments in EE.
d. Comments on Project Cost, Financing, Borrower Contribution, and Dates:
Project Costs and Financing: The project was financed by a GEF grant of US$ 11 million from the GEF. The funds
were fully utilized in support of energy efficiency investments in the building sector . The cost of implementing and
servicing these loans was not included in the total project cost .
Government Contribution: The Government did not contribute directly to any funds, although it did support some of
the same investment programs that the GEO funds supported .
Dates: The closing date was extended three times . In the first restructuring in April 2011 a new project component
was added and the closing date was extended from June 30, 2011 to August 31, 2011 to provide the Government
sufficient time approve the Amended Grant Agreement that included the new project component . In August 2011 the
closing was extended to June 31, 2012; and in May 2012, the closing date was extended to October 31, 2012. These
latter two extensions provided the necessary additional time to implement the restructured components and complete
remaining TA activities.
3. Relevance of Objectives & Design:
a. Relevance of Objectives:
Substantial : Global Environment Objective of increasing public and private sector investments in energy efficiency in
buildings was highly relevant when the project was approved . It remains highly relevant to Poland’s economic
development and priorities in the energy sector, as reflected in the ‘Energy Policy of Poland until 2030’ (adopted on
November 10, 2009). Improving EE is identified as one of the key priorities to achieve the targeted zero -energy
economic growth until 2030 and reduce the energy intensity of Poland ’s economy to the EU-15 level. More
specifically, this long term energy strategy recognizes EE as a key area to : (i) meet the EU climate change targets;
(ii) achieve the national energy savings target of nine percent by 2016; (iii) avoid pollutant emissions; and (iv)
stimulate investments in modern energy -savings technologies and products .
Similarly, the GEO remains highly relevant and consistent with the current Country Partnership Strategy (CPS)
2009-2013 by contributing to growth and competitiveness of the economy through targeting the increase in private
sector investments. And finally, the GEO remains closely aligned with GEF ’s Strategic Priorities for the Climate
Change Focal Area which the Bank continues to support .
b. Relevance of Design:
Modest : At the time of appraisal, the buildings sector remained an underserved market for energy efficiency (EE)
investments. Market barriers, included insufficient access to commercial EE financing for up -front investment costs
and High transaction costs for relatively small EE projects . Both of the project’s original investment components were
highly relevant to the project objectives . The Partial Guarantee Facility was designed to reduce the risks that
commercial banks faced in lending to EE projects by providing partial coverage of risk exposures for such loans . The
Capital Grant Facility provided subsidies for bundled EE projects larger than US$ 250,000.
However, the project design failed to take into account the myriad of alternative subsidized -financing programs
provided through both EU and local programs including :
The EU Cohesion Funds available for EE and RE (about Euro 54 million), which became available after Poland ’s
accession to the EU in 2004 and supported projects with up to 85 percent subsidies;
The EU Guarantee Fund, which provided guarantees for projects co -financed co-financed from EU subsides.
The National Fund for Environmental Protection and Water Management, which inter alia provided soft loans
and grants for EE in the residential and public sector;
The Polish EcoFund Foundation, which provided 10-15 percent grants for EE in the private and public sector,
including heat insulation in buildings; and
The National Credit Guarantee Fund (KFPK), which operated through 27 participating banks, and had simpler
applications procedures and less stringent reporting requirements .
These failings resulted in the need to restructure the project in 2011. After restructuring, the project design remained
relevant to the current energy sector needs . Specifically, the TM Program continued to be the key vehicle in
improving EE in residential buildings, while the relevance of the Energy Performance Contract (EPC) mechanisms for
public buildings were expected to increase in the future, inter alia, due to the Government’s enhanced obligations
under the new EU Directive on energy efficiency (2012/27/EU), which requires member states to encourage the use
of energy performance contracts in the public sector . The Government expects to translate this directive into national
law in 2014. .
4. Achievement of Objectives (Efficacy):
Achievement of Global Environmental Objective : to increase public and private sector investments in energy
efficiency in buildings . Rating : Modest
Overall, the project supported implementation of a total of 478 EE projects, including 32 projects based on EPC,
thereby providing capacity development and raising awareness of commercial banks in possibilities for lending in this
sector. However, the Guarantee Facility failed to stimulate demand for EE services, and the impact of the
Thermal-Modernization (TM) project component is difficult to evaluate, since it simply piggybacked on the already
existing Government TM support program .
Target 1: Overcoming the risk barriers that inhibit commercial bank participation in EE project financing .
Country-wide, there was a 14-fold increase in commercial lending for EE, between 2004 and 2012, with total lending
exceeding the project’s expected outcome target by more than US$ 900 million. However this increase cannot be
attributable to the project’s successful use of the guarantee instrument to overcome the risk barriers as the project
intended. Rather, the key forces driving the expansion of commercial bank participation in EE programs were the
changing market conditions in the housing sector which resulted in increased availability of financing along with a
general reduction of collateral requirements for building owners and cooperatives .
Outputs :
Energy Savings target for the Thermo -Modernization (TM) loans: (Target - 68 GWh/y, Achieved - 58 GWh/y).
Leveraging of Bank funds in the TM project component was lower than originally estimated . As a result the output
fell 15% short of the target.
Energy savings from the EE investments supported under the CGF : (Target 42 GWh/y; Actual 10.6 GWh/y). Projects
financed under this program had longer payback periods and significantly higher average costs than anticipated at
Appraisal.
CO2 reduction: Project investments resulted in a total CO 2 reduction of 26,000 T/y, compared with the outcome
expected at appraisal of 37,600 T/y. The TM project reduced CO2 emissions by 22,000 tons, 15% short of the
expected outcome of 26,000 T/y. The CGF financed projects reduced CO 2 emissions by 4,000 T/y, only a third of the
target of 11,600 T/y.
Outcomes :
The Partial Guarantee Facility was expected to leverage about US$ 39 million in debt financing for 390 EE projects,
went unutilized and was eventually cancelled for reasons already described in Section 3b, Relevance of Design,
Energy Saving EE investments supported by the Capital Grant Facility (CGF): (Target – US$6.67; Actual US$8.67
million) The target was exceeded by 30%. KESCO used the CGF for 32 EE projects in Krakow.
The Thermal Modification project component directly leveraged US$ 43.8 million in debt financing and about US$
3.39 million co-financing from beneficiaries. In doing so, it supported the Government ’s TM highly successful
program, which accounted for between the 84 percent (in 2011) and almost 100 percent (in 2009) of total energy
efficiency loans in the residential sector, during a time period when the total number of EE projects implemented in
Poland has increased by more than 8 times from 3,195 projects in 2004 to 28,115 projects in 2011. By filling a
critical financial gap in this program, the project also helped to avoid delays in implementing 445 EE projects in the
residential sector. Sector participants reported that the GEF contribution helped to sustain a stable and predictable
investment environment for residential housing owners and participating banks under the TM Program in
2011-2012. However, there is no breakdown of data available to demonstrate the exact impact of the contribution .
And more importantly, the TM Investments Component did not stimulate a transition towards more market -based
models for EE financing.
Co-financing contributed by building owners in KESCO ’s GEF supported projects: (Target US$0.67 million; Achieved
US$3.39 million). In the 32 EE projects that utilized the CGF, project beneficiaries contributed five times the amount
of their own funds than had been expected at Appraisal .
Target 2: Demonstrating the feasibility of packaged investments in higher -cost EE measures in buildings:
KESCO developed a simplified EPC model for public buildings, which helped to reduce investment transaction costs .
This model used the ‘deemed savings’ approach, (i.e. contracts that were based on pre -determined savings
estimates), which has been used as a basis for EE projects in two other townships in the region around Krakow .
Outputs:
KESCO implemented 32 GEF supported EE projects based on this simplified EPC model . This was accomplished
without significant delays in identifying and contracting bundled EE projects . The TM Investments Component
supported 26 Projects with investment volumes exceeding US$ 250,000.
Outcome:
The total number of EE projects in Poland that were larger than US$ 250,000 increased from 151 in 2004 to 1,830 in
2012. This 12-fold increase indicates that the feasibility of packaged investments in higher -cost EE measures has
been successfully demonstrated in the Polish market .
At the project level, KESCO was successful in demonstrating the feasibility of packaged investments in the public
buildings sector. By successfully implementing EPCs for energy efficiency investments outside those for street
lighting KESCO helped to increase awareness of the EPC financing mechanism . When measured ex-post, actual
savings almost always exceeded estimates, which has helped to build confidence in EPC for beneficiaries . However,
KESCO’s business model for financing higher cost EE projects was totally dependent on the GEF grants as the
primary driver for sales, and when this support ended, and no new source of concessional funding was made
available, interest in higher cost EE retrofitting building investments in have, essentially, disappeared .
Target 3: Increasing acceptance of the EPC mechanisms .
Outputs: :
The total annual value of energy service projects implemented throughout Poland is estimated at Euro 5-10
million.However, in the public sector, the annual volume of EPC projects is estimated to have decreased . since the
Project appraisal due to unfavorable legislative provisions and few public tenders for EPC ;
Outcome:
Given KESC’s limited geographical scope of operation and the lack of disaggregated data for Krakow, it is not
possible to demonstrate a causal linkage between the GEF Project and regional or national outcome levels . The ICR
concludes that the acceptance of EPC mechanisms in Poland has remained below expectations compared to the
potential market size and the development in other EU member states .
5. Efficiency:
Modest : Over the eight year of implementation, the Project supported 478 EE projects valued at US$ 57.8 million.
Annual energy savings generated amount to about 68.4 GWh year and CO2 emissions were reduced by 26,226 t per
year. The Incremental Cost Analysis showed project -related costs of CO2 abatement of US$ 28.0 per tons CO2
compared with an estimated US$ 8.0 per tons CO2 at appraisal (at zero percent discount rate ). Both are substantially
higher at a 10 percent discount rate. The GEF fund of US$ 11 million leveraged an additional US$ 46.8 million, for a
leverage ratio of about 1-to-4.3, compared with an estimated 1-to-10.7 at appraisal. The difference is primarily due to
the considerable scale-down of the guarantee facility (from US$ 5.7 million to US$ 50,000) and the redirection of the
reserve funds to TM investments .
The Benefit-Cost Analysis indicates economic payback periods of 18.2 year at completion, compared with 5.6 years
estimated at appraisal, primarily due the fact that the actual subprojects included a much lower proportion of window
replacements and other high-pay-back measures than was assumed at appraisal . These pay-back times are not
unreasonable for building components with technical lifetime extending far beyond the 15 years horizon used in the
analysis. Accordingly, annual energy cost savings amounted to US$ 3.2 million compared with an estimated US$
11.6 million per year at appraisal. Only the CGF component has both an ex-ante and ex-post estimated EIRR. This
component accounted for only US$ 2 million of the project expenditures of US$ 11. The EIRR of this component is
estimated at 0.6 percent. This low investment efficiency is mostly due to a conscious choice of including investment
packages with high pay-back-time measures so as to achieve higher effectiveness in terms of savings achieved in
each individual building.
ERR )/Financial Rate of Return (FRR)
a. If available, enter the Economic Rate of Return (ERR) FRR ) at appraisal and the
re-estimated value at evaluation :
re-
Rate Available? Point Value Coverage/Scope*
Appraisal Yes 4.8% 18%
ICR estimate Yes 0.6% 18%
* Refers to percent of total project cost for which ERR/FRR was calculated.
6. Outcome:
The Project objective remains relevant to Poland ’s national and sector priorities . But the project only modestly
achieved its GEO objective. While all the outcome targets were achieved, the measurable contribution of the GEF
Project remained limited, and 5 out of 7 project-specific outputs were below their target values . The slow and limited
progress in achieving the GEO objective was reflected in the project's moderately satisfactory DO supervision ratings
between 2007 and 2011. Combined with the project's modest efficiency, this yields a rating of Moderately
Unsatisfactory for overall Outcome .
a. Outcome Rating : Moderately Unsatisfactory
7. Rationale for Risk to Development Outcome Rating:
Sustainability of EE measures implemented : The energy savings from the EE measures in buildings financed by
the project are expected to be sustained for the technical /economic lifetime of the retrofitted component (more than
15 years). The measurement and verification activities conducted by KESC upon implementation of the EE measures
demonstrated that actual savings were achieved and exceeded deemed savings .
Continuation of KESCO: The Krakow Municipal District Heating Company (MPEC) is committed to KESCO
continued operating as one of its subsidiaries . Recently, ESCO-type pilot programs in public buildings and EE project
based on Public-Private Partnerships, which are repaid from energy savings, have slowly started to increase in
several municipalities, providing more opportunities for ESCOs . However, while these developments, along with the
provisions of the new EU Directive, create new opportunities for KESCO, the company has yet to demonstrate a
sustainable business model for EE services without GEF grant incentives .
a. Risk to Development Outcome Rating : Moderate
8. Assessment of Bank Performance:
a. Quality at entry:
The Project was highly innovative in its design and was among the first World Bank projects piloting
guarantee instruments and EPCs mechanisms to improve EE in the building sector .
However, given the novelty of the EE guarantee instrument as well as the time gap between start of preparatory
work and Project appraisal, the risk assessment and quality at entry would have benefited from market surveys to
confirm interest for the guarantee instruments with a ready pipeline of bankable EE projects .
at -Entry Rating :
Quality -at- Moderately Unsatisfactory
b. Quality of supervision:
The supervision Team had a flexible approach and applied innovative and creative thinking to tackle the
implementation constraints. The Task Team conducted regular implementation support missions, including
thorough supervision of fiduciary and safeguard aspects . Implementation Status and Results Reports (ISRs)
highlighted key issues arising during implementation, and realistically rated the performance of the project both in
terms of achievement of development objectives and project implementation .
The team consisted mostly of only two experts to cover both the broad energy sector issues as well as specific
EE, financial and institutional issues, which may raise the question if it was equipped with sufficient resources to
react quickly and effectively to the challenges experienced during Project implementation . It could have
responded earlier to the need to adapt to the changes in the EE market and to the failure of the project ’s
Guarantee Component, which was rated moderately unsatisfactory one year after the project became effective
and was subsequently downgraded to unsatisfactory during the following 3.5 years until restructuring in 2011. In
addition, some of the key remedial actions identified in ISRs (e.g. revolving audit fund, intermediate facilitator
platform on performance basis) ended up being implemented with significant delays or not at all .
There was also some equivocation on whether to continue or close the project before
the restructuring finally took place . While the Bank eventually concluded that the Project ’s value for the
preparation of the US$ 1.1 billion DPL on EE outweighed its poor performance, these deliberations further
delayed the Bank’s responsiveness to its disbursement problems .
Quality of Supervision Rating : Moderately Unsatisfactory
Overall Bank Performance Rating : Moderately Unsatisfactory
9. Assessment of Borrower Performance:
a. Government Performance:
Overall, the Government confirmed its commitment to EE in buildings by sustaining the TM Program and by
continuing the gradual increase of energy tariffs, which enhanced the commercial viability of EE projects . During
implementation, it helped to foster progress towards the GEO by strengthening the legal and regulatory
environment for EE in buildings in accordance with the relevant EU Directives; and by adopting several key
strategies and policies on EE, including the national EE Action Plans in 2007 and 2011 and the ‘Energy Policy of
Poland until 2030’, which recognized EE as a key priority in the energy sector . However, the Project would have
benefited from a more specific and consistent policy support for the timely adoption a comprehensive legal and
regulatory framework for EE.
Government Performance Rating Moderately Satisfactory
b. Implementing Agency Performance:
During the first years of implementation, BGK faced significant challenges due to its lack of experience with
implementing World Bank Projects, organizational and managerial changes, and the lack of progress under the
guarantee component. BGK concentrated on the administration and negotiation of Guarantee Framework
Agreements, but the mid-term review in 2007 concluded that interactions and targeted support to commercial
banks for deal origination were limited and not sufficient to address poor performance of the guarantee com
ponent. The performance of BGK improved considerably after 2010. After some initial difficulties(discussed in
Section 2.3 above) compliance with Bank safeguard and fiduciary requirements also increased during project
implementation.
Implementing Agency Performance Rating : Moderately Satisfactory
Overall Borrower Performance Rating : Moderately Satisfactory
10. M&E Design, Implementation, & Utilization:
a. M&E Design:
. The M&E framework was designed to focus primarily results in Poland as a whole rather than on the results that
could be attributed to the project . The outcome indicators focused on the number, size, financing and energy savings
of EE building related investments in Poland as a whole, as did four of the ten intermediate outcome indicators . In
addition,most of the outcome indicators were not clearly defined in quantitative terms, The Results framework was
strengthened after the restructuring and selected output indicators for the TM Investments Component were
associated with clear quantitative targets .
b. M&E Implementation:
There were some inconsistencies in reporting related to outcome and output targets . In anticipation of the
restructuring, a local consultant was contracted in 2010 with a focus on outcome indicators and the new TM
Investments Component, which helped to address several gaps in M&E implementation .
c. M&E Utilization:
. Output indicators of the PGF component were used to emphasize the project's poor performance and need to
restructure, but this was not translated into remedial action .
M&E Quality Rating : Negligible
11. Other Issues
a. Safeguards:
The Project was rated Category B . because there were no significant adverse environmental issues associated with
EE improvements in housing that resulted in reduced energy consumption and CO 2 emissions. There were no
significant safeguard issues during implementation .
b. Fiduciary Compliance:
Organizational and managerial changes during the initial phases of the project had an adverse impact on project
implementation and financial management . This problem was rectified after the project implementation agency (BGK)
recruited highly qualified staff and reorganized the Project Management Unit (PMU). As a result, the PMU became
more effective and financial management capacity was greatly improved . The overall financial management rating for
the Project was satisfactory . Project financial monitoring reports generated by the IT system were submitted on a
timely basis and were acceptable . Annual audits of the project financial management statements were received in
due time and contained clean opinion with no accountability or internal control issues .
c. Unintended Impacts (positive or negative):
None
d. Other:
None
12. Ratings :
12. ICR IEG Review Reason for
Disagreement /Comments
Outcome : Moderately Moderately
Unsatisfactory Unsatisfactory
Risk to Development Moderate Moderate
Outcome :
Bank Performance : Moderately Moderately
Unsatisfactory Unsatisfactory
Borrower Performance : Moderately Moderately
Satisfactory Satisfactory
Quality of ICR : Satisfactory
NOTES:
NOTES
- When insufficient information is provided by the Bank
for IEG to arrive at a clear rating, IEG will downgrade
the relevant ratings as warranted beginning July 1,
2006.
- The "Reason for Disagreement/Comments" column
could cross-reference other sections of the ICR
Review, as appropriate.
13. Lessons:
The following lessons were abstracted from the ICR.
1. Innovative instruments should be kept simple with low transaction costs. In a market environment that is not
familiar with innovative instruments, such as ESCOs and EPCs, it is can be easier to start with simple ESCO-type
models to get initial market traction. POE ESCO offered simple contracts, including guaranteed ‘deemed savings’
to public building institutions, which helped to reduce transaction costs and started building awareness among
targeted public institutions.
2. ESCOs need financial safety net in start-up phase. A financial safety net provided by a parent utility can
substantially facilitate access to commercial financing by a start-up ESCO and mitigate risk aversion of new
clients. For instance, guarantees provided using the balance sheet of its owner, MPEC, were key for POE ESCO in
accessing financing from commercial banks.
3. Strong sub-project pipeline is essential. When introducing new market-based instruments, such as guarantees, it
is critical to have a strong, readily available pipeline of bankable EE projects in order to reconfirm existing market
demand for the product, allow a quick start and early dissemination of success stories and address capacity
constraints from financial intermediaries early on and in a targeted manner.
4. Guarantee instruments are often not critical for EE lending. The performance of a guarantee mechanism is
strongly dependent on prevailing conditions in the credit market. While the guarantee instrument aims to enhance
lending to underserved market segments due to high risk perceptions, lack of creditworthiness remains a key
constraint to open access to a new client base. The failed attempts to refocus the PCG into higher risk segments
demonstrated that commercial banks often require similar collateral for guarantees as for loans, inter alia due to
the high transactions costs involved in calling guarantees and recovery at default. Thus, while guarantees may be
used as an additional safety layer for already creditworthy clients, they are not perceived as a critical mean to tap
into a new client base where commercial banks had previous concerns related to collateral.
14. Assessment Recommended? Yes No
15. Comments on Quality of ICR:
This was a good ICR. It explained the project results in the context of the environment in which the project was
implemented, and thoroughly enumerated the reasons for the less than satisfactory outputs . One area where it could
have been more explicit was in the discussion of the project's target outcomes, which, as noted, focused mostly on
outcomes for which the project could not claim to have effected . Considering this project's close relation to the EE
Development Policy Loan mentioned in the ICR, it would have been useful to discuss the synergies and inform the
reader about the current status of this loan .
a.Quality of ICR Rating : Satisfactory