COORDlNATION
Risk and Uncertainty:
Selection Criteria for Projects
Offering Net Positive
Domestic Benefits
Subodh C. Mathur
June 1994
Early Release Version
THE WORLD BANK
1818 H Street, N.W.
Washington, D.C. 20433 USA
Telephone 202 473 1816
Risk and Uncertainty:
Selection Criteria for Projects
Offering Net Positive
Domestic Benefits
Subodh C. Mathur
June 1994
DRAB7'
Abstract
In recent times, the application of standard cost-benefit analysis to development projects
has led to appraisals are biased upwards,and are also poor predictors of the actual returns from
these projects. Similarly, the North American and European experience with energy
conservation projects has shown a clear tendency to under-estimate costs and over-estimate
benefits. Hence, unless appropriate changes are made in the methodology, ex-ante appraisals
of energy conservation projects in developingcountries are likely to be poor guides bf theactual
outcomes from the projects as well as of the host country's willingness to actually implement
the projects.
Consequently, GEF's rule of classifying projects as ineligible for GEF support (Type I
projects) based on the appraisal of net domestic benefitsalone suffers from two potentialflaws.
First, under the present methodology, the appraisals tend to result in over-optimistic
assessments, so that GEF may fail to support projects that may need support. Second, this
classification method fails to take account of the host country's assessment of the project, which
may not be based solely on the appraised net domestic benefits. Thus, GEF may mistakenly
conclude that the host country will implement the project on its own. It is suggested that GEF
revise its decision rule to take account not only of the appraised net domestic benefits, but also
of other factors such as the amount of global environmental benefits at stake and the level of
financial support to be provided by GEF.
TABLEOF CONTENTS
EXECUTIVESUMMARY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . i
1. INTRODUCTION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
2 PROBLEMS IN COST-BENEFIT ANALYSIS OF WORLD BANK PROJECTS
. ... 4
Statistical analysis of World Bank project appraisals ................... 6
Implications for GEF . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
3 COSTS OF ENERGY CONSERVATION PROJECTS
. . . . . . . . . . . . . . . . . . . . 12
Consideration of Government Costs . . . . . . . . . . . . . . . . . . . . . . . . . . . . .12
Consideration of Utility Costs ................................ 13
Consideration of End-user Costs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
Implications for GEF . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
4. BENEFITS OF ENERGY CONSERVATION PROJECTS ................. 17
Improper definition of program impact ........................... 18
Lower than expected participation rates .......................... 19
Presence of "Free Riders" and "Takeback" Effects ................... 23
Equipment failure. misuse and persistence of effects . . . . . . . . . . . . . . . . . . 25
Implications for GEF ..................................... 26
5 UNCERTAINTY. RISK. AND DECISION RULES
. ..................... 28
Uncertainty and risk in theory and project appraisals . . . . . . . . . . . . . . . . . . 28
Government response to uncertainty ............................34
GEF's response to uncertainty ................................ 35
6. SUMMARY AND CONCLUSIONS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .37
Flaws in the application of cost-benefit analysis . . . . . . . . . . . . . . . . . . . . . 37
Flaws in estimating costs of energy conservation projects . . . . . . . . . . . . . . .38
Flaws in estimating benefits of energy conservation projects . . . . . . . . . . . . . . 39
Uncertainty. risk. and decision rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
REFERENCES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
evaluate and possibly provide funds for the proposed project; (ii) Utility costs, which are
incurred by the implementation agency (and associated trade allies or non-profit groups) that
executes, monitors, and possibly conductsongoingor a-post evaluationsof the project; and (iii)
End-user costs, which are incurred by the end-users, who are the beneficiaries of the project,
but who may also have to bear some costs.
6. Appraisals and evaluations of energy conservation programs in North America and
European have frequently failed to consider the full panoply of costs, even when a conscious
effort has been made to be comprehensive. Based on this experience, there is likely to be
significant underestimation of all three categories of costs inn potential GEF projects. In
principle, it should be relatively straightforward to verify that all three categories of costs have
been taken intoaccount in the cost-benefitanalysis of GEF projects. However, in practice, there
may be difficulties in arriving at reasonable estimates of costs, and some costs may be
overlooked completely. Hence, actual v i a l costs of energy conservation projects are likely to
be under-estimated, unless special care has been taken to ensure that this is not the case.
Flaws in estimating benefits of energy conservation projects
7. Based on the experience in North America, it is a common theme that energy savings,
which are the benefits of the energy conservation programs, have been frequently and
significantlyoverestimated in theex-ante appraisals of these programs. In any case, the "actual"
savings from an energy conservation project are often to difficult to calculate precisely because
they are equal to an actual post-installation consumption subtracted from a hypothetical baseline
consumption that would have occurred had the program not been in place, and all other factors
had held constant.
8. There are a number of factors that lead to ex-post energy savings which are less than the
a-anre anticipated savings: (i) improper definition of program impact; (ii) lower than expected
participation rates; (iii) "Free riders" and "takeback" effects;and (iv) equipment failure, misuse
and lack of persistence of effects. While it may be relatively straightforward to ensure that the
proper definition of program impact -- the net program impact -- is used, there may be
difficulties in taking account of the other factors. For instance, the actual participation rate will
depend upon a diversity of factors, such as the discount rate, transaction costs, the priorities of
the end-users, and the nature of the promotional campaign, all of which may be difficult to
estimate or assess on an a-ante basis. Similarly, there may be unanticipated takeback effects,
which implies that the initial declinein energy consumption and total energy costs brought about
by the energy-efficient technology induces end-users to increase their use of the service, e.g.,
users may use their energy-efficient compact fluorescent lamps for longer hours than
conventional lamps. Finally, the history of energy projects in the developing countries indicates
that there is a potential for significant problems in the installation, proper use, and maintenance
of energy efficient technologies and devices, particularly for those with which the local people
have limited familiarity and experience. Thus, even if there are initial savings from energy
conservation projects, these have the potential of declining steadily over time. It would be
prudent for GEF to verify whether the proposed project has taken account of this potential
decline in benefits over time, or taken steps to prevent such a decline.
Uncertainty, risk, and decision rules
9. It is likely that even after the biases in ex-ante estimates of costs and benefits of energy
conservation projects have been reduced or eliminated, there will remain substantial uncertainty
about the actual net benefits of energy conservation projects that GEF may support.
Consequently, any decision-maker, i.e., the Government or GEF, who relies 00 the ex-ante
appraisal to make a decision about an investmentproject faces significant probabilities of getting
'false-positive" and/or "false-negative" results. For example, the Government may end-up
undertaking projects that do not deliver the expected results ("false-positive") and/or it may fail
to undertake projects that would have brought substantial benefits to the country ("false-
negative").
10. Whilethereis an extensiveliteratureon risk and uncertainty,even well-known theoretical
c~nceptshave not been extensively incorporated into project appraisals. At the same time, it
is also clear that many of the theoretical results available in the literature are not sufficiently
practical to be readily applied in the appraisal of projects in developing countriesL
Government response to uncertainty
11. In view of the biases and uncertainty associated with the conventional ex-ante appraisal
of net domesticbenefits, governments may be reluctant to rely solely on such appraisals to make
decisions about undertaking energy-efficiency projects. It is likely that different Governments
would have different responses to the uncertainty associated with GEF projects. One response
of the decision-maker may be to conduct a heuristic, back-of-the-envelopeanalysis to determine
"low-case" or "worst-case" outcomes. An extremely risk-averse decision-maker may wish to
be sure that the worst-case scenario associated with the project is acceptable. Or,a risk-averse
decision-maker may approve only those projects whose "low-case" estimateof the net domestic
benefits exceeds a particular value. Even if no analysis is conducted to determiqe"low-case"
outcomes, a Government may be willing to undertake only those energy conservqtion projects
;hat require low initial capital expenditures, so that the project can be terminatq at relatively
low cost if there are early indications that the project will fail to achieve its projects benefits.
CEF's response to uncertainty
12. Apart from the uncertainty associated with projected net domestic benefits, GEF also has
to consider the uncertainties associated the decision-making rules used by Governments. It is
clear that a host country may not undertake a project even if GEF classifies it as Type I, in
which case the global environmental benefitsassociated with it will not be realized.
13. An attempt by GEF to take account of the Government's decision-rule in formulating
GEF's own decision-makingrule about whether or not to support a particular project would lead
to a moral hazard, because it would provide an incentives for Governments to adopt a stated
policy of "We will not undertake these types of projects on our own" merely in order to secure
GEF support. Thus, GEF will run the risk of supporting projects that do not need GEF support,
i.e., "false positive" results.
14. On the other hand, if GEF continues to classify projects as Type I and Type I1based on
the appraised net domestic benefits, then GEF faces two problems. First, until the project
appraisal methodology is modified to take account of its failings and biases and risks, the
appraised net benefits are seriously flawed,and GEF would run the risk of both "false positive"
and "false negative" results, where "false negativen represents'failing to support projects that
require GEFsupport. Second, even with an appropriatelymodified methodology, a decision rule
based solely on the (correctly) projected net benefits will fail to take account of the potentially
different responses of different Governments to similar projects, i.e., some Governments may
be more risk-averse than others. Therefore, GEF may face significant risks of getting "false
negative" results.
15. Mathematically, it is not possible for GEF to develop a decision-making rule that
simultaneously minimizes the probabilitiesof both "false positive" and "false negative" results.
Therefore, in deciding whether to focuson "false negative" or "false positive" results, GEF will
have to take account of the consequences of these types of results. If substantial global
environmental benefits are at stake, then GEF may be womed about denying a small amount
of support to projectand taking the risk that the project may never be undertaken, i.e., the focus
would be on minimizing "false negative" results. Alternatively, if substantial GEF funding is
required, then GEF may be womed about "falsepositive" results. The implication is that GEF
has to develop its own risk profile, and develop decision-making rules that take account of the
amount of GEF funding at stake and the estimated global environmental benefits.
DRAIT
1. INTRODUCTION
1.1 One of the analytical problems faced by the Global Environmental Facility (GEF) is the
rationale for providing financialsupport for so-called ''Type I" projects. These projects have
two definingcharacteristics: (i) they offer significant global environmental benefits, and (ii) they
appear to offer positive net domestic benefits when evaluated in the standard ewnomic cost-
benefit framework. The second characteristicimplies that Type I projects should be undertaken
by host countries without any need for financial support from GEF based on global
environmental considerations. If these Type I projects are actually undertaken without GEF
financialsupport, then considerable global environmental benefits will be realized without the
use of GEF funds.
1.2 However, there are concerns that Type I projects may not be implemented even though
they appear to offer positive net benefits. If these projects are not undertaken, then the global
environmental benefits associated with the projects will not be realized. Thus, it may be
appropriate for GEF to provide financial support for such projects in order to realize the global
environmental benefits.
1.3 The objectiveof this paper is to identify thefactors that may lead projects to beclassified
as Type I even though they do not actually offer positive net domestic benefits. In accordance
with the terms of reference, the focus of this paper is on energy projects, partidularly energy
conservation projects that may have significant global environmental benefits. Nol new research
has been undertaken for this paper, and it is based on a review and synthesis of the results
available in the literature.
1.4 The basic approach taken in this paper is to consider the a-ante calculated net domestic
benefits associated with a project as an estimate that is subiect to error. There h e two broad
sources of error which lead to over-optimistica-ante assessments of projects relelbantfor GEF:
flaws in the application of cost-benefit analysis to developmeht projects in
general;
flaws specific to energy conservation projects.
Once these errors are taken into account, it may turn out to be the case thata project that
ppears to provide net domestic benefits actually does not do so. The failure to actually provide
positive net domestic benefits would then make a project a potential candidatefor GEF financial
support.
1.5 There has been substantial experience with energy conservation projects, and more
generally demand-side management(DSM)programs in North America and ~ u r d ~while e , such
projects are still relatively few in the developing countries. Hence, it is useful to consider the
problems that emerged in the North American and European experience with the appraisal and
implementationof these projects. As shown below, even after more than a decade of experience
with DSM programs in the U.S. and Europe, there is still a major need to improve the quality
of data collection, without which it is difficult to provide a rigorous appraisal and evaluation of
DSM programs.
Need for Better Data on DSM programs
1.6 Many observers in the U.S. who have been closely involved with DSM programs find
that the level and quality of the data bout these programs is inadequate. For example, after a
detailed analysisabout the type of data used for appraising and evaluating DSM programs in the
U.S., Hirst and Sabo (1992) found that "...the amount and quality of data now available on
DSM programs are far short of what utilities and regulatory commissions need. The current
lack of explicit, widely used definitions of DSM programs is a key deficiency ... we now
discuss DSM programs in a 'tower of Babel,' leading to disparate estimates of DSM potential
and performance." ,
1.7 Further, Hirst and Sabo (1992) concluded that "the program-cost data that utilities report
are often incomplete or not sufficiently detailed to use to compare or assess DSM program
performance. In addition, traditionalaccountingsystemsonly monitor utility expenditures. Costs
borne by the customer and other nonutility parties are often not provided by existing accounting
systems. Knowledgeof these costs is necessary for calculating program cost-effectivenessfrom
the perspective of participating customers and society."
1.8 In order to overcome these data problems, Hirst and Sabo (1992), whose work was
sponsored by the Electric Power Research Institute (EPRI)and the U.S. Department of Energy,
have developed a handbook that addresses the need for additional and better information in two
ways. First, the handbook contains discussions of the key concepts associated with DSM
programs-types, participation, energy and load effects, and costs. Second, the handbook offers
definitions and a sampling reporting form for utility DSM programs, so that there would be
greater consistency in the collection and reporting of data on DSM programs.
1.9 Similarly, Prindle (1991) concluded that "the difficulty with costs analyses in most DSM
studies is that they are based on engineering estimates or other methods with high levels of
uncertainty. More hard evaluation data on the cost, performance, reliability and other attributes
of DSM options is needed. If data can be provided with enough rigour to satisfy utility
regulators and planners, more money will flow into utility DSM programmes." In the absence
of such hard data, Prindle concluded that:
there remains a vast reserve of scepticism in the utility industry about DSM as a
real resource for utility planners. To the extent that the jury is still out on the
size, reliability, and longevity of DSM resources, this scepticism is justified ...
Wild claims are still made about the magnitude of DSM resources .. .
1.10 In the same vein, based on an analysis of the data for several European countries
(Belgium, Sweden, Norway, and Denmark), Bartlett (1993) concluded that "estimates of the
...Therefore,
electricity used for residential lighting in most countriesare subject to large measurement error
the potentialannual electricitysavings from the use of CFLs[compact fluorescent
lamps] cannot be accurately estimated."
Organization of paper
1.11 The rest of this paper is organized in thefollowing way. Section 2 presentsa discussion
of recent concernsabout flaws in theapplicationof cost-benefitanalysis to development projects
in general. Sections 3 and 4 present a discussion of the issues raised by errors in cost and
benefits, respectively, of energy conservation projects, based mainly on the experiencein North
America and Europe. Section 5 discusses therisksassociated with development projects, and the
decision rules relevant in the context of risky developmentprojects. Finally,Section 6 presents
the summary and overall conclusions.
appropriate, the findings of institutional development specialists and staff with
other skills in assessing the likely performance of project-related institutions.
Ensure that the macroeconomic, financial, technical, and behavioralassumptions
underlying the analysis are clearly spelled out.
A. Statistical analysis of World Bank project appraisals
2.7 Pohl and Mihaljek (1992) analyzed the data for 1,015 projects supported by the World
Bank over 1974-1987. The largest number of the projects were in agriculture (40%),followed
by transport (30%),and energy (20%),and a small number of projects in industry and urban
development. Pohl and Mihaljekanalyzed the relationship between "appraisal rates of return"
(estimatesof economic rates of return at the time when projects are designed and appraised)and
"re-estimated rates of return" (estimates of rates of return at the time when the projects are
completed and begin normal operation) 31.
2.8 Based on a statistical analysis, the authors reached three main conclusions:
there is an upward bias in appraisal rates of return;
there is little link between appraisal and re-estimated rates of return;
factorssuch as cost overrunsand completiondelays explain only a very small part
of the difference between appraisal and re-estimated rates.
Upward bias in appraisal rates of return
2.9 The nature of over-optimism in World Bank project appraisals is indicated by the data
in Table 1. It is clear that the appraisal rates are significantly higher than the re-estimated rates;
for example, the median appraisal rate is 18%,while the median re-estimated rate is only 14% .
Further, on average there have also been significant cost ovemns and implementation delays;
for example, on average, these projects took six years to complete, compared to an estimated
average completion time of four years, for an average time delay of two years.
2.10 It is clear that, on average, there was a bias towards optimistic assessments in project'
appraisals undertaken by the World Bank. However, it is worth noting there were a large
number of projects that did not suffer from this optimistic bias. While Pohl and Mihaljek's
statistical analysis did not differentiatebetween over-optimistic and other projects, it clear from
31 The re-estimated rates are not true ex-post rates of return, which can be calculated only
at the end of the project's economic life. Nevertheless, the re-estimated rate is a better indicator
of a project's performance than the appraisal rate becausethe re-estimated rate is based on actual
values for several critical variables, such as investment costs and project completion schedules,
while the appraisal rate is based on estimates.
their plot of the relationship between re-estimated and appraisal rates of return (see Annex 1)
that there were many projects for which the re-estimated rate of return was greater than or equal
to the appraisal rate of return. It followsthat if we examineonly the projects that had optimistic
assessments, then their optimistic bias is relatively greater than the average bptimistic bias
reported in Table 1 4/. In other words, for projects that have an optimistic bias, the bias is
greater than the average bias.
Table 1: Bias in A~~raisalsof Rates of Return of World Bank Proiects
Arithmetic Median Maximum Minimum
Mean
Estimated Ecoxiomic Rate of Return ( X )
At project appraisal 22 18 158 1
At project completion 16 14 128 - 20
Estimated Total Project Cost(US $ million, current prices) ~
I
At project appraisal 86 34 3,193 1
At project completion 102 40 4,045 1
Time Overrun(Years) 2 2 16 1-4
I
Note: Based on 1,015 World Bank projects over 1974-1987 I
Source: Pohl and Mihaljek(1992) I
2.11 An alternative measure of the optimistic bias is provided by the results of the regression
analysis reported by Pohl and Mihaljek. In their basic model, the dependent vhriable was the
re-estimated rate, with the appraisal rate as the explanatory variable, i.e., ,
Re-estimated rule = a + b Appraisal rare
I
In variants of this basic model, a number of other explanatory variables, such as the country's
economic management rating and GNP, and indicator (dummy) variables reprdsenting sectors
(energy, transport, etc.) and regions (Latin America, East Africa, etc.) were albo added.
-
41 The average bias reflects the upward bias in the optimistic projects as well as the
downward bias in conservativeprojects, where the re-estimated rate exceeded the appraisal rate.
- 8 - DRAFT
2.12 In these regression equations, the slope parameter b measures the effect of a change in
the appraisal rate on the re-estimated rate. For example, holding other things constant, if the
appraisal rate increases by 1, say from 12% to 13%, then the re-estimated rate increases by b.
Clearly, if b = 1, then a change of 1 in the appraisal rate will also imply a change of 1 in the
re-estimated rate; further, if b is less than 1, then a change of 1 in the appraisal rate will imply
an increase in the re-estimated rate of less than 1.
2.13 In the results reported by Pohl and Mihaljek, the estimated value of b is in the narrow
range of 0.43-0.46, which implies that when the appraisal rate increases by 1, the re-estimated
rate increases by considerably less than 1. Theextent of the optimisticbias in appraisals implied
by this estimated b is indicated by the following example. Consider two projects X and Y with
appraisal rates of, say 12%(X) and 27% (Y),so that the difference in the appraisal rates is 15%.
With b = 0.45, the difference between the re-estimated rates of the two projects is predicted to
be only 6.75%, and not 15%51 In other words, project Ys high appraisal rate is significantly
biased upwards.
2.14 Ener~vsector: Pohl and Mihaljek did not report the average appraisal and re-estimated
rates by sector. However, they did report the results of the regression analysis by sector. For
the energy sector 61, the estimated value is b = 0.61, which is closer to 1 than the overall
value of b = 0.45. This result indicates that the upward bias in energy sector projects may be
less than the general upward bias in World Bank appraisals.
2.15 Consequencesof U~wardBias: Oneresult of theexcessiveoptimismin theappraisal rates
of return is that there are a large number of projects whose re-estimated rates of return are
below conventionally acceptable rates of return. Specifically, about one-fourth of the projects
had a re-estimated rate of return below 10%; about one in seven (14%) project has a re-
estimated rate of return below 5%; and one in twelve (8%) of the projects had zero or negative
re-estimated rates of return.
Limited link between appraisal and re-estimated rates of return
2.16 There is only a limited link between the appraisal and re-estimated rates of return. . For
example, Pohl and Mihaljek (1992) found that "ninety percent of all projects have appraisal rates
of return in the range of 10-40%,but only about half have re-estimated rates of return within
this range."
2.17 Another indication of the limited link between appraisal and re-estimated rates of returns
is the low explanatory power of the regression equations. For the basic model, Pohl and
Mihaljek report an R2 value of 0.19, SO that the appraisal rate of return explains about only 20%
61 See Table 8, Pohl and Mihaljek (1992).
of the variation in the re-estimated rate of return; in a variant model which includes other
explanatory variables as well as indicator (dummy) variables for sectors and regions, the
explanatory power is 31%. Similarly, for the energy sector alone, the regression equation
(which includes other explanatory variables as well as regional indicator variables) explains only
30%of the variation in the re-estimated rate of return. 21
Reasons for divergence between appraisal and re-estimated rates of return
2.18 Proiect-specific factors: Intuitively, cost overruns and implementation delays are
expected to lower the re-estimated rate of return. However, in the regression analysis, these
explanatory variables were not statistically significantand also had the "wrong" sign. Further,
when nominal cost overruns were decomposed into unexpected inflation and realcost overruns,
the results of the regression analysis indicated that real cost overruns did not have a strong
adverse effect upon re-estimated rates of returns 81.
2.19 Regional differences: There is a clear geographical pattern in the diverpence between
appraisal and re-estimated rates of return. The estimated parameters of the regional indicator
variables imply that, for a given appraisal rate of return, the re-estimated rates of return are the
highest in South Asia, i.e., projects in South Asia show the lowest optimistic bias. South Asia
is followed by East Asia, Latin America, the Mediterranean,and the French African Community
(CFA), with projects in East and West Africa (other than CFA zone) haviqg the greatest
divergence between appraisal and re-estimated rates of return. Pohl and Mihaljbk attribute the
differencein the performanceof projects in CFA countries, compared toother Af$ican countries,
to the institutionalframeworkand the conservative fiscal and monetary policies f~llowedby the
CFA countries.
2.20 The relatively poor performance of projects in East and West Africa (dther than CFA
countries) is also reflected in the occurrence of project failures. Out of the 80 projects that had
-71 It should be noted that low values of are common with cross-section datasets with a
large number of observations; to this extent, the low explanatory power reported by Pohl and
Mihaljek is not a surprise. Further, Pohl and Mihaljek reported only adjusted R2values, and
not the conventional R2values. The adjusted R2 reduces the conventional R2, based on the
number of explanatory variables included in the regression equation, so that the conventional
value is greater than or equal to the adjusted R2values. While the adjusted k2 is useful for
comparing the relativeexplanatorypower of differentregression equations with different number
of explanatory variables, only the conventional R2 can be properly interpreted as the actual
explanatory power of a particular regression equation.
-81 Pohl and Mihaljek indicate that this result may be misleading because there is a possibility
that projects with large real cost overruns reflect mostly expansion of projects, rather than errors
in cost estimates.
negative re-estimated rates of return, 27 were in East Africa. In particular, agricultural projects
in sub-Saharan Africa experienced a high failure rate, so that half of agricultural projectsin East
Africa and more than a quarter of such projects in West Africa had re-estimated rates of return
below 5%, with a significant difference between CFA and non-CFA countries.
2.21 Other ex~lanatorvvariables: The regression analysis indicates that an unexpected
increase in primarv commodity urices tends to increase the re-estimated return, which is
consistent with the fact that many of the agricultural projects involve production of primary
commodities. Further, the regression analysis indicates that better economic management of a
country 91 tended to raise the re-estimated rates, which led Pohl and Mihaljek to conclude that
the adverse effects on project performance of government interventions through price controls,
high tariffs, import restrictions, etc. has been underestimated in World Bank project appraisals.
B. Implications for GEF
2.22 It is clear that in recent years World Bank appraisals of development projects have been
over-optimistic. Thus, if a conventional World Bank appraisal of a proposed GEF project
classifies that the project as economically viable, then there is a clear potential that the project
may not actually be economically viable, i.e., a project that appears to be a Type I project may
actually be a Type I1 project. In particular, over-optimisticassessments are likely unless project
appraisals that have explicitly considered and taken account of:
(i) downside risks;
(ii) the host country's commitment to the project;
(iii) the host country's macroeconomic conditions, policies, and economic
management;
(iv) the capacity of host country institutions to effectively implement the project; and
(v) the success rate of completed projects in the sector, country, and region
Thus, until the Bank issues guidelines about the manner in which these factors should be taken
into account, and until the guidelines are actually incorporated into project appraisals, GEF may
find it prudent to question the validity of the results of project appraisals that ignore these
factors.
2.23 It is also clear that while, on average, there has been an optimistic bias is World Bank
project appraisals, this bias has not been present in a significant number of cases, and some
project appraisals have actually been conservative in their estimateof the project's rate of return.
Thus, there has been only a limited link between the appraised and likely actual economic
viability of a project. In other words, the appraised economic viability of a GEF project is
21 Measured by indices such as an index of price distortion or the Bank's internal ranking
of the quality of a country's economic policies and management.
- 11 - DRAFT
likely to be a poor indicator of the project's actual economic viability 101. It follows that it
would be imprudent for GEF to use the appraised economic viability of a project as a dominant
criterion in determining whether or not the project should or will be undertaken without GEF
financial support.
-101 This situation may be made clearer by the following example. suipose a set off
measurements is taken by a number of different observers to determine the depth of a river, and
the average depth of the river is calculated to be 22 feet. A later review shbws that (i) the
measuring instruments used by the differentobservers had different biases, somb upwards, and
other downwards, and (ii) measurements taken with reliable instruments show that the average
depth of the river is 16 feet, i.e., an average bias of 6 feet. Suppose that a reading taken with
one of the original measuring instruments shows that the depth of the river at a particular point
is 18 feet, but it is not possible to determine whether this particular original instrument used to
take this reading was biased, or the direction and extent of the bias. In this situation, it would
be hazardous to use the average bias to correct for the possible bias is measurement, e.g., it
would be meaningless to conclude that the correct depth is 18 - 6 = 12 feet, where 6 is the
average bias.
DRAFT
3. COSTS OF ENERGY CONSERVATION PROJECTS
3.1 For potential GEF energy conservation projects, domestic costs can be classified into
three categories:
Government costs. These costs are incurred by the official agencies that
consider, evaluate and possibly provide funds for the proposed project before it
is implemented.
Utility costs. These costs are incurred by the implementation agency that
executes, monitors, and possibly conducts ongoing or ex-post evaluations of the
project. On occasion, costs may also be incurred by trade allies, such
manufacturers of CFLsor retail outlets, and/or non-profitgroups, who participate
in the promotional aspects of energy conservation programs.
End-user costs. These costs are incurred by the end-users, who are the
beneficiaries of the project, but who may also have to bear some costs.
3.2 As discussed below, appraisalsand evaluations of energy conservationprograms in North
America and European have frequently failed to consider the full panoply of costs, even when
a conscious effort has been made to be comprehensive. For example, in his analysis of
European lighting program costs, Mills (1991) uses a concept of "total resource cost," which
includes all costs for the lamps, salaries, consultants, advertising, postage, evaluations, etc.
However, the "total resource cost" does not include Government o r end-consumer costs.
Further, Mills states that "Lighting trade organizations and/or individual manufacturers have
helped European utilitiesto organize and run some programs." Yet, it does not appear that these
costs have been included by Mills in "total resource cost."
A. Consideration of Government Costs
3.3 For GEF projects, there is significant potential for the underestimation of Government
costs since they may be mainly of the opportunity cost variety, and not explicit cash
expenditures. GEF energy-related projects have an element of novelty, which implies that they
may require extensive consideration and evaluation by Government officials. Thus, the
opportunity cost value of the time and effort spent in consideration and evaluation may be
significant. Further, in some instances. the Government may engage domestic or international
consultants to assist in this process.
3.4 In the U. S., similar costs are typically incurred as part of the regulatory process that is
used to approve energy conservation programs. Regulatory costs are incurred not only by the
regulatory commissions or agencies but also by all the parties that participate in the regulatory
process. For example, apart from the regulatory commission and its staff, proposed energy
conservation projects may also be incurred by groups such as the Office of Consumer Advocate
DRAFT
(which represents residential consumers), the mass transit agency (which uses large amounts of
electricity to operate trains), or associations of large commercial buildings (which also use
significant amount of electricity). While accounting practices differ at utilities, it is unlikely that
any utility can include these costs in the aggregation of total costs because the utility is unlikely
to have any means of collecting such information, even in the remote event that it had been
compiled by the individual agencies.
3.5 The cost of evaluating the novel elements of GEF energy projects may pkove to be high
for countries that have limited expertise in this regard. For example, a recent review 111
of the energy sector in Tonga concluded that the Government adopt:
a policy that limits the energy options to robust technologies that have been
proven operationally and economically in environments similar to that of Tonga.
One of the principal reasons for this recommendation is the shortage of managerid and technical
skills in Tonga. Thus, the opportunity costs of evaluating potential GEF projects may vary from
country to country.
I
B. Consideration of Utility Costs
I
3.6 Typically, in addition to direct expenditure costs, such as on purchasingCFLs or other
equipment, the utility will also incur indirect costs that may be in the fdrm of explicit
expenditures or opportunity costs 121. The explicit indirect expenditures may be for
consultants or experts used to assist the utility in implementing and/or monitoring the project,
or for promotional measures designed to raise end-user awareness of the project. The
opportunity costs may consist of the value of the time of utility personnel as wkll as overhead
costs associated with the project. I
3.7 Following Hirst and Sabo (1992), utility costs include:
I
AdministrativeCosts, which account for the staff involved in probram planning,
design, marketing, implementation, and evaluation, including liabor expenses,
office supplies, data processing, and such other costs; ~
- Tonga:lssue~andOprionsintheEnergySector,PacificIslandsSeriesho.1,Vol.10,
111
World Bank, August 1992.
-
121 Beny (1989) quantified the administrative costs of program plannihg, evaluation,
marketing, auditing, quality control, data collection, and related activities. Admlinistrativecosts
were about 20% of total costs of residential programs, and about 10-15% for commercial
programs.
- 14 - DRAFT
Marketing Costs, which include all costs directly associated with preparation and
implementation of marketing activities, such as direct mailings, bill stuffers,
media advertising, training sessions, etc.;
Monitoring and evaluation costs, which are incurred for data collection and
analysis to assess the performance of the program;
Equipment Costs, which cover the cost of the equipment purchased directly by
the utility;
Incentives Costs, which cover the costs of the incentives provided by utilities to
participate in energy conservation programs.
3.8 While direct expenditure costs are easy to measure and difficult to hide, they may be
underestimated because of unfamiliarity with local working conditions, particularly with respect
to the projected scheduling of the project. For example, the estimated utility costs may be based
on a schedule that does not account of the delays that frequently occur in that particular country
or region. Or even the promotionalcosts of the project may be partially absorbed in the general
advertising expenses of the utility. Thus, the actual direct utility expenditures may be
significantly underestimated in the project's cost-benefit analysis.
3.9 There is a clear potential for not including or underestimating the indirect expenditures.
Based on a sample of ten utilities in the U.S.,Joskow and Marron (1992) concluded that
... many typesof administrativecosts, includingmeasurementand evaluationof
conservation savings and overhead, are not universallytracked and reported. The
failure to account for such costs can lead to significant underestimatesof the true
costs of utility-sponsored conservation initiatives.
C. Consideration of End-user Costs
3.10 Even though the end-users are usually the beneficiaries of utility conservation programs,
there may be some costs that they have to bear. In some instance, the end-users may have to
bear part of the cost of acquiring new hardware, e.g., households may have to pay part of the
cost of new CFLs. Further, energy efficiency is usually embedded in expensive and long-lived
assets, and end-users may be reluctant to throw away or dispose of inefficient equipment that
still works. In any case, there are some opportunity costs associated with "premature"
retirement of hardware, e.g., the replaced conventional bulbs may have some residual value that
may be lost when CFLs are installed.
3.11 End-users may also incur significant time and effort costs in considering and evaluating
whether or not participate in utility-sponsoredprograms. Joskow and Marron (1992)categorize
these as "customer transaction costs" and report that "None of the programs attempt to measure
customer transaction costs. Yet customer transaction costs are very real economic costs that
DRAFT
should in principle be accounted for in evaluating the societal cost of utility conservation
programs."
3.12 Hamlin (1990) described the cost incurred by end-users in this way:
Consumers must spend time and effort searching for the particular options that
are best for them; then invest in new technology, equipment, or process that will
reduce their energy service costs over time. This requires the consumer to invest
precious time and scarce capital up front and to accept some risk to reap the
benefits of energy efficiency in the form of future operationalsavings. Ahecdotal
observations of customer behavior and survey results shows that the majority of
customers are not aware, willing, nor able to make the necessary investinentsin
time and capital, nor to take the risks. (Emphasis in original.)
I
From an end-user's perspective, savings in energy costs are just one of the fadtors considered
in making decisions on the use of capital and other scarce resources; the end-user also considers
product quality, space and cost requirements for equipment, labor costs, etc. ~
3.13 According to Sioshansi (1991) 131 market research results indiqate that most
consumers face high transaction costs in obtaining timely, credible and relevht information
when purchasing major energy appliances or making decisions energy consedation decisions
-
141. For example, Gruber and Brand (1991) report that 52% of the small and medium-sized
W. German firms they surveyed did not consider subsidies a decisive factor in undertaking
energy conservation programs because ". ..subsidy programmes are often poorl$ adapted to the
problems of small and medium-sized firms. The stafl do not have much ti e to read big
brochures or to811in complicated application forms. (Emphasis added.)" ?'
I
e I
3.14 It appears that residential consumers may feel that they have to incur so e trouble even
when they do not have to make any decisions about equipment purchase or bear any equipment
costs. For example, in the Hood River energy conservation project, even th 0ugh the home
~
I
-
131 Sioshansi's observations are particularly relevant because the author is ehployed a large
U.S. electric utility, Southern California Edison Company. I
I
,
I
- AccordingtoNadel'etd(1993),severalU.S.programshaverecogni?edthelackof
141
information as a major barrier to the adoption of energy-efficient lighting, and taken steps to
disseminate objective information that -consumers can use to evaluate efficient lights. Several
lighting technology demonstration centers are also now open to the public and dehign community
in major U.S. cities. The U.S. EPA has a "Green Lights" program which i$ a high-profile
project designed to promote lighting retrofits in the facilities of the top U.S. cor$orations. This
program providespublicity materials,decision-making tools, technical information, manufacturer
and contract information, information on utility rebates, and publicity for partir$pants.
- 16- DRAFT
energy audit was free and the entire cost was borne by the Bonneville Power Authority, only
85% of eligible homeowners participated.
D. Implications for GEF
3.15 It is clear that in North America and Europe the costs of energy conservation projects
have been underestimated, and that this finding applies to all three categories of costs:
government, utility, and end-user. Hence, unless extensive care has been taken in the economic
appraisal of potential GEF projects, it is natural to expect that the projects costs have been
underestimated.
3.16 Further, it may bedifficulttoeliminate thepotential underestimationof Governmentcosts
associated with GEF projects because the accounting system used by Governmentsmay not be
oriented towards establishing costs incurred in the evaluation of individual projects. In the
absence of substantive data from developing countries, it may be difficultto develop even a
priori rules of thumb to take account of these costs. In any case, there may be significant
variations in the abilities of the host countries to evaluate potential GEF projects.
3.17 The novelty elements associated with potential GEF projects may introduce a factor that
makes it difficult for some countries to undertake some seemingly Type I projects without GEF
support. As noted by the Wapenhans report, the limitation of expertise in host countries poses
risks for development projects in general (see para. 2.5). Thus, the novelty of the projects
implies that there is an element of risk associated with the project, i.e., the project may deliver
less than the projected benefits and may even provide no benefits at all, or there may be
significantcost overruns, or some unexpected snags that reduce the justification for the project.
Thus, the Government may be reluctant to undertake the project without GEF support because
of this element of risk. The nature of this risk, and its implicationsfor Governments and GEF
are discussed in Section 5 of this paper.
3.18 So far as utility costs are concerned, in theory, it should be relatively easy to ensure that
all relevant utility costs are taken into account, e.g., the handbook developed by Hirst and Sabo
(1992) may provide a framework for developing an appropriate accounting system. In the
interim, it may be necessary to develop some rules of thumb to gauge the validity of the costs
reported in potential GEF projects. However, it may be difficult to eliminate tendencies to make
excessively optimisticassumptions about the effort and time required to implement the project.
The nature of the downside risks presented by such tendencies is discussed in Section 5.
3.19 Similarly, in principle, it should be relatively easy to develop estimates of end-user
expenditures on hardware as well as the residual value of equipment that is prematurely
scrapped. It may be more difficultto develop estimatesof the end-user transaction costs because
these are likely to vary significantly according to the particular circumstances of the project.
Nevertheless, efforts should be made to include some estimates of end-user costs in the
aggregate project costs.
4. BENEFITS OF ENERGY CONSERVATION PROJECTS
4.1 The benefits of energy-related projects are usually projected to flow to both the utility
and its customers. The overestimation of benefits may affect either the utility or its customers
or both. Based on the experience in North America, it is a common theme that energy savings,
which are the benefits of the energy conservation programs, have been frequently overestimated
in the ex-ante appraisals of these programs. For example, Nadel and Keating (1991) examined
32 U.S. utility energy conservation programs, and found that savings were overestimated in 27
programs and under-estimated in the other five. For 15 of the 27 overestimated programs, the
actual savings were less than 50% of the projected savings. In particular, eightof 11 residential
programs saved less than half as much energy as predicted. Similarly, Keating 151estimated
that about 15-20% of energy-saving bulbs handed out by U.S. utilities were not being used.
4.2 The experience of a number of energy-related projects in the developing countries also
underscores the tendency to overestimate the impact of such projects. In particular, this
tendency is likely to arise when projects are sponsored or promoted by "enthusiasts" or
"proponents" of particular points of view or technologies or by entrepreneurslfirmswho also
stand to profit from hardware sales associated with the project 161.
4.3 VerificationThe"actual"savingsfromanenergyconservationprojectare
often to difficult to calculate precisely because the actual savings are equal to an actual post-
installation consumption subtracted from a hypothetical baseline consumption that would have
taken place had the program not been in place, and all other factors had held constant. For
example, changes in causal factorssuch as weather, income, work habits,or lifestyleschanges171
bring about changes in energy consumption that are difficult to differentiate from the impact of
energy conservation programs.
Reasons for differences between ex-ante and ex-post energy savings
4.4 There are a number of factors that lead to ex-post energy savings which are less than the
ex-ante anticipated savings. These factors can be classified as:
L5/ Wall Srreer Journal, May 27, 1993, page B9.
-
16/ See Overview: PucrjicRegionalEnergy Assessmenr, Volume 1, Pacific IslandsSeries No.
1, World Bank, 1992.
-
17/ For example, the spread of take-outldeliveredfood and the i-ncreasinguse of microwave
ovens in the U.S. has significantly reduced the energy used by 'households for preparing food
at home.
DRAFT
expected value of $ 65 271. Is this the correct measure of the benefits? An alternative is the
"option price," which is the maximum sure payment that the farmer would be willing to make
in both states. Option price depends upon the individual, and may be more or less than the
expected value of benefits.
5.16 From his theoretical analysis, Graham concluded: (i) Option price is the appropriate
measure of benefit in situations involving similar individualsand collective risk (a dam would
be a case of collective risk; (ii) expected value calculations are appropriate in situations
involving similar individuals and individual risks. These concepts were later extended to the
case of uncertain costs by Freeman (1989). However, these concepts have not been applied to
the case of project appraisals in developing countries.
Pure risk and the capital asset pricing model
5.17 The concept of pure risk has been widely used in the capital asset pricing model
(CAPM), which provides some very useful results on how rational, risk-averse individuals and
markets evaluate risk. One of the key relevant results of the CAPM is that, under the right
circumstances, it is not particularly useful to evaluate the risk of an asset on its own; instead,
it is better to consider the characteristicsof a particularasset in thecontext of the entire portfolio
of assets being held by an individual. For instance, assets whose values tend to move in
opposite directions -- whose returns are negatively correlated -- tend to reduce the overall risk
in the portfolio. For this reason, the selection of a varied portfolio of assets makes it possible
to diversify away the risks associated with particular assets.
5.18 In the CAPM',the risk of an asset has two components: (i) systematic risk, which is
represented by B (Beta), and (ii) unsystematic risk. The parameter Bi measures the riskiness of
a particular asset i relative to the risk in the entire market portfolio 281. If an asset has a Beta
value equal to one (Bi = I), then it is just as risky as the market as a whole; when an asset's
Beta is greater (less) than one, the asset is more (less) risky than the market as whole.
5.19 A fundamental result of the CAPM is that in an efficient market all assets will have the
same rate of return after adjusting for risk, which is stated as
281
- Mathematically, Bi is defined as
where Ri and R,,, the returns of asset i and the market portfolio. Thus, Bi is the covariance
are
of the return on the asset with market return divided by the variance of the market return.
return on a particular asset = risk-free return + risk adjustment
The nature of the risk adjustment is such that assets whose Beta is greater (less) than one will
have higher (lower) rates of return, i.e., the risk adjustment depends on l3, but not on
unsystematic risk 291.
5.20 In contrast to systematic risk, unsystematic risk is the purely random variation in the rate
of return of an investment about its expected value, and is due to the peculiarities of the asset.
As shown by the equation above, a key result of the CAPM is that the portfolio risk in efficient
portfolios is determined by systematic risk, and not by the unsystematic risk of an asset; in
particular, a high unsystematic risk will not lead to a requirement a higher rate of return 3 1 .
5.21 These results of the CAPM are well-known and widely accepted. Yet, it is not common
practice to consider the riskiness of investing in energy conservation investments. Oneexception
to this is the analysis provided by Sutherland (1991).
I
5.22 Sutherland (1991) argues that investing in energy efficiency is risky ih the sense that
actual savings tend to vary significantly from predicted savings 311. Since investors are
risk-averse, investments in energy efficiency are less than what they would be ih a more certain
world. However, based on the CAPM results, the relevant risk is not the raqdom risk of an
individual asset but the risk of the investor's overall portfolio. Sutherland claids that the major
risk of many energy-efficient investments is the random unsystematic ri4k; hence such
investments are probably not risky in the sense of having high l3 values. It follows that the
required rate of return on energy-efficient investments should be compdble to that of
investments in general. Thus, Sutherland concludes, that in general the vie$ that risk is a
market barrier that discourages energy-efficient is an ad hoc notion, which is not firmly
grounded in financial theory. I
I
-
291 Mathematically, in an efficient market, in equilibrium, the return on an:asset is:
,
where Rf is the risk-free return, and fl,(R,,,- Rf) is the adjustment for risk.
I
301 These results of the CAPM are based on some assumptions that are dsually valid for
stock markets in industrialized countries: liquidity of investments, marketability1,and the ability
to reduce risk by holding a diversified portfolio.
I
I
311 According to Sutherland,a study of commercial building retrofits in the U.S. concluded
that very few predictions of energy savings came within 20% of measured results.
Residential sector and small businesses
5.23 Sutherland finds that the CAPM results may not be applicable to the analysis of energy-
efficiency investments undertaken by the residential sector or small, privately held businesses
because the key CAPM assumptions -- liquidity, marketability, and the ability to reduce risk by
holding a diversified portfolio -- may not be valid for such investments. In particular, these
investments tend to be in tangible, illiquid assets, with limited marketability.
5.24 Hieh initial costs: Further, a household may find the risk of the investment (in say,
-
CFLs for low income households, or shell measures for other household) to be significant
relative to the household's total income, and the household may not have a sufficiently
diversified portfolio to diversify away this risk. Low-income households may have a zero, or
even negative, propensity to save, and may therefore be averse to investment assets. Such
households are particularly limited in terms of reducing risk through diversified portfolios.
In other words, residential consumers and small, privately held businesses may require higher
rates of return on energy-efficient investmentsbecause such investments are illiquid, not readily
marketable, and their risk is not easily diversified away.
Is risk an additional cost?
5.25 Whilerisk is a potentiallyserious obstacle to theadoptionof energy-efficient technologies
and devices, it appears inappropriate to consider risk an additional cost element that is
overlooked in cost-benefit analysis. Instead, it appears appropriate to (i) explicitlyconsider both
pure and downside risk in conducting the cost-benefitanalysis, and (ii) develop decision-making
rules that consider not just the appraised domestic benefits but also the risks. For example, the
methodology put forward by Crousillat and Merrill (1992) explicitly considers and emphasizes
the downside risk in undertaking-major power sector investments.
B. Government response to uncertainty
5.26 In view of the biases and uncertainty associated with the conventional ex-ante appraisal
of net domestic benefits, governments may be reluctant to rely solely on such appraisals to make
decisions about undertaking investment projects. It is likely that different Governments would
have different responses to the uncertainty associated with GEF projects. Nevertheless, it is
possible to consider some of the decision-making rules that Governments may use.
5.27 It is likely that a decision-maker will view the projected net domestic benefits of an
energy conservation project with skepticism. One response of the decision-maker may be to
conduct a heuristic, back-of-the-envelope analysis to determine "low-case" or "worst-case"
outcomes. An extremely risk-averse decision-maker may wish to be sure that the worst-case
scenario associated with the project is acceptable. Or, consistent with above discussion of
downside risk, a risk-averse decision-maker may approve only those projects whose "low-case"
estimate of the net domestic benefits exceeds a particular value, irrespective of the potential
benefits associated with base or high case estimates of the net domestic benefits.
5.28 Even if no analysis is conducted to determine "low case" outcomes, a Government may
be willing to undertake only those energy conservation projects that require low initial capital
expenditures, so that the project can be terminated at relatively low cost if there are early
indications that the project will fail to achieve its projects benefits. In other words, a
Government may not be willing to stake an initial large sum of money on uncertain Wects, but
may be willing to undertake projects with similar total costs but whose costs are spread over
time.
5.29 It is also possible that none of the formal models may describe a Government's decision-
making process, which may be based on the experience and "seat-of-the-pants"judgement of the
decision-makers.
C. GEF's response to uncertainty
5.30 Apart from the uncertaintyassociated with projected net domestic benefits, qEF also has
to consider the uncertainties associated the decision-making rules used by Governhents. It is
clear that even if GEF classifies a project as Type I, the global environmental benefitsassociated
with it will not be realized unless the Governmentactually implements the project in^ the absence
of GEF support.
5.31 In principle, GEF could take account of the Government's decision-rule in formulating
GEF's own decision-making rule about whether or not to support a particular projept. In other
words, GEF may consider it prudent to consider supporting projects that a Governn)ent will not
undertake on its own. However, any such recognition presents a moral hazard because it
provides an incentives for Governments to adopt a stated policy of "We will not un dertake these
types of projects on our own" merely in order to secure GEF support. Thus, GEF1willrun the
risk of "false positive" results, i.e., of supporting projects that do not need GEF aupport.
5.32 If GEF continues to use the appraised net domestic benefits as a decision-making rule,
i.e., continues to classify projects as Type I and Type I1 based on the appraised Oet domestic
benefits, then GEF faces two problems. First, until the project appraisal methodology is
modified to take account of the failings, biases and risks indicated by the Wapenharis report and
the experience with energy conservation projects in industrialized countries, the alppraised net
benefitsare seriously flawed. Consequently,any decision rule based on these estimdtes may also
be seriously flawed, and GEF would run the risk of both "false positive" and "false negative"
results, where "false negative" represents failing to support projects that require G/EF support.
I
5.33 Second, even if the methodology is modified to eliminate or reduce the abode problems,
a decision rule based solely on the (correctly) projected benefits will fail to take adcount of the
potentially different responses of different Governments to similar projects, i.e., some
Governments may be more risk-averse than others. Therefore, GEF may face significant risks
of getting "false negative" results.
5.34 It is not possible for GEF to develop a decision-making rule that minimizes both "false
positive" and "false negative" results 321 In deciding whether to focus on "false negative"
or "false positive" results, GEF will have to take account of the consequences of these types of
results. If substantial global environmental benefits are at stake, then GEF may be worried
about denying a small amount of support to project and taking the risk that the project may never
be undertaken, i.e., the focus would be on minimizing "false negative" results. Alternatively,
if substantial GEF funding is required, then GEF may be womed about "false positive" results.
5.35 The implication is that GEF has to develop its own risk profile, and develop decision-
making rules that take account of the amount of GEF funding at stake and the estimated global
environmental benefits.
321 In practical situations, decision makers often have to declare a preference for minimizing
one or the other probability. For example, in the judicial system, the burden of the proof is on
the prosecution, and the defendant has to be proven guilty beyond a reasonable doubt. In other
words, there is an emphasis on reducing "false guilty" verdicts even though this may lead to
frequent "false not-guilty" verdicts. Or, a medical diagnostic test for a disease such as cancer
may be set up so that it minimizes the probability of "false negative" results (test says an
individual does not have the disease even though it is present), but permits the probability of
"false positive" results to stay high, perhaps so that further diagnostic test can be run to verify
whether the individual does have the disease.
6. SUMMARY AND CONCLUSIONS
6.1 The Global Environmental Facility (GEF) is concerned that Type I projects 3 1 may
not be implemented by host countries even though they appear to offer them positive net
domestic benefits, so that the global environmental benefits associated with the projects may not
be realized. Therefore, it is appropriate for GEF to consider the steps to be taken to ensure that
such projects are actually implemented.
6.2 There are two broad problems with the projected the a-ante calculated net domestic
benefits associated with a project, which lead to over-optimisticassessments of projects relevant
for GEF: (i) flaws in the application of cost-benefit analysis to development projects in general;
and (ii) flawsspecificto energy conservation projects. Once these flaws are taken into account,
it may turn out that a project that ayears to provide net domestic benefits actually does not do
so. The failure to actually provide net domestic benefits would then make a project a potential
candidate for GEF support.
A. Flaws in the application of cost-benefit analysis
6.3 Recently, there have been concerns that the application of cost-benefit anlalysis by the
World Bank has led to over-optimisticassessments of development projects. The world Bank's
Wapenhans report co~lcludedthat the appraisal of development projects supported by the World
Bank has been over-optimisticbecause they have not taken full account of the: (i) changes in the
globallevel economicconditions, (ii) the host country's macroeconomicconditionsand policies,
changes in developmental priorities, deficientregulatoryenvironments,and thelacklof or decline
in capacities of local institutions, and (ii) the increasing complexity of projects, which makes it
difficult to implement them effectively, along with a lack of commitment on thd part of host
countries to the projects.
6.4 Based on a statistical analysis of 1,015 World Bank projects, Pohl and Mihaljek (1992)
reached three main conclusions: (i) there is an upward bias in appraisal rates of rewm; (ii) there
is little link between appraisal and likely actual rates of return; and (iii) factors such as cost
overruns and completion delays explain only a very small part of the differdnce between
appraisal and likely actual rates. One result of the excessive optimism in the app+.isalrates of
return is that there are a large number of projects whose likely actual rates of retdrn are below
conventionally acceptable rates of return.
6.5 In response to these over-optimisticappraisals, the Bank has decided to emphasize, inter
alia, (i) host country commitment to projects, and (ii) explicit and systematic recognitionof the
I Type I projects offer significant global environmental benefits, and also positive net
domestic benefits when evaluated in the standard economic cost-benefit framework.
DRAFT
risksassociated with developmentprojects. The Bank plans to issue soon new guidelines on risk
and sensitivity analysis for development projects.
6.6 Since recent World Bank appraisals of development projects have been over-optimistic,
it follows that when a conventional World Bank appraisal classifies a proposed GEF project as
economically viable, there is a clear potential that the project may not actually be economically
viable, i.e., a project that appears to be a Type I project may actually be a Type I1project. In
particular, over-optimistic assessments are likely unless project appraisals have explicitly
considered and taken account of: (i) downside risks; (ii) the host country's commitment to the
project; (iii) the host country's macroeconomicconditions, policies, and economic management;
(iv) the capacity of host country institutions to effectively implement the project; and (v) the
success rate of completed projects in the sector, country, and region where the projectis located.
6.7 Thus, until the Bank issues guidelines about the manner in which these factors should be
taken intoaccount, and until the guidelinesareactually incorporated intoprojectappraisals, GEF
may find it prudent to question the validity of the results of project appraisals that ignore these
factors. In other words, since the appraised economic viability of a GEF project is likely to be
a poor indicatorof the project's actual economic viability, it would be imprudent for GEF to use
the appraised economic viability of a project as a dominant criterion in determining whether or
not the project should or will be undertaken without GEF financial support.
B. Flaws in estimatingcosts of energy conservation projects
6.8 For potential GEF energy conservation projects, domestic costs can be classified into
three categories: (i) Governmentcosts, which are incurred by the official agencies that consider,
evaluate and possibly provide funds for the proposed project; (ii) Utility costs, which are
incurred by the implementation agency (and associated trade allies or non-profit groups) that
executes, monitors, and possibly conductsongoing or ex-post evaluationsof the project; and (iii)
End-user costs, which are incurred by the end-users, who are the beneficiaries of the project,
but who may also have to bear some costs.
6.9 Appraisals and evaluations of energy conservation programs in North America and
European have frequently failed to consider the full panoply of costs, even when a conscious
effort has been made to be comprehensive. Based on this experience, there is likely to be
significant underestimation of all three categories of costs inn potential GEF projects.
6.10 Governmentcosts may be underestimatedsincethey are frequently of theopportunitycost
variety, based on the staff time and effort involved. In particular, the cost of evaluating the
novel elements of GEF energy projects may prove to be high for countries that have limited
expertise in this regard.
6.11 Typically, in addition to direct expenditure costs, such as on purchasing CFLs or other
equipment, the utility will also incur indirect costs that may be in the form of explicit
expenditures or opportunity costs. Based on a sample of ten utilities in the U.S.,Joskow and
DRAFT
Marron (1992) concluded that many types of administrative corn, including measurement and
evaluation of conservation savingsand overhead, are not universally tracked and reported. The
failureto account for such costs can lead to significant underestimatesof the truecosts of utility-
sponsored conservation initiatives.
6.12 Even though the end-users are usually the beneficiaries of utility conservation programs,
they may have to bear transaction costs involved in considering and evaluating whether or not
and to what extent to participate in conservation programs. In some instance, they may also
have to bear part of the cost of acquiring new hardware or the opportunitycost of "premature"
retirement of hardware, e.g., the replaced conventionalbulbs may have some residual value that
may be lost when CFLs are installed. Small and medium-sized firms may also experience
similar difficulties.
6.13 In principle, it should be relatively straightforward to verify that all three categories of
costs have been taken into account in the cost-benefit analysis of GEF projects. HQwever,in
practice, there may be difficulties in amving at reasonable estimates of costs, and dome costs
may be overlooked completely. Hence, actual social costs of energy conservation piojects are
likely to be under-estimated, unless special care has been taken to ensure that this is not the
case.
C. Flaws in estimating benefits of energy conservation projects
6.14 Based on the experience in North America, it is a common theme that energy savings,
which are the benefits of the energy conservation programs, have been frequently and
significantlyoverestimated in theex-anteappraisalsof these programs. In any case, the "actual"
savings from an energy conservation project are often to difficultto calculateprecisely because
they are equal to an actual post-installationconsumption subtracted from a hypothetical baseline
consumption that would have occurred had the program not been in place, and all other factors
had held constant.
6.15 There are a number of factors that lead to ex-postenergy savings which are leSs than the
ex-ante anticipated savings: (i) improper definition of program impact; (ii) lower thah expected
participation rates; (iii) "Free riders" and "takeback"effects;and (iv) equipment failuie, misuse
and lack of persistence of effects.
6.16 The most meaningful measure of the effect of a utility sponsored program is the net
program impact which considers the actions of the participants in the program with respect to
what would have happened if the utility-sponsored program had not come into existence.
However, it is possible that the economic appraisal conducted for the energy conservation
programs being considered by GEF is based on one of the other definitions of the impact, such
as the maximum technical potential, which measures the impact of a 100% penetration of the
most efficient technologies.
6.17 The participation rate, i.e., the ratio of eligible customers who actually participate in a
utility sponsored program, is one of the key determinants of the benefits from energy
conservation programs. The actual participation rate may be lower than the projected
participation rate for a number of reasons: (i) inappropriatediscount rate; (ii) transaction costs,
other priorities, and organizational problems; and (iii) ineffective promotional campaigns.
6.18 It is common to use a real discount rate of about 10% in cost-benefit analysis of
development projects. However, the available evidence indicates that consumers, particularly
low-income consumers, use much higher discount rates in evaluating energy conservation
programs. If a high real discount rate is used for end-use consumers and a lower real discount
rate is used for the utility, a particular project may turn out to be beneficialfrom the perspective
of the utility but not from the perspective of the consumers. Thus, the use of an inappropriately
low discount rate may lead to optimistic estimates of the participation rate. In response to the
use of high discount rates and consequent low participation rates, the U.S. has set minimum
efficiency standards for a number of appliances, which forces consumers to purchase only
relatively energy-efficientappliances.
6.19 The decision to participatein utility sponsored programs also depend upon other factors
that such as the transactions costs, the end-users' other priorities, and their organizational
structure. Further, the promotional campaign instituted by the utility may prove to be
ineffective, particularly in situations where the utility.lacks experience in such promotions.
These factors may be overlooked in projecting the participation rate, thus leading to optimistic
estimates.
6.20 In thecontextof utility-sponsoredprojects, "free riders" are participants who would have
undertaken the proposed measures even without a utility-sponsored program. For example,
customers who would have bought and installed CFLson their own but take advantageof utility
incentives to buy them are free riders. If no account is taken of free riders, then the actions of
the free riders are ascribed to the utility program. While the presence of free riders has been
a concern in the U.S., it has not been so in European lighting programs. Given the novelty of
the technology and devices being promoted by GEF energy conservation programs, free riders
are unlikely to be a major concern for GEF projects. In contrast, GEF programs may induce
"free drivers," who are customers who do not participate in a utility sponsor program but are
influenced by the program and adopt the program recommendations. Thus, free drivers
represent an additional benefit that may not be accounted for in the ex-ante appraisal of DSM
programs.
6.21 In the context of energy conservation programs, "takeback" refers to the change in
energy-related operating practices of a firm or households as a consequence of participating in
a DSM program. For example, a household may increase its use of an airconditioner after a
utility helps pay for a new, more efficient unit, so that actual energy savings may be less than
anticipated. While substantial takeback effectshave not been observed in the U.S.and Europe,
this may be the case in developing countries, particularly where low-income consumers are
involved.
DRAFT
6.22 The history of energy projects in the developing countries, particularly those 011 the
supply side such as generation, transmission, and distribution facilities, indicates that there is
a potential for significant problems in the installation, proper use, and maintenance of energy
efficient technologiesand devices, particularlyfor those with which the local people have limited
familiarity and experience. Thus, even if there are initial savings from energy conservation
projects, these have the potential of declining steadily over time. It would be prudent for GEF
to verify whether the proposed project has taken account of this potential decline in benefitsover
time, or taken steps to prevent such a decline.
D. Uncertainty, risk, and decision rules
6.23 The historical experienceof World Bank projectsas well as energy conservation projects
in the U.S.and Europe shows that, apart from any possible systematic bias, there remain
substantial differencesbetween actual and realized rates of return. Thus, even after the biases
in ex-ante estimates of costs and benefits of energy conservation projects have been duced or
eliminated, there is likely to remain substantial uncertainty about the actual net benefits of
energy conservation projects that GEF may support. For GEF energy-related projects an
additional reason to expect divergence between estimated and actual net benefits is that GEF
projects have an element of novelty, which implies that there is limited experience oh which to
base theapproximations,assumptionsand rules-of-thumb usually required in projectcost-benefit
analysis.
6.24 One relevant implication of the uncertainty associated with ex-ante appraisals is that any
decision-maker, i.e., the Governmentor GEF, who uses theex-ante appraisal to make a decision
about an investment project faces with significant probabilitiesof getting "false-positive" and/or
"false-negative" results. For example, the Government may end-up undertaking projects that
do not deliver the expected results ("false-positive") and/or it may fail to undertake projects that
would have brought substantial benefits to the country ("false-negative").
6.25 While there is an extensiveliteratureon risk and uncertainty, even well-known theoretical
concepts have not been extensively incorporated into project appraisals. At the sanle time, it
is also clear that many of the theoretical results available in the literature are not sufficiently
practical to be readily applied in the appraisal of projects in developing countries; for example,
much of the theoretical discussion relates to situations in which the uncertain outcofne can be
treated as a random variable with a probability distribution. For example, the distinction
between "pure risk," under which there is a possibility of unexpected adverse as well favorable
events, and "downside risk," which focuses on unexpected adverse events only has only recently
been formally explored, and no definitive theoretical or practical results are available yet.
6.26 In contrast, the concept of pure risk is well-developedand widely-accepted, but it has not
commonly been used in project appraisals, in part because it is difficult to provide reasonable
subjective estimates of the probabilities that are required by this approach. Nevertheless, some
of the theoretical results of the capital asset pricing model (CAPM), based on an analysis of pure
risk, are relevant for project appraisal. In particular, in the context of efficient markets, the
- 42 - DRAFT
CAPM shows that it is not meaningful to consider the risk of an asset on its own; instead,
rational individuals evaluate the risk of an asset in the context of an entire portfolioof assets.
The natural implication is that it would be useful to evaluate the riskiness of a particular project
in the context of the entire portfolio of projects being undertaken by a country.
Government response to uncertainty
6.27 In view of the biases and uncertainty associated with the conventional a-ante appraisal
of net domestic benefits, governments may be reluctant to rely solely on such appraisals to make
decisionsabout undertaking energy-efficiency projects. It is likely that different Governments
would have different responses to the uncertainty associated with GEF projects. One response
of the decision-maker may be to conduct a heuristic, back-of-the-envelopeanalysis to determine
"low-case" or "worst-case" outcomes. An extremely risk-averse decision-maker may wish to
be sure that the worst-case scenario associated with the project is acceptable. Or, a risk-averse
decision-maker may approve only those projects whose "low-case" estimate of the net domestic
benefits exceeds a particular value. Even if no analysis is conducted to determine "low-case"
outcomes, a Government may be willing to undertake only those energy conservation projects
that require low initial capital expenditures, so that the project can be terminated at relatively
low cost if there are early indications that the project will fail to achieve its projects benefits.
GEF's response to uncertainty
6.28 Apart from the uncertainty associated with projected net domestic benefits, GEF also has
to consider the uncertainties associated the decision-making rules used by Governments. It is
clear that a host country may not undertake a project even if GEF classifies it as Type I, in
which case the global environmental benefits associated with it will not be realized.
6.29 If GEF takes account of the Government's decision-rule in formulating GEF's own
decision-making rule about whether or not to support a particular project, this would lead to a
moral hazard, because it provides an incentives for Governments to adopt a stated policy of "We
will not undertake these types of projects on our own" merely in order to secure GEF support.
Thus, GEF will run the risk of supporting projects that do not need GEF support.
6.30 On the other hand, if GEF does not take account of the Governments' decision-rules, and
continuesto classify projectsas Type I and Type I1 based on the appraised net domestic benefits,
then GEF faces two problems. First, until the project appraisal methodology is modified to take
account of the failings, biases and risks indicated by the Wapenhans report and the experience
with energy conservation projects in industrialized countries, the appraised net benefits are
seriously flawed, and GEF would run the risk of both "false positive" and "false negative"
results, where "false negative" represents failing to support projects that require GEF support.
Second, even if the methodology is modified to eliminate or reduce the above problems, a
decision rule based solely on the (correctly) projected benefits will fail to take account of the
potentially different responses of different Governments to similar projects, i.e., some
Governments may be more risk-averse than others. Therefore, GEF may face significant risks
of getting "false negative" results.
6.31 Mathematically, it is not possible for GEF to develop a decision-making rule that
simultaneously minimizes the probabilitiesof both "false positive" and "false negative" results.
Therefore, in deciding whether to focus on "false negative" or "false positive" result$,GEF will
have to take account of the consequences of these types of results. If substatid global
environmental benefits are at stake, then GEF may be womed about denying a small amount
of support to project and taking the risk that the project may never be undertaken, i.e., the focus
would be on minimizing "false negative" results. Alternatively, if substantial GEF funding is
required, then GEF may be womed about "false positive" results. The implication is that GEF
has to develop its own risk profile, and develop decision-making rules that take account of the
amount of GEF funding at stake and the estimated global environmental benefits.
- 44 -
REFERENCES
Anderson, Jock R. and John C. Quiggin, Uncertainty in Project Appraisal, Paper prepared for
the World Bank Annual Conference on Development Economics, 1990.
Bates, Robin W., Energy Conservation Policy, Energy Markets and the Environment in
Developing Countries, Environment Working Paper No. 45, World Bank, 1991.
Bartlett, Sarita, "Shedding Light on Residential Consumers," Energy, Vol. 18, No. 2, 1993.
Berry, Linda, The Administrative Costs of Energy Conservation Programs, ORNLICON-294,
Oak Ridge National Laboratory, Oak Ridge, TN, 1989.
Besant-Jones, John E. and Lori Hylan, ed., Managing Risks of Investments in Developing
Countries, Industry and Energy Department energy series paper no. 55, World Bank,
1992.
Caner, Phoebe, "The Drive to Verify Energy Savings," ElectricityJournal, Vol. 5, No. 4, 1992.
Cebon, Peter B., "'Twixt cup and lip: Organizational behavior, technical prediction and
conservation practice," Energy Policy, 1992.
Brown, Ian, "Energy Service Company Investments in Energy-efficientLighting in Hungary,"
Energy, Vol. 18, No. 2, 1993.
Busch, John F., Peter du Pont, Ad Surapong Chirarattananon, "Energy-efficientlightingin Thai
commercial buildings," Energy, Vol. 18, No. 2, 1993.
Crousillat, Enrique O., Incorporating risk and uncertainty in power system planning, Industry
and Energy Department energy series paper no. 17, World Bank, 1989.
Crousillat, Enrique, and Hyde Merrill, The trade- of/risk method :a strategic approach to
power planning, Industry and Energy Department energy series paper no. 54, World
Bank, 1992.
Dubin, Jeffrey A., Market Barriers to Conservation: Are Implicit Discount Rates Too High?,
Social Science Working Paper No. 802, Division of the Humanities and Social Sciences,
California Institute of Technology, July 1992.
Gellings, Clark W., Ahmad Faruqui, and Ken Seiden, "Potential energy savings from efficient
electric technologies," Energy Policy, Vol. 19, No. 3, 1991.
Graham, Daniel A., "Cost-Benefit Analysis under Uncertainty," American Economic Review,
Vol. 71, No. 4, 1981
Graham, Daniel A., "Cost-Benefit Analysis under Uncertainty: Reply," American Economic
Review, Vol. 74, No. 5, 1984
Gruber, Edelgard and Michael Brand, "Promoting energy conservation in small and medium-
sized companies," Energy Policy, Vol. 19, No. 3, 1991.
Hamlin, L. D., "Energy efficiency: the future business opportunity for electric utilities,"
Electricity Journal, Vol. 3, No. 7,1990.
Hartmann, Raymond S. and Michael J. Doane, "Household Discount Rates Revisited," Energy
Journal, Vol. 7, No. 1, 1986.
Hassett, Kevin A. and Gilbert E. Metcalf, "Energy conservation investment: Do consumers
discount the future correctly?" Energy Policy, Vol. 21, No. 6, June 1993.
Hirst, Eric and Carol Sabo, "Defining and reporting data on utility DSM ~ro~rams,~"Energy,
Vol. 17, No. 7, 1992.
Johnson, Karl and Erich Unterwurzacher, "Ensuring market supply and penetration of efficient
lighting technologies," Energy, Vol. 18, No. 2, 1993
Joskow, Paul L. and Donald B. Marron, "What Does a Negawatt Really Cost? ~vidbncefrom
Utility Conservation Programs," Energy Journal, Vol. 13 No. 4 1992.
Joskow, Paul L. and Donald B. Marron, "What Doesa NegawattReally Cost? FurthedThoughts
and Evidence," Electricity Jouml, Vol. 6, No. 6, 1993. I
Krause, F. and Joseph Eto, TheDemandSide: Conceptualand MethodologicalIssues, Least-cost
Utility Planning Handbook for Public Utility Commissioners, Volume 21 National
Association of Regulatory Utility Commissioners (NARUC), Washington, DIC., 1988
Lind, Robert C., Discountingfor time and risk in energy policy, Washington, D.C.: ~esources
for the Future, 1982 I
McInnes, Genevieveand Erich Unterwurzacher, "Electricity end-useefficiency," ~ n ePolicy, ~
r ~
Vol. 19, No. 3, 1991. I
I
Mendelsohn, Robert and William J. Strang, "Cost-Benefit Analysis under Ubcertainty:
Comment," American Economic Review, Vol. 74, No. 5, 1984
I
Mills, Evan, "Evaluation of European lighting programmes," Energy Policy, Vol. 19, No. 3,
1991.
Mills, Evan, "Efficient Lighting Programs in Europe: Cost Effectiveness, Consumer Response,
and Market Dynamics," Energy, Vol. 18, No. 2, 1993.
Nadel, Steven M. and Kenneth M. Keating (1991), "Engineering Estimates versus Impact
Evaluation Results: How Do They Compare and Why," in Energy Program Evaluation:
Uses Methods and Results, Proceedings of the 1991 International energy Program
Evaluation Conference.
Nadel, Steven M., Barbara A. Atkinson and James E. McMahon, "A Review of U.S. and
Canadian Lighting Programs for the Residential, Commercial and Industrial Sectors,"
Energy, Vol. 18, No. 2, 1993.
Ouederni, Bechir N. and William G. Sullivan, "A Semi-varianceModel for IncorporatingRisk
into Capital Investment Analysis," Engineering Economist, Vol. 36, No. 2, 1991.
Pohl, Gerhard and Dubravko Mihaljek, "Project Evaluation and Uncertainty in Practice: A
Statistical Analysis of Rate of Return Divergences of 1,015 World Bank Projects,"
World Bank Economics Review, Vol. 6, No. 2, 1992.
Pouliquen, Louis Y., Risk Analysis in Project Appraisal, World Bank Staff Occasional Paper
Number 11, Johns Hopkins University Press, Baltimore, 1970.
Prindle, William R., "Demand-side management in the 1990s: Time to come of age," Energy
Policy, Vol. 19, No. 3, 1991.
Rave, Klaus, 'A Contract for ~fficientLighting in German Public Buildings," Energy, Vol. 18,
No. 2, 1993.
Reutlinger, Shlomo, Techniquesfor Project Appraisal under Uncertainty, World Bank Staff
Occasional Paper Number 10, Johns Hopkins University Press, Baltimore, 1970.
Schipper,Lee and Dianne V. Hawk, "More efficienthousehold electricity-use: An international
perspective," Energy Policy, Vol. 19, No. 3, 1991.
Simpson, David and James Walker, "Extending cost-benefit analysis for energy investment
choices," Energy Policy, Vol. 15, No. 3, 1987.
Sioshansi, Fereidoon P., "The myths and facts of energy efficiency: Survey of implementation
issues," Energy Policy, Vol. 19, No. 3, 1991.
Sutherland, Ronald J., "Market Barriers to Energy-Efficient Investments,"Energy Journal, Vol
12, No. 3, 1991.
- 47 - DRAFT
Train, Kenneth, "Discount rates in consumers' energy-related decisions: A review of the
literature," Energy, Vol. 10, No. 12,.1985;
Vine, Edward L., "Free Rider Estimation: Refining the Use of Surveys," Energy, Vol. 17,
No. 10, 1992.
Vine, Edward L., "Persistence of Energy Savings: What do we know and how can it be
ensured?" Energy, Vol. 17, No. 11, 1992.
Wapenhans, Willi A. (World Bank Portfolio ~ana~ementTask Force), Efictive
Implementation: Key to Development Impact, World Bank, 1992.
World Bank, EconomicAnalysis of Projects: Towards an Approach to Evaluationfbr the 1990s,
Warld Bank, 1992.
World Bank, Getting Results: The World Bank's Agenda for Improving Development
Efectiveness, Warld Bank, 1993.
DRAFT
Some Draft Notes on Light-bulb programmes
6.32 According to Mills (1991), the residential programmes have had a significant impact on
national lamp sales, but program participation rates show no correlation with the incentivelevel.
The participationrate is the product of a number of factors, including effectivenessof promotion
strategies, type of incentive, and restrictions in some cases on the number of lamps allowed to
each household. The non-residential programmes should be more cost-effective than residential
programmes. This is partly because of the need to contact fewer customers, plus the delivery
of more lamps/customer means lower administrativecosts.
6.33 The average societal costs of conserved energy is C2.l/kWh, including indirect costs of
C0.3IkWh. The payback time to participating households ranged from 0 years (free lamps) to
three years. For the programmesdescribed in this article, administrativeand other "transaction"
Costs contributed CO.3lkWh ($l/lamp) to the total cost of conserved energy.
6.34 According to Mills (1993), between late 1987 and 1992, at least 52 financial-incentive
programs for CFLs were implemented in 11 West European countries, including U.K. and
Ireland. The 7.4 million households eligible for the program received 2.5 million CFLs.
Program target groups ranged from a few thousand households to several hundred thousand.
Data on costs for 40 of these programs from 8 W. European countries (Sweden, Denmark,
France, Netherlands, Finland, Ireland, W. Germany, and Austria). For these programs, the
average cost is 2.1 centslkwh, of which 0.3 cents is indirect cost, where indirect means non-
equipment cost. The average price paid by program participants was $ 111CFL. Non-
participants also benefited because increases in lamp sales prompted manufacturers to lower
prices. For example, in Sweden, the 75,000 rebate checks led to an additional "leveraged"
41,000 sales.
6.35 Mills (1991) found that lower energy costs were cited by only half the participantsas the
reason for participatingin the European CFL programs. Trying a new technology and using the
rebate check were the other frequently cited reasons. Non-participants reported a number of
reasons for not participating: (i) general lack of interest, (ii) excessive lamp prices, (iii) non-
awareness of the program, (iv) lamp sizelweight was excessive.
6.36 Information: Exhibitions, open houses and other low-effort approaches have yielded
minimal impacts compared to programs offering financial incentives. Nonetheless, future
programs should address the problem that consumers often have inaccurate or inadequate
information on CFLs.
6.37 According to Mills (1993), the choice of light sources and the markets for energy-
efficient lighting have changed dramatically in recent decades, and is expected to continue.
Global CFL sales were 114 millionlyearin 1991, and are expected to reach 250 millionlyear by
1995; global incandescent sales per year were over 9 billion in 1991. One reason for the rapid
increase is that an increasing number of parties that are not traditionally involved in promoting
efficientlighting (utilities,government, public interest groups, others) are actively participating.
The key parties have been electric utilities, while lighting retailers in some cases shared in
marketing and providing consumer information. In one case (Sweden), a government body
designed and financed lighting programs carried out by the utilities.
6.38 The major drawback for the accelerated market penetration of CFLs is their high initial
costs, combined with information barriers regarding their cost-effectiveness. In both the
residential and commercial sectors, the lack of information and capital, the reluctance to adopt
unfamiliar technologies, and only moderate interest in energy costs and in reducing expenses
continue to hamper the widespread introduction of energy-efficient technologies.
6.39 According to Brown (1993), a fundamental barrier to achieving cost-effective lighting
efficiency in W. Europe is the lack of investment capital for such technologies. This problem
is magnified in Eastern Europe; this problem is present in Hungary, even though it is in the
fortunate position of having the manufacturing capacity for efficient products such as CFLs.
One possible source of capital is energy service companies (ESCOs), which can provide third-
party financing. These ESCOs should get their funds from western ESCOs which have the
capital as well as the expertise..
6.40 According to Busch et al (1993), energy-efficientlighting would be very useful in Thai
commercial buildings, because electricity use in these buildings is growing rapidly, i.e. number
of buildings is growing rapidly. The authors use a 6% real discount rate and a 20-year time
horizon, though not all components are assumed to have this life. The authors find that for
offices the average electricity price is $0.087, while the CCE (cost of conserved energy) for the
full conservation measure is $ 0.019. The simple payback period for full lighting conservation
measures ranges from less than.one year in hotels and retail buildings to about three years in
offices. The IRR of installingall the lighting measures is 35% for offices, 142%for hotels, and
107% for shopping centers. CFLs, electronic ballasts, and triphosphor narrow-diameter (T8)
lamps prove to be the most economically promising technologies.
6.41 The above figures for Thai commercial buildings are from a societal perspective. The
individual benefits may be less; for example, individual building owners have to pay import
duties on equipment, which are not included as costs in the societal-perspective. Also, the cost
of money may be more than 6%. Nevertheless, even if you use actual market prices, and a
discount rate of 12%, investment in efficientlighting remains cost-effective. Nevertheless, such
efficient systems remain the exception in Thailand. Part of the problem has been a lack of
information about the options, their savings, and the life-cycle costs associated with their use.
Further, in Thailand's very competitive market for commercial space, building owners and
developers are reluctant to consider any measures that will increase initial costs.
6.42 The Government of Schleswig-Hosltein, a state in Germany, undertook to replace all
lamps in public buildings with CFLs. It was estimated that about 600,000 conventional
incandescent lamps could be exchanged. To give the programme publicity, the first bulbs were
replaced by the state Minister of Energy. However, the goal of 600,OO was overambitious.
Under the guidelines of "Phase I" of the contract, about 77 thousand lamps have been installed.
- 50 - DRAFT
6.43 Based on an analysis of the data for several European countries (Belgium, Sweden,
Norway, Denmark), Bartlett (1993) concluded that "estimates of the electricity used for
residential lighting in most countries are subject to large measurement error ... Therefore, the
potential annual electricity savings from the use of CFLs cannot be accurately estimated."
6.44 Further, unlike U.S. residential programs, European programs have not been targeted
at any particular group, which may be a serious flaw of the European programs. In addition,
there may be some loss of energy savings if people install CFLs in low-usage areas; 6% of the
CFLs purchased during NESA's campaign in Denmark were placed in vacation homes perhaps
because the publicity materials did not clearly specify that they should be installed in high-use
ateas.
6.45 Energy consumers do not necessarily volunteer to participate in energy efficiency
improvements that appear (at least on paper) to be cost-effective. A variety of motivational,
information, and risk-avoidance factors are blamed for the divergence between what is
theoretically a sound investmentand what is observed in practice. Over the years, many lessons
have been learned on how to overcomeconsumers' initial apathy to energy efficiency, and lead
them to the desired investment.<Note: These are the benefits of the learning curve. But this
learning curve is not yet available n the LDCs. So, o p e I projects may actually face problems,
with lower ex-post rates of return> Utilitiesenjoy substantial economies of scale in obtaining
and disseminating information to consumers.
6.46 Joskow and Marron (1993) indicate that the results of their 1992 study should not be
interpreted as saying that utility DSM programs are cost-effective. "It would be imprudent to
rush to this judgement without examining further the quality of the cost and energy savings
information reported by utilities. Our analysis suggests that utilities often understate program
costs and overstate program energy savings; as a result reported costs of saved energy are often
too low."
6.47 In the U.S., consumers can choose between similar appliances that differ principally in
energy efficiency. Why does consumer behavior diverge from an economically rational
behavior? This was looked at by LBL (Krause and Eto for NARUC). Factors that avplv when
consumeris about to buy. First, the trade-offs between efficiencyand higher first cost were not
clear for some appliances: an efficient appliance may lack some features that another less
efficient one has. Second, the more efficient appliances may not be available in stores. Third,
the rationalconsumer may lack the information about the costs and benefitsof energy efficiency,
and the transaction costs of obtaining this information may be high. Fourth, the rational
consumer may not have the capital for investment or may not feel financially secure to make
investments with a large payback period. Fifth, the monetary savings are small both in absolute
terms as well as in terms of percentage income. Other factors. Sixth, the consumer's purchase
decisions are heavily influenced by non-cost characteristics, such as noise level, colour, etc.
Seventh, the landlord or contractor may be the buyer, and not the actual user. Conclusion: The
market demand for efficiency investments in new appliances is very weak. When electricity