COORDlNATION Risk and Uncertainty: Selection Criteria for Projects Offering Net Positive Domestic Benefits Subodh C. Mathur June 1994 Early Release Version THE WORLD BANK 1818 H Street, N.W. Washington, D.C. 20433 USA Telephone 202 473 1816 Risk and Uncertainty: Selection Criteria for Projects Offering Net Positive Domestic Benefits Subodh C. Mathur June 1994 DRAB7' Abstract In recent times, the application of standard cost-benefit analysis to development projects has led to appraisals are biased upwards,and are also poor predictors of the actual returns from these projects. Similarly, the North American and European experience with energy conservation projects has shown a clear tendency to under-estimate costs and over-estimate benefits. Hence, unless appropriate changes are made in the methodology, ex-ante appraisals of energy conservation projects in developingcountries are likely to be poor guides bf theactual outcomes from the projects as well as of the host country's willingness to actually implement the projects. Consequently, GEF's rule of classifying projects as ineligible for GEF support (Type I projects) based on the appraisal of net domestic benefitsalone suffers from two potentialflaws. First, under the present methodology, the appraisals tend to result in over-optimistic assessments, so that GEF may fail to support projects that may need support. Second, this classification method fails to take account of the host country's assessment of the project, which may not be based solely on the appraised net domestic benefits. Thus, GEF may mistakenly conclude that the host country will implement the project on its own. It is suggested that GEF revise its decision rule to take account not only of the appraised net domestic benefits, but also of other factors such as the amount of global environmental benefits at stake and the level of financial support to be provided by GEF. TABLEOF CONTENTS EXECUTIVESUMMARY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . i 1. INTRODUCTION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 2 PROBLEMS IN COST-BENEFIT ANALYSIS OF WORLD BANK PROJECTS . ... 4 Statistical analysis of World Bank project appraisals ................... 6 Implications for GEF . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 3 COSTS OF ENERGY CONSERVATION PROJECTS . . . . . . . . . . . . . . . . . . . . 12 Consideration of Government Costs . . . . . . . . . . . . . . . . . . . . . . . . . . . . .12 Consideration of Utility Costs ................................ 13 Consideration of End-user Costs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 Implications for GEF . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 4. BENEFITS OF ENERGY CONSERVATION PROJECTS ................. 17 Improper definition of program impact ........................... 18 Lower than expected participation rates .......................... 19 Presence of "Free Riders" and "Takeback" Effects ................... 23 Equipment failure. misuse and persistence of effects . . . . . . . . . . . . . . . . . . 25 Implications for GEF ..................................... 26 5 UNCERTAINTY. RISK. AND DECISION RULES . ..................... 28 Uncertainty and risk in theory and project appraisals . . . . . . . . . . . . . . . . . . 28 Government response to uncertainty ............................34 GEF's response to uncertainty ................................ 35 6. SUMMARY AND CONCLUSIONS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .37 Flaws in the application of cost-benefit analysis . . . . . . . . . . . . . . . . . . . . . 37 Flaws in estimating costs of energy conservation projects . . . . . . . . . . . . . . .38 Flaws in estimating benefits of energy conservation projects . . . . . . . . . . . . . . 39 Uncertainty. risk. and decision rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 REFERENCES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 evaluate and possibly provide funds for the proposed project; (ii) Utility costs, which are incurred by the implementation agency (and associated trade allies or non-profit groups) that executes, monitors, and possibly conductsongoingor a-post evaluationsof the project; and (iii) End-user costs, which are incurred by the end-users, who are the beneficiaries of the project, but who may also have to bear some costs. 6. Appraisals and evaluations of energy conservation programs in North America and European have frequently failed to consider the full panoply of costs, even when a conscious effort has been made to be comprehensive. Based on this experience, there is likely to be significant underestimation of all three categories of costs inn potential GEF projects. In principle, it should be relatively straightforward to verify that all three categories of costs have been taken intoaccount in the cost-benefitanalysis of GEF projects. However, in practice, there may be difficulties in arriving at reasonable estimates of costs, and some costs may be overlooked completely. Hence, actual v i a l costs of energy conservation projects are likely to be under-estimated, unless special care has been taken to ensure that this is not the case. Flaws in estimating benefits of energy conservation projects 7. Based on the experience in North America, it is a common theme that energy savings, which are the benefits of the energy conservation programs, have been frequently and significantlyoverestimated in theex-ante appraisals of these programs. In any case, the "actual" savings from an energy conservation project are often to difficult to calculate precisely because they are equal to an actual post-installation consumption subtracted from a hypothetical baseline consumption that would have occurred had the program not been in place, and all other factors had held constant. 8. There are a number of factors that lead to ex-post energy savings which are less than the a-anre anticipated savings: (i) improper definition of program impact; (ii) lower than expected participation rates; (iii) "Free riders" and "takeback" effects;and (iv) equipment failure, misuse and lack of persistence of effects. While it may be relatively straightforward to ensure that the proper definition of program impact -- the net program impact -- is used, there may be difficulties in taking account of the other factors. For instance, the actual participation rate will depend upon a diversity of factors, such as the discount rate, transaction costs, the priorities of the end-users, and the nature of the promotional campaign, all of which may be difficult to estimate or assess on an a-ante basis. Similarly, there may be unanticipated takeback effects, which implies that the initial declinein energy consumption and total energy costs brought about by the energy-efficient technology induces end-users to increase their use of the service, e.g., users may use their energy-efficient compact fluorescent lamps for longer hours than conventional lamps. Finally, the history of energy projects in the developing countries indicates that there is a potential for significant problems in the installation, proper use, and maintenance of energy efficient technologies and devices, particularly for those with which the local people have limited familiarity and experience. Thus, even if there are initial savings from energy conservation projects, these have the potential of declining steadily over time. It would be prudent for GEF to verify whether the proposed project has taken account of this potential decline in benefits over time, or taken steps to prevent such a decline. Uncertainty, risk, and decision rules 9. It is likely that even after the biases in ex-ante estimates of costs and benefits of energy conservation projects have been reduced or eliminated, there will remain substantial uncertainty about the actual net benefits of energy conservation projects that GEF may support. Consequently, any decision-maker, i.e., the Government or GEF, who relies 00 the ex-ante appraisal to make a decision about an investmentproject faces significant probabilities of getting 'false-positive" and/or "false-negative" results. For example, the Government may end-up undertaking projects that do not deliver the expected results ("false-positive") and/or it may fail to undertake projects that would have brought substantial benefits to the country ("false- negative"). 10. Whilethereis an extensiveliteratureon risk and uncertainty,even well-known theoretical c~nceptshave not been extensively incorporated into project appraisals. At the same time, it is also clear that many of the theoretical results available in the literature are not sufficiently practical to be readily applied in the appraisal of projects in developing countriesL Government response to uncertainty 11. In view of the biases and uncertainty associated with the conventional ex-ante appraisal of net domesticbenefits, governments may be reluctant to rely solely on such appraisals to make decisions about undertaking energy-efficiency projects. It is likely that different Governments would have different responses to the uncertainty associated with GEF projects. One response of the decision-maker may be to conduct a heuristic, back-of-the-envelopeanalysis to determine "low-case" or "worst-case" outcomes. An extremely risk-averse decision-maker may wish to be sure that the worst-case scenario associated with the project is acceptable. Or,a risk-averse decision-maker may approve only those projects whose "low-case" estimateof the net domestic benefits exceeds a particular value. Even if no analysis is conducted to determiqe"low-case" outcomes, a Government may be willing to undertake only those energy conservqtion projects ;hat require low initial capital expenditures, so that the project can be terminatq at relatively low cost if there are early indications that the project will fail to achieve its projects benefits. CEF's response to uncertainty 12. Apart from the uncertainty associated with projected net domestic benefits, GEF also has to consider the uncertainties associated the decision-making rules used by Governments. It is clear that a host country may not undertake a project even if GEF classifies it as Type I, in which case the global environmental benefitsassociated with it will not be realized. 13. An attempt by GEF to take account of the Government's decision-rule in formulating GEF's own decision-makingrule about whether or not to support a particular project would lead to a moral hazard, because it would provide an incentives for Governments to adopt a stated policy of "We will not undertake these types of projects on our own" merely in order to secure GEF support. Thus, GEF will run the risk of supporting projects that do not need GEF support, i.e., "false positive" results. 14. On the other hand, if GEF continues to classify projects as Type I and Type I1based on the appraised net domestic benefits, then GEF faces two problems. First, until the project appraisal methodology is modified to take account of its failings and biases and risks, the appraised net benefits are seriously flawed,and GEF would run the risk of both "false positive" and "false negative" results, where "false negativen represents'failing to support projects that require GEFsupport. Second, even with an appropriatelymodified methodology, a decision rule based solely on the (correctly) projected net benefits will fail to take account of the potentially different responses of different Governments to similar projects, i.e., some Governments may be more risk-averse than others. Therefore, GEF may face significant risks of getting "false negative" results. 15. Mathematically, it is not possible for GEF to develop a decision-making rule that simultaneously minimizes the probabilitiesof both "false positive" and "false negative" results. Therefore, in deciding whether to focuson "false negative" or "false positive" results, GEF will have to take account of the consequences of these types of results. If substantial global environmental benefits are at stake, then GEF may be womed about denying a small amount of support to projectand taking the risk that the project may never be undertaken, i.e., the focus would be on minimizing "false negative" results. Alternatively, if substantial GEF funding is required, then GEF may be womed about "falsepositive" results. The implication is that GEF has to develop its own risk profile, and develop decision-making rules that take account of the amount of GEF funding at stake and the estimated global environmental benefits. DRAIT 1. INTRODUCTION 1.1 One of the analytical problems faced by the Global Environmental Facility (GEF) is the rationale for providing financialsupport for so-called ''Type I" projects. These projects have two definingcharacteristics: (i) they offer significant global environmental benefits, and (ii) they appear to offer positive net domestic benefits when evaluated in the standard ewnomic cost- benefit framework. The second characteristicimplies that Type I projects should be undertaken by host countries without any need for financial support from GEF based on global environmental considerations. If these Type I projects are actually undertaken without GEF financialsupport, then considerable global environmental benefits will be realized without the use of GEF funds. 1.2 However, there are concerns that Type I projects may not be implemented even though they appear to offer positive net benefits. If these projects are not undertaken, then the global environmental benefits associated with the projects will not be realized. Thus, it may be appropriate for GEF to provide financial support for such projects in order to realize the global environmental benefits. 1.3 The objectiveof this paper is to identify thefactors that may lead projects to beclassified as Type I even though they do not actually offer positive net domestic benefits. In accordance with the terms of reference, the focus of this paper is on energy projects, partidularly energy conservation projects that may have significant global environmental benefits. Nol new research has been undertaken for this paper, and it is based on a review and synthesis of the results available in the literature. 1.4 The basic approach taken in this paper is to consider the a-ante calculated net domestic benefits associated with a project as an estimate that is subiect to error. There h e two broad sources of error which lead to over-optimistica-ante assessments of projects relelbantfor GEF: flaws in the application of cost-benefit analysis to developmeht projects in general; flaws specific to energy conservation projects. Once these errors are taken into account, it may turn out to be the case thata project that ppears to provide net domestic benefits actually does not do so. The failure to actually provide positive net domestic benefits would then make a project a potential candidatefor GEF financial support. 1.5 There has been substantial experience with energy conservation projects, and more generally demand-side management(DSM)programs in North America and ~ u r d ~while e , such projects are still relatively few in the developing countries. Hence, it is useful to consider the problems that emerged in the North American and European experience with the appraisal and implementationof these projects. As shown below, even after more than a decade of experience with DSM programs in the U.S. and Europe, there is still a major need to improve the quality of data collection, without which it is difficult to provide a rigorous appraisal and evaluation of DSM programs. Need for Better Data on DSM programs 1.6 Many observers in the U.S. who have been closely involved with DSM programs find that the level and quality of the data bout these programs is inadequate. For example, after a detailed analysisabout the type of data used for appraising and evaluating DSM programs in the U.S., Hirst and Sabo (1992) found that "...the amount and quality of data now available on DSM programs are far short of what utilities and regulatory commissions need. The current lack of explicit, widely used definitions of DSM programs is a key deficiency ... we now discuss DSM programs in a 'tower of Babel,' leading to disparate estimates of DSM potential and performance." , 1.7 Further, Hirst and Sabo (1992) concluded that "the program-cost data that utilities report are often incomplete or not sufficiently detailed to use to compare or assess DSM program performance. In addition, traditionalaccountingsystemsonly monitor utility expenditures. Costs borne by the customer and other nonutility parties are often not provided by existing accounting systems. Knowledgeof these costs is necessary for calculating program cost-effectivenessfrom the perspective of participating customers and society." 1.8 In order to overcome these data problems, Hirst and Sabo (1992), whose work was sponsored by the Electric Power Research Institute (EPRI)and the U.S. Department of Energy, have developed a handbook that addresses the need for additional and better information in two ways. First, the handbook contains discussions of the key concepts associated with DSM programs-types, participation, energy and load effects, and costs. Second, the handbook offers definitions and a sampling reporting form for utility DSM programs, so that there would be greater consistency in the collection and reporting of data on DSM programs. 1.9 Similarly, Prindle (1991) concluded that "the difficulty with costs analyses in most DSM studies is that they are based on engineering estimates or other methods with high levels of uncertainty. More hard evaluation data on the cost, performance, reliability and other attributes of DSM options is needed. If data can be provided with enough rigour to satisfy utility regulators and planners, more money will flow into utility DSM programmes." In the absence of such hard data, Prindle concluded that: there remains a vast reserve of scepticism in the utility industry about DSM as a real resource for utility planners. To the extent that the jury is still out on the size, reliability, and longevity of DSM resources, this scepticism is justified ... Wild claims are still made about the magnitude of DSM resources .. . 1.10 In the same vein, based on an analysis of the data for several European countries (Belgium, Sweden, Norway, and Denmark), Bartlett (1993) concluded that "estimates of the ...Therefore, electricity used for residential lighting in most countriesare subject to large measurement error the potentialannual electricitysavings from the use of CFLs[compact fluorescent lamps] cannot be accurately estimated." Organization of paper 1.11 The rest of this paper is organized in thefollowing way. Section 2 presentsa discussion of recent concernsabout flaws in theapplicationof cost-benefitanalysis to development projects in general. Sections 3 and 4 present a discussion of the issues raised by errors in cost and benefits, respectively, of energy conservation projects, based mainly on the experiencein North America and Europe. Section 5 discusses therisksassociated with development projects, and the decision rules relevant in the context of risky developmentprojects. Finally,Section 6 presents the summary and overall conclusions. appropriate, the findings of institutional development specialists and staff with other skills in assessing the likely performance of project-related institutions. Ensure that the macroeconomic, financial, technical, and behavioralassumptions underlying the analysis are clearly spelled out. A. Statistical analysis of World Bank project appraisals 2.7 Pohl and Mihaljek (1992) analyzed the data for 1,015 projects supported by the World Bank over 1974-1987. The largest number of the projects were in agriculture (40%),followed by transport (30%),and energy (20%),and a small number of projects in industry and urban development. Pohl and Mihaljekanalyzed the relationship between "appraisal rates of return" (estimatesof economic rates of return at the time when projects are designed and appraised)and "re-estimated rates of return" (estimates of rates of return at the time when the projects are completed and begin normal operation) 31. 2.8 Based on a statistical analysis, the authors reached three main conclusions: there is an upward bias in appraisal rates of return; there is little link between appraisal and re-estimated rates of return; factorssuch as cost overrunsand completiondelays explain only a very small part of the difference between appraisal and re-estimated rates. Upward bias in appraisal rates of return 2.9 The nature of over-optimism in World Bank project appraisals is indicated by the data in Table 1. It is clear that the appraisal rates are significantly higher than the re-estimated rates; for example, the median appraisal rate is 18%,while the median re-estimated rate is only 14% . Further, on average there have also been significant cost ovemns and implementation delays; for example, on average, these projects took six years to complete, compared to an estimated average completion time of four years, for an average time delay of two years. 2.10 It is clear that, on average, there was a bias towards optimistic assessments in project' appraisals undertaken by the World Bank. However, it is worth noting there were a large number of projects that did not suffer from this optimistic bias. While Pohl and Mihaljek's statistical analysis did not differentiatebetween over-optimistic and other projects, it clear from 31 The re-estimated rates are not true ex-post rates of return, which can be calculated only at the end of the project's economic life. Nevertheless, the re-estimated rate is a better indicator of a project's performance than the appraisal rate becausethe re-estimated rate is based on actual values for several critical variables, such as investment costs and project completion schedules, while the appraisal rate is based on estimates. their plot of the relationship between re-estimated and appraisal rates of return (see Annex 1) that there were many projects for which the re-estimated rate of return was greater than or equal to the appraisal rate of return. It followsthat if we examineonly the projects that had optimistic assessments, then their optimistic bias is relatively greater than the average bptimistic bias reported in Table 1 4/. In other words, for projects that have an optimistic bias, the bias is greater than the average bias. Table 1: Bias in A~~raisalsof Rates of Return of World Bank Proiects Arithmetic Median Maximum Minimum Mean Estimated Ecoxiomic Rate of Return ( X ) At project appraisal 22 18 158 1 At project completion 16 14 128 - 20 Estimated Total Project Cost(US $ million, current prices) ~ I At project appraisal 86 34 3,193 1 At project completion 102 40 4,045 1 Time Overrun(Years) 2 2 16 1-4 I Note: Based on 1,015 World Bank projects over 1974-1987 I Source: Pohl and Mihaljek(1992) I 2.11 An alternative measure of the optimistic bias is provided by the results of the regression analysis reported by Pohl and Mihaljek. In their basic model, the dependent vhriable was the re-estimated rate, with the appraisal rate as the explanatory variable, i.e., , Re-estimated rule = a + b Appraisal rare I In variants of this basic model, a number of other explanatory variables, such as the country's economic management rating and GNP, and indicator (dummy) variables reprdsenting sectors (energy, transport, etc.) and regions (Latin America, East Africa, etc.) were albo added. - 41 The average bias reflects the upward bias in the optimistic projects as well as the downward bias in conservativeprojects, where the re-estimated rate exceeded the appraisal rate. - 8 - DRAFT 2.12 In these regression equations, the slope parameter b measures the effect of a change in the appraisal rate on the re-estimated rate. For example, holding other things constant, if the appraisal rate increases by 1, say from 12% to 13%, then the re-estimated rate increases by b. Clearly, if b = 1, then a change of 1 in the appraisal rate will also imply a change of 1 in the re-estimated rate; further, if b is less than 1, then a change of 1 in the appraisal rate will imply an increase in the re-estimated rate of less than 1. 2.13 In the results reported by Pohl and Mihaljek, the estimated value of b is in the narrow range of 0.43-0.46, which implies that when the appraisal rate increases by 1, the re-estimated rate increases by considerably less than 1. Theextent of the optimisticbias in appraisals implied by this estimated b is indicated by the following example. Consider two projects X and Y with appraisal rates of, say 12%(X) and 27% (Y),so that the difference in the appraisal rates is 15%. With b = 0.45, the difference between the re-estimated rates of the two projects is predicted to be only 6.75%, and not 15%51 In other words, project Ys high appraisal rate is significantly biased upwards. 2.14 Ener~vsector: Pohl and Mihaljek did not report the average appraisal and re-estimated rates by sector. However, they did report the results of the regression analysis by sector. For the energy sector 61, the estimated value is b = 0.61, which is closer to 1 than the overall value of b = 0.45. This result indicates that the upward bias in energy sector projects may be less than the general upward bias in World Bank appraisals. 2.15 Consequencesof U~wardBias: Oneresult of theexcessiveoptimismin theappraisal rates of return is that there are a large number of projects whose re-estimated rates of return are below conventionally acceptable rates of return. Specifically, about one-fourth of the projects had a re-estimated rate of return below 10%; about one in seven (14%) project has a re- estimated rate of return below 5%; and one in twelve (8%) of the projects had zero or negative re-estimated rates of return. Limited link between appraisal and re-estimated rates of return 2.16 There is only a limited link between the appraisal and re-estimated rates of return. . For example, Pohl and Mihaljek (1992) found that "ninety percent of all projects have appraisal rates of return in the range of 10-40%,but only about half have re-estimated rates of return within this range." 2.17 Another indication of the limited link between appraisal and re-estimated rates of returns is the low explanatory power of the regression equations. For the basic model, Pohl and Mihaljek report an R2 value of 0.19, SO that the appraisal rate of return explains about only 20% 61 See Table 8, Pohl and Mihaljek (1992). of the variation in the re-estimated rate of return; in a variant model which includes other explanatory variables as well as indicator (dummy) variables for sectors and regions, the explanatory power is 31%. Similarly, for the energy sector alone, the regression equation (which includes other explanatory variables as well as regional indicator variables) explains only 30%of the variation in the re-estimated rate of return. 21 Reasons for divergence between appraisal and re-estimated rates of return 2.18 Proiect-specific factors: Intuitively, cost overruns and implementation delays are expected to lower the re-estimated rate of return. However, in the regression analysis, these explanatory variables were not statistically significantand also had the "wrong" sign. Further, when nominal cost overruns were decomposed into unexpected inflation and realcost overruns, the results of the regression analysis indicated that real cost overruns did not have a strong adverse effect upon re-estimated rates of returns 81. 2.19 Regional differences: There is a clear geographical pattern in the diverpence between appraisal and re-estimated rates of return. The estimated parameters of the regional indicator variables imply that, for a given appraisal rate of return, the re-estimated rates of return are the highest in South Asia, i.e., projects in South Asia show the lowest optimistic bias. South Asia is followed by East Asia, Latin America, the Mediterranean,and the French African Community (CFA), with projects in East and West Africa (other than CFA zone) haviqg the greatest divergence between appraisal and re-estimated rates of return. Pohl and Mihaljbk attribute the differencein the performanceof projects in CFA countries, compared toother Af$ican countries, to the institutionalframeworkand the conservative fiscal and monetary policies f~llowedby the CFA countries. 2.20 The relatively poor performance of projects in East and West Africa (dther than CFA countries) is also reflected in the occurrence of project failures. Out of the 80 projects that had -71 It should be noted that low values of are common with cross-section datasets with a large number of observations; to this extent, the low explanatory power reported by Pohl and Mihaljek is not a surprise. Further, Pohl and Mihaljek reported only adjusted R2values, and not the conventional R2values. The adjusted R2 reduces the conventional R2, based on the number of explanatory variables included in the regression equation, so that the conventional value is greater than or equal to the adjusted R2values. While the adjusted k2 is useful for comparing the relativeexplanatorypower of differentregression equations with different number of explanatory variables, only the conventional R2 can be properly interpreted as the actual explanatory power of a particular regression equation. -81 Pohl and Mihaljek indicate that this result may be misleading because there is a possibility that projects with large real cost overruns reflect mostly expansion of projects, rather than errors in cost estimates. negative re-estimated rates of return, 27 were in East Africa. In particular, agricultural projects in sub-Saharan Africa experienced a high failure rate, so that half of agricultural projectsin East Africa and more than a quarter of such projects in West Africa had re-estimated rates of return below 5%, with a significant difference between CFA and non-CFA countries. 2.21 Other ex~lanatorvvariables: The regression analysis indicates that an unexpected increase in primarv commodity urices tends to increase the re-estimated return, which is consistent with the fact that many of the agricultural projects involve production of primary commodities. Further, the regression analysis indicates that better economic management of a country 91 tended to raise the re-estimated rates, which led Pohl and Mihaljek to conclude that the adverse effects on project performance of government interventions through price controls, high tariffs, import restrictions, etc. has been underestimated in World Bank project appraisals. B. Implications for GEF 2.22 It is clear that in recent years World Bank appraisals of development projects have been over-optimistic. Thus, if a conventional World Bank appraisal of a proposed GEF project classifies that the project as economically viable, then there is a clear potential that the project may not actually be economically viable, i.e., a project that appears to be a Type I project may actually be a Type I1 project. In particular, over-optimisticassessments are likely unless project appraisals that have explicitly considered and taken account of: (i) downside risks; (ii) the host country's commitment to the project; (iii) the host country's macroeconomic conditions, policies, and economic management; (iv) the capacity of host country institutions to effectively implement the project; and (v) the success rate of completed projects in the sector, country, and region Thus, until the Bank issues guidelines about the manner in which these factors should be taken into account, and until the guidelines are actually incorporated into project appraisals, GEF may find it prudent to question the validity of the results of project appraisals that ignore these factors. 2.23 It is also clear that while, on average, there has been an optimistic bias is World Bank project appraisals, this bias has not been present in a significant number of cases, and some project appraisals have actually been conservative in their estimateof the project's rate of return. Thus, there has been only a limited link between the appraised and likely actual economic viability of a project. In other words, the appraised economic viability of a GEF project is 21 Measured by indices such as an index of price distortion or the Bank's internal ranking of the quality of a country's economic policies and management. - 11 - DRAFT likely to be a poor indicator of the project's actual economic viability 101. It follows that it would be imprudent for GEF to use the appraised economic viability of a project as a dominant criterion in determining whether or not the project should or will be undertaken without GEF financial support. -101 This situation may be made clearer by the following example. suipose a set off measurements is taken by a number of different observers to determine the depth of a river, and the average depth of the river is calculated to be 22 feet. A later review shbws that (i) the measuring instruments used by the differentobservers had different biases, somb upwards, and other downwards, and (ii) measurements taken with reliable instruments show that the average depth of the river is 16 feet, i.e., an average bias of 6 feet. Suppose that a reading taken with one of the original measuring instruments shows that the depth of the river at a particular point is 18 feet, but it is not possible to determine whether this particular original instrument used to take this reading was biased, or the direction and extent of the bias. In this situation, it would be hazardous to use the average bias to correct for the possible bias is measurement, e.g., it would be meaningless to conclude that the correct depth is 18 - 6 = 12 feet, where 6 is the average bias. DRAFT 3. COSTS OF ENERGY CONSERVATION PROJECTS 3.1 For potential GEF energy conservation projects, domestic costs can be classified into three categories: Government costs. These costs are incurred by the official agencies that consider, evaluate and possibly provide funds for the proposed project before it is implemented. Utility costs. These costs are incurred by the implementation agency that executes, monitors, and possibly conducts ongoing or ex-post evaluations of the project. On occasion, costs may also be incurred by trade allies, such manufacturers of CFLsor retail outlets, and/or non-profitgroups, who participate in the promotional aspects of energy conservation programs. End-user costs. These costs are incurred by the end-users, who are the beneficiaries of the project, but who may also have to bear some costs. 3.2 As discussed below, appraisalsand evaluations of energy conservationprograms in North America and European have frequently failed to consider the full panoply of costs, even when a conscious effort has been made to be comprehensive. For example, in his analysis of European lighting program costs, Mills (1991) uses a concept of "total resource cost," which includes all costs for the lamps, salaries, consultants, advertising, postage, evaluations, etc. However, the "total resource cost" does not include Government o r end-consumer costs. Further, Mills states that "Lighting trade organizations and/or individual manufacturers have helped European utilitiesto organize and run some programs." Yet, it does not appear that these costs have been included by Mills in "total resource cost." A. Consideration of Government Costs 3.3 For GEF projects, there is significant potential for the underestimation of Government costs since they may be mainly of the opportunity cost variety, and not explicit cash expenditures. GEF energy-related projects have an element of novelty, which implies that they may require extensive consideration and evaluation by Government officials. Thus, the opportunity cost value of the time and effort spent in consideration and evaluation may be significant. Further, in some instances. the Government may engage domestic or international consultants to assist in this process. 3.4 In the U. S., similar costs are typically incurred as part of the regulatory process that is used to approve energy conservation programs. Regulatory costs are incurred not only by the regulatory commissions or agencies but also by all the parties that participate in the regulatory process. For example, apart from the regulatory commission and its staff, proposed energy conservation projects may also be incurred by groups such as the Office of Consumer Advocate DRAFT (which represents residential consumers), the mass transit agency (which uses large amounts of electricity to operate trains), or associations of large commercial buildings (which also use significant amount of electricity). While accounting practices differ at utilities, it is unlikely that any utility can include these costs in the aggregation of total costs because the utility is unlikely to have any means of collecting such information, even in the remote event that it had been compiled by the individual agencies. 3.5 The cost of evaluating the novel elements of GEF energy projects may pkove to be high for countries that have limited expertise in this regard. For example, a recent review 111 of the energy sector in Tonga concluded that the Government adopt: a policy that limits the energy options to robust technologies that have been proven operationally and economically in environments similar to that of Tonga. One of the principal reasons for this recommendation is the shortage of managerid and technical skills in Tonga. Thus, the opportunity costs of evaluating potential GEF projects may vary from country to country. I B. Consideration of Utility Costs I 3.6 Typically, in addition to direct expenditure costs, such as on purchasingCFLs or other equipment, the utility will also incur indirect costs that may be in the fdrm of explicit expenditures or opportunity costs 121. The explicit indirect expenditures may be for consultants or experts used to assist the utility in implementing and/or monitoring the project, or for promotional measures designed to raise end-user awareness of the project. The opportunity costs may consist of the value of the time of utility personnel as wkll as overhead costs associated with the project. I 3.7 Following Hirst and Sabo (1992), utility costs include: I AdministrativeCosts, which account for the staff involved in probram planning, design, marketing, implementation, and evaluation, including liabor expenses, office supplies, data processing, and such other costs; ~ - Tonga:lssue~andOprionsintheEnergySector,PacificIslandsSeriesho.1,Vol.10, 111 World Bank, August 1992. - 121 Beny (1989) quantified the administrative costs of program plannihg, evaluation, marketing, auditing, quality control, data collection, and related activities. Admlinistrativecosts were about 20% of total costs of residential programs, and about 10-15% for commercial programs. - 14 - DRAFT Marketing Costs, which include all costs directly associated with preparation and implementation of marketing activities, such as direct mailings, bill stuffers, media advertising, training sessions, etc.; Monitoring and evaluation costs, which are incurred for data collection and analysis to assess the performance of the program; Equipment Costs, which cover the cost of the equipment purchased directly by the utility; Incentives Costs, which cover the costs of the incentives provided by utilities to participate in energy conservation programs. 3.8 While direct expenditure costs are easy to measure and difficult to hide, they may be underestimated because of unfamiliarity with local working conditions, particularly with respect to the projected scheduling of the project. For example, the estimated utility costs may be based on a schedule that does not account of the delays that frequently occur in that particular country or region. Or even the promotionalcosts of the project may be partially absorbed in the general advertising expenses of the utility. Thus, the actual direct utility expenditures may be significantly underestimated in the project's cost-benefit analysis. 3.9 There is a clear potential for not including or underestimating the indirect expenditures. Based on a sample of ten utilities in the U.S.,Joskow and Marron (1992) concluded that ... many typesof administrativecosts, includingmeasurementand evaluationof conservation savings and overhead, are not universallytracked and reported. The failure to account for such costs can lead to significant underestimatesof the true costs of utility-sponsored conservation initiatives. C. Consideration of End-user Costs 3.10 Even though the end-users are usually the beneficiaries of utility conservation programs, there may be some costs that they have to bear. In some instance, the end-users may have to bear part of the cost of acquiring new hardware, e.g., households may have to pay part of the cost of new CFLs. Further, energy efficiency is usually embedded in expensive and long-lived assets, and end-users may be reluctant to throw away or dispose of inefficient equipment that still works. In any case, there are some opportunity costs associated with "premature" retirement of hardware, e.g., the replaced conventional bulbs may have some residual value that may be lost when CFLs are installed. 3.11 End-users may also incur significant time and effort costs in considering and evaluating whether or not participate in utility-sponsoredprograms. Joskow and Marron (1992)categorize these as "customer transaction costs" and report that "None of the programs attempt to measure customer transaction costs. Yet customer transaction costs are very real economic costs that DRAFT should in principle be accounted for in evaluating the societal cost of utility conservation programs." 3.12 Hamlin (1990) described the cost incurred by end-users in this way: Consumers must spend time and effort searching for the particular options that are best for them; then invest in new technology, equipment, or process that will reduce their energy service costs over time. This requires the consumer to invest precious time and scarce capital up front and to accept some risk to reap the benefits of energy efficiency in the form of future operationalsavings. Ahecdotal observations of customer behavior and survey results shows that the majority of customers are not aware, willing, nor able to make the necessary investinentsin time and capital, nor to take the risks. (Emphasis in original.) I From an end-user's perspective, savings in energy costs are just one of the fadtors considered in making decisions on the use of capital and other scarce resources; the end-user also considers product quality, space and cost requirements for equipment, labor costs, etc. ~ 3.13 According to Sioshansi (1991) 131 market research results indiqate that most consumers face high transaction costs in obtaining timely, credible and relevht information when purchasing major energy appliances or making decisions energy consedation decisions - 141. For example, Gruber and Brand (1991) report that 52% of the small and medium-sized W. German firms they surveyed did not consider subsidies a decisive factor in undertaking energy conservation programs because ". ..subsidy programmes are often poorl$ adapted to the problems of small and medium-sized firms. The stafl do not have much ti e to read big brochures or to811in complicated application forms. (Emphasis added.)" ?' I e I 3.14 It appears that residential consumers may feel that they have to incur so e trouble even when they do not have to make any decisions about equipment purchase or bear any equipment costs. For example, in the Hood River energy conservation project, even th 0ugh the home ~ I - 131 Sioshansi's observations are particularly relevant because the author is ehployed a large U.S. electric utility, Southern California Edison Company. I I , I - AccordingtoNadel'etd(1993),severalU.S.programshaverecogni?edthelackof 141 information as a major barrier to the adoption of energy-efficient lighting, and taken steps to disseminate objective information that -consumers can use to evaluate efficient lights. Several lighting technology demonstration centers are also now open to the public and dehign community in major U.S. cities. The U.S. EPA has a "Green Lights" program which i$ a high-profile project designed to promote lighting retrofits in the facilities of the top U.S. cor$orations. This program providespublicity materials,decision-making tools, technical information, manufacturer and contract information, information on utility rebates, and publicity for partir$pants. - 16- DRAFT energy audit was free and the entire cost was borne by the Bonneville Power Authority, only 85% of eligible homeowners participated. D. Implications for GEF 3.15 It is clear that in North America and Europe the costs of energy conservation projects have been underestimated, and that this finding applies to all three categories of costs: government, utility, and end-user. Hence, unless extensive care has been taken in the economic appraisal of potential GEF projects, it is natural to expect that the projects costs have been underestimated. 3.16 Further, it may bedifficulttoeliminate thepotential underestimationof Governmentcosts associated with GEF projects because the accounting system used by Governmentsmay not be oriented towards establishing costs incurred in the evaluation of individual projects. In the absence of substantive data from developing countries, it may be difficultto develop even a priori rules of thumb to take account of these costs. In any case, there may be significant variations in the abilities of the host countries to evaluate potential GEF projects. 3.17 The novelty elements associated with potential GEF projects may introduce a factor that makes it difficult for some countries to undertake some seemingly Type I projects without GEF support. As noted by the Wapenhans report, the limitation of expertise in host countries poses risks for development projects in general (see para. 2.5). Thus, the novelty of the projects implies that there is an element of risk associated with the project, i.e., the project may deliver less than the projected benefits and may even provide no benefits at all, or there may be significantcost overruns, or some unexpected snags that reduce the justification for the project. Thus, the Government may be reluctant to undertake the project without GEF support because of this element of risk. The nature of this risk, and its implicationsfor Governments and GEF are discussed in Section 5 of this paper. 3.18 So far as utility costs are concerned, in theory, it should be relatively easy to ensure that all relevant utility costs are taken into account, e.g., the handbook developed by Hirst and Sabo (1992) may provide a framework for developing an appropriate accounting system. In the interim, it may be necessary to develop some rules of thumb to gauge the validity of the costs reported in potential GEF projects. However, it may be difficult to eliminate tendencies to make excessively optimisticassumptions about the effort and time required to implement the project. The nature of the downside risks presented by such tendencies is discussed in Section 5. 3.19 Similarly, in principle, it should be relatively easy to develop estimates of end-user expenditures on hardware as well as the residual value of equipment that is prematurely scrapped. It may be more difficultto develop estimatesof the end-user transaction costs because these are likely to vary significantly according to the particular circumstances of the project. Nevertheless, efforts should be made to include some estimates of end-user costs in the aggregate project costs. 4. BENEFITS OF ENERGY CONSERVATION PROJECTS 4.1 The benefits of energy-related projects are usually projected to flow to both the utility and its customers. The overestimation of benefits may affect either the utility or its customers or both. Based on the experience in North America, it is a common theme that energy savings, which are the benefits of the energy conservation programs, have been frequently overestimated in the ex-ante appraisals of these programs. For example, Nadel and Keating (1991) examined 32 U.S. utility energy conservation programs, and found that savings were overestimated in 27 programs and under-estimated in the other five. For 15 of the 27 overestimated programs, the actual savings were less than 50% of the projected savings. In particular, eightof 11 residential programs saved less than half as much energy as predicted. Similarly, Keating 151estimated that about 15-20% of energy-saving bulbs handed out by U.S. utilities were not being used. 4.2 The experience of a number of energy-related projects in the developing countries also underscores the tendency to overestimate the impact of such projects. In particular, this tendency is likely to arise when projects are sponsored or promoted by "enthusiasts" or "proponents" of particular points of view or technologies or by entrepreneurslfirmswho also stand to profit from hardware sales associated with the project 161. 4.3 VerificationThe"actual"savingsfromanenergyconservationprojectare often to difficult to calculate precisely because the actual savings are equal to an actual post- installation consumption subtracted from a hypothetical baseline consumption that would have taken place had the program not been in place, and all other factors had held constant. For example, changes in causal factorssuch as weather, income, work habits,or lifestyleschanges171 bring about changes in energy consumption that are difficult to differentiate from the impact of energy conservation programs. Reasons for differences between ex-ante and ex-post energy savings 4.4 There are a number of factors that lead to ex-post energy savings which are less than the ex-ante anticipated savings. These factors can be classified as: L5/ Wall Srreer Journal, May 27, 1993, page B9. - 16/ See Overview: PucrjicRegionalEnergy Assessmenr, Volume 1, Pacific IslandsSeries No. 1, World Bank, 1992. - 17/ For example, the spread of take-outldeliveredfood and the i-ncreasinguse of microwave ovens in the U.S. has significantly reduced the energy used by 'households for preparing food at home. DRAFT expected value of $ 65 271. Is this the correct measure of the benefits? An alternative is the "option price," which is the maximum sure payment that the farmer would be willing to make in both states. Option price depends upon the individual, and may be more or less than the expected value of benefits. 5.16 From his theoretical analysis, Graham concluded: (i) Option price is the appropriate measure of benefit in situations involving similar individualsand collective risk (a dam would be a case of collective risk; (ii) expected value calculations are appropriate in situations involving similar individuals and individual risks. These concepts were later extended to the case of uncertain costs by Freeman (1989). However, these concepts have not been applied to the case of project appraisals in developing countries. Pure risk and the capital asset pricing model 5.17 The concept of pure risk has been widely used in the capital asset pricing model (CAPM), which provides some very useful results on how rational, risk-averse individuals and markets evaluate risk. One of the key relevant results of the CAPM is that, under the right circumstances, it is not particularly useful to evaluate the risk of an asset on its own; instead, it is better to consider the characteristicsof a particularasset in thecontext of the entire portfolio of assets being held by an individual. For instance, assets whose values tend to move in opposite directions -- whose returns are negatively correlated -- tend to reduce the overall risk in the portfolio. For this reason, the selection of a varied portfolio of assets makes it possible to diversify away the risks associated with particular assets. 5.18 In the CAPM',the risk of an asset has two components: (i) systematic risk, which is represented by B (Beta), and (ii) unsystematic risk. The parameter Bi measures the riskiness of a particular asset i relative to the risk in the entire market portfolio 281. If an asset has a Beta value equal to one (Bi = I), then it is just as risky as the market as a whole; when an asset's Beta is greater (less) than one, the asset is more (less) risky than the market as whole. 5.19 A fundamental result of the CAPM is that in an efficient market all assets will have the same rate of return after adjusting for risk, which is stated as 281 - Mathematically, Bi is defined as where Ri and R,,, the returns of asset i and the market portfolio. Thus, Bi is the covariance are of the return on the asset with market return divided by the variance of the market return. return on a particular asset = risk-free return + risk adjustment The nature of the risk adjustment is such that assets whose Beta is greater (less) than one will have higher (lower) rates of return, i.e., the risk adjustment depends on l3, but not on unsystematic risk 291. 5.20 In contrast to systematic risk, unsystematic risk is the purely random variation in the rate of return of an investment about its expected value, and is due to the peculiarities of the asset. As shown by the equation above, a key result of the CAPM is that the portfolio risk in efficient portfolios is determined by systematic risk, and not by the unsystematic risk of an asset; in particular, a high unsystematic risk will not lead to a requirement a higher rate of return 3 1 . 5.21 These results of the CAPM are well-known and widely accepted. Yet, it is not common practice to consider the riskiness of investing in energy conservation investments. Oneexception to this is the analysis provided by Sutherland (1991). I 5.22 Sutherland (1991) argues that investing in energy efficiency is risky ih the sense that actual savings tend to vary significantly from predicted savings 311. Since investors are risk-averse, investments in energy efficiency are less than what they would be ih a more certain world. However, based on the CAPM results, the relevant risk is not the raqdom risk of an individual asset but the risk of the investor's overall portfolio. Sutherland claids that the major risk of many energy-efficient investments is the random unsystematic ri4k; hence such investments are probably not risky in the sense of having high l3 values. It follows that the required rate of return on energy-efficient investments should be compdble to that of investments in general. Thus, Sutherland concludes, that in general the vie$ that risk is a market barrier that discourages energy-efficient is an ad hoc notion, which is not firmly grounded in financial theory. I I - 291 Mathematically, in an efficient market, in equilibrium, the return on an:asset is: , where Rf is the risk-free return, and fl,(R,,,- Rf) is the adjustment for risk. I 301 These results of the CAPM are based on some assumptions that are dsually valid for stock markets in industrialized countries: liquidity of investments, marketability1,and the ability to reduce risk by holding a diversified portfolio. I I 311 According to Sutherland,a study of commercial building retrofits in the U.S. concluded that very few predictions of energy savings came within 20% of measured results. Residential sector and small businesses 5.23 Sutherland finds that the CAPM results may not be applicable to the analysis of energy- efficiency investments undertaken by the residential sector or small, privately held businesses because the key CAPM assumptions -- liquidity, marketability, and the ability to reduce risk by holding a diversified portfolio -- may not be valid for such investments. In particular, these investments tend to be in tangible, illiquid assets, with limited marketability. 5.24 Hieh initial costs: Further, a household may find the risk of the investment (in say, - CFLs for low income households, or shell measures for other household) to be significant relative to the household's total income, and the household may not have a sufficiently diversified portfolio to diversify away this risk. Low-income households may have a zero, or even negative, propensity to save, and may therefore be averse to investment assets. Such households are particularly limited in terms of reducing risk through diversified portfolios. In other words, residential consumers and small, privately held businesses may require higher rates of return on energy-efficient investmentsbecause such investments are illiquid, not readily marketable, and their risk is not easily diversified away. Is risk an additional cost? 5.25 Whilerisk is a potentiallyserious obstacle to theadoptionof energy-efficient technologies and devices, it appears inappropriate to consider risk an additional cost element that is overlooked in cost-benefit analysis. Instead, it appears appropriate to (i) explicitlyconsider both pure and downside risk in conducting the cost-benefitanalysis, and (ii) develop decision-making rules that consider not just the appraised domestic benefits but also the risks. For example, the methodology put forward by Crousillat and Merrill (1992) explicitly considers and emphasizes the downside risk in undertaking-major power sector investments. B. Government response to uncertainty 5.26 In view of the biases and uncertainty associated with the conventional ex-ante appraisal of net domestic benefits, governments may be reluctant to rely solely on such appraisals to make decisions about undertaking investment projects. It is likely that different Governments would have different responses to the uncertainty associated with GEF projects. Nevertheless, it is possible to consider some of the decision-making rules that Governments may use. 5.27 It is likely that a decision-maker will view the projected net domestic benefits of an energy conservation project with skepticism. One response of the decision-maker may be to conduct a heuristic, back-of-the-envelope analysis to determine "low-case" or "worst-case" outcomes. An extremely risk-averse decision-maker may wish to be sure that the worst-case scenario associated with the project is acceptable. Or, consistent with above discussion of downside risk, a risk-averse decision-maker may approve only those projects whose "low-case" estimate of the net domestic benefits exceeds a particular value, irrespective of the potential benefits associated with base or high case estimates of the net domestic benefits. 5.28 Even if no analysis is conducted to determine "low case" outcomes, a Government may be willing to undertake only those energy conservation projects that require low initial capital expenditures, so that the project can be terminated at relatively low cost if there are early indications that the project will fail to achieve its projects benefits. In other words, a Government may not be willing to stake an initial large sum of money on uncertain Wects, but may be willing to undertake projects with similar total costs but whose costs are spread over time. 5.29 It is also possible that none of the formal models may describe a Government's decision- making process, which may be based on the experience and "seat-of-the-pants"judgement of the decision-makers. C. GEF's response to uncertainty 5.30 Apart from the uncertaintyassociated with projected net domestic benefits, qEF also has to consider the uncertainties associated the decision-making rules used by Governhents. It is clear that even if GEF classifies a project as Type I, the global environmental benefitsassociated with it will not be realized unless the Governmentactually implements the project in^ the absence of GEF support. 5.31 In principle, GEF could take account of the Government's decision-rule in formulating GEF's own decision-making rule about whether or not to support a particular projept. In other words, GEF may consider it prudent to consider supporting projects that a Governn)ent will not undertake on its own. However, any such recognition presents a moral hazard because it provides an incentives for Governments to adopt a stated policy of "We will not un dertake these types of projects on our own" merely in order to secure GEF support. Thus, GEF1willrun the risk of "false positive" results, i.e., of supporting projects that do not need GEF aupport. 5.32 If GEF continues to use the appraised net domestic benefits as a decision-making rule, i.e., continues to classify projects as Type I and Type I1 based on the appraised Oet domestic benefits, then GEF faces two problems. First, until the project appraisal methodology is modified to take account of the failings, biases and risks indicated by the Wapenharis report and the experience with energy conservation projects in industrialized countries, the alppraised net benefitsare seriously flawed. Consequently,any decision rule based on these estimdtes may also be seriously flawed, and GEF would run the risk of both "false positive" and "false negative" results, where "false negative" represents failing to support projects that require G/EF support. I 5.33 Second, even if the methodology is modified to eliminate or reduce the abode problems, a decision rule based solely on the (correctly) projected benefits will fail to take adcount of the potentially different responses of different Governments to similar projects, i.e., some Governments may be more risk-averse than others. Therefore, GEF may face significant risks of getting "false negative" results. 5.34 It is not possible for GEF to develop a decision-making rule that minimizes both "false positive" and "false negative" results 321 In deciding whether to focus on "false negative" or "false positive" results, GEF will have to take account of the consequences of these types of results. If substantial global environmental benefits are at stake, then GEF may be worried about denying a small amount of support to project and taking the risk that the project may never be undertaken, i.e., the focus would be on minimizing "false negative" results. Alternatively, if substantial GEF funding is required, then GEF may be womed about "false positive" results. 5.35 The implication is that GEF has to develop its own risk profile, and develop decision- making rules that take account of the amount of GEF funding at stake and the estimated global environmental benefits. 321 In practical situations, decision makers often have to declare a preference for minimizing one or the other probability. For example, in the judicial system, the burden of the proof is on the prosecution, and the defendant has to be proven guilty beyond a reasonable doubt. In other words, there is an emphasis on reducing "false guilty" verdicts even though this may lead to frequent "false not-guilty" verdicts. Or, a medical diagnostic test for a disease such as cancer may be set up so that it minimizes the probability of "false negative" results (test says an individual does not have the disease even though it is present), but permits the probability of "false positive" results to stay high, perhaps so that further diagnostic test can be run to verify whether the individual does have the disease. 6. SUMMARY AND CONCLUSIONS 6.1 The Global Environmental Facility (GEF) is concerned that Type I projects 3 1 may not be implemented by host countries even though they appear to offer them positive net domestic benefits, so that the global environmental benefits associated with the projects may not be realized. Therefore, it is appropriate for GEF to consider the steps to be taken to ensure that such projects are actually implemented. 6.2 There are two broad problems with the projected the a-ante calculated net domestic benefits associated with a project, which lead to over-optimisticassessments of projects relevant for GEF: (i) flaws in the application of cost-benefit analysis to development projects in general; and (ii) flawsspecificto energy conservation projects. Once these flaws are taken into account, it may turn out that a project that ayears to provide net domestic benefits actually does not do so. The failure to actually provide net domestic benefits would then make a project a potential candidate for GEF support. A. Flaws in the application of cost-benefit analysis 6.3 Recently, there have been concerns that the application of cost-benefit anlalysis by the World Bank has led to over-optimisticassessments of development projects. The world Bank's Wapenhans report co~lcludedthat the appraisal of development projects supported by the World Bank has been over-optimisticbecause they have not taken full account of the: (i) changes in the globallevel economicconditions, (ii) the host country's macroeconomicconditionsand policies, changes in developmental priorities, deficientregulatoryenvironments,and thelacklof or decline in capacities of local institutions, and (ii) the increasing complexity of projects, which makes it difficult to implement them effectively, along with a lack of commitment on thd part of host countries to the projects. 6.4 Based on a statistical analysis of 1,015 World Bank projects, Pohl and Mihaljek (1992) reached three main conclusions: (i) there is an upward bias in appraisal rates of rewm; (ii) there is little link between appraisal and likely actual rates of return; and (iii) factors such as cost overruns and completion delays explain only a very small part of the differdnce between appraisal and likely actual rates. One result of the excessive optimism in the app+.isalrates of return is that there are a large number of projects whose likely actual rates of retdrn are below conventionally acceptable rates of return. 6.5 In response to these over-optimisticappraisals, the Bank has decided to emphasize, inter alia, (i) host country commitment to projects, and (ii) explicit and systematic recognitionof the I Type I projects offer significant global environmental benefits, and also positive net domestic benefits when evaluated in the standard economic cost-benefit framework. DRAFT risksassociated with developmentprojects. The Bank plans to issue soon new guidelines on risk and sensitivity analysis for development projects. 6.6 Since recent World Bank appraisals of development projects have been over-optimistic, it follows that when a conventional World Bank appraisal classifies a proposed GEF project as economically viable, there is a clear potential that the project may not actually be economically viable, i.e., a project that appears to be a Type I project may actually be a Type I1project. In particular, over-optimistic assessments are likely unless project appraisals have explicitly considered and taken account of: (i) downside risks; (ii) the host country's commitment to the project; (iii) the host country's macroeconomicconditions, policies, and economic management; (iv) the capacity of host country institutions to effectively implement the project; and (v) the success rate of completed projects in the sector, country, and region where the projectis located. 6.7 Thus, until the Bank issues guidelines about the manner in which these factors should be taken intoaccount, and until the guidelinesareactually incorporated intoprojectappraisals, GEF may find it prudent to question the validity of the results of project appraisals that ignore these factors. In other words, since the appraised economic viability of a GEF project is likely to be a poor indicatorof the project's actual economic viability, it would be imprudent for GEF to use the appraised economic viability of a project as a dominant criterion in determining whether or not the project should or will be undertaken without GEF financial support. B. Flaws in estimatingcosts of energy conservation projects 6.8 For potential GEF energy conservation projects, domestic costs can be classified into three categories: (i) Governmentcosts, which are incurred by the official agencies that consider, evaluate and possibly provide funds for the proposed project; (ii) Utility costs, which are incurred by the implementation agency (and associated trade allies or non-profit groups) that executes, monitors, and possibly conductsongoing or ex-post evaluationsof the project; and (iii) End-user costs, which are incurred by the end-users, who are the beneficiaries of the project, but who may also have to bear some costs. 6.9 Appraisals and evaluations of energy conservation programs in North America and European have frequently failed to consider the full panoply of costs, even when a conscious effort has been made to be comprehensive. Based on this experience, there is likely to be significant underestimation of all three categories of costs inn potential GEF projects. 6.10 Governmentcosts may be underestimatedsincethey are frequently of theopportunitycost variety, based on the staff time and effort involved. In particular, the cost of evaluating the novel elements of GEF energy projects may prove to be high for countries that have limited expertise in this regard. 6.11 Typically, in addition to direct expenditure costs, such as on purchasing CFLs or other equipment, the utility will also incur indirect costs that may be in the form of explicit expenditures or opportunity costs. Based on a sample of ten utilities in the U.S.,Joskow and DRAFT Marron (1992) concluded that many types of administrative corn, including measurement and evaluation of conservation savingsand overhead, are not universally tracked and reported. The failureto account for such costs can lead to significant underestimatesof the truecosts of utility- sponsored conservation initiatives. 6.12 Even though the end-users are usually the beneficiaries of utility conservation programs, they may have to bear transaction costs involved in considering and evaluating whether or not and to what extent to participate in conservation programs. In some instance, they may also have to bear part of the cost of acquiring new hardware or the opportunitycost of "premature" retirement of hardware, e.g., the replaced conventionalbulbs may have some residual value that may be lost when CFLs are installed. Small and medium-sized firms may also experience similar difficulties. 6.13 In principle, it should be relatively straightforward to verify that all three categories of costs have been taken into account in the cost-benefit analysis of GEF projects. HQwever,in practice, there may be difficulties in amving at reasonable estimates of costs, and dome costs may be overlooked completely. Hence, actual social costs of energy conservation piojects are likely to be under-estimated, unless special care has been taken to ensure that this is not the case. C. Flaws in estimating benefits of energy conservation projects 6.14 Based on the experience in North America, it is a common theme that energy savings, which are the benefits of the energy conservation programs, have been frequently and significantlyoverestimated in theex-anteappraisalsof these programs. In any case, the "actual" savings from an energy conservation project are often to difficultto calculateprecisely because they are equal to an actual post-installationconsumption subtracted from a hypothetical baseline consumption that would have occurred had the program not been in place, and all other factors had held constant. 6.15 There are a number of factors that lead to ex-postenergy savings which are leSs than the ex-ante anticipated savings: (i) improper definition of program impact; (ii) lower thah expected participation rates; (iii) "Free riders" and "takeback"effects;and (iv) equipment failuie, misuse and lack of persistence of effects. 6.16 The most meaningful measure of the effect of a utility sponsored program is the net program impact which considers the actions of the participants in the program with respect to what would have happened if the utility-sponsored program had not come into existence. However, it is possible that the economic appraisal conducted for the energy conservation programs being considered by GEF is based on one of the other definitions of the impact, such as the maximum technical potential, which measures the impact of a 100% penetration of the most efficient technologies. 6.17 The participation rate, i.e., the ratio of eligible customers who actually participate in a utility sponsored program, is one of the key determinants of the benefits from energy conservation programs. The actual participation rate may be lower than the projected participation rate for a number of reasons: (i) inappropriatediscount rate; (ii) transaction costs, other priorities, and organizational problems; and (iii) ineffective promotional campaigns. 6.18 It is common to use a real discount rate of about 10% in cost-benefit analysis of development projects. However, the available evidence indicates that consumers, particularly low-income consumers, use much higher discount rates in evaluating energy conservation programs. If a high real discount rate is used for end-use consumers and a lower real discount rate is used for the utility, a particular project may turn out to be beneficialfrom the perspective of the utility but not from the perspective of the consumers. Thus, the use of an inappropriately low discount rate may lead to optimistic estimates of the participation rate. In response to the use of high discount rates and consequent low participation rates, the U.S. has set minimum efficiency standards for a number of appliances, which forces consumers to purchase only relatively energy-efficientappliances. 6.19 The decision to participatein utility sponsored programs also depend upon other factors that such as the transactions costs, the end-users' other priorities, and their organizational structure. Further, the promotional campaign instituted by the utility may prove to be ineffective, particularly in situations where the utility.lacks experience in such promotions. These factors may be overlooked in projecting the participation rate, thus leading to optimistic estimates. 6.20 In thecontextof utility-sponsoredprojects, "free riders" are participants who would have undertaken the proposed measures even without a utility-sponsored program. For example, customers who would have bought and installed CFLson their own but take advantageof utility incentives to buy them are free riders. If no account is taken of free riders, then the actions of the free riders are ascribed to the utility program. While the presence of free riders has been a concern in the U.S., it has not been so in European lighting programs. Given the novelty of the technology and devices being promoted by GEF energy conservation programs, free riders are unlikely to be a major concern for GEF projects. In contrast, GEF programs may induce "free drivers," who are customers who do not participate in a utility sponsor program but are influenced by the program and adopt the program recommendations. Thus, free drivers represent an additional benefit that may not be accounted for in the ex-ante appraisal of DSM programs. 6.21 In the context of energy conservation programs, "takeback" refers to the change in energy-related operating practices of a firm or households as a consequence of participating in a DSM program. For example, a household may increase its use of an airconditioner after a utility helps pay for a new, more efficient unit, so that actual energy savings may be less than anticipated. While substantial takeback effectshave not been observed in the U.S.and Europe, this may be the case in developing countries, particularly where low-income consumers are involved. DRAFT 6.22 The history of energy projects in the developing countries, particularly those 011 the supply side such as generation, transmission, and distribution facilities, indicates that there is a potential for significant problems in the installation, proper use, and maintenance of energy efficient technologiesand devices, particularlyfor those with which the local people have limited familiarity and experience. Thus, even if there are initial savings from energy conservation projects, these have the potential of declining steadily over time. It would be prudent for GEF to verify whether the proposed project has taken account of this potential decline in benefitsover time, or taken steps to prevent such a decline. D. Uncertainty, risk, and decision rules 6.23 The historical experienceof World Bank projectsas well as energy conservation projects in the U.S.and Europe shows that, apart from any possible systematic bias, there remain substantial differencesbetween actual and realized rates of return. Thus, even after the biases in ex-ante estimates of costs and benefits of energy conservation projects have been duced or eliminated, there is likely to remain substantial uncertainty about the actual net benefits of energy conservation projects that GEF may support. For GEF energy-related projects an additional reason to expect divergence between estimated and actual net benefits is that GEF projects have an element of novelty, which implies that there is limited experience oh which to base theapproximations,assumptionsand rules-of-thumb usually required in projectcost-benefit analysis. 6.24 One relevant implication of the uncertainty associated with ex-ante appraisals is that any decision-maker, i.e., the Governmentor GEF, who uses theex-ante appraisal to make a decision about an investment project faces with significant probabilitiesof getting "false-positive" and/or "false-negative" results. For example, the Government may end-up undertaking projects that do not deliver the expected results ("false-positive") and/or it may fail to undertake projects that would have brought substantial benefits to the country ("false-negative"). 6.25 While there is an extensiveliteratureon risk and uncertainty, even well-known theoretical concepts have not been extensively incorporated into project appraisals. At the sanle time, it is also clear that many of the theoretical results available in the literature are not sufficiently practical to be readily applied in the appraisal of projects in developing countries; for example, much of the theoretical discussion relates to situations in which the uncertain outcofne can be treated as a random variable with a probability distribution. For example, the distinction between "pure risk," under which there is a possibility of unexpected adverse as well favorable events, and "downside risk," which focuses on unexpected adverse events only has only recently been formally explored, and no definitive theoretical or practical results are available yet. 6.26 In contrast, the concept of pure risk is well-developedand widely-accepted, but it has not commonly been used in project appraisals, in part because it is difficult to provide reasonable subjective estimates of the probabilities that are required by this approach. Nevertheless, some of the theoretical results of the capital asset pricing model (CAPM), based on an analysis of pure risk, are relevant for project appraisal. In particular, in the context of efficient markets, the - 42 - DRAFT CAPM shows that it is not meaningful to consider the risk of an asset on its own; instead, rational individuals evaluate the risk of an asset in the context of an entire portfolioof assets. The natural implication is that it would be useful to evaluate the riskiness of a particular project in the context of the entire portfolio of projects being undertaken by a country. Government response to uncertainty 6.27 In view of the biases and uncertainty associated with the conventional a-ante appraisal of net domestic benefits, governments may be reluctant to rely solely on such appraisals to make decisionsabout undertaking energy-efficiency projects. It is likely that different Governments would have different responses to the uncertainty associated with GEF projects. One response of the decision-maker may be to conduct a heuristic, back-of-the-envelopeanalysis to determine "low-case" or "worst-case" outcomes. An extremely risk-averse decision-maker may wish to be sure that the worst-case scenario associated with the project is acceptable. Or, a risk-averse decision-maker may approve only those projects whose "low-case" estimate of the net domestic benefits exceeds a particular value. Even if no analysis is conducted to determine "low-case" outcomes, a Government may be willing to undertake only those energy conservation projects that require low initial capital expenditures, so that the project can be terminated at relatively low cost if there are early indications that the project will fail to achieve its projects benefits. GEF's response to uncertainty 6.28 Apart from the uncertainty associated with projected net domestic benefits, GEF also has to consider the uncertainties associated the decision-making rules used by Governments. It is clear that a host country may not undertake a project even if GEF classifies it as Type I, in which case the global environmental benefits associated with it will not be realized. 6.29 If GEF takes account of the Government's decision-rule in formulating GEF's own decision-making rule about whether or not to support a particular project, this would lead to a moral hazard, because it provides an incentives for Governments to adopt a stated policy of "We will not undertake these types of projects on our own" merely in order to secure GEF support. Thus, GEF will run the risk of supporting projects that do not need GEF support. 6.30 On the other hand, if GEF does not take account of the Governments' decision-rules, and continuesto classify projectsas Type I and Type I1 based on the appraised net domestic benefits, then GEF faces two problems. First, until the project appraisal methodology is modified to take account of the failings, biases and risks indicated by the Wapenhans report and the experience with energy conservation projects in industrialized countries, the appraised net benefits are seriously flawed, and GEF would run the risk of both "false positive" and "false negative" results, where "false negative" represents failing to support projects that require GEF support. Second, even if the methodology is modified to eliminate or reduce the above problems, a decision rule based solely on the (correctly) projected benefits will fail to take account of the potentially different responses of different Governments to similar projects, i.e., some Governments may be more risk-averse than others. Therefore, GEF may face significant risks of getting "false negative" results. 6.31 Mathematically, it is not possible for GEF to develop a decision-making rule that simultaneously minimizes the probabilitiesof both "false positive" and "false negative" results. Therefore, in deciding whether to focus on "false negative" or "false positive" result$,GEF will have to take account of the consequences of these types of results. If substatid global environmental benefits are at stake, then GEF may be womed about denying a small amount of support to project and taking the risk that the project may never be undertaken, i.e., the focus would be on minimizing "false negative" results. Alternatively, if substantial GEF funding is required, then GEF may be womed about "false positive" results. The implication is that GEF has to develop its own risk profile, and develop decision-making rules that take account of the amount of GEF funding at stake and the estimated global environmental benefits. - 44 - REFERENCES Anderson, Jock R. and John C. Quiggin, Uncertainty in Project Appraisal, Paper prepared for the World Bank Annual Conference on Development Economics, 1990. Bates, Robin W., Energy Conservation Policy, Energy Markets and the Environment in Developing Countries, Environment Working Paper No. 45, World Bank, 1991. Bartlett, Sarita, "Shedding Light on Residential Consumers," Energy, Vol. 18, No. 2, 1993. Berry, Linda, The Administrative Costs of Energy Conservation Programs, ORNLICON-294, Oak Ridge National Laboratory, Oak Ridge, TN, 1989. Besant-Jones, John E. and Lori Hylan, ed., Managing Risks of Investments in Developing Countries, Industry and Energy Department energy series paper no. 55, World Bank, 1992. Caner, Phoebe, "The Drive to Verify Energy Savings," ElectricityJournal, Vol. 5, No. 4, 1992. Cebon, Peter B., "'Twixt cup and lip: Organizational behavior, technical prediction and conservation practice," Energy Policy, 1992. Brown, Ian, "Energy Service Company Investments in Energy-efficientLighting in Hungary," Energy, Vol. 18, No. 2, 1993. Busch, John F., Peter du Pont, Ad Surapong Chirarattananon, "Energy-efficientlightingin Thai commercial buildings," Energy, Vol. 18, No. 2, 1993. Crousillat, Enrique O., Incorporating risk and uncertainty in power system planning, Industry and Energy Department energy series paper no. 17, World Bank, 1989. Crousillat, Enrique, and Hyde Merrill, The trade- of/risk method :a strategic approach to power planning, Industry and Energy Department energy series paper no. 54, World Bank, 1992. Dubin, Jeffrey A., Market Barriers to Conservation: Are Implicit Discount Rates Too High?, Social Science Working Paper No. 802, Division of the Humanities and Social Sciences, California Institute of Technology, July 1992. Gellings, Clark W., Ahmad Faruqui, and Ken Seiden, "Potential energy savings from efficient electric technologies," Energy Policy, Vol. 19, No. 3, 1991. Graham, Daniel A., "Cost-Benefit Analysis under Uncertainty," American Economic Review, Vol. 71, No. 4, 1981 Graham, Daniel A., "Cost-Benefit Analysis under Uncertainty: Reply," American Economic Review, Vol. 74, No. 5, 1984 Gruber, Edelgard and Michael Brand, "Promoting energy conservation in small and medium- sized companies," Energy Policy, Vol. 19, No. 3, 1991. Hamlin, L. D., "Energy efficiency: the future business opportunity for electric utilities," Electricity Journal, Vol. 3, No. 7,1990. Hartmann, Raymond S. and Michael J. Doane, "Household Discount Rates Revisited," Energy Journal, Vol. 7, No. 1, 1986. Hassett, Kevin A. and Gilbert E. Metcalf, "Energy conservation investment: Do consumers discount the future correctly?" Energy Policy, Vol. 21, No. 6, June 1993. Hirst, Eric and Carol Sabo, "Defining and reporting data on utility DSM ~ro~rams,~"Energy, Vol. 17, No. 7, 1992. Johnson, Karl and Erich Unterwurzacher, "Ensuring market supply and penetration of efficient lighting technologies," Energy, Vol. 18, No. 2, 1993 Joskow, Paul L. and Donald B. Marron, "What Does a Negawatt Really Cost? ~vidbncefrom Utility Conservation Programs," Energy Journal, Vol. 13 No. 4 1992. Joskow, Paul L. and Donald B. Marron, "What Doesa NegawattReally Cost? FurthedThoughts and Evidence," Electricity Jouml, Vol. 6, No. 6, 1993. I Krause, F. and Joseph Eto, TheDemandSide: Conceptualand MethodologicalIssues, Least-cost Utility Planning Handbook for Public Utility Commissioners, Volume 21 National Association of Regulatory Utility Commissioners (NARUC), Washington, DIC., 1988 Lind, Robert C., Discountingfor time and risk in energy policy, Washington, D.C.: ~esources for the Future, 1982 I McInnes, Genevieveand Erich Unterwurzacher, "Electricity end-useefficiency," ~ n ePolicy, ~ r ~ Vol. 19, No. 3, 1991. I I Mendelsohn, Robert and William J. Strang, "Cost-Benefit Analysis under Ubcertainty: Comment," American Economic Review, Vol. 74, No. 5, 1984 I Mills, Evan, "Evaluation of European lighting programmes," Energy Policy, Vol. 19, No. 3, 1991. Mills, Evan, "Efficient Lighting Programs in Europe: Cost Effectiveness, Consumer Response, and Market Dynamics," Energy, Vol. 18, No. 2, 1993. Nadel, Steven M. and Kenneth M. Keating (1991), "Engineering Estimates versus Impact Evaluation Results: How Do They Compare and Why," in Energy Program Evaluation: Uses Methods and Results, Proceedings of the 1991 International energy Program Evaluation Conference. Nadel, Steven M., Barbara A. Atkinson and James E. McMahon, "A Review of U.S. and Canadian Lighting Programs for the Residential, Commercial and Industrial Sectors," Energy, Vol. 18, No. 2, 1993. Ouederni, Bechir N. and William G. Sullivan, "A Semi-varianceModel for IncorporatingRisk into Capital Investment Analysis," Engineering Economist, Vol. 36, No. 2, 1991. Pohl, Gerhard and Dubravko Mihaljek, "Project Evaluation and Uncertainty in Practice: A Statistical Analysis of Rate of Return Divergences of 1,015 World Bank Projects," World Bank Economics Review, Vol. 6, No. 2, 1992. Pouliquen, Louis Y., Risk Analysis in Project Appraisal, World Bank Staff Occasional Paper Number 11, Johns Hopkins University Press, Baltimore, 1970. Prindle, William R., "Demand-side management in the 1990s: Time to come of age," Energy Policy, Vol. 19, No. 3, 1991. Rave, Klaus, 'A Contract for ~fficientLighting in German Public Buildings," Energy, Vol. 18, No. 2, 1993. Reutlinger, Shlomo, Techniquesfor Project Appraisal under Uncertainty, World Bank Staff Occasional Paper Number 10, Johns Hopkins University Press, Baltimore, 1970. Schipper,Lee and Dianne V. Hawk, "More efficienthousehold electricity-use: An international perspective," Energy Policy, Vol. 19, No. 3, 1991. Simpson, David and James Walker, "Extending cost-benefit analysis for energy investment choices," Energy Policy, Vol. 15, No. 3, 1987. Sioshansi, Fereidoon P., "The myths and facts of energy efficiency: Survey of implementation issues," Energy Policy, Vol. 19, No. 3, 1991. Sutherland, Ronald J., "Market Barriers to Energy-Efficient Investments,"Energy Journal, Vol 12, No. 3, 1991. - 47 - DRAFT Train, Kenneth, "Discount rates in consumers' energy-related decisions: A review of the literature," Energy, Vol. 10, No. 12,.1985; Vine, Edward L., "Free Rider Estimation: Refining the Use of Surveys," Energy, Vol. 17, No. 10, 1992. Vine, Edward L., "Persistence of Energy Savings: What do we know and how can it be ensured?" Energy, Vol. 17, No. 11, 1992. Wapenhans, Willi A. (World Bank Portfolio ~ana~ementTask Force), Efictive Implementation: Key to Development Impact, World Bank, 1992. World Bank, EconomicAnalysis of Projects: Towards an Approach to Evaluationfbr the 1990s, Warld Bank, 1992. World Bank, Getting Results: The World Bank's Agenda for Improving Development Efectiveness, Warld Bank, 1993. DRAFT Some Draft Notes on Light-bulb programmes 6.32 According to Mills (1991), the residential programmes have had a significant impact on national lamp sales, but program participation rates show no correlation with the incentivelevel. The participationrate is the product of a number of factors, including effectivenessof promotion strategies, type of incentive, and restrictions in some cases on the number of lamps allowed to each household. The non-residential programmes should be more cost-effective than residential programmes. This is partly because of the need to contact fewer customers, plus the delivery of more lamps/customer means lower administrativecosts. 6.33 The average societal costs of conserved energy is C2.l/kWh, including indirect costs of C0.3IkWh. The payback time to participating households ranged from 0 years (free lamps) to three years. For the programmesdescribed in this article, administrativeand other "transaction" Costs contributed CO.3lkWh ($l/lamp) to the total cost of conserved energy. 6.34 According to Mills (1993), between late 1987 and 1992, at least 52 financial-incentive programs for CFLs were implemented in 11 West European countries, including U.K. and Ireland. The 7.4 million households eligible for the program received 2.5 million CFLs. Program target groups ranged from a few thousand households to several hundred thousand. Data on costs for 40 of these programs from 8 W. European countries (Sweden, Denmark, France, Netherlands, Finland, Ireland, W. Germany, and Austria). For these programs, the average cost is 2.1 centslkwh, of which 0.3 cents is indirect cost, where indirect means non- equipment cost. The average price paid by program participants was $ 111CFL. Non- participants also benefited because increases in lamp sales prompted manufacturers to lower prices. For example, in Sweden, the 75,000 rebate checks led to an additional "leveraged" 41,000 sales. 6.35 Mills (1991) found that lower energy costs were cited by only half the participantsas the reason for participatingin the European CFL programs. Trying a new technology and using the rebate check were the other frequently cited reasons. Non-participants reported a number of reasons for not participating: (i) general lack of interest, (ii) excessive lamp prices, (iii) non- awareness of the program, (iv) lamp sizelweight was excessive. 6.36 Information: Exhibitions, open houses and other low-effort approaches have yielded minimal impacts compared to programs offering financial incentives. Nonetheless, future programs should address the problem that consumers often have inaccurate or inadequate information on CFLs. 6.37 According to Mills (1993), the choice of light sources and the markets for energy- efficient lighting have changed dramatically in recent decades, and is expected to continue. Global CFL sales were 114 millionlyearin 1991, and are expected to reach 250 millionlyear by 1995; global incandescent sales per year were over 9 billion in 1991. One reason for the rapid increase is that an increasing number of parties that are not traditionally involved in promoting efficientlighting (utilities,government, public interest groups, others) are actively participating. The key parties have been electric utilities, while lighting retailers in some cases shared in marketing and providing consumer information. In one case (Sweden), a government body designed and financed lighting programs carried out by the utilities. 6.38 The major drawback for the accelerated market penetration of CFLs is their high initial costs, combined with information barriers regarding their cost-effectiveness. In both the residential and commercial sectors, the lack of information and capital, the reluctance to adopt unfamiliar technologies, and only moderate interest in energy costs and in reducing expenses continue to hamper the widespread introduction of energy-efficient technologies. 6.39 According to Brown (1993), a fundamental barrier to achieving cost-effective lighting efficiency in W. Europe is the lack of investment capital for such technologies. This problem is magnified in Eastern Europe; this problem is present in Hungary, even though it is in the fortunate position of having the manufacturing capacity for efficient products such as CFLs. One possible source of capital is energy service companies (ESCOs), which can provide third- party financing. These ESCOs should get their funds from western ESCOs which have the capital as well as the expertise.. 6.40 According to Busch et al (1993), energy-efficientlighting would be very useful in Thai commercial buildings, because electricity use in these buildings is growing rapidly, i.e. number of buildings is growing rapidly. The authors use a 6% real discount rate and a 20-year time horizon, though not all components are assumed to have this life. The authors find that for offices the average electricity price is $0.087, while the CCE (cost of conserved energy) for the full conservation measure is $ 0.019. The simple payback period for full lighting conservation measures ranges from less than.one year in hotels and retail buildings to about three years in offices. The IRR of installingall the lighting measures is 35% for offices, 142%for hotels, and 107% for shopping centers. CFLs, electronic ballasts, and triphosphor narrow-diameter (T8) lamps prove to be the most economically promising technologies. 6.41 The above figures for Thai commercial buildings are from a societal perspective. The individual benefits may be less; for example, individual building owners have to pay import duties on equipment, which are not included as costs in the societal-perspective. Also, the cost of money may be more than 6%. Nevertheless, even if you use actual market prices, and a discount rate of 12%, investment in efficientlighting remains cost-effective. Nevertheless, such efficient systems remain the exception in Thailand. Part of the problem has been a lack of information about the options, their savings, and the life-cycle costs associated with their use. Further, in Thailand's very competitive market for commercial space, building owners and developers are reluctant to consider any measures that will increase initial costs. 6.42 The Government of Schleswig-Hosltein, a state in Germany, undertook to replace all lamps in public buildings with CFLs. It was estimated that about 600,000 conventional incandescent lamps could be exchanged. To give the programme publicity, the first bulbs were replaced by the state Minister of Energy. However, the goal of 600,OO was overambitious. Under the guidelines of "Phase I" of the contract, about 77 thousand lamps have been installed. - 50 - DRAFT 6.43 Based on an analysis of the data for several European countries (Belgium, Sweden, Norway, Denmark), Bartlett (1993) concluded that "estimates of the electricity used for residential lighting in most countries are subject to large measurement error ... Therefore, the potential annual electricity savings from the use of CFLs cannot be accurately estimated." 6.44 Further, unlike U.S. residential programs, European programs have not been targeted at any particular group, which may be a serious flaw of the European programs. In addition, there may be some loss of energy savings if people install CFLs in low-usage areas; 6% of the CFLs purchased during NESA's campaign in Denmark were placed in vacation homes perhaps because the publicity materials did not clearly specify that they should be installed in high-use ateas. 6.45 Energy consumers do not necessarily volunteer to participate in energy efficiency improvements that appear (at least on paper) to be cost-effective. A variety of motivational, information, and risk-avoidance factors are blamed for the divergence between what is theoretically a sound investmentand what is observed in practice. Over the years, many lessons have been learned on how to overcomeconsumers' initial apathy to energy efficiency, and lead them to the desired investment. Utilitiesenjoy substantial economies of scale in obtaining and disseminating information to consumers. 6.46 Joskow and Marron (1993) indicate that the results of their 1992 study should not be interpreted as saying that utility DSM programs are cost-effective. "It would be imprudent to rush to this judgement without examining further the quality of the cost and energy savings information reported by utilities. Our analysis suggests that utilities often understate program costs and overstate program energy savings; as a result reported costs of saved energy are often too low." 6.47 In the U.S., consumers can choose between similar appliances that differ principally in energy efficiency. Why does consumer behavior diverge from an economically rational behavior? This was looked at by LBL (Krause and Eto for NARUC). Factors that avplv when consumeris about to buy. First, the trade-offs between efficiencyand higher first cost were not clear for some appliances: an efficient appliance may lack some features that another less efficient one has. Second, the more efficient appliances may not be available in stores. Third, the rationalconsumer may lack the information about the costs and benefitsof energy efficiency, and the transaction costs of obtaining this information may be high. Fourth, the rational consumer may not have the capital for investment or may not feel financially secure to make investments with a large payback period. Fifth, the monetary savings are small both in absolute terms as well as in terms of percentage income. Other factors. Sixth, the consumer's purchase decisions are heavily influenced by non-cost characteristics, such as noise level, colour, etc. Seventh, the landlord or contractor may be the buyer, and not the actual user. Conclusion: The market demand for efficiency investments in new appliances is very weak. When electricity