Report No. 8164 Annual Review of Evaluation Results October 30,1989 Operations Evaluation Department FOR OFFICIAL USE ONLY C) - Document of the World Bank This document has a restricted distribution and r-4y be used by recipients only in the performance of their official duties. Its contents may not otherwise A be disclosed without World Bank authorization. FORA AL . Y THE WORLD BANK WasNngtn, D.C. 20433 U.SA YVES ROVANI Obectw-Gener Operadons EWvauon October 30, 1989 MEMORANDUM TO THE EXECUTIVE DIRETORS AND THE PRESIDENT Sulject: Annual Review of Evaluation Results for 1988 Attached, for information, is a copy of a report entitled "Annual Review of Evaluation Results for 1988", prepared by the Operations Evaluation Department. Attachment This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed without World Bank authoriation. AL ,Y ANNUAL REVIEW OF EVALUATION RESULTS 1988 October 30,1989 Operations Evaluation Department Th,- document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed without World Bank authorization. Abbreviations and Acronyms AGR - Agriculture Department AR - Annual Review of Project Performance ARIS - Annual Review of Implementation and Supervision DFI - Development Finance Institution DFC - Development Finance Corporation EEMNA - Europe, Middle East and North Africa Region ERR - Economic rate of return ESW - Economic and sector work GDP - Gross domestic product IDA - International Development Association LAC - LatIn America and the Caribbean Region OD - Operational Directive OED - Operations Evaluation Department OMS - Operational Manual Statement OPS - Operations Policy Staff PCR - Project Completion Report PMS - Public Sector Managemev* PPAR - Project Performance Audit Report SAL - Structural Adjustment Loan SECAL - Sector Adjustment Loan TA - Technical assistance PREFACE Operations evaluation in the Bank provides a systematic, compre- hensive, and independent review of the Bank's development experience. While the focus is on the Bank's performance in the pLeparation and Imple- mentation of the operations it finances, the implementation capacity and overall policies of the borrowing countries are equally, if not more, important in the successful outcome of Bank-ftirnced operations. This year's review of evaluation results is the fifteenth in the annual series. Like its predecessors, it is based on project completion reports prepared by the Bank's operational staff and on performance audits and studies prepared by the Bank's Operations Evaluation Department (OED). Borrower countries participate in preparing project completion reports and review the audits of performance. The review looks at the Bank's experience over an extended period, taking as the basis for analysis all the Bank-supported operations that have ever been evaluated. It thus reports on nearly 2,000 operations, evaluated between 1974 and 1988. Concentrating particularly on the operations approved in the period 1968-80, it analyzes trends over time by grouping operations according to the year in which they were approved by the Bank's Board (rather than, as in past reviews, according to the year in which they happened to be evaluated). The Bank evaluates the performance of all of its operations at the completion of the implementation phase, shortly after final disbursements. This evaluation compares outcomes at that point in time with expectations at appraisal. It is an opportune Lime to account for the use of resources, assess the extent to which operations achieve their stated objectives, and draw valuable lessons for the future. However, this timing and the methodology used have limitations. While at completion the resources Invested, and the correpponding costs, are more fully known than at appraisal, the eventual lenefits and costs throughout the life of the operation are still unknown. The evaluation process tries to re-estimate benefits and costs, taking account of changes that have taken place in the operation and its environment, and the experience gained since appraisal. Since in most operations the ecoromic activities only mature or develop fully after the Bank has completed its disbursements, the evaluation cannot fully take into account the learning process and adaptation that takes place during this phase. Impact evaluation, done for a small proportion of operations, typically about five years after completion, sheds more light on their developmental impact. In this review OED continues the practice of judging operations satisfactory or unsatisfactory. "Satisfactory" performance denotes the achievement of an adequate economic rate of return in those types of operations for which this measure is applicable, or, for other types of operations, the substantial achievement of the goals stated at appraisal, especially with respect to progress with institution-building and the extent of time- and cost-overruns. The review shows that a significant proportion of projects with a rate of return, at completion, of below 10 percent (normally, though nt in all cases, the cut-off rate below which projects are considered unsatisfactory) may still generate important nonquantifiable benefits and thus be worthwhile to the economies that have undertaken them. To complement its assessments of performance, this review discusses the austainability of completed operations. OED first reported on its assessments of sustainability in the Annual Review of Project Performance Results, 1984. This year's review discusses the most important determinants of sustainability, including macroeconomic factors, and identifies differences in these determinants across sectors. Prospects for sustainability, like operational performance, are assessed at the time the Bank completes its disbursements. At this time OED staff provide a judgment of the ability of operations to maintain an edequate level of net benefits after the investment phase is completed. Inevitably their judgments on sastainability, like those on performance, may need reassessment as an operation and its domestic and international environment evolves. Since the issues surrounding sustainability are multidimensional and diverse, and several aspects of the assessments made are qualitative, not quantitative, the judgments as to sustainability presented in the review (i.e., "likely", "marginal*, "uncertain" and "unlikely") must be carefully interpreted and used. Sustainability is judged "uncertain" where the key factors involved are particularly hard to predict--for example, where an operation faces a particularly uncertain market or policy environment, or where the institution charged with implementation is young and fragile. Being based on judgments made about completed operations, the findings presented in this year's review do not necessarily indicate the status of the Bank's current portfolio. The review does not take stock of all the actions and programs the Bank has implemented in the last decade; indeed, several programs have been d6signed to correct some of the problems discussed here and to change, where appropriate, the directions of investment allocation and policy reforms. The findings do, however, identify important sectoral and regional patterns which help to enrich understanding of the development process. Yves Rovani Director General Operations Evaluation - Mi - TABLE OF CONTENTS Page EECUTIE SRfARY ............................................. .. v 1. INTRODUCTION .......... . ........... 1 A. Outline of Review ......................... 2 2. PERORMNCE8 AN OERMIEW FREfu FIFTEEN YRS OF 3VALUATION ..... 5 A. Performance Analysis ................................... 6 B. Performance of IDA-financed Operations .................... 12 C. Economic Returns ......**...*........................... 13 D. Cost and Time Discrepancies ............................... 16 E. Operations Evaluated in 1988 .............................. 16 3. DETERNINANTS OF PERF***ANCE ..*.............................. 19 A. Operations with Unsatisfactory Performance ............... 20 B. Operations with ERRs Delow Zero ........................... 21 C. Performance in African Agriculture ........................ 22 D. Conclusions, Future Directions ............................ 23 4. SUSTAIABILITT OF DEVELOPMNT OPERATIONS ...................... 27 A. Analyzing Sustainability .................................. 27 B. Sustainability of Bank Operations ......................... 30 C. Conclusions ........................ 35 5. FINANCIAL ASPECTS OF SUSTAMBILITT .......................... 39 A. Financial Sustainability of Projects ...................... 40 B. Financial Linka Between Projects and the Macroeconomy...... 43 C* Conclusions ...................................... ... 47 6. MANAGING NATURAL RESOURCES, STRENGTBENING INSTITUTIONS: TWO ASPECTS OF SUSTAINABILIT .......................... 53 A. The Bank and the Management of Natural Resources .......... 53 B. Institutional Development .............................. 57 - iv - TEET TABLES 2.1 Distribution of Operations Evaluated in 1974-88, by Sector ad Region .******************.*******.********************* 7 2.2 Performance Evaluations for Operations Evaluated in 1974-88 ... 8 2.3 Operations with Unsatisfactory Performance as Percent of All Operations, by Region, 1968-80 ........................ 10 2.4 Investment in Operations with Unsatisfactory Performance, as Percent of Regional Investment Totals, 1968-80 ......... 10 2.5 Amounts Invested per Operation with Unsatisfactory Performance, 1968-80 ............................... 11 2.6 Performance Evaluations for IDA Operations Evaluated during 1974-88 .......................... 13 2.7 Re-estimated ERRs of Evaluated Operations, by Sector and Region, 1968-80 ...................... 14 2.8 Performance Evaluations for Operations Evaluated in 1988....... 17 3.1 Distribution of SatisfactorylUnsatisfactory Operations by Region, 1974-88 ........................................ 19 3.2 Distribution of a Group of Unsatisfactory Operations by Re-estimated Economic Rates of Return, 1974-88 ............ 20 4.1 Sustainability Analysiss Number of Operations Analyzed, by Sector of Economic Activity ............................ 31 4.2 Sustainability of Operations: Regional and Sectoral BreakdownS ........................ 31 FIGURES 1.1 World Bank Commitments, FT 1969-87 ............................ 4 1.2 World Bank Comitments, by Region, FY 1969-87 ................. 4 2.1 Gap Between Appraisal and Re-estimated Economic Rates of Return, by Year of Evaluation, 1974-88 .................... 15 2.2 Gap Between Appraisal and Re-estimated Economic Rates of Return, by Year of Approval, 1968-80 ...................... 15 2.3 Operations Evaluated in 1988, by Year of Approval and Closing ................................................... 16 BOXES 1.1 The Bank in the 1970s ......................................... 3 3.1 African Agriculture: Influences on the Bank's Performance ..... 25 4.1 Sustainability in Education Projects .......................... 38 5.1 Some Common Financial Problems ................................ 50 5.2 Sustainability of Development Finance Institutions ............ 51 6.1 The Bank's Role in Forestry Development ....................... 61 6.2 Some Findings fram Impact Evaluation Reports .................. 62 6.3 The Bank's Involvement in Renewable Resource Management *...... 63 EXECUTIVE SUMMARY 1. This year's review of evaluation results Is the fifteenth in the annual series. Like its predecessors, it is based on project completion reports prepared by the Bank's operational staff and on performance audits and studies prepared by the Bank's Operations Evaluat.on Department (OED).11 While the focus continues to be on the Bank's performance in the preparation and implementation of the operations it finances, the implementation capacity and the overall policies of borrowing countries are equally if not more important in the successful outcome of Bank-financed operations. 2. In tais year's review the approach has evolved in two main respects. First, it takes advantage of improvements in OED's data base to attempt a comprehensive look at how the Bank '2as performed iis mission over an extended period. Second, the review is structured to allow a more detailed and substantive discussion of a group of selected topics. It concentrates on the factors that affect the performance of development operations and on the 4.nfluences that govern the saustainability of these operations. 3. The evaluation of performance of all operations is carried out at the completion of the implementation phase, shortly after final disbursements. This evaluation compares outcomes at that point in time with expectations at appraisal. It is an opportune time to account for the use of resources, assess the extent to which the operations achieved their stated objectives, and draw lessons for the future. However, this timing has limitations since in most operations the economic activities only mature or develop fully after the Bank has completed disbursements. Evaluation cannot therefore take into account the learning process and adaptation that actually takes place during this phase. Impact evaluation, done for a small proportion of operations, typically about five years after completion, sheds more light on their actual and prospective development impact. 4. Most of the operations reviewed in this report were designed and approved in the 1970s. For the Bank, this was a decade both of ambitious growth in lending, from $2.2 billion in 1970 to $11.5 billion in 1980, and of far-reaching policy change. Compared with the previous decade, the economic environment for development in the 1970s was far more changeable and difficult to predict. Many African countries were racked by drought and civil strife, bringing severe food shortages. For many oil-importing developing countries, within and oi-ide Africa, high oil prices, declining terms of trade for commodity producers importing manufactured goods, and the slowdown in export growth gave rise to massive current account deficits. Low real interest rates and pressure to lend from markets flush 11 All references to the Bank in this review include the International Development Association (IDA), and references to loans include IDA credits, unless otherwise noted. - vi - with petrodollars pushed many developing countries into overborrowing. For those which had borrowed at floating rates, grave difficulties ensued when real interest rates rose substantially at the end of the decade. 5. The findings of the review of performance do not necessarily indicate the status of the Bank*s current portfolio, since they are based on judgments made about completed operations. This review does not take stock of all the actions and programs the Bank has implemented in the last decade. Several programs have been designed to correct some of the problems discussed here and to change, whenever appropriate, the directions of investment allocations and policy reforms. 6. The review presents and discusses the most important determinants of sustainability, to further the discussions on this important topic. Since the issues surrounding sustainability are multidimensional and diverse, and several aspects of the assessments made are qualitative, not quantitative, the judgments made on austainability must be carefully interpreted and used. Overview of the Bank's Performance (Chapters 2, 3) 7. One thousand nine hundred and eighty three operations have been evaluated to date. These operations were approved in the period 1958-84; they amounted to a total investment (from all sources) of $178 billion. For the decade of the 1970s, they are highly representative of the number and the distribution of all the operations approved by the Bank's Board. Most of the operations evaluated are projects, but 66 are program and policy-based loans, including 26 structural adjustment loans. 8. In this review OED continues the practice, used in last yearva review, of classifying operations into two performance categories: satisfactory and unsatisfactory. In certain types of projects, notably agriculture, industry and transport, a project economic rate of return (ERR) is calculated at project appraisal and then re-estimated upon project completion. In the case of public utilities, incremental revenues attributable to the project are used as a proxy for benefits in calculating a quasi-ERR. Any ERR is based on a comparison of costs and benefits, measured using shadow prices that indicate real values to the economy. Inherent in the costs and benefits are four complementary indicators of performance considered particularly important by the Bank: sustainability of benefits, progress with institution-building, and the extent of time- and cost-overruns. In the case of operations for which ERRs are not a suitable measure--some in the above categories, as well as human resources, structural adjustment loans (SALs), and technical assistance operations- -the performance assessment is based on a judgment as to the accomplishment of the goals stated at appraisal, giving particular attention to the four indicators just noted. - vil - 9. Projects judged to have adequate ERRs, where applicable, or to have met the other indicators for acceptable performAnce were rated as satisfactory. Those that did not were rated as unsatisfactory. The review shows that a eignificant proportion of projects with a re-eatimated ERR below 10 percent (normally, though not in all cases, used as the cut-off rate below which projects are considered unsatisfactory) may still generate Important nonquantifiable benefits and thus be worthwhile to the economies that have undertaken them. 10. Four fifths of the evaluated operations were rated as performing satisfactorily at the time the Bank completed its disbursements. Overall, the Bank's performance was best in Asia, followed by Europe, Middle East, and North Africa (EMNA), Latin America and the Caribbean (LAC), and Sub- Saharan Africa (Africa). 11. Perfotmance tended to be strong in most of the sectors where the Bank's portfolio was large--particularly pownr, highways, and water supply. 12. Agriculture--also with a large portfolio--was the sector with the highest incidence of unsatisfactory performance; nearly a third of agricultural operations were judged unsatisfactory, and in Africa in this sector 51 percent of operations were satisfactory. It is worth noting, however, that for agricultural operations the average rate of return was no lower than the average ERR for all operations; while a high proportion of agricultural operations were judged unzatisfactory, a high proportion also performed very well. 13. In education, the record is relative'y strong, but in population, health, and nutrition, only 60 percent of operations were judged satisfactory, and in technical assistance, only 71 percent were judged satisfactory. In oil and gas, all of the 25 operations, and in telecommunications, 95 percent of the 58 operations, were rated satisfactory. 14. Industry projects and development finance corporations (DFCs) performed very well in Asia and LAC. In highways (243 projects), more than 84 percent of the projects were rated satisfactory in every region, with especially good performance in EMENA and Asia. Though the numbers of urban development projects reviewed are relatively small (49 in total), these projects have tended to perform well, on average. 15. In policy and program lending, making up a group of 66 operations, 88 percent of the operations were satisfactory. In some of the SALs judged satisfactory, however, the record was mixed, in terms of the implementation of policy reforms and the economic performance achieved. Trends in Performances 1968-80 16. A trend analysis was made for the period 1968-80. Ninety percent, or 1,794, of the evaluated operations were approved during this period, and they constitute a representative sample of total Bank lending in the period. - vill - 17. This group of evaluated operations was analyzed according to the years in which the operations were approved. The grouping by year of approval proves to be a better basis for identifying trends and determinants of performance than OED's former approach of grouping operations according to the years in which they happened to be evaluated. Three trends in performance were analyzed: changes in the percentage of total operations rated unsatisfactory; trends in the percentage of total investment committed to unsatisfactory operations; and trends in the investment cost per unsatisfactory operation. 18. The percentage of operations performing unsatisfactorily tended to be higher than the average (of 19 percent for all regions throughout the period) among operations approved in 1974 and subsequently, and especially in 1978. In Asia, operations performed better than the average in almost all years; in EMERA and LAC, experience was more variable; and in Africa, except for one year, the percentage of operations with unsatisfactory performance was always higher than the average. 19. This analysis also shows that the yearly average investment in unsatisfactory operations, across all regions, increased in the latter part of the period (i.e., 1977-1980). The average amount Invested per unsatisfactory operation, across all regions in the 13 year period, was $82 million. Though unsatisfactory performance was most common in Africa (29 percent of the number of operations in the region), investment costs per unsatisfactory operation were much smaller in Africa than in the other three regions. Relative to the size of economies and per capita incomes, however, the amounts invested per unsatisfactory operation in Africa were large. IDA Operations 20. Of the 1,983 operations evaluated, 827 were financed by International Development Association (IDA) credits (in some instances, in a blend with IBRD loans). Seventy seven percent were judged satisfactory. 21. IDA operations generally performed as well as the population as a whole, except in agriculture, population, health, and nutrition, and technical assistance. Their performance was particularly good in power, oil and gas, and telecommunicati-ns. A large number of the IDA operations are in economies that consit;ently performed poorly during the 1970s. Relative to the other regions. performance was poor in Africa, where 32 percent of the IDA operations were judged unsatisfactory over the entire period. Economic Rates of Return 22. On 1,065 operations approved in 1968-80, the average economic rate of return as re-estimated at the time of completion was 16 percent. This average reflects the ERRs for individual operations weighted by their costs, and thus provides a good indication of the return on the total resources invested in this group of operations. -iI - 23. As reported in the past, there is a continuing large difference between forecast and re-estimated ERRs, both by year of approval and by year of evaluation. The analysis of trends in the gaps between the appraisal and completion ERRs for the 1968-80 period reveals some Itportant tendencie,;. First, in Africa the ERRs estimated at appraisal were always higher than at completion, and there remains a marked tendency to overestimate rates of return. Although this tendency is less prononced in other regions, the findings emphasize the importance of using realistic assmptions in the analysis of returns. Second, in those sectors where low returns are relatively common, there has been an increasing tendency to overestimate the ERRs at appraisal. This is particularly the case in agricultural and urban projects. 24. Given these tendencies, it is important for the Bank to make sure that its review of proposed projects and programs takes deliberate account of the assumptions underlying the forecast returni to such projects and programs. Where technical parameters (such as crop yields) and economic variables (such as prices) are particularly difficult to predict, experience has shown the virtues of providing for strong monitoring and evaluation units. On the basis of implementation experience as documented by these units, the design of operations can be modified once they are in progress. Determinants of Unsatisfactory Performance 25. The economic, social, and political environment of development is complex, not to speak of the civil strife experienced in some countries. Inevitably, some operations wil. have unsatisfactory outcomes. Neverthe- less, this review shown that, to a corsiderable extent, the outcome of operations depends on factors that are in principle within the control of borrower countries' policy makers during the implementation period. This underscores the central importance of the overall policies of borrowing countries, and their implementation capacity, in determining the outcome of Bank-financed operations. 26. An analysis was made of 238 operations whose performance was rated unsatisfactory on the basis of their economic rates of return as re- estimated at the time if their completion. The sectors represented in this group are agriculture, industry, transport and tourism, energy, public utilities, and urban operations. The results show that 30 percent of the unsatisfactory operations were still yielding returns very close to 10 percent. Of the remaining unsatisfactory operations, 34 percent had ERRs below 0 percent, 23 percent had ERRs between 0 and 4 percent, and 13 percent had ERRa of 5 and 6 percent. 27. In transport, 67 percent of the unsatisfactory highway projects had rates of return higher than 5 percent; in ports and railways, 83 percent had rates of return higher than 7 percent. The information suggests that Bank lending for transport has been quite successful, and that even operations ranked as "unsatisfactory" have proved quite worthwhile to the economies of borrowing countries. A similar pattern is found in some infrastructure-intensive projects in other sectors. * x - 28. When individual assessments of performance are aggregated across projects and sectors, it can be seen that many operations failed largely because of two main factorst unfavorable monetary and fiscal policies during their Implementation, and Institutional weaknesses in the borrower countries. Problems internal to operations--such as difficulties in operation and maintenance, Inadequate design in a broader sense, and deficiencies in financial management--were more common determinants of poor performance than factors over which W3vernments had no control-- sch as changes in world commodity prines, or severe weather. Evei where operations were affected by changes in international prices, droughts, and the like, evaluation reports often concluded that results would have been better had better macroeconomic policies been in place. 29. In a large number of the unsatisfactory operations, the principal determinants of performance were institutional. This term covers a very broad rubric. In some cases, institutions at the national and project levels were weak, and their capacity was overstretched by the Bank- supported operations. Often, difficulties stemmed from poor management, of both the financial and the human resources available for the oparations. Other difficulties arose when project designs turned out to be based on an inadequate understanding of the surrounding culture, rural social and institutional structures, and behavioral norms. And in some operations, unrealistic assumptions were made at appraisal as to the macroeconomic policies (for example on interest rates or agricultural prices) that would be in force. 30. The findings emphasize the importance of careful assessments of the institutional and financial aspects of proposed operations. They also highlight the role played by sector work to generate the needed support for policy changes in borrower countries to address those problems that cannot be appropriately addressed at the project level. On a regional basis, this review, like its predecessors, emphasizes the need for continued efforts to improve performance in Africa, particularly in the agricultural sector. Sustainability of Operations (Chapters 4 5) 31. The sustainability of development operations--broadly defined as their ability to maintain an adequate level of net benefits after the investment phase is completed--is of growing concern in the development community generally, and in the Bank's evaluation of its own effectiveness. 32. Evaluation experience shows that austainability depends on whether an operation or an investment or policy program is able to achieve an appropriate balance among the major forms of capitals human, natural, cultural, institutional, physical, and financial. For example, if a project expands the supply of financial and physical infrastructure (such as irrigation works, or school buildings), it will have little chance of being sustainable unless there is enough human and institutional capital (knowledgeable people, organized so that they can operate and manage it efficiently) to use the new facilities to advantage. - xi - 33. In complementing the assessuent of performance, sustainability analysis takes account of the characteristics of institutions and of social and cultural factors which, experience shows, are very important determinants of sustainability in operations. Moreover, sustainability analysis focuses on the costs and benefits of a project over the long term, and helps to address the effects on the project of other projects in the country's portfolio. Explicit consideration of these "portfolio effects" is needed for a realistic assessment of a project's sustainability. 34. Prospects for sustainability, like operational performance, are assessed at the time the Bank completes its disbursements. At this time OED staff provide a judgment of the ability of operations to maintain an adequate level of net benefits after the investment phase is completed. Inevitably their judgments on sustainability, like those on pe&formance, may need reassessment as an operation and its domestic and international environment evolves. 35. In its evaluation work OED classifies operations according to different levels of sustainability: likely, marginal, uncertain, and unlikely. Sustainability is judged uncertain where the key factors involved are particularly hard to predict--for example, where an operation faces a particularly uncertain market or policy environment, or where the institution charged with implementation is young and fragile. 36. Of the 557 Bank-supported operations whose sustainability OED has analyzed (i.e., all the operations that were evaluated in 1986-88), 15 percent were judged unlikely to be sustainable; 52 percent were judged likely to be sustainable, and the rest--fully one third--were judged marginally sustainable or uncertain. Africa has the lowest proportion of operations classified as likely to be sustainable (only 32 percent); Asia has the highest (65 percent), followed by EMENA and LAC (60 and 48 percent, respectively). Since the issues surrounding sustainability are multidimensional and diverse, and several aspects of the assessments made are qualitative, not quantitative, the judgments made in the review about the prospects for sustainability must be carefully interpreted and used. Determinants of Sustainability 37. A review of experience in projects judged to be sustainable highlights the role played by the following six elements: Institutional developments Most of the sustainable operations put a major emphasis on developing the institutional capacity required to sustain benefit flows after their completion. Experience also shows the importance, for the sustainability of individual operations, of the general level of education and institutional development in a country at large. Prospects for sustainability are better, too, in projects where the Bank has been able to en.sure that the goals and intended means of achieving them are compatible with the attitudes and cultural values of their intended beneficiazies. - xii - Macroeconomic policiest Good projects may fail if they have to operate in inhospitable policy environments. Most of the evaluation reports on sustainable projects mention that the macroeconomic policies of the borrowing countries were conducive to their profitability and efficient operation. Management: Most sustainable operations have had adequate management structures, and good managers. Audits indicate that management is a critical factor influencing sustainability in most sectors of the economy, particularly in development finance institutions and industrial and energy projects. Resource mobilization for recurrent costs: The sustainability of most operations depends heavily on their access to enough financial resources to maintain the efficiency and productivity of the physical capital they create. At the time they were evaluated, all the sustainable operations had efficient mechanisms for covering their recurrent costs. Commitmer-s A large number of audits have shown the importance of beneficiaries' participation to the sustainability of projects. Another, broader, dimension is a commitment by government to carry out the institutional and policy reforms needed to encourage sustainable growth. The importance of this latter commitment cannot be overemphasized, particularly as projects become increasingly sensitive to changes in the macroeconomic environment, both domestic and international. Stratexy: Most of the sustainable operations formed part of a clearly defined national development strategy. Without such a strategy, a country's portfolio of investments tends to respond to short-term imperatives, and as a result, projects are much less likely to be sustainable. Only with a clearly defined long-term strategy is there a good enough chance of securing the financial resources needed and of creating the skills and management expertise needed to sustain the operations. Financial Aspects of Sustainability 38. There are many sources of risk to the financial sustainability of operations. Though the Bank's operational guidelines have in general served well, some changes may be needed in the way the Bank applies them if sustainability issues are to be adequately addressed: - xii - - for types of projects that do not directly earn revenue from beneficiaries, present procedures for analysing sustainability are weak, particularly during their design and appraisal; - shortcomings in the handling of financial aspects of projects are sometimes attributable more to weaknesses in the design of covenants and, consequently, weak compliance with these covenants, than to inadequate procedures (a subject that is being addressed by Bank management); - improvements are needed in the way financial reporting is treated, to allow elements with a strong influence on the financial sustainability of operations to be properly monitored. 39. Resource mobilization for projects. Many of the operations evaluated faced severe financial constraints during their implementation phase. In a number of sectors where the Bank has a very large lending program and cost recovery is a central determinant of project sustainability, cost recovery from beneficiaries was often much lower than called for in the project design. 40. Most of the projects OED has evaluated were approved and began their construction phase in the period 1974-80. By and large, this was a period of cheap borrowing and easy liquidity. In the 1980s, however, once most of the projects had entered their implementation phase, pressure over scarce financial resources mounted considerably. Many revenue-generating projects that could have been sustained financially through adequate pricing policies were not. In many of these projects, too, particularly in electric power, demand for their output was lower than had been forecast. Projects dependent on the public budget fell victim to fiscal tightening. Financial shortfalls affected many projects' ability to maintain and operate their assets efficiently, and their services deteriorated. Many resorted to short-term oorrowing, at high costs both to themselves and to the economies at large. 41. Financial sustainability has also been affected by the high incidence of cost overruns. Though cost savings in projects with underruns accrue to the lender, cost overruns are largely the responsibility of the borrower. If the project scope is not reduced or supplementary financing not made available by the Bank, cost overruns are often financed through high-cost short-term borrowing, with damaging effects on the profitability of the project. Based on the extent and seriousness of the assessed risk of cost overruns, contingency plans for their financing will need to be developed. The financing of cost overruns raises a number of macroeconomic issues (e.g., public finance and trade) which may not be adequately handled at the individual project level. - xiv - 42. Importance of public expenditure, investment, reviews. The review of evaluation reports shows that financial sustainability at the project level depends critically on the extent to which other projects in the country's portfolio are financially sustainable. For example, where the profitability of revenue-earning projects is depressed by low tariffs and inefficient systems to recover costs from customers, public subsidies or allocations of scarce foreign exchange may effectively starve projects in public health, primary education, or agricultural research that, unlike public utilities, have no means to recover costs directly from the public. The lesson is to determine which projects ought in principle to recover their costs directly from beneficiaries, and to allow only those which cannot rely on direct cost recovery to compete for the overall marginal savings. 43. Given the importance of these portfolio effects, public expenditure and investment reviews should play a central role in planning for sustainability. Assisting with these reviews provides the right milieu for the Bank to discuss with governments priorities regarding the allocation of scarce financial, human, and institutional resources across all sectors of the economy. Public investment reviews should consider sustainability issues systematically, assessing the distribution of benefits and costs over time, as well as the alternative forms of financing, for the whole portfolio of investments. Investment and expenditure reviews should continue to focus on the financing of recurrent costs and the establishment of an adequate approach to resource mobilization for that purpose. Project Sustainability in the Context of Macroeconomic Adiustment 44. Though no study has been done to quantify the extent to which adjustment programs have directly affected the sustainability of individual projects, the evidence from project evaluation work shows clearly that macroeconomic variables (such as pricing, tariff policies, public finances, and the level of institutional development) are important determinants of project sustainability in most sectors. 45. Two points should be stressed. First, Bank lending in support of adjustment programs strengthens macroeconomic incentives and should thus lead to a more efficient allocation of resources, while by improving the climate for savings and investment it should enhance the borrowing country's capacity to mobilize financial resources. It also helps to strengthen the institutional framework for policy analysis and for investment planning and evaluation. All of this should help to make development efforts more sustainable. Second, however, adjustment programs contain measures (for example, measures to reduce public spending) that have often aggravated the funding problems of projects designed before the adjustment process got underway. 46. Experience shows that the financial interactions between projects and the macroeconomy need much more explicit consideration, both in the design of projects and in the choice of conditions attached to the Bank's - Xv - adjustment lending, than has been customary. Changes in the Bank9s perspective on these matters will have to be accompanied by changes in the relevant guidelines and procedures. 47. Under current circumstances, public sector finance requires more attention than it has received in the past. At the project level, fiscal implications and linkages are often not carefully analyzed. There Is seldom explicit consideration of the links between the financial structure of projects and the country's cost recovery and macroeconomic policies. Moreover, this review shows that sustainability analysis needs to be made an explicit feature of the Bank's sector and macroeconomic work. Recognizing the key role that macroeconomic variables play in the process of achieving sustainability may require the Bank to engage more staff, or to train existing staff associated with country operations, to address these issues adequately. Managing Natural Resources, Strengthening Institutions (Chapter 6) 48. For the pursuit of sustainability at the country level, proper management of natural resources and institutional development are major necessary conditions. Chapter 6 concentrates on these forms of capital, drawing on some of OED's recent studies. Management of Natural Resources 49. Bank-financed operations have made significant contributions to sustainable development, by enabling countries to transform natural resources that are relatively abundant into economic goods. Bank-financed investments have helped to raise the carrying capacity of cities by Improving land, water, and air, and by providing complementary physical and institutional capital. Other contributions to sustainable development- -amounting to several billion dollars thus far--have come through irrigation projects and projects for the management of land, forest, and energy resources. 50. A concern with the capacity of economic systems to survive, in the very long term, usually centers on the way countries manage their natural resources (assumed to be the scarce factor in the long term), and on the potentially negative effects of economic development on the environment. This intimate relationship between the achievement of economic development goals and of suatainability has often been referred to as *sustainable development." 51. Where natural resources are mismanaged, growth and equity-related goals are often compromised over the long term. The results of this year's review, however, also show that in the pursuit of sustainable development, forms of capital other than natural resources must also be taken into account. Thus, there is no point in adopting or financing an environmentally sound resource management strategy unless managers are available to manage it, adequate incentives are in place, and the development process is rooted in the society's values and beliefs. - Zvi - 52. OED's evaluation activities have for some time monitored the environmental aspects of the Bank's lending. The findings suggest three preliminary recommendations: - First, the Bank should continue increasing its ability to assist borrower countries to manage their natural resources, and to establish comprehensive national environmental or resource management strategies. The environmental dimension of Bank lending should be more central to the design of country strategies and to sector work. - Second, the Bank should continue its efforts to help countries incorporate environmental concerns into macroeconomic policy. Whenever appropriate, environmental concerns must become an integral consideration in the design of adjustment lending programs (public expenditure and investment issues will be essential components). In this regard, the Bank's Envi-onment Department should continue to play a leading role. - Third, the Bank's research and policy programs should continue to receive increasing support to find ways of incorporating environmental and natural resource considerations into national accounting systems. Some policies that appear sound within the existing system of national accounting, because of their contribution to national income, may have a disastrous impact on the environment. Institutional Develoment 53. The Bank's experience, as evaluated by OED, has shown the great extent to which the sustainability of development efforts depends on the nature and performance of institutions. It has also shown that while components aimed at strengthening institutions feature in a high proportion of Bank-financed operations, and the approaches used in them are very diverse, many of these components have not had lasting positive effects. Imperfect though the measures of such achievements are, the record calls for even greater attention to the institutional aspects of operations. 54. In view of the fact that the sustainability of development depends heavily on a country's capacity to marage its economy, this year's review discusses some issues in the Bank's assistance for public sector management. A study of free-standing technical assistance in support of public sector management in Africa (concentrating on 20 operations, in seven countries) shows that the technical expertise provided by these operations was often very successful in achieving specific, short-term goals. But in most cases, the operations either failed or only partly - xvii - succeeded in developing sustainable national institutions. Positive results were found only in the case of the introduction of specific systems and procedures (for example, for debt management, budgeting procedures, or statistical data bases). The projects rarely created sustainable change", and most of them failed to build up the basic self-reliance or institutional viability of the recipient entities involved. 55. For the Bank to encourage institutional development successfully through technical assistance, experience suggests the need for a high level of commitment and mutual interest on the part of the donor and the borrower; project design that capitalizes on existing opportunities and exploits the comparative advantage of different donors in a complementary fashion, within the context of an identified, country-specific, institutional development strategy; and a thorough basis in Bank sector work. 56. Attention to legal issues is also important in institutional development. An inadequate recognition of the role of law in institutional development may have contributed to the failure of some institutional development efforts in the past. In Conclusion ... 57. Of the issues raised by the review, five in particular call for attention in the 1990s. They suggest that the Bank should continue tos increase its efforts to help improve development performance in Africa, with particular emphasis on agriculture (para. 3.13ff); expand its work program of studies on and assistance for institutional development (paras. 3.18; 4.36; 6.32ff); make the sustainability of operations a more explicit goal of its lending (para. 4.39); consider the effects on the management of natural resources, as well as the environmental effects, in designing all its operations (para. 6.13ff); and increase its emphasis on financial issues, at the project, corporate, sector, and macro levels, giving particular attention to financial interactions between projects and the macroeconomy (paras. 4.31; 4.37-38; 5.32ff). Though awareness of the first two is already very broad, and they are already an Important feature of departmental work programs, all will require innovative thinking and sustained emphasis. - 1 - 1. INTRODUCTION 1.01 The Annual Review of Evaluation Results, or Annual Review of Project Performance, as it has been called in previous years (AR), synthesizes and provides a unique perspective on a large body of develop- ment operations. Like its predecessors, this year*s review assesses the pe:formance of operations supported by the Bank, and its underlying concern is with the efficiency and effectiveness of these operations. While the focus is on the Bank's performance in the preparation and implementation of the operations it finances, the implementation capacity and overall policies of borrowing countries are equally if not more important in the successful outcome of Aank-financed projects. 1.02 In this year's review, the approach has evolved in two ways. First, improvements in OED's data base now make it possible to review the experience of all the 1,983 Bank operations that have ever been evaluated. This gives the opportunity for a comprehensive look at how the Bank performed its mission over an extended period. (See Box 1.1.) 1.03 Second, the review is structured to allow more detailed discussions around a particular themes the requirements for sustainable development operations. It looks both at the conditions for achieving satisfactory performance in Bank operations and at the conditions for sustaining performance after the investment phase of operations is completed. 1.04 The Bank evaluates the performance of all its operations at the completion of the implementation phase, shortly after final disbursements. This evaluation compares outcomes at that point in time with expectations at appraisal. It is an opportune time to account for the use of resources, aasess the extent to which operations achieved their stated objectives, and draw valuable lessons for the future. However, this timing and the methodology used have limitations, which are explained in the remaining chapters. While at the time the Bank completes its disbursements, the resources invested, and the corresponding costs, are more fully known than at appraisal, the eventual benefits and costs throughout the life of the operation are still unknown. Further, since in most operations the economic activities only mature or develop fully after the Bank has completed its disbursements, the evaluation cannot fully take into account the learning process and adaptation that t&kes place during this phase. Impact evaluation, done for a small proportion of operations, typically about five years after completion, sheds more light on their development impact. 1.05 It should also be noted, first, that the findings described in this report do not necessarily indicate the current status of the opera- tions evaluatedt operations showing poor returns at completion may become more successful later on (for example, if their economic environment changes or if the process of technology adoption accelerates) while, for their part, operations showing high returns may not sustain their success. - 2 - Second, given that the analysis of performance Is based on judgments made about completed operations, the results do not necessarily indicate the quality of the Bank's current portfolio. Third, this review does not take stock of all the actions and programs the Ba"& has implemented in the last decade. Several programs have been designed to correct some of the problems discussed here and to change, whenever appropriate, the directions in investment allocations and policy reforms. A. Outline of Review 1.06 Chapter 2 gives an overview of the performance of 1,983 operations evaluated by OED over the period 1974-88. The results here are presented to disclose what has taken place, rather than to explain it or to draw conclu- sions about what should take place. 1.07 Chaptax 3 discusses the 378 evaluated operations whose performance has been unsatisfectory, and draws lessons and priorities for future studies. It identifies the principal determinants of performance in a group of operations whose economic rates of return were judged to be poor. 1.08 Chapter 4 analyses the influences on sustainability in development operations. It finds that some of the operations realising satisfactory economic returns at the time of completion (i.e. when the Bank complezed its disbursements) seem unlikely to be sustainable much beyond that time. The discussion is based on an analysis of a group of 557 operations whose sustainability OED has assessed; these operations were audited by OED in 1986, 1987, and 1988. On the basis of the experience of a group of sustainable operations, recommendations are offered for Improving sustainability. 1.09 Chapter 5 discusses common problems that have affected the financial sustainability of operations, concentrating in particular on cost recovery and coast overruns; it stresses the need to consider explicitly the links between the financial structure of projects and the borrowing country's macroeconomic policies. 1.10 Chapter 6 discusses issues in the management of natural resources, and technical assistance for public sector management. Annexes to the report provide statistical background material. - 3 - Box 1.1: THE BANK IN nE 1970s Most of the operations reviewed in this Though keeping the bulk of its lending in report were designed and approved in the the fore of project and sector investments, 1970s. For the Bank, this was a decade both the Bank responded by introducing policy- of ambitious growth in lending, from 82.2 based lending to support adjustment programs billion in 1971 to 811.6 billion in 1990 and policies in these countries. rhructural (Figures 1.1, 1.2), and of far-reaching adjustment loans (SALs) concentrate policy change, In the context of rapid attention on macroeconomic policies and evolution of the world economy. From lending associated institutional changes at the mostly for large-scale, capital-intensive, national level; sectoral adjustment loans projects in infrasructure and public (SECAL*) help promote the sectoral policies utilities, during the McNamara years the Bank considered necessary for sustained economic expanded Its purview into new areas, driven growth. by the perception that traditional Instruments of development were doing C-mpared with the previous decade, the relatively little to alleviate widespread and economic environment for development in the growing poverty. The Bank's rural 1970s was far more changeable and difficult development strategy, concentrating on to predict, with the floating exchange rates improving the productivity of small farmers of the major countries; the oil shocks of and introduced in 1972-78, was complemented 1972-78 and 1979 the commodity boom of by a lending program to laerove living 1972-78; and the subsequent recession In conditions for the urban poor, introduced in industrial countries. Many African 1974-76. Given the urgency of the poverty countries were racked by drought and civil problem, the Bank put Its new concern with strife, bringing severs food shortages. For the distributional consequences of many oil-importing developing countries, development into action on a large scale, within and outside Africa, high oil prices, using a range of sometimes unconventional declining terms of trade for commodity approaches, before It or anyone had been able producers importing manufactured goods, and to assess which approaches worked best, in the slowdown in export growth gave rise to which circumstances. massive current account deficits. Commercial lending Increased dramatically. On a different front, the Bank responded to Low real interest rates and pressure to lead the growing balance of payments difficulties from markets flush with petrodollars pushed of oil-importing countries by increasing Its many developing countries Into energy lending and its capacity to advise overborrowing. For those which had borrowed countries on energy matters. at floating rates, grave difficulties ensued when real Interest rates rose substantially By the latter years of the decade, a growing at the end of the decade. number of the Bank's borrowing countries were faced with severe structural Imbalances. . 4- 明陶目りU白陶園日的陶目田u園自… 陶鵬1.&：→婦COm加測知加，m～踏森易wo一→一by師風 ：「―一―て図e「一一一一一ーコ 1‘【／V!7一ZVI 1’『／!6『／1 1一！要5一鵬．..1 査一！4‘一＿な．&&l sf /!’一＿；盛－、・三・、－、・・1 ／ビー．.l＿一：,,..1］巨矛一三」1 の刀刀瑠かn 引曲あ町のカn 乃万m 馴曲毎町 ―曲園陶国田園U自日園国園日剛園日国国■日日国国国国田国国臓日園 - 5 - 2. PERFORMANCE: AN OVERVIEW FROM FIFTEEN YEARS OF EVALUATION An analysis of the performance of nearly of the *perations evaluated, and It Is a 2,600 operations supported by the Bank, representative sample, of total Bank opera- approved between 1969 and 19840 shows; that tIone In that period. The total amount four -^Ifthe of %hem were performing satlefac- Invested In the 1,794 operations, by the Bank tartly at the time the Bank completed Its and other sources of finance, was $168 disbursements. The total amount Invested In billion; of this, 17 percent we* Invested In those operations, by the Bank and other operations Judged to be unsatisfactory. sources of Insnce, was 8179 billion. In Thirty percent of the Investment In Africa general, projects In Infrastructure and urban want to unsatisfactory operations; comparable development performed well, as did power, oil figures for other regions are 19 percent for and gas, end highway* projects, and program EMENA, 20 percent for LAC, and 7 percent for and policy lending, with 05 percent or more Asia. of the operations being Judged satisfactory. Agriculture was the softer with the highest Though poor performanchs was most common In Incidence of unestlefact4ey performance; Africa. where only 78 Ivrcent of the number nearly a third of agricultural operations of operations performed satisfactorily, the were Judged unsatisfactory. 'In education, Investment costs Wwr unsatlafectory operation the record Is strong, but performance was "r* larger in other regions, I.e., 81" very variable In population, health, and million In EMENA, 2185 million In LAC, and nutrition and In technical assistance. 869 million In Asia, compared with Ilas million In Africa. Relative to the size of A trend analysis was am& of 1,794 evaluated African economies zad per capita Incomes, op6rations that were -approved between 1969 however, the amount* Invested per and 1986. This group accounts for 90 percent unsatisfactory operation were large. 2.01 The discussion that follows draws on 15 years of performance auditing and evaluations covering 1#983 operations evaluated between 1974 and 1988. The presentation has three alms: to draw up a balance shoot of the Bank9s achicn- ments in supporting developmentl; to look at patterns and trends in regions and sectors; and to further the analysis of special issues, so as to bring the light of experience to bear on what concerns the Bank and its borrowers today. 2.02 The 1,983 operations were approved during the period 1958-84 and represent a total investment (by the Bank and other sources of fitance' f of $178 billion. Ninety percent of them were approved in 1968-80 (Annex 1, Table 1-1). The group of evaluated operations approved subsequently is rather smalls amounting to only about five percent of the total. Most operations approved since 1980 hw e not yet been completed, ors if they haves their completion reports are not yet available. For the decade of the 1970s, the evaluated operations provide a very good representation of both the number and distribution of all the operations approved by the Bank9s Board. 2.03 Sixty six of the operations evaluated were program and policy- based loans; the rest were projects. Of the 19983 operations, 581 were in Africaq 510 :in Asia, 440 in EMENA, and 428 in LAC. The other 24 were in countries that no longer borrow from the Bank (Table 2.1). Five hundred - 6 - and ninety four of the operations were in agriculture, and within agriculture the largest number were in Africa, closely followed by Asia. Transport operations made up the next most numerous group. A. Performance Analysis 2.04 In this review OED continues the practice, used in last year's review, of classifying operations into two performance categoriest satisfactory and unsatisfactory. In certain categories of operations, notably agriculture, industry and transport, an economic rate of return is calculated at appraisal and then re-estimated upon completion. In the case of public utilities, incremental revenueA. attributable to the project are used as a proxy for benefits in calculating a quasi-ERR. Any ERR is based on a comparison of costs and benefits, measured using shadow prices that indicate real values to the economy. Inherent in the costs and benefits are four indicators of performance considered particularly important by the Banks sustainability of benefits, progress with institution-building, and the extent -f time- and cost-overruns. In the case of operations for which ERRs are not a suitable measure--some in the above categories, as well as human resources, SALs, and technical assistance--the performance assessment was based on a judgment as to the accomplishment of the goals t1lat were stated at appraisal, taking account especially of the four indicators just noted. Projects judged to have adequate ERRs where applicable or to have met the other indicators for acceptable performance were rated as satisfactory. Those that did not were rated as unsatisfactory. 2.05 The limitations of these practices were noted in Chapter 1; the review does show that a significant proportion of projects with a re- estimated ERR below 10 percent (normally, though not in all cases, considered the cut-off rate below which projects are considered unsatisfactory) may still generate important nonquantifiable benefits and thus be considered worthwhile to the economies that have undertaken them. Overall Results 2.06 The results of the analysis of performance show that 378, or 19 purcent, of the evaluated operations were rated unsatisfactory (Table 2.2). Overall, the Bank's performance was best in Asia, where 89 percent of the 510 evaluated operations were rated satisfactory; followed by EMENA, with 84 percent of 440 operations; LAC, with 81 percent of 428 operations; and Africa, with 70 percent of 581 operations. 2.07 Performance tended to be strong in most of the subsectors where the Bank had a large portfolios in highways (243 evaluated operations); education (183 evaluated operations); electric power (180 evaluated operations); development finance corporations (DFCs) (175 evaluated operations); and water supply and waste disposal (113 evaluated operations), at least 85 percent of the operations were judged satisfactory. Studies by OED show that these results for DFCs need some qualification. Several DFC operations were rated satisfactory, -7- Table 2.1: DISTRIBUTION OF OPERATIONS EVALUATED IN 1974-88, BY SECTOR AND REGION Setcor/Operation Africa Asia EMENA LAC Other /l Total Agriculture and rural development 197 189 99 195 4 694 Area development 6 17 12 20 - 129 Livestock 28 18 11 26 1 79 Irrigation 18 68 82 16 - 124 Agroindustries 0 26 12 4 - 42 Other /k 70 78 32 89 8 220 Industry and energy 84 186 168 182 12 522 Industry a 21 89 10 - 84 DFCs 30 48 58 82 8 175 Oil and gas 2 5 17 1 - 25 Power 26 43 88 70 4 18 T*Iecommunications 18 21 s 18 - Se Infrastructure and urban development 198 16 129 188 8 667 Transport 18 68 81 91 5 485 Highways 112 27 48 58 8 248 Railways 18 10 15 11 1 01 Ports 21 22 15 12 1 71 Tourlem and other 9 8 8 19 - as Urban 8 16 11 14 - 49 Water supply and waste disposal 26 22 87 28 1 118 Human resources and technical assistance 99 64 40 46 2 284 Education 73 88 85 87 2 188 Population, health and nutrition 1 9 4 8 - 20 Technical assistance 16 9 2 6 - 82 Program and policy lending 17 25 14 10 - 60 Program loans and import support /S 1 18 2 8 - 24 Emergency reconstruction 7 1 8 4 - 15 SAL* 9 6 8 8 - 26 TOTAL 581 519 440 428 24 1,988 - No operations. /a Finland, Iceland, Ireland, Israel and New Zealand. /b Includes perennial crops, credit, fisheries, land settlement, research and extension, program loan/credit, and agricultural services projects. Ie Includes only nonsector-specific operations. Source: Annex 1 Tab!e 1-1, 1-8. -8- Table 2.2: PERFORMANCE EVALUATIONS FOR OPERATIONS EVALUATED IN 1974-88 (Percentage of operations rated satisfactory) Average Sector/Operation Africa Asia ElENA LAC Other /I all regions Agriculture and rural development 51 83 82 78 75 70 (Area development) 49 78 50 45 - 68 (Irrigation) 88 89 84 78 - 8e Industry and energy 79 95 85 96 109 88 (Industry) Se 9o 79 88 - 81 (DFCs) 75 89 88 88 1e 85 (Oil and gas) le 16G 166 18 - lee (Power) 88 180 84 91 100 92 (Telecommunications) 85 166 166 92 - 95 Infrastructure and urban developeent 68 0 67 79 19 85 (Transport) 82 88 65 8 19 84 (Highways) 85 98 98 88 1ee 69 (Pore) 81 88 1M 67 100 85 (Urban) 19 94 91 71 - 88 (Water supply and waste disposal) 92 91 91 79 1N 88 Human resources and technical assistance 75 98 72 79 10 9 (Education) 79 94 76 88 1IN 88 (Population, health and nutrition) 10 76 25 Se - as (Technical ealsstance) 56 109 186 s9 - 71 Program and policy lending 8 92 98 70 - 8 (SALs) 89 67 88 0 - 73 Al I Sectors 76 89 84 81 96 81 - No operations. /* Finland. Iceland, Ireland, Israel and New Zealand. Source: Annex 1, Table 1-5. despite inadequacies in the companies' provisions for bad debts, write-offs of uncollectable loans, inclusion as income of accrued interest from nonperforming assets, and overstatement of real profits. 2.08 Agriculture, with 594 evaluated operations, was the sector with the highest incidence of poor performance. Performance within this sector was very variable, however, and the average rate of return for agricultural operations was the same, at 16 percent, as the average for all sectors for which ERRs were calculated. In Africa, in agriculture, there were 51 percent satisfactory operations. In area development (129 evaluated operations), performance was often very poor, with only 53 percent of the operations rated satisfactory. In urban development, where only 49 operations have been evaluated thus far, performance was also strong, with 88 percent being rated satisfactory. In oil and gas, all the 25 evaluated operations were satisfactory and in telecommunications, 95 percent of the 58 operations were satisfactory. - 9 - 2.09 Some of the strengths and weaknesses of the record in different regions are worth noting. Within agriculture, irrigation projects performed very well in Asia and EMENA, and poorly in Africa. Industry projects and DFCs performed very well in Asia and LAC, with 88 to 90 percent of these 115 operations being rated satisfactory. In highways (243 projects), more than 84 percent of the projects were rated satisfactory in every region, with especially good performance in EMENA and Asia. All the eight urban operations in Africa were judged satisfactory, and so were more than 90 percent of the urban operations in Asia and EMENA. 2.10 The record in human resources is also creditable in most regionst among the 73 education projects in Africa, four fifths were satisfactory; and among the 36 in Asia, 94 percent were satisfactory. Technical assistance operations did well in Asia and EMENA though poorly in Africa and LAC. But of the four population, healUh, and nutrition projects in EMENA, only one was satisfactory, and of the six in LAC, only three were satisfactory. 2.11 In policy and program lending, making up a group of 66 operations, 88 percent of the operations were judged satisfactory; eight of the nine SALs in Africa, and seven of the eight SALs in ENERA, were judged to have been satisfactory. Several of the SALs that were rated satisfactory achieved rather mixed results, in terms of the implementation of policy changes and overall economic performance. Trends in Performance: 1968-80 2.12 A trend analysis was made of the 90 percent (1,794 in number) of the evaluated operations that were approved in the period 1968-80. For all regions, this is a highly representative sample of the types of operations evaluated. -1so, being highly representative of the numbers of operations approved in that period, this sample affords an opportunity for a comprehensive look at the Bank's performance in the 1970s. 2.13 For the analysis, operations were grouped by their year of approval. This approach is more revealing of trends in performance than OED's former approach of grouping operations according to the year in which they were evaluated. Looking at the results for the sample, grouped according to the years of approval, Table 2.3 shows that the percentage of operations performing unsatisfactorily tended to be higher than the average (of 19 percent for all regions throughout the period) among operations approved in 1974 and subsequently, and especially in 1978. In Africa, except in 1969 the percentage of unsatisfactory operations was always higher than the average of 19 percent, with the highest peaks in 1974, 1978, and 1979. In Asia, operations performed better than the average, except those approved in 1978. Experience in the other two regions was more variable. - 10 - Table 2.8: OPERATIONS WITH UNSATISFACTORY PERFORMANCE AS PERCENT OF ALL OPERATIONS, BY REGION, 1988-88 Approval year Africa Asia EMENA LAC Total 1988 24 9 16 18 16 1969 11 0 88 0 10 1970 26 7 18 22 18 1971 29 7 19 11 16 1972 28 11 8 12 16 1973 20 10 12 29 16 1974 41 17 17 21 28 1976 81 10 17 27 22 1978 86 11 13 18 19 1977 86 12 14 25 22 1978 40 20 28 81 28 1979 41 14 17 19 22 1980 24 4 4 27 12 Average, 1968-78 22 7 17 18 15 Average, 1974-80 84 12 15 24 21 Average, 1968-80 29 10 16 19 19 Source: Annex 1, Table 1-6. 2.14 Analysis on the basis of the total investment costs of operations, rather than their numbers, gives a somewhat different impression of performance (Table 2.4). Between 1968 and 1980, $166 billion was invested (by the Bank and other sources of finance) in the sample of 1,794 evaluated operations. Of this, 17 percent was invested in unsatisfactory operations. The regional averages for the period were: Africa, 30 percent; EMENA, 19 percent; LAC, 20 percent; and Asia, 7 percent. Compared with the average for the 13 year period, the percentages of total investment that were committed to unsatisfactory operations were high in 1970 and 1978-80. Table 2.4: INVESTMENT IN OPERATIONS WITH UNSATISFACTORY PERFORMANCE, AS PERCENT OF REGIONAL INVESTMENT TOTALS, 1968-80 Approval year Africa Asia EMENA LAC Total 1988 11 1 18 7 12 1989 16 a 66 0 17 1970 15 1 16 46 24 191 15 8 18 4 8 19s2 28 6 4 2 6 1978 24 8 28 7 11 1974 s 14 28 7 22 1976 14 7 19 so 89 1976 89 11 6 11 12 1977 42 5 9 81 14 1978 81 9 24 82 22 1979 64 84 28 28 82 1980 25 (1 (1 87 6 Average, 1968-89 80 7 19 20 17 - 11 - 2.15 The average amount invested per unsatisfactory operation, across all regions, in 1968-80 was $82 million (Table 2.5). The average investment cost per unsatisfactory operation in Africa was $30 million, compared with $69 million in Asia, $135 million in LAC, and $144 million in EHENA. Operations in general have tended to be smaller in Africa, and investments in unsatisfactory operations, too, have been smaller in Africa than in other regions. Relative to the size of economies and per capita incomes, however, the amounts invested per unsatisfactory operation in Africa are large. Table 2.6: AMOUNTS INVESTED PER OPERATION WITH UNSATISFACTORY PERFORMANCE, 1968-89 (Millon current US dollars) Africa Asia EMENA LAC Total 1988 8 8 422 88 62 1969 28 0 95 6 66 1970 12 10 81 265 86 1971 11 84 67 57 84 1972 29 40 75 88 88 1978 46 24 172 51 68 1974 88 60 258 89 74 1975 18 85 150 687 161 1976 84 81 46 o 58 1977 44 4? 1ee 82 o 1978 18 so 145 268 91 1979 51 451 269 164 182 1980 87 (1 22 148 88 Average, 1968-89 89 89 144 135 82 Memo Item: Average, 1968-89 for satisfactory operations 28 98 156 129 100 Comparison with Previous Annual Reviews 2.16 A comparison is made here with the results given in AR85 and AR87, which analyzed operations on the basis of their year of evaluation, rather than their year of approval. AR87 reported that 28 percent of all operations evaluated in 1987 were rated unsatisfactory. The same data rearranged by year of approval show that a high proportion of these projects were approved in 1978. The cohort evaluated in 1987 also contained a large number of operations approved in 1979, another relatively poor year (Annex Table 1-2). 2.17 A high proportion of the operations evaluated in AR85 performed very poorly. The operations evaluated in 1985 were dominated by those approved for Africa between 1974 and 1978; as was shown above, many of these were rated with very unsatisfactory performance. Among the - 12 - operations evaluated in AR85, those approved for the Asia region in 1976 and 1977 performed worse than those approved in the two previous years. B. Performance of IDA-financed Operations 2.18 OED has evaluated 827 operations supported by IDA credits. This group represents 42 percent of the total of 1,983 operations evaluated, and a total investment of $36 billion; it includes operations that blended IDA and IBRD resources. The regional distribution of IDA operations, in percentages, is as follows (see Annex 1, Table 1-13): Africa 47 Asia 30 EMENA 16 LAC 7 2.19 The average rate of unsatisfactory performance among IDA-financed operations is 23 percent. Differences in performance are explained by differences in the sectoral and subsectoral distributions of evaluated operations, as well as in individual country conditions. 2.20 On a sectoral basis, IDA-financed operations generally performed as well as the population as a whole, except in agriculture, population, health, and nutrition, and technical assistance. Performance was particularly good in power, oil and gas, and telecommunications. In agriculture and rural development, 33 percent of the IDA operations performed unsatisfactorily, compared to 30 percent for the population. A high proportion of the IDA agricultural operations were in ecological zones that have high climatic risks and in countries with political instability. In the other sectors, there were no major differences in overall performance among regions (Table 2.6). 2.21 Within the group of IDA-financed operations performance was relatively poor in Africa: the share of IDA operations judged unsatisfactory was 32 percent for Africa; 14 percent for Asia; 17 percent for EMENA; and 17 percent for LAC. A large number of the evaluated operations were in countries whose economies consistently performed poorly during the 1970s. - 13 - Table 2.0: PERFORMANCE EVALUATIONS FOR IDA OPERATIONS EVALUATED DURING 1974-88 (Percentage of operations rated satisfactory) All Sector/Operation Africa Asia EMENA LAC Regions Agriculture and rural development 49 88 88 78 87 (Area davelopment) 52 76 43 as 52 (Irrigation) 80 98 85 88 79 Industry and energy 83 91 s 1e 8 (Industry) 50 82 so - 8 (DFCs) 75 8 86 1N 79 (01l, gas, and energy) 100 100 180 - 10 (Power) 19G 100 100 100 100 (Telecommunications) 109 100 100 - 1N Infrastructure and urban development 88 88 88 81 88 (Transport) 82 85 87 88 8 (Highways) 84 91 109 1e 88 (Ports) 92 71 100 9 8o (Urban) 100 67 67 100 87 (Water supply and waste disposal) 85 82 98 0 85 Human resources and technical assistance 74 89 67 of 77 (Education) 79 67 0 980 82 (Population, health and nutrition) 100 75 0 - 8e (Technical assistance) 6e 100 - - 70 Program and policy lending 75 100 100 6o 9 (SALs) 8o - 100 0 71 AVERAGE 68 86 88 88 77 - No operations. Sources Annex 1, Tables 1-18 and 1-14. C. Economic Returns 2.22 As explained earlier, for certain types of operations an ERR is estimated initially at appraisal and then re-estimated at the time the project completion reports are prepared. For the 1968-80 period, the average re-estimated rate of return for 1,065 operations was 16 percent. Disaggregation of this average shows moderate variations across regions and across broad types of operations. The regional averages for the period are 14 percent, 21 percent, 13 percent, and 14 percent for Africa, Asia, EMEMA, and LAC, respaztively. By sectors, the re-estimated average ERR was 16 percent for agriculture, 29 percent for oil and gas, 17 percent for industry, 25 percent for highways, and 13 percent for sites and services (Table 2.7). - 14 - Table 2.7: RE-ESTIMATED ERRS OF EVALUATED OPERATIONS, BY SECTOR AND REGION, 1968-80 Average Re-astimated ERt SfActor Tgriculture and rural development 16 Crop are development 12 Livestock 12 Forestry 27 Fishery 28 Power 11 Oil and gas 29 Oil pipeline as Industry 17 Urban sites and services 18 Tourism 18 Ports 22 Highways 25 Water supply 8 R Africa 14 Asia 21 EMENA 18 LAC 14 TOTAL 16 Notet ERRe are not applicable to development finance corporations, human resources, and technical assistance operations, or to program and policy lending. These operations are not covered in this table. /I Average of Individual project ERR*, weighted by projt6 costs. 2.23 The average ERRs just noted are weighted by project costs and thus indicate approximately the return on the total resources involved. Considering that the measures of benefits usually employed in both power and water supply operations are the financial returns, which traditionally provide a conservative estimate of economic value in these activities, an overall average ERR of 16 percent seems quite acceptable. 2.24 As in previous annual reviews, the frequency distribution of ERRs was studied on a regional and sectoral basis. ERRs as re-estimated at completion varied a great deal across regions and sectors, though most fell between 10 and 15 percent. Those estimated at appraisal mostly fell between 15 and 20 percent in industry, transport and tourism, and agriculture, and in a somewhat lower range in urban and public utilities projects. ERRs tended to vary widely within the group of agricultural operations, but not within other groups of operations. 2.25 For several years OED's annual reviews, analyzing operations in groups according to their years of evaluation, have remarked on an apparently widening gap between ERRs as forezast at the time of appraisal and those as re-estimated at the time of completion. For this year's review, three-year moving averages were calculated for the gap between ERRs - 15 - estimated at appraisal and those eatimated at completion. When these are Figure 2.1: Gap Between Appraisal and Re-estinated calculated according to the year of ERR*, by Yew of Evlueat 1974-88 evaluation, there is an apparent increase in the gap over time. The 0 averages calculated according to the year of approval do not show such an easily discernible increase in the gap. 20. APPOMER Both methods of analysis do show, . however, that there is a continuing as- large difference between forecast and * re-estimated ERRs. (Figures 2.1, 2.2, 22 - and Annex 1 Figures 1-1, 1-2.) 2.26 For projects approved in certain years there are particularly wide gaps between the average appraised 16 and re-estimated ERRs. The gaps tend to be larger in Africa and LAC, and in 14 agriculture, energy, public utilities, ReeWmedERR industry, and urban operations. They 12- call for an assessment of the most - important assumptions that determine the 74 76 78 60 62 84 as so value of ERRs in those sectors and regions. 2.27 First, in Africa in particular, throughout the period consiLdered, the Figure 2.2: Gap Between Appraisal and Re-estimated ERRs estimated at appraisal are ERRs, by Year of Approval, 196840 consistently higher than those re- estimated at completion. This tendency is less pronounced in the other regions, a but the results do suggest a need for greater realism in the assumptions 27 underlying project analysis. Second, in the sectors where returns tend to be 2 ApWWERR relatively low--especially ag:iculture and urban projects--there has been a as growing tendency to overestimate the returns at the appraisal stage. 2.28 Given these two tendencies, it is important for the Bank to make sure that its review of proposed projects and PW-MERm programs takes deliberate account of the I? assumptions underlying the forecast returns to such projects and programs. Is Technical parameters, such as crop yields, and economic variables, such as .. price, are often difficult to predict, as 70 72 74 76 78 so particularly in agriculture. In such circumstances experience has shown the - 16 - virtues of providing for strong monitoring and evaluation units. On the basis of implementation experience as documented by these units, the design of operations can be modified once they are in progress. 2.29 Finally, although only a few impact evaluation reports are available, compared with the number of audis, the preliminary findings of several of these reports show that the ERR tends to decline, rather than rise, in the years after operations are completed. D. Cost and Time Discrepancies 2.30 Estimates of total costs were made at the time of completion for 1,778 of the operations that were evaluated between 1974 and 1988. Of these, 544 operations, or 31 percent, had actual costs lower than those estimated at the time of appraisal. Of the 544 projects with cost underruns, the smallest number were in LAC and the largest number in Africa, followed by Asia. The 1,123 projects with cost overruns were distributed very evenly across the four main regions. 2.31 Most of the operations approved in the 1970s took much longer to complete than expected; very few took less time than expected. There is no statistically significant trend in time overruns by the year of approval. Education projects in all regions had a very high incidence of time overruns; some of the reasons are institutional and fiscal. (Annex 1, Tables 1-7 to 1-12.) E. Operations Evaluated in 1988 Figure 2.3:COuratonhsEvsIauutedl9AB, by Year of Approval and Closing 2.32 Detailed information about the performance of the cohort of operations evaluated in 1988 is given in Annex 2. This section describes the composition and overall performance of the 1988 s gg verMam cohort, to allow comparisons with the *gwgaraoe" cohorts reviewed in earlier annual reviews. In 1988 170 operations were so technically completed, out of which 65 were audited, respecting the minimum 140 permitted ratio of 40 percent. (Two of the 170 operations were not implemented.) 2.33 Lending for the 1988 cohort amounted to nearly $7 billion. Most of these operations were approved in the 1970s, with a strong representation from 1978 (Figure 2.3). Seventy four percent of the evaluated operations were rated " aM 7 M U-1soa a . e - 17 - satisfactory. By sector, performance followed similar patterns to those observed in the cohorts of the last few years (Table 2.8). Table 2.8: PERFORMANCE EVAUATIONS FOR OPERATIWNS EVAUATED IN 1988 Sector Percent Number of satisfactory operatione Agriculture and rural development s8 62 Industry and energy 78 s Infrastructure and urban development 91 42 Human resources and technical assietance 72 26 Program and policy tending le a TOTAL 74 18 Reaion Africa 6 44 Asia 87 89 EMENA 79 48 LAC 78 42 TOTAL 74 168 Note: Excludes the two operations not implemented. Source: Annex 2, Table 2-7. 2.34 In agriculture, 25 of the 62 operations evaluated in 1988 were judged unsatisfactory. Twelve of these were in Africa, five in LAC, and four each in Asia and EMENA. Ten were area development operations; seven were in irrigation and drainage. The most common reasons for failure, affecting 23 of the 25 operations, seem to have been inadequacies in preparation and unrealistic assumptions made on appraisal. Difficulties in implementation, arising out of poor commitment to project objectiTes, institutional weaknesses, or inadequate supervision, were cited in 14 of the 25 operations. 2.35 Five out of the twelve industry operations evaluated in that year were judged unsatisfactory. The major reasons were: - long delays in construction and in reaching expected rates of capacity utilization; - large cost overruns associated with the delays just mentioned, which significantly affected the operations' financial viability; - inadequacies in project preparation and deficiencies in design; - 18 - - inadequacies in project management, including lack of competent managers and skilled professional staff, inadequate project planning and control procedures, and ineffectual coordination between the project management team and the regular management of the parent firms; - delayed or inadequate provision of financial resources; - distortions in sector policies affecting prices, salaries and wages, spending and investment rates, and areas of managerial autonomy; - adverse trends in market prices, which became the single most important determinant of unsatisfactory performance in most of these operations. - 19 - 3. DETERMI 4ANTS OF PERFORMANCE Thia chapter studi.A a group of 288 unsatis- others, there was an overall lack of factory operations and shows that a good appropriate management for the financial as nroportion (30 percent) were yielding well as human resources available for the economic returns close to 1 percent, operations. Only rarely did evaluation (normally, 'hough not in all cases, consid- reports mention factors beyond the control of ered the cut-off rate below which projects policy makers (for example droughts, changes are considered unsatisfactory). However, 82 In world comodity prices) as the principal of the 23 operatione had econoic rates of causes of unsatisfactory performance. return below zero; 88 of this group were in agricultire. The discussion of performance below relies heavily on ecen-mic rates of return for In most of the operations with negative judging operations; the findings are valid, returns, the main determinants of performance but for an adequate understanding they need were poor management, weak institutions, and to be supplemented by an analysis of the difficul.iles with institutional changes and operations' susetainability; this Is offered reforms. In some cases, institutions were in Chapter 4. weak at the national and project levels; In 3.01 Tis chapter identifies and discusses the issues surrounding operations whose performance was judged "unsatisfactory* on the basis of their economic rates of return as re-estimated at the time of tompletion, as described in Chapter 2. Experience with projects that have proved to be successfully sustainable is reviewed in Chapter 4. 3.02 Seventy percent of the number of operations audited in Africa have been judged satisfactory, compared with 89 percent, 84 percent, and 81 percent for Asia, EMENA and LAC, respectively (Tables 2.2; 3.1). Out of 378 operations judged to be unsatiafactory, 170 (45 percent) are in Africa; the poorer performance in Africa partly reflects the large number of unsatisfactory agricultural projects in the Bank's portfolio in the region. Table 3.1: DISTRIBUTION OF SATISFACTORY/UNSATISFACTORY OPERATIONS BY REGION, 1974-88 (Number of operations) Category Africa Asia EMENA LAC Other /I Total Satisfactory 406 454 387 848 28 1,698 Unsatisfactory 170 so 69 82 1 878 Operations for which data not available 8 - - 1 - 4 Operations not implemented 2 - 4 2 - 8 TOTAL 581 68 440 428 24 1,988 - No operations. /I Finland, Iceland, Ireland, Israel, and Now Zealand. Source: Annex 1, Table 1-5. - 20 - A. Operations with Unsatisfactory Performance 3.03 Of the 378 operations judged as unsatisfactory, an analysis was made of 238 whose performance was rated unsatisfactory on the basis of their economic rates of return as re-estimated at the time of completion. The total investment cost of these operations was nearly $20 billion (out of a total investment cost, for all operations reviewed, of $178 billion). The sectors represented in this group of operations are agriculture, industry, transport and tourism, energy and public utilities and urban operations. As explained in Chapter 2, a re-estimated ERR of 10 percent is normally, though not in all cases, used as the cut-off rate below which projects are considered unsatisfactory. 3.04 For this analysis the unsatisfactory operations were arranged according to their rates of return as re-estimated at project completion, into four categories: (1) operations with ERRs of 7 percent and above; (2) those with ERRs of 5 and 6 percent; (3) those with ERRs from 0 percent to 4 percent; and (4) those with ERRs below 0 percent. The results show that a significant proportion of operations were still yielding returns very close to 10 percent (Table 3.2). Of the remaining unsatisfactory operations, 34 percent had ERRs below 6 percent, 23 percent had ERRs betw*en 0 percent and 4 percent, and 13 percent had ERRs of 5 and 6 percent. Table 8.2: DISTRIBUTION OF A GROUP OF UNSATISFACTORY OPERATIONS BY RE-ESTIMATED ECONOMIC RATES OF RETURN, 1974-88 Transport Energy and Range Agricultur Industry A Tourism Pubilc UtiI. Urban Total No. 2F No-.w- NO. I 9; w NWW below 5 6 45 2 1 7 18 7 87 0 0 82 84 0 - 4X 88 26 4 81 7 18 4 21 188 64 28 5 - e% 18 12 4 81 5 9 8 16 1 88 81 18 7X and above 26 18 8 28 86 65 5 28 1 88 71 89 Numbers of operations 143 (62) 18 (8) 5 (28) 19 (8) 8 (1) 288 (1"0) (sectoraI distribution) 3.05 Of the operations with ER'Rs below 0 percent, the largest proportion--80 percent--were in the agricultural sector. Within agriculture, unsatisfactory operations were most common in croplarea development, livestock, tree crops, and irrigation. Among these operations, however, rates of return seem to have varied a great deal. In irrigation, 21 percent of the unsatisfactory operations had ERRs of less than 0 percent. Among the croplarea development operations, nearly 49 percent had ERRs below 0 percent, but 18 percent had ERRs greater than 7 percent. In tree crops, fisheries, credit, and agroindustry, most of the unsatisfactory operations fell into the lowest category. - 21 - 3.06 In transport, 67 percent of the unsatisfactory highway operations had ERRs over 5 percent, and of these 57 percent had rates of return of over 7 percent. In ports and railways, 83 percent had ERRs over 7 percent. The information suggests that Bank lending in the transport sector has been quite successful, and that even operations ranked as "unsatisfactory* have proven to be worthwhile to the economies of borrowing countries. B. Operations with ERRs Below Zero 3.07 Among the 238 unsatisfactory operations there are 82 that had negative ERRs. Except for three operations approved in 1980-82, all these operations were approved during the 1970s. Of the 82 operations, 64 were in agriculture, two in industry, four in highways, one in railways, one in ports, one in tourism, five in power, one in telecommunications, and one in water supply. The total investment cost of these 82 operations was nearly $2 billion, or one percent of the total cost of evaluated operations. Determinants of Poor Performance 3.08 Several findings cut across sectors and should be noted. First, in the great majority of cases, the principal determinants of performance were difficulties with institutional changes and reforms. These varied, depending on the sector in question. S2cond, in several inatances operations failed because the macroeconomic environment was very unfavorable during their implementation. In fewer cases, changes in world commodity prices, over which governments had no control, were decisive. The fact that such exogenous influences were not cited as principal determinants of performance may only reflect the nature of the audit process. Some of the most frequently cited determinants of performance, however, were in principle within the control of governments. This was particularly the case with institutional reforms and changes in monetary and fiscal policies. Problems internal to the operations, such as difficulties in operation and maintenance, inadequate design, and deficiencies in financial management, were also common features. Third, financial problems and constraints were other Important determinants of performance in most of the unsatisfactory operations. 3.09 In the transport operations with vety low rates of return, the most important problems were: - lack of competent management; - weak institutions; - inadequate financial management; - various technical and design issues. - 22 - 3.10 Industrial estates in this group of operati-mus suffered from: - limitations in institutional capacityl - financial instability, caused by charging tenants rents that were too low to cover high administrative costs; - major construction delays. 3.11 Power operations in the group suffered from: - major technical inadequacies in design; - borrowers' poor institutional and managerial performance; - poor financial performance, and other financial problems; - delays and cost escalations, and inadequacies in their tariff structures. 3.12 The determinants of poor performance in agriculture, with a particular emphasis on Africa, are discussed below. C. Performance In African Agriculture 3.13 Evaluation results show that the performance of Bank operations in African agriculture is cause for concern. It is worth noting that many of these operations were experimental in character. They were implemented under difficult conditions, often very Imperfectly understood, in the desire to address urgent problems of rural poverty. Of the 197 operations in African agriculture that have been examined, nearly half were judged to have unsatisfactory performance. 3.14 In studying the experience with African agricultural operations that had ERRs below zero at completion, a distinction was made between the determinants of performance that could have been influenced by policy makers, and those that clearly could not--such as movements in the international prices of primary commodities, or harsh and unpredictable weather. Though perhaps not what one would expect, the analysis shows that the main determinants of unsatisfactory performance were mostly within the domain of policy makers. Only rarely did evaluation reports mention factors beyond the control of policy makers as the principal cause of unsatisfactory performance. Although some operations suffered from changes in international prices, droughts, and the like, the evaluation reports often concluded that performance could have been better if better macroeconomic management had been in place. 3.15 Within agriculture in Nfrica, determinants of unsatisfactory performance vary quite widely f--m one subsector to another (see Box 3.1). The most conspicuous failures were in area development operations, only a - 23 - minority of which succeeded to some degree. With hindsight it is clear that the design of area development operations tended to be unduly complex, and that they would have required exceptional leadership to succeed. Moreover, these operations were to some extent victims of the severe institutional difficulties and inappropriate macroeconomic policies which affected many operations in Africa, not only in agriculture. 3.16 It appears that the main reasons why agricultural operations have failed in many countries of Africa are institutional. They include weak staffing, poor organization and management of project entities and other agencies on which operations relied, and poor understanding of nonmarket incentives, and of the cultures and social structures within which the operations were introduced. Project completion reports and project performance audit reports on Bank operations in Africa almost invariably made the point that institutional goals were not achieved. In regions other than Africa, by contrast, institutional factors were less important; failures in agricultural operations were more commonly attributed to inappropriate sectoral and macroeconomic policies. D. Conclusions, Future Directions 3.17 The economic, political, and social environment of development is complex, not to speak of the civil strife experienced in some countries, and it should be expected that some operations will have unsatisfactory outcomes. The analysis of experience with nearly 2,000 operations has, however, shown the considerable extent to which the fate of operations depends on proper design and on factors that are in principle within the control of the borrowing countries' policy makers during the implementation period. It has also emphasized that there are circumstances where purposive actions by government--for example in pushing through a price policy reform, or, over the longer term, in investing in education or in promoting the diffusion of new technologies--could clearly help to make operations more successful. Many of the areas of policy that were outside the purview of the Bank's dialogue with borrowers at the time these operations were appraised are now the subject of conditionality under SALs and sectoral adjustment loans, while, through its assistance with public expenditure and investment reviews, the Bank bas a vehicle for influencing the performance of the operations it helps finance. 3.18 An inadequate level of institutional development has played a central role in determining the performance of most unsatisfactory operations. In some cases, institutions were weak at the national and project levels. In others, financial as well as human resources were badly managed. Since few organizations are static, institutional issues in operations are often difficult to anticipate, even in operations that are adequately designed. However, an explicit assessment at the appraisal stage of the adequacy of institutional arrangements will establish a sounder basis for implementation. Such an assessment ought to include both a judgment of the operazion's need for a specific form of organization and a judgment of the absorptive capacity of the relevant institutions. - 24 - 3.19 This overview of performance has relied a good deal on economic rates of return as a criterion for judging operations. The limitations imposed by that criterion do not invalidate the findings. Nonetheless, there is a case for a series of future studies that would center on operations for which performance evaluation has not been based on ERRs (e.g., education, nutrition, population, technical assistance, nonproject lending). 3.20 For future evaluations of performance, three areas of study might prove especially worthwhile: - First, the establishment of a framework to identify which issues in the failure of operations are macroeconomic in nature, and to study the relationship batween those issues and the ways in which SALa and SECALs are being formulated. - Second, sectoral studies to identify the types of problems that recur frequently at the sector or country level (e.g., in project design, technology, or the absorptive capacity of institutions), and to assess the extent to which Bank policies and procedures can be expected to resolve those issues. - Third, the development of a mechanism to integrate OED's findings and recommendations with those coming out of the Bank's reviews of projects under Implementation, so that they reinforce one another. The findings presented in the latter sections of this chapter are very similar, if not identical, to those identified during these reviews during the last two years. This implies that even though they are drawn largely from the experience of the 1970s, the conclusions presented here are still central to the current debate on performance. 3.21 Lastly, a concern for performance must be complemented by a concern for sustainability. This is the subject of the next chapter. - 25 - Box 3.1: AFRICAN AGRICULTURE INFLUENCES ON TOB BANKS PERFORMANCE In I livastock operations, project design determining poor performance. Financial tended to pay much too little attention to management tended to be poor, both at the social and Institutional factors. Often, seotoral level and within operations, and understanding seems to have been inadequate exchange rate, pricing, and marketing of the intricate Institutional arrangements policies -nall caused difficulties. (including the structure of property rights) Operations were not exemp free political prevailing in the areas covered. Too little Interference, particularly in the way attention wee given to ways of mobillaing finances were administered and in managing local populations. Cost escalation also posed the estors as such. problems. In Irrigation operations the principal In agricultural credit operations, designs problems stemned from Inadequate design, often gave too little attention to proposed institutional constraints, and the lack of terms of lending and to the internal effI- an adequate system for resource mobillIation clency of the agencies involved, whose staff- and operation and maintenance. Ing and organization were often very weak. In fisheries and agroindustries, problems In tree crop operations, designs were often were similar. Project preparation was often unrealistically complex. Arrangements for deficient, operations were ineffectually interactions with smallholders were unduly managed, Implementing agencies ofte. lacked complicated, Implementing institutions were the skills neded, and the consultants often weak and poorly managed, and labor selected to eist them were often poorly inefficiency posed problems at all levels. chosen. Other problems related to In only a few cases, unfavorable operations' capital structure and lack of International prices were the key factor flexibility. - 27 - 4. SUSTAINABILITY OF DEVELOPMENT OPERATIONS The sustainabtilty of development operations and financial. Experience shows that, is of growing concern In the development repeatedly, the Bank assumed that the community generally and in the evaluation of critical constraint was the scarcity of Bank operations. To complement Its assess- financial and physical capital, and that once ment of performance, this chapter discusses this constraint was lifted, a new balance the sustainability of completed operations. would be achieved among all forms of capital. As in the case of performance, sustainability Often, however, such a balance has not been Is evaluated at the time the Bank completes achieved. In particular, lesser recognition Its disbursements. was given to the need for education and Institutional development, without which The sustainability of most development off icient use could not be made of financial operations depends on the balance they capital, nor, sometimes, could the natural achieve among six types of capital: human, resources under the purview of projects be natural, cultural, Institutional, physical, prcperly managed. 4.01 The sustainability of development operations--broadly defined as their ability to maintain an adequate level of not benefits for a reasonable period after donors' disbursements are completed--is of growing concern in the development coumunity generally. In many countries, large public debt and the economy-wide adjustment process are having important effects on growth and the distribution of its benefits, and on the capacity to sustain the development effort. To improve the sustainability of projects, donors are promoting and helping borrowers carry out needed reforms in policies and institutions. They are prolonging their interven- tions by financing new follow-on projects, where finance is the constraint, and rehabilitating old projects whose physical capital has been depleted. 4.02 In this context, even though a high proportion of its operations are rated satisfactory, the Bank recognizes sustainability as a vitally important consideration in the evaluation of its lending. This chapter describes the record for all 557 Bank-supported operations whose sustain- ability OED he s assessed, discusses the factors that have the greatest influence on sustainability, and reviews issues in the evaluation of sustainability. A. Analyzing Sustalnability 4.03 Most development operations encompass at least six forms of capital: human, natural, cultural, institutional, physical, and financial. A review of experience with a sample of more than 550 operations shows that their sustainability depends upon whether they are able to achieve an appropriate balance among these various types of capital. For example, a project that creates new physical capital (such as irrigation works, or school buildings) will have little chance of being sustainable unless there - 28 - is enough human and institutional capital (knowledgeable people, organized so they can work efficiently) to use the new facilities to advantage. Since the nature and relative scarcity of these forms of capital, and the degree to which one form can substitute -or another, vary widely according to circumstances, the balance that may be best in one case may not be so in another. 4.04 Until relatively recently, the Bank0s operations and its evaluation of their performance tended to be based on the assumption that the critical constraint on development was the scarcity of financial and physical capital, and that once this constraint was lifted, a new, adequate, balance would be achieved among all forms of capital. As the evaluation process evolveC these assumptions began to be questioned. Both audits and evaluation reports showed that in several instances, increases in the scale of physical and financial capital that accelerated economic growth had resulted in a deterioration in the natural environment. Supplies of natural capital (such as water or forest) have proven much more of a constraint than previously recognized. Evaluation reports have also shown that a balance between physical/financial capital and natural capital is only a neceisary, and not a sufficient, condition for sustainability. Failure to marage the natural environment adequately is often related to inadequate ;ducation and training, weak institutions, and poor understandir of sociocultural traits. Often, too, infusions of physical and financil capital have proved much less productive than expected, because of shortages of the other forms of capital. This is particularly evident where the institutions charged with managing development have been weak. Sustainability and Proiect Analysis 4.05 In theory, a concern with sustainability can be incorporated into project analysis based on the economic rate of return. In appraising a project one may assume that (1) the ERR analysis can properly handle distortions and externalities, through shadow pricing, while the effects on the environment may be adequately reflected in the estimates of cost and benefit streams; (2) the project9a financial viability and its fiscal effects, and likewise the organizational capacity of the implementing agencies, can be properly assessed; (3) risks to the environment can be adequately hedged in the design of the project; and (4) appropriate assumptions can be made about the reinvestment of the project's proceeds and the valuation of its assets at the end of the evaluation period. However, for several reasons, sustainability analysis offers important additional insights for decision making. 4.06 Choice of assets for measurement. A project's assets may include a natural resource (such as land or water, in an agricultural project), an institution (such as a primary education system in an education project), or physical infrastructure (such as roads, dams, or canals in transport or public utility projects). Although in principle there should be no discrepancy between the assets used to measure projects' contributions to growth and those used to assess their sustainability, in practice there are - 29 - major differences. In circumstances where the key determinants of a project's sustainability are tinancial and physical capital, performance analysis based on the ERR will give an adequate indication of a project's sustainability. If, for example, the only factor jeopardizing an irrigation project is a lack of funds for its operation and maintenance, performance analysis based on the ERR should yield the same conclusions as sustainability analysis, and the latter is redundant. 4.07 In circumstances where a project'a sustainability is influenced by its effects on natural resources--where, say, the irrigation project threatens to damage the soil and ultimately to reduce the productivity of its target farmers--an indication of the project's sustainability can be obtained by incorporating natural resource considerations into ERR analysis, by: - expanding the matrix of benefits and costs, taking account of nonmarket transactions, spatial external effects (upstream and downstream of the project), external effects on future generations, and the project's potential for causing or avoiding environmer-al disasters or irreversible damage to the environment; - computing shadow prices in a way that gives a monetary value to environmental goods (such as trees, riverine waters); - emphasizing how the project affects assets or resources that are essential to the preservation of the environment, even if the effects are felt only over the long term. 4.08 Project analysis has not been able, however, fully to take account of the level of institutional development or of cultural variables which, experience shows, are extremely important determinants of sustainability in Bank-supported operations. For example, if an irrigation project's operation and maintenance is held back not only by lack of funds but also by lack of engineers, by inadequate standards, faulty cost recovery policies, or inadequate rules for managing and conserving irrigation water, or by beneficiaries' aversion to paying for water or to organizing groups to perform local maintenance work, then sustainability analysis will yield insights additional to those provided by project aralysis. 4.09 As regards natural capital, several methods now available permit economists to take account of externalities related to spatial linkages. The Bank's Environment Department has actively disseminated these methods, and will continue doing so. As regards methods for the analysis of cultural and institutional factors, more work still needs to be done. 4.10 Time dimension. Sustainability analysis is often concerned with the effects of a project over the very long term. To assess whether a project will go on yielding a stream of benefits over the long term, performance analysis based on the rate of return is often of limited help. A high economic rate of return is not a sufficient condition for an - 30 - operation's sustainability. In particular, the discounted cash flow analysis that is used in examining the economic returns of a project tends to be biased against events or resource allocations far in the future. 4.11 Portfolio effects. The sustainability of a project may be reinforced or compromised by other projects in the country's portfolio. Institutional development projects, for example, may enhance the sustainability of all other projects, while projects with large recurrent costs--unless they are good at mobilizing resources to cover these costs- -may starve other projects of the funds needed ftor investment or for operation and maintenance. The combined portfolio effects may in some instances be more important to a project's sustainability than elements of the project itself (see Chapter 5). 4.12 Limitations of the proiect focus. The experience evaluated for this review emphasizes the great extent to which the fate of projects depends on sectoral and macroeconomic policies, and also on the general level of education, command over modern technologies, institutional development, and cultural and societal norms in the borrower country. Though it may be possible to approximate the action of such factore in shadow prices, doing so often means that potential difficulties in implementing the project are glossed over, and potential ways of addressing them are not decided upon. B. Sustainability of Bank Operations 4.13 In its evaluation work OED classifies operations according to different levels of sustainahility: likely, marginal, uncertain, and unlikely. Sustainability is judged uncertain where the key factors involved are particularly hard to predict--for example, where an operation faces a particularly uncertain market or policy environment, or where the institution charged with implementation is young and fragile. 4.14 For the present review, an Pralysis was made of all the 557 Bank- supported operations on which OED has made judgments as to sustainability (Table 4.1). This group, which constitutes all the operations evaluated audited during 1986, 1987, and 1988, represents about a third of the operations audited by OED during the last 15 years, and is more representative than those whose sustainability was analyzed in previous annual reviews. 4.15 A reservation should be noted here about the timing of the judgments made. Most audits are undertaken soon after disbursements are completed, and it may be premature to make judgments at that stage about sustainability. (This is not unique to the analysis of sustainability; evaluation of the performance of long-gestating projects, such as forestry and land management projacts, faces similar problems.) Since the issues surrounding sustainabilit7 are multidimensional and diverse, and several aspects of the assessments made are qualitative, not quantitative, the judgments made on sustainability (i.e., "likely", "marginal", "uncertain", and "unlikely") must be carefully interpreted and used. - 31 - Table 4.1: SUSTAINABILITY ANALYSIS: NUMBER OF OPERATIONS ANALYZED, BY SECTOR OF ECONOMIC ACTIVITY Agriculture and rural development 178 Industry 27 Development finance corporations 6 Poier and energy 6 To scommunications ie Water supply and waste disposal 51 Transport 62 Urban 29 Education so Population and nutrition 4 Nonproject lending and technical assistance 2 Total 657 Source: Annex 1, Table 1-21. 4.16 Of the 557 operations analyzed, 15 percent were judged unlikely to be sustainable; 52 percent were judged likely to be sustainable; and the rest--fully one third--were judged either marginally sustainable or uncertain. Grouping the operations by region (Table 4.2), Africa had the lowest percentage of projects classified as likely to be sustainable (only 32 percent, which is highly correlated with overall performance); Asia had the highest (65 percent), followed by EMENA (60 percent) and Latin America (48 percent). Table 4.2: SUSTAINAILITY OF OPERATIONS: REGIONAL AND SECTORAL BREAKDOWNS (Number of operations, and percentages) Likely Unlikely Marainal Uncertain Total R.egion Africa 48 32 86 28 10 18 87 27 18a Asia 117 88 15 0 9 6 89 22 18 ElENA 77 60 11 9 8 a 32 25 120 LAC 54 40 19 17 15 18 25 22 118 AVERAGE 291 62 88 15 Be 9 183 24 657 Sector Agriculture 91 51 86 20 17 1 84 19 170 Industry 14 62 6 19 4 18 4 15 27 Development finance corporations 87 88 4 7 2 4 18 28 6 Power and nergy 82 64 1 2 1 2 18 82 Se Telecommunications 9 96 1 19 0 0 6 6 10 Water supply and waste disposal 26 51 4 8 6 12 1 29 61 Transport 41 68 12 15 9 11 20 24 82 Urban 5 17 14 48 1 8 9 81 29 Education 22 44 8 6 6 12 19 88 6 Population and nutrition 2 5e 1 25 6 6 1 25 4 Nonproject lending and technical assistance 12 as 2 16 4 26 2 1 26 - 32 - 4.17 In operations in agriculture and education, evaluation reports consistently note that sustainability depends on a very wide range of factors. Though the numbers of projects reviewed are small, sustainability seems to be much easier to achieve in Asian than in African agriculture (more than two thirds of the agricultural projects in Asia were rated sustainable, compared with only one fifth of those in Africa). Of the 29 urban operations reviewed only five were judged likely to be sustainable. 4.18 Looking at the 557 operations by their years of approval, no intertemporal trends are clearly discernible. Determinants of Sustainability 4.19 Project Completion Reports, Audit Memoranda, and any other project reports deemed relevant for the exercise were reviewed to identify the principal determinants of sustainability of operations in different sectors. Following a typology introduced in AR84, judgments made by operational staff and evaluation officers were grouped into the following broad categories of determinants of sustainability: Economic and financial factors: mainly related to resource mobilization and allocation of financial resources at the national and project levels. Technical factors: mainly related to the influence of project design on sustainability. Institutional factors: including institution building and strengthening, technology transfer and adaptation, socioeconomic and cultural factors, and the degree of government support. Policy factors: including the extent to which country economic policies support the goals of Bank-financed operations, and the effects of policy changes. 4.20 The relative importance or frequency of mention of an particular determinant was recorded on a sector basis, and an analysis was carried out to identify any sectoral patterns that emerged. (Annex 1, Tables 1-22 to 1-25.) 4.21 In education and agriculture, sustainability depends on factors in all these categories. In agriculture the overall results are consistent with earlier findings, in that sustainability depends heavily on a strong commitment by governments to the principle of rural improvement, matched by a record of positive action in mobilizing resources and developing the institutional, financial, and human capital that will be needed once donors9 involvement ceases. In certain types of agriculture and education projects, sustainability depends on institutions at two ends of the spectrum: those at the grassroots or beneficiary level and those at the - 33 - national level. Difficulties in finding the right balance between the roles of institutions at these different levels have made many such projects hard to sustain over time (see Box 4.1). 4.22 Within agriculture, the analysis confirms that beneficiaries' participation in decisionmaking and in Implementation increases the efficient use of human, institutional, and cultural capital in livestock projects, rural development, and area-based development projects, and is a major determinant of these projects' sustainability. The analysis also shows that institutional strengthening is central to sustainability in all subsectors; that fiscal policy is central to success in irrigation, treecrop projects, and rural development projects; that government commitment is central in livestock, agricultural research, and extension projects; and that design and technology transfer issues are major determinants of sustainability in rural development, irrigation, agroindustries, and program credit projects. 4.23 Sustainability in development finance companies, transport, power, and oil and gas projects is greatly affected by the policy environment (particularly pricing and exchange rate policies), the availability of public funds, the capacity to mobilize financial resources, and economic conditions in general. Most such operations depend heavily on having access to enough financial resources to maintain the efficiency and productivity of the physical capital they create. In most of the cases reviewed, capital or investment costs were earmarked during construction in order to ensure completion, but funds for operation and maintenance or recurrent costs were not. The extent to which some operations will maintain an adequate level of benefits is thus not evident. In many of these cases, sustainability has been determined by the capacity to compete for the scarce financial capital available in the income or savings accounts of governments. 4.24 Sustainability in transport, urban, and water supply and waste disposal projects, as well as agriculture and education, could be greatly improved by focusing on factors that are often more under the control of those responsible for projects. In most such projects, sustainability critically depends on preparation and design, the choice of technology, operation and maintenance, and availability of raw materials. Both the Bank's regional staff and evaluation officers have emphasized the importance for sustainability of decisions made at the preparation, design, and appraisal stages. In these projects the design issues concern not only the engineering and technical aspects of projects, but also the institutional and human capital aspects. 4.25 For both agricultural credit projects and development finance corporation projects, the most important determinant of sustainability is institutional capability (e.g., the financial viability of the lending institutions) followed by issues relating to cost recovery and financing. Institutional capacity and sector management have been identified as the principal determinants of sustainability in projects in power and transport, as well as in agriculture and oil and gas. - 34 - Experience in Sustainable Projects 4.26 To help advance understanding of the determinants of sustainability, an analysis was also made of the experience of the group of 291 operations that OED had judged likely to be sustainable, as described above (Table 4.1). The analysis of this group of operations draws partly on findings reported in AR86 and AR87, and also on those of impact evaluation reports, on audits reported It special studies on sustainability since 1984, and on the few available country studies that examine sustainability. In all the sustainable projects, six major elements stand outs 4.27 Institutional development. Most sustainable Bank-financed operations put a major omphasis on the development of the institutional capacity required to sustain benefit flows after their completion. Their experience also shows tho importance, for the sustainability of individual operations, of the general level of education and institutional development in the borrowing country at large. Development projects often require people with a very high level of skills. As the complexity of projects increases, the demand for human capital becomes more critical to sustainability. Prospects for sustainability are better in projects where the Bank has been able to ensure that the goals, and intended means of achieving them, are compatible with the attitudes and cultural values of their intended beneficiaries. 4.28 Macroeconomic policies. Good projects may fail if they have to operate in inhospitable environments. Most of the evaluation reports on sustainable projects mention that the macroeconomic policies of the borrowing countries were conducive to their profitability and efficient operation. 4.29 Manaitement. Most successful operations have had good managers able to sustain the development effort. Audits indicate that management is a critical factor influencing sustainability in most sectors of the economy, and particularly in development finance institutions, industry, and oil and gas projects. 4.30 Resource mobilization. The capacity to mobilize financial ree...rces after donor financing stops is as important as the initial capacity to absorb physical and financial capital. At the time they were evaluated, all of the 291 sustainable operations had adequate resources to finance their recurrent and operation and maintenance costs. They used efficient, agreed-upon, mechanisms for sharing the cost of development over time. Examples of this approach include the development of comprehensive cost-recovery mechanisms, establishment of labor-cost allocation plans, identification of responsibility for the cost of project goods or services, establishment of a price system compatible with the accomplishment of national socioeconomic objectives, and identification of the levels and sectoral patterns of nominal or effective rates of protection. 4.31 Two findings related to project financing are worth mentioning here. First, the sustainable projects did not try to impose externally- - 35 - determined financing solutions; the financing mechanisms proposed for resource mobilization were indigenous. Second, to prevent resources from drying up soon after project completion, intermediate solutions were often used, such as the financing of several project "time-slices,' to provide time to mobilize funds from local sources. But such solutions are only temporary. Long-term solutions require that macroeconomic policies be consonant with financing needs at the project level. Though some sectors are more vulnerable than others, the need for financing a development effort often may come into conflict with the need, pursued in many structural adjustment programs, to reduce the fiscal deficit by reducing public expenditures. (See Chapter 5.) 4.32 Borrower and government commitment. A large number of audits have shown the importance of beneficiaries' participation to the sustainability of projects; some have observed that, under some circumstances, comunity participation is the only way to mobilize resources. Several examples are given in OED's study of "Cost Recovery in Irrigation Projects", which shows that successful resource mobilization, during and after implementation, was directly correlated with community participation. 4.33 A broader commitment, on the government's part, is often needed to carry out the institutional and policy reforms needed to make the socioeconomic environment hospitable to sustainable growth. The importance of this commitment cannot be overemphasized, particularly as projects become increasingly sensitive to changes in the macroeconomic anvironment, both domestic and international. 4.34 Strategy. Most of the sustainable operations formed part of a clearly defined national development strategy. The need for a strategy under which individual projects contribute to overall development goals is becoming a common theme in impact evaluation reports. Without such a strategy, a country0s portfolio of investments responds to short-term imperatives, such as balancing the budget, rather than to long-term development goals. As a result, projects are much less likely to be sustainable. 4.35 The explicit recognition of long-term goals is particularly critical to sustainability in operations to improve education; capital markets policy and administration; projects whose goals take a long time to achieve, as in activities to regenerate the environment; and projects with a very long gestation period, for example forestry projects. In this regard, emphasis should be placed on the role played by the Bank's sector work in facilitating the process of reaching consensus among all parties concerned. C. Conclusions 4.36 Of the findings that emerge from this review of the sustainability of operations, four should be stressed. First, given the strong influence of institutional factors in determining sustainability throughout the - 36 - economy, it is Important to intensify existing efforts to establish the necessary framework for carrying out institutional analysis in all the Bank's operations. In.ome cases, Improving project management skills Is critical. In others, it may be necessary to help improve indigenous institutions and organizations both by enhancing their internal management and procedures (e.g., as in most industrial and development finance company projects) and by strengtheninr their links with other institutions. As projects become more complex, and particularly when they involve multiple institutions, institutional capacity becomes increasingly critical to success and sustainability. Some policy reforms undertaken through structural adjustment programs also require sophisticated institutional capabilities for both their design and implementation. Institutional analysis requires an understanding of cultural values, political systems, and the motives of the actors involved in the development process. The explicit recognition of these factors can help ensure that projects are designed or restructured in ways that will improve the overall management of sectors. 4.37 Second, the goals of investment projects and programs need to be consistent with the macroeconomic policies being pursued. Most of the projects evaluated by OED were approved at a time when there were no SALs or SECALs in the countries concerned. Country policy matters tended to be seen as beyond the purview of those designing and appraising projects. Yet the latter also tended to make overoptimistic assumptions about governments' ability and willingness to implement the policy changes that would have provided projects with the "right" set of incentives. Today, the Bank's dialogue with borrower countries has broadened, allowing a more integral consideration of questions of macroeconomic policy and questions of project design and implementation. 4.38 Third, public expenditure and investment reviews should continue to play a central role in planning for sustainability. 'What may appear to be optimal for an individual project may not be beneficial in the context of the economy as a whole. Some trade-offs can only be settled at the national level, when the whole portfolio of investments is analyzed; issues related to financing and resource mobilization, to improved management and monitoring of the environment and of natural resource use, and to poverty and equity are all more amenable to control at the national level than at the individual project level. Questions of project financing in the context of adjustment programs, and the role. of public investment and expenditure reviews in helping to ensure the financial sustainability of projects, are taken up in Chapter 5. 4.39 Fourth, the Bank's future sector work, lending strategies, and advice to borrower countries should recognize the sustainability of operations as an important objective and should consider ways of helping countries to maintain an efficient balance among all forms of capital- -physical, financial, natural, institutional, cultural, and human. In many countries, debt difficulties and shortages of financial capital place a considerable strain on other forms of capital, and make sustainability more difficult. Land and water resources are being seriously depleted in some - 37 - countries, and significant amounts of financial, physical, and human capital may be required to restore the balance. Given the short-term needs and constraints facing many developing countries, the balance may never be restored. It is critical, therefore, that sector work begin to focus on the problems confronting borrower countries in sustaining developmental progress, and that the desirable changes in development strategy, lending, and research priorities be clearly identified. -38 - Box 4.1 a SUSTAINABILITY IN EDUCATION PROJECTS An GED study on the sustainablitty of equipment is poor and deteriorating, and education projects and their long-term curriculum problems persist, both in the effects Is not yet finished but its relevance of curricula In use and In the preliminary conclusions on two education capability of the system to Implement projects, one In Indonesa and one in Sierra changes. The moat Important endogenous Leone, are Illusrative. The project in determinants of sustaiability are Indonesia has ranked well against most shortcomings in design (e.g.$ inadequate sustainability criteria applied; the main training of staff, inattention to curricula), reasons appear to be satisfactory design, and the meet Important exogenous ones are implementation, and government policy inadequate post-completion funding and lack support. However, financial support for of policy support. The lack o? funds for these project.' recurrent costs has been recurrent cost. has been aggravated by the declining and their ability to sustain an expansion of subsequent projects In the adequate level of benefits is eroding. sector and in other sectors as well. The project In Sterra Leone has been judged The study has used the following framework in poor from a sustalnability standpoint. The assessing the sustainability of education maintenance of physical facilities and projects: A. Indicatore B. Factors Influencing Sustainablitty 1. Input Indicators 1. Internal factors Benefits Design characteristics Physical facilities Appraisal procedures Teachers Initial funding Materials A equipment Provision for adequate monitoring 2. Output Indicators Implementation characteristics Benefit. Adequacy of planning A management Delay in, e.g., procurement Enrollment trends Flexibility to change Graduates Supply of teachera students Administrative services 8. Efficiency indicators Post-completion characteristics Adequacy of recurrent funding Efficiency Adequacy of institutional support Textbook: student ratios 2. External factors Retention A completion rates Recurrent cost per student Trends in macroeconomic variables Utilization rates Trends in education sector conditions 4. qualitative Indicators Public expenditure on education Curriculum relevance Distribution of public expend by Project relevance source Adaptability to changes 3istribution of public expend. by recipient sector 5. Spinoff Indicators Tends In subsector conditions Institutions Policy changes Implementation of further projects - 39 - 5. FINANCIAL ASPECTS OF SUSTAINABILITY A sector-by sector review of problems with a maintain and operate their assets bearing on the financial eustainobility of officiently. Their services have Bank-tinanced projects highlights the deteriorated, and many have resorted to difficulties frequently caused by cost over- short-term borrowing, at high costs to runs and by difficulties in meeting recurrent themselves and to the economies at large. costs. Financing for possible cost overruns needs to be arranged with the same care as a The financial sustainability of projects project's basic financial plan. Many demands a focus on finance at both the projects have suffered from severe financial macroeconomic and the sector or project constraints during their Implementation level. At the project level, fiscal phase; cost recovery, whether directly, from implications are often not carefully customers or beneficiaries, or indirectly, analyzed. Nor do fiscal systems encourage out of allocations from government budgets, explicit consideration of the links between has often been much lower then called for in the financial structure of projects and the project design. Because of financial short- country's cost recovery and macroeconomic falls, many projects have been unable to policies. 5.01 Several factors mitigate against projects' financial sustain- ability: - inadequate financial systems at the country level, incapable of making a strong contribution to capital accumulation; - higher-than-expected costs of producing and delivering projects' goods and services; - - the preference of many donors for financing new projects, rather than giving institutional and financial support for operation and maintenance, to improve the efficiency of on- going projects; - lack of rounterpart fund& for projects after the investment phase is completed; and - poor financial returns, or insufficient cash flow, in economic activities that are nonetheless expected to generate substantial resources in the long term. 5.02 The complexities surrounding the financial sustainability of development projects call for an evaluation framework capable of assessing the management and performance of enterprises; trends in the availability of finance, both domestic and international; and changes in projects' operational effectiveness (as suggested, for example, by compliance with covenants, the state of financial management, or data from project monitoring). Such a framework has yet to be developed, but the findings of - 40- this chapter represent a first step in setting priorities and establishing an agenda for action. The analysis that follows draws on experience with the same 557 projects (evaluated in 1986-88) whose sustainability was discussed in Chapter 4 above. It also makes use of special studies and impact evaluation reports. A. Financial Sustainablifty of Projects 5.03 At the project level, several types of problems have recurred throughout the Bank's lending program (see Box 5.1), while others tend to arise more often in some sectors than in others. 5.04 In agriculture, four types of financial problems have been very commons shortages of local funds; inadequate budgetary allocations for recurrent costs; inadequate capacity to mobilize resources and establish sustainable mechanisms for recovering costs; and weak agricultural credit systems. 5.05 In industry and DFCs, the most common financial problems have arisen from weaknesses in mobilizing resources; difficulties in achieving financial and institutional sustainability; high foreign exchange risks; and weak management of liquidity. (See Box 5.2.) Many of the problems can be traced to borrower countries' monetary and fiscal policies and the capital market development strategies being pursued. But in the financial management and policies of development finance operations, a great deal of improvement could in principle be achieved through actions taken at the project level. 5.06 In electric power, the pursuit of the most efficient resource allocation at the sector level often seems to have led to the choice of investment plans of doubtful affordability over the long term. Difficulties commonly noted have been: projects' inability to cover their operating expenses and debt service; poor compliance with covenants; and low levels of cost recovery, attributable to the tariff and pricing policies in force. These issues have been noted in power-sector evaluation work for several years, and probably will be for some time to come. In fact, recent evaluation work suggests that such problems are becoming more common in several countries as government borrowing crowds out the alloca- tion of funds to the private sector, and as the debt problem, both domestic and foreign, becomes more acute. 5.07 In transport, water supply, and urban operations, many projects have been beset by difficulties in mobilizing resources, at both the national and local level; by inadequate systems for pricing the services provided; and by the generally poor financial managimment of entities providing such services. Problems of cost recovery, common to most of these projects, are discussed later in this chapter. - 41 - 5.08 In education, most of the financial difficulties that have arisen are related to public sector finances and to the management of recurrent costs. Cost Overruns 5.09 Cost overruns are a very frequent problem in project implementa- tion, and in some cases they have become an important constraint on financial sustainability at the project and country level. In the earlier part of the period reviewed, cost discrepancies were mainly the result of insufficiently detailed project design. But after 1973, factors outside the direct control of the Bank and its borrowers--inflation, the strength of the US dollar, the high price of imported oil, and declining terms of trade--assumed more and more importance. Based on the extent and serious- ness of the risks of cost overruns occurring, contingency plans will need to be developed. 5.10 The financing of cost overruns raises a number of issues which may not be adequately handled under current Bank procedures. First, internal rates of return, whether economic or financial, are a poor measure of the impact of a cost overrun on a project's profitability. These rates reflect costs and benefits averaged over a project9s life; they do not highlight short-term imbalances which may occur as a result of the overrun and may put the project in jeopardy. 5.11 Second, while cost savings are often captured by a project's funding agency, cost overruns are often financed by the borrower. From the experience documented in project completion reports, there are four typical responses to cost overruns in Bank projects. These can have widely differing effects: - Increased contributions by governments, presumably in the form of additional equity. - Additional long- or short-term borrowing, which increases debt service and reduces profitability. Short-term borrowing to finance overruns is often hazardous because it may result in serious structural imbalances in the way the project is financed. - A change in the project's scope, which is either reduced or shifted toward less costly alternatives. This can be quite damaging because the project's size and scope should have been optimized at appraisal. - A second-phase Bank project which, among other objectives, provides funding for whatever could not be done in the first phase. 5.12 To avoid prejudice to projects, the Bank should ensure that financing for possible cost overruns is arranged with the same care as the - 42 - basic financial plan. First, the risk of overrun and its possible scale should be assessed by reference to the state of engineering, the firmness of the project's cost estimate, and the currencies of procurement. Contin- gency plans might be necessary, whereby the entity, government, and co- financiers structure the likely needs for funds, on the one hand, and debt servicing capability (or dividend potential), on the other. Depending on the extent of the problem, contingency plans should be covered in the Loan and Project Agreements, or be the subject of a separate agreement (as in the Project Funds Agreements of the International Finance Corporation). Cost Recovery 5.13 In a number of sectors where the Bank has a very large lending portfolio (irrigation, energy, low-cost housing, for example), cost recovery is a central determinant of project sustainability. Instruments for recovering costs vary a great deal depending on the type of project and the country's economic environment, legislative framework, public finance policies, development objectives, and physical and ecological factors. Broadly speaking, they are of two typess (1) direct cost recovery through, for example, billing for sales, collecting tariffs, or levying users' fees for services rendered (in the case of some public services, it may be impracticable or undesirable to charge for all cost components); and (2) indirect cost recovery, usually achieved through general taxation (not directly related to the services provided to beneficiaries). 5.14 Whether direct or indirect methods are used, the level of cost recovery is usually governed by considerations of political economy. In irrigation projects, for example, "water charges are often difficult to implement because of strongly held traditional attitudes and values about water access which make water charges difficult to introduce or change" (AGR/OPS Policy Note of March 20, 1984 on "Financing Operations and Maintenance in Irrigation", para. 20). The financial dimensions of cost recovery are only one aspect of a very complex set of institutional, political, and social problems. 5.15 Empirically, how has the level of cost recovery affected the financial sustainability of Bank-supported projects? Many of the projects evaluated faced severe financial constraints during their implementation phase. In a number of sectors where the Bank has a very large lending program and cost recovery is a central determinant of project sustain- ability, cost recovery from beneficiaries was often much lower than called for in the project design. In most public utilities and other public sector enterprises that recover costs directly through earning revenues, cost recovery depends strongly on the pricing policies followed and on demand for their output. In many instances, unduly high forecasts of demand have provoked overinvestment in such entities, leading to upward shifts in supply, with a corresponding lowering of prices. In others, tariff levels or user charges have been kept too low to yield a positive cash flow. Both sets of circumstances militate against financial sustain- ability. - 43 - 5.16 In agriculture, in many instances cost recovery has been poor, and as a result project-related infrastructure is not being properly operated and maintained, so that massive rehabilitation is sometimes needed. The services provided have deteriorated over time, as has these projects' ability to generate the expected benefits. Government efforts to increase revenue have typically been weak. But even when the revenues collected have been large enough, in principle, to cover project costs, their alloca- tion back into agriculture very often has not matched the original objectives of the projects. Implementing agencies are often not financially autonomous, and even where they are, in some cases the beneficiaries of projects are too poor for full cost recovery to be appropriate. These findings suggest that increasing attention should be paid to the public finance aspects of agricultural development and, in particular, to the way in which taxes and subsidies affect the allocation of resources for agriculture at the farm and regional level. 5.17 In many irrigation projects, the revenues generated have not satisfied the intentions of the various covenants; they have continually fallen short of projects' operation and maintenance costs, let alone meeting the capital costs. In some instances, even where all the funds collected from the beneficiaries have been allocated back into the irriga- tion system, they have covered only a small share of the requirements (e.g., for maintenance, repairs). In several irrigation projects, problems have arisen because international prices of the commodities farmers produced have been lower than anticipated. 5.18 In the water supply sector, experience with unaccounted-for water losses has bean particularly disappointing. One noteworthy exception is a water supply project in Botswana, where substantial tariff increases were applied to offset revenue losses caused by drought. The real autonomy given to the parastatal agency administering the project permitted it to choose direct cost recovery instead of subsidies. B. Financial Unks Between Projects and the Macroeconomy 5.19 The findings from this review suggest that to promote the sustain- ability of development investments, public sector finance at the macro level requires much more attention than it has received in the past. At the project level, fiscal implications are often not carefully analyzed; there is seldom explicit consideration of the links between the financial structure of projects and borrowing countries9 cost recovery and macro- economic policies. 5.20 Many project-implementing agencies are not financially autonomous, and many depend on allocations of tax revenues from the general budget. In many projects, funds are collected from beneficiaries but are not earmarked for the use of the project. Projects that finance their recurrent costs out of profits may suffer due to inappropriate macroeconomic policies--for example, if a new government introduces interest rates that are negative in real terms, a development finance corporation will be unable to stay profitable and will lack resources with which to finance its operations. - 44 - 5.21 Most of the projects OED has evaluated were approved and began their construction phase in the period 1974-80. By and large, this was a period of cheap borrowing and easy liquidity. New investments were under- taken on a large scale, often without comprehensive financial plans and often in the context of national policies that were not conducive to their financial health. Borrowers tended to assume that liquidity would still be easy once the construction phase was completed, and that the recurrent costs of operating and maintaining these projects, and meeting their further needs for investment, would easily be met. !>.22 The day of reckoning came in the 1980, once most of the projects had entered their implementation phase. Many revenue-generating projects that could have been sustained financially through adequate pricing policies were not, and pressure over scarce financial resources mounted considerably. In many of these projects, toe, particularly in electric power, demand for their output was lower than had been forecast. Projects dependent on the public budget fell victim to fiscal tightening. Because of financial shortfalls, many projects could not maintain and operate their assets efficiently, and their services deteriorated. Many resorted to short-term borrowing, at high costs both to themselves and to the economies at large. 5.23 For many countries, the recurrent costs of investment projects and the amortization of debt associated with them have become a central issue in the design of macroeconomic adjustment programs. Many of these programs demand more deliberate investment planning and large cutbacks in public sector budgets. These measures will gradually bring expenditures on investments and recurrent costs into line with available resources. Effects of Structural Adiustment Lending on Proiects 5.24 Though no study has been done to quantify the extent to which adjustment programs have directly affected the sustainability of individual projects, the evidence from project evaluation work shows clearly that macroeconomic variables (such as pricing, tariff policies, public finances, and the level of institutional development) are important determinants of project sustainability in most sectors. 5.25 Two points should be stressed. First, Bank lending in support of adjustment programs strengthens macroeconomic incentives and should thus lead to a more efficient allocation of resources, while by improving the climate for savings and investment it should enhance the borrowing country's capacity to mobilize financial resources. It also helps to strengthen the institutional framework for policy analysis and for invest- ment planning and evaluation. All of this should help to make development efforts more sustainable. Second, however, adjustment programs contain measures (for example measures to reduce public spending) that have often aggravated the funding problems of projects designed before the adjustment process got underway. Projects most affected are those with high recurrent cost requirements, a long gestation period, and operation and maintenance requirements that strongly affect their economic rates of return. - 45 - 5.26 Resource allocation. Efforts to Improve rssource allocation in structural adjustment programs have most often concentrated on the removal of price distortions, through adjusting overvalued exchange rates, eliminating negative interest rates, and liberalizing foreign trade, as well as reducing domestic taxes and subsidies that distort optimal resource allocation. Another major thrust of these programs has been to rationalize investment planning. In Turkey, for example, in 1980-83 the number of ongoing projects in the public investment program was reduced from 9,000 to 5,500, giving priority in the allocation of resources to the most efficient projects in export-oriented agriculture, energy, communications, and trans- port. By encouraging efficient project selection and directing investments into the most efficient anq! profitable sectors, SAL reforms have contrib- uted to growth and employment and strengthened the revenue base of many borrowing countries. 5.27 Resource mobilization. The structural adjustment programs of many countries have centered around an Imprevement in public savings perform- ance. Budget deficits have been brought down through a mixture of revenue- increasing and expenditure-reducing measures; in all the structural adjust- ment programs that have been evaluated, the revenue base improved overall. Though trade liberalization reduced the incidence of import tariffs, actual revenues from tariffs increased in some cases, because the volume of imports grew in relation to gross domestic product. Other sources of increased government revenue were improved tax administration and the imposition of new taxes (particularly value added and excise taxes); a general revival of economic growth resulting from an expansion of exports; and improved investment planning and the reduction or elimination of subsidies to state economic enterprises. 5.28 The funding available for projects was augmented by disbursements of SALs and the counterpart funds that were generated from other bilateral and multilateral assistance complementing the adjustment programs. In Cte d'Ivoire, for example, counterpart funds from the SAL were used to finance agricultural state enterprises as well as to finance government contribu- tions to several Bank-financed projects. 5.29 Production incentives. The better climate for savings and invest- ment created by SALs in several countries, with the reduction of inflation and improvement in exchange rate competitiveness, has stimulated export growth and diversification and enhanced the overall rate of economic growth. As a result, demand has been growing for the expansion of infra- structure investments, especially in transport and coamunications. This has opened up important opportunities for public and private investment, but at the same time it has intensified the stress on countries' financial systems and on their management and coordination cadres. 5.30 Institutional framework. Programs supported by SALs have helped to strengthen policy analysis, investment planning, and evaluation capabilities in key ministries and operating agencies of some borrowing countries. In a few cases, they have also helped to strengthen the banking system and capital markets. The sector components included in some - 46 - programs have helped to upgrade the analytical and policy making skills of planning agencies, while tb intensive economic and sector work and the accompanying policy dialogue with borrowers in the preparation of the programs has enhanced understanding of the role of appropriate prices in the incentive structure. For example, in C&te d'Ivoire, the first SAL strengthened and intensified the Bank's dialogue with the government on macroeconomic, sectoral, and institutional policy reforms, and helped to resolve issues in investment programming and rice pricing policy which could not have been adequately addressed in conventional project work. In Turkey, the SAL program was instrumental in the reorganisation of the Ministry of Agriculture, Forestry, and Rural Affairs, while in Mauritius the SAL program helped to Improve the management of nonfinancial public enterprises. 5.31 In Jamaica, the relationship between SALs and project lending grew progressively closer over a series of SALs. Between 1977 and 1984 the Bank approved seven balance of payments operations: one program loan, three export development loans, and three SALa. In addition, five projects were identified and their processing begun. In some areas, the SALs created frameworks that made project lending possible--for example, by improving the viability of certain public enterprises; in other cases, the SAL process identified needs for project lending, such as the Public Adminis- tration Reform Project. Conversely, project implementation revealed policy areas that needed to be addressed, and action programs for this purpose were financed through SALs. Portfolio Effects and the Role of Public Investment, Expenditure. Reviews 5.32 The review of evaluation reports shows that financial sustain- ability at the project level depends critically on the extent to which other projects in the country's portfolio are financially sustainable. For example, for revenue-earning projects whose profitability is depressed by low tariffs and inefficient systems for recovering costs from customers, public subsidies or allocations of scarce foreign exchange may effectively starve projects in public health, primary education, or agricultural research that, unlike public utilities, have no means to recover costs directly from the public. The lesson is to determine which projects ought in principle to recover their costs directly from beneficiaries, and to allow only those which cannot rely on direct cost recovery to compete for the overall mar.inal savings. 5.33 In view of these relationships, public investment reviews should consider sustainability issues systematically, assessing the distribution of benefits and costs over time, as well as the alternative forms of financing, for the whole portfolio of investments. Expenditure reviews should continue to focus on the financing of recurrent costs and the establishment of an adequate approach to resource mobilization for that purpose. To be avoided are investment and expenditure reviews with over- simplified and inflexible guidelines, which could be fatal to slow- gestating environmental projects and therefore detrimental to the sustain- ability of development more generally. - 47 - C. Conclusions 5.34 Several of the outstanding issues discussed are being addressed by Bank management (see, for example, thA "Report of the Task Force on Financial Sector Operations", R89-163, September 7, 1989). Nevertheless three main points with implications for the Bank's work stand out. First, under current circumstances, the financial sustainability of projects demands a focus on finance at both the macroeconomic and the sector or project level. One of the implications for the Bank is that the condition- ality attached to adjustment lending, and the advice offered on country investment programs and individual projects, should be formulated with reference to a common framework. 5.35 Second, given that the financial fate of one publicly-financed project closely depends on the financing allocated to other projects in the portfolio, public expenditure and investment reviews should play a central role in planning for sustainability. 5.36 Third, the many sources of risk to the financial sustainability of operations mean that the financial Implications of Bank operations should be carefully analyzed and monitored. 5.37 While the Bank's procedures, as laid down in Operational Manual Statements (OMS)/Operational Directives (ODs), have in general served well, the conclusions outlined above suggest the need for a review of some of these procedures, especially with regard to the followings - Current 2rocedures for project appraisal do not adequately address the issue of financial sustainability. Cash-flow analyses are not required and the standard funds-flow state- ments are not always a satisfactory substitute. - No systematic assessment is made of foreign exchange avail- ability. This is rather surprising since project completion reports frequently mention the lack of foreign exchange as a reason for a project's unsatisfactory Implementation or performance. - OHS provide hardly any guidance on how to deal with foreign exchange risk, except in DFCs--which are urged to transfer foreign exchange risk to their own borrowers. - OMS provide no guidance for financial analysis in conditions of rapid price inflation, even though rapid inflation is frequently mentioned in project completion reports as a major reason for cost overruns. - The current emphasis on tariff levels and structures as a means to ensure a company's linancial soundness or to promote economic efficiency pricing ignores the fact that tariff I I - 48 - increases may mask operational inefficiencies and that there are many other ways to increase profitability--for example, by reducing costs, increasing turnover, or modifying financial leverage. 5.38 Npnrevenue-earning investments. Present procedures are also weak regarding the sustainability of nonrevenue-earning investments. OMS say simply that "appraisal may include consideration of ... the impact on the government budget" and "for nonrevenue-earning projects ... forecast may be limited to annual expenditures during implementation and the first two years of full operation in order to demonstrate ... the recurrent financial costs of operation and maintenance and any incremental tax receipts resulting from the project." Under such guidelines, neither the impact of individual projects on government resources, nor the crowding-out effect by which some projects pre-empt the government resources needed by others, are examined during appraisal. Procedures need to be developed for the Bank to appraise the mechanism by which adequate funding will be secured to main- tain and operate the asset created by the proposed investment. 5.39 Covenants. Shortcomings in the Bank's handling of the financial aspects of projects are sometimes attributable more to the inadequate design and thus weak enforcement of covenants than to inadequate procedures. The Bank uses financial covenants as a means to insure against risks affecting the realization of a project's expected benefits. As stated in OMS 2.22, para. 4, "financial covenants provide a framework for financial discipline and represent a statement of objectives which the Bank considers important for achieving and maintaining satisfactory financial performance of revenue-earning entities." Frequently the covenants are not enforced. A recent review of project completion reports identified 15 explicit mentions of financial covenants including seven to the effect that the covenants had not been met, and two to the effect that the covenants had had to be changed because the projects could not meet their original conditions. An OED "Review of Cost Recovery in Irrigation Projects" (1986) concluded that the cost recovery covenants in such projects were seldom enforced and that the recoverf record had not been satisfactory because of poor government commitment, unreliability in the supply of irrigation water, which had made users reluctant to pay irrigation fees, and the often heavy burden of direct and indirect taxes already imposed by the government on the farming sector. 5.40 Financial reporting. Financial reports are the means by which the Bank monitors the progress of its investment; they are commonly the source of the data on which economic as well as financial judgments are based. But financial reports, althouga routinely called for in Loan and Project Agreements, are frequently received too late to be useful in project super- vision, and audit opinions often contain such serious qualifications as to cast doubt on the credibility of the data given. The Bank has developed a procedure to monitor compliance and, thus, ensure that delinquent reports are followed up, and that suspect data are clarified to its satisfaction. Even where reports are timely and r.ccurate, countries use a wide diversity of accounting standards and policies in preparing them (or use none at all), so that comprehension is made difficult and comparability between countries may require a number of adjustments to be made. - 49 - 5.41 Staffing. If the Bank is to pay more attention to financial issues, it will need to review the adequacy of the numbers and expertise of the operations staff who will have to address them. As stated in the Bank's reviews of ongoing operations, training may be required in topics such as accounting standards and the financial control of public sector enterprises, and new skills may need to be hired, for example in the sphere of public finance. Professional guidance and leadership for these staff, which has been lacking since most of the financial adviser positions were removed in the Bank's reorganization, will need to be restored, perhaps on a regional basis. - 50 - Box S. 1i SOME COlHON FINANCIAL PROBLEMS * Financing plans have not always been * Projet entitles often did not pursue the designed to best fit the needs of the potential gains from improvements in project and borrower, leading to later off!ciency and productivity. problems of debt servicing. 0 The financial performance of utilities * Many projects* financing plans contained was generally unsatisfactory, often no arrangements to cope with delays in because needed tariff Increases were the provision of counterpart funds. postponed. * Lack of contingency arrangements for * Increasing accounts-receivable created financing cost overruns added to liquidity problems for many project construction delays. entities. * The debt service due on foreign loans was * Arrears on the portfolios of DFCs often increased, in many cases, by severe depressed their profitability. depreciation of local currencies against the currencies owed. * The lack of recognized accounting standards, proper accounting systems, and * The level of cost recovery from project professional auditing hindered the beneficiaries was often lower than called diagnosis of problems and the for in the project design. Implementation of policies. - 51 - Box 5. 2: SUSTAINABILITY OF DEVELOPMENT FINANCE INSTITUTIONS A recent OED study traced the Impact of the The study offers a number of lessons for changing economic environment on development promoting sustainablitty. First, snce DFle finance institutions (DFle) assisted by the are reluctant to rely on economic appraisal Bank in smaller, less Industrialized, techniques in their selection of projects, countries, mostly in Africa. The study, the Bank should reduce Its insistence that based on six case studies as well as DED's DFIs carry out a mechanistic internal review of some 120 DFI projects since 1980, economic rate of return analysis as part of sought to explain why some DFIs have their project work. The Bank also needs to succeeded in maintaining or increasing their work directly with the governments concerned financial and economic sustoinability while to help reduce policy-induced distortions; others have not. In distorted economic this will bring about greater coneonance environments, financial and economic between the results of economic rate of profitability may diverge substantially, and return analysis and those of financial rate though profitability is necessary It is not of return analysis, which DFI do carry out. sufficient to ensure sustainability. Thus, The Bank should, however, be more realistic besides profitability, several other than it has sometimes been In the past In its performance Indicators of sustainability were expectations regarding the effectiveness of examined: financial sector contributions, policy dialogue between DFI and their resource mobilization, resource allocation, governments. promotion and development, and impact on policy. Second, several areas related to the Bank's operational effectiveness-particularly The study concluded that the Institutions appraisal and supervision--should be studied cannot generally be said to have improved. achieved sustainability in a broad sense. Most are capable of performing their basic Third, the quality of DFI projects should not financial function of transferring Investment be allowed to suffer In order to achieve resources to industrial borrowers at a quantity and timetable targets established profit, however small. But given their high under the Bank's lending program. Fourth, level of arrears, and the way they account when dealing with DFIs with majority for overdue Interest Income, their apparent government-ownership, the Bank must ensure profitability may be Illusory. They have that the DFI's autonomy is preserved. Fifth, contributed ittle to financial sector lending to DFIs should be made conditional on development, and their success in mobilizing satisfactory financial performance. The resources, other than from official sources, report concludes that, especially in less has been modest. In only a few instances advanced countries, there remains a case for have they taken on the job of promoting new Bank lending to individual DFIs if they meet entrepreneurs or new activities; where they tests of efficiency, competitiveness, and have, the initiative has generally come from viability. (IThe Sustainability of Develop- the Bank or the government concerned. ment Finance Institutions in an Evolutionary Environment', Report No. 7658, May 15, 1989.) I I - 53 - 6. MANAGING NATURAL RESOURCES, STRENGTHENING INSTITUTIONS: TWO ASPECTS OF SUSTAINABILITY For the pursuit of sustainablilty at the pricing and trade policies, fiscal and country level, proper management of natural resource mobilization programs, and resources and Institutional developmant are environmntal management. The Bank also needs major necessary conditions. This chapter to make a major effort to help borrower concentrates on these forms of capital by countries incorporate environmsntal concerns reporting on recent findings from evaluation into their macroeconomic planning. experience, which cut across countries and sectors and focus particularly on economywide To a great extent, the sustainability of managemnt issues. development depends on the character and performance of Institutions, at all levels. The first part of the chapter examines The second part of the chapter summarizes the experience with natural resources management experience with an important aspect of and its effects on growth and equity institutional development: technical objectives In the long term. It concludes assistance for public sector management. that the Bank must continue to enhance its While components aimed at strengthening capacity to help member countries manage Institutions feature in a high proportion of their natural resources. A better natural Bank-financed operations, and the approaches environment, and better management of natural used in them are very diverse, many of these resources, are goals that are vital to components are judged not to have had lasting sustainable development In many areas of the positive effects. The Bank should give more developing world; they must remain central prominence to carrying out a comprehensive considerations in the advice that the Bank institutional analysis in all its operations. gives on development matters in general. The Institutional issues identified in Making more concerted efforts to help its evaluation work are very diverse, among borrowers pursue styles of development that countries, sectors, and operations, and It are growth-oriented, equitable, and will be Important for the Bank to avoid sustainable may call for changes in the generalized prescriptions. frameworks that the Bank uses in advising on 6.01 This chapter reports on two important aspects of sustainability: the management of natural resources and the strengthening of institutions. The way in which developing countries manage these two forms of capital (as referred to in Chapter 4) will greatly affect the sustainability of economic development, in addition to presenting new challenges and options for the future. Drawing on some of OED's recent studies, this chapter reports briefly on the management of natural resources in some of the Bank's operations (both urban and rural), and on one aspect of the Bank's assistance for institutional development. A. The Bank and the Management of Natural Resources 6.02 A concern with the capacity of economic systems to survive, in the very long term, usually centers on the way countries manage their natural resources (assumed to be the scarce factor in the long term) and on the potentially negative effects of economic development on the environment. - 54 - This intimate relationship betryeen the achievement of economic development goals and of sustainability has often been referred to as "eustainable development". 6.03 Experience shows that where natural resources are mismanaged, growth as well as equity goals are often compromised over the long term. The results of this year's review show, however, zhat in the pursuit of sustainable development, forms of capital other than natural resources must also be taken into account. Thus, there is no point in adopting or financing an environmentally sound resource management strategy unless managers are available to manage it, adequate incentives are in place, and the development process is rooted in the society's values and beliefs. 6.04 For some time OED has given attention to the environmental dimensions of sustainable development, assessing how the Bank's operations affect the access to, use, or management of such natural resources as land, water, forests, air, minerals, and fuels. This has been donet - by evaluating the performance of investments in irrigation, forestry, and energy that directly affect a country's access to natural capital, or by assessing the actual use rates of resources; and - by evaluating the performance of operations that indirectly affect access to and the supply, use, and distribution of natural resources. 6.05 OED's attention to sustainable development has also focused on the particular sector and policy issues, constraints, and trade-offs among growth, equity, and sustainability that arise in the design or implementation of specific projects, and the department has a program of special studies in this area. To complement the information drawn from individual audits, the discussion that follows makes reference also to some special studies and impact evaluation reports. Bank Lending to Improve Access to Natural Resources 6.06 Sustaining the development of cities has been an Important element in Bank lending. In most cities of the developing world, population puts heavy pressure on natural resources. Bank-financed investments have helped to raise the carrying capacity of cities by improving land, water, and air, and by providing complementary physteal and institutional capital. The positive role the Bank has played in sustaining the development of cities, and in the process resolving difficult trade-offs between equity and sustainability, would be a fruitful subject for future study. 6.07 Water and waste-disposal projects supported by the Bank have expanded the carrying capacity of cities while enhancing the quality of life. In this subsector, the projects audited in 1988 alone amounted to more than $1.4 billion of total investment. Most of them were designed to improve or extend access to water and or sewerage services, particularly - 55 - for the urban poor. They also supported institutional improvements in the implementing agencies. 6.08 Audits show that the urban projects evaluated in 1988 have yielded important environmental and socioeconomic benefits. In squatter upgrading projects in Manila and three major Philippine cities, and in the low-income neighborhoods of Cartagena, Colombia, and Managua, Nicaragua, for example, major land reclamation and drainage works, together with the provision of sanitation infrastructure and housing improvements, have greatly improved residents' health and environmental conditions. As well as affecting the use of natural resources within cities, many urban projects also affect the use of the resources (for example forests, agricultural land, energy resources) that are needed to supply urban demand for food, shelter, fuel, and other resource-based goods. 6.09 The availability of raw materials is an important determinant of whether development is sustainable. Through its lending for industry, the Bank has made a substantial contribution to the expansion and improvement of raw materials in borrower countries, particularly in the resource- extracting industries in energy, chemicals, steel, paper, and fertilizer. OED Studies on Natural Resource Management 6.10 Equally important has been the Bank's assistance to borrower countries in finding a style of development that each can sustain. OED has a program of studies to help increase the Bank's understanding of the interplay between sector and country lending strategies, on the one hand, and environmental concerns at the macro level, on the other. These include studies on the management of renewable resources, the environmental effects of development programs financed by the Bank, and an overview of the Bank's role in the forestry sector (see Box 6.1). These studies focus on both the time dimension of the development process and the type of assets against which development performance or returns should be measured. Thus, attention is now being paid to natural assets such as land, water, air, and vegetation relative to other forms of capital. 6.11 A prospective OED study will evaluate the impact of Bank-financed projects on the environment in Brazil. The study will examine several large infrastructure and regional development projects. It will trace the evolution of environmental concerns within the Bank and in Brazil over several decades and explore the progressive integration of environmental considerations into Bank-supported development efforts. The projects to be reviewed include a representative range of investments in physical and social infrastructure, primary productive activities, and environmental protection. 6.12 Several of OED's impact evaluation reports also provide information useful for assessing the trade-offs between growth, equity, and sustainability (see Box 6.2). Recommendations 6.13 From OD's work to date on issues in the management of natural resources, three important preliminary recommendations emerge. First, the - 56 - Bank must further enhance its capacity to help its borrower countries manage their natural resources. The environmental dimension of Bank lending is central to the design of country strategies and should continue to be so. All Bank lending must be subect to thorough environmental screening. An improved natural environment and better natural resource management are critical to sustainable development and therefore must remain central considerations in the advice the Bank gives on development matters in general. 6.14 Second, the Bank should continue its practice of assessing the trade-offs among growth, equity, and sustainability at all stages in the project cycle and on a countrywide basis. It will need to make systematic and concerted efforts to help developing countries find styles of development that are growth-oriented, equitable, and sustainable. The little evidence that OED has at present on this subject shows that identification of such a set of priorities for each country might require important changes in the traditional frameworks used by the Bank in advising on pricing and trade policies, fiscal and resource mobilization programs, and environmental management (see Box 6.3). 6.15 Third, the Bank must make a major effort to help countries incorporate environmental concerns into their macroeconomic policy planning. Environmental aspects of economic development should not be considered in isolation from macroeconomic policies, as has very often been the case in the past. Production of a comprehensive environmental impact statement outlining the effects of given macroeconomic policies is not an adequate approach to ensure the sustainability of economic development; macroeconomic planning itself needs to take cognizance of the probable effects of alternative policies. 6.16 If the sustainability of development is to be a major consideration in the Bank's strategies and objectives, environmental questions must become an integral part of its adjustment lending programs, and such programs must also be planned to contribute to the achievement of sustainable development. Specifically, the merits of policies and institutional reforms should be assessed as these affect the availability and productivity of natural capital over space and time. This review does not suggest that each adjustment loan must have an "environmental component." What it means is that country economic analysis and the design of a country strategy should explicitly identify major environmental issues and consider: (1) whether there are any environmental issues that can or need to be resolved through policy reform appropriately handled as part of an adjustment program and (2) whether policy reforms might have serious environmental effects that need to be considered. 6.17 Sector work, public investment, and expenditure reviews should consider the sustainability of development as a central theme, paying attention to how individual projects or the whole portfolio of investments would contribute to this goal. This would require analyzing the potential effects of changes in public funds on the financing of recurrent costs; the implications of the reform programs for the natural environment; the institutional needs of the proposed programs (including the framework of laws, ruless and regulations); the human resource needs; the effects of - 57 - market and nonmarket incentives on the equilibrium across all forms of capital; and the spatial and intertemporal linkages among investments. 6.18 As stated in the report to the Development Committee, World Bank Support for the Environments A Progress Report (August 1989), "adjustment lending has not, until recently, paid explicit attention to environmental issues. It does, however, offer opportunities for Improving environmental management, for example, in reforms such as reduction in the subsidies for energy or pesticides.... It may be possible in related cases to use adjustment lending in the pursuit of specific environmental-related object!ves.... Adjustment lending reflects the growing recognition of the potentxal structural reforms in achieving environmental objectives...." The report adds that "several Important strategic issues remain unresolved. One is overcoming the limits of economic analysis in justifying projects and policies... . Another is providing the right information and analyses to hasten the integration of environmental concerns in country economic planning... . Another is developing national-income-accounting systems that correctly reflect the economic costs of resource degradation, particularly for countries that depend heavily on the exploitation of a rapidly deteriorating natural resource base." ... It is stated also that "an increased emphasis on environmental considerations in country economic work, and, in parallel, a training program for country economists is now required." 6.19 The Bank's research and policy program must receive increasing support to help find ways to incorporate environmental and natural resource considerations into national accounting systems. In their present form, these systems do not permit analysis of the environmental consequences of alternative economic policies; policies that appear sound from the point of view of their impact on the country's GDP may have a disastrous impact on the environment. B. Institutional Development 6.20 The Bank's experience, as evaluated by OED, has shown the great extent to which the sustainability of development efforts depends on the character and performance of institutions. As reported in earlier annual reviews, it has also shown that while components aimed at strengthening institutions feature in a high proportion of Bank-financed operations, and the approaches used in them are very diverse, many of these components are judged not to have had lasting positive effects. In many of the countries that most need it, there is rather little evidence of lasting achievement in institutional development through Bank-supported operations. 6.21 Imperfect though the measures of such achievements are, they raise questions about the attention the Bank has been giving to the institutional aspects of operations. Also cause for concern is the fact that development projects have tended to increase in size and complexity, requiring a growing diversity of skills from the institutions that implement them. 6.22 In the cohort of operations evaluated in 1988, more than 90 percent (154) were intended to make a significant contribution to - 58 - institutional development. In proportions roughly similar to those in the cohort evaluated the previous year, 36 percent of these operations were judged to have had substantial success, 41 percent partial success, and 23 percent to have accomplished very little in this area (Annex 2, Table 2-10). In the transport sector, many borrowers were reluctant to accept the technical assistance planned; many had only limited capacity to absorb technical assistance, and could not spare qualified staff to act as counterparts. The performance of technical assistance in agriculture was relatively good; the inputs varied from large consulting teams providing hundreds of staff-years to individual short-term assignments of a few weeks' duration. Technical assistance to energy and telecommunications projects seems to have been generally successful. In industry, evaluation reports note that for firms to make the most out of technical assistance, they need to be well managed. In urban operations, a need for more follow- up assistance was noted, particularly in those that called for a high degree of institutional integration during their implementation (as, for example, with the construction and financing arrangements for urban housing). 6.2:, This year's review, of all evaluated operations, has repeatedly shown the importance of countries' capacity to manage their economies, whether in achieving satisfactory performance at the level of individual investments or in pursuing a path of development, at the national level, that is sustainable over time. The rest of this chapter reports some findings of an OED study on one aspect of the Bank's assistance for public sector management. Technical Assistance for Public Sector Management in Africa 6.24 Depending on the borrower country, public sector management (PSM) projects aim to strengthen the institutional capacity of core and sectoral ministries, public enterprises, or public administration in general. The main instrument the Bank now uses to help strengthen public sector management is free-standing technical assistance (TA) loans and credits, particularly those made in the context of its lending for structural and sectoral adjustment. 6.25 In many countries, public sector management is both an important target of SALs and SECALs and an ultimate determinant of their success or failure. There is a growing awareness that, in Africa in particular, adjustment policies must be supported by an integrated institutional development strategy if they are really to succeed. And in numerous adjustrent loans or credits, it is the technical assistance component that provides the funds to meet specific requirements (such as studies) deemed necessary to satisfy conditions relating to public sector management. 6.26 Technical assistance for PSM has been extended especially to African countries, in most of which economic adjustment problems are severe and institutional development is still at an early stage. In 1976-88, lending for PSM in Africa amounted to US$566 million in 66 projects in 30 countries. Substantial increases tr-k place in 1987, when PSK technical assistance to Africa reached US$79 i .ion, and in 1988, when it reached US$143 million. PSM projects are the zastest-growing category of technical assistance in the region. - 59 - 6.27 In a study by OED, management issues were studied throughout the project cycle in twenty freemstanding technical assistance projects in support of public sector management in seven African countries (Cite d'Ivoire, Kenya, Madagascar, Malawi, Niger, Rwanda, and Senegal). The study found that though the technical expertise provided by projects was often successful in achieving specific and short-term goals, the institutional aspects tended to be much more more difficult to carry out. The implemen- tation of those components has only partly succeeded in building capacity within national institutions to perform their functions on a sustainable basis. 6.28 The relative success of some TA projects has been rooted in their careful identification and design and, especially, in a high level of interest and commitment on the part of both Bank and borrower. In many other projects or project components, such as SAL-related policy studies, the Bank developed various initiatives without first ensuring that the counterpart organization would be interested in pursuing them. Often the situation deteriorated further as the projects progressed, and as recipient countries proved increasingly loath to use borrowed funds for TA items such as training and studies. 6.29 In a number of cases where needs for particular TA activities had been assessed, an integrated institutional development strategy had not been prepared and so there was no coherent framework within which to place the proposed TA projects and their various components. A closely related problem was, in several cases, a lack of clarity about what exactly was to be achieved through technical assistance and what the priorities ought to be. Also, the Bank often underestimated the political sensitivity of certain policy or institutional questions; as a result, project designs called for reforms whose achievement was somewhat unlikely. 6.30 For most of the public sector management projects in Africa, the technical assistance provided by the Bank has either failed or only partially succeeded in developing sustainable national institutions. This conclusion is based on an analysis of the performance of the TA in strengthening overall capacity, in improving systems development, and in training staff. Positive results were found only in the case of the intro- duction of specific systems and procedures (for example for debt manage- ment, budgeting procedures, or statistical data bases). The ways in which various components or inputs (such as work of long-term consultants, training) were designed and managed by the Bank and the borrower country authorities also generally proved less than satisfactory or simply average; they rarely created sustainable changes and often failed to build up basic self-reliance or institutional viability in the recipient entities involved. 6.31 In the 20 projects reviewed for the Africa TA study, the record of institutional development is rather weak. It is not surprising that several borrowing countries have become increasingly worried about the opportunity costs of borrowing for TA, unless the latter is provided to an institution that obviously has the potential to absorb and exploit it. If - 60 - a program is to be realistic, project identification, design, and implementation require a greater level of commitment and a greater depth of political and economic understanding of the recipient country (in a national or sectoral context) than were evident in the Bank9s relationship with borrowers in most of the TA projects studied. Some projects (such as Kenya Agriculture I, Madagascar Management I and II, and components of the economic and financial TA project in Senegal), on the other hand, show that the Bank can provide effective TA and that the latter can enhance the capacity of borrower countries on a sustainable basis. Conclusions 6.32 For the Bank to encourage institutional development successfully through technical assistance, experience suggests the need for a high level of commitment and mutual interest on the part of the donor and the borrower; project design that capitalizes on existing opportunities and exploits the comparative advantage of different donors in a complementary fashion, within the context of an identified, country-specific, institutional development strategy; and a thorough basis in Bank sector work. Technical assistance has tended to be most effective where it has concentrated on strengthening technical, managerial, or administrative systems. 6.33 Attention to legal issues is also Important in institutional development (including the regulatory framework in which projects are carried out; the rules that define the powers, functions, and responsibilities of institutions as well as their interrelationships; and legal measures to ensure the sustainability of institutions). An inadequate recognition of the role of law in institutional development may have contributed to the failure of some institutional development efforts in the past. 6.34 Given the strong influence of institutional factors in determining sustainability throughout the economy, the Bank should give more prominence to carrying out a comprehensive institutional analysis in all its operations. Institutional analysis is demanding; it requires an understanding of cultural values, political systems, and the motives of the actors involved in the development process. But specific efforts to assess the institutional dimensions of every proposed operation and instrument would have a high pay-off. - 61 - Box 6.1s THE BANK'S ROLE IN FORESTRY DEVELO!HENT The Bank has been active in the forestry deforestation undermines the livelihood of sector for thirty-five years. During this fare families who, lacking wood, turn to period, perceptions of the role of forests in using crop residues and manure for fuel, development have changed substantially. rather than for fertilizing the soil, which Leading to the sector has grown considerably- then becomes increasingly unusable for -and is expected to continue to do so. agriculture. Forestry projects have been designed to respond to a broad range of needes energy, OED has begun a study to provide an Interim rural development, poverty alleviation, and perspective on Bank experience with forestry environsentally-related imperatives. The projects and to aeist management in economic and social significance of forests identifying directions for future policy and for development is now being explicitly activities in the sector. The study will acknowledged by decision makers. Forests in address sectoral and macroeconomic issues and developing countries have particular will examine the Bank's response to the goals significance for poor households, both rural and objectives outlined in its Forestry and urban, and especially for women in these Sector Policy Paper of 1978. The study will households for whom gathering firewood may complement and draw upon recent analysis of take several hours a day. In some Bank involvement in rural development, ciecusatances, as in parts of Ethiopia, energy, and the environment. - 62 - Box 6.2a SOCH FINDINGS RM IMPACT EVALUATION REPORTS The three Jenka Tria,gle projects were the some regions affected by the projects, high first In a series of six Bank loans to the population density threatens to create severe Government of Malaysia for the development of problems. Continuous expansion of the ores new lands to be planted to oil pale and under cultivation has resulted In the rubber and settled by landless people. The clearing of more land, a smaller area under projects were part of a large government fallow, and shorter follow periode than in development and settlement program which the pest. Yields have fallen significantly, started in 1958. The impact evaluation and some farmers have migrated to other report fot these projects and the findings of regions. Soil erosion, deterioration, and a surver of settlers demonstrate that the acidification problems have been accentuated projects have sustained a not flow of by mechanical clearing of marginal areas, benefits seventeen years after the first resulting in rapid sell erosion; intensive investment. Overall, the report confirms the and exploitative use of land by Immigrant rather high social benefits from the populations; poor weed control due to a settlement schemes, although It notes that shortage of labor; and the burning of these benefits may be unstable if major residues instead of returning them to the fluctuations In prices occur. The report soil. Deforestation has also reduced the also notes the importance that the executing availability of fuelwood and has probably agency gave to the environmental aspects of adversely affected the local climate (Report the projects. Land-clearing practices No. 6878, June 8, 1907). resulted in minimal soil erosion; the original land-use plan was flexibly An Impact evaluation report on four implemented when the slopes were found to be irrigation projects in Mexico (Rio Sinaloa too steep to be planted to oil palm; and the and Panuco) and Morocco (Doukkle I and II) mills were sited for enough from water that In still in preparation notes that a few sources to avoid pollution. However, not all years after completion, she four projects the environmental effects of the project were show a poorer economic performance than benign. As with any agricultural development projected at the time of audit. Weaker project of this nature, the clearing of commodity prices, lower-than-expected forest land clearly reduced the population of efficiency in the use of water, insufficient local spies of forest animals (Report No. attention to operation and maintenance, and 5988, December 23, 1985). damage to the environment and natural resources have been the main factors The Impact evaluation report for cotton affecting performance. In addition, In the development projects in Burkina Faso, CSt Sinaloa project, where heavy selinization and D'Ivoire, and Togo notes that at project waterlogging of soil has been caused by poor completion, concern was expressed that the drainage, an estimated 85 families have rapid agricultural development In the three incurred losses due to the progressive project areas could have adverse effects on abandonment of land or low productivity. the environment. The report adds that in - 63 - Box 6.3: THE BANK'S INVOLVEMENT IN RZ орвга0lеаs Ум / ерагадlаа 01вв owггuns 1я74 п1 1.7 ве п в.е 1е ss б1 1я76 46 1.7 48 б 0.4 48 А 7я 1я7б 64 1.0 � 1 б.б У бя б7 1я77 вб п.е ав 4 s.e 1п 76 е7 1я7в Бе 4.в вs б в.я 18 7ь 78 lятя vб п.в 7п я в.я � аьь бь lяsб 7s п.4 7а � s.в а1 м яа 1я91 б1 �.б 7® 4 0.6 б б1 я4 1У82 б8 4.• б1 я 1.7 10 1б4 б8 1988 146 Z.T 74 б 1.б ц 168 я4 1084 18я 4.7 76 4 /.У 19 16� я1 1я66 73 �.6 б8 а 1.� 41 М я1 1�б 188 s.1 7б 4 1.б 14 414 б9 1я87 166 4.7 б7 б 1.я 1в 1М б0 1я8б 140 $.� 67 i i.0 44 1б8 я1 1я74-б8 1,484 �.8 бя бя 1.А 1я i,б4! 88 Notos ТаЫо охеlида орагаiiом поt 1врlоsмLед апд opaгa4ioas !ог аЬ..�ь да`s игв wиvвiiaЫв. 32 - Annex 1 Table 1-12: DISTREWICH OF TIME-UNDERRUN OPERATIONS EVALUATED DURING 1974-se, BY REGION AND SECTOR (Number of operations) S r/Operstion Africa Asia EMENA LAC Total Agriculture and rural development is is 6 7 41 Area development a 2 9 6 5 Livestock 1 4 0 1 6 Irrigation 2 2 0 1 6 Agroindustries 0 0 0 0 0 Other /1 12 9 0 6 26 Industry and energy 2 2 5 4 is Industry 1 0 2 1 4 OFCs 0 0 1 1 2 Energy 9 0 1 a 1 Power 1 2 1 2 6 Tolocow-ounications 0 0 0 0 0 Infrastructure and urban development 4 3 2 0 9 Transport a 2 2 0 7 Highway* a I 1 0 6 Rai lways a a 0 a Ports 1 1 0 2 Tourism and other Urban Water supply and waste disposal I 1 0 0 2 Human resourc%w ana technical assistance 0 1 0 1 2 Education a 1 6 1 2 Population, health and nutrition a 0 0 0 6 Technical assistance 0 0 0 0 0 Program and policy lending 9 a 0 0 a Program loans and Import support 0 a a 0 4 Emergency reconstruction 0 0 a 0 0 SAL8 a 0 0 a 0 Total 24 25 7 12 es Note: Excludes operations In Finland, Iceland, Ireland, Israel, and Now Zealand. /j Includes perennial crops, credit, fisheries, land settlement, research and extension, program loan/credit and agricultural service* projects. - 83 - Table 1-18: DISTRMBUTION OF DA OPERATIONS EVALUATED DURING 197443, BY REGION AND SECTOR (Number of operations) Sector/Operation Africe Asia EEMA LAC Total Agriculture and rural developmet 143 128 41 21 886 Area developmenst as 7 8 34 Livestock 28 9 8 1 46 Irrigation 18 41 18 8 7 Agroindustries 8 14 8 - 2s Other /1 48 48 15 5 111 Industry and energy 85 48 25 18 118 Industry 4 11 4 - 19 DFCs 16 19 7 1 84 Energy 1 1 4 - 0 Power 9 11 6 9 8 Telecommunications 5 1 4 - 19 Infrastructure and urban development 188 48 40 16 229 Transport 114 20 28 12 175 Highways 88 11 14 9 122 Railweys 11 6 8 1 21 Ports 12 7 4 2 25 Touries and other 8 2 2 - 7 Urban 8 8 8 8 16 Water supply and waste disposal 18 11 14 1 89 Human resources and technical assistance So 28 is 19 124 Education 58 16 15 18 98 Population, health and nutrition 1 4 8 - 8 Technical assistance 14 9 - - 28 Program and policy lending 8 16 5 2 81 Program loans and import support 2 16 8 1 22 Emergency reconstruction 1 - I - 2 SALS 5 - 1 1 7 Total 892 247 129 59 827 Note: Table Includes blend operations. - No operations. /l Includes perenniael crope, credit, fisheries, land settlement, research and extension, program loan/credit and agricultural services projects. -84 - AnnI Table -141 DIMsION O FlsATIsACT PeBoNAc EwUATIS Fa MoA OPEATIONS LUVMMAD DUI 1974-8, SY RESION AM SECTOR (Number of operation) Sector/Operation Africa Ast EOINA LAC Total Agriculture and rural development 76 SS 0 5 in Area development 32 2 4 2 40 Livestock 16 4 1 1 22 Irrigation 8 8 2 2 1 Agroln&astries 2 8 a - 6 Other /a 17 8 1 0 So Industry and anergy a 4 a 6 is Industry 2 2 2 - 6 DFCs 4 2 1 0 7 Energy 0 0 a - 6 Power 6 a I 0 a T*Icomeunlatees - Infrastructure and urban development 28 7 5 8 8 Transport 21 4 8 2 aM Highways 14 1 a 0 15 RaltwaIv y I 2 0 7 Ports 1 2 0 2 5 Tourlen and other 1 1 1 - s Urban 6 1 1 e 2 Water supply and waste diepoeal 2 2 1 1 6 Human resources and technical asitance 18 8 6 1 28 Education 11 2 8 1 17 Population, health and nutrition I 1 8 - 4 Technicale aistance 7 0 - - 7 Program and policy lending 2 0 I 1 8 Program loans and Import support 4 0 0 9 0 Emergency reconstruction I - 0 - 1 SALe 1 - 0 1 2 Total 124 84 22 1B 19o Notes Table Includes bleand operations. - No operations. /g Includes perennial crops, credits, fishertee, land settlement, research and extenslon, p'ogram loan/credit, and agricultural services projects. - 8S - Аnмх - ТвЫ• 1-16t COIPARISON OF ЕУАWАТФ IDA OPERATItW4 6У BECTdt МО REOIdi wI111 TOTAL 2DA E1lAWATED OPERATIONB А10 TOTAL EYAL1lAiEd OPERATIOIS. 1•74-р (рвгемдв•вв. Ьвsвд en ливlвг et оовгвilом) Sвеiлг Atг1w Ав1• Е1ЕМА UtC То�в1 11•гiсиltигв snd гигвl двwlорвмi х et t.eiвl IOA орвмАiом in аеtог 46 80 12 б • ot *еtвl IDA орвга0iопs 1л гв8iеn 8t 4• 8Z 8б х ot Аоt,в1 ерегаАiоав in sво8ог влд грiм 76 Ai 41 28 • о1 toiвl орвгвtiом 1л nебог 46 п! 7 4 бд I11dY1trгr елд олвГ� f ot t,otвl IDA epontioм in мсtог 81 8t 4Z �i х ot totвl IOA еовгвLlом in гepion • 17 1• 17 • о! tоЕв1 орегвtiоnв in иеtог влд гpion 4Z 84 1б 6 • et total opвntioм in воеtог 7 • 6 4 2Z 2nlгsstгue`Уго влд игЬаn дwвlорвмб >i о! totsl ТОА орвгабlем in иоtог б• 17 17 7 • ot total IDA орвгвilом in гpion 84 1в 81 47 б ot teZв1 орегвbiолг in воеЕог алд ге•ien 0• Я L1 14 f ot totвl орвгв0lоn�� in sвеl,ог •i 7 7 8 41 Ниввn гиеигем snd tвehniмl вssiвf.вnoв х ot �оЧ1 IDA еовгвtiом in ввеtог � � 88 16 8 х о! totвl IOA оОвгабiоnв in гpion 17 12 14 17 8 ef tot,вl оовгsАiоав in вееtег влд грiол 7б 62 46 41 б ot tota ( opoмti ом i п вве< ot оwм 11 грi елв 1 ZеЧ 1 07 Ч 2i 14 IOA грiомl t,otвl вв � ot ot totвl IOA орем@lодв 47 80 1б 7 Иоtвк Tota1 IDA орвгвbiопв вnд toLв1 ooвгstlons both гвtвг Оо вvвlwtiвд орвгавiом on1y. • no4 Со totвl nw�Ьвг о! орвгsbiом tог 81и рвгiод еоvвгад. 86 Annex I Table 1-16: DISTRIBUTION OF TOTAL EVALUATED OPERkTIONS BY REGION AND COUNTRY 197444 me. Of No. of NO. Of Rosion/Country operation* Roo"GO/Country operations Region/Country operatiore AFRICA ASIA LAC Benin 11 Bangladesh 45 Argentina Botswana 16 sum 14 Bob a Burkins Faso to China 7 Bsib d- Burundi 13 FIJI 7 Belize Cameroon 29 India ISO Bolivia 21 Central African Republic a Indonesia To Brazil 70 Chad is Kom, Republic of St Caribbean Region I Comer** I Lao, People*s Chile 12 Congo ? Dow. Republic I Colombia 52 Coto 411voire 32 Malaysia 41 Costs RICO 19 East African Cc smu; nity 7 Maldives I Dominica I Ethiopia 25 Nepal 14 Dominican Republic 9 Gabon 4 Papua Now Guinea 17 Ecusdor 20 cambia 7 Philippines 47 El Salvador to Ghana to Singapore 11 Guatemala 9 Guinea it Sri Lanka 22 Guyana 13 Oulftes-Ble"M I Taiwan, China 2 Haiti it Kenya So That land 49 lloodurom 21 Lesotho 0 Western Same 1 Jamaica 21 Liberia 19 TOTAL no Mexico so Madagascar is Nicaragua 14 Malawi 24 P 14 mail 17 Afghanistan 3 porequay 17 Mauritania is Algeria 19 Peru is Mauritius 12 Cyprus 14 Trinidad A Tobago 12 Niger 12 Egypt a Uruguay 9 Nigarls 24 Grown is Venezuela 7 Rwanda 9 Iran 17 70TAL 428 senogs 1 33 Iraq 4 Sierra Loons 9 Jordon is Somalia - is Lebanon 2 Sudan 23 Morocco as Swaziland 6 Own 5 Tanzania so Pakistan 63 Toto is Portugal 9 West African Federation 2 Remals 25 Uganda 9 Spain is Zaire is Syria is Zambia 19 Tunisia 38 Zimbabwe I Turkey 46 TOTAL SU Yomen Arab Republic 19 yeasn' People** Dem. Repbl It is yugeslavis 92 TOTAL 440 Note: Excludes Finland, Iceland, Ireland. Israel, and New Zealand. - 87 - Ann Tble1-17: WIMT AVERAGE RE-ESTMAI ECONOMIC RATES OF RETURN FOR OPRATIONS EVA.UATED DIRING 198-89, BY SECTOR AM REGION Sector/Operation 18 1987 198 Number of ERR Number of ERR Number of ERR operations () operations (s) operations (5) Agriculture an6 rural development 37 18 69 12 46 15 Area development 9 12 11 4 15 8 Lv.e ock 8 8 7 16 1 8 Irrigatlon 18 14 18 16 17 11 Agroindustrie 1 -6 8 12 4 18 Ohar la 6 20 11 18 14 22 Induetry and energy 85 13 19 U 21 14 Indeustry 11 11 4 7 8 13 Energy 8 30 3 16 8 21 PowP 16 11 12 13 6 11 Teecommnleat1one 6 18 n na 4 26 Infrastructure and urban development 86 19 27 26 25 22 Transport 87 28 10 25 14 20 Urban 18 28 8 32 6 16 Water supply and wante dIsposal 18 a 8 6 6 12 Tourism and other 8 11 - - - - TOTAL 138 16 101 16 92 17 Rgion Africa 29 19 22 16 16 18 Asia 41 15 34 20 22 20 EMENA 87 17 27 14 28 16 LAC 81 14 18 18 20 19 Totaa 18 16 101 16 92 17 Note: ERR are not appi Icable to development finance corporation*, human resources, and technical acelatancm operations, or to program and policy lending. Thesw operations are not covered by this table. - No operations. n* Not avallable. L Includes perennial crope, credit, fisheries, land settlement, research and extension, forestry, program loan/oredit, and agricultural services projecta. - 88 - Annex 1 Table 1-18: WEICTD AVERAGE RE-EST1MID ECONOMIC RATES OF R1~ FR OPERATION3 EVALUATED o==Nm 196-88, BY =ECTM AmO REGI (wolshtod by pro~jct cost) 1986 1987 1981 Nmber of ERR Number of ERR Numbar of ERR Sector/Operation operaklone (s) operations (m) operations () Agriculture and rural deviopoent 87 16 54 18 48 14 Area devolopmont 9 11 11 16 10 16 Livestock 8 8 7 14 1 86 Irrigation 18 18 18 14 17 16 Agroindust~le 1 -5 7 18 4 16 Other 0 8 11 11 14 28 Inductry and nargy 84 14 19 11 21 14 Inductry 11 10 4 12 8 18 Energy 8 86 8 22 8 18 Pouer 16 12 12 9 8 16 Telecoumniatlon 8 18 ng n* 4 19 Infratructure and urban dfvflop.e 6 66 17 27 21 26 19 Transport 37 19 18 24 14 27 Urban 18 14 8 18 6 12 Water upply adwaste disposa 18 8 8 6 6 1 Tourim and other 8 11 - - - - TOTAL 187 16 1la 18 92 16 Rgion Africa 29 17 22 16 16 11 Agio 46 17 84 18 22 16 ElENA 87 16 26 9 28 15 LAC 81 15 18 19 26 14 Total 187 10 100 18 92 15 Not: ERR are not appicbleb, to develop.ent f1nance corporations, human resources, and taknical assistance operations, or to program and policy lending. These operations are not covered by this table. Table also excludes projects for which cost or re- ostimated ratms of return data were not availablo. - No operationo. na. Not avalabei. ja Include perannial erops, credit, fisheries, land setlment resrch and extonclon, forestry, program loan/oredit, and agricultural services projecte. Annex -1 Table 1-19: APPRAISAL AID RE-ESTIMATED ECONOMIC RATES OF REtON FOR EVALZATED OPERATIONS, 1974-88 Average ERR* La Evaluation Number of at at ERR year operations Appraisal Completion Sippage /b M 0 00 1974 81 16.1 16.6 -0.7 1976 26 18.6 16.7 1.8 1976 89 15.4 14.7 0.7 1977 69 17.9 16.2 1.7 1976 55 19.6 18.9 4.7 1979 04 19.6 17.6 1.8 198s 62 21.8 16.6 5.8 1981 69 16.9 14.0 4,9 1982 76 19.6 14.1 5.4 18 97 21.0 12.8 6.2 1964 167 21.1 11.6 9.6 19865 1f 24.6 15.9 6.6 1988 184 21.4 15.6 5.8 1987 98 29.6 15.8 18.7 1988 66 25.7 16.5 9.2 Note: Excludes agricultural credit and DFC projects, projects for which an ERR to inappropriate, and projects for which ERRs were not available at both appraisal and completion. is Unweighted average of ERRe for individual operations. Lb Appraisal value minus completion value. ..gp_ Аnмх 1 Твb� 1•пдк OI81RlвUTI01 Of WSATI�ACTdtY QERATl91iS вУ RE-ESTIWTФ ЕСОа�l; RATEB 0F RET181H i�i. f ВУ SECTOR 1U0 81SSECTOR, 1ятМр рваsбlvв E4R е-{>< ERR 6-ве ERR 7>< or Aloksг ERR Totвt Ио. ot Х of йо. ot !{ ot t1o. ot х ot Ие. ot f о� No. ot Sвебег/виЬовебог орвгвбiом бобвl орогабlеnв бобвl орвгабlом бобвl ерогабlом 6oto1 opвгations Оягiеиliигв агиl гигаl доrвlорввеб Iгrips6ioл 6 п1 1е 4п 4 17 6 п1 п4 и �.вбоеь 14 вв 18 41 п е 8 я 8п Сгер/агр дwвlервмб п4 4я е 1в в 1е я 1в вя Рвгимlвl orops 7 47 8 пе 8 пе п 18 16 ьТоговбгr 1 п6 1 пб 1 пб 1 п6 4 F1амг1.. 4 ве 1 пе е е е в а Cг.eiL " i ее 1 п0 е е 1 пе Б Аугоinдмбгlео/ргоевsоiде е ее 1 1е е в а ее и s.�tsl..мs 1 ss е е е е 1 ве п оемго 1 ве в е е в 1 ье п тлtвl вв 16 8я пб 1в 1п пв 1я 149 trsnspoг6 еад tоигlр Иiяhварв 4 1я 8 14 п 1е 1п 67 21 Rai 1вв�rо 1 Т 1 7 1 8 18 87 16 РегСs 1 18 1 18 В в в тб 8 А1 гРогб. е е е е 1 пб а тs 4 Тоигiм 1 1т 1 и п в8 s 88 в Тоо:1 т 1в в 11 ь я 8в вт ы Еквг�r впд риЫ ie иб11 i6ia Роввг 6 8я п 16 8 п8 8 п8 18 Твlвеоввиniмбiепв 1 Ьв е в е е 1 Бе п isbг вирРlу. рвбо disp. 1 пб п 68 е 8 1 п6 4 Тобв1 т 87 4 п1 8 1в 6 пв 1я - 91 - Téhip ý-21: DIMM0TION OF StWAINADIUTT AUESSIIENTS BY REGION ~ SECTOR. OPERATIONS EVAIXATED IN 1~** (namånr of operatlom, and porcentages) Region Likolv UnMol marginal uncortaln Tota l no. M. 4 MO. I Anriculturo and roral dovolope~ Af rIce 0 20 14 34 6 12 14 34 41 MU 48 u 16 14 a 4 la 18 74 EMM 17 65 5 18 a la 8 19 at I.AC 18 sa 7 22 8 19 1 a 82 Tota 1 91 st 88 29 17 16 84 19 Ile Induntry Af rlca as 2 87 0 a Agio 6 83 la 0 a 2 26 a EMENA 6 42 6 4 så 2 17 12 LAC a 75 1 25 0 6 a 9 4 T144 l 14 62 5 19 4 16 4 16 27 DovoloEent f Inanco corporation* Af rico 6 42 2 17 ?. 8 4 a 12 Agio la 71 0 a 6 0 a 29 21 EMENA 4^ 89 9 6 6 a a 29 16 LAC 9 ca 2 26 1 la 8 0 9 Tota 1 37 a 4 7 2 4 la 28 58 Powor and on*roy Africa 0 sa b 0 1 se 2 Agio 18 90 0 6 0 6 2 10 20 EMENA a m a 9 0 a 8 60 18 LAC 6 se 9 0 1 8 6 42 12 Tota l 32 64 1 2 1 2 18 32 60 TolotonewnIcatIce Africa i u 6 9 6 6 2 A*;* 2 10 6 0 6 6 a a 2 EMENA la 0 6 0 b 9 0 a LAC 10 0 b 9 9 b 6 a Tötal 9 m 1 10 6 4 a a la Water aupply and w*~ diepoe&I Af rlco 8 80 2 20 1 10 1 16 16 AGIO 4 40 0 9 2 20 4 40 la EMENA 9 68 1 8 1 8 6 81 16 LAC 7 47 1 7 2 13 5 83 16 Tom l 28 51 4 a 8 12 la 29 el _ 92 _ Аппвх 1 = ТаЫ• 1-21к DISTRIBUTIdi OF SUSTAINABцITY ASSESSI�NTS ВУ REtiION At�O SECTOR. OPERAт20NS EYAWATED IN 1�в-!в (CQл1TIMlED) (пивЬвг ot opoмbions. вnд рвгемtвявв) Rpioп Likв1� Uпlikвly_ Wг� tlп,�игisin,�, T�b1 Tгansooгt . Аtгiи 11 а7 я вв 4 1а е 2е вв As1a в в2 д 0 0 д 6 8� 18 O1ENA 1в 78 1 6 в А 6 28 22 ис в 86 2 12 8 2а 4 24 17 Тоtв1 41 6А 12 18 9 11 2в 24 82 Uгbsn Af гi es е в в 7s 8 е 1 2ь 4 мis 2 28 4 48 1 18 8 80 10 аrЕиА 1 вв 1 вв е е 1 ав в ис 2 17 в se s е 4 аа 12 Тоtв1 6 17 14 48 1 8 � 81 2� EdnиLion А�гiи 4 1д 8 14 4 1д 18 48 21 мiа 11 вб 8 8 2 12 4 24 17 а1ЕИА 4 в7 в е в в 2 8в в ис в sr е в е в в ьв в Tott 1 22 44 8 в в 12 1в 88 bA Poauls�lon snd пutгitioп Аtгiи е е в е ё в е е в Asis 2 180 в 0 д в в 0 Z amrA в в 1 1ед в е е в 1 ис о е в в е е 1 1во 1 тot.t 2 ье 1 zs в е 1 2s 4 Nosproiвei lвrtdina апд tвchnlиl вssistsnм А�гiи 7 в1 1 д 8 27 в д 11 мsа 2 в� в в 1 вв в в а р1ЕЧА 2 в7 1 вв е в д в в ис 1 вз е в в в 2 в7 в Тоtв) 12 88 4 1в 4 2А Z 18 2д 1-,abla 1-22: PRIKIPAL KWUMWS OF =TA~IUTV: MATIVE REWLWY DISTRIMMM IN AWIT STATEMMø OPEUTIM EVAWATED IN l~N Teeltalcal factore IfistItutløml factorg Polity conelderatlong Cc~ le end f Intnelel factors VH (Døglen, proper2t1on, ¥14 Gov* ~ý support, m Pol ley *apport fl Avellebility of funde IngtItatlønal strangth., a man*gemont, VM Design VM ne m £~ le ~ Ulene Devel*~ flage* Å Døgløn VM InstItutlønøl streng" ne VH FInancIng corporatlom Power end enerøy L v" Inskitutlønal monagemont, L Economle conditions Tøloc~ catlone M IN Gover~ support, m Avellebility of funde Inetitutlonal management W~ supply and Å Design VM InstItutløna$ manggement, Å m £«nesle conditiong weste disposel Transport VH Deslon, malfttenance VM InstItatlønal strøn~ Ing nø m £~ le condltløns Urban VM Dø*løn, mint~nco L InstItuklonal strsingthening ne N Finanting hh tion VII Dowlon, mal~ affe VM Gov*rnmøftt support, N Polley changøs VII Avgi lehl Ilty of funde InstituMonøl etr~, Å mone~ Populatlon and natrltion N Døolgn N s~ficiarlog, pørtlelpetloa nø ne A»lløblllty of t'øndø govor~t support Non~ 1«41ng and na N M*n*g~ t M Economle c~1d*rotio« tochnIcal asslskonco VM a very M9h; N = hlgh; Å avorege; L a low; VL a very #*w; ne a not applicablo. Anans..I .able& 1-2i PRINCIPAL DETERMINANTS OF SUSTAINADILITY IN A0RICULTItE: RELATIVE FREQUENCY DISTRIGUTION IN AWIT STATENTS, CPERATIONS EVALUATED IN 19O-88 Agriculture subsector Technical factors Institutional factors Policy considerations Econmic and fInancalt factors ArealRural Development V Design, malntenance V Benerfciares' participation, L Policy support Vi Availability of funds institutional strengthening Livestock L Design VM Government support, L A Economic conditions, Institutional strengthening funding Irrigation V Design, maintenance VW Seneficiaris' participation, L H Availability of funds institutional managemeat Extenslon/research na VN Government support, na n Institutionalstrngthening Tree crop/estate L A no H Availability of funds Forestry L M Institutional strengthening no as Agrolodustry H Design, maintenance N Technology transfer L L Institutional managemt Credit L VWt Znstitutional strengthening L A Program credit VW Design, preparation L H Policy support na V a very high; H high; A average; L a low; VL a very low; na a not applicable. - 95 - Annex 1 Table 1-24: SUSTAINABILITY ASSESSMENTS FOR AFRICA, BY SECTOR, OPERATIONS EVAIATED IN 1988-88 (Percentages) Sector Likely Unlikely Marginal Uncertain Agriculture 20 84 12 84 Industry 88 07 0 8 Development finance corporations 42 17 8 88 Power and energy 8 Be 0 so Telecomunications Be 5 a 0 Water supply and waste disposal 89 26 1 1s Transport 87 86 18 26 Urban 0 75 0 25 Education 19 14 19 48 Population and nutrition 0 0 6 0 Nonproject lending and technical assistance 64 9 27 6 - 96 - Tøtj 1.25 SU7AINAITy ASSES4M1lTS, BY REGION AND SECTOR OEATIM EVAUATED N 1906- (Number of operatlonu) Likely Unilkely Marginol Un aerta Total ~ICA Xgric-ituro and rural dovelopen t 14 a 14 41 Emd-a6ey 1 2 I I 8 DFCs 6 2 1 4 12 Power and eergy 6 1 I 1 2 Tøleo nalone 1 Water supply and vest* disposal 8 2 1 1 le Transport 11 9 4 6 ug Urban a 3 6 1 4 Education 4 3 4 lo 21 Population and nutrition 6 6 9 g Nonproject lending and tochalcal ssstance 7 1 8 6 11 TOTAL 48 se le 87 136 (distrlbl on, 1) (32) (28) (18) (27) (111) ASIA griulturo and rural developmsnt 4S 1S 8 18 74 Industry 6 1 I 2 9 DC* 15 9 0 6 21 power and ery le I g 2 21 Tolocammunlcatløns 2 0 a I 2 ter supply and wast. disposal 4 8 2 4 1 Transport 9 I I 6 18 Urban 2 4 1 8 le Eduation 11 8 2 4 17 Population and nutrl1on 2 6 6 0 2 Nonproject lending ad technlcal assistanse 2 8 1 a 8 TOTAL 117 15 9 $9 1N (dløtrlbuon, 1) (6) (8) (6) (22) (10) EMEN =*rTculture and rural development 17 6 8 6 81 Industry 5 1 4 2 12 DFC9 12 I 6 8 16 Power and energy 8 I I g 16 T8lecommunlcatlono 8 Water supply and maste disposal 9 1 1 6 16 Transport 16 1 I 5 22 Urban 1 1 1 8 Educaolon 4 6 I 2 6 PopuatIon and nutritlon a 1 I g 1 Nonproject lending and technical assistnce 2 1 I 9 8 TOTAL 77 11 8 32 128 (di*trlbutlon, 5) (8) (9) (6) (25) (10) LAC grlculturo and rurol develop~ent 18 7 6 1 82 Industry 8 1 6 6 4 DFC9 5 2 1 I 8 Power and enerøy 6 6 1 5 12 Tolecommunlcations 8 I I I 8 Water supply and waste disposal 7 1 2 5 15 Transport 6 2 6 4 17 Urban 2 6 I 4 12 Education 8 6 I 8 8 Population and nutriløn 9 a g 1 1 Nonproject lending and teonical assistanoe 1 I 6 2 3 TOTAL 54 19 15 26 118 (dietrlbutlon, 9) (48) (17) (18) (22) (118) CROSS-REGIAL TOTAL291 88 1 7 (dl~ributlon, 5) (62) (15) (9) (24) (1W9) - 97 - Annex 1 Table 1-26: PERFORMANCE ASSESSMENTS, BY YEAR OF EVALUATION 1974-88 Percentage of projects Evaluation rated as having year satisfactory performance 1974 86.0 1975 87.7 1976 85.7 1977 89.9 1978 90.7 1979 87.7 1980 88.5 1981 85.2 1982 80.3 1983 84.8 1984 74.1 1985 70.2 1986 81.8 1987 72.4 1988 73.8 Notess OED's traditional method of assessing satisfactory performance is based on achievement of at least a 10 percent ER, or other significant benefits if the ERR was lower, or an evaluator's qualitative judgment about performance if no ERR was calculated. Excludes 8 operations not implemented and 4 operations for which data were unavailable. 98 - Figur* 1.1: Gap Btween Appruis and R-timted Economic Rats of Return, by Region, 1968-0 Afie Aia 20 11 10 18 14 6 121 10 6 U 4 2 -2 68 70 72 74 78 78 80 68 70 72 74 76 78 80 EMENA LAC 15 11 14 10 12 10 8 8 6 i a 44 2 -4 en 70 72 74 76 78 80 88 70 72 74 78 78 80 V /-/a-. - 99 - Figur* 1-2: Gap Btwun Appraiud wd Ro-msted Economic Rates of Rotum, by Setor, 19688-0 AgIIulture and flural Devlopament idustry 15 14 25 12 1010 6 6 70 72 74 76 78 6 70 72 74 76 78 80 Enegy and Publio Uilis Iknu Op~rtions i5 14 7 12 B Js 10 4 6 9I 3 4 2 2 10 8 6 14 12 10 -2 - 100 - Figur 14: Average Apprasul and Re-et*mated Ratus of Retum, by Region, 1968-80 Africa Asia 34 34 32 30 30 28 i 28 Pl>i 4l.Ч Jи1ф Jo1-p Ова-� TilТ 9мsi 1 5оиlАвгn Stвtos wstoг 8вррlу J1 Zсвввг. 1t� 1iN.! WмМ Арг-М рвс-Чi 7At А ColoвQi• пда ееЧоа rraaг sn�1y ss1 а.в wy-n w1г-п о.о-ав ТпА1 г Соlавb�а sгб 9оао4 Маtог 8иро. Sоввг.t Dм1а. 1097 � 1i.N W{м7Ч New7o Jид-К Т4А1 ► trpгnв �tввваоl Sоrогарв 1~ рг.lда9о пви 1б.s wy-N 11ву-�И ApNT Тцs Ер�б Маtвг 5и�1у J1 Soиnpo EaOtnвoгtno 4>� в.f On4rp РвЬ-iА Jвмр 7ilА Чlигаоиа Wму�и МаLвг 8и{q►tr Еарiпигlnя 19р i.7 Арг-i1 JимЧ1 Jим16 7647 Niигapw Увмаw Уаtог SuPP1y IIi 1А11б 1Ч.1 NowT7 Jва-7Ч Oвe-ti 762Т Иiимяиа Rим1 8an1tвLloп 1в9` А.в Nov-77 Fвb-Tf Ооа-Ч Тб47 P611fPprina 4nd Wn11a wabг Su�1y 1б16 p.i Ju1-Tt Ju1-ТЧ Ова-Чб 716f Sгi �aaka Zad Natoг SupPfr i Sогвг+вао 1М1 >Ч1./ Jиn-1Ч 5р-М Чвг-t7 71bб Уовм AR 8aas'в IIп0 Nabr Supply а 5оввм�в � А7Ч 1/.Ч рое-Тб Jsд-1'г Ово-f4 7iq А Уввм P�t А1 Ynkslla Чаtвг SupP1y Rв1иЬ. ц64 i.�f Jwr-11 Jn1-S1 Ова-Чб 76� Vвввn POR агваtаr 'Wм watwг 8иррlу 1fq 1s.4 1by-r Ju1-М Dва-�б T6f4 А ' Imlиввгr впд омгтг ОоwlооsвлЕ Finaпeв Cowaanioв 0вгЬвдов 2пдwОгlаl Ово. i Еарогд Рго*о01оа 1N3i 1/.Ч Wг-f/ двг-бi Sвp,ii 71М Е�дt 4tsb Оwвlервм0 Indwtгlal 8вдk 1М4 6/.Ч Wг-!Ч Wy-iN Jw„/6 7>i2 E�rpL 8гд Оwвlорвм0 ImWaSгial бапk 16U �/.Ч Wг-7� Арг-Tf Wг-р 77iY Квара 4лд IпdusLгial Овvslорвм4 6ank 11Ч 10.� .1и1-Тб Ju1-T6 JимМ 741А , Код{га 8.д Industгlal Ооvвlорвм0 Baak 11>ls �/.Ч Wy-Т7 W1t-77 Овт-{Ч 74�0 МвдвЧввеаг 6aakin'ау Indoвbгiв (ВИ1) 07'l 6.Ч Jaд-f� Уа� Ове-1[ 717У Yвrieo 1s4 8вв11 • Yodiw 8м1о Iedw4гy 1664 47.Ч Арг-7� Ув�►-Тf Sвр-� 76б� Yucias �дд 8а11 • Увдiив 8са1• Indwбгy 1р1 1АЧ./ JимЧ/ l;ор-В1 Jид-iб >liбft Е2:,� E�►ot Abn Qiг Ws Овwlорввд4 41N 1/.Ч Wг-!1Ч W�r-!