77092 Improving Institutional Capability and Financial Viability to Sustain Transport An Evaluation of World Bank Group Support Since 2002 March 2013 Improving Institutional Capability and Financial Viability to Sustain Transport An Evaluation of World Bank Group Support Since 2002 ©2013 The Independent Evaluation Group The World Bank Group 1818 H Street NW Washington DC 20433 Telephone: 202-458-4497 Internet: http://ieg.worldbankgroup.org E-mail: ieg@worldbank.org Some rights reserved 1 2 3 4 5 6 15 14 13 This work is a product of the staff of the Independent Evaluation Group (IEG) with the exception of the “Management Response” and the “Chairperson’s Summary.” Note that IEG and the World Bank Group do not necessarily own each component of the content included in the work. IEG and the World Bank Group therefore do not warrant that the use of the content contained in the work will not infringe on the rights of third parties. The risk of claims resulting from such infringement rests solely with you. The findings, interpretations, and conclusions expressed in this work do not necessarily reflect the views of IEG, the World Bank Group, its Board of Executive Directors, or the governments they represent. IEG and the World Bank group do not guarantee the accuracy of the data included in this work. The boundaries, colors, denominations, and other information shown on any map in this work do not imply any judgment on the part of IEG and the World Bank group concerning the legal status of any territory or the endorsement or acceptance of such boundaries. Nothing herein shall constitute or be considered to be a limitation upon or waiver of the privileges and immunities of IEG or the World Bank Group, all of which are specifically reserved. Rights and Permissions This work is available under the Creative Commons Attribution 3.0 Unported license (CC BY 3.0) http:// creativecommons.org/licenses/by/3.0. Under the Creative Commons Attribution license, you are free to copy, distribute, transmit, and adapt this work, including for commercial purposes, under the following conditions: Attribution—Please cite the work as follows: IEG. 2013. Improving Institutional Capability and Financial Viabil- ity to Sustain Transport: An Evaluation of World Bank Group Support Since 2002. Washington, DC: World Bank. License: Creative Commons Attribution CC BY 3.0 Translations—If you create a translation of this work, please add the following disclaimer along with the at- tribution: This translation was not created by the Independent Evaluation Group or the World Bank Group and should not be considered an official World Bank Group translation. IEG and the World Bank Group shall not be liable for any content or error in this translation. All queries on rights and licenses should be addressed to the IEG, 1818 H Street NW, Washington, DC 20433, USA; fax: 202-522-3125; e-mail: ieg@worldbank.org. ISBN-13: 978-1-60244-227-6 ISBN-10: 1-60244-227-4 Cover image: Bangkok. © iStockphoto.com/oneclearvision Library of Congress Cataloging-in-Publication Data has been requested. Contents ABBREVIATIONS .................................................................................................................................. V ACKNOWLEDGMENTS ........................................................................................................................ VI OVERVIEW ........................................................................................................................................... VII MANAGEMENT RESPONSE .......................................................................................................... XXVII MANAGEMENT ACTION RECORD ................................................................................................. XXIII CHAIRPERSON’S SUMMARY: COMMITTEE ON DEVELOPMENT EFFECTIVENESS ................ XXIX 1. INTRODUCTION................................................................................................................................. 1 Objective and Scope of the Evaluation ..................................................................................................................... 4 Evaluation Framework .............................................................................................................................................. 8 Evaluation Building Blocks ...................................................................................................................................... 12 Organization of the Report ...................................................................................................................................... 13 WORLD BANK GROUP SUPPORT FOR SUSTAINING TRANSPORT .............................................. 15 Sustained Transport as Reflected in Sector and Country Strategies ...................................................................... 15 Sustained Transport in World Bank Operations ...................................................................................................... 17 The Risk of Maintenance and funding Shortfalls at Appraisal ............................................................................................. 17 Sensitivity of Ex Ante Economic Analysis to Maintenance Shortfalls .................................................................................. 18 Sustained transport in Project Objectives............................................................................................................................ 19 Measures to Sustain Transport ........................................................................................................................................... 21 Sustained Transport in Analytic and Advisory Activities.......................................................................................... 24 Sustained Transport in IFC Investments, IFC Advisory Services, and the MIGA Portfolio ..................................... 25 Summary ................................................................................................................................................................. 26 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR SUSTAINING TRANSPORT ................ 29 Assessment of Sustained Transport Outcomes and its Variability .......................................................................... 31 Determinants of Sustained Transport Outcomes .................................................................................................... 34 Sector Policy and Regulatory Framework ........................................................................................................................... 37 Financial Arrangements ....................................................................................................................................................... 37 Institutional Framework........................................................................................................................................................ 44 Sector Management and Capacity Building ........................................................................................................................ 47 Summary ................................................................................................................................................................. 55 EFFECTIVENESS OF IFC AND MIGA’S SUPPORT TO THE PRIVATE SECTOR IN SUSTAINING TRANSPORT ........................................................................................................................................ 59 Factors that Affect Sustained Transport Outcomes ................................................................................................ 64 Effect of IFC Advisory Services on Sustained Transport Outcomes ....................................................................... 68 Coordination between the World Bank, IFC, and MIGA .......................................................................................... 71 Summary ................................................................................................................................................................. 72 CONCLUSIONS AND RECOMMENDATIONS..................................................................................... 73 i CONTENTS Summary of Findings and Recommendations for Sustaining Transport ................................................................. 73 Recommendations for Sustaining Transport ........................................................................................................... 76 Boxes Box 1.1. Sustained Transport, Poverty Alleviation, and Economic Growth ............................................. 2 Box 1.2. Findings on Environmental and Social Sustainability of Transport Operations from Recent IEG Evaluations ............................................................................................................................................. 6 Box 1.3. Measures that are Postulated to Contribute to Sustained Transport .......................................10 Box 3.1. Measuring Sustained Transport in Country Studies ................................................................32 Box 3.2. Risk to Development Outcome Definition and Criteria .............................................................34 Box 3.3. Factors Associated with Less Sustained Transport in Case Study and PPAR Projects ..........35 Box 3.4. Lessons from Tanzania, Ghana, and Yemen on the Implementation Results of Road Funds .40 Box 3.5. User Charges for Urban Transport Involve Costs and Benefits for the Poor ...........................43 Box 3.6. Staff Downsizing in Mozambique’s Ports and Railways Sector ...............................................47 Box 3.7. Building Institutional Capability through Strategic Highway Planning for Road Rehabilitation Programs ...............................................................................................................................................49 Box 3.8. World Bank Support to Improve Governance and Anticorruption in Intercity Highways...........51 Box 3.9. Sustained World Bank Engagement for Sustained Transport Services ...................................54 Box 4.1. Successes and Challenges in Sustaining Privately Operated Transport by Subsector ...........62 Box 4.2. Project-Level Measures Aimed at Enhancing the Sustainability of Private Transport ..............66 Box 4.3. Incentives and Accountability Contribute to Sustained Private Transport Investments............67 Tables Table 1.1. Indicators of Sustained Transport Outcomes Often Used in Projects Supported by World Bank Group ...........................................................................................................................................11 Table 2.1. Country Assistance/Partnership Strategies: The Share that Highlighted Measures to Sustain Transport By Country Income (Percent) ................................................................................................16 Table 2.2. Declining Share of Projects with Sustained Transport Objectives, By Country Income and Subsector ..............................................................................................................................................21 Table 2.3. Percentage of World Bank Transport Projects Supporting Measures to Help Sustain Transport, By Sector Mapping ...............................................................................................................22 Table 2.4. Trend of the Share of Transport Sector Projects with Measures to Sustain Transport By Fiscal Year of Approval and Project Objectives (percent)......................................................................23 Table 3.1. Projects that Implemented Measures to Help Sustain Transport were more likely to Achieve Intermediate Outcomes (Institutional Capability and Financial Viability) ................................................36 Table 3.2. Implementation Status of Financial Arrangements at Project Closure ..................................38 Table 3.3. Lessons of What Worked in Different Subsectors for Sustaining Transport ..........................56 Table 4.1. Share of Sustained IFC Investments, FY2002–11 (n=number of investments; c=number of countries) ...............................................................................................................................................60 Table 4.2. IFC Measures to Help Sustain Transport by Subsector and Country Income (Number of Projects for Which Measures Were Identified, Out of All 53 Operationally Matured IFC investments FY02-11)................................................................................................................................................68 ii CONTENTS Figures Figure 1.1. World Bank Group Commitments and Gross Volume of Guarantees, and Number of Transport Operations Supported, by Fiscal Year of Approval ................................................................. 3 Figure 1.2. Distribution of World Bank and IFC Transport Projects, by Country Income, FY02-11 ......... 7 Figure 1.3. The Results Chain and Framework for the Evaluation .......................................................... 8 Figure 2.1. Percentage of World Bank Operations that Identified Maintenance and Related Funding as Risks at Appraisal, by Country Income and Subsector (n=number of projects)..................................... 18 Figure 2.2. Share of Projects with Objectives to Sustain Transport, By Country Income Category and Transport Subsector (n=number of projects) ......................................................................................... 20 Figure 2.3. Percentage of Projects That Included Sustained Transport in Their Objectives.................. 21 Figure 3.1. Share of Projects with Sustained Transport Outcomes, Based on Field and Desk-Based Assessments of Project Results (n=number of projects, c=number of countries) ................................. 33 Figure 3.2. Distribution of Projects with Changes in the Risk-to-Development Outcome Rating between Project Closure and IEG’s Field Evaluation (PPAR) ............................................................................. 34 Appendixes REFERENCES...................................................................................................................................... 79 APPENDIX A: WORLD BANK GROUP INFRASTRUCTURE AND TRANSPORT TIMELINE ............ 83 APPENDIX B: OVERVIEW OF THE WORLD BANK GROUP TRANSPORT PORTFOLIO ................ 89 APPENDIX C: PORTFOLIO REVIEW AND FIELD- AND DESK-BASED COUNTRY STUDY METHODOLOGIES ............................................................................................................................ 101 APPENDIX D: ADDITIONAL TABLES ............................................................................................... 121 APPENDIX E: LIST OF WORLD BANK GROUP PROJECTS REVIEWED ...................................... 123 NOTES................................................................................................................................................ 131 Evaluation Managers  Caroline Heider Director-General, Evaluation  Emmanuel Y. Jimenez Director, IEGPS  Martha Ainsworth Acting Manager, IEGPS  Midori Makino Task Manager iii Abbreviations AAA Analytic and advisory activities AFR Africa Region APL Adaptable Program Loan CAS Country Assistance Strategy CREMA Contratos de Recuperacion y Mantenimiento DPO Development Policy Operation EAP East Asia and Pacific Region ECA Eastern Europe and Central Asia Region ERR Economic rate of return FY Fiscal Year HDM Highways Design and Maintenance Model IBRD International Bank for Reconstruction and Development IDA International Development Association IEG Independent Evaluation Group IFC International Finance Corporation LAC Latin American and the Caribbean Region LAMATA Lagos Metropolitan Area Transport Authority M&E Monitoring and evaluation MIGA Multilateral Investment Guarantee Agency MNA Middle East and North Africa Region O&M Operations and maintenance OECD Organization for Economic Co-operation and Development OPRC Output- and Performance-based Road Contracts PER Public Expenditure Review PPAR Project Performance Assessment Report PPIAF Public-Private Infrastructure Advisory Facility PPP Public-private partnership PSP Private Sector Participation RONET Road Network Evaluation Tool SAR South Asia Region SSATP Sub-Saharan Africa Transport Policy Program UNDP United Nations Development Program XPSR Extended Supervision Report v Acknowledgments This evaluation was prepared by the Public Sector Evaluation Department of the Independent Evaluation Group (IEG). The evaluation was prepared by a team led by Midori Makino under the guidance of Martha Ainsworth (Acting Manager), Monika Huppi (former Manager), Emmanuel Jimenez (Director), and overall direction of Caroline Heider (Director-General, Evaluation). The team consisted of Victoria Alexeeva, Stefan Apfalter, Isabel Chatterton, Peter Freeman, Ramachandra Jammi, Kavita Mathur, Alberto Nogales, Maria Elena Pinglo, George Tharakan, Dileep Wagle, and Cameron Wilson. Peer reviewers were Jaime Biderman, Raymond Bourdeax, John Hine, and Gregory Ingram. Kenneth Gwilliam and Anil Bhandari provided comments on an early draft. Management and colleagues of the Independent Evaluation Group provided helpful guidance and comments, including Hans Martin Boehmer, Ken Chomitz, Navin Grishankar, Caroline Heider, Ali Khadr, Chad Leechor, Mark Sundberg, Stoyan Tenev, and Cheryl Toksoz. William Hurlbut and Cheryl Toksoz edited the report. Romayne Pereira provided administrative support to the team and assisted with report production. The External Advisory Panel consisted of Professors Ali Huzayyin (Professor of Transport, Cairo University), Rosario Macario (Director of the Master in Transportation Planning and Operations at Instituto Superior Técnico, Lisbon Technical University), Ming Zhang (Associate Professor, University of Texas at Austin School of Architecture), and Mr. Dong-Woo Ha (Director, Transport Division, United Nations Economic and Social Commission for Asia and the Pacific). The team is grateful for their perspectives and feedback on an early draft and on the final report. The responsibility for the results rests with the evaluation team, however. Finally, we wish to acknowledge the generous financial support of the Government of Norway, through the Ministry of Foreign Affairs and the Norwegian Agency for Development Cooperation. vi Overview Improving Institutional Capability and Financial Viability to Sustain Transport The services provided by investments in Objective and Scope of the Evaluation road networks, railways and transit The objective of this evaluation is to systems, and ports for trade by air and assess the effectiveness of World Bank water are important contributors to Group support to countries in poverty alleviation and economic sustaining the provision of transport growth. Sustained transport infrastructure and services and to distill investments can support poverty lessons on the factors contributing to reduction directly, in terms of sustained transport. In the context of improving access and thus economic this evaluation sustained provision of opportunity targeted to the rural poor, infrastructure and services is defined as and through facilitating broad-based the extent to which the policies, growth. But the impact of infrastructure institutional and regulatory framework, investments and the services that arise sector management capacity, and from them can be undermined by poor financial arrangements are in place to operations and maintenance. ensure that transport infrastructure is operated and maintained, enabling a Over the past decade, the World Bank, reliable flow of services over the long International Finance Corporation (IFC), term. The term transport is used and Multilateral Investment Guarantee throughout this report to mean Agency (MIGA) have committed about transport infrastructure and services, $50 billion for operations or guarantees unless otherwise specified. in the transport sector, amounting to 12 percent of the World Bank Group’s total The evaluation covers World Bank, IFC, commitments and guarantee volumes. and MIGA transport operations The performance of World Bank approved over fiscal year 2002-11. It transport operations at project closure includes investments in all six transport has been high—almost 89 percent are subsectors—intercity highways, rural rated moderately satisfactory or better. roads, urban transport, railways, air Yet inadequate operations and transport, and ports and waterborne maintenance —raised as an issue more transport — in all client countries of the than 20 years ago — has remained a Bank Group. The evaluation recognizes concern through the present. the potentially complementary approaches of the three agencies in sustaining transport. The World Bank vii OVERVIEW helps governments improve the transport; IFC investments, MIGA enabling environment for sustained guarantees, and country and sector transport and finance physical strategies approved by the Board of investments. IFC and MIGA Directors from fiscal 2002-11; (ii) review complement these efforts by supporting of World Bank analytic and advisory profitable private sector transport- activities with focus on transport, and related investments. IFC advisory services completed during the same time period; (iii) nine field- Evaluation Questions and Framework based and 11 desk-based country The evaluation addresses the following assessments, which included a detailed four questions: (i) To what extent have review of 68 World Bank projects an Bank Group strategies and operations average of five to six years after closing; aimed to sustain transport? (ii) To what and (iv) a review of 30 IEG field-based extent have countries implemented the transport project evaluations, also policies and regulatory framework, known as Project Performance institutional framework, financial Assessment Reports. mechanisms, and capacity building activities needed to sustain transport Findings: To what extent have Bank Group strategies and operations aimed supported by the Bank Group? (iii) How to sustain transport? effective have these measures been? (iv) What factors have determined success? Sustained transport features prominently as a concern in Bank The analytical framework for this Group strategies and analytic and evaluation traces a results chain, starting advisory activities. with: (i) physical investments and various other measures that help sustain Sustained transport has been a key transport (outputs); (ii) financial feature of the Bank Group’s work since viability and institutional capability its 1996 strategy, Sustainable Transport: (intermediate outcomes); and (iii) Priorities for Policy Reform, and remains a sustained transport (the outcome). The focus in the current strategy, Safe, Clean, impact of sustained transport on and Affordable Transport for Development poverty reduction and economic growth (2008-2012). Country Assistance is beyond the scope of the evaluation. Strategies over the past decade have generally emphasized the need for Methodology sustaining transport, focusing primarily The evaluation employs multiple on roads. A large number and share of analytical building blocks and sources analytic and advisory activities of evidence to answer the evaluation supported by the World Bank also focus questions: (i) a portfolio review of all on sustaining transport. Support for World Bank projects managed by the output and performance-based road Transport Sector and of projects maintenance contracts and plans for managed by other Sectors with more public-private partnerships were on the than 30 percent of funds flagged for increase. viii OVERVIEW The share of World Bank operations maintain the infrastructure, and that with explicit objectives to sustain share has been declining. transport is low and has declined by half. Nearly a third of World Bank operations identified lack of World Bank projects are accountable for maintenance funding as a risk at project their objectives. Only 15 percent of the appraisal. 287 projects managed by the Transport Sector and approved during fiscal 2002- A large share (43 percent) of the projects 11 had explicit objectives to sustain that identified maintenance and funding transport. That share has declined over shortfalls as risks had sustaining the decade from roughly a quarter of transport as an objective, compared to projects to only one in 10. The only 15 percent of projects that did not downward trend persists even when flag these risks. In addition, the projects looking within each country income that identified maintenance and funding group and within the main subsectors shortfalls as risks were more likely to (urban transport, intercity highways, include financial arrangements as a and rural roads). measure to help sustain transport, in their project components. The share of A small and declining share of World projects that identified maintenance Bank operations explicitly support risks at appraisal, however, has been financial arrangements that would declining over time, from 39 percent in contribute to sustained transport the first half of the decade to 24 percent outcomes. in the latter half. To help countries sustain transport, the Ex ante economic analysis rarely takes World Bank has supported measures into account the effect of underfunding related to: (i) financial arrangements for of maintenance on the benefit flows. operations and maintenance; (ii) sector policy and regulatory frameworks; (iii) Sensitivity analysis is normally carried institutional frameworks; and (iv) sector out at appraisal for most projects as a management and capacity building. basis for investment decisions. Almost all World Bank transport However, the effect of maintenance projects support measures for sustaining underfunding, which leads to the transport in their project components. reduction in expected flow of benefits, They typically finance sector has not been systematically included in management and capacity-building these calculations. An implicit activities, for example, such as attention assumption in the calculation of the to Road Authority capacity for economic rate of return at project procurement, contract management, completion is that condition-based and planning. However, only 16 percent maintenance will be carried out support financial arrangements to throughout the life-span of the asset, so secure adequate, reliable, and that the benefit flow is maintained. predictable resources to operate and ix OVERVIEW Projects are rarely designed to minimize concession contracts, for example, to maintenance needs. mitigate the effects of delay in making the land available in a toll road Intercity highways and rural roads construction project. Measures related to projects, which usually depend on enhancing management capacity constrained public resources for included provision of technical advice— maintenance, are not based on for example, to ensure that technical evaluation of cost effective rehabilitation expertise is in place to oversee quality of and maintenance solutions for entire physical works, or redesign the project road networks or subnetworks. to avoid social conflicts—or enhancing corporate governance. For IFC- and MIGA-supported projects, sustained operation of investments is Findings: Has Bank Group support to always an implicit objective. sustain transport been implemented and is it effective? IFC and MIGA support private infrastructure and service providers in Sustained transport varies by country the transport sector through income and subsector. investments, guarantees, and transaction-oriented advisory services. Transport is likely to be sustained in As these private providers are exposed high- and upper-middle-income to commercial risks, IFC and MIGA countries, compared to low- and lower- satisfy themselves at the time of due middle-income countries. The diligence — before financing the subsectors most likely to have sustained operation —that their prospective transport are intercity highways and private sector engagements are likely to ports. Railways have faced difficulty remain financially and operational sustaining their infrastructure and viable in the long run. Thus, sustained services; urban transport, air transport, transport is an implicit objective in all of and rural roads have shown mixed these operations. results. Projects managed by the transport sector had higher sustained When there’s a need to improve the outcomes than those managed by other project design to enhance the sectors. sustainability of the investments, IFC adopts proactive measures. Projects that implemented measures to sustain transport were more likely to IFC has adopted proactive measures in achieve the intermediate outcomes. 32 percent of its transport projects. Most of the measures were related to Sector management and capacity enhancing the financial viability of the building activities had the highest project enterprises, these additional implementation rate by project closure, measures included mobilizing funds followed by institutional frameworks, from commercial lenders, enhancing sector policies, and regulatory financial structures, and modifying frameworks. Financial arrangements x OVERVIEW had the lowest implementation rate but 2002–11 and the three evaluated MIGA if implemented, they had the largest guarantee projects showed evidence of impact on the intermediate outcomes. sustained transport at operational maturity and beyond. Both intermediate Results are based on projects in selected outcomes—management capacity and countries due to lack of systematic financial viability—were achieved in monitoring of transport outcomes most cases. beyond project closure. Although 85 percent of IFC Advisory For more than half of the World Bank Services transactions delivered the projects reviewed in the selected needed outputs for the transaction countries, transport was sustained five process, only about a third were to six years beyond project completion. successfully completed. A Since there is no mechanism to comprehensive assessment of the systematically monitor sustained outcomes of these transactions is, transport beyond project closure, these however, not possible, as these projects results are limited to the findings from have not matured sufficiently and only a the subset of 76 operations supported by few post completion reports are the World Bank. There was very little available. documentation on the impact of analytic and advisory activities. The World Bank Group has leveraged the complementary roles of the three Operations supported by the World institutions in sustaining transport to Bank often have impact at the sector some extent, but more can be done. level. The World Bank’s efforts in creating an Assessment of most projects supporting enabling environment to sustain private the intercity highways subsector sector participation through the Public indicates sustained transport outcomes Private Infrastructure Advisory Facility at the sector level. This is because World (PPIAF) and other lending and analytic Bank support has focused on creating an and advisory activities (AAA) have a enabling policy, financial, and complementary character to IFC’s institutional environment for the entire investments since the World Bank’s road sector to manage and maintain activities focus on creating an enabling road networks. environment for sustained transport, including the policy and regulatory IFC investments and MIGA guarantees frameworks conducive for private sector were effective in sustaining transport participation. At the project level, but only one third of IFC Advisory coordination was found between IFC Services completed their transactions. advisory services and the World Bank. About 80 percent of IFC transport In a few cases, IFC’s due diligence for investments approved and investments built on World Bank operationally matured between fiscal expertise. Beyond these cases, however, little evidence was found that IFC xi OVERVIEW liaised at the project level with the needed for service providers to operate World Bank on a regular basis. and maintain infrastructure services. Findings: What factors have A clear accountability framework and contributed to success? incentive structure for operations and maintenance are critical for sustained Projects with objectives to sustain transport. transport and those that identify maintenance funding risks are more Across IFC- ,MIGA-, and World Bank- likely to be sustained in the long run. supported operations in certain subsectors— including ports and toll Among the projects reviewed that had roads—the key factors that contributed an explicit objective to sustain transport, to improved institutional capability three-quarters achieved it, compared to and financial viability were the half of the operations without an explicit contractual arrangements and user pay objective. If issues related to sustaining principles often featured in public- transport are the focus of project private partnerships. In intercity objectives, they are likely to have highways and rural roads, institutional measures to improve financial viability frameworks can adopt similar features, or institutional capability in the project if they are well designed and operated components and to sustain the according to contractual provision of transport infrastructure agreements/guidelines. and services beyond project closure. Similar results were found for projects A broad-based approach that aligns that identified maintenance funding funding sources has helped allocate risks. They are also more likely to maintenance budgets. sustain transport in the long run. Transport financing that relies on Financial viability and institutional diverse funding sources has improved capability are linked to sustained financial viability, especially in urban transport. transport and intercity highways. But these funding sources need to be The key shortcomings for poorly aligned. World Bank investments sustained transport were associated linked to countries that had public with institutional capability — weak expenditure reviews (PERs), especially institutional frameworks, lack of with explicit mention of transport autonomy, and technical capabilities in maintenance, were more likely to the responsible institutions needed to realize sustained transport outcomes. efficiently design, plan, construct, However, more than half of the PERs operate, and maintain transport. The reviewed failed to highlight the other factor was financial viability — transport maintenance funding issue. difficulty in ensuring adequate, reliable, and predictable financial resources xii OVERVIEW Supporting appropriate government- generating the necessary level of led reform programs in a realistic time demand to make transport services frame through continuous and viable. sequential engagements have been effective. Two key factors affect sustained transport in IFC and MIGA operations: Continuous policy dialogue and the quality of preparation and sequenced World Bank operations to incentive and accountability support government-led reform frameworks. programs, rather than just episodic project engagements, have been In the cases in which transport was not effective. The time frame for the sustained, the key reasons were supported actions should take into attributed to corporate governance, account the capacity of the government sponsor quality, and commercial to carry out the reforms especially in reasons. lower income countries and in Recommendations challenging subsectors like rural roads. Increase the focus on sustaining Political economy considerations need transport infrastructure and services in to be taken into account. project design. The political economy of reforms – who • Increase the focus on sustained benefits, who doesn’t, and how the transport in projects with sector latter can be compensated – is critically reform objectives and components important to understand. In countries through measures to adopt or where political economy was factored strengthen the financial in during the preparation process arrangements for and enhance through stakeholder consultation, institutional capability to plan and inclusive analysis, and communication carry out operations and have also contributed to reducing the maintenance. resistance to reforms, including for the establishment of road funds in • World Bank-supported projects for highways and central coordination the intercity highways and rural agency for urban transport, and staff roads subsector should: (i) rationalization in railways. systematically carry out ex-ante risk Multi-modal planning is more effective analysis and mitigation with regards than fragmented interventions. to operation and maintenance and funding shortfalls; (ii) conduct As evidenced especially in the sensitivity analysis on the effect of railways, multimodal planning and inadequate maintenance on net coordination at the country’s sector benefit flows resulting from level has also been important in transport infrastructure and services; xiii OVERVIEW and (iii) systematically evaluate registration fees, and congestion entire network or subnetwork charges. managed by the road agency to seek cost effective rehabilitation and • In the railways subsector, critically maintenance solutions. assess the viability of investments, with particular attention to: (i) • Link World Bank transport realistic demand forecasts based on investment operations with sector - analysis of potential competition wide reforms that will sustain these from other transport modes; and (ii) investments through support for realistic estimation of rehabilitation complementary development policy and maintenance costs. operations and analytic and advisory activities. • Support to urban transport should include a comprehensive financial • World Bank-supported projects that analysis of the overall urban finance transport components and transport system, including fare are managed by other sectors should integration, tariffs and subsidies, and ensure that transport components the net impact on the poor. are integrated into the operations and maintenance of transport sector Strengthen institutional capability to plans and strategies. sustain transport outcomes. Improve the long-run financial viability • Where complex reforms are planned, of support for sustained transport. encourage continuous and sequential engagement and support • For the subsectors and transport appropriate government-led reform modes that rely on operations and programs in a realistic time frame, maintenance funds from public taking into account the capacity of sources or earmarked funds such as the government to carry out the intercity highways and rural roads: reforms. (i) engage with the client where high level policy decisions related to • Factor in the political economy in the maintenance funding can be taken; reform process by identifying the (ii) within the Bank, put a more key stakeholders and constituencies prominent emphasis on the upfront, carrying out stakeholder adequacy and reliability of transport analysis, ensuring consultation and maintenance funding in public communication during preparation expenditure reviews. of the reform, and adopting ways of compensating the affected groups to • Diversify the sources of financing for minimize the resistance to change or roads and urban transport for more delays in legal or regulatory reliable financing, such as axle approvals. control fees, fuel surcharges, vehicle xiv OVERVIEW • In subsectors that are dominated by the public sector, such as intercity highways and rural roads, mainstream proven models of demand-side governance and commercial principles to ensure that there is a proper accountability and incentive framework in place, such as output- and performance-based maintenance contracting, second generation road funds, and microenterprise models. • In order to strengthen the ability of countries to routinely collect data on, monitor, and assess sustained transport outcomes, support governments to put in place a reliable system to monitor and evaluate such outcomes systematically in all subsectors, particularly for ensuring adequate road maintenance. xv Management Response Management welcomes this evaluation of the World Bank Group support on “sustaining” the provision of transport infrastructure and services, covering the period 2002 through 2011, by the Independent Evaluation Group (IEG). This evaluation is relevant in view of the large and increasing volume of transport lending, which, as IEG reports, represents 12 percent of the World Bank Group’s total commitments and guarantee volumes over the evaluation period (reaching up to 17 percent in recent years), and the renewed focus on the delivery of infrastructure for poverty reduction and economic growth. The first section sets out comments from World Bank management. The second section provides comments from International Finance Corporation (IFC) management. Management’s specific response to IEG’s recommendations, with which it generally agrees, is noted in the attached draft Management Action Record matrix World Bank Management Response This evaluation takes place against the backdrop of a sector, which has about 89 percent of its closed projects rated moderately satisfactory or better over the period under evaluation. Management welcomes the overall positive findings of the review. Among the important elements of this evaluation are: (a) its emphasis on the centrality of transport for poverty reduction and economic growth; (b) the superior performance of transport projects compared to other sector projects of the World Bank; (c) a significant share of transport projects that identified maintenance and funding shortfalls as risks to achieve sustaining transport as an objective; (d) “sustained transport” for five to six years beyond project completion in many Bank projects; and (e) achievement of the objective of “sustained transport” in three-quarters of the transport projects reviewed. The transport agenda features prominently in the Bank Group’s engagement with clients, as a critical means to achieve Inclusive Green Growth. Safe mobility for people is critical for social inclusion. Hence, the Bank has a renewed effort to provide the poor with access to low-carbon transport modes while strengthening transport safety, in particular on the road. Transportation is one of the most difficult areas of environmental sustainability due to an ever increasing demand for transport services and its reliance on fossil fuels. As such, Bank support has promoted lasting and sustainable modal shifts to low-energy modes, both for people and freight movements. These efforts are epitomized in the context of cities development: the transformation of cities towards more compact, livable, and economically-efficient agglomerations goes through the implementation of an effective metropolitan mobility plan, which involve all modes of transportation from non-motorized to mass transit systems. More broadly, xvii MANAGEMENT RESPONSE transport enables access to jobs, education, health care, and other socio-economic activities, thus empowering under-privileged segments of the population and underpinning the development of a resilient social fabric. Transport also remains the backbone of both domestic and international trade. Consequently, improvements in transport and logistics systems are a prerequisite for economic competitiveness and enlarged market access. Moving towards green logistics is therefore a parallel track to improving people mobility, while advancing transport safety and security protects social inclusion and strengthens commercial exchanges that in turn promote economic growth. The estimated demand for transport finance is outstripping current investments by a large margin. The financing needs to avoid a “+4 C degree world” will roughly double because of adaptation costs. Moreover, transport infrastructure having typically long economic lives, it is critical to design them so as to avoid lock-in effects that could slow down the transition to a sustainable development path. Closing the gap in both financing and engineering for new sustainable transport infrastructure requires leveraging additional sources of finance and innovation. Time is of the essence: adapting and developing transport infrastructure is not an option; it is a must to achieve inclusive green growth. Given the broader context of Bank support to the transport sector, IEG’s review focuses on a narrow component — Has the World Bank Group been effective in ensuring that client countries have the appropriate institutional capability and financial arrangements in place so that transport infrastructure is operated and maintained properly over a long life cycle? The evaluation covers a variety of institutional and financial arrangements across a wide spectrum of transport sub-sectors, including ports, railways, urban transport, intercity highways, rural roads, airports, and waterways. Management concurs with IEG that sustained benefits from any infrastructure investment depend on adequate operations and maintenance (O&M). This issue has long been recognized and integrated in the Bank’s engagement in transport with client countries. As IEG’s report shows, a large number and share of analytical and advisory activities supported by the World Bank has focused on sustaining transport. Support for output and performance-based road maintenance contracts and plans for public- private partnerships have been on the rise. In addition, a large number of Bank projects have identified maintenance and funding shortfalls as risks, or even achieved sustained transport as an objective. Notwithstanding the achievements in these areas noted by IEG, the challenges faced by the transport sector are still significant: a lack of funding, poor sector capacity, an inadequate institutional policy and regulatory framework, a focus on hard infrastructure to the detriment of O&M, and short-terminist approach, which can all weaken the transport system and erode the service provision. These are some of the issues that the World Bank Group’s engagement in transport aims to address. xviii MANAGEMENT RESPONSE “Sustained transport” is an issue that cuts across countries. As the evaluation shows, transport is more likely to be sustained in high-income and upper-middle-income countries, as compared to low- and lower-middle-income countries. However, inadequate operations and maintenance is an issue that is not only specific to World Bank client countries: it cuts across developed and developing countries. Putting in place the right policies, institutional and regulatory framework, sector management capacity, and financial arrangements to ensure that transport infrastructure is properly operated and maintained has remained a concern through the present. The World Bank has worked on this issue for more than 20 years, testing and drawing lessons from various approaches. Road funds, for example, which at some point used to be regarded as the way to address the O&M issue in transport, were sometimes dismantled as governance and fiscal issues soon undermined their own sustainability. Over time, the limits to self-financing of infrastructure maintenance and operations became obvious for the transport community as a whole. The World Bank has long developed the tools for appropriate planning. The Bank developed an infrastructure planning toolkit, the HDM (Highway Development and Management) toolkit in the 1980s. This tool is globally accepted as the model for planning in particular road infrastructure investments. Since 2011, the toolkit is maintained and globally disseminated to infrastructure agencies and transport ministries around the world through the World Road Association (PIARC). This planning tool includes a risk analysis on maintenance funding. It will, however, have to be tailored to the size and the socio-geographic context of the project to limit project preparation costs. For instance, the type of network analysis that is routinely applied for highway projects should only be used for rural road and urban transport projects if the project impacts clearly go beyond the local level. Maintenance funding is a public finance issue. Decision-making on the allocation of maintenance funds goes beyond transport sector interventions and transport policy decision-making. Maintenance funding is subject to discretionary decision-making depending on the macro-economic environment and fiscal stance. In simple terms, such decisions are made by the Ministry of Finance, and not the Transport Ministry, within an overall fiscal context. Governments take decisions on resource allocation for maintenance at the level of infrastructure investments and not at the level of a project. As such, it is difficult for a project to integrate maintenance funding as an objective. In management’s view, resource allocation for maintenance can only be made a direct project objective in transport projects that aim at sector reform. “Sustained” transport is more than maintenance funding. Against the backdrop of current challenges of sustainable development, the transport agenda has become much more complex. Transport policy has now to integrate the social costs and benefits of transport. It should put greater emphasis on ensuring transport safety; it should make xix MANAGEMENT RESPONSE sure that developing countries do not get “locked-in” to energy-intensive transport; and it has to adapt to changing weather conditions. Adaptation needs cast a new light on maintenance demands, as the changing weather conditions increase the demand for maintenance efforts. This broader context of “sustained transport” calls for stronger emphasis on sector-level reform, stronger links between transport infrastructure policy and other policy areas, and a longer-term engagement, with appropriate sequencing of projects. IEG’ S RECOMMENDATIONS Management agrees with the IEG’s call to maintain and enhance the emphasis on “sustained transport” and to a stronger focus on sector reform over project-level engagement will increase opportunities to improve funding for maintenance. This will require going beyond the time-span of a project life cycle, into adopting a more programmatic approach to transport interventions, with an appropriate sequencing of projects. This will help better align transport operations with sector- and country-level policy dialogue and make it easier to secure funding for transport infrastructure maintenance by giving transport a bigger role in the dialogue on public finance. This approach will, however, have to be made consistent with the country-based model of the Bank. Internally, this will entail: (a) strengthening project teams’ commitment to using existing planning tools and ensuring their effectiveness through regular review; (b) closer cooperation of the transport sector board with other sector boards on planning and implementation of transport components in non-transport projects; and (c) renewing efforts to identify sources of self-financing for maintenance and operations in client countries. IFC Management Response Management welcomes IEG’s evaluation of the World Bank Group’s support for improving institutional capability and financial viability to sustain transport. The report provides a valuable independent assessment of IFC’s development results in a key sector area within IFC’s strategic focus on infrastructure. The report correctly recognizes the complementary roles of the relevant institutions of the World Bank Group. Both the World Bank, through lending and analytic and advisory activities, and IFC, through advisory services, work with governments to help create an enabling environment conducive for sustained transport and to support public-private partnerships (PPPs). The World Bank provides financing for public sector transport projects, while IFC and MIGA support private sector projects through their respective financing instruments. xx MANAGEMENT RESPONSE Management is pleased with the report’s overall positive findings on IFC’s transport sector interventions. About 80 percent of IFC investment services operations that have reached operational maturity are successful in providing sustained transport. The report also shows that several of these private sector projects have broader effects in sustaining transport beyond the project companies. Some projects served as models for lifting the technical quality and service standards of other major players. Other projects demonstrated the viability of private sector participation in a nascent transport sector, attracting the entry of other players. On advisory services, the report indicates that 85 percent of the transaction delivered specific technical advice. It is, however, premature to assess the outcomes of many PPP advisory services. IFC agrees with the report’s recommendation that is applicable to IFC. The report recommends that in the railways subsector, the World Bank Group should critically assess the viability of investments, with particular attention to: (i) realistic demand forecasts based on analysis of potential competition from other transport modes; and (ii) realistic estimation of rehabilitation and maintenance costs. This recommendation is consistent with IFC's due diligence process and is already the current practice in IFC. xxi Management Action Record IEG Findings & Conclusions IEG Recommendations Acceptance Management Response by Mgt of Recommen- dation Sustained Transport in World Increase the focus on sustaining Bank Group operations transport infrastructure and services in project design We will enhance our focus on sustained transport, within the Sustained transport has been a WB: Agree • The World Bank should purview of a country-based model, by continuing to remain longstanding concern in World engaged in the Country Assistance Strategy (CAS) process, and Bank Group sector and a large increase the focus on sustained transport in projects pursuing the dialogue with client countries on sector reform. share of the country strategies, yet the share of Bank operations with sector reform objectives Transport projects will apply the HDM-4 (Highway Development with explicit objectives to sustain and components through and Management Model), with a focus on the road financing and transport is low and declining. measures to adopt or road funds model, as well as contracting maintenance model, Further, nearly a third of Bank strengthen the financial whenever possible. transport operations identified arrangements for and enhance A guidance note will be developed and endorsed by the Sector lack of maintenance funding as a institutional capability to plan Board to establish criteria for the application of network analysis risk at project appraisal. and carry out operations and with focus on rural and urban transport. An internal review maintenance. process will be established to check and report on the inclusion of World Bank projects with • World Bank-supported projects risk and sensitivity analysis as well as network analysis for objectives to sustain transport for the intercity highways and projects going beyond local level. and those that identify rural roads subsector should: maintenance funding risks are The Transport Sector Board will put in place a mechanism to (i) systematically carry out ex- more likely to support measures review non-transport projects including significant transport ante risk analysis and to improve financial viability or components. mitigation with regards to institutional capability, and more operation and maintenance likely to be sustained in the long and funding shortfalls; (ii) run. conduct sensitivity analysis on xxiii MANAGEMENT ACTION RECORD IEG Findings & Conclusions IEG Recommendations Acceptance Management Response by Mgt of Recommen- dation the effect of inadequate Transport is less likely to be maintenance on net benefit sustained in low- and lower- flows resulting from the middle-income countries, and in transport infrastructure and projects managed by sectors services; and (iii) other than the transport sector. systematically evaluate the entire network or subnetwork Ex ante economic analysis managed by the road agency rarely takes into account the to seek cost-effective effect of underfunding rehabilitation and maintenance maintenance on the benefit solutions. flows. • World Bank-supported projects that finance transport components and are managed Highway projects, which usually by other sectors should ensure depend on constrained public that transport components are resources for maintenance, are integrated into the operations not being designed to minimize and maintenance of transport maintenance needs. sector plans and strategies. xxiv MANAGEMENT ACTION RECORD IEG Findings & Conclusions IEG Recommendations Acceptance Management Response by Mgt of Recommen- dation Financial viability Improve the long-run financial viability of support for sustained transport. WB: Agree The Transport Sector Board will assess the transport sector’s Financial viability is one of two intermediate outcomes • For the subsectors and contribution to PERs and identify opportunities to integrate associated with sustained transport modes that rely transport into PERs. transport. on operations and During design, project task teams will explore more maintenance funds from systematically a variety of sources of funding for maintenance Countries that had public public sources or and operations, including plans for self-finance. expenditure reviews (PERs), earmarked funds such as intercity highways and rural Efforts to realistically evaluate cost and revenue forecasts that especially with explicit mention roads: (i) engage with the cover competition between different modes of transport will be of transport maintenance, were client where high-level enhanced, with particular attention to urban transport and more likely to realize sustained policy decisions related to railways. transport outcomes. However, more than half of the PERs maintenance funding can Ex ante analysis of urban transport projects will include an reviewed failed to highlight the be taken; (ii) within the analysis of subsidy requirements and integration of pricing of transport maintenance funding Bank, put a more different sub-modes of transport. issue. prominent emphasis on the adequacy and reliability of transport maintenance Transport financing that relies funding in public on diverse funding sources expenditure reviews. has improved financial viability in urban transport • Diversify the sources of and intercity highways. financing for roads and urban transport for more reliable Railways have faced difficulty financing, such as axle sustaining their infrastructure and control fees, fuel surcharges, services for both World Bank and vehicle registration fees, and IFC projects because they congestion charges. xxv MANAGEMENT ACTION RECORD IEG Findings & Conclusions IEG Recommendations Acceptance Management Response by Mgt of Recommen- dation generally did not pay enough • In the railways subsector, attention at appraisal to market critically assess the viability demand conditions, such as road of investments, with particular IFC: Agree IFC' s current due diligence process includes a comprehensive competition, and accurately attention to: (i) realistic market assessment and financial projections with sensitivity estimating the capital investment demand forecasts based on analysis. IFC’s market analysis takes stock of historical demand needs to revamp infrastructure. analysis of potential and future developments that could impact the demand and competition from other supply balance, including competition from other modes of While low-income passengers are transport modes; and (ii) transport. Our financial projections include estimates of capital less able to pay, the benefits of realistic estimation of expenditures for major rehabilitations and maintenance based on improved access to jobs and rehabilitation and a technical evaluation. economic opportunities may be maintenance costs. great. The net impact of user fees to finance transport operation and • Support to urban transport maintenance on the poor is should include ambiguous, however, and comprehensive financial depends on the context. analysis of the overall urban transport system, including fare integration, tariffs and subsidies, and the net impact on the poor. xxvi MANAGEMENT ACTION RECORD IEG Findings & Conclusions IEG Recommendations Acceptance Management Response by Mgt of Recommen- dation Institutional Capacity Strengthening Institutional Capabilities to Sustain Transport Enhanced institutional capability is Outcomes WB: Agree Project task teams will use the institutional design tools available, a second • In subsectors that are like the modules “Institutional and Management Structures for intermediate outcome associated dominated by the public Roads” and “Road Financing and Road Funds” of the HDM-4, with sustained transport sector, such as intercity wherever applicable. They will use the outcomes of the recent outcomes. highways and rural roads, analytical work on performance-based contracting to improve stream proven models of maintenance and operations. A clear accountability framework demand-side governance and In the urban transport context, the Bank’s Leaders in Urban and incentive structure for commercial principles to Transport Planning (LUTP) program will be offered worldwide to operation and ensure that there is a proper strengthen institutions and institutional cooperation between non- maintenance is critical for accountability and incentive transport and transport authorities. sustained transport in IFC and framework in place, such as MIGA supported operations, and Management recognizes the importance of moving towards a output- and performance- Bank-supported operations in more programmatic engagement in client countries. This will based maintenance certain subsectors, including ports allow for longer-term cooperation with client countries and offer a contracting, second and toll roads. This is often time frame to build the capacity and create the institutions generation road funds, and addressed through contractual needed for better resource allocation and organization of microenterprise models. arrangements and the user pay maintenance and operations. principle often featured in public- • Where complex reforms are Part of these efforts will take the form of stakeholder private partnerships. In intercity planned, encourage consultations and mobilization to strengthen the voice on the highways and rural roads, continuous and sequential demand side for improved institutional arrangements. institutional frameworks can adopt engagement and support similar features, if they are well appropriate government-led Efforts to improve data collection at the project level on designed and operated according reform programs in a realistic maintenance of infrastructure asset values will be sustained. to contractual time frame, taking into agreements/guidelines. account the capacity of the Continuous engagement in the government to carry out the sector, through policy dialogue reforms. xxvii MANAGEMENT ACTION RECORD and sequenced World Bank operations in the transport sector • Factor in the political has contributed to achieving policy economy in the reform reforms, especially in countries process by identifying the with weak capacity. key stakeholders and constituencies upfront, The political economy of reforms – carrying out stakeholder who benefits, who doesn’t, and analysis, ensuring how the latter can be consultation and compensated – is critically communication during important to understand before preparation of the reform, launching a reform to sustain and adopting ways of transport. Stakeholder compensating the affected consultation, inclusive analysis, groups to minimize the and communication have also resistance to change or contributed to reducing the delays in legal or regulatory resistance to reforms. approvals. There are limited data to assess • Support governments to put in sustained transport outcomes; the place a reliable process for monitoring put in place during systematically monitoring and project implementation is often not evaluating sustained transport continued beyond project closure. infrastructure and services in all subsectors, particularly for road maintenance. xxviii Chairperson’s Summary: Committee on Development Effectiveness The Committee on Development Effectiveness (CODE) met to consider Improving Institutional Capacity and Financial Viability to Sustain Transport: An Evaluation of World Bank Group Support Since 2002 and the Draft Management Response. Summary The Committee welcomed the Independent Evaluation Group’s (IEG) evaluation. They appreciated IEG and management’s general convergence of views and the clarity in the Management Action Record matrix on the recommendations and responses by management. The Committee concurred with the evaluation’s findings and recommendations to i) increase the focus on sustaining transport infrastructure and services in project design, including through systematic ex-ante risk analysis and linking transport investment operations with sectorwide reforms that will sustain these investments; ii) improve the long-run financial viability of support for sustained transport, including greater emphasis on adequacy and reliability of transport maintenance funding in public expenditure reviews; and iii) strengthen institutional capability to sustain transport outcomes, including through improved political economy analysis and clear accountability frameworks and incentive structures for operations and maintenance. The Committee looked forward to the upcoming Progress Report on Transport, which would take into account lessons learned from the evaluation. Anna Brandt, Chairperson xxix 1. Introduction Highlights The World Bank Group has committed nearly $50 billion to transport over the past decade, representing 12 percent of all institutional commitments in lending, investments, and guarantees. Sustained benefits from these investments depend critically on adequate operations and maintenance. Sustaining transport infrastructure and services has been a persistent concern of Bank Group transport strategies, which have prioritized improving the quality of services over increasing the stock of infrastructure. This evaluation assesses the extent to which World Bank, International Finance Corporation (IFC), and Multilateral Investment Guarantee Agency (MIGA) investments in transport infrastructure and services over the past decade have been sustained, the factors associated with improved outcomes, and lessons from the experience. The services provided by investments in road networks, railways and transit systems, and ports for trade by air and water are important contributors to poverty alleviation and economic growth. Sustained transport investments can support poverty reduction directly, in terms of improving access and thus economic opportunity targeted to the rural poor, and also through facilitating broad-based growth (Box 1.1). But the impact of private or public infrastructure investments and the services that arise from them can be undermined by poor operation and maintenance (O&M). Lack of funding, poor sector capacity, and inadequate institutional, policy, or regulatory frameworks which are the key contributing factors, can weaken a transport system and erode service provision (World Bank 1994, 1996, 2006, 2008). A 1988 World Bank Policy Study for the roads subsector estimated that about $88 billion worth of road infrastructure assets had been lost over the preceding two decades owing to inadequate maintenance in 85 developing countries reviewed. This loss of assets could have been averted with preventive maintenance costing less than $23 billion (World Bank 1988). 1 A 1998 study of the impact of road maintenance on vehicle operating costs in 33 countries found that every dollar spent annually on maintenance saves from $1.40 to $44.80 in vehicle operating costs, depending on the number of vehicles (World Bank 1998). 2 1 CHAPTER 1 INTRODUCTION Box 1.1. Sustained Transport, Poverty Alleviation, and Economic Growth Econometric modeling by the International Food Policy Research Institute in five countries— China, India, Thailand, Uganda, and Vietnam, representing different stages of economic development—calculated the number of poor people raised above the poverty line for each additional unit of public spending on different expenditure items. The results show a positive relationship between public spending on rural roads and poverty reduction. The direct effects arise in the form of benefits the poor receive from employment programs directly targeted to the rural poor. The indirect effects arise when government investments in infrastructure (particularly rural roads), agricultural research, health, and education stimulate agricultural and nonagricultural growth, leading to higher employment and income-earning opportunities for the poor and to cheaper food. In urban settings, long commutes resulting from poor transport infrastructure can greatly limit women’s ability to participate in the workforce because of the time constraints associated with childcare. In rural Pakistan and Guatemala, the expansion of rural road networks contributed to increased female mobility and schooling (World Bank 2012). A study of the socioeconomic impact of national highways on rural population conducted in India between 2003 and 2010 showed a threefold increase in the share of income from non- agriculture activities; an 85 percent increase in female labor participation; a twofold increase in the per capita trip rate for education; and about a 50 percent increase in school enrollment. Source: Shenggen Fan 2009, Asian Institute of Transport Development 2011, World Bank 2012 Transport has historically been one of the largest investment portfolios of the World Bank Group 3. The World Bank, comprising of the International Bank for Reconstruction and Development (IBRD) and the International Development Association (IDA); and IFC and MIGA have increased their annual commitments and guarantee volumes in the transport sector from $2.7 billion in fiscal 2002 to $8.5 billion in fiscal 2011 (Figure 1.1). During that decade, 577 operations were approved in the sector, amounting to approximately $50 billion in commitments and about 12 percent of the Bank Group’s total commitments and guarantee volumes. Most of these investments were financed by the World Bank: IBRD and IDA accounted for 89 percent of total commitments and 65 percent of operations, followed by IFC (9 percent of commitments, 31 percent of operations), and MIGA (3 percent and 4 percent, respectively). 2 CHAPTER 1 INTRODUCTION Figure 1.1. World Bank Group Commitments and Gross Volume of Guarantees, and Number of Transport Operations Supported, by Fiscal Year of Approval A. Trends in World Bank Commitments and Number of Projects, by Fiscal Year of Approval Commitments (US$ million) 8000 80 7000 70 6000 60 Number of Projects 5000 50 4000 40 3000 30 2000 20 1000 10 0 0 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 World Bank Commitments World Bank (No. of Projects) B. Trends in IFC Commitments/MIGA Gross Exposure and Number of Projects by Fiscal Year of Approval Commitments/Gross Exposure (US$ million) 3000 18 16 2500 14 Number of Projects 2000 12 10 1500 8 1000 6 4 500 2 0 0 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 IFC Commitments MIGA Gross Exposure IFC (No. of Projects) MIGA (No. of Guarantees) Source: World Bank Group data. 3 CHAPTER 1 INTRODUCTION The performance of World Bank transport operations at closure has been high, yet inadequate operations and maintenance — raised as an issue more than 20 years ago — has remained a concern. Among the 209 projects that closed during fiscal 2002-11 and have been evaluated by the Independent Evaluation Group (IEG), 89 percent were rated “moderately satisfactory” or higher 4 at project closure, compared to 76 percent for all World Bank operations. The 1988 policy study reviewed the financial impact of inadequate maintenance and the 1994 World Development Report highlighted the growing concern that major investments in basic infrastructure facilities were not resulting in service improvements. This prompted the World Bank Group to increase its emphasis on improving the quality of services rather than the quantity of infrastructure, as reflected in its 1996 transport strategy and all subsequent strategies (World Bank 1996). A recent study of transport infrastructure in Africa found that road networks throughout the region are inadequately maintained and that the region is not even close to allocating sufficient budget for the required infrastructure maintenance (Africa Infrastructure Country Diagnostics, 2011). The extent to which the World Bank Group’s support has resulted in sustained provision of transport services has not previously been evaluated. The most recent IEG evaluation of World Bank assistance to the transport sector, covering the period 1995–2005, assessed the effectiveness of World Bank support over a broad range of topics, such as the roles of the public and private sectors, maintenance, institutional development, environmental protection, and the impact of transport on poverty reduction. It stressed that institutional change takes longer than the life cycle of a typical project. This evaluation builds on the findings of the previous one and focuses on the effectiveness of Bank Group support in sustaining the provision of transport infrastructure and services. Objective and Scope of the Evaluation The objective of this evaluation is to assess the effectiveness of World Bank Group support to help countries sustain the provision of transport infrastructure and services. 5 The term “transport“ is used throughout this report to mean transport infrastructure and transport services, unless otherwise specified. “Sustained transport” is defined as the extent to which policies, institutional and regulatory frameworks, sector management capacity, and financial arrangements are in place to ensure that transport infrastructure is operated and maintained, enabling a reliable flow of services over the long term. The outcomes are assessed in the form of benefits flowing from infrastructure and services to users. For intercity highways, for example, this would mean the improvement in road network conditions, which would in turn reduce 4 CHAPTER 1 INTRODUCTION travel time and costs, improve access, and enhance safety for the users. Measures of sustained transport relevant for different subsectors are summarized later in this chapter. Although sustained transport outcomes are evaluated at the project level, these outcomes reflect sector-level achievements in many cases because World Bank projects often support institutional, financial, or policy measures that are national in scope, and not confined to the areas covered by the investments supported under the projects. Assessments of IFC- and MIGA-supported investments, on the other hand, are predominantly at the project level. This evaluation also assesses the institutional and financial aspects of sustaining transport. The 1996 World Bank strategy Sustainable Transport: Priorities for Policy Reform 6 identified four pillars of sustainability: social, financial, economic, and environmental. This evaluation focuses on institutional capability and financial viability in selected projects and the extent to which infrastructure and services are sustained at and beyond project closure. Recent IEG evaluations have highlighted findings on the environmental and social aspects of transport (Box 1.2). Assessment of development impacts of infrastructure, including poverty alleviation, economic growth, and macroeconomic outcomes, are outside the scope of this evaluation. 5 CHAPTER 1 INTRODUCTION Box 1.2. Findings on Environmental and Social Sustainability of Transport Operations from Recent IEG Evaluations Safeguards and Sustainability Policies in a Changing World (2010) The evaluation reviewed the implementation of safeguard policiesa in 232 World Bank– supported projects approved from 1999 to 2008, 47 of which were in the transport sector. Client implementation of safeguards in transport projects was similar to that of the Bank as whole, as shown through the implementation of environmental mitigation plans (60 percent substantial or better for transport versus 63 percent Bank-wide) and the provision of replacement property or compensation to project-affected persons. Climate Change and the World Bank Group (2009, 2010, 2012) The three-phase IEG assessment of Bank Group engagement with climate change suggests that the choices and investments made in transport systems will irreversibly shape the future of the global climate. Transport will be the most difficult sector in which to curb the soaring increase of carbon emissions, as most future transport sector emissions will originate in developing countries. In countries outside of the Organization for Economic Cooperation and Development (OECD), greenhouse gas emissions from transport nearly doubled from 1990 to 2006, and transport’s share of emissions rose from 5.6 to 12.8 percent. To make a dent in carbon dioxide emissions in the transport sector, greenhouse emissions from road transport within and between cities must be reduced. The first phase evaluation (2009), on greenhouse mitigation, found that where diesel prices are less than half the world market rate, emissions are about twice as much per capita as other countries at similar income levels. OECD countries that have maintained high fuel prices for decades (through taxation) have evolved more efficient transport systems. The second phase of the evaluation (2010), which focused on the challenge of low-carbon development, found that growing urban transport demand is leading to severe urban congestion, air pollution, and greenhouse gas emissions. The Bank Group has supported bus rapid transit systems in Bogota and Mexico City, which were replicated elsewhere. Bus rapid transit offers attractive economic returns to cities in time and fuel savings and reduced pollution and congestion. The third phase of the evaluation (2012), which focused on adapting to climate change, highlighted the need to incorporate climate risk into project planning because of the vulnerability of transport infrastructure to climate risks. Note: a. Environmental and social safeguard policies (World Bank) and Performance Standards (IFC and MIGA) are a cornerstone of the Bank Group’s support for sustainable development and poverty reduction. The objectives of these policies, to which the Bank Group has committed and is publicly accountable, are to improve the quality of investments and guarantee operations and to prevent or mitigate undue harm to people and the environment in the development process. Source: IEG 2009, 2010, and 2012. The World Bank, IFC, and MIGA approach sustained transport differently, consistent with their institutional mandates. The World Bank supports governments to create or improve the enabling environment for sustaining transport by, for example, developing sector policies; setting up legal, regulatory, and institutional frameworks; and funding transport infrastructure investments. IFC and MIGA support private investors in the transport sector—mainly through 6 CHAPTER 1 INTRODUCTION support for investments, guarantees, and transaction-oriented advisory services projects that are in an already improved enabling environment for sustaining transport. They also support advisory work on regulatory and general business climate matters. As a result, the transport investments and guarantees of IFC and MIGA predominantly support projects in richer, middle-income countries that are more likely to have an enabling environment in place, while the World Bank’s support is more evenly spread across the country income groups (Figure 1.2). Figure 1.2. Distribution of World Bank and IFC Transport Projects, by Country Income, FY02-11 A. WORLD BANK (417 PROJECTS) B. IFC (120 PROJECTS) High High income, Upper income, 7, 2, 2% middle 1% income, 153, 37% Low income, Upper Low 11, 9% middle income, income, Lower 95, 23% 77, 64% middle Lower income, middle 30, 25% income, 162, 39% Note: For the World Bank, the total number of projects includes projects managed by the Transport Sector and by other sectors. Twenty regional projects are excluded. For IFC, five regional projects are excluded. Source: World Bank Group data. The evaluation seeks to answer four questions: To what extent have Bank Group strategies and operations aimed to ensure sustained transport? To what extent have countries implemented the policies and regulatory framework, institutional framework, financial mechanisms, and capacity building activities supported by the Bank Group to sustain transport? How effective have these measures been? What factors have determined success? The transport sector, as defined in this evaluation, includes urban transport, rural roads, intercity highways, railways, ports and waterborne transport, and air transport in all Bank Group client countries. The evaluation recognizes that adequate maintenance is critical for sustained intercity highways and rural roads, while continuous operations by service providers is critical for sustained urban 7 CHAPTER 1 INTRODUCTION transport, air transport, and ports and waterborne transport. Sustained transport in the railways subsector is affected by both operations and maintenance. The evaluation covers the transport portfolio of World Bank (IBRD and IDA), IFC, and MIGA projects approved during fiscal 2002–11. Evaluation Framework The analytical framework for this evaluation shows the channels through which the three institutions, governments, and the private sector can achieve sustained transport as a contribution to poverty alleviation and economic growth (Figure 1.3). The evaluation assesses the effectiveness of World Bank support to governments and IFC and MIGA support to the private sector through projects that finance outputs to achieve the outcomes in the results chain, that is, sustained transport. The development impacts of sustained transport are outside the scope of this evaluation. Figure 1.3. The Results Chain and Framework for the Evaluation Note: The scope of the evaluation is shaded. Source: IEG Evaluation Team 8 CHAPTER 1 INTRODUCTION The evaluation assesses outputs, financial and institutional intermediate outcomes, and sustained transport outcomes. Sustained transport beyond project closure is the outcome as defined in this report. 7 This outcome is expected to be achieved through sustained institutional capability as well as the financial viability of the country’s transport sector and related institutions supported under the projects (intermediate outcomes). The evaluation also recognizes that development impact could be affected by other factors, including interventions by other donors and financiers, interventions through other sectors such as health and education, the global economy, political economy, and natural disasters. Outputs that are expected to affect sustained transport consist of physical investments and four broad categories of measures that help sustain transport: sector policy and regulatory frameworks, institutional frameworks, sector management capacity, and financial arrangements that are hypothesized to sustain transport (Box 1.3). Physical investments in transport infrastructure are the largest outputs in commitment volume terms across Bank Group–supported projects, and are typically grouped into support for upgrading, rehabilitation, maintenance, disaster, and post-conflict reconstruction, as well as new construction. The measures that help sustain transport are predominantly supported by the World Bank through its lending and non-lending operations to governments. IFC and MIGA work primarily at the project level toward sustaining the private enterprise and only take proactive and corrective measures when they are likely to be needed. These measures are not defined as intended outputs in the results chain, but they nevertheless affect the ability of private enterprises to sustain transport. The Bank Group’s support to the countries through other means, such as country dialogue and operations in other sectors, may influence country priorities. 9 CHAPTER 1 INTRODUCTION Box 1.3. Measures that are Postulated to Contribute to Sustained Transport Sector policies and regulatory frameworks comprise support in defining and implementing appropriate sector policies and strategies and legal and regulatory frameworks. Institutional frameworks include public and private sector arrangements in service delivery, such as the establishment and strengthening of government agencies responsible for the transport subsector, community-based programs, restructuring and corporatization, concessions,a performance-based contracts, and other forms of public private partnerships. Sector management capacity building involves support in building the country’s institutional capacity to plan, budget for, manage, and operate infrastructure and services. This measure has two parts—support for improved systems and mechanisms for sector planning and management, including the adoption of safety and governance measures, implementing agency–level reforms, and provision of training activities and technical studies. Financial arrangements include reforms in budgetary provisions and adoption of user fees, earmarked funds, pricing policies, subsidies, and taxation models introduced to provide funds particularly for the purpose of operations and maintenance. Note: a. Concession is defined as a grant of property by the government to the private sector in return for services for a determined period of time. As elaborated in chapter four, while the risks related to private sector participation are well known, if properly structured it has a high potential to achieve sustained transport and therefore the evaluation captures concession and other forms of public private partnerships as a measure to sustain transport. In assessing the outcomes at or beyond project closure or operational maturity 8, the same criteria are used across World Bank, IFC, and MIGA–supported operations. 9 The outcomes are measured by whether the transport services have the accessibility, capacity, physical condition, and design features necessary to meet the needs of users and are typically expressed in terms of travel cost or travel time. The specific indicators vary depending on the subsector and mode of transport. Certain subsectors, including intercity highways may have sustained outcome indicators that are at the sector level such as condition of the country’s road network, safety, and access to all-weather roads. Examples of indicators for different transport subsectors are listed in Table 1.1. Details of the evaluation framework and additional examples of the indicators by transport mode or subsector are detailed in Appendix C. The evaluation recognizes that physical assets can deteriorate not only because of lack of maintenance but also because of poor construction. Institutional capability of the sector to plan, design, and implement has been assessed to reflect the quality of construction. However, the evaluation cannot attribute sustained transport outcomes exclusively to Bank Group support, given the passage of time and the possibility that other factors have affected the outcomes. 10 CHAPTER 1 INTRODUCTION Table 1.1. Indicators of Sustained Transport Outcomes Often Used in Projects Supported by World Bank Group Urban transport Accessibility to transport services by the population including the poor Level of congestion Average travel speed and costs Rural roads Accessibility year round Coverage of the rural population Intercity highways Network’s physical condition Travel time and costs Accessibility Level of road safety Railways Locomotive availability Wagon turnaround time Rail traffic Ports and Berth productivity waterborne transport Cargo dwell times Hinterland connectivity Level of congestion Air transport Level of congestion Air transport safety Source: World Bank project documents Intermediate outcomes for sustaining transport—institutional capability and financial viability—are defined as follows: Institutional capability—whether the responsible institutions have the right structure, autonomy, and technical capabilities needed to efficiently design, plan, construct, operate, and maintain transport infrastructure services. This includes trained labor (public or private sector) and management systems, such as strategic planning. For IFC- and MIGA–supported projects, the term “management capability” is used, referring to the capabilities of private enterprises rather than that of public institutions supported by the World Bank. Financial viability—whether the responsible public institutions (in the case of World Bank projects) and private enterprises (in the case of IFC and MIGA projects) have access to adequate, reliable, and predictable financial resources needed to operate and maintain the infrastructure services for which they are responsible. Measures that would contribute to financial viability include proper demand, traffic, and tariff revenue projections; construction and operating cost estimates; and budgetary allocations and subsidies. In IFC projects, the financial rate of return of the project enterprise has been used. 11 CHAPTER 1 INTRODUCTION The evaluation also assesses the extent to which IFC and MIGA deployed their tools in a manner that complemented each other’s activities as well as those of the World Bank. The evaluation assesses whether there have been synergies between IFC investment and advisory services or between IFC investments and World Bank operations in supporting sustained transport. The evaluation also reviews the extent to which IFC’s Advisory Services were supporting the World Bank’s work in creating an enabling environment, by influencing the regulatory regime for example. Evaluation Building Blocks Multiple analytical building blocks are used to answer the evaluation questions. The evaluation uses a variety of information sources and analyses to identify the key factors that sustain transport. The main building blocks are: A portfolio review of Bank Group operations approved from fiscal 2002-11. The review included: all World Bank–supported projects managed by the Transport Sector and components of operations managed by other sectors with more than a 30 percent share of commitments for transport; IFC investments and advisory services for transport; and MIGA guarantees for transport investments. Project appraisal documents, Board reports, and reviews of implementation/supervision completion and results reports have been used for this analysis. World Bank–supported analytic and advisory activities managed by the Transport Sector with themes focusing on sustained transport have also been identified. The results of the subset of those operations that had closed (for the World Bank) or reached operational maturity (for IFC and MIGA) were also reviewed. Bank Group sector and country strategies. These were reviewed to assess the extent to which the need for sustaining transport was addressed. Field- and desk-based project assessments. Nine field-based country studies covering 42 projects (in Chile, India, Indonesia, Mozambique, Nigeria, Peru, Poland, Russia, and Senegal) assessed whether the transport infrastructure and services supported under Bank Group projects was sustained beyond project completion or operational maturity. In this context, site visits were carried out and interviews were conducted with World Bank Group task teams, government counterparts, implementing agencies, and other sector stakeholders. In addition, 11 countries were selected for in-depth, desk-based reviews, covering 26 projects in Bolivia, Honduras, Mongolia, Papua New Guinea, Sri Lanka, Tanzania, Tunisia, Turkey, Uganda, Yemen, and Zambia. 10 All projects that closed during the evaluation time period (fiscal 2002-11) were 12 CHAPTER 1 INTRODUCTION also reviewed in these desk- and field-based assessments, even if approved prior to fiscal 2002. IEG’s project evaluations. Findings from field-based independent evaluations of 30 transport projects (Project Performance Assessment Reports, or PPARs) carried out by IEG during the past five years were also used. Together, these sources constitute evidence of the extent to which transport investments have been sustained in the longer run. The list of the projects reviewed as part of field- and desk-based assessments and the list of the projects evaluated in PPARs by IEG are in Appendix E. Organization of the Report The report has five chapters. Chapter two answers the first and second evaluation questions on the extent to which issues related to sustaining transport have featured in sector and country strategies, the composition and scope of Bank Group support for sustaining transport, and the measures for sustaining transport supported under Bank Group operations. Chapter three assesses the effectiveness of the measures that help sustain transport primarily supported by the World Bank in different country contexts. The fourth chapter assesses the role of direct support to the private sector by IFC and MIGA in sustaining transport. It also discusses coordination across the three Bank Group institutions in support of sustained transport. Both chapters three and four discuss factors associated with success. The fifth and final chapter summarizes the evaluation findings and provides recommendations for Bank Group management. 13 World Bank Group Support for Sustaining Transport Highlights The need to sustain transport is largely reflected in sector and country strategies, yet only one in seven World Bank transport operations has an objective focused on sustaining transport infrastructure and services, and the share has been declining. The World Bank has supported countries to sustain transport through a variety of interventions and policy dialogue, including specific investment loans, development policy loans, and analytic and advisory activities. One- third of World Bank transport operations flag the risk of inadequate maintenance at appraisal; fewer than half of these incorporate an objective to sustain infrastructure and services and only 28 percent have included financial arrangements in the project components. More than 85 percent of World Bank transport operations support capacity building and sector management measures; only 16 percent support financial arrangements, and that share has been declining. Projects with objectives to sustain transport and that identify maintenance and funding shortfalls as risks are more likely to include measures for sustaining transport. For IFC- and MIGA-supported private sector investments, sustained operation is an implicit objective, as these projects are exposed to commercial risks and assessed for their viability before they are approved. As needed, IFC uses proactive measures aimed at enhancing project design which also helps to sustain individual investments in the transport sector. This chapter documents how sustaining transport has been reflected in sector and country assistance strategies, project objectives, and lending and non-lending activities. The effectiveness of these activities in sustaining transport beyond project closure will be addressed in chapter three. Sustained Transport as Reflected in Sector and Country Strategies Supporting governments to sustain infrastructure and services has been a key feature of the Bank Group’s transport strategies since the mid-1990s 1. The need to sustain transport first became an explicit focus of Bank Group work in the 1996 strategy Sustainable Transport: Priorities for Policy Reform. That strategy defined two principal challenges for the transport sector: completion of basic infrastructure networks and provision of adequate maintenance for those networks. The strategy 15 CHAPTER 2 WORLD BANK GROUP SUPPORT FOR SUSTAINING TRANSPORT identified four pillars for action—social, financial, economic, and environmental sustainability—and acknowledged the likelihood of trade-offs (for example, between safety and costs, financial returns and user fees, or vehicle standards and air quality). The Bank Group was to focus on institutional and policy reform to help governments fulfill their enabling and supervisory role in a freer transport market. This was to be done through more selective and focused technical assistance for building the capacity and skills needed by the public sector, while the private sector would be expected to take on more of the responsibility for providing, operating, and financing transport services and infrastructure. This strategy was in effect until it was updated by Safe, Clean, and Affordable Transport for Development (2008-2012), which commits the Bank Group to help partner countries establish the governance, strategies, policies, and services that will deliver transport for development in a way that is economically, financially, environmentally, and socially sustainable. As noted in the preceding chapter, this evaluation focuses on the institutional and financial aspects. Most Country Assistance Strategies (CASs) 2 emphasized the need to address institutional and sector capacity issues, focusing primarily on sustained road transport. Two hundred seventeen CASs were prepared for 108 3 countries from fiscal 2002-12. Institutional and sector capacity issues in transport were highlighted in 75 percent and 71 percent of strategy documents, respectively. However, regulatory and financial issues for the sector were discussed in only 36 percent and 26 percent of the CASs, respectively (Table 2.1). There was little variation in the attention given to institutional, regulatory, and policy issues in the CASs across the country income groups. However, the focus on sector capacity and financial aspects was higher in the CASs for lower-income countries compared to those in upper- middle-income countries Table 0.1. Country Assistance/Partnership Strategies: The Share that Highlighted Measures to Sustain Transport By Country Income (Percent) Income group Institutional Regulatory/policy Sector capacity Financial Upper middle income countries 78 37 69 18 Lower middle income countries 76 36 67 32 Low income countries 74 34 81 32 All (n=217) 75 36 71 26 Source: World Bank CASs Forty-five percent of CASs included plans for private sector participation (PSP) or public-private partnerships (PPP). The focus on maintenance and PSP/PPP was primarily for roads and to some extent air transport. In contrast, the CASs rarely 16 CHAPTER 2 WORLD BANK GROUP SUPPORT FOR SUSTAINING TRANSPORT mentioned maintenance or issues related to sustained transport in railways, ports, or waterborne transport. The overall emphasis on measures for sustaining transport (institutional capacity, sectoral capacity, regulatory capacity and financial mechanisms) has remained at about the same level over the past 10 years. There was a slight increase in attention to regulatory/policy and sector capacity measures, but no significant change in institutional or financial measures. Sustained Transport in World Bank Operations THE RISK OF MAINTENANCE AND FUNDING SHORTFALLS AT APPRAISAL Nearly a third of World Bank supported transport projects approved from fiscal 2002-11 identified maintenance and funding shortfalls as risks at appraisal. All World Bank investment projects are required to undertake a risk analysis at appraisal to characterize the level of risk and to point to how the risks will be mitigated. Among the 287 transport projects approved from fiscal 2002-11 and managed by the transport sector, 31 percent identified maintenance or funding shortfalls as a risk. Forty-five percent of these cases were classified as moderate risk, 36 percent as substantial, and 13 percent as high. 4 Risks for maintenance and funding shortfalls were most likely to be cited for projects in low- and lower- middle income countries and in the intercity highways and rural roads subsectors (Figure 2.1). This is in line with the CAS analysis which also pointed out that maintenance funding issues were raised more often in low- and lower-middle- income countries than in upper-middle- or high-income countries. Projects that have identified maintenance and funding shortfalls as key risks have declined from 39 percent during fiscal 2002-06 to 24 percent during fiscal 2007-12. 5 17 CHAPTER 2 WORLD BANK GROUP SUPPORT FOR SUSTAINING TRANSPORT Figure 0.1. Percentage of World Bank Operations that Identified Maintenance and Related Funding as Risks at Appraisal, by Country Income and Subsector (n=number of projects) Percentage of Projects 50% 47% 39% 38% 36% 40% 29% 28% 28% 30% 23% 23% 20% 10% 0% Low Lower Upper Rural Intercity Railways Ports. Urban Aviation income middle middle Roads Highways (n=31) IWW, Transport (n=22) (n=54) income income (n=62) (n=165) TTF (n=58) (n=98) (n=111) (n=36) Source: IEG portfolio review. The total number of projects in this figure is larger than the portfolio size because some projects contained components for multiple subsectors. IWW= Inland Waterways. TTF=Trade and Transport Facilitation. SENSITIVITY OF EX ANTE ECONOMIC ANALYSIS TO MAINTENANCE SHORTFALLS Based on a review of ex ante economic analysis of World Bank highway projects, few if any are assessing the sensitivity of returns on the investment to assumptions on whether the assets are maintained. Bank projects are supposed to conduct ex ante economic analysis of project costs and benefits as a basis for investment decisions. IEG examined the ex ante and ex post economic analysis section of all 51 completed intercity highway projects approved since fiscal 2002. All of these projects used the Highway Development and Management Tools model, 6routinely calculating the financial needs for maintenance and systematically including sensitivity analyses on the basis of variability in demand levels and/or an increase in the cost of maintenance, in addition to the possibility of increased road work estimates. Economic rates of return calculated at project appraisal and at completion have generally shown strong results for highway projects—almost all with rates over 12 percent, and 18 to 31 percent showing rates of more than 50 percent. This is because the model forecasts that benefits will rise automatically with increases in traffic density. 18 CHAPTER 2 WORLD BANK GROUP SUPPORT FOR SUSTAINING TRANSPORT None of the reviewed operations prepared an ex post or ex ante sensitivity analysis of project benefits to variations in the availability of funding for maintenance. Sensitivity analysis is normally carried out at appraisal for most projects based on a 20 percent fluctuation of benefits (traffic density) and costs (material and labor costs), but this analysis has been repeated in only 12 out of 51 projects at completion. An implicit assumption in the calculation at project completion is that condition-based maintenance will be carried out throughout the life-span of the asset, so that the benefit flow is maintained. In environments where there is a history of maintenance underfunding, the benefit/cost analysis is likely overly optimistic and the useful life of the asset and the present value of the net benefits are likely to be shorter and smaller. The review also found that the preparation of roads projects is not underpinned by a systematic assessment of the entire network or subnetwork managed by the road agency, to seek rational allocation of resources and cost effective rehabilitation and maintenance solutions. The review found little evidence to indicate that projects and the related investments were being designed to optimize the limited maintenance funds for the whole road network, even in heavily budget- constrained situations. In a few countries, including Poland, Indonesia, and Sub- Saharan African countries, such assessments have been carried out by the World Bank using a road network evaluation tool 7, but the road agencies in these countries have yet to use the model to deploy their resources in an optimal and cost-effective manner across their road networks. SUSTAINED TRANSPORT IN PROJECT OBJECTIVES While all transport investments are expected to provide sustained services, this seldom appears as an objective in World Bank transport operations. Projects’ formal objectives are important, as they signal the results for which the projects will be held accountable. Only 15 percent of the 287 World Bank transport projects approved from fiscal 2002-11 and managed by the Transport Sector had objectives that focused on improving operations and maintenance or funding for maintenance with a view to sustain transport. 8 Projects in low-income countries had the highest share of objectives to sustain transport (19 percent), followed by lower-middle-income (17 percent) and upper- middle-income countries (13 percent). This result is in line with the larger focus on financial aspects of maintenance in low- and lower-middle income countries, as evidenced in the CAS analysis (Figure 2.2). 19 CHAPTER 2 WORLD BANK GROUP SUPPORT FOR SUSTAINING TRANSPORT Figure 0.2. Share of Projects with Objectives to Sustain Transport, By Country Income Category and Transport Subsector (n=number of projects) Percentage of Projects 20% 19% 19% 17% 17% 16% 15% 13% 13% 9% 10% 5% 3% 0% Low Lower Upper Intercity Urban Railways Rural Aviation Ports income middle middle Highways Transport (n=31) Roads (n=22) (n=36) (n=54) income income (n=165) (n=58) (n=62) (n=98) (n=111) Note: There are projects in Poland, a high-income country with sustained transport in the objective. Source: IEG portfolio analysis of projects managed by the transport sector, approved during fiscal 2002-11. Highway projects were the most likely to focus on sustaining transport in their project objectives, yet no more than one in five projects in any subsector had objectives to improve sustained transport, for which they were accountable. Intercity highways had the largest share of projects (19 percent) that focus on sustaining transport in their objectives, followed by urban transport (17 percent), railways (16 percent), and rural roads (13 percent) (Figure 2.2). About 80 percent of projects that focused on sustaining transport in their objectives were in intercity highways, followed by urban transport, rural roads, and railways. Only one or two projects each in ports and airports had objectives that referred to sustaining transport. The share of projects managed by the Transport Sector with objectives focusing on sustained transport has declined over the decade by about half (Figure 0.3). Whereas 22 percent of transport projects in fiscal 2002-06 had objectives to sustain transport, this had fallen to 10 percent in fiscal 2007-11, a statistically significant result. The downward trend persists even when looking within each country income group and within the main subsectors — urban transport, intercity highways, and rural roads (Table 2.2). 9 The declining share of objectives to sustain transport does not appear to be due to changes in the country or subsector composition of the portfolio. 20 CHAPTER 2 WORLD BANK GROUP SUPPORT FOR SUSTAINING TRANSPORT Figure 0.3. Percentage of Projects That Included Sustained Transport in Their Objectives 35% 30% 25% 20% 15% 10% 5% 0% 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 Note: Projects managed by the transport sector only. Only 4 percent of transport projects managed by other sectors had reference to sustained transport in their objectives. Source: IEG portfolio review. Table 0.2. Declining Share of Projects with Sustained Transport Objectives, By Country Income and Subsector Category FY02-06 FY07-11 Country Income Low income (n=54) 22 15 Lower middle income (n=98) 23 14 Upper middle income (n=111) 17 10 Transport Subsector Highways (n=165)* 25 14 Urban transport (n=58) 27 11 Rural roads (n=62) 14 11 All Projects (n=287)* 22 10 Note: *The difference in share between the two time periods is statistically significant at p<=.05. Source: IEG portfolio review. MEASURES TO SUSTAIN TRANSPORT More than 85 percent of World Bank transport projects finance sector management and capacity building measures, but very few projects support financial arrangements. In working with governments to support their sector reform initiatives and create an enabling environment for sustaining transport, the 21 CHAPTER 2 WORLD BANK GROUP SUPPORT FOR SUSTAINING TRANSPORT Bank assists countries in setting up sector policies and regulatory frameworks, institutional frameworks, sector management capacity building, and financial arrangements. 10 Eighty-six percent of transport projects managed by the Transport Sector provide financing for sector management and capacity (Table 2.3 column 1). This is followed by the establishment and strengthening of the institutional framework (51 percent) and the sector policy and regulatory framework (25 percent). Direct support to establish financial arrangements occurs least frequently (only 16 percent). As described in the next chapter, financial arrangements have often been found to be embedded in institutional frameworks, especially in the context of public-private partnerships, which typically assume adoption of user charges. Table 0.3. Percentage of World Bank Transport Projects Supporting Measures to Help Sustain Transport, By Sector Mapping Projects managed by other All transport projects managed Sectors with > 30% transporta Sustainability-enhancing measures by transport sector (n=287) (n=150) Sector management and capacity 86 17 building Institutional framework (includes PSP) 51 12 Sector policy and regulatory framework 25 6 Financial arrangements 16 3 a.This group includes DPOs with some transport content, managed by other Sectors. Source: IEG portfolio review. Operations managed by the Transport Sector are far more likely than transport projects managed by other sectors to have measures aiming to sustain transport. Of all projects in the lending portfolio with any transport commitments, 36 percent in number and 11 percent by lending volume are managed by sectors other than Transport. More than 80 percent of these projects are managed by five sectors: Urban Development (37 percent); Economic Policy (19 percent); Agriculture and Rural Development (13 percent); and Environment (9 percent), Finance, and Private Sector Development (6 percent). These projects were much less likely to support any of the measures for sustaining transport defined in this evaluation (Table 2.3, column 2). An important subset of operations managed by other sectors is Development Policy Operations (DPOs). There were 57 DPOs that have transport content, approved between fiscal 2002 and 2011 to support the achievement of a set of development results through a medium-term program of policy and institutional actions. Of these, 61 percent were managed by the Poverty Reduction and Economic Management Network, 30 percent were managed by the Sustainable 22 CHAPTER 2 WORLD BANK GROUP SUPPORT FOR SUSTAINING TRANSPORT Development Network 11, and 9 percent were managed by the Finance and Private Sector Development Network. All of these DPOs have included measures for sustaining transport. The measures that figured more prominently than in investment operations were related to sector policy and regulatory frameworks (32 percent) and financial arrangements (49 percent). The share of projects supporting measures involving financial arrangements has declined by half over the decade. Measures for financial arrangements were explicitly included in 24 percent of the transport sector projects approved during fiscal 2002–06, but in only 11 percent of the projects approved during fiscal 2007–11 (Table 2.4). Support for institutional frameworks has also declined somewhat, from 59 percent to 45 percent of projects. Not surprisingly, projects that focused on sustained transport in their development objectives are more likely to include measures for sustaining transport than are projects that did not focus on sustained transport in their objectives. Low-income countries have been receiving the highest share of support for the majority of the measures to sustain transport, particularly sector policy, regulatory frameworks, and institutional frameworks. Comparing the two time periods, fiscal 2002-06 and fiscal 2007-11, the share of projects supporting measures involving financial arrangements had the largest decline (71 percent), followed by institutional framework measures (38 percent), and sector policy and regulatory framework (20 percent). Table 0.4. Trend of the Share of Transport Sector Projects with Measures to Sustain Transport By Fiscal Year of Approval and Project Objectives (percent) Projects with Projects without sustained sustained transport Measures to sustain FY02–06 FY07–11 transport objectives transport (n=117) (n=170) objectives (n=44) (n=243 ) Sector management and 95* 84* capacity building 91* 82* Institutional framework 70** 47** (includes PSP) 59* 45* Sector policy and regulatory 34 23 framework 27 24 Financial arrangements 24** 11** 32** 13** Source: IEG portfolio review. Note: Difference across groups is statistically significant at: **p<=.01; *p<=.05. Projects that identified maintenance shortfall risks were more likely to have sustained transport objectives and to include measures to sustain transport in 23 CHAPTER 2 WORLD BANK GROUP SUPPORT FOR SUSTAINING TRANSPORT their components. Among the projects that identified maintenance and funding shortfalls as risks, about 22 percent have sustained transport objectives compared with 12.5 percent of the projects that did not identify maintenance risks. About 30 percent of projects that had identified maintenance and funding shortfall risks included financial arrangements as a measure to help sustain transport in their project components, while the average was 16 percent for the portfolio. Sustained Transport in Analytic and Advisory Activities From fiscal 2002–11, the World Bank supported 322 analytic and advisory activities managed by the Transport Sector and 828 activities related to transport, but managed by other sectors. Over half of Bank-supported analytic and advisory activities on transport are public expenditure reviews (PERs), investment climate analyses, or upstream work on or in support of establishing public-private partnerships, all of which address factors that enhance sustainable financing. Of the activities managed by the Transport Sector, strategies account for 22 percent, the largest share among the analytic and advisory activities managed by the Sector. Of those, about 30 percent refer to activities and measures to help sustain transport. PERs were the most common analytic and advisory activity managed by other sectors, in particular, by the Poverty Reduction and Economic Management Sector. Those activities comprised about 20 percent of World Bank analytic and advisory activities on transport. Other common themes included transport safety and public-private partnerships, but detailed information was often not available in the files. Therefore it was not possible to categorize the analytic and advisory activities by their measures of focus for sustaining transport. In terms of subsectors, activities related to intercity highways and the general sector were the most common, followed by rural roads. About an equal number of activities supported the urban transport, railways, ports, and aviation subsectors. Out of the 140 PERs that made some reference to the transport sector, fewer than half made explicit reference to transport operations and maintenance funding needs. The scarce documentation in the World Bank’s internal tracking system constrained the review. 12 As a result, it was difficult to find clear patterns or evidence that link analytic work with better implementation of measures to help sustain transport supported by projects. 24 CHAPTER 2 WORLD BANK GROUP SUPPORT FOR SUSTAINING TRANSPORT Sustained Transport in IFC Investments, IFC Advisory Services, and the MIGA Portfolio IFC and MIGA support private infrastructure and service providers in the transport sector through investments, guarantees, and transaction-oriented advisory services. IFC investments in the transport sector typically involve a loan— and to a lesser extent equity—to a concession or a private operator. In general, IFC investment services do not work with governments to support their sector reform initiatives or create an enabling environment, they support projects where due diligence indicates that an adequate enabling environment is already in place. In the event that IFC’s due diligence detects a need to enhance project design or structure, they set corrective measures that are likely to enhance the long-term operation of the investment, and hence its sustainability. IFC advisory services mainly support governments in developing public-private partnership transactions. MIGA provides guarantees that are usually made against specific political risks for an equity investment or non-shareholder loan. Through these activities, IFC and MIGA support enterprises whose maintenance and operational adequacy is either contractually required or an integral part of the business model. Once passed due diligence, it is implied that IFC- and MIGA-supported companies are likely to have the management capacity to operate and maintain transport infrastructure and services, and to be financially viable. Therefore, for IFC- and MIGA-supported projects, sustained operations of the investment is always an implicit objective. The World Bank’s Public-Private Infrastructure Advisory Facility (PPIAF) lays the foundations for IFC’s and MIGA’s efforts by working upstream with countries in making an informed choice between private and public sector implementation and creating an enabling environment for private engagement. During fiscal 2002-11, PPIAF implemented 138 advisory activities in transport, ranging from assisting countries in preparing sectoral strategies and reform road maps, assessing the options for public and/or private engagement, and establishing institutional and regulatory frameworks to facilitating specific pioneering transactions. With almost two-thirds of activities (64 percent) in IDA/blend countries, PPIAF engages particularly in countries where the lack of an enabling environment has traditionally deterred private sector participation. A comprehensive assessment of linkages between PPIAF’s and IFC’s activities is not yet possible, as the former has only recently started to track these. Still, this study has identified several other cases in which the World Bank and IFC collaborated upstream, with IFC providing inputs to the policy dialogue. IFC Advisory Services alone, however, had little focus on creating an enabling environment for private sector participation. 13, 14 25 CHAPTER 2 WORLD BANK GROUP SUPPORT FOR SUSTAINING TRANSPORT IFC’s Advisory Services have supported governments mainly through structuring public-private partnership transactions and to a far less extent by creating an enabling environment for sector reforms. Over the decade covered by this evaluation, IFC carried out 54 Advisory Services operations in the transport sector: 46 supported the formation of concessions mainly in the air transport (35 percent), highways (22 percent), and ports (19 percent) subsectors. The supported concessions involved outsourcing operations and maintenance of transport infrastructure and services to the private sector. Apart from transaction-oriented advisory assignments, IFC also undertakes studies, workshops, and institution- building with the aim to sustain transport in 17 percent of all activities. 15 Summary The World Bank has supported countries to sustain transport through a variety of interventions and policy dialogue including specific investment loans, development policy loans, and analytic and advisory services. Sustained transport as defined in the context of this evaluation features prominently as a concern in Bank Group strategies and analytic and advisory activities, but this concern is not borne out by project-level objectives and interventions. Only 15 percent of all projects managed by the Transport Sector focus on sustained transport in their objectives and that share has declined over the past decade. This does not appear to be due to changes in country or subsector composition of the portfolio. The low and declining share of projects with reference to sustained transport in their objectives is a source of concern because, as described in the next chapter, projects with specific references to sustained transport in their objectives are more likely to have sustained transport outcomes than projects without those references in their objectives. Nearly all World Bank transport operations provide financing for sector management and capacity to sustain transport, but only 16 percent support financial arrangements and even that share has been declining. One-third of approved projects identified maintenance and funding shortfalls as risks at the appraisal stage. While it is difficult to tell ex ante whether this is an under- or over-statement of the risk, among those reporting the risk, one-third included financial management measures in project components. Based on the experience of intercity highways projects, ex ante ex post economic analyses are not factoring in the sensitivity of the cost and benefit streams to shortfalls in operation and maintenance. These highways projects are also not systematically evaluating 26 CHAPTER 2 WORLD BANK GROUP SUPPORT FOR SUSTAINING TRANSPORT the entire network or subnetwork managed by the road agency to seek cost- effective rehabilitation and maintenance solutions. Projects that refer to sustained transport in their objectives and identify maintenance and funding shortfalls as risks are likely to include measures to improve financial viability or institutional capability, but since the shares of such projects have been declining over the past 10 years, concern emerges that the ongoing and future operations may not have as large an impact on sustained transport in the longer run. Private sector transport investments supported by IFC and MIGA are exposed to commercial risks and assessed for their viability before approval. Whenever needed, IFC uses proactive measures aimed at enhancing project design and structure to help sustain individual investments in the transport sector. These measures are discussed in chapter four. 27 The Effectiveness of World Bank Support for Sustaining Transport Highlights Sustained transport varies by country income and subsector. Transport was more likely to be sustained in upper-middle- and high-income countries, and in intercity highways and ports. The outcomes of projects managed by the Transport Sector were more likely to be sustained than were the outcomes of projects managed by other sectors. The sustained transport outcome is linked to the achievement of two intermediate outcomes: financial viability and institutional capability. Projects with a sustained transport objective and projects that identify maintenance risks were more likely to implement measures to sustain transport, and they were more likely to achieve the intermediate outcomes that led to sustained transport outcomes. A clear accountability framework and incentive structure contributed to sustained transport outcomes in landlord port-operator models and toll roads, for which user pay principles were adopted, and in other highway projects, for which output- and performance-based maintenance contracts were used. A broad-based approach that aligns funding sources more coherently, attention to political economy, multi-model and strategic planning, diversification of funds, and continuity and sequencing of engagement are additional factors that have contributed to sustained transport outcomes. This chapter discusses the key factors that have contributed to sustained transport and the extent to which World Bank support has been effective in helping governments to implement measures contributing to sustained transport over the longer term. The World Bank prepares implementation completion and results reports informing on project results at completion; it does not systematically monitor sustained transport outcomes beyond project closure. Evidence for the analysis has therefore been triangulated from two sources that measured some degree of change in sustained transport outcomes beyond the closure of World Bank projects. The outcome — sustained transport — was measured by assessing whether transport infrastructure and services was sustained one to 11 years after projects had closed. First, field-based assessments were carried out for 42 projects in nine 29 CHAPTER 3 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR SUSTAINING TRANSPORT countries and desk-based assessments for 26 projects in 11 countries, for a total of 68 projects in 20 countries. 1 The 20 countries were selected at random, while maintaining regional representation, from among those with a population greater than one million and with at least two World Bank Group operations that were either approved, closed (World Bank projects), or operationally matured (IFC/MIGA projects) between fiscal 2002-11 (Appendix C). 2 Three were low- income countries (15 percent), 10 were lower-middle-income countries (50 percent), and seven (35 percent) were upper-middle-income countries. Fifty-five of the 68 projects were managed by the Transport Sector. The remaining 13 were managed by other sectors. The time elapsed between project completion and field visits ranged between one and 11 years. Second, 30 field-based PPARs prepared by IEG during the past five years were reviewed. They were in 19 countries and took place two to seven years after the projects closed. An equal share (30 percent) were in upper-middle-income, lower- middle-income, and low-income countries, and three (10 percent) were in high- income countries. Twenty projects (20 percent) were in intercity highways, eight (27 percent) were in urban transport, and the remaining two (3 percent) were in the ports subsector. Eight of the 30 PPAR projects were also covered in the field- and desk-based country reviews. Five out of the 30 projects were managed by the Urban Development Sector; the remainder was managed by the Transport Sector. Factors that improved or detracted from sustained transport were assessed for projects and analytic and advisory activities from a sample of 20 country case studies and an additional 13 PPARs. The extent to which transport has been sustained in these countries is not representative of the Bank’s transport engagement in all countries. However, the evaluation uses variability in outcomes across the case study countries to distill evidence on the key factors affecting sustained transport in different contexts and transport subsectors. While the unit of analysis is the project, most projects supporting intercity highways addressed sustained transport outcomes at the sector level. This is because the support that the World Bank has provided to intercity highways has focused on creating an enabling policy, financial, and institutional environment for the sector to manage and maintain entire network of roads. Large, decentralized countries like India, in which the World Bank has supported the highway sector at the state level, are included in this category. Support for railway and air transport projects has also often been at the sector level, especially in smaller countries. World Bank support in urban transport, rural roads, and ports has typically been at a project level, either in a selected city, rural area, or a specific port unless there are 30 CHAPTER 3 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR SUSTAINING TRANSPORT policy and reform elements that are applicable to the rest of the country embedded in technical assistance. Assessment of Sustained Transport Outcomes and its Variability Sustained transport varies by country income and subsector 3. The flow of benefits from World Bank-supported transport operations beyond project completion was assessed by collecting results on the provision of transport services at the time of the field- and desk-based assessments some years after project completion and by comparing them with the results achieved at project completion. Those projects that were assessed as having sustained transport either sustained the results achieved at project completion or exceeded them over time (Box 3.1). By that criterion, about 57 percent of the 68 projects reviewed for this assessment sustained the benefits flowing from infrastructure investments and related services some years beyond project completion. For example, in Peru, the 16,000 km of rural roads supported by the World Bank offer continuous access and good quality roads, according to three impact evaluations. The average time traveled was reduced by 30 percent, further reducing the travel costs. Traffic increased by 132 percent and the cost of public transport was reduced by 77 percent nine years after the program began (Poverty and Economic Policy Research Network, 2010). 31 CHAPTER 3 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR SUSTAINING TRANSPORT Box 0.1. Measuring Sustained Transport in Country Studies The assessment of sustained transport outcomes was carried out through field- and desk- based reviews of the evidence covering 68 World Bank projects in 20 countries. General country and project documents and analytic and advisory activities prepared prior to, during, and after project implementation were reviewed. Recent information on the projects’ institutional and financial performance and key performance indicators were collected. For the field-based studies, government officials, private sector, and/or other sector stakeholders were interviewed and site visits were conducted for selected projects. For desk-based project reviews, information was supplemented by interviews with the project teams. For each case study , the degree to which transport had been sustained beyond project closure was assessed on a four-point scale according to whether: (i) the key outcomes achieved by the end of the project improved beyond project closure; (ii) the key outcomes achieved or nearly achieved by the end of the project were sustained; (iii) the key outcomes achieved by the end of the project were not sustained; or (iv) the key outcomes were not achieved by the end of the project (and have not shown improvements beyond project closure). Where there was no information available to make the above assessment, the rating was “not evaluable” and these projects were excluded from the analysis. Institutional capability and financial viability were assessed similarly, based on the sustained intermediate outcome definition in chapter one. The methodology and framework for the intermediate outcome and outcome assessments are explained in Appendix C. The evaluation also recognizes that although the projects often addressed sectorwide outcomes, not all of these outcomes are necessarily attributable to specific World Bank-supported measures. In the group of projects assessed, transport was more likely to be sustained in upper-middle- and high-income countries and for intercity highways and ports (Figure 3.1). While transport was sustained in two-thirds of intercity highway projects, this was not the case for any of the 15 railway projects. For fewer than half of projects in urban transport, aviation, and rural roads, transport was not sustained in the years following project closure. Upper–middle-income and high- income countries showed the best outcomes, compared with low-income and lower–middle-income countries. In the group of projects reviewed, transport was more likely to be sustained for projects managed by the Transport Sector (64 percent) than those managed by other sectors (23 percent, not shown). 32 CHAPTER 3 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR SUSTAINING TRANSPORT Figure 0.1. Share of Projects with Sustained Transport Outcomes, Based on Field and Desk- Based Assessments of Project Results (n=number of projects, c=number of countries) 100% 78% 80% 67% 60% 57% 57% 52% 46% 40% 33% 33% 20% 0% 0% Low and Upper Intercity Ports Urban Aviation Rural Railways All Lower middle highways (n=7; c=7) Transport (n=6; c=5) Roads (n=15; Projects middle and High (n=36; (n=13; (n=15; c=11) (n=68; Income Income c=14) c=11) c=13) c=20) (n=54; (n=14; c=15) c=5) Source: IEG field- and desk-based reviews of transport projects in 20 countries. Transport projects assessed by IEG in the field over the past five years show similar results. For projects that were the subject of a PPAR, sustained transport was assessed based on the change in the risk to development outcome rating between project closure and the time of the PPAR mission (Box 3.2). Even though such a risk rating is on the outcomes of project objectives rather than on sustaining transport per se, it is a useful proxy which can corroborate the assessment based on field- and desk-based reviews. 33 CHAPTER 3 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR SUSTAINING TRANSPORT Box 0.2. Risk to Development Outcome Definition and Criteria The risk to development outcome is assessed for all closed World Bank projects. It is defined as “the risk, at the time of evaluation, that development outcomes (or expected outcomes) will not be maintained (or realized)” The rating has two dimensions: (i) the likelihood that some changes may occur that are detrimental to the ultimate achievement of the operation’s development outcome; and (ii) the impact on the operation’s development outcomes if some or all of these changes materialized. The criteria used for assessing this rating are technical, financial, economic, social, political, environmental, government ownership/commitment, other stakeholder ownership, governance, and natural disaster exposure. Source: World Bank, “IEG/OPCS Harmonized Evaluation Criteria,” 2006 About two-thirds of projects across all subsectors were able to either retain negligible to low risk-to-development outcome ratings or to reduce risks over time (Figure 3.2). However, in one-third of the cases, the risk increased relative to the IEG rating assigned at project closure. Seventy percent of projects that were assessed to have increased risks were in low-income countries. All of these were intercity highway projects. The reasons for increased risk mainly related to lack of adequate funding for road maintenance or weak institutional capacity of the road agencies. Figure 0.2. Distribution of Projects with Changes in the Risk-to-Development Outcome Rating between Project Closure and IEG’s Field Evaluation (PPAR) Decreased risk at PPAR, 2, 7% Increased Risk at PPAR, 10, Unchanged Risk 33% Rating, 18, 60% Source: IEG analysis. Determinants of Sustained Transport Outcomes Institutional capability and financial viability are key factors affecting sustained transport. Projects that achieved institutional capability and financial viability—the 34 CHAPTER 3 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR SUSTAINING TRANSPORT intermediate outcomes -- were substantially more likely to achieve sustained transport. Seventy percent of those that achieved financial viability also achieved sustained transport, while only 46 percent of projects that did not achieve financial viability had sustained transport outcomes 4. Likewise, while 79 percent of projects that achieved institutional capability also achieved sustained transport, this was true for only 39 percent of projects that did not substantially achieve institutional capability 5. Box 3.3 summarizes the findings from a review of lessons learned from the 30 PPARs and 68 case study projects. Box 0.3. Factors Associated with Less Sustained Transport in Case Study and PPAR Projects Based on results from changes in the risk to project development outcome (for PPARs) and sustained transport outcomes (for the case studies), projects were classified according to whether transport infrastructure and services were either more or less sustained. The prevalence of specific shortcomings was assessed for both groups. Deficiencies in financial viability and institutional capabilities were far more common in projects with less sustained transport outcomes than in projects with more sustained outcomes. Share of Projects with Deficiencies in Institutional Capability and Financial Viability, According to Whether Transport Outcomes were More or Less Sustained (percent) Percentage of Projects 80% 70% 68% 60% 40% 21% 23% 20% 0% Institutional capability Financial Viability Less Sustained (n=56) More Sustained (n=57) Note: Analysis based on 68 case study projects (with 90 observations to account for multiple subsectors in a project) and 30 PPARs. The case study projects were assessed according to Box 3.1. When the project included more than one subsector, each was separately assessed. The PPARs were assessed as more or less sustained based on whether the risk to development outcome rating improved or deteriorated between project closing and the time of the PPAR field visit. (Eight of the 30 projects with PPARs were also case study projects and were assessed using the case study method.) Source: PPARs, IEG field and desk-based reviews. Projects that implemented measures to sustain transport were more likely to achieve the intermediate outcomes that lead to sustained transport. Results of desk- and field-based project assessments show that projects that planned and 35 CHAPTER 3 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR SUSTAINING TRANSPORT subsequently implemented any of the four types of measures to help sustain transport, discussed in Box 1.3 (sector policies and regulatory framework, institutional framework, sector management and capacity building, and financial arrangements) increased the likelihood of achieving intermediate outcomes (institutional capability and financial viability), compared to projects that had planned but not implemented them by the end of the project (Table 3.1) 6. Table 0.1. Projects that Implemented Measures to Help Sustain Transport were more likely to Achieve Intermediate Outcomes (Institutional Capability and Financial Viability) Percent of projects that achieved Percent of Percent of projects projects that Institutional Capability Financial Viability that implemented planned a measure to among implement those that If imple- If did not If imple- If did not Planned a measure planned mented implement mented implement Measures (n=68) them measure measure measure measure Sector policies and regulatory 41 71 53 25 40 50 framework Financial 29 55 33† 11** arrangements 73† 73** Institutional 59** 43* 66 73 0** 8* Framework Sector management 49 50* 75 90 25 0* and capacity building Note: Difference between outcomes for projects that did and did not fully implement a measure is: ** significant at p<=0.01; * significant at p<=0.05; † significant at p<=0.10. "Done" denotes fully implemented at least one measure; "Not Done" denotes either not implemented at all or only partly implemented at least one measure. Source: IEG analysis based on field and desk-based assessments of 68 projects in 20 countries. Projects with sustained transport objectives and projects that identified maintenance risks were more likely to implement measures to achieve sustained transport outcomes and more likely to have sustained transport outcomes than those without such objectives. If issues related to sustaining transport were the focus in project objectives, the project was likely to have measures to help sustain transport in project components. Projects that identified maintenance risks during project preparation were also more likely to include measures such as financial arrangements. Among the projects that had an objective on sustained transport, 36 CHAPTER 3 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR SUSTAINING TRANSPORT three-quarters achieved it, compared to half of the operations without an explicit objective. Similar positive results were found with projects that had identified maintenance risks during preparation, compared to the average project. The remaining part of the chapter will elaborate on the determinants for financial viability and institutional capability, which include implementation of (i) sector policies and regulatory frameworks; (ii) financial arrangements; (iii) institutional frameworks; and (iv) sector management and capacity building. SECTOR POLICY AND REGULATORY FRAMEWORK The World Bank has supported countries in defining and implementing appropriate sector policies, strategies, and legal and regulatory frameworks in the transport sector that are intended to create an enabling environment for enhanced sustained transport outcomes through a variety of lending and non-lending operations. About one-quarter of projects in the portfolio have included measures to develop or strengthen these sector policies and regulatory frameworks. Specific measures included the preparation of bills to enable private sector participation or the establishment of road funds in intercity highways and other laws and regulations to enable port operator models and concessions in railways and air transport. The review found that in countries where DPOs and PERs were complementing specific investment loans, financial viability and institutional capability were more likely to be achieved. It was hard to find clear linkages between the implementation of sector policy and regulatory framework measures in investment operations with intermediate or final outcomes. However, many of these policy elements were included in DPOs for policy reforms and they were prior actions related to allocation of maintenance funding, establishment of a regulatory framework, or introduction of public-private partnerships and user charges. DPOs and analytic and advisory activities, such as PERs, appear to have complemented specific investment operations in facilitating reform processes in the countries reviewed. FINANCIAL ARRANGEMENTS The World Bank has supported various types of financing mechanisms to raise operation and maintenance costs, for example, direct user charges, earmarked funds, and central budget allocations. Such mechanisms vary across subsectors and transport modes (Table 3.2). The countries covered under this review typically financed maintenance of intercity highways and rural roads through central or local budget transfers and earmarked road funds with the exception of toll roads, which used direct user charges. In the case of urban transport, earmarked maintenance 37 CHAPTER 3 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR SUSTAINING TRANSPORT funds in the form of dedicated, ring-fenced funds and user charges for public transportation have often been used. For railways, ports, and airports, user charges have been the norm, while central budget subsidies have filled financing gaps. User charges are often associated with public-private partnerships, especially in toll roads, ports, urban transport services, and railways. The measures have often been used simultaneously in intercity highways and urban transport. Table 0.2. Implementation Status of Financial Arrangements at Project Closure Number of projects Percent of projects Financing that planned to use the that implemented the mechanism Subsector financing mechanism mechanism User Charges Toll roads, urban transport, 10 30 railways, ports Earmarked funds Intercity highways, urban 19 47 (e.g., road funds) transport Budget allocations Intercity highways, other 16 19 modes to fill deficits Note: The categories are not mutually exclusive. Source: IEG portfolio review. Adopting incentives to contribute financially can help ensure sufficient O&M funding. The user-pays principle—supported in landlord-port operator models 7 (such as in Liberia), toll roads (such as in Senegal), and certain segments of urban transport — such as bus rapid transit systems (Bogota, Colombia), and other public transport services (Lagos, Nigeria)—have been effective in enhancing cost recovery. When user charges are collected by the agency responsible for operations and maintenance, the agency has the authority and the incentive to use these revenues primarily for operations and maintenance according to the contractual agreement with the infrastructure asset holder. Earmarking road funds can be effective in applying the user-pay principle but are not a panacea. Successful ones need to ensure reliability of the funding source. While empirical studies have shown that the use of road funds 8 has improved efficiency and the effective use of public resources resulting in improved conditions of road networks through improved maintenance (World Bank 1994, 1997, 1998, 2002, 2006, 2010), the review showed mixed results. The majority of projects that involved road funds supported the creation or strengthening of “second generation” road funds 9 that are financed by fuel levies and managed by boards representing the interests of road users. Twenty-eight projects, about one-third of all completed World Bank projects supporting intercity highways in the period under review, addressed some aspect of existing or new road funds. Actions related 38 CHAPTER 3 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR SUSTAINING TRANSPORT to creating or strengthening road funds were implemented only 47 percent of the time due largely to political resistance to set them up. Once road funds are formally established, the key factor that affected their performance was the government’s efforts to ensure that sufficient funds are raised in a timely manner through appropriately set fuel surcharges and other charges that can include vehicle registration fees and tolls. Box 3.4 provides some examples. In some countries, the vehicle population is too small (Papua New Guinea) or the distances too vast and population densities too low (Mongolia) to provide the needed resources through a fuel levy and other charges. In these cases, public budgetary allocations are necessary. The next section will focus on what has worked to ensure these fund allocations are timely and adequate. Obtaining a legal commitment to future public budgetary allocations for funding O&M, however, does not necessarily lead to adequate funding. The Bank has attempted to improve the adequacy and reliability of government outlays for road maintenance to line ministries or transport agencies through a number of measures. The Bank has typically included covenants in legal agreements or obtained a commitment from the government for annual increases in budget allocations for road maintenance through a letter of sector policy. Out of 19 completed World Bank projects that included covenants as financial arrangement measures, only eight projects complied with the financial commitments at project completion, while the remaining 11 projects did not meet covenant targets at that stage. Of the eight projects that met the covenants by completion, six had kept up this performance as of 2012. Of the 11 projects that did not fulfill the covenants at project completion, five have shown improvement, mainly because of follow-on projects that continued to support the initiated reforms. Prior actions included in DPOs have triggered some increase in road maintenance funding, leading to sustained transport outcomes. DPOs have often included prior actions related to increased spending on road maintenance. Field- and desk-based assessments found that of the seven DPOs that included prior actions related to road maintenance, four had evidence for improved allocation of funding for road maintenance by project closure, and five showed improvement in road condition. 39 CHAPTER 3 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR SUSTAINING TRANSPORT Box 0.4. Lessons from Tanzania, Ghana, and Yemen on the Implementation Results of Road Funds Tanzania’s Central Transport Corridor and Second Integrated Roads projects improved the sustainability of the country’s road network by enhancing the capacity of a modern second- generation road fund and road agency structure funded by a fuel levy that is commensurate with maintenance needs. The country is among the few whose road fund meets the Sub- Saharan Africa Transport Policy Program’s (SSATP) criteria for sustainable road funds. In the fiscal 2007-08 budget, the government decided to increase the fuel levy from $0.08 to $0.16 equivalent per liter of petrol and diesel, which is set at close to the optimal fuel levy for maintenance of $0.15 equivalent per liter. These funds are enough to preserve the entire road network, providing financing not only for periodic and recurrent maintenance, but also for needed rehabilitation. The World Bank has also supported a semi-autonomous road agency with a public-private oversight board. Such entities have helped to ensure that the funds are spent efficiently. The share of the road network in poor condition under the responsibility of the roads agency (TANROADS) was substantially reduced, from 49 percent (14,000 kilometers) in 2003 to 15 percent (4,250 kilometers) in 2009. However, in the Ghana Road Sector Development Project , total revenue for the road fund increased threefold between 2002 and 2007, yet only partially met maintenance needs. In the Yemen Transport Rehabilitation Project, the technical assistance to carry out road fund reforms was ineffective because the road fund failed to become an autonomous agency and remained an implementation arm of the ministry. Critical increases in road user fees that were approved by the parliament were not implemented by the government. Source: IEG 2011. Project Performance Assessment Report: Tanzania - Second Integrated Roads and Central Transport Corridor and Uganda - First and Second Phase Road Development and Roads Sector Institutional Support Technical Assistance Projects; and IEG field and desk based assessments, Financial viability has been secured often through a reliance on diverse funding sources, in terms of types of funds (user fees, licenses, and when necessary, state budget). In Nigeria, the World Bank helped set up a transport fund for the newly created Lagos Metropolitan Transport Authority, LAMATA. 10 The funding came from diverse sources: the Lagos state budget; license fees (hackney permit, road taxes, license plate registration, and vehicle registration); concession fees; and other road user charges (tolls). LAMATA has also entered into an agreement with the Joint Tax Board at the federal level to increase road user charges. These charges are to be shared between LAMATA (50 percent), the state treasury office (40 percent), Motor Vehicle Authority (5 percent), and state Ministry of Transportation (5 percent). However, such comprehensive approaches to funding urban transport have been rare. The road sector in Ethiopia is another example of diverse sources of fund use. The stated sources of road fund revenue in Ethiopia, as per the proclamation, are the 40 CHAPTER 3 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR SUSTAINING TRANSPORT government budget, a fuel levy, an axle-weight-based vehicle license renewal fee, overloading fines, and any other road tariff that may be fixed and approved as necessary. These multiple funding sources need to be well-aligned. Public expenditure reviews can be a useful tool to do so. Securing maintenance funding for the transport sector, particularly for intercity highways and rural roads, has required improvements in broader public sector management, often involving the Ministry of Finance. While more than half of the PERs reviewed failed to highlight the importance of transport maintenance funding, the case studies showed that the countries that had them, especially PERs with explicit mention of transport maintenance, were more likely to have sustained transport outcomes. The review also showed that all five countries that had PERs focusing on road maintenance showed sustained transport outcomes in their highways sectors, while for the five countries whose reviews had little or no analysis of transport maintenance needs, only 40 percent showed sustained transport outcomes. Overall low financial viability points to the value of linking the PERs to sector-level maintenance budget allocation. DPOs in Indonesia were aimed at reducing inefficient public expenditures in infrastructure and implementing governance and fiduciary reforms. As a result, larger budgetary allocations were made available to all transport infrastructure, including roads, and the extent of national roads in good condition improved from 81 to 88 percent. Subsequently, the first infrastructure PERs was carried out. For the transport sector inputs, two policy notes were drafted (on roads and railways), with specific recommendations for how resources can be allocated and used more efficiently using the Road Network Evaluation Tool. Financial arrangements, especially those that rely on user fees and earmarked funds, need to address political realities and social objectives if they are to lead to sustained outcomes. Raising user fees or fuel surcharges can be politically sensitive. Under Chile’s Programmatic Development Policy Loan, user fees and other revenues were not enough to cover O&M costs and neglected citizen’s participation and inadequate communication dialogue to users made it difficult to raise the fees. In the end, the gap had to be covered by the government through a permanent provision of resources. The review found that political issues have been tackled through awareness-raising of the benefits of road funds with the affected stakeholders and inclusion of user representation in the road fund board, which have contributed to reducing resistance. In Zambia, roadside posters advertise the part of the road maintained by 41 CHAPTER 3 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR SUSTAINING TRANSPORT the road fund, increasing its visibility to users. To reduce resistance to increasing fuel levy surcharges, road funds in Zambia and Tanzania have improved transparency by making their accounts public. Another sensitive issue is the impact on the poor and vulnerable of measures to sustain transport financially. User charges can affect access of the poor negatively or positively, depending on how the revenues are used. While low-income passengers are less able to pay, the benefits of improved access to jobs and economic opportunities may be great. Differential pricing can also be used for routes used by the poor, as has been done in urban transport projects in Brazil and Senegal (Box 3.5). However, treatment of this issue in the Transport Sector’s analytic and advisory activities was thin. The review found that the impact of user fees on the poor population has not been systematically assessed at project appraisal or at completion. 42 CHAPTER 3 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR SUSTAINING TRANSPORT Box 0.5. User Charges for Urban Transport Involve Costs and Benefits for the Poor While user fees for urban transport are a potential impediment for the poor, if the revenue is used to operate and maintain existing or expanded services, the poor can benefit greatly. The Belo Horizonte Metropolitan Transport Project in Brazil improved poor residents’ access to employment centers, health centers, and education facilities through an integrated urban transport service of buses and trains. About half of the metro users belonged to low- income households. The extension of the system substantially improved access for the poor. Seven minibus lines with reduced fare (R$0.35 instead of the basic fare of R$1.45) were started to serve slums. The São Paulo Integrated Urban Transport Project, also in Brazil, reportedly saved passengers time and improved service regularity and reliability by integrating fares in the four major stations, while also offering the possibility to use two high-capacity networks at an affordable price. This is extremely important because most São Paulo Metropolitan Train Company passengers originate from low-income areas in the suburbs. Between 2000 and 2004, transport costs for users with one minimum salary decreased by 16.5 percent (from 45.6 percent of income to 29.2 percent), for those with two minimum salaries by 5.5 percent, and with three by 3.3 percent. Moreover, 65 percent of the company’s ticket revenues come from the sale of the “Vale Transporte,” a ticket that by law has to be financed by the employer if the home-to-work trip exceeds six percent of the employee’s salary, further reducing the impact of train fare on the income of poor people. One of the objectives of the Senegal Urban Mobility Improvement Adaptable Program Loan was to keep public transport affordable for the poor. A World Bank–financed study of the leasing scheme reports that although fares for mini-buses increased by 10 percent in 2005, the services of the new mini-buses, which issue color-coded tickets identifying the stages the passenger paid for, were cheaper for most journeys. Fares have been kept low through negotiations by the Executive Council for Urban Transport in Dakar and range from FCFA100 to FCFA240 (approximately US$0.20-0.47) for a trip from the center to Diamniado. Source: Implementation Completion Reviews for Belo Horizonte Metropolitan Transport Project (2004), São Paulo Integrated Urban Transport (2005), Senegal Urban Mobility Improvement Adaptable Program Loan (2009), Dakar Bus Financing Study, IBIS – Transport Consultants, Final Report, June 2008 In modes that rely heavily on funds other than general taxes, it is crucial to have realistic and well-evidenced revenue projections when preparing projects. In railways and some urban transport projects, revenue projections were often not based on realistic estimates of transport demand and maintenance needs. Railway projects that did not achieve financial viability (for example, in Indonesia, Mozambique, Tanzania, and Turkey) generally did not pay enough attention to market conditions and the feasibility of full-cost-recovery rail tariffs, particularly for passengers, but also for export cargoes that needed to remain competitive on world markets. Moreover, multi-modal planning through demand analysis that 43 CHAPTER 3 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR SUSTAINING TRANSPORT incorporates competition from alternative transport modes, such as roads, is often not done, as was the case for the Tanzania railways. INSTITUTIONAL FRAMEWORK Institutional frameworks are transport service delivery models that are tailored to the subsector or transport mode and to country circumstances. Institutional arrangements that are common in the public sector are, for example, a central coordinating authority for urban transport, road agencies for intercity highways, and a decentralized local government/community participatory model in rural roads. Performance-based contracts and concessions have been other forms of institutional frameworks that are common in subsectors that often involve public- private partnerships such as ports, railways, air transport, certain modes within urban transport, toll roads, and in road maintenance activities. Restructuring followed by concessions in railways, toll road concessions, and other forms of public-private partnerships have the potential for coordination or knowledge sharing with IFC, but such collaboration among the World Bank Group has been limited, as elaborated in the next chapter. Institutional frameworks that encourage the adoption of a market-oriented approach with clear accountability and incentive structures tend to improve intermediate outcomes. Institutional arrangements that keep all functions within the public sector, such as road agencies and decentralized, local government–led service delivery models are difficult to coordinate and can be subject to political influence. For example, the Bank has systematically been supporting the creation of a central coordinating authority in urban transport, but out of the 20 case study countries, this model was successfully implemented only in Nigeria. Under the Bank’s Lagos Urban Transport Project, extensive upfront consultations with more than 100 sector stakeholders was carried out by a strong political champion to establish the LAMATA. While similar institutions have been proposed or attempted in other cities (in Algeria, Argentina, Chile, Lebanon, Romania, Tunisia, Uganda, and Vietnam), these have yet to come to fruition. The coordination problem is exacerbated by the very diverse mix of agencies involved. At one extreme are large bureaucracies, such as the national railways that operate a city’s commuter rail services, while at the other are many small transport operators providing para-transit services such as mini-buses and other such micro- enterprises. Organizing all of this into a coherent, efficient, integrated system has posed myriad challenges. 44 CHAPTER 3 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR SUSTAINING TRANSPORT Private sector, microenterprise, or community participation in operations and maintenance has helped create an accountability framework and proper incentive structure. The World Bank’s assistance to governments to prepare appropriate legislation and policies to adopt a landlord port model, for example, was effective. Of the closed ports projects reviewed, those that followed the landlord operator model had the best transport sustaining outcomes, while projects that did not attain sustained transport were those that had delays or difficulty in passing the bills to enable private sector participation. The most notable success was in Liberia. Despite its fragility after the civil war, the port of Monrovia has been transformed from a state-owned enterprise into a landlord port, with a private operator providing commercial services involving general cargo and containers and the government serving as landlord and regulator responsible for public policy. The operator brought management skills and resources to upgrade the efficiency of port services, improving ship turnaround time and cutting theft by 90 percent. The average time in the Monrovia Port for a container is 14 days, compared to averages of 37 days for Sub-Saharan African ports and 11 for OECD countries. The cost per container averages US$1,200 compared with US$1,960 (Sub-Saharan Africa) and US$1,032 (OECD). Output- and Performance-based Road Contracts (OPRC) in roads define the minimum maintenance conditions that have to be met by the contractor through the use of performance measures in the view of reducing the scope for discretionary spending and improving efficiency. But very few projects have shown long-term results to date. In a few successful OPRCs, including Contrato de Recuperacion y Mantenimiento, known as the CREMA contracts in Argentina and Brazil, there is evidence that the approach has worked in preserving and improving road networks effectively and efficiently (Silva and Liautaud, 2011). The key determinants of success for CREMA have been identified as: (i) political will at the highest level to try innovative models for road maintenance; (ii) a strong local construction industry; (iii) strong technical capacity of the implementing agencies to carry out economic analysis of subprojects, environmental assessments, and procurement; and (iv) availability of reliable annual surveys of road conditions. In rural roads, a decentralized community participatory approach has involved the use of microenterprises for operations and maintenance, an approach that has been effective. In Peru, the 18,000 kilometers of road built under Bank-funded projects are properly accounted for and maintained by the microenterprise model. Institutional frameworks should address political economy when implementing staff retrenchment programs, for example, in railways restructuring. In railways 45 CHAPTER 3 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR SUSTAINING TRANSPORT in Mozambique and Poland, the resistance to staff downsizing carried out to cut operational costs in the context of railway restructuring, was overcome by involving stakeholder consultation, rationalization of pay scales, severance payments, incentives, and/or retraining programs (Box 3.6). In Tunisia and Tanzania, on the other hand, the staff downsizing plan did not work because it only supported severance payments while failing to address other measures, such as retraining programs and rationalization of pay scales, leading to reversals of staff retrenchment and financial deterioration of the railways. While concessions are useful institutional reforms that align incentives and enhance accountability, they still have to be based on realistic commercial assessments. For example, restructuring efforts in railways rarely led to concessions mainly due to underestimated road competition and vast capital expenditure needs. In Mozambique (Beira), Senegal, and Tanzania, railways have actually been re-nationalized. The only positive concession example was Camrail in Cameroon. In that case, while there was significant underestimation of investment needs at appraisal and the government was not complying with its passenger service obligations, the concession agreements were amended to improve the financial position of the railway by involving the government in the rehabilitation program and by reducing the debt burden of the concessionaire. The World Bank provided timely support for the revision of concession arrangements in addition to enhancing the capacity of both the government and Camrail with the provision of a framework for coordinated sector investment. 46 CHAPTER 3 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR SUSTAINING TRANSPORT Box 0.6. Staff Downsizing in Mozambique’s Ports and Railways Sector Ports and Railways of Mozambique managed to reduce their staff from 19,387 in 1998 to 1,653. Staff rationalization was completed within the cost estimated at appraisal although the implementation period was 30 months longer. The key activities in the staff rationalization included: (i) diagnosis of staff composition and characteristics by an international consultancy firm; (ii) a survey of 3,020 workers throughout the country to identify the activities they wanted to undertake after retrenchment; (iii) legal consideration of retrenchment solutions and compensation options; (iv) development of a Financial Procedures Manual, stating the requirements for an audit of 100 percent of any invoices and payments relating to the staff rationalization process; (v) creation of specific operational units including: a Staff Technical Unit as an autonomous office to handle the sensitive process of retrenchment and reintegration of surplus staff and the adoption of social mitigation measures; a Project Accounting Unit, and offices for the Organization of Early Retirement; and (vi) following conception of the pilot phase of the reintegration program and its costs, award of a contract with a strategic partner to implement the socio- professional integration of the retrenched workers. Source: IEG 2012. Project Performance Assessment Report: Railways and Ports Restructuring Project SECTOR MANAGEMENT AND CAPACITY BUILDING Sector management and capacity building measures include tools, systems, training, and staffing for planning, design, supervision, management, operation, and maintenance (including road safety measures); the development of procurement and financial management capacity; and the implementation of governance and anticorruption action plans. Sector management and related capacity building measures predominated in intercity highways and rural roads projects and occurred to a lesser extent in urban transport projects. The most common measure was the establishment or adoption of transport planning and management systems, such as asset management systems and the Highways Design and Maintenance Model (HDM), followed by road safety measures. Tools to address governance have increased in recent years. Other common activities included training on contract management in the intercity highways subsector and measures to enhance the capacity of the construction industry although the frequency of these measures has been declining. The Bank has also assisted countries to build sector capacity through working in partnership with regional and global programs. The countries reviewed in Africa had benefited from the Sub-Saharan Africa Transport Program’s Road Maintenance Initiative which was in some cases instrumental in the implementation of their road funds, for example. Global partnerships related to road safety have supported 47 CHAPTER 3 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR SUSTAINING TRANSPORT capacity strengthening, program preparation, knowledge transfer, and improved safety in low- and middle-income countries. While 90 percent of projects that planned sector management and capacity building measures implemented at least one, in many cases projects focused on the completion of physical works and neglected the implementation of such technical assistance components. Insufficient support from government, changes in government, resistance from affected sector stakeholders, and a lack of in-house capacity to use systems have also contributed to low implementation. However, where the measures were successfully implemented, they contributed greatly to enhancing and sustaining the capabilities of transport institutions. In intercity highways, the most effective sector management measures in enhancing institutional capability and sustained transport have been transport planning and management systems, and road safety measures. Strategic planning and overall network analysis and optimization supported by the Bank have improved the cost- effectiveness of intercity highway investments by taking into account the existence of severe budget constraints in the design of rehabilitation programs, requiring that programs aggressively seek lower-cost solutions (Box 3.7). This is largely a question of maintaining road networks and keeping the proportion of the network in poor condition within manageable limits. By reducing investment costs when budgets are constrained, the approach enhances overall network sustainability. In contrast, focusing on just a threshold economic rate of return analysis for individual road sections, without reference to how they fit in the overall sector objectives, has often led to misallocation of resources and does not necessarily lead to financing least-cost solutions. 48 CHAPTER 3 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR SUSTAINING TRANSPORT Box 0.7. Building Institutional Capability through Strategic Highway Planning for Road Rehabilitation Programs Road rehabilitation programs financed by the Bank are intended to help countries catch up with large maintenance backlogs that have built up through a lack of timely maintenance. Typically, a significant portion, say 40 to 50 percent, of the network would be in poor condition at the start of a Bank-funded road rehabilitation program and the program would seek to reduce this figure to a more manageable 10 to 15 percent in a relatively short period of time. Critical to achieving this objective is developing a cost-effective, detailed strategic highway investment plan, which considers tradeoffs among rehabilitation options. In Mozambique, the road sector strategy prepared by the government developed a number of rehabilitation options from low-cost resealing at around $60,000 per kilometer to heavy rehabilitation at a cost of $200,000 per kilometer, with three other options in between. The strategic plan optimized these options. By 2011, the country was in line to meet its target of 77 percent of roads in good and fair condition as a share of total classified roads. Rehabilitation Option Kilometers Cost per kilometer Resealing 500 $60,000 Light Rehabilitation 240 $100,000 Light Rehabilitation with Reseal 520 $120,000 Light and Heavy Rehabilitation 450 $140,000 Heavy Rehabilitation 60 $200,000 Road rehabilitation programs are, however, often designed based on a threshold rate of return for each investment, without recognition of the budget constraints or the objectives of achieving a maintainable network at the lowest possible cost within a reasonable period. In one of the countries covered in this review, its program began with about 30 percent of the network in poor condition, and when it closed eight years later, about 20 percent was still in poor condition. Once the Bank’s assistance ended, the percentage of roads in poor condition increased once again. A lower-cost rehabilitation strategy optimized under expected budget constraints might have yielded a shorter pavement life, but it might have produced a 40 percent cost saving and allowed the road agency to achieve a maintainable network by the end of the program. Source: IEG field and desk-based project assessments, Project Performance Assessment Report: Mozambique-Roads and Bridges Management and Maintenance Project Phase I and Railways and Ports Restructuring Project, 2012, and Maintaining Road Assets: A fresh Look at the World Bank’s 1988 Policy Paper “Road Deterioration in Developing Countries”. Support to improve road safety has translated into enhanced road safety outcomes. Of the 27 projects that addressed road safety, nearly all had baseline values for indicators and end-of-project targets for fatality rates; of these, 21 projects (81 percent) reduced road fatality rates. The World Bank has supported road safety through two approaches: first, through repairing unsafe physical conditions on the road network, referred to as “road crash black spot improvements;” and second, through technical assistance and equipment to improve traffic safety enforcement 49 CHAPTER 3 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR SUSTAINING TRANSPORT and public awareness of safety measures, such as seat belt use, alcohol and speed- related accident risk, and other driver behavior issues. Most projects have shown quick results from fixing readily identifiable problems, such as black spots, but long-term solutions for sustained improved road safety were challenging, particularly those that rely on interagency coordination. In Chile, an exception, the National Commission on Traffic Safety and the National Police and Highway Directorate jointly supported the construction of vehicular and pedestrian overpasses, improving road signs and road markings, and providing lighting equipment. 11 In Senegal, stricter enforcement of licenses and registration has helped to improve safety, keeping unqualified drivers and unsafe vehicles off the roads, and reducing road crashes from 4,074 in 2003, to 3,107 in 2006. 12 In Mozambique, in contrast, road crashes have increased at a higher rate than the number of vehicles on the roads and there is limited progress in drafting a road safety policy and on installation of a system to issue card-type secure driving licenses. 13 The number of projects and programs that addressed governance issues in the transport sector has substantially increased since the Bank’s governance and anticorruption strategy was launched in 2007. Early results of the Bank’s governance and anticorruption engagements in the road sector have generally shown their effectiveness in creating an accountability framework with proper incentives for performance (Box 0.8.8). 50 CHAPTER 3 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR SUSTAINING TRANSPORT Box 0.8. World Bank Support to Improve Governance and Anticorruption in Intercity Highways Intercity highways have led the effort to improve governance and enhance integrity in the transport sector. Indicators have been developed to track vulnerabilities to corruption in roads projects and measures have been identified to reduce the risks of corruption in project design and implementation, including the computerization of procurement and contract management systems; support in establishing systems for monitoring contractors, consultants’ performance, construction costs, and unit rates; enhanced procurement controls to ensure reliability of contract cost estimates; detection of overpricing through bid analysis; enhancement of supervision controls and the introduction of third-party audits; incorporation of complaints-handling mechanisms in bid documents; establishment of vigilance units; and dissemination of information to communities about their service delivery rights. Country case studies in Bangladesh, Moldova, Cambodia, and Azerbaijan carried out under IEG’s evaluation of the World Bank’s country-level engagement on governance and anticorruption found that where the Bank’s governance and anticorruption engagements in the road sector have been implemented, they have generally been relevant and effective. The Integrity Vice Presidency’s investigation of the roads subsector concluded that in addition to expanding project-level preventive measures, more attention should be paid to project supervision, especially in high-risk environments, with a particular focus on the verification of cost estimates and the identification of collusive bidding. The availability of systematic unit cost data facilitates ex post evaluation that may signal problems that need to be addressed to improve sustainability across the Bank’s portfolio of road works projects. Measuring performance through unit cost data more consistently across the Bank’s portfolio may provide important lessons that would enable improved design and sustainability of road works projects. Sources: World Bank. 2011. Curbing Fraud, Corruption, and Collusion in the Roads Sector. Integrity Vice Presidency. Washington, DC. World Bank. 2011. World Bank Country-Level Engagement on Governance and Anticorruption. An Evaluation of the 2007 Strategy and Implementation Plan. Independent Evaluation Group. Washington, DC. 51 CHAPTER 3 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR SUSTAINING TRANSPORT Institutionalizing capacity-building activities in government systems has led to institutional capability for sustained intercity highways. Field- and desk-based studies and PPARs indicate that initiatives launched under World Bank projects rarely survive project closing; they are not institutionalized within project agencies and are often limited to project implementation units. A successful example is in Gujarat State in India, where the World Bank supported the state’s Roads and Building Department to improve its capacity in road network management— planning, procurement, data collection, and analysis—using a road management system, contract management and quality control, and environmental and social safeguards. Sustained transport in Gujarat has been demonstrated through road condition data, increased maintenance funding, and improved administrative efficiency. The government ensured continuity in road agency staffing, supported e- procurement, and provided strong financial oversight. The quality of project roads, using the International Roughness Index, has been maintained at or below the targeted level of four meters per kilometer, five to eight years after completion of road works. The administrative cost as a proportion of the capital and maintenance budget was reduced by almost half, from 30 percent at project approval to 15.7 percent at project completion and to 11.5 percent in 2011. In nominal terms, the budget allocation for routine and periodic maintenance almost doubled between 2001–02 and 2008–09. The review indicates that capacity enhancement of the local construction industry improves the quality of road works and the effective maintenance of roads. In the period under review, 15 closed projects contained components for upgrading the local construction industry, of which 11 reported improvements in the local construction industry, leading to efficient maintenance of the road network. In the state of Karnataka in India, for example, limited experience and skills among contractors were a constraint in the initial years. Most of the local contractors faced technical capacity constraints and lacked proper construction management skills and experience in resource planning and execution. Under the Bank-supported state highways project, training was delivered on modern contract management techniques to enhance the capacity of local contractors. Together with on-the-job experience, it led to improvements in performance in terms of both quality and time. In Mozambique’s Second Road and Coastal Shipping Project, poor quality of local contractors was identified as one of the sector constraints. The government provided for creation of and support for provincial contractors‘ associations with a confederation at the national level. Through the project, contractors were trained, and as a result, a competitive local road construction industry is emerging and a local contractors’ association was formed with over 50 members. 52 CHAPTER 3 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR SUSTAINING TRANSPORT Continuous policy dialogue and sequenced World Bank operations to support government-led reform programs, rather than just episodic project engagements, have been effective. Reforms often take longer to implement than a typical project. Therefore, the time frame for the supported actions should take into account the capacity of the government to carry out the reforms especially in lower- income countries and in challenging subsectors, like rural roads. In addition, proper project preparation, often requiring multiple years and significant human and financial resource requirements, is often necessary to first, ensure the government’s commitment and ownership of the reform agenda that is the key to sustaining transport and, second, to carry out proper appraisals in assessing the institutional capability and financial viability of transport projects. Where countries have weak sector capacity, dedicated sector investment loans that are sequenced have been effective in sustaining transport because they enable continuous dialogue and close supervision of counterpart governments or agencies. Adaptable program loans can be effective for phasing the reform process, but their triggers are often too ambitious, or the sector environment changes, leading to unsuccessful transition into the follow-on phases, as occurred in Uganda’s Road Development Program. Among the projects that are managed by other sectors, DPOs have shown higher implementation rates of desired policies. DPOs dedicated to transport sector policy have not been used widely, however. Examples with sequential and effective World Bank support are highlighted in Box 3.9. In the railways sector, the Bank supported a few policy measures aimed at improving the performance of railways through formalizing mechanisms for compensation for passenger services obligations, compensation for operation and maintenance for railways at reduced tariffs, and policy setting for track access and charges. These measures had mixed results at project closure because the ambitious reform activities were often unrealistically squeezed in the project implementation time frame and because of the time it took to consult with sector stakeholders. Accordingly, significant delays were experienced in the adoption of policies. 53 CHAPTER 3 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR SUSTAINING TRANSPORT Box 0.9. Sustained World Bank Engagement for Sustained Transport Services In Yemen, three Social Funds for Development projects strengthened the capacity of communities to establish (project I), refine (project II), and mainstream (project III) community- led operations and maintenance approaches. At the end of the second project, 90 percent of the roads subprojects had adequate local arrangements for maintenance, with participation from both beneficiary maintenance committees and local authorities. Community contracting was used for roads projects, enabling communities to directly contract local suppliers in a transparent manner, elect committee members, post financial records in public spaces, and hold community meetings—factors that can give credibility to elected committees, encourage households to contribute, and help mitigate risks associated with potential opposition to the project. In Papua New Guinea, where the initial capacity of sector authorities was very poor, the World Bank supported the country’s program together with other donors over an extended period through projects that addressed incremental steps. This led to sustained government commitment and shared goals for implementing reforms required for sustained transport. In Peru, the World Bank has been supporting the institutional arrangements and policies for the country’s rural roads subsector through three rural roads projects over the past 15 years. PROVIAS, the department overseeing the rural roads program in the Ministry of Transport and Communications has been highly effective in planning and executing rural roads projects by scaling up the model mainstreamed through World Bank–supported projects. Following the country’s decentralization process, the proven rural roads maintenance model is now being managed by local governments and municipalities, involving local communities and microenterprises. Results show that the 18,000 kilometers of road built under Bank-funded projects are properly accounted for and maintained by the microenterprise model and community participation. PROVIAS has accurate road inventories and maintenance planning capacity, and the roads rehabilitated under the three World Bank–supported projects have the necessary annual maintenance budgets allocated permanently to the municipalities by a decree. In Chile, the World Bank supported three highways projects over almost 20 years. The Bank’s sustained presence contributed to the modernization of management tools that led to an increase of budget allocation for the sector and strengthening of road sector policies. By focusing on building internal institutional capabilities and emphasizing the use of improved weigh control systems, the projects also set the standard for overall management of Chile’s road network. Source: Field- and desk-based project assessments of Yemen, Papua New Guinea, and Chile. Analytical and advisory activities need to be monitored and self-assessed in terms of their impact. As described in chapter two, the World Bank has used a significant number of analytic and advisory activities to complement the investment support to countries. The review found that all 218 analytic and advisory activities carried out in the 20 countries covered either one or more aspects identified in this evaluation as the measures for sustaining transport. The field- based country case studies searched for the linkages between these measures and 54 CHAPTER 3 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR SUSTAINING TRANSPORT sustained transport outcomes through interviews with task team leaders, government counterparts, and implementing agencies. However, only the PERs were found to be linked to improved allocation of maintenance funds, leading to sustained transport outcomes. This may be because governments cannot distinguish between the effects of analytic and advisory activities and those of loans as also reported in IEG’s evaluation of World Bank economic and sector work and technical assistance, 2000–2006 (IEG, 2008). These findings point to the importance of monitoring the results and impact of analytic and advisory activities, including client feedback from the countries. The use of reimbursable advisory services 14 has shown its effectiveness in the case of air transport in Russia, where the World Bank supported the City of St. Petersburg throughout the process of arranging for a public-private partnership for Pulkovo Airport. According to the client feedback received during IEG’s field visit, the services provided by the World Bank were comprehensive, relevant, of high quality, and delivered at a time when the client required them. Another case where evidence was reported was in Peru, where the technical assistance from PPIAF was influential in helping jump-start the concessions program, revising tariffs in the transport sectors, and generally propelling reforms in transport. In India, prominent analytic and advisory activities covered nearly all subsectors and a significant body of this work complemented the large portfolio of projects in the country’s roads sector, covering nine states and a railways project. Feedback from the government and implementing agencies suggests that this work and related workshops and dissemination activities have helped to raise awareness towards the latest thinking in roads sector management and helped improve institutional capacity and implementation of project activities. Summary Sustained transport outcomes are not systematically monitored beyond project closure, and there is very little documentation on the impact of analytic and advisory activities. The sustained transport outcome analysis was therefore carried out through the review of a subset of the World Bank’s transport portfolio, including 68 project reviews in 20 countries and 33 PPAR results. Sustained transport varies by country income and subsector. Transport is likely to be sustained in high- and upper-middle-income countries, compared to lower- middle- and low-income countries. The subsectors that have generally sustained transport are intercity highways and ports. Railways have faced difficulty 55 CHAPTER 3 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR SUSTAINING TRANSPORT sustaining their infrastructure and services and urban transport, air transport, and rural roads have shown mixed results. Projects managed by the Transport Sector had higher sustained outcomes than those managed by other sectors. In the highway, railways, and air transport subsectors, the institutional, financial, or policy measures supported were generally sectorwide. Subsector-specific measures that were effective in sustaining transport are summarized in Table 3.3. Table 0.3. Lessons of What Worked in Different Subsectors for Sustaining Transport Subsector Examples or effective measures for sustaining transport 15 Intercity Cost effective rehabilitation and maintenance choices Sequential and continuous World Highways for entire network or subnetwork of roads. Bank engagement of policy and Rural Well-functioning asset management system and dialogue according to the capacity Roads mainstreaming of road agency capacity and readiness of the country. Output and performance-based maintenance contracts Dialogue at the public Local government, community, and local enterprise management level and increased participation and their capacity enhancement (rural use of PERs focusing on transport roads) to raise government awareness Urban Integrated tariff structure and attention to affordability and commitment to allocate Transport concerns of the poorer population and related maintenance funds. government subsidy allocations. Diversification of maintenance Establishment of central coordinating agency through funding sources. extensive stakeholder consultations. Proper policy and regulatory Railways Realistic financial projections, multi-modal planning, framework where public -private and government’s readiness for covering deficits. partnerships are attempted. Well-thought-out staff rationalization plan with upfront consultation and communication. Ports User-pay principles in landlord operator model. Note: Excludes air transport because of the small sample size of projects reviewed. The sustained transport outcome is linked to achievement of two intermediate outcomes: financial viability and institutional capability. Projects with a sustained transport objective and identification of maintenance risks are more likely to implement measures to achieve sustained transport outcomes. Projects that implemented measures to sustain transport were more likely to achieve the intermediate outcomes that led to sustained transport. The key factors that contributed to sustained transport outcomes in certain subsectors, including ports, waterborne transport, and toll roads, have been identified as the clear accountability framework and incentive structure through contractual arrangements and application of the user pay principle. Transport financing that relies on diverse funding sources has improved financial viability in urban transport and intercity highways. 56 CHAPTER 3 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR SUSTAINING TRANSPORT The review has found that a combination of investment loans and other instruments, including development policy loans and PERs appear to have contributed to the implementation of reforms and increased allocation of maintenance funds. This indicates the need to align across institutions beyond the sector, especially the finance ministry. Supporting appropriate government-led reform programs in a realistic time frame through continuous and sequential engagement, taking into account the capacity of the sector to carry out the reforms has been effective, especially in lower-income countries and in challenging subsectors like rural roads. Attention to political economy through stakeholder consultation, inclusive analysis, and communication have also contributed to reducing resistance to reforms, including for the establishment of road funds in highways, central coordination agencies for urban transport, and staff rationalization in railways. The review also found that in countries where the World Bank engaged at the sector and multi-modal levels rather than in a fragmented manner, transport was more likely to be sustained. 57 Effectiveness of IFC and MIGA’s Support to the Private Sector in Sustaining Transport Highlights Private sector transport investments supported by IFC and MIGA have to a large extent (83 percent) led to sustained transport services. IFC and MIGA support to making transport more sustained has, however, reached mainly non-IDA countries, as has most foreign direct investment. IFC’s investments have also contributed to sustaining transport beyond the supported project by lifting technical, quality, and service standards across the transport sector. Of the transport transactions supported by IFC advisory services that reached financial closure, too few projects have yet matured to assess their outcomes. The World Bank Group has on some occasions leveraged the complementary roles of the three institutions in sustaining transport. This chapter assesses the extent to which IFC and MIGA support to the private sector for transport investments has sustained infrastructure and services. MIGA and IFC support the engagement of private investors in the transport sector— through their guarantees, investment services, and public-private partnership advisory services. To the extent that the supported private enterprises become sustained transport service providers, IFC and MIGA contribute to sustaining transport directly. Further, such projects can contribute indirectly through effects beyond the project enterprise by setting new quality and service-level standards or by demonstrating the viability of an approach that will enhance sustained transport through wider adoption. Some of IFC’s public-private partnership advisory services also addressed regulatory and legal issues and—together with IFC’s Investment Climate Advisory Services—can contribute to enhancing the enabling environment for subsequent private sector engagement in transport—and hence their likelihood of becoming sustained transport service providers. MIGA’s and IFC’s activities are different from, but complementary to World Bank- supported operations, which engage with governments to establish policy and regulatory frameworks, institutional frameworks, and capacity building of the central transport ministry or agencies. This support to the enabling environment is, however, crucial for IFC and MIGA investments and advisory services, as the extent to which the supported enterprises result in sustained infrastructure and services is often dependent on the regulatory and institutional framework. 59 CHAPTER 4 EFFECTIVENESS OF IFC AND MIGA’S SUPPORT TO THE PRIVATE SECTOR IN SUSTAINING TRANSPORT The assessment of sustained transport infrastructure is based on IFC’s entire investment portfolio in transport; that is, all 53 operationally matured transport investments approved from fiscal 2002-11. 1 From the entire portfolio of 61 approved investments, eight were excluded because they were closed or cancelled before reaching operational maturity. No further reporting exists, so sustainability can no longer be assessed. The majority of the investments evaluated were found in upper-middle-income countries (38), followed by lower-middle-income (9) and a few low-income countries (6). Evidence was obtained for all 53 transport investments from project documents, including related analytic activities prepared prior to, during, and after project due diligence and implementation, complemented by recent information related to the institutional, functional, and financial aspects of the project entity and the key performance indicators used in project documents, such as project appraisal reports, board reports, and supervision reports. The evidence base was further supported by IEG-validated Extended Project Supervision Reports (XPSRs) for 19 IFC investments. The information was supplemented by interviews with the project teams to the extent that they were still staff members of IFC. Of the 15 transport projects supported by MIGA during fiscal 2002–11, seven reached operational maturity. Of these, three had an available Project Evaluation Report or IEG-validated Self-Evaluation Report, used as the evidence base for MIGA’s assessment. 2 About 80 percent of the 53 IFC investments reviewed, and the three evaluated MIGA guarantee projects showed evidence of sustained transport at operational maturity and beyond. 3 Sustained transport outcomes of IFC projects have been assessed through reviewing IFC’s most recent supervision reports which include the key indicators after operational maturity. The same evaluation methodology has been used as in World Bank projects. 4 The two intermediate outcomes for IFC projects are management capability of the private sector and financial viability. The latter has been assessed by reviewing the financial rate of return of the enterprise; in other terms, the assessment determined the extent to which IFC-supported transport services providers were well-operated and financially sound. Broader effects beyond the project enterprise in sustaining transport at the sector level were also captured in this assessment. Box 4.1 illustrates patterns of success and failure typically found in ports, air transport, and railways. Table 0.1. Share of Sustained IFC Investments, FY2002–11 (n=number of investments; c=number of countries) Share of investments that have Years since Reaching Subsector sustained transport (percent) Operational Maturity Roads and highways 100 1.7 60 CHAPTER 4 EFFECTIVENESS OF IFC AND MIGA’S SUPPORT TO THE PRIVATE SECTOR IN SUSTAINING TRANSPORT (n=3; c=3) Ports and shipping 86 2.7 (n=22; c=16 ) Aviation 79 1.6 (n=14; c=10 ) Railways 70 1.9 (n=10; c= 4 ) Urban Transport 50 1.5 (n=2, c=2) Other, including warehousing 50 1.5 (n=2, c=2 ) Total 83 2.1 (n=53 investments) (c=25 countries) Source: IEG portfolio data/ratings. Project-level results show that IFC investments perform well on both management capacity and financial viability. 5 In 64 percent of cases, both management capacity and financial viability were rated as sustained, while in only 15 percent of cases, projects had good management capacity, but low financial viability, indicating inter-linkages between the two intermediate outcomes. Companies with good management capacity tend to generate revenues sufficient to cover the operation and maintenance expenses of their assets and often even the contribution for future investments, thus keeping the company in a financially sound condition. 61 CHAPTER 4 EFFECTIVENESS OF IFC AND MIGA’S SUPPORT TO THE PRIVATE SECTOR IN SUSTAINING TRANSPORT Box 0.1. Successes and Challenges in Sustaining Privately Operated Transport by Subsector Ports. IFC supported the rehabilitation, expansion, and operation of a container and general cargo terminal. The terminal concession was awarded competitively for 20 years in 1999, with the “landlord operator model”a under a framework that fosters competition. The concessionaire was responsible for infrastructure investments and regular maintenance of infrastructure and equipment. IEG found that the institutional capability and financial viability had been sustained after operational maturity. Productivity had significantly increased, as evidenced by an increased turnaround factor. The port’s overall activity rose from about 450,000 tons in 2000 to 870,000 tons in 2010. This experience encouraged concessions of three other major ports in the same country, which, according to the Maritime Chamber, have increased their efficiency in moves per hour by at least 100 percent. Air Transport. This IFC investment upgraded, expanded, and operated an international airport, the larger of two international airports in the country. In 2003, its operation was taken over by a private operator under a 30-year build, operate, and transfer concession granted by the government following a competitive bidding process. Consumer benefits were evaluated as substantial by IEG, reflecting improvements in the airport environment and in passenger processing times for a small additional cost. The first airport concession reduced a potential bottleneck to the development of the tourism industry and attracted investments in new hotels. The project demonstrated the tangible benefits of private sector involvement in airport operation. The Airport Authority is pushing ahead with the redevelopment of another airport. Railways. IFC financed the rehabilitation, operation, and maintenance of a more than 2,000 km railway line under two nearly identical 25-year concessions in two countries. In 2005, following a joint internationally competitive tender process, the two governments awarded the concession to two special-purpose companies. The company decided to raise funding to meet capital expenditures for the first five years, expecting to meet subsequent needs through internal cash generation. The concessionaire estimated that the total capital expenditure for the 25-year concession period would be approximately $370 million, far more than the capital expenditure required under the concessions. According to the IEG-validated XPSR, the sponsor did not have adequate assets to provide for the equity needed. The concessionaire had to pay concession fees to the governments, but due to delays in sourcing equity, the company missed this deadline, leading to a near collapse of the concessions in 2006. Between 2006 and 2011, the concession underperformed due to substantial mismanagement and underinvestment. The concessionaire was in default of the concessions primarily due to nonpayment of concession fees; not investing the required capital expenditures to upgrade the railway; and not meeting freight volume targets and losing share to roads. In 2008-09, the governments issued Default Notices and Notices of Termination due to arrears on concession fees and rent in excess $17.5 million. IFC convinced the governments to withdraw the notices and to discuss a repayment schedule with the concessionaire and its shareholders. A new sponsor invested and brought in a substantially new management team. The restructured concessionaire is in a better position to turn operations around, although its long-term financial viability remains largely unclear. As a result, the project was rated as not sustained. Notes: The examples do not use specific projects names for confidentiality reasons. a. In landlord port model, the port services are provided by private operators while the Port Authority retains responsibility for policy and regulation. Source: IEG Project Evaluation Reports, IEG-validated XPSRs and IEG field- and desk-based studies. 62 CHAPTER 4 EFFECTIVENESS OF IFC AND MIGA’S SUPPORT TO THE PRIVATE SECTOR IN SUSTAINING TRANSPORT The fact that more than 80 percent of IFC-supported investments had sustained outcomes has to be seen in context. The majority of IFC’s transport investments (77 percent by volume) went into non-IDA countries, with 88 percent of all those that reached operational maturity leading to sustained service providers. By contrast, only about a quarter (23 percent) went to IDA countries, where only 67 percent of investments were sustained. Comparing these 23 percent of IDA investments with an overall foreign direct investment net inflow of 16 percent (by volume in US dollars) into IDA countries, IFC’s transport portfolio seems to follow more or less general foreign direct investment patterns. IFC investments also had broader effects in sustaining transport beyond the project enterprise. Typically IFC invested in one (or a few) players in a particular transport subsector of a respective country; its reach to the sector as a whole was limited. 6 However, according to the XPSRs, 43 percent of all IFC-supported transport projects that reached operational maturity were found to have lifted technical, quality, and service standards across several of the major players in the country’s transport sector. About nine percent of projects had knowledge transfer and training effects beyond the project entities. Finally, 35 percent of projects had evidence of demonstration effects – that is, they demonstrated that investments were feasible in a nascent transport subsector and that the regulatory and judiciary frameworks were solid, leading to replication of similar investments in the same subsector. In the early stages of private sector participation, it is difficult for IFC to identify adequate project sponsors. Hence, its investments tend to occur later than other private investors who engage in a country’s reforming transport sector. Analysis of the timing of IFC’s transport investments in an overall sector reform pathway 7 revealed that only about 29 percent (or 15 investments) took place in the early stages of private sector participation, which is less or equal to seven years after countries opened up the transport sector for private investors. 8 Country case studies indicated that IFC has, at times, challenges in identifying project sponsors of adequate quality in the early stages of private sector participation in a given country, that is, sponsors with appropriate financial strength, experience, and integrity. Early engagements would demonstrate that private sector participation is possible in untested regulatory regimes or in distorted markets caused by the competition of a large state-owned enterprise. Early engagements typically also support first-of-a-kind concession arrangements, as well as divestiture or privatization efforts of state-owned enterprises. 63 CHAPTER 4 EFFECTIVENESS OF IFC AND MIGA’S SUPPORT TO THE PRIVATE SECTOR IN SUSTAINING TRANSPORT Investing earlier in a sector reform process does not come to the detriment of sustained transport. Even when IFC invested earlier in sector reform processes— and thus took on more complex and riskier investments—it did not fail in sustaining transport: of the 29 percent of investments that were undertaken during the early stages of sector reform, 80 percent turned out to be sustained, compared to 83 percent of average transport investments. Some IFC-supported transport enterprises have sustained infrastructure and services even in the absence of a proper regulatory framework. Most investments in Russia’s rail cargo sector were well-managed businesses; still these cargo operators are functioning in a tariff system that allows for significant non-infrastructure costs in the access charge, increasing the latter unnecessarily. The World Bank’s recent PERs of Russia calls for a review of the current “discount system,” suggesting instead a bottom-up approach including costs only directly attributable to infrastructure. 9 The regulatory framework allows IFC’s clients to operate successfully in their niches, as profit margins are sufficient and tariffs at least predictable. Privately financed sustained transport can hence be encountered even in countries where the regulatory environment is imperfect. Results at the project level should therefore not necessarily be equated with sustained transport at the sector level. Factors that Affect Sustained Transport Outcomes Key factors that affect the sustained transport outcomes of IFC- and MIGA– supported investments are similar to those identified in the review of the World Bank project portfolio. These are project preparation and incentive and accountability frameworks. IFC and MIGA undertake a thorough assessment—not only of the commercial factors but also of the enabling environment, including regulatory issues, during appraisal of its investments, and this is likely to be the determining factor for IFC’s effectiveness in sustaining transport. IEG’s XPSR ratings further corroborate this finding with 95 percent of transport investments rated satisfactory or better on screening, appraisal, and structuring (one criterion for assessing IFC work quality in XPSRs) compared to 73 percent of all IFC investments. The private sector’s incentives and accountability frameworks, along with its approach to investment planning also contribute to sustained transport. Private sector engagements have inherent motivation and incentives that are aligned with accountability and planning structures. This may be different from the public sector where operators are not subject to incentives in competitive markets to provide 64 CHAPTER 4 EFFECTIVENESS OF IFC AND MIGA’S SUPPORT TO THE PRIVATE SECTOR IN SUSTAINING TRANSPORT reliable services at the least cost. Box 4.2 elaborates on the private sector’s incentives and accountability systems and their impact in sustaining transport. The eight cases of failure in sustaining transport 10 are attributed to poor management capacity or low financial viability. Changes in governments and natural disasters have also been factors. Deficiencies in management capacity often stem from poor corporate governance and inexperienced sponsors. Out of the 10 IFC clients that have poor institutional capacity, six had corporate governance issues and four were attributed to poor sponsor quality. 11 Realistic assessment of commercial viability in project preparation has affected the financial viability of investments. Poor financial viability was found in 50 percent of railway investments that reached operational maturity, representing the relative largest share of the financially nonviable investments (four out of eight) when compared to all other subsectors. Railway investments generally did not pay enough attention at appraisal to market demand conditions, such as road competition, and accurately estimating the capital investment needs to revamp infrastructure, as in the Tanzania and Kenya-Uganda railways. Those IFC- supported railway investments that sustained transport, supported specific segments with a captive demand supply chain in freight, as in investments in Brazil, or a solid customer base, as in Peru and Russia. Building management capacity and enhancing corporate governance have contributed to management capability. In 41 percent of the 53 transport investments that reached operational maturity during fiscal 2002-11, IFC has undertaken proactive measures toward enhancing the design or structure of the projects and hence also their sustainability. These measures can broadly be grouped into activities intended to improve the management capacity or the financial viability of the private sector company engaging in the project. Most of the measures were related to enhancing the financial viability of the project enterprises, and they included mobilizing funds from commercial lenders, enhancement of financial structures, 12 and modification of concession contracts, for example, to mitigate the effects caused by delay in land delivery in a toll road construction project. Measures related to enhancing management capacity are a provision of technical advice—for example, to ensure that technical expertise is in place to oversee the quality of physical works or to redesign the project to avoid social conflicts—enhancing corporate governance; and improving environmental and social standards (Table 4.2, Box 4.3). 65 CHAPTER 4 EFFECTIVENESS OF IFC AND MIGA’S SUPPORT TO THE PRIVATE SECTOR IN SUSTAINING TRANSPORT Box 0.2. Project-Level Measures Aimed at Enhancing the Sustainability of Private Transport Measures that improved management capacity: Enhancing corporate governance. IFC assisted project entities in building up more transparent accounting and reporting systems and making them comply better with corporate governance standards. Enhancing environmental and social standards. IFC was proactive in addressing social matters in anticipation of adverse social conflicts later on that would have jeopardized the sustainability of the projects. Providing technical advice. MIGA contributed by adjusting the design of a toll road to avoid leading it through a densely inhabited area. IFC advised to renegotiate the high fees paid to the government. It advised a company to engage in local or international management oversight, as the firm managing construction did not have technical expertise. Measures that improved financial viabilitya: Improving financial structures. In an urban transport project, IFC created a solid, ring-fenced security package in a corporate finance context to secure cash flows. An aviation investment was phased as the ramp-up of operations took longer than anticipated. IFC organized a lien over receivables in an airport to reduce the risk of nonpayment by the major client of the airport. Modifying concession contracts with governments, as in railways to reduce government concession fees to sustainable levels during restructuring or, as in the case of a highway company, to mitigate the effects caused by a delay in land delivery by the government of Peru. In an airport, the concession agreement was amended to allow payments to be deferred if its major client traffic declines precipitously, for example, in case of liquidation. Protecting the enterprise and IFC from default of their major clients and other potential risks by entering into bilateral agreements with the airport’s major client or into direct agreements with the government to define clearly IFC’s rights in case of default by airport concessionaires. Measures that improved both management capacity and financial viability: Working as an honest broker. In railways, during the financial restructuring processes between the government and sponsors to reach agreement on a feasible concession, or by encouraging cooperation between the parties for restructuring and settlement. Source: IEG assessment of all 53 operationally matured IFC transport investments fiscal 2002-11, based on XPSRs, PERs, due diligence reports, complemented by interviews. a Over and above to providing longer tenure which almost always is given and helps improving financial structure. Government financial support for public-private partnerships played a minor role in sustaining IFC investments, but a stronger one in advisory services. Only 13 percent of transport investments (seven of 53) required government support to make them feasible and sustained. In three toll road investments, the public sector took on explicit direct or contingent liabilities by providing availability payments or minimum revenue guarantees regardless of traffic volumes; in a fourth investment, it supported an investment directly by burdening some of the upfront capital expenditures for construction. Support to retrenchments occurred in two railway investments. 13 Of the seven investments requiring public support, two turned unsustainable. Most of the remaining transport investments, including ports, shipping, and aviation, did not rely on public funding—either because they were 66 CHAPTER 4 EFFECTIVENESS OF IFC AND MIGA’S SUPPORT TO THE PRIVATE SECTOR IN SUSTAINING TRANSPORT commercially viable ports or airports to start with, or they were fully privately owned transport service providers (airlines or shipping companies). Box 0.3. Incentives and Accountability Contribute to Sustained Private Transport Investments Accountability structure and incentives. Private investments are usually embedded in an accountability structure with aligned incentives. Those who invest also earn the profits. In order for them to earn profits, they have to deliver the services agreed upon (for which they are being held accountable by the public side) and hence have a business reason to keep the investment in good shape. With incentives aligned with cash flow, management often tends to be more efficient, leading to improved construction of new assets, improved service delivery, increased innovation and adapted design, and more reliable maintenance, according to World Bank reports (PPIAF 2012). “Lifetime” approach in selecting and planning. Private investors base their investment decision on an all-encompassing calculation, factoring in initial capital expenditure and running costs up to the end of the investment horizon against an expected cash flow. Only this will allow the investor to judge if the return on investment over the entire project lifetime is positive—an intrinsic interest of the private investor. Operations and maintenance are either factored in automatically (in the case of private operators, including privatizations) or are made part of the concessionaire’s responsibilities through the concession contract. This is not necessarily true of other PSP arrangements, such as leases or management contracts, since these involve specific contractual arrangements between the government and the lessor or management company regarding the division of responsibilities for essential investments (in either new assets or rehabilitation of existing assets) that need to be undertaken. Private sector provision of transport infrastructure and services is not always sustained, however. Both private operators and public-private partnerships have failed—with a few spectacular and much publicized failures raising opposition.a The reasons can lie anywhere in the project life cycle, from selection, design, and procurement to supervision and monitoring. Robust feasibility studies, integrated in a broader strategic sector assessment, are equally essential as is the existence of a performance monitoring system, adequate judicial, and regulatory frameworks, capacity to assess contingent liabilities for the public as well as the preparedness to actually pay for service delivery— often found problematic in cultures where transport service delivery had been regarded a public obligation. Hence, private sector participation’s success depends in part on a balanced risk-reward profile, clarity of roles and responsibilities, performance monitoring, and public stewardship to ensure public interests are met. Note: a. Interurban toll motorway private concessions (in France in the early 1970s, CEECs or Mexico in the early 1990s for example), or, to limit the list to English or French experience: Eurotunnel, Orlyval, North Ring Road of Lyon or, more recently, Railtrack (in this case not a public-private venture but a privatization). Sources: World Bank Institute 2012; World Bank Group 2009; Forrer, and others, 2010; European Investment Bank 2003; OECD (2007) 67 CHAPTER 4 EFFECTIVENESS OF IFC AND MIGA’S SUPPORT TO THE PRIVATE SECTOR IN SUSTAINING TRANSPORT Table 0.2. IFC Measures to Help Sustain Transport by Subsector and Country Income (Number of Projects for Which Measures Were Identified, Out of All 53 Operationally Matured IFC investments FY02-11) Roads& Urban Ports & Other Sustainability Aviation Highways Railways Transport Shipping Sector UMIC LMIC LIC Enhancing (n=14; (n=3; (n=10; (n=2; c= (n=22; (n=2; (n=38 (n=9; (n=6; Measures c= 14) c=3) c= 4) 2) c= 16 ) c= 2) c= 14) c=6) c=4) Concession 1 1 1 2 1 amendment Contract 2 1 1 agreement Corporate 1 3 1 4 1 governance Environmental and 2 1 2 5 social Financial fund 1 1 1 3 mobilization Financial 2 1 3 mechanism Improving financial 2 1 3 1 3 2 2 structure(1) Technical advice 2 1 1 2 3 2 1 Total 4 2 4 2 9 1 17 3 2 Investments that have at least one measure Percentage of 29% 67% 40% 100% 41% 50% 45% 33% 33% Investments that have at least one measures /Total Investments Source: IEG analysis of all 53 operationally matured IFC transport investments approved during fiscal 2002-11, corresponding to 44 companies (1) Improving financial structure over and above to providing a longer tenure which almost always improves the financial structure. The lack of public funding was one of the reasons for the low completion rate of IFC advisory services. Of the two-thirds of all IFC advisory services that were not eventually signed, 31 percent would have required some sort of government guarantee, but in 80 percent of these cases, such public support was not available. Effect of IFC Advisory Services on Sustained Transport Outcomes Since assuring finance for maintenance and rehabilitation is an integral part of public-private partnerships or private sector participation—IFC advisory services have the potential of sustaining transport. IFC implemented 54 advisory services in transport during the evaluation period of which IEG reviewed 80 percent (43 68 CHAPTER 4 EFFECTIVENESS OF IFC AND MIGA’S SUPPORT TO THE PRIVATE SECTOR IN SUSTAINING TRANSPORT advisory services). Evidence was obtained from approval documents, intermediate supervision reports, and project completion reports—complemented by six IEG evaluations of advisory services conducted recently. The transactions contained in the sample provide a reasonably broad cross-section of countries and types of assignments, to enable the drawing of general conclusions regarding IFC’s approach and success in contributing to sustained transport. The aim of IFC’s advisory services for public-private partnerships is to bring transactions to commercial and financial closure. In practice, this means to bring a successful completion to what is usually a two-phase process, wherein due diligence and recommendations on the structure of transactions carried out in the first phase lead to a second implementation phase, in which IFC would help organize a transparent, competitive bidding, hopefully to result in a successful bid and award of concession. It is desirable, though not always easy to guarantee, that the winning bidder will be able to secure financing from commercial sources. In carrying out these transactions, IFC makes use of retainer and success fees to the extent possible. For the most part, these are employed to ensure the commitment of the client, as well as to ensure that IFC does not knowingly distort the market through the provision of subsidized services. Success fees do not cover a significant proportion of IFC’s costs: the largest share (about 50 percent) comes from donor contributions. In addition to specific transactions, IFC also seeks to expand the public-private partnership market through studies, workshops, and technical assistance. It has also made efforts to collaborate with the World Bank at the level of Quality at Entry, through World Bank Institute training and through seeking regular cross-support by World Bank staff. It is currently premature to try to assess outcomes of IFC advisory services on public-private partnerships in transport, as too few projects have yet matured. Therefore the evidence base of the recently introduced tracking system is limited to a few post completion reports of which the aggregation of findings is not meaningful. Although 85 percent of the transactions delivered the specific advice for the transaction, 14 only about a third resulted in the completion of a successful bidding process and award of a concession and subsequent financing, a prerequisite for turning into a sustained transport service provider. 15,16 The volatility of governments has been one reason for the relatively low success rate of IFC advisory services. In more than half (56 percent) of IFC’s advisory services in transport that eventually were not successful, government composition was volatile. Key players were liable to change at short notice, as a result of cabinet 69 CHAPTER 4 EFFECTIVENESS OF IFC AND MIGA’S SUPPORT TO THE PRIVATE SECTOR IN SUSTAINING TRANSPORT reshuffles, transfers of civil servants at short notice, election results, and occasional coups d’état with the result that a project can lose a “champion” midway through the process, and with it the support needed to undertake decisions in a timely fashion involving politically sensitive reforms. Conversely, in 30 percent of cases in which the concession was signed and the concessionaire obtained finance, the transaction had been embedded in a broader sector reform of the country. Lack of public sector support, sometimes required to make public-private partnership transactions lucrative for private investors, was another reason for low success. Of all IFC advisory services in transport that did not get signed, about one- third (31 percent) would have required some sort of subsidy element by the government, either to clean up past liabilities of the project enterprise, to alleviate the burden of the upfront capital expenditure of a public-private partnership transaction, or to subsidize cash flow over the project life time, as identified by IFC advisory teams. In four of the five cases, the government did not have the required fiscal space, aggravated by the recent financial crisis. The advisory services transactions that did reach successful commercial closure, did so by structuring transactions that had a significant potential for longer-term developmental impact. The Hajj Terminal in Saudi Arabia, the Queen Alia airport project in Jordan, the Benin port project, the Male Airport privatization in the Maldives, the BA 093 toll road project in Brazil, and the Air Jamaica private sector partnership project, because of due diligence, structuring, and outcomes, resulted in long-term concessions that brought significant revenues to the government through up-front fees, an annual share of concession income, and—equally importantly—commitments to make the investments to rehabilitate and maintain assets over the length of the concessions. As a result, these projects provided a basis for follow-on transactions in the country, or for the structuring of similar transactions in other countries, a kind of “demonstration effect.” However, long- run outcomes in terms of sustained transport are not monitored after the concessions are awarded, so cannot be assessed. IFC’s upstream work, while successful by its own measure, 17 has less relevance for sustaining transport. Apart from transaction-oriented advisory assignments, IFC’s public-private partnership advisory services line also undertakes studies, workshops, or institution-building from time to time (17 percent of all services). 18 The outcomes of these various activities appear to have been successful in most cases. Raising awareness, improving a company’s stakeholder management skills, conducting market reviews aimed at building up a pipeline of transactions, or preparing a business plan that helped pave the way for a subsequent IFC investment may have some incremental value for increasing private sector 70 CHAPTER 4 EFFECTIVENESS OF IFC AND MIGA’S SUPPORT TO THE PRIVATE SECTOR IN SUSTAINING TRANSPORT participation, but contribute only very indirectly to improving the enabling environment, policy frameworks, or sustaining transport. Similarly, IFC’s Investment Climate Advisory Services line has so far focused on creating a business-friendly environment with rather little emphasis on enhancing the enabling environment specifically for private sector participation in the transport sector (about three to- five percent of their services). Coordination between the World Bank, IFC, and MIGA The World Bank Group has sometimes leveraged the complementary roles of the three institutions in sustaining transport. The World Bank’s efforts in creating an enabling environment to sustain private sector participation through PPIAF and other lending and analytical and advisory activities have a complementary character to IFC’s investments, as the World Bank’s activities are more likely to focus on lower-income countries where a weak enabling environment deters private participation. For example, of the 136 PPIAF projects, 64 percent targeted IDA countries. At the project level, coordination between IFC and World Bank was found between IFC advisory services and the World Bank. 19 IFC not only benefits from World Bank sector expertise but it is also an opportunity to align activities across the institutions. In addition, IFC advisory services have sometimes made efforts to work in close collaboration with the World Bank, providing inputs to policy dialogue, and on occasion providing direct technical inputs to governments for updating specific pieces of legislation. 20 In addition, evidence was found that in a few cases, IFC’s due diligence for its investments built on World Bank expertise in a specific subsector, for example, by engaging World Bank experts during due diligence of Russian rail cargo investments. Beyond these cases, however, little evidence was found that IFC liaised with the World Bank on a regular basis for specific projects. However, there are also missed opportunities. IFC investment teams often gather valuable knowledge about country-level conditions, including regulatory issues and legal constraints that would impair private sector participation. This knowledge represents valuable intelligence that could be shared with the World Bank’s analytic and advisory activities. Evidence for IFC-World Bank coordination on such issues was scarce—pointing at a missed opportunity in maximizing the complementary roles of both institutions. Collaboration across the World Bank Group on specific private sector participation transactions is sensitive to conflicts of interest. IFC has Conflict of Interest Guidelines in place governing the relationship between IFC advisory and 71 CHAPTER 4 EFFECTIVENESS OF IFC AND MIGA’S SUPPORT TO THE PRIVATE SECTOR IN SUSTAINING TRANSPORT investment services. The evaluation has not found any evidence that such conflict of interest has been an issue in World Bank Group collaboration in the transport sector, but an increased collaboration between the World Bank and IFC—even though desired for sustained private sector projects—will have to consider conflict of interest issues. Summary Overall, about 80 percent of all IFC transport investments approved and operationally matured from fiscal 2002–11 and the three evaluated MIGA 21 guarantee projects showed evidence of sustained transport at the point of reaching operational maturity and beyond. 22 Two key factors affect such outcomes: the quality of preparation and incentive and accountability frameworks. In the few cases where transport was not sustained, the key reasons were attributed to corporate governance, sponsor quality, and commercial reasons. As IFC and MIGA investments are exposed to commercial risks, these two organizations satisfy themselves prior to taking the decision to invest that the prospective project is likely to be sustained. In cases where IFC’s due diligence identifies the need to improve project design or structure, IFC uses proactive measures — which in turn also helps to sustain the projects. In line with general foreign direct investment flows, IFC’s transport investments reach fewer IDA countries and their effect in sustaining transport is more limited to the project enterprise with some sectoral spill-over effects. An assessment of the actual effects of IFC advisory services on sustaining transport is difficult as too few projects have matured to be assessed for their outcomes, even though it should be noted that IFC advisory services delivered in 85 percent of cases the needed outputs for the transaction process. The World Bank Group has to some extent leveraged the complementary roles of the three institutions in sustaining transport. In particular, coordination is foreseen—and was found—between IFC advisory services and the World Bank. In a few cases IFC investments have built on World Bank expertise. In addition, the work of the World Bank’s PPIAF was found complementary as it has worked with countries in creating an environment conducive to sustained private sector participation—many of which were IDA countries where the lack of an enabling environment traditionally deters private sector participation. 72 Conclusions and Recommendations This chapter summarizes the conclusions of the evaluation and IEG’s recommendations to Bank Group management. Summary of Findings and Recommendations for Sustaining Transport To what extent have the Bank Group strategies and operations aimed to sustain transport? Sustained transport as defined in the context of this evaluation features prominently as a concern in Bank Group strategies and analytic and advisory activities, but this concern is not borne out by project-level objectives and interventions. Only 15 percent of all World Bank projects managed by the Transport Sector focus on sustained transport in their objectives and the share has declined over the past decade. This does not appear to be due to changes in the country or subsector composition of the portfolio. To help countries sustain transport, the World Bank has supported measures related to; (i) financial arrangements for operations and maintenance; (ii) sector policy and regulatory frameworks; (iii) institutional frameworks; and (iv) sector management and capacity building. Nearly all World Bank transport operations provide financing for sector management and capacity to sustain transport, but only 16 percent support financial arrangements, and even that share has been declining. One-third of approved World Bank projects identified maintenance and funding shortfalls as risks at the appraisal stage; while it is difficult to tell ex ante whether this is an under- or over-statement of the risk, among those reporting the risk, only one-third included financial management measures in project components. Based on the experience of intercity highways projects, ex ante and ex post economic analyses are not factoring in the sensitivity of the cost and benefit streams to shortfalls in operation and maintenance. These highway projects which usually depend on constrained public resources for maintenance, are not being designed to minimize maintenance needs. Private sector transport investments supported by IFC and MIGA are exposed to commercial risks and assessed for their viability before approval. Whenever needed, IFC uses proactive measures aimed at enhancing project design and structure to help sustain individual investments in the transport sector. 73 CHAPTER 5 CONCLUSIONS AND RECOMMENDATIONS Has Bank Group support to sustain transport been implemented and effective? Sustained transport outcomes are not systematically monitored beyond project closure, and there is very little documentation on the impact of analytic and advisory activities. The sustained transport outcome analysis was therefore carried out through the review of a subset of the World Bank’s transport portfolio, including 68 project reviews in 20 countries and PPAR results. For IFC, 53 projects, and for MIGA, three projects, for which information was available were reviewed. Sustained transport varies by country income and subsector for World Bank- supported operations. Based on the review of lending and non-lending activities in 20 sample countries, transport is likely to be more sustained in upper-middle- income and high-income countries, compared to lower-middle-income and low- income countries. The subsectors most likely to have sustained transport are intercity highways and ports. Railways have faced difficulty sustaining their infrastructure and services for both World Bank and IFC projects because they generally did not pay enough attention at appraisal to market demand conditions, such as road competition, and accurately estimating the capital investment needs to revamp infrastructure. Urban transport, air transport, and rural roads have shown mixed results. Transport is less likely to be sustained in projects managed by sectors other than the transport sector. Sector management and capacity building activities had the highest implementation rate by project closure, followed by institutional frameworks and sector policies and regulatory frameworks. Financial arrangements had the lowest implementation rate. Projects that implemented these categories of measures are generally more likely to achieve financial viability and institutional capability — the two intermediate outcomes that lead to sustained transport outcomes. Financial arrangements, if implemented, had the largest impact on the intermediate outcomes. Assessment of most projects supporting the intercity highways subsector, in particular, indicate sustained transport outcomes at the sector level. This is because World Bank support has focused on creating an enabling policy, financial, and institutional environment for the entire road sector to manage and maintain their road networks. About 80 percent of IFC transport investments approved and operationally matured from fiscal 2002–11 and the three evaluated MIGA guarantee projects showed evidence of sustained transport at the point of reaching operational maturity and beyond. Both intermediate outcomes—management capacity and financial viability—have been achieved in most cases. Although 85 percent of IFC 74 CHAPTER 5 CONCLUSIONS AND RECOMMENDATIONS advisory services transactions delivered the needed outputs for the transaction process, only about a third were successfully completed. The World Bank Group has to some extent leveraged the complementary roles of the three institutions in sustaining transport. The World Bank’s efforts in creating an enabling environment to sustain private sector participation through PPIAF and other lending and non-lending activities have a complementary character to IFC’s investments since the World Bank’s activities focused on creating an enabling environment for sustained transport, including the policy and regulatory framework conducive for private sector participation. At the project level, coordination between IFC and the World Bank was found. In a few cases, IFC’s due diligence for investments built on World Bank expertise. Beyond these cases, however, little evidence was found that IFC liaised at the project level with the World Bank on a regular basis. What factors have contributed to success? The sustained transport outcome is linked to the achievement of two intermediate outcomes: financial viability and institutional capability. Projects with objectives to sustain transport and those that identify maintenance funding risks are more likely to support measures to improve financial viability or institutional capability, and more likely to be sustained in the long run. A clear accountability framework and incentive structure for operation and maintenance is critical for sustained transport in IFC- and MIGA-supported operations, and World Bank-supported operations in certain subsectors, including ports and toll roads. This is often addressed through contractual arrangements and the user pay principle often featured in public-private partnerships. In intercity highways and rural roads, institutional frameworks can adopt similar features, if they are well designed and operated according to contractual agreements/guidelines. Transport financing that relies on diverse funding sources has improved financial viability in urban transport and intercity highways. These different sources must be aligned well. Countries that had PERs, especially with explicit mention of transport maintenance, were more likely to realize sustained transport outcomes. However, more than half of the PERs reviewed failed to highlight the transport maintenance funding issue. Supporting appropriate government-led reform programs in a realistic time frame through continuous and sequential engagement, taking into account the capacity of 75 CHAPTER 5 CONCLUSIONS AND RECOMMENDATIONS the sector to carry out the reforms has been effective, especially in lower-income countries and in challenging subsectors, like rural roads. The political economy of reforms — who benefits, who doesn’t, and how the latter can be compensated — is critically important to understand. In countries where political economy was factored in during the preparation process through stakeholder consultation, inclusive analysis, and communication have also contributed to reducing the resistance to reforms, including for the establishment of road funds in highways, a central coordination agency for urban transport, and staff rationalization in railways. As evidenced especially in the railways, multimodal planning and coordination at the country’s sector level has also been important in generating the necessary level of demand to make transport services viable. In intercity highways, road network models under budget constraints appear to be effective in identifying cost-efficient investment choices, rather than simple economic rate of return calculations on pre- selected road investments. Two key factors that affect sustained transport in IFC and MIGA operations are: the quality of preparation and incentive and accountability frameworks. In cases in which transport was not sustained, the key reasons were attributed to corporate governance, sponsor quality, and commercial reasons. Recommendations for Sustaining Transport Increase the focus on sustaining transport infrastructure and services in project design Increase the focus on sustained transport in projects with sector reform objectives and components through measures to adopt or strengthen financial arrangements for and enhance institutional capability to plan and carry out operations and maintenance. World Bank-supported projects for the intercity highways and rural roads subsector should: (i) systematically carry out ex-ante risk analysis and mitigation with regards to operation and maintenance and funding shortfalls; (ii) conduct sensitivity analysis on the effect of inadequate maintenance on net benefit flows resulting from the transport infrastructure and services; and (iii) systematically evaluate the entire network or subnetwork managed by the road agency to seek cost-effective rehabilitation and maintenance solutions. 76 CHAPTER 5 CONCLUSIONS AND RECOMMENDATIONS Link World Bank transport investment operations with sectorwide reforms that will sustain these investments through support for complementary DPOs and analytic and advisory activities. World Bank-supported projects that finance transport components and are managed by other sectors should ensure that the transport components are integrated into the operations and maintenance of transport sector plans and strategies. Improve the long-run financial viability of support for sustained transport For the subsectors and transport modes that rely on operations and maintenance funds from public sources or earmarked funds such as intercity highways and rural roads: (i) engage with the client where high-level policy decisions related to maintenance funding can be taken; (ii) within the Bank, put a more prominent emphasis on the adequacy and reliability of transport maintenance funding in PERs. Diversify the sources of financing for roads and urban transport for more reliable financing, such as axle control fees, fuel surcharges, vehicle registration fees, and congestion charges. In the railways subsector, critically assess the viability of investments, with particular attention to: (i) realistic demand forecasts based on analysis of potential competition from other transport modes; and (ii) realistic estimation of rehabilitation and maintenance costs. Support to urban transport should include a comprehensive financial analysis of the overall urban transport system, including fare integration, tariffs and subsidies, and the net impact on the poor. Strengthen institutional capability to sustain transport outcomes Where complex reforms are planned, encourage continuous and sequential engagement and support appropriate government-led reform programs in a realistic time frame, taking into account the capacity of the government to carry out the reforms. Factor in the political economy in the reform process by identifying the key stakeholders and constituencies upfront, carrying out stakeholder analysis, ensuring consultation and communication during preparation of the reform, and adopting ways of compensating the affected groups to minimize the resistance to change or delays in legal or regulatory approvals. In subsectors that are dominated by the public sector, such as intercity highways and rural roads, mainstream proven models of demand-side governance and commercial principles to ensure that there is a proper accountability and incentive 77 CHAPTER 5 CONCLUSIONS AND RECOMMENDATIONS framework in place, such as output- and performance-based maintenance contracting, second generation road funds, and microenterprise models. In order to strengthen the ability of countries to routinely collect data on, monitor, and assess sustained transport outcomes, support governments to put in place a reliable system to monitor and evaluate such outcomes systematically in all subsectors, particularly for ensuring adequate road maintenance. 78 References Ackerman, R, K. Gwilliam and L. Thompson. 1998. The World Bank, Transport and the Environment. Japan Railway and Transport Review 18. “East Japan Railway Culture Foundation.” http://www.jrtr.net. Bandyopadhyay, Arnab, and Natalya Stankevich. 2011. “Institutional Development and Good Governance in the Gujarat Highway Sector—Learning from Gujarat.” Transport Research Support Program. Washington, DC: World Bank and Department for International Development, U.K. Cuttaree, Vickram, and Cledan Mandri-Perrott. 2011. “Public-Private Partnerships in Europe and Central Asia.” Washington, DC: World Bank. DFID. 2007. “Literature review on Private sector Infrastructure Investment.” London: Department for International Development. Fan, Shenggen. 2004. OECD DACT POVNET Agriculture and Pro-Poor Growth Workshop paper: “Infrastructure and Pro-poor Growth.” Washington, DC: International Food Policy Research Institute . Grigg, N. 2010. Infrastructure Finance: The Business of Infrastructure for a Sustainable Future, Hoboken, New Jersey: John Wiley & Sons, Inc. Gwilliam and Kumar. 2003. “How effective are second generation road funds” A preliminary appraisal”. World Bank Research Observer 18 Gwilliam, K. 2011. Africa’s Transport Infrastructure: Mainstreaming Maintenance and Management. Washington, DC: World Bank. Harral, Clell, G. Smith, and William Paterson. 2011. Maintaining Road Assets: A Fresh Look at the World Bank’s 1988 Policy Paper “Road Deterioration in Developing Countries. Washington, DC: World Bank. Heggie, I and Vickers P. 1998. “Commercial Management and Financing of Road. “World Bank Technical Paper 409, World Bank, Washington DC. IEG. 2006b. “Seven Findings for the Nexus of Infrastructure, Agriculture, and the Environment.” Evaluation Brief, August 200, Independent Evaluation Group. Washington D.C. ____. 2007a. “Water and Development: An Evaluation of World Bank Support 1997-2007.” Washington, DC: Independent Evaluation Group. ____. 2007b. “A Decade of Action in Transport: An Evaluation of World Bank Assistance to the Transport Sector, 1995-2005.” Washington DC: Independent Evaluation Group. ____. 2007c. “IFC’s experience in the Transport Sector. Independent Evaluation Group.” Washington, DC: Independent Evaluation Group. ____. 2008. “The Welfare Impact of Rural Electrification: A Reassessment of the Costs and Benefits.” Washington, DC: Independent Evaluation Group. ____. 2008. “Using knowledge to improve development effectiveness” World Bank Economic and Sector Work and Technical Assistance, 2000–2006, Washington DC: Independent Evaluation Group ____. 2010. “Safeguards and Sustainability Policies in a Changing World: An Independent Evaluation of World Bank Group Experience.” Washington, DC: Independent Evaluation Group. ____. 2010a. “Water and Development: An Evaluation of World Bank Support 1997-2007.” Washington, DC: Independent Evaluation Group. ____. 2010b. “Annual Report 2010 Results and Performance of the World Bank Group.” Washington, DC: Independent Evaluation Group. 79 REFERENCES ____. 2011. “Overlooked Links in the Results Chain. Evaluation Brief No. 14.” Washington, DC: Independent Evaluation Group. ____. 2011. “Azerbaijan: World Bank Country-Level Engagement on Governance and Anticorruption.” IEG Working Paper 2011/11: Independent Evaluation Group. ___. 2011. “Bangladesh: World Bank Country-Level Engagement on Governance and Anticorruption.” IEG Working Paper 2011/11: Independent Evaluation Group. ____. 2011. “Cambodia: World Bank Country-Level Engagement on Governance and Anticorruption.” IEG Working Paper 2011/11: Independent Evaluation Group. ____. 2011. “Moldova: World Bank Country-Level Engagement on Governance and Anticorruption.” IEG Working Paper 2011/11: Independent Evaluation Group. ____. 2012. “Project Performance Assessment Report: Mozambique-Roads and Bridges Management and Maintenance Project Phase I and Railways and Ports Restructuring Project” Washington, DC: Independent Evaluation Group. ____. 2012. “Project Performance Assessment Report: India- Gujarat State Highway Project, Karnataka State Highways Improvement Project and Assessment of Analytic and Advisory Activity India Road Transport Service Efficiency Study” Washington, DC: Independent Evaluation Group. IFC 1998. “IFC Beyond 2000. International Finance Corporation.” Washington, DC: International Finance Corporation. ____ 1999. “IFC Strategic Directions, 2000.” Washington, DC: International Finance Corporation. IMF. 2008. “Public Investment and Public-Private Partnerships. Addressing Infrastructure Challenges and Managing Fiscal Risks.” Washington, DC: International Monetary Fund. Jeon, C. and A. Amekudzi. 2005. “Addressing Sustainability in Transport Systems: Definitions, Indicators, and Metrics,” Journal of Infrastructure Systems, March 1, 2005, Vol 11, No. 1.IEG 2006a. New Renewable Energy: A Review of the World Bank’s Assistance. Independent Evaluation Group. Washington D.C. Lee, J. and Hine, J. 2008. Preparing a National Transport Strategy Suggestions for Government Agencies in Developing Countries (Transport Paper 19). Lamech, R. and K. Saeed. 2003. “What International Investors Look for When Investing in Developing Countries: Results from a Survey of International Investors in the Power Sector.” Energy and Mining Sector Board Discussion Paper 6. Washington, DC: World Bank. Lopez H. 2008. “Pro-poor Growth: a review of what we know (and of What we Don’t).” Washington DC: World Bank. MIGA, 2000. “MIGA Review 2000. Multilateral Investment Guarantee Agency.” Washington DC: MIGA. OECD. 2004. “Accelerating Pro-poor Growth Through Support For Private Sector Development: An Analytical Framework. Paris: Organisation for Economic Co-operation and Development. ____. 2006. “Promoting Pro-poor Growth: Key policy messages. Organization for Economic Cooperation and Development.” Paris: Organisation for Economic Co-operation and Development. OED. 1985. “Sustainability of Projects: First Review of Experience.” Operations Evaluation Department. Washington, DC: World Bank ____. 2002. “Power for Development A Review of the World Bank Group’s Experience with Private Participation in the Electricity Sector.” Operations Evaluation Department. Washington DC: World Bank. Potter. B. 1997. “Dedicated Road Funds: A preliminary View on a World Bank Initiative.” Working Paper 97/7. International Monetary Fund, Washington DC 80 REFERENCES Saeed, M. 2006. “Durability and Sustainability of Infrastructure—a State-of-the-Art Report.” Canadian Journal of Civil Engineering. Canada: NRC Research Press Silva M and Liautaud G. 2011. “Performance-based Road Rehabilitation and Maintenance Contracts (CREMA) in Argentina. A review of fifteen years of experiences (1996-2010). World Bank UNDP. 2005. “Making Infrastructure Work for the Poor.” New York: United Nations Development Programme. Valdivia, M. 2010.ntracting the road to development: Early impacts of a rural roads program”, Poverty & Economic Policy Research Network WDR. 1994. “World Development Report: 1994, Infrastructure for Development.” Washington DC: World Bank. World Bank. 2002. “Private participation in Infrastructure: trends in Developing Countries in 1990-2001.” Washington, DC: World Bank. ____. 2003a. “Infrastructure Action Plan.” Washington, DC: World Bank. ____. 2003b. “What International Investors Look For When Investing In Developing Countries.” Washington, DC: World Bank. ____. 2005. “Natural Disaster Hotspots: A Global Risk Analysis.” Washington, DC: World Bank. ____. 2006. “Infrastructure: Lessons from the Last Two Decades of World Bank Engagement.” Washington, DC: World Bank. ____. 2008a. “Safe, Clean, and Affordable Transport for Development—The World Bank Group’s Transport Business Strategy 2008-2012.” Washington, DC: World Bank. ____. 2008b. “Sustainable Infrastructure Action Plan Fiscal Years 2009-2011.” Washington, DC: World Bank. ____. 2009a. “Progress Report Fiscal Year 2009: Infrastructure Recovery and Assets Platform, Sustainable Infrastructure Action Plan.” Washington, DC: World Bank. ____. 2009b. Recent trends in Private Activity in Infrastructure. Washington D.C. ____. 1988. “Road Deterioration in Developing Countries. Causes and Remedies. A World Bank Policy Study.” Washington, DC: World Bank. ____. 1996. Sustainable Development: Priorities for Policy Reform. Development in Practice Series. Washington DC: World Bank. ____. 2002. “Cities on the Move: A World Bank Urban Transport Strategy Review.” Washington DC: World Bank. ____. 2003. “Private Sector Advisory Service.” Washington, DC: World Bank. ____. 2011. “Africa Infrastructure Country Diagnostics.” Washington, DC: World Bank. ____. 2012. “Transformation through infrastructure” World Bank Group Infrastructure Update 2012-2015. Washington, DC: World Bank. ____. 2011. "Transport and the Millennium Development Goals (MDGs).” www.worldbank.org ____. 2005. Harmonized Evaluation Criteria for ICR and OED Evaluations, World Bank ____. 2008. Road User Charges: Current Practice and Perspectives in Central and Eastern Europe, Transport Sector Board, The World Bank Group ____. 2010. A Review of Institutional Arrangements for Road Asset Management: Lessons for the Developing World, Transport Sector Board, The World Bank Group 81 REFERENCES ____. 2011. Railway Reform: Toolkit for improving rail sector performance, World Bank 82 Appendix A: World Bank Group Infrastructure and Transport Timeline Year Strategy/Document/Event A Brief Description 1947 First World Bank Loan First loan was issued to France for $250 million to rebuild infrastructure was approved by the Executive Directors on May 9, 1947. 1972 World Bank Transport Sector A major review and forward-looking statement noted the Working Paper preponderance of World Bank assistance for financing physical assets, and proposed more emphasis on policy, management, and institutional development. It advocated a sector, rather than a modal or project-by-project approach, and in this overall context, proposed priorities under different transport modes: Roads: “feeder” roads, local civil works contracting capacity, technical assistance for design standards, road transport services; Railways: rehabilitation, reform and retreat from over-extended systems; Ports and shipping: coastal and ocean shipping, containerization; Aviation: financing of airports and navigational aids; Urban transport: mass transit systems, urban roads, traffic control systems, terminal facilities. 1983 World Bank Sector Support The Sector Support Strategy Paper focused on improving the Strategy Paper— efficiency of the transport sector through encouraging innovation, Transportation adopting pricing structures that reflected real resource and social costs, and developing appropriate regulatory environments. 1987 Sub-Saharan Africa Transport The SSATP was established in 1987 as an international partnership Policy Program (SSATP) of 35 Sub-Saharan African countries and the Regional Economic Communities of Sub-Saharan Africa, through funding from 11 donor agencies—the European Commission, Sweden, Denmark, Norway, Ireland, United Kingdom, France, African Development Bank, Islamic Development Bank, United Nations Economic Commission for Africa, and the World Bank. The SSATP was established to promote the development and implementation of sound transport sector policies and strategies, through and with transport sector professionals in Sub-Saharan Africa, in support of sustainable economic growth and poverty reduction among its partner countries. 1988 World Bank Sector Strategy The Sector Strategy Paper for Transportation devoted greater Paper on Transportation attention to institutional issues as well as public-private initiatives. Among institutional issues, it addressed sector management at all levels—fiscal and regulatory policies at the government level; physical assets at the level of the sector ministry; and operational and commercial management at the level of the transport operator. It proposed an appropriate mix of public and private initiatives and a policy environment in which transport could effectively respond to demand. In this context, the paper recognized the changing role of government and the Bank’s prospective role in helping harness these changes for a sustained and orderly development of the sector. 83 APPENDIX A WORLD BANK GROUP INFRASTRUCTURE AND TRANSPORT TIMELINE Year Strategy/Document/Event A Brief Description 1988 Road Deterioration in The Bank largely reoriented its highway sector lending and technical Developing Countries— assistance and intensified its efforts to help its clients strengthen road Causes and Remedies asset maintenance. The report concluded that effective road asset maintenance required five key actions: (i) a coalition of private and public stakeholders asserts ‘ownership’ of the road asset development and preservation functions. (ii) road user costs and life-cycle analysis of road agency costs are both taken into account in public policy decisions concerning road development and management. (iii) owners of the road network set up adequate funding mechanisms to sustain maintenance and capital renewal. (iv) road maintenance and renewal works would be carried out by bodies separate from the public road planning and administration bodies, contracted out in competitively tendered contracts; and (v) the more broadly engaged owners of the road network establish strong accountability and incentives aligned with the public interest among road administration, funding sources, planners, executing agencies and contractors. To support accountability, quantitative performance measures are set and monitored regularly. 1994 World Development Report: This report propelled the Bank toward greater use of the private Infrastructure for sector in infrastructure. It concluded that many developing countries Development would benefit through economic growth and poverty reduction if incentives to providers were clarified and strengthened. This could be achieved by giving management more autonomy and focusing accountability on service to customers; by structuring the sectors and relevant regulation in a manner to promote effective competition; and by giving users and other stakeholders more voice and responsibility in planning and regulatory arrangements. Governments would concentrate on creating and maintaining legal and regulatory frameworks to attract private providers. At the same time, they would safeguard the interests of the poor, improve environmental conditions, and coordinate cross-sector interactions. In urging consideration of increased reliance on private sector providers, the report also recognized that private involvement would inevitably grow at substantially different rates in different countries. Those rates depend on private capacities, the ability of government to provide an appropriate regulatory framework, the performance of public sector providers, and political consensus in favor of private provision. 84 APPENDIX A WORLD BANK GROUP INFRASTRUCTURE AND TRANSPORT TIMELINE Year Strategy/Document/Event A Brief Description 1996 Sustainable Transport: The ideas in the World Development Report were expanded on in this Priorities for Policy Reform paper, which found that the dominance of the public sector in the operation of transport services had in some cases resulted in adverse effects, including high costs, lack of flexibility in service provision and assets not being properly maintained to redress these shortcomings the report advocated the Bank to encourage substantial changes in the role of government in transport, reducing its function as a supplier but increasing its function as a regulator. It recognized that governments need to create a proper institutional framework for competition, set economically efficient charges for the use of publicly owned transport infrastructure, and increase community participation in decision making. The theme of “sustainability” was comprehensive and intended to include economic, financial, environmental, and social sustainability. 1996 Cities of Tomorrow Cities for Tomorrow (prepared by the World Bank’s Infrastructure Group, Urban Development) highlighted the issue of demand for vehicle road space outpacing availability, and structured densification of land use, urging greater use of public transport, with due regard to cleaner transport technologies. 1998 Commercial Management and The report provided guidance on tackling the road maintenance Financing of Roads problem by introducing market discipline, in the form of competition that motivates road agency managers to cut waste, improve operational performance, and allocate resources efficiently. The strategic mechanism for promoting competition is commercialization: bring roads into the marketplace, put them on a fee for service basis, and manage them like a business 2000 Cities in Transition: World “Cities in Transition” (prepared by World Bank’s Infrastructure Group, Bank Urban and Local Urban Development) proposed looking at transport in the context of Government Strategy overall urban issues, while intervening selectively, investing widely in urban knowledge generation, dissemination, and capacity building; and working through strengthened public and private partnerships at local, national, and international levels. 2002 Cities on the Move The Bank’s urban transport strategy review concentrated on the social exclusion aspects associated with urban inaccessibility. Its objectives were to develop a better understanding of urban transport problems and to articulate a strategic framework that could be applied in developing countries. The report recognized that urban growth increases transport costs: Economies of agglomeration generate the growth of cities. As cities grow and become richer, vehicle ownership and use grow more rapidly than the available road space, resulting in increased congestion and traffic-generated air pollution. 2003 Infrastructure Action Plan, The Infrastructure Action Plan revitalized the Bank’s commitment to FY04-07 infrastructure in support of economic growth. 85 APPENDIX A WORLD BANK GROUP INFRASTRUCTURE AND TRANSPORT TIMELINE Year Strategy/Document/Event A Brief Description 2004 Public and Private Sector The policy note provides a framework for identifying and assessing Roles in the Supply of the suitability of different models for public and private roles in the Transport Infrastructure and transport sector for various transport modes, including policy and Services: Operational regulatory issues. It summarizes the range of instruments available to Guidance for World Bank the Bank Group for supporting particular models involving transport Staff and logistics as a whole. It also emphasizes that the Bank’s interventions, involving either public, private or both sectors, need to maintain or increase transport access and affordability for the poor; ensure “fiscal space” in the country concerned for public infrastructure loans and other contingent liabilities that may be created by public- private partnerships; promote safety and security in all transport systems; and support environmental and other safeguards. 2008 Safe, Clean, and Affordable The objective of the Bank Group’s transport strategy (which is Transport for Development: consistent with Sustainable Transport) was to help partner countries The World Bank Group’s to establish the governance, strategies, policies and services that will bTransport Business Strategy deliver transport for development in a way that is economically, for 2008-12 financially, environmentally and socially sustainable. The Bank Group would pursue five strategic directions in the next five years: (1) Create the conditions for increased support for transport investment and governance;(2) deepen engagement in the roads and highways subsector; (3) increase engagement in the urban transport subsector; (4) diversify engagement in transport for trade; and (5) transport and climate change: control emissions and mitigate impact. To achieve its objective and the five strategic directions, the Bank Group intended to adjust the way it does business in four main ways: (1) Increase the proportion of Bank Group transport lending made through program approaches; (2) enhance the quality of policy dialogue and sharing of transport knowledge; (3) improve monitoring and evaluation; and (4) capture more synergies across sectors and Bank Group instruments. 2008 Sustainable Infrastructure The Sustainable Infrastructure Action Plan supports a renewed Action Plan, FY08-11 commitment to client countries to improve the reach and quality of infrastructure service delivery in a sustainable manner through increased financing and leverage. Sustainable Infrastructure Action Plan follows up on the World Bank’s Infrastructure Action Plan, FY04- 07, to revitalize the institution’s engagement in infrastructure. The Transport Business Strategy is consistent with the overall goals of the Sustainable Infrastructure Action Plan. 86 APPENDIX A WORLD BANK GROUP INFRASTRUCTURE AND TRANSPORT TIMELINE Year Strategy/Document/Event A Brief Description 2011 Maintaining Road Assets: A The paper (sponsored by UK’s DFID and the World Bank) examined fresh look at the World Bank’s how far the prescriptions offered in 1988 are still valid today, and to 1988 Policy Paper “Road what extent recent developments warrant a rethinking of the Deterioration in Developing recommended approach. Key findings are that the core logic favoring Countries” outsourcing to contractors still holds, to create improved incentives and managerial flexibility supporting minimization of total life-cycle costs. However, contract structure, risk allocation, duration, and governance environment can greatly affect performance. Longer-term contracts can be structured to reward sustained performance and facilitate the requisite funding, but vigilance against corruption is a constant requirement. Modern information and communications technologies offer much promise in supporting greater transparency and accountability. Ultimately, the commitment of the nation’s leaders, public and private, to the integrity of the systems will remain a fundamental determinant of their effectiveness—and the condition of the nation’s roads. 2011 Transformation through This paper updates the Sustainable Infrastructure Action Plan, FY08- Infrastructure—the World 11 as the Group’s strategy for infrastructure. Bank Group Infrastructure The strategy update rests on three pillars—core engagement, Strategy Update, FY12-15 transformational engagement and mobilization of private capital and other sources. “Core engagement” in infrastructure consists of sector- based interventions (physical infrastructure and institutional strengthening) to support access to basic infrastructure services and growth (Pillar 1). This represents 80 percent of the Group’s infrastructure portfolio: it is the bedrock of the Group’s involvement in infrastructure. Building on this strength, the update proposes innovation and increased impact in two areas: transformational engagements (Pillar 2) and mobilization of private capital and other sources (Pillar 3). 87 Appendix B: Overview of the World Bank Group Transport Portfolio The World Bank Group approved 577 lending operations and guarantees in the transport sector for a total value of $50 billion during FY02-11. This corresponds to about 12 percent of the Bank Group’s total commitments and guarantee volumes. Composition and Trends of World Bank Group Transport Sector Projects IBRD and IDA projects have accounted for the largest share of the portfolio in lending volume terms—between 76 percent (in fiscal 2008) and 94 percent (in fiscal 2004) of the total. IFC has accounted for 4 percent (in fiscal 2003) and 13 percent (fiscal 2008) of the portfolio, and MIGA has had the smallest share with a peak in fiscal 2008 at 11 percent (Figure B.1). Figure B.1. World Bank Group Transport Commitments/Guarantees (US$ million) 8000 6000 4000 2000 0 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 IBRD and IDA Transport Commitments IBRD Transport Commitments IDA Transport Commitments IFC Investments Total WB Commitments Source: World Bank, IFC, and MIGA data. The average project size has doubled from $65 million in fiscal 2002 to $130 million in FY11, a trend that has been driven by the World Bank portfolio (doubling from $80 million to $160 million). IFC projects doubled in size in low-income countries, but not overall (Figure B.2). In total, 116 countries received World Bank Group lending support in the transport sector during fiscal 2002-11. More than a quarter of World Bank lending during this period went to India, China, and Brazil. These three countries have been in the top five 89 APPENDIX B OVERVIEW OF THE TRANSPORT PORTFOLIO by volume of lending for the past decade. The top two borrowers combined account for 13 percent of all lending in the transport sector by number of projects and for 29 percent of the total lending volume. For IFC transport investments, the top three countries are Russia, Panama, and Brazil, accounting for 26 percent of all IFC transport investment and 35 percent of volume in the sector. Of these three, only Brazil has been in the top three by volume for the past decade. The majority of transport investment was in Russia —22 projects, mostly in railways and aviation. Figure B.2. Support to the Transport Sector FY02-11: Top Ten Countries by Lending Volume World Bank IFC The top ten countries account for 58% The top ten countries account for 64% of of Total WB Lending Volume Total Lending Volume % of IFC Commitments % of World Bank Commitments 20% 20% 15% 15% 10% 10% 5% 5% 0% 0% Source: World Bank and IFC data. Of the six regions covered, the largest World Bank projects are concentrated in South Asia and Latin America and the Caribbean (Figure B.3). A few of these projects are very large, particularly in India and Brazil (Figure B.4). A few extremely large transport projects have been approved in recent years —South West Roads Project in Kazakhstan ($2.1 billion in fiscal 2009), the Pradhan Mantri Gram Sadak Yojna Rural Roads Project in India ($1.5 billion in fiscal 2011), and the Padma Bridge Project in Bangladesh ($1.2 billion in fiscal 2011)—but such large projects are exceptional. The transport portfolios of IFC and MIGA show similar regional concentrations. The Most prominent region is Latin America and the Caribbean, which accounts for 39 percent of IFC and 40 percent of MIGA transport projects. In the Europe Central Asia Region, IFC accounts for 26 percent and MIGA for 27 percent of transport projects. 90 APPENDIX B OVERVIEW OF THE TRANSPORT PORTFOLIO Figure B.3. Distribution of World Bank and IFC Projects by Region Distribution of World Bank Projects by Distribution of IFC Projects by Region Region 50% 30% 40% 25% 20% 30% 15% 20% 10% 5% 10% 0% 0% AFR EAP ECA LCR MNA SAR AFR EAP ECA LCR MNA SAR FY02-07 FY08-11 FY02-07 FY08-11 Source: World Bank and IFC data Excluding DPOs, the largest World Bank projects are found in the highways and railways subsectors. The largest projects mapped to Sectors other than transport, with at least 30 percent of commitment for transport are the Framework for Green Growth DPO in Mexico ($1.5 billion in fiscal 2010) and the Second Programmatic Loan for Sustainable and Equitable Growth DPO in Brazil ($600 million in fiscal 2006). Figure B.4. Average Project Size Average Project Size, US$ million 350 300 250 200 150 100 50 0 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 Roads & Highways Railways Ports Aviation Source: World Bank data. World Bank lending volumes for transport are dominated by projects mapped to the Transport Sector (Figure B.5). Fifty percent of the projects reviewed and 79 percent of related lending volumes are managed by the Transport Sector. In contrast, 26 percent by number and 10 percent by lending volume are managed by other sectors. Thus, average 91 APPENDIX B OVERVIEW OF THE TRANSPORT PORTFOLIO transport-related lending in projects managed by other sectors is smaller (about $30 million per projects) in comparison to projects managed by the Transport Sector ($ 130 million). This is mainly because transport is only one of the several main interventions in projects managed by other Sectors. Figure B.5. Trends in number of World Bank Projects managed by the Transport Sector and Other Sectors by Fiscal Year of Approval, and Subsector Trends in World Bank Number of Share of Projects managed by the Projects Transport Sector, Subsector FY 02-11 by Fiscal Year of Approval 100% 80% 80 Number of Projects 60% 60 40% 40 20% 20 0% 0 2002200320042005200620072008200920102011 World Bank (No. of Projects) WB Managed by Transport Sector (No.) Managed by the Transport Sector WB Managed by Other Sectors (No.) Managed by other Sectors Source: World Bank data. Most of the World Bank projects that are not managed by the Transport Sector are mapped to the Urban Development and Agriculture Sectors. The trend shows slower growth for projects with transport components but managed by other than for projects managed by the Transport Sector. The number of projects approved annually with transport components managed by other sectors increased from about 15 in 2002 to around 20 in 2011. This recent trend has been relatively stable since 2007. In contrast, the number of projects approved annually and projects managed by the Transport Sector increased from about 20 in 2002 to 50 in 2011. Looking across the transport subsectors, aviation has a higher share of projects managed by other sectors (48 percent), followed by urban transport (44 percent); railways has the smallest share managed by other sectors (16 percent). 92 APPENDIX B OVERVIEW OF THE TRANSPORT PORTFOLIO Figure B.6. Composition of Transport Lending Portfolio by Number of Projects and by Volume Composition of Transport Lending Portfolio Composition of Transport Lending (by number of Projects) Portfolio (by Volume) MIGA Gross MIGA Exposure 3% IFC IFC 22% WB Commitm Managed ents by 8% Transport Sector WB WB 50% WB Managed Managed Managed by by Other by Other Transport Sectors Sectors Sector 26% 10% 79% Source: World Bank data. Transport investments by IFC and guarantees by MIGA tend to support projects in richer countries, while the World Bank’s support is more evenly spread across the income groups (Figure B.7). The numbers of projects supported by IDA and IBRD are evenly divided between upper-middle-income countries and lower-middle-income countries at about 35–38 percent each, while the low-income countries receive 22 percent of the total projects. For IFC, the majority of the projects are in upper-middle- income countries (62 percent), followed by lower-middle-income (22 percent) and low- income (8 percent) countries. MIGA’s transport guarantees show a similar pattern to that of IFC with upper-middle-income countries predominating (80 percent) followed by lower-middle-income (13 percent) and low-income (7 percent) countries. IFC’s transport investments follow global foreign direct investment flows, but their share is “under-weighted” compared to the overall IFC portfolio. About 16 percent of IFC-supported transport investment projects (by commitment volume in U.S. dollars) were in IDA countries during fiscal 2002-11. Comparing this to a foreign direct investment net inflow of 16 percent (again by volume) into IDA countries, this indicates that IFC’s transport portfolio is “neutrally” weighted in IDA countries. But it indicates also that IFC’s transport portfolio is under-weight in IDA countries compared to the overall portfolio, in which 23 percent of total investment volume went to IDA countries. 93 APPENDIX B OVERVIEW OF THE TRANSPORT PORTFOLIO Figure B.7.Distribution of World Bank and IFC Projects Approved by FY and, by Country Income World Bank IFC High income High income Upper middle income Upper middle income Lower middle income Lower middle income Low income Low income 0% 10% 20% 30% 40% 50% 0% 20% 40% 60% 80% FY08-11 FY02-07 FY08-11 FY02-07 Source: World Bank and IFC data. The direction of transport lending changed after the global economic crisis in 2008, 1 in favor of lower-middle-income countries. Breaking the evaluation period into two periods, before and after the end of June 2008, the two periods coincide with the development of the 2007-15 World Bank Infrastructure Strategy which was approved in fiscal 2008. Since then, the number of World Bank projects has increased for low-income and lower-middle-income countries. Similarly, IFC investments have increased substantially for lower-middle-income countries. In terms of World Bank lending instruments, DPOs have increased substantially since the crisis (Figure B.8). Specific Investment Loans (SILs) dominate among the World Bank projects managed by Transport sector, but the majority of projects managed by other sectors are DPOs. Out of the 437 operations supported by the World Bank in the transport sector, 53 percent are SILs, followed by Emergency Recovery Loans (ERL), Adaptable Program Loans (APL), and Sector Investment and Maintenance Loans (SIM). There were 11 closed and 1 active DPOs with more than 30 percent share of commitments in the transport sector. All but one (Armenia) DPO was located in the Africa region. This is an increasing trend as all 13 were approved during the past five years, between fiscal 2007 and fiscal 2011. Only five Sector-Wide Approaches (SWAps) were approved during the evaluation period in Brazil, Indonesia, and Poland, with one Partial Risk Guarantee on Trade Transport Facilitation in the Africa region. 94 APPENDIX B OVERVIEW OF THE TRANSPORT PORTFOLIO Figure B.8. Distribution of World Bank Projects by Lending Instrument and Year of Approval World Bank Projects managed by the World Bank Projects managed by Transport Sector Other Sectors 100% 100% 80% 80% 60% 60% 40% 40% 20% 20% 0% 0% APL DPL SIL TAL Other APL DPL SIL TAL Other FY02-07 FY08-11 FY02-07 FY08-11 Source: World Bank data. In recent years, there has been a sharp increase in the variety and number of recipient- executed grant programs overseen by the World Bank but financed by external sources. Examples include the Asian Financial Crisis Response Trust Fund and the International Reconstruction Fund Facility for Iraq. The Institutional Development Fund figured most prominently with 13 operations, of which 31 percent each were in Africa, EAP, and ECA, and 8 percent were in LAC. The Global Environment Facility supported 10 operations, half of which were in LAC. Other grants include Carbon Offset projects (Table B.1) Table B.1. World Bank Projects by Product Lines Number of projects Percentage IBRD/IDA 366 84.3% Recipient Executed Activities 38 8.8% Institutional Development Fund (IDF) 13 3.0% Global Environment Facility 10 2.3% Carbon Offset 3 0.7% Special Financing 3 0.7% Guarantees 1 0.2% Total 434 100.0% Source: World Bank data. IFC’s transport sector portfolio is dominated by loans. Out of the 121 transport investments supported by IFC during the period fiscal 2002-11, 87 percent were loans, 6 95 APPENDIX B OVERVIEW OF THE TRANSPORT PORTFOLIO percent were equity investments, 4 percent were a combination of loan and equity, and 3 percent were guarantees.(Table B.2). MIGA supported 14 new guarantee projects in the transport sector. Political risk insurance coverage against expropriation was the most frequently found coverage type (86 percent), followed by coverage against transfer restrictions (71 percent), war and civil disturbance (64 percent), breach of contract (43 percent), and to a lesser extent non- honoring of sovereign financial obligation (7 percent). Table B.2. IFC Projects by Instrument Number of projects Percentage Loan 105 87% Equity 7 6% Loan and equity 5 4% Guarantee 4 3% Total 121 100% Source: IFC data. Roads and highways (including intercity highways, rural roads, and urban transport, according to the World Bank project database) account for the largest share in the transport sector both by the number of projects and by lending volume, although it has been declining over time (Figures B.9 and B.10). Figure B.9. World Bank Group: Number of Transport Projects by Subsector (percent) 80% 70% 60% 50% 40% 30% 20% 10% 0% 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 Roads & Highways Railways Ports, IWW &TTF Aviation Source: World Bank and IFC data. Each sector in multi-sector projects is counted as one project. 96 APPENDIX B OVERVIEW OF THE TRANSPORT PORTFOLIO Figure B.10. World Bank Group: Transport Lending Commitments by Subsector (percent) 80% 70% 60% 50% 40% 30% 20% 10% 0% 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 Roads & Highways Railways Ports, IWW &TTF Aviation Source: World Bank and IFC data. Breaking the roads and highways subsector into urban transport and rural roads, the urban transport portfolio has increased, while the focus on rural roads has declined (Figure B.11). Figure B.11. Number of Highways, Urban Transport and Rural Roads Projects (share of all transport projects) 50% 45% 40% 35% 30% 25% 20% 15% 10% 5% 0% 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 Highways Urban Transport Rural Roads Source: World Bank and IFC data. Each sector in multi-sector projects is counted as one project. 97 APPENDIX B OVERVIEW OF THE TRANSPORT PORTFOLIO Trends of Analytic and Advisory Activities in Transport managed by Transport and Other Sectors Figure B.12. Trends in AAA Managed by Transport and Other Sectors, FY 02-11 No. of projects 120 100 80 60 40 20 0 FY02 FY03 FY04 FY05 FY06 FY07 FY08 FY09 FY10 FY11 Managed by Transport Sector Managed by Other Sectors Source: World Bank and IFC data. Performance of World Bank Group Transport Sector Projects World Bank projects that are managed by the Transport Sector and were completed fiscal 2002-11 have 87 percent moderately satisfactory or better IEG outcome ratings 2— one of the highest in the Bank portfolio. Project outcome rating is defined as “the extent to which the operation’s major relevant objectives were achieved, or are expected to be achieved, efficiently.” These project outcome ratings have declined in recent years, however. In fiscal 2002, 100 percent of the exiting projects managed by to the Transport Sector had their outcomes rated as moderately satisfactory or better, but the average for the past five years dropped to 83 percent. For projects with transport components that are managed by other sectors, a similar declining trend was observed for the outcome rating. Thirty-nine percent of the operations that closed during fiscal 2007-11 have significant risks to development outcomes. Risk to development outcome is the risk, at the time of evaluation, that development outcomes or expected outcomes will not be maintained or realized. 3 The IEG Risk to development outcome rating for the projects mapped to the Transport Sector and closed between fiscal 2007-11 showed 16 percent negligible or low, 47 percent moderate, 29 percent significant, and 9 percent high. 4 98 APPENDIX B OVERVIEW OF THE TRANSPORT PORTFOLIO IFC projects have the highest IEG development outcome ratings compared to IFC’s overall portfolio and other infrastructure projects 77 percent of the 47 investment projects evaluated between fiscal 2002 and fiscal 2011 5 had IFC development outcome ratings of moderately satisfactory or better, which is higher than the 65 percent figure for IFC as a whole or the 71 percent of other IFC infrastructure projects. Moreover, the evaluated transport projects have yielded better than average impacts in all four underlying indicators of development outcome.23 However, this trend has become less pronounced in recent years where transport projects’ development outcome ratings are on par with the portfolio overall. The few evaluated outcomes of Advisory Services in the transport sector have been low: two of the five transport Advisory Service projects evaluated between fiscal 2008 and fiscal 2010 6 had satisfactory ratings on development effectiveness. 99 Appendix C: Portfolio Review and Field- and Desk-Based Country Study Methodologies Portfolio Review Methodology Two levels of desk-based portfolio review were carried out. The first-level portfolio review identified and categorized the basic information, characteristics, objectives, and components of all of the following operations approved by the World Bank Group Board of Executive Directors from fiscal 2002-11: 287 World Bank (IDA and IBRD) projects managed by the Transport Sector 150 projects managed by other Sectors with more than 30 percent shares of commitments in the transport sector and all Development Policy Loans; 1,150 World Bank non-lending activities, including analytical and advisory activities (AAA) with elements of transport; 121 IFC investments and 54 IFC advisory services; and 14 MIGA guarantees. Measures that are expected to or aimed to help sustain transport, including policies, regulatory and institutional framework, capacity building activities, and financial arrangements have been identified for each transport subsector. The second-level portfolio review assessed the intermediate outcomes a subsample of the approved projects at the time of project closure for World Bank projects and operational maturity for IFC and MIGA projects. The review also assessed the effectiveness of the specific measures to help sustain transport at the time of project evaluation. Since the World Bank Group uses different methodologies 1 for reviewing the projects supported by IBRD/IDA, IFC, and MIGA, and since the standard IEG reviews are objectives-based, a criteria-based assessment method has been developed to evaluate the results achieved through World Bank Group–supported activities. World Bank non-lending support has been reviewed only in the context of desk and field based country case study projects, with focus on the activities for which there was evidence of significant impact on sustained transport. In the context of the desk and field based country studies, additional projects that were approved before fiscal 2002 but that closed or reached operational maturity during fiscal 2002-11 were also reviewed. 101 APPENDIX C PORTFOLIO REVIEW AND FIELD- AND DESK-BASED COUNTRY STUDY METHODOLOGIES Table C.1. Coverage of Transport Portfolio Review All approved Of which, projects closed projects Ongoing/ recently approved (World Bank) or operationally during FY02- projects as of FY11 matured (IFC/MIGA) 11 Coverage of Lending Operations IBRD/ IDA Projects managed by 287 78 209 Operations Transport Sector Projects managed by Other Sectors with 150 80 70 >30% share of transport IFC investments 121 61 60 MIGA guarantees 15 6 9 Total number of projects 573 225 348 Coverage of World Bank AAA and IFC Advisory Services All Completed Ongoing IBRD/ IDA AAA managed by 322 256 66 Transport Sector AAA managed by 828 675 153 Other Sectors IFC Advisory services 54 33 21 Total number of AAA and AS 1,204 964 240 Source: World Bank and IEG Evaluation Team. Table C.2. Coverage of Desk and Field Based Country Case Study Projects and PPARs Total projects covered Additional Field Based Case Desk Based Case in country studies and projects in Study Projects Study projects PPARs PPARs Projects IBRD/ managed by 77 34 21 22 IDA Transport Sector Projects managed by Other Sectors 13 8 5 0 with >30% share of transport IFC investments 31 27 4 N.A. MIGA guarantees 1 1 0 N.A. Total number of projects 122 70 30 22 Source: World Bank and IEG Evaluation Team. Note: The total number of PPARs is 30 of which eight are already included in the desk and field based country case study projects. 102 APPENDIX C PORTFOLIO REVIEW AND FIELD- AND DESK-BASED COUNTRY STUDY METHODOLOGIES A review of World Bank Group sector and country assistance strategies was carried out to assess to what extent the sustained transport has been reflected. Field- and Desk-Based Country Study Selection To assess the extent to which the World Bank supported projects have sustained transport beyond project completion, field based assessments were carried out for 42 projects in nine countries, and desk based assessments were carried out for 26 projects in eleven countries. 2 The time elapsed between project completion and the field visit ranges between one and seven years. Of the 107 countries with a population over one million and at least one World Bank/IFC/MIGA project that has been approved, closed (IBRD/IDA projects), or operationally matured (IFC/MIGA projects) between fiscal 2002 and FY11, 62 countries have received World Bank Group support in at least two transport subsectors. Of these 62 countries, 20 were selected randomly while maintaining the regional representation, 3 as summarized in Table C.2. Table C.3. Regional Representation Total number of World Bank Group Countries selected Countries selected countries >1 support in at least Regions for field-based for desk-based million population two transport studies review transport lending Subsectors Sub-Saharan Africa 34 16 (26%) 3 3 East Asia and the 11 9 (15%) 1 2 Pacific Europe and Central 25 10 (16%) 2 1 Asia Latin America and 21 13 (21%) 2 2 the Caribbean Middle East and 10 8 (13%) 0 2 North Africa South Asia Region 6 6 (10%) 1 1 Total 107 62 (100%) 9 11 Source: IEG Evaluation Team Of the 20 countries selected, six were subsequently selected for field visits based on the income level representation and the degree of IFC/MIGA interventions, and to obtain a mix of small and large countries. Three additional field-based reviews were carried out in conjunction with PPAR missions to India, Mozambique, and Poland. Table C.2 summarizes the characteristics of the selected countries; a list of projects reviewed as part of the country studies is attached in Appendix E and additional projects that were approved before fiscal 2002-11 but closed or operationally matured during fiscal 2002- 11 were also reviewed in the country studies. 103 APPENDIX C PORTFOLIO REVIEW AND FIELD- AND DESK-BASED COUNTRY STUDY METHODOLOGIES Table C.4. Countries with Field or Desk Reviews Completed or operationally Active/Ongoing (# Countries with Country Income Field Region matured (# of of transport) IFC/MIGA name level studies 4 transport subsectors involvement subsectors) AFR Mozambique LIC 4 2 X X AFR Nigeria LMIC 4 2 X X AFR Senegal LMIC 3 4 X X AFR Tanzania LIC 3 4 X AFR Uganda LIC 2 2 X AFR Zambia LMIC 3 2 EAP Indonesia LMIC 3 5 X X EAP Mongolia LMIC 3 1 Papua New LMIC 2 1 EAP Guinea ECA Poland UMIC 2 1 X Russian UMIC 5 4 X X ECA Federation ECA Turkey UMIC 2 3 X LCR Bolivia LMIC 3 2 X LCR Chile UMIC 5 2 X X LCR Peru UMIC 5 4 X X LCR Honduras LMIC 2 1 X MNA Tunisia UMIC 4 2 X Yemen LMIC 2 3 MNA Republic SAR India UMIC 4 4 X X SAR Sri Lanka LMIC 3 1 X Source: IEG evaluation team. Review of Project Performance Assessment Reports To triangulate the results of desk and field based country case study project assessments, 30 project performance assessment reports (PPARs) prepared by IEG during the past five years have also been reviewed. Of the thirty PPARs, nine (30 percent) each were in upper middle income, lower middle income, and low income countries, and three (10 percent) were in high income countries. Twenty projects (20 percent) were in intercity highways, eight projects (27 percent) were in urban transport, and the remaining two (3 percent) were in ports subsector. 104 APPENDIX C PORTFOLIO REVIEW AND FIELD- AND DESK-BASED COUNTRY STUDY METHODOLOGIES Methodology Used for Assessing Effectiveness in Sustaining Transport The analytical framework used for this evaluation is shown in the results chain in Figure 1.3 of Chapter 1. It also illustrates the complementary roles that the three institutions play in the transport sector, as well as the results chain for the evaluation. This evaluation aims to assess the Bank’s effectiveness in assisting the countries and IFC and MIGA’s effectiveness in supporting the private sector investments in; achieve the outcomes in the results chain, that is, sustained transport. ASSESSMENT OF OUTPUTS Outputs consist of the physical investments and four broadly grouped measures to help sustain transport: sector policy and regulatory frameworks; institutional frameworks; sector management capacity; and financial arrangements. Physical investments in transport infrastructure are the largest outputs in volume of commitments across World Bank, IFC, and MIGA–supported projects, and are typically grouped into support for upgrading, rehabilitation, maintenance, disaster and post-conflict reconstruction, as well as new construction. The four categories of measures to help sustain transport are predominantly supported by the World Bank through its lending and non-lending operations, and in reviewing individual World Bank operation these measures were identified. Examples on the sorts of measures to help sustain transport that can be expected in the six transport subsectors are listed above. URBAN TRANSPORT Policy/Regulatory: Framework for the concessions of public transport services, strengthening public transport cost recovery and subsidy policies, demand management measures to reduce traffic congestion. Institutional Framework: Metropolitan Transit Authority to coordinate overall urban transport system, on-street and off-street parking management and control, transport/land-use planning agency. Capacity Building: Transportation planning studies, traffic management training, bus route franchising and administration, road safety programs, development of bus rapid transit or metro systems. Financial Mechanisms: Electronic toll collection systems, parking concession revenues, route licensing fees, BOT schemes, advertising revenues, municipal finance arrangements 105 APPENDIX C PORTFOLIO REVIEW AND FIELD- AND DESK-BASED COUNTRY STUDY METHODOLOGIES RURAL ROADS Policy/Regulatory: Rural connectivity programs to ensure sustainable all-weather access, poverty targeted rural infrastructure development, attainment of MDGs and access to social services. Institutional Framework: Decentralization to local government, centralized technical support for planning, designs and standards, community driven development, workfare programs Capacity Building: Rural infrastructure planning consultative mechanisms, land acquisition and resettlement, quality control and contract management, maintenance planning and budgeting Financial Mechanisms: Local government budget planning and control, cess and toll revenue mobilization, revenue from local markets. INTERCITY HIGHWAYS Policy/Regulatory: Build, Operate, and Transfer toll roads, axle load controls, road classification and administrative control, design standards, Institutional Framework: Road agency/board/fund, road safety administration, organization of responsibilities central versus regional road departments, safeguards and fiduciary arrangements Capacity Building: Planning and design studies, third party quality control, contract management, construction industry development, electronic tolling systems Financial Mechanisms: Road fund, BOT concessions, toll revenues, securitization of toll roads RAILWAYS Policy/Regulatory: Passenger and freight tariff policies, rationalization of loss- making services, outsourcing and labor redundancy, privatization of non-core services Institutional Framework: Restructuring to enhance management autonomy, lines of business organization and accounting systems Capacity Building: Concessioning, commercial accounting, marketing and business development, intermodal services, modern logistics management and freight forwarding Financial Mechanisms: Tariff setting and cross-subsidies, disposition of surplus assets, land redevelopment, equipment leasing schemes 106 APPENDIX C PORTFOLIO REVIEW AND FIELD- AND DESK-BASED COUNTRY STUDY METHODOLOGIES PORTS AND WATERBORNE TRANSPORT Policy/Regulatory: Port tariff regulation, competition policy for private terminals— for the market or in the market, hinterland connectivity Institutional Framework: Landlord port organization, port conservancy services privatization, port labor and stevedoring services regulation. Capacity Building: Port marketing and business development, concessioning of terminals and container yards, container logistics and document handling systems Financial Mechanisms: Concessioning contracts, competitive port tariff setting AIR TRANSPORT Policy/Regulatory: Competition/ open skies policies, airport terminal concessioning, privatization of aviation services, airline deregulation Institutional Framework: Air traffic control, airports authorities, civil aviation authority. Capacity Building: Airport/ navigational aids, concessioning of terminals, safety emergency services Financial Mechanisms: Competitive airport and air route tariff setting ASSESSMENT OF INTERMEDIATE OUTCOMES AND OUTCOMES The outcome of sustained transport services is expected to be achieved through intermediate outcomes, which are to sustain the institutional capability as well as the financial viability of the country’s transport sector and related institutions supported under the projects. These intermediate outcomes were assessed mainly through the country studies and review of PPARs. A template was developed for the country studies and tailored for assessing the six transport subsectors: urban transport, rural roads, intercity highways, railways, ports and waterways, and aviation. A questionnaire was designed to elicit objective, fact- based assessments, and it was completed using review of relevant project documents, including the project appraisal documents, mission aide memoires, implementation completion and results reports; and interviews with Bank Group staff as well as other relevant stakeholders in the countries (government officials, implementing agencies, beneficiaries, private sector representatives, and other donors and financiers). The questionnaire answers were transferred into spreadsheets for easy comparison between projects and countries. The results were compared and made consistent with results of the portfolio review. 107 APPENDIX C PORTFOLIO REVIEW AND FIELD- AND DESK-BASED COUNTRY STUDY METHODOLOGIES The questionnaire focused on assessing the two intermediate outcomes: institutional capability and financial viability. These two intermediate outcomes were assessed against the outcome, sustained transport, indicators of which are summarized in the table below. Table C.5. Indicators of Sustained Transport Outcomes Often Used in Projects Supported by World Bank Group Subsector Indicators Urban Transport Accessibility to transport services by population including the poor Level of congestion Average travel speed and costs Rural Roads Accessibility year round Coverage of rural population Intercity Highways Network’s physical condition Travel time and costs Accessibility Level of road safety Railways Locomotive availability Wagon turnaround time Rail traffic Ports and Waterborne Berth productivity Transport Cargo dwell times Hinterland connectivity Level of congestion Air Transport Level of congestion Air transport safety Source: World Bank project documents For each case study project review, the degree to which transport has been sustained beyond project closure was assessed on a four point scale according to whether; (i) Scale 4—the key outcome indicators achieved by the end of the project improved beyond project closure; (ii) Scale 3—the key outcome indicators either achieved or almost achieved by the end of the project were sustained beyond project closure; (iii) Scale 2— the key outcome indicators achieved by the end of the project were not sustained ; (iv) Scale 1— the key outcome indicators were not achieved by the end of the project (and have not shown improvements beyond project closure). Where there was no information available to make the above assessment, the rating was “not evaluable.” Similarly, institutional capability and financial viability were assessed. In the case of institutional capability (or management capability in the case of IFC and MIGA supported operations), the capacity of the responsible institutions was assessed using a four point scale according to the definition of this intermediate outcome indicator described below. (i) Scale 4-institutional capability established/strengthened by the end of the project improved beyond project closure; (ii) Scale 3-institutional capability established/strengthened by the end of the project was sustained beyond project 108 APPENDIX C PORTFOLIO REVIEW AND FIELD- AND DESK-BASED COUNTRY STUDY METHODOLOGIES closure; (iii) Scale 2—institutional capability established/strengthened by the end of the project was not sustained; (iv) Scale 1— institutional capability was not established/strengthened by the end of the project (and have not shown improvements beyond project closure). In the case of financial viability, assessment of operations and maintenance funding was made using a four point scale according to the definition of this intermediate outcome indicator described below. (i) Scale 4-financial viability achieved by the end of the project improved beyond project closure; (ii) Scale 3-financial viability achieved or almost achieved by the end of the project was sustained beyond project closure; (iii) Scale 2—financial viability achieved or almost achieved by the end of the project was not sustained; (iv) Scale 1— financial viability was not achieved by the end of the project (and have not shown improvements beyond project closure). Where there was no information available to make the above assessment, the rating was “not evaluable” for both intermediate outcome indicators as well. While the unit of analysis is at the project level, in many cases it is difficult to distinguish the project level outcomes from the spillover or sector-wide effects of specific measures initiated in Bank supported transport projects. The evaluation also recognizes that on the other hand, not all performance indicators can be attributed to any specific World Bank supported measures. The two intermediate outcomes are defined as follows: Institutional Capability—whether the responsible institutions have the right structure, autonomy, and technical capabilities needed to efficiently design, plan, construct, operate, and maintain transport infrastructure services. This includes trained labor (public or private sector) and management systems, such as transport strategic planning and management systems. For IFC and MIGA–supported projects, the term “management capability” is used, referring to the capabilities of the private enterprises rather than that of the public institutions supported by the World Bank. Financial Viability—whether the responsible institutions (in the case of World Bank projects) and private enterprises (in the case of IFC and MIGA projects) have access to adequate, reliable, and predictable financial resources needed to operate and maintain the infrastructure services for which they are responsible. This includes proper demand, traffic, and tariffs revenue projections; construction and operating cost estimates; and budgetary allocations and subsidies. In the case of IFC projects, the financial rate of return of the project enterprises has been used. 109 APPENDIX C PORTFOLIO REVIEW AND FIELD- AND DESK-BASED COUNTRY STUDY METHODOLOGIES For the PPAR assessment, risks to development outcome ratings, defined as “the risk, at the time of evaluation, that development outcomes (or expected outcomes) will not be maintained (or realized)”have been used as a proxy to sustained transport outcomes, recognizing that this definition is based on the outcomes of the project objectives, while the sustained transport outcomes as defined in this evaluation are the sustained benefits from the transport infrastructure and services. A sample questionnaire and subsector-specific questions used for the desk and field based country case project assessments follow. SAMPLE QUESTIONNAIRE 5 I. To what extent has World Bank Group support taken into account the need to sustain transport in (a) its strategies and (b) its operations? A. STRATEGIES 6 i. Do country assistance strategies prepared during 2002-11 make sustained infrastructure in particular transport a priority? Have these priorities evolved over the period? Were there any particular aspects of sustained transport that were emphasized? Provide short extracts from the documents reviewed where they are of particular interest. ii. How has the Private Sector’s role in sustaining transport evolved in World Bank Group country level strategies? iii. Have there been Country Economic Memoranda or Poverty Reduction Strategy Papers that made specific reference to the sustained infrastructure during 2002-11? iv. Has there been a PERs that examined the need to protect infrastructure maintenance expenditures relative to new construction? v. Were there any other documents produced that would be relevant to questions regarding sustained transport supported by the World Bank Group, e.g. project related information on sustained transport in IFC/MIGA strategies and / or project documents? 7 vi. Was there a Transport Sector Strategy produced for the country during this period? Does the current 8 Transport Sector Strategy, if there is one, address sustained transport in particular of maintenance arrangements, or seek to institute mechanisms (e.g. Roads Board, tolled highway concessions, cess/tax on goods for rural roads, cost recovery for urban transport services) to assure sustained transport? vii. Have assessments of sustained transport 9 been carried out by World Bank Group, Government or the investor with regard to one or more of the following:? 10, 11 1) Adequacy of annual funding of maintenance requirements; 2) Staffing / technical capacity of the organization; 110 APPENDIX C PORTFOLIO REVIEW AND FIELD- AND DESK-BASED COUNTRY STUDY METHODOLOGIES 3) Monitoring of infrastructure condition by agencies; 4) Prioritization/programming of maintenance expenditures; 5) Cost recovery through tariffs and tolls; and 6) Budgetary funding levels and the reliability of the process B. OPERATIONS 12 i. Have Loan/Credit/Guarantee agreements included conditions on adequate maintenance funding, on entity finances such as maximum operating ratios, or on needed tariff enhancements. ii. Have Monitoring & Evaluation (M&E) systems for World Bank Group operations included targets for the condition of infrastructure assets 13 or indicators for institutional development? iii. Have investment programs included mechanisms for : a. citizen/user feedback to the operating entity? 14 b. assuring organizational capacity and professional capability? c. timely planning of capacity expansions or improvements to facilities? iv. Have project designs included mechanisms that better assure maintenance, e.g. output based disbursements or long-term O&M concessions? v. Have Governance and Accountability Action Plans (GAAPs) 15 included mechanisms to tackle fraud and corruption and for complaints handling? II. To what extent has the World Bank Group helped the country sustain transport by providing support in the following areas (what worked)? 16 A. Enhancing institutional framework for sustaining transport. In each subsector, describe the institutional framework explaining the roles and relationships between various organizations involved, list those that were assisted by the World Bank Group, what was planned for them, and what was achieved in areas such as: i. Developing national capacity for transportation planning 17 to ensure the transport system continues to serve user needs; ii. Improving coordination of urban transport in major cities, e.g. supporting creation of a Metropolitan Transit Agency; iii. Strengthening rural roads construction/ maintenance, e.g. by establishing a centralized engineering organization to assist village authorities; iv. Improving road sector governance, e.g. through creation of a Roads Board with user representation; v. Improving ports efficiency, e.g. by creation of Landlord Ports which devolve operations to competing private terminal operators; vi. Improving railways’ customer orientation, tariff making and competitiveness, e.g. by introducing lines of business organization and accounting; and vii. Supporting civil aviation to improve safety and economic regulation of air services. 111 APPENDIX C PORTFOLIO REVIEW AND FIELD- AND DESK-BASED COUNTRY STUDY METHODOLOGIES B. Strengthening the Policy and Regulatory Environment to improve sustained transport 18 (elaborate for each subsector in respective columns) by for example: i. Supporting transport regulatory bodies for the economic regulation of the sector with respect to efficiency of tariffs, competition policy, granting of concessions, dispute resolution, etc.; ii. Helping with the preparation of PPP policy manuals and model concession agreements for privately financed/ operated transport infrastructure. C. Enhancing transport sector management capacity through: i. Assuring adequate numbers of trained manpower in various skill areas by, e.g.: 1) Conducting assessments of skill gaps in professional/vocational fields; 2) Establishing training institutes or programs; 3) Funding overseas training of staff; 4) Funding or providing technical support for redundancy programs; etc. ii. Facilitating exchange of technology/ management skills internationally, e.g. through twinning or other partnership arrangements. iii. Establishing information base necessary for effective sector management, such as: 1) Transport studies at the national/state/local (rural or urban) levels to project future demand and capacity constraints; 2) Adoption of cost-effective design standards (e.g. drainage for roads) to sustain transport; 3) Regularly updated unit cost data to improve of cost estimates; 4) Asset management systems to systematically assess maintenance needs and budgets. D. Developing financial arrangements for operations and maintenance of transport infrastructure and services, such as: 1) Establishing a properly sized fuel surcharge to fully fund road maintenance; 2) Conducting tariff studies to enhance revenue, recover costs and discourage public service obligations of transport agencies not adequately compensated by the Government; 3) Developing financial models that balance operating expenditures against affordability; 4) Developing targeted pro-poor subsidies to ensure access to public transport for essential needs—work, education, health; 112 APPENDIX C PORTFOLIO REVIEW AND FIELD- AND DESK-BASED COUNTRY STUDY METHODOLOGIES 5) Introducing schemes to enhance revenues, e.g. advertising at bus shelters, to support socially essential transport services. E. Including socially significant design or operational features: Elaborate on any or all of the following in respective subsector columns: 1) Project seen to benefit large sections of the public and as particularly pro-poor; 2) Project features which met with public resistance modified or eliminated; 3) Socio/Political constraints prevented Govt. fulfilling its obligations to the project; 4) World Bank Group–supported projects established mechanisms to target the poor, identified transmission channels and/or paid attention to the distributional aspects through which it benefitted the poor or under-served populations. Is there evidence that World Bank Group–supported investments produced better (or worse) facilities with regard to socially positive design or operational Features compared to those not supported by the World Bank Group. Counterfactuals would be useful. F. Enhancing private sector participation by: 1) assisting in development of PSP policy manuals in subsectors, e.g. railways, to comprehensively decide types of PSP sought and encouraged; 2) introducing fair/effective land acquisition and compensation policies which allow efficient acquisition of land under “eminent domain” for PSP projects; 3) establishing protections for private investors against shifts in Government policies or programs that adversely affect their projects; 4) issuing guarantees (MIGA, IFC; World Bank) protecting the investor against political risks; 5) assisting Governments in the design and implementation of concessions and to create appropriate policy and institutional frameworks to facilitate PPPs; 6) assisting governments improve transport organization, e.g. creation of transport consortia to coordinate public transport services. III. Have countries been able to sustain transport 19 supported by the World Bank Group? 20 (Before-and-After Analysis plus comparison against counterfactual, if available) A. Institutional Capability (intermediate outcome): How has the performance of World Bank Group–supported institutions changed with regard to their operations and maintenance before and after World Bank Group involvement? For each subsector in 113 APPENDIX C PORTFOLIO REVIEW AND FIELD- AND DESK-BASED COUNTRY STUDY METHODOLOGIES which the World Bank Group has been involved, rate the following aspects on a 4 point scale 21 with regard to its impact on sustained transport: 1) Technical capability to plan and execute maintenance; 2) Adequacy of trained man-power available (in public or private sector); and 3) Availability of spares or consumables for the equipment. Is there evidence that World Bank Group–supported institutions underwent better (or worse) changes with regard to institutional capability compared to those not supported by World Bank Group? Provide a counterfactual if possible. B. Financial Viability (intermediate outcome): How financially viable are World Bank Group–supported projects/entities and how has this changed with World Bank Group support? Rate the changes in the financial viability of each subsector in which World Bank Group was involved on a 4 point scale. Elaborate on any of the following in the respective subsector column: 1) Demand/traffic/revenue projections did not materialize; 2) Construction / Operating cost estimates inaccurate; and 3) Tariffs or budgetary allocations/subsidies inadequate. Is there evidence that World Bank Group–supported facilities/systems underwent better / worse changes with regard to financial viability compared to systems/facilities not supported by the World Bank Group? Elaborate and provide comparison with a counterfactual if possible. C. Sustained Transport (outcome): How has the usefulness of World Bank Group– supported facilities changed with respect to their flow of services to the users? For each subsector in which the World Bank Group has been involved, rate the change in such flow of services in a 5 point scale before and after World Bank Group support. Elaborate on the following key issues in the respective subsector column: 1) Capacity constraint (e.g. lack of demand management causing traffic congestion); 2) Accessibility (e.g. rural roads subject to flooding/ wash outs); 3) Misuse of facility (e.g. damaged condition, unsafe operation). Is there evidence that World Bank Group–supported facilities underwent better / worse changes with regard to delivering sustained transport services compared to systems/facilities not supported by the World Bank Group? Provide a counterfactual if possible. 114 APPENDIX C PORTFOLIO REVIEW AND FIELD- AND DESK-BASED COUNTRY STUDY METHODOLOGIES IV. Have World Bank Group–supported programs influenced development in the sector beyond the Groups projects or programs, either through replication or by providing leadership in donor coordination or other sector-wide means? A. Is there evidence that World Bank Group–supported measures to help sustain transport (e.g. changes in institutional and regulatory frameworks, financial arrangements, etc.) have led to systemic changes in the Government's sector operations? B. Is there evidence that World Bank Group–supported initiatives have been replicated and/or scaled up to sustain the services of the country’s own financed transport infrastructure services? Provide examples of such efforts at replication/scaling up and their outcomes, if possible. C. Is there reason to believe that any of the ongoing World Bank Group operations are likely to address the issues related to sustained transport identified in the sector? V. What Factors Accounted for Success or Failure in Sustaining Transport Infrastructure Services in the country overall or in each of the relevant subsectors during FY02-11 period? A. Factors contributing to Success in sustaining transport, including Private Sector Transport infrastructure: elaborate on those of the following considered to be relevant: • Financial health of concerned agency • Trained staff in adequate numbers • Effective asset management systems • Design standards to ensure functional adequacy • Prevention of misuse through proper enforcement • Public awareness and regular input to responsible agency • Regulatory framework supportive of PSP in infrastructure • Appropriate allocation of risks between govt. and private sector • Clarity and common understandings regarding contractual obligations, division of risks and revenue streams • Structured incentives to spur private operators to maintain facilities efficiently • MIGA mediation activities. • Other: _____________ (specify) B. Factors contributing to shortcomings in sustaining transport, including Private Sector Transport infrastructure: elaborate on those of the following considered to be relevant: • Lack of political commitment for maintenance • Diversion of resources to new construction or other uses • Public grievances ignored—no effective complaints handling • Lack of or ineffective third party review 115 APPENDIX C PORTFOLIO REVIEW AND FIELD- AND DESK-BASED COUNTRY STUDY METHODOLOGIES • Rigid application of procurement rules (specify) • Other or Absence of key success factor _______________ (specify) Urban Transport Questions Q1 Institutional Capability: Key Question: Does the agency responsible for developing, operating and maintaining the system have the necessary capacity to organize and manage the relevant components, e.g. street network, traffic signal systems, the transit systems (bus, rail, other), and coordinate these with the development of the city and its major transportation nodes? If it relies on other agencies to carry out these activities on its behalf, how capable are these other agencies, does the nodal agency have the power to cause these other agencies to carry out their responsibilities, and does it control the funds provided for such activities? If there is no single agency, is there a specific mechanism, for coordinating the inputs of various agencies involved in delivering urban transport facilities and services and how effective has this mechanism been? How capable are these individual agencies with respect to: operational planning, infrastructure/fleet maintenance, reliability of services. Q2 Financial Viability: Key Question: Are the agencies (agency) responsible for delivering urban transport services financially self-sufficient (tariffs fully recover costs?) or do they require periodic infusions of funds from the Government? What are the operating ratios of the main transit agencies? Are fares regularly adjusted? Who controls the transit fares in the city? Q3 Sustained Transport: Key Question: Are the urban transport facilities or services congested now or likely to become congested in the near future—street network, transit systems? What is the average speed of vehicular traffic on city streets during peak periods of the day? What is the fatality rate per 100,000 vehicles registered in the city? 116 APPENDIX C PORTFOLIO REVIEW AND FIELD- AND DESK-BASED COUNTRY STUDY METHODOLOGIES Is there a planning mechanism in place to adjust urban transport plans in line with projections of urban growth, and how well does it function? Are there any demand management efforts underway such as parking controls, restrictions on vehicles in congested areas? Rural Roads Questions Q1 Institutional Capability: Key Question: Do local authorities responsible for the rural road network have the technical and managerial capabilities to discharge their responsibility? How do they foster community participation in rural road maintenance and construction? i. Is there a central agency supporting local authorities with technical and managerial inputs for rural roads? ii. Are there simple published manuals explaining rural roads maintenance techniques for the benefit of local communities? iii. How is maintenance undertaken, by force account works, community labor or through contracts? Is there strong competition among local contractors and what fraction of contracts is awarded on a sole source basis? Q2 Financial Viability: Key Question: Do local authorities responsible for rural roads have a reliable and independent source of funding or are they reliant on annual budget allocations? Is the annual maintenance funding level above $1000/km of rural road, and how much ( percent) of the network is improved (e.g. graded or overlayed) each year? i. Are there specific local tax bases, e.g. shops in local markets, which are designated for local authorities to mobilize resources for rural road construction and maintenance? ii. Are there food-for-work or other such programs to mobilize seasonal labor for road maintenance/construction works? Q3 Sustained Transport: Key Question: What percentage of the rural population is connected (in terms of the norm for the maximum distance to the nearest road) by the rural road network? What percentage is connected year round? i. If 100 percent rural road coverage has not yet occurred, is there a target for when this is to be achieved? 117 APPENDIX C PORTFOLIO REVIEW AND FIELD- AND DESK-BASED COUNTRY STUDY METHODOLOGIES ii. What proportion of rural roads are incomplete, missing bridges, culverts or drainage systems needed to make them fully functional? Inter-City Highways Questions Q1 Institutional Capability: Key Question: Does the Road Agency have the necessary capabilities with respect to road network asset management systems, a maintenance organization on the ground, and qualified and well equipped contractors to discharge its responsibilities? i. Is the Road Agency using operations and maintenance by contract modalities? What percentage of the network is now maintained by contractors? Is this expected to increase and to how much in coming years? ii. Do you have axle load controls on trucks? What enforcement mechanisms do you have in place? On average what percentage of trucks exceed the axle load limits? Q2 Financial Viability: Key Question: What is the maintenance funding requirement to keep the highway network adequately maintained? What proportion of this need has been met in recent years? i. Is there a Road Maintenance Fund? Is it dedicated to maintenance or can it also fund new construction, upgrading or rehabilitation works? Q3 Sustained Transport: Key Question: What proportion of the network is presently in poor condition? If this is above 10 percent are there specific plans to clear the maintenance backlog? What was the reason for the backlog? i. What are the average speeds on the highway network? ii. What fraction of the highway network is controlled access out of the total highway network length? iii. Do you maintain fatality statistics for the highway network in terms of fatalities per 1000 km per year? 118 APPENDIX C PORTFOLIO REVIEW AND FIELD- AND DESK-BASED COUNTRY STUDY METHODOLOGIES Railways Questions Q1 Institutional Capability: Key Question: How does the Railway compare with international comparators on key performance benchmarks, e.g. locomotive availability, rolling stock productivity, employee productivity? i. Does the railway have a strategic planning system to decide on investment/divestment plans which integrates traffic projections, mode share information and track and rolling stock capacity? ii. Does the railway have an established system for traffic forecasting? How well have forecasts performed? iii. How well has the railway managed the elimination of level crossings in coordination with local authorities? Q2 Financial Viability: Key Question: Is the railway generating sufficient surpluses to cover its costs and set aside sufficient funds for depreciation and renewal of assets? i. What is the operating ratio of the railway? ii. Does the railway have a “lines of business” organization and accounting framework? Does it know the relative profitability of various types of traffic? iii. Is the railway able to precisely calculate the cost of public service obligations and require to be compensated by the Government for these required services? Can they raise tariffs independently? Q3 Sustained Transport: Key Question: Is the railway losing market share to road transport in key traffic segments? What is its present mode share for passenger and freight traffic? How has this been changing over recent years and what is expected in the long run? i. How congested are the railways main traffic corridors (traffic density)? ii. What are the main constraints to expanding capacity? 119 APPENDIX C PORTFOLIO REVIEW AND FIELD- AND DESK-BASED COUNTRY STUDY METHODOLOGIES Ports and Shipping 22 Questions Q1 Institutional Capability: Key Question: Is the port organization structure conducive to an efficient port operation, competitive internationally—i.e. landlord port authority with competitively selected terminals operators having strong incentives to provide high quality, cost effective services? i. Do terminal operators exercise monopoly power or are tariffs and service quality competitive? ii. Does the port meet international standards for ships and cargo safety, quick vessel turnaround, and pilotage and other services to shipping? Q2 Financial Viability: Key Question: Do the port authority’s financial statements show a comfortable operating margin, e.g. < 80 percent, without excessive unfunded liabilities such as pension liabilities, or long overdue receivables, particularly from Government agencies who are large users of the port? i. Does the port have large maintenance backlogs, e.g. for channel dredging, that are likely to affect the port operations in the future? ii. Can the port adjust tariffs independent of political considerations to pay for operationally critical expenses that might otherwise be neglected? Q3 Sustained Transport: Key Question: How does the port (or port sector) compare with international benchmarks on key performance indicators: Berth productivity, vessel turnaround times, cargo dwell times? Does the port regularly seek and receive feedback from shippers using the port? i. Are there adequate hinterland connections (road, rail)? Does the port regularly coordinate with regional authorities to ensure its hinterland connectivity serves the needs of shippers efficiently? ii. Do Customs authorities provide speedy clearance for cargo moving out of the port? What is the average delay for customs clearance of a shipping container moving inland through the port? 120 Appendix D: Additional Tables Table D.1. Country Assistance/Partnership Strategies: Occurrence of Measures that Help Sustain Transport (%): FY2002-06 versus FY2007-12 Institutional Regulatory/policy Sector capacity Financial FY02- FY07- FY02- FY07- FY02- FY07- FY02- FY07- Income group 06 12 06 12 06 12 06 12 Lower middle income 83% 72% 30% 38% 60% 72% 26% 26% countries Upper middle income 81% 88% 37% 40% 60% 75% 21% 18% countries Low income countries 78% 75% 26% 42% 78% 83% 25% 38% Average 81% 78% 31% 40% 66% 77% 24% 27% Source: IEG portfolio review. Table D.2. Percentage of World Bank Operations that Included Each Measure that Help Sustain Transport by Country Income Groups Upper middle Low income Lower middle income income Measures to help sustain transport (n=54) (n=98) (n=111) Sector policy and regulatory framework 33 18 27 Institutional framework (includes PSP) 63 53 46 Sector management and capacity Building 91 87 87 Financial arrangements 17 17 15 Note: Six projects in two high-income countries have been omitted. The total number of projects is 263 instead of 287 because these six projects in addition to regional projects are excluded. Source: IEG portfolio review. 121 APPENDIX D ADDITIONAL TABLES Table D.3. Percentage of World Bank Operations that Included Each Measure that Help Sustain Transport by Sector managing the Operations and by Transport Subsector Projects managed by the Transport Sector Board Projects managed by other Sector All Boards with Measures to transport Intercity Rural Ports Urban > 30% help sustain projects highways roads Railways (n= Aviation transport transport transport (n=287) (n=207) (n=62) (n=31) 36) (n=22) (n=58) (n=150) Sector policy and 25 25 24 16 33 45 43 6 regulatory framework Institutional 51 55 56 55 44 73 41 12 framework (includes PSP) Sector 86 89 94 84 94 100 95 17 management and capacity building Financial 16 19 19 23 8 9 24 3 arrangements Source: IEG portfolio review. Table D.4. Projects that Implemented Measures to help Sustain Transport Had Lower Risk to Development Outcome at Project Closing (Percent with Negligible or Moderate Risks) Measure to help sustain transport for projects managed by the transport sector Sector policies and Sector management regulatory Institutional and capacity Financial Was the measure frameworks frameworks building mechanisms implemented? (n=14) (n=33) (n=67) (n=7) Measure was implemented 63 67 73 100 by project closure (n=8) (n=24) (n=48) (n=3) Measure was planned but not implemented by project 33 17% 44 50 closure (n=4) (n=6) (n=9) (n=2) Note: Total number of transport projects = 58.Prior to July 2006, the risk to development outcome rating was not assessed at project closure because it replaced another rating related to sustained transport, Therefore only 58 projects for which the risk to development outcome rating assigned are covered in this analysis, out of the 76 projects reviewed. The ratings are on a four- point scale; negligible to low, moderate, significant, and high. Source: IEG analysis of 9 country studies and 11 desk-based studies. While risk to development outcomes has been rated systematically for 100 percent of closed projects by IEG since July 2006, the achievement level of measures that help sustain transport at project closure was often not available because of lack of information in the ICRs. 122 APPENDIX E LIST OF WORLD BANK GROUP PROJECTS REVIEWED Appendix E: List of World Bank Group Projects Reviewed List of 68 World Bank Transport Projects reviewed under field and desk based assessments in twenty case study countries Project ID Country Project Name Approval Closing Country Income Sector IDA IBRD Region** IEG Project Fiscal Year Fiscal Year Category Board* Commitments Commitments Performance US$ millions US$ millions Assessment Report P040085 Bolivia Participatory Rural Investment 1998 2006 Lower middle ARD - 63 LCR P055230 Bolivia Abapo-Camiri Highway Project 1999 2006 Lower middle TR - 88 LCR P068968 Bolivia Road Rehabilitation and 2002 2011 Lower middle TR - 77 LCR Maintenance P006661 Chile Third Road Sector 1995 2003 Upper Middle TR 120 - LCR P076807 Chile Infrastructure for Territorial 2005 2012 Upper Middle TR 50 - LCR Development P007388 Honduras Transport Sector rehabilitation 1003 2003 Lower middle TR - 65 LCR P057538 Honduras Road Reconstruction and 2001 2008 Lower middle TR - 67 LCR Improvement P009946 India National Highways II 1992 2003 Lower middle TR 153 153 SAR P009995 India Andhra Pradesh State Highway 1997 2004 Lower middle TR 350 - SAR P045600 India States' Road Infrastructure 1997 2002 Lower middle TR 52 - SAR Development Technical Assistance P009972 India Third National Highways 2000 2008 Lower middle TR 516 - SAR P010566 India Gujarat State Highways 2001 2008 Lower middle TR 381 - SAR Yes P070421 India Karnataka State Highways 2001 2008 Lower middle TR 360 - SAR Yes P071244 India Grand Trunk Road 2001 2008 Lower middle TR 589 - SAR Improvement P069889 India Mizoram State Roads 2002 2011 Lower middle TR - 60 SAR P072539 India Kerala State Transport 2002 2011 Lower middle TR 255 - SAR P067606 India U.P. State Roads 2003 2011 Lower middle TR 488 - SAR P073776 India Allahabad Bypass 2004 2009 Lower middle TR 240 - SAR P102737 India Bihar DPL 2008 2010 Lower middle EP 150 75 SAR P004016 Indonesia Strategic Urban Roads 1996 2003 Lower middle TR 87 - EAP P004026 Indonesia Railway Efficiency 1997 2010 Lower middle TR 105 - EAP P003993 Indonesia Sumatera Regional Roads 1998 2006 Lower middle TR 234 - EAP P040578 Indonesia East Indonesia Regional 2002 2010 Lower middle TR 200 - EAP Transport P107163 Indonesia Infrastructure DPL 2008 2008 Lower middle PS 200 - EAP P111905 Indonesia Second Infrastructure 2009 2009 Lower middle UD 200 - EAP Development Policy Loan P115102 Indonesia Third Infrastructure 2010 2010 Lower middle UD 250 - EAP Development Policy Loan P118531 Indonesia Fourth Infrastructure 2011 2011 Lower middle UD 200 - EAP Development Policy Loan P004342 Mongolia Transport Rehabilitation 1994 2002 Lower middle TR - 30 EAP 123 APPENDIX E LIST OF WORLD BANK GROUP PROJECTS REVIEWED List of 68 World Bank Transport Projects reviewed under field and desk based assessments in twenty case study countries Project ID Country Project Name Approval Closing Country Income Sector IDA IBRD Region** IEG Project Fiscal Year Fiscal Year Category Board* Commitments Commitments Performance US$ millions US$ millions Assessment Report P056200 Mongolia Transport Development 2001 2008 Lower middle TR - 34 EAP P001804 Mozambique Second Roads and Coastal 1994 2003 Low TR - 188 AFR Shipping P042039 Mozambique Railways and Ports 2000 2009 Low TR - 100 AFR Yes Restructuring P073479 Mozambique Communication Sector Reform 2002 2009 Low GIC - 15 AFR Project P001785 Mozambique Roads and Bridges 2002 2007 Low TR - 162 AFR Yes Management and Maintenance P049878 Mozambique Economic Management and 2003 2005 Low EP - 120 AFR Private Sector Adjustment Credit P082618 Mozambique Beira Railways Project 2005 2012 Low TR - 110 AFR P070293 Nigeria Privatization Support Program 2001 2010 Lower middle FPD - 114 AFR Credit P074963 Nigeria Lagos Urban Transport 2003 2011 Lower middle TR - 100 AFR P004389 Papua and Smallholder Oil Palm non 1992 2012 Lower middle ARD 27 - EAP New Guinea dedicated P054238 Papua and Second Gazelle restoration 2000 2008 Lower middle TR 25 - EAP New Guinea P004397 Papua and Road maintenance and 2002 2012 Lower middle TR 40 - EAP New Guinea rehabilitation P044601 Peru Rural transport 2001 2007 Upper Middle TR 50 - LCR P035740 Peru Lima Urban Transport 2004 2011 Upper Middle TR 45 - LCR P008615 Poland Szczecin-Swinoujscie Seaway 2001 2008 High TR 39 - ECA Yes and Port Modernization P040795 Poland Railways Restructuring Project 2001 2006 High TR 101 - ECA P078170 Poland Road Maintenance and 2004 2009 High TR 126 - ECA Rehabilitation Program I P088824 Poland Road Maintenance and 2005 2009 High TR 131 - ECA Rehabilitation Program II P096214 Poland Road Maintenance and 2006 2012 High TR 180 - ECA Rehabilitation Program III P008806 Russia Urban Transport 1995 2013 Upper Middle TR 329 - ECA P035764 Russia Bridge Rehabilitation 1996 2004 Upper Middle TR 350 - ECA P046061 Russia Moscow Urban Transport 2001 2009 Upper Middle TR 60 - ECA P002366 Senegal Second Transport Sector 1999 2008 Lower middle TR - 90 AFR P055472 Senegal Urban Mobility Improvement 2000 2009 Lower middle TR - 70 AFR P058070 Sri Lanka North-east Irrigated Agriculture 2000 2005 Lower middle ARD - 27 SAR P002770 Tanzania Second Integrated Roads 1994 2007 Low TR - 170 AFR Yes P078387 Tanzania Central Transport Corridor 2004 2010 Low TR - 122 AFR Yes P005749 Tunisia Rural Roads 1995 2003 Lower middle TR 52 - MNA P046832 Tunisia Municipal Development II 1997 2003 Lower middle UD 80 - MNA 124 APPENDIX E LIST OF WORLD BANK GROUP PROJECTS REVIEWED List of 68 World Bank Transport Projects reviewed under field and desk based assessments in twenty case study countries Project ID Country Project Name Approval Closing Country Income Sector IDA IBRD Region** IEG Project Fiscal Year Fiscal Year Category Board* Commitments Commitments Performance US$ millions US$ millions Assessment Report (Non-dedicated) P043700 Tunisia Transport sector investment 1998 2006 Lower middle TR 50 - MNA project APL I P064082 Tunisia Transport sector investment 2001 2010 Lower middle TR 38 - MNA project APL II P038091 Turkey Road Improvement and Traffic 1996 2013 Upper Middle TR 250 - ECA Safety Project P077328 Turkey Railways Restructuring Project 2005 2012 Upper Middle TR 185 - ECA P065436 Uganda Uganda Phase II of the Roads 2002 2008 Low TR - 51 AFR Yes Development Program P005806 Yemen Multi Mode Transport 1991 2001 Lower middle TR - 30 MNA P041267 Yemen Transport Rehabilitation 1996 2002 Lower middle TR - 37 MNA P070391 Yemen Rural Access Improvement 2001 2007 Lower middle TR - 45 MNA Program I P003236 Zambia Road Sector Investment 1998 2005 Lower middle TR - 70 AFR Program Support P003227 Zambia Railways Restructuring 2001 2005 Lower middle TR - 27 AFR P074445 Zambia Economic Management & 2008 2009 Lower middle EP - 10 AFR Growth Credit 125 APPENDIX E LIST OF WORLD BANK GROUP PROJECTS REVIEWED List of 22 Additional Projects reviewed by IEG in PPARs over the past five years P052590 Argentina National Highways Rehabilitation and Maintenance 1998 2006 Upper Middle TR - 450 LCR Yes P040150 Belize Belize - Roads and Municipal Drainage 2001 2006 Lower middle TR - 13 LCR Yes P038882 Brazil Recife Metropolitan Transport Decentralization 1995 2004 Upper Middle TR - 102 LCR Yes P006564 Brazil Belo Horizonte Metropolitan Transport Project 1995 2004 Upper Middle TR - 102 LCR Yes P055954 Brazil Goias State Highway Management Program Project 2002 2007 Upper Middle TR - 65 LCR Yes P045788 China Tri-Provincial Highway Project, 1998 2007 Upper Middle TR - 230 EAP Yes P070441 China Hubei Xiaogan- Xiangfan Highway Project 2003 2008 Upper Middle TR - 250 EAP Yes P006861 Colombia Urban Infrastructure 1998 2004 Upper Middle UD - 75 LCR Yes P000755 Ethiopia Road Sector Development Program 1998 2005 Low TR - 225 AFR Yes P008417 Georgia Georgia - Municipal Infrastructure Rehabilitation 1994 2000 Lower middle UD - 13 ECA Yes P050910 Georgia First Municipal Development and Decentralization 1998 2003 Lower middle UD - 15 ECA Yes P077368 Georgia Second Municipal Development and Decentralization 2003 2008 Lower middle UD - 15 ECA Yes P040521 Indonesia Second Village Infrastructure 1997 2011 Lower middle TR - 140 EAP Yes P036047 Indonesia Bali Urban Infrastructure 1997 2005 Lower middle UD - 110 EAP Yes P042237 Lao People's Democratic Republic Provincial and Rural Infrastructure 1999 2007 Lower middle TR - 21 EAP Yes P001403 Lesotho Road Rehabilitation and Maintenance 1996 2004 Lower middle TR - 27 AFR Yes P035776 Lithuania Klaipeda Port Project, 2000 2009 Upper Middle TR - 35 ECA Yes P010509 Nepal Multimodal Transit Project 1998 2004 Low TR - 17 SAR Yes P039250 Romania Second Roads Project 1997 2006 Upper Middle TR - 150 ECA Yes P049543 Uganda Uganda Roads Sector Institutional Support Technical assistance Project 1998 2008 Low TR - 22 AFR Yes P065436 Uganda Roads Development-Second Phase 2002 2008 Low TR - 65 AFR Yes P004843 Vietnam Inland Waterways and Port Rehabilitation 1998 2006 Lower middle TR - 54 EAP Yes * ARD: Agricultural and Rural Development; EP: Economic Policy: FPD: Finance and Private Sector Development; GIC: Global Information & Communications Technologies; PS: Public Sector; UD: Urban Development ** AFR: Sub-Saharan Africa; EAP: East Asia and Pacific: ECA: Eastern Europe and Central Asia; LCR: Latin America and the Caribbean; MNA: Middle-East and North Africa; SAR: South Asia 126 APPENDIX E LIST OF WORLD BANK GROUP PROJECTS REVIEWED List of 53 IFC Transport Projects reviewed Project Country Project Project Name Approval Fiscal Country Income Sector Board* Original Project Region** Extended ID Code Year Category Commitments Size Project US$ millions Supervision Report 10024 Peru P1 Peru Rail 2002 Upper middle Railways 9 42 LCR Yes income 10678 Chile P2 San Vicente 2002 Upper middle Ports 15 62 LCR Yes income 11017 Brazil P3 Suape ICT 2002 Upper middle Ports 6 51 LCR Yes income 11109 Dominican P4 Caucedo 2002 Upper middle Ports 30 293 LCR Republic income 11179 Venezuela P5 VRT 2002 Upper middle Ports 9 10 LCR Yes income 11353 Jamaica P6 Montego Bay Arpt 2002 Upper middle Aviation 20 80 LCR Yes income 11383 Latin America P7 UABL River Barge 2002 Lower middle Shipping 25 101 LCR Region income 11646 Pakistan P8 PICT 2003 Lower middle Ports 9 29 MNA income 11720 Mexico P9 DTM 2004 Upper middle Urban 37 37 LCR Yes income Transport 20050 Peru P10 Norvial 2003 Upper middle Highways 18 91 LCR Yes income 20055 Egypt P11 DP World Sokhna 2004 Lower middle Ports 20 100 MNA Yes income 20280 Kenya P12 Kenair 2003 Low income Aviation 15 98 AFR Yes 20337 Russian P13 Kulon 2004 Upper middle OTHER 8 8 ECA Yes Federation income 21363 Cambodia P14 Cambodia Airport 2004 Low income Aviation 10 72 ECA Yes 21396 Russian P15 VSC 2004 Upper middle Shipping 25 46 ECA Federation income 21422 Russian P16 Severstaltrans 2004 Upper middle Railways 40 40 ECA Federation income 21495 Turkey P17 Kusadasi Port 2004 Upper middle Ports 10 18 ECA Yes income 21506 Russian P18 Russkiy Mir 2004 Upper middle Railways 15 73 ECA Yes Federation income 21805 Mexico P19 Manzanillo Port 2004 Upper middle Ports 45 45 LCR Yes income 21887 Brazil P20 TRG EXPANSION 2004 Upper middle Ports 8 16 LCR income 22547 China P21 MS Shipping 2005 Upper middle Shipping 16 21 EAP income 22768 Russian P22 NWSC 2004 Upper middle Shipping 23 47 EAP Yes Federation income 127 APPENDIX E LIST OF WORLD BANK GROUP PROJECTS REVIEWED List of 53 IFC Transport Projects reviewed Project Country Project Project Name Approval Fiscal Country Income Sector Board* Original Project Region** Extended ID Code Year Category Commitments Size Project US$ millions Supervision Report 22831 Dominican P23 Aerodom 2005 Upper middle Aviation 45 266 LCR Yes Republic income 23081 India P24 Sealion Sparkle 2004 Lower middle Ports 5 16 SAR income 23270 Russian P25 Eurosibtrans 2005 Upper middle Railways 30 30 ECA Federation income 23356 Pakistan P26 PICT II 2005 Lower middle Ports 6 6 MNA income 23870 Russian P27 Russkiy Mir II 2005 Upper middle Railways 20 426 ECA Federation income 23882 El Salvador P28 TACA Airlines 2005 Lower middle Aviation 30 30 LCR Yes income 23914 Argentina P29 TRP Project 2007 Upper middle Ports 35 112 LCR income 24082 Romania P30 TTS 2006 Upper middle Shipping 17 17 ECA Yes income 24127 Russian P31 AirTaxi 2006 Upper middle Aviation 15 82 ECA Federation income 24170 Panama P32 MANZANILLO-III 2005 Upper middle Ports 50 50 LCR Yes income 24197 Mexico P33 Irapuato-Piedad 2006 Upper middle Highways 12 14 LCR income 24271 Cameroon P34 AEF 3T 2006 Lower middle Ports 1 1 AFR CAMEROUN income 24306 Jamaica P35 MBJ Phase Two 2006 Upper middle Aviation 20 67 LCR income 24384 Brazil P36 TAM Airlines 2006 Upper middle Aviation 50 50 LCR income 24389 Russian P37 Russkiy Mir IIb 2005 Upper middle Railways 25 54 ECA Federation income 24407 Brazil P38 MRS 2006 Upper middle Railways 50 100 LCR income 24489 Peru P39 Lima JCIAirport 2007 Upper middle Aviation 20 20 LCR income 24534 Dominican P40 Sans Souci Ports 2007 Upper middle Ports 21 46 LCR Republic income 24608 Peru P41 Peru Rail II 2007 Upper middle Railways 8 8 LCR income 24609 Brazil P42 GOL 2006 Upper middle Aviation 50 50 LCR income 24628 Georgia P43 Tbilisi Airport 2006 Lower middle Aviation 27 137 ECA income 128 APPENDIX E LIST OF WORLD BANK GROUP PROJECTS REVIEWED List of 53 IFC Transport Projects reviewed Project Country Project Project Name Approval Fiscal Country Income Sector Board* Original Project Region** Extended ID Code Year Category Commitments Size Project US$ millions Supervision Report 24638 Russian P44 Eurosib Terminal 2006 Upper middle Railways 48 182 ECA Federation income 24669 Jordan P45 CTI Group 2006 Upper middle Shipping 15 15 MNA income 24672 Mexico P46 Vuela 2006 Upper middle Aviation 40 170 LCR income 24766 Kenya P47 KenyaUganda Rail 2007 Low income Railways 32 111 AFR 24951 Dominican P48 Aerodom II 2006 Upper middle Aviation 15 15 LCR Republic income 25034 Burkina Faso P49 BBCOM INT. 2007 Low income Urban 2 9 AFR Transport 25039 Rwanda P50 Intraspeed 2007 Low income OTHER 8 21 AFR 25169 India P51 Ocean Sparkle II 2007 Lower middle Ports 27 86 SAR income 25332 Cambodia P52 Cambodia Airp II 2007 Low income Aviation 18 43 EAP 25835 Russian P53 Chuvash Roads 2007 Upper middle Provincial 50 162 ECA Federation income Roads List of MIGA Transport Projects reviewed Project Contract Enterprise FY issued Effective Host Country Status Evaluation ID Date material available 1810 Manila North Tollways 2001 29.06.2001 Philippines Operationally IEG PER Corporation Mature 4006 Lima Airport Partners 2002 29.05.2002 Peru Operationally IEG PER S.R.L. Mature 7135 Autopistas del Sol. S.A. 2008 20.12.2007 Costa Rica Operationally MIGA Self Mature Evaluation – draft 129 Notes Chapter 1 1Figures are in constant dollars. In 1988 $45 billion worth of road infrastructure assets were estimated to be lost. Preventive maintenance required in 1988 was estimated to be $12 billion. 2The two figures apply to the assumptions of 300 vehicles/day and 10,000 vehicles/day respectively. 3 Summary of the portfolio is attached as Appendix B. 4Project outcome rating is defined as the extent to which the operation's major relevant objectives were achieved, or are expected to be achieved, efficiently. 5 Transport infrastructure and services include the services that benefit the public or commercial customers directly, as well as transport infrastructure that is used by the transport service providers (World Bank 2004). 6This strategy was in effect until “Clean, and Affordable Transport for Development (2008- 2012)” was prepared. Therefore, the 1996 strategy has been applied for the focused portfolio of projects in this evaluation, which included projects approved and closed between FY02 and FY11. 7The sustained transport outcome as defined in this evaluation is different from the outcome rating that is assessed at project completion under the IEG/Operations Policy and Country Services harmonized evaluation criteria, defined as the extent to which the operation’s major relevant objectives were achieved, or are expected to be achieved, efficiently. 8IEG considers that projects have reached operating maturity, generally when operations have recorded at least 18 months of operating revenue, reflected in at least two years of audited financial statements (ex-post evaluation). 9For IFC and MIGA projects, the assessment relies on ratings related to these three aspects of sustainability contained in IFC’s Extended Project Supervision Report (XPSRs), MIGA’s Project Evaluation Reports, IFC’s Development Outcome Tracking System (DOTS), and periodic Project Supervision Reports and Credit Risk Ratings. For World Bank projects, the assessment relies on IEG’s ICR Reviews, Project Performance Assessment Reports, and field- and desk-based country case studies. 10 Selection criteria are summarized in Appendix C. Chapter 2 1 World Bank Group Infrastructure and Transport timeline is described in Appendix A. 131 NOTES 2 The World Bank Group prepares a Country Assistance Strategy (CAS) and in some cases Country Partnership Strategy or Joint Assistance Strategy - for active borrowers from the International Development Association (IDA) and the International Bank for Reconstruction and Development (IBRD). The CAS takes as its starting point the country’s own vision for its development, as defined in a Poverty Reduction Strategy Paper or other country-owned process. The CAS, oriented toward results, is developed in consultation with country authorities, civil society organizations, development partners, and other stakeholders. The purpose of the CAS is to set out a selective program of Bank Group support linked to the country’s development strategy and based on the Bank Group’s comparative advantage relative to other donors. CASs are designed to promote collaboration and coordination among development partners in a country. 3 Appendix D contains more analysis on the CASs. 4 For the remaining six percent, there was no rating for the level of risk. 5 This decline is highly statistically significant (p=.006). 6HDM-4, is the successor to the World Bank Highway Design and Maintenance Standards Model (HDM-III). It is designed to be used as a decision support tool within a road management system. Standard data import and export facilities are provided for linking HDM-4 to various database management systems. Local adaptation and calibration of HDM-4 models can be achieved by specifying default data sets that represent pavement performance and vehicle resource consumption in the country where the model is being used. 2003. Kerali, Henry R. The Role of HDM-4 in Road Management. The Highway Development and Management Series. 7The Road Network Evaluation Tool (RONET) was developed by SSATP (2007) based on the same underlying economic evaluation and engineering principles of HDM-4. RONET uses country-specific relationships between maintenance spending, traffic levels, and road condition, and between road condition and road user costs, to assess the performance overtime of the network under different maintenance standard scenarios. RONET has been primarily applied by the World Bank as part of the Africa Infrastructure Country Diagnostic in forty countries in Sub-Saharan Africa, and in close coordination with a limited number of Road Agencies (Poland, Indonesia, Senegal, Nigeria, Mozambique, Ghana). 8Projects objectives that explicitly mention the words sustain, sustainable, sustainability, or other phrases implying sustainability. 9 While the share of projects with objectives focusing on sustained transport has declined, the transport sector has increased the use of other instruments like development policy loans that complement investment lending in supporting higher level policy reforms. However, only two of these DPOs had reference to sustained transport in their objectives. 10 Country income and sub-sector breakdowns are summarized in Appendix D. 11This includes four DPOs managed by the transport sector, and in the analysis they are part of the projects managed by the transport sector board. 132 NOTES 12The poverty of documentation on AAA has been noted in several previous IEG evaluations and can be traced to deficiencies in filing, reporting, and monitoring of these activities. 13As elaborated in chapter four, while the risks related to private sector participation are well known, if properly structured it has a high potential to achieve sustained transport and therefore the evaluation captures concession and other forms of public private partnerships as a measure to sustain transport. 14IFC’s Investment Climate advisory services focus on creating a business-friendly environment with little emphasis on an enabling environment for private sector participation in the transport sector (about 3-5 percent of their mandates). Likewise, IFC’s public-private partnership advisory services largely focused on launching sustainable transport sector transactions (80 percent) with only some upstream advisory work (10 percent). 15Include a Highway Study in Egypt, a Roads Workshop in Mexico, a capacity-building workshop for the stakeholders of Russky Mir, a Public-Private Partnership Conference in Dubai, strategic reviews of the potential for privatizing state-owned airlines in Samoa and in Jordan, a Maritime and Port study of the MENA Region, a Transport Study of port and airport opportunities for PSP in Liberia, and technical assistance to the Minsheng Shipping Company to help in preparation of a business plan. Chapter 3 1In the 20 countries covered for the assessment, only 26 projects were approved and closed during FY02-11 so an additional 42 projects that were approved prior to FY02 were included in the review. Projects for which sustained transport outcomes could not be assessed primarily due to lack of information were excluded from the analysis. 2Number of countries selected in each region is based on the number and size of World Bank Group support in each region. The countries selected for Sub Saharan Africa region has been rounded up and for Middle East and North Africa the number has been rounded down. 3 An average of five to six years after project completion. 4The difference in achievement of sustained transport among projects that did and did not achieve financial viability is statistically significant at p<=.05. 5The difference in achievement of sustained transport among projects that did and did not achieve institutional capacity is statistically significant at p<=.01. 6 Despite the small sample size, implementation of two of the planned measures was directly and significantly correlated with sustained transport outcomes – sector policies and regulatory framework (60 percent of projects that implemented the measure had sustained transport, compared to 13 percent for those that did not, p<=.05) and an institutional framework (73 percent of those that implemented the measure had sustained transport, compared to 17 percent for those that did not, p<=.01). 133 NOTES 7 Ports around the world are managed along one of the principal management models: (i) Public Service Ports. Service ports have a predominantly public character. Under this model, the port authority offers the complete range of services required for the functioning of the seaport system. The port owns, maintains, and operates every available asset (fixed and mobile), and cargo-handling activities are executed by labor employed directly by the port authority. (ii) Landlord Ports. Landlord port is characterized by its mixed public-private orientation. Under this model, the port authority acts as regulatory body and as landlord, while port operations (especially cargo handling) are carried out by private companies. (iii) Private Service Ports. Fully privatized ports (which often take the form of a private service port) are few in number. In fully privatized ports, port land is privately owned, unlike the situation in other port management models. This requires the transfer of ownership of such land from the public to the private sector (J. Lee and J. Hine, 2008). 8This has traditionally been opposed by macro-economists, both in the International Monetary Fund and the World Bank, because it undermines the fiscal flexibility and responsibility of governments (Potter, 1997). In a formal coordination procedure agreed in 1997 between the Bank and the IMF, the IMF, while expressing reservations, agreed to review the merits of road funds on a case by case basis, and to seek closer coordination between the two institutions in cases where conflicts or disagreements arose. 9 The ‘first generation’ road funds were created by many countries around the world in response to the growing shortage of finance for road maintenance, by attempting to earmark selected taxes and charges and depositing them into a special off-budget account, or road fund, to support spending on roads. These first generation funds typically had a weak legal basis and relied on earmarked revenues, not always related to road use. They were generally managed by the national road agency without the benefit of an oversight board, or being subject to independent technical and financial audits. The ‘second generation’ road funds, which emerged in the late 1990s, were designed to overcome these weaknesses. The administration of the second generation road funds is characterized by separation from the road agency, a sound legal basis, and oversight by a board composed of all stakeholders including road users. These funds are funded by user charges that are directly linked to road use, and are subject to regular technical and financial audits. 10 Nigeria–Lagos Urban Transport Project (P074963). 11 Chile–Third Road Sector project (P006661). 12 Transport SIL 2 (FY99) (P002366). 13 Roads and Bridges MMP (FY02) (P001785). 14Reimbursable Advisory Services (formerly called Fee-Based Services or Reimbursable Technical Assistance) allow the Bank to provide advisory services that the client requests and that the Bank cannot fund in full within the existing administrative budget envelope. This includes, inter alia, economic and sector work, non-lending technical assistance, impact evaluation, research services, external training, and workshops/conferences. 15 Lessons are based on field- and desk-based assessments of 68 projects in twenty countries. 134 NOTES Chapter 4 1As noted earlier, IEG considers that projects have reached operating maturity, generally, when operations have recorded at least 18 months of operating revenue, reflected in at least two years of audited financial statements. 2The current draft Self Evaluation Report (Autopistas de Sol S.A.) is awaiting IEG validation. To the extent to which it will be finalized before CODE submission, the results will remain in this report. 3For IFC only. For MIGA projects, sustainability only up to the point of reaching operational maturity can be assessed based on Project Evaluation Reports as no monitoring information is available thereafter. 4Assessing sustained transport of all operationally mature IFC investments on the same 4- point scale. 5Management Capacity is IFC’s proxy for Institutional Capacity. Note that the three dimensions of sustainability are not assessed as intermediate outcomes for IFC projects as they are not a deliverable per se, as they are with the World Bank. They were assessed for all operationally mature transport investments of IFC when assessing their overall level of sustainability. 6In a few cases, IFC has taken a “programmatic approach” whereby it clustered multiple investments in the same transport sub sector into a “program.” This was done successfully in Russia’s rail cargo subsector with eight IFC investments. Similarly, in a few cases, IFC supported the arrangement of a concession that covered several elements of infrastructure, as it did with three airports in Cambodia. The Lima Airport, in Peru, benefited from the development of other regional airports that otherwise would have been unable to sustain regional operations. this analysis, the pioneering character of an investment was assessed by the time lag 7 In between PSP-enabling reforms, extent of involvement in privatizations or first-of-a- kind/new concessions or arrangements, complemented by factors of market distortion by a dominant state-owned enterprise. 8Interestingly, these “early investments” did not happen disproportionally more frequently in UMIC countries, with 67 percent (or 10 out of 15) early investments in UMIC vs. 72 percent (or 38 out of 53) of the portfolio overall. 9 The World Bank, Russia—Public Expenditure Review, June 8, 2011, Report No. 58836-RU 10 Investments rated 1, that is, not sustained. 11A sponsor is a stakeholder or other investor involved in an IFC project. Because IFC is intended to serve as a catalyst for investment from the private sector, it pursues projects in collaboration with other investors or lenders. 12Over and above to providing longer tenure which almost always is given and helps improving financial structure. 13This analysis did not assess implicit liabilities, where it represents a moral obligation for the government to rescue a project, i.e. reconstruct a vital bridge that used to be operated by 135 NOTES a concession. Secondly, this analysis did not try to assess in how far the fiscal stability of a country was endangered by contingent liabilities of too many or wrongly designed public- private partnerships. 14 Theseare cases where output delivery was rated mostly successful or better in PCRs and evaluation reports. 15 ForIFC Advisory Services, Project Completion Reports are prepared at project closure. Unlike IFC’s investments, where XPSRs, DOTS, and various supervision reports inform about the project success after financial closure up to the point of operational maturity and even beyond, no monitoring system exists for IFC Advisory Services beyond closure. Hence, only the potential to enhance sustainability can be assessed; for example, through the completion of a successful bidding process and award of a concession at the end of which the concessionaire was able to secure commercial financing. 16 Eightprojects were subtracted from the pool of IFC Advisory Services as they did not intend to result in formation of a concession or have a similar result, but rather, they constituted studies or general technical assistance work. 17 As per IFC Completion Report ratings on Output Achievement. 18Including a Highway Study in Egypt, a Roads Workshop in Mexico, a capacity-building workshop for the stakeholders of Russky Mir, a Public-Private Partnership Conference in Dubai, strategic reviews of the potential for privatizing state-owned airlines in Samoa and in Jordan, a Maritime and Port study of the Middle East and North Africa Region, a Transport Study of port and airport opportunities for private sector participation in Liberia, and technical assistance to the Minsheng Shipping Company to help in preparation of a business plan. for IFC’s Advisory Services Work Requiring Intra-World Bank Group 19 Procedures Coordination (2007) 20Examples include the Kenya-Uganda Railway Concession, during the preparation of which IFC provided inputs to help in the revision of the Privatization Law, which was necessary to facilitate the transaction; or the Madagascar Airport project (which did not actually advance beyond Phase 1), during which IFC provided advice to the government on changing the Civil Aviation Code. Of the 15 transport projects supported by MIGA during FY02–11, 7 reached operational 21 maturity. Of these, three had an available Project Evaluation Report or Self Evaluation Report. IFC only. For MIGA projects, sustainability only up to the point of reaching 22 For operational maturity can be assessed based on Project Evaluation Reports as no monitoring information is available thereafter. Appendix B 1The US bear market of 2007–2009 was declared in June 2008 when the Dow Jones Industrial Average (DJIA) fell 20percent from its October 11, 2007 high. 136 NOTES 2Outcome ratings are assessed in an IEG review of the Implementation Completion and Results Report (ICR) which is typically prepared six month after project closure. 3 The rating scales follow a four point scale; negligible to low, moderate, significant, and high. The risk to development outcome rating has two dimensions; (i) the likelihood that some changes may occur that are detrimental to the ultimate achievement of the operation’s development outcome; and (ii) the impact on the operation’s development outcomes of some or all of these changes materializing. The criteria used for assessing this rating are; (a) technical; (b) financial; (c) economic; (d) social; (e) political; (f) environmental; (g) government ownership/commitment; (h) other stakeholder ownership; (i) governance; and (j) natural disaster exposure. 4Prior to FY07, IEG had a different assessment for the sustainability of the projects, which had on average 83 percent likely ratings for projects mapped to the Transport Sector Board. 5 With preliminary 2011 ratings 6 2011 figures not yet available Appendix C 1 Source: IEG Annual Report 2010 Results and Performance of the World Bank Group 2Projects for which sustained transport outcomes could not be assessed were excluded from the analysis. 3Number of countries selected in each region is based on the number and size of World Bank Group support in each region. The countries selected for Sub Saharan Africa region has been rounded up and for Middle East and North Africa the number has been rounded down. 4Field based country studies for Mozambique, Poland, and India were carried out in conjunction with IEG’s PPAR missions and the field based country study for Peru was carried out in conjunction with the country program evaluation. 5Responses to these questions were provided as one spread-sheet for each country. Responses were entered into the appropriate matrix column which are set up as: A. Question; B. Country /Transport sector; C. Urban Transport; D. Rural Roads; E. Inter-city Highways; F. Railways; G. Ports and Waterways; H Air Transport. 6In addition to Country Level strategy assessment, a policy review was conducted to assess World Bank Group (including IFC/MIGA) corporate policies that aim at enhancing the sustainability aspects of projects. For IFC/MIGA operations, sustainability assessments relate mainly to future financial viability of the entities financed; operational capability and physical sustainability are built-in as the private operator would ensure these meet business needs. 7Including IFC / MIGA activities documented as part of the due diligence / underwriting process that aimed at enhancing sustainability at the onset, i.e. the project design phase. 137 NOTES 8In reviewing strategy documents for the period 2002-2011, documents produced in earlier years that continued to be of relevance to World Bank Group operations during the period 2002-2011 were included. 9Adequacy of maintenance was reviewed for each subsector (e.g. urban bus fleets, rural roads maintained by villages, intercity highways maintained by national agency. 10Some of the responses to these were based on interviews with Bank staff currently involved in the country/sector (e.g. transport sector Task Team Leaders) 11Including business case variables that address sustainability aspects for IFC / MIGA projects 12A list of all documents related to loans and other operational products, i.e. Project Appraisal Documents, Loan and Project Agreements, Implementation Completion and Results Reports, Implementation Completion and Results Report Reviews, Project Performance Audit Reports, and Analytic and Advisory Activities was prepared for each country study. There were also informal notes or other documents that individual teams produced for internal use and these were located through the Task Team Leader interviews. 13The condition of “transport infrastructure assets” includes the condition of highway networks, the availability rates of urban transport bus and railway locomotive fleets, depth of water maintained in port approach channels; condition of navigation aids, air traffic controls systems, and airport runways. User input could take various forms: e.g. representation on agency Boards; consultative 14 meetings with user groups; or an effective complaints handling mechanism. 15GAAPs came into use during the latter half of the period under review and hence for earlier projects similar arrangements may have been provided in Financial Management Action plans which focused on fiduciary controls. 16For IFC/MIGA assess both, IFC’s direct support (e.g. through transaction specific and country-wide Advisory Services) as well as IFC (or MIGA) indirect support in providing support in these areas, e.g. through unfunded policy dialogue or by making these sustainability-enhancing measures a condition before disbursement. 17This could be stand-alone institutions for transport planning or through creating capacity in existing institutions to undertake transportation studies. 18Strengthening policy and regulatory environment to sustain transport, refers to the sustainability of sector institutions and private entities in the sector. For instance, concession agreements that continue to be operable are critical to the sustainability of infrastructure concessions. 19InstitutionalCapability (ability to operate and maintain transport) and Financial Viability (adequacy of financial resources) were assessed as intermediate outcomes, in addition to the outcomes which were the sustained or improved benefit flows from the infrastructure and services over time. Social or environmental impacts are not considered here. This section examines whether the facilities and services that were introduced by the 20 World Bank Group and were financed by the World Bank Group were sustained, and; it 138 NOTES hence also examines the effectiveness of the World Bank Group in helping the countries sustained transport. 21For each case study project review, the degree to which transport has been sustained beyond project closure was assessed on a four point scale according to whether; (i) Scale 4— the key outcome indicators achieved by the end of the project were improved beyond project closure; (ii) Scale 3—the key outcome indicators either achieved or almost achieved by the end of the project were sustained beyond project closure; (iii) Scale 2—the key outcome indicators achieved by the end of the project were not sustained ; (iv) Scale 1— the key outcome indicators were not achieved by the end of the project (and have not shown improvements beyond project closure). Where there was no information available to make the above assessment, the rating was “not evaluable.” 22These questions do not include ones for waterways because there were very few projects that dealt with waterways in the portfolio reviewed. 139