Document of The World Bank Report No: ICR00003069 IMPLEMENTATION COMPLETION AND RESULTS REPORT (IBRD-76880, IBRD-78370) ON A LOAN IN THE AMOUNT OF US$493.425 MILLION TO THE STATE OF SÃO PAULO (BRAZIL) FOR A SÃO PAULO STATE FEEDER ROADS PROJECT December 23, 2015 Transport and ICT Global Practice Brazil Country Management Unit Latin America and Caribbean Regional Office CURRENCY EQUIVALENTS (Exchange Rate Effective December 15, 2015) Currency Unit = Brazilian Real (R$) R$3.88 = US$1 US$0.26 = R$1 FISCAL YEAR January 1 – December 31 ABBREVIATIONS AND ACRONYMS AF Additional Financing CAS Country Assistance Strategy CETESB State’s Company for Technology in Environmental Sanitation CPS Country Partnership Strategy DER-SP State Road Administration DNIT Transport Infrastructure National Department DO Development Objective EIRR Economic Internal Rate of Return FUNAI Brazil’s National Indian Foundation GDP gross domestic product HDM Highway Development and Management Model IBRD International Bank for Reconstruction and Development ICR Implementation Completion and Results Report IDB Inter-American Development Bank IMF International Monetary Fund IP Implementation Progress IRI International Roughness Index IRR Internal Rate of Return MICs middle-income countries NPV net present value PAD Project Appraisal Document PDLT State Logistics and Transportation Development Plan PDO Project Development Objective PMS Pavement Management System PPA Multi-Year Plan PPP Public Private Partnership ROW right of way R$ Brazilian Real SEP Secretariat of Economy and Planning (later becoming SPDR) SIAFIM Integrated Financial Administration System SIL Sector Investment Loan SLDP State Logistics Development Plan SLT State Secretariat of Logistics and Transport SMA State Environment Secretariat SPDR State Secretariat of Planning and Regional Development ST State Transport Secretariat (later becoming SLT) TA technical assistance ToR terms of reference UN United Nations US$ US dollar Vice President: Jorge Familiar Senior Global Practice Director: Pierre Guislain Country Director: Martin Raiser Practice Manager: Aurelio Menéndez Project Team Leader: Satoshi Ogita ICR Team Leader: Satoshi Ogita BRAZIL SÃO PAULO STATE FEEDER ROADS PROJECT CONTENTS Data Sheet A. Basic Information B. Key Dates C. Ratings Summary D. Sector and Theme Codes E. Bank Staff F. Results Framework Analysis G. Ratings of Project Performance in ISRs H. Restructuring I. Disbursement Graph 1. Project Context, Development Objectives and Design ............................................... 1 2. Key Factors Affecting Implementation and Outcomes .............................................. 6 3. Assessment of Outcomes .......................................................................................... 12 4. Assessment of Risk to Development Outcome......................................................... 19 5. Assessment of Bank and Borrower Performance ..................................................... 19 6. Lessons Learned ....................................................................................................... 21 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners .......... 22 Annex 1. Project Costs and Financing .......................................................................... 23 Annex 2. Outputs by Component ................................................................................. 24 Annex 3. Economic and Financial Analysis ................................................................. 27 Annex 4. Bank Lending and Implementation Support/Supervision Processes ............ 31 Annex 5. Beneficiary Survey Results ........................................................................... 33 Annex 6. Stakeholder Workshop Report and Results................................................... 34 Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR ..................... 35 Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders ....................... 37 Annex 9. List of Supporting Documents ...................................................................... 38 MAP DATA SHEET A. Basic Information Sao Paulo Feeder Country: Brazil Project Name: Roads Project IBRD-76880, IBRD- Project ID: P106663 L/C/TF Number(s): 78370 ICR Date: 12/23/2015 ICR Type: Core ICR STATE OF SAO Lending Instrument: SIM Borrower: PAULO Original Total US$166.65 million Disbursed Amount: US$493.425 million Commitment: Revised Amount: US$493.425 million Environmental Category: B Implementing Agencies: Departamento de Estradas de Rodagem do Estado de São Paulo (DER-SP) Cofinanciers and Other External Partners: B. Key Dates Revised / Actual Process Date Process Original Date Date(s) Concept Review: 09/27/2007 Effectiveness: 10/28/2009 10/03/2010 Appraisal: 04/29/2008 Restructuring(s): 01/31/2013 06/30/2014 Approval: 07/09/2009 Mid-term Review: 09/30/2011 03/30/2011 Closing: 06/30/2014 06/30/2015 C. Ratings Summary C.1 Performance Rating (by ICR) Outcomes: Satisfactory Risk to Development Outcome: Moderate Bank Performance: Satisfactory Borrower Performance: Satisfactory C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings Quality at Entry: Satisfactory Government: Satisfactory Implementing Quality of Supervision: Satisfactory Satisfactory Agency/Agencies: Overall Bank Overall Borrower Satisfactory Satisfactory Performance: Performance: C.3 Quality at Entry and Implementation Performance Indicators Implementation QAG Assessments Indicators Rating Performance (if any) Potential Problem Quality at Entry Project at any time No None (QEA): (Yes/No): Problem Project at any Quality of No None time (Yes/No): Supervision (QSA): DO rating before Satisfactory Closing/Inactive status: D. Sector and Theme Codes Original Actual Sector Code (as % of total Bank financing) Rural and Inter-Urban Roads and Highways 100 100 Theme Code (as % of total Bank financing) Rural services and infrastructure 100 100 E. Bank Staff Positions At ICR At Approval Vice President: Jorge Familiar Pamela Cox Country Director: Martin Raiser John Briscoe Practice Aurelio Menendez Jose Luis Irigoyen Manager/Manager: Aymeric-Albin Meyer /Eric R. Project Team Leader: Satoshi Ogita Lancelot ICR Team Leader: Satoshi Ogita ICR Primary Author: Mirtha Pokorny F. Results Framework Analysis Project Development Objectives (from Project Appraisal Document) The project development objective is to improve the efficiency of the paved municipal road network (according to the loan agreement). The PDO paragraph in the PAD also included the following description: “Efficiency will be improved as a result of improvements to the institutional framework used by the Borrower in four key areas to deliver adequate transport conditions to users (planning, environmental licensing, participation of private sector, and execution), combined with improvements in the condition of the paved municipal road network.” Revised Project Development Objectives (as approved by original approving authority) N/A (a) PDO Indicator(s) Original Target Formally Actual Value Values (from Revised Achieved at Indicator Baseline Value approval Target Completion or documents) Values Target Years Cumulated reduction of economic costs to users on municipal road network Indicator 1 : (Percentage) Value quantitative or 0.00% 5.70% 7.70% Qualitative) Date achieved 06/08/2009 06/30/2015 06/20/2015 Comments 135% achieved. The results of the economic evaluation demonstrated that the (incl. % cumulated reduction in the road user costs on the municipal roads under the Pro- achievement) Vicinais Program between 2011 and 2014 is 7.7%. Indicator 2 : Satisfaction of road users on paved municipal road network (percentage) Value quantitative or Not applicable 75% 91% Qualitative) Date achieved 06/30/2015 06/30/2015 Comments 121% achieved. These results are based on the interview surveys in 2015 with (incl. % road users of the paved municipal roads rehabilitated in Phases 3 and 4 under the achievement) Pro-Vicinais Program financed by this project and the Borrower. (b) Intermediate Outcome Indicator(s) Original Target Actual Value Formally Values (from Achieved at Indicator Baseline Value Revised approval Completion or Target Values documents) Target Years Indicator 1 : Evolution of condition of municipal paved road network 83% with 97.6% with IRI less 3,025 km with IRI less Value IRI less than 3.5 than 3.5 (good) and than 3.5 (good) and (quantitative (good) 0.4% with 7,115 with IRI more than or Qualitative) and 7% with IRI IRI more than 5 5 (bad) more than 5 (bad) (bad) Date achieved 06/08/2009 06/30/2015 06/11/2015 118% achieved. These results are based on the road surface condition survey in Comments 2015 on the sampled municipal road sections under the Program. When IRI less (incl. % than 3.5, road is considered in good conditions. When IRI more than 5, road is in achievement) bad. Annual decrease of turn-around time for evaluation of environmental Indicator 2 : licensing requests Value (quantitative 0% (240 days) 30.00% 50.00% or Qualitative) Date achieved 06/08/2009 06/30/2014 12/31/2014 Comments 135% achieved. In 2014, the turn-around time for evaluation of environmental (incl. % licensing requests for medium and small projects was 120 days, 50% less than achievement) the baseline value of 240 days. See the details in Section 3.2 and Annex 2. Indicator 3 : Private/external capital mobilized for financing state investment programs Value (quantitative R$11.6 billion R$19.0 billion R$25.6 billion or Qualitative) Date achieved 06/08/2009 06/30/2013 06/22/2015 Comments Target met. The private and external investments toward the state’s projects in (incl. % 2011-2014 was R$25.6 billion in total, including metro, highway, and energy achievement) projects. This indicator is influenced by many factors (see Section 2.3). Agenda agreed of public and private sector actions to reduce logistics Indicator 4 : bottlenecks Value (quantitative N/A Agenda agreed Agenda agreed or Qualitative) Date achieved 06/08/2009 06/30/2014 06/11/2015 Comments Target met. The secretariat of logistics and transport prepared and agreed on the (incl. % agenda of the state logistics development plan along the Tiete river involving achievement) both public and private sectors. Increase of engineering designs bid for road rehabilitation investments Indicator 5 : identified as priority by the pavement management system (PMS) Value PMS partially (quantitative 0.00 70.00 developed or Qualitative) Date achieved 06/08/2009 06/30/2014 06/11/2015 Target partially met. The project developed the road inventory database and Comments road geo-reference system, both of which consist of the important parts of the (incl. % PMS. The development of PMS is still ongoing at ICR and has not been in achievement) operation yet. Number of road km rehabilitated as part of Pro-Vicinais Program Indicator 6 : (Kilometers) Value 2,117.00 11,900.00 11,968.00 (quantitative or Qualitative) Date achieved 06/08/2009 06/30/2014 06/11/2015 Comments 101% achieved. The Pro-Vicinais Program rehabilitated 11,968 km of the state (incl. % paved roads, out of which the project financed 3,300 km. See the details in achievement) Section 3.2. Average percentage delay in civil works contracts concluded during a given Indicator 7 : year under the Pro-Vicinais program Value (quantitative 35% 18% Dropped or Qualitative) Date achieved 06/08/2009 06/30/2014 06/30/2014 This indicator was replaced by Indicator 6 above because the work contract Comments duration was short, and any event (such as a long wet season) could alter the (incl. % required work period, but which has no relation to the performance of DER-SP. achievement) G. Ratings of Project Performance in ISRs Actual Date ISR No. DO IP Disbursements Archived (US$ million) 1 08/28/2009 Satisfactory Satisfactory 0.00 2 05/15/2010 Satisfactory Satisfactory 139.32 3 02/09/2011 Satisfactory Satisfactory 297.40 4 06/28/2011 Satisfactory Satisfactory 412.59 5 01/16/2012 Satisfactory Satisfactory 462.96 6 08/13/2012 Satisfactory Satisfactory 472.93 7 03/22/2013 Satisfactory Satisfactory 485.71 8 11/25/2013 Satisfactory Satisfactory 487.74 9 04/05/2014 Satisfactory Satisfactory 488.74 10 10/27/2014 Satisfactory Satisfactory 489.02 11 06/24/2015 Satisfactory Satisfactory 489.02 H. Restructuring (if any) ISR Ratings at Amount Board Restructuring Restructuring Disbursed at Reason for Restructuring & Approved Date(s) Restructuring Key Changes Made PDO Change DO IP in US$ million This Additional Financing (AF) was approved for a 08/03/2010 N S S 153.75 US$326.775 million loan for the Project’s physical component to scale-up ISR Ratings at Amount Board Restructuring Restructuring Disbursed at Reason for Restructuring & Approved Date(s) Restructuring Key Changes Made PDO Change DO IP in US$ million Bank support to the program and to cover cost overruns. Bank support for project scale- up resulted from the impact of the global crisis which had affected the State’s capacity to contribute to the financing of the program. The PDO and the targets remained unchanged. The restructuring was meant to optimize the use of the proceeds of the parent loan by distributing the unallocated amount between component (1) in support of the State program of rehabilitation of municipal 01/31/2013 N S S 473.68 paved roads, (US$6,084,375), and component (2), in support of institutional strengthening activities, primarily to one of the Beneficiary entities, the State Secretariat of Transport, (US$3,149,000). The PDO and the targets did not change. The restructuring was made to extend the project closing date from June 2014 to June 2015 to allow substantial completion of 06/30/2014 N S S 489.02 several key institutional strengthening activities which had been delayed. The PDO and the targets did not change. I. Disbursement Profile 1. Project Context, Development Objectives and Design 1.1 Context at Appraisal General characteristics. The state of Sao Paulo is a major economic center of not only Brazil but also of its region, with a Gross Domestic Product (GDP) larger than Argentina or Chile, and topped in Latin America only by Brazil and Mexico. Although its economic growth had been historically higher than the national average of Brazil, during the 1990s its share of the national gross production shrank from 39 percent of the GDP in 1985 to 35 percent in 2006. The state identified the need to foster growth, especially after investments were curtailed during the fiscal adjustment period of the mid - 1990's to early 2005. In 2007, as a result of renewed growth, a positive outlook for the economy, and a regained fiscal space, the administration decided to increase the level of investments during the 2008-2011 period by a 2.4 factor (relative to 2003-2007), with a 45 percent share expected to be financed by the private sector. Out of the R$60 billion in investment that occurred over the subsequent four years, nearly R$40 billion were allocated to the transport sector. In addition, the state launched an important institutional capacity-building effort which aimed to promote a professional, efficient, and modern administration, open to citizen control and participation. Sector context. The state government focused on its regional development agenda with special emphasis on the areas of logistics and transport. The state’s transport strategy, as described in the state’s multi-year development plan (PPA) 2008-2011 and the master plan of transport development (PDLT), aimed at reduction of transport costs through rehabilitation of paved municipal roads under the state's municipal paved roads rehabilitation program (the Pro-Vicinais), the development of the main road arteries (rehabilitation and expansion of state roads), a reduction of logistics bottlenecks to support exports (implementation of export corridors), and a gradual rebalancing of the transport matrix. The Pro-Vicinais Program was launched in 2007 and aimed at rehabilitating all deteriorated paved municipal feeder roads in the state, a network of roads covering an overall extension of 12,800 km at an estimated cost of US$1.8 billion. These paved feeder roads constitute the backbone of the road networks in the 645 municipalities in the state, and provide key access between the state’s main transport corridors. Given the limited financial and technical capacity of most municipalities, the state has been financing the progressive paving of these roads and managing their maintenance for the past 30 years. In spite of these efforts, by the mid 2000’s, the condition of the roads had deteriorated to the point where over 60 percent of the roads were in poor condition. This situation prompted the Sao Paulo government to launch this program, which was designed to substantially improve the condition of the municipal paved roads. In the area of public sector management, the Secretariat of Economy and Planning (SEP, later renamed SPDR) focused on consolidating its capacity to: (i) implement result-based management principles in the government business processes, especially in its multi-year 1 planning tool; (ii) manage efficiently the state planning and budget needs; and (iii) support the development and implementation of new partnerships with the private sector (PPP) as well as with multilateral agencies. In the area of environmental protection, the Secretariat of Environment (SMA) worked on the modernization of the state's environmental management system to improve efficiency in analyzing impacts, licensing economic activities, managing and monitoring environmental quality, and protecting and rehabilitating biodiversity. Public sector management improvements were also being made in the Secretariat of Transport (ST, later renamed SLT) by further developing the state's strategic transport plan, improving the state's capacity in structuring and regulating public- private partnerships, and planning and executing public investment programs. Rationale for Bank involvement. The project supported the Country Partnership Strategy (CPS) 2008-2011 on the public sector management element of the macro foundation and on the competiveness pillar. Also, the project focused at the sub-national level - in a state led by a highly motivated governor - on a program which was of high priority, had political support, and which the CPS specifically recommended to target. The Sao Paulo Feeder Roads Project was designed to assist the state in implementing the third phase of the Pro-Vicinais program. (See Map 1.1) At the time of appraisal, work under the Pro-Vicinais Program's first phase (2,100 km financed by the Borrower at a total cost of US$175 million) was nearing completion, work under the second phase (2,550 km at a cost of US$117 million financed by the Borrower and with a US$180 million loan from the Inter-American Development Bank) was ongoing, and under the third phase, bidding of civil works (3,100 km) was underway and was to be supported by the project. In addition, the state was structuring a fourth phase of the program, covering an additional 4,200 km of municipal roads with the objective of undertaking a significant part of these civil works in 2010 (See Map 1.2). At the time of appraisal, the state indicated that it had already requested Federal Government authorization for another Bank loan of US$395 million to finance services and works under the third and fourth phases and was expecting to soon receive the Federal Government's formal authorization to prepare to receive various new loans. In addition to the support of the Pro-Vicinais program, extensive consultations between the Bank and the state of Sao Paulo led to the latter’s request to also include in the project the assistance of the Bank in improving public administration efficiency in planning and managing infrastructure programs, specifically: (i) improving public administration efficiency in planning and managing large infrastructure programs; (ii) modernizing environmental aspects of investment; and (iii) consolidating the state’s capacity in attracting private sector participation in infrastructure.1 1 In early 2008, the State of Sao Paulo and the Bank agreed to process six operations in the urban transport, road transport, water sector, and rural development areas. In addition to these discrete lending operations the state requested technical assistance from the Bank for (i) planning and managing large infrastructure programs; (ii) attracting private sector participation in infrastructure; and (iii) environmental licensing. While 2 1.2 Original Project Development Objectives (PDO) and Key Indicators (as approved) The project development objective (PDO), as stated in the Loan Agreement, is “to improve the efficiency of the Borrower’s paved municipal road network”. The PDO paragraph in the Project Appraisal Document (PAD) also included a description to define the efficiency mentioned in the above PDO: “Efficiency will be improved as a result of improvements to the institutional framework used by the Borrower in four key areas to deliver adequate transport conditions to users (planning, environmental licensing, participation of private sector, and execution), combined with improvements in the condition of the paved municipal road network”. The key indicators were reduction of economic costs to users on the paved municipal road network, and satisfaction of users of the paved municipal road network. 1.3 Revised PDO (as approved by original approving authority) and Key Indicators, with reasons and justification Not revised. 1.4 Main Beneficiaries Road users and road transport services on the municipal roads under the Pro-Vicinais Program directly benefit from the civil works component of the project through decreased transport costs. More specifically, the improved condition of local roads promoted rural communities in enhancing their accessibility to health services, education, and markets. Furthermore, the entire Sao Paulo state and even the country as a whole was expected to see indirect benefits from the project’s positive impact on the capillarity of the road network and on the resulting competitiveness, employment, and economic growth. Finally, the technical assistance component, while initially benefitting the recipient entities, was also expected in the longer run to have a wide impact on the entire population through improved public sector management, reduced logistics costs, and more efficient environmental management. such technical assistance could have been provided under any of the six operations, it was agreed that it would be best suited as a complement to this project (PAD, pg. 7). This was considered reasonable since all of the technical assistances were related to municipal roads directly or indirectly. 3 1.5 Original Components The original project components, as described in the Loan Agreement, were as follows: Component 1: Feeder Roads Rehabilitation (at an estimated cost of US$477.04, of which US$154.65 million was to be financed by the loan) (a) Carrying out of rehabilitation works on approximately 100 km of the Borrower’s paved road sections, and 1,500 km of particular sections of the Borrower’s municipal paved road network that have been identified as eligible road sections under the 3 first phases of the Pro-Vicinais Program including, inter alia: (i) rehabilitation of existing road bases and surfaces, as well as of bridges; (ii) rehabilitation and/or surfacing of shoulders to protect the shoulders’ surfaces from erosion and improve the driving conditions; (iii) upgrading of drainage systems and signalization; and (iv) road safety improvements, including localized curve realignments and critical spots elimination. (b) Consolidating DER-SP’s capacity to manage its road investment programs, including the Pro-Vicinais Program. Component 2: Institutional Strengthening (at an estimated cost of US$23.80 million, of which 12 million was to be financed by the loan) (a) Strengthening of SEP’s institutional capacity to manage the State Investments Program, through the provision of technical assistance, training and equipment for, inter alia: (i) the structuring and initial operation of a monitoring and evaluation system of the State Investments Program; (ii) the definition of a methodology to identify and prioritize projects potentially attractive to the private sector; (iii) the preparation and structuring of projects with private sector participation; (iv) carrying out of improvements to SEP’s capacity in obtaining external financing (including from multilateral agencies); and (v) the elaboration of a strategy for communication with key stakeholders under the State Investments Program, including federal and municipal governments, private sector and civil society. (b) Strengthening of SMA’s and CETESB’s institutional capacity to support the modernization of the Borrower’s environmental management system, through the provision of technical assistance, training and equipment for, inter alia: (i) the definition and implementation of the structure of the Borrower’s unified environmental management organizational structure and related processes; and (ii) the integration, modernization and expansion of related information management systems, including geo-referenced monitoring. (c) Strengthening of ST’s institutional and planning capacity to support the Borrower’s transport sector, through the provision of technical assistance for, inter alia: (i) the carrying out of a study of transport demand for general cargo, including elaboration of the study’s methodology and surveys with research on specific logistics chains; (ii) the identification and selection of public and private sector actions on the demand side to motivate an optimized use of existing transport infrastructure and services; and (iii) the structuring of a methodology to assess economic and social impacts of road investment programs. 4 (d) Strengthening of DER-SP’s institutional capacity to support the Borrower’s road sector, through the provision of technical assistance, training and equipment for, inter alia: (i) consolidating DER-SP’s pavement management system, structuring a bridge management system and monitoring road conditions and traffic level data; (ii) increasing the technical capacity of DER-SP staff; (iii) structuring of traffic monitoring systems; (iv) managing and monitoring environmental and social impacts of road investment programs, and providing technical support to municipalities in environmental management and monitoring of civil works; and (v) supporting municipalities in defining and implementing mechanisms for sustainable maintenance of municipal networks. 1.6 Revised Components In mid-2010, the state requested and the Bank agreed to an Additional Financing (AF) Loan (Loan 7837-BR) in the amount of US$326.775 million to finance the rehabilitation of 1,200 km out of 3,300 km in Phase 4 of the Program. In addition, the AF loan included funds for 700 km in Phase 3, which was originally planned to be financed by the parent loan but which could not be covered due to cost overruns. The cost overruns were mainly caused by: (a) an unfavorable exchange rate during the first year of project implementation; and (b) increased work due to the application of heavier than originally estimated solutions as a result of the improved engineering standard developed under the project to increase the sustainability of the municipal road assets. Accordingly, the extension financed by the parent project decreased from 1,500 km to 800 km. With the AF, the total project support was to reach 2,700 km or 23 percent of the program, against the adjusted 8 percent under the Parent Project. The AF was approved by Bank management on August 3, 2010. The AF component as described in the Project Paper was as follows: Anticipated cost overruns and scale-up and of the Project’s Feeder Roads civil works component (estimated cost US$408.78 million, of which US$326.775 million financed by the loan) Part 1 (a): Feeder Roads Rehabilitation Works (i) Financing of anticipated cost overrun of works to be financed under the Original Project covering approximately 700km of eligible road sections; and (ii) Carrying out of rehabilitation works on approximately 1,200 km of additional specific sections of the Borrower’s municipal paved road network not included in the Original Project, that have been identified as eligible road sections under the 4 first phases of the Pro-Vicinais Program including, inter alia: (A) rehabilitation of existing road bases and surfaces, as well as of bridges; (B) rehabilitation of shoulders to protect the shoulders’ surfaces from erosion and improve the driving conditions; (C) road safety improvements, including horizontal traffic signaling; and (D) rehabilitation and upgrading of drainage systems. 5 Part I (b). Project Management Consolidating DER-SP’s capacity to manage its road investment programs, including the Pro-Vicinais Program. 1.7 Other significant changes In addition to the AF mentioned above, the project was restructured twice, in 2013 and 2014. The restructuring in 2013 was meant to optimize the use of the proceeds of the parent loan by distributing unallocated amounts to both Component 1 and Component 2, as follows: to support the Component 1 State program of rehabilitation of municipal paved roads, US$6,084,375 were reallocated to Component 1, and to support the Component 2 institutional strengthening activities, primarily to one of the Beneficiary entities, the State Secretariat of Logistics and Transport, US$3,149,000 were reallocated to Component 2. In addition to the reallocation, the indicator on the average percentage delay in the execution of civil works contracts was eventually considered not the most appropriate indicator to measure the intermediate outcome2 and was replaced under the AF by one of the Bank Core Indicators, namely the number of km of municipal roads rehabilitated under the program. The restructuring in 2014 extended the closing date of the parent project from June 2014 to June 2015 to allow substantial completion of several key institutional strengthening activities that had been delayed, as explained in Section 2.2 below. The AF loan closed as planned on June 30, 2014. 2. Key Factors Affecting Implementation and Outcomes 2.1 Project Preparation, Design and Quality at Entry A number of important factors in the project preparation, design and quality at entry affected the implementation and outcomes of the project as follows: The project design was built directly on Brazil’s and the state of Sao Paulo’s priorities. The 2008-2011 World Bank Group Country Partnership Strategy (42677-BR), approved by the Bank’s Board of Directors on May 1, 2008, confirms the validity of pursuing Bank assistance to Brazil within the structure defined in the 2003-2007 CAS, including a macro foundation (macro fundamentals, good governance, and public sector management) and three pillars (equity, sustainability, and competiveness) to which the improvement of regional connectivity was closely linked. In addition, the CPS also recommended that the Bank be engaged at the sub-national level with the states, which were led by highly 2 The rationale for the change was that work contracts durations were short and any particular event beyond the handling of DER-SP (such as an exceptionally long wet season) could potentially substantially alter the value of the indicator with no relation to the performance of DER-SP in contracts management. 6 motivated governments, which was the case in the state of Sao Paulo. The State Government maintained a strong commitment to the Pro-Vicinais program over the entire implementation period. The project incorporated the main features identified to improve the Bank’s effectiveness in partnering with middle-income countries (MICs)3. In particular; (i) the project design had to be streamlined to reduce the non-financial costs of doing business, and to that end the project accepted Brazil’s procedures related to financial management, definition and use of technical standards for civil works, as well as identification, prevention, and mitigation of the environmental impact of road rehabilitation; (ii) the project had to offer added value beyond financing, and accordingly, the project was designed to provide hands- on assistance, such as through project preparation, to improve the efficiency of the Pro- Vicinais Program. This assistance included the development of a cost-effective and sustainable solution catalogue which specified different types of road interventions based on existing road and traffic conditions, and also included a horizontal technical assistance component, implemented through the Secretariats of Planning, Environment, and Transport, to plan, finance, and execute infrastructure investment programs; and (iii) because the Bank’s impact can be greater at a program versus an individual project level, the project was structured in support of the program and to engage with the government in program-level discussions on improving its effectiveness and efficiency. The project was heavily client-oriented rather than product-oriented, and was tailored to answer the specific requests of the government of Sao Paulo. The project supported the highly prioritized program of the state, which had a strong political commitment. Furthermore, the project not only met the needs of the state for the implementation of the Pro-Vicinais program, but also helped to improve the state’s position as the normative and regulatory authority of the municipal roads. This was done by providing cross-cutting support in areas transcending the transport sector such as: (i) improving public administration efficiency in planning and managing large infrastructure programs; (ii) consolidating capacity in attracting private sector participation in infrastructure; and (iii) modernizing the state's capacity for evaluating, licensing, and monitoring environmental and social aspects of infrastructure investments. A range of risks related to implementation was identified during project preparation, and appropriate mitigation measures were considered. Those risks, mostly related to government commitment, the municipalities’ willingness to benefit from the program, and the ability of DER-SP to handle procurement, financial management, and safeguarding of policies were considered low or moderate. None of those risks materialized during project implementation. In fact, the impressively rapid implementation of civil works and the resulting positive impact on the PDO can be attributed to a design that relied on the existence of a government-led comprehensive program along with the commitment and 3 Policy briefing based on strategy paper presented to the World Bank/IMF Development Committee during the Annual Meetings in Singapore, September 18, 2006, and prepared by the staff of the World Bank. 7 availability of donor resources and institutional capacity to deal with safeguarding and fiduciary issues. The only risk that materialized - factors contributing to a late start of the technical assistance component - was not identified at the time of appraisal. It is clear that highly complex information management systems that substantially affect the modus operandi of its user require a significant amount of lead time, and are subject to a risk of delays that could be mitigated through a more incremental approach (for example, modularization). Delays in other TAs, however, mostly related to changed circumstances, would have been difficult to foresee. Project components were appropriately related to the objective of the project. The civil works component focused on the physical condition of the municipal road network, while the institutional development component addressed the normative and regulatory aspects that affected it. The four key areas tackled under the project to provide adequate transport conditions to users (planning, environmental licensing, participation of private sector, and execution) benefit not only the municipal road network but also most infrastructure sectors, thus creating a “substantial positive externality”. 2.2 Implementation The implementation of the civil works component of the project was carried out promptly and in a problem-free manner. Although the achievements of the institutional strengthening component were also considerable, diverse factors affected its implementation pace and in some cases its scope. The following paragraphs highlight the main issues and aspects of project implementation: Component 1 - Feeder Roads Rehabilitation. Most of the civil works under this component were completed within three years after the approval of the project. Some key features facilitated its swift implementation and the rapid disbursement of loan funds, including the following: First and foremost, DER-SP maintained a high level of efficiency in executing the Pro- Vicinais Program, which insured that there was a “constant supply” of eligible projects. Because of the project, with borrower and IDB financing, the Program improved 11,900 km of provincial roads in only five years. Realistic eligibility criteria effectively streamlined project execution: These included: (i) an internal rate of return (IRR) greater than 15 percent; (ii) restrictions on the same expenditure not being financed by more than one multilateral agency; (iii) treatment of environmental and social impacts according to DER-SP’s accepted procedures4; and (iv) standardized engineering designs created according to an agreed manual, which was developed by the Bank team. 4 Resettlements were not expected and did not occur. 8 Bank Financing was very accommodating. This included a waiver of the Bank’s rule for a retroactive financing limit of 20 percent of the loan amount and an allowance of retroactive financing up to 35 percent (US$58 million for the parent project and US$114 million for the AF) to finance eligible contracts, which consisted of eligible expenditures primarily in the third phase of the program. As a result, the Bank disbursed US$225 million in FY11 alone, accounting for 46% of the total loan amount of both parent and AF projects. Component 2 - Institutional Strengthening. The technical assistance activities under this component were generally implemented in a satisfactory manner. (The detailed activities and outcomes of the component are summarized in Annex 2). While most of them were fully launched by 2012, some activities, as explained in the following paragraphs, were required to change their scope from the one originally planned, which led to the extension of the project by one year. The total cost of the component was US$10.17 million, about 57 percent below appraisal estimates. The cost reduction is primarily explained by the fact that most of the institutional development component was implemented in the last two years of the project, a time during which a favorable exchange rate reduced its dollar cost. Diverse factors affected the pace of the implementation and the changed scope of the activities under the institutional strengthening component. In particular, these included the following: Circumstances changed and a late decision was made on the alternative use of loan funds. This involved the Technical Assistance (TA) provided to SEP (which later became SPDR) for Component 2 (a) of the project. Due to initial delays, in view of the fact that a change of government had become imminent, it was felt that investment decisions should be left to the next Administration. Eventually the Borrower and the Bank agreed to finance activities leading to the improvement of investment management that were successfully completed in 2015 (see Annex 2). Thus, after the TA was originally suspended, and after further discussions with SPDR for several more years, two additional TAs were launched in 2012-13 to enhance an investment management capacity (see Annex 2). The time originally allocated for the implementation of complex information technology systems was too optimistic. This involved implementation of technical assistance to SMA/CETESB (Component 2 (b)) for the development of a unified environmental licensing system and an integrated geo-referenced system, where the TA was designed to have SMA’s strong support and ownership. Because of the need to (i) assess alternatives, (ii) reach a consensus within SMA on an effort that was to have a deep institutional impact, (iii) develop intricate terms of reference (ToR), and (iv) comply with Bank procurement rules on which SMA/CETESB had little experience, the technical assistance was only launched in mid-2012, and was successfully completed in mid-2015. A pragmatic and gradual approach was applied for developing a large data management system instead of development of the entire system at once, which required longer time than had been estimated. In the case of the TA to DER-SP (Component 2 (c)), the first Pavement Management System (PMS) had been purchased in late 2000 but was never made operational due to its high level of complexity. Alternatively, the project developed 9 the road mapping and geo-referencing system and the road inventory database system of state and municipal road network, which was to be an essential part of the integrated PMS. The project also assisted the DER-SP in introducing the Highway Development and Management Model (HDM) software, a core management module of the PMS. However, the progress of integrating these systems to conclude the PMS was slow, in part because it required a large and fairly effective institution being decentralized into powerful regional offices which had a strong tendency towards maintaining the status-quo. Because of the rapid project implementation, the mid-term review was advanced. By mid- 2011, in less than two years from going into effect, the project had disbursed 76 percent of loan funds and had already committed 96 percent of them. The mid-term review was thus triggered to take place about six months prior to the planned mid-term review date, because disbursement at that time exceeded 50 percent of the loan amount (including the additional financing signed in 2010). The review mostly (i) encouraged DER-SP to improve the quality of works so as to not reduce the number of eligible projects; and (ii) emphasized the need to increase project support activity by agencies, such as SPDR, SLT, and DER- SP, which had been slower in launching TAs under the project, in order to avoid the reallocation of funds among recipients. 2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization The result indicators established for the project were adequate. In general, the selected indicators were highly related to the project’s expected outcomes, which centered on the successful completion of the Pro-Vicinais program. Indicators were simple, precise, and easily measurable. There are several minor shortcomings. As already mentioned in the preceding section 1.7, one of the indicators – the indicator on the average percentage delay in the execution of civil works contracts – was found to be unsuitable for measuring the efficiency in the implementation of civil work contracts having only a short execution period, and was replaced by a more adequate Bank’s Core Indicator - the number of km of municipal roads rehabilitated under the program. The only other exception to the adequacy of the indicators was the indicator related to the increase in private sector investments in infrastructure. This is because the amount of foreign investment largely depends on the country’s business climate, global economic environment, and investor decisions, all of which are mostly beyond the project’s influence. The indicators could have been further improved by including one to measure an achievement of an activity to enhance the capacity of municipalities that are principally responsible for maintenance of municipal roads considering its importance in sustainability. For implementation and utilization, the responsibility for monitoring of results was shared among the different beneficiaries in their respective areas of responsibility. The monitoring and evaluation (M&E) was implemented as planned. Semi-annual reports were regularly submitted to the Bank on time and in proper form, and in general, there were no difficulties in terms of measuring result indicators. These included a satisfaction survey of 2,467 users of light and commercial vehicles (see the details in Section 3.2). 10 2.4 Safeguard and Fiduciary Compliance (a) Safeguards: Environmental. At appraisal, the project was accorded Category B status in view of the fact that rehabilitation and maintenance work was foreseen on existing roads and that no significant negative impacts were expected. As previously mentioned, the Bank accepted Brazil’s procedures for the identification, prevention, and mitigation of the environmental impact of road rehabilitation. DER-SP hired an environmental specialist to ensure that proper procedures were closely followed and that there were no major impacts or safeguard compliance issues arising during project implementation. Social. Neither OP 4.12 on Involuntary Resettlement nor OP 4.10 on Indigenous People were triggered by the project. There was no land acquisition or physical displacement because all civil works and support activities were conducted on lands situated within the existing right of way (ROW). Because of the complete absence of any structures or other household assets along the ROW boundary, even the marginal widening of a road shoulder to improve road safety or drainage did not result in any adverse social effects on any of the land owners present in the project area. With respect to OP 4.10, very few indigenous peoples remain in the state of Sao Paulo - approximately 60,000 in all, or 0.2 percent of the total population - and most of them now live in urban areas. Brazil’s National Indian Foundation (FUNAI) recognizes the existence of 17 indigenous territories in the state, 12 of which are legally demarcated, one of which is in the demarcation process, and four of which have yet to begin this process. None of the project roads are located in the vicinity of any lands legally occupied or claimed by indigenous peoples. Bank missions confirmed that the management of the risks and impacts identified in OP 4.12 was conducted in a satisfactory manner, and that there were no indigenous peoples present in, or with collective attachments to, the project area. (b) Fiduciary: Procurement: Given the experience and institutional capacity within DER-SP, procurement of goods, works, and services under the project was carried out satisfactorily in accordance with agreed procedures. Procurement processes of complex information technology services tended to be lengthy and meticulously reviewed by the supervision team, an effort recognized by the client as key to the smooth performance of the consulting firms. There were no procurement issues reported. Financial Management (FM). The Financial Management Specialist stationed in Brasilia periodically carried out formal FM supervision. The FM performance was rated as satisfactory at the beginning of the project but was downgraded to moderately satisfactory after August 2011 when small shortcomings were observed, including minor inconsistencies between the designated account balances presented by the project and the account balances observed by the external auditors through the state’s Integrated Financial Administration System (SIAFEM). These inconsistencies, however, did not compromise the provision of timely and required information for project monitoring. Another 11 shortcoming was the delay of DER in implementing the controllership department (hiring internal auditors). The audit reports were rated as satisfactory, although they were sent with delays, and in most cases (except for 2013) included unqualified opinions about financial statements. 2.5 Post-completion Operation/Next Phase Municipal governments are responsible for maintaining the municipal roads rehabilitated by the program. All municipalities that adhered to program were to commit to the adequate financing of their roads. Most of the technical assistance activities under the institutional strengthening component in the key areas (planning, environmental licensing, participation of private sector, and execution) will continuously improve the efficiency of the state government after the project is completed. For example, the deliverables prepared by the project are used for daily operations by the state agencies, including the road inventory and geo-referenced information system for DER-SP, the environmental licensing and geo-reference systems for SMA/CETESB, and the Public Private Partnership (PPP) toolkit for SPDR. The Bank approved in 2013 the Sao Paulo Sustainable Transport Project for Brazil. (P127723, Loan 8272-BR), which supports activities, particularly on the transport and logistics planning and environmental management, that were initiated under the subject project. In addition, the technical assistance under the new project includes promoting the road safety agenda, which also includes users of municipal roads. 3. Assessment of Outcomes 3.1 Relevance of Objectives, Design and Implementation Rating of relevance of objective: High The objective of the project remains highly relevant to the government’s and Bank’s current agendas 5 . The efficiency of the tertiary road network is key in improving connectivity, reducing the distance between people, markets, and services, and closing knowledge gaps. The most recent state four-year development plan (the current 2012-2015 Plano Pluri Annual, PPA), identifies as key pillars the sufficiency of transport infrastructure and logistics, the support for high levels of competitiveness, and access to domestic and foreign markets. Similarly, the project objective of improving the efficiency of the municipal road network remains highly relevant in the current CPS 2012-2015, notably under its third and fourth pillars, one of which is to promote regional economic development through improved policies and strategic infrastructure investments, and the other of which is to support the private sector in frontier areas and to further improve 5 As described in the section 1.1 and 2.1, the objective of the project was also highly relevant to the State's development priorities and Bank's strategies at appraisal. 12 sustainable natural resource management and enhancement of climate resilience while at the same time contributing to local economic development and helping to meet rising global food demand. The municipal road network plays an important role in the success of the objectives of both of these pillars. In addition, the continued high relevance of the project was further confirmed by the request of the Sao Paulo government for continuing Bank assistance in the areas of transport and logistics planning and of environmental management initiated under this project. As a result, the Bank approved in 2013 the Sao Paulo Sustainable Transport Project for Brazil (Loan 8272-BR) which focused on increasing the Borrower's transport and logistics efficiency and safety while enhancing its capacity in environmental and disaster risk management. The design of the latest operation also continues the successful approach initiated under the subject project, that is, transcending transport through cross-cutting, horizontal support in areas that are key to the sustainability of infrastructure development in general. Rating of relevance of project design/implementation: High The relevance of the project design and implementation is considered high. The rating takes into account that the main component, the financing of the Pro-Vicinais Program which comprised 98 percent of the loan amount, was structured in support of a state-level program rather than in individual municipal-level projects. This enabled the project to focus on engaging the government on program-level discussions on improving its effectiveness and efficiency. The rapid project implementation that was based on reducing transaction costs, and the effective role of the Bank in strengthening the technical quality of the engineering of the road works, both attest to the soundness of the design and its relevance for similar operations in MICs. The primary outputs and the intermediate outcomes are all consistent with the PDO. The outputs, including the rehabilitation of almost 12,000 km of municipal roads, and TAs of the institutional strengthening component, contributed to the improvement of the efficiency of the municipal road network through better physical municipal road conditions and through the enhanced management capacities demonstrated in the planning, environmental management, and management of both municipal roads in particular and of infrastructure in general. 3.2 Achievement of Project Development Objectives Rating: High The efficiency of the operation of Sao Paulo’s paved municipal roads was highly improved as demonstrated by the following results: Reduction of economic costs to users of the paved municipal road network. The rehabilitation of the municipal roads under the Pro-Vicinais program resulted in an 13 estimated reduction of the costs to users of 7.7 percent, surpassing the original PDO indicator target of 5.7 percent. Degree of satisfaction of road users. In early 2015, DER-SP carried out a survey of road users6 of the network that was improved under the 3rd and 4th phases of the Pro-Vicinais Program. The survey covered 2,047 users comprising 2,078 light vehicles and 380 heavy ones. The results demonstrated that 91 percent of the survey participants are satisfied with the improvements, surpassing the original PDO indictor target of 75 percent. Those who are not satisfied in general reflect the fact that some of the roads do not have a shoulder lane, which was not included in the program’s scope due to low cost effectiveness. Intermediate outcomes and outputs: The successful implementation of civil works under the Pro-Vicinais program, and an institutional strengthening component, with the support of the project and other financiers, contributed to improving the efficiency of the paved municipal road network, as shown below: (a) Rehabilitation occurred on 11,968 km of municipal roads by June 2015, as compared to a target of 11,900 km (including 3,300 km financed by the project and the remainder financed by the Borrower and IDB. See Table 3.1 below for details. Table 3.1: Pro-Vicinais Program Financing Status Costs by Financier (US$ million) World Bank World Bank Extension Parent Additional Phase (km) Project Financing IDB Borrower Total 1 2,117 175 175 2 2,525 180 117 297 3 3,112 160 115 352 608 4 3,297 211 309 402 Concession* 917 Total 11,968 160 326 180 953 1,482 * 916.8 km of the municipal road sections were rehabilitatedor maintained under the state road concession contracts with the private sector (b) Conditions of the paved municipal road network were improved from the 2009 baseline of only 23.6 percent in good condition (IRI<3.5) versus 55.6 percent in poor condition (IRI>5), to the current situation of 97.6 percent of the network in good condition, well above the original target of 83 percent, according to the road surface condition survey on the sampled municipal road sections under Phases 3 and 4 of the Program. 6 The survey was financed by the counterpart funds. 14 As the component 1 (b) of the project, and to a great extent thanks to a project design that allowed the supervision team to concentrate on program rather than project issues, the Bank was instrumental in the development and subsequent DER-SP’s adoption of a process to identify, on the basis of life-cost analysis, the most technically efficient road works based on traffic levels and relevant engineering parameters. These efforts resulted in the adoption of more costly alternatives under the 3rd Phase of the program which included more high-traffic and high-volume roads than under the 4th phase, while ensuring more sustainability in road assets for the long-run. Contributions to the improvement of the efficiency of the institutional frameworks for the municipal road network (and for other infrastructure sectors) were made in the following four key areas: planning, environmental licensing, participation of the private sector, and execution. These were shown by the achievement of the targets of most of the indicators monitoring the institutional strengthening of the project, as follows: Planning: The project (component 2(c)) successfully assisted SLT in preparing the agenda of the State Logistics Development Plan (SLDP), of which municipal roads are an integral component. During the project, the SLT developed the agenda of SLDP along the Tiete River, involving both public and private sectors, and focused on reduction of logistics costs for bio-fuel and for agriculture products such as sugarcane and soybeans. Meanwhile, the State Plan for Logistics and Transport 2030 (PDLT) is still under preparation at ICR. The project financed two activities: (i) a substantive study on freight demand which linked input-output and origin-destination matrices and which is already proving its usefulness in estimating freight volumes when related to a general equilibrium economic model, which was used for the demand projections of the Tiete-River logistics development plan; and (ii) studies on specific issues of the state’s transport planning, such as access to ports and airports, and economic evaluations of specific planned investments. Both activities directly provided important inputs for developing the PDLT. In addition, the project supported a study to assess the impact of the Pro-Vicinais program, which included the user’s satisfaction survey. The study was originally initiated by the request of the Bank. Based on complex econometric factors attempting to link increased economic activity and use of health and education facilities with the specific interventions of the project, the study faced the usual challenge of isolating the impact of the road investments from other external factors. Building on the experiences of this study, SLT is preparing the impact study of the Bank’s new project on state road rehabilitation. Environmental Licensing: The project (component 2(b)) contributed to a decrease in the turn-around time for the evaluation of requests for environmental licensing for work on municipal roads, and to a decrease of other potentially harmful activities through the technical assistance to SMA/CETESB for the development of a unified environmental licensing system and an integrated geo-referenced system The on-line unified licensing system is in place, effectively assisting potential applicants in the process of permit requests. The modules related to the actual issuing of licenses is still being tested, but nevertheless, the parts already implemented for the processing system have had a significant impact on time savings. The geo-referenced system integrated various 15 information related to land use and environmental data on the same map so that SMA/CETESB can more efficiently assess the license requests. As a result, in 2014, the turn-around time for evaluation of environmental licensing requests for medium and small size projects was 120 days, 50 percent less than the baseline, exceeding the original result indicator target of a 30 percent decrease. Participation of the Private Sector: The project (component 2(a)) supported the TA for the preparation of a toolkit for processing PPPs, including those related to municipal roads. The TA was completed successfully in 2015, giving the State Government Secretariat7 a unified online platform for the initiation and processing of PPPs in all infrastructure sectors, increasing the efficiency and transparency of the process. The platform was institutionalized by government decree number 61371 of July 25, 2015. Although it cannot be attributed solely to the project, the private and external investments to the state investment program in 2011-2014 was R$25.6 billion, well above the target of R$19 billion over a 4 year period. Execution: The project (component 2(d)) contributed to enhancing the efficiency of the DER-SP’s execution capacity, though PMS has been partially developed and has not yet been in operation. A first PMS had been purchased in late 2000, but never became operational due to a high level of complexity. Alternatively, the DER successfully installed the database system of the road inventory and geo-references of the state and municipal road network, both of which will include the essential parts of the PMS. The project also conducted a statewide traffic volume survey, the results of which were incorporated into the above system as essential input data. A technical assistance activity is underway to add a management module to these systems to conclude the PMS, and will further DER-SP capabilities to more efficiently process information to achieve optimal pavement management of the state and municipal road network. In addition, the project supported DER-SP in the introduction of the Highway Development and Management Model (HDM-4), which will also be incorporated as a part of the PMS through the purchase of licenses and training courses to support the PMS, In addition, SPDR, with the assistance of the project (component 2(a)), conducted pilot projects for introducing “budget by result” (performance-based budget management) to improve efficiency on executions of the government entities with transparent results and monitoring frameworks. The pilot projects are expected to have a positive impact on public sector investment decisions, of which municipal expenditures is an important component. The TA activities, financed under the project, were related to government expenditures on education and penitentiaries. 7 The PPP unit in SPDR (formerly SEP) was reassigned to the Government Secretariat in 2015. 16 3.3 Efficiency Rating: High An ex-post economic analysis was conducted following the same methodology as the one done at Additional Financing8, which assessed the 3rd and 4th phases of the Pro-Vicinais program. Costs and benefit were computed over a 20-year period from 2011 to 2030 with HDM-4, based on data (works, traffic, and so forth) effectively observed on the sampled municipal road network under the project, including: (i) types of road works applied; (ii) observed unit costs for road works, which took into consideration the cost increase from the original appraisal estimate due to the application of a more sustainable solution; and, (iii) observed traffic volumes before and after the work. Overall, the ex-post evaluation confirmed that the efficiency of the investments under the Pro-Vicinais program remains high. The resulting Economic Internal Rate of Return (EIRR) and Net Present Value (NPV) of the Phase 3 and 4 of the Pro-Vicinais program were respectively 36.6 percent and R$620 million at a 12 percent discount rate over a 20- year period at a 2010 price, which are better than the ex-ante evaluation results at Appraisal with the EIRR of 35.20 percent and the NPV of R$521 million percent (full details are presented in Annex 3). Most of the activities under the institutional development component also contributed to efficiency gains, though it is difficult to quantify these benefits in monetary terms. For example, the environmental management improvements benefit the municipal road development by reducing transaction costs of environmental licensing processes. The efficiency gains of most of the activities, such as the pilot project on performance-based budgets, the PMS, and the PPP toolkit, might benefit the entire state in the long-run through improvements in public policy decision-making processes. 3.4 Justification of Overall Outcome Rating Rating: Satisfactory Project objectives were deemed highly relevant to the country’s priorities aiming at supporting regional development and equity, reducing logistic costs through, among other things, improving accessibility to the core state network and enhancing the capacity of the state for planning and environmental management. The relevance of the project design and implementation is also considered high considering the primary outputs and the intermediate outcomes are all consistent with the PDO. 8 At the appraisal of the parent project, the economic feasibility was assessed based on the road sections of the 2nd phase assuming its homogeneity to the 3rd phase in terms of the road characteristics although the 2nd phase was not financed by the project. Therefore, the methodology of the ex-post evaluation follows one of the ex-ante evaluations at AF which assessed the 3rd and 4th phases. 17 Efficacy was high and the PDO’s indicators were fully met. Transport costs decreased and users were mostly satisfied with the road improvements. Beyond the PDO indicator, two project intermediate result indicators were closely related to the efficiency of the municipal road network and exceeded the targets, and three out of four indicators that were related to the institutional strengthening component were fully met. Efficiency was high as was confirmed by the economic appraisal carried out on selected, but representative, investment sub-projects that showed that the project socio-economic rate of return remained robust, yielding significant benefits for the project beneficiaries as a whole. The overall outcome was rated satisfactory based on the above assessments and a minor risk in sustainability of the development outcome as described in the section 4, although when compared to other similar projects, it appears that this project deserves a high satisfactory rating. 3.5 Overarching Themes, Other Outcomes and Impacts (a) Poverty Impacts, Gender Aspects, and Social Development The project was expected to have a positive impact on poverty and social development. It was thought that improving transport conditions would, on one hand, fuel the economy, thereby entailing job creation and access to markets, and on the other hand, provide better connections between communities, thereby fostering access to social facilities and reinforcing social nets. One of the interesting findings is that according to the user interview survey, 38 percent of users increased their frequency of trips related to work due to improvement of road conditions. In addition, the processing of the Additional Financing took place at a time of decreased economic activity in which the government of Sao Paulo was seeking to implement countercyclical measures, including road expenditures. The project was therefore seen as a vehicle to mitigate the impact of lower economic growth on the most vulnerable sectors of the population. The project did not specifically address gender issues. (b) Institutional Change/Strengthening The project made considerable advances in the modernization of the public sector management of the state of Sao Paulo with the introduction of state-of-the-art tools for user-friendly, interactive and efficient activities, including the processing of environmental licenses, the integration of a large amount of rich and diverse data from different sources and its display into a spatial format, the introduction of transparent and efficient mechanisms for initiating and processing PPPs, and the development of useful models for transport demand projections and planning. (c) Other Unintended Outcomes and Impacts (positive or negative) The project did not produce unintended outcomes. 3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops N/A 18 4. Assessment of Risk to Development Outcome Rating: Moderate The level of risk to development outcomes is rated as moderate considering the following factors: The municipal roads under the program will be maintained by the municipal governments according to the agreement between them and the state, which was one of the requirements for participating in the program. In addition, the municipal roads under the program are still in good condition. The road surface condition survey conducted in 2015 demonstrated that 97.6 percent of the roads remained in good condition four or five years after the works. Meanwhile, the financing of the municipal roads continues to raise doubts in terms of the capacity of the local governments to live up to the actual needs of future maintenance. Although municipal governments are responsible for municipal roads, historically DER- SP has been informally in charge of developing and maintaining those networks, which is not a very clear institutional arrangement from the viewpoint of the long-term road asset management. The outcomes of the technical assistance activities under the institutional strengthening component are considered sustainable. Major deliverables of the project, including the road inventory and geo-referenced information system for DER-SP, the environmental licensing and geo-reference systems for SMA/CETESB, and the PPP toolkit for SPDR, are incorporated into the operational processes in the state agencies. 5. Assessment of Bank and Borrower Performance 5.1 Bank Performance (a) Bank Performance in Ensuring Quality at Entry Rating: Satisfactory As noted in the preceding section 2.1, most of the design issues deserved satisfactory ratings. In particular, what eventually constituted a successful institutional strengthening component (2 percent of the loan amount) not only contributed to the PDO but also generated a substantial positive externality. Although the activities did contribute to the efficiency of the municipal road sector, they clearly addressed issues that were highly relevant to the efficiency of public sector management well beyond the subject of roads, a very positive outcome that was not fully captured in the PDO. Probably the most effective feature of project design was the incorporation of lessons learned with respect to operations involving MICs. The project design was pragmatically streamlined to reduce the non- financial costs of doing business by joining in the financing of a program that already had a proven record of execution with a minimum of new requirements. The project was seen at the time of appraisal as an opportunity for the Bank to re-engage with the state of Sao Paulo in the inter-urban transport sector (the last operation had closed in the 1980s). The 19 rather “non-intrusive” design of the civil works component, together with an institutional strengthening component highly responsive to the demands of the Borrower, underlined the Bank’s strategy to gradually strengthen the possibility of a rich technical dialogue with an accomplished state in Brazil, and also served to demonstrate the Bank’s ability to respond to the MIC agenda by providing high value technical support to a sophisticated client. The project results and the subsequent ongoing and future-planned Bank operations in Sao Paulo confirmed the correctness of that approach. Finally, with the benefit of hindsight, the lead-time for the preparation of very complex IT components was overoptimistic. No major issue on fiduciary was found at entry. (b) Quality of Supervision Rating: Satisfactory Traditional supervisory activities to monitor progress toward mid-term outcomes were carried out, and missions traveled to the field more than twice a year, with additional regular Bank support provided directly by the project team. Beyond the focus on regular supervision, there was a strong interest on the part of the Bank in maintaining active options for supporting institutional activities, even at times when the responsiveness of the recipient agencies faltered. Supervisory missions did play the envisaged productive role of the Bank as a constructive partner providing valuable inputs on technical matters such as the quality of work, the comprehensiveness of terms of references for complicated assignments, and the suitability of the bidding documents. All these issues were reviewed in great detail by a highly competent supervision team and were reported in comprehensive Aide Memoires. Officials interviewed for the Implementation Completion and Results Report (ICR) concurred that the presence of the Bank provided helpful technical advice, and that in some cases it was instrumental in ensuring a bidding process leading to the best choice of consulting firms. Attention to safeguarding and fiduciary aspects of the project was also adequate. A lack of a more proactive Bank role in implementing the aim described in the project components (Part 2(d)(v)) addressing the challenging issues of future financing of municipal roads and of project sustainability was probably the only weak point of this operation. (c) Justification of Rating for Overall Bank Performance Rating: Satisfactory The satisfactory rating is based on the fact that both the design and the supervision of the project were satisfactory as described above. Also the Bank management should be commended for their attention to the Borrower’s needs by agreeing to extend the closing date of a loan that was mostly disbursed to finance important institutional strengthening activities identified late in the execution period. 20 5.2 Borrower Performance (a) Government Performance Rating: Satisfactory The government of Sao Paulo’s support for the implementation of the Pro-Vicinais Program was unwavering throughout the project implementation period, consistently supplying the necessary funding that allowed the remarkable achievement of improving almost 12,000 km of municipal roads in less than five years. (b) Implementing Agency or Agencies Performance Rating: Satisfactory The rating gave substantial weight to the role of DER-SP as the executing agency of civil works under the program, the main component of the project, and as the agency responsible for the coordination of information gathered from SPDR, SLT and SMA on, among others, procurement planning, certification of bills, progress in the activities underway, and achievement of intermediate outcome indicators. The performance of DER-SP on these activities was rated as highly satisfactory, both in terms of its general institutional capacity for executing civil works and of its effective management of the project (technical, fiduciary, and safeguards). The performance of DER-SP on the implementation of its institutional strengthening component was less satisfactory. Effective actions for the implementation of the PMS dragged on until late in the project execution period, in part because of a general trend of diminishing human resources and the need for their replacement in specific areas, particularly in information technology. The performance of the other implementing agencies was mixed. SMA’s performance was highly satisfactory, achieved to a great extent because it was the agency that from the outset had the most clearly defined specific need for and concept of the use of the loan funds. SLT and SPDR, on the other hand, dropped some of the original proposed studies, and it took time until the final scope of their components were defined. (c) Justification of Rating for Overall Borrower Performance Rating: Satisfactory Based on the above considerations, the overall performance of the Borrower is rated as satisfactory. 6. Lessons Learned The highly streamlined project design proved to be a valuable instrument and provided the right amount of flexibility in project implementation that was the key for the success of the Pro-Vicinais Program for a sophisticated client with important needs. The project allowed the Bank’s team to concentrate on supporting the government’s overall program and to 21 provide guidance on the program’s technical and financial aspects, free from the excessive burden that usually is associated with operations centered on individual transactions. The sufficient lead-time and resources for consensus building and prioritization of the policy goals were a key to successfully introducing new management systems involving very complex procedural changes. During the initial years of the project, the needs of some of the implementing agencies focused on internal deliberations on strategies and goals. This required further time to help define future assistances and to develop complex terms of reference for more time-intensive consultancies. It should be emphasized that the long lead-time in defining specifications of the management systems led as a result to successful development of the systems, particularly in the case of the environmental licensing and geo-reference systems for SMA/CETESB as well as the PPP toolkit for SPDR. However, approaches dividing the final objective into modules, and adding them at a pace in line with the life of the project, could be more efficient. The sustainability of the rural road asset could be enhanced through more active engagement of the project in assisting clients in supporting local governments. While the issue of insufficient resources of local governments to maintain their roads is an ubiquitous and well-known challenge, the challenge was more complex with a majority of the Bank’s projects because the projects’ beneficiaries were at the national or state level where they normally do not have any responsibility for maintaining rural roads and therefore have less incentive to invest in this area. The projects would need to proactively motivate these national- and state-level clients to support local governments in assessing and developing mechanisms for sustainable maintenance of their road networks. Standardized engineering designs with a classification of different levels of maintenance interventions based on simple road parameters are an important contributor for implementing large-scale roads rehabilitation programs rapidly and efficiently but without compromising quality engineering. This approach was again confirmed as effective in this project and could be applied not only to rural roads but also to state- and national-level road projects. Future similar operations should encourage this methodology and consider the introduction of specific training for time-to-time updating of the guidelines. 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners (a) Borrower/implementing agencies N/A (b) Cofinanciers N/A (c) Other partners and stakeholders N/A 22 Annex 1. Project Costs and Financing (a) Project Cost by Component (in US$ million equivalent) Percentage of Components Appraisal Estimate Actual Appraisal Feeder Roads Rehabilitation 477.04 (1) 1,010.25 (2) 111.77% Institutional Strengthening 23.80 10.17 -57.27% Total Baseline Cost 500.84 1,020.42 103.74% Physical Contingencies 50.08 0.00 Price Contingencies 45.08 0.00 Total Project Costs 596.00 1,020.42 71.09% Front-end fee IBRD 0.42 0.42 Total Financing Required 596.42 1,020.84 71.16% (1) Refers to the estimated cost of rehabilitating 3,100 km of roads of the 3rd phase of the Program with financing from the Parent Project (Loan 7688-BR) and counterpart fund. (2) Refers to the final cost of financing a total of 6,409km of roads of the 3rd and 4th phase of the Program with the financing from both the Parent Project and the AF jointly with the counterpart fund. (b) Financing Appraisal Actual/Latest Percentage of Source of Funds Estimate Estimate Appraisal (US$ million) (US$ million) Borrower 429.77 527.42 22.72% IBRD 166.65 493.425 196.08% 23 Annex 2. Outputs by Component The following table includes the activities carried out during the project per component linked to the outputs attained: Table: Project Activities and Outputs Component Activities Outputs Component 1 – Feeder roads rehabilitation (actual cost US$1,010.25) Improve state’s feeder road The project supported the rehabilitation network works of almost 3,300km of municipal road sections out of 11,900km under the Program, which reduced transport costs of road users. As demonstrated in section 3.2, the program was financed by various agencies, including the Bank, IDB, and their counterparts. As indicated in the Loan Agreement, the project included financing to state roads if applicable. However, at the end, the project did not finance any state roads. Component 2 – Institutional strengthening (actual cost US$10.17) (a) Support to structuring and management of the state investment program (Economy and Planning Secretariat, SEP, later SPDR) Enhance efficiency of the state’s The project supported the following two PPP policy activities: (i) the development of the PPP toolkits and the new website to accept online requests of PPP project proposals with high transparency (the online process was authorized by the state decree); and (ii) training courses on financial analysis to modernize PPP processes. Introduce the management The consulting service financed by the methodology of “budget by project prepared the monitoring and results” into the state agencies evaluation frameworks of “budget by results” for the pilot agencies. The pilot projects were conducted for the state administration secretariat in 2014 and the education secretariat in 2015. Structuring and management of After an initial delay in launching the TA, the State Investment Program. the time for the change of government became imminent and it was felt that investment decisions should be left to the 24 next Administration. Thus, the TA was canceled. (b) Support to modernization of the state’s environmental management structure (SMA and CETESB) Integrated environmental geo- The environmental geo-reference system reference system of the state of (DATAGEO) was developed through the Sao Paulo following activities: (i) DATAGEO system development, (ii) acquisition of the IT equipment for the DATAGEO team, and (iii) acquisition of digital equipment for environmental police which use DATAGEO in onsite inspection. The DATAGEO integrated various information related to land use and environmental data on the same map so that SMA/CETESB can more efficiently assess the license requests. The SMA is requesting the Bank to upgrade further the DATAGEO with the financing of the new Bank project: Sao Paulo Sustainable Transport Project (P127723). Improved environmental Integrated & Unified Environmental licensing system Licensing System (GAIA) was installed in 2015 through a consulting service financed by the project. The on-line unified licensing system is in place effectively assisting potential applicants in the process of permit requests. The modules related to the actual issuing of licenses is still being tested but nevertheless, the parts of the processing system already implemented have had a remarkable impact on time savings. Development of pilot system of While two consultants were hired to assist collecting site survey data CETESB in developing the pilot system, the strategy of the system development was changed in the middle of the contracts and the services were cancelled. (c) Support to consolidation of institutional and planning capacity in the transport sector (Transport Secretariat, SLT) Logistic demand diagnosis The study of logistics demand in the state was conducted to identify needs in the logistic sector together with the State 25 Development Plan. The study included input-output and origin-destination data, using a general equilibrium economic model. Support to the new State This activity was implemented by hiring Logistics and Transport six individual consultants to work on Development Plan 2030 (PDLT) specific areas in transport and logistics planning, including: (i) a diagnosis cabotage and port system, (ii) an economic study, (iii) an environmental study, (iv) an economic modelling plan, (v) an air transport demand study, and (vi) accesses to the three major airports in the Sao Paulo Metropolitan area. The State Plan for Logistics and Transport 2030 (PDLT) is still under preparation at ICR. Evaluation of the Pro-Vicinais An economic model was developed and an program assessment of impacts of the Pro-Vicinais program was conducted, including user satisfaction surveys. Building on the experiences of this study, SLT is preparing the impact study of the Bank’s new project on state road rehabilitation. (d) Support to improved public sector management capacity in the road sector (State Road Department, DER) Development of integrated road The two road management systems of the management system of state and state and municipal road were developed: municipal roads (i) the Highway Cadastral System (SICAR) including database of road inventory with basic physical characteristics, and (ii) the roads mapping and geo-referencing system (SIRGEO). Both systems will be an essential part of the PMS. Development and facilitation the 8 licenses of HDM-4 software were use of the Highway Development purchased in 2012. In 2015, a 2-week and Management Model (HDM- training session on HDM-4 took place for 4) the 16 DER-SP professionals. Traffic study and analysis The statewide traffic counting surveys on the state road network under the jurisdiction of DER-SP were conducted in 2012-13. The data of traffic counts will be integrated into the SICAR mentioned above. 26 Annex 3. Economic and Financial Analysis Introduction This annex summarizes the ex-post economic analysis of the project. The objective of the analysis is to assess the economic costs and benefits of the 3rd and 4th phases of the Sao Paulo municipal road rehabilitation program (Pro-Vicinais program) implemented by DER-SP, a part of which was funded by Bank Loans 7688-BR and 7838-BR, using the methodology adopted for the ex-ante evaluation conducted at AF 9 . The major project benefits are savings in road-user costs, including vehicle operating costs and travel time costs, and reduction in maintenance and rehabilitation costs. To develop the ex-post analysis model, several modifications from the ex-ante model were made taking into account and based on: (i) the type of works effectively executed; and, (ii) observed traffic volumes and unit costs for works. The evaluation was conducted by using the HDM-4 simulation model. The results show that Economic Internal Rate of Return (EIRR) and Net Present Value (NPV) were respectively 36.6 percent and R$620 million at a 12 percent discount rate at a 2010 price. Methodology The ex-ante economic assessment included an evaluation of the municipal road rehabilitation under the 3rd and 4th phases of the Pro-Vicinais program with the total extension of 6,019 km. The evaluation conducted the detailed analysis on the sampled road sections of 1,053 km, representing 17 percent of all sections under the two phases. This evaluation was accomplished by the DER of São Paulo using the Highway Development and Management Model (HDM-4) that performed a Cost Benefit analysis of roads rehabilitation and maintenance projects using the Consumer’s Surplus approach. The project benefits mainly consisted of the reduction of vehicle operating costs and passenger time costs, together with the reduction of maintenance costs to the road agency when compared to the reference scenario (a “do minimum” scenario including only routine maintenance and reconstruction when the infrastructure triggers a deteriorated situation). Additional benefits to road users which have not been quantified included reduced accidents, improved driving, and riding comfort. Following the same methodology as for the ex-ante economic appraisal, the project externalities on environment (including global warming) that were expected to be negligible or even positive were not taken into account in the analysis. 9 At appraisal of the parent project, the economic feasibility was assessed based on the road sections of the 2nd phase assuming its homogeneity to the 3rd phase in terms of the road characteristics, although the 2nd phase was not financed by the project. Therefore, the methodology of the ex-post evaluation follows one of the ex-ante evaluations at AF which assessed the 3rd and 4th phases. 27 The program was analyzed by assessing the actualized streams of benefits to road users over a 20 year period, and savings on maintenance costs (as compared to the without- program case) net of the costs of the observed road works. The yearly discount rate used was 12 percent. Traffic Volume and Growth The traffic volumes were updated by the traffic count survey on sampled municipal roads under the program. See Table C.1 below for summaries of survey results. The traffic data after the works is available on the 17 road sections in diverse regions, about 30 percent of the study scope. The annual traffic growth rate is assumed as 1 percent considering the recent economic downturn in Brazil. Table C.1: Traffic Counting Survey Results Road Costs The road costs were updated based on the actual applied type of civil works on each road section. All costs are at a 2010 price. The applied unit costs of rehabilitation and maintenance are summarized below in Table C.2. 28 Table C.2: Road Costs Maintenance Routine R$9,360/km Patching R$22.47/m2 Rehabilitation Asphalt Surfacing 30mm R$17.9/m2 Asphalt Surfacing 40mm R$23.46/m2 Asphalt Surfacing 50mm R$29.2/m2 Reshaping R$10.87/m2 Asphalt surfacing 50mm, graded stone Reconstruction 15cm and crushed stone 15cm, R$77.23/m2 Localized recuperation with asphalt surfacing 22mm, graded stone 16.5cm and soil cement 38cm R$90.38/m2 Vehicle Operating Costs The following Table C.3 presents the average vehicle fleet characteristics and operation costs. Table C.3: Vehicle Fleet Characteristics and Operation Costs Medium Heavy Articulated - Description Car Bus Truck Truck Truck Bitrem Rodotrem Treminhão New Vehicle Cost (Real/vehicle) 36,744.00 290,000.00 84,000.00 155,800.00 299,630.00 374,350.00 440,767.00 338,000.00 New Tire Cost (Real/tire) 208.00 1,100.00 840.00 1,100.00 1,380.00 1,380.00 1,380.00 1,380.00 Fuel Cost (Real/liter) 2.50 1.87 1.87 1.87 1.87 1.87 1.87 1.87 Lubricant Cost (Real/liter) 10.00 16.50 16.50 16.50 16.50 16.50 16.50 16.50 Economic Unit Costs Maintenance Labor Cost (Real/hour) 4.08 35.08 12.12 19.84 42.24 56.44 70.30 55.24 Crew Cost (Real/hour) 8.03 24.91 16.60 16.60 16.60 16.60 16.60 16.60 Overhead (Real) 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 Interest Rate (%) 12.00 12.00 12.00 12.00 12.00 12.00 12.00 12.00 Passenger Working Time (Real/hour) 6.49 2.21 1.00 1.00 1.00 1.00 1.00 1.00 Kilometers Driven per Year (km) 23,000.00 96,000.00 60,000.00 84,000.00 96,000.00 96,000.00 96,000.00 96,000.00 Hours Driven per Year (hr) 550.00 1,920.00 1,920.00 1,920.00 1,920.00 1,920.00 1,920.00 1,920.00 Service Life (years) 11.00 13.00 11.00 11.00 11.00 11.00 11.00 11.00 Utilization and Percent of Time for Private Use (%) 100.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 Loading Number of Passengers 1.00 29.00 2.00 2.00 2.00 2.00 2.00 2.00 Work Related Passenger Trips (%) 75.00 75.00 1.00 1.00 1.00 1.00 1.00 1.00 Gross Vehicle Weight (tons) 1.50 10.00 10.50 15.50 27.50 38.00 50.00 49.00 ESA Loading Factor 0.00 2.96 2.96 4.00 7.50 10.83 14.17 10.83 Scenarios The HDM-4 model was run on each lot of work, taking into consideration the following scenarios:  Without project alternative: routine maintenance including vegetation clearing, drainage cleaning, pothole patching (100 percent), and rehabilitation or reconstruction when IRI >7 was topped with asphalt concrete (except for the two first years of the project). 29  With project alternative: the above projected rehabilitation works, the routine maintenance of the without project alternative completed by a periodic maintenance consisting in the resurfacing with asphalt concrete when IRI reaches 4.5. Results Table C.4 below details the net present value (NPV) and economic internal rate of return (EIRR) of the program in comparison to the ex-ante evaluation. Overall, EIRR and NPV of the entire 3rd and 4th phases of the program were respectively 36.6 percent and R$620 million at a 12 percent discount rate at a 2010 price. While the EIRR is somehow similar to the previous values, the NPV decreased from the one at AF. One of the possible reasons was the lower traffic growth rate applied to the ex-post evaluation (1 percent) compared to the ex-ante (3 percent). Table C.4: Net Present Value and Economic Internal Rate of Return of the Program At Appraisal At Additional Financing At ICR (Phase 2) (Phase 3) (Phase 4) (Phase 3 & 4) Net Present Value (R$ million) 521 885 544 620 Economic Internal Rate of Return (EIRR) 35.20% 40.70% 20.50% 36.60% 30 Annex 4. Bank Lending and Implementation Support/Supervision Processes (a) Task Team members Responsibility/ Names Title Unit Specialty Lending Flavio Chaves Natural Resources Mgmt. Specialist. AFTN3 Safeguards Specialist Regis Thomas Cunningham Sr. Financial Management Specialist GGODR Team Member Nicolas Drossos Consultant GGODR Team Member Eric R. Lancelot Sr. Transport. Engr. AFCF1 Team Leader Aymeric-Albin Meyer Operations Adviser OPSPQ Team Leader Marta Elena Molares-Halberg Lead Counsel LEGES Team Member Luis R. Prada Villalobos Senior Procurement Specialist GGODR Team Member Solange P. Van Veldhuizen Program Assistant ECSHD Team Member Adrien J. Veron Consultant LCSTR Team Member Supervision/ICR Flavio Chaves Natural Resources Mgmt. Spec. GWADR Safeguards Specialist Eric R. Lancelot Sr. Transport. Engr. AFCF1 Team Leader Marcilio Augusto Neves Consultant GTIDR Team Member Jason Jacques Paiement Social Development Specialist CRKI2 Team Member Henrique Penna Naves Consultant LCSTR Team Member Jean-Claude Sallier Consultant LCSTR Team Member Sivan Tamir Operations Officer SECPO Team Member Elisabet Vila Jorda Junior Professional Associate LCSTR Team Member Satoshi Ogita Transport Specialist GTIDR Team Leader Gregoire Francois Gauthier Sr. Transport Engineer GTIDR Team Member Virginia Maria Henriquez Team Member Consultant GTIDR Fernandez Rodrigo Alessandro Akira Team Member Consultant GTIDR Nakama Borja Castro Lancharro Consultant GTIDR Team Member Hanayo Taguchi Program Assistant GTIDR Team Member Maria Jose Vilas Boas Pereira Safeguards Specialist Safeguards Specialist LCSSO Weiss Financial Susana Amaral Sr. Financial Management Specialist GGODR Management Specialist Financial Eduardo Franca de Souza Financial Management Specialist GGODR Management Specialist Clarisse Torrens Borges Dall Senior Environmental Senior Environmental Specialist GENDR Acqua Specialist Cristina Oliveira Roriz Consultant LCSEN Team Member 31 (b) Staff Time and Cost Staff Time and Cost (Bank Budget Only) Stage of Project Cycle US$ thousand (including No. of staff weeks travel and consultant costs) Lending FY08 19.79 129.55 FY09 21.00 127.71 FY10 0.35 11.14 Total: 41.14 268.40 Supervision/ICR FY09 - - FY10 25.69 137.13 FY11 17.03 106.22 FY12 15.73 102.23 FY13 15.47 73.20 FY14 15.34 73.82 FY15 8.58 56.82 Total: 97.84 549.41 32 Annex 5. Beneficiary Survey Results N/A 33 Annex 6. Stakeholder Workshop Report and Results N/A 34 Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR The following are the main comments, with the statement of relevant issues and lessons learned that were achieved through the development of the project, made by the client. The World Bank performance The World Bank's role was essential, especially during the project implementation stage in not only providing guidelines, objectives, and goals, but also in the planning and construction of the contractual model (administratively, legally, and technical). The World Bank supported DER-SP in developing and implementing the project, and providing methodologies and procedures to be adopted by the Bank. During implementation, some concerns arose about disagreements between the Bank and DER-SP related to the legal principal of public works and services contracts due to the differences between the Bank’s guidelines and the recommended procedures of the external Supervising and Advisor Organizations to DER-SP, notably the State Court of Auditors (ECA), which led to numerous discussions between the DER-SP procurement team and the Bank’s procurement experts which eventually resulted in the delay of some bidding processes. These disagreements and resulting discussions showed the need to increase the knowledge of the public administration on Bank policies on goods and services contracts and on their legal and political obligations and limits, in order to avoid conflicts and impasses that could compromise the required goals. Contracts management Contract management is supported by computerized systems that help monitor and control administrative actions related to these contracts. Currently, DER-SP has the following systems:  Measurement System – (FCTM) that aims to track physical and financial progress on a monthly basis for each contract;  Financial Management System – (FCVB) that is used in order to manage information related to contractual payments and which are also made available to the Financial Tracking System (SIAFEM), which conducts the budgetary and financial implementation of the State Government; and  Management and Supervision Reports that provide information on the physical progress of contracts. All systems’ operation, maintenance, and improvements fall under PRODESP’s responsibility (Procesamento de dados de Estado de Sao Paulo), a public company linked to the State Department of Public Management, which has been making efforts to restructure methods and procedures to modernize systems. The new systems are being deployed to make managerial information more reliable. However, it still contains restrictions on the physical progress for on-going contracts. 35 Despite the efforts and resources spent in nearly a decade, the DER/SP does not have yet an integrated set of computer systems that are compatible with the size of the public investments involved and that reflect their economic and strategic importance. It is still missing a powerful computational tool which can perform current administrative functions, but which can also add both planning systems and existing costs, as well as others, that must be created to meet the technical, administrative, operational, and strategic agency demands. It is evident too that meeting just the compliance with administrative goals for reducing temporal interregnum between the invitation to bid, the contract signature and the start of work, do not guarantee an optimal process-management and technical-operational performance of the work. On the contrary, technical disputes from the projects and budgets have always been present in the program. Even though there was good quality at the time of their preparation, it was very often necessary to adapt to the reality, since the pavement recovery solutions were adopted from a solution matrix and applied to the results of the visual surveys of road conditions. Considering that design changes are a key driver of increases in the value of contracts, they must be approved by the technical team of DER-SP and the Bank. In this regard, it is essential to improve the processes for review and approval of any design changes, which provides greater flexibility in procedures, to prevent any interruptions and delays in the completion of works. It is also important to improve administrative processes so as not to undermine the fundamental technical elements, the quality of the work, and the adoption of systematic monitoring system to follow all the necessary steps for project implementation, from specialized surveys in the field, through preparation of projects, budgets, procurement, and execution, in addition to the currently monitored activities. Similarly, there is a need for the implementation of consistent systematic management of technical information in an integrated manner and for prompt translation of the contents of studies and elaborate designs and their works. This systematic approach should be linked to other administrative and financial systems to produce management and strategic indicators for the various hierarchical levels of DER-SP. 36 Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders N/A 37 Annex 9. List of Supporting Documents  Project Appraisal Document, June 8, 2009  Project Paper, June 23, 2010 38 MAP 1.1 Roads under the 3rd Phase of the Pro-Vicinais Program 39 MAP 1.2 Roads under the 4th Phase of the Pro-Vicinais Program 40