Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: AP and Telangana Road Sector Project (P096021) Report Number: ICRR0021930 1. Project Data Project ID Project Name P096021 IN: AP and Telangana Road Sector Project Country Practice Area(Lead) India Transport L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IBRD-77920 30-Jun-2015 191,534,537.66 Bank Approval Date Closing Date (Actual) 15-Oct-2009 31-Mar-2019 IBRD/IDA (USD) Grants (USD) Original Commitment 320,000,000.00 0.00 Revised Commitment 191,534,537.66 0.00 Actual 191,534,537.66 0.00 Prepared by Reviewed by ICR Review Coordinator Group Cynthia Nunez-Ollero Peter Nigel Freeman Ramachandra Jammi IEGSD (Unit 4) 2. Project Objectives and Components DEVOBJ_TBL a. Objectives According to the Financing Agreement (FA, p.5), and the Project Appraisal Document (PAD, paragraph 17), the Project Development Objective (PDO) of this project was "to provide better quality, capacity, and safe roads to users in a sustainable manner through enhanced institutional capacity of the Andhra Pradesh government in the road sector." This review will assess the achievements of the following objectives:  to provide better quality, capacity, and safe roads to users in a sustainable manner Page 1 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: AP and Telangana Road Sector Project (P096021)  to enhance institutional capacity of the provincial government in the road sector The following were the original key PDO outcome indicators (PAD, paragraph 18):  improvement in the share of the Core Road Network (CRN) in good condition,  reduction in vehicle operating costs, travel time, and fatality rates per vehicle km travelled.  full operationalization of the Road Development Corporation (RDC) with adequate financial and human resources to manage the CRN, and  ability to launch PPP projects was added as a key outcome indicator b. Were the project objectives/key associated outcome targets revised during implementation? No c. Will a split evaluation be undertaken? Yes d. Components 1. Road Improvement (US$606.79 million at appraisal, US$526.20 million actual) This component was to finance two activities to upgrade and maintain the Core Road Network (CRN) by: (i) upgrading about 429 km of priority state highways (284 km in Phase I and 145 km in Phase II) using public sector financing, and (ii) funding the Long-term Performance-based Maintenance Contract (LTPBMC) to conduct maintenance of 6,241 km of the CRN. 2. PPP Facilitation Support (US$3.07 million at appraisal, US$1.20 million actual) This component financed activities to strengthen the capacity of Andhra Pradesh Road Development Corporation (APRDC) to attract private sector participation in financing, development, and management of selected high traffic density corridors. This component financed the costs of transaction advisers to assist APRDC to carry out feasibility studies and to develop and transact selected PPP roads and (b) capacity building of APRDC to better identify, structure, award, and monitor PPP projects. 3. Institutional Strengthening (US$16.54 million at appraisal, US$9.60 million actual) This component financed targeted technical assistance, training, and advisory services to (i) operationalize APRDC with requisite capacity to manage the CRN and PPP projects and (b) various aspects of project implementation management including the Asset Management Program, Governance and Accountability Action Plan (GAAP), and Institutional Strengthening Action Plan (ISAP). 4. Road Safety. (US$18.65 million at appraisal, US$12.00 million actual) This component financed two activities to help the Government of Andhra Pradesh provide safer road corridors by assisting the concerned agencies to initiate road accidents reduction measures: (i) institutional and policy action plans for improving responsibility and capacities, and (ii) demonstration projects on selected CRN corridors including the implementation of a black spot improvement program. A black spot referred to sites where road traffic accidents were historically concentrated. Page 2 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: AP and Telangana Road Sector Project (P096021) e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project Cost: The total project cost at appraisal was US$645.06 million. Disbursements reached US$549.00 million. Financing: The International Bank for Reconstruction and Development (IBRD) financed this project with a loan of US$320 million of which US$191.5 million was disbursed.US$56 million was cancelled in March 2014 and another US$25 million in April 2017 (see Dates below). The balance of the loan was cancelled. Borrower Contribution: The Government committed and disbursed counterpart funds of US$325.86 million Dates: The project was approved on October 15, 2009 and made effective March 23, 2010. The Mid Term Review (MTR) was conducted on October 8, 2012. The original closing date was June 30, 2015. The project was extended by 45 months and closed on March 31, 2019. There were six (6) level 2 restructurings:  On March 3, 2014 to cancel US$56 million in savings due to exchange rate gains.  On February 26, 2015 to reallocate funds between disbursement categories. In March, 2014, Andhra Pradesh was divided into two - Andhra Pradesh and Telangana. Funds were reallocated to accommodate the implementation of the project in two distinct states. In December 16, 2014, the borrower informed the World Bank of the enactment of Andhra Pradesh’s Reorganization Act 2014, thus carving out Telangana State (Telangana) from Andhra Pradesh. Consequently the activities and the outcomes remained unchanged, but indicators were divided for Components 1 and 2 and duplicated for Components 3 and 4 between the two states.  On April 7, 2017 to amend the results framework, revise components, reallocate among disbursement categories and cancel financing. The project was adversely affected by (a) political uncertainties of the Bifurcation Act, (b) adoption of a new Land Acquisition and Resettlement Act, (c) inferior performance of some road upgrading packages leading to the termination and/or rebidding of three of the nine packages, and (d) lagging progress for Components 3 and 4.10 Restructuring included (a) adding a new implementation agency for the new state of Telangana; (b) deleting the Kakinada-Rajahmundry upgrading package (KR-07) from the scope of the project (62 km) because of prolonged land acquisition; (c) cancelling the corresponding US$25 million; and (d) extending the project closing date by 23 months, from June 30, 2015 to May 31, 2017. The restructuring led to changes in the RF and legal covenants: (a) revision of intermediate outcome and core indicators for Component 1 and (b) reduction of the target indicator number of PPP roads from three to two as one of the roads was reclassified from a state to a national highway.  On May 25, 2017 to At the request of the GOAP and the Government of Telangana (GOT)—and following the satisfactory fulfillment of long outstanding environmental and safeguard issues—the World Bank granted the subject project an additional extension to December 31, 2018, which was needed to allow for the completion of project activities  On July 28, 2017, the project was extended to December 31, 2018, to change targets in the results framework and complete project activities.  On November 29, 2018 to extend to March 31, 2019 and reallocate funds between disbursement categories to complete activities under Components 1, 3, and 4. Page 3 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: AP and Telangana Road Sector Project (P096021) 3. Relevance of Objectives Rationale The transport sector in the state of Andhra Pradesh, together with telecommunication and tourism form close to half of the state's gross domestic product. The state has 188,568 km of roads. The Roads and Building Department (RBD) managed 68,510 km of these. Municipalities managed the rest . About 41,000 km of the RBD-managed roads were state highways and major district roads. In 1997, the state designated its Road Development Corporation to manage a Core Road Network consisting of 10,000 km of the 41,000 km. In 2001, Andhra Pradesh ratified the 2001 Infrastructure Act to define areas of cooperation between the government (public sector) and the private sector through public private partnerships (PPPs). Andhra Pradesh undertook improvements in the road network using its own resources, accessed the Central Road Fund of the Government of India, as well as domestic and international borrowings. In this regard, the PDO was relevant to the sector and the state's efforts at providing better quality, improved capacity in the sector and safer roads. Meanwhile, the RDC lacked capacity to develop, manage, and maintain capital improvements and PPP projects. Road safety was becoming a burgeoning concern as more vehicles were now on the road, bad driving behavior such as speeding was on the rise, and coordination among the various road safety agencies at the local, state, and national levels was wanting. The PDO was relevant to the state's efforts to increase investments, promote safety, and coordinate among the various transport agencies to achieve an efficient transport sector that would be contribute to economic growth. The PDO was relevant to the Government's implementation of its Governance and Accountability Action Plan (GAAP) and Institutional Strengthening Action Plan (ISAP) designed to enhance systems, procedures, and processes before, during, and after project implementation. GAAP and ISAP were designed to help Road Development Corporations to improve information disclosure and transparency in project implementation, better quality control measures, third party monitoring, and include citizens inputs and complaints using mandatory user satisfaction surveys. The PDO remained relevant to the country's development program (12th Five Year Development Plan, 2012-2017) to achieve faster, sustainable, and more inclusive growth. The PDO contributed to the Government's strategic objectives of improved transport connectivity in targeted states. These improvements included connecting all villages with all weather roads and upgrading national and state highways to the minimum two lane standard (indicator 16 of 25 core indicators of the plan). The PDO was also relevant to the World Bank's Country Partnership Framework (CPF) 2018-2022, which promoted three focus areas: (i) promoting resource efficient growth, (ii) enhancing competitiveness and enabling job creation, and (iii) investing in human capital. Under the second focus area, the PDO was relevant to item 2.3 to improve connectivity and logistics. This focus area stipulated that the Government continue to pursue policies aimed at strengthening public sector institutions, improve implementation capabilities, target connectivity constraints by supporting road development and maintenance, and strengthen private sector participation. The PDO remained relevant even as the state was divided into Andhra Pradesh and Telangana in 2014. The new state of Telangana mirrored the institutional set up of Andhra Pradesh in the road sector and the project components were spread between the two states. In sum, the problem being addressed was clearly identified in the state's transport sector strategy, the PDO provided an adequate response to that problem Page 4 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: AP and Telangana Road Sector Project (P096021) even as the transport sector issues were spread across two different states. There was only with minor misalignments with regard to the capacity of the new state of Telangana. Rating Relevance TBL Rating Substantial 4. Achievement of Objectives (Efficacy) EFFICACY_TBL OBJECTIVE 1 Objective - to provide better quality, capacity, and safe roads to users in a sustainable manner Rationale The Theory of Change was premised on using two activities - road improvement and road safety - to achieve better quality, better capacity, and safer roads. Road improvements would be achieved by upgrading priority state highways and maintenance of the core road network using Long Term Performance and Maintenance Contracts (LTPRMCs). Road safety would include demonstration projects showcasing measures that would reduce road accidents, adoption of road safety action plans, and initiation of a black spot improvement program. These outputs would lead to declines in the cost of vehicle maintenance, reduce travel time for passengers, and reduce the number of fatalities from road accidents in the demonstration corridors. OUTPUTS:  274 km of roads were efficiently upgraded  Of the 10,820 km of total classified network 6,800 km in Andhra Pradesh and 4,020 km in Telangana, 88 percent of the roads were in good and fair condition (baseline 40 percent, original target 80 percent, target exceeded). Actual achieved percentage was calculated based on weighted average of actual value for Andhra Pradesh (90 percent) and Telangana (80 percent).  328 km of roads (218 in Andhra Pradesh and 110 in Telangana) were upgraded efficiently and in an environmental and socially sustainable manner (baseline was 0, original target 429 km, revised target 340 km, of which 230 km was in Andhra Pradesh, and 110 km in Telangana, combined target mostly achieved).  6,018 km of roads under Long Term Performance Based Maintenance Contract (baseline 0, revised target 6,018 km of which 4,300 km in Andhra Pradesh, and 1,718 km in Telangana, target achieved)  95 percent of the land was acquired and 59 percent of project affected persons were resettled and rehabilitated (baseline 0, target for land acquisition in Andhra Pradesh was 384 acres and 106.16 acres in Telangana, target mostly achieved; baseline for resettlement and rehabilitation was 1, target for Andhra Pradesh was 2,792 persons and in Telangana was 310 persons, target not achieved)  Improved 36 black spots in the Andhra Pradesh core road network and 8 black spots in Telangana core road network (baseline data collected and black spots identified, revised target was phase II Page 5 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: AP and Telangana Road Sector Project (P096021) implementation and evaluation, revised targets of 36 black spots in Andhra Pradesh and 10 black spots in Telangana, targets achieved for Andhra Pradesh and mostly achieved in Telangana). OUTCOMES:  Vehicle Operating Costs (VOC) was reduced for cars to INR 4.88 per vehicle km relative to the "without project" scenario or 8 percent reduction (baseline for cars, INR 5.41 per vehicle km, original target INR 4.76 per vehicle km or 12 percent reduction, target mostly achieved). For trucks, VOC was reduced to INR 9.59 per vehicle km or 8 percent reduction (baseline INR 11.85 per vehicle km and original target was INR 10.9 per vehicle km or 8 percent reduction, target achieved).  Reduced travel time for bus passengers reduced by 23 percent to 8.30 hours (baseline 10.60 hours, original target 7.80 hours, or 26 percent reduction, target mostly achieved)  The number of road accident fatalities per vehicle km traveled on demonstration corridors was reduced to 1.56 fatalities, in Andhra Pradesh 0.57 per km, in Telangana at 0.99 per km (baseline 2.08 with Andhra Pradesh at 0.6 fatalities per km, Telangana at 1.48 fatalities per km, target 0.83 fatality per km, Andhra Pradesh at 0.47,Telangana at 0.36, target mostly achieved)  80 percent of the core road networks were upgraded and maintained (baseline 40 percent, target 80 percent, 92 percent in Andhra Pradesh and 80 percent in Telangana, achieving target) With most of the outcome indicators for achieving better roads and safer roads, achieved or mostly achieved, the efficacy under this objective was substantial. Rating Substantial OBJECTIVE 2 Objective • to enhance institutional capacity of the provincial government in the road sector. Rationale The Theory of Change was premised on implementing two activities to build institutional capacity in the road sector. These were facilitating Public Private Partnerships (PPPs) and the institutional strengthening components. PPP facilitation included activities that would strengthen the capacity of the government of Andhra Pradesh to develop selected high density traffic corridors using PPP arrangement via toll revenues and viability gap support from the central and state governments. PPP facilitation included (i) transaction advisors to help carry out feasibility studies and develop and transact selected road projects as PPPs; (ii) build capacity of the RDCs to identify, structure, award, and monitor PPP projects. Institutional strengthening activities would provide technical assistance, training and advisory services to operationalize the Andhra Pradesh Road Development Corporation, project implementation support to preparing and implementing an Asset Management Program (AMP), a Government and Accountability Action Plan (GAAP), and Institutional Strengthening Action Plan (ISAP). These outputs would lead to the RDC being fully operational with financial, technical, and administrative resources to manage the Core Road Network and with the capacity to engage the private sector to develop, finance, and manage road concessions. Page 6 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: AP and Telangana Road Sector Project (P096021) OUTPUTS:  selected 2 roads for PPP transactions (baseline 0, original target 3, revised target 2, one each in Andhra Pradesh and Telangana, target achieved)  Institutional Strengthening Action Plan (ISAP) studies and training needs assessment completed as targeted but only in Andhra Pradesh. The IT- Management Information System, Human Resource plan, environmental and social management plans were fully operational in Andhra Pradesh but not in Telangana.  Government and Accountability Action Plan (GAAP) studies completed for Andhra Pradesh, as targeted but not achieved in Telangana (baseline, none, revised target disclosure and complaints handling operational, target achieved for Andhra Pradesh but not in Telangana).  Annual maintenance plans were completed through the Road Management System (RMS), achieving target in Andhra Pradesh but not in Telangana.  Demonstration projects were carried out in at least two road corridors achieving targets in both Andhra Pradesh and Telangana (one each) (baseline, 0)  Comprehensive road safety functions, capacity, and action plans were in place (baseline, road safety lead agency established, formally revised target included new safety policy approved, institutional arrangements in place, and action plan implemented, target fully achieved in Andhra Pradesh and substantially achieved in Telangana). OUTCOMES:  Andhra Pradesh Road Development Corporation RDC was substantially operational with adequate financial and human resources to manage its Core Road Network (baseline, RDC not operational, original target RDC has its own staffing and adequate sources of revenue for CRN, target achieved),  Telangana RDC was formed in 2017 following the division of Andhra Pradesh into two provinces - Andhra Pradesh and Telangana. At project closing, the Telangana RDC organizational structure and financing arrangements were still under examination (target partially achieved). While outcome indicators were achieved for Andhra Pradesh, those that related to Telangana were mostly not achieved, leading to a modest rating for efficacy of this objective. Targets were doubled to accommodate the location of the projects after 2014 when the State of Andhra Pradesh became two separate states - Andhra Pradesh and Telangana. No additional resources were provided. Rating Modest OVERALL EFF TBL OBJ_TBL OVERALL EFFICACY Rationale Page 7 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: AP and Telangana Road Sector Project (P096021) Most of the outcome targets for Andhra Pradesh were achieved or mostly achieved. However, most of the outcome targets for Telangana were not achieved or at best, mostly achieved. Overall Efficacy Rating Primary Reason Modest Low achievement 5. Efficiency Economic efficiency. At appraisal, economic cost-benefit analyses were carried out for both the road upgrading package and the LTPBMC maintenance works. The net present value (NPV) and economic internal rate of return (EIRR) of each road upgrading package used the Highway Development and Management (HDM 4) model. Each periodic maintenance package used the Road Economic Design model. Net benefits were discounted over the life of the investments at 12 percent as the required economic opportunity cost of capital (EOCC). The ICR did not define which institution determined this EOCC. At closing, the ex-post economic analysis for the roads’ upgrade and the LTPBMC maintenance subcomponent (over 90 percent of total project cost) indicated favorable EIRRs and NPVs (ICR, Table 3). At closing, due to (i) implementation period overruns ranging from 6 months to 4 years, (ii) an average of 24 percent increase in construction costs, and (iii) lower-than-anticipated traffic growth led to a low EIRR but still greater than the 12 percent cost of capital justifying its viability (ICR, paragraph 34). Road Upgrade LTPBMC in AP only Appraisal Completion Appraisal Completion EIRR (%) 21.4 17.6 58.5 32.7 Net Present Value ((US$ 412 76 277 42 million) Operational and Administrative Efficiency: The project was designed to be implemented over a 5 year period. A total of 45 month extension was provided to complete a project that cancelled US$56 million in 2014 due to exchange rate gains (INR49 = US$1 at appraisal and INR62 = US$1 by December 2013), and again US$25 million in 2017 because a contract package was cancelled due to protracted land acquisition process and project scope was reduced. Operational efficiency was negatively affected by the slow progress on institutional reform, lack of coordination with Revenue Department as part of land acquisition, and limited capacity of staff, consultants and contractors (ICR, paragraph 47). There were unforeseen technical difficulties brought by the Bifurcation Act which required the replication of outcome indicators in the new state of Telangana and the attribution of outcome indicators for components across the two states. The new Land Acquisition and Resettlement and Rehabilitation Act also led to delays manifested in delays in payment compensation and prolonged litigation by landowners. Conclusion: Time overruns, reduced scope, quality of financial management, and suboptimal procurement processes reduced overall project efficiency. Telangana did not achieve some institutional development and road safety activities under Components 3 and 4. The project’s overall efficiency is rated modest Page 8 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: AP and Telangana Road Sector Project (P096021) Efficiency Rating Modest a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 94.10 Appraisal  21.40  Not Applicable 95.80 ICR Estimate  17.60  Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome On April 7, 2017, the project was restructured, the scope reduced, and US$25 million was cancelled. Two key performance indicators were amended : (i) the reduced scope due to the exclusion of the Kakinada Rajahmundry package from the project revised component 1 target from 427 km to 340 km (230 km in Andhra Pradesh and 110 km in Telangana); and (ii) from 3 to 2 PPP transactions (from two to one for Andhra Pradesh because the third road was reclassified as a national highway and removed from PPP consideration). The reduced scope led to a split rating. Overall Outcome Ratings Rating Dimension Original Objectives Objectives after Restructuring Relevance of Objectives Substantial Efficacy Objective 1-better quality, capacity, and safe roads to users Substantial Substantial for both CRN increase and mostly achieved  CRN increased reduced road fatalities was not targets in reduced road fatalities for both  Road fatalities reduced calculated till project closing. Andhra Pradesh and Telangana Objective 2 -enhance institutional Modest capacity of the provincial Substantial government  1 PPP in Andhra Pradesh  1 PPP in Andhra Pradesh  1 PPP in Telangana  RDC ability to engage  RDC fully operational in  RDC fully operational in Andhra private sector Andhra Pradesh Pradesh  RDC fully operational  RDC not operational in Telangana Overall Efficacy Substantial Modest Page 9 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: AP and Telangana Road Sector Project (P096021) Efficiency Modest Modest Outcome Rating Moderately Satisfactory Moderately Satisfactory Outcome Rating Value 4 4 Amount Disbursed (US$ million) 110.8 80.6 Disbursement Rate (%) 110.8/191 = 58 80.6/191 = 42 Weight Value 4*.58 = 2.32 4*.42 = 1.68 Total Weights 2.32 + 1.68 = 4 Overall Outcome Rating 4 a. Outcome Rating Moderately Satisfactory 7. Risk to Development Outcome The following posed risks to development outcome:  Technical Risks. The software systems introduced under the RAMS, and MIS in Andhra Pradesh would require updating and maintenance. To mitigate this risk, Andhra Pradesh has created a Road Fund and reportedly pursuing institutional strengthening of its road authority. Andhra Pradesh has issued decrees and ratified bills creating this mechanism.  Financial Risks. The pavement conditions of the two road safety pilot corridors deteriorated due to the overloading of freight trucks and lack of proper maintenance. To mitigate this risk, a long-term contract for road maintenance, including the upkeep of roads’ safety features (for example, guardrails, jersey barrier, marking, signs), and a proper overloading control policy need to be in place. In Andhra Pradesh, the Road Building Department has included a long term performance based management contract model for the strategic maintenance of the 4,600 km of state highways it managed. RBD revised the state's maintenance contracts to follow this model.  Institutional Support Risks. The road safety agencies in both Andhra Pradesh and Telangana were established to deliver integrated road safety. Activities covered transport, health, and police. There is a risk that coordination may not continue without sustainable funding sources. To mitigate this risk, Andhra Pradesh created an initial fund while Telangana was in advanced stages of enacting the legislation that would establish this fund.  Risk from unmitigated overloaded freight trucks on the roads. Overloaded freight trucks were observed to have caused the deterioration of the pavements of the two road corridors where road safety measures were piloted (one in each state). To mitigate the risk of further deterioration of road pavements, an appropriate overloading control system using electronic weighbridges needs to be adopted in both states. Page 10 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: AP and Telangana Road Sector Project (P096021) 8. Assessment of Bank Performance a. Quality-at-Entry Lessons learned from the preceding road sector operation in the state of Andhra Pradesh informed project design. Among lessons learned was avoiding delays for land acquisition, and obtaining clearances from various government departments by undertaking these activities before work commenced; and ensuring an adequate and assured flow of maintenance funding and developing capacities for systemic network planning and management for sector sustainability (PAD, paragraph 26). The project was supported by strategic relevance to the country's development plans. The components were realistic and designed to achieve project objectives. The capacity of the implementing agency did not pose risks to implementation. Risks were generously assessed (ICR, paragraph 46) but for two elements that adversely affected the project: land acquisition and the capacities of project contractors and consultants. This was evidenced by changing risk categories for both elements from Moderate to Substantial as mitigation measures proposed for both were simplified and did not adequately consider administrative complexities and landscape of the local road construction industry. In addition, financial management assessment and procurement assessment also appeared to have been generous since implementation resulted in significant delays arising from these two factors (see Quality of Supervision below and Section 10. Other Issues, (b) Fiduciary Compliance). The moderate shortcomings in preparation resulted in the rating below. Quality-at-Entry Rating Moderately Satisfactory b. Quality of supervision The World Bank team carried out twice a year supervision missions between 2010 to 2012 and three in 2013, followed by 4 supervision mission in 2014 and another 3 in 2015. All in all there were 23 supervision missions over the almost 10 year implementation period. The Bank team proactively identified and resolved threats to achieving the PDO. as evident in the frequency of the supervision missions, and in particular after the state of Andhra Pradesh was divided into Andhra Pradesh and Telangana. This division led to technical difficulties for which the project was unprepared. Andhra Pradesh showed its commitment to the project through the appointment of its key staff, while Telangana took some time to address project needs. The government fully disbursed its counterpart financing. According to the ICR, the World Bank’s Aide Memoires were candid pointing out reasons for its sluggish performance, and offered actionable corrective measures to address those areas with an eye to achieving the PDO outcome indicators. The sector's inherent complexities such as poor interagency coordination, contractor performance (such as aggressive bidding, dubious joint ventures), and high personnel attrition rates negatively affected performance. Bank missions diligently offered measures and action plans to rectify problem areas (ICR, paragraph 68). However, fiduciary risk continued to be an issue even as the Bank team offered corrective measures. Procurement and contracting issues were addressed through intensified training and follow up Page 11 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: AP and Telangana Road Sector Project (P096021) (of the eight work packages released, 4 were terminated and rebid (ICR, paragraph 50) and a contract was cancelled because of land acquisition issues in 2017). Transition arrangements after loan closing were evident in the actions taken by Andhra Pradesh on its own initiative. Andhra Pradesh has mandated that multi-year rolling investment and maintenance plans for roads use RAMS and MIS to prioritize activities and allocate maintenance funds. Andhra Pradesh also developed a command and control center integrating RAMS, MIS, and web- and GIS based maps (ICR, paragraph 73). Its Road and Building Department (RBD) also used the long term performance based maintenance contracting model to bid and contract documents for a strategic maintenance plan for 4,600 km of state highways (ICR, paragraph 74). Recall that the RBD managed 68,510 km of state highways and major district roads (60 percent of total roads in Andhra Pradesh) while RDC managed the designated 10,000 km of state highways and major district roads under its core road network. In the case of Telangana, its state Road Development Corporation was established in February 2017 to oversee flagship road development programs but its version of RAMS and MIS were not in place nor was the Road Fund established yet (ICR, paragraph 29). Telangana is also in advanced stages of enacting legislation to ensure funds for its road safety agencies to deliver integrated road safety activities (ICR, paragraph 72). However, there was a lack of realism and unrealistic expectations about the capacity of the newly created state administration in Telangana following the Bifurcation. Quality of Supervision Rating Moderately Satisfactory Overall Bank Performance Rating Moderately Satisfactory 9. M&E Design, Implementation, & Utilization a. M&E Design The design of the M&E framework included clear and measurable indicators, including length of upgraded and maintained roads and savings in VOC and travel time. The theory of change was sound and reflected in the results framework, including the ability to launch PPP projects and the operationalization of the Road Development Corporation. Intermediate results indicators captured the contribution of the activities toward achieving the outcomes. Indicators were specific, measurable, achievable, and time bound. Baselines and targets were available for all indicators (except for the two intermediate outcome indicators added during restructuring - see Implementation below). According to the PAD, Road Users Satisfaction Surveys would be carried out at the start (baseline) mid-term, and near closure of the project to assess the Road Development Corporation's ability to provide effective road services to users (PAD, paragraph 41). A minor shortcoming in design was the indicator used for road safety. M&E design did not anticipate that the project would enhance road design, control road access, and improve intersections. Thus, the indicator for road safety (number of fatalities per vehicle kilometer) was deemed better expressed in the number of fatalities per 100 vehicle kilometer travelled. The M&E design was originally embedded with the implementing agency but the Bifurcation Act of 2014 dividing Andhra Pradesh into Andhra Pradesh and Page 12 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: AP and Telangana Road Sector Project (P096021) Telangana, required the addition of an implementing entity from the state of Telangana. The M&E framework was revisited with regard to enhancing institutional capacity as a result of this 2014 Bifurcation Act. The Telangana road authority was formed in July 2017 leading to project restructuring, revisiting target indicators, revising the results framework. b. M&E Implementation The M&E system was implemented by the Andhra Pradesh Road Development Corporation (RDC). Third-party consultancies collected, reported, and analyzed data and developed in-house M&E capacities by developing and implementing the Road Asset Management System and Management Information System. The Road Users Satisfaction Surveys were reported as completed as part of acheiving the Governance Accountability Action Plan but no details were reported in the ICR. Other M&E measures were introduced later including percentage of land acquired for the project and percentage of resettled and rehabilitated project affected persons in response to concerns over the progress of both items and its negative impact on the project progress. However, these two indicators did not have any targets. The reference Implementation Status and Results Report in the footnote (ISR 14, April 28, 2015) provided targets for land acquired in Andhra Pradesh (384 acres) and Telangana (106.16) and the number of project affected people resettled and rehabilitated (2,792 in Andhra Pradesh and 210 in Telangana). Three indicators—VOC, time savings, and accident reduction—were deferred until the project closed. c. M&E Utilization M&E data on performance and results progress informed the World Bank’s supervision missions. Data was also used in restructuring the project, adjusted activities during implementation, identified slow- moving road upgrading packages for termination and rebidding, adjusted target road lengths following difficulties in land acquisition and site access issues, and revised the number of PPP indicators, among adjustments made. M&E data was used to provide evidence of achievement of outcomes. M&E data was also used to inform the lessons learned from the project operations. M&E Quality Rating Substantial 10. Other Issues a. Safeguards Environmental and Social Safeguards. The project received a category A and triggered the following environmental safeguards:OP/BP 4.01 Environmental Assessment, OP/BP 4.04 Natural Habitats, OP/BP 4.36, Forests, OP/BP 4.10 Indigenous Peoples, OP/BP 4.11 Physical Cultural Resources, and OP/BP 4.12 Involuntary Resettlement. During implementation OP/BP 4.10 was not applied. None of the upgraded roads passed through tribal areas. Compliance with the other safeguards were not reported in the ICR but the Operations Portal reported an unsatisfactory overall safeguards compliance, a moderately satisfactory compliance with OP/BP 4.01 and 4.36; satisfactory compliance with OP/BPs 4.04 and 4.11, and Page 13 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: AP and Telangana Road Sector Project (P096021) unsatisfactory compliance with OP/BP 4.12. The ICR reported that OP 4.12 remained unsatisfactory or moderately unsatisfactory until project closing (ICR, paragraph 60). The Resettlement Policy Framework (RPF) was revised in 2015 to reflect the new Land Acquisition Act and Resettlement Act of January 2014, including retroactive provisions if payments were not dispensed by its due date. The new Act disrupted the payment of compensation and resettlement assistance to project affected persons (PAPs). Overall unsatisfactory safeguards compliance was reported in the Operations Portal. Four hundred twenty (420) acres were acquired from 2,200 private landowners. There were 1,400 families displaced and 1,100 of them were informal settlers who lost their small businesses. Resettlement costs reached about US$19 million (ICR, Annex 5) distributed to about 2,000 untitled holders to maintain their pre-displacement conditions, reestablish their livelihoods, and housing. Requirements of environmental and social safeguards were integrated in engineering design to ensure compliance during implementation. The Andhra Pradesh Road Development Corporation retained and mainstreamed these measures into its contract documents and project processes. The implementing agency engaged seven local nongovernment organizations, a sociologist, and a Deputy Collector to provide land acquisition and resettlement support and coordinate implementation with the state governments. At the district level, monitoring committees headed by joint collectors, oversaw grievances and individual complaints. A ‘Disclosure and Complaint Handling’ mechanism was operated by the Andhra Pradesh Road Development Corporation (RDC) website. There were 341 grievances received, 184 (54 percent) were related to discrepancies in resettlement and rehabilitation entitlements, eligibility for these benefits, changes in eligibility lists, and demands for higher compensation. 166 (48 percent) of these grievances were resolved. The Grievance and Redress Mechanism (GRM) dealt with land acquisition and resettlement related complaints while the Labor Commissioner dealt with labor related complaints. The Labor Association filed a case in the High Court. The Contractor was instructed to pay outstanding labor payments before pending bills were approved. A few outstanding land acquisition and resettlement activities emerged early in the project - from year 1 when there was substantial delay in initiating the land acquisition process; year 2: disclosure of the Resettlement Policy Framework in Andhra Pradesh in 2017 and payment of balance compensation to all vulnerable land losers in Chittur-Puttur Road. There was also outstanding action related to non-payment of wages to 70 unskilled workers in Kurnool-Devananda road in Andhra Pradesh (INR 2.50 Million). According to the ICR, the Bank would follow-up with the implementing agency until these outstanding actions were resolved. Andhra Pradesh did not undertake independent resettlement impact studies to evaluate the effect of resettlement activities while Telangana initiated one in August 2019. Preliminary findings in Telangana indicated that (i) land acquisition took an average of two years, (ii) a portion of non-agricultural families affected by the project, housing conditions and household assets improved over the baseline; (iii) pending issues remained with land acquisition such as payments due to 11 land owners and resettlement support payments due to 33 vulnerable farmers who became landless. Overall compliance with resettlement safeguards was unsatisfactory according to the Operations Portal and reported Moderately Unsatisfactory and Unsatisfactory in the ICR (paragraph 61). Overall safeguards compliance was also reported as unsatisfactory in the Operations Portal but not in the ICR. Page 14 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: AP and Telangana Road Sector Project (P096021) b. Fiduciary Compliance Financial management. Financial management was rated Moderately Satisfactory during most of the life of the project. Reports were submitted on time or with modest delays and provided information on project progress and expenditure patterns. Availability of funds for project payment was generally adequate, with one exception for Telangana in August 2018, which was eventually resolved, and adequate funds were provided to RDC. Internal audits (ICR, paragraph 65) reports were delayed, and when received, observations replicated those in preceding reports. These observations indicated that no follow up actions were taken on the previous internal audit findings. External audit reports showed disallowances, which reached nearly INR 1,020 million (approximately US$14.8 million) by December 2015. Disallowances were related to mobilization advances and technical aspects. The audit process and the team of auditors were not given the required documentation at the time of the visit. The project team met with the Accountant General to ensure that auditors were provided the needed information and to break the cycle of disallowances. In July 2016, a workshop was conducted to clear the disallowances backlog. A significant part of the disallowances was settled in subsequent years. However, the May 2019 Certification Audit of Expenditure report cited unresolved disallowances. Procurement. Procurement had coordination challenges and suboptimal contractor performance (ICR, paragraph 78). During post procurement review, delays in payments to contractors were noted due to administrative delays in approving and clearing payments. Contract modifications (e.g., extension of time) were executed before obtaining prior Bank approval as requested by the Bank team. Site work packages (road upgrade, road maintenance, civil works for road safety) that required close coordination with multiple agencies (e.g., Irrigation, Forest, and Revenue Departments) were negatively affected by the impact of the Bifurcation Act because of a lack of clarity in contract administration. In addition, interagency coordination among Transport, Health, and Police Departments was also a challenge in procuring road safety packages. In addition to coordination, the contractors' performance was slowed by aggressive bidding and pricing. This led to cashflow problems and failure to meet contract obligations. There was a general sense of poor planning and mobilization handicaps by contractors. The implementation of the new Land Acquisition and Resettlement Act resulted in a lack of access to project sites. There were delays in clearing forest trees. These factors led to termination and rebidding of 4 of the 8 work packages that covered the road upgrading, leading to cost overruns and significant time delays. In some cases, due to the complex procurement arrangements contained in contract packages, the implementing agency took up to 3-4 years to terminate and rebid (ICR, paragraph 51). One road improvement package contract was dropped/cancelled because it was too far behind to be completed by project close. Corrective measures were suggested for future consideration: (i) ensuring a stricter qualification process for contractors (technical, financial, prior experience, personnel, equipment) to ensure adequate performance and acceptable quality; (ii) using weighted criteria and combined technical and financial scores to mitigate aggressive bidding; (iii) revisiting current procurement guidelines and contract documents to enforce punitive provisions in case of contractors’ underperformance, including if needed, the exclusion from opportunities to bid on future projects; and (iv) moving away from the traditional contracting process and opting for Engineering, Procurement, and Construction/PPP type modalities that were implemented in other states (ICR, paragraph 78). Page 15 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: AP and Telangana Road Sector Project (P096021) c. Unintended impacts (Positive or Negative) --- d. Other --- 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment No disagreement. Both the ICR and this review concluded MS Outcome Satisfactory Moderately Satisfactory outcome after the split rating. The Data Sheet erronously noted Satisfactory outcome. Risk factors at entry were not adequately addressed. Contracting, procurement issues Bank Performance Satisfactory Moderately Satisfactory and issues associated with compliance with safeguards remained throughout supervision. Quality of M&E Substantial Substantial Quality of ICR --- Substantial 12. Lessons The ICR presented a few lessons and recommendations from this operation that may benefit future similar operations. Some of these are noted below:  Establishing forward and backward linkages to existing systems may better sustain project outcomes. In this project, the Road Asset Management System (RAMS) and management information system (MIS) replaced the ad hoc decision making of Andhra Pradesh in prioritizing budget allocations for investment planning and road maintenance. The RAMS and MIS allowed the state to use a rolling, multi-year planning strategy using evidence from the reports generated by the systems to justify priorities. Andhra Pradesh also established a command and control center that integrated RAMS and MIS with web- and GIS-based maps and linked other systems, such as contract administration and decision support tools for its Road Development Corporation (RDC) and Roads and Building Department (RBD). For example, the RDB in Andhra Pradesh used a long term performance based management contract worth INR3,000 million (equivalent to US$42 million) for a new maintenance plan covering 4,600 km of state highways. This contract was linked to the MIS system to monitor physical and financial progress. The lesson from this project was that the Page 16 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: AP and Telangana Road Sector Project (P096021) capacity of an implementing agency capacity to plan investments and budget for road maintenance may be improved by providing forward and backward linkages to existing systems.  Having a focal point for road safety management at the state level may be useful to achieve efficiencies. In this project, Andhra Pradesh established a cell in the state's transport department tasked with planning and monitoring the impact of road safety initiatives, managing data for road safety indicators, and coordinating with other agencies and constituents. This one-stop shop for road safety measures made available credible data and fostered coordination. The lesson learned from this project was that by creating a focal point for road and safety management, the state achieved better coordination and efficiency in delivering relevant, credible data to end users.  Ensuring fair compensation in land acquisition, resettlement, and rehabilitation avoids implementation delays. In this project, in Andhra Pradesh, by January 2014, only 55 percent of land acquisition was completed, with 24 percent of project affected families receiving resettlement and rehabilitation assistance. By March 2015, a reported three year delay in payment to land owners; and payment of cash assistance to vulnerable families remained outstanding. By June 2019, after the project closed, there were reports in Andhra Pradesh of outstanding compensations to 65 landowners, partial resettlement assistance to 363 project affected persons and additional compensation to 542 landowners who became landless. In Telangana, partial land acquisition compensation to 10 landowners and additional compensation to 33 landowners who became landless remained pending. The Bifurcation Act followed by the implementation of a new Land Acquisition and Resettlement Act, also contributed to the delays in releasing compensation. In some cases, landowners litigated even after being compensated. The lesson from this project was that by adjusting for inflation, fair compensation for land acquisition, resettlement, and rehabilitation for project affected persons fosters confidence in the government's intention to be fair, avoid prolonged litigation to settle differences, and overcome implementation delays.  Continuous capacity building efforts helps new institutions. In this project, the 2014 Bifurcation Act introduced a new implementing entity in the new state of Telangana to replicate the sector in Andhra Pradesh. Resources were allocated to Telangana with little support for capacity building. The unbalanced attention to building the capacity of Telangana was evident in the unmet project targets. The lesson learned from this project was that when new government entities are formed continuous technical assistance and training help build adequate institutional capacity.  Appropriate data should be generated to inform adequate response to road safety issues. In this project, overloading was a leading cause of road accidents involving trucks. To check the overloading violations, electronic weigh bridges were proposed on the demonstration corridors. According to the task team, the two states planned and budgeted for these weigh bridges to support their road safety programs. The lesson learned from this project was that credible data could be used to justify investments in infrastructure such as weigh bridges to optimize road safety. 13. Assessment Recommended? No Page 17 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: AP and Telangana Road Sector Project (P096021) 14. Comments on Quality of ICR The ICR provided a comprehensive narrative of the project. The theory of change was expressed in the results framework. The report was focused on results with a verifiable accounting of the difficulties encountered. Evidence of outcomes was robust, pointing to the impact of the systems installed under the project. Annexes in the ICR supported results claimed by the project. Sections were mutually reinforcing. A minor shortcoming was in lessons learned. There were no lessons from the private sector's participation in the road sector or other possible private sector engagements beyond intercity highways. In addition, the data sheet reported the outcome as satisfactory but the text supported a moderately satisfactory outcome. a. Quality of ICR Rating Substantial Page 18 of 18