ICRR 14103 Report Number : ICRR14103 IEG ICR Review Independent Evaluation Group 1. Project Data: Date Posted : 06/03/2013 Country : China Project ID : P099112 Appraisal Actual Project Name : Anhui Highway US$M ): Project Costs (US$M): 410 451.43 Rehabilitation And Improvement Project L/C Number : L7511 Loan/ US$M): Loan /Credit (US$M): 200 200 Sector Board : Transport Cofinancing (US$M): US$M ): 0 0 Cofinanciers : Board Approval Date : 04/22/2008 Closing Date : 06/30/2012 06/30/2012 Sector (s): Roads and highways (99%); Sub-national government administration (1%) Theme (s): Infrastructure services for private sector development (100% - P) Prepared by : Reviewed by : ICR Review Group : Coordinator : Fang Xu Robert Mark Lacey Soniya Carvalho IEGPS1 2. Project Objectives and Components: a. Objectives: The project development objective stated in the Project Appraisal Document (PAD, page 4) and the Loan Agreement (Schedule 1, page 4) was: "to increase the effective use of road infrastructure in Anhui Province to support its social and economic development ." The outcome target (PAD, page 4) was an "increase in the flows of passenger and freight traffic along key corridors across the province at lower costs and improved safety .� b.Were the project objectives/key associated outcome targets revised during implementation? No c. Components: There were four components: 1) Road rehabilitation (estimated cost US$193.61 million, actual cost US$196.14 million). This component was to support Anhui provincial rehabilitation and maintenance program from 2007-2010 under the Eleventh Five Year Program by financing the rehabilitation of 18 segments totaling about 890km of the road network. Activities under this component consisted of improving pavement conditions of the roads, specific structural strengthening and limited adjustments to the alignment when required to improve safety . Contract supervision was included in this component . 2) Road improvement (estimated cost US$209.85 million, actual cost US$218.98 million). This component was to improve or upgrade about 320km of key provincial and national roads located in the central-eastern and southeastern areas of the province . Contract supervision was included in this component . 3) Pilot on road maintenance using output-performance based road maintenance contracting method (estimated cost US$1.95 million, actual cost US$ 5.10 million). Under this component, Anhui Provincial Communications Department was to implement maintenance using output-performance based road maintenance contract approach in at least two highway segments of the provincial highway network for a cumulative number of 120 kilometers. 4) Institutional Strengthening Program (estimated cost US$ 4.09 million, actual cost US$3.84 million). This component included provision of improved tools for road management (technical, environmental and economic) and a training program. Six sub-activities were included in this project: a study on technological options for the recycling of pavement materials when a pavement needs to be rehabilitated; technical assistance for the pilot on road maintenance contract; preparation of standardized environmental specifications; support Anhui Highway Administrative Bureau in implementing the enhanced version of Chinese Pavement Management System in Anhui Province; socio-economic impacts analysis of key roads to be improved under the project; and, training program and study tour. The original project components were not revised. However, during project implementation, some changes were introduced. Due to the emergency needs of the Beijing Olympic Torch relay in Anhui in 2008, Anhui Provincial Communications Department resurfaced the Jixi Section of S215 under rehabilitation sub-projects with domestic funds. Further works to the section were unnecessary. The Bank loan finally funded the civil works of the remaining 17 rehabilitation roads and 7 improvement roads. The actual component costs cited above are taken from Annex 1, table (a) of the ICR. The project team confirmed that these cost figures were accurate. Although not a separate component of the project, the resettlement and land acquisition costs were estimated at US$ 12.75 million and US$28.87 million respectively at appraisal. d. Comments on Project Cost, Financing, Borrower Contribution, and Dates: Project costs : At appraisal, the estimated project cost was US$ 410 million. At project completion, the total actual cost was US $451.43 million. Both totals include land acquisition and resettlement costs . The cost increase was reflected the higher than estimated cost of both civil works activities and resettlement . financing: The original approved IBRD loan amount was US$200 million, which remained unchanged during Project financing: the course of project implementation . There were no other external sources of financing . contribution: the Borrower's contribution was estimated at appraisal to be US$210 million, and actual Borrower's contribution: contribution was US$ 251.43 million, as a result of increased project financing needs . Dates : the project closed on schedule on June 30, 2012. 3. Relevance of Objectives & Design: a. Relevance of Objectives: This project objective -- to support Anhui, a relatively less developed province's social and economic development -- is fully in line with the Country's priorities and corresponds to the Bank's Country Partnership Strategy (FY2013-16). China's 12th Five Year Plan (2011-2015) calls for focusing on improving the quality of life, rather than just the pace of growth and paying attention to balanced growth . The Country Partnership Strategy (CPS) for Fiscal Years 2013-2016 under its second strategic theme, includes "improving transport connectivity for more balanced regional development." The objectives were also relevant to the Bank's 2006-2010 Country CPS with China. Under the second pillar of the strategy, upgrading of transport infrastructure, while improving network management and traffic safety, was singled out as an element contributing to reducing poverty, inequality and social exclusion . Relevance of objectives is rated as Substantial . b. Relevance of Design: The statement of development objectives is clear and measurable. The causal chain between activities supported by the project and attainment of the development objectives is clear and logical. The road improvement and rehabilitation components would improve the road condition and road connectivity, the performance-based road maintenance contract pilot would indicate ways of improving the quality and efficiency of road maintenance; the institutional strengthening program would increase the capacity to better manage the road assets, All these designed activities (project inputs) could reasonably be expected to bring about enhanced road conditions (output), thereby providing the benefitting communities with better, lower cost transport. This in turn would improve their access to social and economic services, and contribute to the development of the province . The project also included road safety measures, such as improving road surface conditions, applying uniform engineering standards, improving alignment to enhance sight distances, clearing drains, setting up road signs, etc, which would contribute to improved road safety. There was, however, no activity addressing issues of road safety behavior and awareness; these factors weakened the project's contribution to enhanced safety . Relevance of design is rated as Substantial . 4. Achievement of Objectives (Efficacy): The degree of achievement of the project's development objective -- “to increase the effective use of the road infrastructure in Anhui Province and to support its social and economic development .� -- is rated as substantial . Outputs : At project completion, all planned activities had been implemented with satisfactory quality, within budget and on schedule, except one section of road that was taken over by the Province due to the urgency of the Beijing Olympic Torch Relay in Anhui in 2008. The main project outputs are summarized as follows : 1) 840.85 km national and provincial road were rehabilitated, as against the original target of 885.6km (because of the road section taken over by the Province ). 2) 320.6 km of national and provincial roads were improved compared to an original target of 318.1km. 3) Output-performance based road maintenance contracts were piloted as intended on two highway sections totaling 120 km. 4) Testing of asphalt recycling for pavement was conducted as intended on five road sections and the completion report was issued on December 12, 2011. 5) The Chinese Pavement Management System has been used by all 17 municipalities in accordance with the target. 6) An Environmental Specifications Manual for Highway Construction was completed on time by July 2010. 7) 1,230 person-months of domestic training, 32.2 person-months overseas training and 9.67 person-months overseas study tour were conducted . Outcomes : According to the project's outcome target, the increased effective use of the road infrastructure would be measured by an increased traffic flow through the project road infrastructure in a faster, safer and cheaper manner. The evidence as provided by ICR was as follows : a) Traffic volume on rehabilitated and improved roads increased by 7.5% and 7.31% at project completion, both exceeding the appraisal targets (5.1% and 4.5% respectively). During the same time period, fuel prices increased dramatically and economic growth was higher than projected, the former having a potential negative impact of traffic volume while the latter would have a positive one. Depending on which impact is bigger, traffic volume growth may be over- or under-attributed to the project. It seems however reasonable to assume that improved road conditions have generated part of the increased traffic volume . b) Annual accident rates on improved roads and rehabilitated roads decreased, respectively, from 0.32 per thousand vehicle kilometers to 0.28 and from 0.34 to 0.32. These were short of the respective targets of 0.20 and 0.18 due to the continued prevalence of drunk-driving, overloading, weak safety awareness, and inadequate training of new drivers. Road safety behavior and awareness were not part of the project . c) Travel speeds on rehabilitated and improved roads had increased by 16.0% and 25% respectively by project completion. This was on target for rehabilitated roads but below the target of 34.8% on improved roads due to greater speed limit enforcement on the latter . d) The decreases in transport tariffs did not fully achieve the targeted value by project completion mainly due to increased fuel and labor prices in China. However, as the ICR acknowledges, transport tariffs are an inappropriate measure of the impact of the project on transport costs, since they are influenced by a wide range of institutional and political factors and not just by improved road conditions. A more appropriate measure would be the impact of the improvements on transport costs. The ICR did not provide direct information on transport costs. However, the project's Economic Internal Rate of Return, which is a consolidated outcome of reduced vehicle operating costs, reduced accident costs and time savings, was estimated at 30.2% at project completion (see Section 5 below). Since project capital costs were in excess of those estimated at appraisal, this result indicates that the project may have resulted in significant reductions in the three categories of transport cost (vehicle operation, time and accidents ). A number of other factors may have contributed to the degree of attainment of the outcome targets. For example, a higher traffic volume could also be the result of increased vehicle ownership, itself the consequence of rising incomes. Travel speed depends to some extent on local regulation, including enforced speed limits. Accident rates reflect driver behavior and safety awareness as well as road conditions . A follow up study (ICR, page 10) found that travel time on the project roads was shortened, and that the average income of the 10,000 people living along highway G205 increased from RMB1,500 in 2007 to RMB6,500 in 2011. However, this increase could not be attributed to the project alone . 5. Efficiency: At project appraisal, the HDM-4 model was used for the economic evaluations for all 25 road sections either to be rehabilitated or to be improved. The assumptions for the economic evaluation were : a) no diversion of traffic from the railway line; b) traffic growth at 5.1% until 2012, decreasing thereafter to 3.3% to 2020 and 1.7% to 2030; and c) possible additional benefits from enhanced access to social services were not included . The average ex ante Economic Rates of Return (ERR) for road rehabilitation and improvement were 28.2% and 26%, respectively; the Net Present Values for rehabilitation and improvement were RMB 1, 935 million and RMB 2,023 million, respectively. The overall ERR for all road improvements was 27%. At project completion, the same methodology was applied for the 24 road section rehabilitation or improvements supported under the project (one road section improvement being financed outside the project by the Borrower). The results indicate that the ERRs were in the range of 19.9%-51.4%, with the overall ERR for all road sections estimated at 30.2%. The improvement over the appraisal estimate of 27% was mainly due to higher than projected traffic volumes on road sections improved or rehabilitated by the project . There were no significant operational or administrative inefficiencies . The project closed on time. Project construction costs were about 10% higher than the appraisal estimate, due mainly to inflation in material prices . Efficiency is assessed as substantial . ERR )/Financial Rate of Return (FRR) a. If available, enter the Economic Rate of Return (ERR) FRR ) at appraisal and the re- re -estimated value at evaluation : Rate Available? Point Value Coverage/Scope* Appraisal Yes 27% 93.6% ICR estimate Yes 30% 92% * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome: The project objectives are substantially relevant to the priorities of the country and the Bank's Country Partnership Strategy. The causal chain of the project design is mostly clear and logical . The achievement of project objective of increasing the effective use of road infrastructure in Anhui province is substantial, as evidenced by the increased traffic volume, reduced traveling time, lower transport cost and reduced road accidents rate . Efficiency is substantial with an ERR above appraisal estimates and well in excess of the opportunity cost of capital; operational and administrative inefficiencies were not significant . Overall outcome is rated Satisfactory . a. Outcome Rating : Satisfactory 7. Rationale for Risk to Development Outcome Rating: The risk to development outcome is rated as Moderate . There are two important risks: the shortage of maintenance funds and evolving institutional arrangements for maintenance . Regarding the first, an analysis carried out during preparation showed that the resources allocated to maintenance and rehabilitation should be increased from 57% (in 2006) to 62% of the Anhui Highway Administrative Bureau ’s budget to ensure a steady good condition of the roads network (PAD, page 11). However, at project completion, the percentage of the Bureau ’s budget allocated to roads network maintenance and rehabilitation had actually fallen to 55.1% (ICR Data Sheet). This risk could be mitigated by the increased use of asphalt recycling technologies (which will reduce maintenance costs ) and of performance-based contracts (which will increase maintenance efficiency ). Concerning the second risk, there is a tendency throughout China to decentralize highway maintenance responsibility from Provincial Government to local authorities, which usually have less capacity and resources . However, Anhui Province has an established highway administration system at provincial and local levels with well experienced staff who have executed several Bank-supported projects in a timely and satisfactory manner . In addition, in Anhui Province, the decentralization of responsibility has been accompanied by the transfer of resources to the local level . a. Risk to Development Outcome Rating : Moderate 8. Assessment of Bank Performance: a. Quality at entry: Preparation of this project built upon the experience of two prior successful highway operations in Anhui province. The main lesson learned from the previous projects and reflected in the design of this projects was to refocus the sector agenda from road construction to road assessment management and serviceability of the network. The project activities were, therefore, designed to focus on road network rehabilitation and improvement, which was in line with both the Borrower ’s and the Bank's development strategy for the country . The design covered all aspects of roads assets management including improving the condition of the road network, more efficient maintenance contracting methods and technology, and more effective institutional arrangements. Environmental and social issues were fully considered at project preparation stage with a view to minimizing adverse impacts. Implementation arrangement were well thought -out and comprehensive. Anhui Highway Administration Bureau was the lead implementation agency, supported by the Project Executing Office . There were Sub-project Construction Management Offices to manage and supervise the civil works in each city where projects investment took place . However, as noted in the ICR, there were a number of issues with project preparation. First, the large number of contracts (55, covering 24 road sections dispersed over 12 cities and prefectures) increased the difficulty of project management and implementation . Second, the two-year contract period for the two pilot performance -based maintenance contracts was too short to capture fully the benefits of such contracts in terms of improved road asset management . Moreover, the actual cost of the pilot contracts (US$4.8 million) was three times the appraisal estimate (US$1.6 million). Third, the M&E framework as designed was unable to monitor adequately several factors such as changes in road safety policy the regulatory framework for highway use, fuel price fluctuations, and rapid economic development, all of which influenced the outcome of the project (see Section 10a below). Quality at entry is rated as Moderately Satisfactory . at -Entry Rating : Quality -at- Moderately Satisfactory b. Quality of supervision: The ICR’s discussion of the quality of supervision is sparse and lacking in specifics . However, at project completion, i) all contracts had been completed on schedule; ii ) the Bank's loan was fully disbursed; iii ) the Borrower's ICR notes the project's value added (see ICR, Annex 7); and iv) there was full compliance with the Bank's financial management and safeguard policies (see Section 11 below). In addition, the ICR notes (page 14) that during project implementation, the Bank team worked with the client in a constructive way : i) to ensure achievement of project objectives and compliance with safeguard and fiduciary policies; ii ) to improve traffic safety and the quality of rehabilitated and improved roads; iii ) to promote good practices during project implementation, such as provision of training to villagers and partnering with local governments on environmental protection; and iv) to solve implementation issues effectively and efficiently . Quality of supervision is rated Satisfactory . Quality of Supervision Rating : Satisfactory Overall Bank Performance Rating : Moderately Satisfactory 9. Assessment of Borrower Performance: a. Government Performance: The Government in this project refers to the Anhui Provincial Government . The ICR (page 15) notes that the Government fully supported the project from preparation through completion; provided adequate counterpart funds; did its part to ensure that project activities would be completed in a timely manner; and, demonstrated full commitment to the training program -- including the tightly controlled overseas training activities -- as originally intended. Government Performance Rating Satisfactory b. Implementing Agency Performance: The implementing agency was Anhui Highway Administrative Bureau . Its performance was satisfactory as evidenced by: Implementation arrangements were appropriate. There was a Project Executing Office, under which 12 local city/prefecture highway administration bureaus assigned experienced staff to form the Sub-Project Construction Management Offices. In addition, Anhui Highway Administrative Bureau established a Chief Supervision Engineer’s Officer to enhance the supervision of project implementation . The implementation agency maintained a close working relationship with the Bank team to ensure that implementation issues could be resolved in a timely manner. For example, when some contractors expressed their intention to terminate their contracts due to low contract prices, Anhui Highway Administrative Bureau handled the situation well. Consequently, 43 out of 55 civil works contracts and most of the Institutional Strengthening Program were satisfactorily completed one year before the closing date, while the remaining activities were fully completed by May 31, 2012, one month prior to the closing date . The M&E framework was satisfactorily implemented (see Section 10 below). Bank and national safeguards policies were complied with (see Section 11 below). Implementing Agency Performance Rating : Satisfactory Overall Borrower Performance Rating : Satisfactory 10. M&E Design, Implementation, & Utilization: a. M&E Design: Baselines and targets were established for all key performance indicators . The Anhui Highway Administrative Bureau was responsible for the M&E system, delegating where appropriate to the Project Executing Office and the Construction Management Offices. However, indicators did not take into account the influence of potential exogenous factors. Consequently, the achievement of project development objectives could not be accurately monitored and it was difficult to establish the extent to which achievements could be attributed to the project . For example, the average travelling speed was not only affected by the road conditions, but also to a large extent by the traffic volume and speed control . As noted in Section 4 above, some core indicators, such as transport tariffs, were not appropriate. b. M&E Implementation: M&E implementation, as reported by the ICR, was adequate . The data on project implementation and the indicators chosen to measure attainment of the outcome targets were systematically collected and reported . The project progress reports were prepared and submitted in a timely manner . c. M&E Utilization: As already noted in the ICR, the design of M&E was not adequate, therefore, M&E could not be fully utilized to adjust project activities and inform policy decisions . M&E Quality Rating : Modest 11. Other Issues a. Safeguards: The project was classified as Category A for Environmental Assessment purposes due to the new construction segment on the Taohuatan-Tangkou road, which traversed a mountainous area of high scenic value with tourism potential. In addition to Environmental Assessment (OP 4.01), two safeguards policies were triggered: Physical Cultural Resources (OP 4.11) and Involuntary Resettlement (OP 4.12). Environment The PAD (page 19) reports that full Environmental Impact Assessments (EIA) were prepared for all the S322 and 6 road improvement sub-projects; and a simplified EIA for all the 18 road rehabilitation sub-projects. Based on these Assessments, an Integrated Environmental Assessment Report in English was prepared for the entire road program in accordance with guidelines and a table of contents agreed with the Bank. The Anhui Provincial Communications Department had prepared a comprehensive and detailed draft Environmental Management Plan in accordance with Bank policy. The ICR (page 8) reports that compliance with the Environmental Management Plan was satisfactory. Potentially negative environmental impacts caused by construction activities were satisfactorily mitigated in compliance with the Environmental Management Plan . Involuntary Resettlement The PAD (page18) reports that a socioeconomic survey was conducted in the project areas during preparation. The survey showed that the project would require house relocation totaling 31,194m2, affecting 680 people in 175 households. About 20 small-business shops would also be affected. In line with Chinese laws and regulations and with the Bank’s social safeguard policies (OP 4.12), detailed Livelihood Rehabilitation and Resettlement Action Plans were developed. The Anhui Provincial Communications Department was in charge of the implementation of the resettlement program. An independent monitor was appointed for the program implementation. The total actual land acquisition and resettlement cost was US$26.87 million, more than double the appraisal estimate of US$12.75 million. The ICR (page 8) reports that resettlement implementation was satisfactory. Land acquisition and resettlement works began in September 2008 and comprised 2510.09 mu (1.67 million square meters equivalent) of land acquisition and 31,194 m2 of house demolition. The project team provided additional information on the number of people affected by the resettlement, which were 2,550 households with 9,496 persons. The team confirmed that all 412 households affected by house demolition built new houses with adequate facilities. The external monitoring agency concluded that land acquisition and resettlement had been completed by resettlement offices at all levels in line with the Resettlement Action Plan (ICR, page 32). Based on the results of a resettlement follow-up study on 264 affected households, living standards and production conditions were improved as a result of the implementation of the Resettlement Plan. b. Fiduciary Compliance: As reported by the ICR (pages 8-9), procurement was satisfactory, in compliance with the Bank policies and completed as scheduled, despite the large number of contracts under the project. The ICR also reports that fiduciary compliance was satisfactory. Project financial management met acceptable standards during the implementation. Comments of independent auditors on annual financial statements were satisfactorily addressed . c. Unintended Impacts (positive or negative): d. Other: 12. Ratings : 12. ICR IEG Review Reason for Disagreement /Comments Outcome : Moderately Satisfactory The project objectives were Satisfactory substantially achieved at project completion; the project design and objectives are substantially relevant and the project efficiency is substantial . The overall outcome is therefore rated as Satisfactory. Risk to Development Moderate Moderate Outcome : Bank Performance : Moderately Moderately Satisfactory Satisfactory Borrower Performance : Satisfactory Satisfactory Quality of ICR : Satisfactory NOTES: NOTES - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate. 13. Lessons: The ICR draws two lessons from the experience of preparing and implementing this project : A careful choice of M&E indicators, well tailored to the cause and effect relationships between the project interventions and their intended outcomes, is necessary in order to monitor not just progress in physical implementation, but also to gauge how much of the achievement of the project development objectives could be attributed to the project. The pilot performance based road maintenance contract could have yielded more useful lessons if the duration of the contracts had been longer and if they had been larger covering more road sections . 14. Assessment Recommended? Yes No 15. Comments on Quality of ICR: The ICR reported both outputs and outcomes of the project. Its assessment of the achievement of the project objectives was evidence based. It was candid about those areas where there was room for improvement, including the M&E design. However, the information was not presented in a very organized way. For example, the Discussion of the achievement of the project development objectives was a less than systematic mixture of outcomes and outputs. The project cost information listed in Annex 1 (a), Project Cost by Component, was not organized by component, making it difficult to determine the actual cost of each project activity at completion. The Lessons Learned are sparse and of limited general applicability . a.Quality of ICR Rating : Satisfactory