Document of The World Bank Report No: ICR00002553 IMPLEMENTATION COMPLETION AND RESULTS REPORT (IDA-40730) ON A CREDIT IN THE AMOUNT OF SDR 21.0 MILLION (US$ 31.73 MILLION EQUIVALENT) TO THE SOCIALIST REPUBLIC OF VIETNAM FOR A ROAD SAFETY PROJECT JUNE 15, 2013 Vietnam Sustainable Development Unit Sustainable Development Department East Asia and Pacific Region CURRENCY EQUIVALENTS (Exchange Rate Effective February 28, 2013) Currency Unit = Vietnamese Dong (VND) VND 1.00 = US$ 0.000048 US$ 1.00 = VND 20,806 FISCAL YEAR January 1 – December 31 ADB Asian Development Bank CAS Country Assistance Strategy CPGRS Comprehensive Poverty Reduction and Growth Strategy DCA Development Credit Agreement DP Displaced person DRVN Directorate for Roads of Vietnam EMDP Ethnic Minority Development Plan EMP Environmental Management Plan ERR Economic Rate of Return GOV Government of Vietnam ICB International Competitive Bidding IDA International Development Association IFRs Interim Financial Reports JBIC Japan Bank for International Cooperation JICA Japan International Cooperation Agency M&E Monitoring and Evaluation MOET Ministry of Education and Training MOH Ministry of Health MOPS Ministry of Public Security MOT Ministry of Transport MTR Mid-Term Review NCB National Competitive Bidding NPTS National Program for Traffic Safety NRADS National Road Accident Database System NTSC National Traffic Safety Committee PIP Project Implementation Plan PMU Project Management Unit QCBS Quality Cost Based Selection RAP Resettlement Action Plan TSPMU Traffic Safety Project Management Unit TSSR Transport Safety Strategy Review UNDP United Nations Development Program VRA Vietnam Road Administration VRSP Vietnam Road Safety Project Regional Vice President: Axel Van Trotsenburg, EAPVP Country Director: Victoria Kwakwa, EACVF Sector Director: John Roome, EASSD Sector Manager: Jennifer Sara, EASVS Task Team Leader: Van Anh Thi Tran, EASIN ICR Team Leader: Reda Hamedoun, EASIN SOCIALIST REPUBLIC OF VIETNAM Vietnam Road Safety Project CONTENTS Data Sheet A. Basic Information……………………………………………………………………..…. . i B. Key Dates…………………………………………………………………………...……...i C. Ratings Summary…………………………………………………………………………..i D. Sector and Theme Codes…………………………………………………………………..ii E. Bank Staff………………………………………………………………………………….ii F. Results Framework Analysis……………………………………………………………….ii G. Ratings of Project Performance in ISRs…………………………………………………..vi H. Restructuring………………………………...……………………………………………vi I. Disbursement Graph……………………………………………………………..……….vii 1. Project Context, Development Objectives and Design ............................................... 1 2. Key Factors Affecting Implementation and Outcomes .............................................. 4 3. Assessment of Outcomes .......................................................................................... 11 4. Assessment of Risk to Development Outcome......................................................... 15 5. Assessment of Bank and Borrower Performance ..................................................... 16 6. Lessons Learned ....................................................................................................... 18 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners .......... 20 Annex 1. Project Costs and Financing .......................................................................... 21 Annex 2. Outputs by Component ................................................................................. 22 Annex 3. Economic and Financial Analysis ................................................................. 23 Annex 4. Bank Lending and Implementation Support/Supervision Processes ............ 29 Annex 5. Beneficiary Survey Results ........................................................................... 31 Annex 6. Stakeholder Workshop Report and Results................................................... 32 Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR ..................... 33 Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders ....................... 37 Annex 9. List of Supporting Documents ...................................................................... 38 MAP…………………………………………………………………………………...39 A. Basic Information Country: Vietnam Project Name: Road Safety Project Project ID: P085080 L/C/TF Number(s): IDA-40730 ICR Date: 10/08/2012 ICR Type: Core ICR MINISTRY OF Lending Instrument: SIL Borrower: TRANSPORT Original Total SDR 21.00M Disbursed Amount: SDR 15.82M Commitment: Revised Amount: SDR 16.50M Environmental Category: B Implementing Agencies: National Traffic Safety Committee Cofinanciers and Other External Partners: B. Key Dates Revised / Actual Process Date Process Original Date Date(s) Concept Review: 12/17/2003 Effectiveness: 03/09/2006 03/09/2006 12/16/2009 and 06/20/2011 Appraisal: 12/13/2004 Restructuring(s): NA (closing date extensions) Approval: 06/07/2005 Mid-term Review: 02/23/2009 03/06/2009 Closing: 12/31/2009 12/31/2012 C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Moderately Satisfactory Risk to Development Outcome: Moderate Bank Performance: Moderately Satisfactory Borrower Performance: Moderately Satisfactory C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings Quality at Entry: Moderately Satisfactory Government: Moderately satisfactory Implementing Quality of Supervision: Moderately Satisfactory Moderately satisfactory Agency/Agencies: Overall Bank Overall Borrower Moderately Satisfactory Moderately satisfactory Performance: Performance: i C.3 Quality at Entry and Implementation Performance Indicators Implementation QAG Assessments Indicators Rating Performance (if any) Potential Problem Project Quality at Entry No None at any time (Yes/No): (QEA): Problem Project at any Quality of Yes None time (Yes/No): Supervision (QSA): DO rating before Moderately Closing/Inactive status: Satisfactory D. Sector and Theme Codes Original Actual Sector Code (as % of total Bank financing) Central government administration 57 60 General education sector 4 5 Health 5 5 Other social services 12 9 Rural and Inter-Urban Roads and Highways 22 21 Theme Code (as % of total Bank financing) Injuries and non-communicable diseases 67 64 Other social development 33 36 E. Bank Staff Positions At ICR At Approval Vice President: Axel von Trotsenburg Jemal-ud-din Kassum Country Director: Victoria Kwakwa Klaus Rohland Sector Manager: Jennifer Sara Jitendra N. Bajpai Project Team Leader: Van Anh Thi Tran Jerry A. Lebo ICR Team Leader: Reda Hamedoun F. Results Framework Analysis Project Development Objectives (from Project Appraisal Document) The project development objective is to reduce the rate of accidents, injury and death associated with road transport through physical improvement and institutional development to strengthen the management of road transport safety. ii (a) PDO Indicator(s) Original Target Formally Actual Value Values (from Revised Achieved at Indicator Baseline Value approval Target Completion or documents) Values Target Years Indicator 1 : Accident rates per 100 million vehicle-km in project corridors. Value quantitative or 39 29 25 Qualitative) Date achieved 12/31/2004 09/30/2012 03/31/2012 Comments Achieved. The original target was a 26% reduction from the baseline, and (incl. % the actual target achieved a 36% reduction. The outcome was then achievement) achieved at 138%1. Indicator 2 : Injury rate per 100 million vehicle-km in project corridors. Value quantitative or 130 62 107.1 Qualitative) Date achieved 12/31/2004 12/31/2012 09/30/2012 Comments Partially achieved. The original target was a 52% reduction from the (incl. % baseline, and the actual target achieved a18% reduction. The outcome was achievement) then partially achieved at 34%. 2 Indicator 3 : Fatality rate per 100 million vehicle-km in project corridors. Value quantitative or 13 6 5.1 Qualitative) Date achieved 12/31/2004 12/31/2012 09/30/2012 Comments Achieved. The original target was a 54% reduction from the baseline, and (incl. % the actual target achieved a 61% reduction. The outcome was then achievement) achieved at 113%.3 1 Original baseline for accidents was estimated based on the nationwide ratio of reported accidents by the Police to reported fatalities. Using a different methodology, the M&E consultant set a baseline of 8.5 per 100m-vkt, based on data collected in 2009, and an end value of 5.4. For the purpose of this ICR, the M&E end value was converted into the actual end value used above to take into account the methodology used for the PAD and end up with numbers comparable with the baseline. 2 Original baseline for injuries were estimated at ten times the number of fatalities based on experience in developed countries and included all types of injuries, even slight injuries. Using a different methodology (including serious injuries only), the actual data was 2.8 severe injuries per 100m-vkt in 2009, and collected an end value of 2.5. For the purpose of this ICR, this end value was converted into the actual end value used above to take into account the methodology used for the PAD. iii (b) Intermediate Outcome Indicator(s) Original Target Formally Actual Value Values (from Revised Achieved at Indicator Baseline Value approval Target Completion or documents) Values Target Years Performance-based national road safety strategy (phase 2), including Indicator 1 : financing plan, developed and approved by Government. Final strategy Value document Approved by the quantitative or N/A approved by Government Qualitative) Government Date achieved 12/31/2004 09/30/2012 03/31/2012 Comments (incl. % This document is more of an overarching policy paper on road safety. achievement) Indicator 2 : Traffic exceeding posted speed limits (%), reduced Value No target quantitative or 30 established in the 18 Qualitative) PAD Date achieved 12/31/2004 12/31/2012 09/30/2012 Comments 40% reduction from the baseline. The original baseline was (incl. % underestimated during project preparation. A new baseline of 64% was achievement) estimated by the B8.1 consultant in April 2011. Indicator 3 : Motor cycle riders wearing helmets (%), improved Value No target quantitative or 25 established in the 93 Qualitative) PAD Date achieved 01/01/2004 12/31/2012 09/30/2012 272% increase from the baseline. The Helmet Law mandated in 2007 Comments constituted a major step in addressing safety and awareness issues and (incl. % reflected the growing importance of road safety issues in the national achievement) dialogue. Indicator 4 : % of motorcycle drivers licensed, improved Value No target quantitative or 73 established in the 93 Qualitative) PAD 3 Using a different methodology than the one used during preparation, the actual data was 8.8 fatalities per 100m-vkt in 2009, and collected an end value of 5.1. For the purpose of this ICR, the end value was converted into the actual end value used above to take into account the methodology used for the PAD. iv Date achieved 12/31/2008 06/30/2012 09/30/2012 Comments 27% increase from the revised baseline. The original baseline value (95%) (incl. % was overestimated and revised in December 2008. achievement) Indicator 5 : % of commercial vehicle operator licensed, improved Value No target quantitative or 95 established in the 97 Qualitative) PAD Date achieved 12/31/2004 06/30/2012 09/30/2012 Comments (incl. % 2% increase from the baseline. achievement) % of vehicles in demonstration corridors passing first time inspection, Indicator 6 : improved Value No target quantitative or 90 established in the 75.5 Qualitative) PAD Date achieved 11/31/2004 06/30/2012 03/31/2012 Comments 16% decrease from the baseline. The change is likely due to the (incl. % enforcement of more stringent vehicle testing standards achievement) Indicator 7 : Percentage of NRADS completed Value quantitative or 0 100 95 Qualitative) Date achieved 12/31/2004 06/30/2012 09/30/2012 Comments Achieved. The equipment has been installed, and testing/training has been (incl. % completed. achievement) Number of biannual road safety reports prepared in accordance with Indicator 8 : project monitoring and evaluation guidelines Value quantitative or 0 7 7 Qualitative) Date achieved 12/31/2004 06/30/2012 09/30/2012 Comments (incl. % Achieved. achievement) Number of annual reports on effectiveness of treatments and Indicator 9 : interventions prepared in accordance with project monitoring and evaluation guidelines Value quantitative or 0 4 3 Qualitative) v Date achieved 11/31/2004 06/30/2012 03/31/2012 Comments (incl. % Partially achieved (75%). achievement) G. Ratings of Project Performance in ISRs Actual Date ISR No. DO IP Disbursements Archived (USD millions) 1 01/05/2006 Satisfactory Satisfactory 0.00 2 02/26/2007 Satisfactory Moderately Satisfactory 1.56 Moderately 3 05/17/2008 Moderately Satisfactory 2.42 Unsatisfactory Moderately 4 06/16/2008 Moderately Satisfactory 2.42 Unsatisfactory Moderately 5 01/06/2009 Moderately Satisfactory 3.00 Unsatisfactory Moderately 6 09/15/2009 Moderately Satisfactory 3.37 Unsatisfactory 7 01/05/2010 Moderately Satisfactory Moderately Satisfactory 4.99 8 02/16/2011 Moderately Satisfactory Moderately Satisfactory 10.56 9 12/24/2011 Satisfactory Moderately Satisfactory 15.50 10 12/11/2012 Moderately Satisfactory Moderately Satisfactory 20.86 H. Restructuring (if any)  Extension to the Development Credit Agreement Closing Date (December 16, 2009) for 18 months  Second extension to the Development Credit Agreement Closing Date (June 29, 2011) for 18 months vi I. Disbursement Profile vii 1. Project Context, Development Objectives and Design 1.1 Context at Appraisal Country and Sector background at appraisal At the time of project preparation, rapid economic growth was accompanied by rapid traffic growth. From 1990 to 2004, the number of automobiles had increased 2.5 times and the number of motorcycles had increased more than 8 times. Over 90 percent of the 11-12 million motorized vehicles in Vietnam in 2004 were motorcycles. Yet in comparison to those of other Asian countries, motorization levels in Vietnam were still low, indicating that vehicle ownership would continue to increase with economic growth. Road fatalities had increased 39 percent in 2000/2001 and 22 percent in 2001/2002. Vietnam had a reported fatality rate of about 12 per 10,000 vehicles and an accident rate of about 20 per 10,000 vehicles in 2004. These rates were below those of China (about 25-30 fatalities per 10,000 vehicles), but higher than those of Thailand and Malaysia or found in OECD countries (in the range of 1-2 fatalities per 10,000 vehicles). In Vietnam, as in many other countries, under-reporting of accidents and fatalities was significant. The Vietnam Multi-Center Injury Survey of 27,000 households estimated that there were 900,000 road traffic accidents involving injury in 2001, in comparison to reported accidents of 25,000. Similarly, road-related fatalities were estimated to be underreported by about 30 percent. Government Strategy The Government of Vietnam (GOV), at the highest levels and across sectors, was considering improving road safety a high priority. Vietnam’s 2002-2010 National Policy on Accident and Injury Prevention noted that injuries related to road accidents were not just a transport sector issue, but a leading factor in a wider national injury crisis. The Government strategy was spelled out in the National Program for Traffic Safety (NPTS), which covered the 2001-2005 periods. The NPTS called for policy and physical interventions funded by government budgets, driver training and licensing fees, insurance, as well as official development assistance. Its activities included (i) education and publicity campaigns, (ii) road safety audits, (iii) identification and treatment of blackspots, (iv) development of a traffic accident reporting and analysis database, (v) improvement in the driver training and licensing system, (vi) creation of first-aid stations, and (vii) modernization of the motor vehicle registration and inspection system. The National Traffic Safety Committee (NTSC) was already responsible for coordinating the delivery of NPTS activities and reporting on progress. It had a secretariat (Executive Office) in MOT. Eleven ministries were represented on the NTSC at the deputy minister or vice minister level. National government funding for NTSC and road safety activities carried out by line ministries came from traffic fine receipts. The majority (60 percent) of these funds was going directly to the Road and Rail Traffic Police Administration (RRTPA) under the Ministry of Public Security, while other road safety activities and NTSC received 35 percent. The remainder was used for general budget support. Local funding for traffic safety activities also came from fines, with the bulk allocated to traffic police, traffic inspection, and local police departments (52 percent). The remainder was used to fund vehicle inspection centers, local traffic safety departments, 1 and local government road safety activities. The overall level of funding for road safety in 2003 was about US$45 million, split equally between local and national expenditure. Rationale for Bank Assistance The World Bank partnered with the World Health Organization in 2004 to produce the World Report on Road Traffic Injury Prevention, demonstrating both organizations’ recognition of road safety as a major world health issue. The Bank, due to this experience and commitment, and its substantial engagement in the road sector in Vietnam since 1994, was well positioned to work with GOV on road safety. The design of the project also took into consideration the findings of the Transport Safety Strategy Review (TSSR) and was consistent in size and scope with objectives described in the 2004 CAS Progress Report for Vietnam (2004). 1.2 Original Project Development Objectives (PDO) and Key Indicators (as approved) The project development objective was to reduce the rate of accidents, injury and death associated with road transport through physical improvement and institutional development to strengthen the management of road transport safety. The key performance indicators were:  Rates of accident, injury, and death on project road corridors reduced;  Development and government approval of a national road safety strategy specifying achievable performance targets and a financing plan;  Driver behavior and compliance with vehicle and licensing standards on project road corridors improved; and  Monitoring and evaluation framework implemented in project corridors and used to prioritize road safety interventions, and national road accident database system (NRADS) established and updated regularly. 1.3 Revised PDO (as approved by original approving authority) and Key Indicators, and reasons/justification The PDO was not revised. 1.4 Main Beneficiaries The main beneficiaries of VRSP include road users, roadside communities, and businesses in the three demonstration corridors. The main direct benefits of the project come from the reductions in social costs resulting from the reduced numbers and severity of fatalities and injuries caused by road accidents. These benefits include (i) the reduction in the incidence and severity of road accident injury and death and the associated cost to individuals and the Vietnamese economy, (ii) the reduction in the public resources wasted by road accidents, including the temporary or permanent loss and diversion of human resources, the consumption of medical resources, and the damage or destruction of property, and (iii) the improved efficiency of road transport due to the reduced economic inefficiencies resulting from road accident trauma. Beneficiaries also included government agencies engaged in coordinating road safety, as well as the institutions (NTSC, MOT, MOH, MOET, MOPS) that benefited from the implementation support activities and trainings. The benefits include the cost savings 2 from greater effectiveness of road safety interventions, which stemmed from the increased institutional capacity to monitor, evaluate, and manage road safety and collect data at both the national and road network levels. Benefits extended to populations in the whole country through the nationwide awareness campaigns, vehicle inspections, driving licensing and enforcement. 1.5 Original Components (as approved) This US$31.7 million project financed by IDA consisted of three components to be implemented over the 2005-2009 period: (a) The Institutional and Capacity Building Program, (b) The Road Safety Demonstration and Awareness Program, and (c) Road Safety Monitoring and Evaluation Program. Component A - Institutional and capacity building program (US$7.84 million) to provide technical assistance and project implementation support to strengthen the management and technical capacity of the NTSC Executive Office, the Traffic Safety Project Management Unit (TS PMU), the sub-PMUs, and corridor supervision teams. This component was also to support the preparation of a performance based national road safety strategy, including a financing plan. Component B. Road safety demonstration and awareness program (US$20.89 million) to develop and implement comprehensive, integrated safety programs along three high- risk national road corridors. The three demonstration corridors identified for inclusion under the project were: (i)National Highway 1 between Hanoi and Vinh (281 km); ( ii) National Highway 1 between Ho Chi Minh City and Can Tho City (1 5 1 km); and (3) Highway 51 from Dong Nai province to Vung Tau (80 km). This component had 12 subcomponents. The component also supported: (i) the Vietnam Register in upgrading its national vehicle inspection system, computer and communications equipment, software development, and vehicle inspection equipment, including specialized vehicle inspection equipment at three primary national vehicle inspection stations (Hanoi, Ho Chi Minh City, and Vinh Long).; (ii) Vietnam Road Administration VRA with equipment to improve the driver training centers at Chi Linh and in Dong Nai province and related facilities;.(iii) consulting services for VRA to conduct road safety audits and blackspot or black route identification and treatment; (iv) the Traffic Police through the provision of vehicles and equipment to implement a general-deterrence road safety enforcement program in the three demonstration corridors; (v) a program of road user education (MOET) and public awareness (NTSC) in the three demonstration corridors, using targeted social marketing techniques; (vi) Ministry of Education and Training with the provision of school road safety education materials, equipment, and associated training for a program of primary, secondary, and tertiary road safety education in the three demonstration corridors; (vii) Ministry of Health by providing materials, equipment, and training for an emergency medical services program in the three demonstration corridors; (viii) and operating costs and local consulting services. Component C. Road Safety Monitoring and Evaluation Program (US$6.20 million), to support the development of the National Road Accident Database System (NRADS) and a comprehensive program of project corridor monitoring and evaluation. 3 1.6 Revised Components  B1: Some blackspots on Highway 51 and NH1 were removed under other projects (Highway 51 upgrading and NH1 widening).  B5: Purchase of driving license printing system in lieu of equipment in the Dong Nai driver training center  B7: Purchase of a camera system in lieu of cars and equipment from the police  B4.3: Cancellation of subcomponent B4.3after agreement with GOV that equipment for vehicle inspection centers would be provided through GOV funding  B8,2: Reduction in the scope of work in education and awareness raising program under B8.2. Instead of broad based awareness programs, these programs were redesigned to address specific target groups with specific themes.  Reduction in overseas training under A4.2 reducing the training cost from US$552k to US$270k. 1.7 Other significant changes Extension of Closing Dates: The closing date was extended twice: (i) from December 31, 2009 to June 30, 2011 to address the priority time-bound action plan to accelerate implementation that was set during the midterm review mission; and (ii) for 18 months from June 30, 2011 to December 31, 2012 to allow for the conclusion of several on-going training activities and completion of substantial packages critical to the full achievement of the project development objectives. Credit Cancellation. Approximately US$6.9 million equivalent would remain unspent from the credit, due to (i) overlap with other projects and the reduction of number of blackspots treated under the project, (ii) contingencies amounting to approximately US$1.3 million would not be required, (iii) close management of consultants generating saving of up to US$ 1 million and (iv) exchange rate gains between SDR and USD by about US$1.2 million. 2. Key Factors Affecting Implementation and Outcomes 2.1 Project Preparation, Design and Quality at Entry Project preparation and design were heavily based on a Transport Safety Strategy Review (TSSR) that the Bank conducted in 2003. This TSSR assessed government programs and approaches to road safety and set out short to long term strategies to address deteriorating road safety in Vietnam. In June 2003, the Bank and GOV discussed the findings of the TSSR and the main issues in the sector and agreed that a road safety initiative intervention should target the following issues:  Capacity and accountability. Institutionally, the most pressing need in road safety was to strengthen the capacity, resources, and skills of the agencies responsible for delivering the government’s program, including NTSC. The legal framework at the time focused on NTSC’s role in coordination, leaving the responsibility for implementation with central government agencies, other relevant organizations, and provincial and municipal People’s Committees. The team concluded that this limited accountability for overall outcomes and makes coordination difficult. Component A of VRSP, with its focus on management and operating costs support to NTSC, technical 4 assistance to sub-PMU teams, training and the funding of a performance-based road safety national strategy, aimed at correcting this.  Knowledge and awareness of road safety. International experience suggested that projects emphasizing traffic management functions (driver licensing, vehicle inspection/registration…), only had marginal impact on accident and fatality rates in the short term. To rapidly curb accident and fatality rates, international experience suggested that targeted, cost-effective measures such as speed-limit and drunk-driving enforcement, separation of traffic, and motorcycle helmet law enforcement, were more effective. This had to be accompanied by measures to raise public awareness of the importance compliance with road safety laws, in a country where drunk-driving posed a bigger traffic safety problem than was acknowledged, and where laws requiring motorcycle helmets were seen by the public as ineffective. Component B, with its focus on road user awareness, road safety education in schools and vehicle registration and driver testing, targeted precisely this issue. The project design also took into consideration the conclusions of the World Report on Road Traffic Injury prevention. The report recommended to (i) make road safety a political priority, (ii) appoint a lead agency for road safety, give it adequate resources, and make it publicly accountable, (iii) develop a multidisciplinary approach to road safety, (iv) set appropriate road safety targets and establish a national road safety plan to achieve them, (v) create budgets for road safety and increase investment in demonstrably effective road safety activities. In line with these guidelines, VRSP was conceived as a stand-alone road safety project to reflects the high level of political support for this issue in the country, supported accountability and coordination through NTSC, involved MOT and the ministries of Public Security, Education and Training, and Health for implementation, and focused on demonstration corridors to show the effectiveness of road safety initiatives. Despite this careful preparation process, the program was over capacity of the involved agencies in the country which ultimately led to a difficult implementation of the project and a delay of 36 months in its closing. In particular, the project design could have been improved in the following two areas:  An overly complex design: Simplicity of design is a key factor of success in environments where the multiplicity of stakeholders (from NTSC to line ministries, from the Traffic Safety Project Management Unit (TSPMU) to sub-PMUs, from contractors to provincial and central authorities), suboptimal administrative processes, weak institutional capacity, discrepancies in appraisal procedures between ministries, and a wide geographical project area scope are bound to make implementation complex. Contrary to this view, VRSP included 18 sub-components for a $US 31 million project, triggered the procurement of more than 60 packages (ranging from black spot treatment detailed designs and mobile inspection vehicles to the purchase of helmets and a road user awareness campaign), and involved 63 cities and provinces all over Vietnam. This design made it extremely challenging to retain the project focus, to coordinate the preparation of all the components across agencies, and ultimately to allow for a timely and effective implementation of the project.  The lack of clear and effective institutional arrangements: Under the designed implementation arrangements, NTSC, with support of its Executive Office (NTSC- EO), acted as the executive agency responsible for overall project oversight and created the TSPMU for coordinating day to day implementation (including procurement, financial management, and monitoring). On top of this, each of the 6 5 participating agencies (the Ministries of Public Security, Education and Training, and Health, the VRA and the Vietnam Register under the Ministry of Transport, and the NTSC Executive Office) were to establish and staff a sub-PMU to implement its part of the project. Sub-PMUs were responsible for technical decisions and management of their respective subcomponents. The Ministry of Finance, People’s Committees at the provincial level and provincial traffic safety departments also had supporting roles.  These suboptimal arrangements were compounded by the fact that the agencies involved did not always have the adequate authority or resources to conduct their tasks: in particular, NTSC, by mandate an overarching agency for all ministries, had neither the legal authority nor the adequate resources to carry out its leadership role and direct participating agencies, which proved a major obstacle towards the smooth implementation of the project. Moreover, initially tasking the TSPMU (under the authority of NTSC) of procurement responsibilities for packages to be implemented by sub-PMUs (under the authority of the ministries and agencies instituting them) would clearly lead to severe implementation delays. 2.2 Implementation The implementation of the project went through two very distinct phases: lagging and ineffective from loan effectiveness to early 2009, timely and satisfactory from early 2009 to the actual closing date. But the complexity of the project implementation arrangements and design, combined with significant delays in the procurement of activities critical to the execution of the project, resulted in extremely slow disbursements. As a result, VRSP was flagged as a problem project in July 2008, and the closing date was extended twice: from December 31, 2009 to June 30, 2011, and then to December 2012. From March 2006 to March 2009, the overall progress of the project remained extremely low. By June 2008, the majority of activities were either at preparation (80%) or under bidding (12%), with only 8% under actual implementation. By November 2008, or one year before the original closing date of December 2009, the disbursement rate was 8.7%. At the time of the mid-term review in February 2009 (itself postponed several times since early 2008), 13% of the activities were under implementation and 3% had been completed. Three main issues could be identified as being behind the execution setbacks:  Initial procurement delays: The delay in the procurement of the A1 Consultancy proved to be a major setback for project implementation. This subcomponent was designed to finance an integrated package of technical assistance to support capacity building and implementation support across all project components and implementing agencies. The procurement of this A1 Consultancy started in 2007 bus as a result of complaints received regarding procurement handling and the selection process, the package was re-bid in the third quarter of 2008 after experiencing a long delay. The consultant was finally mobilized in November 2009, but the delay had a significant impact on all the components dependent on its support.  Institutional issues: The weak institutional capacity and limitation of resources of NTSC hindered the smooth implementation of VRSP. The NTSC, by mandate, was the key road safety planning and coordination body of various ministries to facilitate road safety initiatives. However, it did not have the authority, the staff 6 nor the expertise to exercise its oversight of the project as defined through the institutional arrangements of the projects. In effect, the NTSC (i) only had advisory powers and was not positioned at the higher level of decision-making, (ii) its resources were markedly insufficient in terms of manpower, access to data, and for conducting studies and analyses, and (iii) its staff did not include a large amount of traffic safety specialists.  Lack of proactive supervision: During the first two years of project implementation, and despite the fact that the disbursement rate was below 8%, official supervision remained extremely limited. Until the project was flagged as a problem status in July 2008, only three official supervision missions were conducted after the Credit signing of the Development Credit Agreement (DCA): in December 2005, in October 2006, and in July 2007. Similarly, only three Implementation Status Reports (ISRs) were completed and filed from 2006 to 2008. This, along with a late mid-term review, led VRSP to be delayed due to recurrent procurement issues and, ultimately, to a delay of three years in project closing. From March 2009 up until the closing of the project in December 2012, the implementation of the project progressed smoothly and efficiently. The diagnosis and recommendations of the mid-term review, as well as the rigorous and proactive follow up enabled by a tightened supervision process were key elements behind the turnaround of the project. The three main factors behind the acceleration of the execution of the project can be identified as the following:  A vigorous supervision of the project, and agreement on a time bound action plan: by July 2009, the 19 targets set by the midterm review to upgrade the project to normal status had been met (with the exception of the award of the A1 consulting services contract). That contract was signed on September 2009. Following this approach, a further series of 9 targets were agreed to be completed by October 2009 and were achieved. All in all, 6 official supervision missions were conducted from March 2009 to the closing of the project. In addition, close daily supervision allowed to provide timely support to the client, and the task team adopted a proactive approach The task team always seeked support from Bank management and asked government leadership to provide timely decisions/responses.  The restructuring of the project, as a significant streamlining effort regarding the scope of the project subcomponents took place after the midterm review, allowing the project to regain its focus. Subcomponent B6 (road safety audits/safety inspections) was added to the contract for the consultant engaged under C2 (demonstration corridors M&E) and there was a reduction in the scope of work in education and awareness raising program under B8.2 to address specific target groups with specific themes. Overseas training under A4.2 was also reduced in order to concentrate efforts on components with the most impact. The Bank and GOV also agreed to cancel the Vehicle Inspection Equipment (B4.3).  Institutional changes: In order to strengthen the capacity of NTSC, GOV passed the Decree No.32 to consolidate its structures and decision-making procedures with the Deputy Prime Minister instated as the Chair of NTSC. Also, a change of leadership was implemented at TSPMU, which had noticeable effects in terms of project 7 ownership, clearer direction and faster delivery of procurement documents. Staffing resources were also strengthened in both NTSC and TSPMU 2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization The M&E framework was designed based on 4Es (Engineering, Education, Enforcement, Emergency) approach and the Road Safety Management Pyramid, which links interventions to intermediate outcomes to primary outcomes in road safety. The primary outcomes of road safety programs are usually the reduction of number of accidents, injuries and fatalities that occur, outcomes which were applied to VRSP. Accidents were classified by severity levels of fatal, severe injury and slight injury. Intermediate outcomes (such as decrease in traffic exceeding posted speed limits, better maintained vehicles, increase in motor cycle riders wearing helmets or increase of safety belt usage) were used to measure compliance with traffic law or standards that are directly related to the primary outcomes and thus the accident scenarios. M&E implementation focused on the collection of data for the demonstration corridors, on the development of a methodology for monitoring road safety improvements and interventions, and on the training local staff in methods and procedures. VRSP collected its own data on the three demonstration corridors on traffic volume, speed, helmet and seatbelt wearing, alcohol usage and vehicle/driver documentation through surveys carried out by the M&E consultant: in total, starting mid-2008, 9 vehicle speed surveys were carried out, as well as 18 helmet and seatbelt usage surveys, 8 alcohol and vehicle/driver licensing surveys in cooperation with the Traffic Police, and traffic volume 8-hour counts each December in cooperation with VRA. Other data were collected from stakeholders, including registration data, MOH injury records, driver´s licensing, vehicle inspection statistics and official nationwide accident statistics from the Traffic Police. For instance, the accident data from the corridors was hand collected by the consultant every quarter from the responsible police stations along the three corridors. Six training courses were also conducted on monitoring, evaluating and reporting of road safety projects and interventions, as well as reporting of primary and intermediate outcomes: a road safety audit course for TSPMU and VRA staff in May 2008, a blackspot identification, investigation and treatment Course in September 2008, an information management course on Project Management for TSPMU and the sub-PMUs in December 2008, two courses for the police on May 2012 to raise awareness of road safety, enforcement and data collection, and a general M&E course for all stakeholders in November 2012. Due to a lack of reliable data at appraisal the M&E framework presented in the PAD omitted critical targets and some baseline values proved to be inaccurate. Issues with the quality of road safety data were recognized in the PAD. To address the need for high quality data the project included the design, development and commissioning of a National Road Accident Database System (NRADS), the intermediate indicator for this activity was achieved by the end of the project. To improve the poor quality of the appraisal data the project included a contract for monitoring and evaluation along the demonstration corridors. Work under this contract confirmed the problem of significant underreporting of injuries and attempted to reconcile data from different sources. 8 However, the results of this M&E work were only available at the end of the project and more attention should have been devoted from the outset to setting up a rigorous M&E framework. Due to the low quality of the baseline figures estimated during preparation (and used in the PAD), difficulties were encountered in measuring changes from these baselines during implementation. Also, key results indicators for Component B (percentage of traffic exceeding posted speed limits, of motorcycle riders wearing helmets, of motorcycle drivers licensed, of commercial vehicle operators licensed, of vehicles in demonstration corridors passing first time inspection) were missing from the PAD and thus undermined the quality of the M&E framework. In the PAD baseline estimates for total number of accidents and injuries were estimated based on factors applied to the number of fatalities as reported by MOPS, the factors applied were derived from experience in developed countries. This approach assumed (i) that the MOPS data for number of fatalities was reliable, and (ii) that the factors applied were appropriate to Vietnam. Both of these assumptions proved to be flawed in that (i) the MOPS data underreported fatalities, and (ii) the factors used to determine the number of accidents and injuries were too low, resulting in further underreporting. To address these issues, the M&E activities developed a revised baseline based on a more rigorous analysis of available data. In particular, greater use was made of MOH data to compliment MOPS data. For the purposes of this ICR, and to be able to compare the measurements with the original baseline, the M&E data reported at the end of the project was converted to be compatible to the baseline indicators from the PAD, using the factor of injuries reported by MOH to the amount of serious injuries reported by MOPS. This approach does not address underreporting of accidents and injuries in the PAD baselines, suggesting that the actual reduction in accidents and injuries was greater than that reported in this ICR. 2.4 Safeguard and Fiduciary Compliance Procurement Procurement was moderately satisfactory, as procurement issues were the main factor behind the significant delays experienced in project implementation. The capacity building and implementation support (A1.1) consultant, who was by design instrumental for management and procurement assistance, was mobilized in October 2009, after being delayed by re-bidding. The first contract for blackspot treatment civil works (B1.1) was signed in April 2010 and the consulting services for the design of the accident database, a key element for identifying blackspot locations and strategic interventions, was signed in October 2010. Commencements of these crucial contracts were behind the originally planned schedule for a minimum of 12 months. This significant delay was mainly caused by the lack of ownership and adequate capacity of the implementing agencies in charge. The original VRSP implementation plan was based on the assumption that an international consultant would be available at the start of the project to provide implementation support (A1.1), particularly for procurement. A large portion of this assistance for management was intended for the preparation of bidding documents for all packages, work plans and for on demand assistance. As a result of the delayed arrival of the implementation support consultant, a large number of packages had to be prepared entirely by the sub-PMUs without external guidance. The signature of the contract with the consultant in October 2009 streamlined the procurement process significantly, but issues remained as the consultant did not attend evaluation meetings, restricted to evaluation members selected by the authorities. Similarly, the consultant could not advise the quality control of the independent body selected (TCQM), which was involved in every payment pre-approval 9 process of the project. TCQM approved all requests from the NTSC, regardless of the amounts involved, generating further delays. The lack of familiarity with WB procurement guidelines and discrepancies in procedures across agencies slowed down project implementation considerably. The absence of formally established and fully functional sub-project implementing agencies, inadequate procurement capacity of the project implementing agencies and inadequate familiarity of the participating agencies with Bank procurement rules, and the high-risk procurement environment were not sufficiently mitigated through the several training activities for the procurement teams. Also, appraisal procedures within GOV remained complex and lengthy, as they involved specialized ministries with their own procedures and having to coordinate their activities through Decision no. 387 (2007). Financial Management Financial management was generally satisfactory over the course of the implementation of the project. While the NTSC was the executing agency for VRSP, internal controls, quarterly financial reports, disbursement progress and follow up with audit issues were managed by TSPMU. The accounting system of TSPMU was assessed as adequate, after a new accounting software (ANA) was introduced in June 2010, even though minor mistakes in general accounting balances persisted. IFRs, expected by the Bank no later than 45 days after the end of each calendar quarter, were submitted timely and were generally of acceptable quality (despite recurrent, but later corrected, errors such as not carrying forward the opening cash balance of Government Counterpart Fund or incorrect reporting of counterpart expenditures). The internal controls as regulated in the FM manual were complied with, and bank, cash, fixed assets were satisfactorily maintained. There were regular occurrences of ineligible expenditures (honorarium fees paid to civil servants for a training course, financing of “receiving/welcoming guest activities�), but these were swiftly refunded in accordance with the Project DCA. External audit reports were submitted to the Bank on time, have unqualified audit opinion and with acceptable quality. Social Safeguards The project triggered the Involuntary Resettlement (OP/BP 4.12 safeguards), but safeguards implementation work (under A2) revealed there were no resettlement issues on the demonstration corridors. The review of potential social impact, based on detailed designs for the corridors, confirmed there was no land acquisition, resettlement nor compensation involved for any of the package or corridors. Environmental safeguards The project triggered the Environmental Assessment (OP/BP/GP 4.01) safeguards. An Environmental Management Plan (EMP) was prepared, providing a framework within VRSP to prevent or mitigate potential environmental impacts. It also provided a monitoring plan to ensure compliance with the EMP. It assessed potential environmental impacts and conducted a screening analysis of the 15 proposed blackspot improvements. The main potential impacts were found to relate to air quality, water quality and noise. It was considered unlikely that either the direct or indirect impacts of the engineering works would result in environmental conditions any worse than prevailing situation, and any environmental impact that is not prevented can be mitigated. The EMP covered the following five technical areas: materials supply (indirect impacts), soils, hydrology, air 10 quality, noise. The supervision consultants for black-treatment contracts provided adequate monitoring for the EMP implementation. No petition on the environment issues was received. The environmental safeguard policies were complied with. 2.5 Post-completion Operation/Next Phase GOV has been discussing mounting an intervention based on this pilot project. . This intervention would not only address the lessons learned in VRSP but also lay further foundation, particularly in the area of road safety and institutional management and capacity building. The overall goal would converge with VRSP’s: to reduce poverty and vulnerability in Vietnam by reducing the incidence and severity of road transport injuries and deaths. The main difference would lie in a greater focus on a select number of effective road safety improvements and on the simplification of implementation arrangements, based on lessons learned from VRSP. The objective could therefore be to reduce the rates of accidents, injury and death associated with road transport through innovative improvement interventions on high risk corridors and institutional development to strengthen the management of road safety. This GOV-led intervention would target quick and improved road safety results in provinces with some innovative Safe System approach interventions on high-risk corridors. In particular, the Government will allocate an annual budget to NTSC to sustain the road safety forum and trigger initiatives in road safety. MOH is preparing a new Master-plan for Ambulance locations; MOET is rolling out school programs to University Level; Traffic Police is proposing to rollout additional traffic camera enforcement equipment. 3. Assessment of Outcomes 3.1 Relevance of Objectives, Design and Implementation The project objectives remain relevant to the development priorities of Vietnam at the time of ICR. By addressing the immediate need to reduce the rates of road accidents, injuries, and fatalities in the country, the project assisted the FY2012-FY2016 CPS and the 2011-2015 Government’s SEDP objectives of (i) strengthening Vietnam's competitiveness in the regional and global economy, (ii) increasing the sustainability of its development, and (iii) broadening access to economic and social opportunity. Reducing road accident-related health shocks contributes to the productivity of the work force, and addressing the administrative, legal, and governance arrangements required to improve road safety and building up cross-sector capacity among transport, health and education pushes Vietnams towards a more sustainable development model. In addition, lowering the health costs and economic inefficiencies resulting from road crashes (which push people into poverty) serves economic opportunity. VRSP is also consistent with the move towards the Safe System approach to road safety recommended by the OECD. By recognizing that road systems could be designed to expect and accommodate human error, by addressing all elements of the road transport system in an integrated way, by requiring acceptance of shared overall responsibilities and accountability between system designers and road users, and by implementing innovative interventions and new partnerships, VRSP helped the country to adopt an approach that is increasingly implemented adopted in other countries. The Safe System approach (i) recognizes that prevention efforts notwithstanding, road users will remain fallible and crashes will occur, (ii) stresses that those involved in the design of the road transport system need to accept and share responsibility for the safety of the system, and those that 11 use the system need to accept responsibility for complying with the rules, (iii) aligns safety management decisions with broader transport and planning decisions that meet wider economic, human and environmental goals, and (iv) relies on innovative rather than "traditional" interventions. 3.2 Achievement of Project Development Objectives Project outcomes indicate that Project Development Objectives have been largely met:  PDO 1: Accident rates per 100 million vehicle-km: the project corridors have demonstrated reductions in accident rates from 39 to 25 accidents per 100 million vehicle-km. The original target (29) constituted a 26% reduction from the baseline, and the actual target achieved a 36% reduction. The outcome was then achieved at 138%.  PDO 2: Injury rate per 100 million vehicle-km: the project corridors have demonstrated a reduction in injury rates from 130 to 107 severe injuries per 100 million vehicle-km. The original target (62) was a 52% reduction from the baseline, and the actual target achieved an 18% reduction. The outcome was then partially achieved at 34% 4.  PDO 3: Fatality rate per 100 million vehicle-km: the project corridors have demonstrated a reduction in fatality rates from 13 to 5.1 fatalities per 100 million vehicle-km. The original target (6) was a 54% reduction from the baseline, and the actual target achieved a 61% reduction. The outcome was then achieved at 113% The PDO was achieved through the implementation of comprehensive, integrated road safety programs along selected high risk demonstration corridors. Each project output contributed to the many factors that cause road accidents along the demonstration corridors and the effective treatment of accident victims. The combined impact of the project outputs led to the project outcomes being achieved. Under Component B of the project sub-components addressed the four ‘E’s of road safety: these being Education, Engineering, Enforcement and Emergency. Education addresses the need to reduce unsafe road user behavior that causes, or increases the severity of, accidents. The project delivered 34,000 safety helmets to 17 schools in 12 provinces along demonstration corridors; provided road safety materials for use in schools; and completed a comprehensive national road safety campaign, including TV commercials and flyers. Engineering addresses the need for roads and road vehicles to be designed, constructed and maintained in such a way that reduces both the number and severity of accidents. The 4 Prior to the project the number of deaths and injuries from road accidents was significantly under reported. This was particularly so for injuries as victims would be quickly removed from an accident scene for treatment and the cause of the injuries were not recorded at the hospital. Hence during the project it became clear that the baseline for injuries was too low but there was no data with which to correct it. Given the substantial under reporting before the project the reduction in numbers of injuries was greater than the measured values suggest and it is likely that a 50% reduction in numbers of injuries was achieved. 12 project removed 40 blackspots on the three demonstration corridors5; trained and certified 150 road engineers as Road Safety Auditors; upgraded vehicle inspection computer systems are operational in 92 inspection centers; delivered two ‘state-of-the-art’ mobile inspection vehicles; and designed a National Vehicle Testing Center. Enforcement addresses the need to establish acceptable road users behavior through the enforcement of laws and regulations related to roads. The project designed, installed and commissioned a traffic enforcement camera system on two demonstration corridors; upgraded equipment for driver licensing system in all 63 Provincial Departments of Transport nation-wide; upgraded driver training equipment and testing facilities at the Chi Linh Driving Training Center; and designed, installed and commissioned new National Road Accident Database System (NRADS). Emergency addresses the need for ensuring that road accident victims are offered timely, appropriate and competent care after an accident to reduce the severity of injuries and number of deaths. The project procured and delivered three ambulances and emergency equipment to 46 hospitals/clinics along demonstration corridors; and completed first-aid responder training for traffic policemen and clinic medical staff along the demonstration corridors. The following key intermediate outcomes were achieved and contributed to the PDO achievement: Performance-based national road safety strategy (phase 2), including financing plan, developed and approved by Government: the strategy was prepared and approved by Government. Traffic exceeding posted speed limits (%), reduced: the number of motorcycles and four-wheel vehicles exceeding posted speed limits has decreased by 40%. Motor cycle riders wearing helmets (%), improved: 94 percent of males, 92 percent of females, and 63 percent of children now wear motorcycle helmets (though the current helmet law does not require children to wear helmets, this represents a significant increase from 34 percent in February 2009). Valid licenses have been issued to more than 93 percent of both commercial vehicle operators and motorcyclists: 93% of drivers now hold valid licenses. % of commercial vehicle operator licensed, improved: 97% of operators are licensed. Percentage of NRADS completed: 95% has been completed, remaining works, mainly maintenance liability, was completed after project closure. Number of biannual road safety reports prepared in accordance with project monitoring and evaluation guidelines: all seven reports were prepared and submitted. Two intermediate indictors were partially achieved or not met: Number of annual reports on effectiveness of treatments and interventions prepared in accordance with project monitoring and evaluation guidelines: was partially achieved as three out of four reports were prepared and submitted. % of vehicles in demonstration corridors passing first time inspection, improved: was not met as the number of failures increased. It is believed that this reduction was as a result of testing procedures being improved such that testing standards were being more strictly applied. By strengthening institutional organization and capacity under Component A of the project has led to significant and sustainable institutional outcomes: the Chairman of 5 Of the three demonstration corridors one, National Highway 51, had to be replaced by the HCMC – Ninh Thuan section of National Highway 1. This change was necessary as a major highway improvement project was announced for National Highway 51. 13 NTSC is now a Deputy Prime Minister; the project assisted in the development of a National Road Safety Implementation Program (RSIP) to support the Vietnam Road Safety Strategy; deployed an information system connecting all provincial traffic safety committees to the NTSC; and procured and installed IT equipment for NTSC Information System have been installed in all 63 PTSCs., The relevant agencies are now replicating project activities in other areas beyond the project corridors. For example, a national information system for vehicle inspection management is in full operation, training for accredited road safety audit is continuing, training for traffic police on first aid is being planned for expressway, new system of driving licensing is now in full operation, enforcement camera is planned for other national highways etc. and a new road safety project is proposed by NTSC and MOT. 3.3 Efficiency The updated Net Present Value (NPV) and Economic Internal Rate of Returns (EIRR) of the Project, using very conservative accident cost values, the currently reported ratio of injuries to fatalities and actual effects of the Project interventions at the completion of the project (presented in Table 9.6 below) show that VRSP is strongly viable. Summary of Updated Economic Analysis At ICR Indicator At appraisal Using accident costs as in PAD by ADB NPV (US$ million) $23.13 $25.89 $20.67 EIRR (%) 34% 77% 59% By using the same methodology and model to realize projections at appraisal stage with actual performance of the Project, both updated NPV and EIRR are higher than those at appraisal phase, indicating that the Project was not only economically viable but also proved to be an efficient use of Bank resources. 3.4 Justification of Overall Outcome Rating Rating: Moderately Satisfactory The overall outcome of the project was moderately satisfactory. Two of the three PDOs were fully achieved at 138% (PDO 1) and 113 % (PDO 3) where the other one (PDO 2) was partially achieved at 34%, leading to sharp reduction in the rate of accidents and death, as well as significant reduction in the rate of injuries in the three VRSP road demonstration corridors. Implementation issues at the start of the project caused these results to be delivered three years later than originally intended, but the extremely effective turnaround of the project starting 2009 ensured that these goals were achieved while using Bank resources efficiently: at respectively USD 23.7 million and 75%, the NPV and EIRR of the project were higher at the close of the project than during appraisal, demonstrating the project’s clear efficiency despite the three years delays. Also, VRSP’s overall development objectives were closely aligned with both GOV and best practice- countries strategies on road safety interventions, and this alignment combined with sound fiduciary and safeguards arrangements, a solid M&E framework, and carefully designed blackspot treatments. However, due to the significant implementation delay at the start of 14 project, many project activities were implemented at a late implementation stage, such as installation of cameras, the road safety awareness building campaign, and the NRADS. 3.5 Overarching Themes, Other Outcomes and Impacts (a) Poverty Impacts, Gender Aspects, and Social Development VRSP helped elevate road safety issues in the national dialogue and has some impact at the institutional level: the helmet law for instance constituted a major step in addressing safety and awareness issues when it was mandated in 2007. Recently the decision to remove low-quality helmets from the market was another step in the right direction. This was also the main driver behind the KPI on helmet usage. Due to the economic losses linked to traffic accident deaths and injuries, the drop in traffic accidents contributed to the reduction of poverty in project areas. Direct impact on gender aspects was not reported. (b) Institutional Change/Strengthening Capacity has improved significantly for some agencies following VRSP’s investments and activities. This is most notable for the TSPMU, throughout the wide range of challenges that have occurred and the implementation support consultant guidance and training. The planning capacity of the sub-PMUs has also been positively affected, but remained dependent on the availability of the staff assigned at various phases of VRSP. This should be addressed in a future road safety project. Moreover, the new NTSC information system will link the NTSC in Hanoi with the 63 PTSCs in the provinces (including Hanoi and HCMC) and will provide a server for data exchange, analysis and knowledge management in addition to a web site. Road safety management training was also provided to NTSC-EO, PTSCs and PMU. (c) Other Unintended Outcomes and Impacts (positive or negative) N/A 3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops N/A 4. Assessment of Risk to Development Outcome Rating: Moderate A key risk to the sustainability of VRSP’s outcomes is the lack of capacity within NTSC, combined with inadequate and single source funding. Lack of funding, especially for the operation and maintenance on the blackspot interventions, could undermine VRSP’s achievements as the allocation of adequate human, equipment and budgetary resources must comply with the functional requirements of the road safety management system. Capacity could remain weak at the NTSC-EO, mainly due to inadequate staffing and limited technical knowledge in several areas of road safety (despite the fact that the implementation support consultant conducted several workshops and discussions on the NTSC organizational structure and emphasized the need to increase the technical staffing of the NTSC). Moreover, the country’s institutional framework for road safety still lacks several critical elements found in good practice countries. For instance, without an operational / management level committee and sector working groups, the work of the NTSC will suffer from a lack of sustained and ongoing co-ordination between meetings. 15 Only a policy level committee has been established (NTSC), supported by a secretariat (NTSC-EO). On the other hand, ownership of VRSP and its objectives have been strong at all levels of government, and there is a clear sense that GOV will dedicate an adequate amount of resources for future road safety initiatives. A follow up project to VRSP, which would demonstrate GOV’s commitment on these issues, was also being discussed at the time of this ICR. 5. Assessment of Bank and Borrower Performance 5.1 Bank Performance (a) Bank Performance in Ensuring Quality at Entry Rating: Moderately satisfactory The Bank performance during the lending phase is rated moderately satisfactory. As noted in section 2.1, the project exhibited weaknesses in two areas at entry: (i) an overly complex design, which included 18 sub-components for a $US 31 million project, triggered the procurement of more than 60 packages, involved 63 cities and provinces all over Vietnam; and (ii) the lack of clear and effective institutional arrangements, with an implementing agency (NTSC) which had not adequate resources or expertise to carry out its leadership role and direct participating agencies. In addition TSPMU (under the authority of NTSC) which had procurement responsibilities for packages to be implemented by sub-PMUs, was not empowered to carry them. This design made it extremely challenging to coordinate the preparation of all the components across agencies, and was a major factor behind implementation delays. The Borrower has characterized the project design as unreasonable in its own ICR. Moreover, more attention should have been devoted to setting up a rigorous M&E framework from the outset. Due to the low quality of the baseline figures estimated during preparation (and used in the PAD), the M&E activities had to rely on a different baseline. Also, key results indicators for Component B (percentage of traffic exceeding posted speed limits, of motorcycle riders wearing helmets, of motorcycle drivers licensed, of commercial vehicle operators licensed, of vehicles in demonstration corridors passing first time inspection) were missing from the PAD and thus undermined the quality of the M&E framework. (b) Quality of Supervision Rating: Moderately Satisfactory The quality of supervision was moderately satisfactory up until the midterm review in early 2009. During the first two years of project implementation, and despite the fact that the disbursement rate was below 8%, Bank official supervision of the project remained limited. Until the project was flagged as a problem status in July 2008, only three supervision official missions were conducted after the signing of the DCA: in December 2005, in October 2006, and in July 2007. Similarly, only three Implementation Status Reports (ISRs) were completed and filed: in January 2006, February 2007, and in June 2008 (this ISR was completed almost a year after the last supervision mission, and 1.5 years after the latest ISR). As a result, and due to the complex institutional arrangements involving a range of agencies and line ministries and to the number of packages in the 16 project design, procurements problems were not thoroughly followed up upon and the closing date had to be extended twice. However, the highly satisfactory quality of supervision starting 2009 led to swift turnaround of the project (as noted by management) and a smoother implementation of the project until completion. Following the midterm review, a 19-targets time bound action plan, to which the extension of the project closing date was conditioned, was rolled out in March 2009 and by July 2009 all 19 targets had been met with the exception of the award of the A1 consulting services contract. Feedback from GOV agencies showed that the proactive, problem-solving centered and trust building approach followed by the Bank supervision team played no small part in the successful turnaround of the project during its last three years. The team in Hanoi provided hands-on oversight of the project with a strong emphasis on implementation quality, monitoring and evaluation, financial management reporting, and, overall, rigorous Aide-Memoire follow up. Comments provided by line ministries, agencies and local authority’s staff have all indicated deep appreciation for the respectful working relationship and the constructive approach that was established, despite the complexity and wide scope of the project. (c) Justification of Rating for Overall Bank Performance Rating: Moderately Satisfactory In summary, the Bank was successful in helping VRSP achieve its development objectives, albeit with moderate shortcomings. As clearly shown through the sharp reduction in the rate of accidents, injury and death associated with road transport in the three VRSP demonstration corridors, the project development objective of the World Bank first stand-alone road safety intervention was met. Bank efforts also ensured that VRSP stayed highly relevant to the GOV road safety priorities and laid a robust groundwork to future road safety efforts based on a Safe System approach. The Bank performance did suffer from quality at entry problems due to an unrealistic design and sub-optimal institutional arrangements, and supervision efforts in the first three years of the project fell short of what was required for a project with so many agencies involved and packages to be procured (spanning areas from road safety education to IT systems to road safety equipment). However, these issues were identified at the midterm review and the Bank performed a quick and effective turnaround of the project starting in early 2009. As such, an overall rating of moderately satisfactory is justified. 5.2 Borrower Performance (a) Government Performance Rating: Moderately Satisfactory Government performance is assessed as moderately satisfactory. The rating reflects shortcomings in coordinating efforts across multiple line ministries in the early stage of implementation, while later recognizing major policy achievements in elevating the importance of road safety. This includes the appointment of the Deputy Prime Minister to chair the National Transportation Safety Committee. (b) Implementing Agency Performance Rating: Moderately Satisfactory Implementing agency performance is assessed as moderately satisfactory. This reflects shortcomings in TSPMU management on various aspects of implementation, including minor issues related to financial management (requiring repayment of ineligible 17 expenditures). While TSPMU was the lead implementing agency for the project, assessment of the six sub-PMUs is considered as follows: Sub-PMU Performance Moderately Unsatisfactory – Due to weak capacity, NTSC-EO delegated much of their responsibilities to other agencies in MOT NTSC-EO resulted in delays in completion of reviews and approvals of submissions from TSPMU. Moderately Satisfactory- the design and procurement of the civil Vietnam Road works for the accident black-spots was delayed. However, once Administration civil works contracts were awarded the work was completed in a timely manner. Satisfactory – sub-component activities were completed in a Vietnam Register timely manner. Moderately Unsatisfactory - Significant delays in procurement of the National Road Accident Database and challenges in meeting Ministry of Police the implementation schedule of the pilot enforcement corridors would impact overall project performance. Moderately Satisfactory- the design and printing of school books Ministry of Education was implemented but there were delays in the procurement. Moderately Satisfactory- the training of first-aid responders was Ministry of Health implemented but there were delays in the procurement of the hospital equipment and ambulances. (c) Justification of Rating for Overall Borrower Performance Rating: Moderately Satisfactory The Borrower’s overall performance is rated as moderately satisfactory. Key investment and institutional strengthening activities were ultimately delivered, though often behind schedule. 6. Lessons Learned A responsible lead organization with effective decision making powers is a prerequisite to manage an effective road safety program. NTSC does not have any authority to streamline the implementation management systems in other agencies. Further, NTSC’s limited capacity prevented NTSC from taking the role of an effective leading road safety agency. As a result, the implementation arrangements for VRSP’s were not conducive to quick decision making and the institutional framework for road safety lacked critical elements found in best-practice countries (no operational / management level committee nor sector working groups). A framework identifying leadership, collaboration and accountability of departments within government and non-government road safety stakeholders needs to be established for future road safety efforts to ensure the development, effective implementation and delivery of the road safety strategy. Models used elsewhere in the world vary from having a (i) lead agency that plays a dominant role in most of the institutional management functions, in addition to taking a lead role in implementation; and (ii) having a lead agency that is responsibility within government for the development of the national road safety strategy but coordinates the implementation of the strategy through the delegation of individual activates to each line agency. Given the strong independence of government agencies in Vietnam it is likely that the first model may not be applicable, as demonstrated by the outcomes under VRSP. Given this future 18 interventions need to build on the achievements of VRSP to develop NTSC as the agency responsible for horizontal inter-governmental coordination, vertical coordination of national, regional and local activities and the coordination of delivery partnerships between government, professional, non-governmental and business sectors. Simplicity of design and implementation arrangements are key for Safe System – inspired road safety approaches, which put the emphasis at once on road user behaviors, road designs, institutional capacity and stakeholders awareness. Simplicity of design is a key factor of success in environments where the multiplicity of stakeholders, suboptimal administrative processes, weak institutional capacity, discrepancies in appraisal procedures between ministries, and a wide geographical project area scope were bound to make implementation complex. Moreover, slow government internal communications, coordination and approval procedures, internal reorganizations and regulatory changes are issues that should be anticipated. In the case of VRSP, they became particularly acute in a context where rigid government control systems were imposed on a multi-sector project involving several agencies and sub-PMUS. To simplify the implementation arrangements, the number of PMUs should be minimized, in comparison to one PMU and six sub-PMUs in VRSP, and responsibility for procurement and quality control should be delegated to a unique PMU for every contract, avoiding multiple agencies internal control mechanisms. Sustainable funding and the availability of adequate knowledge and skills are essential to the delivery of road safety interventions. Without the right mix of adequate funding and human resources for road safety interventions, institutional structures and processes tend to be ineffective and national strategies tend to remain a “paper exercise�. The allocation of appropriate human, equipment and budgetary resources must comply with the functional requirements of the road safety management system. In the case of VRSP, resources at several sub PMUs were inadequate, most notably at the NTSC-EO. Similarly, funding for crucial items such as police resources and equipment was not a high priority. Therefore, a long term investment strategy, relying on multiple income streams to avoid the over-dependency on a single source, would be required to implement the desired results. Additional sources of funding could be considered, as surcharges on traffic fines, motor fuel, weight-distance charges, compulsory vehicle insurance, vehicle license fees, or toll fees from roads. M&E systems need to be based on a comprehensive road safety management information system, including a national road accident database. For the VRSP indicator baselines were estimated based on trends observed in developing countries. As data was collected during the project it became clear that the these trends were not appropriate for Vietnam. Although a temporary data collection system was set up with the assistance of the district police on the pilot corridors, health data also could not be used due to the absence of MOH inputs in the database. The commissioning of NRADS and the NTSC IT systems under VRSP have the potential to provide a range of data that can be used to monitor road safety interventions in the future. However, future interventions should assist NTSC in (i) expanding the use of NRADS to include agencies of MOH and MOT, and (ii) linking databases on vehicle and driver registration and driver violations to the NRADS database. The objective would be to create a family of interlinked databases that allows data on all aspects of each accident to be recorded such that trends in accidents can be used to identify measures needed to reduce both the number and severity of accidents. These interventions could include: black-spot road improvements, upgrading of accident response systems in areas of high accidents, intensified police surveillance to enforce traffic laws in areas of high accident rates. 19 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners (a) Borrower/implementing agencies Editorial comments were provided, ICR text has been revised to address these comments. (b) Cofinanciers Not applicable. (c) Other partners and stakeholders Not applicable. 20 Annex 1. Project Costs and Financing (a) Project Cost by Component (in USD Million equivalent) Actual/Latest Appraisal Estimate Percentage of Components Estimate (USD (USD millions) Appraisal millions) Total Baseline Cost 31.24 27.64 88.5 Physical Contingencies 0.38 0.00 0 Price Contingencies 3.30 0.00 0 Total Project Costs 34.92 27.64 79.2 Front-end fee PPF N/A 0.00 0 Front-end fee IBRD N/A 0.00 0 Total Financing Required 31.73 23.74 87.1 (b) Financing Appraisal Actual/Latest Type of Estimate Estimate Percentage of Source of Funds Cofinancing (USD (USD Appraisal millions) millions) Borrower 3.19 3.90 122 International Development 31.73 23.74 75 Association (IDA) 21 Annex 2. Outputs by Component Component A Institutional and Capacity Building Program Status US$ Amount A1 Capacity building and implementation support Package A1.1 Implementation support to NTSC/TSPMU and pre-F/S of the Vehicle Testing Center Completed 5,136,994 Package A1.2 TSPMU Financial Management system Completed 34,789 Package A1.3 Financial Uditing Completed 46,796 Package A1.4 Early Implementation Support to NTSC/TSPMU Completed 47,600 A2 Safeguards implementation support and independent monitoring Package A2.1 Consulting services for social/envt monitoring of blackspot treatment program Completed 83,076 Package A2.2 Procurement of equipment to support environmental monitoring Completed 35,217 A3 NTSC Equipment to support implementation 4 WD vehicles, computers, office equipment, electrical equipment Completed 379,990 A4 Operating costs and training Package A4.1 TSPMU operating costs Completed 1,496,000 Package A4.2 Project implementation training Completed 552,000 Component B Road Safety Demonstration and Awareness Program Status US$ Amount B1 Blackspot treatment program Package B1.1 Hanoi - Thanh Hoa - 12 blackspots Completed 288,872 Package B1.2 Thanh Hoa - Vinh - 10 blackspots Completed 491,887 Package B1.3 HCMC - Ninh Thuan - 11 blackspots Completed 9,732,111 Package B1.4 HCMC - Can Tho - 7 blackspots Completed 588,330 B2 Resettlement No land acquistion, resettlement, compensation involved Completed 0 B3 Design and Construction supervision Package B3.1 Blackspot treatment detailed design Completed 96,695 Package B3.2 Construction supervision Completed 114,006 B4 Vehicle inspection computer systems upgrade Package B4.1 Computer and communications equipment Completed 448,698 Package B4.2 Software development for vehicle inspection system Completed 158,000 Package B4.3 Inspection equipment Cancelled 0 Package B4.4 Two mobile inspection vehicles Completed 1,479,559 B5 Driver training and testing Package B5.1 Upgraded licensing equipment Completed 893,238 Package B5.2 Upgraded driver training equipment and testing facilities Completed 110,901 B6 Road safety audits and safety inspection Completed 113648 B7 Demonstration corridor safety enforcement Completed 2749265 B8 Road user education and awareness Package B8.1 Consulting for coordinated road user education and public awareness campaigns Completed 994,800 Package B8.2.1 Safety helmets for children and teachers Completed 203,805 Package B8.2.2 Printing and dissemination of road safety materials Cancelled 34,561 Package B8.2.3 NTSC information system Completed 313,438 Package B8.2.4 Road safety promotion vehicle Completed 89,528 Package B8.2.5 Road safety management training Completed 180,000 Package B8.2.6 Implementation of road user education and publci awareness campaigns Completed 534,372 B9 Road safety education program Package B9.1 Teaching materials and books Completed 212,126 Package B9.2.1 Traffic safety teaching facilities Completed 278,918 Package B9.2.2 Other teaching materials and books Completed 212,305 Package B9.3 Design of road safety education materials Completed 460,948 Package B9.4 Teacher training on road safety Completed 100,000 B10 Road accident emergency treatment Package B10.1 Road accident emergency treatment supplies for hospitals and clinics Completed 856,250 Package B10.2 Three ambulances Completed 158,215 Package B10.3 Training and training material Completed 361,000 B11 Operating costs for the demonstration corridors Completed 2,070,000 B12 Local consulting services Completed 137,500 Annex 2. Outputs by Component (continued) Component C Road Safety Monitoring and Evaluation Program Status US$ Amount C1 National Road Accident Database System (NRADS) Package C1.1 Design and Installation of NRADS Completed 1,840,608 Package A1.2 Supply and Installation of the database hardware and software Completed 1,317,495 C2 Demonstration Corridor Monitoring and Evaluation Completed 2,212,770 22 Annex 3. Economic and Financial Analysis The objective of VRSP was to improve road safety in Vietnam by applying an integrated set of interventions to three “high-risk� demonstration corridors. The proposed investments targeted to reduce road user costs and costs to society by reducing the number and severity of fatalities and injuries that result from road accidents. In May 2005, as part of the appraisal procedure, an economic analysis of the VRSP was conducted to justify the economic viability of the project components. This analysis will update assumptions based on actual performance of the Project by 2012 to reconfirm economic benefits of the project. Evaluation Criteria The update analysis applies the same cost benefit analysis method that used in appraisal phase. Because of the integrated nature of the Project and the covariance effects between Project components, the Project has been evaluated as a whole rather than as individual components. A model has been developed for road safety on the three demonstration corridors that takes account of the historic, current and predicted vehicle kilometres travelled, historic, current and predicted accident rates, and estimates of the effect of project interventions. Road accidents, fatalities and injuries are modelled for the demonstration corridors “with� and “without� the Project. The economic evaluation measures the incremental costs and benefits between these two alternatives. The economic evaluation uses a 12 percent discount rate and a 20-year analysis period. Costs and benefits are discounted to 2004 values. The exchange rate in 2004 was 15,500 VND to US$1.00. Project costs and benefits have been expressed in economic terms, rather than using actual market/financial prices. Costs The main costs of the project are the costs of civil works (blackspot treatments) and purchase of vehicles, equipment and materials. These originally were estimated based on historical cost data for similar works and purchases in Vietnam, and are updated with actual annual disbursements. Table 9.1. Economic Project Costs (US$ million) Year PAD Projection Actual Disbursement 2005 10.46 - 2006 6.54 1.38 2007 4.91 0.23 2008 4.86 0.62 2009 0.79 2010 2.97 2011 4.93 2012 5.22 2013 6.23 Total 26.77 22.35 Benefits 23 The main economic benefits of the Project are reductions in social costs resulting from reduced numbers and severity of fatalities and injuries that result from road accidents. There are some time savings to road users from better availability of medical services but these have not been included in the analysis. Environmental effects of the Project are small and mitigation measures are included as part of construction, therefore, environmental benefits or dis-benefits have been excluded. Accident Costs Accident costs used in the Project economic analysis during appraisal phase in 2005 were those documented in Technical Note No. 6 “Road Accident Costing in Vietnam� from the Vietnam Road Safety Improvement Study carried out by Ross Silcock, et al in March 1999 and updated to March 2004 values. Internationally, willingness-to-pay has become the accepted method to measure the value of life. This approach for developed countries gives a value (usually termed the “value of a statistical life�, or the “value of a statistical fatality avoided�) many times higher than the human capital approach. A recent study suggests that a reasonable range for a country’s value of statistical life appears to be 100-200 times GDP per capita. Using the low end of this range for Vietnam would still substantially scale-up the estimate of project benefits and this underscores the conservative nature of the estimates made using the values provided by Technical Note No. 6. Since then there were two other studies on accident costs in Vietnam:  VietnamRoad Traffic Accident Costing as part of the Regional Road Safety Program carried out by Asian Development Bank in association of Southeast Asian Nations in 2004 using the gross output or human capital methodology.  Costing road accidents, a National Research Study conducted by the Transport Development and Strategy Institute (TDSI) in 2006. The study has collected a relatively comprehensive record of accidents from major hospitals, insurance agencies, traffic polices, etc in order to estimate economic value of road accidents. Table 9.2. Accident Costs in Vietnam (2004 price, US$) Type of accident PAD ADB TDSI* Fatality 14,000 11,393 27,006 Injury 400 2,003 10,450 * The costs were calculated in 2006 and adjusted back to 2004 Injury costs were calculated for serious injuries The above injury accident costs include an adjustment for damage only accidents and assume that injuries are 25 percent serious and 75 percent minor. If only serious injuries are being considered, then the injury cost should be increased. This updated analysis keeps using the conservative accident costs as in PAD, but also calculate benefits of the Project using accident costs of the ADB studies for references. 24 Traffic Data The GOV Comprehensive Poverty Reduction and Growth Strategy is based on high growth, which will be accompanied by high growth in traffic and exposure to risk on the road network. Traffic growth in Vietnam has been very high over the past decade and, given the GOV strategy, it is assumed that this trend will continue. At appraisal phase in 2005, limited traffic count information from 1997 to 2003 was obtained from VRA for the demonstration corridors. This was extrapolated to give vehicle kilometers of travel (VKT) for each section of the corridors. The weighted average annual traffic growth (linear) for the corridors using this data, based on linear regression, was 8.6 % (percent of 2003 values). During project implementation, traffic counts have been done regularly under the Package Consulting Services for Demonstration Corridors Monitoring and Evaluation of the Component C2 by Consia Consultant. These data are cross checked with available traffic counts carried out by DVRN. An assessment has been done to estimate impacts of the open of the HCMC - Trung Luong Expressway in 2010 and Cau Gie - Ninh Binh Expressway in November 2011 and to define the average annual traffic growth rate of 8.7% on the demonstration corridors. The actual growth rate is 8.7%, which is very close to the rate assumed in the analysis in PAD. Details on traffic counts are in the Table 9.3. Accident Data The baseline data in 2004 was based on accident data for 2000 to 2003 for the demonstration corridors obtained from RRMU2, RRMU7 and Traffic Police. The ratios of accidents/fatalities and injuries/fatalities obtained from both Traffic Police and NTSC are around 1.6 and 1.5 respectively. This means that, in effect, the Traffic Police are only reporting fatal accidents and a very small proportion of serious injury accidents. There is some indication that even fatal accidents are not all reported. An accident costing study in Indonesia found that fatalities were some 40% greater than reported by Police. The assumed ratio of accidents/fatalities used in Technical Note No. 6 was 6.9 and the ratio of injuries/fatalities was 10.0. These ratios are very conservative compared with the ratios for countries with reasonably good reporting regimes. For example, in New Zealand, recent improvements in reporting of road accidents by the Police gives a ratio of reported accidents/fatalities of nearly 100, a ratio of serious injuries/fatalities of about 5, and a ratio of serious plus minor injuries/fatalities of about 25. The analysis model assumed that the ratio of accidents/fatalities was 3.0 and that of injuries/fatalities was 10.0 in the baseline year of 2004. The combined baseline, prediction and actual accident rates for the demonstration corridors, together with the associated social cost, are given in Table 9.4. 25 Table 9.3. Traffic Data AADT (Total motorized vehicles) NR Chainage Count Station City/Town 1997 2000 2003 2008 2009 2010 2011 2012 1 Km 192 + 500 Phap Van - Cau Gie Thuong Tin 17,650 21,938 24,957 26,660 32,336 1 Km 221 + 000 Dong Van Phu Ly 12,000 15,867 20,946 21,789 20,165 21,863 1 Km251 + 000 Doan Vy Bridge Ha Nam/Ninh Binh 4,062 5,475 7,990 8,831 10,976 11,847 13,218 15,087 1 Km 253 + 700 KCN Gian Khau Gia Vien 8,571 12,000 12,763 12,074 15,662 1 Km 270 + 000 Yen Bridge Ninh Binh 4,930 5,559 9,207 12,616 13,143 14,454 15,928 18,097 1 Km 291 + 100 Bim Son Bim Son 8,000 12,869 19,178 20,044 15,806 18,291 1 Km 299 + 260 Hà Trung Ha Trung 8,188 7,612 12,829 19,596 20,473 15,894 16,579 1 Km 323 + 000 Thanh Hoa Thanh Hoa 8,000 15,259 17,031 17,844 15,329 27,200 1 Km 366 + 300 Tinh Gia Town Tinh Gia 5,934 8,426 9,561 9,519 11,074 1 Km 447 + 300 Quan Hanh Nghi Loc 5,388 7,985 6,123 6,513 7,699 6,991 7,661 1 Km 467 + 300 Ben Thuy Vinh 1,420 10,328 12,432 13,598 15,675 11,334 19,123 1 Km 1925 + 000 HCMC HCMC 80,000 26,439 26,828 14,792 13,542 18,444 1 Km 1945 + 000 Tan An Town Tan An 41,432 33,210 34,776 18,750 13,547 17,926 1 Km 1967 + 500 Trung Luong T-junction My Tho 36,988 26,705 35,619 18,925 12,184 12,956 1 Km 1968 + 700 Trung Luong T-junction My Tho 22,167 40,955 59,998 23,561 25,449 12,783 11,478 10,089 1 Km 2031 + 100 Vinh Long Vinh Long 17,128 9,981 9,973 13,135 16,070 18,019 1 Km 2040 + 100 Vinh Long Vinh Long 17,005 8,487 9,508 11,737 14,304 17,445 1 Km 2068 + 000 Can Tho Can Tho 20,000 9,213 9,622 11,512 9,601 9,554 51 Km 11 + 000 Toll Gate T1 Dong Nai 18,229 19,198 22,565 21,637 24,705 51 Km 16 + 000 Nuoc Trong Bridge Dong Nai 11,227 13,005 19,704 16,617 23,845 27,472 20,405 22,752 51 Km 61 +800 Long Huong Vung Tau 8,037 11,987 15,993 11,365 12,207 14,596 15,552 13,816 51 Km 74 +000 Vung Tau Vung Tau 14,930 13,539 16,451 18,929 8,531 Source: PAD, VRSP-C2: Consulting Services for Demonstration Corridors: Monitoring and Evaluation, CONSIA Consultants * HCMC - Trung Luong Expressway opened in 2010 and Cau Gie – Ninh Binh Expressway opened in Nov 2011. 26 Table 9.4. Accident Rates 2001-2003 2004 2012 2012 Average Baseline Targeted Achieved Social Cost Total (US$ million) 8.1 11.0 6.4 7.7 Cents per VKM 19 23 11 11 Accidents Total 917 1,837 1,692 1,028 Rate per 100 million VKT 22 39 29 25 Fatalities Total 553 603 358 343 Rate per 100 million VKT 13 13 6 5 Injuries Total 836 6,125 3,582 7,196 Rate per 100 million VKT 20 130 62 107 Source: PAD, VRSP-C2: Consulting Services for Demonstration Corridors: Monitoring and Evaluation, CONSIA Consultants Effect of Project Interventions It is common practice to estimate rates of accidents of differing degrees of severity by road type and traffic conditions. The impact of a project is then forecast on the basis of the expected changes to road conditions and traffic volumes. However, accident incidence on a section of road or road network, like the demonstration corridors, is very sensitive to local conditions and design and a relatively high variability of accident outcome from forecast is not unusual. This is partly due to the random nature of accidents. At the appraisal stage, while there was no formal research available to show the effect in the Vietnam environment of the interventions proposed for the Project, estimates were made of the likely accident reduction effect based on the incidence of the accident cause factors being addressed by the Project interventions together with international experience of typical responses. Table 9.5. Best Estimated Intervention Effects Assumed in PAD Year Effect of New Interventions 1 2 3 4 1. Blackspot treatments 3% 6% 9% 12% 2. Driver Licensing - Training/Testing/Enforcement 3% 3% 3% 3% 3. Vehicle Road - Worthiness - 2% 2% 2% 2% Inspection/Enforcement 4. Cycle helmet enforcement 10% 15% 20% 22% 5. Speed enforcement 10% 10% 10% 10% 6. Drink-driving 3% 3% 5% 5% 7. Encroachment 1% 1% 1% 1% 8. School Road Safety Education 1% 1% 1% 1% 9. Emergency Medical Services 2% 2% 3% 4% 27 In this updated analysis, the actual effects of the Project interventions measured by the M&E Consultants are used instead of the above assumptions. Key indicators featuring the baseline of the Project in 2004 and actual achievement of the Project in 2012 (presented in Table 9.4) enable interpolations of data for intermediate year during project implementation in accordance to progress of work and reimbursement. Updated Economic Analysis Results The updated Net Present Value (NPV) and Economic Internal Rate of Returns (EIRR) of the Project, using very conservative accident cost values, the currently reported ratio of injuries to fatalities and actual effects of the Project interventions at the completion of the project (presented in Table 9.6 below) prove that VRSP is strongly viable. Table 9.6. Summary of Updated Economic Analysis At ICR Indicator At appraisal Using accident costs as in PAD by ADB NPV (US$ million) $23.13 $25.89 $20.67 EIRR (%) 34% 77% 59% By using the same methodology and model to realise projections at appraisal stage with actual performance of the Project, both updated NPV and EIRR are higher than those at appraisal phase, indicating that the Project is not only economically feasible but also better than it was predicted. By using other sources of accident costs produced by the ADB via their studies specialised in costing accidents in Vietnam, the updated analysis confirms the robustness of the Project. 28 Annex 4. Bank Lending and Implementation Support/Supervision Processes (a) Task Team members Responsibility/ Names Title Unit Specialty Lending Anthony G. Bliss Consultant SASDT Jerry Lebo Sr Operations Officer CFPTO Nga Thi Quynh Dang Trust Fund Coordinator EACVF Christopher J. De Serio Operations Analyst EASIN Vinh Quoc Duong Consultant EASVS Eva Jarawan Lead Health Specialist EASHH Hung Viet Le Sr Financial Management Specia EASOS Irina Luca Lead Procurement Specialist AFTPW Aurelio Menendez Sector Manager LCSTR Behdad M. H. Nowroozi Sr Financial Management Specia MNAFM Maria Margarita Nunez Sr Highway Engineer LCSTR Richard G. Scurfield Consultant SASSP Anil H. Somani Consultant EASNS Kien Trung Tran Senior Procurement Specialist EASR2 Phuong Thi Minh Tran Sr Transport. Spec. EASVS EASSO - Hong Vu Sr Operations Off. HIS Mei Wang Senior Counsel LEGAM Chaohua Zhang Lead Social Development Specialist SASDS Supervision/ICR Andre A. Bald Sr Urban Spec. AFTU1 Paul Vallely Sr Transport Specialist EASVS Hanh Thi Huu Nguyen Financial Management Specialist EASFM Reda Hamedoun Transport Specialist EASIN Anthony G. Bliss Consultant SASDT Christopher J. De Serio Operations Analyst EASIN Fei Deng Sr Transport. Spec. EASIN Vinh Quoc Duong Consultant EASVS Dung Anh Hoang Sr Transport. Spec. EASVS Hung Viet Le Sr Financial Management Specia EASOS Irina Luca Lead Procurement Specialist AFTPW EASTE - Simon Lucas Sr Transport. Spec. HIS Lan Thi Thu Nguyen Natural Resources Economist EASVS Hoang Xuan Nguyen Procurement Specialist EASR2 Trang Phuong Thi Nguyen E T Consultant EASVS Thuy Thi Bich Nguyen Team Assistant EACVF Anna R. Okola Transport. Engineer LCSTR William D. O. Paterson Consultant EASNS Cung Van Pham Sr Financial Management Specia EASFM 29 Emmanuel Py Infrastructure Specialist EASWE Nguyen Chien Thang Senior Procurement Specialist EASRP Phuong Thi Minh Tran Sr Transport. Spec. EASVS Van Anh Thi Tran Sr Transport. Spec. EASVS Ha Thuy Tran Financial Management Specialis EASFM Thao Phuong Tuong Team Assistant EACVF EASSO - Hong Vu Sr Operations Off. HIS Ly Thi Dieu Vu Consultant EASNS (b) Staff Time and Cost Staff Time and Cost (Bank Budget Only) Stage of Project Cycle USD Thousands (including No. of staff weeks travel and consultant costs) Lending FY04 27.79 75.50 FY05 40.15 106.36 Total: 67.94 181.86 Supervision/ICR FY06 17.85 25.95 FY07 19.4 38.83 FY08 18.12 50.52 FY09 22.31 47.92 FY10 21.24 39.54 FY11 25.65 51.41 FY12 20.63 45.88 FY13 21.61 53.01 Total: 166.81 397.41 30 Annex 5. Beneficiary Survey Results (if any) 31 Annex 6. Stakeholder Workshop Report and Results (if any) 32 Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR I. Summary of Borrower’s ICR 1. Performance Indicators: The key performance indicators served to monitor the outcome of the VRSP. Most of these factors were calculated according to international standards to be able to compare the road safety situation between different countries. In Vietnam the key indicators for the road victims were collected from the Traffic Police. At the end of the project the accident situation on the corridors had improved. The accident risk has gone down to 63% of the initial value as a result of fewer accidents despite increasing traffic. The risks of severe injury went initially up but are now at 89% of the initial value. This figure is also a result of better reporting of accidents. The risk of being killed in traffic on the corridors declined to 58% of the initial value. This is a strong indication that there is a better reporting of injuries in accidents, a number that had been severely under reported. In the future, the introduction of NRADS will improve the monitoring of the data collection as part of the VRSP2. 2. Evaluation of interventions on the pilot corridors: Implementation should carried out carefully. The preparation took a lot of time due to the combination of different agencies. The coordination mechanism of the project should have been clear and specific. The staffs of the management agencies should be trained before implementing projects to ensure capacity building. The differences in implementation procedures need to be clarified and showed clearly in the credit agreement to avoid delays in implementation. 3. Lessons learned  Project design: VRSP is the first large scale standalone road safety project to be prepared in Vietnam and elsewhere in the world. It was complicated by too many coordination activities of different agencies, which led to inconsistencies. There was a lack of leadership with has enough capacity and power. The project was a problem project according to the World Bank. In order to be restored to normal status, to improve the project progress NTSC restructured the project by changing the project leader, delegation of authority of subPMU, changes in TSPMU. Capacity of NTSC was strengthened by the decree No. 32. 33  The splitting of components and resources was not reasonable (consulting services, goods, training...), which led to the lack of the flexibility and harmony in the coordination of the project funds.  The provisions on local tax, domestic spending and foreign spending was not clear and should be defined before bidding to impact the preparation of counterpart funding for the project.  The project design did not perfectly assess the typical agenices specialization in Vietnam.  Implementation:  The implementation agency did not assign enough staff for project management, which led to the long preparation of procedures due to the lack of knowledge of WB requirements.  The sub-PMU staffs did not work full time on the project and lacked knowledge of WB procedures for procurement and approval.  The evaluation procedures of both Vietnam and WB are long due to the discrepancies in procedures. The project was implemented on some pilot corridors, but the intervention specifications were minimal.  The procedures, packages splits and bidding methods were not appropriate, leading to implementation delays. 4. Evaluation of the effectiveness of the borrower in the preparation and implementation of activities:  Capacity building:  NTSC restructured the project to change the project leader and delegated authority to the sub PMU.  A road traffic safety strategy was formulated.  Mechanisms, procedures, and equipment for NTSC were provided to ensure coordination activities in traffic safety.  Staffs were trained to implement Traffic Safety activities for NTSC, study tours in Europe were organized to get meaningful experience for committee staffs and members of the Committee.  More than 300 Traffic Safety Auditors for the Directorate for Road of Vietnam in the technical audit of traffic safety on the Highways were trained. Training materials and training teachers were developed to pursue auditor training courses in the future.  Approximately 1400 staffs of MOH and traffic police were trained in first-aid for traffic accidents.  900 teachers at all levels were trained on teaching road safety in schools.  The Road Accident database system was established over the country, including 63 provinces, and 508 district Police station with nearly 2,000 police officers and soldiers. 34  A monitoring system to test motor vehicle and register them nationwide was established.  Driving license management system for DRVN was developed in 63 provinces over the country.  The interventions in Traffic Safety:  Black spots treatment on 3 demonstration corridors (Ha Noi - Vinh; Ho Chi Minh City - Can Tho; Dong Nai - Ninh Thuan), including 38 black spots. At the end of the project, after all the black spots on 3 demonstration corridors were treated, the number of traffic accidents reduced so much that there were not any serious traffic accidents. Specifically, the number of traffic accidents before treatment (2005 - 2007) was 319; traffic accident after treatment (3/2011- so far) was26 cases.  Pilot training support system equipment for driving & testing schools in Chi Linh District, Hai Duong Province. The system supported training centers for drivers training.  Equipment to address data, vehicle management, and pilot technical standard testing equipment of the mobile vehicle to support monitoring the situation of vehicles in use after registration. The system has been piloted in the main bus stations in Ha Noi and Ho Chi Minh City (My Dinh, Mien Tay) and the 3 demonstration corridors of the project to check the status of vehicles.  Raising awareness for road users by different methods to consider and review the effectiveness of each method.  Teaching facilities for schools on the 3 demonstration corridors including books and documentation for teachers and other teaching facilities for teaching traffic safety for all educational levels (159.620 material books and 5.040 posters for preschool, 1.321.260 books for primary school level, 138.130 books for secondary school level, 51.040 books for high school level, 34.520 books for tertiary education level). These documents have been delivered to 12 Departments of Education and Training (DOET) and 46 Divisions of Education and MOET.  467 emergency materials were provided in the 3 demonstration to 76 institutions, including: 12 hospitals, 9 province general hospitals, 3 emergency centers, 27 district hospitals and 14 municipal medical stations. 5. Implementation:  During the first stage of project implementation (2006 - 2008), the implementation was very slow due to lack of resources, leadership and management mechanisms. However, in 2009, NTSC had strong organization adjustments. The project was put on normal project status afterwards and could be successfully implemented.  The borrower was very active and tried to coordinate all the activities of many different agencies. Need to continue to enhance the capacity of staffs for traffic safety implementation. 35  The awareness campaigns and teachings on traffic safety were very effective, and should be expanded and developed in the future. 6. Bank performance:  The World Bank had an effective team with regards to the coordination, preparation and supervision of project implementation. However, the implementation still faced to difficulties due to unreasonable project design.  The review of submitted documents took a long time due to project design, and complexity of procedure.  Procedures should be adjusted and comments should be given in a timely manner.  Better define project evaluation indicators in the future: the indicators could not be measured and surveyed, therefore it was very difficult to evaluate the effectiveness of the implementation of the project. II. Comments on Draft ICR Editorial comments were provided; ICR text has been revised to address these comments. 36 Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders 37 Annex 9. List of Supporting Documents 1. VRSP Project Appraisal Document 2. VRSP Development Credit Agreement 3. Mission Aide Memoires 4. Implementation Status Reports 5. Midterm Review 6. Borrower Implementation Completion Report 38 MAP 39