Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Tianjin Urban Transport Improvement Proj (P148129) Report Number: ICRR0023557 1. Project Data Project ID Project Name P148129 Tianjin Urban Transport Improvement Proj Country Practice Area(Lead) China Transport L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IBRD-85650 31-Mar-2021 100,000,000.00 Bank Approval Date Closing Date (Actual) 21-Dec-2015 31-Dec-2022 IBRD/IDA (USD) Grants (USD) Original Commitment 100,000,000.00 0.00 Revised Commitment 100,000,000.00 0.00 Actual 100,000,000.00 0.00 Prepared by Reviewed by ICR Review Coordinator Group Hassan Wally Avjeet Singh Avjeet Singh IEGSD (Unit 4) 2. Project Objectives and Components DEVOBJ_TBL a. Objectives The Project Development Objective (PDO) of the Tianjin Urban Transport Improvement Project as articulated in the Loan Agreement (LA, page 5) was identical to the one stated in the Project Appraisal Document (PAD, paragraph 24) and aimed to: Page 1 of 20 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Tianjin Urban Transport Improvement Proj (P148129) "leverage the Tianjin metro system and to promote walking and biking in the urban core (in Heping and Nankai) in order to make transport greener and safer in Tianjin and draw lessons for other large cities." Revised PDO. As part of the August 29, 2019 restructuring, the PDO was revised to include Hebei district an additional beneficiary district. The revised PDO was to: "leverage the Tianjin metro system and to promote walking and biking in the urban core (in Heping, Hebei and Nankai) in order to make transport greener and safer in Tianjin and draw lessons for other large cities." The project will be assessed based on the revised PDO because it was identical to the original PDO except that the scope of the project changed to include Hebei district. Parsing the revised PDO. The PDO will be parsed based on the following three objectives: 1. To make transport greener through leveraging the Tianjin metro system and promoting walking and biking in the urban core (in Heping, Hebei and Nankai). 2. To make transport safer through leveraging the Tianjin metro system and promoting walking and biking in the urban core (in Heping, Hebei and Nankai). 3. To draw lessons for other large cities. b. Were the project objectives/key associated outcome targets revised during implementation? Yes Did the Board approve the revised objectives/key associated outcome targets? Yes Date of Board Approval 29-Aug-2019 c. Will a split evaluation be undertaken? No d. Components The PDO was supported by the following five components: 1. Green Transport Improvement in Heping and Nankai Districts (appraisal cost: US$89.48 million, actual cost: US$76.77 million). This component would finance the redevelopment of the streetscape in certain parts of Heping and Nankai Districts, including the creation of an integrated pedestrian and bike Page 2 of 20 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Tianjin Urban Transport Improvement Proj (P148129) network with infrastructure investments in, inter alia, street pavement updates, drainage improvements, street facilities, and landscape improvements. 2. Metro Access Improvement (appraisal cost: US$89.05 million, actual cost: US$56.60 million). Under this component, the project would finance civil works for selected intersection improvements and interconnection facilities (bike parking, bus connection/terminal, taxi connection, landscaping and park and ride) at about 111 metro stations along existing Tianjin Metro Lines 1, 2, 3, and 9, and Lines 5 and 6 (under construction). 3. Public Bike Sharing System (PBS) Pilot (appraisal cost: US$23.14 million, actual cost: US$0.00 million). This Component would finance the establishment of a pilot public bike-sharing (PBS) system in the core urban area of Tianjin, including, inter alia, the pavement of PBS stations, the development of a PBS management system, and the provision of bikes and other required equipment. 4. Bus Terminal Development (appraisal cost: US$15.99 million, actual cost: US$4.53 million). This component would finance the development of selected bus terminals, including, inter alia, civil works for the pavement of terminals and bus stops, bus parking, car parking, bike parking and service buildings, and the purchase of equipment for bus operation within the bus terminals. 5. Technical Assistance (appraisal cost: US$2.90 million, actual cost: US$3.16 million). Technical assistance would be provided on, inter alia: sustainable green urban transport development; parking management improvement schemes; the effectiveness of the public bike sharing system implementation; multi-channel financing mechanisms for urban transport; and surveys and support for the analysis of and reporting on the project impact. Revised Components. Components were adjusted in the two restructurings to adapt to increased project scope as follows: Component 1. First restructuring: The Component was renamed as Green Transport Improvement in Heping, Hebei and Nankai Districts. The NMT improvement activities was expanded to cover 38 streets in Hebei district, an additional length of approximately 35 km. The activities extended the overall coverage of NMT interventions to 11.7 sq km compared with the original 7.2 sq km. Second restructuring: No changes in the scope activities, however there were large savings in the contracts under Component 1 due to competitive bidding and the savings were used for additional activities under Component 2, elaborated on further below. Component 2. First restructuring: The scope of work (but not the number of stations) under this component was reduced by excluding certain access improvement activities that were being implemented as part of metro construction (57 metro stations access improvement works along Metro Lines 5 & 6 were executed by other agencies), which resulted in savings of around US$32 million. Second restructuring: Metro access improvements at 15 stations were dropped due to some of the changes envisaged around these sites being done by other agencies, reducing the target number of metro stations with access improved to 96. In addition, improvements to a further 107 streets accessing metro stations were added to the scope of this component to utilize savings from Component 1. Component 3. First restructuring: At appraisal, there was no public bike sharing system in Tianjin. However, in 2017, several dockless bike sharing companies established operations in the city, with a total Page 3 of 20 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Tianjin Urban Transport Improvement Proj (P148129) estimated fleet size of 800,000. Thus, this component was cancelled as its activities planned at appraisal were already being effectively carried out through private sector financing. Component 4. First restructuring: This component was renamed as component 3: Bus Terminal Development, due to the cancellation of the former component 3. Three bus terminals were dropped from the scope because agencies had decided that they wanted to use the land for other purposes. Second restructuring: The planned two terminals were dropped and replaced with one new bus terminal. The reason why one terminal was cancelled was because it did not have the support of nearby residents, while the other site was cancelled because the landlord for the site where the bus terminal would be built requested very large land consolidation fees that the government was not willing to pay. Compared to the original plan, the remaining terminal – the Hongqiao Xianyang North bus terminal - was to provide five times more capacity for bus parking and services and an additional vertical parking building with 200 parking spaces to limit roadside parking. Component 5. First restructuring: This component was renamed as component 4: Technical Assistance (TA), due to the cancellation of the former component 3. Three TA activities were dropped (the effectiveness of public bike sharing, multi-channel financing mechanism for urban transport, and surveys and support for the analysis of and reporting on the project impact). The study on the effectiveness of public bike sharing was dropped because Component 3 was cancelled, the multi-channel financing mechanism study was funded through a Global Environment Facility (GEF) Sustainable Integrated Approach Pilot Project, and the survey study on project impacts was subsumed into the scope of two new TAs. These TAs covered the performance evaluation and analysis of economic activity in the areas being improved by the project, and preparation of Project Implementation Completion Report. In addition, given the government’s interest and commitment, “Phase 2” studies were added for both the Sustainable Green Urban Transport Development Strategy study and the Parking Management Improvement Scheme study to build on the findings and recommendations of the “Phase 1” studies. e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project Cost. Total project cost was estimated at US$224.27 million. The actual cost was US$144.77 million (ICR Data Sheet, page 2). The difference between the appraisal and actual cost was due to lower Borrower contribution (see below). Financing. The lending instrument for this project was Investment Project Financing (IPF).The project was financed through an IBRD loan worth US$100.00 million which was fully disbursed according to the ICR Data Sheet (page 2). Borrower Contribution. The Borrower was expected to contribute US$124.27 million of counterpart funding. The actual amount contributed was US$44.77 million. Dates. The project was approved on December 21, 2015 and became effective five months later on May 26, 2016. The Mid-term Review (MTR) was conducted on November 12, 2018 which was in line with expected date on November 30, 2018 as reported in the PAD. The project closed on December 31, 2022 which was 21 months after the original closing date on March 31, 2021. This 21 months extension was needed to "allow time to complete all activities under the project including the new activities added for Hebei Page 4 of 20 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Tianjin Urban Transport Improvement Proj (P148129) District" and "to allow time for completing activities delayed by the outbreak of COVID-19 (ICR, paragraph 20)." The project was restructured twice, both level 2, as follows: 1. On August 29, 2019, when the amount disbursed was US$18.46 million, in order to revise the PDO, revise the Results Framework, change components and cost, extend the Loan closing date by 15 months to June 30, 2022, reallocate funds between disbursement categories, change in disbursements arrangements, and change in implementation schedule. 2. On June 25, 2022, when the amount disbursed was US$68.79 million, in order to revise the Results Framework, change components and cost, extend the Loan closing date by another 6 months till December 31, 2022, and reallocate funds between disbursement categories. While the PDO was revised, this revision only increased the scope of the project to implement green transport improvement activities in Hebei. The three main elements of the PDO remained the same and were the basis for the assessment of outcomes. Therefore, this Review will assess the outcome of the project based on the revised PDO and the restructured outcome targets without the application of a split rating. 3. Relevance of Objectives Rationale  Context at Appraisal. Tianjin is one of the four Municipalities directly under the Central Government and a core growth area in the Beijing-Tianjin-Hebei (Jing-Jin-Ji) region. However, since the 1990s, a car-oriented city design paradigm that prioritized car movement over people’s mobility has dominated Tianjin urban planning and marginalized walking and biking. By the end of 2014, private car ownership had reached 2.4 million, a trebling since 2006, which worsened congestion. In addition, the road death rate in Tianjin was the ninth worst out of 32 province-level administrative units country-wide. This project aimed to increase active mobility, including walking and non- motorized traffic (NMT), and integrate it with the public transportation system. Maximizing the proportion of people practicing active mobility would result in substantial social, environmental, and economic benefits.  Previous Bank Experience. The Bank has solid experience in urban transport operations in major global metropolises, including large Chinese cities. Also, the Bank supports the urban transport program in China, including two previous urban development projects in Tianjin. This was the largest project in the World Bank’s history that fully focused on increasing active mobility and integration between active mobility and public transport, both in terms of financing allocated and the quantity of infrastructure implemented.  Consistency with Bank Strategies. At appraisal, the PDO aligned with the World Bank Group Country Partnership Strategy for China (CPS, FY2013-FY2016). The CPS focused on three main pillars: support greener growth, promote more inclusive development, and advance mutually Page 5 of 20 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Tianjin Urban Transport Improvement Proj (P148129) beneficial relations with the world. The project supported the CPS pillars for greener growth and inclusive development and the CPS sector objective of promoting low-carbon urban transport. The project also supported the World Bank’s goal of boosting shared prosperity. At completion, the PDO continued to align with the World Bank Group’s Country Partnership Framework for China (CPF, FY2020-2025). Specifically, the project promoted greener growth for the CPF engagement area 2.1. Facilitating the transition to a low carbon energy path and 2.5. promoting low-carbon transport and cities.  Consistency with Government Strategies/Priorities. At appraisal, the PDO aligned with China’s 12th Five-Year Plan. The Plan included major investments in urban infrastructure, with a significant portion to improve public transport. At completion, the PDO continued to align with China's National 14th Five-Year Plan for Modern Comprehensive Transport Development (2021-2025). The Plan emphasized green and low-carbon transport, safety improvement, and smart mobility. The project also directly influenced the design of the 14th Five-Year Development Plan for Tianjin Green Transportation (2021-2025) and the 14th Five-Year Plan for Tianjin Comprehensive Transportation (2021-2025). Summary of Relevance of Objectives Assessment. The PDO was clear and pitched at a relatively ambitious level. However, the third element of the PDO lacked relevant outcome-oriented indicators, and one PDO indicator had attribution concerns (see section 9a for more details). Overall, the Relevance of Objectives is rated Substantial. Rating Relevance TBL Rating Substantial 4. Achievement of Objectives (Efficacy) EFFICACY_TBL OBJECTIVE 1 Objective To make transport greener through leveraging the Tianjin metro system and promoting walking and biking in the urban core (in Heping, Hebei and Nankai). Rationale Theory of Change (ToC). To achieve the stated objective, the project supported the redevelopment of the streetscape in certain parts of Heping and Nankai, including creating an integrated pedestrian and bike network with infrastructure investments in street pavement updates, drainage improvements, street facilities, and landscape improvements. Also, the project supported civil works for selected intersection improvements and interconnection facilities at metro stations along existing metro lines. Measures included building bike parking, building/improving bus connection/terminal creating taxi connection, improving squares and parks with landscaping and building park and ride. The project also supported the selected development of bus terminals including civil works for the pavement of terminals, and bus stops, bus parking, car parking, bike parking and service buildings, and the purchase of equipment for bus operations within the bus terminals. The Page 6 of 20 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Tianjin Urban Transport Improvement Proj (P148129) expected outputs were: kilometers of urban street rehabilitated and open to traffic in Heping and Nankai pilot areas, an increase in NMT space ratio in Heping and Nankai pilot areas (%), and a percentage of Km with at least a 3-star rating for pedestrians for project streets in Heping, Hebeie, and Nankai pilot areas, the number of metro stations with access improved, number of bus terminals completed under the project. Intermediate outcomes included Increased walking and biking trips generated by the project in Heping and Nankai pilot areas (daily trips), a decrease in the number of reported accidents involving non-motorized vehicles or pedestrians in Heping and Nankai pilot areas, and an increase in the daily trips and metro ridership generated from metro access improvement in the central city area. The expected outcomes were to promote walking and biking in Tianjin's urban core area, to reduce GHG emissions (tCO2/year) as a result of project activities, and to leverage the Tianjin metro system. These outcomes reflected the objective of making transport greener. The achievement of the PDO was underpinned by the following three critical assumptions: 1. Timely approval of detailed designs; 2. Strong project management capacity; and 3. Strong city leadership support for recommendations. It is worth noting that component 3 (Public Bike Sharing System Pilot) was dropped after the private sector- financed dockless bike-sharing took off in the city in 2017. By 2020, the private sector dockless bike-sharing operators had 534,000 bikes operating in the city compared to the 12,370 bikes that were to be supported by the project (ICR, paragraph 64). The activities reflected in the ToC were directly connected to the outputs, intermediate outcomes, and outcomes in a plausible causal chain. The critical assumptions were logical and realistic. Outputs/Intermediate Results  132.19 km of urban streets were rehabilitated and open to traffic in the six districts where streetscape improvements were made, exceeding the revised target of 75 km.  216,000 square meters of bike lanes with red pavement coloring were constructed or rehabilitated (no target provided).  605,000 square meters of Sidewalks were constructed or rehabilitated (no target provided).  178,000 square meters of parks and squares were constructed or rehabilitated (no target provided).  43,000 square meters of sidewalk were landscaped with grass and small shrubs (no target provided).  3,541 sidewalk trees were planted (no target provided).  96 metro stations had their access improved, fully achieving the revised target.  NMT Space Ratio in Heping, Hebei, and Nankai pilot areas reached 52%, exceeding the revised target of 46%.  One bus terminal was completed under the project, which fully met the revised target but was below the original target of 5. In a further communication, the project team explained that “the savings not used on the bus stations were spent on doing additional urban street rehabilitation. The one bus station completed under the project consolidated many functions of the originally envisioned small stations: more bus parking areas, charging facilities, larger maintenance, and office facilities. Finally, as noted in the 2019 restructuring paper, the city has been developing additional bus stations in parallel with the project.”  Four innovative and influential studies were completed and endorsed: 1. Parking Management Improvement Scheme for the Core Zone of Tianjin (2016); 2. 2030 Sustainable Green Urban Transport Development Strategy for Tianjin (2018); 3. Decision Support Platform for NMT Facilities Page 7 of 20 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Tianjin Urban Transport Improvement Proj (P148129) and Environment Improvement (2021); and 4. Analysis of Short-Term Impact of Project on Tianjin Regional Economic Vitality (2022). Outcomes  By project completion, metro ridership (daily trips) generated from metro access improvement in the central city area reached 175,750, significantly exceeding the revised target of 95,000 (PDO outcome indicator 1). Also, walking and biking trips generated by the project in the Heping, Hebei, and Nankai pilot areas reached 153,613 daily trips, exceeding the revised target of 120,000 daily trips (PDO outcome indicator 2). The GHG emission reduction due to the project was estimated at 34,281 tCO2e/year, significantly exceeding the revised target of 7,500 tCO2e/year (PDO outcome indicator 3). The project met or substantially exceeded the revised end targets for all three PDO-level and intermediate-level indicators related to this objective.  Contributions of the project to making transport greener were mainly through improved infrastructure. The project improved 189 streets and 96 metro stations that were spread over much of the city's central core and are among the most socially and economically significant (ICR, paragraph 30). "All street improvement contributed to increased walking, biking, and metro trips (ICR, paragraph 30)." In addition, the project interventions enabled the integration with station entrances, including bus stops, bicycle parking areas, taxi stands, and public spaces surrounding the stations. Project interventions also enhanced the accessibility of the station coverage range, with particular attention to improving bicycle lanes and sidewalks.  Among the notable achievements of the project were 38 public parks, squares, and gardens built or upgraded.  The ICR noted that illegal parking in bike lanes in Hebei District was still widespread at project closing and represented a key short-term barrier to increasing green transport in Tianjin (ICR, paragraph 33). Summary of Efficacy Assessment. The project supported improvements in infrastructure and delivered four strategic technical studies. These project activities contributed to increased walking, biking, and metro trips. As noted above, by project completion, the project met or substantially exceeded the revised end targets for all three PDO-level and end targets for its intermediate-level indicators related to this objective. Therefore, the efficacy with which this objective was achieved is rated High. Rating High OBJECTIVE 2 Objective To make transport safer through leveraging the Tianjin metro system and promoting walking and biking in the urban core (in Heping, Hebei and Nankai). Rationale Theory of Change (ToC). The same ToC under objective 1 applies. Page 8 of 20 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Tianjin Urban Transport Improvement Proj (P148129) Outputs/Intermediate Results  82.4 km of traffic guardrail barrier was installed (no target provided).  284 streetlights were installed (no target provided).  100% of roads in project streets in Heping, Hebei and Nankai pilot areas had at least 3-star rating for pedestrians exceeding the revised target of 90% and the baseline at 30%. Outcomes  By project completion, the number of reported accidents involving non-motorized vehicles or pedestrians in Heping, Hebei and Nankai pilot areas reached 1,104 accidents per year which was an improvement over both the baseline of 1,201, as well as the revised target of 1,144 accidents per year.  While the above-mentioned result was encouraging, the continued prevalence of illegal parking in bike lanes, especially in Hebei District, continues to be a key short-term barrier to making transport safer in Tianjin. Illegal parking forces bike riders into the motor vehicle travel lane and increases their risk of being killed or seriously injured by someone driving a motor vehicle (ICR, paragraph 41). Summary of Efficacy Assessment. The project contributed to making transport safer in Tianjin through redevelopment of the street scape and improving bike lanes. The project also increased the number of people taking metro for their daily trips as reported under objective 1. The project also exceeded its end target for its PDO outcome indicator and intermediate results indicator pertaining to this objective. Therefore, the efficacy with which this objective was achieved is rated High. Rating High OBJECTIVE 3 Objective To draw lessons for other large cities. Rationale Theory of Change (ToC). To achieve the stated objective, the project provided technical assistance on sustainable green urban transport development, improvement of parking management schemes, and analysis of economic activity in the NMT zone supported by the project. The expected outputs included the number of TA studies completed and endorsed by the Tianjin government and several dissemination events promoting green strategies for large cities. This would contribute to drawing lessons for other cities, leverage the Tianjin metro system, and promote walking and biking in the core urban area of Tianjin. Anticipated long term impact was that the large majority of travel in large Chinese cities is done by green transport modes. The achievement of the PDO was underpinned by one critical assumption: strong city leadership support for recommendations. Page 9 of 20 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Tianjin Urban Transport Improvement Proj (P148129) The activities reflected in the ToC were directly connected to the outputs, and outcomes in a plausible causal chain. The critical assumptions were logical and realistic. The outcome indicator was relatively output- oriented, focusing on dissemination events rather than strategies informed by the lessons. Outputs/Intermediate Results  45 media articles were written about the project (no target provided).  6 TA studies were completed and endorsed by the Tianjin government (no target provided). Outcomes  By project completion,15 dissemination events were organized to promote green transport strategies for large cities, exceeding the revised target of 10 (PDO outcome indicator 6).  The ICR (paragraph 42) reported lessons were provided for other large cities in the following ways: 1. Seven peer-reviewed academic articles written about project-executed studies were published in Urban Transport of China journal; 2. 45 media articles were written about the project between 2019 and shared online17 websites; 3. The project influenced the design of the inaugural National Standards for Urban Pedestrian and Bicycle Transportation System Planning and Design, adopted on October 1, 2021; 4. Tianjin was invited to give presentations at many domestic conferences on the experiences of the project, including the World Transport Convention and the China Urban Planning Annual Conference; 5. The Tianjin Urban Planning and Design Institute is midway through drafting a detailed Chinese-language case study on the lessons learned from the project, aiming to be circulated to transport professionals within Tianjin and China more broadly by December 2023; 6. Tianjin is currently a finalist for the Sustainable Transport Award (STA) for 2024. The STA is the world’s most prestigious award, recognizing achievements in sustainable transport; and 7. The China Academy of Transportation Science (CATS) is in the process of establishing an “Urban and Rural Active Mobility Development Alliance.” Summary of Efficacy Assessment. The project exceeded its end target for the PDO outcome indicator and the end target for the intermediate results indicator pertaining to the objective. The project experience also provided notable lessons that will benefit other large cities in China. The outcome indicator was relatively output-oriented with a focus on dissemination. However, the ICR provides some evidence of the influence of the lessons on the national standards for Urban Pedestrian and Bicycle Transportation System Planning and Design. The ICR notes that the project had intended to hold a high-level workshop at the end of December 2022, inviting transport officials from other Tier 1 Chinese cities, relevant agencies within Tianjin, and private sector stakeholders. However, this workshop could not go ahead due to the country-wide COVID-19 outbreak. The overall efficacy with which this objective was achieved is rated Substantial. Rating Substantial OVERALL EFF TBL Page 10 of 20 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Tianjin Urban Transport Improvement Proj (P148129) OBJ_TBL OVERALL EFFICACY Rationale Overall Efficacy is rated High. The achievement of the first two objectives was rated as High and the third objective as Substantial. The project delivered more infrastructure than anticipated over a wider geographic area and reported relevant lessons that would inform city level and national level policies. Overall Efficacy Rating High 5. Efficiency Economic and Financial Analysis (EFA) ex-ante  The project's economic internal rate of return (EIRR) was calculated by comparing the economic costs and benefits over 25 years, including 5 years of construction and 20 years of operation. The project had an estimated EIRR of 17.6%, which was higher than the World Bank recommended economic opportunity cost of capital (12%). The EIRR for the additional Hebei District scope added at the 2019 restructuring was 16.4%.  The economic benefits considered were (i) savings in passenger travel time cost, (ii) reduction in vehicle operation cost, (iii) reduction in emissions and environment improvement, (iv) reduction in vehicle accidents, and (v) savings in bus operation and maintenance cost.  Sensitivity Analysis. A sensitivity analysis was carried out and showed that EIRRs for the tested cases were higher than 12%; however, if capital and O&M costs are both increased by 20%, and benefits were concurrently decreased by 20%, the EIRR would drop to an estimated 10.9%. ex-post  The ex-post analysis was broadly consistent with the ex-ante economic analysis methodology to ensure consistency and comparability. The ex-post analysis for the Heping, Hebei, and Nankai district scope was estimated at 48.8%. Incorporating the additional NMT improvements done in the Hedong, Hexi, and Hongqiao districts, the project EIRR was 53.54%.  The analysis assumed a construction period of 2016 to December 2022, with economic benefits starting from 2023. The project economic analysis evaluation period was through to 2040. The discount rate used was 12%.  The economic benefits were generated by considering seven factors: (i) travel time savings, (ii) reduced travel costs from mode shift, (iii) reduced travel costs for motor vehicle drivers, (iv) reduced traffic accidents, (v) reduced emissions, (vi) time savings from upgraded bus stations, and (vii) reduced bus maintenance costs. Page 11 of 20 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Tianjin Urban Transport Improvement Proj (P148129)  Several benefits were not considered in the analysis, including reductions in noise and air pollution, savings from fewer premature deaths, and healthcare costs. Therefore, the estimated EIRR was assumed to be conservative.  While the analysis justified the project investments, it could have benefitted from a more holistic economic analysis informed by best practices and not just use the motor vehicle-centric benefit and cost flows (ICR, Annex 4). Implementation Efficiency. The project closed on December 31, 2022, which was 21 months after the original closing date on March 31, 2021. According to the ICR (paragraph 20), additional time was needed to complete all activities under the project, including the new activities added for Hebei District and to accommodate for delays that resulted from the outbreak of COVID-19 and the related movement restrictions and supply chain issues. The project also experienced implementation delays that stemmed from securing institutional consensus on the new urban planning and design approach being piloted by the project (ICR, paragraph 47). The ICR (paragraph 80) noted that competitive bidding generated considerable project cost savings of around US$100.23 million. The savings enabled the construction of 82.19 kilometers beyond what was envisaged at appraisal. Despite the delays, NMT infrastructure improvement works were applied to all six districts of the Tianjin urban core, rather than the two districts envisaged at appraisal and three envisaged at the first restructuring. These additional works substantially improved the economic return on investment (ICR, paragraph 47). Summary of Efficiency Assessment. The EIRR at completion was significantly higher than the one estimated at appraisal (53.54% compared to 17.6%). The project was also completed at a lower cost than envisaged at appraisal. The implementation delays were justified overall due to the project scope expansion and the COVID- 19 restrictions. Therefore, efficiency is rated High. Efficiency Rating High a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 0 Appraisal  17.60  Not Applicable 0 ICR Estimate  53.54  Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome Relevance of Objectives was rated Substantial. Overall Efficacy was rated High. The project met or exceeded all PDO and intermediate-level indicators. It also delivered more infrastructure than anticipated over a wider Page 12 of 20 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Tianjin Urban Transport Improvement Proj (P148129) geographic area and reported relevant lessons that would inform city-level and national level policies. Efficiency was rated High. The EIRR at completion was significantly higher than the one estimated at appraisal (53.54% compared to 17.6%). Based on the assigned rating to the three Outcome criteria, the Outcome is rated Highly Satisfactory. a. Outcome Rating Highly Satisfactory 7. Risk to Development Outcome The following risks could potentially impact the Development Outcome: 1. Government/Stakeholders commitment risk. This risk is dependent on the city's intention to follow a green transport development path, following the recommended strategies of the project financed 2030 Green Transport Development Strategy study. The recommendation of the strategy is substantively included in the 14th Five-Year Development Plan for Tianjin Green Transportation (2021-2025) and the 14th Five-Year Plan for Tianjin Comprehensive Transportation (2021-2025). Also, there is solid and growing support for active mobility from the central government. As a result, the short-term risk to the development outcome is considered low. However, in the short term, Tianjin needs to reduce the prevalence of illegal parking on sidewalks and bike lanes. Also, in the long term, establishing strict policies to disincentivize car ownership, use, and on-street parking is critical to safeguard public space for active mobility (ICR, paragraph 91). 2. Institutional risk. The handover arrangements of the project's assets to local districts are almost complete. This process follows rigorous requirements before a district will accept the new infrastructure to be under its control. As of June 18, 2023, 120 out of 189 roads were fully handed over to the local district administrative agency and had operations and maintenance plans in place. The PMO will continue to facilitate the handover of the remaining sites (ICR, paragraph 92). 8. Assessment of Bank Performance a. Quality-at-Entry  Strategic relevance and approach. This was the largest project in the World Bank’s history that fully focused on increasing active mobility and integration between active mobility and public transport, both in terms of financing allocated and the quantity of infrastructure implemented. The PDO aligned with the Bank's strategies and Government priorities (see section 3 for details).  Technical, financial, and economic aspects. The project was designed with ambitious objectives, clearly structured components, and limited acts to procure. Design was informed by a detailed Feasibility Study financed by the counterpart. However, the project did not include enough emphasis on why and how to reduce speed limits in central Tianjin to complement the streetscape improvements made to reprioritize NMT. The PAD included an economic and financial analysis that provided adequate justification for the project investments. Page 13 of 20 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Tianjin Urban Transport Improvement Proj (P148129)  Poverty, gender, and social development aspects. The project benefited those in poverty and the bottom 40% of household income by improving the modes they used most frequently to access jobs and services. Also, streetscape improvements were associated with significant increases in the number, density, quality, popularity, and average consumption of stores (ICR, paragraph 56). The project tracked indicators disaggregated by gender, and the design incorporated the needs of less-able bodied persons in the infrastructure design.  Environmental and fiduciary aspects. The project design included appropriate environmental and social risk assessments with adequate mitigation strategies. The design featured adequate procurement and financial management (FM) arrangements.  Implementation arrangements. The PMO has had a long history of working on World Bank financed projects. However, the PMO had insufficient technical capability in streetscape renewal and construction contract management, and it also lacked enough staff to manage the project effectively (ICR, paragraph 63). This lack of capacity could have been identified in project preparation and mitigated.  Risk assessment. The overall risk to achieving the PDO was rated Substantial. The risks related to the project and institutional capacity for implementation and sustainability were all rated substantial. Fiduciary and safeguards risks were considered Moderate given the extensive prior experience of the PMO. While key risks were well identified, the risk related to the impact of illegal parking was underestimated.  M&E arrangements. The PMO was responsible for coordinating the relevant agencies in collecting the required M&E data. M&E design had shortcomings in the methodology, attribution, and linkage to the PDO and project activities (see section 9a for details). Summary of QAE Assessment. The project was strategically relevant, with ambitious objectives. The design was relevant and reflected adequate environmental and fiduciary aspects. Implementation arrangements could have benefitted from addressing the lack of capacity at the PMO. While key risks were well identified, the risk related to the impact of illegal parking was underestimated. Finally, M&E arrangements were adequate, but M&E design had shortcomings. Overall, Quality at Entry is rated Satisfactory. Quality-at-Entry Rating Satisfactory b. Quality of supervision  The Bank conducted 13 supervision missions over the implementation period. On average, two missions were carried out each year, while virtual missions were carried out during COVID-19 restrictions. The final mission in December 2022 included a half-day virtual site visit to more than 30 sites around the city, with detailed information about the work done at each site and live video footage of the site.  The project implementation benefitted from extensive technical guidance by the task team, including overall project management approach, detailed design, strategic study outputs, regulation of share-bikes, post-construction quality auditing, and operations and maintenance phase procedures (ICR, paragraph 86). Page 14 of 20 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Tianjin Urban Transport Improvement Proj (P148129)  The ICR (paragraph 86) reported that the Bank task team and the PMO worked well together and maintained rapid and smooth communication throughout the project. Also, reporting by the task team to World Bank management was provided in a timely manner.  However, the Bank team could have worked on executing the second project restructuring slightly earlier. The PMO requested to extend the project in September 2021. However, this request was fulfilled nine months later, in June 2022. In addition, the second restructuring did not reflect changes to the PDO and results framework to account for the additional streetscape improvements being done through component 2. At project completion, through component 2, the project completed 75.52 kilometers of streetscape improvements on 107 streets accessing metro stations in the central Tianjin districts of Hebei, Hedong, Hexi, and Hongqiao. However, the PDO only specified Heping, Hebei, and Nankai as the districts where walking and biking would be promoted. Correspondingly, several Results Framework indicators and end targets should have been revised to reflect the increased scope of the project. Summary of Bank Supervision Assessment. The Bank team successfully guided the project to achieve its PDO. At completion, NMT streetscape improvements were implemented in six districts, when at appraisal, it was only envisaged that they would be done in two districts. However, the Bank team could have worked on executing the second project restructuring slightly earlier. Overall, Bank Supervision is rated as Satisfactory. Based on the assigned rating for QAE and Bank Supervision, Bank Performance is rated Satisfactory. Quality of Supervision Rating Satisfactory Overall Bank Performance Rating Satisfactory 9. M&E Design, Implementation, & Utilization a. M&E Design  The PAD did not include a Theory of Change (ToC) since it was not required by the Bank at the time of appraisal. Nonetheless, the ICR (figure 2) included a ToC constructed based on the detailed project description in the PAD and Results Framework. The ToC in the ICR clearly reflected the relationship between the project activities, outputs, and outcomes in a plausible causal chain. The ToC also included the critical assumptions that underpinned the achievement of the PDO.  The achievement of the PDO was measured by five PDO outcome indicators: 1. Increase in metro ridership with the project compared to without the project (trips); 2. Increase in volume of non- motorized trips by project, including Public Bike Sharing trips and increase in walking and biking trips by the project in Heping and Nankai (trips); 3. GHG emission reduction (tons); 4. Reduction in number of accidents in Heping and Nankai pilot areas (accidents); and 5. System of detailed performance evaluation with citizen engagement in place (number of sites).  The PDO indicators were connected to the stated objectives, measurable and reflected realistic targets. However, the PDO-level indicator “Reduction in number of accidents in Heping and Nankai Page 15 of 20 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Tianjin Urban Transport Improvement Proj (P148129) pilot area (accidents/year)” had attribution issues, given it could be influenced by several factors beyond the control of the project. Also, a key weakness of the Results Framework (RF) design was that it did not have outcome indicators to measure the achievement of objective 3 “to draw lessons for other large cities.” Finally, the ICR (paragraph 70) correctly noted that PDO-level indicators 1 and 2 had a methodological problem related to the small sample size, which might not generate representative data.  The RF included 8 intermediate results indicators (IRIs) to track the progress of the different project activities. The IRIs were measurable and connected to the stated activities and had reasonable targets. However, the intermediate-level indicator “Percentage of km with at least 3-star rating for pedestrians for project streets in Heping, Hebei, and Nankai pilot areas" also used a small sample size. In addition, the RF lacked equivalent indicators for bicyclists. Finally, the RF lacked indicators to track the percentage of km of bike lanes constructed or upgraded that have physical protection from motor vehicle traffic despite the importance of physical protection for the road safety of bike riders (ICR, paragraph 71).  Overall, the M&E design was adequate but could have benefitted from better refinement of the methodology, outcome indicators for the third objective, attention to attribution, and coverage of all activities supported by the project, as discussed above. b. M&E Implementation  The PMO oversaw the overall M&E implementation. Data was diligently collected for the indicators following the methodology set ex-ante (ICR, paragraph 72).  The project conducted additional monitoring and evaluation of the project’s contribution to greening transport in Tianjin that went beyond the requirements of reporting on the Results Framework. This was possible through two of the component 4 technical assistance studies, where the project collected empirical data on the impact that changes in the active mobility environment had on travel behavior and the economic vitality of the street.  Revision of the RF. The targets of the PDO indicators were revised to reflect the increased scope of the project, and the indicator on biking was dropped after this activity was supported by the private sector. In both the first and second restructurings, changes were made to the Results Framework to reflect a change in the project scope and to further improve understanding of the project context. The PDO indicator tracking the “Reduction in number of accidents in the Heping and Nankai pilot area” was revised to track the number of reported accidents involving non- motorized vehicles or pedestrians, instead of all reported accidents, to reflect better the project impact on the safety of non-motorized vehicles and pedestrians. Overall, the changes improved the quality of the design of the project M&E. c. M&E Utilization  The Results Framework was used by the task team to monitor the project progress. However, the ICR did not provide specific examples on how M&E data was used in project management.  The PMO produced a detailed counterpart Implementation Completion Report utilizing data collected throughout the project. The PMO also provided World Bank with the raw data and Page 16 of 20 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Tianjin Urban Transport Improvement Proj (P148129) spreadsheets used to calculate Results Framework indicators so World Bank could verify independently. Summary of Quality of M&E Assessment. M&E design had some moderate shortcomings. M&E implementation was adequate, and several design shortcomings were addressed through the two project restructurings. Utilization was adequate as the project data informed the ICR, but there was no evidence provided on how the M&E data helped the in management decisions. Overall, M&E Quality is rated Substantial. M&E Quality Rating Substantial 10. Other Issues a. Safeguards  The project was classified as an environmental Category B as environmental and social impacts were expected to be site specific, temporary in nature, and can be readily mitigated with good construction management and known mitigation measures. The project triggered two environmental safeguards: Environmental Assessment (OP/BP 4.01) and Physical Cultural Resources (OP/BP 4.11), and one social safeguard: Involuntary Resettlement (OP/BP 4.12).  An Environmental Impact Assessment (EIA) and a stand-alone Environmental Management Plan (EMP), a Social Assessment (SA) and a Resettlement Policy Framework (RPF) were prepared for the project as per the requirements of Bank. The EIA and the RFF were disclosed in Infoshop on September 30, 2015. The EMP was disclosed in Infoshop on October 12, 2015. Compliance with Environmental Safeguards. Environmental protection measures were implemented on the project sites. This was confirmed by field observations of supervision missions and the regular independent environmental monitoring reports prepared by Nankai University. "There were no legacy environmental safeguards issues identified upon project completion" and "environmental safeguards performance was rated Satisfactory (ICR, paragraph 79)." Compliance with Social Safeguards. Implementation of the resettlement policy framework was satisfactory, with no land acquisition related to the project. One of the bus terminals planned for improvement at appraisal, the Nankai Qingnianlu Bus Terminal, was removed from the project scope at the 2019 restructuring as there was an owner of an illegal parking lot that did not give consent for the parking lot area to be used during construction of the bus terminal. Overall, safeguards compliance was rated as Satisfactory at closing. b. Fiduciary Compliance Page 17 of 20 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Tianjin Urban Transport Improvement Proj (P148129) Financial Management (FM). The project FM arrangements and performance were both satisfactory. FM conformed with World Bank requirements (ICR, paragraph 81). Most of the interim financial reports (IFRs) were of acceptable quality, unmodified (clear) audit opinion, and were timely submitted to the World Bank. However, some IFRs were delayed due to COVID-19 restrictions and the change of PMO staffing. FM performance had minor deficiencies, mainly due to delayed project implementation progress, slow disbursement, and some inaccurate project accounting. These deficiencies were addressed by the project in a manner acceptable to the World Bank. Procurement. Procurement was rated Satisfactory, with no significant procurement or contract management issues. Procurement activities followed the World Bank procurement guidelines (ICR, paragraph 80). Procurement processes were completed without significant delays. Competitive bidding generated considerable project cost savings of around US$100.23 million (ICR, paragraph 80). c. Unintended impacts (Positive or Negative) "The streetscape improvements were also found to make the streets more attractive for small businesses. Through a study23 financed by the project, it was found that the streetscape improvements were associated with significant increases in the number, density, quality, popularity, and average consumption of stores (ICR, paragraph 56)." d. Other None. 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment Outcome Highly Satisfactory Highly Satisfactory Bank Performance Satisfactory Satisfactory Quality of M&E High Substantial M&E design shortcomings. Quality of ICR --- High 12. Lessons The ICR included four lessons. The following three are emphasized with some adaptation of language: 1. Shifting the dominant urban planning paradigm from car-centric to green mobility- prioritizing development requires buy-in from stakeholders and enough time to materialize. Page 18 of 20 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Tianjin Urban Transport Improvement Proj (P148129) There was a lot of resistance from project stakeholders to changing the urban planning and design status quo, which was the main factor leading to the delays experienced in the first few years of the project. Five years is too short a project time frame to create innovative streetscape designs for large sections of the core urban area, get consensus on these designs and permits to construct from the relevant stakeholders, get the works built and handed over, and establish new policies and procedures for operations and maintenance. Projects of this type should also have a powerful steering committee, including heads of all relevant organizations, led by the Mayor’s Office. The steering committee should meet frequently throughout the whole project lifecycle to adequately address any implementation issues. 2. To ensure the success of active mobility infrastructure improvements, strict parking management measures need to be simultaneously enforced. Without strict parking management, motor vehicle parking will inevitably increasingly encroach on active mobility infrastructure and reduce the usage rates and safety of this infrastructure. World Bank should include motor vehicle parking management reform, led by the city mayor or head of the municipal transport department, as a standard feature in projects seeking to increase walking and biking. 3. Projects seeking to make cities more friendly for active mobility should take a holistic, sociotechnical systems thinking-based approach rather than focusing overwhelmingly on infrastructure provision. Quality infrastructure is a necessary but insufficient precondition to a city being a place where most people regularly engage in active mobility. While the activities supported under this project significantly improved the friendliness of the city infrastructure, the project only did analytical work with no direct impact on the other indicators of friendliness in terms of the extent to which city management, services, and culture are friendly towards active mobility. The World Bank should take a holistic approach to future projects globally. 13. Assessment Recommended? No 14. Comments on Quality of ICR Quality of Evidence and Analysis. The ICR provided an adequate evidence base to support the achievements reported. The ICR provided clear linking between evidence and findings and used the evidence base to serve the arguments under the different sections, in particular, the discussion on outcomes. However, the assessment of the third objective was not comprehensive enough due to M&E design weaknesses. Lessons. Lessons reflected the project experience and were based on evidence and analysis. Results Orientation. The ICR included an adequate discussion on the achievement of the PDO. The outcome discussion was well balanced between what the project actually achieved on the ground and the achievement of outcome indicators. Page 19 of 20 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Tianjin Urban Transport Improvement Proj (P148129) Consistency with guidelines. The ICR used the available data to justify most of the assigned ratings. Reporting on safeguards lacked an explicit statement on the compliance of the project with the Bank's safeguard policies. Conciseness. The ICR provided adequate coverage of project activities and candidly reported on most shortcomings in a concise form. Summary of the Quality of ICR Assessment. The ICR was well-written and supplemented with data-rich Annexes. It included an adequate discussion on the achievement of outcomes and reflected relevant lessons. Most sections were concise and reflected relevant information. Overall, the Quality of the ICR is rated High. a. Quality of ICR Rating High Page 20 of 20