Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Croatia Railway (P147499) Report Number: ICRR0023998 1. Project Data Project ID Project Name P147499 Croatia Railway Country Practice Area(Lead) Croatia Transport L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IBRD-85000,IBRD-85010,IBRD-85020 31-May-2020 183,670,200.93 Bank Approval Date Closing Date (Actual) 30-Apr-2015 31-Jul-2023 IBRD/IDA (USD) Grants (USD) Original Commitment 183,350,000.00 0.00 Revised Commitment 183,350,000.00 0.00 Actual 183,670,200.93 0.00 Prepared by Reviewed by ICR Review Coordinator Group Maria Shkaratan Avjeet Singh Avjeet Singh IEGSD (Unit 4) 2. Project Objectives and Components DEVOBJ_TBL a. Objectives The Project Development Objective (PDO) was “to improve the operational efficiency and the financial sustainability of the public rail sector in the Guarantor’s territory” (Legal Agreement, page 6). The PDO was stated almost the same way, with a slightly different wording, in the Project Appraisal Document (PAD): “to improve the operational efficiency and the financial sustainability of the public railway sector in Croatia” (PAD, page iii). Page 1 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Croatia Railway (P147499) The PDO was not revised. For the purposes of this Implementation Completion and Results Report (ICR) review, the objective will be assessed as follows: PDO 1: To improve the operational efficiency of the public railway sector in Croatia. PDO 2: To improve the financial sustainability of the public railway sector in Croatia. b. Were the project objectives/key associated outcome targets revised during implementation? Yes Did the Board approve the revised objectives/key associated outcome targets? Yes Date of Board Approval 15-Jun-2019 c. Will a split evaluation be undertaken? Yes d. Components 1. Original components At appraisal, the Project had four components, each supporting one of the four beneficiary entities: Ministry of the Sea, Transport and Infrastructure of Croatia (MSTI, former MMATI); Croatian Railway Passenger Company (Hrvatske željeznice Putnički prijevoz or HZPP); Croatian Incumbent Railway Freight Operator (Hrvatske željeznice Cargo or HZC), and Croatian Railway Infrastructure Manager (Hrvatske željeznice Infrastruktura or HZI). Component 1 Project Coordination and Sector Policy Support (cost at appraisal: US$2.9 million; actual cost: US$7.0 million) was to support MMATI (later MSTI) in improving its institutional capacity as the railway sector operator and in strengthening sector governance. The focus would be on the following: (i) asset management, and (ii) project coordination and sector policy. Component 2 Support to HZPP Restructuring (cost at appraisal: US$58.7 million; actual cost: US$65.0 million) was to support HZPP in improving its operational efficiency and financial sustainability. This would include the following: (i) work force retrenchment: retroactive financing of eligible severance payments, financing the implementation of the HZP’s retrenchment plan for 2015-2018, and supporting retraining and alternative employment; and (ii) implementation of the restructuring plan: rehabilitating the rolling stock to improve fleet efficiency, IT modernization and integration, studies, measures to adapt business processes, and capacity building in management and restructuring. Component 3 Support to HZC Restructuring (cost at appraisal: US$53.4 million; actual cost: US$53.4 million) was to support HZC in improving its operational efficiency and financial sustainability. This would include the following: (i) work force retrenchment: retroactive financing of eligible severance payments, Page 2 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Croatia Railway (P147499) financing of the implementation of the HZC’s retrenchment plan for 2015-2017, and supporting retraining and alternative employment support; and (ii) implementation of the restructuring plan: periodic maintenance of the rolling stock, IT modernization and integration, and capacity building in management and restructuring. Component 4 Support to HZI Restructuring and Enabling Investment to Increase the System Efficiency (cost at appraisal: US$85.7 million; actual cost: US$93.1 million) was to support HZI in improving its operational efficiency and financial sustainability. This would include the following: (i) work force retrenchment: financing the implementation of the HZI’s retrenchment plan for 2015-2019, and retraining and alternative employment support; (ii) installation of railway safety crossings; (iii) improvement of structures presenting safety risks, and reinforcing areas prone to landslides or rock falls; (iv) emergency infrastructure investments, mostly rehabilitation of track and electrical systems along critical corridors; and (v) capacity building in management and restructuring. 2. Changes during implementation Changes in costs: - At the first restructuring of June 2018, the cost of components 1, 2, and 4 grew, in line with the increased scope of activities (please see section Change in Components below). The overall Project cost increased from the appraisal level of US$200.8 million to US$217.2 million, while the IBRD funding remained the same. - At the fourth restructuring in November 2021, the cost of component 4 was raised by US$2.2 million, leading to the increase of the total Project cost from US$217.2 million to US$219.4 million, while the IBRD funding remained the same. There was also a reallocation of funds from component 1 to component 4. - At the fifth restructuring, at government request, unused loan funds were reallocated to the rehabilitation of the Zagreb – Savski Marof railway line. - During the period of 2020-2023, the Project was affected by inflation, which was first caused by the COVID-related supply chain disruptions and later by the factors linked to the Russia’s invasion of Ukraine. This had a negative impact, leading to a non-achievement of some of the outcomes. (ICR, page 11) Change in indicators: - At the first restructuring of June 2018, the main indicator targets, including the target for one PDO indicator, were reduced, signaling a reduced Project ambition. At the same time, the staff retrenchment targets were increased, and there were changes in two indicators with no impact on Project scope or ambition. Specifically: o The following targets were reduced:  For the PDO indicator “Number of annual train-km operated per HZI employee”: from 5,015 to 3,834.  For the intermediate results indicator (IRI) “HZP commercial revenues per employee (Croatian Kuna or HRK)”: from 0.3 to 0.21.  For the IRI “Number of HZC rolling stock rehabilitated (Number)”: from 2,820 to 2,166. Page 3 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Croatia Railway (P147499)  For the IRI “Number of kms of line sections rehabilitated or improved (Kilometers)”: from 338 to 306.25. o The three targets for staff retrenchment were increased. o The indicator “Passenger wagons rehabilitated (Number)” was replaced by the indicator “HZP four new Diesel Multiple Unit (DMU) trains (Number)” because it was decided to invest in new trains instead of rehabilitating the old ones. This change did not entail a reduction in scope or ambition. o The IRI “HZI million train-km per track-km” became a PDO indicator. - At the second restructuring of June 2019, some of the main indicator targets were reduced, and some indicators were replaced by less ambitious ones, signaling a reduction in Project ambition. Importantly, the scope/ambition of two PDO indicators was reduced. Specifically: o The following targets were reduced:  For the PDO indicator “HZI million train-km per track-km (Number)”: from 9,063 to 8,300.  For the IRI “Percentage of assets with resolved legal titles (Percentage)”: from 40 to 30.  For the IRI “Locomotives rehabilitated (Number)”: from 16 to 11. o One IRI target for staff retrenchment was increased, and the PDO indicator “Number of annual train- km operated per HZI employee (Number)” (with the number of staff in the denominator, which was decreased as a result of Project activities, pushing up this result) had a target increase from 3,834 to 4,500. o The PDO indicator “HZC viability ratio”, a quantitative measure of railway efficiency (revenue divided by costs) was replaced by a less ambitious non-quantitative PDO indicator “Completion of a strategic partnership for HZC (transaction finalized) (Yes/No)”. o The PDO indicator “Prioritized railway sector development and infrastructure maintenance program developed (Yes/No)” was replaced by the PDO indicator “Introduction and signing of the multi- annual infrastructure contract (Yes/No)”. This specific change did not constitute a reduction in Project ambition; it was needed because the activity to develop the above program was replaced by the activity of introducing and signing a multi-annual infrastructure contract, which was an EU requirement not yet met by Croatia (Restructuring paper, page 6). - At the third restructuring of August 2020, the main changes were that several indicators were dropped and new indicators were added: o The following indicators were dropped:  The PDO indicator “HZI million train-km per track-km” was dropped. The explanation was that the numerator (train-km) reflected demand for rail services, while the denominator reflected the size of the network, and both are outside of Project control (Restructuring paper, page 10).  The IRI “HZP commercial revenues per employee (Croatian Kuna or HRK)” was dropped. The explanation was that the numerator reflected the demand for rail services and the pricing of transportation, and both factors are outside of the Project’s control.  “Percentage of affected workers who participated in the social mitigation programs reporting satisfaction with the programs including gender disaggregated data” was dropped. Page 4 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Croatia Railway (P147499) o The following indicators were added:  IRI for Citizen Engagement “Customers reporting perception of service improvement over past two years (Percentage)”.  IRI “Preparatory work for centralized traffic control and signaling modernization (Yes/No)” (to monitor a new Project activity: the first phase in preparatory work to develop Croatia’s national program of centralizing traffic control and signaling). o The target for the IRI indicator “Number of HZP staff retrenched (Number)” was reduced from 771 to 376. o The target of the IRI “HZP four new DMU trains (Number)” was increased from four to five. - At the fourth restructuring of November 2021, the PDO indicator “Introduction and signing of the multi- annual infrastructure contract (MAIC)” was revised to “Preparation of the draft MAIC”. The Restructuring paper stated that this change was due to a delay in the preparation of the infrastructure strategy and action plans, which would form the basis for the MAIC. (Restructuring paper, page 7) Change in components: - At the first restructuring of June 2018 (level 2), the scope of components 1, 2, and 4 increased, while the scope of component 3 remained the same. The modifications in components were implemented at government request as a result of changes in government priorities (Restructuring paper, page 4, paragraph 1). The following changes were made: (i) acquiring new DMU trains instead of rehabilitating old locomotives in component 2; (ii) rehabilitating the railway line Savski Marof – Zagreb instead of rehabilitating several smaller lines in component 4; (iii) strengthened Project coordination/capacity support to MSTI and additional consultancy support for financial restructuring of HGC (component 1); (iv) additional consultancy support for the operational restructuring of HZI (component 4); and (v) (component 1). e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project Cost: The appraisal estimate was US$200.7 million, and the actual disbursement was US$218.5 million. Project Financing: The Project was financed by three IBRD loans; all three were original loans (originated at appraisal). The total amount was US$183.35 million at appraisal and US$183.7 million at closure. For the first loan, the appraisal estimate was US$88.7 million and the actual amount at closure was US$88.1 million. For the second loan, the original amount was 46.5 million, and the actual amount at closure was US$46.9 million. For the third loan, the original amount was US$48.2 million, and the actual amount was US$48.7 million. Borrower/Recipient contribution: The Borrower’s contribution was expected to amount to US$17.35 million at appraisal, and actual Borrower contribution was US$34.8 million. Project Dates: The Project was approved on April 30, 2015 and became effective on August 4, 2015. The MTR review was on February 12, 2019. The Project was restructured five times: (i) on September 24, 2018; (ii) on June 15, 2019; (iii) on August 12, 2020; (iv) on November 3, 2021; and (v) on July 10, 2023. The Page 5 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Croatia Railway (P147499) original closing date was May 31, 2020. The Project was extended two times, for the total of three years and two months (or 38 months), to July 31, 2023, which was the date of the Project’s actual closure. Split evaluation. There were changes in Project’s indicators and indicator targets throughout implementation and especially at the second restructuring (June 2019) that signaled reduced Project ambition: the main indicator targets were lowered, and some PDO indicators were replaced by less ambitious ones. Importantly, this included reduced scope/ambition of PDO indicators, which triggers a split evaluation, with the Revised Project defined by the second restructuring. The fourth restructuring also narrowed the ambition by replacing the PDO indicator “Introduction and signing of the multi-annual infrastructure contract (MAIC)” to “Preparation of the draft MAIC”. However, since the revised indicator was output oriented and would not affect the outcome rating (also given the low share of disbursement after the fourth restructuring i.e. 8.2%) , a split evaluation has been conducted only for the second restructuring. 3. Relevance of Objectives Rationale Country and Sector Context. Croatia became a member state of the European Union (EU) in 2013, and this provided many opportunities to deepen reforms, including in the transport sector. As part of this process, the Croatian Railways Holding company was split into three independent companies under the umbrella of MSTI: HZI, HZC, and HZPP. However, the regulatory framework to ensure competition and access to new operators had not been established yet. Financial performance of the Croatian railway system was weak, and the budget support amounted to 1.2 percent of GNP annually. The EU allocated EUR 2 billion to the railway sector and secured the financing until 2027 in order to upgrade the assets and develop international corridors in the TEN-T network. However, more support was needed. The Project sought to complement the EU investments and support the restructuring of the three new companies to keep the railway sector operational. Relevance to the Government Strategies at approval. The Government’s interim transport strategy was adopted in October 2014, giving a very strong orientation towards railway compared to other modes. Railway represented a significant share of traffic and had a strong potential. In 2013, Croatia railway’s share of both cargo and passenger traffic remained above the EU average, with 12 percent of goods traffic and 31 percent of passenger public transport. The Government implemented a major railway sector reform in 2012, compliant with the EU Directives. In June 2012, MMATI adopted a five-year restructuring plan for Croatian Railways Holding, splitting it into three independent entities, limited liability companies owned by the Republic of Croatia - HZI, HZC, HZPP, - and the holding company was dissolved. Following the reform of 2012, the focus of the Government was on the companies’ restructuring. In this context, the WBG could play a key role in helping the Government reach its objectives while complementing the actions and financing of the European Institutions. Relevance to Government Strategies at closure. The Transport Development Strategy of the Republic of Croatia (2017-2030), which was published in 2017 as “a complete national strategic document based on comprehensive traffic data and national traffic model”, underscored that while the the quality of Croatia’s road network was among the world’s best, the quality of the railways network was much lower and needed to be improved. However, the railways infrastructure investment in Croatia had been decreasing since 2014, and modernization of rolling stock was not sufficient. Therefore, railways improvements were a Page 6 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Croatia Railway (P147499) priority of the Strategy, and the two top general objectives (GO) were as follows: (i) a modification of the passenger modal split in favor of passenger transport, including rail (GO1); and (ii) the modification of the freight modal split in favor of rail and water transport. The Strategy also stated seven specific objectives for rail transport, and four of them were addressed by the Project, specifically: (i) to better utilize the railway system in the main Croatian agglomerations and within and between functional regions; (ii) to improve the efficiency of the rail system; (iii) to improve the environmental impact of rolling stock; and (iv) to improve the safety at level crossing with roads. Relevance to the World Bank Group's (WBG’s) Assistance Strategies at approval. The Project was aligned with the WBG’s Country Partnership Strategy (CPS) for FY2014-17. The CPS stated that “Modernized public enterprises with improved governance and performance along with a reduction of the logistics costs in infrastructure (e.g. railways, energy, and information technology) is needed to accomplish Croatia's two edged goal of fiscal responsibility and public sector service delivery.” (CPS, page 8) The Project was aligned with the CPS’s Pillar 2 Competitiveness, sub-pillar Strengthening Public Enterprises , which has a focus on the “structural reforms aimed at the Croatian Railways (HZ) system” (CPS, pages 18- 19). Relevance to the WBG’s Assistance Strategies at closure. The Project was aligned with the WBG’s Country Partnership Framework (CPF) for Croatia FY2019-24, specifically Focus Area 1 Enhancing Public Sector Performance And Institutions, Objective 2 Support Policies that Reduce Fiscal Vulnerabilities, with a Focus on State-Owned Enterprises. The CPF stated that the Bank’s support to SOE restructuring in the transport sector focused on rationalizing expenditures and improving sustainability. In the railway sector, the Bank targeted rationalization of subsidies for the infrastructure and passenger services, improved service delivery, and modernization of the companies’ institutional setup and the railways network assets (CPF, pages 35-36). Related World Bank operations. The Project was a follow-up on the Croatia Railway Modernization and Restructuring Project (P039161) and was based on the analytical work presented in the Croatia Railway Policy Note 2013 (Report No.78689-HR). Incentives to accelerate railway reforms in Croatia were also included in the Croatia - First and Second Programmatic Adjustment Loan Projects (P082278 and P094341). Parallel to the reviewed Project, the Bank produced a Croatia transport sector Policy Note 2019, supporting the implementation of the Government’s Transport Development Strategy 2017. Overall, the Project was aligned with both the Government’s and the WBG’s strategies, specifically, with the Government’s interim transport strategy 2014 and complete transport strategy 2017, as well as with the WBG’s CPS for FY2014-17 and CPF FY2019-24. The project was pitched at an appropriate level of ambition given the prior Bank operations. Therefore, the relevance of objectives is rated as High. Rating Relevance TBL Rating High 4. Achievement of Objectives (Efficacy) EFFICACY_TBL Page 7 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Croatia Railway (P147499) OBJECTIVE 1 Objective To improve the operational efficiency of the public railway sector in Croatia. Rationale The theory of change (ToC) for the Project was not included in the PAD but was developed for the ICR. It showed direct, logical causal links from inputs (activities) to outputs, and to four intermediate outcomes. The ToC showed how the achievement of the PDO would be supported by investments in the following three areas of the planned reforms in the targeted companies (HZPP, HZC, HZI) and the sector overall (MSTI): (i) Rationalization of the workforce (retrenchment) (HZPP, HZC, and HZI); (ii) improving conditions and availability of the rolling stock (HZPP and HZC); (iii) improving operational and financial management(HZPP and HZC); (iv) safety investments and emergency rehabilitation (HZI); and (v) improving asset management and sector governance (MSTI). As a result, the following intermediate outcomes would be reached: (i) improved operations and financial outcomes (HZPP, HZC); (ii) increased productivity and improved financial outcomes (HZI); and (iii) improved efficiency and sustainability in the use of public resources (MSTI). The ToC provided a comprehensive picture of the Project activities and outputs. The critical assumptions were also listed. As discussed in Section 2.e, the Original Project will be assessed based on the targets as presented in the PAD, while the Revised Project will be assessed based on the targets approved at the second restructuring of June 2019. Outputs: Objective 1 outputs were measured by the Original Project using the following indicators: Sector Policy Support. - “Number of project beneficiaries, including women beneficiaries”. This indicator was dropped at the second restructuring of June 2019, and the achievement of its target at closure is unknown. - “Percentage of assets with resolved legal titles”. The achievement at closure was zero, against the original target of 40 percent (established at the first restructuring) and the baseline of zero. The explanation in the ICR is that the target was not met because of a court procedure (arbitration) “between the Consultant and MSTI” (ICR, page 28). At IEG’s request, the team provided additional information, as follows. The baseline and target for this indicator were to be established through consulting services. However, there was a dispute with the consultant regarding an alleged incomplete product delivery. The dispute was to be resolved through mediation. As of the date of completion of this ICR review, the mediation was expected to be concluded by June 2024. The achievement of this indicator was not measured because the baseline and target were not clear. Support to HZPP Restructuring Page 8 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Croatia Railway (P147499) - “DMUs and diesel locomotives rehabilitated (Number)”. The achievement at closure was four locomotives, against the original target of 17 locomotives and the baseline of zero locomotives. The target was barely achieved. - “Passenger wagons rehabilitated (Number)”. This indicator was dropped at the second restructuring of June 2019, and the achievement of its target at closure is unknown. In relation to these two outputs, the team provided the following additional information: at approval, the emphasis was on asset rehabilitation, but during implementation, the project was reoriented towards financing new assets, and therefore instead of repairing locomotives and wagons, new trainsets were financed. At the same time, HZPP used its own resources to rehabilitate 38 DMUs and diesel locomotives, as well as wagons. IEG notes that this information explains why the target was not achieved but does not affect efficacy analysis because repairs financed by HZPP were made outside the project and cannot be attributed to the project. - “EMUs and electric locomotives rehabilitated (Number)”. The achievement at closure was eight locomotives, against the original target of six locomotives and the baseline of zero locomotives. The target was exceeded. Support to HZC Restructuring. - “Wagons rehabilitated (Number)”. The achievement at closure was 3,037 wagons, against the original target of 2,772 wagons and the baseline of zero wagons. The target was exceeded. - “Locomotives rehabilitated (Number)”. The achievement at closure was seven locomotives, against the original target of 48 locomotives and the baseline of zero locomotives. The target was barely achieved. Note: Please see the note under “Support to HZPP Restructuring” above. The same statement applies here. Support to HZI Restructuring and Enabling Investment to Increase the System Efficiency. - “Number of kms of line sections rehabilitated or improved”. The achievement at closure was 284.6 kilometers, against the original target of 338 kilometers and the baseline of zero kilometers. The target was largely reached. - “Fatalities per 10 million train-km (Number)”. The achievement at closure was one fatality, against the target of seven and the baseline of zero. The target was exceeded. - “Million train-km per track-km (Number).” This indicator was dropped at the second restructuring in June 2019, and the achievement of its target at closure is unknown. Outcomes: Objective 1 outcomes were measured by the Original Project using the following indicators: - “Prioritized railway sector development and infrastructure maintenance program developed (Yes/No)”. The related activity was replaced by another one at the second restructuring of June 2019, and this indicator was dropped as a result, therefore its achievement at closure is unknown. Page 9 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Croatia Railway (P147499) - “HZC viability ratio” (revenue/operating costs). The baseline was 0.26, and the target was 1.0. This indicator was dropped at the second restructuring of June 2019, and the achievement of its target at closure is unknown. The ICR explained that HZC required investments from external partners to become viable, and this indicator was replaced by a “completion of a strategic partnership” indicator (ICR, page 10). Note: In relation to this outcome, the team provided the following additional information: HZC went through significant improvements in its financial and operational performance, supported by the project, and by Project closure, the HZC’s working ratio was 1.16, exceeding the target. IEG notes this information, however, there are attribution concerns regarding the contribution of the project towards meeting this target since the underlying data /indicators on the drivers behind the increase in viability ratio are not evident. The link between the project activities and the increase in viability ratio is weak, and other indicators show it. For example, the operating expenses per train-km were actually higher than the baseline, and there was no data on revenues per employee because this indicator was dropped. Rating. The Original Project partially achieved its intended Objective 1 results, and its efficacy rating is Modest. Specifically, the two outcome indicators were later dropped. For one of them, the achievement of the target is unknown. For the second one, the target was exceeded by closure. Out of the ten output level indicator targets, four were largely reached or exceeded, while three were not reached or barely reached, and another three were dropped and their achievement is unknown. Rating Modest OBJECTIVE 1 REVISION 1 Revised Objective To improve the operational efficiency of the public railway sector in Croatia. Revised Rationale Please see the discussion of the ToC under Objective 1, Original Project. Objective 1 Revised outputs and outcomes were measured using the indicators listed below. Note: This is the full list of Project indicators used after the second restructuring of June 2019. It significantly differs from the list of indicators in the original Project because many of the indicators were reformulated and their targets adjusted at the first and second restructurings (September 2018 and July 2019). Outputs: Objective 1 outputs were measured by the Revised Project using the following indicators: Sector Policy Support. - “Percentage of assets with resolved legal titles”. The achievement at closure was zero, against the revised target of 30 percent (revised at the second restructuring) and the baseline of zero. The explanation in the ICR was that the target had not been met because of a court procedure (arbitration) “between the Consultant and MSTI” (ICR, page 28). At IEG’s request, the team provided additional information, as follows. The baseline Page 10 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Croatia Railway (P147499) and target for this indicator were to be established through consulting services. However, there was a dispute with the consultant regarding an alleged incomplete product delivery. The dispute was to be resolved through mediation. As of the date of completion of this ICR review, the mediation was expected to be concluded by June 2024. The achievement of this indicator was not measured because the baseline and target were not clear. Support to HZPP Restructuring - “DMUs and diesel locomotives rehabilitated (Number)”. The achievement at closure was four locomotives, against the revised target of four locomotives and the baseline of zero locomotives. The target was reached. - “EMUs and electric locomotives rehabilitated (Number)”. The achievement at closure was eight locomotives, against the revised target of eight locomotives and the baseline of zero locomotives. The target was reached. - “HZPP’s new trainsets (Number)”. The achievement at closure was five trainsets, against the target of four (established at the second restructuring of June 2019), the target of five (third restructuring of August 2020), and the baseline of zero. The target was reached. - “Customers reporting perception of service improvement over past two years (Percentage)”. The achievement at closure was 4.4 percent, against the target of 10 percent (introduced at the third restructuring of August 2020) and the baseline of zero. The target was not reached. The ICR explained this by the negative impact of COVID 19 on demand for passenger transportation (ICR, page 30). Support to HZC Restructuring. - “Wagons rehabilitated (Number)”. The achievement at closure was 3,037 wagons, against the revised target of 2,150 (first restructuring of September 2018), and the baseline of zero. The target was exceeded. - “Locomotives rehabilitated (Number)”. The achievement at closure was seven locomotives, against the revised target of 16 ((first restructuring of September 2018), revised target of 11 (second restructuring of June 2019), and the baseline of zero. The target was partially achieved. Support to HZI Restructuring and Enabling Investment to Increase the System Efficiency. - “Number of kms of line sections rehabilitated or improved”. The achievement at closure was 284.6 kilometers, against the revised target of 306.25 kilometers ((first restructuring of September 2018) and the baseline of zero kilometers. The target was largely reached. - “Fatalities per 10 million train-km (Number)”. The achievement at closure was one fatality, against the target of seven and the baseline of zero. The target was exceeded. - “Preparatory work for centralized traffic control and signaling modernization (Yes/No)”. The target was not achieved. The ICR noted that the work was under implementation and was expected to be completed in 2025 (ICR, page 33). Outcomes: Page 11 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Croatia Railway (P147499) Objective 1 outcomes were measured by the Revised Project using the following indicators: - “Introduction and signing of the multi-annual infrastructure contract (Yes/No)” (the indicator was introduced at the second restructuring, replacing the indicator “Prioritized railway sector development and infrastructure maintenance program developed (Yes/No)”). This indicator’s target was not reached, and the indicator was replaced (late in the Project implementation, at the fourth restructuring of November 2021) by yet less ambitious indicator “Preparation of the draft multi-annual infrastructure contract (Yes/No)”. In this case, the indicator target was reached. - “Completion of a strategic partnership for HZC (transaction finalized)”. The target was not reached. The ICR noted that by closure, HZC had finalized its restructuring and started its search for a reliable partner (ICR, page 28). Rating. The Revised Project partially achieved its intended Objective 1 results, and its efficacy rating is Modest. Specifically, one of the outcome indicators was only reached in its latest, less ambitious revision (which was introduced at the fourth restructuring of November 2021, late into the Project implementation), and the other was not reached. Even though the indicator on the "Preparation of the draft multi-annual infrastructure contract" was achieved, it is a relatively weak indicator to measure operational efficiency of the sector. The other indicators related to efficiency (reflected under Objective 2 regarding financial sustainability) i.e. operating expenses per train-km or the number of annual train-km operated per HZI employee were also not reached at project closure. Revised Rating Modest OBJECTIVE 2 Objective To improve the financial sustainability of the public railway sector in Croatia. Rationale Please see the discussion of the ToC under Objective 1, Original Project. Outputs: Objective 2 outputs were measured by the Original Project using the following indicators: Support to HZPP Restructuring - “Number of HZP staff retrenched (Number)”. The achievement at closure was 1,724 staff, against the target of 690 staff and the baseline of zero staff. The target was significantly exceeded. - “HZP commercial revenues per employee (HRK)”. This indicator was dropped at the second restructuring of June 2019, and its achievement at closure is not known. Page 12 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Croatia Railway (P147499) Support to HZC Restructuring. - “Number of HZC staff retrenched”. The achievement at closure was 1,507 staff, against the target of 500 staff and the baseline of zero staff. The target was significantly exceeded. Support to HZI Restructuring and Enabling Investment to Increase the System Efficiency. - “Number of HZI staff retrenched”. The achievement at closure was 1,697 staff, against the target of 470 staff and the baseline of zero staff. The target was significantly exceeded. - “Project-supported organization(s) publish reports on inputs and effect of consultation on project including gender disaggregated impact (Yes-No)”. This target was not reached. The ICR explained that the work was under implementation and was expected to be completed in 2025 (ICR, page 33). Outcomes: Objective 2 outcomes were measured by the Original Project using the following indicators: - “HZP operating expenses per train-km (Croatian Kuna or HRK)”. The achievement at closure was 50.1 HRK, against the target of 41 HRK and the baseline of 45 HRK. The target was not reached. The ICR explained that the actual result was 40.8 HRK in 2018 and 40.6 HRK in 2019, already exceeding the target. However, in 2020, COVID-19 led to a significant decrease in demand for passenger transportation, negatively affecting this outcome (ICR, page 25). The ICR stated that the target was achieved when controlling for inflation (ICR, page 13). - “Number of annual train-km operated per HZI employee”. The achievement at closure was 4,156 train-km, against the target of 5,015 train-km and the baseline of 4,032 train-km. The target was barely reached. The ICR explained the low achievement of the target by the following: (i) traffic restrictions on the lines that were being modernized, and (ii) reduction in traffic due to COVID-19 in 2020-2022. The ICR stated that the target “was achieved once rail services and ridership recovered from COVID-19 restrictions” (ICR, page 13). At IEG’s request, the team provided additional information, as follows. The achievement of this indicator was subject to normal evolution of traffic (disrupted by COVID-19) and staff downsizing (implemented, although also negatively affected by COVID-19). Calculations using the (pre-COVID-19) 2018 and 2019 traffic characteristics (train-km) and the number of staff as of 2021 (after the downsizing) resulted in 4,568 train-km per employee for the traffic of 2018 and in 4,675 train-km per employee for the traffic of 2019. With these hypothetical numbers, the original target would have been partially achieved. However, the actual statistical data for this indicator for 2023, provided by the team, was 3,973 train-km, which is below both the original target and the baseline, and shows a downward trend. The target for this outcome indicator was not achieved. Rating. The Original Project partially achieved its Objective 2 intended results, and its efficacy rating is Modest. Specifically, neither of the outcome level targets was reached; and out of five output level targets, three were significantly exceeded (all measuring workforce retrenchment) while one was not reached; and one indicator was dropped, and the achievement of the target is unknown. Rating Page 13 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Croatia Railway (P147499) Modest OBJECTIVE 2 REVISION 1 Revised Objective To improve the financial sustainability of the public railway sector in Croatia. Revised Rationale Please see the discussion of the TOC under Objective 1, Original Project. Objective 2 Revised outputs and outcomes were measured using the indicators listed below. Note: This is the full list of Project indicators used after the second restructuring of June 2019. It significantly differs from the list of indicators in the original Project because many of the indicators were reformulated and their targets adjusted at the first and second restructurings (September 2018 and July 2019). Support to HZPP Restructuring - “Number of HZP staff retrenched (Number)”. The achievement at closure was 1,724 staff, against the revised target of 771 staff (established at the second restructuring of June 2019), revised target of 376 staff (third restructuring of November 2021), and the baseline of zero staff. The target was significantly exceeded. Support to HZC Restructuring. - “Number of HZC staff retrenched”. The achievement at closure was 1,507 staff, against the revised target of 910 staff (first restructuring of September 2018) and the baseline of zero staff. The target was significantly exceeded. Support to HZI Restructuring and Enabling Investment to Increase the System Efficiency. - “Number of HZI staff retrenched”. The achievement at closure was 1,697 staff, against the revised target of 1,098 staff (first restructuring of September 2018), revised target of 900 staff (second restructuring of June 2019) and the baseline of zero staff. The target was significantly exceeded. Outcomes: Objective 2 outcomes were measured by the Revised Project using the following indicators: - “HZP operating expenses per train-km (Croatian Kuna or HRK)”. The achievement at closure was 50.1 HRK, against the revised target of 41.4 HRK (first restructuring of September 2018), and the baseline of 45 HRK. The target was not reached. As reported in the Original Project efficacy section, the actual result was 40.8 HRK in 2018 and 40.6 HRK in 2019, already exceeding the target. However, in 2020, COVID-19 led to a significant decrease in demand for passenger transportation, negatively affecting this outcome (ICR, page 25). The ICR stated that the target was achieved when controlling for inflation (ICR, page 13). - “Number of annual train-km operated per HZI employee”. The achievement at closure was 4,156 train-km, against the revised target of 3,834 train-km (first restructuring of September 2018), revised target of 4,500 train-km (second restructuring of June 2019), and the baseline of 4,032 train-km. The target was partially achieved. The ICR explained the low achievement of the target by the following: (i) traffic restrictions on the Page 14 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Croatia Railway (P147499) lines that were being modernized, and (ii) reduction in traffic due to COVID-19 in 2020-2022. The ICR stated that the target “was achieved once rail services and ridership recovered from COVID-19 restrictions” (ICR, page 13). At IEG’s request, the team provided additional information, as follows. The achievement of this indicator was subject to normal evolution of traffic (disrupted by COVID-19) and staff downsizing (implemented, although also negatively affected by COVID-19). Therefore, the data for years 2020-2022 was skewed due to the COVID-19 impact. Calculations using the (pre-COVID-19) 2018 and 2019 traffic characteristics (train-km) and the number of staff as of 2021 (after the downsizing) resulted in 4,568 train-km per employee for the traffic of 2018 and in 4,675 train-km per employee for the traffic of 2019. With these hypothetical numbers, the revised target would have been achieved. However, the actual statistical data for this indicator for 2023, provided by the team, was 3,973 train-km, which is below both the revised target and the baseline, and shows a downward trend. The target for this outcome indicator was not achieved. Rating. The Revised Project partially achieved its Objective 2 intended results, and its efficacy rating is Modest. Specifically, neither of the outcome targets was achieved, and while the three output level targets were significantly exceeded, all corresponding indicators measured workforce retrenchment, which was only important as a means to achieving the operational efficiency and financial sustainability of the railway sector (measured by intermediate and PDO outcome indicators) and not essential by itself. The key indicator to reflect financial sustainability i.e. the working ratio or financial viability ratio was dropped thus weakening the evidence base. Revised Rating Modest OVERALL EFF TBL OBJ_TBL OVERALL EFFICACY Rationale Original Project: For the Original Project, the rating for efficacy is Modest: the targets under both objectives were partially reached. Overall Efficacy Rating Primary Reason Modest Low achievement OBJR1_TBL OVERALL EFFICACY REVISION 1 Overall Efficacy Revision 1 Rationale Revised Project: For the Revised Project, the efficacy rating is also Modest: the targets under both objectives were partially reached. Even though the project met the staff retrenchment targets, the 'operating expenses per train-km' or the 'number of annual train-km operated per HZI employee' were partially achieved. Moreover, the key Page 15 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Croatia Railway (P147499) indicator to reflect the second objective related to financial sustainability i.e. the working ratio or financial viability ratio was dropped thus weakening the evidence base. Overall Efficacy Revision 1 Rating Primary Reason Modest Low achievement 5. Efficiency Economic analysis At appraisal, the economic analysis covered the following: (i) investments in physical assets, including rehabilitation of track and improvement of safety, and (ii) retrenchment for HZPP, HZC, and HZI. It was based on the Cost-Benefit Analysis (CBA) approach. The analysis assumed benefits to include the following: reduced maintenance costs for HZI, reduced operation costs for HZPP and HZC, improved operation and services for users, and a shift of traffic from road to rail and reduced pollution and congestion as a result. time savings and a decrease in operating costs. The Economic Internal Rate of Return (EIRR) for physical investments was estimated at 12.4 percent. The EIRR for retrenchment was 24.7 percent and the NPV equaled EUR 42.1 million. Since the opportunity cost of capital (discount rate) was 12 percent, both EIRRs exceeded it, making the Project efficient. At closure, the economic analysis also covered physical investments and retrenchment, and was based on the CBA approach. However, a different model was used. The benefits included reduced operation costs of HZPP, HZC, and GZI; reduced maintenance costs for HZI; benefits from new and rehabilitated assets; reduced labor costs and increased productivity; and reduced CO2 emissions. The overall EIRR (for physical investments and retrenchment) was estimated at 11.2 percent. The opportunity cost of capital (discount rate) was 12 percent, and the EIRR was slightly below it. Administrative efficiency Administrative efficiency had deficiencies, especially at the start of implementation when the Project experienced delays (ICR, page 18). The closing date was extended twice, for a total of three years and two months (or 38 months). The Project was restructured five times. The ICR mentioned that the Project design was overly optimistic given the conditions in the country and the components were too complex given the capacity to manage large investments (ICR, page 22). As a result, the Project’s ambition had to be lowered and the RF had to be significantly modified during implementation, weakening performance monitoring. In addition to the risks that could have been mitigated through Project design, the Project faced external challenges, which were mostly outside of its control, and was able to adapt to them using restructurings. Specifically, the following challenges were encountered: a change in government and the introduction of new EU Directives for railways (leading to a change in sector strategy and a one-year delay of Project effectiveness); COVID-19 (leading to a sharp drop in demand for passenger transportation, among other negative impacts); a spike in inflation; and the change of currency from Croatian Kuna to Euro. Page 16 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Croatia Railway (P147499) On balance, considering the EIRR at closure slightly below the opportunity cost of capital and several design and implementation inefficiencies, but also taking into account the project's ability to adapt to negative external challenges, the Project efficiency rating is Substantial, with moderate shortcomings. Efficiency Rating Substantial a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 98.60 Appraisal  12.40  Not Applicable 96.80 ICR Estimate  11.20  Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome The relevance of objectives is rated as High. Efficacy is rated as Modest both pre- and post second restructuring given the partial achievement of the targets. Efficiency is rated as Substantial. The overall outcome for both Original and Revised Projects are rated as Moderately Unsatisfactory; therefore, the Overall Project Outcome rating is Moderately Unsatisfactory. It is noted that 52.8% was disbursed at the time of the second restructuring. a. Outcome Rating Moderately Unsatisfactory 7. Risk to Development Outcome Financial. The Project initiated reforms and physical investments to enable a modern, commercially oriented, and efficient railway system. Much more needs to be done to achieve this expected final outcome, and there is a risk that the necessary continuation of reforms and investments will not occur. However, significant investments are ongoing and expected in the future. At the time of the ICR preparation, 25 major infrastructure projects were being implemented in Croatia, and the country announced plans to invest more than EUR 4 billion in rail infrastructure over the next 10 years through a program set to begin in 2024 and be funded mostly through the EU. The overall reform agenda is also supported, guided by the EU acquis. (ICR, page 23) Page 17 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Croatia Railway (P147499) 8. Assessment of Bank Performance a. Quality-at-Entry The ICR reported that Project preparation was based on lessons learned from other Bank operations and Project-dedicated studies. The relevant stakeholders were diligently consulted. A risk assessment was conducted, including an assessment of the financial management capacity of the three targeted railway companies. Project design was aligned with the national strategy and EU regulations. However, the Project’s PDO and outcome targets were overambitious, and the components were too complex to implement, considering the capacity constraints of the targeted companies. The ICR mentioned that the Project design was overly optimistic given that the final government sector strategy was not in place and the country was undergoing structural changes related to EU accession. The ICR also stated that the components were too complex given the inadequate in-country capacity to manage large investments, including retrenchment, the purchase of rolling stock, and major rehabilitation works. (ICR, page 22) A deeper review of the previous experience in the sector could have helped to identify and mitigate these design flaws. Also, while the Project design was based on an adequate results chain, the outcome indicators were flawed, making it difficult to measure progress in achieving objectives. In addition, while risk analysis was conducted, some known external factors were not taken into account, specifically the change of government (which resulted in a one-year delay in Project effectiveness), the introduction of new EU Directives for railways, and the currency transition from Croatian Kuna to Euro. (ICR, page 22). Given the shortcomings, the quality at entry is rated as Moderately Unsatisfactory. Quality-at-Entry Rating Moderately Unsatisfactory b. Quality of supervision The ICR reported that the supervision was effective, supported by the close collaboration between the Bank and the client and by the client’s commitment. Having a consistent and experienced team allowed for effective communication and the ability to address issues promptly, earning the client’s trust. The supervision also significantly benefited from technical support by the Bank, especially during the retrenchment process and the strategic reforms. The country-based staff was actively involved in the supervision, and their low turnover supported the continuity in Project management. Restructurings were used effectively, improving the Project design and the RF. However, there were implementation inefficiencies, especially at the start when the Project experienced delays. The closing date was extended twice, for a total of three years and two months (or 38 months), and the Project was restructured five times. Also, RF deficiencies made implementation monitoring challenging, and there were multiple changes to the RF late into Project implementation, both due to the need to scale down Project ambition and to improve the RF quality. (ICR, page 23). The Quality of Supervision is rated as Moderately Satisfactory. Page 18 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Croatia Railway (P147499) Quality of Supervision Rating Moderately Satisfactory Overall Bank Performance Rating Moderately Unsatisfactory 9. M&E Design, Implementation, & Utilization a. M&E Design The original RF reflected the logic of Project interventions in the PAD and was sufficiently linked to the PDO. The PDO outcome indicators reflected the core Project results, and the intermediate indicators were linked to the PDO indicators, providing more details and measuring intermediate outcomes and outputs. The indicators measured all types of results: those related to investments in physical assets (e.g., “DMUs and diesel locomotives rehabilitated”), institutional results (e.g., “Prioritized railway sector development and infrastructure maintenance program developed” or “Number of HZP staff retrenched”), and financial results (e.g., “HZC viability ratio”). The ICR reported that the organizational setup of the M&E framework was adequate. The MSTI was responsible for the monitoring and semi-annual and annual management reports, covering progress on results indicators. Each targeted company appointed dedicated M&E staff and submitted semi-annual reports in an agreed format. (ICR, page 19) However, the RF had significant deficiencies and had to be extensively modified during implementation, both to correct the flaws in its design and to reduce the ambition of several intended results. The flaws in the RF design, as the ICR rightly pointed out, included the presence of several standard sector indicators that were beyond the Project control (e.g., “HZP operating expenses per train-km”, “Number of annual train-km operated per HZI employee”, and “Completion of a strategic partnership for HZC”) (ICR, page 19). Also, targets for several indicators were overambitious, including in relation to investments in physical assets and institutional development. The needed changes were significant and included dropping many indicators and introducing new ones, thus limiting the capacity of the RF to continuously monitor Project performance, especially in relation to the second objective (“To improve the financial sustainability of the public railway sector in Croatia”). b. M&E Implementation The abovementioned RF deficiencies were addressed through Project restructurings, especially during the first and the second ones (September 2018 and June 2019), which was late into Project implementation (respectively, three and almost four years after Project effectiveness of August 2015) and when more than one-half of Project financing had been disbursed. Some of the changes were still being implemented at the third and fourth restructurings. As a result of these changes, the final RF had more realistic targets, and its indicators were more attributable to the Project. However, the RF’s comprehensiveness was compromised, especially in relation to the second Project objective (sector financial sustainability) and to Component 1; and the continuity of performance monitoring was partially Page 19 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Croatia Railway (P147499) impaired. Overall, as the ICR reported, the restructurings did not adequately address the RF flaws (ICR, page 20). The ICR reported that the M&E data was collected regularly, and updates on progress towards achieving the targets were included in the 14 Implementation Support and Review (ISR) reports and reported in the Bank's Aide-Memoires (AMs) on a semi-annual basis. When physical missions were not possible during COVID-19, virtual missions were conducted to ensure ongoing monitoring. (ICR, page 19-20) c. M&E Utilization The ICR reported that the RF was utilized to inform Project management and decision-making. The decision to proceed with Project restructurings and closing date extensions was primarily based on M&E data reflecting progress in delivering Project outputs, as well as on the continuous dialogue between the Bank and the Borrower that had been documented in progress reports. (ICR, page 20) M&E Quality Rating Modest 10. Other Issues a. Safeguards Environmental and Social Safeguards. The ICR reported that at appraisal, the Project was assessed as Environmental Assessment Category B (Partial Assessment) due to a potential negative impact of the works on rehabilitation of railway tracks, other rail infrastructure, and the rolling stock. Environmental Assessment (OP/BP 4.01) and Involuntary Resettlement (OP/BP 4.12) were triggered. In relation to the workforce retrenchment, a PSIA was prepared, and social mitigation measures proposed. The Client developed a Project E&S Management Framework (ESMF). Based on the ESMF, the following documents were prepared: an E&S Management Plan (ESMP) for the rehabilitation of the Zagreb-Savski Marof railway line, 7 ESMP checklists for rehabilitation and maintenance of rolling stock, and ESMP checklists for 15 types of infrastructural works. E&S audits were carried out for the already commenced replacement systems. E&S instruments were mostly prepared and consulted on time except for the ESMP checklists for HZPP rolling stock rehabilitation, which were delayed. In addition, there was a significant delay in E&S reporting for all sub-projects. Also, the Project operated without an environmental specialist in the PIU for an extended period, leading to a reduction of the compliance rating to Moderately Unsatisfactory in 2021. The issues were addressed quickly, and the rating was upgraded to Moderately Satisfactory by the end of 2021. During most of the implementation period, E&S performance was rated Moderately Satisfactory. The retrenchment was carried out per the national Labor Code and applicable collective agreements. All three companies established grievance mechanisms. The Institute for Labor Market Development provided Page 20 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Croatia Railway (P147499) professional training and career counselling in ten towns to 86 affected workers from HZPP and HZC who applied for these services. On March 5, 2021, a fatal accident occurred: an employee of a sub-contractor was hit by a passing locomotive during the Project’s track rehabilitation works. The incident was reported to the Bank on April 23, 2021; the Bank team visited the location on May 3, 2021 and completed an Environment and Social Action Plan (ESAP). Following the requirements, HZI prepared a root cause analysis (RCA) of the accident and introduced operational health and safety (OHS) measures, including better oversight, awareness meetings, training, and a risk assessment review. The Bank discussed additional measures with HZI, including fencing/marking safe corridors, additional signalization, monitoring of train speed, stopping works during heavy traffic, and improving signalization. By December 22, 2021, the HZI met the ESAP requirements (except some past due E&S reports, which were submitted by mid-2022). b. Fiduciary Compliance Financial management (FM). The ICR reported that the Project’s financial reports were acceptable albeit in some cases there were delays in submission. The audit reports included clean audit opinions on reporting and use of the loan proceeds, however entity financial statements were qualified by auditors and indicated financial performance issues. During the Project life, the number of audit qualification decreased. FM was rated as Satisfactory at Project closure. (ICR, page 21) Procurement. Procurement followed the WB’s “Guidelines: Procurement of Goods, Works, and Non- Consulting Services under IBRD Loans and IDA Credits & Grants by the WB Borrowers” and “Guidelines: Selection and Employment of Consultants under IBRD Loans and IDA Credits & Grants by the WB Borrowers”. The Bank’s Standard Bidding Documents (SBD) for procurement of Goods and Works were used. Procurement applied advance contracting and retroactive financing to selected Project activities, with reimbursement by the Bank of any payments made by the borrower under contracts signed prior to loan signing. The Bank carried out a posteriori review of the procurement process of the relevant contracts and found it reasonable and in line with the procurement principles. Rating for procurement during implementation was maintained as Satisfactory. (ICR, page 21-22) c. Unintended impacts (Positive or Negative) --- d. Other --- 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment Page 21 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Croatia Railway (P147499) Shortcomings in achievement of Moderately Moderately Outcome both objectives, both pre- and Satisfactory Unsatisfactory post restructuring. Moderately Moderately Bank Performance Shortcomings in quality at entry. Satisfactory Unsatisfactory Quality of M&E Modest Modest Quality of ICR --- Substantial 12. Lessons The following lessons were derived from the ICR (ICR, pages 28-29): 1. The Project’s experience highlighted the WB’s convening power to leverage EU funding to support reforms and infrastructure investments that lay out the basis for sector modernization and competitiveness. Critical to the success of the supported reforms was a broad-based commitment by government and stakeholders, as well as the capacity for reform implementation. A good assessment of the level of commitment and capacity can provide critical information for project design, including the timing, scope, and sequencing of reforms, as well as for risk mitigation. However, there can be external factors and unforeseen circumstances, which, in case of the reviewed Project, included COVID-19, supply chain disruption, and a spike in inflation. The lesson learned from the reviewed Project is that in such operating environment, a flexible project design could help with the necessary adjustment to the changing circumstances and support the utilization of new opportunities when planned activities turn out to be no longer feasible. 2. The Project was negatively affected by the deficiencies of its M&E framework, specifically overambitious targets and indicators that were not fully attributable to the Project. More generally, in large infrastructure projects, often standard, industry-wide performance indicators are included in the RF without considering if they are within the project scope and control. In measuring the outcomes of institutional reform, often RFs rely heavily on output level indicators that do not necessarily measure the envisioned change in the relevant sector institutions. The Project’s M&E framework was restructured multiple times but still contained inadequate and inappropriate results indicators, making it hard to assess progress toward the achievement of the PDO. The Project would have benefited from a presence of experienced M&E specialists on the team and/or in the implementing agency, especially during key implementation points, such as appraisal, mid-term review, and evaluation. The lesson learned is that when RF has significant deficiencies, its adjustment through restructurings can be challenging and if significant changes are made to the RF during implementation, it can impair the RF’s capacity to continuously monitor project performance. 13. Assessment Recommended? No 14. Comments on Quality of ICR Page 22 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Croatia Railway (P147499) The ICR provides sufficient technical details to understand the logic of the Project, the factors of Project performance, the outcomes of the Project, and various aspects of Project implementation. It provides comprehensive and well-supported evidence on various aspects of Project’s evaluation, a clear linking of evidence to findings, and a candid discussion of the Project’s weaknesses. The ICR could have been improved if the Original Project’s efficacy were discussed separately from the Restructured Projects’ efficacy. This way, it would have presented evidence to support the split efficacy ratings. On balance, the ICR quality rating is Substantial, with minor shortcomings. a. Quality of ICR Rating Substantial Page 23 of 23