Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review HT Cultural Heritage and Tourism Sector (P144614) Report Number: ICRR0023465 1. Project Data Project ID Project Name P144614 HT Cultural Heritage and Tourism Sector Country Practice Area(Lead) Haiti Urban, Resilience and Land L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IDA-H9440 31-Aug-2020 17,310,649.08 Bank Approval Date Closing Date (Actual) 19-May-2014 29-Jul-2022 IBRD/IDA (USD) Grants (USD) Original Commitment 45,000,000.00 0.00 Revised Commitment 26,150,000.02 0.00 Actual 17,843,117.59 0.00 Prepared by Reviewed by ICR Review Coordinator Group Katharina Ferl Vibecke Dixon Kavita Mathur IEGSD (Unit 4) 2. Project Objectives and Components DEVOBJ_TBL a. Objectives According to the Project Appraisal Document (PAD) (p.ii) and the Financing Agreement of July 9, 2014 (p.4) the objective of the project was to “a) increase the attractiveness of the cultural heritage sites in North of Haiti for tourists; (b) improve the living environment for the residents living in North of Haiti; and (c) support the Recipient’s capacity to respond promptly and effectively to an Eligible Emergency, as needed. Page 1 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review HT Cultural Heritage and Tourism Sector (P144614) b. Were the project objectives/key associated outcome targets revised during implementation? Yes Did the Board approve the revised objectives/key associated outcome targets? Yes Date of Board Approval 27-Aug-2020 c. Will a split evaluation be undertaken? Yes d. Components The project included five components. According to the World Bank team (June 7, 2023) no actual costs per component were reported since during the June 2020 restructuring only one disbursement category, combining all components, was established. Component A: Physical Cultural Heritages sites and Touristic Circuits in the PNH-CSSR and Cap- Haitian Historic Center (appraisal estimate US$28.0 million). This component was to finance the following activities: i) conservating and rehabilitating the Citadelle Henry, the Palais de Sans Souci, the Chapelle de Sans Souci and Les Ramiers in the Parc National Historique Citadelle-Sans Souci-Ramiers - National History Park (PNH-CSSR) including structural strengthening, refurbishment, presentation, sanitation and safety improvement through the carrying out of works and the provision of goods and consultants’ services; ii) supporting the management of the PNH-CSSR; iii) urban upgrading of the Cap-Haïtien historic city center; iv) strengthening the planning and management capacity of the Natural Cultural Heritage Institute (ISPAN) and provision of technical assistance for the implementation of the activities stated under this component. Component B: Local Investments in Milot, Dondon and Cap-Haitien (appraisal estimate US$7.0 million). This component was to finance the following activities: i) improving the local urban infrastructure in Milot, Dondon and Cap Haitien to carry out investments selected based on a participatory decision-making mechanism through the carrying out of works and the provision of goods and consultants’ services; ii) preparing a solid waste management study in Milot and Dondon, and sanitation and urban planning studies for Cap-Haïtien; and iii) strengthening the capacity of the municipalities located in Milot, Dondon and Cap- Haïtien through the provision of goods, consultants’ services and training. Component C: Inclusive Tourism Sector Development Support (appraisal estimate US$4.0 million). This component was to finance the following activities: i) strengthening the capacity of the Ministry of Tourism (MT) through: a) the development of tourism sector planning and tourism circuits; b) skills development; and c) the upgrading of tourism statistics and enhancing sector coordination through the carrying out of small works and the provision of goods, consultants’ services, and training; ii) strengthening the capacity of the Destination Management Organization (DMO) in: a) developing tourism promotion strategies, marketing, and technical capacity; and b) developing and upgrading tourism products, through the provision of goods, consultants’ services and training; and iii) enhancing the local engagement in cultural heritage tourism through: a) the provision of Community Events and Tourism Grants to Beneficiaries for carrying out sub-projects to develop tourism related services and products; and b) the improvement of Page 2 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review HT Cultural Heritage and Tourism Sector (P144614) the capacity of the local communities to develop tourism related services and products, through the provision of goods, consultants’ services and training. Component D: Contingency Emergency Response (appraisal estimate US$1.0 million, zero disbursement). This component was to finance a provision of support to respond to an Eligible Emergency, as needed. Due to the high risk of a catastrophic event in Haiti, the proposed project would include a provisional component for Contingent Emergency Response (CER), designed as a mechanism for rapid response in the event of an eligible emergency, subject to the request of the government. Component E: Project Implementation, Monitoring and Evaluation (appraisal estimate US$5.0 million). This component was to support Technique d’Exécution (UTE) at the Ministry of Economy and Finance (MEF) and the National Cultural Heritage Institute (ISPAN) in the implementation, monitoring and evaluation of the project. e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project cost. The project was estimated to cost US$45.0 million. Actual cost was US$17.8 million due to the dropping of several activities, cost savings, and use of some financing for the COVID-19 response. Financing. The project was financed by an IDA grant in the amount of US$45.0 million of which US$26.15 million was disbursed and US$18.85 million of financing was cancelled. Borrower Contribution. It was not planned for the Borrower to make any contributions. Dates: The project was restructured six times (all level 2):  On June 30, 2017, the project was restructured to: i) modify the Results Framework to add two intermediate indicators (“establish a Provisional Park Management Authority (P-AGP) in charge of implementing” and “prepare and adopt an appropriate work program of conversation and consultation activities in the National History Park (PNH-CSSR), where most of the projects investment were to take place”); and ii) change in implementation schedule to establish a permanent AGP by 2018.  On June 29, 2018, the project was restructured to: i) drop several activities such as renovation of the old prison, rehabilitation of Antenor Firmin House and Alliance Francaise due to delays in the submission of critical documents; ii) funds from the dropping of several activities were reallocated to component C, inclusive tourism sector development support and to Component E, project implementation, evaluation and monitoring; iii) reformulate and adjust indicators: and iv) align procurement arrangements with the World Bank’s 2017 procurement guidelines.  On June 6, 2020, the project was restructured to cancel US$5.3 million to mobilize resources to support the government in its COVID-19 response under a different World Bank project.  On August 27, 2020, the project was restructured to: i) reduce activities under Component A due to delays in the preparation of technical documents and foreign firms not being able to start work on the project site due to COVID-19 restrictions; ii) cancel US$7.75 million of project financing; iii) extend the project’s closing date by 23 months from August 31, 2020, to July 31, 2022 to allow for Page 3 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review HT Cultural Heritage and Tourism Sector (P144614) the implementation of project activities which were delayed due to significant socio-political upheavals and the COVID-19 pandemic; iv) modify Results Framework accordingly; and v) change disbursement estimates.  On October 9, 2021, the project was restructured to: i) create a new disbursement category to finance the project’s possible resettlement-related expenditures for the upgrading of facades in two main streets of Cap Haitien’s city center due to the government not being able to provide funding.  On March 4, 2022, the project was restructured to: i) cancel US$5.8 million by dropping the rehabilitation of Ancienne Capitainerie under component A due to an unsuccessful call for bits as week as savings made under different activities. Split Rating. A split rating will be conducted because PDO targets were revised during August 2020 restructuring. 3. Relevance of Objectives Rationale Country context: According to the PAD (p. 1) in 2012, two years before project appraisal, Haiti’s Gross Domestic Product (GDP) per capita was US$771, making it the poorest country in the Americas. Recovering from the devastating earthquake in 2010 that resulted in damages and losses of US$7.9 billion (120 percent of the country’s GDP) with needed reconstruction efforts estimated at US$11.3 billion. However, at appraisal, economic growth was increasing mainly due to a pick-up in agricultural production, construction, and the industrial sector, in particular the textile and garment industry. Also, security indicators were improving, and the government aimed to improve climate investment, foster entrepreneurship and public-private partnership for infrastructure. Also, Haiti had the potential to attract new investments in agribusiness, apparel, and tourism. Sector context: Cap Haitien, the second largest city in Haiti, located in the North of the island, was surrounded by historic monuments representing the country’s history and cultural heritage. Cap Haitien and its surrounding area was seen as having considerable potential for tourism and economic development. Conservation and management of cultural heritage were seen as critical for the sustainable development of tourism. Demand for domestic and international tourism to the area was growing. In 2009, receipts from international tourists in the North were US$312 million accounting for about 34 percent of the value of Haiti’s total service exports and 8 percent of GDP that year. Alignment with the government’s strategy: In 2014, the government started to actively foster tourism. The 2013-2014 Tourism Strategy aimed to work with the private sector to improve existing hospitality and convention facilities and constructed new hospitality training centers. The FY14 budget of the Ministry of Tourism was increased by 26 percent and staff to work on various touristic sites was hired. Alignment with the World Bank’s strategy: The objective of the project was in line with three of the focus areas of the World Bank’s most recent Country Partnership Framework (CPF) (FY2016-FY2021): i) enhancing inclusive growth by promoting development in the north of Haiti; ii) strengthening resilience by earthquake-proofing cultural heritage and buildings; and iii) improving governance in the management of Page 4 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review HT Cultural Heritage and Tourism Sector (P144614) the PNH-CSSR and by strengthening agencies managing tourism and cultural heritage. When the CPF was enhanced through a Performance Learning Review (PLR), the two objectives of the project (“to improve the living environment for the residents living in the North of Haiti” and “support the recipient’s capacity to respond promptly and effectively to an eligible emergency”) remained relevant when the project closed. However, the objective “to increase the attractiveness of the cultural heritage sites in the North of Haiti for tourists” was not relevant anymore due to the deterioration of the political and social issues as well as the security situation, exacerbated by the COVID-19 pandemic, and resulting in the need for more basic needs to be addressed. Taking everything together, the relevance of the objective is rated Substantial. Rating Relevance TBL Rating Substantial 4. Achievement of Objectives (Efficacy) EFFICACY_TBL OBJECTIVE 1 Objective Increase the attractiveness of the cultural heritage sites in North of Haiti for tourists. Rationale Theory of Change. The project’s theory of change envisioned that project activities such as making physical investments to improve resilience, restore, conserve, and beautify heritage assets, developing and implementing a park management plan for the PNH-CSSR (including a master plan and business plan) as well as operationalizing the Park Interim Management Authority (AGP), were to result in outputs such as three main historic sites being rehabilitated, a park management plan being implemented, and the AGP being operational. These outputs were to result in the outcome of increased attractiveness of the cultural heritage sites in North of Haiti for tourists, and increased resilience of heritage assets to natural disasters. Also, the theory of change envisioned that longer-term outcomes would include increased economic growth for the local population. These assumptions seem reasonable. The project’s theory of change was adequate and logical without apparent logical gaps. The ICR reported the following outputs that included a target:  An interim authority (AGP) with staffing acceptable to IDA was set up in the PNH-CSSR, achieving the target. However, a permanent AGP was never established, putting the sustainability of an effective Park Management System at risk. The Minister of Culture (MC) appointed a liaison agent charged with the task of developing the AGP. However, when the government changed in 2020, the liaison agent left. It took time to find a new liaison, leaving no time to legally create the permanent AGP.  A National History Park Management Plan was finalized. The plan is not under implementation yet. Therefore, the target of the plan being finalized and under implementation was only partially achieved. Page 5 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review HT Cultural Heritage and Tourism Sector (P144614)  Two touristic routes were developed, not achieving the original target of five tourist routes being developed and promoted through project investments/activities. The route from Sans-Souci to Choiseul was completed and a tourist circuit that includes Rue 16 and 20 was proposed. The original target of five touristic routes was not achieved. Additional outputs reported in the ICR without target values:  Diagnostic studies for the rehabilitation of the monuments were finalized (including topographical surveys, geotechnical and conservation studies).  Urgent conservation works were done at the Henri-Ramiers Citadelle.  ISPAN’s documentation center was constructed. Outcomes:  An effective Park Management system was not established. Therefore, the target of an effective Park Management system contributing to sustainable development of the park was not achieved.  The original target of seven restored buildings with increased resistance to seismic events was not achieved. During the 2018 restructuring, the rehabilitation of the old prison, the Mainson Antenor Firmin and the Alliance Francaise were dropped. In 2022, the upgrading of the Ancienne Capitainerie was dropped due to delays in the procurement of a construction firm. The project only conducted emergency repairs to parts of Citadelle Henr- Ramiers, considered the largest fortress in the Americas.  The number of tickets sold annually to the PNH-CSSR decreased from 12,500 in 2014 to 10,500 in 2022, not achieving the original target of 16,750 tickets.  The number of entities that provided cultural and tourism services in the project areas decreased from 310 in 2014 to 256 entities in 2022, not achieving the original target of 350 entities. The project did not achieve any of the PDO indicator targets. However, diagnostic studies for the rehabilitation of the monuments were finalized and urgent conservation works at the Henri-Ramiers Citadelle were conducted, ISPAN’s documentation center was constructed, and the PNH management Plan was completed and submitted to the World Heritage Center. While these achievements were limited, they may contribute to increasing the resilience and attractiveness of the cultural heritage sites in North of Haiti. Therefore, this objective is rated Modest. Rating Modest OBJECTIVE 1 REVISION 1 Revised Objective Increase the attractiveness of the cultural heritage sites in North of Haiti for tourists. Revised Rationale The objective remained the same, but the targets were adjusted. The theory of change remained the same. Page 6 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review HT Cultural Heritage and Tourism Sector (P144614) Outputs:  Two touristic routes were developed, achieving the revised target of two touristic routes being developed and promoted through project investments/activities. Outcomes:  The revised target of one building being restored was not achieved.  The number of tickets sold annually to the PNH-CSSR decreased from 12,500 in 2014 to 10,500 in 2022, achieving the revised target of 10,500 tickets.  An effective Park Management system was not established. Therefore, the target of an effective Park Management system contributing to sustainable development of the park was not achieved.  The number of entities that provided cultural and tourism services in the project areas decreased from 310 in 2014 to 256 entities in 2022, not achieving the revised target of 288. Most output and outcome indicators under this objective were not achieved. However, the revised target of tickets sold annually to the PNH-CSSR was achieved. Taking everything together, the achievement of this objective was Modest. Revised Rating Modest OBJECTIVE 2 Objective Improve the living environment for the residents living in North of Haiti. Rationale Theory of change: The project’s theory of change envisioned that project activities such as improving local urban infrastructure, preparing solid waste management, sanitation and urban planning studies, strengthening the capacity of municipalities for participatory planning and providing tourism development grants to engage and strengthen local populations’ involvement in and inputs to the tourism sector were to result in outputs such as improved urban infrastructure, solid waste management, sanitation and urban planning studies being prepared. These outputs were to result in outcomes such as strengthened capacity of municipalities, increased income from the tourism sector for the local population and an improved living environment for the residents in the North of Haiti. Also, the theory of change envisioned that longer-term outcomes would include improved quality of life for the local population and inclusive and sustainable local development. The project’s theory of change was adequate and logical without apparent logical gaps. Outputs: The ICR reported the following outputs that included a target: Page 7 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review HT Cultural Heritage and Tourism Sector (P144614)  2,670 meters of streetscapes were upgraded in project target areas, exceeding the original target of 300 meters.  Three infrastructures were properly operated and maintained (Place de Milot, Place de Dondon, and the ISPAN documentation center) following project financing, not achieving the original target of five infrastructures.  47 sub-projects were financed. Out of these, 24 sub-projects promoted handicrafts, cultural activities, or entertainment such as music, theater, and painting. Seven grants supported food processing like manioc and chocolate, and Haitian cuisine. Another 13 sub-projects financed the upgrade and maintenance of tourist sites and destinations while another three sub-projects supported transport and tour guides. The original target of 100 sub-projects was not achieved. 25 percent of the sub-projects were implemented by women, not achieving the original target of 40 percent. 25 percent of the sub- projects were implemented by youth, achieving the original target of 25 percent. The ICR reported the following outputs that did not include a target:  Rehabilitation of facades in Street 16 and 20 in the historic center of Cap Haitien.  According to the Bank team (June 7, 2023) during the restructuring in March 2022, three solid waste management studies were dropped. Outcomes:  The project benefitted a total of 63,450 beneficiaries, exceeding the original target of 35,000 beneficiaries. 35 percent of beneficiaries were female, not achieving the original target of 40 percent. The beneficiaries benefitted from the upgrading of the public squares and improved streetscapes.  76 percent of the population in project areas was satisfied with the quality of infrastructure financed by the project, exceeding the target of 75 percent (this target was not revised). The project was able to make improvements in regard to upgrading streetscapes and satisfying project beneficiaries. However, the project was not able to achieve the target of number of entities providing cultural and tourism services and operationalizing the National History Park Management Plan. Therefore, this objective is rated Modest. Rating Modest OBJECTIVE 2 REVISION 1 Revised Objective Improve the living environment for the residents living in North of Haiti. Revised Rationale The objective remained the same, but the targets were adjusted. The theory of change remained the same. Page 8 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review HT Cultural Heritage and Tourism Sector (P144614) Outputs:  Three infrastructure assets were properly operated and maintained following project financing, achieving the revised target of three infrastructure assets.  47 sub-projects were financed, exceeding the revised targets of 40 sub-projects being financed. 25 percent of the sub-projects were implemented by women, achieving the revised target of 25 percent. Outcomes:  2,670 meters of streetscapes were upgraded in project target areas, exceeding the revised target of 150 meters. The project exceeded all output and outcome targets under the revised objective resulting in Substantial achievement of this objective. Revised Rating Substantial OBJECTIVE 3 Objective Support the Recipient’s capacity to respond promptly and effectively to an Eligible Emergency, as needed. Rationale The contingency fund was not triggered. Therefore, this objective is not being assessed. Rating Not Rated/Not Applicable OVERALL EFF TBL OBJ_TBL OVERALL EFFICACY Rationale The achievement of the first objective with original targets was rated Modest. The achievement of the second objective with original targets was also rated Modest. Overall achievement was Modest. Overall Efficacy Rating Primary Reason Modest Low achievement OBJR1_TBL OVERALL EFFICACY REVISION 1 Page 9 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review HT Cultural Heritage and Tourism Sector (P144614) Overall Efficacy Revision 1 Rationale The achievement of the first objective with revised targets was Modest. The achievement of the second objective with revised targets was Substantial. Overall achievement was Modest. Overall Efficacy Revision 1 Rating Primary Reason Modest Low achievement 5. Efficiency Economic analysis: The PAD (p. 12) conducted a traditional economic analysis for component A (US$28 million, 62 percent of total project financing). The benefits were defined as tourist inflows and associated expenditures over a time horizon of 10 years. The analysis used a discount rate of 10 percent and calculated an Economic Rate of Return (ERR) of 39 percent and a Net Present Value (NPV) of US$18 million. However, the Economic analysis did not take a reduction of tourist inflows into account. Also, it was based on tourist figures of 2015, which were especially high. Furthermore, the project design assumed that cruise line tourists would visit the PNH-CSSR even though it would take approximately six hours to drive from the cruise ship’s docking location to the monuments. Later, the security situation made this even more impossible. Throughout project implementation, component A’s financing was reduced and actual disbursement for component A was US$16.95 million (planned financing was US$28.0 million), 93.5 percent of total financing. The ICR (p. 19) used the same method as at appraisal and calculated an ERR of one percent and a NPV of minus US$146 million, indicating that the project was not a worthwhile investment. Operational efficiency: The project encountered several significant implementation issues due to weak institutional capacity, conflicts between Technical Implementation Unit (UTE), ISPAN and UNESCO, coordination issues between a large number of partners and stakeholders, the local security deteriorating, and the outbreak of the Covid-19 pandemic. All these issues resulted in significant implementation delays, which required the extension of project implementation by a total of 23 months. Also, due to implementation issues several project activities were dropped and financing in the amount of US$13.05 million. When the project closed US$17.84 million was disbursed (39.6 percent of planned financing). Taking everything together, the project’s efficiency rating is Modest. Efficiency Rating Modest a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Page 10 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review HT Cultural Heritage and Tourism Sector (P144614) Rate Available? Point value (%) *Coverage/Scope (%) 62.00 Appraisal  39.00  Not Applicable 0 ICR Estimate  1.00  Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome Relevance of the objective was Substantial given the objective’s alignment with the World Bank’s most recent Country Partnership Framework (CPF) (FY2016-FY2021). Efficacy was Modest both before and after restructuring due to low achievement. Efficiency was Modest due to significant implementation issues, a negative NPV and an estimated ERR at only 1%. Taking everything together, the project’s outcome rating is Moderately Unsatisfactory before and after the restructuring. Therefore, overall outcome is Moderately Unsatisfactory. a. Outcome Rating Moderately Unsatisfactory 7. Risk to Development Outcome Governmental commitment: According to the ICR (p. 32) the government has not committed to restoring the cultural heritage sites based on the preliminary studies completed by this project. If those sites are not being rehabilitated, there is a high risk of those sites not being resistant to seismic events and eventually collapsing. Technical capacity: An interim authority (AGP) was set up and staffed by the project. However, the UTE was not able to timely hire a suitable lawyer to prepare the permanent AGP formalization decree. As a result, the AGP was not established, putting the sustainability of an effective Park Management System at risk. Also, given the frequent changes in government, there is a high risk that staff, trained under the project, will no longer be in their positions. Political and security situation: Haiti continues to remain in a politically and security volatile situation. If this continues, there is a high risk that tourists will not return to the country, which will negatively impact the development of the local tourism industry and the sustainability of small businesses catering to tourists. Exposure to natural disaster: According to the ICR (p. 26) between 2012 and 2022 Haiti was hit by 16 floods, eight hurricanes, two droughts, two earthquakes, two cholera outbreaks, and the COVID-19 pandemic. There is a high risk of further natural disasters in the future and the cultural heritage sites are at risk of being affected by these. Page 11 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review HT Cultural Heritage and Tourism Sector (P144614) 8. Assessment of Bank Performance a. Quality-at-Entry The project was part of a larger approach to developing the North of Haiti and the World Bank coordinated with the Inter Development Bank (IDB) and USAID who made other investments in the area. According to the PAD (p. 12) the overall implementation risk of the project was High. The World Bank team identified several implementation risks and the main risks as low institutional capacity of government agencies and large number of stakeholders. The World Bank team aimed to mitigate these risks by: i) phasing activities over time taking into account the capacity of the different agencies; ii) anchoring the project at the Ministry of Economy and Finance (MEF), which had a convening capacity for other ministries and government agencies; iii) engaging the Unite Techique de Execution (UTE), an agency with a strong track record implementing projects, as the PIU; iv) using a multi-sector team drawing on experience from different parts of the World Bank; and v) leveraging technical assistance from specialized institutions such as UNESCO. However, these mitigation measures were insufficient. Also, the World Bank team did not identify the risk of choosing the UTE as PIU. UTE did not have any technical knowledge in the area. The World Bank team chose the PIU based on its experience in implementing other donor-financed projects and several project preparations advances for other World Bank-financed projects. It was planned that the technical inputs on cultural heritage were to be provided by the National Cultural Heritage Institute (ISPAN), with technical support from UNESCO. Since ISPAN used to manage previous cultural heritage projects in the North, it disagreed with this arrangement. The UNESCO country representation supported ISPAN in this disagreement which impacted project coordination and required the involvement of UNESCO headquarters. ISPAN was responsible for providing technical oversight for component A. However, due to the disagreement, the project suffered from poor coordination, delays in developing Terms of Reference, and revision of technical documents. Also, the ICR (p. 23) stated that UNESCO entered into an implementation agreement with the government to support the implementation of the project. However, the project experienced many delays during the first phase of the project implementation and in 2018, the World Bank concluded that UNESCO’s support was insufficient resulting in the reformulation of the convention to reduce the scope and simplify the list of remaining deliverables. In addition, the World Bank team did not identify the risk of high turnover of decision makers in the government, which resulted in implementation delays. Finally, the World Bank team did not identify the risk of generation of solid waste by the project and the lack of appropriate infrastructure for the treatment, disposal, and management of it. According to the ICR (p. 22) the UTE found it challenging to manage a complex set of activities and coordinate a large number of partners and stakeholders. Also, the UTE lacked knowledge in World Bank financial management, procurement, and social and environmental safeguards resulting in implementation delays. Furthermore, the project’s institutional set up was overly complex and centralized including three different ministries, various entities and beneficiaries. The ICR (p. 30) states that the project assumed that a large part of the visitors to the PNH-CSSR and its monuments were to be cruise ship tourists. However, this assumption proved to be wrong since the cruise ship tourists were not allowed to leave the area where they docked. As a result, the tourists could Page 12 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review HT Cultural Heritage and Tourism Sector (P144614) have only been coming from the Dominican Republic and the diaspora, which was further restricted during the COVID-19 pandemic. Quality-at-Entry Rating Unsatisfactory b. Quality of supervision According to the ICR (p. 25) the World Bank team conducted 17 supervision missions, of which seven were virtual due to the COVID-19 pandemic. Some if these missions included senior management. The project had four different Task Team Leaders (TTLs) over its life span. All TTLs and most support staff were based outside the country. Especially during the early years, project implementation was negatively affected by the frequent turnover of TTLs not allowing for a continuous monitoring and early identification of implementation bottlenecks. Also, the project experienced a high turnover in and lack of assigned procurement staff resulting in the project not receiving procurement supervision for extended periods of time. According to the ICR (p. 31) supervision improved from 2017 onwards when an action plan was developed to create a more decentralized implementation modality. To address procurement related issues, the project received Hands on Extended Implementation Support (HEIS). However, HEIS only began eight months before project closure. The project was restructured a total of six times to address various implementation challenges by relocating funds to strengthen project management and monitoring, dropping activities, and revising the Results Framework. While some of these actions had a positive impact on project implementation some were insufficient. During the project restructuring in 2020, the number of cultural heritage works was reduced further and the coordination with UNESCO was improved, resulting in an upgrade of the PDO rating from Moderately Unsatisfactory to Moderately Satisfactory. However, with the reduction in cultural heritage works, the project’s focus moved away from cultural heritage reservation to urban upgrading and tourism sector support. Despite these restructurings, the project remained a problem project for five of eight years since these actions came too late. The first project restructuring took place almost three years into project implementation. Furthermore, the lack of cooperation and communication between UTE, ISPAN, and UNESCO was not resolved. The ICR (p. 31) stated that the project at times lacked strategic direction, management support and guidance as well as fiduciary support, especially related to procurement, as mentioned above, and safeguards Quality of Supervision Rating Moderately Unsatisfactory Overall Bank Performance Rating Page 13 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review HT Cultural Heritage and Tourism Sector (P144614) Moderately Unsatisfactory 9. M&E Design, Implementation, & Utilization a. M&E Design The project’s objective was clearly specified. The project’s theory of change and how key activities and outputs were to lead to the intended outcomes was sound but not well reflected in the Results Framework. The Results Framework included four PDO indicators to measure the first aspect of the PDO (“increase the attractiveness of the cultural heritage sites in North Haiti for tourists”) and one PDO indicator to measure the second aspect of the PDO (“improve the living environment for the residents living in North Haiti”). While the selected indicators were measurable, several indicator targets were overly ambitious. Also, the project delivered several outputs that were not included in the Results Framework and lacked targets. According to the PAD (p. 11) the UTE was responsible for the project’s M&E activities. Also, an M&E system was to be put in place for different activities of the project to ensure that required M&E data were to be regularly generated and tracked. b. M&E Implementation According to the ICR (p. 26) the UTE did not establish the M&E system as planned due to delays in hiring experts. Also, the planned baseline study was only delivered in January 2022, six months before project closure. During three of the six project restructurings, the Results Framework was modified. Two of the initial outcome targets were reduced, four of the original intermediate outcome targets were reduced and one intermediate outcome indicator was dropped. Also, one intermediate outcome indicator was added. The project restructuring in 2018 reallocated funds to strengthen the project’s M&E activities. As a result, M&E data were collected from different sources and verified during supervision missions. c. M&E Utilization During the initial years of project implementation until 2018, limited data were available to assess progress towards the achievement of the PDO. Also, the ICR (p. 27) stated that M&E data were not used to inform the project restructurings and Mid-Term Review. After 2018, four years into project implementation, performance and results data were used to inform decision making. M&E Quality Rating Modest 10. Other Issues Page 14 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review HT Cultural Heritage and Tourism Sector (P144614) a. Safeguards The project was classified as category B and triggered the World Bank’s safeguard policies OP/BP 4.01 (Environmental Assessment), OP/BP 4.04 (Natural Habitats). OP/BP 4.09 (Pest Management), OP/BP 4.36 (Forests), OP/BP 4.11 (Physical Cultural Resources) and OP/BP 4.12 (Involuntary Resettlement). According to the ICR (p. 28) the project prepared an Environmental and Social Management Plan (ESMP), and Summary Resettlement Plans (SRPs). The ICR further stated that sub-projects under component A were better monitored since they are at the same location as the project office in Cap Haitien than sub- projects under component B and C, which were located in communities further away. The project faced issues with the treatment, disposal, and management of solid of different natures due to the lack of appropriate infrastructure. Also, the project’s budget for safeguard compliance was insufficient to finance remediation actions. As a result, some of the project’s activities were dropped. Also, according to the ICR (p. 25) the lack of a permanent environmental and social safeguard specialist in Cap Haitien from 2014- 2019 as well as the lack of adequate equipment, made the supervision of safeguard compliance challenging. Since the environmental specialist at the PIU resigned and was not replaced, the safeguard rating was downgraded to Moderately Satisfactory. At project closing the project’s environmental and social safeguard performance was rated Moderately Unsatisfactory due to the non-completion of all planned works such as the development and enhancement of the tourist site Chute de Cotard, requalification of the built fronts of Rue 16 and Rue 20, renovation work on the facades of houses around the Place de Dondon, enhancing Caramel beach of Bas-Limbé and redesigning the roof of a commercial shed on the beach of Saint Michel de Camp Louise. As a result, a post-closure action plan had to be developed and the borrower had to commit to complete all sub-projects in line with applicable operational procedures and relevant mitigation measures. According to the World Bank team (June 7, 2023) the World Bank required environmental and social monitoring and follow-up on construction sites. In addition, a section on environmental and social aspects was added to the half-yearly reports at the World Bank's request. The ICR (p. 28) stated that by September 2022, all triggered safeguard policies and mitigation measures were implemented successfully. Furthermore, the ICR (p. 28) stated that the project’s Grievance Redress Mechanism (GRM) was managed by the PIU’s environmental and social specialist. Complains could be submitted through the telephone, WhatsApp and the internet. As of July 2022, the GRM had received 18 complaints. According to the World Bank team (June 7, 2023) a complaint about Cotard Falls (chute de Cotard) was only successfully resolved after the project closing with the Government’s own resources. This was a complaint made by the local community about poorly executed work. After project closure, the PIU carried out remedial work resulting in the community being satisfied. b. Fiduciary Compliance Financial Management. According to the ICR (p. 29) the project complied with the financial management arrangements throughout implementation. The project assigned a Financial Management Specialist who worked exclusively on the project and with the World Bank to address bottlenecks in a timely manner. The project’s Interim Financial Reports (IFRs) and financial audits were submitted as requested by the Page 15 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review HT Cultural Heritage and Tourism Sector (P144614) financing agreement and were accepted by the World Bank. The ICR stated that even though the IFRs were sometimes submitted with a delay, it did not affect project implementation. According to the World Bank team (June 7, 2023), in March 2019, the auditors included one recommendation regarding the need to establish a monitoring tool for contracts. The external auditor’s opinions were unqualified. The project’s Financial Management rating was rated Satisfactory at project closure. Procurement. According to the ICR (p. 29) the project experienced procurement issues throughout implementation. During the first four years of project implementation the UTE did not have sufficient procurement capacity. Between January and mid-2018 the project did not have procurement staff assigned to it. Also, the project’s procurement process was very centralized when it came to the authorization of payments, going through several levels and requiring approval by the Ministry of Finance. This process slowed procurement down and resulted in implementation delays. According to the World Bank team, the June 2018 restructuring paper included adjustments in procurement aiming to align all procurement arrangements under this operation with the mandatory World Bank’s new procurement framework “World Bank Procurement Regulations for IPF Borrowers (July 2016 revised November 2017)”. The World Bank team also stated that the World Bank identified shortcomings and addressed these through restructurings such as the June 2018 and the August 2020 restructurings. Measures taken included: i) developing a plan to improve internal procedures; ii) setting up a project management computer system; and iii) transferring all contracting activities to STEP. The World Bank provided Hands-on Expanded Implementation Support (HEIS) between October 2021 and project closure (July 2022). However, this happened too late in project implementation and did not allow for any significant improvements. When the project closed, there were 16 ongoing activities (total amount of US$2.3 million) that still had to be completed (they were completed by January 2023). The project’s procurement performance was rated Moderately Satisfactory at project closure. c. Unintended impacts (Positive or Negative) NA d. Other --- 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment Moderately Moderately Outcome Unsatisfactory Unsatisfactory Page 16 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review HT Cultural Heritage and Tourism Sector (P144614) The rating is in error. The ICR Moderately Moderately Bank Performance rates Bank Performance as MU, Satisfactory Unsatisfactory the same as IEG's assessment. Quality of M&E Modest Modest Quality of ICR --- Substantial 12. Lessons The ICR (p. 33-35) provided several lessons learned, which were adapted by IEG:  Assessing cooperation dynamics between local agencies may allow for better project implementation and sustainability. The project design of this project included the cooperation of three different agencies, UTE, ISPAN, and UNESCO. Due to the project choosing UTE as implementing agency, ISPAN and UNESCO were unwilling to corporate initially. Also, UTE lacked the necessary technical capacity and fiduciary experience to successfully implement the project. As a result of these two issues, the project experienced significant implementation bottlenecks and delays.  Equipping a PIU with the necessary capacity before the project becomes effective and ensuring that it has a presence in the implementation area as well as decision power can positively impact project implementation. In this project, the PIU was based in the capital while the project was implemented in the North, not allowing a strong presence in the project area. Also, the PIU was not allowed to approve all operational transactions, which had to go through the government, resulting in substantial delays.  Using Trust Funds to provide Technical Assistance can be a useful tool to conduct the necessary technical studies. This project was not able to start the actual works on the monuments due to most time being spent on procuring and conducting the required architectural, topographic, and geotechnical studies. A lesson learned added by IEG:  Assessing local circumstances when designing a project to increase revenues from tourists, as local circumstances can positively impact project outcomes. This project was based on the assumption that tourists would leave their cruise ship and travel about six hours by land to visit historic sites. This assumption was unrealistic and would have benefitted from being assessed into more detail. 13. Assessment Recommended? No Page 17 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review HT Cultural Heritage and Tourism Sector (P144614) 14. Comments on Quality of ICR The ICR provided an adequate overview of project preparation and implementation, and it was internally consistent and relatively concise. The economic analysis was adequate, and the lessons learned are useful for future World Bank engagement in this area. The ICR only provided limited information on critical areas such as procurement and safeguard compliance despite significant issues. Also, the ICR did not explain what the Community Fund was. Finally, the ICRR did not conduct a split rating. While the ratings do not change, as the ICR explains, the significant change to the design of the project warrants the benefit of a split evaluation and it would have provided useful insights on the challenges even after project changes were made. Thus, the ICR is rated Substantial, but only marginally so. a. Quality of ICR Rating Substantial Page 18 of 18