Document of The World Bank FOR OFFICIAL USE ONLY Report No: ICR00005352 IMPLEMENTATION COMPLETION AND RESULTS REPORT TF0A8246 ON A SMALL GRANT IN THE AMOUNT OF USD 0.9 MILLION TO THE Republic of Turkey FOR Supporting Critical Infrastructure Resilience and Disaster-Risk Awareness in Turkey (P168167) February 3, 2021 Urban, Resilience And Land Global Practice Europe And Central Asia Region Regional Vice President: Anna M. Bjerde Country Director: Auguste Tano Kouame Regional Director: Steven N. Schonberger Practice Manager: Christoph Pusch Task Team Leader(s): Esse Ayse Erkan Orenbas ICR Main Contributor: Esse Ayse Erkan Orenbas ABBREVIATIONS AND ACRONYMS AFAD Turkey Disaster and Emergency Management Agency BCP Business Continuity Planning CI Critical Infrastructure CPF Country Partnership Framework DRM Disaster Risk Management EU European Union EOI Expression of Interest GA Grant Agreement GDP Gross Domestic Product GFDRR Global Facility for Disaster Reduction and Recovery ISR Implementation Status Report M&E Monitoring and Evaluation NDP National Development Plan NGOs Nongovernmental Organizations PDOs Project Development Objectives RFP Request for Proposals SDG Sustainable Development Goal STEP The World Bank’s Systematic Tracking and Exchanges in Procurement TOR Terms of References TUBITAK Scientific and Technological Research Council of Turkey OIZ Organized Industrial Zone TABLE OF CONTENTS DATA SHEET.......................................................................... ERROR! BOOKMARK NOT DEFINED. I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES ......................................... 3 II. OUTCOME........................................................................................................................................ 5 III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME .......... 10 IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME ............................................................................................................................................. 12 V. LESSONS LEARNED AND RECOMMENDATIONS ........................................................ 13 ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS .................................................. 15 ANNEX 2. PROJECT COST BY COMPONENT ......................................................................... 23 ANNEX 3. RECIPIENT, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS .......................................................................................................................................... 24 The World Bank Supporting Critical Infrastructure Resilience and Disaster-Risk Awareness in Turkey (P168167) DATA SHEET BASIC INFORMATION Product Information Project ID Project Name Supporting Critical Infrastructure Resilience and P168167 Disaster-Risk Awareness in Turkey Country Financing Instrument Turkey Investment Project Financing Original EA Category Revised EA Category Not Required (C) Not Required (C) Organizations Borrower Implementing Agency Republic of Turkey Disaster and Emergency Management Authority (AFAD) Project Development Objective (PDO) Original PDO The Project Development Objective is to inform building resilience into critical infrastructures and support disaster risk awareness at the community level. FINANCING FINANCE_TBL Original Amount (US$) Revised Amount (US$) Actual Disbursed (US$) Donor Financing TF-A8246 900,000 895,187 895,187 Total 900,000 895,187 895,187 Total Project Cost 900,000 895,187 895,187 Page 1 of 24 The World Bank Supporting Critical Infrastructure Resilience and Disaster-Risk Awareness in Turkey (P168167) KEY DATES Approval Effectiveness Original Closing Actual Closing 04-Sep-2018 29-Jan-2019 30-Jun-2020 31-Aug-2020 RESTRUCTURING AND/OR ADDITIONAL FINANCING Date(s) Amount Disbursed (US$M) Key Revisions 29-Jun-2020 0.35 Change in Results Framework Change in Loan Closing Date(s) Change in Implementation Schedule KEY RATINGS Outcome Bank Performance M&E Quality Moderately Satisfactory Satisfactory Substantial RATINGS OF PROJECT PERFORMANCE IN ISRs Actual No. Date ISR Archived DO Rating IP Rating Disbursements (US$M) 01 27-Mar-2020 Satisfactory Satisfactory 0.24 ADM STAFF Role At Approval At ICR Regional Vice President: Cyril E Muller Anna M. Bjerde Country Director: Johannes C.M. Zutt Auguste Tano Kouame Director: Ede Jorge Ijjasz-Vasquez Steven N. Schonberger Practice Manager: David N. Sislen Christoph Pusch Task Team Leader(s): Esse Ayse Erkan Orenbas Esse Ayse Erkan Orenbas ICR Contributing Author: Esse Ayse Erkan Orenbas Page 2 of 24 The World Bank Supporting Critical Infrastructure Resilience and Disaster-Risk Awareness in Turkey (P168167) I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES Country Context at Appraisal 1. Turkey’s economic performance since 2000 has been impressive, transforming it into a middle-income country as well as the world’s 17th largest economy. This progress was achieved despite the 2008-2009 global financial crisis. Poverty declined by more than half during the 2002–2012 period, from 44 percent to 21 percent of the population. During this time, Turkey experienced dramatic urbanization, opened to foreign trade and finance, harmonized many of its laws and regulations with European Union (EU) standards, and greatly expanded access to public services, including critical infrastructure and superstructure. However, Turkey’s achievements were challenged by multiple election cycles, regional conflict, challenging international relations, and a failed coup attempt in July 2016 that adversely affected Turkey’s economy and led to double-digit inflation, a large current account deficit, and an increase in unemployment. The Turkish economy has experienced positive adjustments starting from late 2019. Current account imbalances have declined, banks have reduced their external exposure, and portfolio flows have started to recover. However, these achievements have been undermined by COVID-19 pandemic and the economic outlook has worsen. Sector Context 2. Turkey is one of the world’s most disaster-prone countries, where about 95 percent of Turkey’s territory, 70 percent of its population, 83 percent of its GDP, and 76 percent of industrial plants are located near an active fault line. Earthquakes have claimed the highest number of lives and caused the greatest economic loss for the country. Several earthquakes have occurred since 1900 with approximately 94,000 fatalities, a total affected population of 7 million, and direct losses of USD 50 billion1. Climatic events are also on the rise due to climate change with increasing frequency and intensity of urban floods, hail, drought and temperature extremes, which pose the greatest risk to rapidly growing cities in Turkey. The risk is greater for industrialized cities, often located along the coastlines, in flood plains, or along seismic rifts, with their concentration of assets and residents which are vulnerable to disasters. In 2016, the port of Mersin, the second largest port in the country was flooded, disrupting imports and exports to Turkish cities within a 600km radius and international trade. In summer 2017, two rainfall/hail episodes disrupted the largest port, Istanbul, and caused widespread disruption including both physical damage and loss of service in the transport and power systems. 3. Since the late 1990s, there has been increasing recognition by both governments and donors of the need to address disaster and climate risks in medium-term development frameworks, in legislation and institutional structures, and in sectoral policies. Despite substantial efforts that were channeled to seismic risk reduction investments over the last decade, few investments have addressed disaster and climate change risks faced by lifeline and critical infrastructures. In 2013, the Corporate Social Responsibility Association of Turkey together with the United Parcel Service and the World Economic Forum, developed a disaster preparedness initiative for small and medium size enterprises, entitled “Business Disaster Resiliency Program for Turkey� or “Sağlam Kobi�. The initiative also established a ‘Resilient Businesses Platform’ which furnished some 1,400 enterprises with Business Continuity Planning (BCP) training tools2 to ensure businesses are better prepared when the next disaster strikes. Rationale for World Bank Support 4. The Bank has been instrumental in supporting the disaster risk management (DRM) agenda of countries to put in place mechanisms needed to build resilience, mitigate risks, and improve preparedness capacities to respond to disasters all over the world and has a long collaboration history of technical and financial support for resilient cities and infrastructure in the Republic of Turkey. Numerous and pioneering World Bank financed 1 Erdil M. (2013), Earthquake Risk in Turkey, ScienceMag, Vol. 341, Issue 6147, pp 724-725. 2 saglamkobi.org Page 3 of 24 The World Bank Supporting Critical Infrastructure Resilience and Disaster-Risk Awareness in Turkey (P168167) projects3 that were accomplished in the country led to a shift from post-disaster reconstruction and recovery interventions to ex-ante programs, including strengthening the disaster risk management capacities and enhancing the resilience of its critical infrastructure (CI) against natural hazards that can be applied in multiple sectors at different scales. The Bank engagements with the Government focused on mainstreaming disaster risk management into priority sectors’ policies and interventions and increasing urban resilience through sector specific engagements at all levels of the Government. The World Bank Turkey DRM Team, through Building Resilience in Turkey Technical Assistance scheme (TF018854, closed in June 2018), tackled CI and BCP fronts by facilitating key analytical studies in partnership with regional planning and investment authorities as well as organized industrial zones (OIZ). Building on the interest induced by the results of these studies4 and the need to drill further into operational continuity, the Bank supported implementation of the Critical Infrastructure Resilience and Disaster-Risk Awareness in Turkey Project financed by a Global Facility for Disaster Reduction and Recovery (GFDRR) Grant5 in the amount of US$0.9 million. This Project was seen as a vehicle to contribute to efforts for building resilience into lifeline utilities and critical infrastructures and the development of business continuity planning, which have not been fully addressed in Turkey yet. The implementing agency for the Project was Turkey Disaster and Emergency Management Agency (AFAD). Higher Level Objectives to which the Project Contributes 5. The Project was aligned with the Sendai Framework for Disaster Risk Reduction 2015-2030 targets “ to substantially reduce the damages to be suffered by critical infrastructures disaster damage to critical infrastructure and disruption of basic services, among them health and educational facilities, including through developing their resilience by 20306�. The Project contributed to the Sustainable Development Goal 9 of the United Nations to build resilient and well-functioning infrastructures to increase competitiveness as well as to support economic development7. 6. The Project was aligned with the ongoing World Bank Country Partnership Framework (CPF) for Turkey (FY18-FY21). The Sustainability pillar of improving sustainability and resilience of cities, supported policy advice and financing resilience of cities through strengthening facilities for disaster resilience and improving disaster awareness and preparedness. Moreover, the Project supported the goals of the Tenth Five-Year National Development Plan (NDP) of the Government of Turkey for 2014-2018, Pillar 2.3 on Livable Spaces and Sustainable Environment which underlined disaster-resilient, environmentally sensitive and economically competitive settlements, businesses and communities as a key area. The Project is also fully consistent with the Strategic Plan of AFAD (Objectives 2.3, 3.4, 3.5 and 4.1 on enhancing preparedness, mainstreaming disaster risk awareness, promoting community-level risk champions respectively) and the Turkey Disaster Response Plan. Project Development Objectives (PDOs) 7. The Project development objective is to inform building resilience into critical infrastructures and support disaster risk awareness raising at the community level. Key Expected Outcomes and Outcome Indicators 8. The Project aimed at achieving the following PDO level results indicators and sub-indicators: 1) Stakeholder involvement stimulated on understanding risk and risk identification 1.1. Number of critical infrastructures which piloted CI risk assessment control checklists 1.2. Number of enterprises/organizations which informed formulation of OIZ level area BCP 3 Earthquake Rehabilitation and Reconstruction Project (Completed in 2000), Emergency Flood and Earthquake Recovery Project (Completed in 2004), Emergency Recovery Loan, Marmara Earthquake Emergency Reconstruction Project (Completed in 2007), Istanbul Seismic Risk Mitigation and Emergency Preparedness Project (Closed in 2015), Project and Disaster Risk Management in Schools Project (Ongoing). 4 High-level Critical Infrastructure Risk Assessment (CIRA) in the Region of Çukurova (Turkey) and Lifeline Utilities and Business Continuity Planning in Tuzla Istanbul, which were developed by the World Bank under Building Resilience in Turkey technical assistance. 5 Funded by the Government of Japan through GFDRR. 6 https://www.preventionweb.net/sendai-framework/sendai-framework-for-drr/at-a-glance 7 Goal 9-Build resilient infrastructure, promote inclusive and sustainable industrialization and foster innovation. https://www.globalgoals.org/9-industry-innovation-and-infrastructure Page 4 of 24 The World Bank Supporting Critical Infrastructure Resilience and Disaster-Risk Awareness in Turkey (P168167) 1.3. Number of end-users who benefit from the DRM awareness raising zones established in the Scientific and Technological Research Council of Turkey (TUBITAK) science centers 2) Design/planning capacity of the client and partners increased 2.1. Number of sectorial risk assessment methodologies developed and endorsed 3) Percentage of female beneficiaries reached via project activities Components 9. The Project consists of the following three components: a. Component 1. Supporting Disaster Risk Awareness Raising. Under this component, the Project carried out activities to support the establishment of disaster risk awareness raising zone in the Science Center of Konya Metropolitan Municipality8, including the design, procurement and installation of disaster risk awareness raising equipment. b. Component 2. Critical Infrastructure. This component supported studies to identify the level of criticality of infrastructures in the energy and health sectors by: i) carrying out of a multi-hazard risk assessment, ii) developing a web-based tool that can be used online by CI asset owners/operators to conduct disaster risk assessments of CIs, and develop recommendations to address the gaps identified and thus reduce vulnerability of assets, lives and savings against disasters, and iii) producing a report of policy recommendations to identify CIs and reduce disaster risks, conducting studies to prepare draft legislation and disseminating the existing good practices. c. Component 3. Business Continuity Planning. This component financed analytical work to develop a Manual on Preparation and Implementation of Business Continuity Plans and was piloted in the two selected organized industrial zones in Hatay and Koceli provinces. It also included trainings to ensure the sustainability of BCPs. Significant Changes During Implementation 10. The delays were inevitable in successful completion of the tasks due to the pandemic. Some Project activities were postponed and/or disrupted due to the COVID-19 outbreak and corresponding delays in material supply and logistics problems, travel restrictions imposed by the Government, limitation on working hours by curfews and restriction of international travels which resulted in inability of international consultancy firms to travel to Turkey. In June 2020, the Bank extended the closing date of the Project by two months, from June 30, 2020 to August 30, 2020, upon the request of the Government, to ensure completion of all Project contracted activities and achievement of the Project development objectives. 11. At the design stage, most of the Project activities were initially planned to be face-to-face to enable active participation to the events by many firms. However, due to COVID-19 precautions, restrictions on travel, face- to-face meetings, on-site visits, part of the meetings and trainings had to be changed into online modality. The design of the Project provided the PIU team the flexibility to conduct such activities virtually which helped to reach out to a large number of companies. II. OUTCOME Assessment of Achievement of Each Objective/Outcome Relevance of PDOs 12. Relevance of Objectives is rated High. The PDO is highly relevant to Turkey’s development agenda, through implementation, and remains relevant at closing. Enhancing disaster risk awareness and informing building resilience into critical infrastructures are part of the CPF’s (FY18 -FY21) third strategic objective- 8This science center was under the administration of the Scientific and Technological Research Council of Turkey (TUBİTAK) dur ing the appraisal stage of the project. Then, TUBİTAK handed over the management of Center to Konya Metropolitan Municipality in May 2019. Page 5 of 24 The World Bank Supporting Critical Infrastructure Resilience and Disaster-Risk Awareness in Turkey (P168167) deepened sustainable development, aiming to address improving sustainability and resilience of Turkish cities under thematic area 8. The Project, through establishing a disaster risk awareness raising zone, aimed at empowering community-based disaster awareness and increasing disaster risk information which individuals, students, public institutions and nongovernmental organizations (NGOs) benefited from. Furthermore, although it is not explicitly stated in the PDO, the number of females benefitting from the Project activities was one of the PDO level indicators which is fully aligned with CPF objectives to ensure that service delivery in supported cities targets female beneficiaries. 13. The Project has supported Turkey in reaching its national development goals set out in the 11th NDP9 are fully aligned with increasing disaster risk awareness in public, creating disaster resilient settlements, increasing resilience of infrastructures and preventing or reducing the losses of lives and assets, as mentioned in the 11th NDP. Moreover, AFAD Strategic Plan mandates AFAD to draft critical infrastructure risk assessment methodologies for at least three sectors and coordinate the actual conduct of risk assessments with the designated line Ministries. According to the 11th NDP, it is expected that the country’s industrial sector will play a central role industrial Zones and Technology Development Zones, and Free Zones will be improved and thus, these regions will contribute more effectively to the competitiveness and efficiency of the industry sector. 14. The Project is also relevant to the EU legislation requirements which Turkey needs to align with as an EU candidate state. In particularly, in compliance with the EU Directive No. 114 of December 8, 2008 on the identification and designation of European critical infrastructure and the assessment of the need to improve their protection, Turkey is required to prepare a strategy to protect critical infrastructures. The Project supported this activity, and the output would be instrumental to adopt and implement this EU directive in Turkey. Achievement of PDOs (efficacy) 15. The PDO consisted of two distinct objectives: i) informing building resilience into critical infrastructures, and ii) supporting disaster risk awareness at the community level. The PDO outcome indicators provided definitive targets to measure the achievement of the PDO (binary targets: Yes or No), except one indicator which measured the percentage of female beneficiaries who benefitted from project activities. 16. Achievement of Objective 1: Informing building resilience into critical infrastructures Rating: Substantial. 17. This objective was achieved through Component 2 and 3 which consisted of two main tasks: identifying the level of criticality of critical infrastructures in selected pilot sectors and the development of the BCP. 18. The Project supported the development of a risk assessment methodology and the control checklists for critical infrastructure in health and energy sectors10 in close consultation with the Ministry of Health and Ministry of Energy and Natural Resources. Building on these, the Project, through its Component 2, supported development of a web-based software which would be administered by AFAD. To date, risk assessments of 20 critical facilities were piloted drawing on the developed methodology, far exceeding the initial target of 3. Furthermore, to increase capacity on the CI resilience, the Project Component 3 financed online meetings and workshops for representatives of 22 critical infrastructures and respective ministries. A virtual workshop with 53 participants was organized by AFAD in July 2020 to evaluate the results of pilot assessments. 19. Under the BCP task, the Project contributed to the development of the Guideline on Preparation & Implementation of Business Continuity Plans for OIZs. It aims at supporting OIZs in building resilience to quickly adapt to disruptions due to any emergency or possible disaster, while maintaining business operations, the delivery of products/services, protecting people, assets and overall reputation. The Guideline focuses area-based BCP which will also provide an opportunity for individual business to adapt it to a firm or asset level. 9 During the implementation period, implementation of 10th NDP ended and 11 NDP was put in effective as a new plan. 10 It included subsector, such as hospitals and pharmaceuticals in the health sector, and refinery, sub-stations, energy production, storage plants, pipeline and transmission and distribution lines in the energy sector. Page 6 of 24 The World Bank Supporting Critical Infrastructure Resilience and Disaster-Risk Awareness in Turkey (P168167) 20. The process of preparation and implementation of the BCP was further piloted with the collaboration of two pilot OIZs11 in Hatay (İskenderun) and Kocaeli Provinces. Meetings and pilot assessments in these OIZs, which were facilitated by the Project, provided a learning experience for OIZs and enabled OIZ managements and firms to raise their disaster risks awareness, enhance their understanding of potential risks and impact of disasters on their businesses as well as the importance of taking measures to mitigate these risks in the area/premises. A multi-stakeholder and interactive environment were created to discuss the consequences of disruptions in operations faced by the OIZ and possible recovery strategies. It was noted by AFAD that the management of one OIZ and board members considered the need of vulnerability assessment of the buildings within their OIZ. 21. To ensure the sustainability of OIZs’, BCPs, the delivery of “Train the Trainer� regarding a BCP for AFAD and relevant stakeholders. The 3-day training was given to 16 participants online (due to COVID-19 restrictions) August 4-6, 2020. This training was aimed to equip participants with the necessary knowledge and ability to support the OIZ to develop and conduct a BCP exercise. The trainings materials will be further used by AFAD and will provide an opportunity to implement the similar events and trainings nationwide. 22. All three outcome indicators and three corresponding sub indicators related to the outcome of informing building resilience into critical infrastructure were achieved, as evidenced by the Table 1, with some target values surpassed. Out of three related Intermediate Result Indicators, two of them were fully achieved (please see Table 2). The grounds paved by the Project supported activities will lead to even more visible results, if they continue to be used and implemented by the client. The instruments and tools developed by the Project supported are intended to be employed by AFAD and other sectoral ministries. In particular, the web-based software tool developed with the Project will support AFAD and related ministries in developing a critical infrastructure mapping at the national scale, as the number of risk assessments increases over time. AFAD plans to continue expanding and updating this software by adopting the risk assessment methodology and the control checklists for the CI, for an additional 10 sectors12 that were identified by AFAD. By that, the risk information regarding the critical infrastructure will be further expanded to the additional sectors and, thus, contribute to the next important step for informing resilient interventions. In order to reach out to a larger number of firms and OIZs, AFAD also plans to disseminate the BCP Guidelines and promote its implementation in OIZ through organizing a series of meetings and publishing the Guidelines on its web site. All that is expected to lead to even more visible results and enhance the Project impact. Thus, the outcome rating is Substantial. Table 1. PDOs and Outcomes Objective 1: Informing building resilience into critical infrastructures Outcome Indicator Target Actual 1. Stakeholder involvement Number of critical infrastructures piloted CI Critical infrastructure risk assessment control stimulated on understanding risk assessment control checklists- 3 checklist was piloted in 20 facilities risk and risk identification Number of enterprises/ organizations which 6 firms contributed to formulation of OIZ level informed formulation of OIZ level area BCP- 4 BCP 2. Design/planning capacity of 2 sectoral risk assessment methodology for Number of sectorial risk assessment the client and partners health and energy sectors were developed methodologies developed-2 increased 3. Percentage of female Percentage of female beneficiaries reached via 63 percent of participants in the project activities beneficiaries reached via project activities- 50 were women project activities 11 İskenderun OIZ and Kocaeli Machinery Specialized OIZ. 12 10 sectors are energy, transport, water management/dams, communication, critical public servies, critical industrial production/ commercial places, health, agriculture and food, and culture and turism. Page 7 of 24 The World Bank Supporting Critical Infrastructure Resilience and Disaster-Risk Awareness in Turkey (P168167) Table 2. PDOs and Intermediate Result Indicators Outcome Indicator Intermediate Result Indicators Target Actual 1. Stakeholder involvement Number of businesses/enterprises which benefit 8 businesses benefited from formulation of the stimulated on understanding from formulation of the pilot area business pilot area business continuity plan risk and risk identification continuity plan- 40 2. Design/planning capacity Number of institutions which participate into CI 20 institutions participated in CI resilience of the client and partners resilience training- 10 training increased Number of partner/client institutions which 30 institutions benefitted from the exchange of benefit from exchange of best practices- 10 best practices 23. Achievement of Objective 2: Supporting disaster risk awareness at the community level Rating: Substantial 24. This objective was achieved through Component 1. The most critical achievement of this outcome area was the design and establishment of the DRM zone, which covers simulators, platforms for making experiments, playing games, and panels on which visitors can simulate disasters. The designs, which were undertaken by the leadership of AFAD, considered the accessibility of the elderly and disabled people by tactile paving walkways and special audio devices for blind people, and sign language for deaf people. Thus, the Project allowed for the inclusion of all people in the community in the disaster risk awareness activities. 25. The corresponding indicator and sub-indicator only partially achieved this objective (Please see Table 3 and 4). The targets were associated with the number of people or community groups visiting the DRM zone. As of January 29, 2021, 5,316 people visited the DRM zone, and 49 percent of the visitors were female. The number of visitors achieved is below target13. Due to the Government’s strict measures to combat the COVID-19 pandemic, the mobility of people was reduced, and the number of visitors was also reduced as a result. However, as the country recovers and eases some restrictions for reopening of these places, it is expected that the targets will surpass the initial targets. Therefore, the under-achievement of the planned target value of number of visitors, and non-governmental organization and community groups should not be interpreted as a failure of the project objectives. Furthermore, school aged children and students are important target group, which will benefit from DRM zone and can make a significant contribution to enhancing disaster awareness and preparedness by sharing the knowledge with their family members and classmates. The DRM zones will facilitate equipping them with the knowledge and skills which may save their lives. Moreover, the DRM zone developed under the Project will be a showcase for AFAD for the establishment of further platforms in different locations across the country. As the designs of the DRM zones have been developed in such a way that they can be adjustable and flexible as per the size of the space and scope of the content, this model can be replicated and established in different communities and locations. Active involvement of AFAD in the design and implementation of activities to establish DRM zone contributed to AFAD gaining experience to establish such DRM platforms at this scale. AFAD will assume responsibility to guide the design and establishment of DRM zones to be established by interested municipalities and other local organizations. As the number of DRM zones increases, they would be integral part of the Turkey disaster risk management systems in future. Thus, the achievement of the outcome 2 is rated Substantial. Table 3. PDOs and Outcomes Objective 2. Supporting disaster risk awareness at the community level Outcome Indicator Target Actual 1.Stakeholder involvement Number of end-users who benefit from the stimulated on understanding risk and DRM awareness raising zones established- 5,316 people visited the center as of January risk identification 50,000 29, 2021. 3.Percentage of female beneficiaries Percentage of female beneficiaries reached via 49 percent of participants in the project reached via project activities project activities- 50 activities were women. 13 Around 250,000 people visit Science Center annually in normal time. Based on this, it was targeted 50.000 visitors/users during the design stage. Page 8 of 24 The World Bank Supporting Critical Infrastructure Resilience and Disaster-Risk Awareness in Turkey (P168167) Table 4. PDOs and Intermediate Result Indicators Outcome Indicator Intermediate Result Indicators Target Actual 1. Stakeholder involvement Number of NGOs/community-based Since official opening of the DRM stimulated on understanding risk and organizations reached out to via visibility awareness raising zone could not be done yet risk identification activities on DRM awareness raising zones -4 due to the cancellation of the events by COVID-19 pandemics, the DRM zone is not visible between NGO and community groups. 26. The overall efficacy rating is assessed as Substantial given two outcome areas achieved Substantial results (Please see Table 5). Table 5. Overall Efficacy Rating Outcome Area Rating Outcome Area Rating Objective 1: Informing building resilience into critical infrastructures Substantial Objective 2. Enhancing disaster risk awareness at the community level Substantial Overall Efficacy Substantial Efficiency 27. Efficiency of the Project is rated as Substantial to achieve the PDO. The Project has reached its objectives within its budget and with a 2-months extension of the closing date, which happened due to the COVID-19 impact on the pace of the project implementation. Establishment of an operational DRM awareness zone, developing a BCP Guideline, creating a platform for criticality assessment are all new and innovative activities in Turkey. They will establish a precedent for future programs in critical infrastructure resilience and raising disaster risk awareness which are added value in this project. AFAD plans to scale up the activities introduced and piloted by the Project nationwide. That will provide indirect benefit in the long run as more people and firms benefit from them. In addition, the benefit of capacity building activities for beneficiaries including AFAD as well as disaster risk awareness activities will also accrue in the longer term. 28. Of the USD 900,000 approved for the Project implementation by AFAD, USD 0.89 million was disbursed as of October 31, 2020. The two-month extension of the Project closing date did not call for the budget increase and had a positive impact regarding the efficiency of the Project in terms of use of the funds. Delays in the Project implementation thing date extension were due to the COVID-19 impact on the progress, as well as the fact that some tasks were linked to each other, and, therefore, the payments to the consultancy firms and the contractor could not be paid until the tasks were completed satisfactorily. Therefore, without the 2-month extension, the unused project resources would have to be refunded, while the project would have remained unfinished and the expected outcomes would not likely have been achieved. 29. All activities were designed, executed and monitored by AFAD’s own staff. The AFAD team satisfactorily facilitated and engaged with stakeholders to have them actively participate in project activities. AFAD reviewed all planned outputs on time with joint site visits before COVID-19 outbreak. Overall Outcome Rating 30. The overall outcome of the project is Moderately Satisfactory due to its high relevance and substantial efficacy and efficiency (Please see Table 6). The Project’s minor shortcomings in the efficacy were related to the slowdown of activities due to the unexpected COVID-19 pandemic which unfavorably affected not only achieving some specific outcomes but also the overall project implementation. The likelihood of achieving the objective is high once restrictions are eased and normal mobility is restored Nevertheless, the Project demonstrated substantial results in its intended outcomes. Page 9 of 24 The World Bank Supporting Critical Infrastructure Resilience and Disaster-Risk Awareness in Turkey (P168167) Table 6. Overall Outcome Ratings Relevance of PDO High Efficacy Substantial Efficiency Substantial Overall Outcome Rating Moderately Satisfactory Other Outcomes and Impacts 31. Institutional Strengthening. As the implementing Agency, at the beginning of the Project, AFAD had no previous experience in managing the World Bank projects and this led to the agency facing a steep learning curve, especially in fiduciary aspects of the Project. During the Project implementation, the Bank team closely supported the AFAD staff in managing procurements, financial management, to strengthen their institutional capacity and skills. 32. AFAD developed another Project by securing financing from Directorate General for European Civil Protection and Humanitarian Aid Operations, based on the outputs of the critical infrastructure activities. With the new project, risk assessment and criticality methodology will be tested in the İzmir Aliağa region where important energy facilities are located. This will be an opportunity to improve the methodology and expand its use across Turkey. 33. At first, the BCP instrument was a very new topic for OIZ administrations. The AFAD team sometimes faced criticism and was questioned why a BCP is needed when there is already an emergency plan. As the Project moved forward, it became clear that BCP were different from the existing contingency plans. OIZ administrations’ views have started changing, and, at the time of the Project completion, they have expectations for the dissemination of these plans. 34. AFAD noted that there were examples of BCPs for businesses in different sectors around the world, but a plan covering a large number of businesses on the scale of the OIZ was very rare. For this reason, the BCP guideline which is one of the outputs financed by the Project has been put forward as a unique resource not only for Turkey, but the international community. 35. Gender. While the Project did not have any solely women focused interventions, it did have targets for female beneficiaries and participation in the Project activities. In the workshops and training, the number of female participants exceeded the number of participating men and surpassed the overall target of 50 percent with 56 percent. This could be attributed to women’s empowerment and employment as experts in the areas of DRM, resilience of critical infrastructure and business continuity areas, which are new to Turkey. The AFAD also established a team, with a majority of women members. III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME Key Factors During Preparation 36. The objectives of the Project and its activities were jointly identified by AFAD and the Bank team. Given the modest budget and firm timeline of the Project14, the Project team targeted the activities, which would be reference works for future CI risk assessment, BCP and DRM zones and were at the right level of ambition for this Project. The results framework of the Project was simple, straightforward and fully aligned with the PDO. 37. The project experienced initial delays totaling 4 months. The Project was approved by the Bank in August 31, 2018 and the Grant Agreement (GA) between the Republic of Turkey and the Bank was signed on January 23, 2019, which became effective on March 5, 2019. The signing of the GA, inclusion of the project into the public investment program, establishment and endorsement of the designated account and specimen holders respectively consumed considerable time which could have been spent on the actual implementation. 14 Based on the conditions of the Trust Fund financing this project, the project activities should have been completed no later than June 30, 2020. Page 10 of 24 The World Bank Supporting Critical Infrastructure Resilience and Disaster-Risk Awareness in Turkey (P168167) 38. Risk ratings and mitigation measures were accurate at the appraisal stage. The Project Paper accurately evaluated the overall implementation risk rating as Moderate, since AFAD was new to managing the WB financed Project that was governed by the Bank policies and procedures, although they have experience in conducting projects with regard to Turkish regulations. Improvement of staff capacity in line with the needs of the AFAD particularly in fiduciary areas was clearly pointed out in the Project Paper. The risk mitigation measures to strengthen the capacity were mainly relying on hiring external individual consultants or appointing staff internally from other related departments of AFAD. However, implementation of the Project continued with existing capacities with intensive hand on support from the Bank side. 39. Risk mitigation measures regarding project management were identified at design stage to allow smooth implementation. Some of the mitigation measures for procurement related project risks were: (a) ensuring that regular support/advice including training was provided by the Bank team to AFAD during the initial implementation phase; (b) developing the initial Procurement Plan together with AFAD; (c) supporting the preparation of the template bidding documents/ together with the technical specifications and terms of references in advance; (d) conducting regular supervision and implementation support missions by the World Bank, and (f) initiation and implementation of procurement activities through simplified methods agreed between AFAD and the Bank. Key Factors During Implementation 40. In order to compensate for the initial time delays, AFAD and the Bank team started to work on developing a draft procurement plan, financial management systems of the Project as well as terms of references (TORs), expressions of interest (EOI) and request for proposals (RFP) for consulting services under the three components. All that allowed to compensate for the time spent between the GA signing and grant effectiveness and avoid any significant delays in the Project implementation. Pursuant to the signature of consultancy contracts and official kick-off of analytics and good contracts, the AFAD team accelerated the execution of the Project. On the other hand, identification of respective ministries and OIZ managements and timely contact with them during the preparation stage contributed to the readiness of the Project and helped to mitigate risks related to stakeholders. 41. Since the early of 2020, the COVID 19 pandemic outbreak was unexpected shock in Turkey as well as in the world. The delays experienced in the project implementation, which were beyond the control of AFAD and had a negative impact on the implementation speed of the Project, were due to: i) material supply and logistics problems under component 1: ii) travel restrictions imposed by the Government and limitation on working hours by curfews imposed from March to June 2020; iii) inability of internationally contracted firms to travel to Turkey15; and iv) cancellation of planned face-to-face meetings, workshops, trainings and site visits which influenced the timing of deliverables and disbursement. These challenges led to the need for more implementation time than the Project’s original time frame. The Bank granted the extension of the Project to August 30, 2020 based on AFAD’s extension request by 2 months, as per the conditions of Trust Fund. Implementation of planned activities for consultancy services resumed virtually until the project closing due to the pandemic’s outbreak. 42. Although the Project was designed as recipient executed, the implementation period required close follow-up and support from the Bank Task Team to the AFAD in particular on preparing the TORs, EOI, RFPs and technical documents, reviewing of project deliverables and preparing financial reports. In addition, support was provided by the Bank team through participating site visits missions, meeting with consultancy firms as well as day-to-day communications. The AFAD expressed their appreciation of support provided by the World Bank Task Team during the implementation process and even in the grace period to finalize the Project’s closing activities. 15 Two consultancy firms, which received contracts for component 2 and 3, were Italian, and could not come to Turkey. Page 11 of 24 The World Bank Supporting Critical Infrastructure Resilience and Disaster-Risk Awareness in Turkey (P168167) IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME Bank Performance 43. Quality at entry. The Bank performance at entry is assessed as Satisfactory. The Bank team worked together with AFAD on the design of the Project, components and fiduciary arrangements. Based on previous World Bank analytical works in this area, the Project design was solid and appropriate from a technical and financial perspective. 44. Quality of supervision. Quality of supervision is assessed as Satisfactory. The Bank provided intensive support to the client throughout implementation. The Bank team was involved in providing technical support during the preparation of the TORs, the review of project outputs, and participation in the meetings, as requested by AFAD, and was instrumental in helping AFAD to enhance its skills and institutional capacity. The procurement and FM support continued in the implementation period. Procurement Plan revisions were carried out to adjust to changes in components. The FMRs, which covered the financial status of the contracts and projects, were submitted by AFAD as per the GA requirements. Furthermore, to update the project status in the Bank systems, the Task Team furnished one Implementation Status Report (ISR), approved by the Bank management on March 27, 2020. While it should be acknowledged that the Bank team overlooked preparation of an initial ISR which was due in three months after the grant approval, this shortcoming, however, did not lead to any implementation bottlenecks since the Bank team did frequent follow-ups on the Project implementation progress with AFAD. That allowed for the timely introduction of the correcting measures, whenever needed. 45. Quality of Monitoring and Evaluation (M&E). The M&E overall was satisfactory. The design of the results framework, indicators, and targets was fit for purpose, and the M&E implementation and utilization were found to be adequate. On the other hand, the binary nature of the PDO indicators are easy to measure, however, it makes it difficult to assess the quality of outcomes. Environmental Safeguards, Social Safeguards, and Fiduciary Compliance Environmental and Social Safeguard Compliance 46. The environmental category of the project was defined as C, since the Project financed no civil works, which may have had adverse impacts on the environment. Therefore, there was no issue relating to compliance with environment and social safeguards. Fiduciary Compliance 47. Procurement. The activities under the Project followed the ‘World Bank Procurement Regulations for IPF Borrowers’ (July 2016- revised November 2017) (‘Procurement Regulations’). The World Bank’s Systematic Tracking and Exchanges in Procurement (STEP) system was used to prepare, clear, and update the project’s Procurement Plan and to conduct all procurement transactions of the project in line with the provisions stipulated in the Grant Agreement. All contracts were subject to post-review with regards to procurement in accordance with the procedures set forth in paragraph 4 of the World Bank Procurement Regulations for IPF Borrowers. Close follow-up and assistance from the Bank’s procurement team were provided to address procurement issues on time. 48. The procurement arrangements followed by AFAD were generally in compliance with the Bank procedures. However, delays were observed for the revision and update of the procurement plan as well as uploading the mandatory documents into STEP by AFAD. The Bank Team provided hands-on support to enter necessary information into STEP. 49. One procurement complaint was received for the consultant’s selection for “Consulting Services for Critical Infrastructure Methodology Development Study� under Component 2. The Bank team reviewed the selection procedure followed by the implementing agency and found that there weren’t any issues to object to the outcome of the selection. Page 12 of 24 The World Bank Supporting Critical Infrastructure Resilience and Disaster-Risk Awareness in Turkey (P168167) 50. Financial Management. The Project remained in compliance with the Grant Agreement covenants on financial management. The Bank’s financial management team provided timely support to AFAD during the implementation of the project regarding the financial management issues. The interim unaudited financial reports were submitted in a satisfactory format and content. The Bank’s team conducted financial management supervision in January 2020 on the basis that the project should be supervised by the Bank FMS once in its lifetime in a field review in line with the WB guidelines for small grants. The financial management rating for the Project was satisfactory with regard to supervision results. The Bank team secured an exception to the annual audit requirement due to the small number of transactions and amount in 2019, and thus the 2019 audit was conducted with the 2020 audit in December 2020. The Auditors did not identify any findings that could have financial implications on the Project so that they issued a clean audit opinion on the project financial statements. Risk to Development Outcome 51. The Project supported activities remained of high strategic relevance with the Government priorities on building resilience and increasing awareness at the community level. The risk to development outcome relates to the sustainability of the outcomes, as this sustainability would be achieved through the meaningful expansion of measures against critical infrastructures and mainstreaming of the business continuity plan notion. The Project supported outcomes have introduced these concepts in the form of manuals and online platforms to increase awareness and knowledge of end users, and, accordingly, they can be taken to the next level via sustainable financing investment projects. Therefore, given that capacity has been built in-house, AFAD’s commitment to continue the use of the products developed under the project and establishing such financing mechanisms to increase disaster resilience, in collaboration with respective ministries are positive indications for the sustainability of the Project supported outcomes. V. LESSONS LEARNED AND RECOMMENDATIONS 52. The Project preparation and implementation have generated important lessons and recommendations that can serve as an input to future operations in RETF projects. The lessons are summarized as follows: 53. Familiarization of the unexperienced client with the Bank’s procurement rules and procedures takes time, which needs to be taken into account during the operation’s design and implementation timeline, and requires attention and support from the Bank teams to build the client’s capacity and ensure smooth project implementation. AFAD faced a need to go through the steep learning curve, and significant efforts had to be made to increase its capacity in financial management and procurement, since the agency had no experience in implementing and managing the World Bank projects. There was substantial need from the Bank side to help the clients with proper planning and provide close hands-on and implementation support on procurement and contract management, as needed. Therefore, sufficient time for capacity development activities, the client team’s skill compositions and required time for implementation support should be taken into consideration during the design of the Project. 54. Establishing the necessary set-up for implementation of a recipient executed trust fund-financed projects in the Turkish context was considerably time consuming. The GFDRR funds was approved by the Bank Management in August 2018. However, the GA between the Republic of Turkey and the Bank was only signed on January 29, 2019, since it took significant time to include the project into the public investment program, and this time could have been spent instead on the Project implementation. The required time for project implementation should consider decision delays during preparation. 55. In addition, due to the COVID-19 outbreak which appeared in early March 2020, all project activities were suspended for a while. The consulting companies which were selected for Components 2 and 3 were Italian firms. They could not travel to Turkey due COVID-19 related restrictions and continued their work via telecommuting. The corresponding meetings and trainings were also conducted virtually. Similarly, the contractor for Component 1 could not go onsite due to travel restrictions and lockdowns. However, the contractor continued to produce and install equipment at a slower pace. It can be noted that although the working conditions worsened, the client’s teams adopted new working conditions to complete activities. Page 13 of 24 The World Bank Supporting Critical Infrastructure Resilience and Disaster-Risk Awareness in Turkey (P168167) 56. Designing the projects in a more flexible way to allow for the adjustments by implementing agencies against unexpected situations serves to fulfill emerging needs of projects to achieve its objectives. As mentioned before, due to COVID-19 restrictions, AFAD had to cancel or postponed planned project activities. To overcome these, the workshops, trainings and meetings which were planned as face to face were substituted for virtual or online modality given the pandemic circumstances. Most of the activities were conducted online, except activities for DRM Zone establishments. AFAD was confident that the Project was successfully implemented under this new implementation arrangements. Therefore, the project should include design elements which ensure flexibility for project implementation. 57. The Bank and client should ensure that necessary measures are taken for stakeholder engagement from the beginning of the Project design stage. AFAD engaged the relevant ministries, organized industrial zone managements, and other related agencies as well as requested feedback from them. Effective engagement of the stakeholders made it possible especially for the organized industrial zones to keep an open mind to different resilience options, and collaborate in providing data, attending workshops, training and meetings, and providing feedback. . Page 14 of 24 The World Bank Supporting Critical Infrastructure Resilience and Disaster-Risk Awareness in Turkey (P168167) ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS A. RESULTS INDICATORS A.1 PDO Indicators Objective/Outcome: Objective 1: Informing building resilience into critical infrastructures Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Stakeholder involvement Text No Yes Yes Yes stimulated on understanding risk and risk identification 03-Sep-2018 30-Jun-2020 30-Jun-2020 31-Aug-2020 Number of critical Number 0.00 0.00 3.00 20.00 infrastructures which piloted CI risk assessment control 03-Sep-2018 30-Jun-2020 30-Jun-2020 31-Aug-2020 checklists Number of Number 0.00 4.00 4.00 6.00 enterprises/organizations which informed formulation 03-Sep-2018 30-Jun-2020 30-Jun-2020 31-Aug-2020 of OIZ level area BCP Number of end-users who Number 0.00 50,000.00 50,000.00 5,316.00 benefit from the DRM awareness raising zones 03-Sep-2018 30-Jun-2020 30-Jun-2020 31-Aug-2020 Page 15 of 24 The World Bank Supporting Critical Infrastructure Resilience and Disaster-Risk Awareness in Turkey (P168167) established in TUBITAK science centers Comments (achievements against targets): Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Design/planning capacity of the Text No Yes Yes Yes client increased 03-Sep-2018 30-Jun-2020 30-Jun-2020 31-Aug-2020 Number of sectorial risk Number 0.00 0.00 2.00 2.00 assessment methodologies developed and endorsed 03-Sep-2018 30-Jun-2020 30-Jun-2020 31-Aug-2020 Comments (achievements against targets): Sectoral risk assessment, which were developed through Project contributed to capasity increase of AFAD. AFAD through sofware which developed has an opportunıty to condut more assessment which can help risk mapping of facilities in Turkey. Also, AFAD can adapt this metholody to other critical sectors that it identied which provide oppurtinity to conduct more assessement in diffeent sectors. Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Percentage of female Percentage 0.00 50.00 50.00 56.00 beneficiaries reached via project activities 03-Sep-2018 30-Jun-2020 30-Jun-2020 29-Jan-2021 Page 16 of 24 The World Bank Supporting Critical Infrastructure Resilience and Disaster-Risk Awareness in Turkey (P168167) Comments (achievements against targets): The percentage of the women who particiapted in project activites for critical infrastrure component and BCP compoent was % 63. Due to the Covid-19 pandemic restrictions in place since March 2020, as of Janaury 29, 2021, 5,316 people visited the DRM zone and 49 percent were female. The average value of all activies was added as achievement. Objective/Outcome: Objective 2: Supporting disaster risk awareness at the community level Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Stakeholder involvement Text No Yes Yes Yes stimulated on understanding risk and risk identification 03-Sep-2018 30-Jun-2020 30-Jun-2020 31-Aug-2020 Number of critical Number 0.00 0.00 3.00 20.00 infrastructures which piloted CI risk assessment control 03-Sep-2018 30-Jun-2020 30-Jun-2020 31-Aug-2020 checklists Number of Number 0.00 4.00 4.00 6.00 enterprises/organizations which informed formulation 03-Sep-2018 30-Jun-2020 30-Jun-2020 31-Aug-2020 of OIZ level area BCP Number of end-users who Number 0.00 50,000.00 50,000.00 5,316.00 benefit from the DRM awareness raising zones 03-Sep-2018 30-Jun-2020 30-Jun-2020 31-Aug-2020 established in TUBITAK science centers Comments (achievements against targets): Page 17 of 24 The World Bank Supporting Critical Infrastructure Resilience and Disaster-Risk Awareness in Turkey (P168167) Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Percentage of female Percentage 0.00 50.00 50.00 56.00 beneficiaries reached via project activities 03-Sep-2018 30-Jun-2020 30-Jun-2020 29-Jan-2021 Comments (achievements against targets): The percentage of the women who particiapted in project activites for critical infrastrure component and BCP compoent was % 63. Due to the Covid-19 pandemic restrictions in place since March 2020, as of Janaury 29, 2021, 5,316 people visited the DRM zone and 49 percent were female. The average value of all activies was added as achievement. A.2 Intermediate Results Indicators Component: Component 1 Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Number of NGOs/community- Number 0.00 4.00 4.00 0.00 based organizations reached out to via visibility activities on 03-Sep-2018 30-Jun-2020 30-Jun-2020 29-Jan-2021 DRM awareness raising zones Comments (achievements against targets): Since the DRM Awareness zone has fully operationalized towards the end of project and low number of visitors due to COVID-19, this target could not been achieved. It is noted by the AFAD that the official opening of the DRM Awareness Zone did not done yet and the programs for NGO or other community groups could not organized due the COVID limitations. Page 18 of 24 The World Bank Supporting Critical Infrastructure Resilience and Disaster-Risk Awareness in Turkey (P168167) Component: Component 2 Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Number of institutions which Number 0.00 10.00 10.00 20.00 participate into CI resilience training 03-Sep-2018 30-Jun-2020 30-Jun-2020 31-Aug-2020 Comments (achievements against targets): Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Number of partner/client Number 0.00 10.00 10.00 30.00 institutions which benefit from exchange of best practices 03-Sep-2018 30-Jun-2020 30-Jun-2020 31-Aug-2020 Comments (achievements against targets): 30 institutions benefitted from exchange of best practices through meetings, trainings, and workshops. Component: Component 3 Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Number of Number 0.00 40.00 40.00 8.00 businesses/enterprises which benefit from formulation of the 03-Sep-2018 30-Jun-2020 30-Jun-2020 31-Aug-2020 Page 19 of 24 The World Bank Supporting Critical Infrastructure Resilience and Disaster-Risk Awareness in Turkey (P168167) pilot area business continuity plan Comments (achievements against targets): AFAD plans to conduct meeting to disseminate of outputs produced under the Project starting from January 2020 which could not have been organized during the project implementation. These planned meetings by AFAD's own effort are expected to contribute to increase the number of firms reached which would exceed the target. Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Number of partner/client Number 0.00 10.00 10.00 30.00 institutions which benefit from exchange of best practices 03-Sep-2018 30-Jun-2020 30-Jun-2020 31-Aug-2020 Comments (achievements against targets): 30 institutions benefitted from exchange of best practices through meetings, trainings, and workshops. Page 20 of 24 The World Bank Supporting Critical Infrastructure Resilience and Disaster-Risk Awareness in Turkey (P168167) B. ORGANIZATION OF THE ASSESSMENT OF THE PDO Objective/Outcome 1 Informing building resilience into critical infrastructures 1. Stakeholder involvement stipulated on understanding risk and risk identification. Outcome Indicators 2. Design/planning capacity of the client and partners increased. 3. Percentage of female beneficiaries reached via project activities. 1. Number of institutions which participate in Critical Infrastructure resilience training. Intermediate Results 2. Number of businesses/enterprises which benefit from the formulation of the pilot area business continuity plan. Indicators 3. Number of sectorial risk assessment methodologies developed and endorsed. 4. Number of partner/client institutions which benefit from exchange of best practices. 1. At least representatives of the 22 institutions participated in the workshops and meetings for critical infrastructure resilience trainings and checklist development and testing focus group meetings. 2. Methodology for risk assessment for health and energy sectors, 12 subsectors and criticality checklists was developed. 3. The software for criticality assessment and measures to be done were developed and installed into AFAD IT systems. 4. Business Continuity Planning Guideline was developed. It covers the steps of developing BCP, business impact analysis to assess the impact of a disruption on the lifeline utilities, their capacity and supply-chains, and recovery strategies. Key Outputs by Component 5. Around 70 representatives of businesses and OIZ managements participated in development of BCP. In-depth interviews (linked to the achievement with six companies were conducted with regard to business continuity assessment. of the Objective/Outcome 1) 6. The 3-days trainer’s training was given to 16 participants online (due to COVID-19 restrictions) August 4-6,2020 on BCP. Four AFAD staffs received certification on business continuity. 7. The number of facilities which benefit from formulation of the pilot area business continuity plan was 8 during the project implementation. AFAD aims at organizing meetings starting from January 2021 for the firms in OIZs to disseminate the BCP Guideline and its implementation. Through this, the goal is that a significantly larger number of firms could be reached which exceeds target of 40 firms. Objective/Outcome 2. Supporting disaster risk awareness raising at the community level 1. Stakeholder involvement stipulated on understanding risk and risk identification. Outcome Indicators 2. Percentage of female beneficiaries reached via project activities. Page 21 of 24 The World Bank Supporting Critical Infrastructure Resilience and Disaster-Risk Awareness in Turkey (P168167) Intermediate Results 1. Number of end-users who benefit from the DRM awareness raising zones established in TUBITAK science centers. Indicators 2. Number of NGOs/community-based organizations reached out to via visibility activities on DRM awareness raising zones. 1. Disaster Risk Awareness Zone was established. Key Outputs by Component 2. Necessary equipment, platforms and panels were installed. (linked to the achievement 3. Training for the maintenance of the equipment was given to staff working in the Konya Science Center. of the Objective/Outcome 2) 4. 5,316 people visited the center as of January 29, 2021. Page 22 of 24 The World Bank Supporting Critical Infrastructure Resilience and Disaster-Risk Awareness in Turkey (P168167) . ANNEX 2. PROJECT COST BY COMPONENT Amount at Approval Actual at Project Percentage of Approval Components (US$M) Closing (US$M) (US$M) Supporting Disaster Risk .45 .34 76 Awareness Raising (RETF) Critical Infrastructure (RETF) .25 .31 124 Business Continuity Planning .20 .24 120 (RETF) Implementation .10 0.096 96 Support/Supervision (BETF) Total 1.00 0.986 Page 23 of 24 The World Bank Supporting Critical Infrastructure Resilience and Disaster-Risk Awareness in Turkey (P168167) ANNEX 3. RECIPIENT, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS 1. The draft Implementation Completion Report was shared with the AFAD by the Bank team to get their feedback. They agreed that the report reflected the achievements of the project. AFAD is currently preparing their project completion report. 2. AFAD team pointed out that reducing the vulnerability of businesses against disasters or emergencies is possible with Business Continuity Plans in place. Business continuity strategy and vision will be taken into account in all future subsequent works. It is aimed at disseminating the results of this study to other OIZs with training and workshop organizations. 3. As the project moved forward, it became clear that there were examples of business continuity plans for businesses in different sectors around the world, but a plan covering a large number of businesses on the scale of the OIZ was very unusual. For this reason, the BCP guideline, which is the one output of the project, has been put forward as a unique resource for the international community as well as for Turkey. 4. Simulators, experiments, games, and panels which visitors can use to simulate disasters, were originally designed under Component 1. The AFAD can replicate them easily in future projects in various platforms, centers, schools or parks in other localities, using the knowledge gained from the project. 5. Accessibility for disabled persons to the disaster awareness zone was achieve by having a tactile paving walkway and special audio device for blind people and sign language for deaf people. This project also contributes to increase the disaster risk awareness of disabled people. Page 24 of 24