Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Somalia - WAPR (P167826) Report Number: ICRR0023733 1. Project Data Project ID Project Name P167826 Somalia - WAPR Country Practice Area(Lead) Somalia Water L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IDA-D4970 28-Feb-2023 29,522,160.61 Bank Approval Date Closing Date (Actual) 01-Jul-2019 28-Feb-2023 IBRD/IDA (USD) Grants (USD) Original Commitment 42,000,000.00 0.00 Revised Commitment 31,530,581.60 0.00 Actual 29,526,183.29 0.00 Prepared by Reviewed by ICR Review Coordinator Group Ihsan Kaler Hurcan Peter Nigel Freeman Ramachandra Jammi IEGSD (Unit 4) 2. Project Objectives and Components DEVOBJ_TBL a. Objectives According to both the International Development Association (IDA) Financing Agreement (p.6) dated October 8, 2019, and the Project Appraisal Document (p.15) dated June 4, 2019, the objective of the Water for Agro- Pastoral Productivity and Resilience Project (WAPR) was “to develop water and agricultural services among agro-pastoralist communities in dry-land areas of Somalia.” Page 1 of 25 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Somalia - WAPR (P167826) b. Were the project objectives/key associated outcome targets revised during implementation? Yes Did the Board approve the revised objectives/key associated outcome targets? No c. Will a split evaluation be undertaken? No d. Components The project consisted of four components as defined in the Financing Agreement (pp.6-7) and the Project Appraisal Document (pp.15-20). It was designed to scale up activities piloted in Puntland and Somaliland under the previous Water for Agropastoral Livelihoods Pilot Project (WALP – P152024) and start new pilots in the states of Galmudug and South West while continuing with capacity development support. A. Support the Development of Multiple Use Water Sources (Appraisal cost: US$15.0 million; revised cost at the project restructuring: US$12.0 million; actual cost: US$12.0 million) This component consisted of two subcomponents aimed at financing investments in key water management infrastructure for harvesting, storing, and delivering water for people, livestock, and agriculture in dry-lands of Somalia: A.1. Construction of New Community Water Points. This component was to finance the construction of small sand and subsurface dams in dry riverbeds, surface water storage infrastructure (such as berkads- water basin to store surface water between 3 and 12 cubic meters—and haffir dams—earth dams at runoff concentrating flow path without clear stream bed), area infiltration interventions (such as semicircular bunds or soil bunds), and rock catchments. If these options were not available, this subcomponent was to finance the construction of boreholes for groundwater extraction. A.2. Rehabilitation of Community Water Points. This subcomponent was to finance the rehabilitation of existing water infrastructure and boreholes. The subcomponent was also to finance the construction of zero- emission standpipes or shallow wells with hand or solar pumps and watering troughs for livestock. To ensure the availability of water supply during droughts, selected project sites could include multiple interventions. The Wadi Evaluation Tool, which was to be used for the diversification of water resources, extensive ground truthing, and groundwater assessment to be implemented under this subcomponent, was expected to lead to an increase in the supply of water and mitigate the risk of droughts and climate change. B. Institutional and Capacity Development. (Appraisal cost: US$6.0 million; revised cost at the project restructuring: US$5.0 million; actual cost: US$5.0 million) This component was to finance activities for the development of knowledge systems and institutions to support more integrated and sustainable agriculture and water services in the ministries, agencies, and project states. The component consisted of two subcomponents: B.1. National and State Institutional Capacity Building. This component was to finance the following technical assistance activities: (a) Development of construction standards, rangeland management guidelines, key feasibility studies for preparing project interventions, engineering surveys, and hydrological assessments for project areas; (b) provision of improved extension services to farmers and pastoralists, Page 2 of 25 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Somalia - WAPR (P167826) management models, and cost-recovery mechanisms;(c) collection of better data to improve knowledge of hydrogeology and groundwater exploration; and (d) support to a technical assistance agency that would assist national and state government agencies in selecting, training, and monitoring nonstate actors for local project implementation. B.2. Community Development and Demand Mobilization. This subcomponent was to finance a holistic community engagement to support the project activities through a continuous dialogue on the development needs of the community, the resources they have, their priorities to manage those resources, and how to ensure equitable access and sustainability. C. Supporting Sustainable Land Management and Livelihoods Development Around Water Points. (Appraisal cost: US$9.5 million; revised cost at the project restructuring: US$8.0 million; actual cost: US$8.0 million) This component aimed at creating and strengthening productive livelihoods among target communities in the project area. It consisted of two subcomponents: C.1. Integrated Landscape Management. This component was to finance the following activities: (a) Cash-for-work community-led soil and water conservation measures such as terracing of irrigable land degraded or endangered by erosion, planting trees and other vegetation in selected upland areas, enhancing rangeland management to introduce rotational grazing and stocking rate limits, and supporting improved management and sustainable use of existing forest and vegetation resources; and (b) promotion of the uptake of alternative energy solutions through awareness building, demonstrations, and financing. C.2. Agriculture and Livestock Support. This subcomponent was to finance the following activities: (a) Development of demonstration plots (community gardens and fruit tree groves); (b) distribution of improved seeds and other inputs; (c) introduction of high-efficiency micro-irrigation systems and soil micro-nutrient assessment; and (d) provision of training to communities to increase the production of more nutritious food and the use of climate-smart farming techniques. These activities were to be implemented using the water resources developed under the first component and community development plans developed under the second component. D. Project Management, M&E, Knowledge Management, and Learning. (Appraisal cost: US$9.0 million; revised cost at the project restructuring: US$7.0 million; actual cost: US$7.0 million) D.1. Project Management. For an efficient implementation of the project, this subcomponent was to finance the following: (a) The incremental operating costs for managing the project; (b) the cost of procurement and hiring of financial management specialists; and (c) outreach and communications on the government’s role and leadership in the project to the broader Somali community. D.2. M&E, Knowledge Management, and Learning. This subcomponent was to hire a qualified and experienced international firm to provide quality enhancement and implementation support to the project through support to the Somali authorities in financial management, procurement, monitoring, and supervision of the project. This subcomponent was also to support the enhancement of the existing web- based Management Information System and the incorporation of new modern technologies such as geo- tagging of site investments, collection of field data with tablets/smartphones, and application of geospatial imaging for quantifying before and after comparisons for specific indicators, and technical work such as flood mapping. Page 3 of 25 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Somalia - WAPR (P167826) D.3. Contingent Emergency Response. This component was to finance immediate and rapid response to an eligible crisis, if needed, such as natural disasters, severe economic crises, or other crises and emergencies in the country. A formal declaration of a national emergency by the government and a formal request by the Ministry of Finance to the World Bank were required to trigger this Immediate Response Mechanism. Revised Components The project could not implement activities in Somaliland because of the delay in the effectiveness of the portion of the Grant for Somaliland amounting to US$7.5 million. As a result of a deal brokered by the World Bank, the Federal Government of Somalia and Somaliland signed the subsidiary agreement on July 2022, as a disbursement condition rather than at effectiveness. As there was insufficient time left to implement the project activities in Somaliland before project closing, the project was restructured in November 2022 to cancel most of those activities such as the construction of 30 new water points, rehabilitation of 10 water points, establishment of community management systems, and provision of agricultural services to the farmers. At the project restructuring, seven existing water points in the other three states were added to the project scope for rehabilitation. The cancelled activities in Somaliland were transferred to the follow-on Somalia Water for Rural Resilience Project. e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project Cost: The project cost was originally estimated at US$42.0 million including US$2.5 million for price and physical contingencies. At the project restructuring in November 2022, the estimated project cost was revised down to US$32.0 million because of the cancellation of most of the activities in Somaliland and the amount corresponding to physical and price contingencies. The project closed on February 28, 2023 with an actual cost of US$29.53 million. The project team confirmed that the difference between the revised and actual project costs was because of the changes in the exchange rates. Financing: At appraisal, the IDA grant amount was estimated at US$42.0 million fully financing the estimated project cost including physical and price contingencies. An amount equal to US$7.5 million was cancelled because of the cancellation of the majority of the project activities in Somaliland at the first restructuring, and US$2.5 million because of the cancellation of the price and physical contingency item. Therefore, The IDA grant amount was revised down to US$32.0 million. By project closing in February 2023, the project had disbursed US$29.53 million. All project funds were accounted for at the time of project evaluation. Borrower’s contribution: At appraisal, no borrower’s contribution was estimated, and none materialized at project closing. Project Restructuring (November 4, 2022): The project was structured four months before the project closing date to cancel US$10.0 million of the grant for the activities that could not be implemented in Somaliland because of delayed grant effectiveness (see Revised Components above for the activities cancelled). Because of the cancellation of these activities, target values of some intermediate results indicators were revised down such as the number of new water points constructed from 75 to 45, number of water points rehabilitated from 25 to 22, and the number of community management systems established from 75 to 45. Page 4 of 25 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Somalia - WAPR (P167826) Dates: The project was approved on July 1, 2019. The IDA Financing Agreement was signed on October 8, 2019, and the US$28.0 million portion of the grant became effective on December 13, 2019 following the signing of the subsidiary grant agreements between the Federal Government of Somalia and the states of Puntland, Galmudug, and Southwest. The remaining US$14.0 million of the grant could only become available for disbursement on July 1, 2022, because of a significant delay in the signing of the subsidiary grant agreement between the Federal Government of Somalia and Somaliland (Please see the Revised Components paragraph above for the explanation of this delay). The Mid-Term Review was conducted in November 2021. The project closed on its original closing date of February 28, 2023. Reasons not to undertake a split assessment of the project’s performance: At the project restructuring, the project’s scope was substantially scaled down because of the cancellation of activities in Somaliland and a grant of US$7.5 million was cancelled because of the delay in the effectiveness of grant for Somaliland. The cancellation of the grant amount was commensurate with the contraction of the project scope and did not affect adversely affect the outcomes expected from the project’s intervention in the other three states. Therefore, this review will not undertake a split assessment of the project’s outcome and will assess the performance based on the revised targets. 3. Relevance of Objectives Rationale The project objective was highly aligned with the World Bank’s current strategy as defined in the Country Partnership Framework (CPF) for Somalia, FY19-FY22 (at the time of this review, the World Bank, together with the Federal Government of Somalia, was in the process of preparing the next CPF for Somalia). The project sought to address the development problems of insufficient availability of water, because of climatic shocks, in drylands of Somalia for domestic use, livestock, and crop cultivation and insufficient knowledge of crop cultivation and livestock production suitable to these drylands. These created a major obstacle to the achievement of the full potential of crop farming and livestock production and forced nomadic households to move across regions and into neighboring countries resulting in conflicts. The 2018 World Bank Group’s Systematic Country Diagnostic (SCD) identifies this as one of the two drivers of fragility, conflict, and violence (FCV) in the country—vulnerability to climatic shocks, predominantly recurrent drought, famine, and floods, which eroded livelihoods, drove unsustainable exploitation of natural resources that were already degrading rapidly, and caused extreme food insecurity and population displacement. The project was to address these problems through the construction and rehabilitation of water points in the drylands, improvement of land and pasture management, and training and provision of equipment in crop farming and livestock production. The project also sought to address the development problem of insufficient government capacity in service delivery and low levels of community participation in development planning. The project was to address these weaknesses through the implementation of the project by the central and state-level governments supported by capacity building technical assistance (unlike other donors’ projects that were implemented by the United Nations’ agencies or non-governmental organizations) and continued engagement and knowledge sharing with the local communities. These activities were expected to address the second FCV driver identified in the SCD—constrained and limited capacity of the government in service delivery systems to restore and supply public services, such as water services and agricultural services, which in turn lead to citizens’ lack of trust in state institutions (PAD, pp.7- 8 and ICR, p.5). The project objectives directly correspond to the Focus Area 2 of the CPF, i.e., “Restoring Economic Resilience and Opportunities” and support the achievement of Objective 2.4. Increase access to Page 5 of 25 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Somalia - WAPR (P167826) water for rural resilience and productivity. They also correspond to the Focus Area 1, i.e., “Strengthening Institutions to Deliver Services” and support the achievement of Objective 1.1. Strengthen public finance accountability and institutional development. The project objectives were highly relevant to the country context but output-oriented. Although the government capacity and fragility issues posed as high risks for the achievement of the project objectives, the level they were pitched were appropriate for the development status in the country. The country is still emerging from a three-decade long conflict and vulnerability of the rural communities in drylands is still an important driver of FCV. The project objectives were relevant to support the development in Somalia by addressing the root causes of fragility and aligned with the economic development objectives of the Somalia National Development Plan (SNDP) for 2020-2024 to “revive Somalia’s livestock and agriculture sectors through investments in water harvesting and storage infrastructure, access to credit and extension services, and promotion of climate-smart agriculture” (ICR, p.14). The objectives were also aligned with the social development objectives of the SNDP to develop human capital through provision of clean provision of water to households, strengthen public service delivery, and develop capacities and modalities for consultation and engagement with local communities, private sector, and civil society stakeholders (ICR, p.14). However, the project objectives were output-oriented rather than outcome-oriented. The Project Appraisal Document clearly identifies various immediate outcomes expected from the project’s intervention, but these outcomes are not included in the formulation of the project objectives. The project objective to develop water and agricultural services among agro-pastoralist communities in dryland areas of Somalia is closer to the output level in the results chain. The formulation of the project objective does not clearly define what development outcomes are expected from the project’s intervention. The World Bank funded the Water for Agropastoral Livelihoods Pilot Project (WALP - P152024) between 2015 and 2018 as the first engagement in rural infrastructure sector in Somalia in more than two decades. WALP’s objective was more challenging but complex: To improve the pastoral and agro-pastoral communities’ access to, and management of, small-scale water sources and to enhance the capacity of the government to implement small-scale water interventions in targeted arid lands of Somaliland and Puntland. The Water for Agro-pastoral Productivity and Resilience Project (WAPR - the project under this review) was the second rural infrastructure project the World Bank funded in Somalia and aimed at scaling up access to water in Somaliland and Puntland covered under WALP and expand the intervention to two more states— Galmudug and South West. Therefore, these two projects are closely linked to each other, which is followed by a third project, i.e., Somalia Water for Rural Resilience Project approved in December 2022. However, the project objective of the WAPR is less challenging although it is the second project in the World Bank’s engagement in rural infrastructure in Somalia, which benefited extensively from the lessons learned during the implementation of the first engagement under WALP. While the formulation of the project objectives was output-oriented and less challenging compared to the objective of the earlier engagement in the rural infrastructure sector, the outcomes expected from the project’s intervention were clearly identified in the project documents that would have been expected to contribute to the addressing of the two fragility drivers identified in the CPF and support the objectives in the SNDP. Considering the FCV status of the country, overall relevance of the project objectives is rated High. Rating Relevance TBL Rating High Page 6 of 25 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Somalia - WAPR (P167826) 4. Achievement of Objectives (Efficacy) EFFICACY_TBL OBJECTIVE 1 Objective To develop water services among agro-pastoralist communities in dryland areas of Somalia. Rationale Theory of Change for Objective 1 The project inputs (IDA grant and technical assistance) were to be used to finance the construction of water harvesting and delivery infrastructure that were identified in coordination with the communities in the drylands of Somalia and capacity building at the federal and state government and community levels to operate and maintain these water systems. The project outputs were to consist of various water harvesting infrastructure (such as sand dams, subsurface dams in dry riverbeds, surface water pan dams, and boreholes for underground water extraction) and water delivery infrastructure constructed under the project through the use of Wadi Evaluation Tool developed under the previous WALP Project. These outputs were expected to lead to the expected outcomes of increased and sustained availability of water to be consumed by local agro- pastoral communities for drinking and sanitation, watering their livestock, and irrigating land for crop farming. The outcomes expected from the training of the federal and local government officials, continuous engagement with the local agro-pastoral communities, and implementation of awareness activities were increased ownership and adequate day-to-day operation and maintenance of the water infrastructure built under the project by local communities supported by the federal and local governments to ensure the sustainability of project results beyond project closing. However, the project did not address the more technical and complex maintenance of the water systems built under the project; the state governments do not have a framework to support the operation and maintenance of the infrastructure and a system to monitor water quality (although the quality of water was expected to be high because of the absence of any environment polluting industrial activity in the project areas and the natural filtration of the water through the sand structures to be built under the project, a water quality monitoring system is essential to ensure the quality of water for human and livestock consumption and irrigation). Additionally, the project’s theory of change did not include the establishment of a financing mechanism to cover the cost of the operation and maintenance of the water systems should the amount of water user fees fall short of covering those costs (this shortcoming is addressed in the follow-on WRR Project through subsidies). Overall, the causal links from project activities and outputs to the expected outcomes were direct and valid, and the achievement of the project objective could be attributed to the project’s intervention, but there were weaknesses in the theory of change in ensuring the sustainable operation and maintenance of the water systems and monitoring of the quality of water that are expected to be addressed under the follow-on project. Outputs  Improved community water points constructed under the project: The achievement was 34 new water harvesting structures built under the project against the revised target of 45. The original target was 75 including the water points to be built in Somaliland that were cancelled at the project restructuring (see Revised Components in section 2.d Components). The 34 new water harvesting structures consisted of the following: a) eight haffir dams in Galmudug; b) seven haffir dams and one Page 7 of 25 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Somalia - WAPR (P167826) sand dam in South West; and c) eight haffir dams, seven sand dams, and three subsurface sand dams in Puntland. Eleven water sources in Puntland could not be developed because of cost escalations caused by the COVID-19 pandemic and Russia’s invasion of Ukraine disrupting the global supply chains (ICR, p.28).  Improved community water points rehabilitated under the project: The project rehabilitated 15 water structures (three boreholes in Galmudug; one haffir dam and three boreholes in South West; and eight boreholes in Puntland) against the revised target of 22. The original target was 25. The rehabilitation activities included the repair of non-functioning components, expansion of water storage capacity, and construction of additional water kiosks and livestock troughs. All boreholes rehabilitated under the project were fitted with solar pumping systems. Some boreholes were also fitted with diesel- fired generators with solar pumping systems to ensure continuous operation of the boreholes.  Number of community managements systems established (i.e., community committees): By project closing, 54 community communities had been established and operational against the revised target of 45. The original target was 75. These communities included water management, village, farmer, women, youth, social, and pastoralist development committees across three states consisting of 769 females and 1,178 males. The project supported the establishment of these committees through trainings in project planning and implementation monitoring, and operation and maintenance of the project outputs after project closing. Outcomes  Number of people provided with access to improved water sources under the project: As a result of the water systems built under the project, 362,731 people gained access to water in the drylands of three states—84,700 people in Galmudug, 155,503 people in Puntland, and 122,528 people in South West. The original target was 250,000 people, which was not revised despite the cancellation of project activities in Somaliland. Although the project completed fewer water supply sources than planned, the project brought water to more people than originally targeted because of the advanced site selection approach using the Wadi Evaluation Tool and truthing of selected water sources by the communities through an elaborate community consultation process led by the states’ project implementation units (ICR, p.16).  Percentage of project beneficiaries satisfied with project investments: The target was 60 percent of the beneficiaries being satisfied with the project investments. The surveys conducted with sampled beneficiaries from different project sites showed that 68 percent of the respondents were satisfied with the project investments. The following satisfaction dimensions were included in the survey covering both water and agriculture activities completed under the project: Increased water quantity during the dry season, increased water quality during the wet season, reduced time spent to fetch water, the ability of the state government to provide water for human consumption and domestic uses, the ability of the state government to support the provision of water for livestock and agricultural purposes, agricultural inputs distributed (seeds and tools) livestock treatment services provided by mobile veterinary clinics, and rangeland rehabilitation. This indicator does not report satisfaction levels for water and agricultural services separately. Therefore, this indicator shows the overall satisfaction of the beneficiaries with the project’s intervention.  According to the findings of the surveys conducted at project closing, a total of 1,551,304 animals and 845 farms had gained access to water through the water systems constructed under the project. The project fell short of developing the number of water sources it targeted because of the disruptions in the global supply chains caused by the COVID-19 pandemic and Russia’s invasion of Ukraine that resulted in Page 8 of 25 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Somalia - WAPR (P167826) higher construction costs. However, the project successfully developed or rehabilitated 49 water sources that provided access to water for about 363,000 people (higher than the target of 250,000) in the drylands of the three project states. This was achieved through the utilization of the geospatial Wadi Evaluation Tool in identifying the most feasible water sources to develop in close cooperation and coordination with the local communities. It is estimated that the 49 water resources developed under the project can supply 1.60 million cubic meters of water during the dry season, which is significantly more than the 0.78 million cubic meters of water needed by the beneficiaries (including irrigation and livestock watering) during the entire dry season (ICR, p.51). According to the information provided by the World Bank project team during a meeting on January 18, 2024, some agro-pastoral communities settled around the water sources developed under the project because of the sustained availability of water during the dry season. The majority of the agro-pastoral communities find it easier to sustain their traditional way of nomadic life because of the availability of water sources developed under the project that help them plan their seasonal migration routes in the drylands. It is expected that the availability of water during the dry season will prevent those agro-pastoral communities travel to distant areas within Somalia or cross borders into neighboring countries in search of water. As discussed in the following section, the increased availability of water significantly contributed to livestock production and crop farming in the drylands. The project also significantly contributed to the institutional capacity strengthening in Somalia at the federal and local government and community levels. The communities were involved in the planning and construction of the water systems supported by the federal and local government authorities. The implementation of the project by the local governments resulted in the establishment of a total of three project implementation units in three states and one at the federal government level. The project implementation units employed 69 members of staff and were supported by the project-hired consultants. These project implementation units will continue to develop new water sources in cooperation with the local communities under the follow-on WRR Project. The establishment of the community committees was highly effective in ensuring the involvement of the agro- pastoral communities in the development of the Community Investment Plans outlining their water and agricultural priorities to be financed by the project. These committees closely worked with the states’ project implementation units. The strong involvement of the communities in the decision making process related to their own development path resulted in their higher ownership of the water systems, which manifested itself in the efficient day-to-day operation of these systems and sustained availability of water. Although it is hard to measure, the close cooperation and coordination between the communities and federal and local state authorities must have contributed to increasing trust between the citizens and the government. While there were weaknesses in the project’s theory of change to ensure larger maintenance of the water systems both technically and financially and the project could not implement its activities in Somaliland because of the deteriorating political relations between the Federal Government of Somalia and Somaliland, the project was nevertheless substantially successful in developing water resources in the drylands and strengthening institutional capacity for the sustained provision of water for the agro-pastoral communities in the drylands. This should also help address the fragility drivers in Somalia (i.e., population displacement and weak government authority) contributing to the stabilization of the country. Overall, the project’s efficacy in achieving the project objective to develop water services among agro-pastoralist communities in dryland areas of Somalia is rated Substantial. Page 9 of 25 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Somalia - WAPR (P167826) Rating Substantial OBJECTIVE 2 Objective To develop agricultural services among agro-pastoralist communities in dryland areas of Somalia. Rationale Theory of Change for Objective 2 The theory of change for Objective 2 was built on the premise that the project would develop water sources in the drylands of the project states that would contribute to the development of agricultural services. The project inputs (IDA funds and technical assistance) were then to be used to train the farmers in the vicinity of the water sources in improved agricultural technologies for crop farming and livestock production at the fields schools and demonstration farms to be developed under the project. The project was also to finance the construction or rehabilitation of veterinary clinics to provide veterinary services and the distribution of seeds and agricultural equipment, while providing training to animal health workers in the three states. The project outputs would be the availability of agricultural services to the farmers in the project areas. The outcomes expected from the project outputs would be the number of people benefiting from these services and adopting improved technologies. The long-term outcomes would be increased food and livestock production that would generate additional household income and improve the households’ resilience to better withstand climate shocks. As the Objective 2 was output-oriented (i.e., development of agricultural services), the results framework did not include indicators measuring the outcomes expected as a result of the development of agricultural services such as improved food and livestock production or improvement in the socio-economic welfare of the agro-pastoralist communities in the project areas. The theory of change for Objective 2 did not include interventions that would ensure the sustainability of the agricultural services after project closing, which was to be covered by the follow-on project. Overall, while there were weaknesses in the theory of change for Objective 2 in the sustainability of project outcomes after project closing and the outcomes expected from the project outputs were not adequately captured by the results framework, the causal links in the results chain from project activities and outputs to the expected outcomes (which are closer to the output level) were direct and valid, and the achievement of Objective 2 could be attributed to the project’s intervention. Outputs The project’s results framework included only one intermediate results indicator titled “Farmers/client reached with agricultural services.” As that indicator captures the access aspect of agricultural services, it is included in the Outcomes section below. The project achieved the following outputs that were not captured by the results framework but reported in the ICR (pp.17-18 and 45-46):  Veterinary clinics - constructed: The project constructed six new satellite veterinary clinics including laboratories in Puntland. Page 10 of 25 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Somalia - WAPR (P167826)  Veterinary clinics – rehabilitated: The project financed the upgrading or rehabilitation of seven veterinary clinics, which included physical structure improvements and purchase of laboratory equipment and 64,000 clinic items, including veterinary drugs.  Veterinary clinics – mobile: The project purchased fully equipped mobile veterinary clinics (two in Galmudug, two in Puntland, and one in South West).  Treatment and vaccination of livestock: The project financed the treatment and vaccination of a total of 1,378,305 goats, sheep, and camels for deworming and various infections through the use of mobile veterinary clinics (34,514 in Galmudug, 1,319,599 in Puntland, and 24,192 in South West).  Establishment of farmer field schools: The project financed the construction of 50 farmer field schools (10 in Galmudug, 30 in Puntland, 10 in South West) to provide training on improved farming technologies for agro-pastoralists.  Setting up of demonstration farms/centers: To disseminate knowledge on improved agricultural practices to farmers, the project set up nine demonstration centers (four in Galmudug, four in Puntland, and one in South West).  Distribution of farming tools and equipment and seeds: The project distributed farming tools and equipment (such as hoes and machetes for farm preparation) and drought-tolerant seeds variety to farmers in the three states.  Trainings: The project financed trainings sessions for community animal health workers in three states. These workers are expected to provide the same training to farmers. The ICR did not provide the number of workers trained at these trainings.  Restoration of communal rangelands: The beneficiary communities in the project states identified, through a community-led participatory process, 23 degraded communal rangelands (883 hectares) that required restoration. The project provided funds to these communities through cash-for-work program for restoration works including gullies and bunds to enhance infiltration and reduce soil erosion and planting of trees. Outcomes  Farmers/clients reached with agricultural services: The project aimed at reaching at 80 percent of the 150,000 farmers in the project areas with agricultural services to equip them with the skills and knowledge, so that they could leverage the increased amount of water in crop farming and livestock production. The achievement was 78 percent corresponding to 117,575 framers (17,664 in Galmudug, 63,941 in Puntland, and 36,420 in South West). Of the total number of people who benefited from agricultural services, 90,913 people used the mobile veterinary clinics and livestock husbandry advisory services, 21,919 people attended trainings, and 4,733 people received farming inputs (seeds and equipment). The agricultural services included good livestock husbandry practices, techniques for land preparation using conservation agriculture approach and planting of dryland variety of crops, drought-tolerant crop or seed varieties, different soil and water conservation measures at on-farm and catchment levels, approaches to better crop harvesting and storage, and pest control. With the support of the project-hired specialized consultants, expert staff from the ministries of agriculture, livestock, and environment of the federal government and states provided these services and trainings to farmers at the farmer field schools and demonstration farms set up under the project (see the entries below).  Number of farmers/clients adopting improved technology: This indicator measures the number of project beneficiaries who benefited from the increased availability of water and practiced different technologies introduced by the project to engage in more crop farming and livestock production Page 11 of 25 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Somalia - WAPR (P167826) activities. The achievement was 85,015 farmers adopting improved technology against the original target of 110,000. The main factor in the adoption of improved technologies was the availability of water because of the project’s intervention. Each beneficiary adopted a different set of technologies introduced under the project. At 47 percent adoption rate, early land preparation, correct spacing, early planting, and timely and proper weeding were the main technologies adopted by the beneficiaries. These were followed by improved seed varieties and crop rotation technologies at 46 percent adoption rate. The remaining five technologies and their adoption rates were as the following: Drought-tolerant crops (44 percent), soil conservation (41 percent), field pest control (39 percent), gulley reclamation (38 percent), and bio pesticide (32 percent).  For the results of the beneficiary survey, please see Outcomes under Objective 1. Benefiting from the availability of water resources, the project successfully developed agricultural services for agro-pastoralist communities through the construction or rehabilitation of veterinary clinics and laboratories, provision of extensive training in agricultural technologies suitable for drylands, establishment of demonstration farms and field schools, and distribution of seeds and agricultural equipment. A high number of livestock were vaccinated against diseases or treated at the permanent or mobile veterinary clinics. The project activities resulted in a significant number of farmers adopting improved technologies in agriculture and livestock husbandry, but the number was lower than the target (85,015 against the target of 110,000). The findings of the survey conducted at project closing indicate that there is a high demand for agricultural services by the agro-pastoral communities. However, the project did not sufficiently address the absence of financial mechanisms and a system for continuous technical support and capacity building to ensure the sustainability of agricultural services after project closing. The follow-on project will address this shortcoming both in water and agricultural services through the establishment of a subsidy system and the possible introduction of local customary laws for the management of these services. Therefore, when assessed from a multi-phase intervention in developing water and agricultural services in the drylands of Somalia, the project under review was successful in achieving the outcomes expected from the project’s intervention, which will be further developed under the follow-on project. While there were weaknesses in the theory of change in ensuring the sustainability of the project’s outcomes and the achievement of number of people adopting improved technologies was lower than the target, the project was successful in providing a high initial funding to improve the veterinary services, introduce improved agricultural technologies, and build institutional capacity, which will be further supported under the follow-on project. Overall, the project’s efficacy in achieving the project objective to develop agricultural services among agro-pastoralist communities in dryland areas of Somalia is rated Substantial. Rating Substantial OVERALL EFF TBL OBJ_TBL OVERALL EFFICACY Rationale Page 12 of 25 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Somalia - WAPR (P167826) While the number of water resources developed under the project fell short of the target, the project was successful in increasing the availability of water through water harvesting structures and boreholes and bringing water to a higher number of people than originally targeted. The project was also successful in developing agricultural services in the project areas through trainings and provision of veterinary services and distribution of seeds and agricultural equipment. Although not covered by the project objective, the close coordination between the federal and state government authorities and the agro-pastoral communities in the development and implementation of the project supported by extensive training contributed to the development of trust amongst stakeholders. Therefore, the project directly contributed to the efforts in addressing the two fragility drivers in the country—population displacement because of insufficient water resources and weak government authority. Overall Efficacy Rating Substantial 5. Efficiency Economic Analysis A cost-benefit analysis was conducted at appraisal to assess the economic viability of the project’s intervention in developing water sources and improving the livelihoods of the agro-pastoral communities in Somalia’s drylands. Quantifiable economic benefits included in the analysis consisted of the following: a) avoiding payments for water from expensive tanker services; b) economic value of improved access to water for livestock and agriculture; c) time savings from fetching water; d) reduction in school absenteeism; e) direct and indirect health benefits because of access to cleaner water; f) reduction in health care costs because of the reduction in waterborne illnesses; and g) avoiding the loss of livestock deaths and livestock quality because of lack of access to water during dry seasons. Some other economic benefits that could not be quantified because of a lack of reliable data were excluded from the cost-benefit analysis such as economic benefits related to a stronger enabling environment for economic development and overall systemic improvement with regards to equity, efficiency, and shared prosperity, and those related to the other project activities in community facilitation, citizen’s engagement, local development, technical and vocational skills, and improved project management. The total cost of the project was included in the analysis as costs together with the operation and maintenance cost of the water systems for 15 years. The cost of greenhouse gas emissions was also included in the analysis. However, the analysis did not include economic benefits that could accrue from improved livestock health because of the availability of improved veterinary services, implementation of improved agricultural technologies, increased agricultural productivity because of the use of dryland suitable seeds and equipment. Similarly, the analysis did not include the cost of sustaining agricultural services beyond project closing. Therefore, the cost-benefit analysis was restricted mostly on the impact of increased availability of water on humans, livestock production, and crop farming but captured a wide range of economic benefits from health improvement and reduction in school absenteeism to the economic value of access to water on livestock production and crop farming. The calculations resulted in an Economic Internal Rate of Return (EIRR) of 135 percent and a Net Present Value (NPV) of US$144.26 million at a discount rate of 15 percent. These high values are because of the significant economic impact of the availability of clean water during dry seasons on Page 13 of 25 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Somalia - WAPR (P167826) daily lives of agro-pastoral communities, livestock production, and crop farming, absence of which was a major development and conflict issue in Somalia. At project closing, the same methodology was used to conduct a cost-benefit analysis. There were no major changes in the economic benefits included in the analysis, but only the cost of water sources developed under the project (28.81 percent of the total actual project cost) was included in the analysis rather than the total project cost as was done at appraisal. The lifetime of the water systems was also taken as 30 years differing from 15 years included in the cost-benefit analysis at appraisal. Therefore, the cost-benefit analysis conducted at project closing was not fully comparable to the analysis conducted at appraisal because of differing investment costs and longer lifetime of the water systems. With these assumptions, the calculations resulted in a significantly higher post-project EIRR of 279 percent. The ICR does not report a post-project NPV. The higher number of project beneficiaries (362,731 against the target of 250,000) was another reason for a higher EIRR. The ICR (p.22) reports that if the beneficiary number is taken as 250,000, the EIRR drops to 146 percent, still slightly higher than the appraisal EIRR. Operational and Administrative Efficiency The project implementation significantly benefited from extensive preparatory studies for design discussions that included “macro-level characterization of water resource challenges, spatial analysis of the potential for wadi harvesting using the Wadi Evaluation Tool in the project areas, and studies on the suitability of different harvesting technologies” (ICR, p.25). The incremental implementation strategy adopted at appraisal to mitigate the implementation related risks was critical in achieving effectiveness quickly and completing the project activities in Galmudug, Puntland, and South West by the original project closing date, while the grant effectiveness for the activities in Somaliland was significantly delayed because of political issues. However, the project faced initial implementation delays because of lack of experience of the project implementation unit staff in World Bank-financed projects; the preparation of safeguards documents resulted in a three-month project implementation delay, but these delays did not adversely affect the achievement of the project outputs and outcomes. The commitment of the federal and local governments to the implementation of the project was high. The turnover of the project implementation unit staff was minimal. The national project coordinator’s leadership was essential in successfully completion of the project (ICR, p.27). The project was designed to facilitate the full involvement of the project beneficiary agro-pastoral communities in the design and implementation of the project, which was successful in both achieving the project outcomes and building institutional capacity at the community level. During the first half of the project implementation, there were frequent noncompliance issues with the environmental and social management plans (ESMP) at project sites because of the insufficient knowledge of the local contractors about the World Bank’s safeguards requirements. These resulted in frequent implementation delays at certain project sites but were addressed through the implementation of the ESMP template in the form of a checklist. While there were some methodological differences in the cost-benefit analyses conducted at appraisal and project closing and some minor shortcomings in the operational and administrative efficiency of the project that led to project implementation delays, the project completed its activities and achieved the expected outcomes by the original project closing date benefiting a higher number of people than originally targeted. The cancellation of the activities in Somaliland was a significant shortcoming in fully achieving the project outcomes, but the reasons that led to the cancellation of these activities were beyond the control of the project. Overall, the project’s efficiency in achieving the project objective to develop water and agricultural services among agro- pastoralist communities in dry-land areas of Somalia is rated Substantial. Page 14 of 25 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Somalia - WAPR (P167826) Efficiency Rating Substantial a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 100.00 Appraisal  135.00  Not Applicable 28.81 ICR Estimate  279.00  Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome The project objective was highly relevant to the World Bank strategy and country context that aimed at addressing the development problem of the lack of water for the agro-pastoralist communities in the drylands of Somalia. While the project could develop fewer water sources because of the cancellation of project activities in Somaliland and the cost escalation caused by the COVID-19 pandemic and the war in Ukraine (disrupting global supply chains), the project was successful in providing a higher number of people with access to water and a substantial number with access to agricultural services. Therefore, the efficacy of the project in achieving the project objective is rated Substantial. The project’s efficiency in achieving the project objective is also rated Substantial while there were some minor shortcomings in the operational and administrative efficiency and differences in the methodology of the cost-benefit analyses at appraisal and project closing. Overall, the project outcome is rated Satisfactory. a. Outcome Rating Satisfactory 7. Risk to Development Outcome Technical: As discussed under the efficacy section above, the project supported the capacity development of the community committees in simple day-to-day operation of the water systems built under the project. However, currently there is no established system for regular and emergency maintenance of these water systems should there be a need for major (for example reconstruction of sand dams) or complex maintenance (for example repair of solar pumps). The World Bank will continue supporting the operation and maintenance of the existing water systems under the follow-on Project, but the absence of an established system for regular and emergency maintenance of the water systems that would ensure the availability of quality of water supply stands out as a high risk for the sustainability of the project outcomes. Page 15 of 25 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Somalia - WAPR (P167826) Financial: Fees collected from the water users cover the cost of the operation of water systems. According to the findings of the survey conducted at the end of project closing, several communities had to rely on state government subsidies to operate the water systems because the collection of end user fees did not fully cover the costs. This is not a sustainable financing mechanism as the state governments might have sufficient funds to allocate to the communities for the operation of the water systems. Similarly, the agricultural services developed under the project may not be sustained in the absence of sufficient funds from the federal and state governments. The follow-on project will continue to support the water and agricultural services through subsidies, but insufficient funds to support these services stand out as high risk for the sustainability of the project outcomes in the long-term. Fragility: Somalia is an FCV status country. The project was implemented under high security measures. A worsening of the security situation could force agro-pastoral communities abandon their settlements around the water sources developed under the project or prevent them accessing these sites. This could lead to insufficient operation and maintenance of the water systems and cessation of agricultural services developed under the project. Although there are positives signs that the federal and state governments are in the process of building institutional capacity in governing the country and the states, the possibility of a worsening security situation is still high. 8. Assessment of Bank Performance a. Quality-at-Entry The development of water sources and agricultural services in the drylands of Somalia was of high strategic importance for Somalia as it directly addressed one of the fragility drivers in the country— displacement of population because of the lack of water. The project was designed based on findings of the Somalia Resilient Multiple Use Water Services ASA and the experience gained and lessons learned in the pilot WALP Project to scale up and expand the geographical reach of water delivery and agricultural services in the drylands. The design of the water harvesting structures was appropriate to the geography and climate in the project areas; the project used proven technologies in water harvesting. The focus on expanding veterinary services through the use of mobile veterinary clinics was significantly appropriate to reach the agro-pastoral communities with a semi-nomadic life in the remote areas of drylands. The project activities were to directly address the poverty and social development issues in the agro-pastoral communities through the development of clean water resources and direct support to improve livestock production and crop farming. Environmental and fiduciary aspects of the project were sufficiently assessed justifying the viability of the project. Because of the extensive preparatory study (using remote sensing technology and community-led ground truthing) for design discussions with the authorities and the agro-pastoral communities increased the project implementation readiness prior to effectiveness. Prioritizing the needs of the agro-pastoral communities and ensuring their full involvement in the preparation of the development plans were effective in successful implementation of the project. The project was to be implemented through the project implementation units set up under the state governments (not through United Nation agencies or non-governmental organizations) was highly effective in developing institutional capacity and strengthening trust between the state governments and citizens. Risks were sufficiently identified, and mitigation measures were in place; the incremental implementation approach adopted in project design was critical in completing the project activities in Page 16 of 25 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Somalia - WAPR (P167826) three states while the effectiveness of the grant for Somaliland had been significantly delayed because of political reasons. The project components financed activities that were to directly lead to the achievement of the project objective, and the project implementation units were adequately staffed but their lack of experience in implementing a World Bank-financed project were to be addressed through the World Bank trainings and support from the project-hired consultants. However, the M&E design had moderate shortcomings in adequately capturing the project outputs and outcomes, especially related to the development of the agricultural services (see 9. M&E Design, Implementation, & Utilization below). The project design also lacked a concrete plan for the sustainability of the water and agricultural services after project closing; the follow-on Project will address this shortcoming. Overall, the World Bank satisfactorily identified, facilitated project preparation, and appraised the project; the quality at entry is rated Satisfactory. Quality-at-Entry Rating Satisfactory b. Quality of supervision The World Bank project team carried out eight implementation support missions between early 2020 and project closing in February 2023 mostly virtually because of the COVID-19 travel restrictions. When in- person missions were held, the meetings with government counterparts were held at the Mogadishu International Airport because of security concerns. Mission dates had to be rescheduled frequently because of the same reason. Despite these challenges, the project team’s focus on the development impact of the project was satisfactory. The supervision inputs and processes were adequate. The monthly project management team meetings held with the federal and local government project implementation units were highly effective in helping the World Bank project team proactively identify the project implementation bottlenecks and address them through trainings given to the project implementation staff and facilitating the support of the project-hired consultants. The Implementation Status and Results Reports and Aide Memoires candidly reported the project performance. The project team conducted the Midterm Review (MTR) of the project in November 2021 as planned. The discussions held during the MTR led to some effective decisions to improve the project implementation. For example, the decision to hire a Quality Enhancement and Implementation Support (QEIS) firm, despite some challenges in procurement, was critical in improving the supervision of project implementation and M&E data collection and verification through the development and use of the Management Information System and a beneficiary survey conducted at project closing. The MTR also resulted in the decisions to disaggregate some indicators according to gender to capture the project’s impact on gender and introduce the recruitment of female Junior Professional Officers and interns at the federal and state project implementation units. At the MTR, it was also agreed to restructure the project, but the restructuring was delayed because of the expectation that an agreement could have been reached between the Federal Government of Somalia and Somaliland to sign the subsidiary agreement so that the implementation of the activities in Somaliland could have started. The World Bank project team played an important brokering role in the dialogue and the eventual signing of a subsidiary agreement between the Federal Government of Somalia and Somaliland. However, because there was insufficient time left to implement the activities in Somaliland before the project closing in February 2023, the project was restructured in November 2022 to cancel most of the activities to be implemented in Somaliland. Those cancelled activities will be implemented under the follow-on project. Page 17 of 25 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Somalia - WAPR (P167826) The project team’s supervision of fiduciary and safeguard aspects of the project were adequate. The non- compliance issues with the ESMPs at the project sites were addressed through the implementation of the ESMPs as a checklist by the project implementation units and contractors. The World Bank project team identified some weaknesses in the internal controls and addressed them through the implementation of an action plan (see section 10.b. Fiduciary Compliance). Based on the successful achievement of the project results, the project team prepared the follow-on project to continue with the further scaling up of water and agricultural services in the drylands and establish mechanism to ensure their sustainability beyond the end of the World Bank support. Overall, the quality of the World Bank project team’s supervision of the project implementation to achieve the project objectives was Satisfactory. Quality of Supervision Rating Satisfactory Overall Bank Performance Rating Satisfactory 9. M&E Design, Implementation, & Utilization a. M&E Design The project objective was clearly defined but output-oriented. Although the outcomes expected from the project’s intervention were identified in the project documents, they were not included in the formulation of the project objective. The theory of change explaining the causal chains from project activities and outputs to the expected outcomes and the achievement of the project objective was robust, but it was partially reflected in the results framework. While the indicators in the results framework sufficiently encompassed the results in increasing the availability of water in the project areas, the other indicators were insufficient to capture the outputs and outcomes related to the development of agricultural services. Most of the outputs in agricultural services were reported in the project documents but only the number of people benefiting from these services and adopting improved technologies in agriculture were captured by the results framework. Despite these shortcomings, the indicators in the results framework were specific, measurable, achievable, relevant, and time bound. Targets for each indicator were identified. The M&E implementation arrangements were clearly defined; the project implementation unit at the federal government level was to have an M&E coordinator and the project implementation units in each state M&E officers who would be responsible for effective M&E and reporting. M&E data collection was to include geo-tagging of project investments and geospatial imaging to make a before and after the project comparison. The project was to hire a Quality Enhancement and Implementation Support (QEIS) firm “to support the baseline surveys, verify project outputs, develop a web-based Management Information System (MIS) to enable real-time data loading and monitoring and support the endline survey” (ICR, p.30). The survey to be conducted after project closing was to provide more data about the impact of the project on the socio-economic welfare of the project beneficiary agro-pastoral communities. Page 18 of 25 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Somalia - WAPR (P167826) b. M&E Implementation The project hired a QEIS firm in August 2021 after a significant delay. This resulted in the delayed completion of the baseline surveys in May 2022. The ICR (p.29) reports that as the project implementation units in each state and submitted quarterly and annual M&E reports that included data on initial project conditions and progress towards achieving the project results, the delay in the completion of the baseline surveys did not adversely affect the assessment of the project’s progress. The M&E reports were submitted regularly but with occasional delays and quality issues at the early stages of project implementation. Following the trainings provided by the World Bank project team to the M&E officers at the project implementation units, the quality of M&E reporting improved. The MIS was established as planned and was critical for real-time data recording and measuring the indicators in the results framework (The MIS can be reached at the link given below). Starting from September 2021, The QEIS firm verified project results quarterly using the data from 25 percent of the project sites. The attention of the federal and local state project implementation units and the QEIS firm was sufficient in the effective implementation of M&E. However, the shortcomings in the results framework in adequately capturing the project outcomes were not addressed during project implementation. This shortcoming was significantly compensated by the detailed data gathered through the project end beneficiary survey. The Ministry of Planning, Investment and Economic Development at the federal government level conducted the survey with the assistance of the QEIS firm. The project beneficiaries of agropastoral communities were significantly involved in designing the development plans; hence, they significantly contributed to the defining of the indicators’ targets and assessing their achievement. The M&E functions and processes are highly likely to be sustained because of the implementation of the follow-on project and the availability of the MIS. c. M&E Utilization The M&E findings were sufficiently communicated to the project stakeholders including the Federal Government of Somalia, the governments of the three states, the World Bank, and the project beneficiary agro-pastoralist communities. The M&E findings did not lead to a significant shift in the implementation direction of the project but informed the extent of the single project restructuring to ensure the effectiveness of the in achieving the project objectives and cancellation of the unused funds. The M&E data gathered through the results framework and the project end survey were sufficiently used to provide evidence of the achievement of project outcomes rather than simply reporting the application of inputs or the achievement of outputs. The success of the project evidenced by the M&E findings was critical in informing subsequent interventions. Overall, while there were some moderate shortcomings in the results framework in adequately capturing the project outcomes, which were not addressed during project implementation, the project’s M&E system as designed and implemented, including the project end beneficiary survey, was sufficient to assess the achievement of the project objectives and test the links in the results chain. The M&E quality, therefore, is rated Substantial. M&E Quality Rating Substantial Page 19 of 25 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Somalia - WAPR (P167826) 10. Other Issues a. Safeguards At appraisal, the project was classified as Category B under Environmental Assessment (OP/BP 4.01) and triggered the Natural Habitats (OP/BP 4.04), Pest Management (OP/BP 4.09), Physical Cultural Resources (OP/BP 4.11), Involuntary Resettlement (4.12), Safety of Dams (OP/BP 4.37), and Projects on International Waterways (OP/BP 7.50) safeguard policies. Environmental Assessment (OP/BP 4.01): At appraisal, the project was classified as Category B because of site-specific, temporary, and reversible environmental and social impacts of the project activities such as the following: (a) landscape degradation; (b) loss of flora and fauna due to land clearance; (c) inefficient waste management during the water infrastructure construction; (d) water pollution and contamination at haffir dams; (e) noise, dust, and vibration from construction and maintenance; (f) inadequate occupational health and safety practices; and (g) social conflicts between preexisting and new land users. The project prepared an Environmental and Social Impact Framework (ESIF) at appraisal and disclosed it in Somalia and on the World Bank’s InfoShop on January 15, 2019, after extensive consultations with the local agro- pastoral communities. Initially, there were delays in the preparation of the safeguards policies including Environmental and Social Management Plans (ESMPs) because of insufficient experience of the project implementation unit staff in World Bank-financed projects. During the early implementation phase of the project activities, some civil works frequently stopped because of non-compliance with the ESMPs (specifically with the health and safety measures and implementation of grievance redress procedure), which was later addressed through the use of the ESMP templates in the form of a checklist. This proved to be an effective solution to ensure the project’s compliance with the safeguard policies. There were no other noncompliance issues during project implementation. Natural Habitats (OP/BP 4.04). The project triggered this safeguard policy because the project’s intervention was expected to result in altering the seasonal streams. The development of the water sources did not result in a major alteration of seasonal streams. The project was compliant with this safeguard policy. Pest Management (OP/BP 4.09). The project triggered this safeguard policy because agricultural intensification was to occur as a result of the project’s intervention. The project resulted in the restoration of rangelands and development of crop farms. The project was compliant with the requirements of the Pest management safeguard policy. Physical Cultural Resources (OP/BP 4.11): At appraisal, there was no known archaeological or sacred sites in the project areas, but the project triggered this safeguard policy to prevent any accidental loss or damage to cultural sites that could be found by chance during construction. Project activities did not affect any physical cultural resource. Involuntary Resettlement (OP/BP 4.12): Although the project activities were not expected to result in any involuntary displacement of people, the project triggered this safeguard policy as a precautionary measure because the subprojects to be financed by the project had not been identified at appraisal. A Resettlement Policy Framework was prepared and disclosed in Somalia and the World Bank’s InfoShop on January 15, 2019. The project activities did not require any resettlement of land acquisition. However, some communities donated land for demonstration farms and construction of water points following the conclusion of memorandums of understanding. Page 20 of 25 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Somalia - WAPR (P167826) Safety of Dams (OP/BP 4.37). The project triggered this safeguard policy because the project was to finance the construction of small dams. All the sand dams constructed under the project satisfied this safeguard policy. Projects on International Waterways (OP/BP 7.50). The project triggered this safeguard policy because it involved the construction of small-scale and sub-surface dams and surface water harvesting infrastructures. However, the Africa Regional Vice President approved an exception for the notification of the riparian states on February 15, 2019 because it was determined that the water harvesting infrastructure to be constructed under the project would not affect the flows of rivers Juba and Shebele (the project did not cover these international waterways) and all the tributaries proposed for the construction of small-scale dams run exclusively within Somalia, the lowest downstream riparian in the region. b. Fiduciary Compliance Financial Management The project implementation units were adequately staffed with qualified and experienced financial management specialists. A team from the Accountant General’s office supported the project implementation units in financial managements. Project’s interim financial reports acceptable to the World Bank were submitted regularly and on time. The external audit reports were unqualified and submitted on time except the one for 2021, which was submitted one month late because of the delay in recruiting of an external technical assistance to the Office of the Auditor General. However, the World Bank project team identified some internal control shortcomings such as weak controls in project operating costs and advances to staff, absence of internal audit oversight, and delays in processing payments to vendors (ICR, p.31). These shortcomings were addressed through an action plan by project closing. There were no known issues of corruption or misuse of funds associated with the project. All project funds were accounted for at project closing. Procurement Procurement was implemented in accordance with the World Bank’s procurement policies and procedures in a decentralized model by the project implementation unit in each three states. The World Bank project team supported the preparation of a Procurement Risk Assessment and Management System, a Procurement Strategy for Development, and a Procurement Plan for the first 18 months of the project implementation. However, because of a lack of experience of the project implementation units in procurement in accordance with the World Bank policies and procedures, there were delays in the procurement of some contracts. For example, because of the long process for the procurement of a quality enhancement and implementation support firm, the baseline surveys could only be carried out in May 2022. Following the mid-term review, the project implementation units’ procurement performance improved as a result of the training of the staff by the World Bank project team. The prior review of all contracts at the federal and state government levels contributed to the implementation of procurement in accordance with the World Bank policies and procedures. Page 21 of 25 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Somalia - WAPR (P167826) c. Unintended impacts (Positive or Negative) Somalia did not have structured schemes for salary and per diem payments and incentives to government officials. During project implementation, staff of the project implementation units (government staff) contested the differences in allowances and salaries paid to them and the consultants externally hired by the project. This led to the development of new national salary and allowance structures for the federal and state government staff. This development is expected to contribute to the institutional strengthening in the country. d. Other None. 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment Outcome Satisfactory Satisfactory Bank Performance Satisfactory Satisfactory Quality of M&E Substantial Substantial Quality of ICR --- Substantial 12. Lessons This review has drawn five lessons based on the information in the ICR. A multi-phased approach starting with advisory services and analytics (ASA) followed by a pilot project and a scale-up project can gradually and significantly increase the development impact and implementation efficiency of interventions in an FCV context. Somalia is an FCV status country. The security situation and weak governance are major barriers to project implementation. Initially, the World Bank engaged with Somalia through an ASA (Resilient Multiple Use Water Services) assessing the feasibility of water source development in rural areas. A pilot project (Water for Agropastoral Livelihoods Pilot Project) was designed based on the findings of the ASA and implemented in Somaliland and Puntland. Based on the success of the pilot project and the experience gained during its implementation, the Water for Agro-pastoral Productivity and Resilience Project was designed and implemented to scale up the development of water systems and agricultural services and expands the size of the intervention territory. The ASA laid the foundation and principles of the follow-on investments. The two investment projects gradually increased the development impact to a larger community. The projects’ development of impact was significant as evidenced through the findings of the beneficiary survey conducted after the closing of the WAPRP. A third investment project is currently being implemented to further scale up the Page 22 of 25 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Somalia - WAPR (P167826) development impact by reaching to more agro-pastoral communities and supporting the establishment of mechanism for the sustainability of the development outcomes. High community engagement at the project design and implementation phases can increase the likelihood of the successful completion of the project activities. The project developed modalities for consultation and engagement with local agro-pastoral communities starting from the project design phase. The community committees established under the project were critical in the preparation of the community development plans. The communities were also involved in the selection of the water source development sites. The involvement of the agro-pastoral communities in project preparation and implementation increased their ownership of the project. The project activities were successfully completed within the original project implementation timeframe with the full support of the local agro-pastoral communities. The communities are responsible for the day-to- day operation and simple maintenance of the water systems. The project was a good example of the positive impact of community engagement on the successful implementation and completion of the project activities. Project implementation through federal and local governments can be highly effective in supporting the development of institutional capacity in governance in FCV countries. The common method of project implementation in Somalia has been through the United Nations agencies or non-governmental organizations. This project was implemented by the federal and state governments. Project implementation units were established at the federal government and each of the three states involved in the projects. The government employees worked at these project implementation units were trained in various aspects of project preparation and implementation and assumed full responsibility of the project. The relevant ministries also benefited from the trainings and other capacity development activities provided by the project. The interaction of the government officials with the local agro-pastoral communities contributed to the development of trust between the government and citizens. The experience gained during the implementation of the project will be retained by the government employees and will be further strengthened through the implementation of follow on Somalia Water for Rural Resilience Project. An incremental implementation strategy can be critical in mitigating the delayed effectiveness risk in projects to be implemented in different local jurisdictions simultaneously. The project was to be implemented in four different states. One of the conditions of project effectiveness was the signing of the subsidiary grant agreement between the federal government and each of the four project implementation states. Under the incremental implementation strategy, it was decided to start project implementation in the states that fulfilled the effectiveness conditions rather than waiting for all four states to fulfill the effectiveness conditions. This proved to be a highly effective approach in ensuring the start of the project activities in three states although the signing of the subsidiary agreement between the federal government and Somalia significantly delayed the project effectiveness for Somaliland because of political reasons. The activities to be implemented in Somaliland were eventually transferred to the follow-on project. The use of remote sensing data and spatial overlay of different thematic data layers in identifying the potential sites for water resource development can be highly instrumental in increasing the development impact of a project. The project utilized the Wadi Evaluation Tool (WET), an innovative approach developed under the pilot WALP for the selection of the most suitable sites for water resource development. The WET is based on “spatial analysis of available remote sensing data and spatial overlay of up to 30 different thematic data layers” (ICR footnote 15 Page 23 of 25 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Somalia - WAPR (P167826) on p.10). The data are used to create a map showing potential water development sites within a defined region of interest. The map is used to rank the water development sites according to their water harvesting potential. The project developed fewer water sites than planned, but the use WET ensured that the sites with the highest water harvesting potential were developed. These water sites reached to a higher number of project beneficiaries than targeted. The number of livestock benefiting from water harvested in those sites was also significantly higher than originally estimated. 13. Assessment Recommended? Yes ASSESSMENT_TABLE Please Explain The project constitutes the scaling-up phase of an earlier pilot intervention in the development of water and agricultural services in the drylands of Somalia. Both projects had successful results and a third project with the same objectives is currently under implementation. The project series is extremely important in improving the daily socio-economic lives of the agro-pastoral communities living in the drylands of Somalia. The projects also contribute to institutional capacity development at the federal and state government levels. Therefore, they contribute to the addressing of the fragility drivers in Somalia—population displacement because of lack of water and weak government capacity. An assessment following the completion of the third project in the series would provide invaluable data about the success factors or shortcomings of the projects and the sustainability of the projects’ outcomes following the end of the World Bank support. 14. Comments on Quality of ICR The ICR is candid and provides a detailed overview of the project. The report is focused on what occurred as a consequence of the project. It provides an accurate and substantiated set of observations supported by evidence from the project end survey and the project’s results framework to the extent possible given the latter’s weaknesses; the ICR together with its annexes provides an adequate evidence base to support the achievement of the project’s outcomes. The photographs provided in Annex 8 were significantly helpful in understanding the development impact of the project. There is a logical linking and integration of the various parts of the report. Its narrative is sufficiently evaluative. The report mostly follows and adequately responds to the Bank guidance and its Appendix L, Preparation of ICR in FCV Setting for IPF Operations both regard to ratings (specifically Outcome and M&E Quality ratings) and the performance narrative. However, the main report was longer than the length suggested in the Bank guidance, although some sections could have benefited from a more evaluative discussion such as the sections on Efficiency, Environmental, Social and Fiduciary Compliance, and Bank Performance. The entries in the “Lessons and Recommendations” are based on specific experiences of the project and would be useful in informing subsequent similar interventions, but they are mostly in the form of findings rather than lessons. While there were some moderate shortcomings in the ICR's responding to the Bank guidance and in the formulation of the lessons learned, the overall quality of the ICR is rated Substantial. Page 24 of 25 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Somalia - WAPR (P167826) a. Quality of ICR Rating Substantial Page 25 of 25