Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PK Sindh Irrigated Agri Productivity (P145813) Report Number: ICRR0024144 1. Project Data Project ID Project Name P145813 PK Sindh Irrigated Agri Productivity Country Practice Area(Lead) Pakistan Agriculture and Food L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IDA-56040 31-Dec-2021 184,400,510.10 Bank Approval Date Closing Date (Actual) 20-Mar-2015 30-Sep-2023 IBRD/IDA (USD) Grants (USD) Original Commitment 187,000,000.00 0.00 Revised Commitment 186,989,713.11 0.00 Actual 184,400,510.10 0.00 Prepared by Reviewed by ICR Review Coordinator Group Hassan Maher Amin Avjeet Singh Avjeet Singh IEGSD (Unit 4) Waly 2. Project Objectives and Components DEVOBJ_TBL a. Objectives The Project Development Objective (PDO) of the Sindh Irrigated Agriculture Productivity Enhancement Project (SIAPEP) as articulated in the Financing Agreement (FA, page 5) was identical to the one stated in the Project Appraisal Document (PAD, paragraph 11) and aimed to: "Improve irrigation water management at tertiary and field levels in Sindh." Page 1 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PK Sindh Irrigated Agri Productivity (P145813) Parsing the PDO. The PDO will be parsed based on two objectives: 1. To improve irrigation water management at the tertiary level in Sindh. 2. To improve irrigation water management at the field level in Sindh. b. Were the project objectives/key associated outcome targets revised during implementation? No c. Will a split evaluation be undertaken? No d. Components The PDO was supported by the following four components: 1. Community Water Infrastructure Improvement (appraisal cost: US$120.20 million, actual cost: US$174.83 million). This component included three sub-components: 1.1. This aimed to assist the Government of Sindh efforts to improve and rehabilitate tertiary distribution level water courses. Activities under this sub-component included farmer mobilization, establishment of Water Course Associations (WCAs) and their registration, survey and design, and construction. About 5,500 water courses were to be improved through the provision of lining (corresponding to 30% of water course length). The farmers would co-finance 24% of the costs through provision of skilled and unskilled labor. 1.2. This aimed to increase the resilience of the rural poor to floods through the strengthening of flood mitigation measures, including the establishment of shelters. It financed the provision of approximately 380 permanent flood shelters of three types in the seven most vulnerable districts in the irrigated areas of Sindh to increase their resilience to water related disasters such as flooding and heavy rains. Land for construction would be largely public or community owned land located in non-cultivable areas, preferably on higher ground. Once constructed, these shelters would be transferred to the Provincial Disaster Management Authority (PDMA) and their affiliates at the district level for operations and maintenance; some would be used for community activities. 1.3. This sub-component financed the provision of supervision and implementation assistance consultants (PSIAC) for project design and construction supervision for activities under component 1. PSIAC would assist the Project Implementation Unit (PIU) in construction supervision, engineering and designs, procurement, financial management, quality control, and the timely completion of strategic studies and pilot projects. 2. Promotion and Installation of High Efficiency Irrigation Systems (appraisal cost: US$65.90 million, actual cost: US$21.72 million). This component included two sub-components: 2.1. This aimed to support the installation of HEIS. This included drippers and bubblers for growing high value crops on irrigated and irrigable land; provide technical assistance packages to farmers on operations Page 2 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PK Sindh Irrigated Agri Productivity (P145813) and maintenance of HEIS; and provide additional training and assistance to farmers in the use of HEIS by specialists and consultants. 2.2. This aimed to support the establishment of HEIS demonstration sites and distribution of kitchen garden HEIS kits. This consisted of drip irrigation equipment for female-headed households and landless farmers. It would include 48 demonstration sites of 2 ha (5 acres), two in each district, and the distribution of approximately 10,000 kitchen garden HEIS kits consisting of drip irrigation equipment for a plot of 10m x 10m, including a small water storage of 160 liter with a conveyance pipe for the daily provision of water. The 100 kitchen garden HEIS kits for demonstration purposes would be provided free of charge; individual households requesting a HEIS kit would need to contribute 10% of the investment cost of US$100. 3. Improved Agriculture Practices (appraisal cost: US$24.40 million, actual cost: US$14.42 million). This component included three sub-components: 3.1. This would provide laser guided land leveling equipment and associated deep ripping equipment, and will facilitate training in the use of deep ripping equipment. 3.2. This would facilitate training to farmers to improve crop production and irrigation agronomy practices. 3.3. This would provide Technical Assistance and Training (TAT) Consultants to provide training to field teams, public sector staff and farmers in HEIS installation, operation and repair, crop diversification, crop production under HEIS, soil measurement and fertilizer application, and to support piloting of activities under sub-component 2.2 and training of trainers for operators in laser land leveling and deep ripping under sub-component 3.1. 4. Project Management, Monitoring and Evaluation, and Strategic Studies (appraisal cost: US$31.70 million, actual cost: US$24.83 million). This component included two sub-components: 4.1. This would provide support for project monitoring and evaluation and carrying out impact assessments. 4.2. This would provide support for the Project Implementation Unit and Project district offices and field offices, and for carrying out strategic and feasibility studies for pilot activities. e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project Cost. The total project cost was estimated at US$242.20 million. The actual cost according to the ICR Data Sheet (page 2) was US$184.40 million. The difference was due to the lack of any borrower counterpart funding. Financing. The project was financed through an IDA Credit worth US$187.00 million. The actual amount disbursed was US$184.40 million (ICR Data Sheet, page 2). Borrower Contribution. The Borrower was expected to contribute US$55.20 million. None of this amount materialized according to the ICR Data Sheet. In a further communication, the project team explained that the ICR Data sheet was system-generated with no option to edit/amend the numbers. The team explained that the borrower contributed PKR million 6,264.40 (equivalent to US$38.17 million) as reported by GoS, and farmer’s contribution was estimated at PKR million 7,764.88 (equivalent to US$51.40 million). Page 3 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PK Sindh Irrigated Agri Productivity (P145813) Dates. The project was approved on March 20, 2015 and became effective five months later on August 21, 2015. The Mid-Term Review (MTR) was conducted on April 29, 2019 which was five months later than the MTR date specified in the PAD on November 30, 2018. The project closed on September 30, 2023 which was 21 months beyond the original closing date on December 31, 2021. The project was restructured twice, both Level 2 restructuring, as follows: 1. On November 11, 2021, when the amount disbursed was US$129.29 million, in order to extend the loan closing date by twelve months from December 31, 2021 to December 31, 2022. This extension was needed to accommodate delays in project implementation resulting from the impacts of COVID-19 and associated restrictions and lockdowns, the locust plague of 2020 which forced implementing agencies to divert resources towards pest control, and severe floods in August-September 2022 that required the client to focus on emergency response and relief activities (ICR, paragraph 20). 2. On November 10, 2022, when the amount disbursed was US$175.22 million, in order to extend the loan closing date by nine months from December 31, 2022 to September 30, 2023. This extension was needed to ensure the full utilization of IDA credit and completion of slower-moving project activities, including support to HEIS and repair of irrigation infrastructure damaged by the 2022 floods (ICR, paragraph 20). 3. Relevance of Objectives Rationale Context at Appraisal. Sindh is Pakistan’s second most populous province with a population of 43 million people. Agriculture is the primary livelihood source for over 60% of the population, which resides in rural areas. While Sindh includes over 5.8 million hectares of irrigated land and more than 800,000 farms, the average farm size is small, productivity low, and yields lag other countries with similar agro-climatic conditions. Irrigated agriculture in Sindh suffers from poor management of water resources and inefficient water use. Key concerns include climate change, low surface water delivery efficiency, water distribution inequities, insufficient storage capacity and control structures, wasteful on-farm water use, inadequate drainage infrastructure resulting in water shortages, waterlogging, salinity, and environmental degradation, poor operation and maintenance (O&M), low-cost recovery, and a restrictive investment climate. Further, water theft and water scarcity affect tail-end farmers and lining watercourses is one way of assuring equitable water availability. This project aimed to improve irrigation water management at tertiary and field levels in Sindh which was expected to improve the management of water resources and increase the efficiency of water use. Previous Bank experience. The World Bank has extensive experience in irrigation projects and strong knowledge of irrigation issues. This project benefited from the Bank experience in irrigation and drainage sector operations in countries such as Mexico, Turkey and Uzbekistan, Kazakhstan, and the Kyrgyz Republic. Also, the experience and lessons from Pakistan's Sindh On-Farm Water Management Project were relevant to this project. Overall, the Bank was in a strong position to design and oversee the implementation of this project-given the rich experience and in-house irrigation expertise. Consistency with Bank Strategies. At appraisal, the PDO was in line with World Bank Group’s Country Partnership Strategy (CPS, FY2015-FY2019) for Pakistan. Specifically, the project would contribute to the Page 4 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PK Sindh Irrigated Agri Productivity (P145813) result areas of private sector development and inclusion. The project would directly contribute to the CPS Outcome 2.2: Increased productivity in farms and selected irrigation schemes, by supporting water use efficiency and encouraging technology that was expected to promote crop diversification and increased productivity. The project would also contribute to Outcome 3.3: Increased resilience to disasters in targeted areas, through supporting efficient management of the country’s water resources and contributing to adaptation to future climate change scenarios. The project was expected to contribute to the Bank’s twin goals of reducing poverty and promoting shared prosperity by directly benefiting around two million people (out of which 37% were below the poverty line). At completion, the CPS mentioned above was still used as a new CPS was not yet approved. The 2020 Systematic Country Diagnostic (SCD) acknowledged that promoting equity and inclusion, through increased productivity and environmental sustainability in agriculture, was a crucial pathway to sustained growth. Finally, the PDO was in line with the Pakistan Country Climate and Development Report (November 2022) which underscored the significance of resilience to floods and droughts, and agricultural recovery from the 2022 floods. Consistency with Country Strategies and priorities. At appraisal, the PDO was in line with Pakistan’s Vision 2025 and the Sindh Development Plan, where the important role of water in various sectors of the economy and its effective management in the future economic growth of the country and the province was emphasized. At project completion, the PDO continued to be in line with the Sindh Agriculture Policy (2018-2023) which reaffirmed the commitment to raise agricultural growth; reduce poverty, address food and nutritional insecurity, sustainably manage natural resources; and promote climate smart and resilient agriculture. The policy also recognized the pivotal role of irrigation and agriculture in achieving these objectives. The PDO was also in line with the National Water Policy (2018) which aimed to achieve by 2023: increased water use efficiency, reduce conveyance loss, increase surface water storage, replace, and refurbish water storage and irrigation infrastructure, as well as improve water accounting and mechanisms for data collection. Finally, the PDO was in line with the goals of the National Climate Change Policy (2021), which sought to enhance institutional capacity, coordination, resilient infrastructure, and early warning and response systems for effective disaster risk management, to achieve sustainable and equitable growth. Summary of Relevance of Objectives Assessment. The statement of objectives was clear, focused, and pitched at an adequate level of ambition relative to the targets set and the Bank's experience. However, the objective was narrow and did not reflect the overall activities covered by the project, most notable was increased agriculture productivity. At completion, the PDO continued to be relevant and in line with the Government strategies and priorities. While a new CPS was not yet approved at the time of completion, the PDO was in line with the Pakistan Country Climate and Development Report (November 2022) which underscored the significance of resilience to floods and droughts, and agricultural recovery from the 2022 floods. Therefore, the Relevance of Objectives is rated Substantial. Rating Relevance TBL Rating Substantial Page 5 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PK Sindh Irrigated Agri Productivity (P145813) 4. Achievement of Objectives (Efficacy) EFFICACY_TBL OBJECTIVE 1 Objective To improve irrigation water management at tertiary level in Sindh. Rationale Theory of Change (ToC). To achieve the stated objective the project, the project supported the following activities: Support community water infrastructure improvement which included community water course improvement through the provision of lining for tertiary level water distribution system and establishing water course associations to manage tertiary lines. Lining was expected to cover an average of 30% of tertiary canal length. The lining of water courses decreases conveyance losses and prevents seepage to groundwater aquifers; helps to deliver water faster to the farmers’ fields, so they can take advantage of the full duration of an irrigation cycle; and ensures equity in water distribution. In addition, the project would support flood mitigation to the poor through financing the construction of 380 flood shelters of three types in the most vulnerable seven districts of the Sindh. The project would also fund flood shelters which were critical infra-structure for the protection of the landless and the very poor, particularly women, children, elderly and disabled along with their belongings. The project would also support the establishment of water course associations to improve water course management. The expected outputs of the above-mentioned activities included: flood shelters established, improved water course infrastructure in place, and water course associations established. The expected intermediate outcome was: improved water course management. All of the above would contribute to achieving three PDO outcomes: increased WC conveyance efficiency, improved irrigation water management at tertiary level, and increased crop intensity. Anticipated long-term outcomes included: increased community resilience, increased crop productivity and diversification, and improved farmers' incomes and livelihoods. Overall, the activities reflected in the ToC were directly connected to the outputs, intermediate outcomes and outcomes in plausible causal chains. Even though, the ToC in the ICR lacked the critical assumptions that underpinned the achievement of the PDO, the project team reported the following potential critical assumptions in follow-up by IEG: Farmers support WC lining and infrastructure improvements and participation in WCAs; Farmers are willing and able to invest in HEIS and other agricultural technologies; WCAs continue to be operational beyond the end of the project period, undertaking necessary operations and maintenance as needed; and Farmers adopt and sustain improved agricultural practices promoted during trainings and as part of farmer field schools (FFSs). Also, the connection between the construction of flood shelters and the stated objective was not clear. Finally, the project design could have benefited from the Page 6 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PK Sindh Irrigated Agri Productivity (P145813) inclusion of an agribusiness/marketing activities, which would have improved the financial sustainability of project investments. Outputs/Intermediate Results  6,060 improved watercourses were fully functional (including a functioning Watercourse Association) exceeding the target of 5,500.  6,060 operational Water Course Associations (WCAs) were created and/or strengthened exceeding the target of 5,500. WCAs were expected to focus on operating, maintaining, and improving water canals and ancillary structures; improving water supply from surface and sub-soil water sources; managing water allocation within the command area and establishing schedules of water delivery; locating, constructing, maintaining, and improving field drains; and organizing labor for general or emergency repairs. In a further communication, the project team explained that while there was no written documentation for the operational status of the WCAs, the project management unit reported to the team that all 6,060 units were carrying out the basic functions of regular meetings, watercourse maintenance and repair, and water distribution decisions.  218,160 farmers (of which 10.6% were females which was below the target of 25%) were benefitting from improved watercourses exceeding the target of 198,000 farmers.  842,340 hectares (ha) were improved in the total command area exceeding the target of 550,000 ha.  847,186 ha were provided with irrigation and drainage services exceeding the target of 564,000 ha.  Number of people with access to shelters (none) compared to target of 106,650.  The construction of 380 flood shelters did not materialize. According to the ICR (paragraph 50): "the (project) design did not account for the community buy-in required to secure public or community- owned non-cultivable land to construct these shelters, nor for the interdepartmental coordination required to construct and maintain them." In a further communication, the project team explained that "the funds that were earmarked for the construction of the 380 flood shelters were repurposed to provide flood emergency assistance goods in response to the 2022 floods. SIAPEP provided 27,100 tents and 2,200,000 mosquito nets to be distributed during the flood relief operations." Outcomes  By project completion, 5,993 water courses had an increase in water course conveyance efficiency of at least 15% exceeding the target of 5,500 water courses (PDO outcome indicator #1). The ICR (Annex 1) noted that this indicator was estimated on the basis of a 10% sample of water courses supported under the project. Of the 6,060 water courses that were lined, 5,993 demonstrated an increase in water course conveyance efficiency of 18% on average, while 67 water courses showed an increase in water course conveyance efficiency of less than 15%. Direct project beneficiaries reached 2,176,822 exceeding the target of 2,000,000 (PDO outcome indicator #2), which represented the reach of the project’s direct benefits to the local population through activities across all components (irrigation and drainage services, laser leveling, deep ripping and farmer field schools).  Cropping intensity in command areas covered by water courses (lined under the project) increased by 27% from a baseline of 121.7% to 148.7% exceeding the target of a 20% increase (PDO outcome indicator #4). Summary of Efficacy Assessment. The project contributed to improving water management at the tertiary level through funding lining of water canals and supporting the establishment of water canal associations. Lining water canals improved conveyance efficiency through preventing seepage to groundwater aquifers. It Page 7 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PK Sindh Irrigated Agri Productivity (P145813) is also plausible to assume, given the improved conveyance, that water was delivered faster to the farmers’ fields and better equity in water distribution was achieved. As noted above, the project exceeded its targets for the three PDO outcome indicators pertaining to this objective. Also, most targets for intermediate results indicators were exceeded. However, construction of flood shelters did not materialize. Also, there was no evidence in the ICR to demonstrate that establishing water course associations helped management of tertiary level in Sindh. In a further communication, the project team explained that WCAs contributed to improved management "by coordinating water distribution to its members, organizing cleaning and maintenance labors, and ensuring minimization of any water loss in the management of this resource." Therefore, the efficacy with which this objective was achieved is rated Substantial with moderate shortcomings. Rating Substantial OBJECTIVE 2 Objective To improve irrigation water management at the field level in Sindh. Rationale Theory of Change (ToC). To achieve the stated objective the project, the project supported the following activities: 1. Support the promotion and installation of High Efficiency Irrigation Systems (HEISs). The project would support small and medium-sized HEISs for 2 to 10 ha (5 acres to 25 acres) farms. HEIS systems were expected to improve irrigation efficiency and use less water. To better demonstrate the technology the project would support 48 HEIS demonstration sites, and finance the associated training of department staff and farmers including women in the installation, operation and maintenance of the HEIS. The project would also support 100 kitchen garden HEIS kits for demonstration purposes for poor families to produce vegetables. 2. Support improved agriculture practices in order to increase the benefits of enhanced water availability from water course lining and high efficiency irrigation systems. The project would support farmers’ training in improved crop production techniques and in acquiring relevant technologies for use on their farms. The project would also finance laser land leveling and deep ripping. Laser land leveling was expected to contribute to about 30% savings of irrigation water, uniform seed germination, uniform uptake of fertilizers, and thereby increased yields. Deep ripping breaks soil hard pans and when followed by laser land leveling it helps water conservation and increase in crop yields. The project would also support improved agriculture production technology including establishing 1,500 farmers field schools (FFS) for about 10 crops on about 30% of the project renovated water courses with the members of the water course associations. The expected outputs of the above-mentioned activities included: HEIS established in target areas for high value crops, kitchen gardens with HEIS established for female farmers, farmer plots received laser land levelling and deep ripping, and farmers trained on improved irrigated agricultural practices. Page 8 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PK Sindh Irrigated Agri Productivity (P145813) The expected intermediate outcomes included: improved field conditions for more efficient irrigation, and improved irrigated agriculture production practices adopted in WC areas. All of the above would contribute to achieving two PDO outcomes: improved irrigation water management at the field level, and increased crop intensity. Anticipated long-term outcomes included: increased community resilience, increased crop productivity and diversification, and improved farmers' incomes and livelihoods. Overall, the activities reflected in the ToC were directly connected to the outputs, intermediate outcomes and outcomes in plausible causal chains. The critical assumptions were the same as provided by the team under Objective 1. Outputs/Intermediate Results  772 females were provided with HEIS which was below the target of 2,600.  4,934 ha were established under HEIS which was significantly below the target of 14,000 ha. This shortcoming was attributed to three main factors: (i) the level of investment required from the beneficiary (40%) for the relatively costly installation of the equipment, and (ii) power issues related to operating a HEIS, whereby the cost of electricity and/or fuel and the prevalence of power shortages dissuaded many potential users, and (iii) the need for specialized technical knowledge and inputs to run a HEIS irrigated farm (ICR, paragraph 31).  15,100 households were provided with HEIS in kitchen gardens exceeding the target of 10,000 ha.  452,143 ha received laser levelling, benefitting 55,361 farmers, significantly exceeding the target of 160,256 ha. The project provided 1,807 laser leveling units.  49,920 ha received deep ripping which was below the target of 106,837 ha. The ICR (paragraph 33) explained that farmers' demand for deep ripping units was limited as farmers could not attribute the benefits with the high initial cost given the availability of cheaper traditional alternatives.  51,484 farmers (of which 18.7% were females) were trained on improved agronomic practices and technologies exceeding the target of 37,500 farmers. This included 2,000 farmer field schools (FFSs) focused on irrigation water management, 772 HEIS trainings trained growers and operators on cleaning, fertigation, water meter readings, valve control and record keeping.  70,858 ha were under improved agronomic practices exceeding the target of 52,500 ha. Outcomes  By project completion, cropping intensity in command areas covered by water courses (lined under the project) increased by 27% from a baseline of 121.7% to 148.7% exceeding the target of a 20% increase (PDO outcome indicator #4). However, in areas covered by HEIS the project achieved an 11% increase in cropping intensity compared to the end target of 50% (PDO outcome indicator #5). Nonetheless, this shortfall did not impact the project overall efficacy significantly since HEIS activities covered only 0.6% of the overall command area covered under the project.  The project also provided 230,626 water users (of which 15% were females) with new/improved irrigation and drainage services exceeding the target of 210,000, but female water users fell short of the 25% target (PDO outcome indicator #3). Page 9 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PK Sindh Irrigated Agri Productivity (P145813) Summary of Efficacy Assessment. The project aimed to improve irrigation water management at the field levels in Sindh through installing High Efficiency Irrigation Systems (HEISs) the provision of kitchen gardens, the expansion of laser land levelling, and the extension of deep ripping. The project exceeded its outcome targets with regards to water users provided with new/improved irrigation and drainage services, but fell short of its target for the increase in crop intensity for areas under HEIS. The project met or exceeded several intermediate outcome targets. Most notable was the number of farmers trained on improved agronomic practices and the area under laser land levelling. However, the project fell significantly short of achieving its target for areas under HEIS, and areas that received deep ripping. Therefore, the efficacy with which this objective was achieved is rated Modest. Rating Modest OVERALL EFF TBL OBJ_TBL OVERALL EFFICACY Rationale Overall Efficacy is rated Substantial with moderate shortcomings. The project contributed to improving water management at the tertiary as lining water canals improved conveyance efficiency through preventing seepage to groundwater aquifers. It also helped to deliver water faster to the farmers’ fields and ensured equity in water distribution. The project exceeded its targets for the three PDO outcome indicators pertaining to this objective. However, construction of flood shelters did not materialize. Also, there was no evidence in the ICR to demonstrate that establishing water course associations helped management of tertiary level in Sindh. The project also contributed to improving water management at the field level through installing High Efficiency Irrigation Systems (HEISs), the provision of kitchen gardens, the expansion of laser land levelling, and the extension of deep ripping. The project exceeded one outcome target pertaining to this objectives and fell short of the other (cropping intensity under HEIS areas). The project also fell significantly short of achieving its target for the area under HEIS and the areas that received deep ripping. Overall Efficacy Rating Substantial 5. Efficiency Economic and Financial Analysis (EFA) Ex-ante  The economic internal rate of return (EIRR) of the project for the base case was estimated at 37.7% with a net present value (NPV) of US$268.10 million, under an opportunity cost of capital of 12%. The Page 10 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PK Sindh Irrigated Agri Productivity (P145813) economic benefits of project interventions included in the analysis were: (i) increased area under irrigation and cropping intensity; and (ii) increased crop yields.  Financial analysis was carried out on the basis of crop budgets for 14 crops. The results showed that net farm incomes (i.e., after full O&M costs were deducted) increased considerably in all cases. However, the calculated farm incomes were not sufficient to recover investment costs for 2 ha and 4 ha HEIS farms with row crops in unimproved water courses.  A financial cost-benefit analysis was carried out for the various HEIS models. The results showed that on-farm HEIS for orchards was financially viable with Financial Internal Rates of Return [FIRR] of above 12%, except for the 2 ha and 4 ha farms in unimproved water courses.  Sensitivity Analysis. The results of sensitivity analysis showed that the project’s economic viability was robust to adverse changes in project costs and benefits. The project was also robust to changes in incremental benefits and only becomes uneconomic if incremental benefits were reduced by 67%. Under the extreme situation of a 20% reduction in incremental benefits and a 20% increase in project costs, with a three-year delay in benefits, the project remained economically viable with an EIRR of 15.5%.  The total estimated project cost per beneficiary household was US$1,284 and the average cost per water user provided with new/improved irrigation and drainage services was estimated at US$836.  Overall, the PAD (Annex 6) included detailed EFA that provided a sound justification of the project investment at appraisal. Ex-post  The overall project economic rate of return (ERR) was estimated at 47.7% exceeding the EIRR of 37.7% estimated at appraisal. This was based on the conservative assumption that 50% of farmers reached by the project realize on average 50% of the incremental benefits as estimated in the Impact Assessment Reports (ICR, paragraph 40).The analysis at completion largely followed the same methodology applied at appraisal to ensure consistency and comparability.  Total project cost per beneficiary household was US$1,069 as compared to US$1,284 estimated at appraisal (83%), and the average cost per water user provided with new/improved irrigation and drainage services was US$576 as compared to US$836 estimated at appraisal (69%).  The average cost per water course improved by the project was US$27,552, which was 38% above the average cost estimated at appraisal US$19,937. The ICR (paragraph 39) attributed this to the impact of inflation and the fact that more than half of the water courses were lined 50% compared to the appraisal target of 30%.  Implementation efficiency. The project closed 21 months beyond its original closing date. The project experienced delays due to the impact of the COVID-19 and associated supply chain disruptions combined with a locust plague in 2020. Also, the massive floods in Sindh in 2022 caused extensive damage to project interventions, and required the Government to focus on emergency response and relief activities. The ICR (paragraph 20) noted that the irrigation infrastructure required repairs following the 2022 floods.  Overall, the ICR (Annex 4) included a detailed EFA that justified the project investments. Summary of Efficiency Assessment. The overall project economic rate of return (ERR) at completion was estimated at 47.7% exceeding the EIRR of 37.7% estimated at appraisal. Also, the total project cost per beneficiary household and the average cost per water user provided with new/improved irrigation were 83% and Page 11 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PK Sindh Irrigated Agri Productivity (P145813) 69% of their respective appraisal estimates. While the project experienced a significant delay of 21 months, this was largely due to external factors beyond the project control as noted above. Therefore, Efficiency is rated Substantial. Efficiency Rating Substantial a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 0 Appraisal  37.70  Not Applicable 0 ICR Estimate  47.70  Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome Relevance of Objectives was rated Substantial. Overall Efficacy was rated Substantial with moderate shortcomings. The project contributed to improving water management at the tertiary water course as lining water canals improved conveyance efficiency. The project also contributed to improving water management at the field level by installing High Efficiency Irrigation Systems (HEISs), the provision of kitchen gardens, the expansion of laser land levelling, and the extension of deep ripping. The project exceeded its targets for four out of five PDO outcome indicators, but fell short of one. Also, the envisioned construction of flood shelters did not materialize, and the targets for the area under HEIS and the area that received deep ripping were not achieved. In addition, there was no evidence in the ICR to demonstrate that establishing water course associations helped management of tertiary level courses. Efficiency was rated Substantial. The project economic rate of return at completion was estimated at 47.7% exceeding the 37.7% estimated at appraisal. Based on the assigned ratings for the three Outcome criteria, Outcome is rated Moderately Satisfactory due to moderate shortcomings pertaining to the operation's Efficacy. a. Outcome Rating Moderately Satisfactory 7. Risk to Development Outcome The following risks could potentially impact the Development Outcome: Page 12 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PK Sindh Irrigated Agri Productivity (P145813) 1. Institutional risk. The sustainability of Water course Associations (WCAs) and their ability to manage O&M of Water courses (WCs) continues to be one of the main risks related to the WC improvement outcomes. The WCAs remain the focal point for coordination between on-Farm Water Management field teams and farmers, so the likelihood that they will cease to exist is low. The benefits of improved WCs are highly valued by all farmers, therefore, their willingness to support WCA's O&M activities is likely to remain high. However, for this to happen, WCA members need to remain actively engaged and water management field teams keep these platforms active and relevant to farmers. That said, past experience in other regions with similar interventions show mixed results with regard to the ability of WCAs to maintain WCs without support from the government or external agencies. 2. Financial risk. The lack of financial support could result in farmers abandoning new practices or technologies beyond the project period. Currently, a significant proportion the of the HEIS, laser land leveling, and deep ripping is subsidized by the project, which could limit scale-up or further take-up of these technologies beyond the project. Further, the implementation team comprising individual consultants was dissolved at the time of project closure. While farmer investment in some interventions does provide some reassurance on farmer responsibility to maintain technologies, loss of continuity and handholding support could jeopardize the sustainability of the afore mentioned project activities. The following risk is emphasized by IEG: 3. Environmental risk. The project areas continue to be prone to severe floods and potential negative effects due to climate change. During implementation the project experienced severe floods in 2022 which caused damage to some project funded infrastructure. The project design needs to reflect mitigation measures to accommodate the impacts of climate change, specifically, irrigation infrastructure need to be designed to withstand floods that could potentially impact the Sindh province in the future. 8. Assessment of Bank Performance a. Quality-at-Entry  Strategic Relevance and Approach. The project was strategically relevant and in line with the Government priorities. The PDO was also in line with the Bank strategy at appraisal (see Section 3 for details). Increasing irrigation efficiency was vital to maximize the use of available water resources and increase agricultural production in Sindh province of Pakistan. The project reflected the importance of effective management in the future economic growth of the country and the province.  Technical, Financial and Economic Aspects. While the project design reflected a focused and realistic objective, the targets and the scope of interventions were conservative and did not explicitly incorporate agriculture productivity enhancement. Design focused on irrigation water management, but lacked a broader focus on sustainable irrigated agriculture systems for a more holistic response to current and future sectoral challenges including groundwater management and soil salinity (ICR, paragraphs 49 & 68). Design reflected well structured components with activities that supported improving irrigation management combined with improved field-level agricultural practices for increased productivity. However, constructing flood shelters lacked deeper analysis of the requirements of their successful implementation. This included Page 13 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PK Sindh Irrigated Agri Productivity (P145813) the interdepartmental coordination required to construct and maintain shelters and securing community-owned non-cultivable land to construct these shelters. Also, the promotion of HEIS required deeper analysis of its suitability for implementation under the local environment. The economic and financial analysis in the PAD was detailed enough and provided adequate justification of the project investments.  Poverty, Gender, and Social Development Aspects. The project was designed to target small farmers and poor communities, and project activities aimed to improve agricultural practices at the farm level. The project also promoted kitchen gardens KG which contributed to poverty reduction. The project was also expected to increase agricultural output and increase disposable income among beneficiaries. The deign featured activities that specifically targeted women including kitchen gardens and ensuring that women benefited from project promoted services such as HEIS, laser land leveling and deep ripping. However, the project overestimated the participation of women in project activities. Women participation in project activities was lower than expected, which according to the ICR (paragraph 55) was attributed to the conservative socio-cultural norms and a mismatch of household configurations used to estimate indicator targets.  Environmental and Fiduciary aspects. The project design reflected adequate environmental and fiduciary aspects.  Implementation Arrangements. While implementation arrangements were complicated involving several layers of government and spanning across 24 districts in the province, early mobilization of the PIU and adequate leadership and government commitment ensured effective implementation.  Risk Assessment. The project’s overall risk rating at appraisal was Substantial. Identified risks related to: implementation capacity, governance, stakeholders, and design. Risk identification in the PAD was adequate and reflected mitigation measures. However, implementation of HEIS proved more challenging than anticipated and the risk related to the implementation of flood shelters was not anticipated at appraisal.  M&E Arrangements. M&E arrangements featured systems, such as Geographic Information System (GIS), Management Information System (MIS), Information Communication and Technology (ICT) to monitor activities and verify data. Also a Grievance Redressal Mechanism (GRM) was developed to establish a social accountability system under the project. The results framework (RF) included adequate indicators to monitor project activities and assess achievement of outcomes. However, some indicators could have been better formulated, for example, the choice of cropping intensity as a measurement for improved irrigation management was not the best choice given that irrigation management could be improved without a change in cropping intensity in the case of orchards. Also, the mix of beneficiary types households, household members (total), farmers and females headed households, made beneficiary calculation inconsistent (ICR, paragraph 68). Summary of Quality at Entry (QAE) Assessment. The project was strategically relevant and in line with the Government priorities. While the project design reflected a focused and realistic objective, the objective was narrow and did not reflect the wide set of activities supported by the project. Also, the project design lacked a broader focus on sustainable irrigated agriculture systems for a more holistic response to current and future sectoral challenges including groundwater management and soil salinity. Women participation in project activities was overestimated against a conservative socio-cultural environment. Implementation arrangements and fiduciary and environmental aspects were adequate. However, the risk related to the challenges associated with flood shelters implementation was overlooked, and the risk associated with the implementation of HEIS was underestimated. M&E Page 14 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PK Sindh Irrigated Agri Productivity (P145813) arrangements were adequate, but there were some concerns with regards to the suitability of outcome indicators as noted above. Overall, QAE is rated Moderately Satisfactory due to moderate shortcomings pertaining to design, risk assessment and to a lesser extent M&E. Quality-at-Entry Rating Moderately Satisfactory b. Quality of supervision  The project was implemented under challenging conditions due to the COVID-19 outbreak in early 2020 followed by a locust plague in the same year and severe floods in 2022. The Bank conducted 17 implementation support missions. Implementation benefited from the locally based task team as this allowed regular missions for technical and implementation support. Locally based fiduciary and safeguards specialists enabled close collaboration with the PIU.  The project experienced four changes in Task Team Leadership during implementation. Despite effective handover and ensuring transfer of project knowledge during transitions, the four changes in TTL leadership was "less than ideal scenario, in the Pakistan portfolio (ICR, paragraph 71)."  The project benefited from the close collaboration between the Social GP and Pakistan Gender team on women’s interventions and ways to overcome limited female participation. However, the ICR (paragraph 72) noted that "collaboration or cross-support in project design between the Agriculture and Water GPs was limited."  Although a detailed restructuring of the project was prepared with the counterpart in 2021, including changes in the RF, components, and costs, it was not formalized because the revision process took too long. Eventually two restructurings were approved and both reflected only the extension of the closing date (see Section 2e).  The project team attempted to address the limited uptake of HEIS by farmers, however, all suggested solutions failed to overcome the farmers’ reluctance to HEIS. The project achieved limited results against its envisioned HEIS target (4,934 ha were established under HEIS which was significantly below the target of 14,000 ha). Summary of Quality of Supervision Assessment. The Bank supervision faced challenging implementation events including the outbreak of COVID-19, a locust plague, and the 2022 floods. The Bank team provided support to address implementation challenges, but the two planned restructurings only reflected an extension of the Loan closing dates. Despite the Bank's attempts to address HEIS uptake issues, low farmer uptake of this technology persisted. Overall, the Quality of Bank Supervision is rated Moderately Satisfactory due to moderate shortcomings related to the implementation of some project activities. Based on the assigned ratings to QAE and Bank Supervision, the Overall Bank Performance rating is Moderately Satisfactory due to moderate shortcomings pertaining to QAE and Quality of Bank Supervision. Page 15 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PK Sindh Irrigated Agri Productivity (P145813) Quality of Supervision Rating Moderately Satisfactory Overall Bank Performance Rating Moderately Satisfactory 9. M&E Design, Implementation, & Utilization a. M&E Design  The PDO was composed of two objectives (see Section 2), which were assessed based on five PDO outcome indicators.The PDO outcome indicators were measurable, relevant, included reasonable targets, and had baseline measurements where applicable. However, measuring the improvement in crop intensity as a proxy for improved water management did not reflect an accurate assessment. Since water management could be improved without changing crop intensity as in the case of orchard crops. The results framework (RF) could have benefited from integrating more specific outcome indicators related to water use efficiency such as crop per drop. Also, the RF lacked indicators to measure productivity and diversification, despite that the project supported several activities that would impact productivity and diversification.  The design reflected a mix of beneficiary types, including households, household members, farmers, and female-headed households. This, in turn , made beneficiary calculations inconsistent, which led to a mismatch of household configurations used to estimate indicator targets.  The RF included 15 intermediate results indicators (IRIs) to track the progress of the different project activities. The IRIs were measurable, reflected reasonable targets, and were connected to the project activities. However, the RF lacked an indicator to qualitatively measure the functionality of water course associations.  Overall, M&E design had some notable shortcomings as reflected above. There were concerns on using the crop intensity as a proxy measure for improved water management and the RF lacked an indicator to quantitatively measure the functionality of water course associations. Finally, the mix of beneficiary types led to a mismatch of household configurations used to estimate indicator targets. b. M&E Implementation  According to the ICR (paragraph 58) the implementation of M&E activities was through individual consultants at the PIU and district level, and an agency was engaged to support the various assessments. The project's M&E consultants monitored field implementation and compliance through regular visits, including spot checks and participation in key activities, such as the trainings conducted for farmers and other stakeholders.  M&E implementation benefited from using several platforms including a Geographic Information System (GIS), Management Information System (MIS), Information Communication and Technology (ICT) to ensure robust periodic monitoring of implementation. Also, the project maintained and hosted a public domain that reflected regular reports and a performance dashboard to serve as a social accountability mechanism. While a geospatial monitoring dashboard was established, and updated with data from the field, remote sensing was not used on the ground (ICR, paragraph 57). Page 16 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PK Sindh Irrigated Agri Productivity (P145813)  In addition to regular monitoring activities of improved cropping techniques, a mid-term review, and an impact assessment at completion, were also carried out. Also, the project conducted an impact assessment of water courses improvements using a pre-post approach on a 10% sample basis, that reflected head, middle, and tail farmers of the water course.  Overall, M&E implementation was successful and generated enough data to assess the project outcomes. The system provided necessary data and information to monitor project performance. Also, the digital MIS and geospatial monitoring dashboard allowed auditing and verification of the construction of civil works. However, M&E design shortcomings were not addressed in the two project restructurings. c. M&E Utilization  According to the ICR (paragraph 59) "the project team relied on monthly, quarterly, and annual reports produced by the M&E consultants to track implementation against planned activities."  The M&E system reflected systematic data collection which helped assess the project performance during the Implementation Support Missions, as well as audit civil works to ensure transparency and accountability (ICR, paragraph 59). Summary of M&E Quality. M&E design had moderate shortcomings as noted above. While implementation was successful, M&E shortcomings were not rectified during implementation. Utilization was evident in some management decisions including the extension of water course lining to 50% rather than 30% and using Precast Concrete Parabolic Segments (PCPS) for lining rather than bricks. Overall, M&E Quality is rated Substantial with moderate shortcomings pertaining to M&E design. M&E Quality Rating Substantial 10. Other Issues a. Safeguards  Environmental Category and Safeguards. This project was classified as a “Category B” under OP/BP 4.01 with a partial assessment as the environmental impacts of the project interventions were expected to have positive environmental impacts including: increased water use efficiency, reduction in loss of arable land due to use of water high in salinity and reduction in pesticide usage. Three environmental and one social safeguard policies were triggered: Environmental Assessment (OP/BP 4.01), Pest Management (OP/BP 4.09), Projects on International Water Ways (OP/BP 7.50), and Involuntary Resettlement (OP/BP 4.12). In compliance with the Bank's safeguard policies, the borrower prepared an Environmental Impact Assessment (EIA), an Integrated Pest Management Plan (IPMP) and a Resettlement Policy Framework (RPF). The approved safeguards documents were disclosed on March 4, 2014. Page 17 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PK Sindh Irrigated Agri Productivity (P145813)  Compliance with Environmental and Social Safeguards. According to the ICR (paragraphs 62 & 63) "the project successfully complied with all the environmental safeguard requirements" and "successfully complied with social safeguard requirements." b. Fiduciary Compliance  Financial Management (FM). FM performance was satisfactory for most of the project implementation and FM risk rating remained Moderate. The books of accounts for project transactions were appropriately maintained and fund flow remained smooth throughout the project's life. An external firm conducted regular internal audits of the project during its implementation. However, FM performance rating experienced a downgrade to Moderately Satisfactory (MS) due to delays in recovering advances from non-performing Water User Associations. This situation was rectified through close and regular follow-ups and all advances were successfully recovered by the first quarter of FY 2023. At completion, FM performance was rated Satisfactory.  Procurement. The project used Systematic Tracking of Exchanges in Procurement (STEP), and most of the project funds were disbursed through community-driven procurement approaches and matching grants. To enhance transparency, the project introduced digital wallets for beneficiaries, but the uptake limited (ICR, paragraph 65). Two major consultancies (M&E and PSIAC) experienced challenging contract management because firms provided less qualified staff than contractually mandated. The ICR (paragraph 65) noted that "timely action ensured services were delivered as per contract terms." At completion, procurement performance was rated Moderately Satisfactory. c. Unintended impacts (Positive or Negative) None. d. Other The ICR (paragraph 48) stated that: "the significant impact of WCs was evident in the Government of Sindh’s commitment to sustaining project developments in response to the 2022 floods. The Government provided PKR2,151.34 million (US$) from its own funds to repair 1,593 WCs (26% of WCs rehabilitated by the project) and 56 water storage tanks (10% of the storage tanks established by the project), which were damaged by this natural disaster. The civil works related to the repair continued during the last months of the project, and beyond its close on 30 September 2023, until 30 November 2023. SIAPEP was also able to finance – for total of US$7.86 million – the provision of flood emergency response goods, including 27,100 tents and 2,200,000 mosquito nets." 11. Ratings Page 18 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PK Sindh Irrigated Agri Productivity (P145813) Reason for Ratings ICR IEG Disagreements/Comment Moderately Outcome Moderately Satisfactory Satisfactory Moderately Bank Performance Moderately Satisfactory Satisfactory Quality of M&E Substantial Substantial Quality of ICR --- Substantial 12. Lessons The ICR included four lessons. the following three are emphasized with some adaptation of language: 1. To effectively address issues related to the water-agriculture nexus, a more holistic approach that incorporates cross-cutting interventions combining technical, institutional, and capacity-building activities, is required. SIAPEP focused on infrastructure improvement and supply, and capacity-building for beneficiaries. A stronger emphasis on institutional aspects and interventions targeting agricultural productivity (which naturally complements improved irrigation water availability and is its intended outcome) could have enhanced project outcomes. This requires the Bank to emphasize more collaboration and/or cross-support in project design between the Agriculture and Water GPs. 2. To ensure comprehensive assessment of outcomes, projects need a clear ToC combined with a strong M&E design that informs adjustments to strategy and strengthens outcomes, and provides rich evidence for future operations. Key data on project activities that extends beyond indicator results or activity lists, based on a thorough understanding of causal relationships within the project, can be used to analyze and adjust project processes. While the project collected data on numerous parameters, a number of indicators (which would have benefitted project assessment) were not measured or monitored including: the operation and capacity of WCAs, HEIS operational costs, the effect of HEIS on input savings (water, fertilizer and labor), crop diversification, use and upkeep of laser land leveling (LL) and deep ripping (DR) equipment, the effects of LL and DR on water savings, the delivery of tents and nets, and the activity of the Project Supervision & Implementation Assistance Consultants (PSIACs). Also, data on adoption of practices and technologies promoted by the project beyond its direct beneficiaries would have been useful to verify and assess spill-over effect. 3. To maximize benefits and strengthen the sustainability of production, irrigation project need to incorporate elements of agribusiness and agricultural marketing. Field interactions indicated that there is room to encourage entrepreneurship among project beneficiaries/farmers and foster a culture of calculated risk-taking for productive investments, rather than relying on subsidies. Capacity building, collective marketing schemes and improved market linkages could bolster irrigation and agriculture projects. Within the project cycle, timeliness of interventions beyond improving irrigation water management is also crucial. For instance, activities such as the farmer field schools implemented in this project should be conducted early in the implementation cycle rather than towards its end, since they require time and encouragement for adoption. Similarly, Page 19 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PK Sindh Irrigated Agri Productivity (P145813) activities related to marketing through producer organizations need to be implemented shortly after project implementation begins to allow enough time to establish the relevant marketing links. The following lesson is emphasized by IEG: 4. To ensure successful implementation of activities targeting women in a society that exhibits a conservative socio-cultural behavior, women activities need to be tailored to fit the cultural norms, and targets need to be realistically set at appraisal. The project experience demonstrated that featuring activities geared towards women did not automatically mean that women would participate as envisioned. Project design needs to include a comprehensive social analysis to clearly understand the role of women within the society and design relevant activities that would ensure women participation. 13. Assessment Recommended? No 14. Comments on Quality of ICR  Quality of Evidence. The ICR reflected an evidence base that was adequate to assess the project outcomes. The ICR could have benefited from reporting survey results to reflect the beneficiaries' opinion on the project achievements. In a further communication the project team shared with IEG the final impact assessment.  Results Orientation. The ICR included an adequate discussion on the achievement of the PDO.  Quality of Analysis. The ICR included an adequate assessment of outcomes even though there were some shortcomings in the M&E design.  Internal Consistency. Various parts of the ICR were internally consistent and logically linked and integrated.  Lessons. Lessons reflected the project experience and were based on evidence and analysis.  Consistency with guidelines. The ICR used the standard structure defined in the Guidelines and used available evidence to justify the assigned ratings.  Conciseness. Overall, the ICR provided a clear and concise coverage of project activities to the extent possible, and candidly reported on most shortcomings. However, the ToC lacked the critical assumptions that underpinned the achievement of the PDO. Summary of the Quality of ICR Assessment. The ICR included an adequate assessment of outcomes. The lessons drawn by the ICR were relevant. Most sections were concise and reflected relevant evidence. Overall, the Quality of the ICR is rated Substantial. a. Quality of ICR Rating Page 20 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PK Sindh Irrigated Agri Productivity (P145813) Substantial Page 21 of 21