Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Natl Horticulture & Livestock Project (P143841) Report Number: ICRR0022816 1. Project Data Project ID Project Name P143841 Natl Horticulture & Livestock Project Country Practice Area(Lead) Afghanistan Agriculture and Food L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) TF-13820 31-Dec-2018 176,326,960.00 Bank Approval Date Closing Date (Actual) 12-Apr-2013 31-Dec-2020 IBRD/IDA (USD) Grants (USD) Original Commitment 190,000,000.00 190,000,000.00 Revised Commitment 176,326,960.00 176,326,960.00 Actual 176,326,960.00 176,326,960.00 Prepared by Reviewed by ICR Review Coordinator Group Richard Anson Ebru Karamete Christopher David Nelson IEGSD (Unit 4) 2. Project Objectives and Components DEVOBJ_TBL a. Objectives The project development objective (PDO), as stated in the Trust Fund Grant Agreement (TFA, 2012) and the Project Appraisal Document (PAD, 2012), was to: “To promote adoption of improved production practices by target farmers, with the gradual rollout of farmer-centric agricultural services systems and investment support”. While the National Horticulture and Livestock Productivity Project (NHLP Project) had one level-2 restructuring (following the additional financing/AF in 2016), the PDO remained the same. The main revisions Page 1 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Natl Horticulture & Livestock Project (P143841) included: (a) adjustments in the Results Monitoring Framework to improve reporting under the revised project and to enhance alignment with the country’s priorities; (b) scaling-up of several project activities, and therefore increasing several of the outcome targets, with respect to the same PDO and components (described in section 2 (e)), with some added sub-components; and (c) extending project closing from the original end date of December 31, 2018 to December 31, 2020. For purposes of assessing the extent to which the PDO was achieved (Section 4), this review parses the PDO into two objectives: Objective 1: To promote adoption of improved production practices by target farmers; and Objective 2: To implement gradual rollout of farmer-centric agricultural services systems and investment support. b. Were the project objectives/key associated outcome targets revised during implementation? Yes Did the Board approve the revised objectives/key associated outcome targets? No c. Will a split evaluation be undertaken? No d. Components 1) Horticultural Production (Original allocation: US$56.0 million; Revised allocation with AF: US$ 125.4 million; Actual cost: US$ 102.2 million). This component aimed to support sustainable increases in horticulture production by increased numbers of smallholders, including women, and included expanded extension services to smallholders to increase production. With the support of the AF (in 2016), original activities were expanded, and there were some new activities and increased targets, resulting in three inter- connected subcomponents: (a) new orchard establishment, rehabilitation and technology transfer and expanded extension services to increase production practices by increased numbers of smallholders (with various targets increased); (b) establishing kitchen gardens and microgreen houses; and (c) provision of various types of marketing support to the participating farmers, including: different harvesting and post- harvesting training to targeted male and female farmers; distributing farm tools to improve product quality and efficiency in the harvest and marketing of horticultural products; and construction of raisin drying houses; and strengthening market linkages between producers and traders; 2) Animal Production and Health (Original allocation: US$ 29.7 million; Revised allocation with AF: US$ 53.4 million; Actual cost: US$ 37.7 million). This component aimed to promote increased livestock production, marketing support, and animal health, and also introduced new activities under the livestock program involving dairy, fisheries and beekeeping. The livestock support involved supporting small ruminants, notably, poultry, and also addressing the animal health needs of peri-urban and pastoral households. This component, with support from the AF, also provided enhanced extension which included poultry groups, which received regular animal production and health training, and strengthened livestock market linkages, including supporting the private sector on developing milk collection infrastructure; and Page 2 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Natl Horticulture & Livestock Project (P143841) (3) Implementation Management and Technical Assistance Support (Original allocation: US$ 28.9 million; Revised allocation with AF: US$ 39.45 million; Actual cost: US$ 36.4 million). This component aimed to provide the required technical and management support to help ensure efficient and effective implementation of the project (including planning, M&E, procurement and financial management). With the support of the AF, the TA support was scaled up to better manage the Project’s expanded geographic scope, activities and numbers of beneficiaries. e. Comments on Project Cost, Financing, Borrower Contribution, and Dates (i) Project Cost: The total project cost at approval was US$ 114.6.0 million. Together with the Additional Financing (AF) of US$ 90.0 million, the total project cost was US$ 204.6 million. The actual project cost at closing was US$ 186.1 million (or about 91% of estimated total costs). The difference was due to some cancellation of the grant amount (US$ 13.7 million) and to a decrease in the counterpart contribution from the beneficiaries (see ICR, Data Sheet and Annex 3). (ii) Financing: At approval, the Trust Fund grant (TF-13820) was US$ 100 million, with beneficiary contribution of US$ 14.6 million. The AF (2016) provided an additional US$ 90 million, and also involved an additional beneficiary contribution of US$ 1.0 million. By the closing of the project, total financing (and disbursements) was US$ 186.1 million (with total trust fund managed by the World Bank of US$ 176.3 million and beneficiary counterpart contribution of US$ 9.7 million). (iii) Borrower Contribution: There was no Borrower contribution. (iv) Dates: The project was approved on April 12, 2012, became effective on December 22, 2012. A mid- term review was not carried out. The original closing date was December 31, 2018, with the actual closing date being December 31, 2020 (i.e., an extension of 24 months).This extension was due primarily to enable implementation of significant additional financing, which was coupled with increased targets, and also due to some implementation delays, which were exacerbated by the COVID conditions in Afghanistan. (v) Restructurings and Significant Changes During Implementation: The project had one restructuring, at the time of processing the Additional Financing/AF (in 2016). Although the PDO remained unchanged throughout the project implementation period, the Results Framework, during the restructuring for the AF was modified, both in terms of adding some indicators and increasing some targets, to streamline and ensure better alignment with the PDO, and to include targets on gender-disaggregated number of beneficiaries. The original two PDO indicators that tracked results on the adoption of improved production practices remained unchanged, coupled with a slight increase in targets, and some modifications in wording. However, the PDO indicator to assess the success of farmer-centric service delivery, “total expenditures directly reaching to beneficiary farmers”, was dropped. A new indicator was introduced: “number of producers with improved post-production facilities/ tools and market access”. This indicator aims to capture the outcomes from newly introduced activities for strengthening market access. The change was made for two reasons: the dropped indicator was more output-related, and the newly introduced indicator aimed at measuring the progress against the planned marketing activities introduced with the AF. The other original PDO indicator to track results involved the rollout of farmer-centric service delivery system: “target farmers satisfied with agricultural and rural advisory services”, remained with the original target, but with some modification in wording (ICR, paras. 11 and 12). The ICR (paras. 14 – 16) highlights the specific Page 3 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Natl Horticulture & Livestock Project (P143841) adjustments in the 3 project components arising from the above revisions, including scaling up existing activities and several targets, and adding two project subcomponents. Other important revisions/improvements of the project, mainly to sharpen the operationalization of achieving the PDO and the objectives of the three components, included the following aspects (ICR, paras. 17 and 18): (i) Intermediate Result Indicators (IRIs): The IRIs were revised with the 2016 AF: 13 new IRIs were introduced, and 10 IRIs were removed from the Results Framework (RF). The other 15 IRIs were also modified and their targets were raised to reflect the project’s scaling up under the AF. The original RF, with over 25 IRIs, was complex, and the complexity remained even after the AF as the number of IRIs increased with the AF; (ii) Technical Assistance: The AF also made provision for additional technical assistance and capacity building support to the Ministry of Agriculture, Irrigation and Livestock (MAIL) to formulate technical studies to inform implementation and policy development, including the piloting of mainstreaming of NHLP service delivery systems below the central level; (iii) Implementation Arrangements: (a) the engagement of the MAIL and Provincial-level Directorate of Agriculture, Irrigation and Livestock (DAIL) extension staff at the district level to implement NHLP activities, with visits to target communities to deliver extension messages and best practices; (b) enhanced coordination with various development partners/programs; (c) the appointment of a capable International Finance Advisor to oversee overall financial management of the project; and (d) the improvement in farmers’ contribution record system, monitoring of project fund flows and the internal control system. As part of the Bank’s broader portfolio adjustment discussions for COVID-19 response (carried out jointly by GoIRA and World Bank in June 2020), it was agreed to cancel US$12.5 million. The cancellation was done through global restructuring at the country level; no project level restructuring was required to reflect cancellation. Rationale for Restructuring/Enhancements: The ICR provides a sound rationale for the above restructuring and associated improvements, which were triggered by the additional financing, without changing the PDOs. Key changes under the AF were intended to scale up NHLP’s regular project activities and to strengthen the links between producers and traders, with a focus on post-harvest handling and processing. Promoting Farmer Learning Resource Centers (FLRCs) were very important to strengthen the extension system and to improve delivery of key extension and advisory services to smallholder farmers. The inclusion of a market access agenda under both Components (1) and (2) as standalone subcomponents was intended to strengthen the market linkages between producers and traders of horticulture and livestock products, as well as marketing services and integration of smallholders into target value chains. The inclusion of market access interventions also was intended to focus not only on improving productivity, but also on increasing value addition in the horticulture and livestock subsectors. Other changes were meant to correct some constraints at design and implementation stages, without any significant change to the logic of Theory of Change (ICR, para. 19). In retrospect, some of increased targets were overly ambitious. 3. Relevance of Objectives Page 4 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Natl Horticulture & Livestock Project (P143841) Rationale The project’s objectives were substantially relevant to addressing the country’s key developmental challenges and to contributing to the implementation and achievement of the country’s main developmental strategies. At the country level, the Afghanistan National Development Strategy: 2008 – 2013, assigned high priority to the agricultural sector and to increased agricultural productivity. At the sectoral level, the National Agriculture Development Framework (NADF), developed by the Ministry of Agriculture, Irrigation, and Livestock (MAIL), in 2009, provided the main strategy framework for this project. During the project’s implementation period, the project continued to be fully consistent with the Government’s updated development strategy: Afghanistan National Peace and Development Framework (ANPDF). The project also was and remained during implementation strongly aligned with and addressed key elements and pillars of the Bank’s strategy documents, and supporting analytical reports, overlapping with two Bank strategy documents for Afghanistan, including: (i) Interim Strategy Note (ISN) 2012-2014 (extended to FY16); Afghanistan Diagnostic Report (2016); and (iii) Country Partnership Framework (CPF) FY17-20 (extended past FY20 as per the Performance and Learning Review). At closing, the NHLP, including the support from the AF, was fully aligned with CPF FY17-20, directly with Pillar 2, and particularly with Objective 2.4, regarding increased agricultural productivity. During implementation, there were not changing needs, aside from worsening security conditions and limited institutional capacities, which did not warrant a change in the project’s design and implementation arrangements, while ensuring adequate technical support was provided. In addition, the Project also indirectly contributed: (a) to objectives 1 and 2 (building the legitimacy and capacity of institutions; and equitable service delivery), as outlined in the ISN FY12 – 14; and (b) to Pillar 1 of the CPF FY17-20, with respect to building strong and accountable institutions, especially involving Objective 1.3: Improved service delivery through enhanced citizens’ engagement with the state; and cross-cutting issues of Gender and Climate Change. Also, the project built on and expands the achievements and good performance of the previous project funded by the Bank (Horticulture and Livestock Productivity (HLP) Project, which closed in 2012). The ICR correctly concludes, that the project’s strong and sustained relevance and full alignment to the country’s vision/strategies and to the Bank’s CPF throughout implementation and to project closing, the relevance of the PDO is rated “Substantial”. Rating Relevance TBL Rating Substantial 4. Achievement of Objectives (Efficacy) EFFICACY_TBL OBJECTIVE 1 Objective To implement gradual rollout of farmer-centric agricultural services systems and investment support. Rationale Page 5 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Natl Horticulture & Livestock Project (P143841) The project’s original design included a results framework in the Project Appraisal Document (PAD), but did not include a theory of change (ToC) because it was not required at the time the PAD was written. The ICR reconstructed a ToC for the project, which was consistent with its overall PDO, core outcomes and components (ICR, Figure 1). The ICR figure highlights the rationale for and operational logic of this objective 1 to help achieve “NHLP’s higher level objective to contribute to the overarching goal of increased productivity and overall production of horticultural products and improved production and health” (ICR, para. 5). Theory of Change: The prioritized interventions with respect to this objective 1 included 5 types: a) carrying out improved extension system support for horticultural production, with respect to: improved extension messages on orchard management; value addition and marketing; (b) providing increased investment support for horticultural production, with respect to: orchard development and management; value-addition; and marketing; (c) providing improved extension system support and coverage for animal production and health, involving: organizational support for CIGs; extension messages on animal production and health; (d) productive investment support for animal production and health, with respect to: investment support for adoption of improved technologies; establishment of an animal health surveillance and control system (SHSCS); vaccination program; central veterinary investigation laboratory; intervention models; and (e) providing improved policy and institutional support, technical assistance and “enabling environment” for expanded role of private sector (and thereby contributing to improved delivery systems). At the same time, given the country’s public sector service delivery weaknesses and FCV challenges, and the resulting lag in expanding the envisioned private sector engagement, the implementation arrangements could have included additional technical assistance to enable a more realistic transition to and role of public service providers and an expanded private sector. Key IRIs/Outputs (ICR, Annex 1): (in the event an indicator was added at the time of AF/restructuring, the year is shown below) (i) No. of horticultural packages extended to farmers and beneficiaries: Original Target: 600; Actual: 500; 83%; For females: Original Target: 200; Actual: 0: 0%; For males: Original Target: 400; Actual: 500; 125%; (ii) No. of farmers supported with improved post-harvest knowledge & tools: Original Target: 35,000; Actual: 15,615: 45%; For females: Original Target: 5,000; Actual: 4,297, 86%; For males: Original Target: 30,000; Actual: 14,473; 48%; (iii) No. of farmer groups/associations linked with traders/merchants and processor groups: Original Target: 200; Actual: 28; 15%; (iv) No. of beneficiaries with farmer groups/associations linked with traders/merchants and processors: Original Target: 8,000; Actual: 1,167; 15%; Page 6 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Natl Horticulture & Livestock Project (P143841) (v) No. of animals annually vaccinated against TADs/zoonoses/production diseases by target farmers: Ruminants Original Target: 15 million; Actual: 15.2 million; %: 101%; Poultry: Original Target: 1 million; Actual: 0.6 million; 60%; (vi) No. of Government policy, strategic planning supported and approved: Original Target: 5; Actual: 5; 100%; (vii) No. of Livestock investment packages provided to farmers and beneficiaries: Original Target: 400; Actual: 244: %: 61%; For females: Original Target: 200; Actual: 189; 95%; For males: Original Target: 200; Actual: 55; 28%; (viii) No. of Poor women provided with semi-commercial 100 bird model: Original Target: 5000; Actual: 3000; 60%; (ix) No. of Ultra-poor women provided with 30 bird improved backyard layer: Original Target: 35000; Actual: 25000; 71%; (x) No. of Warm-water aquaculture production systems supported: Original Target: 6; Actual: 20; 333%; (xi) No. of Beekeeping enterprises supported: Original Target: 500; Actual: 500; 100%; (xii) No. of Farmers supported with improved value addition, knowledge, tools and facilities: Original Target: 94400; Actual: 64933; 69%; For females: Original Target: 78450; Actual: 48423: %: 62%; For males: Original Target: 15950: Actual: 16644;104%; (xiii) * No. of Provinces supported in Private Sector driven dairy development: Original Target: 4; Actual: 0; 0%; (xiv) * No. of Groups supported in private sector driven dairy development: Original Target: 50: Actual: 0; 0%; (xv) No. of Groups of farmer groups/associations linked with traders/merchants and processors: Original Target: 1050; Actual: 302; 29%; (xvi) No. of farmers of farmer groups/associations linked with traders/merchants and processors: Original Target: 40000; Actual: 6005; 15%; (xvii) * No. of Periodic surveys/assessments conducted/implemented Mgt-TA Support: Original Target: 3; Actual: 2; 67%; (xviii) No. of training modules for gender awareness developed (with strategy): Original Target: 4: Actual: 4; 100%; Key Outcomes (ICR, Annex 1) Page 7 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Natl Horticulture & Livestock Project (P143841) (i) Percentage of target clients/farmers satisfied with agricultural services: Original Target: 75; Actual: 94: 125%; (ii) No. of targeted clients/farmers satisfied with agricultural services: Males: Original Target: 289040; Actual: 259685; 90%; (iii) No. of targeted clients/farmers satisfied with agricultural services: Females: Original Target:176320; Revised Target (2016):189347; Actual: 147238; % of Revised: 78%; (iv) No. of producers with improved post-production facilities, tools and market access: Original Target: 181140 (2015); Actual: 106066; %: 59%; In summary, the efficacy with which Objective 1 was achieved is rated Modest, because most of the output and outcome targets were not achieved (i.e., only 22% of output targets and 25% of outcome targets were achieved, vis-à-vis the above results). Despite the roll-out of farmer-centric approaches and processes for improved service delivery, NHLP fell short of achieving most of its targets for investment and marketing support. NHLP achieved the end targets for the extension outreach in both horticulture and livestock components. Accordingly, the project’s indicators demonstrate partial achievement of Objective 1. The main reasons are due to various types of institutional capacity and implementation weaknesses (for further details, see ICR, paras. 21 – 27). Rating Modest OBJECTIVE 2 Objective To promote adoption of improved production practices by target farmers. Rationale Similar to Objective 1, the rationale for Objective 2 is reflected in the project’s original design and in the Theory of Change/ToC reconstructed in the ICR (Figure 1 and para. 5), with respect to the various types of project-funded activities/interventions, which contributed to generating sustainable IRIs/outputs, and then generating sustainable outcome with respect to promoting farmer adoption of improved production practices by target farmers (see key indicators below). The project’s AF/restructuring also contributed to achieving sustained outputs and outcomes, thereby contributing to the longer-term PDO of increased productivity and overall production of horticultural products and improved production and health. Theory of Change: The ToC (ICR, Figure 1) identifies and illustrates the results chain and linkages between the main challenges and the priority inputs/interventions and resulting outputs/intermediate results, which then help generate the core PDO-level outcome and the higher-level outcome, with respect to objective 2, namely: To promote adoption of improved production practices by target farmers. The prioritized interventions with respect to this objective included 4 types: Page 8 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Natl Horticulture & Livestock Project (P143841) (a) promoting farmer adoption of improved technologies/practices with respect to: orchard management; value addition and marketing; (b) promoting farmer adoption of increased investments in: orchard development and management; value- addition; and marketing; (c) promoting farmer adoption of improved animal production and health practices, involving: organizational support for CIGs; extension messages on animal production and health; and (d) promoting farmer increases in productive investments and adoption of improved practices for animal production and health, with respect to: investment support for adoption of improved livestock technologies; adopting the practices arising from the establishment of an animal health surveillance and control system (SHSCS); vaccination program; central veterinary services; and adopting improved livestock intervention models. Based on various assumptions highlighted in the ICR Figure 1, these interventions generated the strategic outputs/intermediate results involving farmer adoption of improved technologies involving: orchard development (rehabilitated, new); new areas of vegetables planted; improved backyard layers adopted and installed; animals vaccinated; numbers of farmers adopting improved horticulture and livestock technologies and practices; numbers of Farmer Learning and Resource Centers used by increased numbers of farmers; numbers of farmers linked with market traders/processors. With these IR/outputs, they helped to generate the broad PDO 1 level outcome of increased adoption of improved production practices by horticultural and livestock farmers. Key IRIs/Outputs (ICR, Annex 1): (in the event an indicator was added at the time of AF/restructuring, the year is shown below): (i) No. Producers reached with TA on improved production practices: Original Target: 254,000; Actual: 408,263;160%; For females: Original Target: 90,000; Actual: 134,620; 150%; For males: Original Target: 164,000; Actual: 273643; 167%; (ii) No. of Farmer Learning and Resource Centers Established: (2015): Original Target: 250; Actual: 36; 14%; (iii) Hectares/has. Of orchards rehabilitated (directly & indirectly): Original Target: 106,442; Actual: 32,143; 30%; (iv) No. of Beneficiaries of Orchard Rehabilitation: Original Target: 200,000; Actual: 99,963; 50%; (v) No. of Has. Of New Orchards: Original Target: 19,000; Actual: 32,524; 171%; (vi) No. of Beneficiaries of New Orchards: Original Target: 70,200; Actual: 99,963; 142%; Page 9 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Natl Horticulture & Livestock Project (P143841) (vii) No. Has. Grape trellising and H/M density orchard systems supported: Original Target: 1,030; Actual: 801; 78%; (viii) No. of Beneficiaries of Grape & H/M density orch. Systems supported: Original Target: 3,600; Actual: 3,928; 109%; (ix) No. of units/plots of commercial, off-season vegetables and high value aromatic crops supported: Original Target: 5,700; Actual: 4,600; 81%; For females: Original Target: 5,000; Actual: 4,297; 86%; For males: Original Target:700; Actual: 303; 43%; (x) No. of new kitchen gardening schemes supported: Original Target: 90000; Actual: 143657;159%; (xi) No. of Has. new area brought under high value horticulture under results of improved irrigation technologies: Original Target: 2500; Actual: 2805; 112%; (xii) No. of beneficiaries new area under high value horticulture under results of improved irrigation technologies: Original Target: 10000; Actual: 11372; 114%; (xiii) No. of target farmers exposed to livestock practices & understand its benefits: Original Target:180000; Actual:180390;100%; For females: Original Target: 120000; Actual: 112645: 94%; For males: Original Target: 60000; Actual: 67745; 113% (xiv) DAIL staff visit to farmers: Visits as % of target: 66%; (xv) No. of MAIL/DAIL staff trained & exchange program within NHLP for training: Original Target: 5000; Actual: 9329; 186%; For females: Original Target: 2000; Actual: 2212: 111%; For males: Original Target: 3000; Actual: 7117: 237%; Key Outcomes (ICR, Annex 1) (i) % of target farmers adopt elements of horticulture technology packages promoted by NHLP: Original Target: 70; Revised (2015): 75; A: 61; % of Revised: 81%; For Females: Original Target: 75; Actual: 65; 87 %; For Males: Original Target: 65; Revised Target: 75; Actual: 56; % of Revised Target: 75%; (ii) % of target farmers adopt elements of livestock technology packages promoted by NHLP: Original Target: 65; Revised Target (2015): 70; Actual: 64; % of Revised: 91%; For Females: Original Target: 65; Revised Target (2015): 70; Actual: 71; % of Revised Target: 101%; For Males: Original Target: 65; Revised Target: 70; Actual: 56; % of Revised Target: 80%; In summary, the efficacy with which Objective 2 was achieved is rated Modest, because a large percentage of the output and outcome targets were not achieved (60% of output targets achieved and none of outcome targets achieved 100 %, per above assessment). At closing, 61% of project farmers adopted improved horticulture technology packages, which is 81% of the revised target. Improved livestock technology Page 10 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Natl Horticulture & Livestock Project (P143841) packages were adopted by 64% of the livestock producers, which is 91% of the revised target. More importantly, the total beneficiary outreach was well below than planned at appraisal, resulting in a lower absolute number of horticulture farmers and livestock producers than envisioned. The main reasons are due to various types of underlying institutional capacity and implementation weaknesses, including insufficient dissemination, limited training and extension support, weak research-extension system linkages, and various challenges associated with decentralized and in-country government department driven implementation modality (ICR, see paras. 35 – 40). It is noteworthy that the ICR concluded that “for those beneficiaries that the project did reach, the result was transformative” (ICR, para. 39). These results were further substantiated by the Project’s independent external evaluation study and independent third-party monitoring. Rating Modest OVERALL EFF TBL OBJ_TBL OVERALL EFFICACY Rationale The overall efficacy of the extent to which the original objectives (which remained the same), and their original and revised targets arising from the 1 restructuring (at the time of the AF in 2015) were achieved is rated Modest. Most of the targets for the main outputs and outcomes for both objectives were not met (only 14 of 39 output/outcome targets were met), although in most cases, the targets were close to being reached, and hence both objectives were rated Modest. It is noted that the AF in 2015 introduced some increases in several of the targets. It is also noteworthy that the project’s decrease in the absolute number of farmer beneficiaries from improved investment and marketing support for both horticulture and livestock components, and thus the number of beneficiaries for the adoption targets, fell below the target levels. At the same time, the external evaluation study confirmed important increases in production and productivity of target beneficiaries. Accordingly, the ICR and supporting studies also highlight the positive aspects of the project’s contributions to strengthening Government’s capacities for improved delivery of extension and marketing services, which provides a solid foundation for further improvements (see also ICR, paras. 39 and 40, Annex 7). Overall Efficacy Rating Primary Reason Modest Low achievement 5. Efficiency Overall, the project’s performance and results demonstrated an efficiency rating of Substantial, based on various evidenced-based tools and analyses applied and presented in the ICR (ICR, paras. 42 – 50, and Annex 4 presents further details). The methodology and results used to assess efficiency involved applying economic and financial analyses, with respect to the two components assessed at appraisal, as follows: Page 11 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Natl Horticulture & Livestock Project (P143841) (a) Economic analyses: the Bank’s ex-ante appraisal included a cost-benefit analysis (CBA) involving two components assessed at appraisal (generating an ex-ante economic rate of return/ERR of 24.9%, based on net incremental benefits): (i) the economic returns on establishing 8,000 has. of new orchards and on rehabilitating 6,000 has. of existing orchards (of various high value crops); and (ii) improved livestock and poultry production. For the ICR, an ex-post efficiency economic analysis was carried out for the same 2 components, based on: (i) information collected from the project’s documents, M&E database, the end-line survey, secondary data sources and from “multiple and independent sources”; (ii) inclusion of all relevant project costs, including administrative costs; (iii) use of crop and activity models, involving horticulture and livestock producers, as basis for generating incremental values; (iv) excluding some important non-quantifiable benefits (arising from export of fruits, from GHG emissions and training/capacity building); (v) applying constant 2020 values for prices and costs, applying a SCF at 0.9 and valuing family labor. This economic analysis generated an ERR of 28.1%, which compares favorably with the ex-ante estimate, and also reflects lower investment costs than envisioned at appraisal. The sensitivity analysis showed sensitivity to a delay in achieving the benefits (e.g., delay of 2 years resulted in an ERR of 22.6%). With the inclusion of climate co-benefits from the reduction of GHG emissions into the economic analysis, the ICR estimated the ERR of the project to be 29.2 percent; (b) Financial analyses: The ex-post farm modelling results showed that the project generated a positive impact on beneficiaries’ net income per household, varying according to the farm model and cropping pattern (e.g., farmer with new orchards: $546 - $3,646; farmer with existing orchards: $291 - $649; livestock and extension components, household annual incremental income of benefits: $103 for animal health; $486 from poultry). The evaluation study confirmed very positive feedback by beneficiary farmers on their tangible incremental income benefits; (c) Implementation Efficiency: Notwithstanding the positive economic and financial results of the project as cited above, the ICR correctly highlighted various findings which demonstrate poor implementation efficiency, including: severe start-up delays; unfilled project management positions for several years, worsened by high staff turnover; weak contractual management system; deficient M&E system and capacities; poor procurement capacities and performance. An in-depth fiduciary review by the Bank’s Integrity Department revealed significant weaknesses in project management and fiduciary project controls (e.g., weak asset management; deficient integrity in staff hiring; differences in financial reports and records; and shortcomings in internal and external audits). At the same time, compared to the previous project (HLP), NHLP demonstrated some efficiency gains in terms of a much lower implementation cost per beneficiary (i.e., $282 vs. $54); however, the ICR indicated the various challenges of the revised approach, which was not effective due to severe institutional service delivery weaknesses, and therefore, did not demonstrate improved efficiency and effectiveness. The ICR correctly recognizes that the PDO, scope, targets, and implementation modalities (relying entirely on Government entities at national and local levels) were overly ambitious, given the low capacities in a fragile, conflict and violence (FCV) environment. In summary, while the quantitative analysis showed substantial economic and financial efficiencies, the overall efficiency of the project is rated modest, due to serious operational, procurement, fiduciary (paras. 84 and 85) and administrative inefficiencies that negatively impacted project results. Efficiency Rating Modest Page 12 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Natl Horticulture & Livestock Project (P143841) a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 0 Appraisal  24.90  Not Applicable 0 ICR Estimate  28.10  Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome The overall outcome rating is based on the assessment of the 3 dimensions as summarized above, namely: (a) Substantial rating for Relevance of Objectives: This rating is based on the project’s initial and continued (throughout implementation) strong alignment of objectives and activities with: Government’s national and agrarian sector policies and strategies and contribution to key targets; and the Bank’s Country Partnership Strategy (interim and 2017 – 2020), and various analytical reports. Also, the country’s FCV challenges reinforced the relevance of the project; (b) Modest rating for Efficacy: Most of the targets for the main outputs and outcomes for both objectives were not met (only 14 of 39 output/outcome targets were met), although in most cases, the targets were close to being reached, and hence both objectives were rated Modest. It is noted that the AF in 2015 introduced some increases in several of the targets. It is noteworthy that the project’s decrease in the absolute number of farmer beneficiaries from improved investment and marketing support for both horticulture and livestock components, and thus the number of beneficiaries for the adoption targets, fell below the target levels. At the same time, the external evaluation study confirmed important increases in production, productivity, and family incomes of target beneficiaries; the ICR also highlights the shortcomings of the public sector service delivery system, which the project was endeavoring to address and strengthen, but in retrospect, was overly optimistic in relying mostly on the public delivery system, rather than a more realistic blend of providing targeted technical assistance and of relying on and strengthening of the public delivery system, including stronger research-extension linkages; and (c) Modest rating for Efficiency: The ICR ex-post quantitative analyses demonstrated favorable economic and financial efficiencies and indicators; and apparent improved efficiency gains compared to the previous phase of this project (e.g., lower cost per direct beneficiary, offset by the ineffectiveness of the public sector delivery system), while also taking into account the added challenges of implementation in a FCV environment, including some significant procurement and fiduciary challenges cited in the ICR (paras. 84 – 87). Based on the IEG guidelines (IEG, 2017, p. 46 – 48), a split evaluation was not undertaken, mainly because the PDOs remained the same, and the Additional Financing and the addition of two subcomponents (within existing components) enabled the project to increase its geographical scope, and to increase some of its outcome and intermediate result targets; accordingly, the assessment of the entire project is based on the revised outcomes, outcome targets and achievements (see above). a. Outcome Rating Page 13 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Natl Horticulture & Livestock Project (P143841) Moderately Unsatisfactory 7. Risk to Development Outcome Overall, there is substantial risk to sustaining the project’s development outcomes, especially in the light of the complex and increasing FCV environment throughout the country, as highlighted in the ICR (paras. 96/97): (a) Continuation/sustainability of the improved extension system: NHLP aimed at increasingly transferring financial and managerial authority and responsibilities to counterparts from MAIL, with the aim of ensuring the sustainability of improved knowledge transfer and extension systems to the relevant directorates of MAIL. However, the ICR states that this objective has “not happened”. The immediate risk is Government ensuring sufficient resources to maintain the extension system, especially in the current context of national government’s acute budget crisis. Moreover, the technical and financial support for MAIL is declining from donors, which is posing an additional challenge to maintaining the extension system built by NHLP. Among the NHLP trained extension staff, some have started doing their own farming, some have opened agribusiness shops, and some have moved to other relevant projects. (b) Worsening Security and Fragility: Given the worsening FCV conditions in Afghanistan, the ICR correctly highlights the serious risk of security and fragility affecting the sustainability of project’s outcomes. The ICR cites various concerns affecting the agricultural production and marketing environment, including: unstable and rising prices of key inputs and food commodities; the shortage of extension experts; declining trader and investor confidence. Conflict and insecurity are weakening the value chains for the development of the horticulture and livestock sector, as well as weakening various entities which could provide continued extension and infrastructural support to NHLP beneficiaries. Also, these challenges conditions are preventing development partners from getting re-engaged to support development projects/programs in all sectors. Due to the complex FCV conditions currently prevailing in Afghanistan, the Bank is discussing and deciding on the authorizing environment for future WBG engagements in Afghanistan. The IEG evaluator was informed by the Bank’s project team that “any proposals, plans and portfolio discussions, including addressing the sustainability challenges of this project, must await Bank management decisions on the authorizing environment for re-engagement by the Bank.” 8. Assessment of Bank Performance a. Quality-at-Entry The quality-at-entry of NHLP is rated Moderately Unsatisfactory, based on the following evidence (ICR, paras. 88 and 89): (i) While the project was well aligned with the Bank’s strategy for Afghanistan and with the Government’s developmental strategies and priorities, the project’s design/scope and associated targets could have Page 14 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Natl Horticulture & Livestock Project (P143841) been more realistic. The project design was building the results and lessons of the predecessor project (Horticulture and Livestock Project: HLP), with a huge scale-up from 11 districts to 100 districts, and with a change in the implementation modalities of relying almost entirely on relatively weak government and community institutions, and very limited technical support. While the re-orientation to getting Government entities taking the major role, in retrospect, the shift was overly optimistic, especially to establish and deliver farmer-centric approach to service delivery; (ii) The ICR correctly highlights the weaknesses in the project’s M&E arrangements, and the need for a more focused results framework, with fewer number of performance indicators (covering 25 indicators); (iii) While the project design risk assessment recognized the significant fiduciary and social risks, the mitigation measures were insufficient. The project and overall operating environment involved weak capacities and internal controls involving financial management and procurement. Given poor fiduciary performance, which was further detailed by an in-depth fiduciary review of the project, these assessments showed that the project’s mitigation measures (including periodic training and international procurement advisor) were insufficient. Quality-at-Entry Rating Moderately Unsatisfactory b. Quality of supervision The quality of supervision was Moderately Unsatisfactory, based on the candid assessment provided in the ICR (paras. 90 – 94), and the project’s independent evaluation, included the following aspects: (i) There were significant shortcomings in the Bank’s oversight of the project’s M&E system, especially to ensure timely and quality data collection, measurement and assessment of key outcomes, reflecting a weaker focus on assessing developmental impacts; (ii) NHLP failed to carry out a mid-term review, due to procurement and management neglect; this omission severely limited management capacity to follow an evidenced-based approach to project management and decision-making, which required addressing during implementation critical design and implementation issues; (iii) At closing, project management failed to prepare an agreed Project Completion Report (PCR); (iv) The Bank’s project team failed to pursue the Project’s management (and higher-level Government authorities) to carry out these assessments, and to ensure realistic project/M&E ratings (which were mostly Satisfactory, until mid-2019); (v) In late 2019, the Bank’s project team finally recognized the serious implementation weaknesses, including the weak M&E data quality, and then pursued various actions to strengthen the M&E system and to arrange the external project evaluation; however, time was too short to enable the project’s turnaround, which was further complicated by COVID conditions; Page 15 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Natl Horticulture & Livestock Project (P143841) (vi) The Bank's team raised various fiduciary concerns throughout implementation, involving inadequate internal controls and compliance mechanisms of the financial and procurement units, including misprocurement of 7 contracts. Some of the agreed corrective measures were implemented (para. 92); (vii) The ICR recognizes the AF operation in 2016 was “poorly appraised”, with the expanded project scope and adoption targets being overly ambitious, also in the light of the slow progress in overall disbursements; (viii) The project ratings did not realistically reflect the project’s poor performance. The PDO and Implementation Progress ratings were satisfactory throughout implementation (until 2020 and 2019, respectively). Also, notwithstanding the numerous fiduciary issues and weak M&E system, the ratings were overly optimistic. The ICR makes a strong statement: “The Bank team could have been more candid while assigning ratings to reflect weak implementation progress”. Quality of Supervision Rating Moderately Unsatisfactory Overall Bank Performance Rating Moderately Unsatisfactory 9. M&E Design, Implementation, & Utilization a. M&E Design The M&E system was designed to serve as a management tool to support decision making, learning, accountability, and results monitoring with the following two key components: (i) concurrent monitoring of physical and financial progress; and (ii) monitoring PDO and intermediate outcome indicators through periodic outcome surveys. However, as candidly indicated in the ICR (para. 78), the project’s M&E system exhibited three design shortcomings: (i) there were technical gaps in data collection and design, particularly given the absence of an external M&E agency and the proposed international TA for supporting a much needed M&E system; (ii) it did not specify regular thematic studies to conduct to aid improvement of project performance and results; and (iii) the initial RF was complex and the definition of some of the PDO indicators were ambiguous which had to be modified during the restructuring in 2016. Moreover, the scope of the indicators was too broad and did not fully capable of the attribution with project activities and results. b. M&E Implementation Overall implementation performance of the M&E system was mixed, including (ICR, para. 79): (i) The recruitment of all staff for the M&E unit posted in the regions and headquarters took long time, and the team was not fully onboard, even before the AF in 2016; Page 16 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Natl Horticulture & Livestock Project (P143841) (ii) the Management Information System (MIS) started collecting data from the second year of the project and continued throughout the life of the project; (iii) the M&E team prepared regular quarterly and annual progress reports, and submitted them to the Bank project team for its review and comments; (iv) in the early stage of the project, the quality and reliability of MIS data, particularly the double counting of beneficiaries, was not flagged by the Bank team, and thus, the project continued to rely on the unreliable MIS data to measure the project’s performance. Therefore, the project maintained its satisfactory outcome till 2018 when the quality issues in the M&E data has been flagged by the World Bank M&E specialist; (v) the Bank’s downgrading in ratings in recent ISMs was mainly because the project did not have quality data on tracking results; (vi) in the final stage of the project, the Bank’s Task Team provided active hands-on support to the NHLP M&E team to overcome the data challenges. Accordingly, there have been significant gains in creating a comprehensive beneficiary database, which is largely free of overlaps. c. M&E Utilization (i) The collected MIS data was utilized to prepare periodic reports and track project achievements; (ii) Output information crucially informed management and decision-making. Data on outcomes did inform the upscaling of the project in 2016 (AF). During the decision on AF in 2016, the PDO indicators were on track to achieving their targets; and (iii) However, the identification of data quality issues in 2018 (particularly overlapping in beneficiaries’ data in the MIS) made it difficult utilizing pre-AF M&E data for measuring reliably project performance. M&E Quality Rating Modest 10. Other Issues a. Safeguards The project was classified as environmental category “B” (partial assessment), and triggered various safeguards policies/plans: Pest Management (PMP); Environment and Safeguard Management Framework (ESMF); Environment and Safeguard Management Plan (ESMP); and Grievance Redressal Mechanism (GRM). By the project closing, the overall safeguards rating (and those for the first three triggered policies with ratings), was Moderately Satisfactory. Overall, the project adhered to these safeguards, as follows: (i) The project prepared and implemented relevant site specific ESMPs using the ESMF tools and the PMP plan for the subprojects; Page 17 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Natl Horticulture & Livestock Project (P143841) (ii) The NHLP project team developed and applied some good IPM approaches for managing pest and diseases in the relevant project areas; (iii) The NHLP team carried out an Environmental and Social Safeguards auditing with the help of an independent consultant. The auditing report highlighted some achievements and issues regarding environmental safeguards compliance; (iv) a comprehensive and effective GRM system was established and operational during implementation, with 367 grievances registered and resolved; and (v) The ICR highlights 6 safeguard issues which remained unresolved at closing (ICR, para. 82) involving shortcomings in: getting the needed data for various safeguard reports; safeguard and pest management plans; and safeguard staff in place. b. Fiduciary Compliance (i) Financial Management: There was mixed progress, involving both the Government’s Project and Bank teams, and which became more apparent through an in-depth fiduciary review report and the results of an external audit report, carried out toward the end of the project (2019 & 2020) (ICR, para.85-88): (a) The Government’s project team submitted Interim Unaudited Financial Reports (IUFR) and audited Financial Statements on time with acceptable quality; (b) the project team’s responsiveness to audit and supervision observations improved over time, and by project closure, there were no pending unaddressed audit observations; (c) the project’s disbursement rate remained satisfactory (with a cumulative disbursement of $176.25 million, against a commitment of US$176.33 million); (d) various FM issues raised during the Bank’s supervision missions (ICR, para. 86) were addressed by the Government through adequate corrective actions; (e) during the AF stage, it was agreed that the project’s FM system would be strengthened through various actions, involving: appointment of an international financial advisor to oversee FM and build project staff capacity; establishment and management of a proper system to record/manage farmers’ contributions; closely monitor fund flows; improving internal control system. However, these issues persisted until closing; (f) the in-depth fiduciary review identified US$1.96 million as ineligible expenditure, mainly due to non-procurement non-compliance, which has been refunded by the Government. Also, the Bank’s team exhibited some shortcomings in its FM role, including (ICR, para. 87):(a) its supervision missed or overlooked some red flags; (b) its supervision identified fiduciary issues and red flags, but it failed to take corrective actions; (c) it did not ensure that the MAIL appropriately followed up on and rectified identified issues. (d) In the highly risky operational environment in Afghanistan, the “Bank could have done more to ensure fiduciary compliance via stronger oversight as well as implementation support”. (ii) Procurement: The ICR highlights the project’s unsatisfactory procurement performance (para. 84): (a) long delays, integrity, inability to hire qualified procurement staff; (b) the project exhibited several concerns of potential fraud and corruption on the part of MAIL and NHLP, which were raised publicly and with the Bank, and which prompted the in-depth fiduciary review of the project (ref. above); (c) the Bank as the ARTF trustee with the fiduciary responsibility, initiated the in-depth review of NHLP in April 2019 to verify expenditure eligibility, economical and efficient use of funds and operations of the project control Page 18 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Natl Horticulture & Livestock Project (P143841) mechanisms. The project in-depth fiduciary review identified several deviations from agreed procurement procedures and arrangements and some of the observed deviations were declared as ineligible expenditures. As a result, at closing the Bank rated procurement as “Unsatisfactory”. c. Unintended impacts (Positive or Negative) Not Applicable d. Other The ICR highlights 5 other positive aspects contributed by the project: gender; institutional strengthening; mobilizing private sector financing; poverty reduction and shared prosperity; and prevention of COVID-19 pandemic. While recognizing some attribution challenges with respect to the precise role and contributions of this project to these other strategic benefits, the nature/scope of these “other” benefits are summarized below, based on evidence presented in the ICR (paras. 52 – 60). (i) Gender (para. 52 – 53): The project’s documents and external evaluation study provides strong evidence that the project contributed significantly to women’s increased employment activities, incomes and empowerment, helping to close the gender gap by female inclusion in target beneficiaries, as follows: (a) women comprised 45% of total project beneficiaries (264,192, exceeding the target of 220,400, involving horticulture/kitchen gardens, poultry, livestock and market access); (b) a good gender balance in and enabling access to various training activities; (c) building capacities of a large number of female extension workers and female para-vets; (d) provision of nutrition training to females farmers; and (e) providing a platform to enable women to come together, to learn from each other. The success stories of the project’s support to women beneficiaries were featured in various World Bank blogs and feature stories (ICR, Annex 6). (ii) Institutional Strengthening (para.54 - 56): Some progress was made on strengthening various institutions, but the objective to strengthen capacities at various levels, using multiple tools, was not “fully realized”. The main achievements included the following:(a) Extension staff from central and provincial governments (MAIL and DAIL, respectively) were directly involved in providing extension services and capacity building of farmers, promoting the adoption of improved agricultural and livestock practices; (b) the project promoted local ownership by institutionalizing Veterinary Field Units (VFUs) for enhanced effectiveness and improved provision of services to farmers; (c) the project strengthened farmer-based institutional capacity for horticultural development, including extension, marketing and integration into value chains, through the establishment and strengthening of common interest groups (CIGs), at the community level; the project was not able to use CIGs to achieve Producer Marketing Organizations; (d) MAIL was able to: take over and sustain most of the project’s livestock activities; administer the vaccine campaign and all sanitary mandates; supervise 25 provinces for the brucellosis control program and 26 districts for the sanitary mandate; (e) good progress toward institutionalizing the project’s extension component, through: institutionalizing Community Development Councils (CDCs); strengthening the role and capacities of lead farmers and VFUs, as key local actors and trainers to providing regular and improved extension services to Page 19 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Natl Horticulture & Livestock Project (P143841) larger numbers of farmers; and (f) demonstration plots/structures which farmers will use to learn improved horticultural practices. (iii) Mobilizing Private Sector Financing (paras. 57 and 58): While the ICR recognized that supporting the private sector is a VITAL requirement to sustaining many of the project’s benefits, the ICR reported that many of the key actions were not achieved, including: (a) the project did not pursue prior to project closure (in 2020) actions to link producers with processors and markets (e.g., the project failed to establish milk collection points/centers at clusters to enable farmers to sell surplus milk and other products); (b) the project did not support 4 peri-urban private sector dairy driven entrepreneurs; (c) the project did not provide support to 50 private sector dairy entrepreneurs; and (d) the project did not track/measure mobilizing private sector financing explicitly. At the same time, the ICR reported the following private sector progress/actions supported by the project: (a) project beneficiaries did put their own money for some of the activities implemented by the project, mobilizing their private capital to cost share in some investments; (b) several project-supported activities have contributed to strengthening the development of the private sector (e.g., PMOs were established following trainings, and linked to Farmer Service Center, to build linkages with private sector stakeholders; target producers were organized into producer groups; PMOs were trained in marketing skills); (c) the project’s support to horticulture producers catalyzed IFC investments; and (d) women beneficiaries were connected with McCormick, thereby enhancing their prices and market outlets. (iv) Poverty Reduction & Shared Prosperity (ICR, para. 59): The ICR highlights the project’s contributions: (a) The project’s focused on rural areas, where incidence of poverty is highest; the project’s coverage area targeted the provinces with high rural poverty rates; (b) The project’s selection criteria was pro-poor, with targeting and engaging women being explicit in both the initial project design and AF; (c) The project’s external evaluation study provides positive and tangible evidence regarding the “significant positive changes in beneficiaries’ livelihoods and incomes (e.g., the project’s horticultural activities generated significant increases in job creation; the project’s support of improved technologies and farmer practices, supported by improved extension, increased farmers’ productivity and incomes; the poultry activities increased incomes of poor and ultra-poor women; there were income increases arising from the project- supported bee keeping and fishpond beneficiaries). (v) COVID-19 Support (para. 60): After the breakout of COVID-19 pandemic, the project contributed to increased awareness and developed extension material and distributed it to lead farmers, extensions workers and veterinary field units, to inform the farmers to take precautionary measures. The project also provided equipment and testing kits for the COVID-19. 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment Moderately Moderately Outcome Unsatisfactory Unsatisfactory Moderately Moderately Bank Performance Unsatisfactory Unsatisfactory Quality of M&E Modest Modest Page 20 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Natl Horticulture & Livestock Project (P143841) Quality of ICR --- Substantial 12. Lessons The ICR presents nine main lessons arising from this project (ICR, paras. 98 – 103). The five most relevant lessons which include project-specific conclusions, and which can be applicable to other similar projects in Afghanistan and in other countries, especially in a FCV environment, are summarized below. (a) Lesson 1: Ensuring demand driven and participatory methods, and relying on strengthed public services are effective and sustainable in achieving the intended results, as long as they are complemented by adequate and effective capacity building approaches/activities for existing and relevant government and community organizations.The project made a drastic change from the previous Bank-financed HLP project by relying entirely on existing government and community organizations to implement the investment packages, with the active engagement of community organizations (but with some limited external technical support). The very weak capacities of Government and communities in a very challenging FCV environment, led to the project’s moderately unsatisfactory performance. Thus, the project experience highlighted the importance of adopting a more realistic and gradual reduction of external TA support, phased/linked to the progress in building local capacities; (b) Lesson 2: Promoting active and meaningful participation of women in project design and implementation, coupled with adequate consideration of local social and cultural norms, are important parameters for the sound design and effective implementation of agricultural- based projects in low- income countries. It was noteworthy that the project targeted a large proportion of beneficiaries to be women (40% of total beneficiaries) through appropriate and women- specific horticultural and livestock activities (e.g., kitchen gardening, micro greenhouses, poultry). The project also provided training support to women beneficiaries, with special attention to the selection of venue, transportation, and facilitation; (c) Lesson 3: Sound and functional M&E system from early stages of implementation helps with identifying challenges in project design and implementation and with ensuring realistic targets and sound design for the AF phase. The ICR showed that NHLP’s weak M&E system was a key factor behind many of the project’s shortfalls and failures to address key challenges (e.g., overly ambitious project scope/targets, which were enabled by poor quality data, contributing to false expectations and to poor decisions involving unrealistic scale-up of targets during the AF phase in 2016). These unreliable data quality and resulting M&E weaknesses contributed to poor decisions, resulting in unrealistic scale-up and targets during the project’s restructuring and AF stage in 2016. Accordingly, the experience of this project highlights the importance of ensuring from the design stage a functional and reliable M&E system of the project to help guide key design and subsequent restructuring/AF decisions; (d) Lesson 4: Promoting sound design and implementation of the project’s fiduciary arrangements, especially in a challenging FCV environment, is vital to help ensure efficiency, effectiveness and accountability of project results. The project’s in-depth fiduciary review highlighted various challenges and corresponding specific lessons which underscore the importance of ensuring these specific fiduciary aspects are addressed adequately during the project’s initial Page 21 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Natl Horticulture & Livestock Project (P143841) design stage and throughout implementation. The nature and scope of these specific issues are highlighted in the ICR (para. 103). This ICRR focuses on highlighting 3 specific lessons which can be used as a check-list for the design stage of fiduciary aspects of similar projects in other countries, contextualized to the specific country/project conditions. These lessons include: (i) importance of ensuring consolidation and other appropriate mechanisms (e.g., voucher system) of procurement packages, where feasible, as it would reduce the number of contracts under post-procurement review, which, if not reduced, could increase the governance risks; (ii) importance of ensuring robust monitoring, including third-party monitoring and added fiduciary oversight by the Bank’s project team to help mitigate risks; (iii) importance of the Bank project team providing adequate and effective oversight on key aspects involving human resource management, staff hiring, payroll, financial reporting, without micro-managing; and (e) Lesson 5: With COVID or other crisis conditions, this project highlighted the importance of the Bank developing an “operational contingent instrument and mechanism” for fast crisis response, without disrupting the implementation of on-going projects. With the COVID epidemic, the Bank’s on-going project portfolio was adjusted to make funds available and deployed to support COVID emergency response, especially with respect to emergency food security response. There was an absence of existing Bank financing instruments to address this COVID crisis. Therefore, this re-allocation of funds from existing projects, including NHLP, resulted in the following adverse effects: “led to significant disruptions of the Bank portfolio of the projects, increased delays in the implementation of NHLP, and the agriculture portfolio was largely reduced” (ICR, para. 104). 13. Assessment Recommended? No 14. Comments on Quality of ICR Overall, the quality of the ICR is Substantial. The ICR is well written, consistent with the IEG guidelines, analytical and results-focused, concise, provides relevant lessons, supported by adequate evidence to justify the various assessments, and candid, especially revealing Government and Bank weaknesses during project design, AF and implementation phases. The most notable shortcoming of the ICR is the limited clarity regarding the Bank’s strategy to enhancing the project’s prospects for sustaining many of the project’s notable achievements in: (a) establishing and strengthening improved farmer-centric systems for promoting improved horticultural and livestock production and productivity; (b) introducing strategic initiatives supported through the AF, especially regarding the promotion of production-market linkages, and an expanded role of an incipient private sector. While the ICR’s section on “Risk to Development Outcome” (paras. 96 and 97) recognizes the serious current threats to sustain the current extension system and the country’s current security and fragility “crisis”, this section in the ICR should have been expanded, to address more explicitly the above cited serious threats to sustaining most of the project’s achievements, including a possible framework of possible follow up and realistic “interim” proposals. At the same time, it is recognized that the project is operating in a very complex FCV Page 22 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Natl Horticulture & Livestock Project (P143841) environment, which is affecting all of the Bank’s portfolio (and no doubt, other development partners). Accordingly, the additional clarifications and mitigation measures would need to address various types of project achievements, involving the relevant entities, and to highlight the Bank’s broader/country-level and portfolio-level engagement, especially considering the current very challenging FCV environment; the Bank's team is awaiting Bank management decisions on the “authorizing environment” to enable the Bank’s re- engagement in Afghanistan (see Section 7). a. Quality of ICR Rating Substantial Page 23 of 23