Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review NI Caribbean Coast Food Security Project (P148809) Report Number: ICRR0022189 1. Project Data Project ID Project Name P148809 NI Caribbean Coast Food Security Project Country Practice Area(Lead) Nicaragua Agriculture and Food L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) TF-18703 30-Dec-2019 33,830,640.38 Bank Approval Date Closing Date (Actual) 20-Feb-2015 30-Dec-2019 IBRD/IDA (USD) Grants (USD) Original Commitment 33,900,000.00 33,900,000.00 Revised Commitment 33,830,640.38 33,830,640.38 Actual 33,830,640.38 33,830,640.38 Prepared by Reviewed by ICR Review Coordinator Group Richard Anson J. W. van Holst Christopher David Nelson IEGSD (Unit 4) Pellekaan 2. Project Objectives and Components DEVOBJ_TBL a. Objectives The project development objective (PDO), as stated in the Grant Agreement (GA, 2015), for this Nicaragua Caribbean Coast Food Security Project, was "to enhance food and nutritional security in selected communities of the Caribbean Coast of the Recipient." For purposes of assessing the extent which the PDO was achieved in Section 4, this review will parse the PDO into two objectives, namely: Page 1 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review NI Caribbean Coast Food Security Project (P148809) Objective 1: to enhance food security in selected communities of Caribbean Coast of the Recipient; Objective 2: to enhance nutritional security in selected communities of Caribbean Coast of the Recipient. According to the internationally recognized definition of food security, it means “access to sufficient food for an active and healthy life”. Nutritional security means access to food that meets dietary needs. The project’s results framework lacked performance indicators to measure progress toward “food security” and/or “nutritional security”. Rather, the results framework focused on the extent to which farmers used improved technologies and increased their productivity. These latter measures were the basis for food and nutritional security indicators for this project funded under the aegis of the Global Agriculture and Food Security Program (GAFSP). b. Were the project objectives/key associated outcome targets revised during implementation? No c. Will a split evaluation be undertaken? No d. Components Component 1: Innovation Development Plans for Strengthening Natural and Non-Natural Resource- Based Productive and Marketing Capacity (Original allocation: US$31.85 million; Actual: US$33.49 million). This component aimed to boost the productive and marketing capacities of targeted farmers and rural enterprises by financing the participatory design and implementation (by formal and informal groups) of Innovation Development Plans (IDPs). There were 5 supporting and complementary subcomponents: 1.1: communication campaigns; 1.2: training; 1.3: various types of assessments; 1.4: strengthening organizational and business capacities of targeted producer groups, and fostering smallholder linkages to the market; 1.5: provision of technical assistance for food sanitary and related services. The component also provided financing to cover IDP investments for natural and non-natural resource-based activities. Four types of IDPs were supported: (a) family agriculture; (b) artisanal fisheries; (c) agricultural/agro-industrial ventures; and (d) non-agricultural microenterprises (ICR, para. 15, and Annex 6); Component 2: Strengthening service provision for sustainable production, food security, and nutrition (Original allocation: US$6.42 million; Actual: US$4.84 million). This component financed the strengthening of sector capacity to provide services to IDP beneficiaries in two areas/subcomponents: 2.1: technology generation/validation and transfer to improve productivity and quality of agricultural production by enhancing the institutional capacity of the Nicaraguan Institute of Agricultural Technology (INTA); and 2.2: nutritional education and communication, and nutrition-smart agriculture activities and food handling practices, targeting key subgroups (e.g., pregnant and breastfeeding women and children under five years old) (ICR, para. 16); Component 3: Project management, monitoring and evaluation (Original allocation: US$3.73 million; Actual: US$5.47 million). This component funded project management capacity and project-related activities of the implementing Ministry (MEFCCA), including: 3.1: incremental and operating costs; 3.2: essential equipment and goods; 3.3: comprehensive monitoring and evaluation system; 3.4: environmental and social safeguards, 3.5: financial management and procurement project-related activities. Page 2 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review NI Caribbean Coast Food Security Project (P148809) Revised Allocations Among Components: While the components were not formally revised, there were several modest changes in the allocation of funds among components, including: (i) increase of funds under component 1, to finance operational costs supporting IDP implementation; (ii) decreased funds under component 2, which were transferred to component 3; and (iii) increased funds under component 3. The PDO and outcome targets were not revised. e. Comments on Project Cost, Financing, Borrower Contribution, and Dates (i) Project Costs: The total project cost at approval was US$42.0 million. The actual cost at project closing was US$43.68 million. The small difference (+4%) was due to: the increased contribution by beneficiaries (+ $1.58 million), attributed to their interest in securing Innovation Development Plan (IDP) support; and a small increase in Borrower counterpart contribution ($0.1 million). (ii) Financing: At approval, the GAFSP grant (TF-18703) was US$33.9 million, with a Government counterpart financing commitment of US$5.8 million and a local beneficiary commitment of US$2.3 million, totaling US$42.0 million. By the end of the project, total financing was US$43.68 million, due to an increased contribution of US$1.58 million from local beneficiaries; (iii) Borrower/Recipient Contribution: At approval, the Borrower counterpart contribution was supposed to have been US$5.84 million, with a final contribution of US$5.94 million. (iv) Dates: The project was approved on February 20, 2015, became effective on February 23, 2015. A mid-term review was carried out in September 2018. The original and actual closing dates were the same: December 30, 2019. 3. Relevance of Objectives Rationale The project objectives were to "to enhance food and nutritional security in selected communities of the Caribbean Coast of the Recipient." The emphasis on “selected communities” with these needs included indigenous, afro-descendants and Mestizo communities within the Caribbean Coast (ICR, para. 6, Annex 6, Table 2). The two sub-objectives (food security and nutritional security) were complementary, and their supporting components demonstrated high relevance to the basic needs of farmers and consumers, by addressing strategic elements of the Government's national and sectoral/thematic policies and strategies. These two sub-objectives also supported the implementation of the Bank's country strategies for Nicaragua over two periods (Country Partnership Framework/CPF for FY13-17, and the on-going CPF, FY18 - FY22). This relevance is evidenced in several key Government and Bank documents, including the following (see PAD, Annex 6; ICR, paras. 6 and 25): (a) Government National Food and Nutritional Security Policy (2009): This policy outlines the challenges and objectives, targets and operational strategies to address both food and nutritional security in an integrated manner, supported by targeting strategies in terms of priority geographical areas and vulnerable groups. In this project, the vulnerable groups were defined in the PAD as including "male and female small- Page 3 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review NI Caribbean Coast Food Security Project (P148809) and medium-size producers (at least 20 percent are women), male and female rural laborers, young adults, and artisanal fishers with no or limited assets and equipment"; (b) Government National Plan for Human Development/NPHD (for 2 separate periods: 2012 - 2016 and 2018 - 2021): The PDO had strong linkages to the NPHD, which emphasized the importance of access to safe and nutritious food for poor families (2012-2016); and Strategy No. 6 (2018-2021) regarding integral development of the Caribbean Coast; with Strategy No. 10: strengthening family, community and cooperative economies, including food and nutritional security, as part of the broader productive sector strategy; and with the component Caribbean Coast Development Strategy (c) Government National Policy for Mitigation and Adaptation to Climate Change (2019): This policy consolidates the key elements for addressing climate change, including geographical and thematic priority areas and strategic interventions (building on previous guiding documents, considered during project design); (d) The Bank's previous CPF (FY13-17) and the on-going CPF (FY18-22) that pay special attention to various themes which are addressed by the project, including: (i) raising incomes by sustainably improving agricultural productivity, competitiveness and diversification (CPF, FY13-17, para. 79); (ii) disadvantaged groups and lagging territories within Nicaragua which have not participated fully in the benefits of growth; prioritizes interventions which target vulnerable population groups (e.g., youth, women, subsistence farmers, indigenous and Afro-descendent communities, with a special focus on rural areas in the Caribbean and Central regions; (iii) current CPF Pillar 1: investing in human capital, especially with respect to disadvantaged groups; (iii) objective 4: improved business productivity and financial inclusion and rural small and medium enterprises and female entrepreneurs; (iv) increasing agricultural/livestock productivity among targeted beneficiaries in the Caribbean and Dry Corridor regions. While the overall thrust of the project’s objective is clear and highly relevant to the Government and Bank development strategies, there were two elements that weakened the full relevance and clarity of the objectives in the ICR. They were: (a) the ICR used various definitions of and assumptions for achieving food and nutrition security, which undermined the clarity of the objectives; and (b) there was were no metrics in the project’s results framework to indicate the expected level of enhanced food security and an inadequate dietary diversity score (DDS) to measure nutrition security which undermined the objectives' relevance; The project design was based on the assumption that increased food production by the target population would lead to improved food and nutrition security but how much food production would satisfy that objective was also unclear. Accordingly, the relevance of objectives is rated “substantial” rather than "high". Rating Relevance TBL Rating Substantial 4. Achievement of Objectives (Efficacy) EFFICACY_TBL Page 4 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review NI Caribbean Coast Food Security Project (P148809) OBJECTIVE 1 Objective To enhance food security in selected communities of Carribean Coast of the Recipient. Rationale Theory of Change (ToC): While the project's design included a results framework (RF), the PAD did not develop a ToC because it was not required at the time the PAD was written. The ICR reconstructed a ToC that was consistent with the GAFSP project's "premise that agricultural practices, technology, education and communication will lead to increased productivity of nutrition smart crops and products, which will in turn lead to improvement in food security and nutritional status of beneficiaries" (paras. 9 - 11, and illustrated in Figure 1). However, the ToC does not explain how the two broad and inter-related objectives of food and nutrition security would be achieved. The project supported the generation and dissemination of improved and appropriate technologies relevant to various innovation development plans (IDPs) funded by the project, intended to increase crop and livestock productivity, coupled with improved dissemination strategy/activities to help ensure expanded adoption of the improved technologies by the targeted farmers (including women-headed households), enabled by improved institutional coordination and technology delivery systems. As stated above, an important implicit project design assumption in the ToC was that increased food production through higher productivity would contribute directly to improved food and nutrition security of producer and consumer households. However, the literature on food security concludes clearly that people attain food security when they have access to food, which can be achieved through increased own food production, higher incomes, and trade or barter. The text of the ICR, though not the ToC, acknowledges that increased supply of food does not automatically mean expanded access to food (paras. 9 and 11), while nevertheless also stating that the "project objectives of enhanced food and nutritional security were fully achieved and directly attributable to the activities supported by the project." The ICR goes on to state in the same paragraph that "The selected communities of the Caribbean Coast of Nicaragua now have improved access to food with improved nutritional value" (para. 39). Evidence on outputs and outcomes towards the achievement of Objective 1 is found in the ICR, paras. 26 - 35, and Annexes 1, 6 and 8. Outputs: (1) Producers' organizations strengthened: Innovative Development Plans (IDPs) created 54 "solidarity groups", which contributed to more effective and sustainable IDPs; (2) Network of community promoters was fostered, which contributed to enhanced IDP implementation; (3) Fostering and improving processes of value addition for the different types of IDPs; (4) Increased availability of certified seeds and genetic material, and "IDP support for agricultural/agro- industrial ventures strengthened the quality and volume of seeds produced along with their appropriate treatment, storage solutions and marketing. This greatly benefited the two Technological Development Centers (TDC) managed by INTA" (ICR, para 38), Outcomes: Page 5 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review NI Caribbean Coast Food Security Project (P148809) (1) Number of beneficiaries (or Innovation Development Plan/IDP target beneficiaries) who have adopted an improved agricultural technology promoted by the project: target: 8,000; actual: 10,675; +33%; (2) Number of female IDP clients who have adopted an improved agricultural technology promoted by the project: target: 1,600; actual: 5,188; +224%; (3) Average percentage increase in agricultural/livestock productivity among all direct IDP beneficiaries: target: 10%; actual: 78%; +680%; (4) Number of families who implemented IDPs: target 14,000; actual: 14,286; +2%; families implementing IDPs led by women: target 2,800; actual: 6,994; +149%; (5) Percentage increase of production volume (agricultural products and fisheries): target: 15; actual: 123; +720%; (6) Percentage increase in market access (through establishing product marketing partnerships/agreements): target: 40; actual: 71; +77%; ditto, by women-headed households: target: 40; actual: 71; +77%; (7) Percentage of volume of farm produce under improved post-harvest management: target: 30; actual: 66; +120%; (8) Diversified Production as Percentage of Traditional/Subsistence Production: target: 15; actual: 45.5; +203%; (9) Improved technologies adopted by and applied to artisanal fishery IDPs, which contributed to increased fish catches and marketable surpluses. The ICR correctly defined food security as “having access to sufficient quantity of food” (ICR, para 26), but the results of this project provide mostly evidence that the quantity food produced in the Caribbean Coastal region had increased. There is no explicit evidence in the ICR that this project enabled consumers in the Caribbean Coastal target communities (mostly poor small-scale famers, and many landless with food deficits) through own production and/or trade, to achieve enhanced food security, namely: to have “access to a sufficient quantity of food”. Based on this definition of food security (also endorsed in paragraph 9 in the ICR), this project did not fully achieve Objective 1, despite having achieved substantially increased productivity and production of food crops, which the GAFSP approach asserts would lead to improvement in food security. In an exchange with IEG during preparation of this ICRR the Bank's task team provided considerable evidence of increases in food and agricultural production, arguing that these increases translated into increased availability of food for subsistence farmers and other target groups. However, neither the ICR nor the project task team responses provided any clear evidence of enhanced food security among beneficiaries at project closure. There was, moreover, no explicit indicator defined in the PAD or during implementation to measure food security, and the project's M&E system made no attempt to assess enhancements in food security among the diverse beneficiaries, some of whom were landless laborers and highly dependent on wage income for achieving their food security. Recognizing the ICR’s own definition of food security as "having access to sufficient quantity of food" (para 9), this review acknowledges that the core formal indicator measuring the enhancement of food security by the Page 6 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review NI Caribbean Coast Food Security Project (P148809) project was based on the GAFSP proposition that increased food production will improve food security. Based on credible evidence in the ICR that agricultural productivity and food production in the Caribbean Coast area increased substantially due to this project, this review has therefore rated the efficacy with which the project achieved Objective 1 as substantial with moderate shortcomings because of the as yet unverified impact of increased food production on enhancing food security. Rating Substantial OBJECTIVE 2 Objective To enhance nutritional security in selected communities of Carribean Coast of the Recipient. Rationale Theory of Change: As stated above, the ICR presented one integrated ToC to cover both objectives. The ToC shows how prioritized activities, which were driven by the 4 types of Innovation Development Plans (IDPs), together with improved support services, generated strategic outputs, which in turn, generated the outcomes that contributed to enhanced nutritional security in the targeted communities and for beneficiary farm households. The TOC does not, however, present a precise pathway on how nutritional security (defined in paragraph 26 of the ICR as “having access to food that meets dietary needs”) for the different types of project beneficiary families would be achieved, except through "nutrition sensitive production practices". The achievement of nutritional security was to be measured by increases in the dietary diversity score (DDS) and the observations that IDPs adopted nutrition sensitive agricultural production practices and nutrition-related training (ICR, para. 37). Supporting evidence for the achievement of Objective 2 is found in the ICR, paras. 26 -38, Annexes 1, 6 and 8. The following shows the most relevant indicators of improved nutrition in terms of outputs and outcomes: Outputs These are the same as for Objective 1 Outcomes (1) The percentage increase in dietary diversity score (DDS) for women and children of direct beneficiary families: target: 80; actual: 91; +13.7%. Combined technologies leading to increased productivity, diversification and nutrition quality of food produced, together with improved technical assistance which devoted explicit attention to promoting improved nutrition in food production and consumption practices at the household level, contributed to the improved DDS. However, the DDS is only a partial outcome indicator of improved nutritional security, namely "food that meets dietary needs" (ICR, para 26). A comprehensive indicator of nutritional security would also have the same access requirements as food security (namely access to foods with high nutritional value), as well as education on dietary needs. Page 7 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review NI Caribbean Coast Food Security Project (P148809) (2) Percentage of diversified production (involving increased varieties of high nutritional value in crops varieties, Annex 6): target: 15; actual: 45; +200%. (3) Percentage of volume of farm produce under improved post-harvest management (whereby results were reported by the Bank project team to have promoted enhanced nutrition security at household level): target: 30; actual: 66; +120%; (4) Percentage of IDPs adopting nutrition sensitive practices: target: 50; actual: 100; +100%; (in terms of selection of crops and food preparation) Overall there is some indirect evidence of the enabling conditions for “enhanced nutrition security”, even though the extent of the enhancement probably varied considerably between households because of the wide income distribution in the population in the poor coastal project areas. This variability was not assessed in the ICR or in the Impact Evaluation Study. The indicators mentioned above involve relevant aspects of nutrition measures, but their robustness and depth are weak in terms of providing precise measures of “enhanced nutrition security”. On the basis of the weak evidence in the ICR on whether the project enhanced the dietary needs of beneficiaries, this review concludes that the extent to which Objective 2 (enhanced nutrition security) was achieved is modest. Rating Modest OVERALL EFF TBL OBJ_TBL OVERALL EFFICACY Rationale Overall Efficacy is rated substantial, but with moderate shortcomings, based on the following factors: Enhanced food security was substantially achieved with moderate shortcomings, and enhanced nutritional security was only modestly achieved, A summary of the justification for these ratings follows. (a) Enhanced Food Security. This review concluded that the GAFSP assertion that increased production of nutrition-smart crops and products will in turn lead to improvement in “food security” was not tenable since the objective of enhanced food security is based on adequate access to food and not based on increased production of food. Nevertheless, increases in agricultural productivity and corresponding higher food outputs and outcomes were arguably attributable to the activities supported by the project, and over time, they could contribute to enhanced food security. As shown above, most of the targets of the GAFSP performance/outcome indicators for both objectives where exceeded; 13 of the 13 outcome indicators involving both objectives exceeded their end-targets. Nevertheless, based on the ICR’s own definition of food security as "having access to sufficient quantity of food" (para 9), but acknowledging that the core indicator measuring the enhancement of food security by the project was based on the GAFSP proposition that increased production will achieve food security, this review rated the efficacy with which the project achieved Page 8 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review NI Caribbean Coast Food Security Project (P148809) Objective 1 as "substantial with moderate shortcomings" because of the as yet unverified impact of increased food production on enhancing food security. (b) Enhanced Nutritional Security. There is evidence in the ICR (Annex 4) that the targeted food producers (the main beneficiary population sub-groups) in the Caribbean Coast of Nicaragua increased their food production productivity and diversification of agricultural production, with improved market outlets, and incomes were estimated to have increased. It was presumed that these producers increased their access to food with improved nutritional value. There is evidence in the ICR that the dietary diversity score (DDS) for women and children of direct beneficiary families in the project area improved by almost 14%, but the DDS is not an adequate indicator of nutritional security since it only measures the balance among food groups. Although, there is some evidence of nutritional improvement in communities, in general the evidence that nutritional security was enhanced is weak. This review has therefore rated the efficacy with which Objective 2 was achieved as "modest". Overall Efficacy Rating Substantial 5. Efficiency Overall, the project performance and results demonstrated an efficiency rating of substantial, based on various evidenced-based elements and tools applied. The detailed assessment presented in the ICR shows that the project achieved and surpassed its financial and economic rate of return targets, together with demonstrating other positive measures of efficiency. This evidence of the economic and financial analysis (EFA), which followed a similar methodology used at appraisal, together with other efficiency indicators, is summarized below (based on the ICR, paras. 40- 45, Annexes 4 & 8, which drew on the results of the 4 impact evaluation studies). (1) Economic Rate of Return: The Economic Rate of Return (ERR) at appraisal was estimated to be 17%; at project closing, the ICR estimated an ERR of 22.6% (para. 40); (2) Increased Farm Incomes and Employment Levels: Based on representative farm models, using data from farms in the project’s target area, family incomes were expected to grow between 1.5 and 6 times, with significant increases in family employment, by enhancing farm activities and reducing farmers' dependence on lower paying and erratic off-farm employment. Similar farm models were used for the ICR, which drew on the findings from the impact evaluation studies, to illustrate the benefits to the 11,713 beneficiary farmers. The results showed increases in net family income per year of two to six-fold (from an average of $320 to $1,300 per year), and increases in farm employment. Similarly, the fishery models, representing 1,500 direct beneficiaries, showed three-fold income increases. Other models were used in the ICR to illustrate increases in agro- industrial IDPs, showing financial rates of return exceeding 100%, and ERRs of 8-34% (paras. 41 – 42); Page 9 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review NI Caribbean Coast Food Security Project (P148809) (3) Increased Value of Production, Employment and Returns to Labor: the total value of production generated by the project beneficiaries grew about 3.8 times, family labor increased by 73%, and the average return per day of work increased by 32.5% (ICR, para. 43); (4) Sensitivity Analysis: The ICR demonstrated a robust sensitivity to changes in farmer adoption rates, drops in percentage of farmers not sustaining their benefits, and decreases in farmgate product prices (e.g., if market prices agricultural products were to drop by 15%, the overall ERR would not decline to below 10%); and (5) Other efficiency performance measures: The ICR concluded that project design and implementation arrangements were adequate and flexible, and implementation performance was efficient, as reflected in the performance of procurement, staff continuity, no cost overruns, good disbursement performance and no need to extend the project closing date. Efficiency Rating Substantial a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 95.00 Appraisal  17.00  Not Applicable 95.00 ICR Estimate  22.60  Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome Based on the project's "substantial" relevance, "substantial" efficacy with moderate shortcomings and "substantial" efficiency, this review concludes that there were moderate shortcomings in the achievement of its objectives. Therefore, the project’s overall outcome is rated "Moderately Satisfactory". The rationales for the ratings of the three core elements mentioned above are as follows: (1) Substantial relevance of the PDO. This was based on the strong alignment of the PDO (enhanced of food and nutritional security) with the Government's national development and sectoral/thematic policies and strategies, and with the Bank's country partnership strategies (over 2 periods), including the targeting of low income communities and farmers located in the Caribbean Coast areas. As noted in Section 3 of this review, the relevance of objectives was undermined by the lack of an indicator of enhanced food security despite an accurate recognition of its meaning in the ICR, and the weak indicator measuring enhanced nutrition security, while recognizing the expected contribution of increased food production productivity to the project's beneficiaries Page 10 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review NI Caribbean Coast Food Security Project (P148809) (2) Substantial overall efficacy with moderate shortcomings. The GAFSP project development indicator for increased food productivity and production, as well as several other performance indicators, were clearly achieved On the other hand there was no evidence that increased food production had achieved the expected enhanced food security among the project beneficiaries. The achievement of food security was therefore rated substantial with moderate shortcomings because of the as yet unverified expected impact of increased food production on the enhanced food security of beneficiaries. The achievement of enhanced nutritional security was rated modest because of the weak measures to assess whether enhanced nutrition had been achieved or could be expected to be achieved among the beneficiaries. Hence, this review has rated overall efficacy was rated substantial with moderate shortcomings. (3) Substantial efficiency This rating was based on the positive estimated economic and financial outcomes for farmers, and the overall efficient implementation performance of the project. a. Outcome Rating Moderately Satisfactory 7. Risk to Development Outcome There is moderate risk to sustaining the outcomes and contribution to the project’s eventual impacts. The ICR presents a candid assessment of the risks, together with the specific and verifiable mitigation measures, as summarized below (paras. 78 and 79). (1) Risk of beneficiaries not sustaining their improved production/food and nutrition security practices: The project included design and implementation mitigation measures, to be implemented by a well-established Government ministry/MEFCCA, including: (a) strengthening of the community-based promoters' network (which also are promoting other community development initiatives, hence, part of an integrated approach); (b) organizational and technical strengthening of the beneficiaries, including their producer cooperatives/associations; (c) promotion of the beneficiaries' alliance with private sector stakeholders, along the value chain, although limited in scope and depth, due to the subsistence dominance of the economy and the limited role of private sector in value chain development in the Caribbean Coast; (d) improving post- harvest practices, thereby creating added incentives to expand and sustain the changes; (e) actively promoting coordination among the various institutions that support producers in the Caribbean Coast through various multi-sectoral programs/projects/mechanisms (e.g., including the National System of Production, Consumption and Commerce/SPCC; and MEFCCA); (f) strengthening various sector institutions supporting agricultural development in the Caribbean Coast region; (g) improving technological packages (e.g., seeds and vegetative materials) suited and adapted to the agro-climatic conditions of the Coast; and (h) providing improved sanitary and phytosanitary services to farmers in the Coast; (2) Risk of an absence of or weak continued technical and financial support: Through MEFCCA and other relevant public sector agencies (at central and regional levels), there are various parallel and additional initiatives which are institutionalized, most also extending beyond the project life, which are expected to help consolidate, expand and sustain the project benefits, including: (a) strengthening and restructuring of beneficiary producer and marketing entities; (b) additional targeted financial and technical assistance to IDP- supported agro-industrial ventures and microenterprises, to ensure they meet competitive business standards and that they can access finance to expand and sustain their operations; (c) continued provision of appropriate agricultural technologies (by INTA and other entities), including: provision of extension and Page 11 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review NI Caribbean Coast Food Security Project (P148809) phytosanitary services; access to existing seed banks; research and innovation farms, germplasm banks; capacity building on nutrition, targeting women; and strengthening key entities located in the Caribbean coast. The sustainability of the above support is vital to sustained improvements in food and nutrition security of a very poor target population. At the same time, the IEG's interview with the Bank's project team and their response revealed some degree of uncertainty regarding the project's sustainability, especially with respect to clear plans to expand the role of the private sector. 8. Assessment of Bank Performance a. Quality-at-Entry This project was built on successful experiences and lessons from previous World Bank-supported investments in the Caribbean Coast, particularly the series of projects supporting agricultural technology and land administration. The project was expected to establish synergies with other World Bank operations, including (PAD, para. 10): Improving Community and Family Health Care Services Project; Sustainable Rural Water Supply and Sanitation Project; Rural Roads Infrastructure Improvement Project; and the Rural Telecom Project. The project’s quality at entry is rated Moderately Satisfactory by this review, based on the following evidence (ICR, para. 75): (i) the Bank's project team played a key role in formulating the GAFSP proposal, which was strongly aligned with the country's strategic priorities (both Government’s national and agrarian sector strategies and the Bank's CFP), through the design of a relevant and timely operation, with strong targeting of vulnerable communities located in 15 municipalities in the Caribbean Coast; (ii) Project design to achieving the PDO objectives focused on enhancing food and nutrition security through an integrated approach, leveraged by the innovation development plans (IDP) support, which also created incentives for the participants to take a central role. Also, the integrated approach included explicit and coherent outcomes, outputs and activities which helped generate the and contribute to specific and verifiable results/outcomes involving food and nutritional security, although limited in the measurement and achievement of efficacy of improved access to food, with respect to achieving objective 1 (see above); (iii) Project design devised appropriate and sound institutional arrangements and roles, given the challenging environment, and weak capacities; however, the explicit project measures to strengthen institutional capacities in the Caribbean Coast to help sustain improved services for improved food and nutritional security were implicit (e.g., Bank’s task team provided verbal communication to IEG evaluator about strengthening various local services entities); (iv) Bank technical inputs and preparation processes were adequate and timely, with technical support by experienced team members, also familiar with country conditions, including key role of the country office; (v) environmental and social aspects/safeguards were adequately assessed, including an active role of indigenous community leaders, and the preparation of a quality Indigenous Peoples Planning Framework (IPPF), which guided the preparation of 33 IPPs during implementation; (vi) the gender aspects, including the project's strategy for ensuring women received adequate IDP support and an active role in the IDP process and activities, were a key feature of project design; (vii) Assessment of risks, together with appropriate and sound mitigation measures, were sound and proven to be relevant. Page 12 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review NI Caribbean Coast Food Security Project (P148809) Notwithstanding the above positive design features, the readiness to implement the project was uneven. The most notable shortfall was the delay in establishing functional financial management procedures and arrangements; these were completed during year 2, after the Bank provided financial support to MEFCCA and its 3 regional offices. Another shortcoming at entry was the lack of an explanation of why the food security objective would be achieved if the project focused on productivity gains leading to increased production and improving food availability and nutrition, with little mention of specific aspects and measures to achieving improved and sustainable access to food and improved nutrition by the target communities, which is the core element of food and nutritional security in the literature (PAD, para. 9). Quality-at-Entry Rating Moderately Satisfactory b. Quality of supervision The quality of Bank supervision support was satisfactory, based on the evidence provided in the ICR (para. 76), including: (i) Bank supervision focused on facilitating timely and appropriate technical and operational approaches to developing and implementing the IDPs; (ii) the Bank's project team forged a strong collaboration with the GAFSP Coordination Unit; (iii) the Bank's project team accorded high priority to establishing a strong M&E system; this included providing timely technical support to devising the baseline study, and high quality benefit monitoring system (BMS), including several impact evaluation studies (via additional GAFSP funds); active technical support by the Bank's health/nutrition specialists, and provision of trust fund for training MEFCCA specialists; (iv) regular and timely supervision and other technical support missions (10 and 13, respectively). Since most of the mission members were country- based, this enabled the Bank team to be responsive and well-focused to their MEFCCA counterparts;(v) The Bank's supervision/ICR and performance assessments were of high quality, candid and comprehensive; and (vi) The continuity of Bank staff throughout the project's implementation also contributed to project performance (ICR, para. 76). The main shortcoming was the Bank team’s neglect to formally revise some of the RF targets, thereby resulting in the apparent significant overachievement of some of the project's targets (due to underestimated baseline figures for two key outcome targets (ICR, para. 66, and explanation provided by the Project Team to the IEG Evaluator, Note dated February 4, 2021). Quality of Supervision Rating Satisfactory Overall Bank Performance Rating Moderately Satisfactory 9. M&E Design, Implementation, & Utilization Page 13 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review NI Caribbean Coast Food Security Project (P148809) a. M&E Design Overall, the M&E design was sound. (i) The project’s M&E design was based on a sound results framework/RF. The RF provided the basis for designing a sound M&E system; the results indicators at various levels were well linked --- impact, outcome, intermediate outcome and output performance indicators, which also provided a common basis to help define implementation priorities, facilitate decision-making, and guide implementation adjustments (ICR, para 64); and the ToC laid out clearly the chain of activities that were intended to lead to higher productivity in food production. As stated above (under Efficacy), there were various weaknesses in the explicit linkages and measures for achieving food and nutrition security. (ii) The PAD described clearly the main project indicators, at various levels, with a focus on relevant and verifiable outcomes and outputs, especially with respect to the IDPs (ICR, Annex 6), enabling the partial measurement of progress toward the two core objectives of food and nutritional security. (iii) The M&E scope was consistent with project design and implementation requirements, especially by devoting emphasis to the M&E of the different types of IDPs during their cycles. These aspects also posed the main challenges, which were managed effectively. (iv) The design of the M&E system gave importance to the early design and implementation of the baseline study (BL), the beneficiary monitoring system (BMS), and the carrying out of an independent impact evaluation study (in several components). b. M&E Implementation Overall, M&E implementation proceeded smoothly, based on the following evidence presented in the ICR (para. 65): (i) the BL and BMS developed and implemented early during the project's implementation, constituted a key element to supporting implementation activities and associated decision-making processes. These instruments also contributed to the measurement procedures and estimation of two of the PDO outcome indicators, albeit with limitations in assessing the extent of their achievement; (ii) Baseline Study and Evaluations: there was a high quality BL study, which was developed early during implementation, and developed jointly by various institutional actors, with Bank technical support. It was based on a sound methodology, based on sample of IDP beneficiaries, for more in-depth monitoring and for informing the subsequent impact evaluation studies. These evaluations were carried out in two phases: at mid-term review and at project closure; (iii) Beneficiary Monitoring System (BMS): It was an innovative, real time, geo-referenced system enabling each community to track IDP progress. The design was robust and applied good practices. The design was robust and applied good practices; (iv) The ICR affirmed the high quality of the M&E implementation, which is verified by the availability of reports that were of good quality and delivered on time. It is notable that MEFCCA "adopted the M&E system" for its own institutional M&E system, to support its other programs and projects (ICR, para. 65). Page 14 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review NI Caribbean Coast Food Security Project (P148809) c. M&E Utilization The Project's M&E information was utilized moderately effectively, based on the evidence presented in the ICR (para. 66), as follows: (i) Project monitoring data, progress and evaluation reports were a "valued input" into management decision-making, and as input for research and various reporting deliverables. However, the ICR recognizes that the data generated by the BL and BMS were not used to adjust the targets of several project indicators. Also, it was recognized only at the end of project implementation/closing (end of 2019), with the completion of IDP execution, that the data regarding the achieved targets became clear; (ii) The differences between the estimated targets at appraisal and achieved targets at completion revealed that two PDO outcome indicators exceeded significantly their target values because project design used baseline figures which were too high, and did not adjust the target values during implementation: "Baseline study and the Beneficiary Monitoring System/BMS revealed that agricultural/livestock productivity rates reported by IDP beneficiaries were actually much lower than estimated/assumed during project preparation. This information should have prompted a significant increase in the target above 10%." (ICR, para. 66) (e.g., with reference to: "increased agricultural/livestock productivity among all direct beneficiaries", exceeding the target by 680%; and "increased production volume", exceeding the target by 720%). (iii) M&E activities and outputs provided meaningful inputs for guiding project implementation, including generating progress and evaluation reports which were used to some extent for decision-making by various entities (ICR, para. 66). M&E Quality Rating Substantial 10. Other Issues a. Safeguards The project was classified as Category B (partial assessment) and triggered various environmental safeguards, including: OP/BP 4.01 Environmental Assessment; OP/BP 4.04 Natural Habitats; OP/BP 4.36 Forests; OP4.09 Pest Management; and OP/BP4.11 Physical Cultural Resources. The ICR concludes that "overall, the Project's compliance with triggered environmental safeguards was satisfactory" (para. 69). The ICR provides relevant supporting evidence, including: (i) an Environmental Assessment (EA) and Environmental and Social Management Framework (ESMF) were prepared, consulted and disclosed, in accordance with the Bank's policies and directives; (ii) the promotion and application of climate smart agriculture and sustainable production/fishing practices; (iii) an Environmental Social Management Plan (ESMP) was prepared & disclosed for each of the 58 IDPs; Page 15 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review NI Caribbean Coast Food Security Project (P148809) (iv) through the implementation of the ESMPs, there were two positive effects: (a) strengthening the institutional capacities for environmental management in the Caribbean Coast; and (b) the technologies were aligned with recommended adaptation/mitigation measures, and climate smart agricultural practices; (v) the quality & timeliness of the ESMP documents were satisfactory, complying with Bank requirements. The main actions carried out in compliance with the Bank’s Environmental Safeguards which were triggered by the project (covering 5 OPs/BPs) were carried out satisfactorily (para. 69). The Project triggered the social safeguard policy OP/BP 4.10 Indigenous Peoples. The ICR concluded that the Project compliance with the social safeguard policy triggered was satisfactory. Relevant evidence is summarized in the ICR (para. 70). The Project developed and implemented a grievance redress mechanism, in consultation with communities, and was implemented effectively (para. 71). A total of 129 incidents were recorded, 81 of which were complaints. The ICR concludes that all complaints were satisfactorily resolved prior to project closing (and report available in project files). b. Fiduciary Compliance (i) Financial Management FM performance aspects during preparation were "mixed", due to capacity constraints. During implementation, the ICR concludes that FM performance ranged from "moderately satisfactory" early in the project’s implementation to becoming "satisfactory", based on improved capacities of key actors/entities, and on the evidence presented in the ICR (see paras. 73 and 74). (ii) Procurement Aspects Procurement was carried out by MEFCCA at the central level, as well as by its three decentralized regional offices, through “delegated administrative” modalities, which reflected institutional capacities. The ICR concludes that, notwithstanding procurement delays at the outset due to capacity constraints, the project's procurement performance was "satisfactory" (para. 72). c. Unintended impacts (Positive or Negative) N/A d. Other Page 16 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review NI Caribbean Coast Food Security Project (P148809) The ICR highlighted 4 other positive aspects arising from the project. While recognizing some attribution challenges, the nature/scope of these benefits are summarized below, based on evidence presented in the ICR, paras. 47 - 53), and also presented in the Project's impact evaluation studies, dated 2019. (i) Gender: The proactive gender approach implemented by the project contributed to tangible results, as reflected by the following evidence: (a) The project was guided by a gender strategy, which was developed during the first year of the project’s implementation (2016, ICR, para. 47). This strategy was strongly aligned with the Government's and the Bank's broader gender policies/strategies (National Development Plan and CFP, respectively); (b) the Project promoted gender equality through several integrated activities, especially through the IDPs; (c) IDPs provided direct support to 6,994 women (ICR, para. 47), which comprised about 50% of the support to beneficiaries being channeled to female heads of households; (d) a prominent role of women in the key decision-making platforms of the IDP's participatory approach, including women's prominent roles in the: executive boards of the IDPs; rural development promoters' network; commercialization promotion events; and (e) a prominent gender-sensitive approach embedded in the project's M&E system, and corresponding performance indicators. (ii) Institutional Strengthening: The ICR concludes the project promoted "substantial" institutional strengthening, based on the following evidence (ICR, paras. 48-51 and Annex 7): (a) effective functioning and strengthening of the Project's overall coordination and implementation institutional roles and arrangements, which enabled: several public national (especially MEFCCA) and regional entities to carry out vital technical functions/activities; effective decentralization of project implementation to MEFCCA's three regional delegations (which also manage other projects/activities); (b) effective strengthening of the MEFCCA regional delegations, thereby enabling enhanced capacities for continued decentralized and expanded development support to the Caribbean Coast; (c) institutional strengthening reaching the Indigenous and Communal government, whereby project activities reached 17 of the 23 indigenous territories, 208 local authorities in IDP approval and implementation, including the role of the two autonomous governments for the Caribbean Coast (RACCS and RACCN); (d) enhanced sanitary and phytosanitary surveillance/services, including increased number of specialists, building capacities of technical teams and provision of these services to all communities; and (e) strengthening to artisanal fisheries modernization, including the strengthening of the National Fisheries and Aquaculture Institute (INPESCA), and its expanded and more effective role and support to fishery communities; (iii) Mobilizing Private Sector Financing (ICR, para. 52): The Project-supported IDPs fostered “in-kind” counterpart contributions from private sector beneficiaries, varying between 10 and 15 percent of the total cost of each IDP. The actual amount of their contributions exceeded the initial target ($3.4 million vs. $2.3 million, or an increase of 48%); (iv) Poverty Reduction and Shared Prosperity: The Project contributed to poverty reduction among agricultural producers in the poorest areas of Nicaragua, which had the highest poverty levels and incidence of extreme poverty (para. 53). The IDP approach provided tangible interventions for promoting increased productivity and production levels of staple food crops production and several types of nutrition interventions, to contribute to increased incomes and well-being of the targeted population groups. With the IDP support, the net agricultural family income per year of project beneficiaries, arising from the IDP support, was stated by the ICR to have increased between two to six-fold (from about $320 to $1,300 (ICR Page 17 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review NI Caribbean Coast Food Security Project (P148809) para. 53, Annex 4, although the details of how these figures were estimated are not clear in the ICR). To the extent that incomes of farmers and other members of the targeted communities were increased through on- farm employment, it is likely that the food security of their households will have been enhanced. However, the ICR did not conduct an explicit assessment of the food and nutrition security impact arising from the increased incomes. 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment While the relevance of the project's objectives and its efficiency were rated substantial, the efficacy with which the project's Objective 1 was achieved was rated "substantial with moderate shortcomings", Outcome Satisfactory Moderately Satisfactory and the extent to which Objective 2 was met was rated "modest", which this review assessed as having an outcome with moderate shortcomings and an overall outcome rated moderately satisfactory. Quality at entry for the project was rated Moderately Satisfactory (see Section 8), Bank Performance Satisfactory Moderately Satisfactory which brought the rating for the Bank Performance as a whole to the same rating. Quality of M&E Substantial Substantial Quality of ICR --- Substantial 12. Lessons The ICR presents 6 lessons (ICR, paras. 80-85). The aspects of the lessons that have broader application than this project have been summarized below. IEG suggests one additional lesson. (1) Flexible and Customized Targeted Interventions: The ICR referred to the importance of following “flexible implementation arrangements” for a project targeting a culturally diverse population. The ICR adds that most of the project's targeting and implementation instruments had been developed and agreed prior to project launching. The lesson is that there are benefits in having clear and agreed selection criteria for community and beneficiary participation prior to the project's effectiveness, which enabled the implementation agency/MEFCCA and its 3 regional delegations to Page 18 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review NI Caribbean Coast Food Security Project (P148809) carry out an efficient and successful project, by targeting the expected benefits to the targeted beneficiaries. Accordingly, an additional aspect of this lesson involves customizing the project’s interventions to the priority needs of the targeted communities and direct beneficiaries (e.g., contributing to the enhancement of nutritional security of children in poor communities of the Caribbean Coast region). (2) Efficiency Benefits of Sound and Strengthened Implementation Roles and Arrangements: The ICR correctly highlights the importance of clear institutional roles and arrangements, including M&E system, for implementing the project’s integrated approach, especially through MEFCCA's three regional offices/delegations. Moreover, these arrangements enabled a strengthened regional multi- agency participation and coordination, including customizing the design, implementation, sustainability and scaling up IDPs (including cost sharing arrangements), according to the different types of beneficiaries and their priority needs and capacities. The specific lesson is that these arrangements, coupled with institution capacity building of key entities and implementation processes, contributed to both fiscal and implementation efficiencies in generating, scaling up and sustaining the project results. (3) Effective Collaboration is a Key to Project Success: The ICR highlighted some essential conditions and elements for successful projects. This project was successful because of the strong commitment at all levels of responsibility from: (a) committed and decisive Government leadership, at both central and regional levels, which was further enabled by formal cooperative agreements between MEFCCA and relevant public service entities (ICR, Annex 7); (b) effective collaboration with and participation of the Autonomous Regional Authorities and indigenous communal governments, consistent with Government legislation; (c) sound M&E system; and (d) the continuity of the Bank and Government technical teams. The lesson is that collaboration between various implementing institutions through transparent communication, clear allocation of responsibilities and participation, consistent with Government legislation of roles, formal cooperation agreements amongst key entities and a functional M&E system can result in successful project outcomes. (4) Explicit and Measurable Integrated Strategy to Enhanced and Sustained Food and Nutrition Security: The ICR emphasized the project’s integrated strategy and supporting interventions which promoted enhanced food and nutrition security, as evidenced partially by various complementary outcome measures outlined in Annexes 1 and 6 of the ICR. The main lesson is that a focus on food productivity increases and qualitative indicators are not sufficient to achieve and sustain robust and measurable improvements in food and nutrition security for the target population (which included landless, laborers, urban poor); IEG suggests one additional lesson which may have application to similar other projects. It is vital for the project's design to accurately and rigorously define key concepts embedded in project objectives. The project’s objective was to “enhance food and nutritional security”. The ICR defined food security in a multitude of ways. Without providing an explicit rationale, it was assumed that interventions which would increase food productivity/production would lead to increased food and nutritional security of targeted households. In practice, increased and diversified food production does not automatically lead to improved access to food and nutrition by all persons in the target population. Given the structural poverty in areas like the Caribbean Coast, the project ICR did not provide relevant and robust evidence on the extent to which food and nutrition security Page 19 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review NI Caribbean Coast Food Security Project (P148809) for the target group (including landless and laborers) was achieved, and yet the ICR claimed (without evidence) that food security had been "fully" achieved because food production and its diversification had increased. The lesson is that, in the absence of a cogent and precise definition of a project’s objective, and the indicators with which results are measured, it is not possible to conclude the extent to which the objective is achieved. 13. Assessment Recommended? No 14. Comments on Quality of ICR Overall, the ICR is well written, consistent with the Bank's ICR guidelines, analytical (especially the EFA analysis), candid, results-focused, and most sections were supported by adequate evidence. The evidence included the analyses of the data presented in the Results Framework (Annex 1), a Project Impact Evaluation Study, with two supporting and complementary evaluation studies. Together these assessments provided timely inputs and evidence-based information for preparing the ICR. Annex 8 of the ICR aimed to provide a summary of the main findings of the impact evaluation. However, because of its brevity, Annex 8 did not include the substance of important and relevant findings (e.g., increased farmer incomes, and the potential linkages between increased incomes and enhanced access to food and improved nutrition). Despite acknowledging alternative definitions of food security, the ICR chose to focus the theory of change (ToC) and the assessment of the productivity of food production and domestic food supplies, rather than assessing in more explicit manner, expanding access to food as the basis for achieving and assessing food and nutrition security by all beneficiary target groups. The ICR does not question why, during project implementation, PDO indicators were not amended to include measures of enhanced food security. The ToC also did not include explicit reference to strengthening extension systems needed to scale up and sustain increases in productivity, and also to scaling-up/sustaining food and nutrition security. On the other hand the Bank project team clarified that "In Nicaragua, there is no formal national agricultural extension system per se. Rather, diverse extension/technical services in rural areas are delivered by 6 institutions under the Public Sector Innovation System, with different levels of rural capacity" (Bank project team response to IEG questions, February, 2021). The lessons outlined in the ICR focused on some important issues based on evidence from the project. a. Quality of ICR Rating Substantial Page 20 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review NI Caribbean Coast Food Security Project (P148809) Page 21 of 21