Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PE-National Ag. Innovation Program (P131013) Report Number: ICRR0022758 1. Project Data Project ID Project Name P131013 PE-National Ag. Innovation Program Country Practice Area(Lead) Peru Agriculture and Food L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IBRD-83310,TF-12180 01-Sep-2019 39,257,162.13 Bank Approval Date Closing Date (Actual) 17-Dec-2013 26-Feb-2021 IBRD/IDA (USD) Grants (USD) Original Commitment 40,000,000.00 350,000.00 Revised Commitment 39,257,162.13 336,670.10 Actual 39,257,162.13 336,670.10 Prepared by Reviewed by ICR Review Coordinator Group Hassan Wally April B. Connelly Christopher David Nelson IEGSD (Unit 4) 2. Project Objectives and Components DEVOBJ_TBL a. Objectives The Project Development Objective (PDO) of the National Agricultural Innovation Project (PSNIA) as articulated in the Project Appraisal Document (PAD, paragraph 14) was to: "create the adequate conditions in the SNIA in order to support the effectiveness of its member organizations in providing or developing improved agricultural technologies." Page 1 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PE-National Ag. Innovation Program (P131013) The PDO in Loan Agreement (page 5) was identical to the one in the PAD except where underlined: "create the adequate conditions in the Borrower's SNIA in order to support the effectiveness of its member organizations in providing or developing improved agricultural technologies." Parsing the PDO: The PDO will be parsed into two sub-objectives mentioned below as Objective #1 and Objective #2. Objective #1: To create the adequate conditions in the Borrower’s SNIA in order to support the effectiveness of its member organizations. Objective #2: To enhance the effectiveness of SNIA member organizations in developing and providing improved agricultural technologies. b. Were the project objectives/key associated outcome targets revised during implementation? No c. Will a split evaluation be undertaken? No d. Components The PDO was supported by four components as follows: 1. Strengthening Capacities of the National Institute for Agriculture Innovation (INIA) to Lead the National System for Agricultural Innovation (SNIA) (appraisal estimate: US$23.34 million, IBRD expected contribution: US$2.59 million, actual cost: US$23.34 million). The project would contribute to strengthening INIA’s institutional capacities to lead and coordinate the SNIA, through inter alia: a) The establishment of SNIA’s technical secretariat. The Technical Secretariat would support INIA’s role as the lead government agency for the SNIA. It is in charge of planning, budgeting, evaluating, and reporting on activities related to the SNIA in consultation and under the final supervision of the Project’s Steering Committee. The Technical Secretariat would report directly to the head of INIA. It would manage four units: the Policy Support Unit, the Knowledge Management Unit, the Market for Innovation Services Unit, and the Stakeholders Relationship Unit. b) The establishment of an innovation awards program within INIA to recognize excellence in agricultural innovation. An award system would be established to recognize excellence in agricultural innovation. Awards would recognize the best initiatives aimed at improving productivity, sustainable natural resource management, environmental management, food security or business opportunities. Any individual or group of individuals, as well as organizations could compete. Awards would be granted at an annual national innovation fair. Over 5 years a total of 80 awards would be distributed, each with a maximum cash prize of US$20,000. Page 2 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PE-National Ag. Innovation Program (P131013) 2. Consolidating the Market for Innovation Services (appraisal estimate: US$57.91 million, IBRD expected contribution :US$24.16 million, actual cost: US$57.81 million). The project would support the development of a market for innovation systems in the Borrower’s territory, through: (a) The provision of support to Beneficiaries for the carrying out of: (i) Adaptive Research Subprojects; (ii) Extension Subprojects; and (iii) Community Seeds Enterprise Subprojects. (b) The carrying out of awareness campaigns to promote the availability of the competitive funds described in (a) above, especially among women and indigenous population groups. 3. Strategic Capacities in the SNIA (appraisal estimate: US$40.31 million, IBRD expected contribution: US$12.75 million, actual cost: US$40.31 million). The project would be enhancing the capacity of the system to produce and facilitate innovation, through: a) The provision of support for the carrying out of: (i) Strategic Research Subprojects; and (ii) Capacity Building Subprojects. b) The financing, on a competitive basis, of Scholarships to Eligible Students for postgraduate studies and short term internships. c) The carrying out of awareness campaigns to promote participation in the Subprojects described in (a) and the Scholarships described in (b) above. 4. Project Management (appraisal estimate: US$ 3.57 million, IBRD expected contribution: US$ 0.5 million, actual cost: US$7.14 million). This component would strengthen the institutional and organizational capacity of INIA required for the successful implementation of project activities, including compliance with procurement, safeguards, financial management, and monitoring and evaluation requirements. A Project Implementation Unit (UEP) would be established within INIA to implement PNIA and its two projects, PSNIA; and PINIA. The UEP would be led by a Project Director and will host planning and budget specialists, fiduciary, legal, safeguard management and M&E specialists and the required support staff. e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project Cost. The total cost of the project at appraisal was US$125.2 million. The actual cost as reported in the ICR data sheet (page 2) was US$100.7 million. The difference between the appraisal amount and the actual amount was mainly due to lower than expected borrower's contribution (see below). Financing. The project was financed through a five-year IBRD Loan of US$40.00 million to the Government of Peru. The ICR (footnote #2) explained the final IBRD contribution included US$336.67 thousand from a Government of Spain Trust Fund provided as a project preparation grant and used to support some design work. The actual amount disbursed according to the ICR data sheet (page 2) was US$39.26 million (98% of the expected amount at appraisal). Borrower Contribution. The Government of Peru (GoP) was expected to contribute US$54.50 million of counterpart funds through resources from the Treasury, and the Local Farmer Organizations were expected to contribute US$30.75 million. The actual amounts reported in the ICR data sheet (page 2) were US$37.37 million (69% of appraisal estimate) and US$24.14 million (78% of the appraisal estimate) for the GoP and Page 3 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PE-National Ag. Innovation Program (P131013) Local Farmer Organizations, respectively. The ICR does not explain the reason for the shortfall in government and Local Farmer Organizations actual contributions. Dates. The project was approved on December 17, 2013, but was declared effective on September 7, 2012. The ICR (footnote#2) explained that the Effectiveness date in the ICR Data Sheet was the Effectiveness date for the grant from the Government of Spain Trust Fund, which preceded the Board Approval date for the IBRD loan. The Mid-Term Review (MTR) was conducted on September 19, 2017. The MTR was conducted five years after effectiveness. While the PAD did not specify an exact date for the MTR, the MTR for this project was relatively late relative to other Bank-financed project. The project closed on February 26, 2021 which was 18 months beyond the expected closing date on September 1, 2019. According to the ICR (paragraph 23) "an extension of the project closing date was needed to make up implementation time that had been lost due to the 13-month delay in project effectiveness," and "to provide sufficient time for completion of final project activities and the recruitment of a consultant to carry out the final impact evaluation following procurement delays caused by the COVID-19 crisis." The project underwent three Level 2 restructurings as follows: 1. On October 9, 2018, when the amount disbursed was US$32.17 million, in order to authorize a 12 months extension of the loan closing date from September 1, 2019 to September 1, 2020. 2. On May 18, 2020, when the amount disbursed was US$38.53 million, in order to reallocate funds between disbursement categories. 3. On August 18, 2020, when the amount disbursed was US$39.53 million, in order to authorize a 6 months extension of the loan closing date from September 1, 2020 to February 26, 2021. The changes (extension of the closing date and reallocation of funds) introduced in the three restructurings were relevant and had no effect on the Theory of Change. 3. Relevance of Objectives Rationale Context at Appraisal. The National Agricultural Innovation Institute (INIA) was established in 1978. INIA was responsible for agricultural research, technology transfer, technical assistance, conservation of genetic resources, and production of seeds and breeding of species of high genetic value. However, it slowly lost significance, experienced flat or falling budgets and was in need of substantial strengthening. In 2008, through a Legal Decree 1060, INIA was tasked with leading the National Agricultural Innovation System (SNIA), with the support and the guidance of a newly to be established National Commission for Innovation and Training in Agriculture (CONICA). Until 2013 INIA has made little progress on this new mandate. This project would support the National Agricultural Innovation System (SNIA) and its actors through the institutionalization within INIA of a Technical Secretariat that will coordinate the SNIA. The Technical Secretariat would operate a series of competitive grant funds, supporting the empowerment of producer organizations, strengthening strategic competencies, and supporting skills development for agricultural Page 4 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PE-National Ag. Innovation Program (P131013) innovation. The project would consolidate a decentralized agricultural innovation system and institutionalize its leadership. Previous Bank Experience. From 2002 to 2010 the Bank supported the Innovation and Competitiveness Program (INCAGRO), operated by Ministry of Agriculture and Irrigation (MINAGRI) through a series of competitive funds to strengthen strategic and adaptive research as well as technical assistance and capacity building. This project (PSNIA) evolved from the earlier INCAGRO program to create adequate conditions for agricultural innovation system. Consistency with the Bank Strategies. At appraisal, the PDO was in line with objectives of the Bank's Country Partnership Strategy (CPS, FY2012-FY16). Objective 3 of the CPS focused on sustainable growth and productivity. Specifically, results area # 3.2 concerns sustainable rural development and water resources management. Further, lending for the purpose of strengthening the SNIA was included in the Indicative World Bank Program for FY12-16 (Table 4 of the CPS). At completion, the PDO remained in line with the Bank's Country Partnership Framework for Peru (CPF, FY2017-FY21). Specifically, Pillar I: Productivity for Growth, where objective 3 emphasized the importance of facilitating the absorption of skills and technology to increase productivity, and one of the expected CPF results was increasing the number of farmers that have adopted improved agricultural practices. Furthermore, the CPF under Pillar III: Strengthening the Management of Natural Resources, where objective 8 called for increasing the efficiency of water use and reverting and preventing agricultural land degradation. The project activities centered around strengthening the capacity of the SNIA to generate and promote the successful uptake of agricultural innovations that were designed to contribute to these expected results. Consistency with the Government Strategies. At appraisal, the PDO was in line with the Government’s strategy that aimed to promote rural development in all regions of the country with emphasis on enhancing productivity through innovation in poor regions. At completion, the PDO remained relevant to the Government priorities for the agriculture sector, namely, promoting innovation in the agricultural sector. The relevance of the PDO was further emphasized when the Interim President on March 24, 2021 signed a decree creating a new PROINNOVATE program, "with the mandate to support at central level the types of innovation activities that have been carried out at sectoral level under PNIA and similar programs (ICR, paragraph 27)." While the PDO was consistent with the Bank strategies and Government priorities, the statement of objectives was complicated and lacked clarity. Specifically, the term “adequate conditions in the Borrower’s SNIA” was problematic for assessing the PDO, since the basis for assessing the project outcomes was not clear. The PDO also lacked a connection to the ultimate goal of the project which was improving agricultural productivity and increasing farmers' incomes. Therefore, Relevance of Objectives is rated Substantial. Rating Relevance TBL Rating Substantial Page 5 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PE-National Ag. Innovation Program (P131013) 4. Achievement of Objectives (Efficacy) EFFICACY_TBL OBJECTIVE 1 Objective To create the adequate conditions in the Borrower’s SNIA in order to support the effectiveness of its member organizations. Rationale Theory of Change (ToC). To achieve the stated objective, the project would support capacity-building and reorganization of the National Agricultural Innovation Institute (INIA). This was expected to increase the number of active institutions in the National Agricultural Innovation System (SNIA), promotion of competitive selection and funding of innovations, and generation and dissemination of new knowledge and innovation. The expected outcomes included: strengthening the capacity of INIA to lead SNIA, strengthen the links with the national science and technology system, and SNIA member institutions become satisfied with the system effectiveness and cooperation. The anticipated long-term impact would be a sustainable, equitable, and pluralistic innovation system. The achievement of the stated objective was underpinned by the following assumptions: 1. The weak institutional capacity of INIA would be strengthened and adequate technical assistance would be provided for sub-projects; 2. The insufficient level for knowledge generation and dissemination of innovations would be increased; and 3. The government and regional leadership budget commitments for the innovation agenda would be strengthened and continue after the project closure. The activities reflected in the ToC were directly linked to the PDO in a plausible causal chain and the stated assumptions were logical and realistic. However, the third assumption was not directly related to achieving the objective, but to its sustainability post completion. Also, the ToC could have benefitted from clarifying the relationship between INIA and SNIA. Outputs The following outputs were reported in the ICR (Annex 1) unless referenced otherwise: Strengthening Capacity of INIA to lead the SNIA 1. 5 agricultural policy papers were submitted for approval to the Ministry of Agriculture and Irrigation (MINAGRI) (target: 8, not achieved). 2. 100% of SNIA’s Technical Secretariat funded by INIA (target: 75%, exceeded). 3.169 submissions to the award program (target: 100, exceeded). 4. 229,247 downloads per year from the website of the Technical Secretariat (target: 60,000, exceeded). 5. 88% of institutions participating in meetings and events (target: 60%, exceeded). Strategic Capacities in the SNIA (Extension and Technical Assistance Capacity Building; Postgraduate Fund) 1. 136 papers were accepted in indexed scientific journals (target: 60, exceeded). 2. 76,887 training days to extension and agriculture professionals (target: 7,000, exceeded). Page 6 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PE-National Ag. Innovation Program (P131013) 3. 57 agricultural professionals received a Master’s Degree through scholarship programs (target: 31, exceeded). Thesis research topics selected by the scholarship recipients were generally well aligned with INIA priorities and included, among others: (i) sustainable use of natural resources (17 theses), (ii) agricultural productivity and competitiveness (16 theses), (iii) genetic improvement (7 theses), and (iv) value-addition products (5 theses). Other thesis research topics included use of information and communication technologies in agriculture, animal health, industrial competitiveness, and climate change and environmental impacts (ICR paragraph 40). 4. 25 agricultural professionals (women) received a Master’s Degrees through scholarship programs (target: 11, exceeded). 5. 291 agricultural professionals received internships / exchanges (66 national and 189 international), through scholarship programs (target: 80, exceeded). Internships were financed in areas such as: (i) innovation in agribusiness (17%), (ii) coffee value chain (15%), (iii) animal reproduction including genetic improvement using biotechnology techniques (14%), (iv) agri-food industry (6%), (v) and climate change including links with livestock and agroforestry (7%) (ICR, paragraph 40). 107 agricultural professionals (women) received internships / exchanges through scholarship programs (target: 28, exceeded). 6. The project financed 36 group internship benefitting 295 individuals in diverse stakeholder groups: (i) producer associations (11), (ii) small agricultural companies (9), (iii) agricultural cooperatives (8), (iv) educational institutions (4), (v) non-governmental organizations (2), (vi) water users association (1), and (vii) professional associations (1). More than one-third of the beneficiaries (109 or 37% of the group) were women (ICR, paragraph 40). 7. 745,212 potential agricultural professional beneficiaries (target: 450,000, exceeded). Outcome To strengthen the SNIA’s capacity to effectively support its member organizations, the project created a technical secretariat within INIA as the agency leading the SNIA, established a national agricultural innovation awards program led by INIA to recognize and give visibility to agricultural innovation throughout Peru, and strengthened the cadre of agricultural scientists and extension professionals in Peru by offering competitively awarded scholarships for graduate study and internships. The following elements are discussed to assess the achievement of the stated objective: 1. Improvement in the capacity of INIA to exercise its role as governing body of the SNIA.  The legal status of INIA was changed to a Specialized Technical Organization (OTE) and became effective on February 8, 2018. Five policy papers (target:8) were prepared with project resources and approved by MINAGRI to change INIA's policy framework. However, the organizational changes called for in the policy papers were partially implemented. According to the ICR (paragraph 34) through the OTE status, INIA adjusted its operational policies and streamlined its administrative procedures. Despite the change in the legal status of INIA, there was resistance by established interest groups to change the status quo (ICR, paragraph 34). The ICR (paragraph 34) attributed this to the opposition of many unionized staff to possible changes in their employment conditions, among other factors. According to the ICR (paragraph 34) "this effectively restricted the level of autonomy achieved by INIA and limited its ability to pursue the national innovation agenda independently from MINAGRI."  After the change in legal status of INIA to OTE, the project transferred the PIU established Technical Secretariat to INIA. According to the ICR (paragraph 35) the Technical Secretariat was fully staffed in the third year of the project and functioned effectively and "provided a strong impetus for accelerating Page 7 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PE-National Ag. Innovation Program (P131013) the implementation of project activities." To enhance the presence of INIA throughout the national territory and support SNIA members at the regional level in driving innovation, the project established six regional offices and five sub-regional offices. According to the ICR (paragraph 36) the regional offices contributed to the successful implementation of the competitive grant program by providing close implementation support to grant recipients.  The project also supported the establishment of 20 Agricultural Innovation Technical Commissions and organized 54 regional meetings with local SNIA actors. The Technical Commissions facilitated the development of 20 Regional Agricultural Innovation Agendas in which research, development and technology transfer actions were identified for economically important value chains in each region. 2. Strengthening links with the national science and technology system. The project's steering committee included the Directors of the National Council for Science, Technology, and Innovation (CONCYTEC). This ensured that the project's efforts to promote innovation at the sectoral level could be complemented by efforts made at the central level to promote innovation more broadly throughout the entire economy. 3. Promotion of agricultural innovation through competitive grants, scholarships and internships, and agricultural innovation fairs and awards.  The project established a postgraduate fund to strengthen the strategic competencies among professionals in the agricultural sector working on research and development and innovation. The fund awarded scholarships for postgraduate degree study and internships (82 Masters scholarships, of which 57 (70%) were successfully completed). According to the ICR (paragraph 40) "almost all scholarship recipients have found employment within INIA or in SNIA member institutions, indicating that they are contributing to strengthening the system." The project also supported individual and group internships (see details under outputs). Despite a delayed start in 2018, the post graduate fund was successfully implemented. The ICR noted that the start of the Covid-19 pandemic "prevented some graduate students from completing their theses and some internship grant recipients from completing their internships."  To enhance visibility for the innovation agenda throughout the country, the project organized the national innovation fairs (known as AGRONOVAs) combined with a national prize competition to recognize agricultural innovation. The fairs served to bring together many different SNIA actors and showcase their innovative projects to a large public. The national prize competition was designed to incentivize participating institutions to publicize their innovations, stimulate new initiatives, and identify opportunities to scale up innovations. 169 innovation projects were nominated, of which 20 were awarded prizes in four categories. 4. Dissemination of knowledge generated through sub-projects, thesis research, and commissioned studies.  The outcomes from the research sub-projects were disseminated through 136 articles (227% of the target) published in scientific journals. In addition, technical sheets were prepared for 86 sub-projects detailing the innovation process followed under the sub-project and describing in detail the innovation generated and its possible uses. Finally, 248 records have been created (57 for Masters degrees and 191 for internships) containing detailed information about the research conducted under each degree program or internship. Page 8 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PE-National Ag. Innovation Program (P131013)  Technical information was published and shared among SNIA member institutions through the creation of an online information platform for SNIA member institutions. According to the ICR (paragraph 44) "SNIA members and other users were downloading materials at a rate of nearly 230,000 downloads / year, far exceeding the targeted rate of 60,000 downloads/year. In addition to the above-mentioned achievements, an independent survey conducted shortly before project closing showed that 85% of the 893 SNIA member institutions surveyed reported being satisfied with the effectiveness of the SNIA and the level of cooperation among SNIA institutions. Satisfaction was measured using a set of eight questions, applying the Liker scale to generate a satisfaction level value between 1 and 5. A SNIA member institution was considered satisfied if the sum of the values of the questions reached at least 70% of the maximum score possible. On this basis, the efficacy with which Objective 1 was achieved is rated Substantial, despite some minor shortcomings. Rating Substantial OBJECTIVE 2 Objective To enhance the effectiveness of SNIA member organizations in developing and providing improved agricultural technologies. Rationale Theory of Change (ToC). To achieve the stated objective, the project would support the development of a competitive grant program for: collaborative sub-projects, strategic research sub-projects, capacity building sub-projects. The project would also fund and develop scholarships and internships. These activities would result in the generation of innovative technologies (including improved seeds and germplasm) which would be transferred to new end-users. Activities were expected to strengthen the capacity of extension service providers and value chain actors as well as strengthen the the research capacity of future SNIA researchers. The expected outcomes included: developing a market for innovation services, generate new technologies from demonstration research sub-projects, increased adoption of innovations among farmers and other value chain actors, and generate, develop and publish new knowledge. The anticipated long-term outcomes included: a sustainable, equitable and pluralistic innovation system, increased incomes of farmers and researchers benefiting from grants programs, and increased agricultural competitiveness and higher agricultural productivity. The achievement of the stated objective was underpinned by the following assumptions: 1. The weak institutional capacity of INIA would be strengthened and adequate technical assistance would be provided for sub-projects; 2. The insufficient level for knowledge generation and dissemination of innovations would be increased; 3. The low level of adoption of innovations among farmers would be raised, including effective targeting of women and indigenous people and 4. The government and regional leadership budget commitments for the innovation agenda would be strengthened and continue after the project closure. Page 9 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PE-National Ag. Innovation Program (P131013) The activities reflected in the ToC were directly linked to the PDO in a plausible causal chain and the stated assumptions were logical and realistic. However, the fourth assumption was not directly related to achieving the objective, but to its sustainability post completion. Outputs The following outputs were reported in the ICR (Annex 1) unless referenced otherwise: Consolidating the Market for Innovation Services 1.541 total collaborative sub-projects (target: 400, exceeded). 2. 367 collaborative extension sub-projects (target: 240, exceeded); 95% of which led to an increase in productivity and/or production. 84% facilitated greater access to markets, and 81% achieved improved quality of production (ICR, paragraph 45). 3. 129 collaborative adaptive investigation sub-projects (target: 120, exceeded) focusing on technological innovations to increase efficiency in fertilization, post-harvest processing, management of crops, and commercial management, among others (ICR, paragraph 45). 4. 45 collaborative seed sub-projects (target: 40, exceeded) where 82% of seed growers reported an increase in productivity and/or production, while over 60% reported improved quality of their products. In addition, 27% improved their access to markets (ICR, paragraph 45). 5. 41,244 producers participated in sub-projects (target: 40,000, exceeded). 6. 33% of producers participating in sub-projects were women (target: 35, 94% achieved) 7. 31% of producers participating in sub-projects were self identified indigenous (target: 20, exceeded). Strategic Capacities in the SNIA (Funds for Strategic Research) 1. 59 strategic research sub-projects were co-funded (target: 60, 98% achieved). Strategic research sub- projects focused on innovation in product processing, digital systems for crop management, genetic improvement, climate change adaptation and mitigation measures and pest management, among others, and benefitted a wide range of agricultural value chains, most importantly the coffee, quinoa, potato, cocoa, sugar cane, and livestock sectors (ICR, paragraph 50). Outcome The project sought to enhance the effectiveness of SNIA member organizations in developing and providing improved agricultural technologies through building capacity in the SNIA including INIA to enable the design and successful implementation of competitively funded sub-projects. These sub-projects would benefit producers and other agricultural value chain actors as well as SNIA member institutions, and make available a large number of validated innovations and support the dissemination of the new knowledge generated. Assessing this objective is challenging because the two PDO outcome indicators do not fully capture all of the dimensions contributing to the objective. The ICR (paragraph 45) explained that the assessment of this objective relied on Intermediate Results Indicators as well as evidence from the final impact evaluation. The degree to which Objective 2 was achieved is assessed based on the following three elements: Page 10 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PE-National Ag. Innovation Program (P131013) 1. Capacity of SNIA including INIA built that enables the successful implementation of competitively funded sub-projects and strategic research sub-projects.  541 (target: 400) collaborative sub-projects were financed on three rounds using a demand-driven competitive grant mechanism overseen by PNIA Technical Secretariat. The collaborative sub-projects involved 74 value chains, including coffee, cocoa, milk, beef, guinea pigs, banana, potato, sugar cane, sheep, quinoa, wool, and avocado, among others. Out of the 541 sub-projects only10 sub-projects were not completed representing a success rate of over 98%.  The project enhanced effectiveness of SNIA member organizations in developing and providing improved agricultural technologies deemed strategic for the nation. Strategic research sub-projects were designed to address priority themes identified by INIA and CONICA. These sub-projects were implemented by SNIA member institutions as well as by INIA. Strategic research sub-projects focused more on upstream, basic research, such as DNA fingerprinting of indigenous potato varieties, genetic characterization of disease pathogens, and development of cultural media for tissue culture (ICR, paragraph 51). 2. Benefiting producers and other agricultural value chain actors.  31,634 farmers adopted new technologies promoted by sub-projects exceeding the target of 20,000 (PDO outcome indicator #2). According to a survey of 466 sub-project participants conducted shortly before project closing, 76.7% of sub-project participants adopted at least one innovation developed under the project (ICR, paragraph 54).  Adoption rates were estimated via representative surveys of farmers and other actors who participated in adaptive research, extension, and seed enterprise sub-projects. The ICR (paragraph 54) explained that: "adoption was defined in terms of three criteria: (i) there is clear evidence that direct beneficiaries are using an innovation promoted by the Project independently and at their own discretion; (ii) the innovation introduced by the Project generates net benefits greater than those generated by traditional technologies; and (iii) the innovation is being replicated by indirect beneficiaries at the local level." 3. Making available a large number of validated innovations.  111 new technologies emerging from research projects were demonstrated on farm exceeding almost double the original target of 61 (PDO outcome indicator #3). This points to the success of the project in improving the effectiveness of the SNIA member organizations in developing and providing improved agricultural technologies from research projects to be demonstrated on-farm.  According to the ICR (paragraph 56) "750,000 producers and agriculture value chain actors could potentially benefit from these innovations in the medium and long term, showing the value-added of the sub-projects in driving innovation as a complement to the demand-based competitive grant funds." The evidence provided in the ICR and discussed above point to the success of the project in enhancing the effectiveness of SNIA member organizations in developing and providing improved agricultural technologies. The project exceeded its targets for its two outcome indicators as well as for several intermediate/output level indicators. Therefore, the efficacy with which Objective 2 was achieved is rated Substantial. Rating Page 11 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PE-National Ag. Innovation Program (P131013) Substantial OVERALL EFF TBL OBJ_TBL OVERALL EFFICACY Rationale Overall Efficacy is rated Substantial with minor shortcomings. The project almost fully achieved its objectives. By project completion, the project achieved its intended development objectives and outcomes, with all three PDO Outcome Indicators exceeded, 10 Intermediate Results Indicators exceeded, and two Intermediate Results Indicators substantially achieved. However, the envisioned organizational changes for INIA were partially implemented due to the resistance by established interest groups to change the status quo (ICR, paragraph 34). Overall Efficacy Rating Substantial 5. Efficiency Economic and Financial Analysis (EFA) ex-ante  The EFA at appraisal estimated the Economic Internal Rate of Return (EIRR) on the project investments at 26%. The evaluation horizon for the benefits of the project was set at 20 years and a discount rate of 9%.  The scope of the analysis of the benefits of the project was limited to Component 2 due to the methodological difficulties faced by the estimation of the economic and financial benefits generated by the institutional consolidation supported by Component 1 and, to a lesser extent, the strategic research funded by Component 3 (PAD, Annex 6).  The estimated annual increase in producer’s income as a result of the project was 31%.  The sensitivity analysis in the PAD focused and the impact of inclusion or exclusion of different costs on the project's IRR. It would have been more helpful if the analysis factored in the impact of increasing input costs and an increase or decrease in crop prices on the IRR, among others. ex-post  The ex-post assessment of the project’s efficiency was based on the cost-benefit analysis carried out as part of the ICR. The analysis estimated the overall Economic Internal Rate of Return (EIRR) for the project at 31.2% with an Economic Net Present Value (ENPV) of US$86.20 million and a benefit/cost ratio of 1.48. The Financial Internal Rate of Return (FIRR) was estimated at 24.6% with a Financial Net Present Value (FNPV) of US$48.20 million and a benefit/cost ratio of 1.26. Page 12 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PE-National Ag. Innovation Program (P131013)  The benefit streams considered by the ex-post cost-benefit analysis were the activities under components 2 and 3. The estimated incremental value of production and income realized by project beneficiaries as a result of taking up innovations developed and made available by the project or as the result of having acquired new skills from participating in training activities financed by the project.  The cost-benefit analysis included both financial analysis carried out using actual market prices inclusive of taxes, subsidies, and other distortions, as well as economic analysis carried out using economic prices adjusted in some instances to correct for taxes, subsidies, and other distortions known to be present in the Peruvian economy.  Sensitivity analysis showed that the IRRs were somewhat sensitive to changes in technology adoption rates, production costs and producer revenues. The IRRs were relatively insensitive to changes in other parameters including the amount of training provided to extensionists and researchers. Accelerating the rate of adoption of agricultural technologies validated and made available by the project was shown to lead to significantly higher returns. Administrative and Institutional Efficiency  The project closed on February 26, 2021 which was 18 months beyond the expected closing date on September 1, 2019. According to the ICR (paragraph 23) "an extension of the project closing date was needed to make up implementation time that had been lost due to the 13-month delay in project effectiveness," and "to provide sufficient time for completion of final project activities and the recruitment of a consultant to carry out the final impact evaluation following procurement delays caused by the COVID-19 crisis."  At Project closing, the total cost of component 4 (Project Management) was US$7.14 million. This was double the US$3.57 million estimated at appraisal. According to the ICR (paragraph 64) "project management costs exceeded projections for two main reasons: First, the two extensions of the project closing date meant that PIU staff salaries and operational costs were incurred for 18 months longer than expected. Second, when it became apparent that many collaborative sub-projects were experiencing implementation delays, including due to lack of technical knowledge and/or lack of experience with fiduciary procedures, additional short-term consultants were hired to provide hands-on technical assistance, support administrative processes, and ensure thorough documentation of results." Assessment of Efficiency The ex-post EFA showed that project generated attractive returns on investment with an EIRR of 31.2% at completion compared to 26.2% at appraisal. Also, the sensitivity analysis suggested these returns could be considered robust under a range of plausible scenarios. While the 18 months delay was expected to delay the benefits streams of the project, this was not expected to significantly lower the ex-post EIRR. In fact the ICR (Annex 4) noted that "for all of the benefit streams, the timing of costs and benefits was adjusted to reflect the duration of the sub-project or activity implementation period and the expected distribution through time of the benefits." The increase in the cost of the project management was clearly explained by the ICR- as noted above and seems plausible. On this basis, Efficiency is rated Substantial. Efficiency Rating Substantial Page 13 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PE-National Ag. Innovation Program (P131013) a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 0 Appraisal  26.20  Not Applicable 0 ICR Estimate  31.20  Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome Relevance of Objectives was rated Substantial. Overall Efficacy was rated Substantial. The project almost achieved its intended development objectives with all three PDO Outcome Indicators exceeded, 10 Intermediate Results Indicators exceeded, and two Intermediate Results Indicators substantially achieved. Efficiency was rated Substantial with an EIRR of 31.2% at completion compared to 26.2% at appraisal. Based on the assigned ratings for the three criteria (Relevance of Objectives, Overall Efficacy and Efficiency), Outcome is rated Satisfactory. a. Outcome Rating Satisfactory 7. Risk to Development Outcome The following risks were discussed in the ICR that could potentially impact the development outcome: 1. The risk that the progress achieved in strengthening INIA and the SNIA could be undermined if the government fails to follow through with policy reforms and additional supporting investments. According to the ICR (paragraph 100) "PNIA was viewed as the first phase of what was expected to be a multi-phase effort, and if the current government decides to pursue a different innovation strategy, the sustainability of PNIA progress may be at risk." 2. The risk related to uncertain macroeconomic prospects. The COVID-19 pandemic hit Peru extremely hard and exacted a heavy toll on lives and livelihoods (ICR, paragraph 102). With the country facing a deep recession and a period of fiscal austerity, investment in agriculture innovation will need to be strengthened to sustain the positive achievements of the project. Page 14 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PE-National Ag. Innovation Program (P131013) 8. Assessment of Bank Performance a. Quality-at-Entry The PDO was in line with the Government and Bank priorities for the agriculture sector as discussed above in section 3. The Peruvian government was pursuing a two pronged approach in which INIA’s capacity to do research as well as the functioning of the agricultural innovation system would be improved. The Ministry of Agriculture established the National Agricultural Innovation Program (PNIA). PNIA was made up of two main projects: a project to create adequate conditions for agricultural innovation system (PSNIA-funded by the Bank), evolving from the earlier INCAGRO program; and a project to improve the strategic research services of INIA (PINIA-funded by the Inter-American Development Bank). The design of PSNIA was largely based on earlier experiences in Peru with the INCAGRO program and its lessons. Notable lessons reflected in the project design included: a decentralized implementation structure to ensure even coverage across the country and to reach marginalized farm communities, and establishing the Technical Secretariat in INIA to ensure institutional sustainability of the agricultural innovation system (PAD, paragraph 33). Implementation arrangements ensured a strong presence at regional and sub-regional level (ICR, paragraph 95). Design also featured a competitive grants approach to ensure that the funded activities were responsive to the demands of the intended beneficiaries. However, the implementation readiness was negatively impacted by political changes in Peru, which resulted in delayed effectiveness until May 2015 (ICR, paragraph 75). Also, procurement arrangements were weak and resulted in delays in the recruitment of firms and consultants needed for different project activities (ICR, paragraph 75). Nine risks were flagged at appraisal with an overall risk rating of Moderate. The risks related to three main areas: project risks, implementing agency risks, and stakeholder risk. The ICR (paragraph 96) reported that "close implementation support to sub-projects was underestimated, with the result that this function was underbudgeted." Also, as noted above, procurement arrangements were weak and might have been overlooked as a potential risk at appraisal. Also, the Bank underestimated the challenges associated with implementing institutional reforms. The ICR did not explicitly discuss risks that materialized during implementation nor did it report on the appropriateness of the mitigation measures mentioned in the PAD. M&E suffered from design weaknesses (see section 9 a for further details). The Results Framework (RF) included PDO Indicators that were "imperfectly aligned with the elements of the PDO, some performance indicators were at output level, and some were not precisely defined (ICR, paragraph 96)." Based on the above-mentioned assessment, Quality at Entry is rated Moderately Satisfactory due to moderate shortcomings related to implementation readiness, assessment of risks, and M&E design. Quality-at-Entry Rating Moderately Satisfactory Page 15 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PE-National Ag. Innovation Program (P131013) b. Quality of supervision  The project was implemented in a period of political instability in Peru. During the six-year implementation period, the project was overseen by nine Ministers of Agriculture, five Heads of INIA, and four Executive Directors of PNIA, and according to the ICR (paragraph 75) "each of whom came with a slightly different vision for the project."  The Bank task team organized 12 supervision missions, which included field visits once the first round of sub-projects was approved (ICR, paragraph 97). According to the CR (paragraph 98) "the Bank task team effectively supported the PIU in identifying and anticipating emerging challenges and in finding effective solutions."  The project implementation benefited from continuity in leadership as the project had only two Task Team Leaders (TTL); one who led preparation, and a second who supported the project throughout the 7.5 years of implementation. Implementation also benefited from the close and continuous support to the PIU by the multi-disciplinary task team (including a co-TTL) who were based in Lima.  The Bank maintained good rapport with the Inter American Development Bank (IADB). According to the ICR, paragraph 97) "the Bank team maintained an effective working relationship with the IADB task team implementing the PSNIA sister project, including carrying out several joint supervision missions."  The team restructured the project three times to extend the closing date and reallocate loan proceeds to accommodate internal and external circumstances that the project experienced during implementation. The high turnover in the PIU staff was a challenge that was counterbalanced by the stability in the project leadership on the Bank side.  Supervision efforts could have provided more attention to safeguards. According to the ICR (paragraph 98) "the task team did not sufficiently convey the implications of the ESMF to the PIU, the first round of sub-projects was launched without ESMPs for a small number of sub-projects in which they were required." This problem was swiftly resolved without an adverse environmental or social impact occurring (ICR, paragraph 98). Finally, the Bank team could have addresses M&E design weaknesses relating to the RF at an earlier stage of implementation through a formal restructuring that included a comprehensive revision of the RF. Quality of Supervision is rated Satisfactory. The Bank team led the project through a challenging political environment and managed to achieve the desired outcomes despite the implementation challenges. Based on the assigned ratings for QAE and Quality of Supervision, the Overall, Bank Performance is rated Moderately Satisfactory. Quality of Supervision Rating Moderately Satisfactory Overall Bank Performance Rating Moderately Satisfactory Page 16 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PE-National Ag. Innovation Program (P131013) 9. M&E Design, Implementation, & Utilization a. M&E Design  The Project Appraisal Document (PAD) did not require a Theory of Change (ToC) or results chain. Nevertheless, the ICR included an ex-post ToC that was constructed based on the PDO and the results indicators of the PAD. Overall, the ToC in the ICR was sound and reflected the relation between the project inputs, outputs, outcomes and long-term outcomes.  The statement of objectives was complicated and lacked clarity. Specifically, the term “adequate conditions in the Borrower’s SNIA” was problematic for assessing the PDO, since the basis for assessing the project’s outcomes in relation to "adequate conditions" was not clearly defined.  The PDO was to be assessed through three PDO indicators. Based on the parsing of the PDO, objective #1 was to be assessed through one PDO level outcome indicator "Institutions participating in the SNIA that report satisfaction with the effectiveness and cooperation in the system." Objective #2 was to be assessed through two PDO level indicators: "small and medium sized farmers adopting new technologies promoted by subprojects, including: not less than 35% were women, and not less than 20% were self-identified as indigenous people" and "new technologies emerging from research projects that were demonstrated on farm." The ICR (paragraph 77) correctly pointed out that "the PDO Indicators were imperfectly aligned with the key elements of the PDO." This required the ICR to consider other elements of the project to provide a comprehensive assessment of the objectives.  The Results Framework (RF) included twelve intermediate results indicators. According to the ICR (paragraph 77) "several indicators were proved difficult to measure initially because key terms were not closely defined, which complicated M&E activities until the indicators were adjusted during implementation." Also, the baselines for some indicators had to be constructed retroactively during implementation because they lacked robust baselines prior to implementation (ICR, paragraph 77).  Overall, M&E suffered from notable weaknesses related mainly to the RF. These had to be addressed during implementation to accurately and comprehensively assess the achievement of the project outcomes. b. M&E Implementation  M&E implementation benefited from establishing a system for collecting, verifying, and reporting M&E data that was comprehensive and internally consistent (ICR, paragraph 77). However, initial results for some activities were not reported to the Bank until mid-2016, more than one year after effectiveness.  A web-based M&E system, conceptualized as a module was established as a part of the larger project management information system. However, the development of the M&E system took longer than expected because the M&E systems developed under earlier projects such as INCAGRO were outdated. Also, The ICR (paragraph 78) noted that the development of the M&E system was also partially delayed due to "the difficulty of identifying a competent software development firm."  Establishing the web-based M&E system provided the opportunity for external users to access information about the project. Page 17 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PE-National Ag. Innovation Program (P131013)  Revisions to the RF. The ICR explained that "the adjustments were minor and designed to facilitate measurement of selected indicators, so a formal restructuring of the Results Framework was deemed unnecessary." This Review finds that the RF could have benefitted from a formal restructuring given the notable shortcomings as mentioned above. c. M&E Utilization  According to the ICR (paragraph 79) the "project management made effective use of the M&E data throughout implementation." Project management monitored the progress realized against agreed milestones and attended to lagging activities.  According to the ICR (paragraph 80), the M&E system was used in combination with other management information systems to: "monitor progress in the implementation of sub-projects, ensure the orderly closing of sub-projects once they were completed, and carry out project management functions (administration, personnel management, financial management, procurement)."  The management of sub-projects financed through the competitive grant program benefited from the provision of real-time view of the results through the M&E system. This allowed adjustments to be made during successive rounds of competitive grant funding to improve the targeting (ICR, paragraph 79). For example, imbalances in the distribution of competitive grant awards across regions or among commodities was corrected through adjustments to the eligibility criteria and scoring weights. This allowed better alignment of the allocation of competitive grant resources to the project development objective.  The ICR noted that the efficiency analysis at completion relied on data generated by the project's M&E system. Overall, the Quality of M&E is rated Modest. M&E design suffered notable shortcomings relating to the RF. These shortcomings were not fully addressed during implementation. The assessment of the project outcomes was mainly based on elements defined at completion rather than through a full revision of the RF via a formal restructuring. Finally, utilization was adequate and the project data was used to effectively make relevant management decisions. M&E Quality Rating Modest 10. Other Issues a. Safeguards The project was classified as a Category B, since its components were found to have relatively limited environmental impact potential. It triggered the following five safeguard policies: Environmental Assessment (OP/BP 4.01); Natural Habitats (OP/BP 4.04); Forests (OP/BP4.36); Pest Management (OP4.09); and Indigenous Peoples (OP/BP 4.10). According to the (PAD, paragraph 66) the project was expected to have an overall positive impact on the environment. An Environmental Management Framework (EMF) was prepared by the Borrower, reviewed by the Bank and disclosed by October 8, 2013 in country and by Page 18 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PE-National Ag. Innovation Program (P131013) October 29, 2013 on the Bank’s website. An Environmental and Social Management Framework (ESMF) and Indigenous Peoples Plan (IPP) were prepared and disclosed on October 29, 2013. The potential social risks of the project were assessed to be limited (ICR, paragraph 89). Compliance with Environmental Safeguards. According to the ICR (paragraph 88) "the project was compliant with environmental safeguards." The final supervision mission confirmed that 100% of the sub- projects were regularly reporting the required environmental performance indicators in the system (ICR, paragraph 85). The ICR (paragraph 87) pointed out two environmental safeguards compliance issues that required attention by the PIU: first, the possible location of several sub-projects within national protected areas (NPAs), and second, the planting by some of these sub-projects of an invasive tree species (Pinus radiata). Further investigation of those two issues revealed that the sub-projects were located outside the NPAs, but within their buffer zones, and the corresponding NPA authorities were informed and were actively engaged in ensuring that sub-project activities were compliant with all applicable regulations. Also, the sub- projects involved in the planting of Pinus radiata trees were doing so only in locations with a historical presence of this species, so no negative environmental impacts were expected to result (ICR, paragraph 87). Compliance with Social Safeguards. According to the ICR (paragraph 91) "the project was compliant with social safeguards." After an initial delay, social safeguards were monitored continuously until project closing. The first set of sub-project proposals were not screened properly. This was corrected by screened the first set retroactively as required. According to the ICR (paragraph 90) the retroactive screening revealed that "no adverse social risks were identified." A project-specific Grievance Redress Mechanism (GRM) was established and the resolution of complaints was monitored and reported to the Bank supervision teams. The ICR did report on the number of complaints received by the GRM. b. Fiduciary Compliance Financial Management (FM). FM risk was rated Moderate. While the project financial management system benefited from the fiduciary systems that were developed under the earlier INCAGRO projects, the project's FM struggled to manage and control the fund flows to sub-projects. This resulted in some initial delays in disbursement of funds to sub-projects. The PIU made changes to FM procedures that facilitated fund flows to sub-projects for subsequent rounds of sub-projects and for the scholarship program. The Bank and IADB agreed to joint auditing arrangements for the project, which were conducted following IADB policies and procedures (ICR paragraph 93). According to the ICR (paragraph 93) "the auditors issued unqualified (clean) opinions on the financial statements of the project." Procurement. Procurement risk was rated Substantial. The project implementation unit (PIU) procurement staff benefited from regular formal and informal training provided by the Bank procurement specialist. Also, the PIU procurement staff regularly participated in Bank procurement training sessions offered to project implementing entities. The project experienced procurement delays related to the recruitment of firms to conduct the Impact Evaluation Baseline Study and Mid-term evaluation. While these delays slowed implementation progress and delayed the MTR, they did not materially affect the overall performance of the project (ICR, paragraph 94). Page 19 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PE-National Ag. Innovation Program (P131013) c. Unintended impacts (Positive or Negative) --- d. Other --- 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment Outcome Satisfactory Satisfactory Quality at Entry suffered from moderate shortcomings related Bank Performance Satisfactory Moderately Satisfactory to implementation readiness, assessment of risks, and M&E design. The RF suffered from shortcomings particularly at the Quality of M&E Substantial Modest level of the PDO outcome indicators. Quality of ICR --- Substantial 12. Lessons The ICR included four lessons. The following two lessons are emphasized with some adaptation of language: 1. Competitive grants can be used as an effective tool for directing resources to promote innovation in response to user demand. This will require measures in place to ensure the technical quality of the activities that are funded, and provided that the allocation of resources can be protected from elite capture. The project experience demonstrated that the competitive grant program was successful in directing resources toward demand-driven innovation needs and leveraging matching resources from beneficiaries. 2. To ensure the successful outcome of institutional reforms, project preparation activities must carefully assess the political economy considerations to properly calibrate the risks associated with proposed institutional reform efforts. The Bank underestimated the challenges associated with implementing institutional reforms. While the project made significant progress in strengthening INIA, yet the institutional reforms achieved under the project fell short of expectations. The planned institutional reforms of INIA did not go as far as expected because of resistance by powerful interest groups (especially the labor unions operating in the sector) that the authorities were unwilling or unable to confront. Page 20 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PE-National Ag. Innovation Program (P131013) 13. Assessment Recommended? No 14. Comments on Quality of ICR Quality of Evidence. The ICR acknowledged that the Results Framework suffered from weaknesses (see section 9a). In addition to the RF, the ICR relied on complementary information from the independent impact evaluation, the Borrower’s ICR, and the project databases to fully assess the project outcomes. Quality of Analysis. The ICR provided clear linking to the extent possible between evidence and findings and used the evidence base to serve the arguments under the different sections, in particular the discussion on outcomes. Lessons. Lessons reflected the project experience and were based on evidence and analysis. Results Orientation. The ICR included a comprehensive discussion on the achievement of the PDO. The ICR discussion was balanced between reporting on the achievement of outcome indicators and what the project actually achieved on the ground. Consistency with guidelines. The ICR successfully used the available data to justify most of the assigned ratings. Discussion of outcomes was adequate. The efficiency analysis was robust enough and provided a convincing justification of the project investments. Conciseness. The ICR provided comprehensive coverage of the implementation experience and candidly reported on shortcomings. The reporting on safeguards was brief and did not include an explicit statement on compliance. Also, the ICR did not report on the status of the final audit reports for the project. However, the discussion of M&E design could have benefited from further details with regards to the RF shortcomings. Finally, the ICR did not explicitly discuss risks that materialized during implementation nor did it report on the appropriateness of the mitigation measures mentioned in the PAD. Overall, the Quality of the ICR is rated Substantial with only minor shortcomings a. Quality of ICR Rating Substantial Page 21 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PE-National Ag. Innovation Program (P131013) Page 22 of 22