Document of The World Bank FOR OFFICIAL USE ONLY Report No: ICR00005483 IMPLEMENTATION COMPLETION AND RESULTS REPORT IBRD-83310 ON A LOAN IN THE AMOUNT OF US$ 40 MILLION TO THE Ministry of Economy and Finance of Peru FOR THE National Agricultural Innovation System Support Project November 2, 2021 Agriculture and Food Global Practice Latin America and Caribbean Region CURRENCY EQUIVALENTS (Exchange Rate Effective {Feb 25, 2021}) Currency Unit = Peru Sol PEN 3.6478 = US$1 US$ 1.44665 = SDR 1 FISCAL YEAR July 1 - June 30 Regional Vice President: Carlos Felipe Jaramillo Country Director: Marianne Fay Regional Director: Anna Wellenstein Practice Manager: Preeti S. Ahuja Task Team Leader(s): Michael Morris, Griselle Felicita Vega ICR Main Contributor: Michael Morris, Griselle Felicita Vega ABBREVIATIONS AND ACRONYMS National Council of Science and Technology CONCYTEC (Consejo Nacional de Ciencia y Tecnología) National Commission for Innovation and Training in Agriculture CONICA (Comisión Nacional de Innovación y Capacitación Agraria) CPF Country Partnership Framework CPS Country Partnership Strategy DiD Difference in Difference EIRR Economic Internal Rate of Return EMF Environmental Management Framework IADB Inter-American Development Bank ICR Implementation Completion Report Innovation and Competitiveness Program INCAGRO (Proyecto de Innovación y Competitividad para el Agro) National Agricultural Innovation Institute INIA (Instituto Nacional de Innovación Agraria) ISR Implementation Status and Results M&E Monitoring and Evaluation MEF Ministry of Economy and Finance (Ministerio de Economía y Finanzas) Ministry of Agricultural Development and Irrigation MIDAGRI (Ministerio de Desarrollo Agrario y Riego) Ministry of Agriculture and Irrigation MINAGRI (Ministerio de Agricultura y Riego) MTR Mid-term Review NPAs National Protected Areas PAD Project Appraisal Document PDO Project Development Objective Project for the Improvement of the National Agricultural Innovation PINIA Program’s Agricultural Innovation Strategic Services (IADB-funded) Program for Innovation and Agricultural Services PISA (Programa de Innovación y Servicios Agropecuarios) PIU Project Implementation Unit National Agriculture Innovation Program PNIA (Programa Nacional de Innovación Agraria) PSNIA National Agricultural Innovation System Support Project (WB-funded) Information System for Monitoring and Evaluation SISEV (Sistema Informático de Seguimiento y Evaluación) National Agricultural Innovation System SNIA (Sistema Nacional de Innovación Agraria) TF Trust Fund WBG World Bank Group TABLE OF CONTENTS DATA SHEET ............................................................................................................................ I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES ....................................................... 1 A. CONTEXT AT APPRAISAL .........................................................................................................1 B. SIGNIFICANT CHANGES DURING IMPLEMENTATION ................................................................8 II. OUTCOME ...................................................................................................................... 8 A. RELEVANCE OF PDOs ..............................................................................................................8 B. ACHIEVEMENT OF PDOs (EFFICACY) ........................................................................................9 C. EFFICIENCY ...........................................................................................................................18 D. JUSTIFICATION OF OVERALL OUTCOME RATING ....................................................................20 E. OTHER OUTCOMES AND IMPACTS .........................................................................................20 III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME ................................ 21 A. KEY FACTORS DURING PREPARATION ...................................................................................21 B. KEY FACTORS DURING IMPLEMENTATION .............................................................................22 IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME .. 23 A. QUALITY OF MONITORING AND EVALUATION (M&E) ............................................................23 B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE .....................................................24 C. BANK PERFORMANCE ...........................................................................................................27 D. RISK TO DEVELOPMENT OUTCOME .......................................................................................28 V. LESSONS AND RECOMMENDATIONS ............................................................................. 29 ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS ........................................................... 31 ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION ......................... 48 ANNEX 3. PROJECT COST BY COMPONENT ........................................................................... 50 ANNEX 4. EFFICIENCY ANALYSIS ........................................................................................... 51 ANNEX 5. BORROWER, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS ... 62 ANNEX 6. SUPPORTING DOCUMENTS .................................................................................. 65 ANNEX 7. PROJECT IMPACT EVALUATION SUMMARY........................................................... 66 ANNEX 8. TYPES OF COLLABORATIVE SUB-PROJECTS FINANCED ........................................... 71 ANNEX 9. CASE STUDY: COLLABORATIVE SUB-PROJECT ........................................................ 72 The World Bank National Agricultural Innovation (P131013) DATA SHEET BASIC INFORMATION Product Information Project ID Project Name P131013 National Agricultural Innovation Country Financing Instrument Peru Investment Project Financing Original EA Category Revised EA Category Partial Assessment (B) Partial Assessment (B) Organizations Borrower Implementing Agency Instituto Nacional de Innovacion Agraria (INIA), Ministry Ministry of Economy and Finance of Agriculture and Irrigation, Office of Programming and Investments Project Development Objective (PDO) Original PDO To create the adequate conditions in the Borrower's SNIA in order to support the effectiveness of its member organizations in providing or developing improved agricultural technologies. The World Bank National Agricultural Innovation (P131013) FINANCING Original Amount (US$) Revised Amount (US$) Actual Disbursed (US$) World Bank Financing 350,000 336,670 336,670 TF-12180 40,000,000 38,920,492 38,920,492 IBRD-83310 Total 40,350,000 39,257,162 39,257,162 Non-World Bank Financing 0 0 0 Borrower/Recipient 56,953,930 56,953,930 37,374,747 Inter-American 1,000,000 0 0 Development Bank Local Farmer Organizations 30,750,000 30,750,000 24,139,047 Total 88,703,930 87,703,930 61,513,794 Total Project Cost 129,053,930 126,961,092 100,770,956 KEY DATES Approval Effectiveness MTR Review Original Closing Actual Closing 17-Dec-2013 07-Sep-2012 19-Sep-2017 01-Sep-2019 26-Feb-2021 RESTRUCTURING AND/OR ADDITIONAL FINANCING Date(s) Amount Disbursed (US$M) Key Revisions 09-Oct-2018 32.17 Change in Loan Closing Date(s) 18-May-2020 38.53 Reallocation between Disbursement Categories 18-Aug-2020 39.53 Change in Loan Closing Date(s) KEY RATINGS Outcome Bank Performance M&E Quality Satisfactory Satisfactory Substantial The World Bank National Agricultural Innovation (P131013) RATINGS OF PROJECT PERFORMANCE IN ISRs Actual No. Date ISR Archived DO Rating IP Rating Disbursements (US$M) 01 11-Apr-2014 Satisfactory Satisfactory .33 02 07-Nov-2014 Satisfactory Satisfactory .34 03 26-May-2015 Satisfactory Moderately Satisfactory .34 04 31-Dec-2015 Satisfactory Moderately Satisfactory 1.64 05 26-Jun-2016 Satisfactory Moderately Satisfactory 5.70 06 13-Dec-2016 Satisfactory Moderately Satisfactory 6.14 07 13-Jun-2017 Moderately Satisfactory Moderately Satisfactory 8.41 08 24-Dec-2017 Moderately Satisfactory Moderately Satisfactory 16.76 09 06-Jun-2018 Moderately Satisfactory Moderately Satisfactory 22.56 10 14-Nov-2018 Moderately Satisfactory Moderately Satisfactory 32.17 11 24-Apr-2019 Moderately Satisfactory Moderately Satisfactory 36.44 12 23-Oct-2019 Moderately Satisfactory Moderately Satisfactory 38.53 13 14-Mar-2020 Moderately Satisfactory Moderately Satisfactory 38.53 14 29-Sep-2020 Satisfactory Moderately Satisfactory 39.53 SECTORS AND THEMES Sectors Major Sector/Sector (%) Agriculture, Fishing and Forestry 100 Agricultural Extension, Research, and Other Support 97 Activities Public Administration - Agriculture, Fishing & Forestry 3 Themes Major Theme/ Theme (Level 2)/ Theme (Level 3) (%) The World Bank National Agricultural Innovation (P131013) Private Sector Development 100 Jobs 100 Finance 15 Finance for Development 15 Agriculture Finance 15 Social Development and Protection 10 Social Inclusion 10 Indigenous People and Ethnic Minorities 10 Human Development and Gender 10 Gender 10 Urban and Rural Development 65 Rural Development 65 Rural Markets 25 Rural Infrastructure and service delivery 40 Environment and Natural Resource Management 32 Climate change 32 Adaptation 32 ADM STAFF Role At Approval At ICR Regional Vice President: Hasan A. Tuluy Carlos Felipe Jaramillo Country Director: Susan G. Goldmark Marianne Fay Director: Juergen Voegele Anna Wellenstein Practice Manager: Laurent Msellati Preeti S. Ahuja Michael Morris, Griselle Felicita Task Team Leader(s): Michael Morris Vega ICR Contributing Author: Michael Morris The World Bank National Agricultural Innovation (P131013) I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES A. CONTEXT AT APPRAISAL Context 1. In 2013 when the National Agricultural Innovation System Support Project was appraised, Peru was emerging as one of the fastest growing and most stable economies in Latin America. Having benefited for nearly a decade from prudent macroeconomic policies, inefficiency-reducing policy reforms, positive terms of trade, and strong direct foreign investment flows, the Peruvian economy had almost doubled in size, and gross domestic product (GDP) was growing at an average annual rate of 6.3 percent, the highest 10-year average growth in Peru’s history. At the same time, significant headway had been made in reducing poverty and curbing child malnutrition. Moderate poverty fell from 45.5 percent in 2007 to 19.3 percent in 2011, while extreme poverty dropped from 42 percent in 2004 to 24 percent in 2010. 2. Peru’s economic progress has been impressive, but the fruits of that progress have been distributed unevenly. Income gains and improvements in human development indicators have been concentrated in the country’s fast-growing towns and cities; meanwhile, much of the countryside has been left behind. The urban- rural poverty gap has widened over time, and today poverty is associated mostly with rural areas, where 60 percent of the poor reside. Consistent with higher levels of poverty in rural areas, human development indicators are lower; for example, child mortality and child malnutrition rates are about twice as high in rural as in urban areas, and educational outcomes are significantly lower. 3. The agriculture sector in Peru has enormous potential to drive future growth and further reduce poverty. Because poverty in Peru is concentrated in rural areas, and because most rural households depend on agriculture as a major livelihood source, growth originating in the agriculture sector can be particularly effective in reducing poverty. To some extent, this is already happening: over the past 10 years, Peru’s agriculture sector has grown at an average annual rate of 3.3 percent, contributing around 7.0 percent to GDP on average, and accounting for between 15 and 20 percent of total exports. But while the sector’s contribution to growth has been robust, the impacts on poverty have been more limited. The strong performance of the agriculture sector has been driven mostly by the rapid expansion of the agro-export industry that is concentrated along the Pacific coast (Costa region). Agricultural growth has been much more modest in the Andean highlands (Sierra region) and the Amazonian lowlands (Selva region). 4. Unlocking the potential of Peruvian agriculture will not be possible unless several challenges can be overcome. Land and labor productivity levels are very low, especially in the Sierra and Selva regions, and yields of many crops are low compared to the yields being achieved in other Latin American countries. Many producers work small and fragmented production plots with fragile soils, often plagued with salinity problems and subject to erosion. Management of water resources is a perennial problem—either scarcity of water due to competition with urban water use (especially in the Costa region) or excess of water due to inadequate drainage (especially in the Selva region). Climate change is blamed for the increasing frequency and severity of extreme weather events that impact agricultural performance. The El Niño phenomenon is particularly influential in Peru, as it affects the frequency of frosts and heat waves, as well as the severity of droughts and floods, impacting food crops such as maize, potato, and wheat. 5. Mixed success with agricultural innovation and insufficient adoption of improved technology has exacerbated these challenges and significantly limited agricultural productivity growth. In 1978, the National Agricultural Innovation Institute (Instituto Nacional de Innovación Agraria; INIA) was established to support Page 1 of 72 The World Bank National Agricultural Innovation (P131013) agricultural research, technology transfer, technical assistance, conservation of genetic resources, breeding of species of high genetic value, and production of seed. After achieving some initial successes in the 1980s and 1990s, INIA lost significance, weakened by falling budgets and constant reorganizations. In 2008, the Government attempted to reinvigorate INIA; through Legal Decree 1060, INIA was tasked with leading the National Agricultural Innovation System (Sistema Nacional de Innovación Agraria; SNIA) under the guidance of a newly established National Commission for Innovation and Training in Agriculture (Comisión Nacional de Innovación y Capacitación Agraria; CONICA). The high expectations that accompanied the launch of INIA unfortunately were not borne out, however, and in the years that followed, INIA made little progress on advancing its ambitious mandate. 6. The absence of a high-performing national agricultural innovation system was identified as a major constraint limiting agricultural growth in Peru. Diagnostic assessments performed as part of project preparation showed that a critical factor holding back the potential of the agriculture sector was lack of innovation. The spread of improved agricultural technology depended critically on a well-functioning SNIA, which includes INIA as the lead public agency, private and public universities, research centers, private sector agribusiness, public and private suppliers of agricultural advisory services, and producer organizations. Several problems with the SNIA were identified during preparation: (i) Low levels of investment: During the first decade of the 2000s, public spending on research and technology transfer in Peru was about 0.17 percent of agricultural GDP, well below the average for Latin America (1.14 percent). (ii) Weak institutional environment: The SNIA was poorly articulated, had little capacity to respond to the demands of its clients, and experienced chronic difficulties in establishing priorities and concentrating efforts on strategic topics. (iii) Ineffective leadership: INIA, the agency tasked with leading the SNIA, was operating from inadequate and obsolete infrastructure, lacked sufficient qualified researchers and administrators, and was using outdated and ineffective policies and procedures. 7. The National Agricultural Innovation Program (Programa Nacional de Innovación Agraria; PNIA) was created by the Ministry of Agricultural Development and Irrigation (MIDAGRI) with the dual objectives of strengthening the SNIA and improving the capacity of INIA to carry out research. 1 PNIA was to be implemented through two donor-supported projects using parallel funding: (i) a World Bank-supported project to consolidate the SNIA (PSNIA), and (ii) an Inter-American Development Bank-supported project to improve the strategic research services of INIA (PINIA). The relationship between PNIA, PSNIA, and PNIA is depicted in Figure 1. By incorporating the two donor-funded projects within a single government program, MINAGRI hoped to increase the level of overall funding available for PNIA, achieve better coordination of the support it was receiving from the development banks, and reduce fragmentation within its own organizational structure. 1 Until 2020, the Ministry was called the Ministry of Agriculture and Irrigation (Ministerio de Agricultura y Riego - MINAGRI). Page 2 of 72 The World Bank National Agricultural Innovation (P131013) Figure 1. Relationship between PNIA, PSNIA, and PINIA 8. PSNIA builds on the experience of the earlier World Bank-supported Agricultural Innovation and Competitiveness Program for Peruvian Agriculture (Programa de Innovación y Competitividad para el Agro Peruano; INCAGRO), which between 2002 and 2010 provided competitive grants to strengthen strategic and adaptive agricultural research, as well as technical assistance and capacity building. Although successful at bringing together research institutes, universities, non-governmental organizations (NGOs), producer organizations and private agribusiness firms, INCAGRO was not sufficiently institutionalized in MINAGRI or INIA to attract recurring government funding following the ending of the IBRD-funded projects and therefore did not have a lasting impact on the strengthening of the overall innovation system. 9. Rationale for World Bank Support. Through INCAGRO, the World Bank had served as a long-term strategic partner supporting Peru’s agricultural innovation system. The Bank was able to draw from the lessons learned from the INCAGRO project, particularly regarding the use of competitive grants to direct funding to end- user needs, using rigorous technical criteria to evaluate proposals and maintaining transparency to protect against elite capture. The Bank also was able to bring specific experience from similar interventions in other countries in Latin America and other parts of the world. Theory of Change 10. Figure 2 illustrates the Theory of Change (ToC) as derived from the PAD based on the Project’s objectives, indicators, activities, and institutional arrangements. PSNIA was designed to address three key problems facing the SNIA at the time of appraisal: (i) the weak institutional capacity of INIA, (ii) the insufficient level of support for knowledge generation and dissemination of innovations, and (iii) the low level of adoption of innovations among farmers, including women and indigenous people. Page 3 of 72 The World Bank National Agricultural Innovation (P131013) Figure 2. PSNIA Theory of Change Page 4 of 72 The World Bank National Agricultural Innovation (P131013) Project Development Objective 11. The Project Development Objective (PDO) was “to create the adequate conditions in the Borrower's SNIA in order to support the effectiveness of its member organizations in providing or developing improved agricultural technologies.” Key Expected Outcomes and Outcome Indicators 12. The PDO-level results indicators were the following: PDO 1: Institutions participating in the SNIA that report satisfaction with the effectiveness and cooperation in the system, measured by an independent party PDO 2: Farmers adopting new technologies promoted by sub-projects PDO 2 (a): Farmers adopting new technologies promoted by sub-projects – women PDO 2 (b): Farmers adopting new technologies promoted by sub-projects – self-identified indigenous PDO 3: New technologies emerging from research projects that have been demonstrated on farm 13. Targeted beneficiaries: The Project aimed to reach public institutions such as INIA, MINAGRI, and the National Council of Science and Technology (CONCYTEC); regional and local governments; private firms; research institutions, universities, and agricultural advisory service providers, as well as farmers and community-based organizations, NGOs, indigenous people, and women-based organizations. Project Components 14. As designed, the Project consisted of four components. Component 1: Strengthening the capacity of INIA to lead the SNIA. Appraisal estimate: US$23.34 million, of which IBRD: US$2.59 million. Actual: US$ 23.34 million, of which IBRD: US$1.33 million 15. The objective of Component 1 was to strengthen INIA to assume its coordinating and leadership role within the SNIA. Prior to the Project, INIA had invested very little effort in seeking to strengthen the SNIA as a multi-actor national network; INIA had operated with a strong inward-looking orientation, focusing primarily on its own research programs. Component 1 was to do this through: (a) Establishment of SNIA’s Technical Secretariat. A Technical Secretariat was to be established to support INIA’s role as the lead government agency for the SNIA. The Secretariat would manage planning, budgeting, evaluating, and reporting on activities related to the SNIA in consultation and under the final supervision of the Project Steering Committee. The Secretariat was to manage four units: (1) The Policy Support Unit would draft the national strategy for agricultural innovation and the research and innovation policies to create an enabling environment for agricultural innovation. (2) The Knowledge Management Unit would collect, store and disseminate information related to technological innovation, and support knowledge management within the SNIA. (3) The Market for Innovation Services Unit would manage all the competitive funds and through its decentralized offices (DOs) would provide support to regional innovation processes. (4) The Stakeholder Relationship Unit would coordinate the relations with the private sector, producer organizations and the regional governments, and would establish formal mechanisms of coordination, including the development of local innovation networks. Page 5 of 72 The World Bank National Agricultural Innovation (P131013) (b) Establishment of an INIA Innovation Awards Program. A national agricultural innovation awards program was to be established within INIA to recognize excellence in agricultural innovation and highlight INIA’s role as leader of the national innovation system. Awards were to be granted annually at national innovation fairs organized by the Project. The PNIA PIU was the organizing entity within INIA of the innovation awards program and was responsible for managing calls for nominations, recruiting independent experts to evaluate the nominations and select winners, and organizing major ceremonies during to honor the winning entries and award prizes. Component 2: Consolidating the Market for Innovation Services. Appraisal estimate: US$57.91 million, of which IBRD: US$24.16 million. Actual: US$57.81 million, of which IBRD: US$ 23.72 million 16. The objective of Component 2 was to support the development of a market for innovation services in Peru by setting up a competitive grant facility to co-finance what were called collaborative sub-projects. Collaborative sub-projects were to be implemented by coalitions of research organizations, agribusiness firms, civil society organizations, and producers and other value chain actors. The competitive grants facility contained windows for three types of collaborative sub-projects: (1) Adaptive Research Sub-projects focused on testing promising innovations and adapting them to local circumstances; (2) Extension and Technical Assistance Sub-projects focused on transferring to end-users innovations introduced from other locations and providing technical assistance to facilitate their uptake; and (3) Community Seed Enterprise Sub-projects were designed to help improve seed production and distribution of improved germplasm by forming or strengthening community- or producer-operated small-scale seed enterprises. 17. Collaborative sub-projects to be financed through the competitive grant program would be demand- driven based on regional priorities, inclusive with respect to women and vulnerable groups, and designed to build local capacity for sustained agricultural innovation. To promote collaboration and help build the SNIA, the eligibility criteria stated that proposals could be submitted only by coalitions of applicants working in partnership. To ensure commitment and leverage private-sector resources, sub-project beneficiaries would be required to match the grants provided through the Project with their own resources, with the percentage match set according to the type of sub-project and the characteristics of the beneficiaries. Sub-project proposals would be evaluated by panels of independent national and international experts based on clearly defined criteria. The selection process would ensure an equitable distribution of the funds among the regions of the country. Awareness campaigns would promote the availability of the competitive funds, especially among women and indigenous population groups. Component 3: Strategic Capacities in the SNIA. Appraisal estimate: US$40.31 million, of which IBRD: US$12.75 million. Actual: US$40.31 million, of which IBRD: US$13.46 million 18. The objective of Component 3 was to support the enhancement of the SNIA to facilitate innovation. This was to be done by using competitive grants to support three additional activities: (1) Strategic Research Sub-projects focused on priority themes defined by INIA and CONICA as strategically important for the country. Compared to the collaborative sub-projects, strategic sub-projects were to involve larger grants (up to US$500,000), be disbursed over longer periods (up to 3 years), and focus on more upstream research. To promote collaboration, strategic research sub-projects would also be implemented through partnerships, usually involving research institute, universities, and INIA’s own research programs. Page 6 of 72 The World Bank National Agricultural Innovation (P131013) (2) Capacity Building Sub-projects would provide training for extension service providers and individual extension agents. Capacity-building sub-projects were expected to be implemented by universities, vocational training centers, private firms, and civil society organizations specialized in training and capacity building. (3) Scholarships were to be awarded for postgraduate study and short-term internships. A Scholarship Fund was to be established within INIA to which eligible applicants would be able to apply for funding to support fully or partially graduate study in agriculture-related fields, as well as participation in shorter term internships and study tours, both individually and through groups. Component 4: Project Implementation. Appraisal estimate: US$3.57 million, of which IBRD: US$0.5 million. Actual: US$ 7.14 million, of which IBRD: US$0.41 million 19. The objective of Component 4 was to strengthen the capacity of INIA to ensure the successful implementation of project activities, including compliance with procurement, safeguards, financial management, and monitoring and evaluation (M&E) requirements. A Project Implementation Unit (PIU) was to be established within INIA to implement PNIA and its two constituent projects, PSNIA and PINIA. 20. Relationship between World Bank and IADB Projects: As mentioned above, the government’s PNIA was supported by two donor-supported projects using parallel funding: (i) a World Bank-supported project to consolidate the SNIA (PSNIA), and (ii) an IADB-supported project to improve the strategic research services of INIA (PINIA). When the two projects were being prepared, the Bank and IADB teams explored the possibility of co-financing a single combined project; this idea was eventually ruled out due to differing institutional requirements. Because the Bank-supported PSNIA Project had no formal relationship to the IADB-supported PINIA project, this ICR covers only the PSNIA Project. Nevertheless, the government viewed the two projects as a single PNIA. To ensure coherence with the Project Appraisal Document (PAD), loan agreement, and other Bank project documents, the components and sub-components referred to in this ICR are those of the PSNIA project, which are different from the components and sub-components of the government’s PNIA. The PNIA PIU managed the two bank-supported projects jointly while maintaining separate administrative, fiduciary, and reporting systems, so the arrangement did not pose major problems for the two banks. But it is important to note that PSNIA and PINIA did not have separate identities in the eyes of external audiences, who saw only a single government PNIA to which the two banks were contributing. 21. Project Financing: The total cost of PSNIA at appraisal, as reflected in the PAD, was US$125.2 million, to be financed through a US$40 million loan from IBRD, a contribution of US$54.5 million from the Government of Peru, and US$30.75 million in own contributions from subproject beneficiaries. A notional amount of US$1,000 in IADB financing was also included in the PAD as a placeholder, to create space for eventually merging the contributions from the two banks, but later when it became clear that the IADB contribution would go forward as parallel financing, this amount was removed. Overall expenditure at closing was US$100.7 million, of which US$39.2 million from IBRD 2 (98 percent of the appraisal estimate), US$37.4 million from the Government of Peru (69 percent of appraisal estimate), and US$24.1 million in own contributions from sub-project beneficiaries (78 percent of appraisal estimate). 2 Although not recorded in the PAD, the final IBRD contribution included US$ 336,670 from a Government of Spain Trust Fund provided as a project preparation grant and used to support some design work. The Effectiveness date shown in the ICR Data Sheet is the Effectiveness date for the grant, which preceded the Board Approval date for the IBRD loan. Page 7 of 72 The World Bank National Agricultural Innovation (P131013) B. SIGNIFICANT CHANGES DURING IMPLEMENTATION Revised PDOs and Outcome Targets: Neither the PDO nor the outcome targets were changed. Revised PDO Indicators: The PDO Indicators were not revised. Revised Components. The project components were not revised during implementation 22. Other Changes. The Project underwent three Level 2 restructurings. The first, in October 2018, authorized a 12-month extension of the closing date to September 1, 2020. The second, in May 2020, authorized a reallocation of credit proceeds. The third, in August 2020, authorized a 6-month extension of the closing date to February 26, 2021. Rationale for Changes and Their Implication on the Original Theory of Change 23. The rationales for the restructurings were as follows: • First restructuring: An extension of the project closing date was needed to make up implementation time that had been lost due to the 13-month delay in project effectiveness. The extension allowed: (i) completion of some collaborative sub-projects that had suffered implementation delays due to adverse weather conditions, and (ii) completion of activities to consolidate the SNIA being parallel financed by IADB (especially the rehabilitation and upgrading of 13 INIA experiment stations located throughout the country). • Second restructuring: The reallocation of loan proceeds allowed unused funds from the competitive grant program to be redirected (i) to strengthen the team charged with ensuring the satisfactory closing of innovation sub-projects (US$500,830 additional), and (ii) to cover a minor overrun in spending on capacity strengthening activities, especially the financing of international and national graduate study, internships, and study tours (US$480,480 additional). • Third restructuring: A second extension of the project closing date was needed to provide sufficient time for completion of final project activities and the recruitment of a consultant to carry out the final impact evaluation following procurement delays caused by the COVID-19 crisis. These restructurings had no effect on the Theory of Change. II. OUTCOME A. RELEVANCE OF PDOs Assessment of Relevance of PDOs and Rating 24. The Project was appraised and implemented partly under the World Bank Group’s (WBG) Country Partnership Strategy (CPS) for the Republic of Peru for the period FY12-FY16 (Report No. 66187-PE) and partly under the Country Partnership Framework (CPF) for the Republic of Peru for the period of FY17-FY21 (Report No. 112299-PE). 25. Under the CPS (FY12-FY16), the WBG focused on supporting the Government of Peru’s commitment to boost productivity and address environmental challenges as the basis for inclusive and sustainable development. The PDO related directly to CPS Strategic Priority (iii) Sustainable growth and productivity and to CPS Results Areas 3.1 Promoting Productivity through Enhanced Labor Skills and 3.2 Sustainable Rural Development and Water Resource Management. The Project objectives were closely aligned with the Government’s strategy supported by the CPS “to promote rural development in all regions of the country with emphasis on enhancing Page 8 of 72 The World Bank National Agricultural Innovation (P131013) productivity through innovation in poor regions” (CPS FY1-FY16). The CPS stated that the WBG would support MINAGRI in its strategy of strengthening the agricultural innovation system, which is what PSNIA set out to do. In addition, under the CPS there was a strong focus on scaling-up successful experiences of prior WBG operations in the country; the competitive grant program implemented under PSNIA was an example of a scaled-up activity, building on the success that had been achieved with the competitive grant components of the earlier INCAGRO project. 26. By project closing, the CPS (FY12-FY16) had been succeeded by the CPF (FY17-FY21), but the PDO remained highly relevant. The CPF under Pillar I: Productivity for Growth, Objective 3 emphasizes the importance of facilitating the absorption of skills and technology to increase productivity, and one of the expected CPF results is increasing the number of farmers that have adopted improved agricultural practices. Furthermore, the CPF under Pillar III: Strengthening the Management of Natural Resources, Objective 8 spells out a series of objectives including increasing the efficiency of water use and reverting and preventing agricultural land degradation. The Project activities—centered around strengthening the capacity of the SNIA to generate and promote the successful uptake of agricultural innovations— were designed to contribute to these expected results. Relevance of PDO Rating 27. Relevance of the PDO was High when the Project was prepared, and it remained High throughout the life of the Project. There were no shortcomings in the relevance of the PDO to the CPF current at closing. Then as now, improvements in the capacity of the SNIA were urgently needed to increase the rate of innovation in the agricultural sector and drive the productivity gains that can lead to inclusive and sustainable growth. Promoting innovation in the agricultural sector was a leading priority for the Government of Peru when the Project was prepared, and it remains a leading priority at the time of this review. This is evidenced by the signing by Interim President Francisco Sagasti on March 24, 2021 of the Supreme Decree N° 009-2021-PRODUCE creating within CONCYTEC a new PROINNOVATE program, with the mandate to support at central level the types of innovation activities that have been carried out at sectoral level under PNIA and similar programs. B. ACHIEVEMENT OF PDOs (EFFICACY) 28. The following assessment of the Project’s efficacy is based on the outcomes reported in the Project’s Results Framework (RF), as well as complementary information from the independent impact evaluation (summarized in Annex 7), the Borrower’s ICR, and project databases. 3 29. To evaluate the Project’s efficacy, the PDO is unpacked into two objective outcomes: (1) to create the adequate conditions in the Borrower's SNIA in order to support the effectiveness of its member organizations, and (2) to enhance the effectiveness of SNIA member organizations in developing and providing improved agricultural technologies. 30. Assessment of the degree to which the PDO was achieved is complicated by the fact that some of the terms in the PDO are not precise. What exactly are “adequate conditions in the Borrower’s SNIA”? To provide a basis for assessing the Project’s outcomes, “adequate conditions in the Borrower’s SNIA” is interpreted to mean the existence of an active network of SNIA member organizations that participate in SNIA activities, benefit from coordination services provided by a functional Technical Secretariat, develop common research agendas, and 3 For more information on project results, Annex 1 provides the full list of project indicators and achievements, and Annex 7 summarizes the main findings of the independent impact evaluation. Page 9 of 72 The World Bank National Agricultural Innovation (P131013) through their participation in the SNIA allow their members to engage in innovation activities, gain access to new knowledge, and acquire new skills. Many of the expected project outcomes that contribute to adequate conditions are “soft” outcomes that do not necessarily lend themselves easily to measurement (e.g., institutions strengthened, capacity reinforced, coordination improved, participation enhanced), so it is fortunate that the Project’s RF includes not only quantitative measures of many of the above features but also participant satisfaction measures that capture performance outcomes more qualitatively. 31. Based on the progress achieved against the performance indicators as reported in the Project’s RF (discussed below), information obtained from the Project’s M&E system, and evidence generated through the final impact evaluation, both objective outcomes in the PDO were substantially achieved. To summarize: Objective / Outcome 1: At closing, the SNIA Technical Secretariat was fully funded by INIA, 85 percent of the SNIA member institutions surveyed reported satisfaction with the effectiveness and coordination in the SNIA, and 88 percent of SNIA member institutions that were invited to events and meetings organized by the SNIA sent representatives to participate in the events. 4 Objective / Outcome 2: At closing, the Project had supported 541 collaborative sub-projects under Component 2 with participation from 41,244 beneficiaries, resulting in 31,634 beneficiaries (77 percent) adopting new technologies. In addition, 59 strategic research sub-projects focused on priorities identified by INIA and CONICA had been completed successfully. In addition, the Project had provided 76,887 days of training to agricultural scientists and extension professionals, and it had financed scholarships that allowed 57 students (including 25 women) to complete Master’s degree programs; both types of activity were associated with income increases among the beneficiaries (see Annex 7). Assessment of Achievement of Each Objective / Outcome Objective / Outcome 1: To create the adequate conditions in the Borrower’s SNIA in order to support the effectiveness of its member organizations. (rated: Substantial) 32. To strengthen the SNIA’s capacity to effectively support its member organizations, the Project set out to create a technical secretariat within INIA as the agency leading the SNIA (Component 1), establish a national agricultural innovation awards program led by INIA to recognize and give visibility to agricultural innovation throughout Peru (Component 1), and strengthen the cadre of agricultural scientists and extension professionals in Peru by offering competitively awarded scholarships for graduate study and internships (Component 3). PDO Indicator 1: Institutions participating in the SNIA that report satisfaction with the effectiveness and cooperation in the system, measured by an independent party. (Target: 500; Actual: 762) 33. As measured through an independent survey conducted shortly before project closing, 85 percent of the 166 SNIA member institutions surveyed reported being satisfied with the effectiveness of the SNIA and the level of cooperation among SNIA institutions. Satisfaction was measured using a set of eight questions, applying the Liker scale to generate a satisfaction level value between 1 and 5. 5 A SNIA member institution was considered satisfied if the sum of the values of the questions reached at least 70 percent of the maximum score possible. 4 Measurement of Outcome 1 is made difficult by the fact that the SNIA is comprised of numerous member institutions (research institutes, universities, private companies, civil society organization, producer groups, trade associations, etc.) whose number changes constantly as some members drop out and new members appear. The end-of-project target for PDO Indicator 1 of 500 SNIA members was designed to show that by project closing, the SNIA would be large and flourishing. 5 1: Totally unsatisfied, 2: Unsatisfied, 3: Neither satisfied nor dissatisfied, 4: Satisfied and 5: Totally satisfied. Page 10 of 72 The World Bank National Agricultural Innovation (P131013) The survey questions captured perceptions about the important outcomes to which the Project contributed: (i) improvement in the capacity of INIA to exercise its role as governing body of the SNIA; (ii) strengthened links with the national science and technology system; (iii) promotion of agricultural innovation through competitive grants, scholarships, and internships; and (iv) generation of new knowledge through sub- projects, thesis research, and commissioned studies. (a) Improvement in the capacity of INIA to exercise its role as governing body of the SNIA 34. INIA with greater autonomy to implement the institutional reforms needed to more effectively lead the SNIA. A legal covenant to the Project credit agreement was that within 30 months of project effectiveness, MINAGRI would take the necessary steps to convert the legal status of INIA to that of a Specialized Technical Organization (Organismo Técnico Especializado; OTE). The change in INIA’s legal status became effective on February 8, 2018. Conversion to OTE status provided an opportunity for INIA to adjust its operating policies and streamline its administrative procedures. While the change in legal status represented important progress, efforts to maintain the momentum of reforms then slowed, due to resistance by established interest groups to change the status quo. With the aim of developing a new policy framework for INIA, five policy papers were prepared with project resources and approved by MINAGRI, short of the eight policy papers that had been envisioned. At the time of project closing, the organizational changes called for in the policy papers had been implemented only partially, due to several factors including the opposition of many unionized staff to possible changes in their employment conditions. This effectively restricted the level of autonomy achieved by INIA and limited its ability to pursue the national innovation agenda independently from MINAGRI. 35. INIA with strengthened capacity to carry out its mandate as the lead agency tasked with coordinating the SNIA. The Project supported the establishment of a Technical Secretariat led by an Executive Director. Initially established within the PIU, the Technical Secretariat was later transferred to INIA as intended, after the change in legal status to OTE was completed. Challenges were initially experienced in recruiting and retaining an Executive Director for the Technical Secretariat, which for a while restricted its ability to advance project activities. After it had become fully staffed in the third year of the Project, the Technical Secretariat functioned effectively, and it provided a strong impetus for accelerating the implementation of project activities. To encourage the full integration of the Technical Secretariat into the SNIA, its operational costs were 100 percent financed by INIA (exceeding the target of 75 percent of funding to be provided by INIA). 36. INIA with enhanced presence throughout the national territory and supporting SNIA members at the regional level in driving innovation. The Project established six (6) Regional Offices and five (5) Sub-regional Offices. These decentralized offices played a key role in: (i) driving regional innovation policy; (ii) supporting the preparation of proposals for the various types of competitive grants; (iii) providing technical support and extension services to farmers and producer organizations in the implementation of the competitive grants; and (iv) supporting the project M&E function. The Regional Offices contributed greatly to the successful implementation of the competitive grant program by providing close implementation support to grant recipients. During the life of the Project, PNIA staff were frequently redeployed between Regional and Sub- regional Offices, to align with the evolving demand for their services and to ensure strong technical support and close monitoring and evaluation of sub-projects. 37. Strengthened institutional coordination promoting consolidation of the SNIA at sub-national level. The Project supported the establishment of 20 Agricultural Innovation Technical Commissions and organized 54 regional meetings with local SNIA actors. The Technical Commissions were established to create spaces for dialogue aimed at promoting the creation of networks and inter-institutional cooperation in agricultural research and development and innovation at the sub-national level. The Technical Commissions also facilitated Page 11 of 72 The World Bank National Agricultural Innovation (P131013) the development of 20 Regional Agricultural Innovation Agendas in which research, development and technology transfer actions were identified for economically important value chains in each region. (b) Strengthened links with the national science and technology system 38. Strengthened links between PNIA and the national science and technology system. The PNIA Steering Committee included members deliberately selected to serve as liaisons. The most important of these were the Directors of the National Council for Science, Technology, and Innovation (Consejo Nacional de Ciencia, Tecnologia, y Innovacion; CONCYTEC). A succession of CONYTEC Directors served on the PNIA Steering Committee and participated in PNIA strategy discussions; their presence ensured that efforts being pursued under PNIA to promote innovation at the sectoral level could complement efforts being made at central level to promote innovation more broadly throughout the entire economy. (c) Promotion of agricultural innovation through competitive grants, scholarships and internships, and agricultural innovation fairs and awards 39. Strengthened strategic competencies among professionals in the agricultural sector working on extension and innovation. The Project supported capacity building sub-projects funded through the competitive grant program. The capacity building sub-projects provided 76,887 days of technical training to agricultural scientists, extension agents, technicians, and other professionals (1,098 percent of target). 6 The technical training targeted 16 different value chains. Key areas for technical training included the development of strategic competencies in organic agriculture, raising of small animals, innovation in value and supply chain efficiency, and post-harvest management. In addition to strengthening the country’s endowment of trained extensionists available to deliver advisory services, these capacity building activities directly benefited the participants themselves; the final impact evaluation determined that extensionists who participated in PNIA-supported training programs experienced income gains averaging 198 percent (nominal terms). 40. Strengthened strategic competencies among professionals in the agricultural sector working on research and development and innovation. The Project established a Postgraduate Fund that awarded scholarships for postgraduate degree study and internships. These activities did not start until early 2018 due to delays in preparing the Project’s Capacity Building and Training Plan, which was a condition for launching the scholarship program. Despite the late start, the Postgraduate Fund achieved notable results: • Scholarships: The Project awarded 82 Masters scholarships, of which 57 (70 percent) were successfully completed. In all, 32 men and 25 women (227 percent of the end-of-project target) obtained Masters degrees (184 percent of the end-of-project target). 25 scholarship recipients did not receive degrees, in most cases because the outbreak of the Covid-19 pandemic led to suspension of their academic programs. The thesis research topics selected by the scholarship recipients were generally well aligned with INIA priorities and included, among others: (i) sustainable use of natural resources (17 theses), (ii) agricultural productivity and competitiveness (16 theses), (iii) genetic improvement (7 theses), and (iv) value-addition products (5 theses). Other thesis research topics included use of information and communication technologies in agriculture, animal health, industrial competitiveness, and climate change and environmental impacts. Almost all scholarship recipients have found employment within INIA or in SNIA member institutions, indicating that they are contributing to strengthening the system. 6 The end-of-project value far exceeded the target due to the decision taken in response to higher-than-expected demand to expand significantly the number of individual and especially group internships financed by the Project. Page 12 of 72 The World Bank National Agricultural Innovation (P131013) • Individual internships: The Project financed 255 individual internships (66 national and 189 international), far exceeding the end-of-project target of 80 (365 percent). Internships were financed in areas such as: (i) innovation in agribusiness (17 percent), (ii) coffee value chain (15 percent), (iii) animal reproduction including genetic improvement using biotechnology techniques (14 percent), (iv) agri-food industry (6 percent), (v) and climate change including links with livestock and agroforestry (7 percent). • Group internships: The Project financed 36 group internship benefitting 295 individuals in diverse stakeholder groups: (i) producer associations (11), (ii) small agricultural companies (9), (iii) agricultural cooperatives (8), (iv) educational institutions (4), (v) non-governmental organizations (2), (vi) water users association (1), and (vii) professional associations (1). More than one-third of the beneficiaries (109 or 37 percent of the group) were women. 41. In summary, despite being launched later than expected, the Postgraduate Fund was successfully implemented, and the overall achievements far exceeded the expected goals. The results would have been even more impressive if the COVID-19 pandemic had not prevented some graduate students from completing their theses and some internship grant recipients from completing their internships. 42. Enhanced visibility for the innovation agenda throughout the country. The Project raised the visibility of the agricultural innovation agenda by organizing and implementing national innovation fairs (known as AGRONOVAs) and a national prize competition to recognize agricultural innovation. The Project supported the organization and implementation of two national innovation fairs, which are expected to become annual events led by INIA. The first, AGRONOVA 2018, was held in Lima on April 26-27, 2018 and featured a large program of speakers, included booths displaying the results of 40 innovation sub-projects, attracted a large number of visitors, and received extensive media coverage. 7 The second, AGRONOVA 2019, was held in Tarapoto, San Martín Region, on October 24-25, 2019 and was similar in format and equally successful in terms of bringing together many different SNIA actors and exposing their innovative projects to a large public. 8 The National Innovation Prize Competition (CARAL) organized and supported by INIA was held on June 25, 2020 to give visibility to and share knowledge about successful agricultural innovations. The competition was designed to incentivize participating institutions to publicize their innovations, stimulate new initiatives, and identify opportunities to scale up innovations. All told, 169 innovation projects were nominated, of which 20 were awarded prizes in four categories. 9 (d) Generation of knowledge through sub-projects, thesis research, and commissioned studies 43. Expanded set of knowledge available for use by actors in the SNIA. The Project invested not only in generating new knowledge but also in documenting, archiving, and disseminating that knowledge. The innovation sub-projects, thesis research activities, and commissioned studies generated a rich body of new knowledge that can help advance Peru’s innovation agenda for years to come. The outcomes from the research sub-projects were disseminated through 136 articles (227 percent of the target) published in scientific journals. The published articles describe the results of high-quality research carried out on topics identified as priorities in the National Strategic Plan for Science, Technology, and Innovation. In addition, technical sheets were prepared for 86 sub-projects detailing the innovation process followed under the sub-project and describing in detail the innovation generated and its possible uses. Finally, 248 records have been created (57 for Masters 7 AGRONOVA 2018. https://www.youtube.com/watch?v=v2DbyXKxUSE&t=37s 8 AGRONOVA 2019. https://www.youtube.com/watch?v=fj5tzKzbWZo; https://www.youtube.com/watch?v=xP505qfpDHk 9 MINAGRI lanza Premio INIA-CARAL 2020. https://www.youtube.com/watch?v=uWqc27YnEHE Page 13 of 72 The World Bank National Agricultural Innovation (P131013) degrees and 191 for internships) containing detailed information about the research conducted under each degree program or internship. 44. Technical information published and shared among SNIA member institutions. The Project supported the creation of an online information platform for SNIA member institutions. The information platform has been very successful; during the final three months of the Project life, SNIA members and other users were downloading materials at a rate of nearly 230,000 downloads / year, far exceeding the targeted rate of 60,000 downloads/year (382 percent). Objective / Outcome 2: To enhance the effectiveness of SNIA member organizations in developing and providing improved agricultural technologies (rated: Substantial) 45. As reflected in the Theory of Change, the Project sought to enhance the effectiveness of SNIA member organizations in developing and providing improved agricultural technologies by (i) building capacity in the SNIA including INIA to enable the design and successful implementation of competitively funded sub-projects that would (ii) benefit producers and other agricultural value chain actors as well as SNIA member institutions, and (iii) make available a large number of validated innovations and support the dissemination of the new knowledge generated. Because the PDO Indicators do not fully measure all of these dimensions, they are evaluated using Intermediate Results Indicators as well as evidence from the final impact evaluation. The degree to which Objective / Outcome 2 was achieved is assessed based on the following three outcomes: (a) Capacity of SNIA including INIA built that enables the successful implementation of competitively funded sub-projects and strategic research sub-projects 46. Enhanced effectiveness of SNIA member organizations in developing and providing improved agricultural technologies. The Project supported 541 collaborative sub-projects designed to import or develop locally, validate, and/or disseminate innovations deemed promising for Peruvian agriculture (substantially exceeding the target of 400). The sub-projects were financed using a demand-driven competitive grant mechanism that had separate windows for three categories of sub-project: (i) adaptive research; (ii) extension and technical assistance; and (iii) community seed enterprises (Component 2). 47. The PNIA Technical Secretariat financed three rounds of competitively selected sub-projects using the matching grant mechanism. Initiated in September 2015, July 2016, and May 2017, they involved the following steps: (i) launch and dissemination of the call for proposals; (ii) receipt of proposals; (iii) evaluation of proposals by independent technical experts and publication of results; (iv) negotiation and finalization of selected proposals; and (v) contract signing. An online, web-based system was developed to manage the receipt of proposals, evaluation of proposals, monitoring of implementation progress, and documentation of results. The system, once established, facilitated the sub-project selection process, provided the PIU with real-time information about the implementation status of sub-projects, captured a wealth of data about sub-project activities, facilitated financial management of the competitive grants, and supported the closing of completed sub-projects and documentation of results. The web-based system was not yet in place when the first call for proposals was launched, but it was up and running when the second and thirds calls were launched. Development of the system was complex, but once it was operational, evaluation of proposals, management of sub-projects, and documentation of results were greatly facilitated. As evidence of its utility, the system remains in use within INIA, and it has been adapted by the Ministry of Production for the management of a similar competitive grant program that is supporting innovation in the fisheries and aquaculture sector. Page 14 of 72 The World Bank National Agricultural Innovation (P131013) Table 1. Implementation performance of collaborative sub-projects Target Achieved Achieved Sub-project type (number) (number) (%) 1. Adaptive research 120 129 108 % 2. Extension and technical assistance 240 367 153 % 3. Community seed enterprise pilots 40 45 113 % Total 400 541 135 % 48. The collaborative sub-projects involved 74 value chains, including coffee, cocoa, milk, beef, guinea pigs, banana, potato, sugar cane, sheep, quinoa, wool, and avocado, among others (for additional details, refer to Figure 1 in Annex 8). The rigorous nature of the proposal evaluation process and the high quality of the implementation support provided by the PIU once the sub-projects got underway is reflected in the fact that out of the 541 sub-projects approved, only 10 sub-projects were not completed. This represents a failure rate of less than 2 percent. 49. Highlights of the collaborative sub-project implementation experience: • 129 adaptive research sub-projects (108 percent of target) focusing on technological innovations to increase efficiency in fertilization, post-harvest processing, installation of crops, and commercial management, among others. • 367 extension and technical assistance sub-projects (153 percent of target), 95 percent of which led to an increase in productivity and/or production. 84 percent facilitated greater access to markets, and 81 percent achieved improved quality of production. • 45 community seed enterprises (113 percent of target). Some 82 percent of seed growers reported an increase in productivity and/or production, while over 60 percent reported improved quality of their products. In addition, 27 percent improved their access to markets. 50. Enhanced effectiveness of SNIA member organizations in developing and providing improved agricultural technologies deemed strategic for the nation. The Project supported 59 strategic research sub- projects designed to address priority themes identified by INIA and CONICA. The strategic research sub-projects were driven by nationally identified innovation priorities as determined by INIA and CONICA, and they were implemented by SNIA member institutions as well as by INIA (maximum of 30 percent of sub-projects). Strategic research sub-projects focused on innovation in product processing, digital systems for crop management, genetic improvement, climate change adaptation and mitigation measures and pest management, among others, and benefitted a wide range of agricultural value chains, most importantly the coffee, quinoa, potato, cocoa, sugar cane, and livestock sectors. Table 2. Implementation performance of strategic research sub-projects Target Achieved Achieved Sub-project type (number) (number) (%) 1. Strategic research 60 59 98 % Page 15 of 72 The World Bank National Agricultural Innovation (P131013) 51. Highlights of the strategic research sub-project implementation experience: • 59 strategic research sub-projects were successfully completed out of 60 originally approved for funding (98 percent of target). Strategic research sub-projects were implemented through partnerships usually including universities, national and/or international research institutes, and in many cases INIA. Unlike the collaborative sub-projects which were aimed at testing, validating, and disseminating innovations, the strategic research sub-projects tended to focus on more upstream, basic research, such as DNA fingerprinting of indigenous potato varieties, genetic characterization of disease pathogens, and development of cultural media for tissue culture. (b) Benefit producers and other agricultural value chain actors PDO Indicator 2: Farmers adopting new technologies promoted by sub-projects. (Target: 20,000; Actual: 31,634; Exceeded: 158 percent) 52. To ensure that the collaborative sub-projects would benefit producers and other value chain actors, the Project mounted a comprehensive outreach effort to raise awareness about the competitive grant program and attract a high level of participation. The competitive grant program was promoted through workshops held throughout the national territory, notices placed in print and broadcast media, social media campaigns, posters and placards placed in public venues, and mass mailings, among others. These efforts were very successful, as evidenced by the robust responses to the three calls for sub-project proposals and the receipt of proposals from every region in the country. By project closing, 41,244 producers and other agricultural value chain actors had participated in one or more of the 541 collaborative sub-projects that were successfully completed. Of all the participating producers, 33 percent were women (slightly below the target of 35 percent) and 31 percent were participants who self-identified as indigenous (significantly exceeding the target of 20 percent). 53. To ensure that participation in the collaborative sub-projects would benefit producers and other value chain actors, project staff conducted rigorous evaluations of the sub-project proposals and selected only those meeting high quality standards, provided co-financing in the form of competitive grants, and provided technical assistance and hands-on training throughout the sub-project implementation cycle. Evidence from the project records supported by the results of the final impact evaluation confirm that participation in the sub-projects produced benefits for most participants. An estimated 31,634 producers and other agricultural value chain actors adopted new technologies promoted under the sub-projects, far exceeding the target of 20,000 (158 percent). As described in the following section on Efficiency, adoption of the new technologies in a high majority of cases generated attractive economic returns in the form of increased revenues and/or reduced costs. 54. Adoption rates for innovations developed under the Project were estimated via representative surveys of farmers and other actors who participated in adaptive research, extension, and seed enterprise sub-projects. The methodological approach was based on the general concept of technology adoption, which refers to the knowledge, processes and/or products of exogenous origin that producers incorporate into their production systems. Adoption was defined in terms of three criteria: (i) there is clear evidence that direct beneficiaries are using an innovation promoted by the Project independently and at their own discretion; (ii) the innovation introduced by the Project generates net benefits greater than those generated by traditional technologies; and (iii) the innovation is being replicated by indirect beneficiaries at the local level. According to a survey of 466 sub-project participants conducted shortly before project closing, 76.7 percent of sub-project participants had adopted at least one innovation developed under the Project. The adoption rate varied depending on the sub- project type and funding cycle; adoption was highest among participants in the seed enterprise sub-projects and lowest among participants in the adaptive research sub-projects (Table 3). Page 16 of 72 The World Bank National Agricultural Innovation (P131013) Table 3. Adoption rates of innovations, by sub-project type and funding cycle (percent) Funding cycle Total Sub-project type 2015 2016 2017 (n = 466) (n = 154) (n = 156) (n = 156) Adaptative research (n = 100) 37.5 75.3 59.3 64.2 Extension (n = 320) 87.2 77.6 79.8 79.9 Seed enterprise pilots (n = 46) 53.8 95.1 90.0 86.2 Total (n = 466) 77.1 79.1 73.9 76.7 (c) Make available a large number of validated innovations PDO Indicator 3: New technologies emerging from research projects that have been demonstrated on farm (target: 61, actual: 111; exceeded: 182 percent) 55. Large number of validated innovations made available through the SNIA. The Project provided co- financing, training, and technical assistance to ensure that new technologies would be widely tested in the field by large numbers of sub-project participants. Evidence that the Project improved the effectiveness of the SNIA member organizations in developing and providing improved agricultural technologies is provided by the large number of new technologies emerging from research projects that were demonstrated on-farm. Some 111 distinct technological innovations emerging from strategic and adaptive research sub-projects were successfully demonstrated on farms, almost double the original target of 61. 56. Large numbers of validated innovations are available to potential future beneficiaries. The Project invested in documenting the innovations. As part of the closing process, sub-project beneficiaries were required to prepare data sheets (fichas técnicas) containing detailed information about each innovation that had been evaluated and validated, including technical specifications, recommendation domains, recommended management practices, and economic aspects. The data sheets provide a rich trove of information that has the potential to deliver benefits for producers and other value chain actors in the years to come. Many of the innovations continue to attract attention from potential users, including extensionists, researchers, and other value chain actors. It is estimated that up to 750,000 producers and agriculture value chain actors could potentially benefit from these innovations in the medium and long term, showing the value-added of the sub- projects in driving innovation as a complement to the demand-based competitive grant funds. Justification of Overall Efficacy Rating 57. Efficacy Rating. Overall Efficacy is rated Substantial. The Project almost fully achieved its objectives. Despite experiencing initial implementation challenges due to a high rate of staff turnover in INIA and in the PIU, after a management team was installed that was given enough time to manage the work program properly, the Project made up lost ground. By the closing date, the Project had achieved its intended development objectives and outcomes, with all three PDO Outcome Indicators exceeded, 10 Intermediate Results Indicators exceeded, and two Intermediate Results Indicators substantially achieved. Page 17 of 72 The World Bank National Agricultural Innovation (P131013) C. EFFICIENCY 58. The following assessment of the Project’s efficiency is based on ex-post cost-benefit analysis carried out for this ICR. The ex-post cost-benefit analysis relied on information from three sources: (i) a survey carried out by a consultant engaged by the PIU of 70 collaborative sub-projects randomly selected from among the 541 collaborative sub-projects financed using the competitive grant mechanism; (ii) a survey carried out as part of the independently conducted final impact evaluation of 190 researchers and extension agents who had benefited from capacity strengthening activities financed by the Project; and (iii) project records. (For more complete details on the efficiency analysis, please refer to Annex 4). 59. The benefit streams considered by the ex-post cost-benefit analysis are the estimated incremental value of production and income realized by project beneficiaries (producers, extensionists, and researchers) as a result of taking up innovations developed and made available by the Project or as the result of having acquired new skills from participating in training activities financed by the Project. (For details about how the benefit streams were calculated, please refer to Annex 4.) 60. Several other benefit streams were not considered, including (i) future productivity gains that may be realized by producers and other agricultural value chain actors as the result of a strengthened SNIA; (ii) future productivity gains that may be realized by producers and other agricultural value chain actors due to the presence of a new cadre of more knowledgeable and more capable researchers, extension agents, and producers; and (iii) future productivity gains that may be realized by producers and other agricultural value chain actors thanks to the greater availability of information and knowledge about the innovations developed and validated under the Project. While difficult to quantify and value, these additional benefit streams could be significant, and for that reason the results of the ex-post cost-benefit analysis can be considered conservative. 61. The cost-benefit analysis included both financial analysis carried out using actual market prices inclusive of taxes, subsidies, and other distortions, as well as economic analysis carried out using economic prices adjusted in some instances to correct for taxes, subsidies, and other distortions known to be present in the Peruvian economy. The results of the financial analysis show the returns to project-supported activities from the perspective of implementing entities, participants, private enterprises and beneficiaries, while the results of the economic analysis show the “true” returns to Peruvian society and therefore are likely to be of particular interest to policy makers. Assessment of Efficiency and Rating (a) Returns to overall Project 62. For the overall Project investment, the estimated values of standard measures of project worth are presented in Table 4. The large positive net present values (NPVs) and high economic internal rate of return (EIRR) show that overall the resources invested in the Project generated positive returns on investment, both financially and economically. The estimated Economic Internal Rate of Return (EIRR) EIRR of 31.2 percent is higher than the EIRR estimated at the time PNIA was prepared (EIRR = 26 percent). It is higher than EIRRs reported by similar initiatives in Peru and elsewhere in the region, including the INCAGRO project (EIRR = 24.2 percent) and the Program for Innovation and Agricultural Services (Programa de Innovación y Servicios Page 18 of 72 The World Bank National Agricultural Innovation (P131013) Agropecuarios; PISA) in Bolivia (EIRR = 18.8 percent). It falls toward the high end of the ranges reported in global studies of returns to investment in agricultural research. 10, 11 63. Sensitivity analysis was carried out to determine the robustness of these results under a range of possible changes in key parameters. The IRRs are somewhat sensitive to changes in technology adoption rates, production costs, and producer revenues, and they are relatively insensitive to changes in other parameters including the amount of training provided to extensionists and researchers. A particularly noteworthy finding is that accelerating the rate of adoption of agricultural technologies validated and made available by the Project (which could be achieved by increasing the amount of extension advisory services and technical assistance provided to producers and other value chain actors) would lead to significantly higher returns. (For more complete details, please refer to Annex 4.) Table 4. Ex post cost-benefit analysis – Summary results Financial indicators Financial Net Present Financial Internal Rate Payback Incremental NPV Benefit/Cost Value (FNPV) of Return (FIRR) period per beneficiary Ratio (US$ million) (percent) (years) (US$) 48.2 24.6 1.26 7.3 1,121 Economic indicators Economic Net Economic Internal Payback Incremental NPV Benefit/Cost Present Value (ENPV) Rate of Return (EIRR) period per beneficiary Ratio (US$ million) (percent) (years) (US$) 86.2 31.2 1.48 6.8 2,007 (b) Efficiency of the Project management function 64. At Project closing, the total cost of Component 4 stood at US$7.14 million, considerably higher than the US$3.57 million projected at appraisal. Project management costs exceeded projections for two main reasons. First, the two extensions of the project closing date meant that PIU staff salaries and operational costs were incurred for 18 months longer than expected. Second, when it became apparent that many collaborative sub- projects were experiencing implementation delays, including due to lack of technical knowledge and/or lack of experience with fiduciary procedures, additional short-term consultants were hired to provide hands-on technical assistance, support administrative processes, and ensure thorough documentation of results. In general terms, the PIU performed well throughout the life of the Project. The PIU was staffed by qualified and highly engaged professionals, most of whom were committed to the Project and carried out their activities with great dedication in the face of a constantly changing institutional environment. The PIU management team ably oversaw the successful implementation of a large and diverse work program, made certain that activities and 10 Alston, J., Chang-Kang, C., Marra, M., Pardey, P. and T. Wyatt. 2000 Meta-analysis of returns to agricultural research and development: Ex Pede Herculem? IFPRI Research Report 113. 11 Pardey, P., Alston, J. and R. Piggott (eds.). 2006. Agricultural research investment in the developing world: Too Little, too late? IFPRI. Page 19 of 72 The World Bank National Agricultural Innovation (P131013) outcomes were well documented, and ensured that fiduciary functions were carried out in compliance with WBG and government policies and procedures. 65. Efficiency is rated Substantial, based on the results of the ex-post cost-benefit analysis reported above, which in aggregate show that the investment in PNIA generated attractive returns. D. JUSTIFICATION OF OVERALL OUTCOME RATING 66. Overall Outcome is rated Satisfactory, based on the ratings of High for Relevance, Substantial for Efficacy, and Substantial for Efficiency. 67. Relevance: There were no shortcomings in relevance to the CPS at the time of preparation and to the CPF at project closing, and there is clear evidence of alignment of the PDO to the original CPS and to the current CPF strategic objectives. Efficacy: The Project fully achieved its intended outcomes, and many performance indicators were exceeded. Efficiency: The Project generated attractive returns on investment, as measured by ex-post cost-benefit analysis, and sensitivity analysis suggests these returns can be considered robust under a range of plausible scenarios. E. OTHER OUTCOMES AND IMPACTS Inclusion of vulnerable groups: Women 68. The Project was successful in promoting inclusion, achieving high levels of participation by women. Women were explicitly targeted by the competitive grant program and the scholarship program. Efforts were made to strengthen the capacity of women to compete for competitive grants, for example by organizing special orientation sessions that targeted women’s associations and by making available staff in the regional and sub- regional offices to review and comment on draft proposals being prepared by women’s groups. After the first wave of competitive grants was awarded and evidence emerged of lower-than-expected participation by women, adjustments were made to ensure greater participation by women, including the awarding of additional points to sub-project proposals submitted by women’s groups. The Project’s success in engaging women is evidenced by the fact that all but one of the end-of-project targets for women’s participation were exceeded. Inclusion of vulnerable groups: Indigenous people 69. By effectively targeting ‘native communities’ (the term used by local communities in Peru to self-identify as indigenous), the Project had positive impacts on indigenous people. Implementation of the Indigenous Communities Plan ensured that indigenous populations had knowledge about and access to the Project. The Project contracted a social specialist who was tasked with implementing measures and strategies to promote participation by indigenous peoples, including making sure Project-related materials were available in local languages, targeting orientation sessions to local community groups, and making available staff in the regional and sub-regional offices to review and comment on draft proposals being prepared by indigenous groups. The sub-projects implemented by indigenous groups proved effective. The impact evaluation determined that 202 sub-projects (37 percent of the total) involved participation by indigenous populations. Institutional strengthening 70. The Project strengthened the capacity of INIA in three main ways. First, the capacity of many INIA scientific staff was strengthened through their participation in graduate programs, internships, and training financed by the Project. Second, INIA benefited from many of the management and administration systems developed by the Project that remain in use at the time of this review, including the project management information system, the system for managing competitive grant programs, and the sub-project monitoring and Page 20 of 72 The World Bank National Agricultural Innovation (P131013) evaluation system. Third, physical infrastructure, machinery and equipment were significantly upgraded and modernized at INIA headquarters in Lima, as well as in 13 INIA experiment stations located throughout the country (by design, most of the funding for this work came via the parallel-funded IADB project). Regional strengthening 71. The Project served as an important model for the government’s expressed commitment to decentralization. By establishing regional and sub-regional offices throughout the country, PNIA demonstrated that responsibility for priority setting, decision making, and day-to-day management of the innovation agenda could be delegated to officials at sub-national level. By engaging in dialogue and forming working collaborations at the regional level, it empowered local actors by giving them a greater role in the allocation of public resources. The fact that the criteria for awarding competitive grants incorporated priorities that had emerged from the regional innovation roundtables gave enhanced voice to local stakeholders and partners, most notably the regional and municipal governments. Mobilizing private sector financing 72. The Project was successful in mobilizing private sector financing, thanks in large part to the competitive grant program, which was designed as a matching grants program and required beneficiary groups to mobilize a counterpart contribution. By project closing, the total value of beneficiary contributions was estimated at US$24.1 million. In most cases, the beneficiary contributions were provided in cash, but consistent with the goal of reaching disadvantaged and low-income groups, provision was made to allow beneficiary contribution to be provided in kind (usually materials or labor). Poverty reduction and shared prosperity 73. The independent impact evaluation demonstrated positive impacts on beneficiary wellbeing and household income: Data collected through a survey of 826 beneficiary households (treatment group) and 649 non-beneficiary households (control group) were analyzed using the Difference-in-Difference (DiD) method to determine the impact of the Project on several measures of household income. Agricultural income among beneficiary households increased 34 percent more than among the control group households (a difference of US$900), and total household income among beneficiary households increased by about 18 percent more than among the control group households (a difference of US$435). Likewise, the adoption rate of technological innovations among the beneficiary households was 61 percent higher than in the control group households. These results confirm that the Project had positive impacts as expected (for additional details, please refer to Annex 7. Project Impact Evaluation Summary). III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME A. KEY FACTORS DURING PREPARATION 74. Project preparation was influenced by several factors: • Uneven agricultural development and lagging regions. At the time when the Project was prepared, the government was making significant progress in reducing poverty and improving standards of living nationwide. Progress was uneven, however. While the Costa region was experiencing a boom, the Sierra and Selva regions were lagging. The authorities were searching for opportunities to boost agricultural productivity and promote development in the lagging regions, including pursuing the innovation agenda which was to be complemented by public investments in land titling and irrigation infrastructure. Page 21 of 72 The World Bank National Agricultural Innovation (P131013) • Strong government commitment to the innovation agenda. The Project was prepared at a time when the government was concerned about the lack of innovation in the economy generally, and it was committed to implementing a broad-based strategy to promote innovation including at sectoral level. Consistent with this commitment, it seized on the idea of promoting agricultural innovation in the lagging regions by establishing a program to reinforce the national innovation system and strengthen the market for innovation services. • Push to decentralize. The government’s desire to advance the innovation agenda coincided with an effort to promote decentralization by devolving responsibility for public services away from ministries and agencies based in Lima to the regions. Officials in MEF strongly encouraged the team that prepared the Project to provide for decentralized project offices at regional and sub-regional level, to ensure an active presence at local level. The demand-driven approach of the competitive grant program was seen as a way of pushing resources out to stakeholders and partners. • Spirit of collaboration between the multilateral financial institutions. In a good practice example of close collaboration, staff from the WBG and from the IADB worked with government agencies to design a project supported through parallel financing. Thanks to the close working relationship, the teams from the two banks were able to speak with a single voice, which provided added leverage when it came to discussing critical design issues with government counterparts. B. KEY FACTORS DURING IMPLEMENTATION 75. Project implementation was affected by several factors in the external environment. Factors subject to the control of the implementing entities: • Lack of consistent leadership: The Project was implemented during a period of political instability in Peru, characterized by multiple changes in government that resulted in frequent changes in senior management of MINAGRI, which trickled down to affect INIA and PSNIA. During its six-year life, PSNIA was overseen by nine Ministers of Agriculture, five Heads of INIA, and four Executive Directors of PNIA, each of whom came with a slightly different vision for the Project. • Vacillating commitment by INIA: Shortly after the Project was approved by the World Bank Board of Directors in December 2013, the leadership of INIA changed following a change in government. By the time the loan agreement was signed in April 2014, a new Head of INIA had taken office, and he questioned the project concept, design, and implementation arrangements. This caused a delay in the appointment of an Executive Director for the Project, which was a condition of effectiveness. As a result, effectiveness was not declared until May 2015, by which time yet another new Head of INIA had taken office. • Frequent interruptions in counterpart funding: Throughout its life, the Project suffered from frequent interruptions in counterpart funding, caused in many instances by the unfamiliarity of newly arrived Executive Directors with the intricacies of government budgeting processes. Funding requests were not always submitted to the Ministry of Economics and Finance (MEF) in time to be included in the annual budget, requiring special requests to be submitted that were not always approved. • Bottlenecks in procurement: Procurement capacity in INIA was weak. Delays were experienced as the INIA procurement team struggled to manage the procurement especially of the major works related to the rehabilitation of INIA laboratory facilities and experiment stations funded through the IADB project. At a smaller scale, weaknesses in the procurement function impacted PSNIA through delays in the recruitment of firms and consultants needed to make key contributions, such as the design of computer- Page 22 of 72 The World Bank National Agricultural Innovation (P131013) based management systems, recording of a baseline for the impact evaluation, preparation of background documents for the Mid-term Review, and design of the agricultural innovation awards competition. Factors outside the control of the implementing entities: • Inclement weather: In early 2017, an especially severe El Niño effect gave rise to extreme weather events that caused widespread damage in many parts of the country (flooding in some regions, droughts in other regions). This disrupted the implementation schedule of a considerable number of collaborative sub- projects, either delaying their start-up or preventing their timely completion. Because of these delays, it was not possible to complete the affected sub-projects by the original closing date. • COVID-19 pandemic: In early 2020, the outbreak of the Covid-19 pandemic disrupted activities throughout Peru as many people fell ill and the authorities enacted social mobility restrictions. These restrictions impacted the completion of many sub-projects and complicated the closing of sub-project financial accounts and the documentation of results, activities that had to be conducted virtually. Furthermore, the travel restrictions related to the pandemic including the closing of Peru’s borders left many students, interns, and trainees being supported under the scholarship program stranded away from their homes, including in many cases outside the country. Twenty-seven scholarships for Masters degree study had to be canceled because the scholarship recipients would not have been able to complete their approved courses of study due to the mobility restrictions. IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME A. QUALITY OF MONITORING AND EVALUATION (M&E) M&E Design 76. The design of the Project’s M&E system was sound. The system put in place for collecting, verifying, and reporting M&E data was comprehensive and internally consistent. Dedicated M&E staff based in the central office in Lima were supported by M&E Specialists based in the regional offices, all using a custom M&E system. The time needed to hire and train M&E staff and put in place a functional M&E system was underestimated, however, as some M&E positions remained unfilled for several months, and the M&E system did not become fully operational until after the MTR. 77. If the design of the Project’s M&E system was sound, the Results Framework suffered from several shortcomings. First, the PDO Indicators are imperfectly aligned with the key elements of the PDO. Second, several indicators were proved difficult to measure initially because key terms were not closely defined, which complicated M&E activities until the indicators were adjusted during implementation. The adjustments (which were recorded in Aide Memoires) were minor and designed to facilitate measurement of selected indicators, so a formal restructuring of the Results Framework was deemed unnecessary. Third, robust baselines for some indicators were not established prior to implementation; the baselines had to be constructed retroactively during implementation. M&E Implementation 78. After the Project was launched, the M&E specialists working out of the regional offices got to work quickly and were conscientious in recording progress and reporting results. It soon became apparent, however, that the central M&E unit in Lima lacked a user-friendly system for receiving, recording, validating, and analyzing data coming in from the regional offices. The decision was taken to develop a web-based M&E system, conceptualized as a module that would form part of the larger project management information system. Page 23 of 72 The World Bank National Agricultural Innovation (P131013) Development of the M&E system took longer than expected, in part because digital technology was evolving rapidly and the M&E systems developed under earlier projects such as INCAGRO were of limited value, and in part due to the difficulty of identifying a competent software development firm. Initial results for some activities were not reported to the World Bank and recorded in ISRs until mid-2016, more than one year after effectiveness. Once established, the web-based M&E system was excellent: it held a large amount of geo- referenced information and featured a gateway that allowed external users to access information about the Project. M&E Utilization 79. Project management made effective use of the M&E data throughout implementation, even during the early phases of implementation while the web-based M&E system was still under development and M&E data had to be maintained in EXCEL files. Project administrative and financial staff were able to monitor the progress realized against agreed milestones, which enabled them to step in whenever an activity was off track and required attention. Equally important, the M&E system provided a real-time view of the results of the sub- projects financed through the competitive grant program, which allowed adjustments to be made during successive rounds of competitive grant funding to improve the targeting. For example, when the M&E system revealed imbalances in the distribution of competitive grant awards across regions or among commodities, adjustments were made to the eligibility criteria and scoring weights to better align the allocation of competitive grant resources to the project development objective. Data kept in the M&E system were also used to carry out the efficiency analysis reported in this ICR. 80. The Project’s M&E system was used effectively in combination with other management information systems to: (i) monitor progress in the implementation of sub-projects, (ii) ensure the orderly closing of sub- projects once they were completed, and (iii) carry out project management functions (administration, personnel management, financial management, procurement). The different systems once established provided project management with detailed real-time information about the implementation status of project-supported activities. The systems proved particularly useful during the closing of sub-projects and the recording of sub- project results. Justification of Overall Rating of Quality of M&E 81. Based on the successful collection of M&E data and the effective utilization of M&E data and analysis during implementation but recognizing also the design shortcomings and the delay in launching the web-based M&E system, overall Quality of M&E is rated Substantial. B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE 82. The Project was classified as Category B and triggered the following World Bank safeguard policies: Environmental Assessment (OP/BP 4.01); Natural Habitats (OP/BP 4.04); Forests (OP/BP4.36); Pest Management (OP4.09); and Indigenous Peoples (OP/BP 4.10). Environmental 83. OP 4.01 was triggered to assess the Project’s potential environmental risks and adverse impacts in its area of influence, and to describe the process for mitigating and managing these risks throughout implementation. OP 4.04 was triggered because even though sub-projects were to be implemented in locations traditionally characterized by agricultural landscapes, some of these locations overlapped with buffer zones around Natural Protected Areas (NPAs). OP 4.36 was triggered because the priority commodities being targeted included forest species, and it was expected that some sub-projects would be located in forest habitats. Finally, Page 24 of 72 The World Bank National Agricultural Innovation (P131013) OP 4.09 was triggered in anticipation that some sub-projects might result in increased use of pesticides or the introduction of pesticides into new areas. 84. An Environmental Management Framework (EMF) was prepared, compliant with national regulations and with OP 4.01, and disclosed in country and on the World Bank’s website in September 2013, prior to appraisal. The EMF included the legal and regulatory framework applicable to the Project, baseline information, an assessment of environmental risks and potential adverse impacts, environmental criteria for screening sub- project proposals, and an environmental supervision and monitoring plan. 85. In accordance with the EMF, the Project integrated good practices on natural resource management. Throughout implementation, a full-time Environmental Specialist employed in the PIU monitored the use of good practices with the help of environmental performance indicators. Corresponding information was regularly uploaded to and analyzed in the Project’s management information system. During the final supervision mission, it was confirmed that 100 percent of sub-projects were regularly reporting the required environmental performance indicators in the system. 86. To document how environmental risks and impacts were being managed in accordance with the provisions of the EMF, the Environmental Specialist prepared a report which included several detailed case studies showcasing selected sub-projects. The report was finalized in November 2020. In addition to describing the environmental management practices followed throughout the life of each sub-project, the report includes a section on lessons learned about environmental management good practices that may be considered in the design of future similar/related projects. 87. The close monitoring of environmental safeguards compliance turned up two issues that required attention by the PIU: (i) the possible location of several sub-projects within national protected areas (NPAs), and (ii) the planting by some of these sub-projects of an invasive tree species (Pinus radiata). The Environmental Specialist investigated both issues and worked with local authorities and sub-project beneficiaries to resolve them. The exact location of the concerned sub-projects relative to the boundaries of NPAs was determined, and vegetative zoning maps were analyzed to better understand whether the use of Pinus radiata might pose a threat to native vegetation and water balances. It was determined that: (i) sub-projects were located outside the NPAs but within their buffer zones, and the corresponding NPA authorities had been informed and were actively engaged in ensuring that sub-project activities were compliant with all applicable regulations; and (ii) sub- projects involved in the planting of Pinus radiata trees were doing so only in locations with a historical presence of this species, so no negative environmental impacts were expected to result. 88. The Project was compliant with environmental safeguards and was rated Satisfactory throughout implementation, except during two supervision periods (October 2018, April 2019), when compliance was downgraded to Moderately Satisfactory due to a lack of current information about the small number of sub- projects mentioned above. The rating was upgraded after the Environmental Specialist had investigated and documented compliance. Social 89. The Project triggered one World Bank social safeguard policy: Indigenous Peoples (OP/BP 4.10). As required by the triggering of the policy, an Environmental and Social Management Framework (ESMF) and Indigenous Peoples Plan (IPP) were prepared and disclosed on October 29, 2013. The potential social risks of the Project were assessed to be limited. 90. The PIU recruited a Social Specialist, who monitored the screening of sub-project proposals, ensured that the Project was living up to its commitment to include vulnerable groups including women and indigenous Page 25 of 72 The World Bank National Agricultural Innovation (P131013) people, and advised on mid-course corrections as needed to ensure adequate involvement of these groups in project activities. After an initial delay, social safeguards were monitored continuously until Project closing. Due to the inexperience of the PIU in implementing Bank projects, combined with some initial turnover in the Bank Social Specialist appointed to the Project, the procedures called for in the ESMF and IPP were not implemented systematically when the first set of sub-project proposals was evaluated. After the oversight was detected, the sub-projects were screened retroactively as required, and no adverse social risks were identified. During the evaluation of subsequent rounds of sub-project proposals, systematic screening was done for potential adverse social impacts, and social management plans were prepared as needed for individual sub-projects. 91. The Project was compliant with social safeguards and was rated Satisfactory throughout implementation, except during three supervision periods (June 2017, December 2017, June 2018), when compliance was downgraded to Moderately Satisfactory due to the temporary failure to screen properly the first set of sub-project proposals. Grievance Redress Mechanism 92. The Project originally relied on INIA’s existing Grievance Redress Mechanism (GRM). In response to a recommendation made by the World Bank Social Specialist during an early supervision mission, a project-specific GRM was established. Grievances could be submitted online through the PNIA website, in writing, by telephone, or in person through the PNIA central office or any of the regional and sub-regional offices. The PIU established an effective procedure for processing and responding to grievances, which involved redirecting grievances to the area within the PIU with responsibility for the issue raised. Resolution of complaints was monitored and reported to World Bank supervision teams. Financial Management 93. Financial management (FM) supervision missions rated FM performance as Satisfactory throughout the life of the Project. As part of project preparation, an assessment was carried out of INIA’s capacity to implement FM actions for the Project, and FM risk was rated Moderate. Recognizing that significant amounts of project funding would be channeled to sub-projects and implemented by third parties, the project preparation team identified as a key requirement the establishment of a strong financial management system with clear procedures and adequate controls to manage fund flows. The team drew lessons from the fiduciary systems that had been developed under the earlier INCAGRO projects, whose financial management systems were considered best practice for the management of competitive grant programs. Despite this effort, for several months following start-up, PNIA’s fiduciary systems struggled to manage and control the fund flows to sub-projects, which resulted in some initial delays in disbursement of funds to them. The PIU moved proactively to address the weaknesses, making changes to FM procedures that facilitated fund flows to sub-projects for subsequent rounds of sub-projects and for the scholarship program. The Bank and IADB agreed to joint auditing arrangements for the Project, which were conducted following IADB policies and procedures. The auditors issued unqualified (clean) opinions on the financial statements of the Project. Procurement 94. Procurement reviews rated procurement performance as Satisfactory throughout the life of the Project. As part of project preparation, an assessment was carried out of INIA’s capacity to implement procurement actions for the Project, and Procurement risk was rated Substantial. The Project’s Operational Manual provided detailed procurement guidelines, including: (i) the methods for the procurement of goods, non-consulting services and consultants, (ii) a clear definition of responsibilities applicable for each kind of procurement, (iii) filing procedures, (iv) management of the Procurement Plan and (v) templates for each procurement method. Page 26 of 72 The World Bank National Agricultural Innovation (P131013) Since PIU staff were unfamiliar with World Bank procurement policies and procedures, the Bank Procurement Specialist provided regular formal and informal training, and PIU procurement staff regularly participated in Bank procurement training sessions offered to project implementing entities. Some procurement delays were encountered during implementation, for example in the recruitment of firms to conduct the Impact Evaluation Baseline Study and Mid-term Evaluation. These delays slowed implementation progress in some instances and were one factor that caused the MTR to be scheduled later than originally planned, but they did not materially affect the overall performance of the Project. C. BANK PERFORMANCE Quality at Entry 95. The World Bank team that prepared the Project made a strong effort to design a project that was aligned well with the government’s desire to promote innovation at sectoral level, not only by strengthening capacity in the public sector, but also by promoting a private market for innovation services. The institutional arrangements, which ensured a strong presence at regional and subregional level, were designed to support the government’s decentralization agenda, and the reliance on the competitive grant instrument aimed to ensure that the Project’s activities would be responsive to the demands of the intended beneficiaries. Lessons learned from previous projects were considered during the design process—including both projects that had been implemented in Peru (specifically INCAGRO), and those implemented in other countries. The soundness of the design and the implementation arrangements is reflected in the fact that despite initial delays in effectiveness and start-up, the Project met all of its development objectives and exceeded many of its outcome targets. 96. Despite the strengths of the project design and planned implementation arrangements, minor shortcomings at entry must be recognized. The Results Framework had weaknesses: the PDO Indicators were imperfectly aligned with the elements of the PDO, some performance indicators were at output level, and some were not precisely defined. Also, the effort needed to provide close implementation support to sub-projects was underestimated, with the result that this function was underbudgeted. Quality of Supervision 97. The World Bank team that supervised the Project provided close support throughout implementation. Noteworthy features of the supervision effort included: (i) Regular supervision missions: The World Bank task team organized 12 supervision missions, which included field visits once the first round of sub-projects had been approved; (ii) Continuity in leadership: The Project had just two World Bank Task Team Leaders (TTL); one who led preparation, and a second who supported the Project throughout the 7.5 years of implementation; (iii) In country presence: While the TTLs were based in Washington, most other members of the multi-disciplinary task team (including a co-TTL) were based in Lima, so they were able to provide close, continuous support to the PIU; and (iv) Strong partnership with IADB. The Bank team maintained an effective working relationship with the IADB task team implementing the PSNIA sister project, including carrying out several joint supervision missions. 98. During supervision missions, the World Bank task team effectively supported the PIU in identifying and anticipating emerging challenges and in finding effective solutions. The task team responded quickly to developments in institutional circumstances affecting INIA, as well as to changes in the external environment that called for adjustments in the Project’s implementation modalities (the latter included especially the COVID- 19 pandemic). In response to the evolving internal and external circumstances, the team restructured the Project three times to extend the closing date and reallocate loan proceeds. The continuity in close supervision support contributed enormously to project performance because the constant turnover in key staff within the PIU created the need for repeated training of newcomers who were not familiar with the Project or with World Bank Page 27 of 72 The World Bank National Agricultural Innovation (P131013) operational policies and procedures. One area in which the supervision effort could have been better is safeguards: Because the task team did not sufficiently convey the implications of the ESMF to the PIU, the first round of sub-projects was launched without ESMPs for a small number of sub-projects in which they were required. As discussed above, after the oversight was identified, the task team moved swiftly to address the problem, which was resolved without an adverse environmental or social impacts having occurred. Justification of Overall Rating of Bank Performance 99. Based on the overall strong project design (recognizing the minor weaknesses in the Results Framework and in the planned implementation arrangements that had to be addressed during implementation), as well as the overall strong performance of the World Bank task team during implementation (taking into account the minor issue with safeguards supervision), overall World Bank performance is rated Satisfactory (S). D. RISK TO DEVELOPMENT OUTCOME 100. Sustainability of outcomes. Sustainability of outcomes will be determined by the interplay of two offsetting sets of factors. On the positive side, improvements in systemic capacity resulting from project- supported activities can be expected to contribute to the sustainability of the Project’s achievements even after closing. These include: (i) gains in the institutional capacity of INIA to lead the SNIA; (ii) substantial advances in the development of the market for innovation services, which now includes hundreds of private promoters (“proyectistas”) with experience in designing and implementing innovation sub-projects; and (iii) strengthened technical and managerial skills of producers and other value chain actors, including thousands of beneficiaries who have adopted improved technologies and practices, as well as SNIA member institutions. On the negative side, despite the progress that has been achieved in strengthening INIA and the SNIA, these gains could be threatened if the government fails to follow through with policy reforms and additional supporting investments. At appraisal, PNIA was viewed as the first phase of what was expected to be a multi-phase effort, and if the current government decides to pursue a different innovation strategy, the sustainability of PNIA progress may be at risk. 101. Performance and reputational risks. Several risks to the sustainability of project outcomes remain. The most salient risk is related to the uncertainty around the government’s commitment to the PNIA program. MINAGRI and INIA remain keenly interested in implementing a second phase of the Project, and the Minister of Agriculture has several times approached MEF with a formal request for funding, including through borrowing from the World Bank and the IADB. At the time this review was prepared, MEF had not formally responded to MINAGRI’s and INIA’s request. 102. Uncertain macroeconomic prospects. Despite aggressive containment measures taken by the government, the COVID-19 pandemic hit Peru extremely hard. At the time of this review, the pandemic continues to exact a heavy toll on lives and livelihoods. A combination of adverse shocks has slowed growth momentum in the Peruvian economy, causing one of the deepest recessions the country has faced in its history. It is likely that the new government will focus public investment on strengthening the public provision of health, education, and social protection services. The business case for investment in agriculture innovation will need to be strengthened to convince policy makers about the importance of continuing support for innovation in the agriculture sector during a period of fiscal austerity. Page 28 of 72 The World Bank National Agricultural Innovation (P131013) V. LESSONS AND RECOMMENDATIONS Lessons Learned 103. The multi-actor, public-private innovation network model promoted under PNIA can be an effective complement to the traditional linear model of innovation implemented by public agencies only. PNIA succeeded in mobilizing a large constellation of public and private actors in support of the agricultural innovation agenda. The work done under PNIA to strengthen the SNIA assured the participation of public agencies at regional and municipal level, research institutes and universities, producer organization, private firms, and civil society organizations in the identification, validation, adaptation, and/or dissemination of innovations, many of which did not originate within INIA. The number of innovations generated through the multi-actor network far exceeded the number that could have been generated by INIA research programs alone, even if they had received an equivalent level of funding. While not always well appreciated within Peru, PNIA is often cited internationally as a successful example of the multi-actor innovation network, and the experiences of PNIA have informed the design of several other World Bank-supported projects in other regions. It is recommended that the World Bank continue to promote this model in situations where the institutional context is conducive, in recognition of its ability to mobilize a large constellation of actors to support the innovation agenda. 104. Competitive grants can be an effective tool for directing resources to promote innovation in response to user demand. The competitive grant program implemented under PNIA was successful in directing resources toward demand-driven innovation needs and leveraging matching resources from beneficiaries. The success of the competitive grant program can be attributed to many factors: (i) public awareness campaigns that were effective in familiarizing potential beneficiaries with the opportunities that would be available; (ii) clear eligibility requirements that ensured the Project’s resources were directed to the intended beneficiaries; (iii) an objectively implemented evaluation procedure that ensured only the best sub-project proposals were selected; and (iv) a transparent approval process that was insulated from political interference and protected against elite capture. Requiring beneficiary groups to mobilize counterpart funding ensures that they have ownership and increases the likelihood that the activities being funded will be sustainable. It is recommended that the Bank continue to promote the use of competitive grants, provided measures can be put in place to ensure the technical quality of the activities that are funded, and provided that the allocation of resources can be protected from elite capture. 105. Project M&E systems must be linked with effective communications strategies to create awareness of project results and build ownership. Despite the many positive results achieved under PNIA, it has been difficult to build ownership outside of INIA and MIDAGRI for the agriculture innovation agenda. In large part this is because many of the Project’s results were achieved in remote rural locations, outside the view of the public in general and of policy makers in particular. This begs the question of how can gains such as those realized under PNIA be better documented and more effectively communicated in ways that convince policy makers that the investments are worthwhile? It is recommended that the design of future Bank-supported operations make provision for a strong communications function, so that project results captured in the M&E framework can be brought to the attention of the relevant authorities within the implementing entities and beyond, to build lasting political support and ensure sustainability. 106. The World Bank and the Inter-American Development Bank underestimated the challenges associated with implementing institutional reforms. PNIA made significant progress in strengthening INIA—by building human capacity, modernizing laboratories and research stations, and streamlining administrative procedures. Still, it must be recognized that the institutional reforms achieved under PNIA fell short of expectations. Converting the legal status of INIA to OTE offered an opportunity to overhaul INIA’s operational policies and Page 29 of 72 The World Bank National Agricultural Innovation (P131013) procedures, build a younger and more qualified labor force, phase out legacy activities that are no longer delivering results, and launch cutting-edge innovation activities, yet in the end many of the changes that were made were really quite modest. The planned institutional reforms of INIA did not go as far as expected because of resistance by powerful interest groups (especially the labor unions operating in the sector) that the authorities were unwilling or unable to confront. It is recommended that task teams engaged in future project preparation activities carefully assess political economy considerations to properly assess the risks associated with proposed institutional reform efforts. . Page 30 of 72 The World Bank National Agricultural Innovation (P131013) ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS A. RESULTS INDICATORS A.1 PDO Indicators Objective/Outcome: Conditions created in Borrower's SNIA to strengthen its effectiveness Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion PDO 1. Institutions Number 0.00 500.00 762.00 participating in the SNIA that report satisfaction with the 01-Apr-2014 01-Apr-2014 26-Feb-2021 effectiveness and cooperation in the system, measured by an independent party. Comments (achievements against targets): Exceeded (152 percent). This indicator was estimated through a satisfaction survey applied to a representative sample of SNIA institutions, taking into account a universe of 893 institutions. Different questions were applied using the Likert scale considering a score between 1 to 5 (1= lowest satisfaction and 5=maximum satisfaction with the effectiveness and cooperation in the system). The scores assigned to individual questions were summed, and institutions achieving 70 percent or more of the maximum possible value were classified as satisfied with the effectiveness and cooperation in the system. The following criteria were taken into account in designing the questions to evaluate the satisfaction level: (i) effectiveness of SNIA's coordination capacity, (ii) Effectiveness of the SNIA to promote agricultural innovation in the country in an articulated manner, and (iii) Leadership in SNIA's management. Page 31 of 72 The World Bank National Agricultural Innovation (P131013) Objective/Outcome: Effectiveness of SNIA members supported to provide or develop improved agricultural technologies Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion PDO 2. Farmers adopting new Number 0.00 20000.00 31,634.00 technologies promoted by sub-projects 01-Apr-2014 01-Apr-2014 26-Feb-2021 PDO 2 (a). Farmers adopting Percentage 0.00 35.00 39.00 new technologies promoted by sub-projects - women PDO 2 (b). Farmers adopting Percentage 0.00 20.00 42.00 new technologies promoted by sub-projects - self- identified indigenous Comments (achievements against targets): Exceeded (158 percent). The disaggregated values for technology adoption rates among women (111 percent) and indigenous people (210 percent) also exceeded the targets. This indicator measured the number of beneficiary producers who have adopted new technologies that were generated, adapted and/or promoted by PNIA through the following sub-projects: Adaptive Research (AR), Extension Services (EX) and Seed Enterprise Pilots (SE). The number of beneficiaries, who adopted those technologies was estimated according to the results of a survey applied to a sample considering a universe of 41,244 project beneficiary producers. The adoption rates were estimated considering the following criteria: (i) there was evidence that beneficiary farmers use the technology independently at their own decisions and risks, (ii) the technology is being replicated by indirect beneficiaries at the local level, and (iii) the new technology generated greater benefits than traditional technologies. The aggregated technology adoption rate for all sub-projects financed by the Project was Page 32 of 72 The World Bank National Agricultural Innovation (P131013) estimated at 76.7 percent. In disaggregated terms, the adoption rate for each subproject typology had the following values: (i) Adaptive Research: 64.2 percent, (ii) Extension Services: 79.9 percent, and (iii) Seed Enterprises: 86.2 percent. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion PDO 3. New technologies Number 0.00 61.00 111.00 emerging from research projects that have been 01-Apr-2014 01-Apr-2014 26-Feb-2021 demonstrated on farm Comments (achievements against targets): Exceeded (182 percent). This indicator measured the number of new technologies generated and/or adapted and demonstrated in farmers' fields as a result of the implementation of the Strategic Research and Adaptive Research sub-projects. New technologies were understood as any process, mechanism, knowledge, material, tool and/or equipment which was innovative and was generated as a direct result of the implementation of Strategic Research and Adaptive Research sub- projects. This result was based on the analysis of 188 subprojects. 18 and 93 technologies were deployed in farmers' fields that were attributable to Strategic Research and Adaptive Research sub-projects, respectively. The high rate of achievement can be explained in part because the proportion of sub-projects that generated at least one new technology was very high, and in part because many sub-projects ended up generating more than one new technology A.2 Intermediate Results Indicators Component: Strengthening Capacities of the National Institute for Agriculture Innovation (INIA) to Lead the National System for Agricultural In novation (SNIA) Page 33 of 72 The World Bank National Agricultural Innovation (P131013) Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Result 1. Agricultural policies Number 0.00 8.00 5.00 papers submitted for approval to MINAGRI 01-Apr-2014 01-Apr-2014 26-Feb-2021 Comments (achievements against targets): Partly achieved (63 percent) This indicator was intended to measure the degree to which the Project was able to consolidate institutional strengthening activities through policy reforms and legislative changes. Policy documents prepared with Project support and submitted to MINAGRI for approval included: (i) Supreme Decree No. 004-2018-MINAGRI, on the basis of which modifications were made to the "Regulation of Organization and Functions of the National Institute for Agricultural Innovation," changing the legal status of INIA to become a Technical Agency specialized in agriculture innovation; (ii) Diagnosis and proposal to strengthen the National Agriculture Innovation System; (iii) Proposal for a new "National Agricultural Innovation Policy;" (iv) Proposal for a new "National Plan for Agricultural Innovation;" and (v) Ministerial Resolution No. 0222-2019-MINAGRI, whereby the National Commission for Innovation and Training in Agriculture (CONICA) was created by Legislative Decree No. 1060, which regulates the National Agricultural Innovation System and establishes a procedure for the appointment CONICA members. The target was only partly achieved because the component on institutional strengthening of SNIA experienced implementation delays, and because of a lack of political support for more substantial reforms of INIA. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Result 2. Share of SNIA's Percentage 0.00 75.00 100.00 Technical Secretariat funded by INIA 01-Apr-2014 01-Apr-2014 26-Feb-2021 Comments (achievements against targets): Page 34 of 72 The World Bank National Agricultural Innovation (P131013) Exceeded (133 percent) This indicator was intended to measure the degree to which INIA would be willing to invest its own resources to ensure the continued financing of the SNIA following the end of the Project. By the project closing date, the General Directorate of Agricultural Innovation (DGIA) within INIA was leading the Technical Secretariat of the SNIA, and the Technical Secretariat was being fully funded through the DGIA budget. The DGIA budget is funded using INIA resources, in line with Article 8(d) and Article 46 of the INIA charter. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Result 3. Submissions to Number 0.00 100.00 169.00 award program 01-Apr-2014 01-Apr-2014 26-Feb-2021 Comments (achievements against targets): Exceeded (169 percent) The Innovation Prize Awards Program, known as “CARAL,” was established with Project funding as a way to increase public awareness of the importance of the agricultural innovation agenda. The first round of the CARAL Program was carried out between January and May 2020. During the awards ceremony, the Minister of Agriculture recognized 20 winning projects from among the 169 projects that had been nominated (14 winning projects were led by SNIA member institutions, and 6 winning projects were led by INIA). As established by the competition rules, the winning institutions received grants valued at US$ 20,000, US$ 15,000 or US$ 10,000, depending on the category and the ranking the proposal had obtained. The end-of-project value exceeded the target significantly as a result of the high level of participation of SNIA member institutions and INIA researchers in the CARAL Program. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Page 35 of 72 The World Bank National Agricultural Innovation (P131013) Result 4. Downloads/year Number 0.00 60000.00 229,247.00 from the website of the Technical Secretariat 01-Apr-2014 01-Apr-2014 26-Feb-2021 Comments (achievements against targets): Exceeded (382 percent) The SNIA website established under the Project was the main tool used by the Technical Secretariat of SNIA to engaged with SNIA member institutions and the more general public. The website integrated multiple functions and played a fundamental role as a platform for enhanced interaction among SNIA members. The end-of-project value far exceeded the target due to the extensive use made of the website, and also because the project implementation period ended up lasting longer than expected. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Result 5. Percentage of Percentage 0.00 60.00 88.00 institutions participating in meetings, events, etc. (of the 01-Apr-2014 01-Apr-2014 26-Feb-2021 total inventory) Comments (achievements against targets): Exceeded (147 percent) This indicator was intended to measure the participation of SNIA member institutions in official national and/or regional events organized to foster discussion and facilitate coordinated decision-making on issues related to agricultural research and innovation. The end-of-project value considerably exceeded the target due to the significant mobilization and participation of SNIA member institutions in the events organized by the Technical Secretariat. Page 36 of 72 The World Bank National Agricultural Innovation (P131013) Component: Consolidating the Market for Innovation Services through Competitive Funds Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Result 6. Collaborative sub- Number 0.00 400.00 541.00 projects - total 01-Apr-2014 01-Apr-2014 26-Feb-2021 Result 6 (a). Collaborative Number 0.00 240.00 367.00 sub-projects - extension 01-Apr-2014 01-Apr-2014 26-Feb-2021 Result 6 (b). Collaborative Number 0.00 120.00 129.00 sub-projects - adaptive research 01-Apr-2014 01-Apr-2014 26-Feb-2021 Result 6 (c). Collaborative Number 0.00 40.00 45.00 sub-projects - seeds 01-Apr-2014 01-Apr-2014 26-Feb-2021 Comments (achievements against targets): Exceeded (135 percent) This indicator measured the number of sub-projects that fully achieved their objectives. In total, 541 subprojects completed the entire implementation cycle and obtained the expected results. The end-of-project value significantly exceeded the target because more sub-projects were financed than expected at appraisal. Specifically: Page 37 of 72 The World Bank National Agricultural Innovation (P131013) • 129 adaptive research sub-projects successfully completed the entire implementation cycle and obtained the expected results. The end-of-project target was exceeded as more sub-projects were financed than expected (108 percent) • 367 extension service sub-projects completed the entire implementation cycle and obtained the expected results. The end-of-project target was exceeded as more sub-projects were financed than expected (153 percent) • 45 seed enterprise pilot sub-projects successfully completed the entire implementation cycle and obtained the expected results. The end-of- project target was exceeded as more sub-projects were financed than expected (113 percent) Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Result 7. Producers Number 0.00 40000.00 41,244.00 participating in sub-projects - total 01-Apr-2014 01-Apr-2014 26-Feb-2021 Result 7 (a). Producers Percentage 0.00 35.00 33.00 participating in sub-projects - women Result 7 (b). Producers Percentage 0.00 20.00 31.00 participating in sub-projects - self identified indigenous Comments (achievements against targets): Achieved (103 percent) This indicator measured the number of direct beneficiaries reached by the 541 sub-projects that completed the entire implementation cycle and achieved the expected results. Specifically: Page 38 of 72 The World Bank National Agricultural Innovation (P131013) • 33 percent of the direct beneficiaries were women. This percentage was slightly lower than expected (94 percent) • 31 percent of the direct beneficiaries self-identified as indigenous people. This percentage was higher than expected (155 percent) Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Result 8. Strategic research Number 0.00 60.00 59.00 subprojects co-funded 01-Apr-2014 01-Apr-2014 26-Feb-2021 Comments (achievements against targets): Substantially achieved (98 percent) This indicator measured the number of sub-projects which were successfully implemented and achieved the expected targets. 59 adaptive research sub- projects completed the implementation process. One adaptive research sub-project had to be cancelled due to the outbreak of the COVID19 pandemic and the related travel restrictions, which prevented the planned program of activities from being carried out. . Component: Strengthening Strategic Capacities within the National Agricultural Innovation System Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Result 9. Papers accepted in Number 0.00 60.00 136.00 indexed scientific journals 01-Apr-2014 01-Apr-2014 26-Feb-2021 Page 39 of 72 The World Bank National Agricultural Innovation (P131013) Comments (achievements against targets): Exceeded (227 percent) This indicator measured the number of scientific publications produced as a result of project-supported research subprojects and accepted for publication in indexed journals. At project completion, articles had been accepted by scientific journals such as Frontiers in Plant Science, Spermova, Agricultural Water Management, and American Journal of Potato Research. . Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Result 10. Training days to Number 0.00 7000.00 76,887.00 extension and agriculture professionals 01-Apr-2014 01-Apr-2014 26-Feb-2021 Comments (achievements against targets): Exceeded (1,098 percent) This indicator measured the number of research and extension agents trained by the Project, for the purpose of strengthening the human capital of SNIA member institutions and improving the ability of SNIA to promote, generate and disseminate research and innovations in the Peruvian agriculture sector. The indicator captured the achievements of 27 competency-based training sub-projects, and it reflected also the strengthened capacity of the professionals who completed training courses with support from the Project. Scholarships financed under the Project enabled 81 professionals to obtain Masters degrees, and internships financed under the Project allowed 291 professionals to receive training at the national and international level. The end-of-project value far exceeded the target because during implementation in response to higher than expected demand the decision was taken to increase capacity strengthening activities (more than 250 scholarships were awarded, an increase of more than 300 percent over the original target out of 116 scholarships). Page 40 of 72 The World Bank National Agricultural Innovation (P131013) Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Result 11a. Agricultural Number 0.00 31.00 57.00 professionals who receive Master's Degrees through 01-Apr-2014 01-Apr-2014 26-Feb-2021 scholarship programs Result 11a (a). Agricultural Number 0.00 11.00 25.00 professionals who receive Master's Degrees through 01-Apr-2014 01-Apr-2014 26-Feb-2021 scholarship programs - women Comments (achievements against targets): Exceeded (184 percent) This indicator measured the number of professionals trained in specialized topics related to agricultural research and innovation. 57 researchers completed their Master's degree through scholarships financed by the Project. As a result, the end-of-project value exceeded the target. Efforts made by the Project to encourage women to compete for scholarships for graduate study were successful. Out of the 57 professionals who received their Master's degree, 25 were women, exceeding the end-of-project target value (227 percent) Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Result 11b. Agricultural Number 0.00 80.00 291.00 Page 41 of 72 The World Bank National Agricultural Innovation (P131013) professionals who receive 01-Apr-2014 01-Apr-2014 26-Feb-2021 internships / exchanges through scholarship programs Result 11b (a). Agricultural Number 0.00 28.00 107.00 professionals who receive internships / exchanges 01-Apr-2014 01-Apr-2014 26-Feb-2021 through scholarship programs: females Comments (achievements against targets): Exceeded (365 percent) This indicator measured the number of research and extension professionals who received training through internships and research exchanges financed by the Project. At project completion, 291 professionals had received training. The end-of-project value far exceeded the target because when it became evident that demand far exceeded expectations, the decision was taken to finance group internships and research exchanges. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Result 12. Farmers included Number 0.00 450000.00 745,212.00 among the potential beneficiaries of the selected 01-Apr-2014 01-Apr-2014 26-Feb-2021 research projects Comments (achievements against targets): Exceeded (166 percent) Page 42 of 72 The World Bank National Agricultural Innovation (P131013) This indicator was intended to measure the number of beneficiaries who could potentially adopt the technological innovations made available through the strategic research sub-projects. It was included in the Results Framework because it was recognized that innovations produced by strategic research sub- projects were unlikely to have made their way into farmers' fields by the project closing date, due to the lag needed for adoption. The potential beneficiaries were estimated by researchers who led the strategic research subprojects, considering the scope of the innovations that were being developed and the stage of the research. Page 43 of 72 The World Bank National Agricultural Innovation (P131013) B. KEY OUTPUTS BY COMPONENT Objective/Outcome 1 – Create the adequate conditions in the borrower’s SNIA in order to support the effectiveness of its member organizations. 1. Number of Institutions participating in the SNIA that report satisfaction with the Outcome Indicators effectiveness and cooperation in the system, measured by an independent party. Component 1: Strengthening Capacity of INIA to lead the SNIA 1. Number of Agricultural policies papers submitted for approval to MINAGRI 2. Percentage share of SNIA’s Technical Secretariat funded by INIA 3. Number of submissions to award program 4. Number of downloads/year from the website of the Technical Secretariat 5. Percentage of institutions participating in meetings, events, etc. (of the total inventory) Component 3: Strategic Capacities in the SNIA (Extension and Technical Assistance Capacity Building; Postgraduate Fund) Intermediate Results Indicators 1. Number of papers accepted in indexed scientific journals 2. Number of training days to extension and agriculture professionals 3. Number of agricultural professionals who receive Master’s Degrees through scholarship programs 4. Number of agricultural professionals who receive Master’s Degrees through scholarship programs – women 5. Number of agricultural professionals who receive internships / exchanges through scholarship programs 6. Number of agricultural professionals who receive internships / exchanges through scholarship programs: females Page 44 of 72 The World Bank National Agricultural Innovation (P131013) Component 1: Strengthening Capacity of INIA to lead the SNIA 1. No. of agricultural policy papers submitted for approval to MINAGRI: 5 (target: 8) 2. Percentage of SNIA’s Technical Secretariat funded by INIA: 100 (target: 75 percent) 3. No. of submissions to award program: 169 (target: 100) 4. No. of downloads/year from the website of the Technical Secretariat: 229,247 (target: 60,000) 5. Percentage of institutions participating in meetings, events, etc. (of the total inventory): 88 (target: 60 percent) Component 3: Strategic Capacities in the SNIA (Extension and Technical Assistance Capacity Building; Postgraduate Fund) Key Outputs by Component (linked to the achievement of Objective/Outcome 1) 1. No. of papers accepted in indexed scientific journals: 136 (target: 60) 2. No. of training days to extension and agriculture professionals: 76,887 (target: 7,000) 3. No. of agricultural professionals who receive Master’s Degrees through scholarship programs: 57 (target: 31) 4. No. of agricultural professionals who receive Master’s Degrees through scholarship programs – women: 25 (target: 11) 5. No. of agricultural professionals who receive internships / exchanges through scholarship programs: 291 (target: 80) 6. No. of agricultural professionals who receive internships / exchanges through scholarship programs: females: 107 (target: 28) 7. No. of potential agricultural professional beneficiaries: 745,212 (target: 450,000) Page 45 of 72 The World Bank National Agricultural Innovation (P131013) Objective/Outcome 2 – Providing or developing improved agricultural technologies. 1. Number of farmers adopting new technologies promoted by sub-projects 2. Percentage of Farmers adopting new technologies promoted by sub-projects – women 3. Percentage of Farmers adopting new technologies promoted by sub-projects – Outcome Indicators self-identified indigenous 4. Number of new technologies emerging from research projects that have been demonstrated on farm 5. Number of farmers included among the potential beneficiaries of the selected research projects Component 2: Consolidating the Market for Innovation Services 1. Number of collaborative sub-projects – total 2. Number of collaborative sub-projects – extension 3. Number of collaborative sub-projects – adaptive 4. Number of collaborative sub-projects – seeds Intermediate Results Indicators 5. Number of producers participating in sub-projects – total 6. Percentage of producers participating in sub-projects – women 7. Percentage of producers participating in sub-projects – self identified indigenous Component 3: Strategic Capacities in the SNIA (Funds for Strategic Research) 1. Number of strategic research subprojects co-funded Component 2: Consolidating the Market for Innovation Services Key Outputs by Component 1. No. of collaborative sub-projects – total: 541 (target: 400) (linked to the achievement of Objective/Outcome 2) 2. No. of collaborative sub-projects – extension: 367 (target: 240) 3. No. of collaborative sub-projects – adaptive investigation: 129 (target: 120) Page 46 of 72 The World Bank National Agricultural Innovation (P131013) 4. No. of collaborative sub-projects – seeds: 45 (target: 40) 5. No. of producers participating in sub-projects – total: 41,244 (target: 40,000) 6. Percentage of producers participating in sub-projects – women: 33 (target: 35) 7. Percentage of producers participating in sub-projects – self identified indigenous: 31 (target: 20) Component 3: Strategic Capacities in the SNIA (Funds for Strategic Research) 1. Number of strategic research subprojects co-funded: 59 (target: 60) Page 47 of 72 The World Bank National Agricultural Innovation (P131013) ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION A. TASK TEAM MEMBERS Name Role Preparation Willem Janssen Task Team Leader(s) Selene del Rocio La Vera Procurement Specialist(s) Patricia De la Fuente Hoyes Financial Management Specialist Mary Lisbeth Gonzalez Social Specialist Raul Tolmos Social Specialist Supervision/ICR Michael Morris, Griselle Felicita Vega Task Team Leader(s) Selene del Rocio La Vera, Lucy Violeta Castro Procurement Specialist(s) Nelly Ikeda Financial Management Specialist Kennan W. Rapp Social Specialist Rahmoune Essalhi Team Member Elke Pinedo Castillo Procurement Team Juan Paulo Rivero Zanatta Team Member Ximena Rosio Herbas Ramirez Environmental Specialist Maria Jose Carreras Gamarra Team Member Javier Augusto Avila Molero Social Specialist Page 48 of 72 The World Bank National Agricultural Innovation (P131013) B. STAFF TIME AND COST Staff Time and Cost Stage of Project Cycle No. of staff weeks US$ (including travel and consultant costs) Preparation FY12 2.525 21,586.75 FY13 26.382 140,434.12 FY14 29.056 125,596.27 Total 57.96 287,617.14 Supervision/ICR FY14 5.855 21,508.97 FY15 26.512 139,577.57 FY16 31.543 159,298.95 FY17 21.715 109,154.44 FY18 28.286 185,577.92 FY19 25.632 177,707.93 FY20 21.775 139,766.15 Total 161.32 932,591.93 Page 49 of 72 The World Bank National Agricultural Innovation (P131013) ANNEX 3. PROJECT COST BY COMPONENT Amount at Approval Actual at Project Percentage of Approval Components (US$M) Closing (US$M) (US$M) Strengthening Capacities of the National Institute for Agriculture Innovation (INIA) 23.34 23.34 100 to Lead the National System for Agricultural In novation (SNIA) Consolidating the Market for Innovation Services through 57.91 57.81 99.8 Competitive Funds Strengthening Strategic Capacities within the 40.31 40.31 100 National Agricultural Innovation System Project Management 3.57 7.14 200 Total 125.13 128.60 102.8 Page 50 of 72 The World Bank National Agricultural Innovation (P131013) ANNEX 4. EFFICIENCY ANALYSIS 1. Ex-post cost-benefit analysis was conducted to determine project efficiency. The cost-benefit analysis included both financial analysis carried out using actual market prices inclusive of taxes, subsidies, and other distortions, as well as economic analysis carried out using economic prices adjusted in some instances to correct for taxes, subsidies, and other distortions known to be present in the Peruvian economy. The results of the financial analysis show the returns to Project-supported activities from the perspective of implementing entities, participants, private enterprises, and beneficiaries, while the results of the economic analysis show the “true” returns to Peruvian society, which will be of interest particularly to policy makers. Project costs and benefits 2. The ex-post cost-benefit analysis considered incremental costs and benefits attributable to the Project, which were identified based on the activities taking place during the life of the Project as well as developments expected to occur in future years, as reflected in the theory of change. The benefit streams considered by the ex-post cost-benefit analysis are the estimated incremental value of production and income realized by project beneficiaries as a result of taking up innovations developed and made available by the Project or as the result of having acquired new skills from participating in training activities financed by the Project. The main expected incremental costs and benefits for each type of sub-project and activity, organized by the Project components, are summarized in Table 4.1. Table 4.1. Summary of costs and benefits used in efficiency analysis Subprojects/activities Benefits/Costs Component 1. Strengthening the capacity of INIA to lead the SNIA Incremental benefits were not identified for Component 1, since it is assumed that the activities implemented under this component are aimed at creating the enabling institutional capacities and regulatory conditions whose direct and indirect benefits are reflected in the benefits generated by activities financed under Components 2 and 3. Costs associated with the implementation of Component 1 were included in the overall cost-benefit analysis as part of total project costs. Component 2. Consolidating the market for innovation services Adaptive research sub-projects Benefits: Increased revenues due to increased yields, increased production, improved quality, reduced losses, and/or increased prices Costs: Labor (60% of total cost), tradable goods and services used as Extension sub-projects production inputs (40% of the total cost). Labor characterized as (i) unskilled (extension sub-projects) or (ii) semi- Seed enterprise pilots skilled (adaptative research sub-projects, seed enterprise pilots) Component 3. Strategic capacities in the SNIA Benefits: Increased revenues projected for future adopters of innovations, Strategic research sub-projects plus increased incomes of researchers directly involved in implementation of strategic research sub-projects. Page 51 of 72 The World Bank National Agricultural Innovation (P131013) Subprojects/activities Benefits/Costs Costs: Increased labor and input costs projected to be incurred by future adopters of innovations. Note: It is assumed that the innovations developed by strategic research sub-projects will be adapted and then disseminated using the same mechanisms found in adaptive research and extension sub-projects. Benefits: Increased income of extensionists after receiving training provided by the Project through capacity building sub-projects. Capacity building sub-projects Costs: Opportunity cost of training, calculated as income foregone during participation in capacity-building sub-projects. Benefits: Increased income of agricultural professionals as a direct result of the training. Scholarships and internships Costs: Opportunity cost of graduate study or internship, calculated as income foregone during participation in graduate program or internship activity. 3. For all of the benefit streams, the timing of costs and benefits was adjusted to reflect the duration of the sub-project or activity implementation period and the expected distribution through time of the benefits. 4. Several additional benefit streams were not considered, including (i) future productivity gains that may be realized by producers and other agricultural value chain actors as the result of a strengthened SNIA; (ii) future productivity gains that may be realized by producers and other agricultural value chain actors due to the presence of a new cadre of more knowledgeable and more capable researchers, extension agents, and producers; and (iii) future productivity gains that may be realized by producers and other agricultural value chain actors thanks to the greater availability of information and knowledge about the innovations developed and validated under the Project. While difficult to quantify and value, these additional benefit streams could be significant, and for that reason the results of the ex-post cost-benefit analysis can be considered conservative. Data sources and sampling strategy 5. The ex-post cost-benefit analysis relied on information from three sources: (i) a survey carried out by a consultant engaged by the PIU of 70 collaborative sub-projects randomly selected from among the 541 collaborative sub-projects financed using the competitive grants mechanism; (ii) a survey carried out as part of the independently conducted final impact evaluation of 190 researchers and extension agents who had benefited from capacity strengthening activities financed by the Project; and (iii) project records. (i) Component 2 6. To estimate the costs and benefits associated with activities financed under Component 2, a sample of 70 sub-projects was randomly selected from among the 541 collaborative sub-projects using a stratified sampling methodology to ensure adequate representation of all three types of collaborative sub-projects. The overall composition of the sample is shown in Table 4.2. The breakdown by commodity groupings is shown in Table 4.2. Page 52 of 72 The World Bank National Agricultural Innovation (P131013) Table 4.2. Composition of sample of collaborative sub-projects Sub-projects completed Sub-projects sampled Sub-projects sampled (number) (number) (percent) Adaptative research 129 18 14 % Extension 367 43 12 % Seed enterprise pilots 45 9 20 % Total 541 70 13 % Table 4.3. Disaggregation of sample by commodity groupings Sample size Subprojects/commodity groupings (subprojects) Adaptative research 18 Livestock (milk, meat, and/or wool) 6 Permanent crops 8 Annual crops 4 Extension 43 Livestock (milk, meat, and/or wool) 10 Permanent crops 25 Annual crops 8 Seed enterprise 9 Livestock (milk, meat, and/or wool) 6 Annual crops 3 Total 70 7. Primary data collection activities were carried out by independent consultants, with assistance from PNIA M&E staff, who facilitated contacts with representatives from the 70 collaborative sub-projects included in the sample. Interviews were carried out between December 2020 and February 2021 using customized survey instruments developed for each type of sub-project (adaptive research, extension, seed enterprise pilots). The survey instruments included questions designed to allow comparison of the “with project” and “without project” scenarios. For every sub-project, data were collected on yields, production volume, value of sales, prices paid and received, production costs and other variables. The data collected at sub-project-level were supplemented with information taken from the project records to allow sub-project-level results to be aggregated to the overall project level, including total number of sub-projects financed by the Project, direct and indirect investment costs, project implementation period, technology adoption rates, etc. Page 53 of 72 The World Bank National Agricultural Innovation (P131013) (ii) Component 3 8. To estimate the costs and benefits associated with activities financed under Component 3, data were obtained from two main sources. • Strategic research sub-projects: Strategic research sub-projects differ from collaborative sub- projects because the innovations generated usually have not yet been developed into commercial products, so it is not possible to measure directly the costs and benefits associated with adoption. According to the theory of change, a certain share of the innovations generated through strategic research sub-projects will eventually be adapted for use in specific contexts and disseminated with the help of extension programs. The cost-benefit analysis for strategic research sub-projects therefore was able to use the data obtained through the sample of 70 collaborative sub-projects, combined with assumptions about the probability of research success, period required for adaptive research and local validation, and adoption rates. • Capacity building sub-projects and scholarships and internships: Costs and benefits for the capacity strengthening activities related mainly to the incomes of beneficiaries – both the income foregone during the time the beneficiaries participated in the capacity strengthening activity (cost), or the additional income they reported earning following their participation (benefit). Information about the incomes earned by participants in project-supported capacity building activities was obtained through a survey of 190 researchers and extensionists carried out by the consulting firm that prepared the final impact evaluation of the Project. (For additional details on the final impact evaluation, see Annex 6.) Analytical approach 9. The ex-post cost-benefit analysis was designed to estimate standard measures of project worth including Net Present Value (NPV) at private and social prices over a 10-year period, Financial and Economic Internal Rates of Return (EIRR and FIRR), Benefit/Cost ratio, payback period, and incremental benefits per beneficiary in terms of present value. These indicators were estimated: (i) for the overall Project, (ii) by project components, (iii) by sub-project type, (iv) by commodity grouping / value chain, and (v) by type of training provided to extension and research agents. Sensitivity analysis was carried out to explore the likely impacts of possible changes in key variables such as benefits, costs, technology adoption rate, training days, success training rate, social discount rate and evaluation period. Key assumptions and baseline parameters 10. All cash flows were estimated using nominal prices. The exchange rate for converting from soles to US dollars was determined considering the annual average value that prevailed during project Page 54 of 72 The World Bank National Agricultural Innovation (P131013) implementation. In the baseline scenarios, the cost of capital was set at 10.5 12 and the social discount rate at 8 percent, 13 and the evaluation period was 10 years. 11. The streams of costs and benefits associated with collaborative sub-projects and strategic research sub-projects were adjusted to reflect the average implementation periods for each type of sub- project (the values were estimated through independent consultancies) and the associated technology adoption rates (estimated through surveys of participants and shown in Table 4.4). Likewise, the cash flows associated with the capacity building sub-projects and the scholarships and internships initiative were adjusted based on data collected through the survey of extensionists and researchers about the duration of training activities and the rate of success. Table 4.4. Innovation adoption rates, by sub-project type and funding cycle (percent) Funding cycle Total Sub-project type 2015 2016 2017 (n = 466) (n = 154) (n = 156) (n = 156) Adaptative research (n = 100) 37.5 75.3 59.3 64.2 Extension (n = 320) 87.2 77.6 79.8 79.9 Seed enterprise pilots (n = 46) 53.8 95.1 90.0 86.2 Total (n = 466) 77.1 79.1 73.9 76.7 12. The conversion factors used to convert from market prices to social prices were those stipulated by the Peruvian public investment system: (i) currency: 1.08; (ii) tradable goods: 0.867; (iii) investment: 0.83; (iv) operating costs: 0.8475; (v) skilled labor: 0.81; (vi) semi-skilled labor: 0.665; and (vii) unskilled labor: 0.5. 13. Positive externalities generated by the Project, such as technology spillovers and environmental and social benefits, were not included in the analysis due to the complexity of quantifying those externalities and the limited availability of information. Overall results – Baseline analysis 14. For the overall Project investment, the estimated values of standard measures of project worth are presented in Table 3. 12 Minimum cost of capital for the investments in the agriculture sector. 13 Recommended value for the Public investment system. Page 55 of 72 The World Bank National Agricultural Innovation (P131013) Table 4.4. Ex post cost-benefit analysis – Summary results Financial indicators Financial Net Present Financial Internal Rate Payback Incremental NPV Benefit/Cost Value (FNPV) of Return (FIRR) period per beneficiary Ratio (USD million) (percent) (years) (USD) 48.2 24.6 1.26 7.3 1,121 Economic indicators Economic Net Economic Internal Payback Incremental NPV Benefit/Cost Present Value (ENPV) Rate of Return (EIRR) period per beneficiary Ratio (USD million) (percent) (years) (USD) 86.2 31.2 1.48 6.8 2,007 15. The large positive net present values (NPVs) and high economic internal rate of return (EIRR) show that overall, the resources invested in the Project generated positive returns on investment, both financially and economically. The estimated EIRR is higher than the EIRR estimated at the time PNIA was prepared (EIRR = 26 percent). It is higher than the EIRRs reported by similar initiatives in Peru and elsewhere in the region, including the INCAGRO project (EIRR = 24.2 percent) and the Program for Innovation and Agricultural Services (Programa de Innovación y Servicios Agropecuarios; PISA) in Bolivia (EIRR = 18.8 percent). It falls toward the higher end of the ranges reported in global studies of returns to investment in agricultural research. 14, 15 Overall results – Sensitivity analysis 16. Sensitivity analysis was carried out to determine the robustness of these results under a range of possible changes in key parameters. First, the analysis was re-run assuming a project evaluation period of 20 years, to allow more time for innovations generated by the Project to disseminate and for income increases earned by more highly trained researchers and extensionists to accumulate. As shown in Table 4.5, extending the evaluation period increases the return on investment. 14 Alston, J., Chang-Kang, C., Marra, M., Pardey, P. and T. Wyatt. 2000 Meta-analysis of returns to agricultural research and development: Ex Pede Herculem? IFPRI Research Report 113. 15 Pardey, P., Alston, J. and R. Piggott (eds.). 2006. Agricultural research investment in the developing world: Too Little, too late? IFPRI. Page 56 of 72 The World Bank National Agricultural Innovation (P131013) Table 4.5. Sensitivity analysis – Effects of a change in evaluation period 10 years 20 years Financial indicators Financial NPV 48,159,367 178,039,847 Financial IRR 24.6% 33.3% Benefit/Cost ratio 1.26 1.57 Payback period 7.3 7.3 FNPV/beneficiary family 1,121 4,144 Economic Indicators Economic NPV 86,233,165 285,524,537 Economic IRR 31.2% 38.9% Benefit/Cost ratio 1.48 1.91 Payback period 6.8 6.8 ENPV/beneficiary family 2,007 6,646 17. Sensitivity analysis was also carried out to test the robustness of the results to changes in other key parameters. Table 4.6 shows how the EIRRs change in responses to progressive decreases and increases in the values of key parameters, including producer revenues, production costs, technology adoption rates, and the duration of training activities and their success rates. Table 4.6. Sensitivity analysis - Effects of changes in selected key parameters EIRR values (percent) Parameter -20% -10% -5% 0% 5% 10% 20% Producer revenues 19.37 25.65 28.51 31.22 33.80 36.27 40.92 Production costs 35.55 33.43 32.33 31.22 30.08 28.91 26.50 Technology adoption rate 24.78 28.10 29.68 31.22 32.71 34.17 36.97 Training days – Scholars and interns 31.29 31.25 31.23 31.22 31.20 31.18 31.15 Training days – Extensionists 31.43 31.33 31.27 31.22 31.16 31.11 31.00 Success rate of training 30.56 30.89 31.05 31.22 31.38 31.54 31.87 18. The sensitivity analysis revealed that the EIRRs are somewhat sensitive to changes in producer revenues, production costs, and technology adoption rates, but they are relatively insensitive to changes in other parameters including the amount of training provided to extensionists and researchers. Results – Disaggregated 19. To estimate the returns to different types of activity financed by the Project—information that could improve the design of an eventual follow-on operation or other future projects and programs to Page 57 of 72 The World Bank National Agricultural Innovation (P131013) support agricultural innovation in Peru and elsewhere—the measures of project worth were calculated separately by project component, by collaborative sub-project type, by commodity grouping / value chain, and for the different activities financed under Component 3. (c) Returns by Project component 20. Returns to individual project components are shown in Table 4.7. Table 4.7. Economic and financial indicators by project component Financial indicators FNPV FIRR B/C Payback period NPV/beneficiary Component 1 --- --- --- --- --- Component 2 31,789,232 31.0% 1.33 6.4 770.8 Component 3 18,118,656 19.8% 1.18 7.7 --- Economic indicators ENPV EIRR B/C Payback period NPV/beneficiary Component 1 --- --- --- --- --- Component 2 51,210,072 37.5% 1.68 6.1 1,242 Component 3 37,505,469 28.3% 1.61 7.5 --- 21. Component 1: Separate cost-benefit analysis was not carried out for Component 1, since it is assumed that the activities implemented under this component are aimed at creating the enabling institutional capacities and regulatory conditions whose direct and indirect benefits are reflected in the benefit streams generated under Components 2 and 3. The costs of Component 1 were however included in the cost-benefit analysis for the overall Project. 22. Component 2: The collaborative sub-projects financed under Component 2 generated attractive returns. Aggregating across all three types of collaborative sub-projects, it is estimated that after 10 years have elapsed and the Project-validated innovations will have had time to diffuse, the investments made under Component 2 to consolidate the market for innovation services will have generated an estimated FIRR of 31.0 percent and an estimated EIRR of 37.5 percent. 23. Component 3: The investments made under Component 3 to strengthen the strategic capacities in the SNIA also generated attractive returns. The strategic research sub-projects, capacity strengthening sub-projects, and scholarships and internships program will generate an estimated FIRR of 19.8 percent and an estimated EIRR of 28.3 percent. 24. Component 4: Separate cost-benefit analysis was not carried out for Component 4, since it is assumed that the activities implemented under this component contributed to the implementation of the activities carried out under Components 1, 2, and 3. The costs of Component 4 were however included in the cost-benefit analysis for the overall Project. Page 58 of 72 The World Bank National Agricultural Innovation (P131013) (d) Returns by type of collaborative sub-project financed under Component 2 25. By collaborative fund type: To determine the relative profitability of different types of collaborative sub-projects financed through competitive funds, the measures of project worth were calculated separately by type of sub-project. Returns to different types of collaborative sub-projects are shown in Table 4.8. Table 4.8. Economic and financial indicators by type of collaborative sub-project Financial indicators NPV FIRR B/C Payback period NPV/beneficiary Adaptative research sub-projects 962,243 10.6% 1.00 8.3 126.6 Extension sub-projects 26,067,974 34.8% 1.23 6.2 903.9 Seed enterprise sub-projects 5,623,124 48.3% 1.41 6.0 2901.5 Economic indicators NPV EIRR B/C Payback period NPV/beneficiary Adaptative research sub-projects 3,811,692 18.5% 1.16 7.4 501 Extension sub-projects 42,012,400 43.3% 1.47 5.8 1,457 Seed enterprise sub-projects 8,235,429 58.2% 1.62 5.5 4,249 26. By agricultural commodity grouping / value chain: To determine payoffs to investment in different agricultural sub-sectors, the measures of project worth also were calculated separately by commodity grouping / value chain. The returns on investment were found to be positive for all of the main commodity groupings and value chains, with exception of the value chain for camelid fiber (alpaca and vicuna). Investment in collaborative sub-projects targeting permanent crops such as coffee, cacao, and fruits generated the highest returns (EIRR of 46.9 percent on average). Investment in collaborative sub- projects targeting annual crops such as potato, quinoa, and maize also generated attractive returns (EIRR of 35.5 percent on average). Attractive returns were also generated by investments in collaborative sub- projects targeting livestock and livestock products (average EIRRs of 47.4 percent, 24.9 percent, and 16.0 percent for meat, dairy, and dual-purpose livestock). Page 59 of 72 The World Bank National Agricultural Innovation (P131013) Table 4.9. Economic and financial indicators by commodity grouping / value chain Financial indicators NPV FIRR B/C Payback period NPV/beneficiary Livestock – camelids fiber 227,519 0.8% 0.77 9.9 371 Livestock – dual purpose 612,679 19.5% 1.20 7.6 577 Livestock – meat production 2,336,273 30.8% 1.17 6.5 1,169 Livestock – dairy production 13,956 10.6% 1.00 8.8 3.8 Permanent crops 22,656,927 38.2% 1.40 6.0 904 Annual crops 2,923,639 24.4% 1.16 7.0 490 Other products 2,755,164 33.7% 1.29 6.3 961 Economic indicators NPV EIRR B/C Payback period NPV/beneficiary Livestock – camelids fiber 115,402 3.4% 0.88 9.4 188 Livestock – dual purpose 1,233,163 24.9% 1.44 7.0 1,161 Livestock – meat production 5,280,337 47.4% 1.43 5.7 2,641 Livestock – dairy production 1,777,222 16.0% 1.18 7.9 485 Permanent crops 35,540,867 46.9% 1.69 5.6 1,418 Annual crops 6,570,667 35.5% 1.40 6.2 1,101 Other products 4,801,989 44.1% 1.56 5.8 1,676 (e) Returns by type of sub-project/activity financed under Component 3 27. Returns to the different types of activities financed under Component 3 are shown in Table 4.7. Table 4.10. Economic and financial indicators by Component 3 activity Financial indicators NPV FIRR B/C Payback period NPV/beneficiary Strategic research sub-projects 11,410,809 17.9% 1.18 8.0 ---- Capacity building sub-projects 1,996,224 22.0% 1.44 6.5 1,639 Scholarship and internships 1,661,439 20.3% 1.33 8.3 4,774 Economic indicators NPV FIRR B/C Payback period NPV/beneficiary Strategic research sub-projects 26,851,292 22.9% 1.44 7.7 ---- Capacity building sub-projects 2,836,261 23.9% 1.65 6.5 2,329 Scholarship and internships 2,496,187 21.9% 1.51 7.4 7,173 Page 60 of 72 The World Bank National Agricultural Innovation (P131013) 28. Strategic research sub-projects: Measures of project worth were calculated separately for the strategic research sub-projects. Compared to the collaborative sub-projects, strategic sub-projects pose challenges, because the innovations they produce usually have not yet been developed into commercial products, so additional assumptions are necessary regarding costs, benefits, and adoption timeline. Still, based on assumptions investments made in strategic research subprojects generated attractive returns (FIRR of 17.9 percent on average, EIRRs of 22.9 percent on average). 29. Capacity building sub-projects: Measures of project worth were calculated for the capacity- building sub-projects. For these sub-projects, benefits were estimated based on the increases in income reported by 190 researchers and extensionists who benefited from Project-supported training and capacity building activities. Investments in the capacity building sub-projects generated attractive returns (FIRR of 22.0 percent, EIRRs of 23.9 percent on average). 30. Scholarships and internships: Returns were calculated on the investments made through the Scholarship Fund in graduate training and internships. Again, benefits were estimated based on the increases in income reported by researchers and agricultural professionals who benefited from capacity building financed through the PNIA Scholarship Fund. The return on investments in scholarships and internships was determined to be attractive (FIRR of 20.3 percent, EIRR of 21.9 percent). Efficiency analysis – Summary and conclusions 31. The results of the ex-post cost-benefit analysis support the following general conclusions: • The overall project investment generated positive returns, in both financial and economic terms. The estimated FIRR exceeds the cost of capital, and the estimate EIRR exceeds the social discount rate used in Peru. • The measures of project worth estimated by the ex-post cost-benefit analysis surpass the equivalent measures estimated at appraisal using ex-ante cost benefit analysis. The better-than- anticipated results are due mainly to the conservative assumptions used during appraisal for increases in yields and production attributable to innovations attributable to the Project. • All types of collaborative sub-projects financed under Component 2 (adaptive research, extension, seed enterprise pilots) generated positive returns. • Collaborative sub-projects focused on all commodity groupings and value chains generated positive returns, with the exception of the small number of sub-projects dedicated to work on improving the quality of camelid wool. These sub-projects did not generate positive returns to the extremely long period needed to realize benefits from animal breeding. • Capacity strengthening activities supported under Component 3 generated positive returns. The survey of 190 researchers and extensionist who benefited from Project-supported training reported significant increases in their incomes that they attributed to the training. Page 61 of 72 The World Bank National Agricultural Innovation (P131013) ANNEX 5. BORROWER, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS Borrower’s comments on the PNIA ICR Valentin Cobeñas, Director of Credits, Ministry of Economics and Finance We submit for your consideration the following comments: 1. According to the IBRD loan contract, the project is called: Consolidation of the National Agrarian Innovation System Project. a 2. In the Implementation Completion Report, the project is referred to as: National Agricultural Innovation System Support Project. a 3. The amount disbursed reported as US $ 38,920,493 coincides with the Treasury Department’s Disbursement Record. A report of the Disbursement Record is attached. 4. The IBRD loan became effective on March 12, 2015. The ICR reports the effectiveness as September 9, 2012. b 5. The ICR refers to MINAGRI. Today the reference should be MIDAGRI. c 6. Regarding sub-projects and other activities that could not be concluded by the project closing date due to the Covid-19 pandemic, by means of Directorial Resolution No. 0099-2021-INIA (09.30.2021), pending actions such as contracts have been continued after the closing of the IADB loan (financed the Consolidation Project of the National System of Agrarian Innovation) and IBRD loan (financed the Project for the Improvement of Strategic Services of Agrarian Innovation) for the National Program of Agrarian Innovation (PNIA). 7. We agree with the Implementation Completion Report and the reported results. Notes: a The name of the project as translated into Spanish appears slightly differently in the Spanish version of the loan agreement (an unofficial document prepared as a courtesy for the Borrower) and in the ICR. b As noted in the ICR, the effectiveness date appearing in the World Bank system refers to the effectiveness date of the Spanish Trust Fund that financed several background studies during project preparation. c As noted in the ICR, during project implementation, the name of the Ministry of Agriculture and Irrigation (MINAGRI) was formally changed to the Ministry of Agricultural Development and Irrigation (MIDAGRI). Page 62 of 72 The World Bank National Agricultural Innovation (P131013) Implementation Entity’s comments on the PNIA ICR Consolidated comments transmitted by Juan Adolfo Altamirano Quispe, Vice Minister of Policies and Agricultural Development Ministry of Agriculture Development and Irrigation It is my pleasure to address you, to greet you cordially and at the same time let you know that the World Bank Project Manager sent the Vice Ministry of Policies and Supervision of Agricultural Development of the Ministry of Agricultural Development and Irrigation the preliminary version of the Implementation Completion and Results report for the National Program for Agricultural Innovation (PNIA). In this regard, after the review carried out by the technical areas of the National Institute of Agricultural Innovation, some comments were made that are attached. In general, it is concluded that the document contains abundant consolidated and very important information, and I am pleased to express our agreement on its content. Comments deemed relevant for the ICR a Topic discussed in the ICR MIDAGRI comment Lack of consistency in Indeed, the lack of continuity in the top management of MINAGRI (now programmatic leadership MIDAGRI) and in the leadership of INIA did not help in the beginning of the project to establish a properly established leadership. For these and other reasons, especially of an administrative nature, the program was implemented with certain delays in its operational plan. In this context, one aspect to consider is that the management of normative instruments of vital importance for the SNIA, such as the National Policy for Agricultural Innovation and the National Plan for Agricultural Innovation, were prepared in the third year after the project began, when this should have been done from the beginning. To date, the national agricultural innovation policy continues to be in the process of being approved by CEPLAN. Effectiveness of the multi-actor It is affirmed among the lessons learned that the model of the public-private network innovation model innovation network and of multiple actors promoted under the PNIA can be an effective complement to the traditional linear model of innovation implemented only by public organizations. This is a relevant aspect that, in addition to the results shown in the report, we consider important for the strengthening of the governing body and the consolidation of the SNIA, by managing the implementation of a second stage of the PNIA, in a program that prioritizes the execution of projects aimed at strengthening the operational capacity and institutional framework of INIA, and the execution of innovation projects, in coordination with the actors of the SNIA, and in response to the demand of producers and the market. Note: a Many comments suggested minor corrections in the language of the Spanish version of the ICR that was provided for review. A few comments highlighted topics that might be interesting to explore in future. Page 63 of 72 The World Bank National Agricultural Innovation (P131013) Co-Financier’s comments on the PNIA ICR Alvaro Garcia Negro, Inter-American Development Bank, Lima Regarding the PNIA ICR, I have reviewed the attached document, and it rightfully describes the implementation experience of the project, the results that were achieved, and the lessons learned. I think it is important to highlight, as stated in Section V Lesson Learned (114), that even though great progress was made by INIA, there are still a lot of challenges facing the institution in the path towards achieving top notch recognition, locally and internationally. The IADB remains committed to the regional innovation agenda and hopes there will be future opportunities to collaborate with the World Bank in exchanging knowledge and supporting the government’s innovation agenda. Page 64 of 72 The World Bank National Agricultural Innovation (P131013) ANNEX 6. SUPPORTING DOCUMENTS This Implementation Completion Report was prepared based on information taken from project records as well as the information contained in the following documents: A. World Bank Documents 1. Project Appraisal Document 2. Project Information Document 3. Loan Agreement 8331-PE 4. Implementation Status and Results (ISR) Reports 5. Implementation Support Mission Aide Memoires 6. Restructuring Paper – 1st Restructuring 7. Restructuring Paper – 2nd Restructuring 8. Restructuring Paper – 3rd Restructuring B. Other documents 1. PNIA Annual Project Reports 2. PNIA Baseline Study Report 3. PNIA Mid-Term Review Intermediate Evaluation Report 4. Measurement of PDO1 and PDO 3 Report 5. Measurement of PDO2 Report 6. PNIA Final Impact Evaluation Report 7. Borrower’s Implementation Completion Report 8. PNIA Innovation Ecosystem – Final Results (PPT) 9. Project Concept Note for PNIA 2 (document) 10. Project Concept Note for PNIA 2 (PPT) Page 65 of 72 The World Bank National Agricultural Innovation (P131013) ANNEX 7. Project Impact Evaluation Summary 1. As part of the final evaluation process, an independent firm was engaged to conduct the final impact evaluation of the Project. The final impact evaluation was carried out between October 2020 and March 2021. This period coincided with the Covid-19 pandemic, which forced the firm to carry out data collection activities virtually and resulted in reductions in sample sizes from what had originally been envisioned. The reliance on virtual methods also made it especially difficult to contact all of the respondents who had participated in earlier baseline surveys, which reduced the size of the panel data set used for some of the analysis (see Table 7.1). Table 7.1. Sample size for the impact evaluation Baseline survey (2016-2017) Endline survey (2020-2021) Target population Beneficiary group Control group Beneficiary group Control group Component 1 SNIA member institutions 500 n.a 219 n.a Component 2 Producers 1,536 1,344 826 649 Component 3 Extension agents 330 n.a 62 n.a Researchers/scholarships 407 n.a 128 n.a n.a: not applicable 2. Despite the decrease in the sample size in the endline survey, the data comprised a usable panel dataset. In view of the diverse types of investments made under the Project, the firm used multiple methodologies and relied on data from several different sources (summarized in Table 7.2). Table 7.2. Methodological approaches used for final impact evaluation Beneficiary Methodology Impact Indicators SNIA member Before-and-after • Level of knowledge of the SNIA’s actors on the activities institutions carried out by INIA and SNIA • Perception of the SNIA’s performance on the consolidation of agricultural innovation. • Perception of INIA's role as the SNIA’s governing body Producers Difference in • Technology adoption Difference • Total household income • Agricultural income Page 66 of 72 The World Bank National Agricultural Innovation (P131013) Beneficiary Methodology Impact Indicators Researchers Before-and-after • Researchers whose researchers finalized in new technologies • Researchers who publish articles oriented to generate technologies in the agriculture sector • Researchers’ annual income • Professionals trained through scholarships and internships who have conducted research • Annual income of professionals who accessed to scholarships and/or internships Extensionists Before-and-after • Annual income 1. Ability of INIA to lead the SNIA (Component 1) 3. To assess the impacts of the Project on strengthening the ability of INIA to lead the SNIA, perceptions about the performance of INIA and the SNIA were asked of SNIA member institutions in the baseline survey (n = 500) and endline survey (n = 219). Level of knowledge of SNIA member institutions about the activities carried out by INIA and the SNIA The survey revealed that the Project had a positive effect on the knowledge of the activities carried out by INIA and by the SNIA. The percentage of SNIA member institutions who stated that they are aware of the activities carried out by INIA had increased from 77 percent at the time of the baseline to 87 percent in the months prior to closing. Similarly, the percentage of SNIA member institutions who indicated that they knew about activities carried out by the SNIA had increased from 41 percent to 70 percent. Perception of SNIA member institutions about the performance of the SNIA The survey revealed that most SNIA member institutions are satisfied or very satisfied with the performance of the SNIA. The level of satisfaction was highest regarding the way in which the SNIA uses its convening power to generates shared spaces and coordination mechanisms to encourage discussion, knowledge and resource exchange, and creation of innovations (74 percent approval, up from 59 percent in the baseline). The level of satisfaction was lowest regarding the work done by INIA and the SNIA to improve coordination between the organizations that carry out agricultural research and the organizations that promote the technology adoption and transfer or directly support agricultural production (59 percent approval, up from 53 percent in the baseline). Perception of INIA's role as the governing body of the SNIA The survey revealed that a majority of SNIA member institutions are satisfied or very satisfied with the performance of INIA in leading the SNIA. Nearly six out of 10 SNIA member institutions (57 percent) reported being satisfied or very satisfied with INIA's performance in its role of SNIA’s governing body. In addition, 60 percent of SNIA member institutions stated that compared to a few years ago, INIA influence as leader of the SNIA has improved. Page 67 of 72 The World Bank National Agricultural Innovation (P131013) 2. Consolidating the market for innovation services (Component 2) 4. To assess the impacts of the Project in developing agricultural innovations and making them available for the benefit of producers and other value chain actors, data were collected from 1,475 producer households (826 beneficiary households and 649 control group households), which were asked about technology adoption, changes in agricultural income, and changes in total household income. The values of the impact indicators were estimated using a Difference-in-Difference (DiD) econometric model with fixed effect, which allowed the impact values associated with beneficiary households to be compared directly to the values associated with control group households. The model was specified as follows: , = , + + Where: , = impact indicators = independent variables = dichotomous variable that takes on a value of 1 if the respondent belongs to the beneficiary household and 0 if the respondent belongs to the control group = dichotomous variable that takes on a value of 1 if the observation belongs to the endline and 0 if it belongs to the baseline = Error term , = ratio coefficient between the impact indicators and independent variables 5. The estimated absolute values of the impact indicators associated with beneficiary households were compared directly to those associated with the control group households using the following methodological approach (see table 7.3) Table 7.3. Comparison of impact indicators, beneficiary households vs. control group households Beneficiary Control household Difference Impact indicator household (project impact) Baseline Endline Baseline Endline Technology adoption rate bi,b0 Ybi,b1 ci,c0 ci,c1 Yd = (Ybi,b1 - Ybi,b1) – (ci,c0 - ci,c1) Agricultural income bi,b0 Ybi,b1 ci,c0 ci,c1 Yd = (Ybi,b1 - Ybi,b1) – (ci,c0 - ci,c1) Total household income bi,b0 Ybi,b1 ci,c0 ci,c1 Yd = (Ybi,b1 - Ybi,b1) – (ci,c0 - ci,c1) 6. The DiD analysis showed that compared to households in the control group, households that participated in Project-supported collaborative sub-projects adopted innovations at a higher rate, realized larger increases in agricultural income, and realized larger increases in total household income (Table 7.4). Page 68 of 72 The World Bank National Agricultural Innovation (P131013) Table 7.4. Estimated effect on impact indicators Model (,) Estimated effect Statistical significance Agricultural technology adoption 0.374** p < 0.05 Agricultural income 0.341*** p < 0.01 Total household income 0.182* p < 0.10 Agricultural technology adoption The household survey revealed that the Project generated a statistically significant increase in the use of (i) simple and (ii) advanced technologies in agricultural production processes by beneficiary households, compared to the control group (95 percent confidence level). Beneficiary households were found to have adopted simple technologies at a rate 38 percent higher than households in the control group, and advanced technologies at a rate 61 percent higher than household in the control group. Agricultural income The household survey revealed that the Project had a statistically significant impact on the agricultural income earned by beneficiary households (99 percent confidence level). The agricultural income earned by beneficiary households was found to have increased by 33.8 percent, considerably more than the XX percent reported by households in the control group. Total household income The household survey revealed that the Project had a statistically significant impact on the total income of beneficiary households (90 percent confidence level. The increase in total income of beneficiary households was estimated to be 16.8 percent higher than the increase in total income of households in the control group. The impressive impacts of the Project in raising the incomes of producers would have been even higher had the Covid-19 pandemic not happened during the final year of implementation. Approximately three- quarters of respondents reported that their agricultural enterprises had experienced decreases in production and sales, as well as increases in production costs, because of Covid-19. 3. Strengthening strategic capacities within INIA and in the SNIA (Component 3) 7. To assess the impacts of the Project in strengthening strategic capacity within INIA and in the SNIA, panel data were collected from 128 researchers and 62 extensionist who had participated in a strategic research or capacity building sub-project, or had received a scholarship or an internship. These respondents were asked about the impact of the Project on their productivity as innovators, their scientific publications, and their salaries / incomes. Production of new research products The survey of researchers and extensionists revealed that the Project had a significant impact in increasing the quality of research outputs. Four out of five respondents (81 percent) reported their research had resulted in a new technology, product, patent, or knowledge product during the previous year. Only 61 Page 69 of 72 The World Bank National Agricultural Innovation (P131013) percent of respondents reported similar results during the year preceding the baseline study. Publication of scientific articles and reports The survey of researchers and extensionists revealed that the Project had a significant impact in increasing the quantity of their scientific publications. The results show a significant increase in the percentage of researchers who produced publications aimed at generating technologies related with the agriculture sector. This percentage increased at 10 percent in comparison with the project baseline. Annual income of professionals who accessed to scholarships and internships The survey of researchers and extensionists revealed that the Project had a significant impact in increasing the incomes of those who had received a scholarship or internship through the Project. The annual income of researchers who received a scholarship or internship (mostly these were scholarships that allowed completion of Master’s degrees) increased from US$7,576 to US$18,182. On average, the annual income of extensionists who had received a scholarship or internship (mostly these were internships that provided for technical training) increased from US$4,560 to US$13,632. 4. Efficiency of the Project management function (Component 4) At Project closing, the total cost of Component 4 stood at US$7.14 million, considerably higher than the US$3.57 million projected at appraisal. Project management costs exceeded projections for two main reasons: (i) the two extensions of the project closing date meant that PIU staff salaries and operational costs were incurred for 18 months longer than expected, a period when salaries and benefits had increased because of changes in the labor market and due to inflation, and (ii) when it became apparent that many collaborative sub-projects were experiencing implementation delays, including due to lack of technical knowledge and/or lack of experience with fiduciary procedures, additional short-term consultants were hired to provide hands-on technical assistance to sub-projects, support administrative processes, and ensure thorough documentation of sub-project results. The resources spent on hiring additional consultants were well spent, because the enhanced support provided to sub-projects played a major role in ensuring the very high sub-project completion rate. The cost of Component 4 was included in the calculations of overall project efficiency, and all measures of project worth remained attractive. As noted previously, the PIU performed well throughout the life of the Project. The PIU was staffed by qualified and highly engaged professionals, most of whom were committed to the Project and carried out their activities with great dedication in the face of constantly changing institutional environment. PNIA management ably oversaw the successful implementation of a large and diverse work program, made certain that activities and outcomes were well documented, and ensured that fiduciary functions were carried out in compliance with World Bank and government policies and procedures. The esprit de corps established within the PIU proved especially important during the periodic phases when political support for PNIA waned and budget allocations decreased. Conclusion 8. In summary, the results of the final impact evaluation carried out by an independent firm revealed that the Project had significant positive impacts in with respect to its development objectives. Page 70 of 72 The World Bank National Agricultural Innovation (P131013) ANNEX 8. Types of collaborative sub-projects financed Under Component 2 (Consolidating the Market for Innovation Services), 541 collaborative sub-projects were financed supporting 74 value chains (Figure 2). Figure 2: Value chains supported through competitive sub-projects (no. of sub-projects per value chain) 16 16 Source: PSNIA subproject Monitoring and Evaluation System - SISEV Page 71 of 72 ANNEX 9. Case study: Collaborative Sub-project Mushroom production in Lambayeque Region Development challenge Incahuasi District in Lambayeque Region suffers from a high rate of poverty. Lack of education and the absence of improved farming technologies constrain the productive capacity of many households. On average in Lambayeque Region, a typical family of five lives off a monthly income of only S/ 350. During the 1990s, the National Watershed Hydrographic and Soil Conservation Management Program implemented a reforestation project in the area. Today more than 1,200 ha are planted to pine trees in Incahuasi District. The Bosque de Pino represents a rich natural resource whose potential has not been fully captured up until now. Sub-project details In 2016, Simbiosis, a social-environmental enterprise, partnered with local households and secured financing through PNIA for a sub-project whose objective was to generate new sources of revenue from the mushrooms that grow in association with these pine trees. The total budget for the sub-project came to approximately S/ 280,000, of which 70 percent was provided by PNIA and 30 percent was contributed by the beneficiaries. The sub-project, which was implemented over a period of 15 months, benefited 420 people (120 directly, 300 indirectly). Innovations The sub-project focused on three innovations: (1) Establishment of a mushroom supply chain; (2) Introduction of new mushroom species from Spain as a means of diversification; and (3) Development of a modular solar dryer that is easy to operate, ergonomic, low cost, and produces short-term results. Results Simbiosis helped the participating households increase the productivity and the profitability of their mushroom production enterprises. Yields achieved by mushroom producers (95 percent of whom were women) increased from 915 kg/ha to 1,367 kg/ha on average. Meanwhile, post-harvest losses were reduced from 20 percent to 4 percent thanks to the increased efficiency of the solar dryer. As a result, production per household rose from 62 kg to 201 kg of dried product per growing season. Taking advantage of increased and higher quality supply, Simbiosis facilitated the sale of dried mushroom products to two agro-exporting firms. Because these firms sell into lucrative overseas markets, they are able to offer more attractive prices to the producers, and the price of the dehydrated product increased from S/ 10.00 per kg to S/ 13.80 per kg. Lessons learned The sub-project illuminated the potential for mushroom production to become a sustainable income source for poor rural households and provide a possible pathway out of poverty. It demonstrated the positive impact that the empowerment of women and girls can have on the generation of income within smallholder farming communities. Last but not least, the sustainable business model can be replicated in other parts of the country that feature similar pine forests (covering an estimated 90,000 hectares), so the potential for scaling up is high. Page 72 of 72