Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AR Rural Inclusion (P106685) Report Number: ICRR0023744 1. Project Data Project ID Project Name P106685 AR Rural Inclusion Country Practice Area(Lead) Argentina Agriculture and Food L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IBRD-80930 28-Feb-2021 50,610,675.99 Bank Approval Date Closing Date (Actual) 11-Jun-2015 30-Nov-2022 IBRD/IDA (USD) Grants (USD) Original Commitment 52,500,000.00 0.00 Revised Commitment 52,500,000.00 0.00 Actual 50,610,675.99 0.00 Prepared by Reviewed by ICR Review Coordinator Group Shashidhara Laxman Vibecke Dixon Avjeet Singh IEGSD (Unit 4) Kolavalli 2. Project Objectives and Components DEVOBJ_TBL a. Objectives The Project Development Objective is to increase the socio-economic inclusion of rural poor (small producers, indigenous people, and rural workers) by: (a) strengthening their organizational, planning and management capacity to achieve poverty-reduction goals; (b) improving their access to community infrastructure and services; and (c) piloting a new model for developing sustainable access to markets (Legal agreement (LA, p. 5). The PDO formulation in the PAD was identical (p.4). Page 1 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AR Rural Inclusion (P106685) The PDO will be parsed as follows: The PDO formulation is: to increase the socio-economic inclusion of rural poor (small producers, indigenous people, and rural workers). This is expected to be achieved through the following sub-objectives: (a) to strengthen the rural poor’s' organizational, planning and management capacity to achieve poverty- reduction goals; (b) to improve rural poor’s' access to community infrastructure and services; and (c) to pilot a new model for developing sustainable access to markets. For the sake of the analysis in this ICRR, the three sub-objectives will be assessed as objectives in section 4, and an assessment of how and to what degree the achievement of these three sub-objectives have led to the achievement of the PDO will be provided in a summary paragraph in section 4. b. Were the project objectives/key associated outcome targets revised during implementation? Yes Did the Board approve the revised objectives/key associated outcome targets? No c. Will a split evaluation be undertaken? No d. Components Component 1: Capacity Development (Appraised: US$13.83 million (Bank: US$2.10 million); Actual: US$1.57 (Bank US$0.4 million)). Capacity-building activities for the purpose of increasing the impact of development actions on targeted rural groups. It included two subcomponents: a. the provision of technical support to communities and organizations of small producers, indigenous peoples and rural workers, aimed at improving their organizational, planning, and management capacity; and b. the provision of technical support to strengthen the capacity of targeted rural groups through the preparation of studies, strategies and planning instruments for local development. Component 2: Rural Livelihood (Appraised: US$43.0 million (Bank: US$39.00 million); Actual: US$44.57 million (Bank US$41.31 million)). Demand driven investments/activities aimed at improving the living conditions of the targeted poor rural communities. It has two subcomponents: Page 2 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AR Rural Inclusion (P106685) a. support to demand driven investments/activities (rural investments subprojects (SIR)) for i) improving the living conditions of targeted poor rural communities; (b) small-scale rehabilitation or construction of rural community infrastructure; (c) support to improve agricultural production and marketing, (d) activities for food security and natural resource management; and (e) construction or improvement of household water and sanitation systems. b. Access to Infrastructure and Public Services: infrastructure, goods, and services for targeted communities. Component 3: Access to Markets (Appraised: US$16.27 million (Bank: US$11.00 million); Actual: US$10.7 million (Bank: 8.00 million)). Provision of support for the identification, formulation and implementation of Productive Alliances subprojects. It has two subcomponents: a. Alliance promotion and preparation: activities for the promotion and preparation of Productive Alliance Subprojects, including, inter alia,: (i) the realization of promotional campaign targeted at potential producer organizations and buyers, (ii) the formulation of alliance profiles reflecting joint expressions of interest of producer organizations and potential buyers; (iii) the provision of support for the process of evaluating , ranking and approving Productive Alliances subprojects; (iv) the provision of training to Producer Organizations on the design and negotiation of alliance business plans which may eventually result in Productive Alliances; and (v) the provision of support for the preparation of productive alliance subproject agreements and related documentation, all acceptable to the Bank; and b. Alliance implementation: carrying out productive alliance-driven investments/activities (such as, small scale civil works and the improvements of the production processes and production quality standards of production organizations) for purposes of developing sustainable access to markets for producer organizations based on productive alliances. Component 4: Project Management (Appraised: US$3.27 million (Bank: US$0.27 million); Actual: US$3.48 million (Bank: US$0.90 million). Provision of operational support to assist the Borrower in the coordination, implementation, administration, monitoring and evaluation of the Project. The following changes were made in two components during restructuring.  Component 2: to maximize development impact, the financing of medium-scale scale works with a ceiling of USD800,000 per investment was included.  Component 3: to generate additional demand for alliances and to diversify into different value chains, the geographical scope was expanded from two poor provinces to all the provinces participating in the project. e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project costs Page 3 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AR Rural Inclusion (P106685) The project was appraised at US$76.5 million. The costs were revised to US$66.74 million, and actual costs were US$59.98 million. The costs were revised down during the 2019 restructuring because of the Government of Argentina’s (GOA) reduced ability to provide counterpart financing. Financing Investment project financing of US$52.5 million over a five-year period. The project was financed through an IBRD loan (loan 8093-AR). Borrower contribution The total contribution from the borrower and the communities was US$ 9.4 million against the planned US$24 million. The government contributed US$3.85 million against the planned US$16.6 million, which was revised down to US$6.84 million. Local communities contributed US$5.5 million against the planned US$7.4 million). Dates The project was approved on June 11, 2015, and became effective on April 07, 2016. A Mid Term Review (MTR) was held on June 22, 2018. The project was restructured on June 06, 2019. It was to close originally on February 28, 2021, but closed on November 30, 2022. The first extension by 16 months (from February 28, 2021, to June 30, 2022) was to maximize the probability of achieving the PDO targets. The second extension, by five months, was to complete the ongoing investments and fully achieve the objectives of the project. The level two restructuring, done in 2019 when 15 percent of the funds had been disbursed, included the following:  Components: inclusion of medium scale rural investments under component 2 and expansion of geographical coverage of component 3  Institutional arrangements; included the option of on-lending to provinces in addition to project transferring funds to provinces as grants  Triggering of OP 7.50 and updating of safeguard instruments  Disbursement categories and reallocation of funds;  Procurement arrangements  Results framework; and  Inclusion of the safeguards incident response requirement Split rating The restructuring did not change the PDO. The World Bank’s commitments remained the same; Planned funding for Component 1 declined from USD13.83 million to USD4.2 million primarily because of GOA’s reduced ability to make counterpart funding available. The scope remained the same except for the increased geographical coverage of one component. Some changes were made to outcome indicators, largely to make them easier to measure. The project became more ambitious, although the target for one indicator was decreased (no. of community organizations functional after 12 months). As the PDOs did not Page 4 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AR Rural Inclusion (P106685) change, same outcomes were expected, and the project became more ambitious, this review will not do a split rating. 3. Relevance of Objectives Rationale Country context After rebounding from the economic crisis in 2002, Argentina was successful in reducing poverty and decreasing income inequality. The growth, however, was accompanied by macroeconomic imbalances, inflation, fiscal deficits, and limited international reserves, which threatened the sustainability of the gains made in reducing poverty and inequality. Argentina’s poverty and income inequality were some of the lowest in Latin America, but regional disparities within the country in poverty and access to resources persisted. The less well-developed and poorly integrated northern provinces had poverty rates two to three times higher than the national average and lagged the rest of the country in social services and basic infrastructure. About 1.5 million people, a third of the rural population of about 4.5 million, who made up about 11 percent of the total population, were poor despite an agricultural sector that made Argentina one of the top food producers in the world. The rural poor are mostly small family producers, indigenous people, and rural workers who share the rural space with larger landowners and producers of export commodities. Indigenous people, who comprise 2.5 percent of the population, are present in all provinces of Argentina but more so in the northern provinces. They are one of the largest and most vulnerable groups of the rural poor in Argentina. Nearly a quarter (23.5 percent) of indigenous households live with unsatisfied basic needs, compared to 13.8 percent for the rest of the population. Country strategy The government recognized that family farms play an important role in agricultural and rural development and that small family producers are key actors in the national food system. The PDO aligns with the Government’s commitment to promote growth with equity and inclusion. Argentina’s goals for the agricultural sector (Agri-food and Agribusiness Strategic Plan (PEAA 2020)) emphasized supporting small producers, increasing productivity through investments in basic rural infrastructure, provision of technical assistance, the use of appropriate technologies, and increased access to markets. The Project was initially designed in 2011 when Argentina prepared a series of projects to improve the livelihoods of the rural poor and increase productivity and market access of family producers. Thus, the PDO complements the objectives of other non-Bank financed projects implemented by the Ministry of Agriculture. The PDO subsumes the priorities of the administration that came to power in 2019 to reduce hunger and improve access to water. Bank strategy The PDO was consistent with several result areas within the three strategic themes of the Country Partnership Strategy (CPS – FY15-18): raising the productivity of small and medium producers in lower- income regions and increasing access to basic services for the bottom 40 percent in low-income provinces Page 5 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AR Rural Inclusion (P106685) and areas, for example. At Project Closing, the PDO aligned with the Country Partnership Framework (CPF) for FY19-FY22, which sought to help Argentina achieve macroeconomic stability while reducing poverty through private sector-led growth. One of the objectives was to make the rural economy climate smart because Argentina’s position as a world leader in many food exports was threatened by the increasing frequency and intensity of climate events, with a large segment of farmers in Argentina, especially family farmers, remaining vulnerable to these risks. Although the PDO did not refer to climate- smart support, the Project’s activities, outcomes, and targeted beneficiaries are relevant to reducing small farmers' productive and social vulnerabilities to climate change. The PDO was also consistent with priorities in the 2022 Country Climate and Development Report (CCDR), the 2021 Water Security Diagnostic, and the 2018 Systematic Country Diagnostic (SCD) for Argentina. The CCDR noted higher expected changes in temperature in northern Argentina, with its large vulnerable population. It identified water storage, irrigation, and deepening of waterways as immediate priorities for water security. Similarly, the Water Security Diagnostic identified priority investments to improve water security by 2030. The World Bank supported two projects since 1997: Small Farmers Development Project (PROINDER) and the Provincial Agricultural Development Project (PROSAP). Satisfied with the results of PROINDER, the Government requested the World Bank to continue the support, incorporating the previous projects' lessons and piloting an approach to improve market access for small family producers. The PDO is pitched at a level that adequately addresses a development problem, i.e., to increase the socio- economic inclusion of the rural poor. In addition, it clarifies the necessary and expected outcomes that will lead to the achievement of the PDO, i.e., (a) strengthening their organizational, planning, and management capacity to achieve poverty-reduction goals; (b) improving their access to community infrastructure and services; and (c) piloting a new model for developing sustainable access to markets. The ICR states that socio-economic inclusion would be achieved by (a) improvements in living standards and access to basic social infrastructure, mainly water for household consumption and production, and (b) the economic inclusion of rural producers based on new productive infrastructure, production and marketing equipment, and the acquisition of new knowledge. Aligned with country and bank strategies and pitched at an appropriate level to address a development problem, PDO relevance is rated high. Rating Relevance TBL Rating High 4. Achievement of Objectives (Efficacy) EFFICACY_TBL OBJECTIVE 1 Objective Page 6 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AR Rural Inclusion (P106685) To strengthen the organizational, planning and management capacity of rural poor (small producers, indigenous people, and rural workers) to achieve poverty-reduction goals. Rationale Theory of Change To strengthen the rural poor’s organizational, planning, and management capacity, the project's inputs were technical support to targeted beneficiaries (communities and small producers’ organizations, indigenous peoples, and rural workers’) and to prepare studies, strategies, and planning instruments for local development. This was expected to lead to outputs such as beneficiaries’ organizations able to formulate eligible subproject proposals and for them to be able to manage their organizations commercially and technically. This was in turn expected to lead to the following outcomes: Beneficiary community organizations would demonstrate sustained functionality. Project outputs  399 community organizations were strengthened by the project, (exceeding the target of 300). The ICR provides limited information as to how these organizations were strengthened including what criteria were used to measure “strengthened organizations”. Project outcomes  121 community organizations were functional 12 months after subproject completion, exceeding the target of 100, and  84% subproject proposals were considered eligible for financing (over total number of proposal presented), exceeding the target of 80% The project built capabilities of or strengthened community organizations – building both new ones and developing those existing – exceeding the revised target for the number of community organizations that it aimed to strengthen. The ICR does not provide information on what the project did to strengthen the organizations or what “strengthening” meant. According to the bank team (discussed in a meeting on January 18, 2024) activities to strengthen organizations depended on their needs, which varied considerably, and they used about ten different indicators relating to production, governance, and finance to determine whether the organizations were strengthened. The outcomes, in terms of strengthened organizations, are measured in terms of whether the community organizations continued to function 12 months after the completion of the subprojects they undertook and the quality of subprojects they prepared, reflected by the proportion that was eligible for financing. The project exceeded the target on both: more organizations than targeted were functional after 12 months and a higher than targeted proportion of their proposals were eligible for financing. This objective goes only so far as to build capabilities of community organizations to come together and put up proposals to take actions that can improve their wellbeing. How that capability helped them would be addressed by the next two objectives. The ICR offers additional information from a survey of 349 of the 419 community organizations that implemented special projects. The survey did not require the respondents to answer all the questions and was completed by representatives, who, the ICR notes, may not have highlighted all the benefits. According to the information, 56 percent of them claimed that they benefited from organizational improvements in terms Page 7 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AR Rural Inclusion (P106685) of formalization of their organization, increased participation of members, and improvements in administration and management. The project technically supported community organizations to improve their capacity to organize, plan, and implement subprojects, thus strengthening them. The strengthened organizations were functional 12 months after they implemented subprojects. Their improved capabilities were reflected by the quality of proposals they had prepared to improve their livelihoods, measured by their eligibility for financing. Additional information suggests that at least one half of the community organizations were satisfied with the project benefits. The achievement of this subobjective is thus rated substantial. Rating Substantial OBJECTIVE 2 Objective To improve rural poor’s (small producers, indigenous people, and rural workers) access to community infrastructure and services. Rationale Theory of Change: To improve the rural poor’s access to community infrastructure and services, the projects inputs were investments in agricultural production and marketing, food security, natural resources management and in household water and sanitation systems. This was expected to lead to outputs such as infrastructure and services to improve agricultural production in place and household water and sanitation systems in place. This was in turn expected to lead to improved agricultural productivity (increased volume of sales and sustained market relationships) and improved living conditions. Project outputs  428 community subprojects completed, exceeding the target of 210  111 indigenous people’s community subprojects completed, exceeding the target of 90 Project outcomes  478 rural investment subprojects achieved at least 75% of the expected processes, exceeding the target of 240  95 percent of the rural investment subprojects (SIRs) were operational), exceeding the target of 70 percent73,150 direct project beneficiaries, exceeding the target of 50,000  11,829 families benefited with at least one rural investment project, exceeding the target of 9,000  37,020 beneficiaries of community projects, exceeding the target of 31,500  22,125 beneficiaries of indigenous people subprojects, exceeding the target of 13,500 Page 8 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AR Rural Inclusion (P106685) The project supported subprojects of communities that could potentially benefit them. It did so exceeding the targets for the total number of projects completed and those of indigenous communities and the number of families that potentially could benefit from the community subprojects. The outcome was measured by two indicators. One of them is the number of subprojects that had completed at least the third stage of the four implementation stages: i) approval by the committee, ii) agreement signed, iii) 100 percent of the amount disbursed, and iv) the project completed, with final statement of expenditures. Before the indicator was modified at restructuring, it better reflected how the communities benefited from subprojects: rural investment projects that meet at least 75 percent of the expected results – expected results were to be individually determined depending on the nature of the subproject. The modified indicator is of lower evaluative value than an intermediate results indicator that gives the number of subprojects completed. The other outcome indicator gives the proportion of subprojects that were operational 12 months after their completion – ‘operational’ is not defined. At the end of the Project, not all subprojects had reached the 12-month maturity date post-disbursement. The outcome indicators do not suggest how and to what degree members of the communities gained access to improved and/or new infrastructure and services. The choice of investment was left to the communities, without prioritizing the infrastructure and services to which access would be increased. The government’s request to fund two of its programs to increase access to water and increase local food supplies to reduce hunger brought some focus to the investments made. The ICR reports (p. 66) that around 85 percent of the subprojects supported were to increase access to water. The subprojects targeted rural populations with little or no access to water and built concrete cisterns that could also harvest water or developed other sources of water. The ICR notes anecdotally that this positively affected the beneficiaries’ living condition, by increasing the surface areas of surrounding orchards. In some cases, young families were reported to have returned to the project-supported communities. To suggest the nature of outcomes, the ICR also draws on information from sources other than the project’s M&E system: the borrower completion report, the bank’s ISRs, surveys completed by 349 community organizations, and 25 interviews with beneficiaries and local actors. According to the additional information supplied by the ICR from the survey of community organizations,  53 percent reported improvements in access to infrastructure,  64 percent reported that their value chains had been strengthened in some way, and  56 reported that productivity had increased While there are no M&E measurements to verify or substantiate these figures, the results from these beneficiary surveys suggest that the project had a positive effect on the communities. While the project exceeded the target for the share of subprojects that benefited indigenous people, the target of 50 percent female beneficiaries was not achieved: women participated in 37 percent of non-indigenous rural investment projects, 42 percent of the indigenous person rural investments, and 45 percent of the beneficiaries of productive alliances. To help community organizations gain access to infrastructure and services – leaving it to them to decide what they would be – the project supported community subprojects, exceeding the target for the number of subprojects supported and the potential beneficiaries reached. The project has delivered the targeted outputs, exceeding them in nearly all the cases. The evidence presented on whether the projects benefited members of the communities is inadequate. It has met the target to include indigenous people but narrowly missed the target to include women. The original indicator of how the communities benefited from subprojects i.e. rural investment projects that meet at least 75 percent of the expected results better reflected the outcomes rather Page 9 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AR Rural Inclusion (P106685) than just the number of subprojects completed. The outcome indicator do not suggest benefits in terms of improved access to infrastructure and services, but the outputs are likely to have led to the desired outcomes. The rating for this sub-objective is modest. Rating Modest OBJECTIVE 3 Objective To pilot a new model for developing sustainable access to markets (for small producers, indigenous people, and rural workers). Rationale Theory of Change: To pilot a new model for developing sustainable access to markets, the project provided the following inputs: identify, formulate, and pilot producer groups and market agents and buyers (Productive Alliances, PAs), invest in small-scale civil works, production processes and quality standards. It was expected to lead to outputs such as functioning pilot-Productive Alliances in place. This was expected to lead to outcomes such as the PAs’ ability to demonstrate capacity to increase the volume of sales and to sustain market relationships and to demonstrate sustainability of productive cycles without project support. The project investments were to include a high ratio of the poorest and most vulnerable rural poor, i.e. women and indigenous groups. Project Outputs  46 productive alliances completed, exceeding the target of 40 Project outcomes  75 percent productive alliance subprojects operational and/or maintained 12 months after their completion, exceeding the target of 70, and  90% increase in the volume of sales of the organizations participating in productive alliances exceeding the target of 30%. 2,801 producer families linked to markets through productive alliances, exceeding the target of 1,000  14,005 beneficiaries of productive alliances, exceeding the target of 5,000. The project assisted producer groups to get into alliances with market agents to gain access to markets, exceeding both the number of alliances supported and the number of families and individuals expected to benefit them. Twelve months after their completion, three quarters of the supported alliances were operational, which is not clearly defined in the ICR. According to the bank team (discussed in a meeting in January 2024) operational meant that the organizations were planning, carrying out their activities, sharing information, and surviving beyond the initial support. The alliances are reported to have benefited the organizations by helping them increase their sales by 90 percent. The ICR notes that the technical assistance provided to alliances resulted in new products such as health certified honey for export, byproducts from dairy Page 10 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AR Rural Inclusion (P106685) effluence, tea products for exports, resulting in increased volume of sale. It is not clear how increase in sales volume was estimated for organizations that developed new products. The project activity helped producer organizations get into alliances with market agents to improve their access to market by providing them with material support needed to make the investments necessary for the alliance to be effective. As a result, a higher number than the targeted alliances emerged. The alliances are reported to have enabled organizations to supply to market products that they hadn’t supplied before. One indicator suggests that the alliances were operational 12 months after completion and the other that the groups were able to increase their sales. The additional evidence provided by the ICR (in addition to what can be derived from the project’s M&E system), merits a Substantial rating for this sub-objective. Rating Substantial OVERALL EFF TBL OBJ_TBL OVERALL EFFICACY Rationale The PDO was to increase socio-economic inclusion of the rural poor by strengthening community organizations and assisting them to provide themselves with infrastructure and services and get into alliances with market agents. The PDO was defined at a broad level and the three subobjectives built on each other to achieve socioeconomic inclusion: the first sought to strengthen the community organizations so that they are better able to plan and implement projects that improve their living conditions; the second objective was to support the subprojects proposed by community organizations to improve their access to infrastructure and services, and the third was to help of the community organizations get into alliance with market agents to improve their access to markets. The evidence on achievement of the subobjectives is mixed partly because the project’s M&E system was relatively weak and provided limited evidence of relevant outcomes. The ICR thus draws upon additional sources to document and demonstrate the project achievements. The overall Efficacy rating is Substantial with moderate shortcomings. Overall Efficacy Rating Substantial 5. Efficiency Economic and Financial Efficiency: At appraisal, the analysis simulated the benefits from subprojects likely to be proposed by beneficiaries in different provinces of the country. As it was a pilot initiative, the analysis of Productive Alliances Subprojects was based on illustrative models for small-scale producers in the northeast of Argentina (NEA) Region. The simulations concluded that the marginal economic return on investment would range between 31 percent and 50 percent for Rural Investment Subprojects and between 24 percent and 41 percent for Productive Alliance Page 11 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AR Rural Inclusion (P106685) Subprojects. The expected economic benefits were primarily associated with increased agricultural production and producers’ income, estimated based on 11 production models (corn, cumin, orange, pepper, potato, pumpkin, soybean, sugar cane, sunflower, tomato, and livestock) and the current structure of small-scale production systems by province. The ICR replicated the methodology. The required information was collected from a representative sample of 55 subprojects in agriculture (crops, horticulture, livestock, and milk production), handicrafts, yerba mate, and tea. Information was also collected from groups that received support for increasing access to water, in which case the analysis focused on the benefits such as reduced time spent fetching water and reduced transport expenses. The estimated IRR from rural investment subprojects ranged from 11 to 34. For productive alliance subprojects, estimated IRR ranged from 13 to 38 percent. The estimates for the project and components were as follows: full project: 16.90; production subprojects (component 2): 20.31; subprojects to develop water (component 2): 19.84; all investments in component 2: 18.94; and productive alliance subprojects: 24.56. Sensitivity analysis showed that combining a cost increase of 10 percent and benefit reduction of 10 percent yields an IRR of 12.3 percent. Productive Alliance subprojects (SAPs) under Component 3 were affected by inflation and exchange rate fluctuations. The prices used during formulation of SAPs were no longer valid by the time procurement took place. Around 78 percent of SAPs (18 out of 23) approved by September 2020 were unable to make the total investments proposed in their business plans, resulting in the non-fulfillment of planned objectives. The PIU had to fill the gaps and revised the operational Manual to incorporate an automatic price adjustment mechanism. Administrative Efficiency: A major drought in 2017, which impacted the agricultural sector and the fiscal revenues, worsened Argentina’s challenging macroeconomic conditions soon after the project became effective. GOA’s efforts to reduce its fiscal deficit limited the project implementation unit’s (PIU) ability to function effectively and its ability to make counterpart contributions to execute the Loan. The delays in implementation, budgetary constraints at the federal level, and various institutional reforms slowed down the project from the beginning. The project was also impacted by broad institutional restructuring of GOA in 2018, which changed the institutional arrangements for the project implementation unit (PIU). The Bank restructured the Project in 2019 to enable the government to lend the funds to provinces rather than transferring them as grants, without showing them on its books. As the COVID-19 pandemic led to restrictions on interactions, the Project developed three initiatives to mitigate the effects of social distancing on the pace of project implementation: (a) it established a temporary virtual committee to evaluate and approve Component 2 Rural investment subprojects (SIRs) and Component 3 SAPs; (b) it adopted prototypes of small water infrastructure works developed by National Agricultural Technology Institute (INTA) for SIR-National Water Plan (PNAs); and (c) it developed an ad-hoc intervention plan for the Patagonia region, which, due to the low density of family farmers, was not originally considered under the project. The program, however, was administratively efficient, despite the delays, because it reached more beneficiaries at lower than planned costs. Page 12 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AR Rural Inclusion (P106685) Efficiency Rating Substantial a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 0 Appraisal  0  Not Applicable 0 ICR Estimate 0  Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome The PDO was highly relevant to priorities of the government and for the world Bank’s support strategy. Efficacy was Substantial with shortcomings due to limited evidence of the achievements. Although many factors delayed the implementation of the project, it reached the targeted number delivering acceptable returns, thus earning a substantial rating for efficacy. The overall rating is thus moderately satisfactory. a. Outcome Rating Moderately Satisfactory 7. Risk to Development Outcome Organizational Risk: Nearly all of the activities of this project were implemented through producer organizations. Whether they are cohesive enough to survive the pressures of collectively managing enterprises is a risk, although the project outcomes suggest that the organizations were operating 12 months beyond the life of the project. The prospects of obtaining benefits might encourage individuals and communities to come together, but continuing to work together, contributing to the group activities including maintaining infrastructure where required may prove more difficult. Business/Economic Risk: Economic uncertainty in terms of inflation and fluctuations in exchange rates put the enterprises at risk. While the project was being implemented, inflation necessitated the project to intervene with additional funds to help some groups implement their projects. Strategic Risk: The strategy chosen to provide access to markets for the poor is to create and provide productive alliances with traders or other market agents in which they would get into contractual relationships. These relationships may not hold unless they continue to be mutually beneficial. Whether they Page 13 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AR Rural Inclusion (P106685) will remain so depends on whether producers can increase productivity to reduce their production costs and achieve the quality demanded. Government Ownership Risk: The government has demonstrated commitment with its record of implementing projects that seek socioeconomic inclusion. The macroeconomic difficulties the government faces, however, poses risks to its ability to continue to support such projects. Institutional Risk: Even if the federal and provincial governments commit to nurturing community organizations, weak institutional capacity may limit the effectiveness of their interventions. The provincial implementation units, for example, faced challenges paying competitive salaries to attract capable staff. 8. Assessment of Bank Performance a. Quality-at-Entry The project was strategically relevant, focusing on the most vulnerable rural populations. It also aligned with the priorities of the country to bring about social inclusion, by improving living conditions, production infrastructure and access to markets for the rural poor. The project was appraised in a way that the expected outcomes could be achieved by basing the strategy on a key lesson learned from the previous project that poor rural producers often lacked the level of organization needed to access formal markets. The project would therefore enable them to reach a minimum level of formalization they need; and those with preexisting links would be helped to develop formal linkages through productive alliances, a model which had been tested successfully in other countries in Latin America. Although the PDO was broad and high level, the project interventions conveyed what “socio-economic inclusion” meant and what was required to achieve it. However, the PDO outcome indicators measured mostly outputs and processes, rather than outcomes in terms of socioeconomic inclusion of vulnerable rural populations. Project preparation was lengthy, from an early concept in 2008 and a redesign in 2011 to approval in 2015. It filled a space in GOA's development ecosystem that was not covered by other operations, which amounted to US$175.5 million in multilateral support. The risk of overlapping with other projects was identified in the PAD's Operational Risk Assessment Framework (ORAF). The final design, focusing on the areas in the northeast and northwest of Argentina and piloting of productive alliances, minimized the risk. The project identified several risks and assessed them as moderate: participation of and consultation with indigenous people may not be activated adequately; fiduciary risks from large number of subprojects, and that flow of counterpart funds may slow down activities at the provincial level. They took measures to minimize them: for example, representatives of indigenous people were included in provincial evaluation committees that made decisions on subprojects. Since the indigenous groups required specific attention, 30 percent of the resources were earmarked for them. However, no such efforts were made to include women, except for reserving 20 percent of the capacity building for women. Page 14 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AR Rural Inclusion (P106685) There were some gaps in assessing risks. Fiscal constraints that the government may face and potential disruption from electoral change were not anticipated, despite Argentina’s history of macroeconomic instability. Potential counterpart funding shortfalls were mentioned but mitigation measures were generic. Institutional arrangements were well-considered but they turned out to be less effective. Implementation by provinces and collaborations with technical institutions to overcome the capacity limitations in provincial implementation units, which was done following the restructuring proved to be a better model for implementation. The project was designed in a way that the project outputs contributed to outcomes that addressed Argentina’s problems. The results framework was not adequate to assess the outcomes. The failure to anticipate the fiscal risks required a restructuring. The rating, therefore, is moderately satisfactory. Quality-at-Entry Rating Moderately Satisfactory b. Quality of supervision Adequate supervision was provided during implementation, with 19 supervision missions in all during the 8 years of project operation. Missions were adequately staffed with appropriate skills. The COVID-19 pandemic restrictions forced the GOA and Bank teams to conduct four missions virtually from July 2020 to May 2022. They were critical for analyzing the project’s progress and supporting implementation units at federal and provincial levels in advancing the identification, formulation, follow-up, and strengthening of subprojects. The Bank conducted a timely MTR which led to the 2019 Restructuring. The restructuring included effective solutions to the problems the project was facing. They included the provision for sub-lending funds to provinces – rather than transferring them as grants – and a bigger role for provinces in implementation in collaboration with technical institutions that could effectively support many of the subprojects. The projects also found ways to integrate water and hunger programs to align even more with the priorities of the country. Aide-memoires and follow-up documents suggest that the bank team increased the intensity of supervision to ensure that the project was on track to achieve its outcomes. However, two extensions were required to bring the sub projects to a stage where they could be counted as outcomes. The World Bank’s Country Management Unit also supported the project team in identifying viable options for making progress by communicating directly with the Ministry of Finance. The World Bank team failed to obtain an evaluation of the project because of limited availability of appropriate human resources in the PIU and the intense pressure to implement project activities by the closing date. Page 15 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AR Rural Inclusion (P106685) Although supervision resulted in an appropriate restructuring of the project, accommodating the changing conditions and priorities, because of the failure to suitably modify the RF during restructuring and organize an evaluation to assess the outcomes, supervision is rated moderately satisfactory. Quality of Supervision Rating Moderately Satisfactory Overall Bank Performance Rating Moderately Satisfactory 9. M&E Design, Implementation, & Utilization a. M&E Design The results framework was designed to track project outputs and outcomes to guide implementation. It reflected the theory of change but it was not sufficient to measure the projects’ expected results at higher levels. The outcome indicators primarily measured the status of community organizations, their subprojects, and alliances along with the number of individuals reached. They offered little or no information on whether they delivered the intended benefits. The key assumption behind the design, that if the organizations or subprojects are operational, they will automatically benefit the members –, does not always hold. The framework comprised PDO indicators that in some cases measured outputs with limited evaluative value. For example, the PDO level indicator, the share of subprojects that have achieved 75 percent of the processes, included subprojects that were not yet operational. The intermediate indicators were in most cases measures of project outputs. For example, community organizations strengthened or community subprojects completed. One of the project objectives was to improve access to community infrastructure and services, but the framework did not include any indicators to measure that. Even after the 2019 restructuring when developing access to water became a priority, the framework lacked a core water indicator. No baseline was planned. In 2016, the Bank and the government agreed to apply a methodology that compared beneficiary cohorts via a specific survey to evaluate the impact of rural investment and productive alliance subprojects, with a focus on net income gains, cost reduction, and increased productivity. The MIS system was to be housed in Rural Sector Coordination Unit (UCAR) (later General Directorate of Sector and Special programs (DIPROSE)) which had the experience with previous operations (including Provincial Agricultural Services Project (PROSAP), PRODEAR, and Inclusive rural development program (PRODERI)). Provincial project teams were to monitor subproject implementation, collect data, and transmit the data to provincial coordination units within UCAR. Financing agreements under Component 2 and Component 3 required beneficiary communities and producer groups to directly send M&E information to provincial coordination units. Page 16 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AR Rural Inclusion (P106685) b. M&E Implementation Although inadequate to assess outcomes, the system generated information. Monitoring data was collected and analyzed directly from the execution report of each subproject and from information provided by provincial project teams to the provincial coordination units and UCAR/ DIPROSE. Given the steady stream of monitoring data, Bank Implementation and Supervision Reports (ISRs) rated the systems as satisfactory or marginally satisfactory. Although the project was modified to accommodate changing circumstances and priorities, the results framework and the M&E processer were not suitably changed. Some indicators that were hard to measure were modified but indicators were not introduced to reflect the new priorities. Not enough emphasis was placed on monitoring and evaluation in a difficult implementation context. In 2018, the Bank proposed a with-and without-subproject evaluation survey of SAP beneficiaries, as they had baseline data for sales volumes, but the result was a weak impact evaluation with insufficient data. In 2021, the PIU presented a plan for evaluating the sub-projects, but the delayed start-up of those SPs meant the achievements could not be measured. This is mainly due to the budget restrictions and institutional changes that limited DIPROSE’ resources, but both DIPROSE and the Bank failed to recognize that RF quality would impede a full accounting of the Project outcomes and richness. At closing, the Borrower Completion Report collected and evaluated robust data through questionnaires tailored to different beneficiary cohorts and types of investments, through random and representative sampling. c. M&E Utilization M&E findings were communicated to stakeholders. Up to date data supported decision-making on project management and facilitated the transition of projects from the federal level to provincial levels. The M&E also influenced the direction of implementation. The results informed the 2019 project restructuring and provided the basis for collecting information to estimate the returns on investments. The M&E information also informed the design of the subsequent Climate intelligent and Inclusive Agri- food system Systems Project (CIAF) Project (P176905) and is feeding into broader WB analytic work on the PA model, including a study related to gender. The M&E system was designed to monitor project implementation and facilitate an assessment based on the theory of change, but there were significant shortcomings that limited its value for examining impacts. It wasn’t modified adequately with restructuring. It, nevertheless, informed the design of a subsequent project. The rating is Modest. The ICR provided additional evidence from beneficiary surveys which was used to justify a Substantial (with shortcomings) rating of Efficacy despite the Modest rating of M&E Quality. M&E Quality Rating Modest Page 17 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AR Rural Inclusion (P106685) 10. Other Issues a. Safeguards Environmental The Project was classified as Category B. It triggered the following environmental safeguard policies: Environmental Assessment (OP/BP/GP 4.01), Natural Habitats (OP/BP 4.04), Pest Management (OP 4.09), Physical Cultural Resources (OP/BP 4.11), Forests (OP/BP 4.36), and Safety of Dams (OP/BP 4.37). The 2018 restructuring triggered Projects on international waterways (OP 7.50) International waterways (OP 7.50): an exception to the notification requirement under paragraph 7 of OP 7.5 was authorized on May 3, 2019. Safeguard instruments, Environmental and Social Management Framework (ESMF) and Indigenous Peoples Planning Framework (IPPF) and Resettlement Policy Framework (RPF) were updated. As the location and type of subprojects, because they were demand-driven, would not be known before Project implementation, the Borrower prepared an Environmental and Social Management Framework (ESMF), dated October 21, 2014, and disclosed in Argentina and on the World Bank Group's external website. By 2018, the Bank reported persistent limitations in compliance with the Project’s environmental and social (E&S) commitments and was concerned about counterparts' limited knowledge of the safeguards instruments. The Bank and DIPROSE developed a plan to strengthen safeguards performance, including a workshop to strengthen capacity for early identification of environmental and social issues during the formulation of subprojects, to ensure they were addressed with the support of safeguards specialists. By June 2021, the Bank recommended that DIPROSE, in preparation for Project Closing, record the adjustments made in environmental and social (E&S) aspects during special projects formulation, resulting in better quality special projects being presented for financing. The Bank noted in 2022 that the environmental impact of Socio-Economic Inclusion in Rural Areas (PISEAR) subprojects was low but that additional studies were needed to document impacts and mitigation measures. The following findings are relevant for each safeguard: Natural Habitats (OP 4.04): Training and TA on management and preservation of natural resources was provided in SPs to ensure compliance. Pest Management (OP 4.09): Many SPs, notably those that improved crop production, received training and TA in biological or environmental control methods to reduce reliance on chemical pesticides, as well as agroecological technologies such as green manure and traditional practices, thereby minimizing or avoiding the use of agrochemicals. Physical Cultural Resources (OP 4.11): No works were carried out on physical patrimonial sites of cultural or religious significance, nor did any surveyed works come across archaeological or paleontological remains that required the intervention of specialized institutions for their protection. Page 18 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AR Rural Inclusion (P106685) Forests (OP 4.36): Subprojects did not affect forest areas, nor did they reduce them: no clearing of native forests or large-scale removal of vegetation was carried out. Safety of Dams (OP 4.37): Although subprojects to improve access to safe water were appraised, none of them involved dam construction, relying instead on drilling and constructing wells, or on confinements on slopes. Social PISEAR was expected to achieve mostly positive social development outcomes and impacts. Project design specified that, as a cross-cutting concern in all project activities, PISEAR would pay attention to social inclusion, specifically of vulnerable groups (indigenous people, women, and youth). Mechanisms to promote social inclusion comprised: (a) earmarking 30 percent of Component 2 for indigenous people, community Subprojects; (b) reserving a minimum 20 percent of Component 1 for capacity building of women/youth; and (c) adopting positive discrimination (affirmative action) in SP prioritization. PISEAR triggered both social safeguards, namely Indigenous Peoples (OP/BP 4.10) and Involuntary Resettlement (OP/BP 4.12),43 although there was no involuntary resettlement of individuals caused by Project works and activities. The Borrower prepared an Indigenous People’s Planning Framework (IPPF) to address activities which could potentially affect/involve indigenous people. In addition, each province with subprojects targeting indigenous communities, designed its specific plan observing their needs and characteristics. Prior, free, and informed consultations were carried out as well as participatory diagnoses following the planned protocol. The informed participation of IP in developing projects of interest to their communities in itself represented enhanced social inclusion. Grievance Response Mechanism (GRM): The Project rolled out a citizen engagement system at the ministerial level, accessible online, by phone, and by mail, at the beginning of 2020. Beneficiaries were informed about this system by the provincial Implementation Units (UEPs). Stakeholders were also able to process grievances at specific addresses associated with Plan Agua and the PAs. By November 2022, the Project's GRM had processed 193 grievances, of which none remained outstanding at Closing. b. Fiduciary Compliance Procurement Procurement under PISEAR was affected by the broader challenges of restrictions on budgets and human resources, with personnel reductions in UCAR through to 2019. Subsequently, efforts were made to hire more staff, but barriers to hiring specialized personnel and acute salary restrictions caused further attrition of the project team, including procurement specialists, which seriously delayed procurement processing. By late 2021, the Project lacked a Procurement Specialist. The Bank reiterated the importance of having a designated specialist and a consolidated Procurement team to conduct planned processes and avoid implementation delays in the closing stages. Page 19 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AR Rural Inclusion (P106685) Moreover, with average annual inflation during the Project's lifetime close to 50 percent per year, suppliers presented budgets with a validity of no more than one month, i.e., less than the time required to process the relevant contracts. Without adjustment mechanisms in medium-term contracts, the high rate of inflation discouraged the presentation of offers. The 2019 Restructuring revised the institutional structure for project implementation, adding an execution modality that resulted in both centralized and decentralized procurement.49 UCAR was responsible for procurement at the central level, while UEPs coordinated and supervised procurement by beneficiary communities under SAPs and SIRs. Although procurement was rated MS for most of the Project, the last two periods evaluated saw a drop in the rating to MU, due to the inflation and staffing factors mentioned above. During its last two years, PISEAR managed to reverse the slow pace of execution, with all procurement processes completed by Project Closing, but the exit rating for procurement remained MU. Financial Management Financial Management was rated Satisfactory throughout the Project’s lifetime. UCAR generally maintained financial management practices acceptable to the Bank, project accounts were regularly updated, and Interim Financial Reports (IFRs) were submitted on time. The Bank team supervised FM processes closely, facilitating and requiring adjustments as needed to guarantee the correct evolution of the Project. DIPROSE’s capacity to manage and monitor project implementation was affected by budgetary and staffing restrictions, and an action plan was put in place to correct these deficiencies. Following the change in Administration, the Bank recommended a marked acceleration in requests for reimbursement to release loan resources and maintain the pace of project execution/disbursement in the closing stages of the Project. As of end-March 2023, all the funds were fully documented by SP beneficiaries to DIPROSE, so there is no amount to be reimbursed to the Bank. Compliance with the project’s Financial Management (FM) arrangements were maintained throughout project implementation ending with a Satisfactory FM rating and a Substantial FM risk. Interim financial reports (IFRs) and financial audits were submitted as requested by the financing agreement and were accepted by the Bank. Some IFRs and financial audits were delayed but it did not affect the capacity to provide timely and reliable provision of information required to manage and monitor the execution of project´s activities. According to the bank team (discussed in a meeting on January 18, 2024) the final audit is still pending. c. Unintended impacts (Positive or Negative) New institutional relationships were generated at the territorial level. Provincial governments began to coordinate with territorial and national institutions on livelihood activities, as they took on project implementation. d. Other --- Page 20 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AR Rural Inclusion (P106685) 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment Moderately Outcome Moderately Satisfactory Satisfactory Inadequate risk assessment and Bank Performance Satisfactory Moderately Satisfactory mitigation; and weak results framework Quality of M&E Modest Modest Quality of ICR --- Substantial 12. Lessons The following lessons are derived from the ICR with some modification of language. Even when targeting of vulnerable groups is explicitly included in the objectives and the results framework, the desired outcomes may not be achieved if those priorities are not mainstreamed in decision making processes at all levels. Many community organizations requested capacity strengthening for women and youth and opportunities for them to develop subprojects. The project organized training for them but left the rest to the provinces, where local priorities ruled. A specific gender sub-indicator included in the framework was not achieved. The seemingly simple step of including representatives of indigenous persons in the provincial evaluation committees helped in reaching their targeted share. If the result framework is designed primarily to monitor project execution and an impact study is depended on for information on outcomes and to link them to outputs, the prospects for a completed evaluation can be improved if an independent evaluation with a baseline study is commissioned at the beginning of the project. When project implementation is pushed to the closing date with tightening financial and capacity constraints, doing an evaluation becomes the last priority. The Bank’s efforts to have a study done were not successful. Wherever provision of technical assistance to individuals or groups of beneficiaries is critical to a project activity, the prospects of delivering relevant assistance and maintaining it beyond project life may be improved by engaging technical and development organizations that already work with beneficiaries, rather than build the capabilities in project implementation units. When budgetary limitations were faced, the activities to strengthen beneficiaries’ technical capacities were the first to be cut, or delegated to provincial implementing units, which also lacked the required technical knowledge. Outsourcing technical assistance and training to technical organization (INTA and SAFCI) made it possible to close the knowledge gap, although the opportunity was missed to engage more local organizations with a presence in the territory and with contact with local producers. Such linkages could have increased technical support to ensure the development of strong subprojects and their sustainability after project closing. Page 21 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AR Rural Inclusion (P106685) This review adds another lesson: Where there are considerable doubts about whether a onetime project support to individuals or groups is adequate to achieve the project objective, the chances of success can be improved by considering options to deliver support and monitor outcomes beyond the life of the project. Success of this project depends on continued functioning of community organizations, the viability of whatever enterprises they are engaged in, and maintained alliances to deliver the project outcomes. Closing the project before the outcomes are achieved jeopardizes the outcomes as well as the learning. 13. Assessment Recommended? No 14. Comments on Quality of ICR The ICR comprehensively elaborates on project design, restructuring, project activities, outputs, and outcomes. The report is candid in noting that the PDO was too broad and the results framework was not comprehensive enough to effectively assess the project impact, taking a results orientation. The report is informative, offering details in footnotes and annexures and cross-referencing wherever necessary. It is internally consistent. In the absence of an impact study and valuable information for assessing outcomes from the results framework, the report draws on various other sources. However, they may not be adequate to rate various aspects of the project suitably. In discussing the ratings, it focuses on the outcomes of the projects. The lessons are based on the experience of the project, but they could have been better articulated. a. Quality of ICR Rating Substantial Page 22 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AR Rural Inclusion (P106685) Page 23 of 23