ICRR 12581 Report Number : ICRR12581 IEG ICR Review Independent Evaluation Group 1. Project Data: Date Posted : 03/22/2007 PROJ ID :P007918 Appraisal Actual Project Name :Natural Resources Project Costs 79.1 60.1 Management Project US$M ) (US$M) Country :Paraguay Loan/ Loan US$M ) /Credit (US$M) 50.0 47.0 Sector (s):Board: ): RDV - Other social US$M ) Cofinancing (US$M) 0.0 0.0 services (28%), General agriculture fishing and forestry sector (28%), Roads and highways (25%), Micro- and SME finance (12%), Central government administration (7%) L/C Number :L3708 FY ) Board Approval (FY) 94 Partners involved : None Closing Date 12/31/2002 03/31/2006 Evaluator : Panel Reviewer : Division Manager : Division : Christopher D. Ronald S. Parker Alain A. Barbu IEGSG Gerrard 2. Project Objectives and Components a. Objectives As stated in the Loan Agreement and the Staff Appraisal Report, the project had four objectives : (1) To establish the institutional framework necessary to deal with the major agricultural and natural resource problems of the project area in the Departments of Alto Parana and Itapua . (2) To generate relevant information through research and implement an area -wide system of natural resource protection. (3) To assist small farmers in the project area to establish sustainable and diversified agricultural production systems, to adopt natural resource conservation measures through appropriate land use planning, and to improve their productivity and incomes through processing and marketing of agricultural products . (4) To encourage the direct participation of the rural community living in the project area in the management of natural resources through the stimulation of cooperatives, local government, and other institutions . This would seek to alter the attitudes of the rural community toward natural resource management (NRM) and to develop a consensus on its importance and value. b. Components (or Key Conditions in the case of Adjustment Loans ): The project had five components at appraisal, and a sixth component was added in 1996 shortly after loan effectiveness: (a) Agricultural development (US$46.4 million at appraisal, $20.8 million actual): To assist in restructuring agricultural production towards sustainable, conservation -oriented agriculture. (b) Indigenous communities (US$3.4 million at appraisal, $2.9 million actual): To provide assistance to the remaining indigenous population in the project area . (c) Infrastructure development (US$15.4 million at appraisal, $17.7 million actual): To realign, rebuild, improve and maintain existing roads to reduce their negative environmental effects and to follow and support agro -ecological land use zoning. (d) Institutional development (US$5.1 million at appraisal, $4.7 million actual): To decentralize project management by encouraging local and more democratic participation . (e) Pilot agricultural credit (US$8.8 million at appraisal, $2.7 million actual): To finance 70 percent of individual sub-projects, up to a limit of $5.0 million, in a pilot credit program. (f) Community rural investments (zero at appraisal, $11.3 million actual): A stand-alone social fund to support pilot community productive activities to help alleviate poverty in three Departments outside the original project area, which was later expanded to ten Departments . This component was conducted separately from the other components until after the project's restructuring in 2000. The project was restructured in 2000 based on the findings of the MTR that was carried out in 1998-99. The objectives were not changed . The restructuring focused primarily on streamlining the institutional arrangements in order to encourage speedier, more effective and transparent operations . Most significantly for the subsequent success of the project, it was decided to shift the institutional strengthening priority away from the Ministry of Agriculture (MAG) and to the local and municipal organizations, given MAG ’s slow progress toward strengthening and decentralization. The pilot agricultural credit component was discontinued at this stage . c. Comments on Project Cost, Financing, Borrower Contribution, and Dates The project closed 3 1/4 years behind schedule. Institutional problems coupled with counterpart financing difficulties and numerous changes in government appointments -- including 13 different Ministers of Agriculture during the life of the project -- significantly delayed the early period of project implementation . Although the restructured project gained momentum, the project needed almost twelve rather than the original eight years to be satisfactorily completed . 3. Relevance of Objectives & Design : The objectives of the project were relevant, but the initial design was flawed . The objectives responded to urgent NRM and poverty issues in rural Paraguay . The central aim was to introduce a coherent strategy of watershed management by providing technical assistance, technology, and research results to small farmers in selected rainfall catchment areas in the Department of Alto Paran á and the northern part of the Department of Itapúa in Eastern Paraguay in order to reverse the environmental degradation that had gotten worse since deforestation accelerated in the region in the 1970s. It appropriately focused on a dynamic sub -region of Eastern Paraguay where the country ’s richest agricultural soils are found and where agricultural growth and colonization were contributing to increasing deforestation and serious agro -environmental problems. The original project design was too complex and unrealistic . This followed a complex and outdated integrated rural development (IRD) approach and tried to address too wide a range of issues . It tried to simultaneously carry out institutional strengthening, create regional units, decentralize, and coordinate many other institutions . The restructuring of the project in 2000 after the MTR made the design simpler and more focused on the achievement of the objectives. This removed the highly complex "decentralized" -- or more accurately "deconcentrated" -- institutional arrangements for implementation centered on the Ministry of Agriculture (MAG), and focused on working directly and effectively with beneficiary groups and local governments in order to achieve the objectives on the ground. These new institutional arrangements effectively bypassed the continued institutional instability within the MAG and enabled beneficiaries and local governments to participate more actively in project implementation . 4. Achievement of Objectives (Efficacy) : (1) Institutional framework (achieved ). After the MTR, the project succeeded in introducing an unprecedented level of decentralization into the agricultural sector in Paraguay and helped to establish the local -level institutional framework necessary to address major agricultural and natural resource issues in the project area . When combined with the high profile roles for communities and municipalities, as well as project -generated committees and associations, these activities also greatly contributed to fulfilling the fourth objective as well . (2) Research (achieved ). The project adapted and disseminated existing technologies as part of elaborating and implementing microcatchment and farm management plans . The project successfully carried out a series of activities to establish and consolidate the San Rafael Park (a protected area) within the limitations set by the undefined legal status of the area. (3) Agricultural production (achieved ): A longitudinal beneficiary survey that was carried out in a sample of seven targeted microcatchments over the 2000-2005 period demonstrated the widespread adoption of conservationist agricultural practices (such as green fertilizer, contour lines, soil decompaction and reforestation ) and significant improvements in productivity. Average yields of major crops increased in most microcatchments by more than 20 percent on average, including improvements in soybean yields by 20 percent, cotton by 11 percent, wheat by 7 percent, maize by 15 percent, cassava by 25 percent and rice by 44 percent. Less progress was made in terms of diversification due to very favorable prices for traditional crops, especially soybeans . (4) Participation (achieved ): The participatory methodology used by the project, which was pioneering in the Paraguayan context, is reported to have created substantial social capital . The self-governing and participatory neighborhood organizations formed under the project have interacted effectively with municipal and departmental governments, and actually administered and managed the funds destined for the microcatchment areas -- a first for Paraguay. The project allowed neighborhood organizations to successfully discuss, negotiate, and find solutions to their problems with government in an unprecedented way . Overall, improved NRM and more sustainable approaches to agriculture are now an accepted paradigm in Paraguay, while participatory methodologies for decision -making and working with small farmers and indigenous people are more commonplace. These achievements are remarkable considering that Paraguay was at an early stage with regards to environmental matters in general (and sustainable agriculture and protected areas in particular ), and that institutional capacity-building at the local level was needed when the project was begun . 5. Efficiency : An economic analysis was not carried during project appraisal . A post-project estimate of the economic rate of return (ERR) was carried out based on the limited baseline data available in the SAR, records of project outcomes, and data collected through interviews and from technicians . The analysis for the whole project was based on total investment costs and the results achieved by Components (a), (c) and (f) on agricultural production, infrastructure development and community rural investments . Benefits included increased production and lower farm costs that resulted from decreased soil erosion and soil fertility loss, as well as lower road maintenance costs and vehicle operation costs from investments made in the rehabilitation of 396 km of roads. Costs included all on-farm and off-farm investment and recurrent costs, extension to farmers, and community development costs and training . Under these assumptions, project investments generated a net present value of US$ 11.2 million and an overall ERR of 19.2 percent. 6. M&E Design, Implementation, & Utilization: No baseline data were collected, and very few performance indicators were defined at appraisal, which seriously handicapped later project monitoring and evaluation . The Planning Directorate of the Ministry of Agriculture, which was assigned responsibility for project M&E in the SAR, failed to assume this responsibility . Although the project provided some training for M&E, it failed to develop an efficient and effective M&E system with results -based indicators. The lack of an adequate M&E system, including social monitoring and performance indicators, failed to provide timely notice of implementation problems that were arising and that could have been averted with a better M&E system. A longitudinal beneficiary survey was carried out in a sample of seven targeted microcatchments over the 2000-2005 period, which demonstrated widespread adoption of natural resource conservation practices and significant improvements in productivity . 7. Other (Safeguards, Fiduciary, Unintended Impacts--Positive & Negative): The project was a category A project . Therefore, an environmental assessment was completed about 18 months before project appraisal. The EA concluded that the project would promote more sustainable and environmentally-benign agricultural methods for the region . The only component found to have specific risks of negative environmental effects was the infrastructure sub -component for rural road improvement . The ICR asserts that the project was a pioneer in addressing safeguard issues, particularly with regards to indigenous people and general stakeholder participation -- but does not provide any evidence to this effect . 8. Ratings : ICR ICR Review Reason for Disagreement /Comments Outcome : Satisfactory Satisfactory Institutional Dev .: Modest Modest Sustainability : Likely Likely Bank Performance : Satisfactory Satisfactory But only moderately so, because of the unsatisfactory quality at entry . Borrower Perf .: Unsatisfactory Unsatisfactory Quality of ICR : Satisfactory NOTES: NOTES - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - ICR rating values flagged with ' * ' don't comply with OP/BP 13.55, but are listed for completeness . 9. Lessons: (1) It is vital for microcatchment area -based projects, which are aimed at improving sustainable NRM and generating income in poor communities, to create and /or strengthen beneficiary organizations at the local level and to involve both local governments and beneficiary groups in an effective and transparent process of decentralized and participatory planning and decision -making. The highly participatory and more interdisciplinary microcatchment area-based approach (carried out under the agricultural development component ) was more effective in building both social capital and community involvement in NRM than the more limited menu CDD -style approach (carried out under the community rural investments component ). (2) Projects that are attempting to address complex problems like sustainable NRM in a weak institutional environment should utilize the simplest possible institutional arrangements for implementation, consistent with the objectives of the project. Project goals and objectives should also be highly focused and limited in order to maximize their achievement. It is crucial to establish strict criteria for targeting beneficiaries and areas as well as clear and transparent criteria and decision -making processes for allocating resources . (3) When working with indigenous groups, it is critical to work with and strengthen indigenous organizations, while respecting the culturally-defined decision-making mechanisms of each ethnic group . (4) An efficient and effective M&E system, including social monitoring and performance indicators, is a crucial management tool to ensure focused actions and quality controls in multi -disciplinary projects such as microcatchment area-based projects whose activities span many technical and institutional domains . Baseline date must be collected during project preparation . 10. Assessment Recommended? Yes No Why? This was a pioneering project in the context of Paraguay that claims to have brought about an effective paradigm shift in favor of more participatory approaches to diversified agricultural production in harmony with sustainable NRM and environmental protection . 11. Comments on Quality of ICR: The ICR is satisfactory overall, including a frank discussion of the severe problems (including unsatisfactory quality at entry) facing the project up to the MTR and a realistic rating for institutional development impact . However, the treatment of M&E and safeguard issues is weak, especially for a category A project . The ICR acknowledges that M&E was deficient from the start, but dosn't say how the project attempted to improve the situation sufficiently to produce the evidence that has been marshalled in support of the project's achievements . The ICR also asserts that the project was a pioneer in addressing safeguard issues, particularly with regards to indigenous people, but does not provide any evidence to this effect .