Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Scaling-Up Participatory Sustainable (P130222) Report Number: ICRR0023460 1. Project Data Project ID Project Name P130222 Scaling-Up Participatory Sustainable Country Practice Area(Lead) Lao People's Democratic Republic Environment, Natural Resources & the Blue Economy L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IDA-65350,IDA-H8520,TF-15286 31-Aug-2018 35,935,786.19 Bank Approval Date Closing Date (Actual) 31-May-2013 31-Jul-2022 IBRD/IDA (USD) Grants (USD) Original Commitment 19,000,000.00 12,830,000.00 Revised Commitment 36,829,789.31 12,829,789.31 Actual 35,935,786.19 12,829,789.31 Prepared by Reviewed by ICR Review Coordinator Group Alejandra Eguiluz Vibecke Dixon Avjeet Singh IEGSD (Unit 4) 2. Project Objectives and Components DEVOBJ_TBL a. Objectives The Project Development Objective (PDO) of the LA-Scaling-Up Participatory Sustainable Forest Management (SUPSFM), as stated in the Financing Agreement, is ¨To execute REDD+ activities through participatory sustainable forest management in priority areas and to pilot forest landscape management in four provinces. (Financing agreement, Schedule I), which is the same as stated in the PAD (page 3). Page 1 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Scaling-Up Participatory Sustainable (P130222) REDD+ stands for Reducing Emissions from Deforestation and Forest Degradation, Fostering Conservation, Sustainable Forest Management, and Enhancement of Forest Carbon Stocks. The revised PDO is the following: “To strengthen participatory sustainable forest management in targeted production forest areas, and forest landscape management in targeted provinces” (PAD for additional financing, page 31). It will be parsed, for the purposes of this review, as follows:  To strengthen participatory sustainable forest management in targeted production forest areas  To strengthen forest landscape management in targeted provinces Since the substance of the PDO did not change with the restructuring, a split evaluation will not be done. b. Were the project objectives/key associated outcome targets revised during implementation? Yes Did the Board approve the revised objectives/key associated outcome targets? Yes Date of Board Approval 31-May-2013 c. Will a split evaluation be undertaken? No d. Components Component 1: Strengthening and Expanding participatory sustainable forestry management (PSFM) in Production Forest Areas (estimated cost at appraisal: IDA US$ 12.22 million, Forest Investment Program (FIP) US$ 8.15 million, and Government of Laos US$ 4.98 million. Actual cost at closing: IDA US$12.67 million, FIP US$ 6.02 million) Subcomponent 1A: Developing Partnerships to Increase Implementation Capacity, aimed to support capacity building and skills development, including technical forest management, facilitation, and communication skills, improved financial management, gender equity, and monitoring and evaluation. It would also support the development of strategic partnerships and collaborative arrangements with other stakeholders and contract agencies and finance the creation of a Forest Certification Unit (FCU) within the Department of Forestry (DoF). Subcomponent 1B: Community Engagement in PSFM and Village Livelihood Development, aimed at supporting the elaboration and implementation of site-specific Community Action Plans (CAPs) by the participating villages, following the Community Engagement Framework (CEF) and identifying options for generating sustainable livelihoods (including forest-based livelihoods). Page 2 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Scaling-Up Participatory Sustainable (P130222) Component 2: Piloting Forest Landscape Management (FLM) (Estimated cost at appraisal: IDA US$1.17 million, FIP US$0.78 million, and GoL US$0.48 million. Actual cost at closing: IDA US$ 0.02 million, FIP US$ 0.02 million). Subcomponent 2A: Developing Methodologies and Frameworks for FLM aimed to support the development and adoption of a landscape approach for forest and biodiversity resources management in four northern provinces in Lao PDR. Activities would include support for coordinated management planning, elaboration of methods for REDD+ related Measurement, Reporting, and Verification (MRV) and Reference Emission Levels (REL) at the landscape scale, and improved coordination on forest law enforcement. The implementation was to be in close collaboration with the Finland supported project on strengthening national spatial planning capacity in Lao PDR for sustainable natural resource management. Subcomponent 2B: Establishing Forest Landscape Pilots aimed to finance the planning and implementation of the CEF and PSFM in selected village forests. These provincial pilots were to include strengthened coordination among agencies responsible for forest protection and law enforcement and identifying overlapping and adjacent development activities, minimizing and mitigating impacts of such activities on selected forest landscape areas. Component 3: Enabling Legal and Regulatory Environment (Estimated cost at appraisal: IDA US$1.97 million, FIP US$1.31 million, and GoL US$0.80 million. Actual cost at closing: IDA US$ 1.93 million, FIP US$ 0.78 million). Subcomponent 3A: Strengthening Legal and Regulatory Frameworks was to support the development of a legal and regulatory framework for the implementation of PSFM and FLM, and for improved monitoring and reporting on timber revenue benefit sharing, domestic timber processing and sales, international timber trade, and tracking the share of timber coming from certified sources Subcomponent 3B: Strengthening Forest Law Enforcement and Governance was to strengthen forest law enforcement and governance by implementing the National Forest Law Enforcement Strategy 2020 by the Department of Forest Inspection (DOFI). Targeted support was to be provided for monitoring salvage logging and improving capacity for assessing environmental compliance. Subcomponent 3C: Creating Public Awareness for Climate Change and REDD+ was to support GoL’s efforts to raise public awareness about climate change and REDD+ by designing a national communication strategy and a public awareness campaign. Component 4: Project Management (Estimated cost at appraisal: IDA US$3.19 million, FIP US$2.13 million, and GoL US$1.3 million. Actual cost at closing: IDA US$ 8.49 million, FIP US$ 6.01 million). This component covered project management at the national and sub-national levels, technical assistance (TA), and monitoring and evaluation (M&E). A parallel TA from the Government of Finland was to provide national and international consultants and capacity building and training to support the expansion of the project in Priority Forest Areas (PFAs), strengthen forest law enforcement and governance, support forest sector policy reform, build capacity for participatory land use planning and tenure strengthening, support development of sustainable livelihoods, and undertake analytical work as required to meet the overall objectives of the project. Changes to components (at 2020 additional financing/3rd restructuring) Page 3 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Scaling-Up Participatory Sustainable (P130222) Component 1. The financing for subcomponent 1A was discontinued because the initial objective of developing and adopting a landscape approach for forest and biodiversity resources management was deemed unattainable. Component 2. Subcomponent 2A was dropped. The concept of Forest Landscape Management Plans was very novel, and the lack of definition in the project's proposal of interventions didn't help attract stakeholders' buy-in. Subcomponent 2B was revised. Originally titled “Establishing Forest Landscape Pilots,” it was renamed “Establishing Forest Landscape Investment Plans” to develop practical and simplified investment plans for selected priority landscapes in selected provinces through consultations with diverse public and private stakeholders. A new intermediate indicator was added to the results framework to track this activity. More specifically, activities to be implemented under the AF included the following: (i) prepare Forest Landscape Investment Plans for priority landscapes in selected provinces; (ii) prepare assessments to support landscape investment development, including identifying overlapping and adjacent development activities, and minimizing and mitigating impacts of said development activities on selected forest landscape areas; (iii) support dialogue, consultations, and multi-sector platforms on landscapes, land use, and REDD+; (iv) develop an integrated monitoring framework across land uses in the forest landscape; and (v) build institutional and leadership capacity for landscape-level action and management. Component 3. Subcomponent 3C (Creating Public Awareness for Climate Change and REDD+) was dropped. The activities, however, continued to be financed under the REDD Readiness support projects implemented by the Department of Forestry (DOF). A split rating will not be done because a) the PDO was amended to clarify the original project objectives without materially changing them; and (b) the changes to the indicator targets (i.e., for the outcome-level indicators in 2018) were made to facilitate measurement and reporting of the outcomes better. e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project cost. At appraisal, the total project cost was US $39,39 million. IDA financing was US $19 million (IDA-H8520). The Forest Investment Program Grant was US $12.83 million (TF-15286). The revised amounts following the AF of 2020 are as follows: IDA-H8520 was US $ 17,9 million; TF-15286 was US $12,83 million, and IDA-65350 was US $ 4,96 million. At project closing, the actual amounts were IDA- H8520 US $ 12,83 million; TF-15286 was US $12,83 million, and IDA-65350 was US $ 5,14 million. Financing. At appraisal, the lending instruments for the project included: (i) Investment Project Financing (IPF) with an IDA grant (H852-LA) of US $ 19 million; (ii) A grant from the Forest Investment Program (FIP) under the Strategic Climate Fund (SCF) (TF-15286) for the amount of US $ 12.83 million; (iii) Parallel financing from the Government of Finland for technical assistance for US$ 14.5 million. In 2017, the Finnish financing terminated, with the withdrawal of EUR 3.3 million. The financing amount was thus reduced to EUR 7.6 million. In 2020 there was additional financing of US $5 million IDA credit (IDA-65350); Borrower contribution. The planned contribution of the Government Of Laos (GoL) was US $7.56 million at appraisal. The contributions were in nature, and no data was provided regarding the actual amount of the contribution at the project closure. Dates Page 4 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Scaling-Up Participatory Sustainable (P130222) The IDA grant H852-LA was approved on May 31, 2013. The FIP grant TF015286 was approved on August 8, 2013. Project closure was May 2018 originally, but successive extensions pushed closure to July 2022. In 2020 there was additional financing of US $5 million IDA credit (IDA-65350). Successive restructurings took place in May 2018, May 2019, February 2020 and August 2021. The restructuring of May 2018 adjusted the wording and definitions of the indicators. Changes were made with regard to the closing date and the implementation schedule. The revision of the indicators does not materially affect the scale, scope, or nature of the project’s objectives but rather helped ensure that project achievements can be measured appropriately. The changes improved the quality of the RF by making it more adequate and relevant and by improving the link between the indicators and the project intervention in order to increase attribution. Of the five revised outcome-level indicators, the target value for 2 of them was decreased: : (i) Forest area brought under management plans (ha) because the severe security situation made the areas inaccessible, and ; (ii) People in the forest and adjacent community with monetary/non- monetary benefit from the forest, whose baseline was modified to 0 and the target was adjusted to reflect project benefits only and not other forest benefits outside the scope of the project (clearer attribution). The main reason for the extension was the delay in disbursements of grants for the villages under component I (although the grants had been disbursed to 70% of the districts, only 56% of the villages had received them) The restructuring in May 2019 changed the closing date (7-month extension to March 31, 2020) and reallocated amounts between disbursement categories. IDA proceeds and Forest Investment Program (FIP) grants were consolidated into one expenditure category to streamline the disbursement of the two grants and to facilitate the preparation of the US$ 5 million additional financing package. The restructuring in February 2020 integrated the AF (IDA-65350) of US$ 5 million credit and clarified the wording of the PDO (which was originally activity-oriented) and increased five of the six PDO indicator targets to match the actual numbers already achieved to incentivize further progress. It also extended the loan closing date by 15 months (until August 31, 2021). The additional financing and extension compensated for the foreign exchange rate loss of approx. US$1.5 million to ensure continuity of support in anticipation of a new Investment Project Financing (IPF). The restructuring in August 2021 extended the closing date to July 2022. 3. Relevance of Objectives Rationale Context at Appraisal At the time of appraisal (2013), the Lao People’s Democratic Republic (Lao PDR) had experienced significant economic growth and poverty reduction during the previous five years. The country's economy grew at an average annual rate of 6.5 percent from 1990 to 2009 (PAD, page 1) thanks to foreign direct investment, resulting in a 30 percent decrease in poverty. The country attracted significant investments in mining, hydropower, and agriculture, but unsustainable resource management and increasing inequality between urban and rural areas posed challenges. The forestry and agriculture sectors contributed 30% of the Gross Domestic Product in 2008-2010 and provided 75% of total employment, but deforestation and Page 5 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Scaling-Up Participatory Sustainable (P130222) forest degradation threatened these economic benefits. In rural areas, it is estimated that non-timber forest products contribute between 30-70% of income for forest-dependent households. (PAD, page 2) The government was strengthening the legal, policy, and institutional framework for sustainable forest management, aiming to increase forest cover, generate sustainable forest products, and promote environmental conservation and protection. At appraisal, the National Assembly was leading a process to finalize a new Land policy and revise the 2013 Land law; concurrently revisions of the Forest law were also underway. (Note: the Land law and Forest law were approved in 2019). The GoL had placed a moratorium on the grant of new concessions in mining and rubber plantations until December 2015. In June 2011, the National Assembly approved the establishment of a new ministry called the Ministry of Natural Resources and Environment (MoNRE). A National REDD+ Task Force had been established with inter-ministerial representation. Consistency with Country Strategy At appraisal, the Government’s Forestry Strategy 2020 aimed to: (i) improve the quality of the existing forested areas to about 70% of the total land area; (ii) to generate a sustainable stream of forest products; (iii) to preserve unique and threatened habitats; and (iii) to promote environmental conservation and protection (PAD, page 2). The project was fully aligned with the GoL 7th National Socio-Economic Development Plan for 2011-2015 (NSEDP7) and its subsequent iterations, NSEDP8 (2015-2020) and NSEDP9 (2020-2025), all aiming to improve living standards by way of the implementation of the Sam Sang (3-builds) policy that prioritizes poor villages for development assistance. The project also aimed to assist the country with reducing deforestation and forest degradation, including formulating the national REDD+ strategy and enhancing readiness for future emission reduction payments. By supporting REDD+, the project was also to contribute to global climate change mitigation efforts, including reducing emissions from deforestation and forest degradation. Lao was one of the first 14 countries that became a REDD+ country participant under the Forest Carbon Partnership Facility (FCPF) in 2008. The project was to support the international commitments undertaken in this regard. At the time of appraisal, an inter-ministerial REDD+ Task Force had been established to coordinate REDD+ activities at the national level. A national REDD+ readiness process was also underway, with which SUPSFM shared programmatic and institutional linkages. Consistency with Bank Strategies At appraisal and during the first 3 years of implementation, the project was designed to support Objective 2: Sustainable Natural Resource Management under the Country Partnership Strategy (CPS) FY2012-2016, particularly its Outcome 2.3: Sustainable Management and Protection of Forests and Biodiversity. During the following 4 years of implementation, the project was aligned with the CPF 2017-2020, that supported the GOL’s 8th National Socio-Economic Development Plan (NSEDP) for 2016-2020 and followed the priorities identified in the Systematic Country Diagnostic (SCD) for Lao completed in 2017 (that had sustainable and efficient management of natural resources as the first development pathway);. The CPF 2017-2020 targets three focus areas for WBG engagement in Lao PDR, namely: (i) supporting inclusive growth, (ii) investing in people, and (iii) protecting the environment. Previous Bank experience Page 6 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Scaling-Up Participatory Sustainable (P130222) The WB has provided long-standing assistance to Lao PDR’s forestry sector since 1995, promoting participatory sustainable forest management (PSFM). SUPSFM represents the third major investment project supporting the sector, following the Forest Management and Conservation Project (FOMACOP; 1995-1999) and the Sustainable Forestry for Rural Development Project and its Additional Financing (SUFORD; 2003-2012). These successive projects have all revolved around sustainable and participatory management of production forests with progressively greater geographic scope from two provinces under FOMACOP to 13 provinces under SUPSFM. Lessons learned from the previous project oriented this one in several important areas: (a) explicitly incorporating and monitoring forest carbon emission reductions, (b) introducing performance payments for forest carbon sequestration, (c) focusing additional efforts on developing sustainable livelihood options, and (d) fostering inter-agency coordination at the landscape scale. The Government of Finland and the World Bank have provided coordinated investment support for natural resources management and participatory approaches in the forest sector for several years under SUFORD. The World Bank was, at the time of approval, engaging in parallel with several other similar projects in Forestry Management, such as the Argentina P132846, approved in 2015, and the Congo PDR P124085, approved in May 2012. Level of ambition of the PDO. The original PDO (To execute REDD+ activities through participatory sustainable forest management in priority areas and to pilot forest landscape management in four provinces) was pitched at a low level of the results chain. The ICR notes that it was output oriented and ambiguous, with an emphasis on the activities. The revised objective (To strengthen participatory sustainable forest management in targeted production forest areas and forest landscape management in targeted provinces) was relatively results focused though since this was the World Bank’s third project in the area, it could have been relatively ambitious. However, several of the identified indicators were at outcome-level (such as i) people with monetary- and non-monetary benefits from the forest, ii) enhanced carbon storage from forest protection; and iii) reduced emission from deforestation and forest degradation, etc.), indicating that the project aimed at higher level outcomes. Rating Relevance TBL Rating Substantial 4. Achievement of Objectives (Efficacy) EFFICACY_TBL OBJECTIVE 1 Objective To strengthen participatory sustainable forest management Rationale Theory of change (ToC) Page 7 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Scaling-Up Participatory Sustainable (P130222) The project’s ToC was not made explicit in the PAD but is described in Figure 1 of the ICR (page 8). The original wording of the PDO was relatively activity oriented and ambiguous rather than at the outcome level and was subsequently revised. The project’s underlying ToC is: to achieve the ultimate goal of the outcome of improved management of production forest areas at the landscape level (pitched as if it were outcome level); interventions in the following three results are to be reached: (i) improvement of the participatory project management of forests in priority areas; (ii) improvement of forest land management in 4 provinces; (iii) enabling legal and regulatory environment. The ToC is clear and convincing and does not have any logical gaps. Outputs  76% of beneficiaries perceived having received technical services of adequate quality (beneficiaries are both from the VLD grants and the production groups), according to the impact surveys undertaken. The target was 83%. This target has been 91% achieved.  97% of participants of the CEF process perceived that the support was adequate, achieving the target of 97%.  10,750 ha of forest were brought under the Village Forest Management Agreement (VFMA), which is the instrument selected by the government to strengthen tenure in village forests, achieving the target of 10.750 ha.  All grants were disbursed to villages, and the activities were under implementation at the project closure. The target of 100% of the grants disbursed was achieved.  The target of reaching 112,000 ha under certification (Controlled Wood Standard) was not achieved, as only 50,534 ha was certified (the low achievement of this output is due to COVID-related travel difficulties and the logging ban, as there was no incentive to cover the cost of the certification without the financial benefits of the timber harvesting in sight). The GoL imposed a logging ban at the beginning of the project implementation, which was not waived as expected, and it was still in operation at project closure, resulting in a significant decrease in the revenue flows from timber harvests.  The percentage of the Strategic and Tactical Enforcement Patrol Program field activities that were to result in enforcement actions was 85% (target). The project achieved 82% (The target was almost 96%).  The following reforms in forest policy, legislation, and other regulations were supported as targeted; completed or ongoing policy or legal work on village forestry, salvage logging, preparation of the pending new Forest Law (including forestry legal compendium, wildlife and aquatic legal compendium, penal code, TLAS, Forest Certification Systems), implementation of PMO-15, National REDD+ Strategy, approval of PMO5 (wildlife law enforcement), and PMO9 allowing forest plantation in PFAs. Target achieved.  Increased investigations of breaches of forestry law and wildlife and aquatic law referred to criminal courts (target was doubled and 100% achieved), with increased inter-agency coordination envisaged under AF that would likely increase the number of successful investigations. Outcomes The Forest area brought under management plans was 976,211 ha, achieving the target of 976,211 ha. However, the revenue that was to be generated from the productive timber activities under these plans was hindered by the logging ban imposed by the GoL, and that remained until the end of the project. The management plan preparation process (including delineation of different land-use types and associated Page 8 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Scaling-Up Participatory Sustainable (P130222) mapping) was generally conducted in a participatory manner, involving stakeholders from central to village levels. The process was perceived to be adequate by 97 percent of the participants. 117,400 people in the forest and adjacent communities received monetary or non-monetary benefits from project interventions, achieving the target of 117,400. The indicator measures the direct beneficiaries of the grants, as opposed to populations benefiting from forests more broadly (as per restructuring in 2018). Impact assessments were undertaken and concluded on substantial socioeconomic impacts from livelihood diversification, reduced reliance on shifting cultivation, increased income and saving, and improved access to healthcare and education. However, the ICR points to some methodological weaknesses of the assessments, reducing the validity of the conclusions. A follow-up survey conducted with village grant beneficiaries found that 95 percent of the respondents perceived the quality of VLDG technical services as either adequate or very adequate. The project-related law enforcement activities contributed towards the outcome indicator of the reduction of the rate of annual forest loss. The rate of annual forest cover loss in targeted Production Forest Areas dropped from a baseline of 0.27 percent to 0.18 percent, not quite achieving the original target of 0.15 percent but achieving the revised target of 0.18 percent. 33,500 tons of CO2e of enhanced carbon storage was calculated to have been achieved from improved forest protection and restoration, compared to a baseline of 1,445, exceeding the original target of 14,227 and almost achieving the revised target of 35,000 tons of CO2e. The real impact could be smaller than reported, for the following reasons: (i) the calculation of the final figure used assumptions that belonged to the previous project (regarding the success rate of the VLLG, of 90%); (ii) it used an overestimated conversion rate from shifting cultivation to forest generation. The project’s output targets were significantly achieved (except for the forest surface under certification), contributing to achieving the main outcome targets for forest management, enhanced carbon storage, and reduced forest loss. The benefits to communities are measured by the grants delivered and their level of satisfaction with the quality of services. Objective 1 was thus substantially achieved. Rating Substantial OBJECTIVE 2 Objective To strengthen forest landscape management in targeted provinces Rationale Theory of Change: The establishment of forest landscape investment plans (input) is expected to lead to an increased forest area brought under the forest landscape management plans (output) and ultimately to strengthen the forest landscape management (outcome). The concept of a forest management plan and the process of elaborating it were not well defined in the PAD, and some ambiguity remained during implementation. Stakeholders perceived the interventions in this area as experimental. During the third restructuring (and AF), subcomponent 2A was dropped, and Subcomponent 2B was renamed as “Establishing Forest Landscape Investment Plans” to develop practical and simplified investment tools for Page 9 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Scaling-Up Participatory Sustainable (P130222) selected priority landscapes in selected provinces through consultations with diverse public and private stakeholders. A new intermediate indicator was added to the results framework to track this activity, but the one outcome-level indicator for this objective remained the same: Forest area brought under forest landscape management plans (ha) Outputs Four forest land management frameworks were developed against the target of four. No other output indicators were identified under this sub-objective. Outcome 3,380,170 ha for the four provinces targeted of Bokeo, Luang Namtha, Oudomxay, and Xayaboury were brought under forest landscape management plans, exceeding the original target of 2,682,000 ha and achieving the revised target of 3,380.170 ha. Although this indicator is reported to be achieved in the RF, the ICR pointed out (p 20) that what had been counted as “plans” only were provincial maps that would serve as inputs to elaborate the actual plans (ICR, page 20). This outcome can thus not be counted as fully achieved. The international definition of a Forest Management Plan is broad and can include a wide range of topics related to the management and use of land. In Lao PDR, there was little awareness of this notion when the project started. Although an attempt was made to identify suitable entry points in coordination with relevant stakeholders, none gained traction from the decision-makers. Towards the end of the project, what gained momentum was the preparation of provincial investment maps at the landscape level, identifying potential adverse impacts on the forest sector (e.g., pulp mills, large-scale agriculture projects, road construction, and hydropower development). This process experienced delays due to COVID-19 and its associated domestic travel restrictions; thus, the maps remained incomplete at the project closure. The unfinished work is to be taken over and completed by the new WB Lao Landscapes and Livelihoods (LLL) Project, approved in January 2021. The elaboration of land management investment plans (input) and subsequent forest area involvement (output) were the only interventions toward achieving Objective 2 (strengthened forest landscape management). The land management plans, from lack of precision and buy-in, were not completed, and only provincial investment maps had been elaborated by the end of the project. The outcome target was thus not fully achieved. Given these shortcomings, the rating is Modest for Objective 2. Rating Modest OVERALL EFF TBL OBJ_TBL OVERALL EFFICACY Rationale The rating for Objective 1 is Substantial and the rating for Objective 2 is Modest. The overall rating is Substantial with moderate shortcomings. Page 10 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Scaling-Up Participatory Sustainable (P130222) Overall Efficacy Rating Substantial 5. Efficiency Economic efficiency Ex-ante analysis. At appraisal, it was estimated that to generate a rate of return of 12%, using a discount rate of 10 percent over a 25-year time, the present value of benefits needed to be US$52.2 million. The present value of net benefits from participatory sustainable forest management in the 41 PFAs, and benefits from non- timber forest products collection, forest restoration (including the provision of fuel wood, fodder, and carbon sequestration), and village activities was examined and resulted in approximately US%91.8 million. The assumption was that even if the prices for carbon and timber were further depreciated, the present value of benefits still justified the project. A financial analysis was carried out at appraisal to assess whether the targeted beneficiaries would get sufficient monetary benefits to justify their adherence to and participation in the project. The returns were estimated based on the projected share of benefits from revenues from timber production, net returns to forest restoration activities, cash returns to NTFP collection, and net returns from special activities that the village grant could finance. The net present value of the return was 73.2 million (well over what is required to generate a 12 percent return, using a discount rate of 10 percent over 25 years). This financial return was, however, sensitive to changes in the volume of timber extracted and the price of timber. If both were to decrease significantly, the conservatively estimated financial returns would not justify the investment (and this is what happened). Ex-post analysis. The logging ban in place throughout project life (since 2013) had the immediate unexpected consequence of decreasing the revenue flows from timber harvests, a substantial part of the expected benefit flows emanating from forest management, harvesting production, and supply-chain activities. However, the ban is deemed to have contributed to accelerating forest restoration and increasing average carbon storage in the priority forest areas (PFAs). Interventions under Component 1 also played an important role in increasing carbon storage and reducing emissions. Ex-post economic analysis is not conclusive. Depending on the price of carbon used, the project's economic returns vary, with an NPV figure ranging from US$631,000 to US$97.5 million. The model becomes more financially sustainable when the carbon price is US$40/tCO2e or higher. As stated on the ICR: the project’s economic contributions may have been significantly larger or smaller than the original estimate, depending on the carbon price used (ICR, page 21). The ICR states that tangible financial benefits to project stakeholders were limited but does not analyze them in detail. It can only be assumed that the economic efficiency is overall modest, given that ex-ante, it had been found that the financial return was sensitive to changes in the volume of timber extracted and the price of timber and that if both were to decrease significantly, the conservatively estimated financial returns would not justify the investment (PAD, page 11, paragraph 46) Operational and administrative efficiency Page 11 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Scaling-Up Participatory Sustainable (P130222) The project experienced delays in implementation. During the first years, the main delay was experienced in the village forest restoration grants due to the blockages in the flow of funds from the district to the village level. The project implementation arrangements were based on the vertical GoL administration system with four layers: central, provincial, district, and village, which turned out to be a barrier to efficient disbursement of the grants, mainly due to inadequate financial management (FM) capacity at the provincial and district levels. It was one of the main reasons for the first project restructuring and extension in May 2018. Another significant delay occurred in relation to the procurement of 75 vehicles and 755 motorcycles in 2017 due to a disagreement between DOF and the Ministry of Finance on the treatment of import duties (paid as part of the GoL contribution or exempt). The decision-making process took almost two years, causing a significant delay in project implementation. The COVID pandemic also caused some delays (mainly due to the restrictions on public gatherings and the consultants’ travel) and was the main reason for the last project extension. The extended project timeframe was partly due to inefficiencies in project implementation and to accommodate the AF. Efficiency Rating Modest a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 0 Appraisal 0  Not Applicable 0 ICR Estimate 0  Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome Based on the Substantial Relevance of Objectives, Substantial Efficacy (with moderate shortcomings), and Modest Efficiency, the outcome rating is Moderately Satisfactory. a. Outcome Rating Moderately Satisfactory 7. Risk to Development Outcome Economic risk: Whereas the logging and unprocessed wood export bans contributed to strengthening the regulatory framework against illegal logging, the risk is that, in the short and medium term, they can negatively affect the revenue flows from sustainable timber harvests. Page 12 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Scaling-Up Participatory Sustainable (P130222) Economic risk: The current ongoing macroeconomic crisis could increase pressure on forest resources for livelihoods; unsustainable exploitation of natural resources tends to intensify during severe economic crises, as it is often a last resort for people deprived of other livelihood options. Stakeholder ownership risk: While SUPSFM made strides in raising awareness of national and local stakeholders about participatory and sustainable forest management, the 2019 forestry law introduced a new management approach (village forest management), which aimed to transfer rights and responsibilities of forest management to the village level. As this new concept is not spelled out in the forestry law, there is a risk of creating confusion and conflict between the old and new management approaches, hampering stakeholder ownership. Institutional risk: Lao PDR lacks an effective institutional mechanism for the coordination of competing land uses, and there are widespread perceptions among national and subnational authorities that agriculture provides an excellent opportunity to improve livelihoods compared to forests (particularly for degraded forest lands). The process of developing landscape-level investment maps initiated under SUPSFM represents an opportunity to convene stakeholders across sectors and facilitate transparent decision-making on land use. Such an inter-sectoral decision-making process risks being damaged if not anchored by a firm and sustainable institutional mechanism. 8. Assessment of Bank Performance a. Quality-at-Entry The overall design of SUPSFM was built on and informed by preceding WB forestry sector projects in the country, particularly the previous one, SUFORD. It used the lessons learned on important aspects of project implementation, including community engagement, alternative livelihoods, sustainable management of PFAs, and the legal and regulatory framework. The project appropriately selected key stakeholders to work with and identified risks and mitigation measures. The WB team secured FIP co- financing and Finnish parallel financing complementary to the IDA grant. Some significant shortcomings in design were that the original PDO was activity-oriented. The interventions related to Objective 2 were insufficiently developed for such a novel concept; the forest landscape management plans were novel in the country and challenging to put in place without clear definition and consensus among stakeholders, as well as a well-defined path for elaborating them. Quality-at-Entry Rating Moderately Unsatisfactory b. Quality of supervision The World Bank team conducted 17 supervision missions during the nine years of project implementation. They provided strong technical support to the project and responded promptly and effectively to issues during project implementation. On several occasions, the WB team showed adaptive management Page 13 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Scaling-Up Participatory Sustainable (P130222) approaches to cope with changing needs and minimize negative impacts. For instance, when EUR 3.3 million in Finnish parallel financings were withdrawn in 2017, the WB team made a decision to take over some of the critical Finnish TA components, including the international consultant team, as part of the PIU, using IDA resources. It mobilized the US$5 million in additional financing. It extended the project implementation period while advancing the preparation of the new forestry sector investment project: the Lao Landscape and Livelihood (LLL) project (which was approved before the end of SUPSFM and incorporated lessons from the latter). The revision of Component 2 as part of the Additional Financing restructuring paper in 2020 could have been more thorough by providing an analysis of what achievements would be realistic to expect under the second objective. More relevant and clearly defined indicators could also have been identified. Quality of Supervision Rating Moderately Satisfactory Overall Bank Performance Rating Moderately Satisfactory 9. M&E Design, Implementation, & Utilization a. M&E Design The original PDO was relatively output-oriented, and the original PDO indicators, although relevant, somewhat lacked precision in their definition and linkage to the project interventions. At appraisal, it was planned that the project would finance targeted annual and periodic studies to assess project performance, and joint implementation support missions with Finland will evaluate performance against a wider range of indicators that will be incorporated into the Project Implementation Plan. Communities were expected to play a direct role in monitoring and evaluating project performance. A full-time monitoring and evaluation adviser was foreseen as part of the technical assistance team. b. M&E Implementation Led by the PIU (National Project Management Office, NPMO), the M&E implementation was generally conducted in a satisfactory manner. The PDO was revised to enhance its outcome orientation, and some indicators were better defined. Comprehensive semi-annual and annual reports were produced in a timely manner. Progress reports and accompanying PowerPoint presentations were also prepared, summarizing key implementation progress and issues in preparation for WB supervision missions. The ICR highlights the fact that in addition to routine M&E implementation, the PIU conducted numerous thematic studies to understand better the impacts of project interventions in key areas, including (i) forest cover change, (ii) poverty alleviation impacts of the VLDGs, (iii) effectiveness of capacity building, (iv) community engagement surveys, and (v) a summary report of law enforcement activities. Page 14 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Scaling-Up Participatory Sustainable (P130222) The project contributed to strengthening local M&E capacity, as many of the thematic studies were produced in partnership with local institutions (such as universities and district-level government officials), increasing their research and reporting capacity on the forestry sector. c. M&E Utilization At the national level the project made active use of M&E information to inform project implementation and ensure timely delivery of outputs (ICR, page 27). Although useful thematic studies were undertaken at the central level by NPMO, sub-national implementation entities often found it hard to use the findings to inform project implementation on the ground mainly due to their technical capacity constraints. Data reported on RF indicators was used to adjust targets and level of ambition of all indicators during 1st and 3rd restructuring, thus improving the quality of the RF and its capacity to reflect the project's reality (with adapted and realistic indicators and targets). M&E Quality Rating Substantial 10. Other Issues a. Safeguards Given the nature of project activities and associated impacts on vulnerable and ethnic groups’ livelihoods, the category of Environmental Safeguards is A. Seven Environmental and Social Safeguard Policies were triggered for the project: Environmental Assessment (OP 4.01), Natural Habitats (OP 4.04; triggered due to the presence of designated forests in the project area), Forests (OP 4.36), Pest Management (OP.4.09; due to anticipated pesticide use under the project), Physical Cultural Resources (OP 4.11)36, Indigenous Peoples (OP 4.10; due to project engagement with indigenous communities), and Involuntary Resettlement (OP 4.12). The ICR does not clearly state whether each safeguard policy was complied with. Environmental and social management instruments were updated from the previous project and implemented for SUPSFM, including the Environmental and Social Impact Assessment (ESIA) (disclosed on April 1, 2013), the Environmental Management Plan (EMP) (disclosed on April 1, 2013), and the Community Engagement Framework (disclosed on April 1, 2013), which integrates the Process Framework, the Ethnic Group Development Framework, and the Resettlement Policy Framework into a single document. The environmental and social standards were complied with during implementation, and no significant environmental or social incident was reported. However, external factors and actors continued to pose threats to forest areas supported by the project. These included public and private sector investment projects for, among other things, hydropower, mining, and road development. Encroachment by local communities into project areas for agricultural production and rubber plantation also remained a challenge. A study on forest encroachment conducted toward the end of project implementation identified key drivers of encroachment. It provided recommendations on mitigation measures, including policy development, Page 15 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Scaling-Up Participatory Sustainable (P130222) enhanced community engagement (consultation and Free Prior and Informed Consent), and village livelihood support. To continue to strengthen institutional and technical capacity for implementing and monitoring safeguard compliance, different measures were put into place, such as the need to emphasize vulnerable and ethnic groups, women, and poor households as beneficiaries of the improved agricultural extension services. For these measures to be sustainable and expand across the country, continued government commitment to mobilizing financial and human resources would be required. b. Fiduciary Compliance Financial Management Overall, FM capacity improved under the project at the central level within the Department of Forests (DOF). However, DOF project staff still needed support from FM consultants to perform complex FM tasks, such as preparing interim unaudited Financial Reports (IFRs), annual financial statements, cash flow forecasts, and expenditure statements at the project closure. English language skills also remained a barrier, thus requiring assistance from consultants to facilitate communication between DOF finance personnel and the WB team on disbursement matters. High staff turnover was observed at both central and provincial levels; having access to adequate profiles to replace FM staff was challenging. Some recurring issues were the following: (i) delays in liquidating advances (an issue that remained unsolved until the end of the project); (ii) processing payments to contractors, untimely reporting of expenditures by provinces, and (iii) inadequate supporting documents for expenditures. The submission of IFRs and audit reports was generally timely. The quality of the IFRs steadily improved, and the audit reports were generally timely and received unqualified audit opinions. Procurement The procurement capacity improved under the project within DOF (the improvement is shown in the ISR ratings over time). The WB procurement guidelines were generally complied with. However, the NPMO still needed support from a procurement consultant to handle large and complex cases toward the end of project implementation. One of the procurement processes (purchase of vehicles) took an extremely long time due to the authorities’ indecisiveness regarding the treatment of the taxes involved. c. Unintended impacts (Positive or Negative) Early signs show that the strengthened legal framework facilitated the mobilization of private-sector financing. The project was instrumental in facilitating public-private sectoral stakeholder dialogue. This significantly contributed to the Prime Minister Order 9 issuance in 2018 (Decree No 9/2018), which promotes investment in environmentally and socially sustainable plantation forests, including private sector investment for restoration and reforestation within PFAs. This provision was supplemented later by the 2019 Forestry Law, which authorizes the establishment of private plantations in degraded forest areas in Page 16 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Scaling-Up Participatory Sustainable (P130222) PFAs. This regulatory reform culminated in recent cases of private financing mobilization, including a 60,000ha eucalyptus plantation within a PFA awarded to a private company in 2020 d. Other --- 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment Moderately Outcome Moderately Satisfactory Satisfactory Moderately Bank Performance Moderately Satisfactory Satisfactory Quality of M&E Substantial Substantial Quality of ICR --- Substantial 12. Lessons The following lesson in the ICR was particularly relevant: Village-level support programs are more likely to be successful when backed by strong and effective implementation arrangements. The modality to deliver assistance to the villages must be adapted to the country’s institutional hierarchy and decentralization level. In this project, the highly decentralized design of the village grants and the very diversified portfolio of (on-demand) micro- scale agricultural production systems posed a challenge to track and assess the overall extent of efficacy. The following lesson is drawn by IEG: If a novel and pilot concept is introduced as a key project intervention without enough precision of its goals or sufficient technical preparation, it might cause ambiguity and confusion among stakeholders and reduce their buy-in. In this project, the concept and modus operandi around the Forest Landscape Management Plans weren’t detailed enough, which created a perception of ambiguity. The project was tasked with exploring ways to operationalize FLM through stakeholder consultations. A clearer understanding of what piloting forest landscape management would entail in Lao PDR, backed by dedicated analytical work at the design stage, could have made the project design more realistic. 13. Assessment Recommended? No Page 17 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Scaling-Up Participatory Sustainable (P130222) 14. Comments on Quality of ICR Quality of Evidence. The ICR presents a very thoughtful and complete analysis of evidence. The quality of evidence is substantial, based on the surveys conducted to assess the perception of the quality of the services and processes among beneficiaries and based on the interviews undertaken during the ICR mission. The data from the RF is analyzed very thoroughly to get a deep understanding of the link between the project’s intervention and the impact. Quality of Analysis. There is a clear link between the narrative, the ratings, and the evidence. The ToC is made explicit and depicted graphically in detail. The economic efficiency is thoroughly analyzed, although it is left inconclusive, as it presents the variable of the carbon price to be used as the key issue that will determine the project’s economic efficiency. A minor shortcoming relates to the relevance section, which could have elaborated more on how the project aligned with the Bank’s strategy during implementation (beyond the first CPS for Lao PDR, which only covered the first years) and mentioned the Bank’s experience in the Forestry sector in other countries at the time of appraisal. The ICR is results-oriented. The discussion on achievements adequately balanced between reporting on the achievement of outcome indicators and what the project achieved on the ground (with rigorous criticism of the targets in the RF). It provides a candid, accurate, and substantiated set of observations aligned to the project development objective, never losing sight, in the analysis provided, of the question of real impact. It digs deep to reveal all the project’s shortcomings and sometimes gives options on how things could have been done, giving the reader valuable information to understand the broader context. It is concise and provides comprehensive coverage of the implementation experience. It follows the guidelines, seeks to triangulate data to reach conclusions, and is focused on results. The lessons are clear and well- explained with concrete project examples. The main learnings of the project are identified. a. Quality of ICR Rating Substantial Page 18 of 18