Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review JO-Badia Ecosystem and Livelihoods (P127861) Report Number : ICRR0021107 1. Project Data Project ID Project Name P127861 JO-Badia Ecosystem and Livelihoods Country Practice Area(Lead) Jordan Environment & Natural Resources L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) TF-12887 30-Jun-2017 14,678,555.00 Bank Approval Date Closing Date (Actual) 27-Nov-2012 30-Jun-2017 IBRD/IDA (USD) Grants (USD) Original Commitment 3,330,555.00 3,330,555.00 Revised Commitment 3,330,555.00 3,330,555.00 Actual 3,330,555.00 3,330,555.00 Prepared by Reviewed by ICR Review Coordinator Group John Redwood Ridley Nelson Christopher David Nelson IEGSD (Unit 4) 2. Project Objectives and Components a. Objectives As stated in both the Project Appraisal Document (PAD) and the legal agreement, the project's development objective was "to support sustainable livelihoods and enhance ecosystem services through participatory approaches in selected areas of the Jordan Badia." b. Were the project objectives/key associated outcome targets revised during implementation? No Page 1 of 14 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review JO-Badia Ecosystem and Livelihoods (P127861) PHEVALUNDERTAKENLBL c. Will a split evaluation be undertaken? No d. Components 1. Community-Centered Ecotourism in the Northern Badia (Appraisal Cost: US$ 2.32 million; Actual Cost: US$ 3.19 million). The project would establish a 250 km ecotourism corridor from west to east connecting the Al Azraq and Shaumari reserves with the proposed Protected Area (PA) of Burqu'. The corridor would be developed around a concept of low-volume, high-value community-centered ecotourism, and provide income-generating opportunities to the At Ruwaished sub-district and nearby communities in the northern Badia, while contributing to conservation of the area's biodiversity. The design, implementation, and management of the corridor would be determined in close consultation with concerned communities and laid out in an Ecotourism Development Plan (ETDP). It was expected that an ecolodge (10-12 room occupancy) and 1-3 campsites would be established along the corridor at sites determined by the ETDP. The component had two subcomponents: (i) Establishment of an Al Azraq/Shamari-Burq' ecotourism corridor; and (ii) Fostering community engagement in the planning, development, and operation of the ecotourism corridor. 2. Adaptive Rangeland Management and Alternative Livelihoods Support in the Southern Badia (Appraisal Cost: US$ 9.08 million; Actual Cost: US$ 9.07 million). The project would finance the establishment and/or rehabilitation of two strategically sited and improved hafirs, two community-based rangeland reserves, and capacity building and support for alternative livelihoods activities. The component had three sub-components: (i) Construction of long-lasting multipurpose water harvesting structures (hafirs); (ii) Establishment and/or rehabilitation of improved rangeland services; and (iii) Maintaining and enhancing livelihoods in targeted communities. 3. Project Management, Monitoring and Evaluation (Appraisal Cost: US$ 3.28 million; Actual Cost: US$ 3.26 million). The Project Management Unit (PMU) in National Center for Agricultural Research and Extension (NCARE) under the Ministry of Agriculture (MOA) would carry out the following activities: (i) manage, monitor, and evaluate implementation of project components; (ii) ensure liaison, communication, collaboration, and joint-problem solving among NARE, the Royal Society for the Conservation of Nature (RSCN), and the Hashemite Fund for the Development of the Jordan Badia (HFDJB); (iii) report implementation progress and financial management (FM) performance to the project Steering Committee and the World Bank; (iv) ensure timely auditing of project accounts; (v) ensure appropriateness of procurement and FM activities as per agreed terms; (vi) ensure adherence to and implementation of approved safeguard instruments; and (vii) act as the Secretariat of the Steering Committee and chair the project's Technical Working Group (TWG). e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project Cost. Actual project costs (US$ 15.52 million) were 6.3 percent higher than estimated at appraisal (US$ 14.6 million). This was entirely due to the increased costs of Component 1 (Community- centered Ecotourism in the Northern Badia) which rose by 37.5 percent due to a doubling of the Grantee's contribution for this component compared with the appraisal estimate (from US$ 850,000 to US$ 1.7 million). There are some minor discrepancies, however, between the cost data in the Financing section Page 2 of 14 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review JO-Badia Ecosystem and Livelihoods (P127861) on page 2 of the ICR and those contained in Annex 3 (Project Cost by Component) on page 40. The project team subsequently confirmed to IEG that the correct actual project costs were US$ 14.68 million total. Financing. The GEF grant of US$ 3.33 million was nearly totally disbursed (99.7 percent) according the Annex 3 but was totally disbursed according to the figures on page 2. Actual counterpart funding was US$ 11.35 million according to the project team. Borrower Contribution. The Grantee's total contribution was 8 percent higher (US$ 900,000) than anticipated at appraisal and was almost entirely allocated to Component 1. This increase was due to additional co-financing secured by RSCN, which reportedly (ICR. para 22) "helped in enhancing the results of the project beyond the expected targets." The ICR did not clarify where these additional resources came from, but in a subsequent communication the project team indicated that these additional resources came mainly from the Khoubara Fund and the Royal Court as cash and were used to scale up the design and construction of the eco-lodge compared to the original plans. In addition, land and buildings were donated by the Government and were used to establish a reception and rest area complex for visitors before they arrive at the eco-lodge. Dates. Even though there was a one year start-up delay after grant effectiveness (March 28, 2013), the project closed as scheduled on June 30, 2017. The initial delay was attributed to the combination of: (i) delays in finalizing the two main project contracts; (ii) internal government discussions on the signing of the contracts; (iii) Cabinet reshuffling in August 2013, which reopened these discussions; and (iv) procedural delays in paying the advances on the contracts to RSCN and HFDJB. 3. Relevance of Objectives Rationale The project's objectives were highly relevant throughout implementation and continue to be so. The World Bank Group's current Country Partnership Strategy for Jordan for the FY2017-2022 period, issued in June 2016, identifies as one of the three objectives under its second pillar ("to improve the equity and quality of service delivery") the "improved management of the water and energy sectors....with a view toward ensuring costs are spread equitably, infrastructure systems are operated sustainably, and resources are managed efficiently" (para.73, pg. 20). In this context, it affirms that "the WBG will support activities which aim at mitigating climate change and protecting the environment" and, more specifically, that the present project, which is specifically mentioned by name, "will continue to help create jobs in one of the poorest poverty pockets through ecotourism and to protect Badia's ecosystems" (para. 76, pg. 21). According to the ICR (paras. 27-28, pg. 11), moreover, the project's objectives remain relevant to the country's recent updated Rangeland Strategy (2013/14) and National Tourism Strategy (2011-2015), which "prioritizes ecotourism in general but more specifically the development of tented camps and eco-lodges, and the promotion of eco- friendly practices within the tourism industry," as well as with Jordan's new 2025 strategy, "which calls for investment in new tourism destinations, careful land use planning and water harvesting in light of the fact that Jordan has limited natural resources and the need for the generation of employment and livelihood opportunities." This strategy is also referred to in the latest CPS (paras. 42-43) as is the "country's situation of extreme water scarcity" (para. 39, pg. 11) and that "climate change impacts on Jordan will be severe." In Page 3 of 14 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review JO-Badia Ecosystem and Livelihoods (P127861) addition, it stated that an analysis of climate and disaster risks "presents opportunities to introduce and double down on efficiency and conservation measures for water and energy sectors; diversification of rural employment; better urbanization and human settlement planning; cleaner energy/tech jobs; as well as climate-smart agriculture." Finally, the CPS indicates (para. 80, pg. 22) that "the WBG will support lagging regions and target improvements in local economic development through a subnational engagement." The project design also remains relevant including both the activities intended to promote eco-tourism and thus local employment and income generation through participatory processes and local infrastructure and other improvements, and to strengthen rangeland management, including through improved water resource management and the rehabilitation of degraded ecosystems, as well as by enhancing local livelihoods in a part of the country in which there appear to be few other opportunities and which appear to be interventions that the Government will seek to continue in the Badia region. The results framework is likewise adequate although it could have contained more specific indicators with respect to the improvement of sustainable livelihoods (see section 9 on Monitoring and Evaluation below). Rating High 4. Achievement of Objectives (Efficacy) PHEFFICACYTBL Objective 1 Objective To support sustainable livelihoods through participatory approaches in selected areas of the Jordan Badia. Rationale Project achievements in relation to this objectives are assessed at both the output and outcome levels. Outputs. • Ecotourism Development Plan (ETDP) developed and approved by the Steering Committee -- this plan, which was approved in Year 3 of project implementation and was prepared with community inputs on the choice of income generating activities -- was the master plan on the basis of which the Ecotourism corridor was established. • First ecotourism destination established. Achieved in Year 4, excursions to Wadi Dahek and Tal Qurma are operational as is the new visitor facility at Shaunari reserve, which is the starting point for the ecotourism corridor, as per the ETDP. These facilities reportedly "draw visitors who contribute to enhancement of local livelihoods through purchase of local goods and services" (ICR, pg. 32). However, no information is provided in the ICR as to the number of visitors to these "facilities" or the amount of additional income acquired by local beneficiaries as a result. • Two Bedouin campsites were purchased with needed equipment, rangers were employed to manage rangeland reserve, and a community liaison officer was employed to assist communities and address their concerns. • Burqu' eco-lodge established. This facility was completed and furnished in June 2017 and reportedly Page 4 of 14 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review JO-Badia Ecosystem and Livelihoods (P127861) began operation in September 2017. However, no data is presented in the ICR with respect to its actual use prior to the ICR submission in December 2017. The ICR does indicate (para 36, pg. 12) that RSCN projections for 2018 estimated that the10-room lodge would accommodate a total of 644 guests and earn 21,584 Jordanian dollars (roughly US$ 24,606), of which the expected community share would be JoD 16,188 (75 percent, or US$ 18,454), although it does not indicate how this percentage was determined. • More than 3,500 members (as opposed to an appraisal target of 1,000) in 28 cooperatives (compared with an appraisal target of 12) have reportedly benefited from GEF-funded "equipment and alternative livelihoods" and HFDJB-funded small grants, including for training. The alternative livelihood activities supported included (ICR, pg. 35) "an electric shop, a telecom shop, a wedding accessories shop, kindergarten, handicrafts, sports ground, and other activities financed through revolving funds established by the project." Other project-funded income generating activities (IGAs), according to the ICR (para 38, pg. 13) were a sewing and embroidery workshop, a yogurt production workshop, a catering kitchen, traditional bread making, cultural events, and tent rentals, although the actual amount of additional income generated is not mentioned. It is not clear how many IGAs were actually funded with project resources, although the theory of change diagram in the ICR (page 7) mentions 28, which implies one per cooperative. • Two sets of training workshops were carried out by RSCN in December 2015 and May 2016 for local cooperatives on: ecotourism services; focusing on project management; proposal writing; computer and accounting skills; strategic planning; leadership; and, sales and marketing to support livelihoods initiatives; were held. These benefitted 135 people (compared with a target of 115). Outcomes • Increased ecotourism-related incomes to communities along the Al Azraq/Shumari.Burqu' corridor -- The target of US$570,000 was largely (94 percent) met as actual increased income at the end of the project was reportedly US$536,545. According to the ICR (pg. 30) this figure reflects income generated "from project activities towards establishing and supporting the corridor including local labor, rent paid locally, material and services purchased locally, salaries paid to local staff and income to project supported IGAs." However, it also stated that "the data provided for this was not verified or audited but drawn from RSCN's record" so it is unclear how accurate and reliable it is. Furthermore, it does not reflect actual incremental income generated by the operation of project-funded facilities such as the new ecotourism destinations and the new eco-lodge, whose implementation was delayed, nor does it clearly indicate the share of this income that was generated by the various new productive and other activities supported. From the standpoint of supporting sustainable livelihoods, this is important, as it appears that some of the reported income sources may only reflect project implementation activities (e.g., rent paid locally, materials and services purchased locally) which, while of relevance in the short-term, would likely not have continued. Unfortunately, the project's ex-post economic analysis does not shed further light in this regard. Conclusion: While there are some questions regarding the actual amount of sustainable new employment and income generated by the project, it succeeded in implementing its planned outputs in support of improved local ecotourism-related livelihood opportunities in the Azraq/Shaumari-Burqu' corridor. There are Page 5 of 14 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review JO-Badia Ecosystem and Livelihoods (P127861) indications, moreover, that these are very likely to increase (e.g., when the eco-lodge comes into full operation) over the medium and longer-term that would contribute to further enhancing local livelihoods. Relevant to sustaining longer-term benefits, participatory approaches were applied for the development and implementation of the ETDP and the associated ecotourism corridor as well as for identification and development of new ecotourism-related and other income generation possibilities in the project area. Despite some shortcomings in the way the outcome in relation to this objective is assessed in the ICR (see Section 5 on Efficiency and Section 9 on Monitoring and Evaluation below), likely project achievements in relation this objective are considered Substantial. Rating Substantial PHREVDELTBL PHEFFICACYTBL Objective 2 Objective To enhance ecosystem services through participatory approaches in selected areas of the Jordan Badia. Rationale As with the previous objective, both outputs and outcomes will be considered in this assessment. Outputs • 3,000 ha of range reserves were rehabilitated, exactly meeting the appraisal target and reportedly protecting natural habitats, through managed rest periods and replanting efforts through seed sourcing and vegetation coverage improvements. As result of project interventions, vegetation cover progressively increased in three (as compared with an appraisal target of two) rangeland reserves in 2014-2016, although this process was interrupted in the Al Hashmieh reserve in 2017 and to a lesser extent in the Al Husseinieh reserve in 2017 due to a severe drought. According to the ICR (pg. 34), "biomass ground surveying and sample collection has been done annually since 2014, showing overall increased values in vegetation terms." However, these increases were not given values in the ICR. • Two hafirs (reservoirs) were constructed in Year 3 with an actual capacity of water retention of 152,000 m3 with up to 12 months of water retention capacity, thereby exceeding the appraisal target of 50,000 to 100,000m3 and 6 months of water retention capacity. As the result of rainwater run-off harvesting, these facilities provide the surrounding communities with more accessible and less costly sources of water supplies for animal watering and fodder production than were available prior to the project. They are especially important in periods of drought, which are expected to become more frequent and intense as the result of global climate change. • Eight rangers "from the community" were trained to provide technical knowledge on healthy rangelands, faunal and floral species and their interactions, grazing carrying capacity and conservation. According to the ICR (para 48, pg. 15) as a consequence of this training "the rangers ascertained rotational grazing patterns for active grazing and rest periods, which are communicated to communities and herders... thus, Page 6 of 14 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review JO-Badia Ecosystem and Livelihoods (P127861) ensuring the sustainability of the rangelands and enhancing this important ecosystem service." However, as suggested above the extent of this sustainability seems to be very sensitive to drought impacts. Outcome The project's outcome indicator in regard to enhanced ecosystem services (i.e., increase in vegetation cover outside of Protected Areas) was the same as one of its intermediate results indicators. Thus, there is no separate indicator of enhanced ecosystem services at the outcome rather than output level. The ICR (paras. 47 and 49, pg. 15) argues that the additional vegetative cover generated by the project "provided species- abundant, species-rich and more secure pastures for livestock" without presenting specific evidence to support this statement. What exactly this means in terms of enhanced ecosystem services, in short, is not altogether clear. The ICR also states that the reserves were adversely affected by the drought in May 2017 and that this was "further compounded by the fact that a declaration to open the rangelands was made by the Minister of Agriculture" in March 2017. However, it does not spell out the implications of this declaration in terms of its potential impact on the project-supported rangelands, including the duration of its applicability (i.e., was it later reversed once the drought situation passed?), thus calling the sustainability of these results into question. In addition, while it does indicate that a participatory approach was followed, stating that "grazing agreements between communities and the HFDJB were signed to ensure the sustainability of rangeland management," how exactly this would be ensured is not specified. The water retention facilities, on the other hand, which also benefited from a community consultation process, have resulted in improved productive use of scare rainwater supplies in parts of the project area on a larger than initially anticipated scale and in a way which is likely to be sustainable. Conclusion: Here too, the project achieved its intended outputs although the links between outputs and the desired outcome could have been more clearly specified in the ICR and there was some lack of quantification, for example on biomass changes. This notwithstanding, and recognizing that it is not easy to measure the outcome of "enhanced ecosystem services," especially at a short time frame after project closing, participatory approaches were followed and project achievements with respect to this objective were Substantial. Rating Substantial PHREVDELTBL PHOVRLEFFRATTBL Rationale Both objectives were rated as Substantial, resulting in an overall Substantial efficacy rating. Overall Efficacy Rating Substantial 5. Efficiency Page 7 of 14 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review JO-Badia Ecosystem and Livelihoods (P127861) The economic analysis in the PAD included an estimate of the revenues expected to be generated by the project-provided Eco-lodge and the monetary value likely to accrue to the users of the hafirs in the form of savings resulting from no longer having to purchase and transport water for long distances (see paras. 31-34, pp.12-13). Annex 6 of the ICR presented an incremental cost analysis as required by the GEF. However, the details of the economic analysis are not presented, nor is an overall rate of return. The ICR followed a similar approach, although it did not replicate the incremental cost analysis. However, what was presented in the ICR was not really an ex-post analysis of costs and benefits as it was based on projected rather than actual observed revenues from the Eco-lodge. However this was understandable since the facility had just been opened. It is also not clear from the ICR whether the figure for the savings by herders as a result of the hafirs was based on ex-ante estimates or actual observed savings. According to the analysis from these projections, the project has an internal rate of return (IRR) of 25 percent. The ICR (para. 62, pp 18-19) concludes that "if project implementation is effective and efficient, project-supported investments will bring substantial economic benefits to the beneficiaries." (emphasis italics IEG). There is therefore a qualification here. On the other hand, the sensitivity analysis does suggest that the projected IRR is fairly robust to an increase in the "baseline" discount rate (from 6% to 8%) and to a decrease in benefits of 10%. The origin of the 6% threshold discount rate is not given a specific reference. The ICR text says "as suggested by the World Bank". In a subsequent communication, the team reported that the reference originated with the undated "Technical Note on Discounting Costs and Benefits in Economic Analysis of World Bank Projects". This note suggests a 5% rate but the project team raised it to 6% based on country real GDP growth projections from the IMF. The team reported that, "Based on the future projections, we found it reasonable to increase the discount rate to 6%." It is not clear to IEG whether there is a consistent and formal process being followed for determining the discount rate. However, in this case, the project IRR projected is well above that threshold. In the subsequent communication to IEG, the project team acknowledged that, due to implementation delays, it had not been possible for the ICR to obtain actual data on income generated by the project facilities. However, it stated that costs and prices had now been updated to reflect the actual numbers and noted, "assumptions have been validated through interviews during project life (while benefits started to accrue) and towards its end." The team also identified the specific assumptions on which the projection of benefits was based, now it seems more firm than at the time of the ICR. These were: (i) the eco-lodge will contain 10 rooms; (ii) the two campsites will contain 6 "rooms" each for 3 people, so higher than the original estimated occupancy; (iii) all facilities will operate 10 months a year; (iv) the eco-lodge will require 11 permanent staff members; (v) each permanent campsite would require 8 staff members, and each wild campsite would require 5 staff members, clarifying that the two actual campsites are "in between wild and permanent" -- more specifically, they are Bedouin camps but erected in set locations; (vi) all facilities will generate revenue from accommodation, food and beverage, and tourism activities; (vii) fixed costs include annual depreciation and staff cost; variable costs include cost per room per night based on double occupancy, with utilities, disposable materials, breakfast, etc; and (viii) revenues are calculated based on a moderate occupancy rate, starting with 20 percent for the eco-lodge and 25 percent for the permanent and wild campsites and with an annual increase of around 10 percent over 5 years. However, the communication also observed that whether this latter assumption will prove to be the case "remains to be seen." IEG finds these assumptions reasonable. In addition, in the further communication the project team noted that the project supported other interventions which also would contribute to ecotourism revenues apart from the eco-lodge. The project supported the ecotourism corridor in other ways through its assistance to the Al Azraq and Shaumari Reserves. This has led to an increase in the number of visitors and associated revenues. More Page 8 of 14 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review JO-Badia Ecosystem and Livelihoods (P127861) specifically, the team stated that "the amount of new ecotourism related income to communities along the Al Azraq/Shaumari-Burqu' corridor at the time of the ICR was confirmed as US$ 536,545 for 2017 and an additional US$ 95,834 have been reported since the ICR. Revenues from the trails are US$ 390,216 for 2017 and US$ 121,133 for the first three months of 2018, which indicates a stronger start for 2018 (already 31% of 2017 revenues)." There are other indications of project efficiency including the fact that there were no cost overruns. Reallocations were made based on cost savings. No extension of the project was required even after the earlier one-year implementation delay. The team credits this positive performance to the continuity of the Bank team that "built trust and relationship with the client which...improved efficiency." On the basis of the additional information and the considerations summarized above, efficiency is rated Substantial. Efficiency Rating Substantial a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 0 Appraisal 0 Not Applicable 88.00 ICR Estimate  25.00 Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome The Relevance of project objectives is rated High and both Efficacy (both objectives) and Efficiency are rated Substantial. The project outcome is therefore rated Satisfactory. a. Outcome Rating Satisfactory 7. Risk to Development Outcome The risk to development outcome over the short and medium-term appears to be modest. The facilities implemented by the project are now in place and fully operational. NCARE will continue to be responsible for operation and maintenance of the hafirs and range reserves at least for three years following project closure and will provide the resources needed to retain the 8 rangers hired under the project for the three reserves Page 9 of 14 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review JO-Badia Ecosystem and Livelihoods (P127861) benefited under it. In addition, DFDJB will support project-created cooperatives for the next ten years. In a subsequent communication with IEG, the project team stated that "now that the eco-lodge is complete, this combined with the facilities along the route will both directly and indirectly results in sustainable income and employment of communities in the sub-district. The team can say this with high confidence given: (i) RSCN's long-term commitment to maintaining the investments and promoting them in the tourism sector, as this is their mandate and part of a long-term plan to declare the area 'protected;' (ii) RSCN is also maintaining and committed to continue and maintain the IGAs that the project established in the At-Ruwaished communities which will continue to generate income and employment as they expand due to increased demand; and (iii) the communities themselves have high interest to continue and engage as there are almost no income sources in this part of the country." Over the longer-term, however, this may be less certain since, as the ICR points out (para 106, pg. 28), "the risk of political instability and security cannot be downplayed as they are very probable in Jordan, especially in the northern and eastern parts of the country that are particularly subject to the inflow of refugees from neighboring Syria and Iraq". 8. Assessment of Bank Performance a. Quality-at-Entry The project was well prepared and was built on lessons learned from other projects. The design was straightforward and adequate and potential environmental, social, and institutional risks were properly identified and mitigated. The results framework/theory of change was also adequate, although the economic analysis could have been presented more clearly and in greater detail, as the ICR itself recognized (para. 98, pg. 26). The initial implementation delay was not the result of poor quality at entry, but rather due to factors on the Grantee's side beyond the Bank's control. Quality-at-Entry Rating Satisfactory b. Quality of supervision The Bank carried out an average of two supervision missions a year, all of which included field visits. The project team consisted of the same TTL, procurement, financial management, operations, and social safeguards specialists at appraisal and throughout implementation, representing an unusually high degree of continuity throughout the project's life, which is commendable. The project team reportedly addressed implementation problems in a timely and effective way and helped to ensure that, despite the one-year start-up delay, the project was able to close as originally scheduled. Quality of Supervision Rating Satisfactory Overall Bank Performance Rating Page 10 of 14 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review JO-Badia Ecosystem and Livelihoods (P127861) Satisfactory 9. M&E Design, Implementation, & Utilization a. M&E Design M&E design was generally adequate and a specialist was hired -- initially on a part-time and later (2014) full- time basis -- as a member of the PMU. The M&E system was expected to track project progress and outcomes with respect the indicators contained in the results framework. However, it could have contained more specific indicators regarding additional employment and income expected to be generated as the result of project- supported ecotourism investments. Monitoring of vegetation cover in the rangeland areas to be rehabilitated and upgraded under the project was also anticipated in connection with component 2,but this refers to a project output rather than outcome (enhanced ecosystem services). b. M&E Implementation The M&E system was operational by October 2013 and was applied both with respect to overall project and component implementation and results and with respect to evolving vegetation cover in the three project- supported reserves, as indicated in Table 1 (Range Reserve Biomass from May 2014 to May 2017) on page 16 of the ICR. Implementation data was provided by all three of the project agencies (NCARE, RSCN, and HFDJB) as well as directly by the PMU. The monitoring of vegetation cover was undertaken by NCARE by using the FAO Collect Earth tool. c. M&E Utilization Twice yearly reports utilized the results framework to assess project implementation progress and were reportedly (ICR, para. 90, pg. 24) submitted in a "timely fashion." Eight such reports were submitted during the life of the project, and monitoring data with respect to the evolution of vegetation cover in the three project-assisted rangeland reserves, as noted above, were used to illustrate project results in this regard in the ICR. The framework was also utilized by the ICR to assess project results. M&E Quality Rating Substantial 10. Other Issues a. Safeguards The project triggered OP 4.01 (Environmental Assessment) under which it was classified in Category B, OP 4.04 Natural Habitats), OP 4.11 (Physical Cultural Resources), and OP 4.12 (Involuntary Resettlement). An Environmental and Social Impact Assessment (ESIA) was carried out and an Environmental and Social Page 11 of 14 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review JO-Badia Ecosystem and Livelihoods (P127861) Management Plan (EMSP) was prepared. The ESIA found that the project's potential negative environmental and social impacts were likely to be minor, temporary, and site-specific, and could be easily mitigated through implementation of the ESMP and the project's Ecotourism Development Plan (ETDP). OP 4.12 was triggered, according to the PAD (para. 47, pg. 15) because "land issues in Jordan are somewhat complex and some small potential exists for social discontent to arise during project implementation." These potential concerns apparently did not arise. According to the ICR, (see para. 94, pg. 25), the ESMP was "taken seriously by the team" and "this contributed significantly to the success of the project and also ensured that the impacts of possible ecological degradation and pollution as well as negative impacts on the community were addressed adequately." However, it did not explicitly state that the ESMP was properly implemented or that all of the triggered Bank environmental and social safeguard policies were fully complied with. In a subsequent communication with IEG, the project team added the following: "we as the Bank team can confirm that safeguards compliance discussed, assessed (and, accordingly recorded in the Aides Memoirs) during each WB mission and closely monitored through field visits. Safeguards instruments were prepared as needed by the Recipient, approved by the Bank, translated into Arabic and disclosed, and the project was in full compliance with WB safeguard policies. As confirmed during field observations, grievances were addressed appropriately by the executing entities: NCARE, RSCN, and HFDJB. One of the missions even included a lead social development specialist from the Bank, independent from the project, who undertook a review of social safeguards and social development aspects including GRM [which IEG assumes refers to the grievance mechanism]. His assessment has recorded these aspects as satisfactory." b. Fiduciary Compliance Procurement. The two large works contracts, with NCARE and RSCN, respectively, were implemented satisfactorily and no problems with procurement were reported in the ICR (see para. 96, pg, 26) Financial Management. A FM action plan was developed, implemented, and supported technically by the Bank. There were no instances of non-compliance. The PMU hired a qualified part-time financial specialist after the original Financial Officer left the project. An external auditor was hired to carry out project audits and issued unqualified audit opinions. Management letters did not report any significant issues with internal controls. c. Unintended impacts (Positive or Negative) The project's recognition of traditional collective rights of Bedouin tribes was considered to be a key contributing factor to project success. Indigenous knowledge was used to select the location of the hafirs and reserves, to select the species to be planted in the reserves, and was taught at the ranger school, thereby reportedly resulting in "regained awareness and appreciation of such knowledge." (ICR, para. 72, pg. 21) In addition to the restoration of the rangelands using native species which had disappeared from the area for some time, several new species not previously been identified in the region emerged and their seeds are also now expected to be utilized for future rehabilitation efforts. The restoration of Qasr (Castle) Burqu' due to project advocacy was also an unexpected positive outcome. Page 12 of 14 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review JO-Badia Ecosystem and Livelihoods (P127861) d. Other Poverty. According to the ICR (paras. 70-71, pg. 20), the project had a positive local poverty reduction impact and it refers to a socio-economic survey undertaken toward the end of the project. As a result, there was evidence of "increased job availability, increased availability of funding for businesses (through several revolving funds), and increasing capacities of local cooperatives to establish and run income generating activities in what was and still is considered a poverty pocket in Jordan." However, no figures are presented, and, thus, it is not possible to assess the actual extent of these benefits. Gender. The project had a PDO indicator target with respect to the gender of its beneficiaries that 51 percent would be female. However, on closing, only 41 percent of actual project beneficiaries were women. This shortfall is attributed (ICR, pg. 31) to "the higher percentage of male beneficiaries from herding-related activities under component 2 and the overall cultural/traditional behavior of targeted communities." This notwithstanding, the ICR (para 67, pg. 20) argues that the 41 percent figure is "significant, considering the highly conservative Bedouin society," suggesting that the original target was probably too optimistic. It also affirmed that local women benefited from capacity building, increased awareness, skills, income, and employment as a result of the project, while some new women-only cooperatives were also established in Al Jafr. In a subsequent communication with IEG, the project team also pointed out that "RSCN pays attention to employ women at various positions in its eco-lodges, for example, Azrak eco-lodge has several workshops where women sell handmade silver jewelry, gift boxes, uniforms for rangers, etc." Institutional Strengthening. The project strengthened the capacity of the implementing agencies involved. RSCN has now been approached by several international and domestic development entities to undertake additional social development activities in northeastern and other parts of the Badia, and NCARE has also increased its ability to implement donor-funded projects and strengthened its ties with local communities in the areas benefited by the hafirs in the southern part of the region, while HFDJB has reportedly gained "prestige" with southern Bedouin communities due to its participation in the project. 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment Outcome Satisfactory Satisfactory --- Bank Performance Satisfactory Satisfactory --- Quality of M&E Substantial Substantial --- Quality of ICR Substantial --- 12. Lessons Lessons that can be drawn from this project, partly taken from the ICR, are: (i) A straightforward and easily implementable project design is especially important in environmentally and culturally sensitive areas. In this case, the project's relatively simple design took these characteristics into account and contained appropriate interventions both in terms of local ecological differences and the Page 13 of 14 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review JO-Badia Ecosystem and Livelihoods (P127861) nomadic nature of part of the beneficiary population. (ii) In project areas where local culture can make full project participation by women problematic, there is a need to carry out thorough assessment of local cultural and other constraints to the achievement of the gender objectives and targets. In this case, the difficulty in fully attaining the gender target as a result of cultural resistance was underestimated and thus, in retrospect, proved to have been overly ambitious. (iii) In a community participation project, it is particularly important for success and sustainability to provide adequate resources up front to build beneficiary confidence and participation in project decision making. In this case, the need to build trust was essential and the project's ability to do so was a contributing factor to its successful, if somewhat delayed, implementation. 13. Assessment Recommended? No 14. Comments on Quality of ICR The quality of the ICR is generally good. However, it could have provided more specific information with respect to the economic analysis used to support the Efficiency section, particularly in the Economic Analysis Annex. This is especially the case with respect to how the benefits were estimated and how likely they are to be achieved since they were based on ex-ante assumptions and projections rather than observed ex-post project results. Also, no reference was given for the origin or derivation of the 6% discount rate. The project team subsequently provided additional and updated information to supplement that contained in the ICR, which IEG has taken into account in its ratings for the project. The Efficacy section would have been stronger had more specific data on outcomes been presented, especially in relation to the ecosystem conservation objective. Some of the lessons learned could have been more informative and comprehensive and also couched more in generic terms for wider applicability. Given the pressure of refugees on the project area from neighboring countries (Syria and Iraq), it is surprising that more attention was not given to how this might have impacted, or how it did impact, project implementation and results, and how it might impact in the future. Alternatively, if this was not an issue nor expected to be, the ICR could have explicitly stated this. a. Quality of ICR Rating Substantial Page 14 of 14