Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review EITAP2 (P160719) Report Number: ICRR0023551 1. Project Data Project ID Project Name P160719 EITAP2 Country Practice Area(Lead) Sierra Leone Energy & Extractives L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IDA-D2680 31-Dec-2022 15,088,551.61 Bank Approval Date Closing Date (Actual) 22-Nov-2017 31-Dec-2022 IBRD/IDA (USD) Grants (USD) Original Commitment 20,000,000.00 0.00 Revised Commitment 15,588,545.22 0.00 Actual 15,102,583.61 0.00 Prepared by Reviewed by ICR Review Coordinator Group Dileep M. Wagle Peter Nigel Freeman Ramachandra Jammi IEGSD (Unit 4) 2. Project Objectives and Components DEVOBJ_TBL a. Objectives The project development objective (PDO), as per the Financing Agreement (Schedule 1, p.4) was “to strengthen governance, knowledge and sustainability of the extractives sector in Sierra Leone”. This was identical to the wording of the PDO in the PAD (p.20). The PDO was not revised. Page 1 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review EITAP2 (P160719) For the purposes of this ICR review, each of the sub-objectives (governance, knowledge and sustainability) will be assessed separately. b. Were the project objectives/key associated outcome targets revised during implementation? No c. Will a split evaluation be undertaken? No d. Components 1. Original components Component 1: Improving Mineral Sector Governance (cost at appraisal: US$2.99 million; actual cost: US$ 3.23 million) financed efforts to improve the overall governance of the sector and removal of key constraints to sector development. The following sub-components were involved: (a) Strengthening extractive sector governance, transparency, accountability and administration; (b) Strengthening legal and regulatory frameworks; (c) Supporting implementation of the recently-updated Minerals Policy and Artisanal Mining Policy; (d) Conservation of resources, environmental monitoring and public participation, and (e) Capacity building of the Petroleum Directorate, through provision of personnel training and hardware for key beneficiary agencies. Component 2: Enhancing Geological Knowledge (cost at appraisal: US$11.53 million; actual cost: US$ 7.78 million) funded coverage of geological maps and acquisition of airborne geophysical data. The expected outcome of this component was to provide basic and reliable geological information necessary to facilitate the promotion of private investments in the mining sector and to support the planning of the socio- economic development of the country. Sub-components included: (a) Airborne geo-physical surveys, (b) Geological mapping and geodata integration, and (c) supervising airborne geophysics campaign for quality control of flights and data. Component 3: Artisanal Mining (cost at appraisal: US$3.18 million; actual cost: US$1.47 million) aimed at developing measures to support and encourage formalization, regulation and improved standards of production of artisanal miners, including the following subcomponents: (i) Strengthening governance in artisanal mining, (ii) Strengthening health, safety and environmental management of the artisanal mining sector, and (iii) Green Gold pilot to support traceability of supply chains and safe channels to achieve fairtrade compliance in the long run. Key activities included the completion of the artisanal mining baseline survey to enable the National Minerals Agency (NMA) increase the availability of data on status and categorization of artisanal mining in the country. As part of building resilient livelihoods in artisanal mining communities, five miner-farmer cooperatives were to be established in these communities. Also, to support capacity building of the Mines Directorate of the NMA in monitoring mining activities, the project provided the Directorate with inspection equipment. Component 4: Project Management (cost at appraisal: US$1.96 million; actual cost: US$ 2.67 million) This component aimed at supporting the Government in managing and coordinating the Project, and in building its procurement, financial management, safeguards management and evaluation capacity through the provision of technical advisory services, training, acquisition of goods and operating costs. Page 2 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review EITAP2 (P160719) Component 5: Contingency for Disaster Risk Management (cost at appraisal: US$ Nil; actual cost: US$ Nil). The component was included to provide support to respond to an Eligible Emergency, including: (a) designing and carrying out relevant sector investments, and (b) providing supervision for carrying out said investments. 2. Changes in components during implementation The project components remained unchanged during implementation. At Mid-Term Review (MTR) in February 2021, the Government initially requested a restructuring and closing date extension, which would have reoriented the support of artisanal mining from formalization and promotion of Green Gold (i.e. fairtrade principles) towards development of good practices for alternative livelihood promotion and rehabilitation of abandoned mine sites. However, the restructuring was aborted when other urgent needs for IDA resources emerged, and an amount of SDR 3.48 million was cancelled prior to completion, so as to reallocate funds within the country portfolio. e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project Cost: The actual project cost was US$15.15 million, compared to the appraisal estimate of US$20 million. Project Financing: The project was financed by an IDA grant (IDA-D2680) of SDR14.2 million (US$20 million equivalent). Borrower/Recipient contribution: The Borrower did not contribute to the project’s financing. Project Dates: The project was approved on November 22, 2017 and became effective on March 01, 2018. The mid-term review was on February 22, 2022. The closing date was December 31, 2022, and the project closed on schedule. 3. Relevance of Objectives Rationale Country and Sector Context, Relevance to Government Strategies: Sierra Leone has a population of about 7 million, nearly 40 percent of whom stay in urban areas. The country’s economy has been driven principally by agriculture and mineral production. Economic outcomes were severely affected by the slump in global commodity prices and the Ebola crisis (2014-15), with growth declining sharply from 4.6 percent in 2014 to a negative 21.1 percent in 2015, putting the Government budget under extreme pressure. Earlier, during 1991-2002, the country had gone through a devastating civil war, fueled in part by conflicting interests in extractive industries (specifically alluvial diamonds). In its aftermath, until 2014, growth in the minerals sector (especially of bauxite, gold and iron ore) resumed until the Ebola crisis and the sharp Page 3 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review EITAP2 (P160719) decline in international prices of iron ore, which resulted in an 84 percent reduction in iron ore production in Sierra Leone. As such, the country had failed to achieve most Millennium Development Goals (MDGs) by 2015, when the project was under preparation. Living standards had remained low, with basic infrastructure lacking. Sierra Leone ranked 179th out of 188 countries on the UN’s Human Development Index in 2016, with widespread poverty, reflected in over half the population living on less than US$1.25 a day. To achieve the Sustainable Development Goals (SDGs) by 2030, it was clear that the Government would need to improve development work across all sectors. However, despite favorable geography and abundant resources, and after hundreds of millions of dollars in soft loans and grants had been provided, Sierra Leone continued to have development outcomes that rated among the worst in the world. Strengthening the management of mineral resources was identified by the 2017 Systematic Country Diagnostic (SCD) as one of four pathways by which Sierra Leone could accelerate growth and reduce poverty. This provided the overarching rationale for development of the Extractive Industries Technical Assistance Project (EITAP) Phase 2, which built on EITAP Phase 1, whose main objective had been to support the Government in its efforts to strengthen institutions and improve the regulatory framework for the extractives sector. However, the SCD also indicated that two foundational requirements – governance and fiscal space – needed to be addressed, without which the situation was unlikely to change. If these requirements could be addressed, the technical interventions supported by the project had the potential to unlock accelerated growth. During the past decade, the country did make commendable efforts to modernize the legislative and institutional framework for the extractive industries. The Ministry of Mines and Mineral Resources (MMMR) was the responsible government agency for policy-making and management of the minerals sector, while the Office of the President created the Petroleum Directorate to regulate oil and natural gas. The National Minerals Agency (NMA) became a semi-autonomous sector regulator, while the Environmental protection Agency (EPA) gained the mandate to regulate and monitor environmental and social matters. However, the SCD pointed out that in practice the content and implementation of the legal and regulatory framework for the extractive industries were rife with inconsistencies, being subject to interpretation, in turn leading to protracted disputes. These disputes highlighted the governance challenges facing the sector, seriously questioning its level of accountability and resulting in a significant loss of potential revenue. The SCD estimated that had the country’s fiscal regime been fully implemented during the boom years (2012-14), revenue collection could have amounted to 11.8 percent of GDP, more than triple the actual revenue collected (3.7 percent). Relevance to the World Bank Group's (WBG’s) Assistance Strategies: The project’s development objectives were aligned with the objectives of inclusive growth and improved governance, as expressed in the Country Assistance Strategy (CAS) of 2010-13. A CAS Progress Review was added in 2012, and explicitly added the extractive sector as a third strategic pillar, reflecting the fact that the mining sector was seen as one of the sources of growth. Additionally, as mentioned earlier, the need to strengthen governance of mineral resources was seen by the SCD as one of four pathways to growth and poverty reduction, and maximization of revenue from the mining sector identified as one of the nine priorities identified by the report. The project’s PDO remained relevant to the priorities of the subsequent Country Partnership Framework (CPF), 2021-26. An important aspect of Objective 1.1 of the CPF (Focus Area 1), concerned with “Strengthening macroeconomic stability, fiscal and financial management”, was the need to manage Page 4 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review EITAP2 (P160719) volatility from the extractives sector, so as “to attract legitimate investment” (CPF, para 64). This called for complete, reliable and readily available geological data, which would strengthen the Government’s ability to negotiate better mining investment agreements and improve investor confidence in the minerals sector. Also part of Focus Area 1, under Objective 1.2 (“Improve government accountability for results in use of public finances”), it was expected that efforts to support governance and transparency in the extractives sector would build upon previous successful engagement on mining policy and legislation governing the sector. As pointed out by the ICR (p.12), the minerals sector had seen steady growth in production volumes and increasing share of minerals exports in total exports, reflected in almost a doubling of annua; production value between 2017 and 2022 – from US$403 million to US$770 million. The growing importance of the extractive industries sector to Government revenue collection pointed to the high relevance of activities aimed at strengthening its governance, to ensure sustainability of revenues. Based on the above, relevance is rated High. Rating Relevance TBL Rating High 4. Achievement of Objectives (Efficacy) EFFICACY_TBL OBJECTIVE 1 Objective Strengthening governance of the extractives sector in Sierra Leone Rationale The theory of change (ToC) for this objective showed a broadly causal link from inputs to outputs and to expected PDO outcomes of this project. The inputs consisted of: (i) Support to Government to strengthen legal and regulatory frameworks and implement the Minerals, and Artisanal Minerals, Policies; (ii) Capacity building support to the Petroleum Directorate; (iii) Completing the coverage of geological maps through airborne geophysical survey, geological mapping and geodata integration; (iv) strengthening governance in artisanal mining, and health, safety and environmental management of the artisanal mining sector; and (v) Support to Green Gold pilot aimed at strengthening traceability of supply chains and sales channels to ensure fairtrade compliance. These activities were expected to lead to the outcomes of strengthened governance, knowledge and sustainability of the extractives sector in Sierra Leone. Key assumptions underlying the ToC included internal macroeconomic conditions and international markets would remain stable; that MMMR would assume the project coordination function, making use of the existing PIU; and that sustained political support to the objective if improving governance, knowledge and sustainability of the mining sector would be available. Outcomes would be measured by a combination of three outcome indicators and a total of 15 intermediate indicators. A key limitation was that the outcome indicator for the objective of improving sector governance – namely, implementation of the Policy Actions Matrix of the Minerals Policy – had a relatively ambiguous achievement target. The target was expressed in Page 5 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review EITAP2 (P160719) terms of percentage of policies implemented, with no details provided on which policies implemented, their relative priority, and how this would be measured in practice. Another limitation of the ToC was that no clarity was provided on whether the various actions supported by the project were of sufficient critical mass to have a significant impact on the development outcomes sought to be achieved. Outputs - Number of core staff of MMMR’s Policy Directorate attending policy training per year was 5, against a target of 15. This was part of the effort to improve overall governance of the extractives sector through strengthening MMMR’s policy-making capacity. It should be mentioned that delays in recruitment of Policy Directorate staff meant that staff could not fully benefit from sustained capacity building support before project closing. - Revision of licensing procedures was not achieved by project closing. Delays in revision and approval of the Mines and Minerals Act of 2009 meant that the NMA could not proceed with the planned revision of licensing procedures prior to project closing. - Revision of the Mines and Minerals Act, which was integral to enhancing the overall governance of the extractives sector, was achieved in 2022, despite delays. - Citizen engagement: National consultation on new policies took place as planned. This refers to consultations with civil society, private sector and other stakeholders as regards the policy development and legal drafting of the Mines and Minerals Development Act (MMDA), 2022, which was a key necessary action. Outcomes: The outcome indicator: Implement the Policy Actions Matrix of the Minerals Policy, which had a target of 30 percent implementation by November 2017 (baseline: 10 percent), was achieved by project closing. As indicated earlier, neither the PAD nor ICR provided clarity on which specific actions were implemented nor of their relative priority. That said, the preparation of the MMDA, approved August 2022, was one of the project’s key objectives. The separation of the Ministry’s policy planning functions from those of sector regulation had led to the creation of NMA in 2013, but a legal and administrative vacuum had been created without a revision of the previous legislation (i.e. the Mines and Minerals Act of 2009), which gave rise to multiple governance challenges. The MMDA of 2022 provided clarity on institutional roles and responsibilities, and clear functions of the political, administrative and regulatory leadership of MMMR and NMA. A new Policy Directorate was established at MMMR, which helped improve mineral sector governance through policy, legal reviews, coordination and research. The project provided logistical support to the Directorate, which was set up with a core complement of 5 staff (including Director). The project supported EITI implementation, which led to successful re-validation of EITI compliance in 2022, with a high score of 87.5 out of 100 (a rating shared only with 6 other African nations). The project also provided institutional capacity building and training to staff at the Petroleum Directorate in such areas as data management and Page 6 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review EITAP2 (P160719) interpretation and business management. The Directorate continues to conduct offshore bidding rounds, though there has yet been little evidence of commercially viable oil and gas reserves. Rating Substantial OBJECTIVE 2 Objective Enhancing geological knowledge of Sierra Leone Rationale Please see the TOC discussion under Objective 1. Outputs: - 31 new or revised maps of 1:100,000 scale were completed by project closing, well in excess of the target of 15. - Only 10 exploration licenses were issued, against a target of 175. The baseline was 146 licenses, mostly inactive or non-performing, which were cancelled – along with a moratorium on issuance of new licenses in 2019 - thereby drastically reducing their number to 10 by closing. - 10 investment promotions were conducted at international conferences, against a target of 8. This was despite the fact that award of new licenses remained suspended – and as such would have generated no new private sector investment response. The efforts would thus have served only to position the country as an open investment destination, awaiting an updating of the Government’s policy regarding the moratorium. - Citizen engagement: Sensitization ahead of flyovers was successfully carried out. Outcomes: - The outcome indicator: Complete a nationwide airborne geophysical survey was carried out to the extent of 100 percent (original baseline was 5 percent). The NMA was the main agency responsible for implementing this objective, and made substantial progress across several large activities, including geoscience data acquisition through a nationwide high resolution airborne magnetic and radiometric geophysical survey, development of new maps and analytical products, and public launch of the Geodata Web Portal in August, 2022 (ICR, p.16). It is assumed that sustainability would be assured for this activity, as NMA has special income sources (from sale of geo data, for instance) that are not controlled by the Ministry of Finance, which would allow it to maintain the equipment, such as servers, printers, plotters, software licenses, etc. While the project was successful in achieving its primary objective of supporting the Government in carrying out the nationwide airborne geophysical survey, thereby enhancing the country’s geological knowledge, a sub-objective, reflected in the intermediate indicators, of attracting private sector investors was derailed to Page 7 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review EITAP2 (P160719) some extent by the Government’s decision to impose a moratorium on issuance of new exploration licenses, resulting in a stoppage of the flow of greenfield investment funds. This decision based to some extent on restrictions imposed by the Covid-19 pandemic, affecting international travel and public assembly (ICR, para 26), followed on steps taken earlier by the Government to cancel inactive and speculative exploration licenses. This had in turn led to a drastic reduction in the number of licenses, such that – at project closing – active licenses were actually lower than the baseline number. However, on the positive side, the Government did resolve key mining sector disputes and renegotiated Mineral Lease Agreements, which have contributed to increasing production and exports. Rating Substantial OBJECTIVE 3 Objective Strengthening sustainability of the extractives sector in Sierra Leone Rationale Please see the TOC discussion under Objective 1. Sustainability of the sector here refers principally to artisanal mining, which is widespread in Sierra Leone, focusing on efforts to increase formalization of miners. According to the PAD (p.14), there were between 300,000 and 400,000 artisanal miners in the country, compared to approximately 14,000 employed in large-scale mines, half of whom were unlicensed and underregulated. Artisanal mining activity was associated with numerous environmental (degradation of land) and health related concerns. During implementation of EITAP1, which preceded this project, diggers of all ages were observed to be undertaking arduous work in unhygienic, and sometimes dangerous, conditions. Due to the informal nature of the artisanal mining sector, numerous operators had been forced into exploitative relationships with middlemen, becoming trapped in a vicious circle of borrowing and indebtedness (PAD, p.18). Outputs: - With a view to the formalization of artisanal miners, 2 cooperatives were established by closing (baseline = 0), against a target of 5. As such, this indicator was only partially achieved. - A baseline study of artisanal mining was completed by closing, as planned. - The target of revising legislation on artisanal mining was achieved. - Efforts to implement the Green Gold pilot, involving (a) training gold miners on cleaner technologies, (b) completing a study on the Green Gold value chain, and (c) developing and implementing the Green Gold branding campaign, were all unrealized. This was because the Green Gold pilot was itself not implemented, as the Government’s priorities shifted away from promotion of artisanal mining towards mine site rehabilitation. Outcomes: Page 8 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review EITAP2 (P160719) - The outcome indicator was related to issuance of new mining licenses. The cumulative target was 1,133 licenses, or 10 percent increase over the baseline of 1,030 licenses issued. The target was exceeded, as a cumulative total of 1,463 licenses was achieved by closing. Based on the failure to implement the Green Gold pilot, and the relatively limited progress made in establishing cooperatives, notwithstanding the satisfactory issuance of mining licenses, the efficacy of this objective is rated Modest. Rating Modest OVERALL EFF TBL OBJ_TBL OVERALL EFFICACY Rationale For Objective 1, the efficacy is rated Substantial. Progress was reported to have been made in implementing the Policy Actions Matrix (though details were not provided), as well as in preparation and passage of the MMDA. Objective 2 was similarly rated Substantial, the project making considerable progress in enhancing geological knowledge acquired via an airborne geophysical survey. Objective 3 was however rated Modest, on account of relatively limited progress having been made towards ensuring sustainability of the artisanal mining sector. Based on this, Overall efficacy is rated Substantial. Overall Efficacy Rating Substantial 5. Efficiency Economic Analysis Considering that the project financed mainly technical assistance activities (promotion of governance, enhanced geological knowledge and sustainability), no economic/cost-benefit analysis could be conducted, either at appraisal, or at closing. As mentioned in the PAD (p.28), though project activities were expected to be much higher than the proposed investment, quantifying these benefits was not a straightforward exercise since the project did not finance any discrete investments generating clear separate cash flows. Operational and Administrative Efficiency From an operational and administrative perspective, the project saw a few activities that incurred cost overruns; most activities however realized cost savings. On the other hand, several project activities scheduled in the work plan did not come to fruition – eventually, only 68 percent of such activities were implemented. Project execution suffered considerable delays on account of the Covid-19 pandemic and disbursements plateaued in Page 9 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review EITAP2 (P160719) 2020. The Government’s decision to abort the planned restructuring which would have reoriented support of artisanal mining from formalization and promotion of Green Gold, led to the cancellation of SDR 3.5 million, for reallocation within the country portfolio. The actual rate of disbursements was only 76 percent by the end of the implementation period – a total of SDR3.48 million being canceled prior to closing. Because of delays in implementation and significant shortfalls in activities executed, overall efficiency is rated Modest. Efficiency Rating Modest a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 0 Appraisal 0  Not Applicable 0 ICR Estimate 0  Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome Based on a High relevance of objectives, Substantial efficacy, and Modest efficiency, the overall outcome is rated Moderately Satisfactory. a. Outcome Rating Moderately Satisfactory 7. Risk to Development Outcome Political: Sierra Leone experienced an 11-year long civil conflict between 1991-2002, and although the country has made commendable progress in rebuilding key institutions and infrastructure since the return to democracy in 1996, the political landscape has been impacted by polarized loyalties and a high degree of socio-political tensions. Elections in 2023/2024 will be a test of the resilience of the reforms instituted so far. Macroeconomic. This risk arises from Sierra Leone’s high dependence on the global economy. The possibility of another slump in global commodity prices, or an economic slowdown in China, could have a Page 10 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review EITAP2 (P160719) noticeable impact on the sector. That said, despite recent global fluctuations and the Covid-19 pandemic, the country’s mining sector has been growing and able to attract investments. Stakeholders: The possibility of some stakeholders withdrawing their commitment to ongoing reforms should no World Bank support be forthcoming is a real risk. However, it is mitigated by the extent of the reforms supported under the project, especially with the establishment of the Policy Directorate of the MMMR, responsible for planning, formulation and stakeholder coordination. Further, a revamped Multi-Stakeholder Group (MSG) under the EITI process is now well-positioned to complement stakeholder coordination and consensus-building for improved sector governance. Overall, the risk that the project’s outcome will not be sustained is Moderate. A key risk arises from the conflicting and vested interests in the mining sector, including a multi-layered system of land ownership. This risk is mitigated by a combination of governance reforms in support of regulatory agencies (especially NMA and EPA) and the transparency mechanisms facilitated by the EITI. The risk that institutional reforms supported by the project might be overturned does exist but can be considered relatively low, given the sector’s importance to the economy and its share of state revenues. 8. Assessment of Bank Performance a. Quality-at-Entry The ICR indicates (p.26) that the Bank team had a clear engagement strategy in place, thanks to the relationships and experience, as well as policy support, accumulated from the EITAP1 project. This provided a head start to the project, which resulted in the three sector policies being adopted within a year of implementation. EITAP1 had supported institutional and administrative reforms to refocus the role of the State and remove inefficient or discretionary regulation. The regulatory framework that had been put into place under that project included strengthening the key institutions MMMR, NMA and the Petroleum Directorate. EITAP2 provided the Government the opportunity to effectively address key strategic bottlenecks. In addition, implementation arrangements were already in place, including a central PIU, at the preparation stage of EITAP2. There were no structural changes required later during implementation. Quality-at-Entry Rating Satisfactory b. Quality of supervision The project was adequately supervised, with twice-yearly missions to Freetown, the capital, being conducted on a regular basis. Performance appears to have been candidly reported in the Implementation Status and Results reports (ISRs). The project was for instance downgraded from Satisfactory to Moderately Unsatisfactory in November 2019, on account of delays in implementation. Virtual missions, in lieu of field-based missions, were scheduled and conducted with the client during the Covid-19 Page 11 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review EITAP2 (P160719) emergency. A Mid-Term Review (MTR) was conducted in February 2021, which resulted in development of an Action Plan, with recommendations to improve implementation. The ICR reports (p.27) that the Bank team and the PIU were able to collaborate effectively during supervision and implementation. The team was able to provide timely technical support, assist in capacity building of project staff and provide constant feedback on strategic issues. The Bank team and PIU were however unable to complete the requested project restructuring in timely fashion, as the intended mine site rehabilitation work proved too extensive and complex to be implemented during the implementation period. Baseline assessments and work plans could not be completed expeditiously enough, and necessary safeguards assessments could not be completed. As a result, the intended work plan and restructuring stalled and was eventually aborted. Quality of Supervision Rating Moderately Satisfactory Overall Bank Performance Rating Moderately Satisfactory 9. M&E Design, Implementation, & Utilization a. M&E Design The project's Results Framework (RF) broadly reflected the logic of project interventions, though there were some deficiencies. PDO indicators were designed to be relevant without being overly prescriptive; however, long-term outcomes were not adequately monitored (ICR, p.24) – such as openness to new investors or facilitation of sustainable artisanal & small-scale mining (ASM). The indicators themselves closely reflected the Theory of Change, one group tracking governance, another geo-science knowledge creation, and a third, designed to monitor formalization and value creation to the benefit of artisanal miners. However, as pointed out in the ToC discussion (Section 4), the key governance outcome indicator suffered from a lack of clarity on exactly which policy actions (from the Policy Actions Matrix) were being measured. b. M&E Implementation The ICR reports (para 53) that the PIU consistently updated the indicator tables, providing details on the status of each activity or report. The data were reflected in the World Bank’s regular ISRs, 10 of which were prepared during implementation of the project. The PIU conducted regular meetings with each implementing agency, whereby they conducted monitoring of activities and indicators. Qualitative information on implementation status of each indicator was also reported. Page 12 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review EITAP2 (P160719) c. M&E Utilization The M&E design allowed the borrower to have a simple and clear platform to use the indicators to track progress. Reports produced were used to ensure that results were disseminated to the appropriate stakeholders. For instance, the public launch of the Geodata Portal ensured that data from the geophysical surveys were made accessible to the broader public and to potential investors. M&E Quality Rating Substantial 10. Other Issues a. Safeguards Environmental Safeguards: The project was assigned a Category B at appraisal, and an Environmental Assessment (OP 4.01) was triggered. As this was a Technical Assistance project without any direct investments in mining activities, nor any civil works or mineral processing, no significant environmental or social impacts were expected to arise. Some indirect environmental and social risks could in principle result from improvements in the policy and regulatory environment, which could in turn lead to future increases in industrial and artisanal mining activities, as well as petroleum exploration. However, the ICR indicates (p.25) that no significant environmental or social safeguards issues were identified during implementation. The ICR reports that the environmental and social safeguards policies were complied with throughout project implementation. The Government prepared the strategic environmental and social assessment (SESA), which was part of the project’s legal requirements, in 2017, to assess the environmental and social risks and impacts of mining regulation and policies. Recommendations contained in the SESA included enhancement of the environmental and social governance of the sector, including revision of the Mines and Minerals Act, 2009, and Environmental Protection Act, 2008, to clarify responsibilities and collaboration between the two agencies. During the project implementation period, 7 of the 11 recommendations made were fully or partially implemented, many of them being intended to be achieved over a long-term horizon. The remaining four recommendations, which saw limited progress, were dependent on revision of the MMDA. According to the ICR (p.26), the SESA was the basis for several activities supporting improvement of the country’s environmental and social management capacity, especially by supporting the EPA and NMA to build on their environmental inspection procedures and sector environmental capacities, to train and equip the mine monitors to effectively monitor and control artisanal mining activities. b. Fiduciary Compliance Financial Management (FM): According to the ICR (p. 26), the project’s financial management was satisfactory and was supported by up-to-date accounts through the implementation period. Financial covenants were fully complied with, the project providing acceptable financial reports, including audits, on a timely basis. The 2021 audit was completed on time and the final audit was expected after ICR completion. Auditors noted that there were no major internal control deficiencies or general accountability issues. That said, the ICR notes that no internal audit report was received by the World Bank in respect of Page 13 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review EITAP2 (P160719) the project’s activities, though the final supervision mission in October 2022 had mentioned to the client that the project’s operations should have been reviewed by the Internal Audit Unit regularly. Procurement: According to the ICR (para 59), procurement was conducted in a satisfactory manner. The PIU was able to successfully complete most of its planned procurements in compliance with World Bnk Procurement Guidelines, despite numerous challenges arising from the impact of the Covid-19 pandemic, which had stalled activities. Several planned procurement activities had to be cancelled on account of the unsuccessful project restructuring. The PIU Procurement Specialist was able to work closely with beneficiary agencies to mitigate impacts of delays in the development of their individual procurement plans. The project also benefited from a decision made early on to frontload some large procurement items, such as the conduct of the nationwide geophysical survey. c. Unintended impacts (Positive or Negative) --- d. Other --- 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment Moderately Outcome Moderately Satisfactory Satisfactory Moderately Bank Performance Moderately Satisfactory Satisfactory IEG disagrees with the ICR's statement that the M&E should be rated Modest because it was unable to achieve modification of Quality of M&E Modest Substantial implementation modalities as activity execution and disbursement rates slowed. This would be beyond the scope of an M&E framework. Quality of ICR --- Modest 12. Lessons The following lessons were mostly derived from the ICR with some modifications by IEG (ICR, pages 28-29): 1. A simple and straightforward project design facilitates successful implementation: In a fragile country such as Sierra Leone, the risk of disruptive external factors is much higher than in Page 14 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review EITAP2 (P160719) other jurisdictions. A simple design and well-prepared procurement and implementation plans cam help minimize the risk of delays or cost overruns. EITAP2’s results were most consequential in the case of the airborne geophysical survey and in the legal reforms initiated earlier under EITAP1, whereas more complex activities, such as the Green Gold pilot failed to get off the ground. 2. In environments susceptible to high turnover of political and administrative personnel, World Bank project and supervision teams need to be able to better manage and respond to changing client expectations during implementation: The MMMR, which hosted the PIU, went through four Ministers and three Permanent Secretaries during the life of the project, the EPA through three Executive Chairpersons, and NMA through two Director-Generals. Each of these had different views and perspectives on project priorities, and in three cases suggested significant shifts in project design to accommodate their priorities – which in some ways significantly delayed or impacted aspects of the operation (the fate of the Green Gold pilot is a case in point). 3. The project highlights the need for more equitable benefit-sharing among mining communities and the population at large: The attempted Green Gold pilot ad mie sector rehabilitation works put the need for cross-sector collaboration into perspective, especially emphasizing the importance of broad-based collaboration with agriculture and environmental agencies. For that reason, it is recommended that future support to the mining sector be closely aligned with other sectors directly involved in community-based development and natural resource management. 13. Assessment Recommended? No 14. Comments on Quality of ICR The ICR is clearly written, concise and generally consistent with guidelines. It provides an adequate description of project components (though some additional detail of the Green Gold pilot and on the reasons for the Government moratorium on new licenses would have been useful), and of the theory of change, although it is not especially critical in assessment of the project’s results indicators. The analysis is evidence-based and candid, overall. When discussing Bank Performance, it could usefully have provided additional information in support of its conclusions, for instance on the extent to which the Bank identified and facilitated preparation and appraisal of the project, as well as the adequacy of supervision resources and inputs. a. Quality of ICR Rating Modest Page 15 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review EITAP2 (P160719) Page 16 of 16