Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Liberia Land Administration Project (P162893) Report Number: ICRR0023683 1. Project Data Project ID Project Name P162893 Liberia Land Administration Project Country Practice Area(Lead) Liberia Urban, Resilience and Land L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IDA-D2370 31-Oct-2022 6,941,337.48 Bank Approval Date Closing Date (Actual) 28-Sep-2017 31-Jan-2023 IBRD/IDA (USD) Grants (USD) Original Commitment 7,000,000.00 0.00 Revised Commitment 7,000,000.00 0.00 Actual 6,941,337.48 0.00 Prepared by Reviewed by ICR Review Coordinator Group Azada Hussaini Vibecke Dixon Avjeet Singh IEGSD (Unit 4) 2. Project Objectives and Components DEVOBJ_TBL a. Objectives The Project Development Objective (PDO) was to strengthen the institutional capacity of the Liberia Land Authority and establish a land administration system, as stated on page 4 in the Legal Agreement (LA) and page 17 in the Project Appraisal Document (PAD). The PDO will be parsed as follows: 1. To strengthen the institutional capacity of the Liberia Land Authority Page 1 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Liberia Land Administration Project (P162893) 2. To establish a land administration system b. Were the project objectives/key associated outcome targets revised during implementation? No c. Will a split evaluation be undertaken? No d. Components The project had 4 components: 1. Support to the Liberia Land Authority (LLA) (US$ $2.29 million, actual: US$ 2.44 million). The component activities were to focus on operationalizing of the LLA and its core responsibilities, including: developing land laws/regulations and administrative procedures; developing business systems, plans and processes, and a strategic investment strategy for financial sustainability; capacity building; communication and awareness raising; acquisition of office equipment, furniture, supplies and vehicles necessary for conducting the work of the LLA and the project, and minor renovation of LLA’s offices. 2. Piloting of Registration of Customary Land (US$1.12 million, Actual: US$ 0.88 million). Initially, this component was titled "Support for inventory and analysis of tribal land certificates" and aimed to support assessments and pilots. It aimed to establish a central database including inventory of tribal land certificates in additional counties, integration of all inventory data, and analysis of the data. At the time of restructuring, activities around inventory of tribal certificates were replaced by a pilot activity to test different approaches and methods for the systematic registration of customary land rights. The restructuring also replaced the relevant indicators in the results framework. 3. Development of a land administration system (US$ 1.57 million, Actual: US$ 1.2 million). This component aimed to support assessments and pilots to develop methods for collecting, recording, and managing land rights information. Proposed activities included development of strategies and options, establishment of a land administration system in Liberia, establishment of a geodetic network as well as the plan for decentralization of land related services. The first restructuring revised the target for “number of decentralized LLA offices established and operational” from 10 to 5 due to high costs of implementation. 4. Project Coordination, Monitoring and Evaluation (US$1.60 million, Actual US$ 2.04 million). This component aimed to support operational costs required for project implementation within the framework of the LLA. The project also planned to undertake a baseline study and social assessment, and the mid-term and end-of-project evaluations. The project was restructured twice in March 2020 and June 2022. The first restructuring revised component 2 activities and related results indicators. It also lowered the target for rehabilitation of local offices from 10 offices to 5 offices. The second restructuring in June 2022 was a no-cost extension of the closing date by three months due to implementation delays in part caused by Covid-19 and delays in the first year of implementation. A split evaluation will not be undertaken since around 30% was disbursed at the time of restructuring when the scope of offices to be rehabilitated was reduced. and it would not have any implications on the outcome ratings. The outcome indicators remained the same. Page 2 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Liberia Land Administration Project (P162893) e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project costs: A total of US$ 7 million was allocated to the project through IDA. This included allocations to the four components as well as a Project Preparation Advance (PPA) of US$ 0.42 million. The total funding envelope as well as component allocations remained the same following the two restructurings in March 2020 and June 2022. The actual project cost at the time of closing was US$ 6.9 million. Financing and borrower contribution: The project was financed through a World Bank IDA grant of US$ 7 million. There were no borrower contributions or co-financing by other partners. Dates: The project was approved on September 28 2017 and became effective on March 22, 2018. The expected closing date was October 31, 2022 which was later extended by three months until January 31, 2023. This extension was requested to allow for the completion of remaining activities. The project MTR took place on April 12, 2021. 3. Relevance of Objectives Rationale Country and sector context. At the time of project appraisal, Liberia faced significant land tenure challenges including contradictory natural resource policies, unclear legal framework, lack of an effective land information system, and low professional and technical capacity in land administration and management. Tenure insecurity had a disproportionate impact on women and smallholder farmers. Alignment with Government strategy. Government of Liberia (GoL) took a few measures to establish a land tenure system. This included: i) establishing the Liberia Land Commission (LLC) in 2009 which ended its term in 2016 and was replaced by a new entity – the Liberia Land Authority (LLA); ii) introducing the Land Rights Policy in 2013 and approving the Land Rights Act in 2018 (a year after the project approval). The PDO was aligned with government efforts as it sought to support the operationalization of the new institution and law. Alignment with Bank strategy. The World Bank Country Partnership Strategy (CPF) FY13-17 highlighted land tenure as a key challenge to sustainable and inclusive growth, as well as a driver of fragility, and a key priority for women's economic development. The CPS included land administration in the list of pipeline operations. The Country Partnership Framework (CPF) FY19-24 retained a focused on land administration and included the project in its list of operations under pillar 3 "Narrowing the Infrastructure Gap to Foster More Equitable Development Nationwide" and objective 8 "More-resilient urban environments and rural communities". The CPF results framework included the following related indicators: (i) Land area under sustainable landscape management practices; (ii) Number of counties with a complete tribal land certificate inventory; (iii) Policies and processes for identification and recording customary land rights are developed and adopted; and (iv) A tribal land certificate inventory database has been established and is being used by the Liberia Land Authority Page 3 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Liberia Land Administration Project (P162893) The PDO was aligned with the CPF objectives and results. A few of the relevant indicators were also reflected in the project’s results framework although at the time of the Country Portfolio Learning Review (PLR) in 2022, the results indicators were not updated. The PLR reflected on how the project outcomes could be sustained through a long-term engagement plan in the sector. Previous World Bank Experience. The World Bank had been involved in the land sector in Liberia since 2009, through a previous operation, the Rehabilitation and Reform of Land Rights and Related Land Matters SPF (P117010). Through this operation, the Bank supported Liberia’s Land Rights Policy, drafting of the Land Rights Law, and developing Land Administration and Urban Land Use Policies. The Bank also supported the capacity building of the Liberia Land Commission and digitized 50,000 land deeds. In terms of comparative advantage, support for developing a land administration system around the new law was aligned with the Bank’s global and country experience in the land sector, however, the PAD did not reflect on whether procuring vehicles and refurbishing offices were aligned with Bank’s comparative advantage as a broker of global knowledge and technical expertise or of the procurement activities could have been completed by the government or other agencies. Level of the PDO. The PDO, “To strengthen the institutional capacity of the Liberia Land Authority and establish a land administration system”, was pitched at a relatively lower level in the results chain to address a development problem. Considering the project builds on a previous operation, and the theory of change discusses higher-level outcomes such as higher land tenure security, and improved land services, the objective could have reflected a higher-level result. However, given the context, as the LLA was a newly established entity and the land law was approved two years into the project, the PDO can be considered realistic in expecting what could be achieved. Rating Relevance TBL Rating High 4. Achievement of Objectives (Efficacy) EFFICACY_TBL OBJECTIVE 1 Objective To strengthen the institutional capacity of the Liberia Land Authority (LLA) Rationale Theory of Change The Liberia Land Authority was established in 2016 and subsumed the functions of three existing entities: the Department of Lands, Surveys, and Cartography under Ministry of Mines and Energy, the Deeds Registry under the Center for National Documents and Records Agency, and the relevant functions from the Ministry of Internal Affairs. The project’s inputs included a range of activities to improve LLA’s capacity such as supporting the LLA to develop business systems, plans, and processes; preparing an investment strategy for financial sustainability; delivering trainings; conducting awareness raising campaigns, procuring vehicles, furniture, and equipment for LLA and local offices; and renovating the new main LLA building and 10 county Page 4 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Liberia Land Administration Project (P162893) offices. Following the March 2020 restructuring, the target for renovation of local offices was reduced from 10 to 5 county offices. These activities were expected to lead to outputs such as government staff trained in land administration and services; regulations and processes developed; investment strategy prepared; office buildings renovated and furnished; logistics equipment made available; county-level LLA offices established. The Results Framework (RF) included an indicator to track technical staff and management hired in LLA departments. Although the project was not involved in the hiring process, it contributed by developing the LLA HR manual, organizational development study, organizational manual, and a board of commissioner policy manual. The activities and outputs above were expected to lead to outcomes such as enhanced institutional capacity of the LLA, improved awareness of the public about the updated legal and institutional arrangements, and functional systems for service delivery. In the long term, the project was expected to contribute to improved land services in Liberia, reduced conflict and disputes, and improved tenure security. A few assumptions around the ToC were not explicitly discussed or addressed, particularly in the areas of public awareness and staff capacity. For example, awareness campaigns can be less impactful due to structural barriers such as literacy rates, internet connectivity, and physical accessibility of the local offices for the rural population. Similarly, it’s unclear whether short-term training to LLA staff would be adequate in increasing staff technical capacity in the LLA and local offices. Issues such as high staff turnover, lack of staff with higher education credentials, and availability of long-term learning or exchange programs may affect capacity-building outcomes. Outputs  The project developed a business plan that identified potential revenue streams and developed 10 procedures (including an organization manual, human resources manual, internal communication strategy, training policy, Board of Commissioners policy manual, performance management system, M&E manual, ICT policy, County Land Office, Standard Operating Procedures). This exceeded the target of one business plan and four administrative procedures.  The RF included an indicator on LLA staff capacity (baseline: 50%, target: 80%). The ICR reports that staff capacity increased to 71%. It is difficult to interpret this achievement as the indicator did not have a clear calculation methodology. The 71% achievement rate also seems optimistic given that only 15% of the 429 trained staff completed training assessments.  429 staff were trained in land-related disciplines, against a target of 400. Of the 429 staff, 99 of the trainees were female. The ICR did not reflect on the quality and outcome of the training.  145 training certificates were issued (no target was established for this indicator).  The RF included an indicator of public awareness increased (baseline: 32%, target: 57%). The calculation methodology around this indicator was unclear. The ICR reports an achievement rate of 51%, what that entails is however unclear. In addition to the achievements recorded in the Results Framework the ICR reported that LLA’s headquarters was relocated to a four-story building that was renovated, furnished, and equipped with solar powered ICT, and five County Land Offices were renovated. Outcomes Page 5 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Liberia Land Administration Project (P162893)  LLA departments are functional, with managers and minimum technical staff in place (Achieved the target of “Yes” from a baseline of “No”). While indicators to measure functionality and efficacy were absent from the Results Framework itself, the project did measure an increase in deeds registration and a significant reduction in service delivery time, both good indicators of functionality and efficacy:  Deeds registration increased on average by 49% during the during the period of implementation  Service delivery time was reduced from 44 days to 10 days The project restructuring introduced changes to the results framework, but this did not change the overall theory of change in a significant manner. The original project document prioritized activities around tribal land inventory to shed light on the extent of tribal lands and the amount of land that needed to be transferred to private land. The restructuring paper and ICR provided sufficient rationale that the inventory was no longer necessary to inform the tribal land formalization process and that the intended results had at least partly been achieved. The rationale around lowering the target for county offices from 10 to 5 seems reasonable given the operating environment and inflation across countries following the Covid-19 pandemic. Overall, the project achieved the expected results by putting in place a physical and organizational structure for the LLA through the renovation of the building, procuring furniture and vehicles, supporting the hiring and training of technical staff, and developing related business procedures. At the same time, as discussed above, the ToC included several assumptions not addressed in the project design, and the results framework included indicators with unclear calculation methods and output-oriented benchmarks that do not capture the quality, functionality, and sustainability of the LLA structures put in place. Despite this, as shown above, the project managed to report on relevant indicators to demonstrate the efficiency of the institutional strengthening. The outcome rating for this objective is thus Substantial with moderate shortcomings. Rating Substantial OBJECTIVE 2 Objective To establish a land administration system Rationale Theory of Change To establish a uniform and decentralized land administration system, the project planned the following activities: to develop new regulations and policies; prepare an inventory of tribal land certificates; carry out assessments and pilots in key areas of the land policy; establish a geodetic network; prepare a plan for decentralization of land services and develop an institutional framework for management of government and Page 6 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Liberia Land Administration Project (P162893) public land. Following the March 2020 restructuring, activities around tribal certificates were replaced with pilots on registration of customary land titles with the aim of developing an upscaling strategy. Expected outputs included a land administration regulation developed; geodetic network developed; and tribal certificate inventory database established. These outputs were expected to enhance information processing systems, establish a functional regulatory framework, and decentralize land management services at the county level. These outputs were in turn expected to contribute to outcomes such as improved land services in Liberia, reduced conflict and disputes, and improved tenure security which were identified as high level outcomes in the results chain. It is noteworthy that the objective formulation is at a lower level of the results chain (at output level: land administration system established). It was assume that once the land administration system was put in place it would automatically be functional and effective. The ToC had some logical gaps. For example, preparing a decentralization plan and renovating 5 out of 15 county offices may not be sufficient for decentralized land services functions. The need for hiring qualified staff in local offices, improving local coordination mechanisms with local authorities, and setting up dispute resolution mechanisms for local conflicts were not discussed or addressed. Similarly, the new indicators on “gender-sensitive upscaling strategy” and gender-disaggregated Customary Land Development and Management Committees (CLDMCs) assumed that the barriers facing women would be resolved through their participation in the CLDMCs. Outputs  Four CLDMCs were established (target fully achieved)  A gender-sensitive upscaling strategy was developed (target achieved)  Four communities were surveyed and registered based on gender-sensitive community by-laws (target achieved)  A network of geodetic and survey control points was established and functional (target achieved)  The system, capacities, and budget for maintenance of geodetic network were not put in place. The system and capacities were developed for the geodetic network, but no budget for maintenance the network was approved. (target partially achieved). The ICR team clarified that the maintenance budget is required only for one aspect of the network i.e., the modern ‘Continuously Operating Reference Station (CORS) which requires regular maintenance and payment of internet services.  Five LLA offices were established and operational (target achieved)  Two national land laws/regulations were adopted (measured without targets). The project did not assign a target for this indicator, and the significance of this progress is therefore unclear. The ICR reported on the following additional outputs that were not captured in the Results Framework, and hence had no target values:  formalization of customary land rights completed in four communities with 29,529 community members  a statutory deed was issued to each of the four communities based on the boundaries that were collectively identified Outcomes Page 7 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Liberia Land Administration Project (P162893)  A policy and processes for the identification and recording of customary land rights were developed and adopted (target achieved)  A system for recording and managing land rights information was established (target achieved) According to the ToC presented in the ICR, the above outcomes were expected to lead to the objective “land administration system established”. The ToC assumed that once the systems and platforms are established, they would be functional and effective. While the RF did not capture the functionality aspects of the system, the project reported on relevant outcome indicators (as outlined under Objective 1 above), demonstrating the functionality of the system. Some evidence on efficacy of objective 2 was presented in the cost-benefit analysis section in terms of number of deeds registered and time savings in the land registration process. At the same time, it is unclear to what extent these changes can be attributed to the project impact alone and to what degree other external factors such as increased demand, and improvements through other agency involvements (Embassy of Sweden, USAID, UN, and FAO) may have contributed to the results as well. The project achieved all output targets, and most outcome targets. While relevant indicators to measure the functionality and effectiveness of the land registration system were lacking in the Results Framework itself, the project did measure both an increase in deeds registration and a significant reduction in service delivery time, both good indicators of functionality and efficacy. The objective “to establish a land registration system” is thus rated Substantial. Rating Substantial OVERALL EFF TBL OBJ_TBL OVERALL EFFICACY Rationale The project achieved most of the planned outputs and the three PDO-level outcome indicators. Although the results framework did not track outcome-level results, the ICR reported an increase in deeds registration and a significant reduction in service delivery time, both good indicators of strengthening institutional capacity. The methodology for assessing some of the underlying output indicators in terms of increased staff capacity and awareness among public of LLA and its mandate was not clear. The overall efficacy is rated as Substantial. Overall Efficacy Rating Substantial 5. Efficiency Page 8 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Liberia Land Administration Project (P162893) Economic and financial efficiency During appraisal, the project carried out three types of cost-benefit analyses: i) cost-benefit analysis based on the assumption that US$2 million would be allocated to LLA; ii) a financial analysis that concluded that LLA could not be self-sustaining in the short to medium term; and iii) a fiscal impact analysis that asserted the LLA could acquire benefits through real estate tax. In the cost-benefit analysis, the estimated economic rate of return (ERR) for the project was 25 percent with a net present value (NPV) estimated at US$72.6 million at appraisal. The project team informed the IEG (on September 12, 2023) that the operating budget was allocated to LLA, and a business plan had been developed with options for revenue generation. These steps increase the likelihood of LLA's financial sustainability over the medium to long term. However, the financial and economic analysis done at appraisal could not be replicated in the ICR given the lack of data on financial revenues and the shift in activities at the time of restructuring (fiscal analysis had been estimated based on tribal land inventory which was dropped). The efficiency analysis at the time of appraisal relied heavily on estimations that were not directly connected with the project's performance and achievement of results indicators. At the time of restructuring the Bank could have updated the financial analysis to incorporate changes to design in terms of their impact on the financial viability and efficiency of the project. The ICR did not present an assessment of unit cost of each result achieved to inform a more practical efficiency analysis as this information was not available. The ICR efficiency analysis used project examples from other countries as a proxy to demonstrate the high returns to project activities such as decentralization in Mongolia's land management system, or public awareness campaigns in Namibia. While these are helpful benchmarks, the exact nature and depth of activities are unclear. For example, the decentralization efforts supported by the project covered five out of 15 counties and four pilot communities, while the geographic coverage of the Mongolia project may not have been comparable. The number of deeds registered increased over the project duration, and the time required to carry out registrations also decreased from 44 days to 10 days, indicating cost savings in the land registration process. The ex-post CBA based on estimated time savings showed a present value of US$4.1 million, which shows economic gains though much lower than the estimated net present value of $72 million as calculated during appraisal. Administrative efficiency The cost of project components remained at the original estimated amounts even though the scope was reduced. Inflation and Covid-19 pandemic led to cost increases. Under component 2, the target for county office renovation was reduced from 10 offices to 5. A few other activities, not reflected in the RF, were also canceled. Some of those activities were taken up by other operations. The project was extended by three months to allow for the completion of the remaining activities. Many of the project activities were implemented by local and international consultants rather than civil servants and LLA staff. The counterfactual scenario would have been to use existing government staff and agencies to implement project activities. This was not feasible since the LLA was a newly established entity without prior experience in Bank operations. Page 9 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Liberia Land Administration Project (P162893) Given the lack of robust economic analysis and shortcomings in the implementation efficiency given the cost overruns, the overall efficiency rating is Modest. Efficiency Rating Modest a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 100.00 Appraisal  25.00  Not Applicable 0 ICR Estimate 0  Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome As the analysis above reflects, the project was relevant and responsive to the country context and aligned with both government and Bank strategies, hence Relevance is rated High. It achieved substantial results toward enhancing the institutional capacity of the LLA and toward establishing a functional land administration system in Liberia, as reflected in the rating of a Substantial Efficacy. The project demonstrated a modestly efficient use of resources given cost overruns during implementation period. The overall Outcome is thus rated Moderately Satisfactory. a. Outcome Rating Moderately Satisfactory 7. Risk to Development Outcome Institutional Risk: The LLA is a newly established entity and there are substantial risks to retention of the capacity developed both in terms of maintenance of goods and supplies (cars, ICT hardware and software, generators, reconstructed buildings) and in terms of soft capacity (management and staff recruitment and training). Staff turnover could jeopardize the training outcomes, lack of commitment or corruption could lead to misuse and deterioration of the platforms put in place. Financial Risk: There is a moderate risk that lack of budget availability may lead to reduced involvement of consultancy firms and local contractors, which the project relied heavily on for implementation. The current level of support from donors may not continue. Without adequate financing, the government would not be Page 10 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Liberia Land Administration Project (P162893) able to allocate operation and maintenance budget to the LLA and the decentralization plan and upscaling strategy may not be fully implemented. Political Risk: Liberia was removed from the World Bank Group's list of Fragile and Conflict-affected States (FCS) in FY22 mainly due to changes in classification thresholds for FCS rather than significant improvements in Liberia’s institutional environment. There was a marginal increase in the Country Policy and Institutional Assessment (CPIA) score driven by improvements in economic policy management and quality of public administration. There continues to be a moderate risk of political changes that may lead to interventions in LLA functions and in the application of regulations/policies supported by the project. Social Risk: Given a history of land disputes, contradictory policies, and inequalities in land ownership in Liberia, there is a substantial risk that the new land law approved by the government and regulations developed through project support are challenged by certain social and interest groups. Although a project GRM was established for project implementation, there is a risk that the national grievance redressal mechanism falls short of ensuring fair and just dispute resolution that advocates for the poor and marginalized communities and individuals. Technical Risk: Although the project delivered over 3,600 training days, continued training and updates are needed to retain and update technical capacity. The geodetic network and M&E systems established also require regular upgrading. Inadequate continued attention to these aspects may affect the level of performance necessary to meet user expectations and technical requirements. 8. Assessment of Bank Performance a. Quality-at-Entry The project followed a previous operation, Rehabilitation and Reform of Land Rights and Related Land Matters SPF (P117010). The project design reflected and incorporated lessons learned from the previous operation, as well as international experience in the land management sector including the need for long- term capacity building and upskilling of land administration staff. The LLA was a new partner and the use of PPA helped to initiate some of the activities. The PAD laid out a clear implementation support plan including an 18-month procurement plan and financial management arrangements. The project assumed the validity of the Land Rights Policy (2013) and anticipated the approval of the Land Rights Act (LRA) which was passed in 2018, a year after project approval. The project’s strategic approach through an incremental implementation plan was pragmatic and helped to focus on LLA capacity building in the first two years and land administration policy support in the last years of implementation following the approval of the land law. The PAD drew on existing research to identify gender gaps in land ownership, particularly under customary norms. While the analysis was reflected in the project document, a clear course of action and outcome-oriented benchmarks were missing on gender. Indicators tracked women’s participation in training programs and used broad descriptions i.e., gender-sensitive upscaling strategy without specifying what outcomes would be achieved for women in terms of asset ownership or voice and agency. The Page 11 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Liberia Land Administration Project (P162893) response to the vulnerability of specific population groups was not clearly articulated in the project design. It was unclear how the project would support these communities beyond the blanket land policy and related procedures. The project identified and discussed two Safeguards triggers and put in place appropriate mitigation measures. However, a functional GRM was missing at the design stage and for several years into implementation. Similarly, stakeholder consultation and communication plans were postponed until the implementation phase despite the availability of a PPA. As part of component 1 activities, the project developed an M&E framework for LLA’s operations. The ICR and PAD did not share details on the project-level M&E arrangements. It was noted that the PIU would lead this function. The results framework lacked clear outcome indicators, as well as definitions and calculation methods for several indicators. A baseline study was not completed at the time of appraisal. Since the PPA could not be used to pay staff time, the project did not hire core implementation team in the PIU needed for procurement and project management functions. The LLA had no prior experience in project implementation with the Bank. These factors contributed to procurement delays in the first two years of implementation. For these reasons, Quality at Entry is thus rated Moderately Satisfactory. Quality-at-Entry Rating Moderately Satisfactory b. Quality of supervision While there were implementation delays in the first two years due to several factors discussed in the ICR including lack of adequate technical staff, the task team took steps to address bottlenecks. Procurement trainings were carried out, a procurement specialist was seconded to the PIU, an international consultant was hired to facilitate key functions, and key agencies were engaged to release timely payments. The Bank team carried out project restructurings at two points to align the project with the new law and adjust targets and implementation timeline in response to emerging challenges. ICR Annex 4 notes that the financial efficiency analysis was not updated at the time of restructuring when the project scope changed, and the appraisal analysis was no longer applicable. Also, the shortcomings in the Results Framework were not adequately addressed, and some indicators lacked a target value until project completion (ICR, para 51). The Bank team included two task team leaders who remained engaged in the project throughout implementation to maintain consistency. The team closely collaborated with the inter-agency platform on land administration and closely coordinated activities. The ICR shared examples of how the Bank's convening and supervision helped to address implementation challenges. However, the ICR notes (para 56) that the central level GRM was established with delay and insufficient staffing. The Quality of Supervision is rated Moderately Satisfactory. Page 12 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Liberia Land Administration Project (P162893) The Overall Bank Performance rating as Moderately Satisfactory given moderate shortcoming at design. Quality of Supervision Rating Moderately Satisfactory Overall Bank Performance Rating Moderately Satisfactory 9. M&E Design, Implementation, & Utilization a. M&E Design While an explicit ToC was not presented in the PAD, the ICR provided this based on the original components and results indicators. As discussed in section 4, the ToC had a logical flow although it relied on a few implicit assumptions. Key aspects of a “functional” land administration system were not measured in the RF. The second outcome "land administration system established" mainly measured outputs but did not track longer-term results on the functionality and effectiveness of the system and the established platforms. During the 2017 restructuring, the indicators introduced at that point were relatively at the output level. There were other shortcomings including a lack of clear methodology for measuring some indicators. The ICR and the PAD did not provide detailed information on M&E design and arrangements. The PIU was expected to lead M&E arrangements, and the project developed an M&E framework for broader LLA activities (going beyond the project implementation arrangements). b. M&E Implementation The PIU was responsible for the project’s M&E functions. The PIU was newly established and the LLA itself was a new institution, therefore M&E capacity had to be developed over time. As the ICR efficiency section reflected, by the time of completion, the client’s MIS/M&E system was not able to respond to information requests from consultants hired to support the ICR completion and cost-benefit analysis leaving an incomplete overview of benefits and results achieved. The M&E manual for LLA was developed shortly before project completion. The last ISR updated all indicators. Some indicators on training and public awareness could not be meaningfully updated during ISRs due to unclear measurement criteria. The project did not carry out a baseline survey during project preparation or completion. As the Bank team clarified on November 18, 2023, the team agreed with the client to collect individual baselines for the indicators on awareness raising and capacity improvements. This data was meant to serve as the baseline for these indicators that required a baseline. However, the collected data was not of the expected quality. The project restructurings did not address shortcomings in results indicators discussed above and these issues affected the quality of results reporting. Page 13 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Liberia Land Administration Project (P162893) c. M&E Utilization The M&E findings were used to provide evidence for some outcomes such as time and efficiency improvements in land registration processes. All indicators in the RF were expected to be updated on a quarterly basis, however several indicators did not report any updates until the last year of project implementation. The LLA’s M&E and MIS capacity was low as it was a newly established entity. There was no evidence that the M&E data was used to inform project management and decision-making. The ICR did not share information on the extent to which M&E findings were shared with stakeholders, and whether they informed subsequent interventions. While there were shortcomings in the Quality of the M&E, the project provided sufficient additional evidence of achievements at both output and outcome levels for a Substantial rating of Efficacy. M&E Quality Rating Modest 10. Other Issues a. Safeguards The project was classified as Category B. The project triggered OP 4.01 (Environmental Assessment) and OP 4.36 (Forests). The extent and impact of building renovations were expected to be minimal. The Bank team informed IEG on September 12, 2023, that ESMPs were prepared for the HQ and county offices, a consultant was hired in the PIU to oversee the activities. Contractors were trained on the code of conduct. Given that renovations were minor no major issues came up. The Forests OP was triggered as a cautionary measure, E&S specialists reviewed the legal documents prepared with project support and did not observe any major issues. The 2017 restructuring included an updated ESMF and stakeholder engagement plan to take into account potential risks of the new activities. The GRM was established late in implementation. As the restructuring paper on June 20, 2022 indicates, field teams were being deployed at that time to carry out the pilot activities. This was around less than a year before project closing, which is quite late for a mechanism that is meant to address grievances arising during project implementation. b. Fiduciary Compliance Financial Management (FM) The project used existing FM arrangements through the Project Financial Management Unit (PFMU) which was managing fiduciary services for World Bank operations at the time. The FM arrangements were therefore considered adequate at the time of approval. There were, however, delays in implementation in part related to late payments. The FM rating was downgraded between October 2018 and April 2019 due to inadequate records in the Systematic Tracking of Exchanges in Procurement (STEP), late appointment of evaluation panels, and delays in approval of workplan and budget. A series of trainings on FM were Page 14 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Liberia Land Administration Project (P162893) delivered which addressed some of the issues. The team informed IEG (on September 12, 2023) that external audits were carried out with unqualified results. The Financial Management was rated as MS at project closing. Procurement The project included an 18-month procurement plan to be managed by the PIU, however, procurement staff needed to be hired and this process was delayed. As with FM, the procurement rating was downgraded in 2018 due to high staff turnover, late appointment of evaluation panels, slow processing of payments, and other delays and shortcomings related to following Bank systems. The ICR states that trainings were delivered, LLA’s procurement manager was seconded to the PIU as a short-term solution, and an international procurement specialist was recruited. The procurement was rated as MS at project closing. c. Unintended impacts (Positive or Negative) None d. Other --- 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment Efficiency was rated modest given lack of adequate economic Outcome Satisfactory Moderately Satisfactory analysis, and shortcomings in implementation efficiency Moderate shortcomings in Bank Performance Satisfactory Moderately Satisfactory quality at entry and supervision Quality of M&E Modest Modest Quality of ICR --- Substantial 12. Lessons The inability to put in place a functioning GRM at the start of implementation could decrease the likelihood of timely citizen engagement around land administration, particularly in an environment with a history of land conflicts. The project GRM did not record any grievances, and this could be because it was established quite late during implementation. Translating a national law to operational policies and regulations without appropriate citizen engagement measures at the design stage could leave concerns unidentified and unaddressed. Page 15 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Liberia Land Administration Project (P162893) Institutional development programs and pilot schemes may not adequately capture results and lessons without a holistic M&E system. The project did not have a robust system to triangulate evidence from various sources to inform management and provide evidence for results. While it is difficult to measure the long-term outcomes of a capacity development effort, the pilot schemes were introduced to reflect lessons and results in more depth. The project ICR reflected on time-saving and an increase in the number of land registrations, but it did not include detailed assessments and did not reflect how the pilot schemes will lead to scale-up at the national level. The following additional lessons are derived from the ICR with some modifications in language: A broadly defined project design and incremental approach allows for flexibility in contexts of legal transition. The ICR also reflects on challenges caused by lack of clarity, including lack of disbursements in the first two years and insufficient remaining time to generate and incorporate lessons learned from pilot activities and policy documents. It could be useful if sufficient time is allocated to adjust to changes and reflect preliminary results. Long-term engagement and a streamlined, coordinated approach among partners enhance the potential for sustainability of the newly established land agencies in fragile contexts. While donor coordination was not visible initially, toward the end of the project, there was a more formal and coordinated approach through an established platform led by the LLA's Project Director. However, it’s unclear if the Bank’s capacity building support to the LLA and local departments will continue through follow-up operations. 13. Assessment Recommended? No 14. Comments on Quality of ICR The ICR provides a good overview of the project, presents a reconstructed ToC, discusses changes through restructurings, and uses multiple sources of information to substantiate the results achieved. The report is concise and consistent in terms of information presented across various sections. It notes areas where data was not available and discusses shortcomings in results indicators. There were some areas where information was missing in the ICR, for example on the number of grievances received and addressed through the GRM, the type of training certification issued by the project, and whether the project was compliant with the anticipated safeguards triggers. The ICR drew on some data beyond the project RF to capture outcomes achieved, for example using LLA data on land registration numbers and timelines. The ICR mentioned pilots carried out in four communities and reflected briefly on findings, but detailed lessons learned were not shared. It was also unclear how the four communities were selected and what their significance was in terms of national scale-up. The ICR is balanced in assessing results and shortcomings. Overall, the quality of the ICR is rated Substantial. a. Quality of ICR Rating Page 16 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Liberia Land Administration Project (P162893) Substantial Page 17 of 17