Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Mali - Poverty DPO Series (P157900) Report Number: ICRR0022606 1. Program Information Country Practice Area (Lead) Mali Macroeconomics, Trade and Investment Programmatic DPF Planned Operations Approved Operations 0 0 Operation ID Operation Name P157900 Mali - Poverty DPO Series L/C/TF Number(s) Closing Date (Original) Total Financing (USD) IDA-60650 31-Dec-2017 56,050,190.00 Bank Approval Date Closing Date (Actual) 30-May-2017 31-Dec-2017 IBRD/IDA (USD) Co-financing (USD) Original Commitment 50,000,000.00 0.00 Revised Commitment 50,000,000.00 0.00 Actual 56,050,190.00 0.00 P161619_TBL Country Practice Area (Lead) Mali Poverty and Equity Operation ID Operation Name P161619 Mali Second Poverty DPO ( P161619 ) Page 1 of 24 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Mali - Poverty DPO Series (P157900) L/C/TF Number(s) Closing Date (Original) Total Financing (USD) IDA-60650,IDA-62820,IDA-D3390 30-Jun-2019 57318757.30 Bank Approval Date Closing Date (Actual) 05-Jul-2018 30-Jun-2019 IBRD/IDA (USD) Co-financing (USD) Original Commitment 60,000,000.00 0.00 Revised Commitment 59,934,794.73 0.00 Actual 57,318,757.30 0.00 Prepared by Reviewed by ICR Review Coordinator Group Sara Johansson de Paul Holden Jennifer L. Keller IEGEC Silva 2. Program Objectives and Pillars/Policy Areas EVOBJ_TBL a. Objectives The First Poverty Reduction and Inclusive Growth Support Operation had two stated development objectives: (a) Foster inclusive growth, and (b) Support pro-poor decentralized transfers and social protection. The second operation maintained these development objectives. The broadness of the first development objective, in particular, makes the articulation of a credible results chain (and evaluation of efficacy) impossible. For the purposes of this ICRR, the objectives that can be logically inferred from the content of the DPO series' associated prior actions (and results indicators) are as follows: (1) Foster agricultural productivity (2) Improve rural infrastructure performance and accessibility (3) Improve micro-finance sector performance (4) Improve targeting and efficiency of social protection and government transfers b. Pillars/Policy Areas Page 2 of 24 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Mali - Poverty DPO Series (P157900) The policy pillars were identical with the development objectives. For the purposes of this ICRR, the policy pillars have been aligned to the four objectives above, which more accurately reflect the areas of action. Prior actions associated with the four pillars are as follows: Table 1. Pillars and Prior Actions Pillar 1/Objective 1: Foster agricultural productivity DPO 1 DPO2 PA1b: The Recipient has enhanced the security of land tenure by: (i) adopting two application decrees PA1a: The Recipient has enhanced security of land for the Agricultural Land Act; (ii) appointing members tenure through the promulgation of the Rural Land to at least 300 sub-district agricultural land Act, dated April 11, 2017 commissions; and (iii) adopting through the Council of Ministers the Domains and Land Policy. PA2a: The Recipient has, through the Ministry of PA2b: The Recipient, through the Ministry of Economy and Finance, Ministry of Agriculture and Agriculture has launched the e-voucher scheme for Ministry the distribution of agricultural input subsidies by: (i) of Livestock and Fisheries, introduced an e‐vouc completing the registration of farmers in four sub- her scheme for the distribution of agricultural inp districts (Yanfolila, Koutiala, Niono, Bla); and (ii) uts subsidies, through the adoption of a revised notifying beneficiaries via text messages about the procedures manual, dated October 2016, governing date at which the e-vouchers will be distributed. the modalities of distribution of such subsidies. Pillar2/Objective 2: Improve rural infrastructure performance and accessibility PA3a: The Recipient has, through the Ministry of PA3b: The Recipient, through the Ministry of Digital Economy and Finance, issued a ministerial decision, Economy and Communications, has: (i) included dated October 17, 2016, establishing a commission obligations in terms of wholesale supply and facility in charge of awarding a 4G license to existing sharing in the 4G licenses for the two active mobile operators and, through the Ministry of Digital operators; and (ii) launched the call for proposals for Economy and Communication, issued a ministerial a fourth mobile telecommunication license through order, dated March 30, 2017, creating the an open international bidding process offering commission in charge of awarding a fourth mobile market access conditions in line with international telecommunications license, through an open good practices. international bidding process. PA4a: The Recipient has, through the Office of the Prime Minister, facilitated the extension of the mobile telephone network to isolated areas, making operational the Universal Service Fund Management Agency (AGEFAU), through: PA4b: The Recipient, through the Board of i) Adoption of the decree of appointment of AGEFAU Administrators of the AGEFAU, has adopted Director General; AGEFAU’s work plan for 2018 which includes investments to extend telecom services to previously ii) Creation and provisioning of AGEFAU account; uncovered areas. and iii) Adoption of a decree fixing the percentage rate and modalities of payment of compulsory contributions payable to AGEFAU (the fund). Page 3 of 24 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Mali - Poverty DPO Series (P157900) PA5b: The Recipient, through: (i) the Ministry of Energy and Water, has initiated the reform process aimed at enhancing the efficiency of the electricity sector by endorsing the audit of EDM, New reform area in DPO 2 including the proposed set of reform actions; (ii) EDM, has initiated the process aimed at improving revenue collection by requiring the use of smart meters by high voltage users and large low voltage users. PA6: The Recipient has through the Ministry of Economy and Finance and the Ministry of Equipment, Transport and Regional Integration, issued an inter‐ministerial order, dated October 20, DPO 1 Trigger dropped 2016, fixing the percentage of the Road Maintenance Fund to be allocated to the routine maintenance of priority rural roads. Pillar 3/Objective 3: Improve micro-finance sector performance PA7a: The Recipient, through the Ministry of PA7b: The Recipient, through the Ministry of Economy and Finance, has issued ministerial orders, Economy and Finance, has strengthened financial dated August 31, 2016, withdrawing the licenses of inclusion by: (i) appointing a liquidator for two of the those microfinance institutions that have been microfinance institutions whose licenses have been audited and needed to be liquidated pursuant to its withdrawn; and (ii) completing the 2017 supervision micro finance sector emergency plan, dated March program of the CCS/SFD in accordance with its 2015. annual calendar. Pillar 4/Objective 4: Improve targeting and efficiency of government transfers and social protection PA8a: : The Recipient has, through its Council of Ministers, advanced the decentralization process by adopting a draft 2017 budget edition, dated PA8b: . The Recipient has produced an electronic September 21, 2016, providing for the transfer of cadastral map for Bamako. budgetary resources to local governments pursuant to its competencies and resources transfer plan in favor of local governments. PA9a: The Recipient, through the Ministry for PA9b: The Recipient, through the Ministry for Solidarity, Humanitarian Action and Reconstruction Solidarity, Humanitarian Action has: (i) made of the North, has adopted a unified vision of its social available online a Unified Social Registry, including a protection policy, through: (a) adoption of a mechanism to integrate beneficiaries of various Presidential Decree, dated November 1, 2016, a social safety net programs; and (ii) made pproving the National Social Protection Policy, al JIGISEMEJIRI beneficiaries eligible for free medical ong with its accompanying social protection action care at the CSCOMS by distributing RAMED plans 2016‐2018; (b) adoption of a Presidential insurance cards. Decree, dated September 5, 2016, creating the National Council of Strategic Orientation of Social Protection (“National Council”), and issuance, through MSAH, of a ministerial order, dated March 23, 2016, creating the Unified Social Registry Page 4 of 24 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Mali - Poverty DPO Series (P157900) Steering Committee (“Steering Committee”); and (c) commencement o f operations of both the National Council and th e Steering Committee as evidenced by Minutes of Proceedings dated October 7, 2016 and September 29, 2016. PA10: The Recipient has made provision in its 2017 budget edition, dated September 21, 2016, for budgetary allocations to expand the coverage of Unchanged social safety net programs, including JIGISEMEJIRI and RAMED. c. Comments on Program Cost, Financing and Dates Program Cost: The cost of DPF1 was estimated at US$ 50 million, and DPF2 at US$ 60 million, for total cost for the programmatic series of US$110 million. The actual amount disbursed under DPO1 was US$ 56,050,190, while the actual amount disbursed under DPO2 was US$ 57,318,757, for a total disbursement of US$ 113,368,947. Differences between approved amounts and actual disbursements are attributed to changes in the local currency/US$ exchange rate. Financing: Both DPF1 and DPF2 were financed through IDA grants. Dates: DPF1 was approved on May 30, 2017, became effective on December 31, 2017, and closed as scheduled December 31, 2017. DPF2 was approved on July 5, 2018, became effective on October 10, 2018, and closed as scheduled on June 30, 2019. 3. Relevance of Design a. Relevance of Objectives The two original DOs of supporting pro-poor growth, transfers and social protection would be relevant to the World Bank’s twin goals of ending extreme poverty and boosting shared prosperity in any low-or middle- income country. However, these objectives were set at such a high level that they cannot credibly be included in a results chain evaluated. Using the four derived objectives of the operation (as expressed in section 2a above), PRISGO operations were relevant to the substantial development challenges faced by Mali, in particular achieving peace and security, increasing agricultural productivity, and improving governance and service delivery (Systemic Country Diagnostic [SCD], p.xix, Country Partnership Framework [CPF] p. 2). Taken together, the breadth of the objectives covered (finance, land reform, infrastructure, social protection) may have been too ambitious given limited institutional capacity and the challenging policy environment. 1. Foster agricultural productivity. Raising agricultural productivity central to poverty reduction in Mali, for several reasons. These include on the one hand, high food insecurity and the concentration of poverty in rural households (90 percent) and agricultural sector activity. On the other hand, non- agricultural sectors likely have limited potential for growth and job creation over the short term, given Page 5 of 24 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Mali - Poverty DPO Series (P157900) institutional constraints and the low skill levels of the population. Higher agricultural productivity would underpin food security and earnings increases. 2. Improve rural infrastructure performance and accessibility. Improvements in rural infrastructure will help the rural population access markets (labor, produce) as well as services. Strengthening of infrastructure contributed to poverty reduction between 2000-2010 in Mali (SCD, p.11) and would underpin the development of agricultural value chains necessary to lower rural poverty. Still, prior to the operation, more than one fifth of the population lacked access to telecommunication services because of limited infrastructure and high tariffs. The road density in Mali is among the lowest in Africa, and electricity supply is limited and unreliable. Strengthening rural infrastructure would be necessary to lower transactions costs and increase access to markets (affecting objective 1 above) and access to social services (affecting objective 4 below). 3. Improve financial stability of micro-finance sector. The micro-finance sector has a significant reach in Mali, much beyond that of the formal banking sector, and is serving close to one million people. Access to micro-finance (both credit and savings) can assist poor and rural households with consumption smoothing and provide sources for further investment in farm or small-scale off-farm activities (Objective 1). Confidence in the microfinance sector – and hence its potential for poverty reduction - was curtailed by fragmentation, poor management (the vast majority of institutions were in financial distress), and limited oversight. 4. Improve targeting and efficiency of government transfers and social protection. Dissatisfaction with governance and service delivery is a root cause of continued instability in Mali. Decentralization of government decision making would be an essential component for improving public management and increase the participation and representativeness of the poor, thereby making policies more pro- poor. Previous operations had supported public financial management (PFM) reforms which included strengthening local government’s PFM capacity (Project Document P157900 [PD1], p.2). The high frequency of shocks – weather, security, health, commodity prices – also implies that strengthening social protection systems was and continues to be a priority in Mali. These four objectives fit well with the World Banks CPF 2016-2019, which aimed to address the main drivers of fragility through three pillars: (i) Improve governance, (iii) create economic opportunities and (iii) Build resilience. In particular, PRISGO supported CPF objectives focused on strengthening public resource management, at local levels, improving productive capacity and market integration in rural areas, improve infrastructure and connectivity, and strengthening safety nets. These pillars were in turn aligned with the Government’s main policy framework for 2016-2018, “Cadre Stratégique pour la Relance Economique et le Développement Durable du Mali” (CREDD). More specifically, the PRISGO series supported the first strategic axis, promoting inclusive and sustainable growth, with a focus on promoting agriculture and increasing food security (PRISGO Objective 1), strengthen rural infrastructure (PRISGO Objective 2) and engendering economic diversification (PRISGO objective 3). PRISGO Objective 4 also supported the second and third strategic axis of access to basic social services, especially expanding social protection. b. Relevance of Prior Actions Rationale Page 6 of 24 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Mali - Poverty DPO Series (P157900) Pillar 1/Objective 1. Foster agricultural productivity, especially for poor households. Supported by PA1a, 1b (land reform) and 2a, 2b (e-vouchers). The causal chain for these PAs was that improving the security of land tenure would promote investment in the agricultural sector, and that the e-vouchers would support the increased use of fertilizer and other inputs, thereby increasing crop yields. Rationale. Competing demands for land, including between foreign investors, farmers and pastoralists, was a source of instability and conflict in Mali. A fragmented legal framework and the parallel systems of customary and statutory laws resulted in low security of tenure. This, in turn, affects incentives and opportunities for investment and thereby overall agricultural productivity. The Agricultural Land Act (ALA) constituted a major step towards necessary land reform, as it provided an integrated legal framework which, inter alia, laid out conditions of access to land, identified agricultural land management bodies, and a mechanism for managing and resolving rural land disputes and conflicts. The approval of ALA had been stalled for a decade due to strong vested interests related to the allocation of government-owned land. PA1a. By supporting the promulgation of the Act, PA1a laid the necessary foundations for strengthening land ownership which in turn should help support the use of land for collateral and increase both access to credit and incentives for investment. This would contribute to higher productivity for both small-scale farmers and the large-scale private sector investment needed to increase agricultural exports. On this account, PA1a is rated Satisfactory. PA1b. To support the implementation of ALA, PA1b supported the adoption of two application decrees and the establishment of rural land commissions. These commissions represented a decentralized implementation and oversight of the ALA, especially for resolving land disputes. PA1b focused on appointing members to these commission. This signified an initial but very incomplete step towards operationalizing land commissions for achieving land dispute resolution. Rural land commissions would still require funding essential for their functionality in resolving land conflicts and an effective enforcement mechanism for decisions taken by commissions. The activity does not deal with women's limited access to land under customary law which would be critical to support an inclusive policy and its application. PA1b also supported the adoption of the Domains and Land Policy covering urban land. Although this formed part of the overall land reform, the action was unrelated to the objective of raising agricultural productivity. PA1b is rated Moderately Unsatisfactory. Note: PA2a/b could also be considered under Objective 4, improving the efficiency and targeting of Government transfers. The evaluation below includes both aspects: input subsidies as productivity enhancers, and e-vouchers as efficient transfers. PA2a. Improved access to inputs was part of a broader scheme of interventions to increase agricultural productivity, especially among the poor, who face credit constraints in accessing inputs, know-how and technology (SCD, p. 20, 35). Mali has a large and expensive subsidy scheme, accounting for 2 percent of total Government expenditures. Since its introduction, subsidies had been distributed through paper vouchers, a system which was poorly targeted, ineffective, opaque, and highly resource demanding. An e- vouchers scheme piloted in Northern Mali had proved more successful in reducing costs, shortening implementation time, targeting and tracking poor small holder producers, and increasing transparency in the distribution, and had resulted in reasonably high uptake with 74% of vouchers claimed. PA2a supported the gradual transformation of the paper subsidy scheme into an e-voucher scheme by ensuring that the legal Page 7 of 24 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Mali - Poverty DPO Series (P157900) basis for the implementation of e-vouchers was in place, through the formal adoption of a revised procedures manual for distribution of vouchers. PA2a is rated Moderately Satisfactory. PA2b put in place a phased implementation of the e-vouchers scheme by (i) focusing on 4 sub-districts (ii) identifying the steps needed to be able to deliver the e-voucher, namely registration of farmers and notification via text message about distribution. The first round of e-voucher distribution had been undertaken too late to be relevant to the farmers. These key steps formed a credible contribution to the distribution of the subsidy through e-vouchers. Several important limiting factors had been identified during the pilot that would need to be overcome for effectiveness of the scale-up, including (i) low coverage of mobile networks in poorer rural segments, (ii) high farmer illiteracy (especially women) and (iii) low ICT capacity of implementing agency, the National Council for Agricultural Research (NCRA) (World Bank, 2019: Digitizing Agriculture). PA2b is rated Moderately Satisfactory. PAs under Pillar 1 are rated Moderately Satisfactory. Pillar 2/Objective 2: Improve rural infrastructure performance and accessibility Supported by 3a, 3b (competition in telecommunications market), 4a, 4b (extending telecommunications services in rural areas), 5b (energy sector reform) and 6a (road maintenance). The causal chain underlying these PAs is that rural infrastructure and communications are essential elements in translating improved agricultural productivity into higher revenue from crop sales. Improved telecommunications allow farmers to identify where the most profitable markets are PA3a and 3b sought to support higher competition in the telecommunications sector to achieve broader geographic coverage and more affordable services. Digital access could help connect rural populations to economic opportunities and facilitate the delivery of public services. However, digital inclusion is also strongly linked to the availability of electricity, a reform area which was not included in the first operation. Given the existing situation of a de facto duopoly at the time of PRISGO1, PAs to increase competition in the sector, and introducing measures to increase effectiveness in 4G licenses for existing operators, are important for a more competitive telecommunications sector, which should deliver lower ICT costs for private firms and individuals in areas where service was already available. However, the PAs are essentially procedural and they do not themselves result in any increase in competition. PA3a established the commissions in charge of preparing licenses for 4G and the commission in charge of launching the bidding. PA3a is rated Moderately Unsatisfactory. PA3b was revised in PRISGO2 (from "awards license" to "launches call for license"), to account for slower than expected advancement on the licensing procedures. PA3b focused instead on (i) ensuring that measures to establish a level playing field between incumbents and new entrants were included in the 4G licenses for existing operators and (ii) that the process for an additional service provider license was decisively underway, as indicated by a public call for proposal. The modification reflected a step back in the results chain towards achieving more competition. A3b is rated Moderately Satisfactory. PA4a and 4b aimed to support the extension of telecommunication services to rural areas, which would be an essential part of achieving more inclusive growth. PA4a and b were supported by TA from the WB to AGEFAU to reinforce investments in disadvantaged areas. Lack of extension of services to rural areas reflected limited profitability for mobile operators in these areas. The Government had created a dedicated Universal access fund to ensure access to ICT across its territory, financed through mandatory contributions from telecom operators. However, by 2014, the fund amounted to 23 CFAF billions but had not disbursed Page 8 of 24 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Mali - Poverty DPO Series (P157900) anything, lacking any management or formal structures to do so. PA4a and 4b thus focused on the establishment and operationalization of an agency to manage the fund (AGEFAU), which provided the basis for the implementation of the universal access strategy. PA4a established the basic modalities of the AGEFAU (appointment of GD, establishment of an account, fixing compulsory contributions). Given the low- capacity environment of Mali, AGEFAU would need significant technical support in moving from agency establishment to actual policy implementation. PA4a is rated Moderately Satisfactory. Indeed, due to slower than expected progress, PA4b was revised compared to the original trigger, from "published financial statements detailing the investments to extend telecom services", to "adopting a work plan" with a specified budget for such investments. The modification reflected a significant step back from action planned. This introduced important risk for delays due to incentive and capacity constraints, although TA was provided by the Bank to mitigate these risks. PA4b is rated Moderately Unsatisfactory. PA 5b supported the financial viability of the energy sector. Electricity access constitutes an important constraint to inclusive growth and is very limited in Mali, with only half (48) percent of the population covered (WDI, 2019 data). In February 2017, the Government and the World Bank agreed on a reform strategy to address serious financial imbalances within the energy sector and particularly the electricity company (EDM), which had resulted in serious underperformance and limited private sector investment. The strategy/action plan was supported by the World Bank, through different investment projects. To support policy reforms necessary for the effectiveness of these investments, and on the request of the Government, PA5b was added to PRISGO 2. PA5b required (i) the Government to endorse the audit of the EDM (for which the pre- subsidy deficit ran at 56 billion CFAF in 2016), and (ii) the application of smart meters for the largest electricity consumers, to improve billing efficiency. A financially stable energy sector is a basic condition for sustainable and predictable energy supply. Given the deep-seated problems in the electricity sector, mainly related to poor governance and vested interests, government endorsement of audit (PA5bi) would not meaningfully contribute to necessary reform measures. PA5b (ii), more directly supported improving the financial situation of EDM but would need to be accompanied by an action plan for installing smart meters. PA5b is rated Moderately Unsatisfactory. PA6a supported improved road maintenance. A better and more extensive road network is essential to improve access to markets and basic public services for rural households. The poor state of rural roads (5 percent in "good condition" prior to the operation [PD1, p. 1]) was partly due to inadequate maintenance. The Road Authority (RA) was responsible for maintenance work, which was financed through the Road Maintenance Fund (RMF). However, there was no systematic process for maintenance and the ad-hoc approach jeopardized road sustainability. It was thus relevant that PA6a focus on improving resource allocation by requiring that the Government earmark a share of the RMF for rural roads. This was followed by an indicative trigger (for PRISGO2) for public financial statements confirming these RMF allocations. While earmarking of resources provided a potential source of financing for rural road maintenance, actions did not specify how the RMF would spend the money and strengthen its operations. PA6a is rated Moderately Unsatisfactory. PAs under Pillar 2 are rated Moderately Unsatisfactory. Pillar 3/Objective 3. Improving financial stability of Micro-Finance Sector. Page 9 of 24 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Mali - Poverty DPO Series (P157900) Supported by PA7a and PA7b (liquidation of insolvent institutions). The causal chain underlying these prior actions was that rural microfinance institutions were important in providing financial services that allowed rural residents to accumulate savings and borrow for investment in agricultural activities. The PRISGO series supported the Government's Emergency Plan for the Micro Finance sector, approved in 2015. As noted above, the micro-finance sector is important in serving rural clients. However, the high prevalence of underperforming institutions had lowered confidence in the micro-finance sector and reduced households' willingness to use financial services, thus reducing financial inclusion (PD1, p.15). A critical problem was that the national supervisory body (CCS‐SFD) lacked capacity and independence to fulfil its monitoring mandate to ensure viable and reliable institutions. Improving oversight, closing down non-viable institutions and establishing mechanisms for compensating depositors, were therefore central to rebuilding confidence in the microfinance sector. PA7a supported the withdrawal of licenses of non-performing institutions, representing the first decisive step in the essential liquidation process to improve micro-finance sector stability. PA7a is rated Moderately Satisfactory. PA7b supported (i) the actual liquidation of at least two institutions and (ii) the CCS-SFD supervision program of the sector to identify performance/viability issues. Both PA7a and 7b thus provided essential and credible steps towards increasing confidence in the micro-finance sector. However, background work by the World Bank (in 2015) suggested that public confidence in the sector hinged on liquidating (and compensating the depositors of) a few specific larger institutions that dominated the market and were in severe distress. PA7b specified that at least two institutions needed liquidation, but did not specify criteria (e.g., by market share) to ensure that the most problematic institutions would be liquidated. PA7b is rated Moderately Satisfactory. PAs under Pillar 3 are rated Moderately Satisfactory. Pillar 4/Objective 4. Improve targeting and efficiency of government transfers and social protection. Supported by PA8a and PA8b (budget decentralization), PA9a and PA9b (Social Protection Policy including beneficiary registry), and PA10 (Safety net programs). The causal chain underlying these prior actions was that government transfers were an important instrument to reduce the incidence of poverty in rural areas. By improving the ability of the program to more precisely identify and target recipients, the PAs contributed to alleviating rural poverty. PA8a supported the decentralization objective through earmarked resources in the 2017 draft budget for local government transfers, in concordance with the 2015 Peace Agreement under which the Government committed to allocating 30 percent of its budget to local authorities [PD1, p. 17]. PA8a thus constituted a credible step towards decentralization by focusing directly on achieving resource transfers towards local authorities. To strengthen the pro-poor approach (which was measured in the related RI), PA8a could have specified that decentralization should include the poorest localities. PA8a is rated Satisfactory. Further, PA8b focused on improving tax collection at local levels, which was also essential to support higher independence of local governments in resource management and their ability to execute pro-poor spending. The development of a cadastral map, piloted for Bamako but with the objective of extending nationwide, constituted a basis for improved property tax collection. However, by focusing on Bamako, it would not - over the short period of implementation - reach other localities, least of all the poorest localities. PA8b is rated Moderately Satisfactory. PA9a supported a more inclusive social protection policy which focused on consolidation, scale-up and targeting of SP, and a move away from food support to more effective transfers. With the launch of the Page 10 of 24 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Mali - Poverty DPO Series (P157900) JIGISEMEJIRI, a cash transfer program, in 2014, the Government aimed to increase the effectiveness of SP. Given the high fragmentation, weak coordination, and poor targeting of existing safety nets, supporting the new SP policy was an important step towards more pro-poor policies. PA9a also laid out critical and sequential actions (i, approving the Social Protection Policy, ii, creating a National Council of Strategic Orientation of Social Protection, iii, creating a unified beneficiary registry) from policy approval to operationalization in a logical results chain. PA9a is rated Satisfactory. PA9b supported the publication of the Unified Social Registry online where prospective beneficiaries can register (PA9bi) and linking beneficiaries of JIGISEMEJIRI to free health care (PA9bii). The Unified Social Registry (PA9bi) was the main mechanism for improving coordination, targeting and transparency of transfers, and its operationalization hence essential for implementing agencies as well as beneficiaries. The linking of cash transfer beneficiaries and health care provision (PA9bii) constituted a pilot for how the registry could be used to create synergies and expand coverage of services to the poor. PA9b is rated Satisfactory. PA10a was linked to and supported PA9a, and focused on the expansion of the coverage of Social Safety Nets programs, including prioritization of the Cash transfer program and the Health Care program (RAMED). By requiring earmarked budgetary resources, PA10a thus underpinned expanded coverage of social safety net programs. PA10a is rated Satisfactory. PAs under Pillar 4 are rated Satisfactory. With the exception infrastructure, pillars are rated moderately satisfactory (pillar 1, agricultural productivity and pillar 3, microfinance stability) or satisfactory (pillar 4, social protection) on the relevance of the Prior Actions. Most PAs provide logical steps in a results chain towards the intended objectives. However, several PAs, especially under pillar 2, were focused on adoption and promulgation of strategies, plans and processes, which were not sufficiently action oriented and could not therefore be expected to make measurable progress on the achievement of objectives Rating Moderately Satisfactory 4. Relevance of Results Indicators Rationale Table 2. Results Indicators, PAs, baseline-targets, status Status Baseline (2015) RI (number and description) Associated PA (Dec 2019 unless Target (2018/2019) otherwise indicated Pillar 1: Foster Agricultural Productivity RI1: Functionality of land Baseline: 0 commissions (percent of land PA1b 65% commissions which hold Target (June 2019): 80% Page 11 of 24 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Mali - Poverty DPO Series (P157900) meetings at least once every three months) RI2. Agricultural subsidy 72,790 (2019) beneficiaries who receive their Baseline: 0 PA2b subsidies through e-vouchers Target (June 2019): >100,000 51,634 (end 2020) (as opposed to paper vouchers) Pillar 2: Improve Rural Infrastructure Performance and Accessibility Baseline (2015): CFAF 108 RI3: Average in tariff per PA3a CFAF 78.5 (2018) minute for personal domestic Target (2018): CFAF 90 calls PA3b CFAF 79.5 (2019) Target (2019): CFAF 90 Baseline (2015): 0 RI4: Number of localities PA4a 31 (2019) benefiting from extension Target (2018): 16 projects of telecom services PA4b 60 (2020) Target (June 2019): 16 RI5: Number of large low voltage Baseline (2015): 0 60 (refers to number of and high voltage users with PA5 smart meters smart meters Target (June 2019): 6,000 installed). RI_: Execution rate of Baseline (2015): 15% maintenance needs for priority PA6 Dropped rural roads Target (2018): 40% Pillar 3: Improve Financial stability of Microfinance Sector Baseline (2014): 8.8% PA7a RI7: Non-performing loans ratio Target (2018): 4.5% 7.1% of all microfinance institutions PA7b Target (June 2019): 4.5% Pillar 4: Improve targeting and efficiency of social protection and government transfers RI8: Percentage of budgetary Baseline (2014): 15% resources allocated to the three PA8a 20% (2019) poorest regions in total Target (2018): 20% budgetary resources transferred PA8b 19% (2020) to regions. Target (June 2019): 25% RI9: Number of households PA9a Baseline (2015): 65,000 receiving cash transfers (direct 79,168 (2019) transfers, high-labor intensive PA9b Target (2018): 100,000 work, and income-generating 91,630 (2020) activities) PA10 Target (June 2019): 150,000 Pillar 1/Objective 1. Foster agricultural productivity, especially for poor households. Progress measured by RI1-RI2. RI1. RI1 was an attempt to measure the operationalization of land commissions, and as such, the implementation of the ALA with the ultimate purpose of increasing land tenure security. Active Land Page 12 of 24 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Mali - Poverty DPO Series (P157900) commissions holding regular meetings however does not imply anything about the outcomes of the meetings and does not convincingly measure any impact for the intended beneficiaries (smallholders), whether through increased land security, agricultural productivity increases or any outcomes in between. The ICR notes that the IBM phone survey could have been leveraged to collect more pertinent information which could have motivated a more precise indicator. RI1 may also be difficult to monitor reliably. R1 is rated Moderately Unsatisfactory. RI2. RI2 (number of recipients of e-vouchers) measured the expansion of the e-voucher system in number of beneficiaries. This indicates whether e-voucher expansion took place, but does not capture whether the e- vouchers were better targeted or whether they were used to finance productivity enhancing measures or activities. Data on use of e-vouchers for fertilizers and extension services by beneficiaries belonging to poorer households (or communities) would have contributed to an understanding of the project’s efficacy. RI2 is rated Moderately Unsatisfactory. Relevance of RIs under Pillar 1 is rated Moderately Unsatisfactory. Pillar2/Objective 2. Improve rural infrastructure performance and accessibility. Progress measured by RI3-RI5. RI3 provided a succinct measure of the objective of PA3a and b, namely a lower tariff for personal domestic calls. RI3 is an indicator of progress towards making infrastructure services more accessible, especially to more vulnerable populations, by making them affordable. However, it does not capture the sustainability of the lower tariffs or the extent to which lower tariffs were the result of increased competition in the sector (which was the intent of the associated prior actions. It is a fully verifiable indicator. RI3 is rated Moderately Satisfactory. RI4 measured the number of localities that had received telecom services through the AGEFAU. The indicator reflected fully the ultimate objective that PA4a and b contributed to by supporting the operationalizing of the AGEFAU and the proof of investment in additional localities. It had a clear baseline and should be verifiable from the government. RI4 provides no indication of the characteristics of the beneficiary localities such as rural location or vulnerable communities. RI4 is rated Moderately Satisfactory. RI5, the installation of smart meters reflects momentum towards improving the financial viability of the energy company. Moreover, although RI5 had a clear baseline and would be verifiable through EDM, it did not specify the percentage of electricity consumption covered by smart meters or what “large” users implies). The indicator had a clear baseline and should be verifiable during and at the end of the operation, through EDM's accounts and monitoring documentation. RI5 is rated Moderately Satisfactory. RIs under Pillar 2 are rated Moderately Satisfactory. Pillar3/Objective 3. Improve financial stability of microfinance sector. Progress measured by RI7. RI7, by measuring the ratio of non-performing loans, constituted a relevant indicator for the policy objective of supporting a more sustainable microfinance sector, and follows logically as an outcome from the two prior actions focusing on liquidation of nonviable institutions. As pointed out in the ICR, an indicator focusing on uptake of (micro-)financial services might have been even more relevant for the financial inclusion objective. RI7 had a clear baseline (8.8 percent of all loans in 2014) and target (4.5 percent in 2018) and would be verified through CCS-SFD monitoring data which the PRISGO also supported by supporting supervision activities. RI7 is rated Satisfactory. Page 13 of 24 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Mali - Poverty DPO Series (P157900) RI under Pillar 3 is rated Satisfactory Pillar4/Objective 4. Improve targeting and efficiency of government transfers and social protection: Progress measured by RI8-RI9. RI8, by measuring the share of budgetary transfers to the poorest as share of all regional transfers, captured whether budgetary decentralization process was pro-poor which was part of the underlying reform logic, (although the pro-poor focus was not actually specified in the PA). However, RI8 did not measure an increase in the decentralization of resources, only whether they were pro-poor (in relative terms). The share allocated to poor governments could improve while transfers to poor localities fell in absolute terms, which would go against the overall objective. RI8 is rated Moderately Satisfactory. The RI related to PA8b was dropped in PRISGO2 and not replaced by an adequate indicator to capture whether local governments' own resources increased or not, leaving progress on strengthening tax collection at local level unmonitored. RI9 measured the increased coverage of the cash transfer program, but not whether it was pro-poor. Given that the targeting of these cash transfers was judged relatively successful at the outset (PD, p. 22), RI9 likely measures a pro-poor expansion. It does not measure other aspects e.g. expansion on health care access for cash transfer beneficiaries (PA9b), which would be an important indicator of improved coordination across sectors, efficiency, and pro-poor policies. RI9 is rated Moderately Unsatisfactory. There is no results indicator to measure progress on budgetary resources allocated to safety nets (PA10a). RIs under Pillar 4 are rated Moderately Satisfactory. Table 3. Indicators and efficacy Overall Efficacy Pillar Efficacy Results indicator Relevance rating Efficacy Rating Rating Rating Moderately Unsatisfactory Pillar 1/Objective 1: Foster agricultural productivity, especially for poor households Moderately Moderately RI1 Modest Unsatisfactory Satisfactory Moderately RI2 Substantial Unsatisfactory Pillar 2/Objective 2: Improve rural infrastructure performance and accessibility Moderately Moderately RI3 High Satisfactory Satisfactory Moderately RI4 Substantial Satisfactory Moderately RI5 Negligible Satisfactory RI_ Page 14 of 24 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Mali - Poverty DPO Series (P157900) Pillar 3: Improve microfinance sector performance Highly RI7 Satisfactory Negligible Unsatisfactory Pillar 4: Improve targeting and efficiency of government transfers and social protection Moderately Moderately RI8 Modest Satisfactory Unsatisfactory Moderately RI9 Modest Unsatisfactory Rating Moderately Unsatisfactory 5. Achievement of Objectives (Efficacy) EFFICACY_TBL OBJECTIVE 1 Objective Foster agricultural productivity, especially for poor households PA1a, 1b, 2a, 2b, RI1, RI2. Rationale By 2019, 65% of land commissions met regularly. By 2020, the share had increased to 70 percent, still short of the 80 percent target. Given the limited relevance of PA1b and R1 for increasing agricultural productivity, and lack of information on progress on intermediate indicators such as agricultural credit, or land security, the Efficacy of RI1 is rated Modest. By 2017 (PRISGO1), the e-voucher system had been put in place reaching 17500 beneficiary households. In 2019, the number had quadrupled to over 70,000 beneficiaries, thus reaching round 70 percent of the target of at least 100,000. At the same time, the ICR reports that the scheme was highly successful in introducing targeting mechanisms and increasing transparency, and that it successfully reached the poorest farmers as well as female farmers (ICR, p. 29), although R2 does not measure targeting aspects. Participation in the e- voucher scheme fell again in 2020 (to around 50,000), due to security concerns, but the scheme remains in place and is being expanded geographically (ICR, p. 29). The efficacy of RI2 is rated Substantial. The overall efficacy of pillar 1 under Objective 1 is rated Moderately Satisfactory. Progress on land reform as well as the basis for a better targeted input distribution moved ahead, albeit slower than foreseen, with functional commissions and a successive expansion of e-vouchers. However, neither of these achievements necessarily reflect genuine improvement in either input use, land security or overall agricultural productivity. Rating Page 15 of 24 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Mali - Poverty DPO Series (P157900) Moderately Satisfactory OBJECTIVE 2 Objective Improve rural infrastructure performance and accessibility PA3a, 3b, 4a, 4b, 5b, RI2, RI4, RI5. Rationale RI3. A provisional agreement with a fourth telecom service provider was made by end-2019, but the fourth license had not been awarded at the time of writing of the ICR. However, the expected entry of a new competitor appears to have led to incumbents lowering tariffs and increasing coverage. By 2019, the tariff had fallen to 78.5 CFA per minute, down from 108 in 2017 and considerably further than the target of 90 CFA/min. The indicator increased marginally in 2020, to 79.5. The efficacy of RI3 is rated High. RI4. The start-up of AGEFAU was slow, and by mid-June 2017, no projects had been undertaken. Once operational [in 2018? Check with TTL] disbursement towards investment projects took off, however. The number of localities benefitting from extension services reached 31 by mid-June 2019, at almost 200 percent of the target (16). Progress continued and by mid-June 2020, 60 localities had received extension services projects. There is no information on where these localities were based or their poverty levels, however, and so synergies with e-vouchers or financial inclusion reforms could not be verified. The efficacy of RI4 is rated Substantial Reforms in the electricity sectors did not move ahead as planned, possibly reflecting an unrealistic target in the context of a major sector overhaul involving significant vested interests and governance problems. By 2019, only 60 smart meters had been installed, amounting to one percent of the target of 6,000. Progress stalled completely during the following year, during which no new meters were put in place. The efficacy of RI5 is rated Negligible. The overall efficacy of policy pillar 2 is rated Moderately Satisfactory. This rating represents two very different outcomes: on the one hand, significant success in lowering user tariffs and increasing the expansion of telecom services. However, there was no progress at all in the electricity sector, which was in a major crisis, with significant implications for other areas of reform, including ICT expansion. Though not rated here as the area was dropped, there was no progress on rural roads maintenance which would also be a critical component for poverty targeted infrastructure improvements. Rating Moderately Satisfactory OBJECTIVE 3 Objective Page 16 of 24 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Mali - Poverty DPO Series (P157900) Improve microfinance sector performance PA7a, 7b, R7. Rationale RI7. The microfinance sector performance did not sustainably improve during the operation. The ICR indicates that the ratio of NPLs did improve by 2017, to 4.9 percent, but worsened again in the subsequent period and reached 7.1 percent in 2019, representing very limited progress overall and a movement in the wrong direction. The ICR does not provide information on intermediate activities such as the supervision or liquidation processes, or on overall performance of the micro-finance sector in terms of outreach. According to WDI data, access to finance (account or mobile) by the poorest 40 percent increased from 15.4 in 2014 percent to 31.3 percent in 2017, but the share who borrowed money fell from 44 to 41 percent. The efficacy of RI7 is rated Negligible. The overall efficacy of Pillar 3 is rated Highly Unsatisfactory, on account of the lack of progress on micro- finance sector performance or financial inclusion. Rating Highly Unsatisfactory OBJECTIVE 4 Objective Improve targeting and efficiency of government transfers and social protection PA8a, 8b, 9a,9b, 10a, RI8, RI9. Rationale [Need more information from TTL] RI8. The worsening security situation resulted in crowding out of non-military expenditures which affected all budgetary priorities. The ICR does not provide information on actual spending on sub-government transfers. The share of budgetary resources to the poorest three communes (RI8) did increase, from 15 percent in 2015 to 20 percent in 2019, but fell short of the target of 25 percent (reaching 80 percent). Given that RI8 only partially captures the PA and overall objective of improving the efficiency and targeting of transfers, coupled with the limited progress, the efficacy of RI8 is rated Modest. RI9. The number of households receiving cash transfers increased only slowly, from around 60,000 in 2015, to 80,000 by 2019 and 91,000 in 2020, but remained much below the target of 150,000. No additional information is available on (i) the characteristics of these households, and so targeting cannot be evaluated, (ii) progress on program coordination/expanding access to health care (PA9bii), or (iii) budget allocations or budget execution towards safety nets, to judge progress. On account of limited progress in combination with the limited relevance of RI2 in capturing the objectives, the efficacy of RI9 is rated Modest Page 17 of 24 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Mali - Poverty DPO Series (P157900) The overall efficacy of Pillar 4 is rated Moderately Unsatisfactory. This reflects the lack of prioritization of the social protection sector in the face of significant security problems, lack of evidence on improved targeting and efficiency, and the fact that RIs only partially measured progress on the associated PAs. Rating Moderately Unsatisfactory OVERALL EFF TBL OLD Overall Achievement of Objectives (Efficacy) Rationale The overall efficacy of the PRISOG series in achieving objectives is rated Moderately Unsatisfactory. The rating does not represent uniformly modest achievements but instead very mixed outcomes. The PRISGO series achieved solid progress in a few areas (targeting the poor with agricultural productivity measures, telecommunications services affordability and extension). However, overall efficacy was significantly held back by negligible or modest progress in key reform areas such as energy reform, safety net expansion, decentralization, and strengthening micro finance sector performance[SJdS1] . As noted in the ICR, p. 9, limited progress on reforms under PRISGO1 led to revised triggers (6 revised, two dropped, one replaced) in PRISGO2, several with more modest ambitions. In a few cases (rural roads, tax collection) there was no progress at all leading to a drop of the reform area. These modifications created a gap relative to expected outcomes and progress towards impact. Overall Efficacy Rating Moderately Unsatisfactory 6. Outcome Rationale The overall outcome of the operation is rated Moderately Unsatisfactory due to the limited progress in a number of reform areas. The relevance of Prior Actions was moderately satisfactory as they set out specific reform activities, in a logical sequence between PRISGO 1 and PRISGO 2, that could credibly support progress towards agricultural productivity for the poor, fostering access to infrastructure in rural areas, micro-finance sector improvement, and strengthening social protection. By contrast, the PRISGO series did not achieve the intended progress towards objectives. Whereas there was measurable progress on fostering agricultural Page 18 of 24 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Mali - Poverty DPO Series (P157900) productivity for the poor and a mixed record on infrastructure, the operations did not make headway on financial exclusion (where the situation worsened) or social protection, where progress on decentralization or extending services to the poor in different areas was too slow. a. Rating Moderately Unsatisfactory 7. Risk to Development Outcome Progress towards the four development objectives of the PRISGO series include higher access to telecommunications services and – although more modestly – the e-voucher expansion, the operationalization of rural land commissions, and the expansion of the cash transfer safety net scheme. Each of these face moderate risks (internal to each area/sector) and high risks (external to sector/area). Given capacity constraints, poor governance and vested interests, continued engagement by the World Bank or other development partners would be a strong mitigating factor for sustaining progress. To continue to foster competition in the telecommunications sector, the entry of the fourth license operator will be important. Although an additional provider had been identified and selected (by December 2019), it not been approved by the time of the ICR, 16 months later. The Government continues to support the e-voucher expansion (ICR, p- 29), adding new districts and addressing mobile ownership and coverage constraints. The Government’s commitment is the strongest mitigating factor for the significant vested interests in the input and subsidy distribution. The reform was also supported by a WB DPF (P167547), approved in FY20. Expansion of rural land commissions to more remote areas, and their effectiveness in resolving land disputes, will hinge on continued support to ensure that these commissions are capacitated with technical and financial resources. As per the PD of the new DPF operation, rural commissions relied on ad-hoc funding from localities until recently, but there will now be a recurrent spending item in the National Budget. Through various donors, technical support is now being provided to about 100-150 rural land commissions (PD for P167547, p. 34). Finally, the expansion of the safety net program, while slower than foreseen at appraisal, has been progressing steadily after the end of the PRISGO. It is being supported by a World Bank Safety Net project (P165064), due to close in FY23, which has received additional financing due to good project performance. Development progress achieved by the PRISGO (and other interventions) remain threatened by considerable security threats in Mali, as well as COVID-19 related impacts on the socio-economic situation more generally, but also directly on program implementation. The impact of the pandemic is being addressed through an emergency operation (P174457). 8. Assessment of Bank Performance a. Bank Performance – Design Page 19 of 24 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Mali - Poverty DPO Series (P157900) Rationale As noted in section 3a above, the PRISGO series was grounded in reform priorities established by the analytical work prepared for the Mali SCD (2015). ASA focusing on rural finance, agricultural productivity, telecommunications, and PFM, supported the development of prior actions. In addition, the PRISGO series built on significant synergies as it was supporting reforms that were part of other WB projects such as the interventions related to e-vouchers and safety nets and was linked to TA projects in the area of decentralization, finance and telecommunications. The e-voucher intervention was a scale-up of a successful pilot in North Mali. There were also valuable synergies between components. For example, ICT opportunities created synergies between different program components such as telecommunications, e-vouchers, micro-finance and cash transfers. In the project design and risk analysis, the PRISGO aimed to incorporate lessons learned from previous operations, namely (i) the importance of selectivity in policy focus (ii) the need to take incremental steps in each reform (iii) the importance of agricultural productivity to poverty reduction and (iv) the critical role of Government commitment especially in areas with strong vested interests (e.g., micro-finance and e-vouchers). At the same time, the ambition of the policy reforms in the PRISGO, given the well-known capacity and governance constraints in Mali, may have been unrealistic or required more explicit links between PAs and expected outcomes. Moreover, the monitoring framework suffered from imprecision, in particular regarding RIs under pillar 4 and pillar 1. Lack of targeting or precision thus compromised the relevance of M&E framework. The ICR team reported difficulties in finding data, due to weak Government monitoring. The Bank team recognized the risks facing Mali, in particular the high security and political risks endemic in Mali, as well as risks related to vested interests in different areas. The team also pointed out the difficulties in hedging against such risks, given high turnover among senior counterparts in the Government. It also highlighted the substantial risks related to complexity of the program and weak institutional capacity. As discussed above, the synergies with other ongoing WB projects, including TA, could have helped mitigate these risks (although they did not in several areas). For the second operation, the project document put greater emphasis on the direct risks due to governance/vested interests inherent in e-voucher, decentralization and energy reforms, and the risks associated with high turnover among key Government counterparts. The mitigating strategy remained the same, namely ensuring strong links to World Bank sector projects to facilitate dialogue and supervisions and leverage capacity building and technical assistance. In Mali, there are established coordination mechanisms with external development partners, for both budget support and sector technical support and dialogue. PRISGO operations had been discussed and vetted within this dialogue, to ensure coherence, synergies and complementarity (PD1, p.19; PD2, p.23). Rating Moderately Satisfactory b. Bank Performance – Implementation Rationale Page 20 of 24 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Mali - Poverty DPO Series (P157900) The implementation of PRISGO1 was affected by an overall deterioration in the security situation coupled with political instability, which required the World Bank team to adjust the program to retain relevance of the program. There were significant modifications undertaken to adapt PRISGO2 to lessons learned under PRISGO1. Between operations, two triggers were dropped, one (road maintenance) as the team judged that there was no likelihood of progress, and one (telecommunications) because the trigger was beyond the control of the Government. One trigger (synthetic tax) was completely revised as deemed irrelevant, and another six triggers needed more precision, or were made less ambitious – which then compromised efficacy of the operation. In addition, a new reform area – energy – was added, although the short timeframe of the second DPO likely precluded progress in such a complex area. RI1 was also revised (from cadastral map to “functionality of land commissions”). The adjustments made by the team were necessary in view of the worsening security situation. Partly, however, they must be assigned to some weaknesses in the design, e.g. for the dropped trigger for telecom, or the synthetic tax. Four supervision missions per year were undertaken, and supervision was done jointly with task team leaders of relevant sector projects in agriculture, telecommunications, and social protection. The team coordinated with IMF around the energy component in the second DPO and met with the donor community for debriefings at beginning and end of each mission. Nonetheless, M&E was sporadic during the operations (ICR, p. 28). Given the synergies between PRISGO and other projects, monitoring information should have been available from other project M&E, but there was insufficient focus on ensuring regular updating. This was particularly serious given that the Government capacity to monitor was weak. The ICR team reports difficulties in collecting data, even for the final ICR. At the same time, the team successfully implemented low-cost, rapid so-called iterative beneficiary monitoring (IBM) surveys for monitoring the implementation of e-vouchers and, later, land reform as well as health vouchers. The e- voucher survey provided clear evidence that the first round of e-vouchers had not been distributed as intended, which led to the revised trigger (PA2b) with specified implementation steps for the second DPO. During the implementation, the Bank showed commitment to the reform agenda by delaying PRISGO1 to ensure that the land act was enacted by Parliament (PA1a, RI1), and delaying the PRISGO2 due to the inadequate distribution of e-vouchers (PA2b, RI2). Rating Moderately Satisfactory c. Overall Bank Performance Rationale World Bank prepared a DPO series that was well anchored in Mali’s development priorities and by and large was underpinned by clear and credible prior actions. The Bank team also incorporated lessons learned from previous operations and ongoing analytical work and sought synergies with other development interventions in Mali. The policy context in Mali must be seen as extremely challenging, and the program faced strong headwinds as armed conflict and communal violence spread outside of Northern Mali, affecting public service Page 21 of 24 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Mali - Poverty DPO Series (P157900) delivery, political will, program M&E and more. Nonetheless, the Bank did not sufficiently account for the political economy of underlying reforms at the outset (but adapted in the context of the second operation) or put in place sufficient mitigating measures to address issues related governance and vested interest. Some revisions reflected problems in the initial design rather than changing circumstances. During implementation, the Bank responded to changes in program outlook and remained committed to the dialogue underpinning reforms. At the same time, follow-up monitoring and evaluation was insufficient. Overall Bank Performance Rating Moderately Satisfactory 9. Other Impacts a. Social and Poverty Poverty in Mali is concentrated in rural areas, and intimately connected with agricultural productivity. By focusing on inclusive growth and resilience, as expressed in reforms focused on rural areas, productive opportunities, financial inclusion and safety nets, the PRISGO series therefore had a strong social/poverty focus. By addressing land issues, PRISGO also addressed a significant source of conflict. The ICR and underlying M&E data do not contain information related to poverty impact and RIs generally did not incorporate pro-poor dimensions. The ICR notes that the e-voucher scheme successfully reached the poorest farmers, however, and the share of budgetary decentralization transfers allocated to the poorest regions increased as well. b. Environmental The PRISGO series could have a positive impact on environment through various activities, but there is no data to evaluate impact at this time. It will depend, inter alia, on how components related to agricultural practices as well as cash transfers result in behavioral changes that in turn affect esp. rural conditions such as land use, conservation methods, and input use. c. Gender The PRISGO series represents mixed results in terms of addressing gender issues more decisively. The safety net and e-voucher programs did contain specific measures to increase inclusion of women (e.g., by registering female household members as farmers as well as male). However, the complications related to women's access to land under customary law were not addressed, and the participation of women in land commissions was very limited. The PRISGO framework did not measure female participation in pertinent areas such as microfinance, cash transfers, etc. The ICR indicates that women were well represented among beneficiaries of the e-voucher program but provides not details to this effect. Page 22 of 24 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Mali - Poverty DPO Series (P157900) d. Other none. 10. Quality of ICR Rationale The ICR has several strengths. It is well written and has a clear narrative. It provides a review of the country and sector context for different reforms. It provides a succinct and clear account of the design of two operations where substantial modifications were made; the reader can easily follow these modifications and their rationale. Ratings are well motivated and logical. The ICR team faced significant challenges in collecting information in the context of a low capacity counterpart with limited monitoring capacity. The ICR could have provided a clearer analysis of the relevance of the PAs, which are rated MS in the ICR, motivated by the revisions towards more modest PAs in the second operation. In a few instances (e.g. Land reform) the ICR reduces the rating for the PAs on account of not achieving the outcome. Yet, it should be sufficient that the PAs contribute meaningfully to outcomes. The ICR could have provided more information on the Bank's implementation performance, in particular monitoring, and the coordination under implementation with other important stakeholders. Overall, the ICR is relatively brief, and more details and references would have been helpful to understand fully the assessment. a. Rating Substantial 11. Ratings Reason for Ratings ICR IEG Disagreement/Comments Moderately Moderately Outcome Unsatisfactory Unsatisfactory Moderately Bank Performance Moderately Satisfactory Satisfactory Relevance of Results Moderately --- Indicators Unsatisfactory Quality of ICR --- Substantial 12. Lessons Page 23 of 24 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Mali - Poverty DPO Series (P157900) The ICR provides the following lessons, with which the ICRR concurs with some modification of language. 1. The scope of the PDO should reflect the limited range of feasible PAs and the constraints of the implementation timeframe. In the PRISGO series, PDOs are set at such a high level that is impossible to evaluate them against performance criteria. The PDO needed to be more narrowly specified so that reforms underpinning them can be more closely linked to the objectives. 2. Overambitious triggers had to be downgraded in the PRISGO2 and ended up being input focused due to limited progress. In setting triggers, acknowledging capacity constraints as well as political and security risks is essential . 3. An explicit results chain provides a focus for assessment of program design. As pointed out by the ICR, many reforms focus on inputs and activities and leave gaps detailing how these activities will contribute to the achievement of ultimate objectives such as land security and effective use of agricultural inputs. A clearer results chain would have clarified each step (and associated risks) towards achieving outcomes and objectives and how to measure progress. 4. Results indicators need to track progress towards achieving the associated objective. This requires avoiding RIs that focus on partial results, achieving inputs or simply measuring whether activities were completed. 5. Technology can be leveraged to facilitate M&E in conflict affected areas. The IBM phone survey was used to significant effect to evaluate the first pilot of the e-voucher scheme, and was used also for the operation (e.g. on land commissions). Although those not covered by mobile network would not be accessible through this approach, the IBM survey provide a low cost and effective means to gather information, also at mid-term. 6. Addressing complex policy areas such as the energy sector requires a robust political consensus. To aim to achieve meaningful progress over three years, in areas such as energy reform where progress has been stalled during many years prior, is unrealistic. DPOs may not be the appropriate instrument for achieving such reforms. 13. Project Performance Assessment Report (PPAR) Recommended? No Page 24 of 24