Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AF Incentive Program DPG (P164882) Report Number: ICRR0022109 1. Operation Information Operation ID Operation Name P164882 AF Incentive Program DPG Country Practice Area (Lead) Afghanistan Macroeconomics, Trade and Investment Non-Programmatic DPF L/C/TF Number(s) Closing Date (Original) Total Financing (USD) IDA-D3410,TF-A9090 15-May-2019 297,578,652.56 Bank Approval Date Closing Date (Actual) 14-Jun-2018 15-May-2019 IBRD/IDA (USD) Co-financing (USD) Original Commitment 90,000,000.00 210,000,000.00 Revised Commitment 300,000,000.00 210,000,000.00 Actual 297,578,652.56 210,000,000.00 Prepared by Reviewed by ICR Review Coordinator Group William F. Steel Judyth L. Twigg Sengphet Lattanavong IEGEC 2. Program Objectives and Pillars/Policy Areas DEVOBJ_TBL a. Objectives The program document (PD, p. 6) states the program development objectives (PDOs) as: “(i) strengthening the policy framework to support state effectiveness, private investment, and social inclusion; and (ii) improving the policy and institutional framework for public financial management.” Page 1 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AF Incentive Program DPG (P164882) For the purpose of this ICRR, the PDOs have been disaggregated into the following distinct objectives: 1. Strengthen the policy framework to support state effectiveness; 2. Strengthen the policy framework to support private investment 3. Strengthen the policy framework to support social inclusion 4. Improve the policy and institutional framework for public financial management b. Pillars/Policy Areas The operation was structured around two pillars: (1) Strengthening the policy framework to support state effectiveness, private investment, and social inclusion; and (2) Improving the policy and institutional framework for public financial management. Pillar 1 had six Policy Actions (PAs):  PA 1: Formalizing and defining responsibilities for integrating national information technology (IT) infrastructure necessary to roll out the E-money and digital payment system.  PA 2: Amendment of the Civil Servants Law that: (i) establishes a career track for specialist staff; (ii) enables positive gender discrimination within civil service appointments; (iii) introduces merit-based and tailored recruitment into the civil service; (iv) establishes Deputy Minister positions within the civil service framework; and (v) enables transfers, sanctions, transparent grievance redress, and civil service renewal.  PA 3: Cabinet submission to the National Assembly of an insolvency law that adequately addresses insolvency proceedings, reorganization provisions for insolvent companies, creditor rights, and treatment of insolvency in viable businesses.  PA 4: Partnership Agreement with the national electricity utility (DABS) to improve its performance over 2018- 2020.  PA 5: Publication of a regulation on Managing Affairs of Informal Urban Properties allowing for the distribution of Property Documents to occupants of informal settlements in urban areas.  PA 6: Cabinet approval of an Irrigation Policy and a Dry Lands Agriculture Policy to increase the resilience of agriculture-based livelihood activities against climate change and strengthen management of water Pillar 2 had four PAs:  PA 7: Revised Budget Circular 1 that: (i) clearly defines the time-bound process and requirements for project proposal submission from ministries and agencies; and (ii) requires all project proposals to include cost estimates for operating and capital expenses needed for the project life-cycle.  PA 8: Roll-out of an electronic taxpayer management system by making fast-track tax filing available in Large Taxpayer Office (LTO).  PA 9: Afghanistan Revenue Department (ARD) to review and revise LTO and Medium Taxpayer Office (MTO) client lists to ensure correct categorization of taxpayers and risk-commensurate compliance and enforcement efforts. Page 2 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AF Incentive Program DPG (P164882)  PA 10: Revised Operations and Maintenance (O&M Policy) to: (i) improve O&M expenditure management and (ii) mandate piloting of the revised O&M Policy by four line ministries in preparing their 2019 budgets. c. Comments on Program Cost, Financing and Dates The USD 300 million Afghanistan Incentive Program Development Policy Grant (IP-DPG) included grants of SDR 62.6 million (USD 90 million) from IDA and USD 210 million from the Afghanistan Reconstruction Trust Fund (ARTF), a multi-donor funding mechanism for reconstruction support, which the government and the World Bank wished to convert into a standard Development Policy Financing operation. The project was approved on June 14, 2018, and became effective on August 7, 2018. Tranche 1 of USD 87,578,653 of IDA funds was disbursed on August 7, 2018, and Tranche 2 of USD 210 million from ARTF was disbursed on December 6, 2018. The project closed on schedule on May 15, 2018. The actual amount disbursed was USD 297.6 million; with the difference being accounted for by changes in the SDR/USD exchange rate. 3. Relevance of Design a. Relevance of Objectives The PDOs were directly relevant to the government’s strategies as stated in the Afghanistan National Peace and Development Policy Framework (ANPDF) and Fiscal Performance Improvement Plan (FPIP). These strategies were intended to continue and deepen the reforms undertaken under reconstruction since the fall of the Taliban in 2001. Despite substantial progress, the economy, society and political stability remained fragile and conflict-affected. However, objectives were set at a very high level, making their achievement within the time frame of a single operation unlikely. More modest or focused objectives would have been better aligned with the scope of the prior actions. This was particularly the case with the “state effectiveness” objective. The first PDO and pillar addressed several of the priorities identified in the ANPDF, i.e. (PD, para. 23): “i) improving governance and state effectiveness; ii) building social capital and nation building; and iii) poverty reduction and social inclusion.” The second PDO and pillar addressed the three key areas of public sector finance management (PFM) reform in the FPIP, namely: “(i) improving investment performance; (ii) ensuring a more accurate, transparent and accountable budget; and (iii) building capacity to manage reforms in the areas of human resources, administration, finance, IT and communications.” The objectives were well aligned with the World Bank’s Country Partnership Framework (CPF for FY17-20, para. 31). The CPF, which remained in force at closing, “is organized under three pillars: (i) building strong and accountable institutions; (ii) supporting inclusive growth; and (iii) expanding and deepening social inclusion. The first pillar is intended to lay the foundation for the subsequent pillars by strengthening the institutions needed to implement activities, in particular to build a professionalized civil service.” b. Relevance of Prior Actions Rationale Page 3 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AF Incentive Program DPG (P164882) There were 10 PAs, including three Prior Actions and seven Tranche Release conditions; of which six supported Pillar 1 and four supported Pillar 2.. Policy Actions 1, 2, 4 and 6 relate to strengthening the policy framework to support state effectiveness. Amendment of the Civil Servants Law (PA2) was aimed at inducing competitive selection of civil servants and Deputy Ministers, as well as increasing the proportion of female civil servants hired (which in turn would enhance social inclusion of women). This would differentiate between technical and administrative sectors, establish a career track for staff, enable positive gender discrimination, introduce merit-based recruitment, establish professional Deputy Minister positions, and make other changes to encourage civil service professionalization. This was a critical first step in a medium-term reform program, but implementation as of the ICR was mixed. It was expected that the resulting increase in the number of qualified and educated civil servants would raise public and private sector confidence in the capacity of government, leading to more effective state management and supporting increased private investment. Similarly, a decree providing a consistent government approach to formalizing IT infrastructure (PA1) was expected to lead to a rollout of digital financial services that would enhance financial inclusion, including women, thereby contributing to achievement of the outcomes of state effectiveness, access to finance, private investment, and social inclusion. PAs 1 and 2 were also consistent with CPF Objectives 1.2 (Improved performance of key government ministries) and 2.1 (Improved business regulatory environment and access to finance). To help address infrastructural constraints on state effectiveness and private investment, PA4 mandated an agreement with DABS to reduce debt and improve performance (especially electricity losses), and PA6 sought Cabinet approval of an Irrigation Policy and a Dry Lands Agriculture Policy. These measures supported CPF Objectives 2.3 (Increased power generation and access to electricity) and 2.4 (Increased agricultural productivity). Policy Action 3 relates to strengthening the policy framework to support private investment. PA3, submission of an Insolvency Law to the National Assembly, was intended to help improve the business environment for private investment by enabling efficient firms to survive while weeding out inefficient ones. This was consistent with CPF Objective 2.1 (Improved business regulatory environment and access to finance). However, lack of an insolvency legislative framework was not a pressing priority for the private sector (ICR p. 16) or a significant constraint to private investment and was therefore of only limited relevance to achievement of the associated objective being only indirectly relevant to the desired outcome. It was therefore of limited relevance of the de facto second objective. Policy Action 5 supported social inclusion by facilitating land tenure rights for returnees and internally displaced persons, and in particular to recognize women as owners on property documents, thereby promoting social inclusion. Overall, the relevance of PAs for Pillar 1 (Strengthening the policy framework to support state effectiveness, private investment, and social inclusion) is rated Satisfactory. Page 4 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AF Incentive Program DPG (P164882) Under Pillar 2, improved public expenditure management was promoted by PA7 for a Revised Ministry of Finance (MoF) Budget Circular requiring strategic fit screening for new investments, and PA10 for Cabinet approval of a revised O&M Policy to apply norms-based methods that would reduce mismatch between budget allocation and needs. Improved public revenue management was supported by PA8 to roll out an electronic taxpayer management system for electronic filing and PA9 to update large and medium taxpayer lists, which were expected to make tax administration more efficient and reduce the risk of corruption. These measures supported CPF Objective 1.1 (Improved public financial management and fiscal self-reliance). The ICR (pp. 14-15) notes that it was recognized that some PAs would only provide initial steps toward objectives, not directly generate the desired outcomes within a limited time frame, and that continuing support was being provided through other projects to achieve desired outcomes (in particular, IP-DPGs in 2019 and 2020). Overall, the relevance of PAs for Pillar 2 (Improving the policy and institutional framework for public financial management) is rated Satisfactory. Rating Satisfactory 4. Relevance of Results Indicators Rationale As noted, the high level objectives make the identification of appropriate results indicators that can assess outcomes over the period of the project difficult. Relevance therefore is largely derived from the adequacy of RIs in tracking the impact of PAs. Pillar 1 (Strengthening the policy framework to support state effectiveness, private investment, and social inclusion) had eight results indicators (RIs), of which one was specifically related to private investment and one to social inclusion. Pillar 2 (Improving the policy and institutional framework for public financial management) had five RIs, of which two related to public expenditure management and three to public revenue management. Three RIs [stated below] were linked to Prior Action 2: the Recipient’s Cabinet has approved and submitted to the National Assembly an amendment of the Civil Servants Law. RI 2 and RI 4 were appropriate indicators of overall progress toward the intended result of developing a more qualified, educated and professional civil service in order to raise public and private sector confidence in the capacity of government and lead to more effective state management, while RI 3 indicated whether women were benefiting from competitive recruitment of civil servants: Page 5 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AF Incentive Program DPG (P164882)  RI 2: Number of civil servants competitively selected in line with the new recruitment process; with a Baseline (2017) = 0 and a Target (2019) = 500. [The target number appears too small to have the desired impact, but, fortunately, a much greater number was achieved.]  RI 4: Proportion of line ministries for which the position of Deputy Minister for Administration & Finance has been competitively selected into a civil service position; with Baseline (2017) of 0% and Target (2019) of 100%.  RI 3 (Proportion of female civil servants at Senior Management Group level recruited through the revised process among competitively recruited civil servant positions) was directly related to improving the quality of the civil service by recruiting qualified female candidates who had previously been excluded, and to improving social inclusion of women. There were no female recruits at Baseline (2017), with a Target of 15% of new recruits by 2019. Digitization of government payments was considered to be important for improving state effectiveness, and was supported by RI 1 (Technical functional specification developed by Asan Khedmat (AK) for an interface of integrated government IT system). Although the PA included delineating “responsibilities for integrating national IT infrastructure necessary to roll out the E-money and digital payment system,” which are important for the objectives of increasing financial access and inclusion (especially of women), there were no indicators related to digital payments and inclusion. Two RIs represented partial indicators of progress in improving infrastructure to support greater state effectiveness: (i) RI 6 Distribution level losses in Kabul and Herat Provinces as a percentage of electricity supplied from substations, to be brought below the Baseline (2017) rates of 20% (Kabul) and 26% (Herat) as part of performance targets under a partnership agreement between MoF and DABS (PA4); and (ii) RI 8 Irrigated wheat productivity, targeted to rise above the Baseline (2017) of 2.49 tons per hectare as a result of improvements expected to ensue from Cabinet approval of an Irrigation Policy and a Dry Lands Agriculture Policy (PA6). These were suitable as measurable indicators during the one-year project time period, given the time lags that can be expected between adoption of a policy, measures to implement it, and achievement of actual improvements in infrastructure. However, these indicators are minimal (essentially targeting only that losses and productivity not worsen) and at best only indirectly relevant to measuring substantive improvements in infrastructure. The ICR (p. 19) notes, for example, that “irrigated wheat productivity is driven by multiple factors.” [Furthermore, as discussed in Section 5, the methodology for measuring electricity distribution losses changed and could not be calculated retroactively to the baseline, so RI 6 proved ineffective.] Results Indicator 5 relates to strengthening the policy framework to support private investment. RI 5 (Distance to Frontier score against the Resolving Insolvency measure in the Doing Business Indicators) attempts to capture the impact of PA 3 (national Insolvency Law), from a Baseline (2017) of 23.2 to a Target (2019) of 33.2. However, it does not capture use of the insolvency framework in practice, let alone improvements in Afghanistan’s investment climate and is therefore of limited relevance. Results Indicator 7 relates to strengthening the policy framework to support social inclusion. RI 7 (Number of property documents recognizing women as full or part owners of land) was a well-chosen indicator of whether the new Regulation on Managing Affairs of Informal Urban Properties was in fact being implemented to allow for the distribution of Property Documents to returnees and internally displaced persons in informal urban settlements. The focus on women represents the most vulnerable segment of the intended beneficiaries, since they were excluded from ownership as of Baseline (2017). The Target (2019) was for women to receive 150,000 or at least 50% of total property documents issued. Page 6 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AF Incentive Program DPG (P164882) Two indicators were reasonably well related to improving public sector expenditure management (Pillar 2): (i) RI 9 (Proportion of approved projects by share of total investment budget that have undergone strategic fit screening), targeted to rise from nil at Baseline to 80% in 2019 (consistent with the Budget Circular to be issued under PA7); and (ii) RI 13 (Number of ministries that develop budget proposals for O&M expenditures using a norm-based method), targeted to rise from none to 4 in 2019 (consistent with the revised O&M policy under PA 10). Three indicators were adequately related to improving public sector revenue management. (i) The Target for RI 11, for the proportion of LTO clients with access to fast-track filing to rise from 2 percent at Baseline (2017) to 100% in 2019, indicated whether PA8 to roll out an electronic taxpayer management system was in fact making fast-track tax filing available. The expected result of increased revenue collection was indicated by (ii) RI 10 (Domestic revenue collected by the LTO and MTO) and (iii) RI 12 (LTO share of total ARD revenue). Eight RIs represented reasonable indicators of implementation of the six PAs that supported Pillar 1 to improve the policy framework for state effectiveness, private investment and social inclusion – although they were related more to outputs at the policy level than achievement of the intended outcomes (which were to be further supported under other ongoing operations). Five indicators adequately captured important elements of both improved expenditure management and improved revenue management, thereby signaling progress toward the objective of improving public sector financial management (Pillar 2). While the indicators were related to the PAs, and targets were articulated, the main limitation of the results framework, as noted by the ICR (p. 28), is that “the results matrix did not fully capture the outcomes of a well-designed and implemented operation.” Adequate data were available to monitor all the indicators in a timely manner. However, changed methodology in measuring electricity distribution losses made it impossible to compare one indicator to the baseline. Rating Moderately Satisfactory 5. Achievement of Objectives (Efficacy) EFFICACY_TBL OBJECTIVE 1 Objective Strengthen the policy framework to support state effectiveness Supported by PAs 1, 2, 4, and 6; RIs 1-4, 6 and 8. Rationale The Civil Servants Law was amended May 23, 2018 (PA2) resulting in competitive selection of 28,667 civil servants by 2019 under the new recruitment process, greatly exceeding the target of 500. Although some progress was made in recruitment of female civil servants, only 10.2% of the new recruits were female, falling Page 7 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AF Incentive Program DPG (P164882) short of the target of 15%. Political resistance prevented competitive recruitment of any Deputy Ministers into civil service positions. A decree to formally delineate AK responsibilities for integrating national IT was fulfilled on 11 November 2018 (PA1). This helped overcome conflicting Ministerial approaches and provide a basis for a harmonized approach to digitizing government payments. AK’s development of a technical functional specification for an interface of integrated national IT infrastructure fully met RI#1. The ICR did not report whether rollout of E- money and digital payments had actually begun. a Partnership Agreement between DABS and MoF was signed in November 2018 to address DABS’ unsustainable debt and improve its technical and financial performance over 2018-20 (PA4). While this represented an improvement in the support framework, achievement of the RI for reducing electricity distribution losses could not be assessed due to a change in the methodology, which could not be retrofitted to the Baseline value. Nevertheless, the ICR states that “overall like-for-like reduction in losses from 2017 to 2019 are estimated at between one and two percent, indicating that the targets were likely met for both cities.” The Cabinet approved an Irrigation Policy and Dry Lands Agriculture Policy on October 30, 2018 (PA6), to provide overall guidance on water and irrigation management. Irrigated wheat productivity rose by 5 percent from 2017 to 2019, fully meeting the target. However, as the policy was only approved in the final quarter of 2018, the full impact from 2017 (base year) cannot be attributed to the prior action. Nevertheless, as noted in the ICR, this indicator was not well suited to measuring actual improvement in the infrastructure or the impact of the new policies. Summary: There were positive but mixed results in terms of achievement of indicator targets and objectives. Three indicator targets were fully achieved and one was likely to have been achieved; one was only partially achieved; and one was not achieved. Rating Moderately Satisfactory OBJECTIVE 2 Objective Strengthen the policy framework to support private investment Supported by PA 3 (and indirectly by PAs 1, 4 and 5); RI 5 (and RI 6). Rationale A new Insolvency Law was passed on May 23, 2018The improvement in the Doing Business “distance- to- the-frontier” for Resolving Insolvency was fully achieved, rising from 23.2 at Baseline to 51.8 in 2019, well above the target of 33.2. However, this indicator does not measure whether perceptions had actually changed as a result or whether or not the reformed insolvency regime was being used (let alone whether private investment was rising), on which no data are provided. The ICR (p. 16) notes that “private sector representatives report that the experience on the ground remains much as it was” and that further work is needed to implement the law. Page 8 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AF Incentive Program DPG (P164882) Rating Moderately Satisfactory OBJECTIVE 3 Objective Strengthen the policy framework to support social inclusion Supported by PA 5 (and indirectly by PA1 [women access financial services] and PA2 [female civil servants recruited]); RI 7. Rationale The Regulation on Managing Affairs of Informal Urban Properties (approved on May 23, 2018), allowed for the distribution of property documents to occupants of informal settlements on urban land, including refugees and internally displaced persons, and specifically providing for recognition of women’s ownership, as a step toward increasing access to services, finance and economic opportunities. Implementation of the Regulation on Managing Affairs of Informal Urban Properties (PA5) was impeded by technical, administrative and capacity issues, as well as burdensome fees, and the issuance of 23,248 Occupancy Certificates to women fell well short of the RI Target of 150,000. Nevertheless, 63.5% of the Certificates issued did recognize women as full or (predominantly) partial owners, exceeding the Target of 50%. While this represents a positive, if small step toward greater social inclusion of women, no evidence is provided as to whether those receiving Certificates are better able to access services, finance or other opportunities. Only 3% of Certificates recognized women as full owners, and it is not established whether being listed as a joint owners confers actual benefits. Similarly, although there was some progress in recruiting female civil servants (see Objective 1), the indicator value of 10% as of 2019 fell short of the target of 15% of new recruits. Rating Moderately Unsatisfactory OBJECTIVE 4 Objective Improve the policy and institutional framework for public financial management Supported by PAs 6-10; RIs 9-13. Rationale The MoF issued a revised Budget Circular on July 14, 2018 (PA7), which defined minimum quality standards, set timelines, and provided detailed guidelines and standard templates for project costing. These processes were applied to the 2019 budget process, and the target of 80% of new projects undergoing strategic fit screening was fully met. Page 9 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AF Incentive Program DPG (P164882) The ARD implemented an electronic taxpayer management system and made fast-track tax filing available to all taxpayers registered in the LTO by September 19, 2018 (PA8). By the end of 2019, all taxpayers were able to use the system, meeting the target (actual use was 34.5%, but as of 2020, all LTOs will be required to use it). The resulting increase of Afs44.5 billion in revenues from the LTO and MOT fully met the target. Furthermore, the ARD reviewed and recategorized the client lists of LTO and MTO taxpayers by September 19, 2018 (PA9). Data and coding problems with client lists of LTO and MTO taxpayers rendered the Baseline indicator obsolete for RI?, but comparable data were used to estimate that the LTO share of total ARD revenue rose by 7 percentage points, only partially achieving the targeted nine percentage point increase. The revised O&M policy was piloted by four line ministries when preparing their 2019 budgets, thus meeting the target for RI13. Summary: Four targets were fully met, one partially, and there was evidence of improvements on both the expenditure and revenue sides of PFM. Rating Satisfactory OVERALL EFF TBL OLD Overall Achievement of Objectives (Efficacy) Rationale While noting the misalignment between the level at which objectives were set (too general and high level to be achievable by the close of the operation) and the more modest scope of PAs, this ICRR finds that achievement of one objective was rated as satisfactory, two as moderately satisfactory, and one as moderately unsatisfactory (based on PDOs as unpacked in Section 2(a)). Eight of the 13 results indicators were fully achieved, three were partially achieved, and one was not achieved (with one difficult to assess because of changes in methodology). On balance, achievement of objectives is rated as moderately satisfactory. Overall Efficacy Rating Moderately Satisfactory 6. Outcome Rationale The relevance of prior actions (design) was rated as satisfactory, and the achievement of objectives (efficacy) as moderately satisfactory. Some of the results were more in the nature of intermediate outputs or processes, representing improvements in the framework to support the intended outcomes (such as private investment and Page 10 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AF Incentive Program DPG (P164882) social inclusion), which were not measured directly (the relevance of RIs was rated as moderately satisfactory). The ICR (p. 14) recognized the “expectation that limited progress would be achieved towards outcome-level results by the time of the ICR” and that the actions taken in connection with this operation only “represented a necessary first step of a broader reform agenda.” This fact would have been better captured with more focused objectives. Two further IP-DPG operations have been implemented (2019) and approved (2020) to support “further progress towards implementation of reforms in all policy areas included in the 2018 DPG” (ICR, p. 28). This suggests that progress toward outcomes may continue, although indicators from the 2019 operation were not reported in this ICR. In this context, the overall outcome is rated as moderately satisfactory. a. Rating Moderately Satisfactory 7. Risk to Development Outcome Risks to development outcomes appear high in several dimensions. Security: The government lacks sufficient control to deliver services to substantial portions of the country. Although the United States and the Taliban agreed on a peace deal in February 2020, talks between the Taliban and the government have broken down, and conflict is likely to resume. The need to manage security risks diverts government attention from continued implementation of reforms. Macroeconomic: Despite some progress in fiscal policy and the macroeconomic framework as of 2018, the PD recognized a “challenging” fiscal outlook and high risk of debt distress. The ICR (p. 26) notes that “Afghanistan remains heavily dependent on aid,” and that aid was expected to reduce (even before the Covid pandemic), especially if there is no settlement with the Taliban. Political: The reforms supported by the operation were generally not controversial and had broad political support, so there is reasonable expectation that implementation will continue. The main exception was Deputy Ministers; the refusal to recruit them competitively into the civil service indicates a political inclination to control rather than professionalize government administration. Unresolved disputes over the 2019 Presidential election mean that political risks continue, and could spill over into the security situation. Institutional: The ICR (p. 26) notes that the ability to continue implementing reforms is limited by “capacity constraints [that] remain across government agencies.” Mitigating Factor: The main factor supporting continued progress in implementing reforms that were initiated under the IP-DPG is the existence of complementary support operations, both sectoral and the IP-DPG operations in 2019 and 2020. The ICR (p. 7) states that “all policy reform actions were carefully selected to leverage and support parallel World Bank investment project financing engagements (recipient-executed) and technical assistance activities (Bank-executed) of each sector team.” However, the current status of these operations is not provided in the ICR. 8. Assessment of Bank Performance Page 11 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AF Incentive Program DPG (P164882) a. Bank Performance – Design Rationale Project design benefitted from past experience and was oriented to support the government’s objectives, as stated in the ANPDF and FPIP, as well as the priorities of the World Bank’s CPF 9although, as noted, there was a misalignment between the scope and ambition of the PDOs and of the PAs). There was substantial consultation with relevant stakeholders. The design was innovative in providing a set of tranche release conditions for the bulk of the financing (from the ARTF), with delays in satisfying conditions intended to lead to decreased funding. This approach to providing incentives for timely implementation and clear benchmarks for technical assistance to support compliance was intended to reduce the risks of “low capacity and political volatility” (PD para. 27) that had been identified from the experience of the ARTF. Macroeconomic, political, security and institutional capacity risks were adequately identified at the outset, though not necessarily mitigated in the design. Mitigation was relied on support from World Bank investment projects which was appropriate in view of the one-year time frame of the operation. The focus on relatively non-controversial reforms (with the exception of having Deputy Ministers competitively recruited instead of politically appointed) generally managed to reduce political opposition, although this may have come at the cost of the relevance of prior actions to achieving meaningful impact. . Objectives were set at too high and general a level, presenting challenges in the articulation of appropriate results indicators and achievable targets. Other results indicators focused earlier in the results chain than necessary (e.g., taxpayer access to the system) Moreover, the link between some results indicators and the desired eventual outcomes was not always clear, especially with respect to social inclusion and private investment (for which the impact of the PAs may have been negligible). The links from the indicators for electricity losses and wheat productivity to infrastructure development and the objective of state effectiveness were not evident. Although the detailed results chain was not always explicit, there was adequate discussion in the PD (and ICR) of expected intermediate results and outputs, especially with respect to PFM. Gender considerations were explicit in the incorporation of actions to give women greater access to land ownership and to be competitively recruited into the civil service. Poverty and social aspects were assessed; although the PAs were not expected to have much direct impact on poverty, the PD (para. 72) stated that “no policy actions supported by the operation are expected to have significant unmitigated negative poverty and social effects in areas currently under Government control.” Likewise, although the Irrigation and Dry Land Agriculture Policies were thought to have some potentially positive effects in mitigating the effects of climate change, overall, the assessment in the PD (para. 79) concluded that “the policy and institutional reform actions supported by this operation are not expected to have significant effects on the environment, forests, or other natural resources.” Rating Moderately Satisfactory Page 12 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AF Incentive Program DPG (P164882) b. Bank Performance – Implementation Rationale The ICR (p. 25) states that the team “provided critical project support to reforms in the areas of land, IT, financial sector, and business regulatory environment….[and] was able to constantly track progress with reforms, coordinate between government and other development partners, engage in high-level policy dialogue, and mobilize technical support” to overcome obstacles to implementation of the reforms. This was evidently successful in mitigating institutional capacity risks for most of the PAs (other than moving away from political appointment of Deputy Ministers). The ICR (p. 25) also states that the team had “significant in-country presence and strong relationships with key agencies, [and] was able to constantly track progress with reforms, coordinate between government and other development partners” (including the Incentive Program Working Group), indicating a high level of stakeholder coordination and consultation. One of the two TTLs was present in Kabul throughout, and some six team members were also based there (another four in Dubai). The TTLs monitored implementation on a monthly basis, and referred issues to the technical teams working on the projects responsible for implementation in the relevant areas. The RIs were adequately monitored. In two cases, data problems or changes in methodology invalidated the original baseline estimates. In one case, recalculation of the baseline showed that the apparent achievement of the target value actually fell short of the expected increase. In the other case, an alternate indicator was attempted, but was not considered adequate to make an assessment. Rating Satisfactory c. Overall Bank Performance Rationale With some gaps in linkages between results indicators and intended objectives and minor weakness in team continuity, overall Bank performance is rated moderately satisfactory. Overall Bank Performance Rating Moderately Satisfactory 9. Other Impacts a. Social and Poverty Page 13 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AF Incentive Program DPG (P164882) The ICR (p.23) concludes that “more time is required before full poverty, gender, and social impacts of supported policies can be known.” b. Environmental The ICR (p. 23) states that it is too soon to document any direct or indirect environmental impact. c. Gender The ICR notes positive first steps toward increasing the number of women in the civil service (through implementation of the new Civil Service Law) and recognizing women’s occupancy rights in informal settlements (through issuance of Occupancy Certificates under new regulations governing informal settlements), although the targeted results were only partially achieved. It is too soon to document the impact of reforms in IT infrastructure and the consequent rolling out of E-money.. d. Other The ICR (p. 24) notes that the operation contributed to “important institutional reforms…. the professionalization and capacity of the civil service, tax administration and e-money.” 10. Quality of ICR Rationale The ICR is well written, providing a consistent and detailed narrative of the operation’s objectives, achievements and limitations. It is generally results-oriented albeit constrained by the short duration of the operation, making use of the available evidence on the results indicators; there is little supplementary data provided to verify that achievement of targets was in fact achieving the desired outcomes. A results chain is implicit in the detailed discussion of achievement of objectives, although the links to intended outcomes are not always apparent. The well-articulated discussion of lessons learned provides clear, relevant guidance that is applicable to development policy operations, elsewhere as well as in Afghanistan. a. Rating Substantial Page 14 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AF Incentive Program DPG (P164882) 11. Ratings Reason for Ratings ICR IEG Disagreement/Comments Moderately Outcome Moderately Satisfactory Satisfactory Moderately Bank Performance Moderately Satisfactory Satisfactory Relevance of Results --- Moderately Satisfactory Indicators Quality of ICR --- Substantial 12. Lessons The following lessons drawn by the ICR (pp. 27-28) follow from the analysis and may be relevant to development policy operations elsewhere, as well as in Afghanistan: Multiple tranches with timing-dependent penalties for delays can be effective "in incentivizing government to actively identify potential causes of delay in reform actions and take prompt corrective measures, and in providing some predictability regarding likely budget support disbursements." Clear assignment of responsibility for implementation and regular monitoring help to build government ownership, which is critical to reform progress. “Regular monitoring meetings… discussed the progress of each policy reform action associated with tranche release conditions and identified the bottlenecks. Through this government-led monitoring mechanism, remedial actions were swiftly taken by the government to address administrative bottlenecks as well as by the Bank and development partners to provide necessary technical support to facilitate the progress of targeted reform.” Multi-donor engagement and oversight can facilitate effective policy dialogue, leading to selection of appropriate policy actions, monitoring progress, providing needed technical assistance, and addressing bottlenecks. A “joint monitoring mechanism ensured that responsible ministries were held accountable for the progress of reform actions.” Development policy operations can serve a valuable purpose in focusing scarce government capacity. “The operation’s focus on key policy priorities helped sustain policy attention and scarce capacity on high- priority reforms, when the attention of government agencies was being distracted by various reform programs, emergencies, and coping with the difficult political and economic environment. However, single tranche operations present challenges in articulating a results framework that can capture outcomes achieved by the close of the operation suggesting the need for more modest and/or focused objectives. 13. Project Performance Assessment Report (PPAR) Recommended? No Page 15 of 15