Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Afghanistan COVID-19 REACH (P174119) Report Number: ICRR0023256 1. Project Data Project ID Project Name P174119 Afghanistan COVID-19 REACH Country Practice Area(Lead) Afghanistan Social Sustainability and Inclusion L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IDA-D7110,TF-B3543 31-Dec-2021 180,955,126.06 Bank Approval Date Closing Date (Actual) 04-Aug-2020 31-Dec-2021 IBRD/IDA (USD) Grants (USD) Original Commitment 280,000,000.00 125,000,000.00 Revised Commitment 244,752,869.99 89,752,869.99 Actual 180,955,126.06 89,381,332.99 Prepared by Reviewed by ICR Review Coordinator Group Cynthia Nunez-Ollero Christopher David Avjeet Singh IEGSD (Unit 4) Nelson 2. Project Objectives and Components DEVOBJ_TBL a. Objectives According to the Financing Agreement (FA, p.5) and the Project Appraisal Document (PAD, paragraph 19), the Project Development Objective (PDO) of this COVID-19 Relief Effort for Afghan Communities and Households (REACH) project was "to provide emergency support to selected households through communities in project areas during the COVID-19 outbreak." This review will assess project performance against this singular objective. Page 1 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Afghanistan COVID-19 REACH (P174119) b. Were the project objectives/key associated outcome targets revised during implementation? Yes Did the Board approve the revised objectives/key associated outcome targets? No c. Will a split evaluation be undertaken? Yes d. Components The following four components were to cover about two-thirds of the country in three geographic areas. The following three agencies were implementing relevant projects in these areas: (i) the Community Charter Afghanistan Project - Rural (CCAP-Rural), -CCAP-Urban (CCAP-Urban), (EZ-Kar), and (iii) Kabul Municipal Development Project (KMDP). The remaining one-third of the country was to be covered by CCAP. Relief packages were to be distributed to households in respective target areas. Earlier projects like the National Solidarity Program (NSP) and EZ-Kar had established and used community groups such as Community Development Councils (CDCs), Gozar Assemblies (GAs) or Councils (GCs) to interface with beneficiaries in the target areas. Gozars are sub-district level administrative units. GAs are elected bodies at Gozar level usually constituted through CDC representatives. For this project Ad hoc COVID-19 CDCs and Ad hoc COVID-19 GAs or GCs were formed solely for this project and where no CDCs or GAs existed before. These Ad-hoc CDCs and GAs were not part of regular CDCs and GAs mobilization activities supported by CCAP, EZ-Kar, or NSP. The project was to contract Facilitating Partners (FPs) to support these community groups to: (i) identify eligible beneficiary households; (ii) receive and manage Emergency Relief Grants in their bank accounts; (iii) procure the goods to be included in the in-kind relief package; (iv) distribute the relief packages; and (v) report to the implementing agencies. 1. REACH Rural: Household Support in Rural and Peri-urban Areas (US$136.0 million at appraisal, US$105.0 million actual) This component was to finance the procurement of relief support in the form of food and hygiene products for an estimated 2.5 million households (or 93 percent of all households) in target rural and peri-urban areas. The target area covered 275 districts and 19 provincial capital cities outside Kabul. These provincial capital cities were managed by the Ministry of Rural Rehabilitation and Development (MRRD). In kind relief packages valued at AFN4,000 (or US$50) containing essential food staples and hygiene products were to be distributed in a single tranche to each eligible household. These packages were estimated to be 50 percent of monthly food ration of an average Afghan household. 2. REACH Urban: Household Support in Provincial Capital Cities (US$56.0 million at appraisal, US$30.0 million actual) This component was to finance the provision of relief support in the form of food and hygiene products and/or cash to be delivered to an estimated 453,000 households (or 90 percent of all households) in 14 provincial capital cities, including an estimated 1.45 million returnees living in settlement camps. An estimated 10 percent of households were pre-determined to be affluent using standard exclusion criteria and were to be excluded from receiving relief packages. In-kind relief packages for urban areas, valued at AFN8,000 (or US$100) to be distributed to eligible households in two tranches. The first tranche of AFN4,000 (or US$50) per household was to be distributed in kind, or cash, based on the preference of communities and subject to pre-determined local circumstances. The second tranche of the next AFN4,000 (US$50 equivalent) per household was to be rolled out later in cash depending on the Page 2 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Afghanistan COVID-19 REACH (P174119) trajectory of the crisis. Mobile money technology was to be explored for this second tranche, depending on factors such as the number of households with acceptable documents for receiving mobile transfers, and the capacity of mobile money operators (MMO) to expand their cash-out point networks. 3. REACH Kabul: Household Support in Kabul Municipality (US$80.0 million at appraisal, US$40.9 million actual) This component was to finance the provision of relief support in the form of food and hygiene products and/or cash to be delivered to approximately 629,000 households (80 percent of total households) in Kabul municipality. Following the same modality and exclusion criteria of component 2 above, about 20 percent of affluent households were to be excluded from receiving relief packages. In-kind relief packages for Kabul municipality, valued at AFN8,000 (or US$100) was to be distributed to each eligible household in two tranches as in component 2 above. 4. Project Management, Communication, and Monitoring (US$8.0 million at appraisal, US$4.9 million actual) This component was to finance the following activities of the three implementing agencies: (i) project management and institutional support; (ii) dedicated strategic communication; (iii) grievance management; and (iv) project monitoring support, including gender and safeguards compliance monitoring. These three implementing agencies were to ensure that communications, grievance management and monitoring were closely coordinated and reported. e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project Cost: The original total project cost was US$280 million. The final project commitment amount was reduced to US$244.8 million. The grant disbursed US$181 million. The balance was cancelled after the Bank suspended operations in the country following the Taliban takeover of the government on August 15, 2021 (see Dates below). The Bank cancelled all undisbursed balance of ARTF commitments at the country portfolio level to repurpose the funds for a new set of operations after the suspension of activities in the country. Financing: The International Development Association (IDA) financed the US$155 million grant and the Afghanistan Reconstruction Trust Fund, managed by the World Bank on behalf of its donors financed another US$125 million. Borrower Contribution: None Dates: The grant was approved on August 4, 2020 and was made effective on September 14, 2020. The project closed as planned on December 31, 2021. No restructuring dates were noted in the ICR data sheet. On April 4, 2022, a Cancellation Notice of Undisbursed Balance from Closed Projects was released and on April 8, 2022 the Omnibus Notice of Cancellation cancelled US$35.2 million in ARTF financing. On August 15, 2021, the Taliban took over the government and all donors, including the World Bank suspended their activities in the country. Split Rating: A reverse split rating of the outcome will be undertaken due to the collapse of the government when the Taliban took over in mid-August 2021, four months before the project closed. A split rating will capture the outcome of the project before disbursements were suspended on August 15, 2021, and then provide an assessment of progress after the takeover. Page 3 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Afghanistan COVID-19 REACH (P174119) 3. Relevance of Objectives Rationale Context: Afghanistan was vulnerable to the rapid spread of COVID-19 because of limited access to information, poor and vulnerable households who subsist on daily earnings, lack of access to water and sanitation, weak health systems and ongoing violent conflict. The COVID-19 crisis in Afghanistan led to loss of life, deterioration of health, and economic stress. The government adopted the widely used containment measures to control the spread of the virus - social distancing, limiting mobility of residents through lockdowns, and closing of borders. These disrupted economic activities and led to loss of employment and income. Remittances were interrupted. Imports, particularly of household items, were disrupted, feeding inflationary pressures. Exports decreased because of border closings. Fiscal pressures increased as government resources were expected to decline by as much as 30 percent below budget. The national poverty rate was expected to rise from 61 to 72 percent (another 1.9 to 6 million people in poverty). The UN projected that 12 million people or 32 percent of the population would face acute food insecurity. The collapse of the government in August 2021 triggered further economic crisis and a withdrawal of most international aid. Country Plans: Recognizing the need for urgent actions against COVID-19, the government undertook a national program called Dastarkhan-e-Milli to serve as the platform for solidarity and address food insecurity. The program would distribute relief packages to reach a third of the country through the ongoing Community Charter Afghanistan Project (CCAP, see below) and the other 2/3 through this project. The program identified the following eight key priorities for short- and medium-term recovery: (i) a national health program, including expanding the scope and scale of basic healthcare and hospital reform and expansion; (ii) a national human security program, involving expansion of community driven development and reform of humanitarian programs; (iii) a national human capital program, to enhance use of technology for distance learning; (iv) a national infrastructure program, to accelerate access to electricity and internet; (v) a diaspora reintegration program to assist returning refugees and migrant workers; (vi) a national stimulus program to balance social and spatial development; (vii) a national recovery program to assist provincial, municipal, district, civic organizations, and the private sector; and (viii) an enhanced regional connectivity program to expand links with neighboring countries (PAD, paragraph 14). The First Vice President was to lead an inter-ministerial task force to manage the government's overall COVID-19 response efforts. World Bank Country Partnership Framework: The PDO was relevant to the objectives of the World Bank Country Partnership Framework (CPF) for FY17-FY20. The CPF period was extended for an additional two years to cover FY21 and FY22. The Bank updated the CPF results matrix to reflect increased emphasis on climate resilience, women’s empowerment, and private investments. Three pillars supported the CPF strategy: (i) building strong and accountable institutions; (ii) supporting inclusive growth; and (iii) social inclusion. The PDO contributed to the achievement of the following strategic priorities in the CPF (i) enhanced social protection for poor and vulnerable people; and (ii) ‘Improved government and community capacity to manage and respond to natural disasters. In addition, the PDO was relevant to the four priorities - agriculture, basic health, education, and livelihood - identified in the Bank's Afghanistan Approach Paper 2.0. This paper served as the blueprint for continued engagement with Afghanistan after the Taliban takeover. The approach would finance, conduct analytical work, and coordinate convening opportunities to protect the vulnerable, help preserve human capital and key economic and social institutions, reduce the need for future humanitarian assistance, and improve gender equality outcomes. The ARTF donors would Page 4 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Afghanistan COVID-19 REACH (P174119) decide the Recipient Executed grants to United Nations (UN) agencies and potentially international and national non-governmental organization (NGOs). These grants were to be made off budget and without the involvement of the interim Taliban administration (ITA) administration. World Bank Experience in the Sector and in the Country: The World Bank has assisted Afghanistan with programs and projects using a community driven development approach (CDD). Foremost of these was the National Solidarity Program (NSP). The NSP established Community Development Councils (CDCs) to deliver services to an estimated 90 percent of 35,000 rural communities in the past 14 years. The Citizens’ Charter Afghanistan Project (CCAP) succeeded the NSP. CCAP was to deliver a minimum service standards package of core infrastructure, emergency support, and social services to participating communities. CCAP used three mechanisms to deliver services to beneficiaries. One, CCAP continued the use of CDCs. Two, CCAP established around 10,000 Grain Banks or local grain reserves. Social Inclusion Grants (SIGs) from these "banks" were given as food or cash to the more needy members of the community. Three, CCAP launched a cash-for-work community asset building program called Maintenance and Construction Cost Cash Grant (MCCG) to reach internally displaced or returnee populations through the Eshteghal Zaee Karmondena (EZ-Kar) project. The social safety net programs that the Bank has used in other countries were relevant to the design of this project (see Section 8 Assessment of Bank Performance at Entry below). Overall, the relevance of the objective is rated High. The project was an emergency operation that directly supported the government's COVID-19 response. The PDO was highly relevant and contributed to achieving the objectives of the Bank's CPF and Approach 2.0 adopted after the Taliban takeover of the government. This paper provided the blueprint for continued delivery of humanitarian and other assistance to Afghanistan. Rating Relevance TBL Rating High 4. Achievement of Objectives (Efficacy) EFFICACY_TBL OBJECTIVE 1 Objective To provide emergency support to selected households through communities in project areas during the COVID-19 outbreak. Rationale Theory of Change (TOC): A results chain was drawn up at appraisal to serve as the original TOC. This same TOC was used at closing. The TOC provided a reasonable causal link between inputs, outputs, and expected outcomes. The objective - to mitigate the COVID-19 induced economic hardship and food insecurity among vulnerable households - was to provide emergency relief packages to benefit Afghanistan households with incomes of US$2 per day or lower. These target beneficiaries covered an estimated 90 percent of households Page 5 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Afghanistan COVID-19 REACH (P174119) and resided in three different geographical areas: (i) rural and peri-urban areas; (ii) provincial capital cities; and (iii) Kabul municipality. Wealthy households were to be excluded from receiving assistance. Inputs were financing of activities to generate a roster of beneficiaries, conduct of a communications campaign, training, procurement of food packages for distribution, preparing standard contracts with local traders and retailers. A “Targeting from the Top” approach without need for face-to-face interaction was to be used with pre-identified objective and subjective criteria to assess households to be excluded from the list of eligible beneficiaries Outputs were to consist of the list of eligible households, roll out of a communications campaign, hiring and training of staff, distribution of relief goods, in cash or in-kind, to beneficiary households. Designated Community Development Councils (CDCs), Gozar Assemblies (GAs) or Gozar Councils (GCs) and where these do not exist, Ad hoc COVID-19 CDCs, and Ad hoc COVID-19 GAs or GCs were to distribute the relief packages. The project was to contract Facilitating Partners (FPs) to support these community groups, where needed, to: (i) identify eligible beneficiary households; (ii) receive and manage Emergency Relief Grants in their bank accounts; (iii) procure the goods to be included in the in-kind relief package; (iv) distribute the relief packages; and (v) report to the implementing agencies. Distribution of relief packages will follow internationally accepted good practice regarding composition of a nutritious food basket (including carbohydrates, protein, and fat) and basic hygiene products. Outcomes were to avoid harmful, unsustainable coping strategies and maintain health of the beneficiaries. The project was expected to sustain private sector distribution networks made up of local small retailers and wholesalers. The outcomes were to be expressed as the number of households in those three different target areas, and reporting female-headed households separately. Because of the rapid response nature of this emergency operations, outcomes were limited to intermediate outcome level (beneficiaries) rather than those that would have resulted from longer term engagement, such as improved health of beneficiaries or sustaining private sector distribution networks. The following critical assumptions, which if actualized would increase the likelihood that the outcomes would be achieved: (i) community leaders adhere to project criteria for choosing households with special attention to female-headed households, agree on households to be targeted by the project, and will ensure that transfers are distributed fairly; (ii) adequate staff, including females can be hired and trained in a timely manner. (iii) food aid with acceptable quality can be procured and delivered in a short period; (iv) food items will be shared among all household members (including women, children, and the elderly). OUTPUTS: The following outputs were achieved in the three target areas  1,936 field staff or facilitators were trained in COVID-19 compliant and gender-sensitive distribution of transfers to households, substantially achieving the target of 2,000 staff members but only 16 percent of those trained were female, not achieving the target of 25 percent.  92 percent of grievances were reported to have been addressed (or 7,959 cases out of 8,664 reported). This exceeded the target of 70 percent.  360 communities were provided with support within 6 months of project effectiveness. This did not meet the target of 14,000 communities.  The government's COVID-19 communications campaign reached 185,000 households not meeting the target of reaching 2.8 million households. Page 6 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Afghanistan COVID-19 REACH (P174119)  No survey was conducted to find out the share of beneficiaries who would have expressed satisfaction with the support received, not meeting the target of 80 percent. OUTCOMES:  294,288 households receiving in-kind support in rural or peri-urban areas (or 17 percent of the original target of 1,760,000 households). This included 5,090 female-headed households or 3 percent of the target of 176,000 female-headed households. Many female-headed households were listed under the name of a male household member. A Third-Party Monitoring Agent (TPMA) conducted post- distribution door-to-door monitoring and reported that 99 percent of female-headed households received assistance. The project was expected to reach substantial progress, with a total of 3,739 communities mobilized (19 percent of the target), 689,330 beneficiary households identified (20 percent of the target), and grants disbursed to 2,583 communities (13 percent of the target).  179,872 households receiving in-kind or cash support in provincial capital cities (50 percent of the original target of 360,000 households) 1,072 (almost 1 percent of the target 36,000) were female- headed households. While the number of communities was smaller in the target cities, the average number of beneficiary households in each community was greater than initially estimated. Work in urban areas started after April 7, 2021 when the national budget included the urban budget. About half of the target value was met in about four months, another 22 percent of households were identified and waiting for assistance. The project was expected to achieve the target for the urban targets.  No households or female households in Kabul municipality received in-kind or cash support, not meeting the target of 500,000 households or 50,000 female headed households. However, 953 Ad- hoc GAs (108 percent of the initial estimate) had been mobilized, of which 862 (97 percent of the initial estimate) had submitted beneficiary household lists, and the FPs had verified the beneficiary lists for 854 of these Ad-hoc GAs (96 percent of the initial estimate) which consisted of 923,183 beneficiary households (184 percent of the target). The targets were achievable despite the delayed start and TPMA continuing to confirm the beneficiary lists. Overall, the efficacy of the project to achieve the objective before the collapse of the government is rated Substantial. There were some indications that targets would be achieved if the project was not suspended in the last four months of implementation. Two of the four output indicators were substantially achieved or exceeded, the rest were unmet. Rating Substantial OBJECTIVE 1 REVISION 1 Revised Objective The objective was not revised. Revised Rationale Revised Theory of Change: The TOC was not revised. The revisions refer to the efficacy of the project to achieve the objective over the entire implementation period but with substantial decrease in the capacity to achieve them because of the collapse of the government on August 15, 2021. In the last four months of the Page 7 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Afghanistan COVID-19 REACH (P174119) project, the government continued to deliver the relief packages to the target beneficiaries in the various categories (i.e., rural, urban and peri urban provincial capitals) and in the Kabul Municipality. The project closed as planned on December 31, 2021. The same inputs, outputs, and outcomes as the original above applied here as well. The assumption was that the project would continue according to its original implementation strategy, but this was not possible with the Taliban takeover of the government. REVISED OUTPUTS:  After August 15, 2021, the TPMA, NGOs, and implementing agencies reported that the distribution of assistance continued in the field once the fighting ended. For example, the TPMA monitored food distribution by several CDCs in Jalalabad in late September and October 2021. Field reports showed that some assistance was distributed post-August 15, 2021 but because of delayed reporting and submission of receipts because staff stopped working after the collapse of the government, reported figures were likely underestimated. REVISED OUTCOMES:  Households receiving in-kind support in rural or peri-urban areas remained at 17 percent of the target households by December 31, 2021. Post August 15, 2021, MRRD, NGOs, CDCs and the TPMA reported that limited food distribution continued but there were data gaps because of delays in data processing and reporting once NGO and government staff stopped being paid after project suspension. The project was expected to reach substantial progress, with a total of 3,739 communities mobilized (19 percent of the target), 689,330 beneficiary households identified (20 percent of the target), and grants disbursed to 2,583 communities (13 percent of the target). The TPMA conducted field monitoring sampling and confirmed the figures reported by MRRD and by IDLG. Almost all households visited by TPMA door-to-door included in a beneficiary list (97 percent, 128,538 out of 132,867) and most post-distribution respondents later reporting having received assistance (74 percent, 1,822 out of 2,467).  51 percent of target was achieved in the urban areas indicating a cumulative 88 percent of target was achieved but this was unverified.  No further achievements in Kabul Municipality. Overall, the efficacy of the project to achieve the objective in the four months following the government collapse and suspension of the project is rated Modest. There were some indications that around 18 percent of targets may have been cumulatively achieved, but additional progress was largely unverified. Revised Rating Modest OVERALL EFF TBL OBJ_TBL OVERALL EFFICACY Rationale Page 8 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Afghanistan COVID-19 REACH (P174119) The overall efficacy of the project to achieve the objective is rated Substantial with moderate shortcomings. The period covered up to the government collapse and suspension of disbursements showed that the targets were reasonably expected to be achieved and the existing project platforms were exceptionally efficient in getting relief out to households. Overall Efficacy Rating Substantial OBJR1_TBL OVERALL EFFICACY REVISION 1 Overall Efficacy Revision 1 Rationale With little additional achievements in the four months following the government collapse, the overall efficacy of the revised project to achieve the objectives is rated Modest. According to the TPMA's Final Project Report, almost all beneficiaries received the same quantity of assistance throughout the program (1,790 out of 1,795), with 76 percent of CDCs in REACH areas distributing goods with no visible defects (766 out of 1,003). Overall Efficacy Revision 1 Rating Primary Reason Modest External shock 5. Efficiency Economic Efficiency: No economic efficiency analysis was conducted at appraisal or at closing. As an emergency response, relief packages were to be provided to all eligible households in the target areas across the country. The project activities involved relief support consisting of essential food staples and hygiene products and cash (mobile money, vouchers, or sim cards) delivered to rural and urban households. The preferred door-to-door distribution of relief packages was to ensure social distancing. Designated collection points observing social distancing would also be established. The extent, complexity, and uncertainty of the pandemic did not lend itself to quantifying the economic impacts of COVID-19 and of the emergency response. A qualitative assessment was applied. Benefits in the form of emergency transfers under this project was to support around 3.3 million households (1.1 million, urban and 2.2 million, rural). The emergency relief transfers to eligible households (whether in in-kind or cash) justified the social and economic rationale for the project because Afghanistan lacked formal safety net structures. These transfers were deemed a viable “immediate relief” measure to support the government's response to the spread of COVID-19, i.e., social distancing. This assistance was to partly meet the immediate cash and food needs of poor and vulnerable households (PAD, paragraph 61). The cost of the relief packages was a combination of in-kind and cash to be distributed to eligible households. Relief package for rural areas was worth at AFN4,000 (US$50) while those for urban areas was worth AFN8,000 (about. US$100) distributed in two tranches. The first tranche of AFN4,000 per household was in kind relief packages and contained essential food staples and hygiene products equivalent to 50 percent of monthly food ration of an average Afghan house. Universal coverage was ensured by excluding affluent households from the Page 9 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Afghanistan COVID-19 REACH (P174119) list of beneficiaries. The grant disbursed 63 percent of the funds over a 75 percent implementation period (12 of 16 months) and achieved 18 percent of targets before the August 15, 2021 suspension. The project was expected to achieve targets, especially in rural and urban areas outside of Kabul where 474,160 poor households or about 3.3 million Afghans had received immediate food and in-kind assistance, including 6,162 vulnerable female heads of household (see Section 4 Efficacy above). Another US$5 million was paid directly to Facilitating Partners after August 15, 2021 and included in the overall disbursement. The project shared operational costs with CCAP, spent an estimated 11 percent of total project cost, achieving a cost-effectiveness ratio within the range experienced in Afghanistan (about 19 percent of the project costs). Administrative and Operational Efficiency: The project used three implementing agencies the Ministry of Rural Rehabilitation and Development (MRRD) who was implementing CCAP; the Independent Directorate of Local Governance (IDLG) who was implementing the EZ-Kar project, and the Kabul Municipality who was implementing the Kabul Municipal Development Program (KMDP). These agencies used Community Development Councils (CDCs) and Gozar Assemblies to identify beneficiaries and serve as distribution points to deliver one- or two-tranche relief packages (in-kind or cash or a combination of the two) to target beneficiary households. Experienced non-governmental organizations (NGOs) were used as facilitating partners (FPs). The following factors led to initial implementation delays. First, social distancing meant smaller groups, rather than large gatherings would be used to identify beneficiary households, distribute, and monitor relief packages. More efficient modalities using mass assemblies were avoided to register beneficiaries, collect cash at a commercial bank or money agents, or exchange vouchers at specific shops. Second, certain political events occurred in late August 2020. Parliament rejected the government-initiated Dastarkhan-e-Milli program, the basis for this project, because several projects were cancelled to fund it; and there was fear of misuse. Third, some date-sensitive budgetary processes and deadlines were not met that cascaded to delays in allocating and releasing of funds for operational needs of implementing agencies. Fourth, high turnover and uncertain contracts for key staff of implementing units were experienced early on. However, the use of existing implementation arrangements helped implementation to proceed once funds were released except in Kabul Municipality. KMDP was not part of CCAP and inexperienced in the implementation arrangement that used FPs, CDCs, GAs, This project and CCAP reported and addressed 494 alert notices on household exclusion or inclusion of ineligible households, and 453 red flags on procurement failures, lack of COVID-19 protective measures, unverified households, civil unrests, and cash or good diversion (ICR, footnote 45). Overall, the efficiency of the project is rated Substantial. The project was expected to reach its expected outcome even as the implementation period was truncated by 4 months (See Section 4, Efficacy above). Efficiency Rating Substantial a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 0 Appraisal 0  Not Applicable Page 10 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Afghanistan COVID-19 REACH (P174119) 0 ICR Estimate 0  Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome Until August 15,2021 Post August 15. 2021 Relevance of Objective High Efficacy PDO - To provide emergency support to selected households (1) in Rural and Peri Urban Areas S M (2) in Provincial Capital Cities S M (3) in Kabul Municipality N N Overall Efficacy S M Efficiency Substantial Outcome MS MU Numerical Value of Outcome 4 3 Disbursement (cumulative, in US$ million) 175.9 280 Disbursement (per period covered) 175.9 104.1 Share of Disbursement (in percent) 63 37 Numerical value of outcome x share of disbursement 4 x 0.63 3 x 0.37 Weighted Value of Outcome 2.52 1.11 Final Outcome 2.52 + 1.11 = 3.63 ~4 = MS a. Outcome Rating Moderately Satisfactory 7. Risk to Development Outcome The fragile and conflict affected (FCV) environment in Afghanistan elevated the following high risks for the limited project outcomes:  Political risk. In this project, the international community suspended its aid and security assistance after the August 15, 2021 Taliban takeover of the government. Government services were halted. Aggregate demand fell. The exchange rate depreciated. Linkages to the international financial system was disrupted. Afghanistan lost access to international reserves. A financial crisis ensued. Page 11 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Afghanistan COVID-19 REACH (P174119) Aid used as hard currency to finance imports (such as electricity, food, fuel, and medical supplies) dried up. Trade deficit rose to around 35 percent of gross domestic product (GDP). Fiscal contraction and disruptions to private sector activity were expected to reduce economic output over the year from August 15, 2021 by as much as 30 percent. The political situation remains uncertain. Improvements initiated under the project may not survive uncertain political structures that the Taliban.  Institutional risk. In this project, the institutional arrangements that implementing agencies used to reach poor households may not survive as a social safety net mechanism. The agencies had used experienced non-governmental organizations (NGOs) as facilitating partners (FPs), and mobilized community organizations (CDCs, GAs, and ad-hoc CDCs/GAs) to rollout and implement the project. The international community continues to use off budget financing of development partners to deliver humanitarian assistance using and sidestepping the Interim Taliban administration. Another institutional arrangement that may not survive the present administration was the multi- layered M&E system used to monitor this project's progress and results with the involvement of community leaders (Mullahs/imams, district (nahia) representatives, teachers, youths), FPs, local governments, and the PMU.  Security risk. In this project, both the government and donors wanted to cover the entire country, including 47 extremely insecure districts that were not under the government's control, to avoid increasing social tension. The project experienced a high level of violence and criminal activities that posed security risks to project staff, FPs, service providers, and beneficiaries. As security deteriorated, and the Taliban took over the government, it remains unclear how the new interim Taliban administration will address security and access to donor financed activities.  Extreme poverty and Inflation risk. In this project, REACH beneficiary households may have avoided the initial impact of the inflationary shock from the increase in basic food prices caused by scarcity. Annual inflation for a package of basic household goods is now around 40 percent. More than two-thirds of households are unable to cover basic food and non-food needs. About one-third of households are unable to cover food needs. Harmful coping mechanisms have emerged - such as borrowing at high interest rates, consumption or sale of assets, or reduced investments in human capital. Long-term consequences of these practices may lead to furthering poverty. The international community is implementing the blueprint under Approach Paper 2.0 to continue to assist the country's poor. 8. Assessment of Bank Performance a. Quality-at-Entry The Bank team identified the project as an emergency response to COVID-19 with reasonable and achievable objective and outcomes designed for rapid implementation. Informed by lessons from previous and ongoing CDD operations in the country, the design replicated the implementation arrangements of the ongoing CCAP. This was to allow the rapid roll out of the project since the implementing entities and target participants were familiar with Bank policies and procedures. However, Kabul municipality was not familiar with the CCAP arrangements. This led to implementation delays. Poverty, gender, and social development needs were adequately addressed. For example, “targeting from the top” provided processes to exclude unqualified households from receiving relief packages. The Environmental and Social Assessment adequately identified key risks and prepared measures to mitigate Page 12 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Afghanistan COVID-19 REACH (P174119) these in the Environmental and Social Commitment Plan (see Section 10 Other Issues below). The team devised a financial management system that assured rapid flow of funds. Simplified community procurement procedures were to be used. The M&E system was adequately designed (see Section 9 M&E below). Key risks, such as those related to social exclusion, corruption, lack of security, and the spread of COVID-19 were adequately assessed and matched with mitigating measures. Additional COVID-19 specific measures such as social distancing, crowd management at distribution points, and security protocols formed part of design. A stakeholder engagement plan featured outreach, consultation, and grievance redress that provided beneficiaries with transparency and accountability. Overall, quality at entry is rated Satisfactory with minor shortcomings because of the capacity of Kabul Municipality to implement the project as designed was not adequately assessed at entry. This led to in implementation delays in this target area (see below). Quality-at-Entry Rating Satisfactory b. Quality of supervision The Bank team proactively addressed challenges to achieving outcomes and was diligent in its fiduciary role. For example, the team advised on maintaining the integrity of beneficiary rosters, from guiding Kabul Municipality to correct deficiencies, to advising urban communities on coping with an influx of internally displaces persons (IDPs). The team recommended hiring facilitating partners to implement activities where operational staff were unavailable. At crucial times, the team maintained daily virtual communication with those leading the government’s COVID-19 response. Bottlenecks in financial flows were escalated to expedite resolution. In addition to the 2 supervision missions over the course of the 16-month implementation period, virtual missions were conducted to ensure delivery of relief to beneficiaries in the face of additional COVID-19 restrictions, and escalating insecurity. The team fulfilled its fiduciary role with the help of a TPMA who reported weekly on key aspects and raised red-flag alerts that implementing agencies would rectify. Overall, the Bank performance at Supervision is rated Satisfactory. There were issues beyond the Bank team’s control, including the Taliban takeover, with the TPMA and CDCs that continued to be active to deliver the project as planned. Overall, the Bank performance is rated Satisfactory. Quality at entry is rated Satisfactory with minor shortcomings in assessing the capacity of the Kabul Municipality to use the unfamiliar CCAP features of engaging beneficiaries. Supervision is rated Satisfactory. Quality of Supervision Rating Satisfactory Overall Bank Performance Rating Page 13 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Afghanistan COVID-19 REACH (P174119) Satisfactory 9. M&E Design, Implementation, & Utilization a. M&E Design The M&E design was sound. The objective was clearly stated. The emergency nature of the project was reflected in the Theory of Change that provided the causal link between the inputs (financing) outputs (relief packages) and simplified outcome indicator (households supported) to achieve the PDO. This indicator encompassed the target areas with reasonable, achievable targets at the rural, peri-urban, urban, provincial capital and Kabul municipal levels. Intermediate results indicators were adequate to capture the contribution of the operation’s four components and outputs toward achieving PDO-level outcomes. The indicators were specific, measurable, achievable, relevant, and time bound. Baselines and targets were available for all indicators. Data collection methods were to be undertaken at the community and neighborhood levels. The community leaders (Mullahs/imams, nahia representatives, teachers, youths), FPs, local governments, and PIUs were to use pre-developed forms to validate beneficiaries and manage distribution of packages. A Third Party Monitoring Agent (TPMA) was to be engaged and the TPMA (both in-person and by phone) would conduct spot-checks and case-specific inspections. An MIS was built on that of CCAP, with Kobo-toolbox applications to collect field-data through portable devices. Sampling was to be undertaken by conducting periodic phone surveys. b. M&E Implementation The Independent Directorate of Local Governance (IDLG), Kabul Municipality, (KM) and the Ministry of Rural Rehabilitation and Development (MRRD) implemented the M&E system, embedded in the Operations Manual, including forms that had to be filled, entered in the MIS, and approved by designated parties. The M&E system tracked implementation status, such as community mobilization, beneficiary identification, grant disbursement, procurement of goods to be included in the assistance packages, and distribution of the packages. Correct processing led to next steps. The TPMA implemented monitoring activities, submitted weekly progress reports, and provided red flag alerts, and their status. Red flags were issues such as unverifiable households, procurement failures, non-adherence to COVID-19 precautionary measures, missing data on grievance reporting. Regular daily or weekly meetings were conducted among the Bank team, the TPMA, and the implementing agencies. c. M&E Utilization The Bank team and the Project Implementation Units (PIUs) used the M&E, MIS, and TPMA generated data and reports to track progress, movement of populations, and adequacy of funds for relief packages by target area. Red-flag alerts were addressed by the PIU with Bank team follow-up. Data reviews that detected inconsistencies in beneficiary lists led to its correction before release of assistance grants. The M&E data was used to verify: (i) beneficiary lists; (ii) distribution of assistance to beneficiaries; and (iii) compliance to applicable operational policies and procedures. Page 14 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Afghanistan COVID-19 REACH (P174119) The overall rating of Quality of M&E is rated “High.” The M&E design was adequate to meet the needs of the project. Data was collected as planned at multiple levels. The M&E data was used to address issues as they emerged and informed corrective measures as needed. M&E Quality Rating High 10. Other Issues a. Safeguards Environmental and Social Safeguards: An Environmental and Social Assessment identified the following five Environmental and Social Standards (ESS) triggered by the project: ESS 1, Assessment and Management of Environmental and Social Risks and Impacts Relevant; ESS 2, Labor and Working Conditions Relevant; ESS 3, Resource Efficiency and Pollution Prevention and Management; ESS 4, Community Health and Safety; and ESS 10, Stakeholder Engagement and Information Disclosure. The Environmental and Social Commitment Plan addressed key occupational health and safety risk. A grievance redress mechanism (GRM) and a hotline were also established to enable project-affected people to a raise concerns and grievances as well as to request clarifications on various aspects of the project and its potential impacts (ICR, paragraph 62). The project benefited from the familiarity of the core staff and FPs with the Bank’s policies and procedures related to environmental and social framework (ICR, paragraph 51). During project implementation, the Stakeholder Engagement Plan (SEP) and the Environmental and Social Management Plan (ESMP) were updated with some delay. The performance ratings for the five ESS were Moderately Satisfactory (ICR, paragraph 59). There were breaches of COVID-19 prevention measures during the distribution of relief assistance packages as reported by the TPMA. Visual communications materials were redeployed to address these instances. b. Fiduciary Compliance . Financial Management: The Bank team helped to establish financial management that enabled swift flow of funds to communities upon release of the funds by the Ministry of Finance. The project benefited from the the familiarity of the core staff and FPs with the Bank’s policies and procedures related to financial management (ICR, paragraph 51). The Bank team rated the project financial management performance Moderately Satisfactory throughout because the implementing agencies were delayed in completing some of the agreed actions, such as the submission of the quarterly financial reports and cash flow projections. The signing of the Memorandum of Understanding (MoU) with Da Afghanistan Bank to set up bank accounts for the GAs in Kabul was also delayed. Monthly financial management (FM) check-in meetings were conducted. There is an outstanding (unaccounted) amount of approximately US$165 million in the designated accounts of the project. It remains unclear if those funds have been disbursed for assistance post-August 15 (ICR, paragraph 60). Procurement: The project benefited from the the familiarity of the core staff and FPs with the Bank’s policies and procedures related to procurement (ICR, paragraph 51). Community procurement methods Page 15 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Afghanistan COVID-19 REACH (P174119) were applied to the procurement of food and other essential goods. No major misuse was reported or observed. The project contracted Facilitating Partners (FPs) to support community groups to: (i) identify eligible beneficiary households; (ii) receive and manage Emergency Relief Grants in their bank accounts; (iii) procure the goods to be included in the in-kind relief package; (iv) distribute the relief packages; and (v) report to the implementing agencies. The TPMA reported that a limited number of CDCs had both the Bid Quotation and the Purchase Order forms available during distribution. However, after distribution the financial review of statements of expenditure showed that relevant documents were available at government offices in the majority of cases. The TPMA reported that the quality and quantities of the procurement items were aligned to the approved procurement plans. CCAP and this project reported 453 red flags on procurement failures, lack of COVID-19 protective measures, unverified households, civil unrests, and cash or good diversion (ICR, footnote 45) c. Unintended impacts (Positive or Negative) --- d. Other --- 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment Moderately Outcome Moderately Satisfactory Satisfactory Bank Performance Satisfactory Satisfactory Quality of M&E High High Quality of ICR --- High 12. Lessons The ICR provided four lessons from the operation, slightly rephrased below (ICR, paragraphs 67- 70):  Using existing community structures facilitates rapid identification of beneficiaries and distribution of assistance. In this project, community level structures used by previous projects to mobilize the community and deliver services facilitated rapid, more accurate, and transparent identification of beneficiary households. Decentralized data was collected more efficiently and accurately. According to the TPMA, the community driven approach helped to minimize conflict and tensions. Community leaders facilitated the inclusion of households that may not have been included otherwise. Community-based procurement and distribution modalities helped to expedite the distribution of assistance. FPs helped communities distribute assistance on their own, rather than waiting for the local authorities or Page 16 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Afghanistan COVID-19 REACH (P174119) the PIU. CDCs’ existence over the past 18 years and their continuation even after the political crisis in August 2021 are evidence of its resilience.  Simplified beneficiary identification may reduce operation costs and exclusion errors. In this project, all households with incomes of US$2 per day or lower, covered an estimated 90 percent of all households. The project “Targeted from the Top” to efficiently exclude wealthy households who do not require support using a set of visibly assessable and verifiable criteria, rather than the eligibility criteria used to identify eligible households. About 10 percent of households were excluded but given an opportunity to present their case if they were assessed in error. This process (i) minimized social tensions from the grievances of excluded needy households; (ii) reduced costs and time to identify the beneficiary households; and (iii) reduced exclusion errors if using eligibility criteria of needy households.  Door-to-door delivery may help reach female-headed households. In this project, delivering assistance to almost all female-headed households was achieved. The FPs and communities organized door-to-door delivery for female-headed households according to social norms. Available female staff was limited, women’s mobility was restricted, and safe and convenient place and time cannot be made available for women to pick-up their assistance. The TPMA reported that in areas they monitored, door-to-door delivery method reached 99 percent of female-headed households.  Partnering with the humanitarian community may optimize humanitarian assistance. In this project, close communications with the humanitarian community helped to coordinate the content, modality, coverage, and the timing of assistance. This led to a framework for cooperation and coordination. The NGOs as FPs played important roles in both humanitarian and development assistance. They shared information on risks and operational viability of the assistance, enhanced implementation, and risk mitigation measures. NGOs familiar with the unique operational context of KM during the design phase may have improved the delivery design of the project. 13. Assessment Recommended? No 14. Comments on Quality of ICR The ICR was prepared to capture the lessons of the truncated emergency operation. The ICR followed the guidelines. Together with the annexes, the report presented a clear narrative of how results were achieved. The report was internally consistent and mutually reinforcing. Analysis provided a clear narrative of the operation. The evidence supported the ratings. Lessons were derived from the operation. The report provided a candid assessment of challenges the operation faced, and the efforts made by all parties, both the Bank and the government, to ensure that rapid implementation matched the emergency nature of the project. The quality of the ICR is rated High. a. Quality of ICR Rating Page 17 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Afghanistan COVID-19 REACH (P174119) High Page 18 of 18