Document of The World Bank FOR OFFICIAL USE ONLY Report No: ICR00004922 IMPLEMENTATION COMPLETION AND RESULTS REPORT ON A CREDIT IN THE AMOUNT OF SDR 39.1 MILLION (US$60 MILLION EQUIVALENT) TO THE SOCIALIST REPUBLIC OF VIETNAM FOR A SOCIAL ASSISTANCE SYSTEM STRENGTHENING PROJECT June 30, 2020 Social Protection and Jobs Global Practice East Asia And Pacific Region This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed without World Bank authorization. CURRENCY EQUIVALENTS (Exchange Rate Effective December 31, 2012) Currency Unit = Vietnamese Dong VND 20,839.85 = US$1 US$1 = SDR 0.65 FISCAL YEAR January 1 – December 31 ABBREVIATIONS AND ACRONYMS BFS Beneficiary Feedback Survey C4D Communication for Development CEMA Central Committee for Ethnic Minority Affairs CLR Completion and Learning Report CPF Country Partnership Framework CPMU Central Project Management Unit CPS Country Partnership Strategy DFID Department for International Development DivLISA Division of Labor, Invalids, and Social Affairs DOLISA Department of Labor, Invalids, and Social Affairs EMDP Ethnic Minorities Development Plan FM Financial Management GoV Government of Vietnam ICR Implementation Completion and Result Report IT Information Technology M&E Monitoring and Evaluation MIS Management Information System MOET Ministry of Education and Training MOF Ministry of Finance MOLISA Ministry of Labor, Invalids, and Social Affairs MTR Midterm Review PAD Project Appraisal Document PDO Project Development Objective PIM Project Implementation Manual PPMU Provincial Project Management Unit PPC Provincial People’s Committee SA Social Assistance SASSP Social Assistance System Strengthening Project TA Technical Assistance VNPost Vietnam Post Regional Vice President: Victoria Kwakwa Country Director: Ousmane Dione Regional Director: Daniel Dulitzky Practice Manager: Philip O'Keefe Task Team Leader: Nga Nguyet Nguyen ICR Main Contributor: Jorge Avalos TABLE OF CONTENTS DATA SHEET ....................................................................... ERROR! BOOKMARK NOT DEFINED. I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES ....................................................... 5 A. CONTEXT AT APPRAISAL .........................................................................................................5 B. SIGNIFICANT CHANGES DURING IMPLEMENTATION (IF APPLICABLE) ..................................... 13 II. OUTCOME .................................................................................................................... 16 A. RELEVANCE OF PDOs ............................................................................................................ 16 B. ACHIEVEMENT OF PDOs (EFFICACY) ...................................................................................... 17 C. EFFICIENCY ........................................................................................................................... 24 D. JUSTIFICATION OF OVERALL OUTCOME RATING .................................................................... 26 E. OTHER OUTCOMES AND IMPACTS (IF ANY) ............................................................................ 27 III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME ................................ 28 A. KEY FACTORS DURING PREPARATION ................................................................................... 28 B. KEY FACTORS DURING IMPLEMENTATION ............................................................................. 29 IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME .. 30 A. QUALITY OF MONITORING AND EVALUATION (M&E) ............................................................ 30 B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE ..................................................... 31 C. BANK PERFORMANCE ........................................................................................................... 33 D. RISK TO DEVELOPMENT OUTCOME ....................................................................................... 35 V. LESSONS AND RECOMMENDATIONS ............................................................................. 36 ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS ........................................................... 39 ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION ......................... 45 ANNEX 3. PROJECT COST BY COMPONENT ........................................................................... 47 ANNEX 4. EFFICIENCY ANALYSIS ........................................................................................... 48 ANNEX 5. BORROWER, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS ... 50 ANNEX 6. SUPPORTING DOCUMENTS (IF ANY) .................... ERROR! BOOKMARK NOT DEFINED. The World Bank Social Assistance System Strengthening Project (P123960) DATA SHEET BASIC INFORMATION Product Information Project ID Project Name P123960 Social Assistance System Strengthening Project Country Financing Instrument Vietnam Investment Project Financing Original EA Category Revised EA Category Not Required (C) Not Required (C) Organizations Borrower Implementing Agency Socialist Republic of Vietnam Ministry of Labor, Invalids and Social Affairs Project Development Objective (PDO) Original PDO The project development objective is to support the government of Vietnam in strengthening the social assistance system by developing innovations in management and service delivery nationwide, and by piloting these innovations in the four project provinces. Page 1 of 52 The World Bank Social Assistance System Strengthening Project (P123960) FINANCING Original Amount (US$) Revised Amount (US$) Actual Disbursed (US$) World Bank Financing 60,000,000 32,981,691 27,103,925 IDA-53540 Total 60,000,000 32,981,691 27,103,925 Non-World Bank Financing 0 0 0 Borrower/Recipient 2,500,000 0 1,000,000 Total 2,500,000 0 1,000,000 Total Project Cost 62,500,000 32,981,691 28,103,925 KEY DATES Approval Effectiveness MTR Review Original Closing Actual Closing 22-Jan-2014 22-Jul-2014 21-Nov-2016 31-Dec-2019 31-Dec-2019 RESTRUCTURING AND/OR ADDITIONAL FINANCING Date(s) Amount Disbursed (US$M) Key Revisions 05-Sep-2017 23.76 Change in Results Framework 04-Sep-2019 30.60 Change in Components and Cost Cancellation of Financing Reallocation between Disbursement Categories 01-Jan-2020 27.50 Change in Components and Cost Cancellation of Financing Reallocation between Disbursement Categories KEY RATINGS Outcome Bank Performance M&E Quality Moderately Satisfactory Satisfactory Substantial Page 2 of 52 The World Bank Social Assistance System Strengthening Project (P123960) RATINGS OF PROJECT PERFORMANCE IN ISRs Actual No. Date ISR Archived DO Rating IP Rating Disbursements (US$M) 01 07-Jul-2014 Satisfactory Satisfactory 0 02 29-Dec-2014 Satisfactory Satisfactory 1.02 03 10-Mar-2015 Satisfactory Satisfactory 1.02 04 03-Dec-2015 Satisfactory Satisfactory 6.03 05 28-Jun-2016 Satisfactory Satisfactory 12.61 06 28-Dec-2016 Satisfactory Satisfactory 18.51 07 29-Jun-2017 Satisfactory Satisfactory 21.42 08 28-Dec-2017 Satisfactory Satisfactory 26.11 09 22-Jun-2018 Moderately Satisfactory Moderately Satisfactory 28.51 10 03-Dec-2018 Moderately Satisfactory Moderately Satisfactory 29.60 11 13-Jun-2019 Moderately Satisfactory Moderately Satisfactory 30.60 12 03-Mar-2020 Moderately Satisfactory Moderately Satisfactory 27.50 SECTORS AND THEMES Sectors Major Sector/Sector (%) Public Administration 5 Other Public Administration 5 Social Protection 95 Social Protection 95 Themes Major Theme/ Theme (Level 2)/ Theme (Level 3) (%) Page 3 of 52 The World Bank Social Assistance System Strengthening Project (P123960) Social Development and Protection 100 Social Protection 100 Social Safety Nets 53 Social protection delivery systems 47 ADM STAFF Role At Approval At ICR Regional Vice President: Axel van Trotsenburg Victoria Kwakwa Country Director: Victoria Kwakwa Ousmane Dione Director: Xiaoqing Yu Daniel Dulitzky Practice Manager: Jehan Arulpragasam Philip B. O'Keefe Puja Vasudeva Dutta, Christian Task Team Leader(s): Nga Nguyet Nguyen Bodewig ICR Contributing Author: Jorge Avalos Page 4 of 52 The World Bank Social Assistance System Strengthening Project (P123960) I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES A. CONTEXT AT APPRAISAL Context 1. Despite Vietnam’s overall progress in poverty reduction, poverty has remained concentrated among ethnic minorities living in isolated areas and households with low human capital. In 2012, during project preparation, ethnic minority households and households with low education and skills faced chronic poverty and had not benefited as fully as other groups from the economic growth in the 1990s. Education, health, and nutrition outcomes remained significantly lower than the overall population for poor and ethnic minority households, perpetuating intergenerational poverty. Vietnam’s secondary school enrollment rates were low and stunting among children under 5 years was 23 percent in 2010, with massive variations observed between top and bottom quintiles. The multidimensional child poverty rate was estimated at around 30 percent, or 7 million children, overall, and 60 percent among ethnic minority groups.1 2. At project inception, the social assistance (SA) system had important policy and implementation gaps that hampered long-term poverty and vulnerability reduction. Vietnam’s SA strategy, policies, and programs had transitioned from emphasizing in-kind and geographically targeted programs to cash and near-cash programs targeted to households. Over the past 20 years, programs were added or replaced the old ones depending on the need and budget capacity without following a comprehensive and long- term vision. Despite many cash transfer programs, there was lack of adequate coverage of some vulnerable groups such as children under 3, pregnant women from poor households, and low benefit levels.2 The Government had a policy to provide education allowances to children ages 3–15 years conditioned on school attendance. In terms of delivery, the SA programs suffered from inefficiency in several ways: (a) Fragmentation of social assistance programs (around 14 cash transfer programs in 2012, see Table 1) with overlapping beneficiary populations, business processes (targeting, enrollment, beneficiary management, payment, monitoring and evaluation [M&E], and so on), with each having its own budgeting and delivery mechanisms. (b) Weak M&E system, which mainly relied on administrative data and periodic reports, and surveys which were rarely used for management and policy decision-making. (c) Beneficiary data not being digitalized systematically and not being managed centrally. For each program, each province had its own plan to develop its databases and management information systems (MISs), and the progress was uneven between programs and provinces. Page 5 of 52 The World Bank Social Assistance System Strengthening Project (P123960) (d) Lack of institutional structures at the local level to link beneficiaries to supply-side health, education, and poverty reduction services and provide support through case management and counseling services. Table 1. SA Programs/Interventions, 2011 No. Names of the Policies/Programs Payment Management Number of Modality Agency Beneficiaries (people) Decree No. 67/13 on regular SA cash support to 9 Cash/monthly DOLISA 23,923 1 groups of disadvanataged people Decision No.112/2007/QD-TTg and Decision Cash/semiannual CEM 15,305 2 101/2009/QD-TTg on education support for poor and near-poor ethnic minority households’ children Decree No. 49/2010/ND-CP on tuition fee exemption Cash/semiannual DOLISA/DOET 29,598 for children in kindergartens, general schools in 3 mountainous and remote areas or orphans, and children with disabilities or in poor households. Decision No. 239/2010/QD-TTg to support full-day Cash/monthly DOET — 4 kindergarten school attendance at age 5 Decision No. 167/2008/QD-TTg on housing support Cash/once DOLISA — 5 for poor homeless/ temporary domicile households Decision No. 1592/QD-TTg on housing, production Cash and in-kind CEMA — 6 land and clean water assistance for EM households Decision No. 81/2009/QĐ-TTg providing support to Cash/once CPC — 7 the poor during Tet holiday Decision No. 268/2011/QD-TTg on electricity subsidy Cash/quarterly DOLISA 58,158 8 for poor households Decision No. 289/2008/QD-TTg on kerosene subsidy Cash/quarterly DOIT — 9 for EM poor households; Decision No. 471/QĐ-TTg on emergency support to Cash/once DOLISA 58,096 10 poor and near-poor HHs for unexpected serious problems Decision No. 1956/2009/QD-TTg on vocational Cash/semiannual DOLISA 13,173 11 training support for rural workers Decision No. 85/2010/QĐ-TTg, Support the semi- Cash/quarterly DOET — 12 boarding students in the most disadvantaged areas for food and accommodation Decree No. 74/2008/QĐ-TTg, Land allocation and Cash and in-kind PPC 12,756 13 grant to the minority HHs without land in Mekong Delta Decree No. 02/2002/ND-CP and Decision No. Cash and in-kind DOLISA 188,808 102/2009/QD-TTg on subsidies for commodities and 14 direct assistance for people in areas with economic difficulties Note: CEMA = Central Committee for Ethnic Minority Affairs; CPC = City People’s Committee; DOET = Department of Education and Training; DOLISA =Department of Labor, Invalids, and Social Affairs; DOIT = Department of Information and Technology; HHs = Households; PPC = Provincial People’s Committee. Page 6 of 52 The World Bank Social Assistance System Strengthening Project (P123960) 3. Addressing program fragmentation and absence of an integrated beneficiary registry were key to improving the SA system. Program fragmentation created considerable inefficiencies both from the perspective of the implementers and beneficiaries. At appraisal, there were more than a dozen cash transfer programs (Table 1). These included most prominently: (a) monthly cash support to the disabled, orphans, single parents, and lone elderly without means under Decree No. 136; (b) a monthly cash transfer to poor households to offset increases in energy prices under Decision No. 268; (c) a cash transfer to children from poor households in school for nine months per year delivered through the education system under Decree No. 49; and (d) a cash transfer program to support ethnic minority and poor Kinh secondary school students in dormitories under Decree No. 12. These were managed by various government agencies (Ministry of Labor, Invalids, and Social Affairs [MOLISA], Ministry of Education and Training [MOET], CEMA, subnational PPCs, and so on) without any form of coordination. Management, monitoring, and payment delivery arrangements for these programs were inadequate and varied between programs and management agencies. The absence of a national database (that is, beneficiary registry) used to track and manage SA beneficiaries resulted in duplication of efforts by government agencies in developing separate beneficiary lists for each program. Within MOLISA, there were around 40 MISs developed for various programs, at the central and subnational level. While monitoring and oversight at the local (commune) level were strong, provincial- and central-level mechanisms for bottom-up information management were underdeveloped because of the absence of an MIS. It was very common that the local governments (district and commune) maintained various lists of beneficiaries (poor households for example) used for different policies and programs. 4. Payments were usually made by local commune officials or local agents (often local retirees) visiting beneficiaries and in some cases by schools. Cash withdrawals were made at the district level (by DivLISA) and then distributed and handed over to commune officers to pay the beneficiaries. For some education support programs, DivLISAs transferred cash to schools to pay the beneficiary households. Conflicts of interest and mismanagement of beneficiary lists were common issues. Payment delivery also varied a lot across programs from monthly to quarterly to semiannual. For education programs, cash allowances were given to students twice a year following semester cycles. Most often, schools got funding based on a request list made at the beginning of the payment cycle and payment was made at the end of cycle. Surplus funding, because some students were not available when the payments were being made, was not returned to the program administrators. The program management M&E therefore was a serious problem and reporting was often done relying on administrative data. 5. These payment practices created a huge problem for both local staff and beneficiaries. It was not safe to bring cash from the district to commune level (subject to risks of robbery and misuse) and funds were often kept at the homes of officials when they could not be disbursed to all beneficiaries. For both local staff and beneficiaries, multiple visits to the commune center to receive benefits, when households were entitled to several programs, were costly both in time and labor and also took more time for local officials to issue cash payments and do reconciliations. The payments for SA programs were manual and involved periodic commune-level gatherings to collect cash at the relevant payment points (primarily post offices). Finally, the payment of social protection programs relied on their own infrastructure and in isolation from national payments and other financial service providers’ infrastructure. 6. By project appraisal in 2013, the Government of Vietnam (GoV) had placed an increased emphasis on strengthening the social protection system in response to these challenges. This was Page 7 of 52 The World Bank Social Assistance System Strengthening Project (P123960) evident in the approval of Resolution 15 on Social Protection Development (2012–2020) by the Central Committee of the Communist Party in May 2012, which included the development of SA into a guaranteed minimum income program and further development of its social insurance scheme. Resolution 15 highlighted the need to consolidate and modernize the management of programs and policies, including a modern payment delivery and integrated beneficiary database, to ensure less fragmentation and overlap of programs. 7. The Social Assistance System Strengthening Project (SASSP) sought to strengthen Vietnam’s SA system by: (a) building a national beneficiary database (comprising poor and near-poor households and SA beneficiaries) and an MIS to manage and track beneficiaries and payments; (b) aligning SA programs and reducing fragmentation by supporting the consolidation of three cash transfer programs into the Opportunity Program in the four project provinces; and (c) improving service delivery by introducing a new payment mechanism through an independent payment provider, systematic communication and monitoring, and a social collaborator network to provide information at the commune level and encourage households to invest in their children’s education, health, and nutrition in the four project provinces. The SASSP is also piloting expansion of coverage of the Decree No. 136 programs to support three new groups of beneficiaries: pregnant women, children ages 0–3 years, and children ages 3–15 years not attending school from poor households (referred to as ‘SASSP additional beneficiaries’). Thus, the project aimed to expand the SA coverage of poor households who were not served by SA programs in 2012, with a focus on early years and human capital development. The project funded benefits for these additional beneficiaries while the Government continued to pay for the existing program beneficiaries. The lessons from piloting the project innovations in the four project provinces, taken as a whole, were intended to inform program and administrative consolidation nationwide over time. The SASSP supported MOLISA, the main project implementation agency, to implement Resolution 15 in developing solutions for SA modernization and consolidation of programs and processes. Theory of Change (Results Chain) 8. The project was designed to strengthen Vietnam’s SA system by improving the management and service delivery of SA programs and piloting innovations in the four project provinces to create evidence and draw lessons for formulating policies to apply these innovations nationwide. To achieve this impact, the project design was intended to support achievement of a set of major outputs, including: (a) a beneficiary registry and an MIS (POSASoft) to better manage SA and poverty reduction programs; (b) investments to improve delivery of consolidated programs in the four project provinces; and (c) assessment of the implementation of the pilot to draw lessons and recommendations for program and administrative consolidation nationwide. The main intended outcome was an improved delivery system for SA programs in the four piloted provinces and evidence plus recommendations for program consolidation and management improvement nationwide, as shown in Figure 1 in the project’s Theory of Change. 9. The core set of activities in the project’s Component 1 (1–5 in Figure 1) were intended to help put in place critical elements of a strengthened management and delivery system for SA nationwide, including development of a beneficiary registry and an MIS. The project supported digitalization of the existing SA beneficiaries and poor and near-poor households in the beginning of 2016 following the 2015 Poverty Census. In parallel, an MIS platform (called POSASoft) was developed to manage SA beneficiaries, poor, and near-poor households and serve as a platform for harmonizing and consolidating household Page 8 of 52 The World Bank Social Assistance System Strengthening Project (P123960) and individual information to serve management and delivery of programs in the four project provinces of Quang Nam, Lam Dong, Tra Vinh, and Ha Giang. The MIS platform was first deployed in the four project provinces and was then rolled out to other provinces nationwide. With the beneficiary registry and MIS, it was the first time that Vietnam had a national database of beneficiaries of the SA program and poor and near-poor households. The MIS included all key building blocks of a SA program that could help carry out program operations and transactions more efficiently, including enrollment of beneficiaries, managing beneficiary lists, budget planning, payment and payment reconciliation, reporting and monitoring, grievance redressal, and so on. This was intended in turn to help improve program management and increase the quality of services to beneficiaries. 10. Considering that using the registry and information management system was technically a new field for the client, the Investment Project Financing framework to carry out a pilot to draw lessons for national replication was the appropriate investment instrument=. This allowed the World Bank to intervene, as needed, in case project activities went off-track. The project was supplemented by technical assistance (TA) to support business process assessment and reengineering, change management, and preparing reports on lessons learned and recommendations as well as a road map for program consolidation by the Government. It also supported consultation workshops to increase awareness, skills, and capacity to carry out innovations. Finally, the project supported the non-project provinces to put in place the necessary preconditions (guidelines, training, software, and hardware) to use the strengthened system for SA management and delivery in a phased manner. 11. Besides the investment in beneficiary registry and MIS, the project was designed to pilot some innovations in program delivery in four project provinces, including: (a) establishing a social collaborator network to conduct outreach, communication, and social mobilization efforts; (b) carrying out communication for development (C4D) activities; and (c) using an independent payment modality for all project programs benefit payments. Social collaborators worked with communities and families to enhance their knowledge about the programs, and their entitlements and responsibilities, and to enhance parenting knowledge and encourage healthy behavior. This in turn would help generate demand for the use of health, education, and poverty reduction services. Social collaborators incentivized women to complete perinatal care, take children for preventive checkups, and send them to school. C4D activities (Activity 3 in Figure 1) supported advocacy at the national and subnational levels to ensure uptake of the Opportunity Program. Behavior change communication material delivered by social collaborators (Activity 4 in Figure 1) was used to engage with the beneficiary households. C4D activities were essential to introducing the Opportunity Program and raising awareness of the consolidation of three existing cash transfers and the new service delivery processes (the independent payment provider and the feedback and redress mechanism). In addition, C4D activities increased awareness of local leaders to the inclusion of the ‘SASSP additional beneficiaries’ in the Opportunity Program. 12. With regard to piloting a new payment modality in the four project provinces, the project supported introduction of an independent payment provider (Vietnam Post [VNPost]) to deliver payments of SA cash benefits instead of using government staff. Before the project, cash support payments were made by local commune staff of DivLISA or commune officers. The new payment modality worked well in terms of reducing the workload for government staff so that they could focus more on their key tasks, improving transparency, making timely payments, and reducing time and cost for beneficiaries. This was highly appreciated by both the program administrative staff and beneficiaries. Building on this experience, MOLISA encouraged all provinces to use the independent payment service Page 9 of 52 The World Bank Social Assistance System Strengthening Project (P123960) provider for SA and social insurance benefit payments. As a result, by project closure, 58 out of 63 provinces had moved to this payment model. Figure 1. Project Theory of Change 13. Another project activity under Component 2 was the launch of the Opportunity Program. The project expanded coverage of SA beneficiaries to three additional groups: (a) pregnant women; (b) children aged 0–3 years; and (c) children aged 3–15 years not attending school and from poor households. The consolidated program, named the ‘Cơ hội thoát nghèo truyền kiếp’ or ‘Opportunity to move out of inter-generational poverty’ program (‘Opportunity Program’ in short), would combine cash benefits under Decrees No. 12 and 49 and Decision No. 268. The project invested in expanding opportunities to the three underserved vulnerable groups, pulling all cash benefits for households into a household package under the Opportunity Program, and getting them paid through the independent payment provider—VNPost. Piloting all innovations in four provinces was expected to provide evidence to develop recommendations for the Government to expand program consolidation nationwide. The lessons from consolidating the three cash transfer programs into the Opportunity Program, expanding coverage to unserved poor households, and improving payment delivery, grievance redressal, and monitoring, were intended to generate evidence to improve program management nationwide. Page 10 of 52 The World Bank Social Assistance System Strengthening Project (P123960) Project Development Objective (PDO) Key Expected Outcomes and Outcome Indicators 14. The PDO, according to the Loan Agreement and Project Appraisal Document (PAD) was “to support the government of Vietnam in strengthening the social assistance system by developing innovations in management and service delivery, and by piloting these innovations in the four project provinces.” The PDO had two intended outcomes or specific objectives as measures of strengthening of the SA management and delivery system. The efficacy of both outcomes was measured separately according to the respective indicators: (a) number of provinces with digitized household information in the national registry; and (b) process consolidation as measured by the number of provinces using the national registry and MIS. 15. The PAD acknowledged the challenges in measuring and monitoring the project’s results. Particularly, the scope of project interventions was framed as, “…while there is consensus on the direction towards cash transfer program consolidation and the project will help put in place the pre-conditions, actual progress on program consolidation will be subject to high-level policy decisions”. An ambiguity in the PAD was the balance between policy consolidation of SA programs (which was reliant on national level policy decisions beyond the scope of the project) and consolidation of delivery systems for such programs. The project components and indicators were clearly more focused on the latter, while the PAD discussion at points suggested a focus on the former. As a result, PDO-level outcome indicators focused on progress in information systems and business process consolidation. 16. The Results Framework (RF) and monitoring was originally framed with two PDO indicators and four intermediate indicators to capture progress in implementing the innovations at the national level and tested in the pilot provinces. These included milestones such as the creation of a new national registry and establishment and operation of a new program MIS, as well as indicators of system and service delivery improvements in the pilot provinces such as the share of benefit payments made on time and beneficiary satisfaction. The two PDO level indicators were “number of provinces with digitized household information in the national registry” and “process consolidation as measured by the number of provinces using the national registry and management information system (MIS).” 17. The project design raises three challenges with respect to measuring and assessing results. One, it is difficult to identify a composite indicator for measuring institutional capacity and system strengthening and intermediate process indicators were used to capture some of these dimensions. Two, measurement systems for these process indicators were weak and needed first to be established in the first year of the project before results could be measured accurately. Three, though the project had ambitions to influence national SA delivery systems (and did so in the end), the project activities under Component 2 were confined to the four project provinces, resulting in somewhat of a disconnect between the original balance of funding across the components and the PDO level indicators. The PDO analysis that follows focuses on the creation of a new national registry and establishment and operation of a new program MIS, as well as indicators of system and service delivery improvements in the pilot provinces such as the share of benefit payments made on time and beneficiary satisfaction. Page 11 of 52 The World Bank Social Assistance System Strengthening Project (P123960) Components 18. The project was structured in three components. Component 1: Strengthening the Social Assistance and Poverty Reduction System (Original US$27 million; Actual US$18.4 million) 19. This component supported: (a) a national database of poor/near-poor households and SA beneficiaries; (b) an integrated MIS for nationwide use; (c) a social collaborator network in the project provinces and capacity-building activities; (d) C4D activities in the project provinces; (e) a grievance redress mechanism in the project provinces; (f) an M&E system and beneficiary feedback survey (BFS) report in the project provinces; and (g) a report on lessons and recommendations from the pilot in the project provinces for policy formulation at the national level. Component 2: Launching a Consolidated SA Program (Original US$30 million; Actual US$6.2 million) 20. This component tested the mechanisms developed under Component 1 and the consolidation of three cash transfer programs into the Opportunity Program in four project provinces. Subcomponent 2(a) provided funds to support benefits for three additional beneficiary groups: pregnant women, children ages 0–3 years, and children ages 3–15 years not attending school from poor households. Subcomponent 2(b) funded the service fee of an independent payment provider to deliver the payment of the Opportunity Program cash benefit plus the cash benefit of Decree No.136.3 Benefits under the consolidated programs (except the three additional beneficiary groups funded by the project) were financed from the state budget. Provision of remuneration for village-level social collaborators was also covered under this component. Table 2. Consolidation of Programs in Four Project Provinces Existing Program Type Payment Consolidation Programs Frequency Decree No. 136 Cash transfer to six groups of SA beneficiaries (elderly, Monthly No, stays as people with disability, orphans, and other groups) Decree No. 136 Decree No. 49/74 Cash transfer and fee exemption for poor students Semiannual Decision No. 12 Food and housing subsidies and cash transfer to Monthly secondary education students in extremely Yes, disadvantaged socioeconomic areas consolidation as Decision No. 268 Cash transfer to offset energy price rises (electricity Quarterly the Opportunity subsidy) for poor households Program New Cash transfer for pregnant women, children below 3 beneficiaries years, and children ages 3–15 years not attending Monthly SASSP additional school living in poor households beneficiaries Note: Existing program/policies names and description as of 2014. Page 12 of 52 The World Bank Social Assistance System Strengthening Project (P123960) Component 3: Project Management (Original US$5.5 million; Actual US$3.5 million) 21. This component provided project management and capacity-building support to the Central Project Management Unit (CPMU) and the four Provincial Project Management Units (PPMUs) established by the SASSP to oversee the development and testing of project interventions at the national and provincial levels. This component also supported the monitoring of project performance through progress reports, spot checks, and BFSs. 22. The total project cost, which was originally estimated as US$62.5 million, consisted of US$60 million from a specific investment loan and US$2.5 million as counterpart funds from the central and local governments. The counterpart funds were used to pay salaries of the CPMU and PPMU teams and other related expenses to enable project implementation by the Government. B. SIGNIFICANT CHANGES DURING IMPLEMENTATION (IF APPLICABLE) Revised PDOs and Outcome Targets 23. The PDO and outcome targets were not revised. Revised PDO Indicators 24. The PDO indicators were not revised. Revised Components 25. While the PDO, components and activities remained unchanged during the project’s life, the project costs by components were reduced and the Results Framework was revised and one monitoring indicator was added during the first project restructuring. The project was restructured three times: the first restructuring in September 2017 revised the intermediate indicators and adding one more indicator under Component 2 in the RF. The second was in September 2019 and the last in December 2019. The project canceled about SDR 17.2 million (US$24 million equivalent) through the 2nd and 3rd project restructurings and about SDR 2.5 million (US$3.49 million) at the project account closure. The project costs by component at approval and actual are in Annex 3. 26. A key question in assessing the project is the cancellation of a substantial portion of project funds due to under-spending on the cash benefits under Subcomponent 2(a). The IDA financing was designed to pay the SASSP additional beneficiaries under the Opportunity Program over three years (2016–2018), based on a simulation of 210,113 poor households with children ages 0–15 years. However, during implementation there was a serious reduction in both the number of additional beneficiaries registered and length of support per beneficiary as compared to the appraisal stage. While this resulted in a substantial cancellation of project funds, it had relatively limited impact on the achievement of PDO level objectives, which were focused on national delivery platforms, and similarly modest impacts on the province-level intermediate indicators. Despite that, it is important to understand the sources of the under-spending to make an informed assessment of the implications for the overall project rating. These are discussed below. Page 13 of 52 The World Bank Social Assistance System Strengthening Project (P123960) 27. The number of additional beneficiaries of the Opportunity Program – and hence disbursements under Component 2 - turned out to be much lower than estimated for several reasons: (a) The Government adopted a much lower poverty rate than originally anticipated, about 9.00 percent, 7.85 percent, and 6.70 percent in 2015, 2016, and 2017, respectively as compared to 15 percent projection for 2015. This was primarily a function of better than expected aggregate progress in poverty reduction across the country, a welcome outcome but one which significantly impacted the number of households eligible for project-funded benefits; (b) The estimated number of additional beneficiaries was further reduced by around 2 percent as the Ministry of Finance (MOF) only agreed to support income-based poor households rather than those considered poor according to other economic and social dimensions. As a result of these two factors, the estimated number of additional beneficiaries in the four provinces reduced sharply to around 82,000 for 2016; (c) The number of additional beneficiaries registered was further reduced during implementation at the local level as there was no consensus in some localities on supporting poor children who were not in school and poor pregnant women (due to a belief that this would make women tend to have more children, and that it may provide an incentive to remain out of school). As a result, the total number of registered and supported beneficiaries in four project provinces between 2016 and 2018 was 69,736 or about 33 percent of the original number of beneficiaries anticipated at appraisal. 28. In addition to the number of beneficiaries being sharply reduced relative to costings, the periods of benefit payment were also shorter than expected. While the project costing anticipated additional beneficiaries being be supported for three years, in practice the actual periods of payment were somewhat shorter. Policy makers and practitioners in Vietnam were very precise in deciding how long the project-financed beneficiaries would be supported. For example, pregnant women were only supported for a maximum of 9 months instead of 12 months, and children were also supported for 9 months each year (school time) instead of 12 months. Children and pregnant women were often supported only from the time they fulfilled requirements and submitted the reporting documents (certificate from local authority in case of children are out of school and medical certificate from health center with stamp, and so on). These requirements made the process complicated for beneficiaries and lessened their eligible time. Later on, the project was able to get local authorities to agree on simpler requirements (e.g., no stamp needed). Thus, the reduction in cash benefits to the SASSP additional beneficiaries in turn resulted in a substantial surplus of project funds. 29. The results indicators were revised to provide more accurate intermediate indicators under Component 2, as shown in table 4. These revisions were made to reflect the fact that the specific Opportunity Program was discontinued after three years, but that the underlying SA programs continued to implement. Page 14 of 52 The World Bank Social Assistance System Strengthening Project (P123960) Table 3. Changes in Results Indicators (1st restructuring approved on September 6, 2017) Original Revised Share of benefit payments for the Opportunity Program Share of benefit payments made on time made on time Share of beneficiaries reporting satisfaction with the Share of beneficiaries reporting satisfaction with the Opportunity Program in project provinces payment process Number of registered individual additional beneficiaries (new) 30. Two original indicators were revised in the RF. The original indicator designed to capture timely payment of a single household package of the Opportunity Program was revised to capture timely payment made for each cash transfer program, including Decree No. 136 (see table 2) as the independent payment provider was used to pay not only benefits under the Opportunity Program, but also Decree No.136. The indicator for beneficiary satisfaction was also revised accordingly. Beneficiaries refers to households receiving benefits from any of the abovementioned programs (table 2). 31. A new indicator to track the number of individual SASSP additional beneficiaries who actually got included was added. Additional beneficiaries included pregnant women, children ages 0–3 years, and/or children ages 3 to under 16 years not going to school and from poor households. Poor households refer to income-poor households, as defined by MOLISA.4 Actual coverage was low as compared to the original estimation because of the reasons outlined above. However, coverage improved significantly by the second year of implementation, with the number of additional beneficiaries registered increasing from 6,371 in October 2015 to 28,880 in late 2016, then falling to 21,947 in 2017, and 18,909 by June 2018, reflecting the decline in the number of poor households following the annual update of the poor list. MOLISA acknowledged the importance of monitoring this figure and of the extend coverage for these groups was improved. Again, as the number of beneficiaries was naturally reduced, the indicator should monitor the proportion of additional beneficiaries registered as compared to actual number of beneficiaries, not to a fix number. Rationale for Changes and Their Implication on the Original Theory of Change 32. The major change in project implementation was the cancellation of funds which were not fully used to support additional beneficiaries. The major reason was because of the Government’s decision to have a smaller number of poor and near-poor households, and this led to a reduction in the number of additional beneficiaries. This unexpected change was out of the project’s control. Another reason was the delay in implementation and reluctance of local staff to include additional beneficiaries (poor out-of- school children). The implication for the theory of change is that it failed to allow for unexpected changes during implementation because of policy changes. With hindsight, this should have been taken into consideration at the design stage, or alternatively have triggered earlier action on restructuring or revision of indicators. In addition, the expected length of the periods for project-financed benefit payments should have been more clearly understood and agreed during project preparation, both in terms of likely time to register beneficiaries and in terms of the actual periods of payment and the apparent mismatch between Page 15 of 52 The World Bank Social Assistance System Strengthening Project (P123960) the Bank’s teams understanding (which was the basis for project costings), and that of the clients, who preferred a more parsimonious definition of periods of eligibility for benefit payment. 33. The second change in the project was the addition of one monitoring indicator to assess progress in expanding the coverage of additional beneficiaries. There was no monitoring indicator on implementation of the Opportunity Program, except a requirement on reporting the lessons learned and drawing recommendations of program consolidation for future government policy. The implication for the theory of change is that if program consolidation between various line ministries was considered an important project intervention/innovation, there should be some indicator to monitor its implementation progress and achievement. II. OUTCOME A. RELEVANCE OF PDOs Assessment of Relevance of PDOs and Rating Relevance of the PDO is High. 34. The PDO was highly relevant to the Government’s development agenda and the World Bank’s country strategy at appraisal and remained relevant during implementation. (a) Relevance to the GoV objectives. The project responded to the needs of Vietnam’s laws, policies, and strategies relating to modernization of social assistance system which were in place at the time of approval and remained so during implementation. The project supported MOLISA’s implementation of Resolution No. 15-NQ/TW (dated June 01, 2012) of the Central Communist Party XI on some social issues for the period 2012-2020, by: (i) developing solutions for the modernization of social assistance system; (ii) consolidating programs; (iii) streamlining processes for SA program implementation; and (iv) piloting the innovations in four project provinces. The project was successful in developing a database of social assistance beneficiaries and an integrated management information system that has been rolled out nationwide and continues to perform as a key platform of social assistance programs. As a result, the project has set a solid foundation for MOLISA to go further in consolidation of its information systems and create a national social security database for managing data of social assistance, poverty reduction, revolutionaries’ benefits, and social insurance as outlined in the Prime Minister Decision 708/QD-TTg (dated May 25, 2017). The project was relevant as it contributed in significant part to improve and implement social security programs, in general, and social assistance programs in particular. It also contributed to capacity building of MOLISA and four project provinces’ DOLISAs to better manage, monitor and evaluate social assistance programs. These technical and administrative structures will continue to maintain their relevance to monitor and manage SA programs after project closing. (b) Relevance to the CPS/CPF objectives. The World Bank supported the GoV in strengthening the SA system, which is anchored in the Opportunity Pillar of the Country Partnership Page 16 of 52 The World Bank Social Assistance System Strengthening Project (P123960) Strategy (CPS) for FY12–FY16 (Report No. 65200-VN). In the Independent Evaluation Group Completion and Learning Report (CLR) review, dated May 17, 2017, the objective on improved systems for targeting social protection was measured by whether a pilot beneficiary database has been created in at least four project provinces, using existing data on poor, near-poor, and beneficiary households. The CLR concluded that this objective was achieved as the four project provinces were using the national beneficiary database as of November 2016. In the Country Partnership Framework (CPF) for FY18–FY22 (Report No. 111771-VN), the project supported Focus Area ‘Invest in People and Knowledge’ and Objective 7 to improve integration and efficiency of SA, pension, and health insurance systems. This objective was measured by: (i) the number of provinces integrated into the MOLISA national database/MIS (target: 60); and (ii) share of benefit payments for the administrative Opportunity Program made on time in the four project provinces (target: 80). In the Performance and Learning Review (PLR) of the CPF for FY18–FY22 (dated April 19, 2019, Report No. 134020-VN), the targets were assessed as achieved, as 60 provinces were integrated into MOLISA’s centralized MIS and a 93.6 share of benefit payments for the Opportunity Program were made on time in the four project provinces.5 By project completion, 60 out of 63 provinces had updated the data of beneficiaries using the new MIS at the national level, a major achievement of system consolidation from a system that was highly fragmented across and within provinces at the outset. B. ACHIEVEMENT OF PDOs (EFFICACY) Rating: Substantial 35. The overall efficacy rating of the project is the result of the achievements of their indicators, which are rated Substantial (see details in annex 1 and in the following paragraphs). 36. The efficacy of the PDO achievements is Substantial. The PDO aims to achieve one clear project level outcome - to strengthen Vietnam’s social assistance system. It proposed to accomplish this through: (a) developing innovations in management nationwide – by building a national database (i.e., beneficiary registry) and its MIS platform; and (b) developing innovations in service delivery in project provinces – by establishing a network of social collaborators and a grievance redress mechanism, supporting C4D activities, and using an independent payment provider to deliver social assistance cash benefits. 37. With regard to developing innovations in management and delivery, several innovations were introduced under the project to improve management and delivery of the SA programs. A key achievement was that a national MIS was developed, which covered all beneficiaries of SA programs and all members of poor and near-poor households. By project closure, 60 provinces had completed digitizing household information in the national registry, which surpassed the target by 50 percent. The MIS was first developed in the four project provinces and was then rolled out to 60 out of 63 provinces, representing a real breakthrough in the information management of SA programs relative to the pre- project situation. This newly built system has also played a significant role in implementing the Government’s safety net package in response to COVID-19, in terms of having a beneficiary registry and payment system in place to deliver timely support. With regard to piloting innovations in the four project Page 17 of 52 The World Bank Social Assistance System Strengthening Project (P123960) provinces, an independent payment provider (VNPost) was selected to deliver payment of SA cash benefits instead of using government staff as previously. The new payment modality worked well in terms of reducing the workload for government staff so that they could focus more on their key tasks, also improving transparency, making timely payments, and reducing time and cost for beneficiaries. This was highly appreciated by both the program administrative staff and beneficiaries. MOLISA encouraged all provinces to use the independent payment service provider for SA and social insurance benefit payments. As a result, by project closure 58 out of 63 provinces had moved to this payment model, a major step forward in having a more modern payment system for G2P payments. 38. The project supported digitalization of beneficiaries’ information managed under POSASoft starting from 2015. The project supported data collection and digitalization for SA beneficiaries and poor and near-poor households following the 2015 Poverty Census. Early 2016 data collection, standardization, and cross-checking were carried out in all provinces. After these national digitalization efforts, provinces took on the responsibility of updating the data into the system. By project closure, 58 out of 63 provinces had completed digitalizing 100 percent of beneficiaries’ 2018 data in their provinces into the system. This was equivalent to approximately 92 percent of the SA beneficiaries and poor and near-poor householdrecords nationally being digitalized. However, only 50 percent of the provinces managed to update data for 2019 for SA program beneficiaries. However, the COVID-19 crisis and introduction of emergency payments by GoV pushed some more provinces to quickly update information and use the system for payment of programs, relying on the platform that was first built under the project. 39. A clear achievement of the project in establishing the beneficiary registry has been reduction in both inclusion and exclusion errors in targeting of SA benefits. Regarding exclusion errors, even though there were no data to exactly measure the magnitude of the reduction in the number of exclusion of eligible beneficiaries, field visits detected many cases where women and children were certified as disabled but they were not enrolled to receive benefits because of the limited understanding of procedures and entitlements. Regarding inclusion errors, as the registry, MIS, and new payment mechanism were in operation in the four project provinces, in many places VNPost staff reported that there were a number of cases where beneficiaries failed to show up to receive the benefits for three continuous months. After three months, VNPost reported these cases to the local program managers and they checked with the local authority at the community and village levels and found that the beneficiaries had been dead for some time. In other cases, the system also helped identify that before the project, many schools often did not return money that should have been paid to the students because the students had dropped out by the time payment was to be made. 40. In addition, the policy and delivery innovations developed under the project have been mainstreamed into GoV national policies. The experiment of these innovations in the four project provinces provided lessons and recommendations to the Government for consolidation of SA program administration and for expansion of SA coverage to poor pregnant women and children under 3. This has fed into national policies. First, the Government has included a target of covering pregnant women and children under 3 from poor households as SA beneficiaries under the Master Plan for Social Assistance Reform which is the policy roadmap for the SA system for the coming decade. Second, the issuance of Decision No. 708/2018/ QD-CP on developing a national integrated information system covering all six major beneficiaries’ groups, including poor, near-poor, SA, unemployment insurance, social insurance, and health insurance was a clear example of reform action which built on the positive experience of the project. Third, the harmonization of SA management piloted under the project was institutionalized Page 18 of 52 The World Bank Social Assistance System Strengthening Project (P123960) through the nationwide use of POSASoft which was mandated under MOLISA’s Circular No. 11/2019/TT- LĐTBXH in July 2019 on “Regulations on development, management, update, and exploitation and use of social assistance and poverty reduction databases” (see below). Key Performance indicators 41. The project was successful in achieving its two PDO indicators, exceeding the original target values in both cases. At project completion, 60 provinces had digitalized beneficiaries’ information in the national beneficiary database and deployed the MIS, which exceeded the end targets of 40 and 50 for each indicator, respectively. Table 4. PDO indicators PDO Indicators Baseline Target Value Achieved Value Process consolidation 0 40 60 Information consolidation 0 50 60 Source: SASSP Results Framework, January 2020. 42. Four out of five intermediate results indicators were achieved or surpassed the targets, while one was at 86 percent of the target. The database and the MIS were rolled out to all provinces, and this was institutionalized by MOLISA’s Circular No. 11/2019/TT-LĐTBXH in July 2019 on “Regulations on development, management, update, and exploitation and use of social assistance and poverty reduction databases”. The circular guaranteed the sustainable and effective use of the new system at both the national and subnational levels over the long term. The average share of benefit payments made on time in the project provinces was 93.7 percent, with 96.5 percent of beneficiaries reporting being satisfied with the payment process, surpassing the targets by 10 percent and 20 percent, respectively. One indicator that fell short of its (86 percent) target was the number of SASSP additional beneficiaries registered (18,909 beneficiaries enrolled instead of the annual target of 22,000) as the Opportunity Program closed by June 30, 2018. Table 5. Intermediate Results Indicators Intermediate Results Indicator Baseline Target Value Achieved Value Strengthening the social assistance and poverty reduction system Creation and operation of No digitized database National beneficiary Achieved national beneficiary database database created and ready to be linked to MIS of other SA programs New program MIS established Manual/semiautomated MIS ready for rollout Achieved and in operation systems in additional provinces Launching a consolidated social assistance program Share of beneficiaries reporting 93.7% satisfaction with the payment 65% 85% Achieved, exceeded the process target Share of benefit payments 96.5% made on time 0% 80% Achieved, exceeded target Number of registered individual 18,909 0 22,000 additional beneficiaries Partially achieved Source: SASSP Results Framework, January 2020. Page 19 of 52 The World Bank Social Assistance System Strengthening Project (P123960) Other Aspects of the Achievement of the PDO 43. The modernization of the management of SA beneficiaries has brought several benefits to program implementers and beneficiaries. The registry was created to serve as a source of harmonized information for management of beneficiaries. Previously, administration of the SA programs was paper based, which resulted in inefficiencies in managing, delivering payment, and tracking beneficiaries. Manual encoding limited district officers to only being able to add or remove records of beneficiaries once a year. Reporting tools for tracking key indicators (for example, approved/rejected applicants and coverage payments) nationally or by locality did not exist. The developed MIS platform offered many advantages for program administrators and beneficiaries alike. For program users, POSASoft generated efficiencies through modern administration of benefits and record keeping. It allowed more frequent updates, increased flow of information for planning purposes, remedied fraud and errors, and generated significant savings on administration costs. The registry also served monitoring purposes and reporting of information from the commune to central level, which used to take time previously. Beneficiaries are now able to submit inquiries on the status of their benefits (for example, missing or delayed payment) and expect an expeditious response given the ready availability of data. Finally, POSASoft increased the quality of information and transparency of implementation of SA programs as data validation and verification processes were incorporated into stages of POSASoft (including error checks already programmed into the software). Almost all provinces in Vietnam now use the database and MIS to register SA beneficiaries and poor/near-poor households, process payment orders, and monitor beneficiaries. 44. The implementation of the MIS also faced some challenges. The first challenge is related to the annual updating of the beneficiary data. The POSASoft registry includes SA beneficiaries and the poor and near-poor households. For the SA programs, updating of beneficiaries is being done regularly, while poverty data are updated once a year based on annual update of poor and near-poor lists. The poverty census is conducted every five years and then updated annually through a complex process which lacks quality control mechanisms and is unevenly implemented across provinces. At the same time, Government has been pushing very hard on cutting down the number of staff under national, organizational, and administrative reform. This has seriously affected work performance at the local level. Understaffed district-level offices face difficulties in processing and updating large volumes of information given the high burden of work. Thus, updating the database to reflect annual changes in poverty status, as well as regular changes in SA beneficiary applications, was a challenge and varied a lot across locations. It took more than a year for the 2015 poverty data to be updated in the registry. Additionally, high turnover of staff in some districts and human resource capacity to manage, maintain, and operate POSASoft have also proven to be a challenge. The project had foreseen this and had designed retraining as well as developed an online manual to help deal with high staff turnover issues, but this coupled with staff reduction made updating data a significant challenge. Experience shows that in provinces and districts where the local leaders/managers paid attention to issues, it was easy to operationalize POSASoft. For the rest, it took a longer time beyond the project’s closing date. 45. The project established a network of social collaborators with the aim of facilitating beneficiaries’ understanding of the project; programs; their entitlement; and their responsibilities in childbearing, caring, and education. They also provided support to poor households and SA beneficiaries through links with health services. Social collaborators were also in charge of communication activities at the commune level. A communications strategy was developed to promote awareness, understanding, and acceptance of reforms introduced by the project at the national and local levels. A formal feedback Page 20 of 52 The World Bank Social Assistance System Strengthening Project (P123960) and redress system associated with the project was built on an existing hotline that MOLISA is using to receive beneficiary feedback and grievances for other existing programs. Spot checks, BFSs, and administrative reports were the main M&E tools of the project. These M&E activities played an important role in providing timely information for the CPMU’s managers and MOLISA leaders in making relevant decisions to solve implementation issues that were raised during project implementation. Findings from assessments and reports were presented in various workshops and events which also provided good feedback for the Government in developing related policies and decisions. 46. Experience from piloting program consolidation under the project provided lessons for national consolidation of the SA programs, which were presented in one of the project reports on ‘Recommendations and road-map for consolidating SA programs in Vietnam’. This was placed as one deliverable of the project and was successfully achieved. The project was designed to pilot all project innovations, taken as a whole, in the four selected provinces. The findings and lessons were drawn not only on the policy front, such as its feasibility to expand coverage of SA program 136 to two out of three additional beneficiary groups. And the third group —'children out of school from poor households’—was not in the end supported as it was considered a conflict with the Government’s existing policy to support children to attend school. Lessons and recommendations were also drawn on the need to restructure business processes, which should be guided by a decree from the central government so that POSASoft could be fully explored and have greater impacts (beyond programs managed by MOLISA). The evidence generated from the pilot demonstrated the importance of having a strong information system and strengthened delivery system for improving SA service quality and reducing poverty in the medium to long term. Moreover, challenges in implementing consolidation of programs between different ministries also call for the Government’s attention to learn from these difficulties in future attempts to consolidate programs and calls for attention and leadership from higher levels of the Government beyond the provincial and ministry levels. 47. A major success of the project was the introduction and support of VNPost as an independent payment provider of SA benefits. The project successfully piloted the third-party benefit payment for beneficiaries of Decree No. 136, ‘SASSP additional beneficiaries’ (financed by the project) and education and electricity subsidies. Before the SASSP, payment delivery relied on local commune labor officers or village leaders who visited the beneficiary households without any specific notice or payment frequency. The introduction of a third-party agency, VNPost, increased the convenience and satisfaction of beneficiaries as they now enjoy more payment points,6 transparency, and predictability in receiving their cash benefits. The BFS7 (2018) found 80 percent satisfaction with VNPost as beneficiaries benefited from fixed payment schedules, suitable facilities at payment locations, and appropriate waiting time, among others. The BFS 2018 found that beneficiaries have higher awareness of their benefit amounts; certainty about their payments (schedule, location, and amount); and a reduction in transaction costs (travel costs, travel time, and waiting time), though beneficiaries living in remote areas still face challenges (see table 7). This has proved that this is the right direction and paved the way to further strengthen payment modality for social protection programs in Vietnam. Page 21 of 52 The World Bank Social Assistance System Strengthening Project (P123960) Table 6. Performance Comparison: Before/After of Awareness, Payment, and Utilization of Service Indicators Indicator Unit Baselinea BFS [Reference Year] [2015] [2018] Awareness b Aware of eligibility criteria % 68 58 Aware of benefit amount % 65 89 Aware of feedback mechanism % 4 12 Payment c Beneficiaries receive payment on time % n.a. 93 Transportation cost to payment location VND 10,900 5,200 Travel time to payment location (average) minutes 37 20 Waiting time at payment location minutes n.a. 30 Utilization of health and education services d Children growth monitoring or checkup (0–5 years old) % 59 65 Child had routine immunization (past 12 months) % 69 59 School enrollment (3–15 years old) % 83 81 Note: a. Avalos, J. 2017. Baseline Report, Social Assistance System Strengthening Project (SASSP) . World Bank, Hanoi. b. Baseline estimates include average for four programs (Decree No. 136, Decree No. 49/74, Decision No. 12, and Decision No. 268). BFS 2018 estimates include these programs plus the SASSP additional beneficiaries. c. BFS 2018 estimates only cover information for Decree No. 136 and ‘SASSP additional beneficiaries”. d. Estimates are for illustration only. Baseline estimations include non-beneficiary households surveyed within the SASSP provinces, while BFS 2018 includes only the SA beneficiaries. 48. Implementation of other innovations brought about in project provinces showed mixed results and lessons learned. The project put in place a sound M&E system to monitor implementation and evaluate results. Most of the information related to effectiveness comes from spot checks, BFSs, and household surveys conducted throughout project implementation. A finding from surveys was that the social collaborators were not able to effectively increase parenting knowledge and link beneficiaries to services. While 71 percent of beneficiaries knew their social collaborator, only 16 percent of beneficiaries recall participating in parenting sessions or meetings where social collaborators encouraged the use of benefits to access maternal or children health care services (BFS 2018). An important reason for this less than satisfactory role of social collaborators was that their monetary incentives (allowance) were too low compared to their responsibilities.8 This resulted in turnover and frequent retraining sessions, and therefore mixed performance was not a surprise. Given the fact that the Government was not ready to pay social collaborators a higher allowance, the lesson learned was to rely as much as possible on the existing network of social collaborators created by other programs such as population programing, grassroots health support programs, and so on. Page 22 of 52 The World Bank Social Assistance System Strengthening Project (P123960) Justification of Overall Efficacy Rating 49. All PDO indicators and almost all intermediate indicators were achieved, and some of them significantly surpassed original target values. This reflects a good track record in implementing the project components and activities which included national-level operations (that is, registry and MIS). Achievement in developing innovations in management nationwide (objective a) is High. The project has modernized the management and monitoring of SA beneficiaries nationwide; this has demonstrated important benefits for administrators, implementers, and beneficiaries especially during COVID-19. Achievement of developing innovations in service delivery nationwide (objective b) is Substantial. Despite the new MIS, some other improvements in service delivery have shown more limited impacts nationwide because of the delay in updating beneficiary data into the system. The piloted activities related to service delivery in the project provinces were able to help the project owner draw lessons for future decisions in consolidating SA programs nationwide. Taken together, these ratings indicate that the project’s overall efficacy rating is Substantial. Table 7. Summary Indicators’ Achievement (Based on data from table 6) Kind of Indicator Rating Category and Correspondent Interval PDO Intermediate Total Rating Results Indicator PDO 1: Number of provinces with digitized household information in the national registry Surpassed 1 2 3 Achieved/substantially achieved (>85%) Partially achieved (65 %–84%) Not achieved (<65%) Total Indicators PDO 1 1 2 Percentage of surpassed, achieved and substantially 100 100 100 achieved PDO 2: The number of provinces using the national registry and management information system (MIS) Surpassed 2 2 Achieved/substantially achieved (>85%) 1 1 2 Partially achieved (65%–84%) Not achieved (<65%) Total Indicators PDO2 1 3 Percentage of surpassed, achieved, and substantially 100 100 100 achieved PDO 2: all PDOs Surpassed 1 4 5 Achieved/substantially achieved (>85%) 1 1 2 Partially achieved (65%–84%) Not achieved (<65%) Total Indicators PDO 2 2 5 7 Percentage of surpassed, achieved, and substantially 100 100 100 achieved Page 23 of 52 The World Bank Social Assistance System Strengthening Project (P123960) C. EFFICIENCY Assessment of Efficiency and Rating Efficiency rating is Substantial 50. Efficiency improvements brought about by the project generated substantial savings, though not all can be quantified. While it will take time to realize the full potential benefits of the SA system as staff and managers need time to adapt fully to the new system, the modernization of administration of the SA programs supported by the project resulted in efficiency gains across several dimensions, including: (a) Establishment of an MIS to modernize SA benefit administration and prevent, detect, and remedy fraud and errors improved efficiency for all programs. The new system not only helped detect and remove ghost beneficiaries (inclusion errors) but also helped detect potential beneficiaries who would have been left out without the project (exclusion errors). Regarding the former, there were cases where people had been dead for a long time but were still on the entitlement list or where children had dropped out of school after being included in the beneficiary list at the beginning of the school year. Regarding the latter, there were many cases where disabled children got certificates and were qualified for benefits but were left out as their parents and/or guardians did not know the rules and what to follow. These people have learned about their entitlement through the project’s C4D activities. (b) Improvements in the payments of benefits clearly improved efficiency and generated savings. The new payment mechanism, which builds on the MIS, increased efficiency of the SA system, as both program administrators and beneficiaries can save time and money by delivering and receiving payments in a more professional and independent way. Table 6 shows that the new payment mechanism reduced transaction costs (travel costs, travel time, and waiting time). (c) The use of the MIS developed under the project helped reduce the timing for monthly reconciliation, which used to be done manually, and improve transparency and avoid conflicts of interest and poor management (as the same staff no longer make the decision on entitlement and making payment). 51. Given that a number of the efficiency improvements under the project are supported by more qualitative assessments, cost-effectiveness estimations are limited to the rolling out of the registry and POSASoft (MIS), and new payment modality. The latter present a challenge in itself given that estimating and comparing registries is not straightforward. Registries incur different types of administrative costs and the costs are spread out over time.9 52. The cost efficiencies generated by POSASoft obviously outweigh the costs of developing and operating the system over time. The project has significantly improved the efficiency of managing SA beneficiaries in Vietnam through its support of the national beneficiary database and MIS. Before the project, the management of SA programs was manual and paper based, which resulted in a highly Page 24 of 52 The World Bank Social Assistance System Strengthening Project (P123960) inefficient and non-transparent system. Provincial- and district-level staff could not track and manage beneficiaries given that most of the information was not digitalized, centralized, and easily accessible. The registry and MIS were designed to integrate beneficiaries’ information from several programs to generate economies of scale, efficiencies, and savings on administration costs and allow for cross-checking of benefit double dipping, which can be significant because the processes of management for each program can be costly. The project has strengthened and standardized programs’ transactions, monitoring, and reporting and reduced errors, all contributing to the improved efficiency of the SA programs. Some of these cost efficiencies include reducing paper and storage costs across all districts in the country; identifying and eliminating cash benefits that were duplicated; and reducing processing time and cost for monitoring, reporting, and payments. It is expected that overall and over time resource savings are higher than the US$15 million invested on hardware, designing and developing the system, and providing training to operate it nationwide. 53. Even without fully quantification of project benefits, it is clear that the benefits relative to investments were substantial. Table 9 gives quantified annual impacts for a selection of the outcomes identified in the sections above. It is clear that, even without quantification of all benefits, the core MIS and national database investments, and their linkage to the improved payment system, more than pay for themselves. Table 8. Potential Annual Savings from Improved Systems Savings From Unit Unit Cost (VND) VND, millions US$, millions Administrator Preparing monthly beneficiary 800 200,000 14,400 0.62 list (save 50% of 12 staff-days per month per district) Reconciliation and accounting 800 200,000 28,800 1.24 (save 9 staff-days per month per district) Reporting and other 863 200,000 122,978 0.82 transactions (save 9 staff- days) Beneficiary Savings on travel cost 4,000,000 5,700 273,600 11.69 Multiple travel and waiting 6,000,000 10,900 78,480 3.35 time Total savings annually — — — 17.72 Note: Estimation of potential annual saving were calculated based on the following assumptions: a. At the district and provincial levels before the project, one staff responsible for making (i) 1 staff-day preparing payment list; (ii) 3 staff-days preparing reports—around 19 sets of quarterly, semiannual, and annual reports/tables to six various departments/divisions; and (ii) 3 staff-days for monthly manual reconciliation of payment with district treasury; three programs per district. b. For clients, 10 percent of households have on average 2 or more beneficiaries or two types of benefits --> save one time; average travel cost was VND 10,900; two-thirds of households receive monthly benefits, average reduction in travel cost was VND 5,700. Page 25 of 52 The World Bank Social Assistance System Strengthening Project (P123960) Table 9. Cost Breakdown for National Database and MIS (POSASoft) Item Amount (US$) Design and development of database and MIS 4,027,669 Computers and other hardware 4,264,726 Training and technical support 6,741,912 Total 15,034,307 Note: Total cost is VND 339,427,000,000. Exchange rate is VND 22,576.83 = US$1. 54. The cost of developing the MIS and database is reasonable in relation to the volume of beneficiaries it can serve and is comparable internationally. The estimated cost at US$2.5910 per household is minor relative to the savings generated by improved monitoring and management among SA programs. As a comparison, Turkey’s Integrated SA system cost US$1.311 per household, excluding training cost. Deducting training costs from POSASoft will result in a per household cost of US$1.43. The database and MIS enable administrators to have timely access to program indicators for operational management and policy making, as it also supports planning, assessing of potential demand, M&E, reporting, and other analytics. 55. There were minor shortcomings in cost-effectiveness of implementation. Updating of information of the SA beneficiaries and of poor and near-poor households into the registry was slow and did not allow program administrators to use the data (that is, list of poor) for identifying potential beneficiaries for the SASSP additional beneficiaries. There were also some delays in procurement processes, which caused delays in conducting the BFSs. However, all BFSs and spot checks were submitted on time before project closure and provided enough time for findings and lessons to be used and reflected in the assessment reports on the pilots submitted to the Government. D. JUSTIFICATION OF OVERALL OUTCOME RATING 56. The previous three sections have documented that ratings pertaining to the project’s relevance, efficacy, and efficiency were High, Substantial, and Substantial, respectively. The development objective of the SASSP was and remains highly relevant to Vietnam’s development priorities. The relevance of the project design is rated High, reflecting appropriate interventions to strengthen the SA system. 57. Efficacy is rated Substantial. Achievement of the objective in piloting innovations in program management and drawing lessons for replication nationwide is rated High, particularly with regard to building a national database and MIS to carry out program transactions, manage beneficiaries, and monitor/report progress and achievement. Achievement of the objective in piloting innovations in service delivery is rated Substantial because of clear improvement in service delivery (for example payments, reconciliation, reporting, and grievance and complaints redressal, and so on) in the four project provinces. The project’s report on recommendations and road map for national program consolidation was Page 26 of 52 The World Bank Social Assistance System Strengthening Project (P123960) completed as designed, discussed at the regional consultation workshop with both the central and local stakeholders and policy makers. The findings from this report also revealed that policy consolidation for existing cash transfers under various government line ministries should not be simply considered physically as a sum of programs but will require reengineering of business processes and will need to be enforced by legal regulation at the central level (for example, a decree). 58. Efficiency is rated Substantial. As analyzed earlier, over time, the cost efficiencies that were generated by POSASoft outweigh the costs of developing and operating the system (tables 9 and 10). 59. Taken together, these ratings indicate a Moderately Satisfactory outcome rating. While a simple average of the above ratings may suggest a Satisfactory overall rating, it is nonetheless important to take into account the significant cancellation of funds, primarily due to the savings on cash benefit disbursements in the four project provinces. While these were due to factors such as rapid poverty reduction which are ultimately positive, the partial disbursement is a factor in determining the overall rating. E. OTHER OUTCOMES AND IMPACTS (IF ANY) Gender 60. While gender is not a strong determinant of household poverty in Vietnam, the SASSP’s design was gender informed. The project supported a gender-sensitive intervention by expanding coverage to additional beneficiaries (poor pregnant women, children ages 0–3 years, and children and 3–15 years not attending school) under the Opportunity Program. The proposed recipients of the cash benefit, as in other cash transfer programs, were women, with the broad goal of improving women’s status within the household. Results from the BFS 2018 show that ‘SASSP additional beneficiaries’ spent their benefits to buy food for their children under five years. This can be attributed to the communications and information activities conducted during enrollment and throughout project implementation by social collaborators. The project also considered gender-sensitive factors to select social collaborators. Recognizing the context of provinces, a large majority of social collaborators were women (sometimes from a Women’s Union or health workers), which created favorable conditions for participation in community activities such as parenting meetings.12 Institutional Strengthening 61. The project built the capacity of MOLISA and DOLISA technical staff responsible for operation fo the SA system nationally and in the pilot provinces, as well as project management teams at the CPMU and PPMUs Throughout project implementation, combined with the World Bank’s Programmatic Social Protection AAA (P132776 [FY13–FY16] and P161141 [FY17–FY20]), capacity-building activities were systematically implemented with an emphasis on transferring knowledge to operate and expand the MIS platform, improve payment systems, and more generally seek to improve the business processes of SA programs. More broadly, key activities such as staff trainings, development of online operation manuals and Q&A sessions, social marketing/communication strategies, and M&E exercises carried out throughout Page 27 of 52 The World Bank Social Assistance System Strengthening Project (P123960) the project life helped strengthen the effectiveness of MOLISA as a SA agency. Other activities under the PAAA enhanced project implementation and informed dialogue with the client in relation to social protection. Mobilizing Private Sector Financing Not applicable. Poverty Reduction and Shared Prosperity 62. The SASSP intervention aimed to improve the SA system with the ultimate goal of reducing poverty and boosting shared prosperity. The interventions financed by the SASSP are expected to contribute to sustained impact of the SA system on poverty reduction in the medium to long term. 63. During the outbreak of the COVID-19, the GoV announced an income support package of VND 61,000 billion (US$2.6 billion) on March 31, 2020. The package will provide support to poor, near-poor, SA beneficiaries, and national devotees (or merit people). The package will also support employees who are not qualified for the unemployment insurance benefit, including those from the informal sector such as small and household businesses, part-time workers, and those on unpaid leave or with reduced income because of the COVID-19 pandemic’s impacts on the country. 64. The national beneficiary registry and MIS platform built under the SASSP provided great support the COVID-19 safety net package implementation. By the end of project implementation, POSASoft was rolled out to all provinces and modernized the management of SA beneficiaries. The local governments have improved registries and have been able to scale up the Government’s emergency measures to SA beneficiaries nationwide and to poor and near-poor households in the four project provinces and Can Tho province. At least four of the pilot provinces, and later other provinces (totally around 50 percent of provinces), could use/update information into POSASoft to extract payment lists and share the same with local Treasury Offices and payment providers for delivering additional benefits to the SA beneficiaries. Other provinces partially updated the system (only SA beneficiaries), but during COVID-19, some provinces were able to update poverty data into the system and use it to scale up support. In addition, the system could facilitate business administration and monitoring, including planning, record keeping, frequent updating, and preventing fraud and errors. Other Unintended Outcomes and Impacts Not applicable. III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME A. KEY FACTORS DURING PREPARATION 65. The long-term collaboration between the World Bank and MOLISA in the past 10 years on social protection reform ensured appropriate support during project preparation. The World Bank provided TA to support social protection system strengthening, which could exploit benefits from greater Page 28 of 52 The World Bank Social Assistance System Strengthening Project (P123960) integration and coordination between SA and social insurance policy and delivery. This included bringing several government departments and agencies to work together and taking a systematic approach in policy analysis and decision-making, balancing between policy reforms and delivery system’s strengthening between SA and social insurance, and building capacity for implementation and change management. These programmatic Advisory Services and Analytics activities informed project preparation and design. 66. Involvement and proactive participation of various stakeholders at both the central and provincial levels right from beginning was a key factor during preparation. Right from the beginning of project inception and identification, several field trips and consultative workshops were organized at the central and local levels to select provinces to be included in the project pilot, select programs to be consolidated, select independent payment providers, agree on selecting additional beneficiaries, and discuss and decide on project implementation arrangements. The inclusion of the SASSP additional beneficiaries required strong and persistent negotiation by the World Bank team, World Bank leadership support, and the Government high-level leader’s support and commitment. The active participation of local-level stakeholders was very important in procuring local support for introducing innovations into program implementation. B. KEY FACTORS DURING IMPLEMENTATION 67. Again, during implementation, active engagement of local stakeholders and continuous support of the central-level government agencies such as MOF, Ministry of Planning and Investment, MOET, and MOLISA were an important factor. The implementation of the Opportunity Program and the development of an MIS as well as the introduction of other innovations at the local level required an enforcement by an inter-ministerial regulation—a circular. The new circular (number 26) developed at the beginning of the project (jointly by the MOF, MOLISA, and MOET) allowed for implementing the following in the four selected provinces: (a) consolidation of three existing programs into a household package; (b) inclusion of ‘SASSP additional beneficiaries’ and being supported by the project; (c) use of an independent payment provider and use of the project fund to make payments to additional beneficiaries and existing programs covered by the project; (d) collection of individual information from SA beneficiaries and poor and near-poor households and building of a national database managed under POSASoft; and (e) development of a new MIS called POSASoft to manage the covered programs’ operations and transactions. However, the policy consolidation for the three existing cash transfer programs under the Opportunity Program in the four provinces was challenging. There were frequent policy changes by different ministries, suspension of registration and delays in payment during the transition period, and different budgeting mechanism and benefit payment frequency. As a result, the project ensured that benefit payments were made as per schedule in the respective policy document, but it was not in practice an integrated monthly package and the benefit paid varied every month. 68. The local management and staff perception and reluctance in including children ages 3–15 years who were out of school was an important challenge in implementation. The main reason was that the Government already had an existing program to support children to stay in schools. School attendance was used as a condition for getting benefits. As a result, local people considered this new policy as being in conflict with the existing policy. They were also concerned that this new policy might encourage children to drop out of school as they might not need to stay in school to get this support anymore. In contrast, the addition of two other groups (pregnant women and children under 3 years) were more Page 29 of 52 The World Bank Social Assistance System Strengthening Project (P123960) accepted at the local level, and in 2017 these two groups were formally included as potential beneficiaries in the Government’s Master Plan for Social Assistance Reform. 69. Staff turnover and reduction at the local level because of Government administration reform (which ordered a 10 percent reduction in staffing by 2020 which was strictly enforced) seriously affected project activities at the district level for data updating and identification and enrollment of additional beneficiaries. Specifically, the district level was the most affected by this administrative reform. Before the project, DOLISA and DivLISAs employed many contracted staff. During the administrative reform (2014–2020) all provinces were under great pressure to cut down staffing according to Decree 108/2014/ND-CP. As a result, local offices released all the contracted staff and retained the permanent ones, who were often at managerial positions. This posed a huge challenge for the project during implementation as local staff had to take on additional tasks because of staff restructuring while also having to carry out project activities, including updating data into the system and learning to use the new system. IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME A. QUALITY OF MONITORING AND EVALUATION (M&E) M&E Design 70. An M&E plan was designed to ensure that project activities were on track and inform the CPMU’s implementation of project activities. A framework with indicators, tools, information sources, timing, responsibilities, and reporting was drafted to achieve successful implementation. In addition, the following set of well-designed M&E tools were developed: (a) internal spot checks for beneficiaries; (b) monthly brief surveys (called checklists) for the payment service provider (VNPost), social collaborators, and MIS operator for fast access of monitoring activities; (c) POSASoft and other administrative records, (d) household surveys (for example, baseline and end-line surveys); and (e) BFSs. These tools covered stakeholders at different levels (district, commune, and village officers) and SA beneficiaries. M&E Implementation 71. All the M&E activities and surveys were completed within the project implementation timeline. The MIS served its purpose as a monitoring tool. Regular administrative reporting improved owing to the standard reporting formats in POSASoft, giving the CPMU and stakeholders the ability to extract relevant data, as needed. Spot checks were conducted to inform program innovations brought about by the project. For instance, regular spot checks were used to monitor VNPost’s performance in delivering payments. Two indicators, timeliness and satisfaction of beneficiaries, were bound to targets of 80 percent and 85 percent, respectively, that had to be met to pay VNPost’s service fees. Beneficiary or community feedback surveys were also implemented to have an in-depth understanding of beneficiaries’ perspectives toward the processes of SA programs across the delivery chain. Two approaches to capturing beneficiary feedback were attempted: (a) through a community meeting (qualitative) and (b) through a comprehensive BFS. A delay with the implementation of BFS 2 was mitigated by combining conducting and procuring BFS 2 and BFS 3. Page 30 of 52 The World Bank Social Assistance System Strengthening Project (P123960) 72. However, rather than conducting a second round of the household survey in June 2018, it was decided that it would be more useful to have an in-depth process evaluation by June 2019. A second round of the household survey was planned for August-September 2018 to examine the impact of the Opportunity Program. However, given the implementation challenges with respect to C4D and collaborators under Component 2, the potential impact on utilization of education and health services was likely limited. Though the project was expected to impact service delivery outcomes, these indicators were difficult to measure. Attribution was made more difficult by the fact that some of the innovations introduced by the project, namely the database, POSASOFT and use of VNPOST had already been adopted by some non-project provinces. Thus, a process evaluation towards the end of the project would a better tool to examine the impact of these innovations on social assistance delivery. M&E Utilization 73. The M&E results informed decisions and adjustments throughout project implementation. Spot checks provided timely information about implementation issues, the baseline survey presented characteristics of SA beneficiaries in project provinces and identified key gaps in service delivery before the SASSP, and BFSs collected feedback from beneficiaries about innovations introduced by the project. The CPMU was able to process and use information to inform implementation. For example, corrective measures were taken after the spot checks identified issues in enrollment procedures for ‘SASSP additional beneficiaries’ (lack of support from local officials and lack of identification and supporting documents). BFS identified the challenges of social collaborators in their role of providing parenting support and knowledge (that is, challenges to organize and hold meetings), which were discussed and which the CPMU tried to address. Finally, BFS and spot checks provided satisfaction ratings of the delivery of payments, a requirement which VNPost needed to comply with to continue as a payment service provider for the project. Justification of Overall Rating of Quality of M&E 74. M&E is rated Satisfactory. An M&E system was embedded from the project design stage to track the performance of project activities and the programs/policies it supported. The main M&E activities included regular reports from POSASoft; spot check surveys to monitor program procedures; and community feedback surveys and household surveys to conduct a process evaluation, monitor service delivery and human development outcomes indicators, and provide baseline data to support future evaluations. The project benefited from rich quantitative and qualitative feedback collected from beneficiaries, social collaborators, commune labor officers, and district labor officers. For example, the spot checks and BFSs, which track the improvements and challenges of the project activities, provided valuable information to make decisions to improve project operations. The last BFS report was completed and submitted to the CPMU and the World Bank. B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE 75. Environment. This is a Category C project. The project did not finance any civil works and therefore, did not have any significant, irreversible, or minor environmental impact. 76. Social. Barriers at every stage of SA delivery prevent ethnic minority groups from accessing the SA programs. The challenges that ethnic minority groups face is related to their location, language, Page 31 of 52 The World Bank Social Assistance System Strengthening Project (P123960) capacity, and norms.13 The project prepared the Ethnic Minority Development Plan (EMDP) and a Social Assessment as part of the project preparation. Key elements of the EMDP were integrated into the overall project design and implemented appropriately (for example, consultations with ethnic groups and information dissemination in ethnic minority languages). Potential ethnic minority beneficiaries and beneficiaries had limited knowledge of program parameters and benefits, and as a result SA coverage was lower for these households. Low awareness of eligibility criteria and benefit levels and lack of information in the ethnic minority language limited the ability of potential ethnic minority beneficiaries to apply for the SA programs and receive their payments on time.14 The project activities tried to address some of these challenges in the four project provinces by streamlining the complex and fragmented SA system. For instance, communications and outreach activities conducted by social collaborators tried to address challenges with regard to language, information about program criteria, and clarity of messages. Delivery of benefits through VNPost improved accessibility for ethnic minority beneficiaries living in remote villages, as payment locations increased in numbers and reached communes and villages in some provinces. 77. Financial management. At appraisal, the project financial management (FM) risk rating was High. The following actions were agreed upon to mitigate the risks: (a) adoption of a Joint Ministry Circular (between MOLISA, MOET, and MOF) setting forth the FM framework of the project to guide the implementing agencies at all levels in the FM aspects for Component 2 of the project and clarifying the roles and responsibilities of relevant authorities and agencies including guidance on budgeting, financial reporting, and auditing; (b) use of a Project Implementation Manual (PIM) with standardized operational guidelines and procedures for implementation of the Opportunity Program; (c) development and installation in all project entities of an MIS to capture the data of beneficiaries, payment, and use of funds; (d) segregation of functions of targeting of beneficiaries, payment, and verification; (e) implementation of a set of actions to ensure transparency of the implementation of the Opportunity Program; and (f) internal audit function. 78. During implementation, MOLISA complied with the FM-related legal covenants of the Financing Agreement. This included the following: (a) a Joint Circular No. 26/2014/TTLT-BLDTBXH-BTC-BGDĐT was issued on September 22, 2014, by MOLISA, MOF, and MOET guiding the FM framework, effective from November 6, 2014, and amended in 2016; (b) the project developed a PIM setting forth the guidelines and procedures for implementation which includes the FM; (c) the project provinces signed a transfer agreement with VNPost for third-party payment; (d) the project provinces installed the MIS to capture the information of beneficiaries, serve as a basis for generating the payment instructions for VNPost, and generate the required financing reports; (e) MOLISA submitted to the World Bank quarterly unaudited interim financial reports in the form and substance acceptable to the World Bank; and (f) MOLISA submitted to the World Bank the annual audited financing statements covering the project. 79. The FM capacity of MOLISA and DOLISA significantly increased during the life of this project. The World Bank team provided FM training to the implementation agencies at all levels. The third-party payment proved to be a success as it made it easier for beneficiaries to get benefits and improved the Page 32 of 52 The World Bank Social Assistance System Strengthening Project (P123960) transparency of transfers as well as reduced the workload of DOLISA. The FM rating is Satisfactory at the completion of the project. 80. Procurement. At appraisal, the project procurement risk was rated Substantial because MOLISA lacked experience in implementing World Bank-financed projects and its staff was unfamiliar with the World Bank procurement rules and procedures. During implementation, MOLISA adopted the PIM including a sufficiently detailed procurement section. The PPMU recruited qualified procurement consultants to support the procurement. The World Bank team provided training to the CPMU and other implementation agencies on procurement rules and procedures and provided hands-on support on request. The procurement was carried out in accordance with the World Bank’s Anti-Corruption Guidelines. Despite some delays at the initial stage, the overall procurement performance maintained a Satisfactory rating till project completion. Despite some delays at the initial stage, the CPMU effectively used the PIM, procurement training was provided to staff, and there was assistance from the individual procurement consultants who were recruited to accelerate and keep the project implementation progress on track. Therefore, the overall procurement performance maintained a satisfactory rating till the project completion. C. BANK PERFORMANCE Quality at Entry 81. The World Bank team actively engaged in policy dialogue and TA in social protection in Vietnam, which became instrumental in developing the project. During preparation, the World Bank team facilitated partnership among MOLISA, United Nations Children’s Fund, and the United Kingdom Department for International Development (DFID), and provided TA financed by both the World Bank’s budget and the DFID trust fund. The World Bank team organized training and study tours for the government counterparts to visit a few countries that operate large SA programs. The project preparation team ensured that the project design was relevant and met the Government’s development priorities and the World Bank’s CPS at the time. The design followed a clear results chain that laid out activities and outputs, with two levels of outcomes. Emphasis was also placed on the project being innovative at both the national and provincial levels and producing experience through a pilot in selected provinces. The phased approach in implementation was appropriate to pilot the system modernization in the four participating provinces and to scale up nationwide based on the pilot experience. The design took into consideration the recommendations of the various World Bank reviews, comments from peer reviewers, and lessons learned from other World Bank- and donor-supported projects. The preparation team, with relevant technical experts, identified the appropriate risks and incorporated design features to mitigate them. An implementation support plan was also developed. Two points that with hindsight could have had closer attention in the PAD were: (i) fuller discussion of the policy limitations at the provincial level on what policies could be integrated under the Opportunity Program without involvement of national MOLISA; and (ii) a clearer unbundling of the different layers of the term “program consolidation” with respect to the Opportunity Program, in order to clarify that it consisted of policy level, and front- and back-end delivery platforms, with the latter being the more pertinent for improving efficiency and client experience. The failure to cover out-of-school children with benefits during implementation also suggests that there was not a meeting of minds during preparation on that point, which may reflect a need for closer attention to counterpart views. Based on this, the quality at entry is rated Satisfactory. Page 33 of 52 The World Bank Social Assistance System Strengthening Project (P123960) Quality of Supervision 82. The World Bank team provided strong support to MOLISA, CPMU, and DOLISA throughout project implementation. The World Bank team carried out regular implementation support missions, including a midterm review (MTR), and this was supplemented by the interim technical support missions as needed. The World Bank team reported the project progress toward achieving project objectives, and progress by each component, performance on fiduciary management, compliance with safeguards, and kept the government counterparts and the World Bank management informed of progress and key issues through meetings, Aide Memoires, and various reports. Three task team leaders sequentially led the implementation support with co-task managing arrangements. The task team leader based in the Hanoi office maintained continued interactions between the Government and the World Bank through informal meetings and exchanges in between the official implementation support missions. In addition to the task team leader, the fiduciary and safeguards team members, all in-country, promptly responded to client requests and strengthened the capacity of the Central and Provincial PMUs. The World Bank team facilitated early identification of potential bottlenecks and mobilized the necessary technical expertise to resolve them. The mobilization of additional technical expertise was particularly useful for the development of information systems and M&E. 83. The World Bank team applied adaptive management in response to rapid economic growth and social development in Vietnam which was in transition from an IDA to IBRD country. The World Bank supported the Government team to revise the Results Framework to ensure its alignment with project interventions. Due to funds surplus identified at the MTR stage relative to the origin plan, the World Bank team worked with MOLISA to find solutions to expand the project interventions to an integrated social protection system for better results and provided consistent support in seeking the central government approvals. Whenever factors beyond the World Bank team and the implementation agency were identified, the World Bank team reported it to the country and sectoral management for coordination with the central ministries. The Bank team made every effort when the under-spending on cash transfers became apparent to explore options for reallocation of those funds to other purposes, but this was at a time when Vietnam was approaching its legislated public debt ceiling, and the Ministry of Finance was focused on reducing borrowing where possible. Based on this, the quality during implementation is rated Satisfactory. Justification of Overall Rating of Bank Performance Rating: Satisfactory 84. Considering the World Bank’s sustained efforts in supporting project implementation and making relevant and timely interventions, the World Bank’s performance is rated as Satisfactory. The World Bank supported the GoV, at the central and local levels, to overcome the project design constraints and the delays in the first years of implementation by (a) promoting adjustments in the project management; (b) enabling local governments, vendor, and service provider teams to build a new MIS and payment systems by training and advising the project teams on time and in an effective manner; and (c) implementing a sound M&E system to measure the project outcomes and made timely decisions and interventions. Page 34 of 52 The World Bank Social Assistance System Strengthening Project (P123960) D. RISK TO DEVELOPMENT OUTCOME 85. The risk to the development outcome is Moderate. 86. Technological options. The MIS platform was designed to support beneficiary management of the Opportunity Program and Decree No. 136, totaling five groups of beneficiaries. POSASoft achieved its goal to improve the management of beneficiaries nationwide. However, a prerequisite to continue using the MIS effectively is the timely update of beneficiary information in the system. Information updating was regularly done and used for program management on the SA side, but not on the poor and near-poor households. The updating on poverty status was done once a year at the end of each calendar year, creating a huge burden on updating a large volume of information needed each year at the district level. The project team tried to find a solution to allow data updating at the commune level so that it could reduce the workload at the district level, and an off-line solution was developed. However, the capacity of staff at the commune level in using this off-line tool varied a lot across communes. Moreover, with rapid development in technology and needs of potential users at the national and local levels, the platform may need to be updated to keep up with the information technology (IT) needs of user programs and upcoming reforms, though this is a common challenge in all such systems. Currently, the platform supports one major umbrella SA program (Decree No. 136 which delivers six benefits to SA beneficiaries). In the future, POSASoft needs to be linked and be able to share information with other registries, support cross-verification with the Social Security System, and build real-time connections with payment delivery providers. The risks identified throughout project implementation (for example, lack or high turnover of district-level IT staff to manage the MIS and the slow pace of updating data into the registry) may persist and limit the benefits of the MIS. The technical risk is significantly mitigated, however, by the issuance of MOLISA’s Ministerial Circular in July 2019 on ‘Regulations on development, management, update and exploitation and use of SA and poverty reduction databases,’ which guarantees the sustainable and effective use of the new system at both the national and subnational levels. 87. Sustainability. The project introduced and supported three additional groups of beneficiaries for three years, which ended in June 2018. While Prime Minister’s Decision No. 488/QD-TTg in May 2017 provides a legal framework to introduce a national policy to support pregnant women and young children from poor households, including these groups of potential beneficiaries in SA programs needs to be through a Government decree. It is expected that revision of Decree No. 136, which is under way, will take care of this. The project also piloted using a social collaborator network, which was discontinued in the project provinces since June 2018 when the pilot ended. It was recommended that the use of social collaborators for the SA programs should rely on and take advantages of the existing network of social collaborators from other programs such as population programing and grassroots health support programs instead of putting in place another network that requires additional financing from the Government. The project activities supported the recent COVID-19 pandemic as many provinces used the MIS and quickly updated data to support implementation of the COVID safety net package by the Government. After COVID-19, the Government has made a stronger push toward integrated information system and e-payment in which the move to use an independent payment provider under the project was an important achievement (that is, introduction of VNPost). The usage of an independent payment provider such as VNPost has been continued in the project provinces and further expanded to the other 55 provinces. Despite these recent efforts, the sustainability of the innovations introduced thus remain low depending on the enforcement from high level and sub-national level following implementation of Decision 708 on developing an Integrated Social Security System and revision of Decree 136. Page 35 of 52 The World Bank Social Assistance System Strengthening Project (P123960) V. LESSONS AND RECOMMENDATIONS Lessons Learned 88. Good coordination between MOLISA, MOF, MOET, and provinces brought positive results. The MOF and MOET were consulted, and they actively engaged in the process of project preparation and implementation. Consultation with and participation by various departments from the project provinces played a critical role in ensuring that the project implementation arrangements were in line with local practices and government regulations. In addition, these agencies also participated in supervision, monitoring, and consultation workshops to assess project progress and helped in finding solutions to deal with problems and challenges that arose during project implementation. On the other hand, they also provided timely advice on project management to ensure that relevant adjustments could be made so that the project activities remained relevant to the changing circumstances. 89. Involvement and proactive participation of various stakeholders at both the central and provincial levels right from beginning was a key factor during preparation to ensure proper and smooth changes during implementation. A major change introduced under the project was in the payment modality—replacing payment by local staff with an independent payment agency. The careful consultation with the local authority and staff on the proposed change has resulted in less opposition and smooth operation later on. Another example was consultation on additional beneficiaries. Some sign of opposition in supporting out-of-school children was observed during initial consultation and this become an obstacle during implementation. The lesson learned is that local staff’s feedback and opinions during preparation should be taken into consideration more seriously and should be dealt with up front. 90. Strengthening a delivery system through building a modern MIS and robust payment modality is the right thing to do even though there were challenges in implementation. It can serve as a vehicle to strengthen institutional capacity, standardize service delivery, and improve efficiency and effectiveness. The new MIS supports the management of a national beneficiary database, monitoring fund flows, regular payment, and produces automated reports for monitoring and reporting. The new MIS was piloted in the four project provinces and rolled out to 60 provinces. It serves as a foundation for Vietnam to develop an integrated social protection system, facilitate linking and data sharing for decision-making, and transform the payment system. The payment agency—VNPost—selected by the project has helped increase payment points at locations convenient to the beneficiaries who were satisfied with timely payments. Using the list of beneficiaries created by the new MIS, the service provider significantly reduced the administrative burden of DOLISA in the province. The third-party benefit payment approach was quickly adopted by other 55 provinces nationwide. 91. A system strengthening project needs more time for the concerned agencies and staff to gain familiarity, especially when implementation is highly decentralized, and in the context of government administrative reform, which reduces staff and increase staff turnover significantly. The Government did not make enough effort and did not have the capacity to operationalize the new MIS right from the beginning of the project because of the lack of experience and staff skills at the central and local levels. This was one of the reasons for implementation delay in the first year. As a lesson learned, it is recommended that capacity building and financial allowance arrangements for project management at all levels is important to be planned right from project preparation. Mobilization of adequate and qualified Page 36 of 52 The World Bank Social Assistance System Strengthening Project (P123960) staff at the CPMU and PPMUs is crucial to the success of implementation. In addition, a plan for maintaining and rewarding trained and skilled staff to take on responsibility to maintain system operations at the local levels plays an important role. Therefore, during project preparation, capacity building and staff training and retaining strategy and relevant institutional arrangements should be carefully designed. 92. As a strengthened delivery system is put in place, appropriate adjustments in resource allocation and staffing should be planned according to the new demand for operation of the system and the need to regularly update the beneficiary database. The project proved that the lack of designated staff and frequent staff turnover in provinces and districts could have negative effects on the system administration and database updates. Moreover, before consolidation, the budget for annual poverty update was managed by different units (National Poverty Reduction Office and its local offices) so that after consolidation no budget was available to cover the costs of updating data in POSASoft, which was managed by the CPMU. As programs are consolidated, budget management under existing programs should also be considered in advance during project preparation so that adequate funding could be made available to cover actual expenses for consolidating program activities. 93. Using the existing local social collaborator networks to the extent possible. The project established a village-level social collaborator network, but with mixed experience. The collaborators were volunteers and no previous qualification was required. The remuneration for the collaborators for their services was also low, which could not incentivize them to provide quality services to the beneficiaries. The functioning of social collaborators turned out to be limited. In the context of lack of human resources at local levels, it is recommended to take advantage of the existing network of local social collaborators from other programs such as population and grassroots health programs. 94. The information system developed under the project provides a solid foundation toward the development of an integrated social protection system. Recommendations have been made to build a dynamic delivery system, link and share the database within MOLISA and other relevant agencies, and enhance the infrastructure capacity of the information system. 95. The World Bank’s efforts in bringing global knowledge and TA were value-added to the specific project outcomes. The World Bank team mobilized resources through project preparation and implementation to provide TA from general know-how, training, study tours to the expertise in key areas, such as targeting, information systems, data management, and SA policy. The World Bank team supported MOLISA to set up the M&E system, which was crucial to evaluate the project innovations and will continue to inform future reform of Vietnam’s SA programs. 96. Last but not least, strong commitment of the project executing agency to changes and innovations designed by the project is key. To make any development project a success, the project executing agency must be a change agent or a change pioneer. Recommendations 97. Based on the project results, MOLISA should organize workshops/meetings with the Government and stakeholders (including the potential service providers) to disseminate the Page 37 of 52 The World Bank Social Assistance System Strengthening Project (P123960) implementation results, commitments among the parties, lessons learned, difficulties, challenges, and recommendations for program consolidation and further delivery system strengthening nationwide. 98. Reasonable financial mechanisms and incentives for the operation of the newly strengthened delivery system nationwide are needed. This will help not only to maintain operation of the new system, but also to facilitate further expansion of innovations in service delivery and consolidation with other programs that MOLISA is managing, as well as data sharing with the social insurance program managed by the Vietnam Social Security. 99. At the MOLISA level, additional resources and staffing would be needed for the designated departments which take over the responsibilities to maintain and operate the system. Especially, the SA department will be in charge of using the system to manage the SA programs at the central level and provide timely information for decision-making and planning to the MOLISA leaders and managers. This department will need skilled staff as specified in the Government’s Project Completion Report. It would be helpful to anticipate these needs at project preparation and have them to be reflected in certain form of commitment from both central and sub-national governments. . Page 38 of 52 The World Bank Social Assistance System Strengthening Project (P123960) ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS A. RESULTS INDICATORS A.1 PDO Indicators Objective/Outcome: Support the government of Vietnam in strengthening the social assistance system Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Information consolidation as Number 0.00 50.00 60.00 measured by the number of provinces with at least 01-Jul-2014 31-Dec-2019 31-Dec-2019 partially digitized household information in the national beneficiary database Comments (achievements against targets): Achieved, exceeded the target. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Process consolidation as Number 0.00 40.00 60.00 Page 39 of 52 The World Bank Social Assistance System Strengthening Project (P123960) measured by the number of 01-Jul-2014 31-Dec-2019 31-Dec-2019 provinces using the national beneficiary database and MIS Comments (achievements against targets): Achieved, exceeded the target. A.2 Intermediate Results Indicators Component: Strengthening the social assistance and poverty reduction system Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Creation and operation of Text No digitized database National beneficiary National beneficiary national beneficiary database database created and database created and ready to be linked to ready to be linked to MIS of other SA MIS of other SA programs programs 01-Jul-2014 31-Dec-2019 31-Dec-2019 Comments (achievements against targets): Achieved target. Page 40 of 52 The World Bank Social Assistance System Strengthening Project (P123960) Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion New program MIS Text Manual/ semi- MIS ready for roll-out MIS ready for roll-out established and in operation automated systems in additional provinces in additional provinces 01-Jul-2014 31-Dec-2019 31-Dec-2019 Comments (achievements against targets): Achieved target. Component: Launching a consolidated social assistance program Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Share of beneficiaries Percentage 65.00 75.00 85.00 93.70 reporting satisfaction with the payment process 30-Jun-2015 29-Jun-2018 29-Jun-2018 29-Jun-2018 Comments (achievements against targets): Achieved, exceeded the target. Page 41 of 52 The World Bank Social Assistance System Strengthening Project (P123960) Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Share of benefit payments Percentage 0.00 75.00 80.00 96.50 made on time 01-Jul-2015 29-Jun-2018 28-Jun-2018 29-Jun-2018 Comments (achievements against targets): Achieved, exceeded the target. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of registered Number 0.00 22000.00 22000.00 18909.00 individual additional beneficiaries 30-Jun-2015 29-Jun-2018 29-Jun-2018 29-Jun-2018 Comments (achievements against targets): Partially achieved target. Page 42 of 52 The World Bank Social Assistance System Strengthening Project (P123960) B. KEY OUTPUTS BY COMPONENT Objective/Outcome: To support the government of Vietnam in strengthening the social assistance system 1. Process consolidation as measured by the number of provinces using the national beneficiary database and management information system (MIS); and Outcome Indicators 2. Information consolidation as measured by the number of provinces with at least partially digitized household information in the national beneficiary database Component 1. Strengthening the Social Assistance and Poverty Reduction System • Creation and operation of national beneficiary database; • New program MIS POSASoft established and in operation. Intermediate Results Indicators Component 2. Launching a consolidated social assistance program • Share of beneficiaries reporting satisfaction with the payment process • Share of benefit payments made on time; • Number of registered individual additional beneficiaries Component 1. Strengthening the Social Assistance and Poverty Reduction System • a national database of poor/near-poor households and SA beneficiaries Key Outputs by Component • an integrated MIS for nationwide use (linked to the achievement of the Objective/Outcome 1) • a social collaborator network in the project provinces and capacity building activities • C4D activities in the project provinces; • a grievance redress mechanism in the project provinces; • an M&E system, BFS report in the project provinces; Page 43 of 52 The World Bank Social Assistance System Strengthening Project (P123960) • a report on lessons and recommendations from the pilot in project provinces for policy formulation at the national level. Component 2. Launching a consolidated social assistance program • Additional Beneficiaries identified and registered; and cash benefits payment to Additional Beneficiaries in project provinces • benefit payment to social assistance beneficiaries through an independent service provider (VN Post) in project provinces • provision of service fees to the VN POST of the Project Provinces • Provision of remuneration for village-level social collaborators Component 3. Project management and M&E • operation of the CPMU and PPMUs; • activities related to project implementation and M&E. Page 44 of 52 The World Bank Social Assistance System Strengthening Project (P123960) ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION A. TASK TEAM MEMBERS Name Role Preparation Christian Bodewig Task Team Leader Puja Vasudeva Dutta Task Team Leader Hoai Van Nguyen Procurement Specialist Cung Van Pham Financial Management Specialist Tuan Anh Le Social Specialist Thuy Cam Duong Social Specialist Supervision/ICR Nga Nguyet Nguyen Task Team Leader Hoai Van Nguyen Procurement Specialist Cung Van Pham Financial Management Specialist Pablo Ariel Acosta Team Member Thao Cong Nguyen Social Specialist Son Van Nguyen Environmental Specialist Thu Ha Le Counsel Van Cam Nguyen Team Member Nga Thi Nguyen Team Member Page 45 of 52 The World Bank Social Assistance System Strengthening Project (P123960) B. STAFF TIME AND COST Staff Time and Cost Stage of Project Cycle No. of staff weeks US$ (including travel and consultant costs) Preparation FY11 8.975 18,851.75 FY12 7.507 19,234.31 FY13 12.725 61,140.55 FY14 19.825 87,633.28 FY15 0 0.00 Total 49.03 186,859.89 Supervision/ICR FY14 13.025 42,994.36 FY15 44.454 157,465.24 FY16 29.900 122,155.04 FY17 20.800 91,073.90 FY18 16.295 108,360.92 FY19 15.650 106,264.32 FY20 17.170 122,311.13 Total 157.29 750,624.91 Page 46 of 52 The World Bank Social Assistance System Strengthening Project (P123960) ANNEX 3. PROJECT COST BY COMPONENT Components Amount at Approval Actual at Project Percentage of (US$, millions) Closing (US$, Approval (US$, millions) millions) Strengthening the social 27.00 18.40 68 assistance and poverty reduction system Launching a 30.00 6.20 21 consolidated social assistance program Project management 5.50 3.50 64 Total 62.50 28.10 45 Note: US$5.5 million under Project Management consists of US$3 million IDA Credit and US$2.5 million counterpart funds, while the actual cost of US$3.5 million includes US$2.5 million IDA financing and US$1 million counterpart funds. Page 47 of 52 The World Bank Social Assistance System Strengthening Project (P123960) ANNEX 4. EFFICIENCY ANALYSIS 1. Efficiency improvements brought about by the project generated savings but are not readily quantifiable. The modernization of administration of SA programs supported by the project resulted in efficiency gains. Establishment of an MIS to modernize SA benefit administration and prevent, detect, and remedy fraud and errors improved efficiency for all programs. Improvements in service delivery (payment delivery, grievance redress) are also likely to improve efficiency and generate savings. However, conceptual and data limitations prevent quantification of these gains and savings. As such, cost- effectiveness estimations presented are limited to the rolling out of the registry and POSASoft (MIS), which presents a challenge in itself given that estimating and comparing registries is not straightforward. Registries incur different types of administrative costs and the costs are spread out over time. 2. The cost for the development of POSASoft can be grouped in three categories. The three categories include design and development, computer and hardware, and training and technical support. Table 4.1. Cost Breakdown for MIS (POSASoft) Item Amount (US$) Design and development of database and MIS 4,027,669 Computers and other hardware 4,264,726 Training and technical support 6,741,912 Total 15,034,307 3. It is estimated that rolling out the database and POSASoft in the country cost approximately US$2.59 per household registered in the database. The total cost of rolling out the database and MIS across 5.8 million15 SA beneficiaries and poor/near-poor households in the country was approximately US$15 million. This included the firm’s cost to design and develop the software and database and fixed assets (computers, servers) for all 58 provinces. 4. Thus, the cost of developing the MIS and database was minimal in relation to the volume of beneficiaries it can serve. The estimated cost at US$2.59 per household is minor relative to savings by improved monitoring and management among SA programs. As a comparison, Turkey’s integrated SA System cost US$1.3 per household,16 excluding training costs. Deducting training costs from POSASoft would result in a per household cost of US$1.43. The database and MIS enable administrators to have timely access to program indicators for operational management and policy making; as such it also supports planning, assessing potential demand, M&E, reporting, and other analytics. 5. The MIS platform was developed by a third party with strong government ownership. While the design and development of the software for the database’s information system was contracted out to an external firm, the process and arrangements were government led and government owned. The MIS platform builds on extensive multisectoral consultation and coordination across offices within MOLISA, Page 48 of 52 The World Bank Social Assistance System Strengthening Project (P123960) such as Department of SA and the Poverty Reduction Office, and provincial offices. This means that the ownership and knowledge of the database and POSASoft span a broad spectrum of interested stakeholders, including those responsible for the SA programs. The MIS is built on internal capacity for designing, managing, and maintaining the system, which allows for greater flexibility as the system evolves over time. 6. Going forward, the vision for POSASoft is to continue its expansion to serve other social protection programs and beyond. The database is well-positioned to support additional social protection programs such as those targeted to the poor, including education, health and nutrition, and livelihood programs. To achieve this goal, POSASoft will need to build on the existing foundations and strengthen areas addressed in spot checks and evaluation reports: adequate human resources (district staff with IT skills and capacity to run the MIS) and appropriate guidelines and processes to encode and upload data regularly. Improvements and updates of the MIS capabilities would allow for automated cross-checks between government databases (for example, with the social security database operated by Vietnam Social Security), which would improve the efficiency in administering programs. This will allow policy makers to have timely/regular access to key program indicators contributing to improved efficiency. Page 49 of 52 The World Bank Social Assistance System Strengthening Project (P123960) ANNEX 5. BORROWER, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS MINISTRY OF LABOUR, INVALIDS AND SOCIAL AFFAIRS CENTRAL PROJECT MANAGEMENT UNIT OF SOCIAL ASSISTANCE SYSTEM STRENGTHENING PROJECT 9th Floor, Song Hong Building, 2 Tran Hung Dao Street, Hanoi, Vietnam Tel: (84-4)3933.1957. Fax: (84-4)3933.3037. Email: cpmu@sassp.org.vn Hanoi, May 28, 2020 To: Mrs. Nguyen Nguyet Nga, Task Team Leader (TTL) The World Bank in Vietnam, 63 Ly Thai To, Hanoi, Vietnam C/c: Ms. Nguyen Cam Van (Team Assistant) Dear Mrs. Nguyen Nguyet Nga, Vietnam: Credit 5354VN - Social Assistance System Strengthening Project Center Project Management Unit of comment on World Bank’s ICR CPMU of Social Assistance System Strengthening Project has studied ICR and has some comment on this report as below: 1. Overall: There are some typing mistakes and unclear words in the report. Detailed comments please read in ICR. 2. About CPMU’s viewpoint on integrating: In Government ICR as well as State Audit Report, we have presented this point. Detail as: • According to the approved Feasibility Strategy (FS), the overall goal of the Project is to "build a management and implementation system of social assistance programs aimed at ensuring a minimum standard of living for all people, contributing to reducing Sustainable poverty in the spirit of Resolution No. 15-NQ / TW”. This overall goal expressed by 05 specific objectives, including the goal of “Proposing a roadmap for consolidating poverty reduction and social asistance policies/programs managed by various government agencies into a single household package“. Therefore, the initial goal set out for the Project was proposing a road map for consolidation of poverty reduction and social assistant policies/programs based on experience from piloting consolidation of four existing policies and program for 3 additional beneficiaries under the Opportunities Program. Basing on the results of implementating Opportunity Program, the Project would propose to the Government the direction of consolidation of various programs into a single household package. So the output of the Project is not a consolidated support program. • Result from pilot program shown that Opportunity Program has been basically successful due to the following reasons: (i) Information consolidation has been achieved through collecting and putting information for all households and their members under the POSASoft Page 50 of 52 The World Bank Social Assistance System Strengthening Project (P123960) database for the whole country, each family and member has unique identity number in MIS POSASoft; (ii) Social Assistance Payment has been implemented by an independent payment service provider (PSP) - VNPost (Minister of Labour, Invalids and Social Affair (MOLISA) signed Minute of Understanding with VNPost at central level, and DOLISAs signed contract with VNPost at local level for benefit payment service). In the past, the payment was carried out by commune officer. But after 5 years of pilot, the payment modality through independant service provider (VNPost) that was applied in 4 project provinces, has been extended to 60/63 provinces across the country. This could be considered as a significant achievement in reforming payment method toward more independent PSP; (iii) 2 out of 3 groups of addinational beneficiaries have been included as additional beneficiaries in Government Master Plan for Social Assistance Reform under Decision No. 488/QĐ-TTg dated April 14th, 2017 by Prime Minister. • The only commiment that was not achieved is one on monthly payment for all programs. Funding for education allowance and electricity subsidies were only made available by local government on semi-annual and quaterly basic. Therefore these benefits were not able to consolidated monthly, but were consolidated at time that funding was made available. In addition, payment was still made by the same payment service provider. • In the Government's Project Completion Report, the road map for policies consolidation and experience lessons have been complited. It recommended that certain sequence should be taken to consideration when consolidating SA programs, starting from consolidation and integration of information, to business process consolidation, and policy consolidation. • Moreover, the piloting the use of MIS POSASoft in conducting daily transactions of social assistance programs especially benefit payment in 4 project provinces has achieved its goal. The system has been operating in these four piloted provinces. During COVID-19, these provinces have used POSASoft to indentify beneficiaries and make payment quickly to existing beneficiaries. Many other provinces are in process of updating data, some provinces (such as Cao Bang, Quang Tri,) have also used POSASoft for COVID-19 income support payment. 3. Thus, in general, the objectives set out in the Feasibility Study have been achieved. An important lesson drawn from this project is that consolidation of policies/programs managed by different sectors (such as MOET, MOLISA and other ministries/sectors) should be done and this is right thing to do. However, it should be noted that policy/program consolidation will only be possible when it is enacted by government regulation, probably a decree or higher. Views on 3 additional beneficiary groups: 4. We think that the project has partially achieved it objective regarding the pilot of expanding benefits to 3 additional groups in the Opportunities Program. In practice, two out of three groups have been included as potential benefiaciaries in Master Plan for Social Assistance Reform and Developemnt in Decision No. 488/QĐ-TTg dated April 14, 2017 by the Prime Minister. This must be considered to be successful in leading to changes in project design and intervention, and it should not be considered as unsuccessful pilot. Page 51 of 52 The World Bank Social Assistance System Strengthening Project (P123960) 5. In addition, it would be useful to carry out assessment of the possible impacts the project may have but as it may take interventions some time (5 years, for example) to make impacts so it could not be feasible to see any impacts during project time. Page 52 of 52