Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Statistical Capacity Building Project (P144139) Report Number: ICRR0022438 1. Project Data Project ID Project Name P144139 Statistical Capacity Building Project Country Practice Area(Lead) South Sudan Poverty and Equity L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IDA-55370 30-Jun-2020 4,533,922.18 Bank Approval Date Closing Date (Actual) 16-Jul-2014 30-Jun-2020 IBRD/IDA (USD) Grants (USD) Original Commitment 9,000,000.00 0.00 Revised Commitment 9,000,000.00 0.00 Actual 4,533,922.18 0.00 Prepared by Reviewed by ICR Review Coordinator Group Judyth L. Twigg Clay Wescott Sengphet Lattanavong IEGEC (Unit 1) 2. Project Objectives and Components DEVOBJ_TBL a. Objectives The project's development objective was to strengthen the capacity of the statistics producers in the territory of the Recipient to produce and disseminate official statistics (Financing Agreement [FA], Schedule 1, page 4). The PDO statement in the Project Appraisal Document (PAD) is consistent with the FA, with a slight modification replacing "the territory of the Recipient" to "South Sudan." This review will use the PDO in the FA. Page 1 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Statistical Capacity Building Project (P144139) b. Were the project objectives/key associated outcome targets revised during implementation? No c. Will a split evaluation be undertaken? No d. Components The project had five (5) components: 1) Legal Reforms and Institutional Development ($0.34 million at appraisal; $0.31 million actual). This component had three sub-components involving the following: (i) update Recipient’s legal and regulatory framework governing statistics; (ii) improve governance and accountability of statistics producers, facilitating producer cooperation and strengthening the dialogue between official statistics producers and user community through relevant forums and working groups; and developing policies, protocols, and memoranda for collaboration; and (iii) reorganization of National Bureau of Statistics (NBS) structures, policies, and procedures to align with corporate priorities. All the sub-components were to be achieved through training and provision of technical advisory services. 2. Human Resource Development and Training ($0.86 million at appraisal; $0.6 million actual): This component was to support building the capacity of data producing agencies, including: (i) the human resource function of the NBS (review of scheme of service of government statisticians, implement performance-based compensation system, and improve human resource policies and procedures); (ii) training plan for NBS staff and managers; (iii) statistics curriculum for public universities in South Sudan and placement program for students to NBS; (iv) mentoring program for key staff and managers of NBS; and (v) assessment of competencies and training needs. 3. Information and Communication Technology and Statistical Infrastructure Development ($1.2 million at appraisal; $0.39 million actual). This component was to build the recipient's capacity on information and communications technology (ICT) and statistical infrastructure, including: (i) production of sample frames for surveys; (ii) development of registers to supplement or facilitate statistical operations; (iii) development of standards, concepts, definitions, and protocols for statistical operations; (iv) database development; (v) ICT investments; (vi) development of Geographical Information Systems (GIS); (vii) development of census tools; and (viii) establishment of quality assurance frameworks for official statistics. 4. Data Development and Dissemination ($4.7 million at appraisal; $1.14 million actual). The component was to support the main data generating activities for this project, including the following: (i) statistical surveys and censuses; (ii) poverty maps and other analytical products; and (iii) data compilation frameworks, such as the system of national accounts. 5. Project Management ($1.9 million at appraisal; $2.069 million actual). This component was to support strengthening capacity for project planning and budgeting, management, coordination, and monitoring and evaluation (M&E) activities. e. Comments on Project Cost, Financing, Borrower Contribution, and Dates The project was to be financed through an investment loan of $9 million. There were exchange rate losses of $0.847 million between the U.S. dollar and SDR. At closure, the total amount disbursed was $4.53 Page 2 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Statistical Capacity Building Project (P144139) million, or 55% of what was planned, due to planned actions being cancelled due to political volatility and conflict. No Borrower contribution was envisaged. A Project Preparation Advance of $1.97 million became effective on March 4, 2014. The project was approved in July 2014 and became effective on February 13, 2015. It closed at the original closing date of June 30, 2020. There were no restructurings. 3. Relevance of Objectives Rationale The project's objectives were clearly articulated. They were highly relevant to country context, government strategy, and Bank strategy at appraisal through closing. At the time of approval, South Sudan was emerging from 50 years of conflict. Complementary to the other activities of nation building, there was a pressing need for reliable data to guide the government's reconstruction and rehabilitation efforts. Implementation of the priorities articulated in the government's 2011-2018 South Sudan Development Plan (SSDP) would not be monitorable without timely, reliable, and accurate statistical data. The objective of strengthening the capacity for statistics production and dissemination was therefore highly relevant to country context. The project's objectives were also highly relevant to government strategy. Prior efforts to develop the statistical system had been patchy, uncoordinated, and partly driven by availability of external resources. There had been few opportunities to develop a long-term, sustainable, coherent, and realistic strategy for development of a national statistical system. Development of such a system was listed as an important priority under the governance programmatic area of the SSDP. In 2013, the government adopted a National Strategy for Development of Statistics (NSDS) 2014-2019 as a vehicle for bringing structure and medium- term strategic planning to the statistical service. The Strategy's main objectives were strengthening the legal and institutional framework for the statistics system, improving human resource capacity for production and management of statistics, developing statistical infrastructure, developing and managing data, and improving physical infrastructure and equipment. The NSDS was the guiding document for the project. Implementation of the NSDS was to establish the NBS as the apex organization of the country's statistical system. NBS was to set standards and policies, coordinate all official statistics production, and serve as a one-stop shop for users of official statistics. At the time of appraisal, the Bank aimed to support two clusters of activity in the country: improving economic management and governance for effective local service delivery, and expanding productive employment opportunities. Support for the statistics sector fell under the first cluster. One of the pillars of the Bank's early analytical and advisory program was governance-improving activity to build public sector capacity. Prior support had included strengthening the South Sudan Center for Census and helping with the development of poverty studies, census statistics and evaluation, statistical capacity, and the use of innovative techniques for opinion polling. The project's objectives were a logical next step in the Bank's support, appropriately pitched for the country's development status and capacity. The Bank's Interim Strategy Note, published in January 2013, noted that building systems for data collection and utilization to inform policy decisions was a key area of focus in the short to medium term. The Country Engagement Note (CEN) for the FY2018-2019 period responded to the crisis facing South Sudan following an intensification of violence, aiming to support stabilization, recovery, and peace where opportunities arose. Page 3 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Statistical Capacity Building Project (P144139) Support for statistical capacity building, particularly data collection, was critical to achievement of the CEN's first objective to support basic services for vulnerable populations. The Bank's Country Engagement Concept Note for FY2021-2022 highlights the importance of investing in institutions, underscoring the importance of strengthening national capacity for data collection and dissemination (ICR, p. 11). Rating Relevance TBL Rating High 4. Achievement of Objectives (Efficacy) EFFICACY_TBL OBJECTIVE 1 Objective Strengthened capacity of statistics producers in South Sudan to produce official statistics Rationale The theory of change for this objective was comprehensive and credible. It assumed that the key challenges to be overcome were an inadequate legal and regulatory framework governing statistics, out-of-date statistical methodologies, inadequate human resources and institutional capacity for statistical systems, and lack of necessary technological, physical, and statistical infrastructure. Capacity to produce and disseminate statistics was to be developed through an integrated support package delivered to the national statistical system, focusing on building human capital as well as physical and statistical infrastructure at the NBS and ministries, departments, and agencies (MDAs). Support for legal reforms and institutional development, in the form of development of a legal framework for official statistics, review of the existing statistics act, and drafting of a bill to amend that legislation, was reasonably expected to lead to strengthened capacity. Provision of technical assistance and training for NBS and data producing agencies, through development of organizational structures, human resource policies, and training for NBS and other MDAs, would also produce relevant capacity strengthening. The development of a quality assurance framework for statistics that would include new policies, procedures, and tools, as well as investments in ICT and plans for a new headquarters building for the NBS, were to provide the necessary technological and physical infrastructure to support the new governance frameworks and enhanced human resources. Outputs The project financed the development of an NBS human resources strategy, training needs assessment, and training plan. A total of 557 staff from NBS and MDAs (of whom 110 are female) were trained in basic statistics and computer applications. The government approved and implemented a 40% salary increase for NBS staff. Statistics units were established at key MDAs, which hired new statistics graduates as part of the project's graduate placement program. The NBS worked with the University of Juba to develop a statistical Page 4 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Statistical Capacity Building Project (P144139) training program including a curriculum, syllabi, and an internship program. A Statistics Bill was developed and revised, and the ICR (p. 14) reports that it has undergone a second reading and is ready to be enacted by the Parliament. The project team confirmed that the bill has not yet passed, at least in part because South Sudan’s Parliament was dissolved and/or not at all active for a large portion of the last two years of the project’s life. A national compendium of statistical standards, classification, concepts, and definitions was developed and adopted to improve harmonization and consistency in data sets among producers of statistics. A Crisis Recovery Survey (CRS) and Integrated Business Enterprises Survey (IBES) were conducted with project support. However, the planned Household Budget Survey (HBS)—whose budget was 28% of the overall project budget—was not conducted due to ongoing political volatility and insecurity. According to the ICR (p. 16), the HBS was the most important and relevant planned survey activity under this objective, as it would have provided key statistical data needed for compiling national accounts (including GDP). Outcomes The Statistical Capacity Score (SCI) is a Bank rating system of countries' statistical capacity. South Sudan's SCI improved from 26.0% in 2013 to 45.6% in 2020, exceeding the target of 44.0%. Scores improved across all three components of the SCI: methodology, source data, and periodicity. The Tool for Assessing Statistical Capacity (TASC), developed by the United States Census Bureau, measures institutional capacity, census/survey planning and management, mapping, sampling, questionnaire design and testing, field operations, data processing, data analysis and evaluation, and data dissemination. South Sudan's TASC score improved from 47.5% in 2015 to 54% in 2020, exceeding the target of 52.3%. The TASC assessment assigns equal weight to sub-ratings for institutional capacity and combined operational modules (including mapping, sampling, questionnaire content and testing, field operations, data processing, data analysis and evaluation, data dissemination, publicity and mobile data capture summary, and administrative records). The ICR (p. 13) reports that progress on the institutional capacity score drove overall improvement in the TASC indicator, with disproportionate gains in the data confidentiality/protection, human/physical capital, and stakeholder coordination dimensions, all of which were driven by the project's interventions. Rating Substantial OBJECTIVE 2 Objective Strengthened capacity of statistics producers in South Sudan to disseminate official statistics Rationale The theory of change for this objective was logical and credible. It assumed that the key challenge to be overcome was low quality and limited access to statistical data. The project's support for a new NBS website, a GIS web-database, and revived micro-data library, as well as the conducting of new surveys whose data Page 5 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Statistical Capacity Building Project (P144139) sets and analytical reports would be documented, archived, and made publicly available, was to increase capacity to disseminate statistical data and analysis. Outputs An NBS ICT needs assessment was conducted. Computers and licenses for a GIS laboratory were purchased, and a GIS web-database with GIS data-layers and a revived micro-data were made available on- line. The number of GIS data-layers accessible through the GIS web-database increased from 4 in 2013 to 16 in 2020, meeting the target of 16. Eight micro-datasets were made available on-line (currently available in the World Bank Microdata Library), almost meeting the target of nine datasets. The NBS website (https://ssnbs.org) was upgraded. The project supported statistical advocacy and sensitization through printing and distributing the NSDS, a statistical compendium, NBS strategic plans, annual reports, statistics regulations, a code of practice for official statistics, an ICT policy, statistical yearbooks, and CRS and IBES data sets and analytical reports. The NBS library was renovated and equipped with computers, furniture, and statistics books and materials; it is now open to the public. Where internet connectivity was a challenge, the NBS disseminated products through workshops and physical distribution of its products to relevant users. The project supported the NBS's first User Satisfaction Survey in 2017. Outcomes The ICR uses the SCI and TASC scores as indicators of achievement for both statistics production and statistics dissemination. The SCI includes a dimension for data dissemination, but no specific information on this dimension is provided. Furthermore, no information is provided on the results of the 2017 User Satisfaction Survey. Nevertheless, the output indicators on statistical advocacy, sensitization, and distribution are sufficient, together with the overall SCI and TASC ratings, to constitute evidence of substantial achievement for this objective. Although not directly a part of the project's objectives, the project team also provided information on the use of statistics for evidence-based policy making. CRS data fed directly into the Bank's South Sudan Poverty Assessment. Data from the CRS were incorporated into the Project Targeting Index online platform that was developed to support a sound methodological approach to the selection of project sites (responding, in part, to observations that projects were being placed only in government-controlled areas). Data from the CRS also fed into the updated Country Engagement Note and the Risk and Resilience Assessment. The CRS was the first survey that collected comprehensive welfare data on the country's internally displaced population, and the project team reported that there were many engagements on using this data for programming with donor partners in Juba, particularly UNICEF. Similarly, IBES data fed directly into a recent overarching policy document for the government's Jobs, Recovery, and Peacebuilding agenda, one of four formal input pieces assessing various aspects of jobs and employment in order to formulate policy for recovery. The project team did not report on use of statistics by universities, think tanks, or non-governmental organizations. Page 6 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Statistical Capacity Building Project (P144139) Rating Substantial OVERALL EFF TBL OBJ_TBL OVERALL EFFICACY Rationale The level of improvement on two key statistical capacity indices, as well as information on statistics dissemination activities, are convincing evidence of substantial achievement of both objectives under extremely challenging circumstances. Even without the planned HBS, the project achieved significant outputs (training and regulatory) that led directly to achievement of intermediate outcomes (national compendium of standards, CRS, IBES); a convincing results chain led from Bank-financed interventions to improvements in the SCI and TASC. Improvement in the TASC rating was driven mostly by gains on the institutional capacity dimension, which was directly supported by the project. The PAD and ICR note that several other development partners (DPs) were active in South Sudan's statistics sector, but no information is provided in the ICR on division of labor. The project team later added that project planning and implementation carefully avoided duplication of efforts with other DPs. The NSDS was adopted in close collaboration between the Bank and other DPs. The main DP working with NBS was the United Nations Population Fund (UNFPA), working mainly to update the sampling frame for the national census. The Bank team coordinated closely with UNFPA both during missions and virtually in between missions. The strong conceptual linkages between the project's implemented activities and observed outcomes, as well as smooth collaboration with DPs, make it plausible to attribute observed outcomes, at least in part, to the project. Overall Efficacy Rating Substantial 5. Efficiency A traditional cost-benefit analysis was not attempted at appraisal, as it was not possible to assign a specific value to enhanced ability to make better decisions based on improved statistical data (PAD, p. 14). The PAD observed that South Sudan was a relatively high-cost environment for statistics, meaning that few administrative procedures were in place to record household interactions with government, and internet-related data sources were scarce or incomplete. This meant that the only way to obtain statistics on social and economic phenomena was to survey households and enterprises at the source, at home or at the place of business. The most cost- effective way to conduct such surveys is to maintain adequate sampling frames from which to draw appropriately-sized samples and administer questionnaires to a representative subset of the population. Page 7 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Statistical Capacity Building Project (P144139) This project's interventions to build capacity to create and maintain adequate sampling frames, and to draw efficient samples, were intended to save resources and improve data reliability over time. The ICR echoes the PAD in not attempting a traditional economic analysis. It notes that the completion of the CRS and IBES, as well as improvement in the informational foundation of the country as evidenced by improvement in the SCI and TASC scores, can be expected to improve targeting of public services and infrastructure and to encourage investment that will create jobs. In terms of operational efficiency, the ICR (p. 17) notes that the sequencing of project activities was carefully designed and implemented, with training and capacity-building preceding data-generating activities. In addition, the budget for implemented surveys was efficiently used. The cost per interview for similar surveys taken in Ghana ($251), Democratic Republic of Congo ($692), and Ethiopia ($90) was higher than that for the IBES in South Sudan ($68). However, there were significant operational inefficiencies due to the country's political situation and insecurity. Some areas of the country were inaccessible to NBS staff, presenting challenges to survey field work. Low oil prices and increased insecurity strained the government's fiscal space, producing "extremely high" staff turnover at the NBS (ICR, p. 19), low staff commitment, and regular electricity/power and internet service outages (ICR, p. 23). The planned 2014 Population and Housing Census was not conducted due to ongoing volatility; that census was to have provided an updated sampling frame for the 2015 National HBS, which was also not conducted due to the security situation. The government requested in December 2017 that the project be restructured to reallocate funds from the HBS to other activities, including expanded legal reform, institutional development, training, and additional traditional surveys. Across 2018 and 2019, delays in acting on this restructuring request were due to negotiations between the Bank, government, and potential implementation partners about moving to third party implementation across the Bank’s entire portfolio in South Sudan. The project team for this project advocated for keeping its implementation within the NBS, arguing that achieving the objectives would be difficult, if not impossible, with implementation from another entity; ultimately, it was agreed not to move to a third-party implementer for this project only. Shortly after this decision was taken, the government went into arrears with the Bank, resulting in portfolio-wide suspension of disbursements in the country across 2019-2020. The project restructuring was never finalized, and it closed without disbursing funds for HBS and some other planned activities. Project efficiency is therefore rated Modest. Efficiency Rating Modest a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 0 Appraisal 0  Not Applicable 0 ICR Estimate 0  Not Applicable Page 8 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Statistical Capacity Building Project (P144139) * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome The project's objectives were highly relevant to Bank strategy, government strategy, and the country's development needs. Achievement of both objectives is rated Substantial, based on improvements in key indices of statistical capacity as well as evidence of gains in the dissemination of statistical data and analysis. Efficiency is rated Modest due to impacts of the volatile political and security situation on project implementation, including inability to implement a major planned household budget survey. These ratings are indicative of moderate shortcomings in the project's preparation and implementation, producing an Outcome rating of Moderately Satisfactory. a. Outcome Rating Moderately Satisfactory 7. Risk to Development Outcome Many of the key risks identified during project design remain relevant after project closure: continued conflict and macroeconomic instability, the latter resulting in part from the dependence of public revenue on oil prices; uncertainty around flow of government resources to the statistics sector; the commitment of MDAs to development of statistical capacity; and the weak capacity of NBS and MDAs' statistics units. The project achieved important institutional strengthening through updating of legal frameworks around the production of statistics, improved governance of official statistics producers, and reorganization of the structures, policies, and procedures of the NBS. Those gains are likely to be sustained. The academic collaboration with Juba University, as well as the ICT and statistical infrastructure developed to support statistical business processes, continue to function as the project intended. The Parliament's failure to enact timely passage of the Statistics Bill stands in the way of sustainable institutional reform and organizational change; passage of the Bill would increase stakeholder responsibility and accountability. The ICR does not provide information on the barriers to passage of the Bill or on the likelihood of its adoption in the future. 8. Assessment of Bank Performance a. Quality-at-Entry Project design was based on the goals, gaps, and needs outlined in the NSDS 2014-2019 as well as intensive consultations with staff and management of NBS and other statistics producers, main users of statistical products, development partners, and other stakeholders (PAD, p. 15). Design was in line with international best practice as recommended by the Partnership in Statistics for Development in the 21st Century, and it was similar to the design of many other statistical capacity building projects financed by the Bank over the preceding decade. Overall risk at appraisal was rated High. The most important identified risks at appraisal pertained to financial management and procurement capacity. Risk mitigation included measures for targeted technical assistance to support weak functions and areas; development Page 9 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Statistical Capacity Building Project (P144139) of detailed manuals, policies, and operating procedures in these areas; strong support from the Bank for overall project implementation; and efforts to build a coalition of senior government officials to support and facilitate implementation of the NSDS. In retrospect, governance risk was underestimated, and more consideration could have been given to the fragility context: the government's fiscal situation and management capacity, the overall security situation, and the Parliament's willingness to adopt the statistics bill. Several other development partners were active in the statistical sector in South Sudan, most notably UNFPA and Statistics Norway, which had a long-standing cooperation with the NBS. According to the PAD, the Bank and Statistics Norway coordinated their programs to focus on separate areas of the NSDS, but it was acknowledged during project preparation that further coordination was needed in some areas, especially business statistics and ICT development (PAD, p. 50). Quality-at-Entry Rating Moderately Satisfactory b. Quality of supervision Factors under the Bank's control: The Bank team remained committed to project implementation despite the challenging context. Implementation Status Reports (ISRs) were produced regularly. Beginning in 2018, this was the only Bank-financed project in South Sudan that was not conducted through third-party implementation. Frequent changes in task team leadership (five in total over the course of the project) were disruptive, though the ICR (p. 23) asserts that there were no resulting information or decision gaps in the project implementation process. Additional fiduciary safeguards implemented by the Bank in August 2016 slowed disbursement and implementation of activities; the Bank team provided support for implementing staff in procurement and project management in order to resolve issues as they arose. The ICR does not explain why project restructuring was not acted upon more expeditiously after the government requested it at the end of December 2017, as the portfolio-wide suspension of disbursements did not occur until 2019. Factors outside the Bank's control: The volatile security situation negatively affected the implementation of project activities. The Bank team attempted to mitigate this challenge, in part, by adopting a flexible statistical sampling approach that allowed substitution of insecure areas with secure areas, prioritizing surveys like the IBES that focused on urban (more secure) areas, and hiring firms to cover inaccessible areas. The tight fiscal situation produced delays in government payments to contractors and civil servants. The Bank responded by facilitating direct payments to contractors, for which thresholds were lowered. However, this approach was not possible for payments to civil servants; as a result, civil servants "showed low motivation, unavailability, and contract terminations" (ICR, p. 23). The project attempted to compensate for delayed salary payments by providing the project team with improved work environments including consistent electricity, internet, and container offices. Quality of Supervision Rating Moderately Satisfactory Page 10 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Statistical Capacity Building Project (P144139) Overall Bank Performance Rating Moderately Satisfactory 9. M&E Design, Implementation, & Utilization a. M&E Design The project's objectives and theory of change were clear and monitorable. According to the project team, the Bank calculated baselines and progress on the SCI using inputs from the country poverty economist. The TASC involved a more complicated data collection exercise across many different modules and questions. For the baseline, a consultant who set up the TASC was hired and went to Juba to work with NBS and build capacity to collect and process these scores. The use of composite indicators of statistical capacity to measure achievement of the first objective (production of statistics) was appropriate, though additional outcome indicators -- including data from the appropriate constituent elements of the SCI and/or TASC -- could have provided more precise measurement of the second objective (dissemination of statistics). Attributable assessment of project achievement would have been strengthened by the specification of intermediate outcome indicators more in line with the project's scope, measuring results that were directly dependent on project interventions. The output indicator that measured the number of surveys conducted under the project provided no indication of the relative importance of these surveys, and therefore it did not capture the magnitude of the gap created by inability to carry out the planned HBS. The NSDS Coordinating Team (NCT) of the NBS was responsible for monitoring and reporting, drawing data from self-assessments, third-party assessments, and direct feedback from beneficiaries. Baselines and targets were to be derived from the TASC. b. M&E Implementation Baselines and targets were developed as planned. The indicators of project performance were thoroughly reviewed and assessed during each implementation support mission.The NCT regularly produced progress reports covering information on budget, capacity building, data production and dissemination, and fiduciary issues. These reports were candid about implementation challenges. The Bank monitored progress through regular implementation support missions. However, there were significant M&E implementation challenges. Full levels of M&E staffing were only in place for the first two years of the project's lifetime. After that point, some NCT staff resigned because of issues with salary payments, uncertainty created by the lack of approval of project restructuring, the suspension of disbursements across the Bank's portfolio in the country, and a general lack of implementation progress. The ICR does not provide information on how M&E reporting managed to continue under these conditions; the project team explained that the PIU was able to complete data collection for the composite outcome indicators through close supervision from the Bank. c. M&E Utilization The ICR (p. 24) states that data on project indicators were reported in ISRs and used to provide evidence on achievement of outcomes. There is no information provided in the ICR on the way that M&E data and analysis were used to inform project implementation, communicated to stakeholders, or Page 11 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Statistical Capacity Building Project (P144139) influenced strategic redirection or resource allocation. The project team later added information on use of project data and analysis to inform project implementation (the Project Targeting Index) as well as national policy and strategy through the South Sudan Poverty Assessment, Risk and Resilience Assessment, and Jobs, Recovery, and Peacebuilding agenda (see Section 4, Objective 2). M&E Quality Rating Substantial 10. Other Issues a. Safeguards The project was Environmental Assessment category C. No safeguard policies were triggered. The project team added that there were a number of safeguards implemented against the misuse of data, including:  Informed consent for all individual, household, and enterprise respondents in any survey;  Full anonymization of datasets in line with the Bank's data privacy policies;  Programming of questionnaires and storage of responses online in the Bank's Survey Solutions system; and  Anonymization, cleaning, and processing of data from both the CRS and IBES in line with Bank policies, and current storage and dissemination through the Bank's Microdata Library, which provides necessary background documentation along with the datasets. b. Fiduciary Compliance At commencement, the project was staffed by an experienced team that provided high-quality administrative and procurement documentation, enabling initially smooth review and clearance procedures. After the conflict in the summer of 2016, enhanced fiduciary measures were undertaken by the Bank, increasing the use of prior review procurements and more scrutiny of Interim Financial Reports and Withdrawal Applications. These enhanced measures were put in place not only for this project, but across the entire Bank portfolio for the country. At that point, project implementation became "markedly more sluggish" (ICR, p. 25). Financial management performance was rated Satisfactory until 2016, when it was downgraded to Moderately Satisfactory because of the impact of the conflict and subsequent tightened supervision measures. The final project audit flagged weaknesses in the system of internal control, leading to inability to account for 64 tablets valued at $15,506. This amount was therefore considered ineligible expenditure that should have been refunded to the Bank before the disbursement deadline date of December 31, 2020. The project team later stated that, to its knowledge, this amount still has not yet been Page 12 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Statistical Capacity Building Project (P144139) refunded. As a result, overall project financial management performance was considered Moderately Unsatisfactory. c. Unintended impacts (Positive or Negative) None reported. d. Other --- 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment Moderately Outcome Moderately Satisfactory Satisfactory Shortcomings in risk assessment at appraisal, and delay in restructuring the project Bank Performance Satisfactory Moderately Satisfactory when it became clear that key activities could not be implemented. Quality of M&E Substantial Substantial Quality of ICR --- Substantial 12. Lessons The ICR (pp. 28-31) provides numerous detailed lessons emerging from each stage of the project's life cycle. They include: Successful implementation of development projects in fragile contexts depends not only on addressing development gaps, but also on consideration of specific fragility and conflict dynamics. In this case, delays in project restructuring revealed lack of flexibility in adjusting to evolving realities, capacity/resource constraints, and risks on the ground. In fragile settings, the need may arise to incentivize project staff to overcome salary and staffing limitations. In this case, there was high turnover because NBS staff were not paid enough by the government to complete implementation of project-financed activities, a situation that grew worse as the macro-fiscal state of the country deteriorated. When outcomes are at risk due to deterioration of the security and/or fiscal context in a project that is otherwise performing well, the Bank could consider working with the client to find ways for key government staff to continue to receive adequate salaries. Page 13 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Statistical Capacity Building Project (P144139) Careful sequencing of activities is an important element of project design. In this case, prioritizing human capacity development during the project's initial phases fostered the development of ownership among statistical staff at the NBS and MDAs' statistical units; despite later challenges, this early strengthening enabled continuation of statistical production even in the face of ongoing conflict. 13. Assessment Recommended? Yes ASSESSMENT_TABLE Please Explain To explore further use of statistical data for improving service delivery, prioritizing the use of scarce resources, holding government accountable at all levels, and empowering individuals, in line with the findings of the 2021 World Development Report: Data for Better Living. Further assessment could also examine issues of data sharing, data re-use, and overall privacy safeguards. 14. Comments on Quality of ICR The ICR is clear, candid, and evidence-based. It follows established guidelines. It presents a comprehensive and plausible theory of change for the project, and it interrogates thoroughly the quality and adequacy of output and outcome indicators in the results framework. The ICR cogently assesses the various types of factors that impacted implementation progress: issues inherent in project design, factors under the Bank's control during supervision, and exogenous factors that were outside the Bank's control. Its extensive lessons, derived from the project's experience as described in the main text of the ICR, should prove highly useful for the design and implementation of other projects in the sector, the country, and other fragile contexts. There were some gaps in the ICR's information and analysis, including the reasons that the project was not restructured in a timely manner after the government's request in late 2017, the activities and division of labor with other development partners in the sector, and details on M&E implementation and utilization. Within the context of an otherwise strong document, these shortcomings are relatively minor, and the Quality of the ICR is rated Substantial. a. Quality of ICR Rating Substantial Page 14 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Statistical Capacity Building Project (P144139) Page 15 of 15