Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review ZR Catalytic Proj. Strengthen the INS (P125509) Report Number: ICRR0021991 1. Project Data Project ID Project Name P125509 ZR Catalytic Proj. Strengthen the INS Country Practice Area(Lead) Congo, Democratic Republic of Poverty and Equity L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) TF-16628 30-Jun-2017 11,037,433.06 Bank Approval Date Closing Date (Actual) 09-Jun-2014 28-Jun-2019 IBRD/IDA (USD) Grants (USD) Original Commitment 11,800,000.00 11,800,000.00 Revised Commitment 11,800,000.00 11,037,433.06 Actual 11,040,413.00 11,037,433.06 Prepared by Reviewed by ICR Review Coordinator Group Hjalte S. A. Sederlof Judyth L. Twigg Malathi S. Jayawickrama IEGEC (Unit 1) 2. Project Objectives and Components DEVOBJ_TBL a. Objectives The Project Development Objective (PDO) of the Catalytic Project to Strengthen the National Statistics Institute (INS) was to strengthen the capacity of INS to generate and disseminate statistical information (Grant Agreement, p. 5). b. Were the project objectives/key associated outcome targets revised during implementation? Page 1 of 11 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review ZR Catalytic Proj. Strengthen the INS (P125509) Yes Did the Board approve the revised objectives/key associated outcome targets? No c. Will a split evaluation be undertaken? Yes d. Components The project had three components: Component 1: Increasing INS technical capacity (estimated cost at appraisal US$0.86 million; actual cost US$0.97 million). The component was to increase the share of INS professional staff with formal degrees or on-the-job training in relevant areas: statistics, demography, economics, and information technology. This was to be achieved through: (i) pre-service formal training for competitive entry into regional statistical schools and subsequent scholarships tied to five-year binding contracts to work in the national statistical system; and (ii) on-the-job and thematic training for staff in various statistical tasks, such as survey data collection, quality control, and analysis. Component 2: Strengthening the non-technical environment (estimated cost at appraisal US$2.94 million; actual cost US$7.35 million). The component was to improve the working environment by: (i) improving general management in INS through management and leadership training, and coaching to apply new skills; (ii) fiduciary capacity development through the provision of relevant training, financial management software and information technology systems, and support to potential needs for reorganization of the fiduciary practice at INS; (iii) improving the physical working environment by rehabilitating buildings used by INS technical departments in Kinshasa; and (iv) providing funds for office supplies, communication, gas, and other operating expenses for the Project Management Team. Component 3: Statistical production and dissemination (estimated cost at appraisal US$7.32 million; actual cost US$2.70 million). The component was to help develop critical statistical infrastructure, including: (i) the key cartography and pilot census phases of a population and housing census that was to allow the development of a master sampling frame for surveys and sector planning and monitoring; (ii) improved national accounts by including the urban informal sector; and (iii) improved access to and interpretability of statistical information by developing a statistical dissemination policy based on official principles of United Nations statistics and the African Charter of Statistics; provision of software for archiving data and training in the use of the software; quality control; enhanced internet bandwidth to disseminate the information through the INS web site; and anti-virus software. Significant Changes During Implementation The project was restructured twice (both were Level 2 restructurings): A first restructuring was undertaken in June 2017. It included the following: Page 2 of 11 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review ZR Catalytic Proj. Strengthen the INS (P125509)  The three original outcome indicators were found to be too ambitious. They were revised to more accurately reflect plausible outcomes, focusing on staff training rather than actual production of statistical products, and two new indicators were added (these changes are discussed under Section 4).  Funds were reallocated to allow completion of an underbudgeted reconstruction of INS offices in Kinshasa. The funds were drawn from statistical production (the cartography of the population census and the support for national accounts), which was truncated in the project and partly reallocated to another Bank project (Statistics Development Project (P150148), $45 million, 2015- 2020).  The closing date was extended by 22 months to respond to delays at project start-up. Effectiveness was delayed by almost ten months due to processing of the grant in the legislature.  The demolition and reconstruction of the INS building triggered OP/BP 4.11 due to potential impacts of that activity on cultural resources. A second restructuring took place on January 3, 2019, extending the closing date to June 28, 2019 to allow ongoing work to be completed. e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project cost. At appraisal, the total cost of the project was US$11.8 million. Actual project cost at closing was US$11.02 million. Financing. The project was financed with a grant from the Statistics for Results Facility Catalytic Fund for an amount of US$11.8 million, of which US$11.02 million was disbursed. Borrower contribution. The government provided US$584,120. Dates. The project was approved on June 9, 2014 and became effective on January 20, 2015. Its original closing date of June 30, 2017 was postponed twice, a first time on June 28, 2017 to January 3, 2019, and a second time from January 3, 2019 to June 28, 2019, at which time it closed. 3. Relevance of Objectives Rationale Project objectives were relevant to country needs, government policy, and Bank strategy at the time of project development and implementation, building up the national statistical system in order to generate necessary socio-economic information for informed policy making. Government policy was articulated in a national strategy for the development of statistics, 2013-2017, that highlighted three major deficiencies: human resources, institutions, and access to statistical information. The Bank’s Country Assistance Strategy for FY2013-2016 recognized the need for better information to meet its objectives: the ability to measure growth through national accounts; address vulnerability and resilience through better knowledge of the population and its living conditions; and promote good governance by rapidly producing relevant Page 3 of 11 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review ZR Catalytic Proj. Strengthen the INS (P125509) data. Subsequently, a World Bank Systematic Country Diagnostic was issued for 2018, as well as a National Development Strategy, both underlining the importance of statistical capacity building in the DRC. Rating Relevance TBL Rating Substantial 4. Achievement of Objectives (Efficacy) EFFICACY_TBL OBJECTIVE 1 Objective Strengthen capacity of the INS to generate statistical information (original outcome targets) Rationale Originally, achievement of this objective was to be measured by two key outcome indicators: the number of national surveys completed using the new master sampling frame; and the annual production of a national accounts report, including informal and industrial sector estimates. The key outputs were increasing the proportion of INS professional staff with formal degrees or on-the-job training in statistical methods. These activities – upgrading staff and data production - were to prove too ambitious to be accomplished over the original project period, leading to the restructuring and a redefinition of project indicators and targets. Against the original output targets, 248 people benefited from technical training sessions, not meeting the original target of 991. 9% of beneficiaries trained by the project were admitted to regional schools of statistics, not meeting the original target of 18%. The original outcome targets for completing national surveys using the new master sampling frame and producing national accounts annually using updated informal sector and industrial estimates were not achieved. Given only preliminary progress toward achievement of activities that would have resulted in achievement of the original outcome targets, achievement of this objective under those targets is rated Negligible. Rating Negligible OBJECTIVE 1 REVISION 1 Revised Objective Strengthen capacity of the INS to generate statistical information (revised outcome targets) Page 4 of 11 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review ZR Catalytic Proj. Strengthen the INS (P125509) Revised Rationale Under the revised project, the scope of the project was made considerably less ambitious, with the outcomes defined largely as training of staff and completion of sampling frames rather than actual production of statistical products. New outcome indicators and targets were introduced (including the conversion of some output indicators to outcome indicators) as follows:  The number of new INS staff with formal degrees in statistics, informatics, or demography increased from zero in 2014 to 23 in 2018, equal to the revised target.  The number of INS staff trained in core fields of statistical work increased from zero to 248, exceeding the revised target of 180.  Demolition and reconstruction works for INS facilities were fully completed and deemed satisfactory.  Completion rates towards sampling frames: o Completion rates of activities required prior to starting the field work for the cartography of the population census were 50 percent at project closure, compared to a target of 100 percent; o Completion rate of activities required to put into place a master business sample frame for enterprise statistics reached 80 percent, equal to the target. Revised Rating Substantial OBJECTIVE 2 Objective Strengthen capacity of the INS to disseminate statistical information (original and revised targets) Rationale The objective in both its original and revised form was to be achieved by a combination of initiatives achieved under Objective 1 supporting the introduction of statistical products on the INS website. By project closing, a total of eight products had been introduced, compared to an original target of 33 and a revised target of 10. However, the website is no longer active due to lack of funds to maintain it with the closing of the project. Achievement of this objective is therefore rated Negligible under both the original and revised outcome targets. Rating Negligible OVERALL EFF TBL OBJ_TBL OVERALL EFFICACY Page 5 of 11 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review ZR Catalytic Proj. Strengthen the INS (P125509) Rationale With negligible achievement of the first objective and negligible achievement of the second objective under the original outcome targets, efficacy under the original outcome targets is rated Negligible. With substantial achievement of the first objective and negligible achievement of the second objective under the revised outcome targets, efficacy under the revised outcome targets is rated Modest. Overall Efficacy Rating Primary Reason Modest Low achievement 5. Efficiency Factors that point to an efficient use of project resources included spending on measures that were likely to underpin the targeting of future investments to high-return projects; investments in staff training to raise the quality of the national statistical system and reduce its dependency on external technical assistance; and providing a better working environment for INS staff with improved infrastructure and management skills. Factors that signaled inefficiencies included the long delays in implementation generated by slow startup and procurement challenges, the reduction in the scope of the project while spending the original amount; and 12 months to restructure, while requiring a second restructuring. The ICR also points to other inefficiencies: failure of the project’s governing committees to meet and staff to be hired (paras. 52-53); failure to sequence project activities as planned (para.68); and failure to budget properly for INS operating costs (para.74). In the absence of a formal analysis, and with evidence of significant implementation inefficiencies, Efficiency is rated Modest. Efficiency Rating Modest a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 0 Appraisal 0  Not Applicable Page 6 of 11 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review ZR Catalytic Proj. Strengthen the INS (P125509) 0 ICR Estimate 0  Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome In accordance with OPCS/IEG guidelines, overall Outcome is derived by weighting the outcome rating under the original/revised objectives/outcome targets according to the percentage of Bank funds disbursed under each. In this case, 65% of Bank funds were disbursed under the revised outcome targets. Based on Substantial relevance, Negligible efficacy, and Modest efficiency, Outcome under the original outcome targets is rated Unsatisfactory. Based on Substantial relevance, Modest efficacy, and Modest efficiency, Outcome under the revised outcome targets is rated Moderately Unsatisfactory. With the weighting favoring the latter, overall Outcome is rated Moderately Unsatisfactory. a. Outcome Rating Moderately Unsatisfactory 7. Risk to Development Outcome At project startup overall risk was rated substantial, reflecting weak capacity in the INS, risks related to the census, insecurity in parts of the country, and financial resource constraints. All of these were also present during implementation and remain as substantial risk factors in the future. The build-up of a census for a master sampling frame is still underway, as is the updating of the national accounts, but resources for operating costs have already fallen short of needs as their financing has shifted from the project on to the government. 8. Assessment of Bank Performance a. Quality-at-Entry The project was relevant, addressing both government and Bank concerns over data needs for better policy making, and it was designed to directly address weaknesses in the national statistical system. Still, it would prove to be too ambitious to accomplish over the three-year (subsequently five-year) project period, and outcome indicators and targets had to be reset in 2017 to be more in line with implementation capacity. While project preparation included consideration of political and institutional risk, governance mechanisms for statistical institutions were still being shaped. Incremental recurrent costs were being met by the project but would subsequently not be fully taken over by the government due to budget Page 7 of 11 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review ZR Catalytic Proj. Strengthen the INS (P125509) shortfalls and consequently shifting priorities (for instance the INS website, which was a key component for achieving the project’s dissemination objective). Project-level investment budget shortfalls would also arise, especially in estimating the INS building's reconstruction costs (a shortfall that at least in part drove the restructuring of the project). Quality-at-Entry Rating Unsatisfactory b. Quality of supervision The fragile institutional environment in the country, including capacity constraints, appears to have placed significant challenges on supervision. The task team did respond to these challenges, as reflected in the restructuring of the project to be more in line with capacity limitations in the statistical system. The ICR points to continuous implementation support to the INS to reduce implementation risks, including maintaining fiduciary propriety, ensuring environmental and social management standards, and ensuring collaboration with other donors. Quality of Supervision Rating Satisfactory Overall Bank Performance Rating Moderately Unsatisfactory 9. M&E Design, Implementation, & Utilization a. M&E Design Targets in the results framework, i.e. for key indicators for monitoring progress and determining outcomes, proved to be too ambitious to accomplish over the three-year (subsequently five-year) project period, and the results framework was revised during implementation to better reflect capacity constraints to developing censuses and sampling frames within the time period of the project. Notably, the work underlying a new population and housing census, necessary for developing a master sampling frame, was only partly completed, and the original outputs could not be achieved under the project. Instead, outcome targets more in line with reality on the ground and still likely to contribute towards the project objectives were introduced, such as changes in the number of skilled staff, rehabilitated INS facilities, and progress on the main components for a population and housing census and more comprehensive national accounts. b. M&E Implementation All indicators in the results framework were reported on in the context of the Bank's supervision system. To some extent this reporting, as well as data collection, were influenced by frequent staff Page 8 of 11 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review ZR Catalytic Proj. Strengthen the INS (P125509) changes in the INS, delaying the advancement of the project. According to the ICR, the pace of implementation improved after the restructuring of the project. c. M&E Utilization Information flowing from the monitoring of indicators was used to better focus supervision interventions by the task team. M&E Quality Rating Modest 10. Other Issues a. Safeguards Prior to project restructuring, the project’s environmental assessment category was rated B in anticipation of the demolition and reconstruction of the INS building. Its census activities triggered OP/BP 4.10 (Indigenous People). For the INS facility, mitigating measures were put in place, and the government developed an Environmental and Social Management Plan that was validated by the Bank. The ICR does not make a direct statement about compliance with the Bank's safeguard policies, though it notes that the project's draft Grievance Address Mechanism, developed by the Bank, was handled informally by the INS, meaning that there was no verification that informally handled complaints were addressed properly. b. Fiduciary Compliance Financial management. The ICR (para 87) is quite brief on financial management, simply noting that the Borrower complied with the legal covenants, having put into place a project accounting system prior to project startup. Procurement. The ICR does not discuss procurement beyond noting that technical assistance was enlisted to reduce risk in procuring the works contract, in monitoring construction, and in monitoring safeguard mitigation measures. c. Unintended impacts (Positive or Negative) None noted. Page 9 of 11 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review ZR Catalytic Proj. Strengthen the INS (P125509) d. Other --- 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment There was negligible dissemination of statistical Moderately Moderately Outcome information and evidence of Satisfactory Unsatisfactory significant implementation inefficiencies. Shortcomings in quality at entry Moderately Moderately influenced subsequent Bank Performance Satisfactory Unsatisfactory performance despite supervision efforts during implementation. There were shortcomings in the Quality of M&E Substantial Modest results framework, notably key outcome indicators. Quality of ICR --- Modest 12. Lessons The following lessons are drawn from the ICR: Capacity constraints influence the sequencing of activities. In this case, while project design had project components feeding into each other – staff training and an improved working environment were prerequisites for statistical production and dissemination – sequencing was too ambitious in view of implementation capacity. More care could have been exercised in assessing the balance in terms of requirements for implementing the different components. Successful project preparation matches key indicators with realistic time frames for achievement. In this case, the initial set of outcomes depended on actions that, while desirable and likely to be achieved over some longer time period, were unlikely to be completed during the life of the project, requiring the introduction of new outcome indicators and targets. Incremental recurrent cost implications can inhibit successful outcomes. The experience of the operation, especially the unsatisfactory outcome on dissemination of statistical information, underlines the importance of being clear about the recurrent financing implications of a completed project. 13. Assessment Recommended? No Page 10 of 11 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review ZR Catalytic Proj. Strengthen the INS (P125509) 14. Comments on Quality of ICR The ICR provided enough information for the project to be assessed, and for results and ratings of the operation summarized and evaluated. Still, the discussion around efficacy could have been better structured around the objectives, and the analysis of efficiency could have more strongly made the distinction between efficiency of the project (the extent to which project resources were used wisely) and efficiency in the sector. a. Quality of ICR Rating Modest Page 11 of 11