Document of The World Bank FOR OFFICIAL USE ONLY Report No: ICR00005717 IMPLEMENTATION COMPLETION AND RESULTS REPORT IBRD-4867-RU ON A LOAN IN THE AMOUNT OF US$10 MILLION TO THE Russian Federation FOR THE Second State Statistical System Development Project (P092429) June 30, 2022 Poverty And Equity Global Practice Europe And Central Asia Region CURRENCY EQUIVALENTS (Exchange Rate Effective {Feb 15, 2022}) Currency Unit = Russian Ruble RUB 1 = US$0.013 US$1 = RUB 75.43 FISCAL YEAR January 1 - December 31 Regional Vice President: Anna M. Bjerde Country Director: Renaud Seligmann Regional Director: Lalita M. Moorty Practice Manager: Salman Zaidi Task Team Leader(s): Buyant Erdene Khaltarkhuu ICR Main Contributor: Olga Shabalina ABBREVIATIONS AND ACRONYMS AF Additional Financing BEA Bureau of Economic Analyses BOP Balance of Payments CIS Computer Information System COVID Coronavirus disease CPF Country Partnership Framework CPI Consumer Price Index CPS Country Partnership Strategy DAP Digital Analytical Platform DQAF Data Quality Assessment Framework EED Electronic Economic Descriptions UIMIS United Inter-Ministerial Information System FM Financial Management FTP Federal Targeted Program GDP Gross Domestic Product GEF Global Environment Facility GFS Government Finance Statistics GOR Government of Russian Federation GP Global Practice GRP Gross Regional Product HBS Household Budget Survey IBRD International Bank for Reconstruction and Development ICC Interagency Coordination Council ICLS International Conference of Labor Statisticians ICR Implementation Completion and Results Report ICT Information and Communication Technology IOI Intermediate Outcome Indicator IEG Independent Evaluation Group (World Bank) ILO International Labor Organization IMF International Monetary Fund IOT Input-Output Tables IPF Investment Project Financing ISR Implementation Status and Results Report IT Information Technology KEP World Bank – Russian Federation Knowledge Exchange Program LA Loan Agreement LFS Labor Force Survey M&E Monitoring and Evaluation MDG Millennium Development Goal MIC Middle Income Countries MOED Ministry of Economic Development MOF Ministry of Finance MTR Mid-term Review NSO National Statistical Office NSSS National State Statistical System OECD Organization for Economic Cooperation and Development OPCS Operations Policy and Country Services PAD Project Appraisal Document PDO Project Development Objective PHRD The Japan Policy and Human Resources Development Grant PIU Project Implementation Unit POM Project Operational Manual RF Russian Federation ROSC Report on the Observance of Standards and Codes (IMF) Rosstat Federal Service of State Statistics of the Russian Federation RRI Regulatory and Reference Information (system) RUB Russian Ruble SDDS Special Data Dissemination Standard SDG Sustainable Development Goals SEEA System of environmental and economic accounting SNA System of National Accounts SPI Statistical Performance Indicators SPEED System for preparing electronic economic descriptions SSS State Statistical System STASYS 2 Second State Statistical System Development Project TACIS Technical Assistance for Commonwealth of Independent States TBS Territorial Bodies of Statistics TF Trust Fund TTL Task Team Leader UFAP United Foundation of Algorithms and Programs UNSD United Nations Statistical Division USRC Unified System for Records Collection WB World Bank TABLE OF CONTENTS DATA SHEET .......................................................................................................................... 1 I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES ....................................................... 6 A. CONTEXT AT APPRAISAL .........................................................................................................6 B. SIGNIFICANT CHANGES DURING IMPLEMENTATION (IF APPLICABLE) ..................................... 14 II. OUTCOME .................................................................................................................... 17 A. RELEVANCE OF PDOs ............................................................................................................ 17 B. ACHIEVEMENT OF PDOs (EFFICACY) ...................................................................................... 18 C. EFFICIENCY ........................................................................................................................... 26 D. JUSTIFICATION OF OVERALL OUTCOME RATING .................................................................... 30 E. OTHER OUTCOMES AND IMPACTS (IF ANY) ............................................................................ 30 III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME ................................ 31 A. KEY FACTORS DURING PREPARATION ................................................................................... 31 B. KEY FACTORS DURING IMPLEMENTATION ............................................................................. 32 IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME .. 33 A. QUALITY OF MONITORING AND EVALUATION (M&E) ............................................................ 33 B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE ..................................................... 35 C. BANK PERFORMANCE ........................................................................................................... 36 D. RISK TO DEVELOPMENT OUTCOME ....................................................................................... 39 V. LESSONS AND RECOMMENDATIONS ............................................................................. 41 ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS ........................................................... 43 ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION ....................... 110 ANNEX 3. PROJECT COST BY COMPONENT ......................................................................... 112 ANNEX 4. ECONOMIC ANALYSIS ......................................................................................... 113 ANNEX 5. MONITORING AND EVALUATION FRAMEWORK.................................................. 114 ANNEX 6. SUMMARY OF PROJECT RESTRUCTURING ........................................................... 140 ANNEX 7. BORROWER, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS . 148 ANNEX 8. SUPPORTING DOCUMENTS (IF ANY) ................................................................... 159 The World Bank Second State Statistical System Development Project (P092429) DATA SHEET BASIC INFORMATION Product Information Project ID Project Name P092429 Second State Statistical System Development Project Country Financing Instrument Russian Federation Investment Project Financing Original EA Category Revised EA Category Not Required (C) Not Required (C) Organizations Borrower Implementing Agency Russian Federation Federal State Statistical Service (ROSSTAT) Project Development Objective (PDO) Original PDO The main objective of the project to assist the government of the Russian Federation to produce reliable, timely and accurate economic, financial, socio-demographic and other data for policy formulation and decision making in line with international good practiceand frameworks. Page 1 of 159 The World Bank Second State Statistical System Development Project (P092429) FINANCING Original Amount (US$) Revised Amount (US$) Actual Disbursed (US$) World Bank Financing 10,000,000 9,963,039 9,963,039 IBRD-48670 Total 10,000,000 9,963,039 9,963,039 Non-World Bank Financing 0 0 0 Borrower/Recipient 40,000,000 121,000,000 118,658,000 Total 40,000,000 121,000,000 118,658,000 Total Project Cost 50,000,000 130,963,039 128,621,039 KEY DATES Approval Effectiveness MTR Review Original Closing Actual Closing 28-Jun-2007 22-Apr-2008 03-Feb-2011 25-Aug-2012 31-Dec-2021 RESTRUCTURING AND/OR ADDITIONAL FINANCING Date(s) Amount Disbursed (US$M) Key Revisions 17-May-2012 4.19 Change in Loan Closing Date(s) 25-Jan-2014 8.79 Change in Results Framework Change in Components and Cost Change in Loan Closing Date(s) Change in Financing Plan 19-May-2015 9.38 Change in Implementing Agency Change in Results Framework Change in Components and Cost Change in Loan Closing Date(s) Change in Financing Plan Reallocation between Disbursement Categories 16-Feb-2018 9.75 Change in Results Framework Change in Components and Cost Change in Loan Closing Date(s) Change in Implementation Schedule 17-Sep-2020 9.91 Change in Loan Closing Date(s) Page 2 of 159 The World Bank Second State Statistical System Development Project (P092429) KEY RATINGS Outcome Bank Performance M&E Quality Satisfactory Satisfactory Modest RATINGS OF PROJECT PERFORMANCE IN ISRs Actual No. Date ISR Archived DO Rating IP Rating Disbursements (US$M) 01 27-Jun-2008 Moderately Satisfactory Moderately Satisfactory 1.00 02 16-May-2009 Moderately Satisfactory Moderately Unsatisfactory 1.00 03 30-Jan-2010 Moderately Satisfactory Moderately Satisfactory 1.00 04 24-May-2011 Moderately Satisfactory Moderately Unsatisfactory 2.32 05 25-Dec-2011 Moderately Satisfactory Moderately Satisfactory 3.66 06 26-Jun-2012 Satisfactory Satisfactory 4.19 07 19-Mar-2013 Moderately Satisfactory Moderately Satisfactory 7.67 08 02-Nov-2013 Moderately Satisfactory Satisfactory 8.28 09 07-Jul-2014 Satisfactory Satisfactory 9.18 10 11-Feb-2015 Satisfactory Satisfactory 9.38 11 23-Sep-2015 Satisfactory Satisfactory 9.41 12 21-Apr-2016 Satisfactory Moderately Satisfactory 9.41 13 27-Jan-2017 Satisfactory Satisfactory 9.45 14 23-Oct-2017 Satisfactory Satisfactory 9.67 15 01-Feb-2018 Satisfactory Satisfactory 9.75 16 09-Nov-2018 Satisfactory Satisfactory 9.81 17 23-Sep-2019 Satisfactory Satisfactory 9.85 18 14-Apr-2020 Satisfactory Satisfactory 9.89 19 18-Dec-2020 Satisfactory Satisfactory 9.94 20 22-Jul-2021 Satisfactory Satisfactory 9.97 Page 3 of 159 The World Bank Second State Statistical System Development Project (P092429) SECTORS AND THEMES Sectors Major Sector/Sector (%) Public Administration 100 Central Government (Central Agencies) 100 Themes Major Theme/ Theme (Level 2)/ Theme (Level 3) (%) Economic Policy 20 Fiscal Policy 10 Fiscal sustainability 2 Public Expenditure Policy 2 Tax policy 2 Subnational Fiscal Policies 4 Macro-financial policies 6 External Finance 2 Monetary and Credit Policies 2 Macroeconomic Resilience 2 Economic Growth and Planning 4 Migration, Remittances and Diaspora 4 Engagement Private Sector Development 33 Jobs 4 Youth Employment 4 Enterprise Development 29 MSME Development 29 Page 4 of 159 The World Bank Second State Statistical System Development Project (P092429) Public Sector Management 43 Public Administration 14 Administrative and Civil Service Reform 14 Data Development and Capacity Building 29 Data production, accessibility and use 29 Human Development and Gender 4 Labor Market Policy and Programs 4 Labor Market Institutions 4 ADM STAFF Role At Approval At ICR Regional Vice President: Shigeo Katsu Anna M. Bjerde Country Director: Klaus Rohland Renaud Seligmann Director: Cheryl W. Gray Lalita M. Moorty Practice Manager: Asad Alam Salman Zaidi Task Team Leader(s): Misha V. Belkindas, Olga Shabalina Buyant Erdene Khaltarkhuu ICR Contributing Author: Olga Shabalina Page 5 of 159 The World Bank Second State Statistical System Development Project (P092429) I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES A. CONTEXT AT APPRAISAL Country context 1. The second State Statistical System Development Project (STASYS 2) was prepared during a time when the Russian Federation (RF) was advancing its political and economic transformation outlined in the Socio-Economic Program for 2005-2008. The Russian economy posted the highest growth and lowest inflation in 2006 and 2007 with Gross Domestic Product (GDP) growth over 8 percent and Consumer Price Index (CPI) under 10 percent. For the first time in years, manufacturing posted an impressive growth of over 7 percent, and all sectors of the economy reported an increase compared to the period before 2006. Consistent with the economic trend, the growth for real income, wages of the population, and retail trade were in double digits in early 2007 at 11.5 percent, 18.5 percent, and 13.6 percent respectively - outpacing GDP growth significantly.1 Thus, the key goals of the development strategy were to continue to promote economic growth, to achieve ambitious social targets, and to ensure significant improvements in the efficiency of federal government agency performance. This vast agenda was data- and information-intensive. Comprehensive, high quality, and reliable statistics would be required to monitor progress in achieving these significant policy initiatives, to formulate its new directions or work out the needed solutions. 2. The Country Partnership Strategy (CPS) for 2006-2009 for the RF envisioned a partnership with the Russian government for realization of (a) sustainable rapid growth, (b) improving public sector management and performance, (c) improving the delivery of social services, and (d) enhancing Russia’s global role. A steady supply of trustworthy and readily available social and economic statistics lays the foundation of evidence-based decision making and eventual achievement of the CPS goals. The STASYS 2 project was designed to assist production of statistical data needed to support, monitor, and evaluate three goals of the CPS, namely progress towards sustainable economic growth, implementation of administrative reform and performance-based budgeting, and social programs for increasing population welfare and poverty reduction. Sector context 3. Successful statistical capacity building initiatives typically involve a series of interventions over a long period of time, during which endogenous methodological, technological, infrastructural, and institutional capacity evolves. The implementation of earlier government targeted statistical programs, grants from Technical Assistance for Commonwealth of Independent States (TACIS) and the Japan Policy and Human Resources Development Grant (PHRD), and first World Bank (WB) STASYS project built a good starting ground for the development of new national state statistical system (NSSS) of the RF. These included significant changes introduced in the statistical system reflecting the socio-economic development, upgrades to the information technology (IT) basis for data production, launched the work to develop essential elements of the System of National Accounts (SNA) and complete the Balance of Payments 1 Russian Economic Report #13 December 2006; Russian Economic report # 14 June 2007 Page 6 of 159 The World Bank Second State Statistical System Development Project (P092429) (BOP) compilation, made methodological changes in different sectors of statistics, and started the adoption of the international standards in statistical practice. 4. Further strengthening of statistical systems was needed due to continuation of market and administrative reforms, acceleration of social transformation, increased focus on effectiveness of budget expenditures and regional development. Adoption of the new Statistical Law in 2007 and the long-term nature of statistical transformations implies further development and advancement of the NSSS. 5. At project appraisal, the following NSS systemwide issues were noted: (i) insufficient coordination of data production among the federal executive bodies due to absence of unified methodology; (ii) disparate capacity of data producers to come up with data matching the international standards; (iii) non-homogeneity of information and difficulties in obtaining data from respondents; (iv) absence and/or lack of indicators needed to reflect the new socio- economic developments and processes in the country; and (v) pending methodological issues and problems arising from Russia’s accession to international organizations and world statistical community. 6. Issues important to the sector that required considerable enhancement could be highlighted as follows: (i) SNA development; (ii) reforming of information basis for social statistics; (iii) re-establishment/renewal of system of data collection for socio-demographic indicators and forecasting; (iv) setting-up data collection for broader dimensions of labor force (absence of data on professional structure and of labor demand and supply); (v) absence of information on new services; (vi) accurate reflection of price dynamic in new and developing economic areas and activities, as well as inflation indicator in economy as a whole; (vii) creation of new indicators for enterprise sector performance; (viii) development of new statistics (tourism, information society, etc.). 7. Therefore, existence of considerable number of methodological and organizational issues in Russian statistics provided rationale to continue its transformation. Moreover, due to exposure of national economy to the external world and necessity to shift to the international standards for reflection of economic activity and public services, further improvements became necessary in the very content of statistical data, data collection and dissemination, and calculation methodology. 8. The STASYS 2 project was initiated as an instrument of accelerating adjustment of Rosstat and NSSS to (i) new information needs of users of statistical information, with special reference to the needs of Governments, businesses, researchers and the public, (ii) new information environment of the economy, society and statistics, created by dynamic development and wide use of modern information and communication technologies (ICT), (iii) progress in technological and organizational modernization of the information systems of government agencies and businesses, and (iv) impact of globalization processes on the information infrastructure of the country. STASYS 2 project was playing a catalytic role for development, speeding up all the processes and bringing Rosstat on a higher level of performance. 9. The STASYS 2 project was based on a sound background analysis, including Implementation Completion Report (ICR) for STASYS project. Preparation and quality at entry benefited from a well described development strategy – Statistical Master Plan (SMP) which was prepared with the Bank assistance to provide a long-term vision and development framework. The SMP was supported by a Federal Targeted Program (FTP) “Development of State Statistics in Russia in 2007-2011” (extended till 2012) and the STASYS 2 project. The FTP, prepared and implemented Page 7 of 159 The World Bank Second State Statistical System Development Project (P092429) by the Government of Russian Federation (GOR), was reinforced by STASYS 2 project activities in the areas where WB project mechanisms and instruments had advantages. 10. Furthermore, the project was integrated into subprogram “Official Statistics” of the State Program “Economic Development and Innovation Economy” (2015–2024). Additional activities were introduced into STASYS 2 implementation plan in 2018 to reflect new national tasks and objectives outlined in “Government Strategy of information society development” for 2017-2030 and “The Program for the digital economy of the Russian Federation”. Compared with the first STASYS operation, the design of STASYS 2 project included coordination and alignment to several government initiatives and related programs ensuring close links to overall GOR’s objectives in statistical development. Rationale for Bank support 11. The rationale for WB involvement was underpinned by (i) its objective of improvement in public sector performance in which statistics is an essential part; (ii) WB’s ability to lead a long-term effort in support of the development of NSS worldwide; this is based on the Bank’s unique expertise and resources in developing and guiding comprehensive, integrated and targeted projects on statistical capacity building as demonstrated from several Statistical Capacity Building Program (STATCAP) projects under implementation at the time of approval; (iii) consistency with national and international commitments; and (iv) WB capacity to be both the major user of economic and social data, and producer of well-known international data series. Theory of Change (Results Chain) 12. Theory of change (TOC) was not a requirement therefore was not formally presented as a standalone section in the Project Appraisal Document (PAD) 14 years ago2. At the same time, the logic behind the operation exists and could be presented as capacity building activities such as improved methodology in-line with international best practice, increased IT infrastructure, and well-trained staff will strengthen the statistical system and help the government to produce and disseminate the necessary data of higher quality for policy formulation and decision-making. 13. Therefore, the design of STASYS 2 project was closely aligned with the government’s FTP 2007-2011 and its key objective to establish a well-coordinated statistical system to ensure production of timely, comprehensive, and reliable macroeconomic statistics by integrating all statistical resources at the national scale for the purpose of data use in analysis of current conditions, development trends, and forecasting of national economy, financial system, demographics, and society. In addition, STASYS 2 project activities directly supported FTP’s three out of six tasks: - Task 1 “Optimization of the content of official statistics to be used for decision-making based on the international best practice” with regards to National Accounts, Financial, Enterprise, Natural Resources and Price statistics (STASYS 2 - Component A), Social and Demographic statistics (STASYS 2 - Component C), - Task 4 in part of “Upgrading the system for collecting, processing, storing, and disseminating statistics by modern technologies” (STASYS 2 - Component B), and 2As per Operations Policy and Country Services (OPCS) guidelines, the TOC became mandatory for PADs as of May 2018. The STASYS 2 project was approved by the World Bank’s Board of Directors on June 28, 2007. Page 8 of 159 The World Bank Second State Statistical System Development Project (P092429) - Task 5 “Creation of continuous professional training for statistical personnel mainly in developing training materials and courses for training of data providers and users” (STASYS 2 - Component D). 14. All activities under STASYS 2 project had impact on intermediate outcomes and contributed to the achievement of PDO. In many cases, the gain was multipurpose and interrelated, and added to the necessary conditions of reaching the PDO by increasing data accuracy and reliability or data timeliness and reliability simultaneously. Introduction of international standards led to increase in both data accuracy and reliability. 15. Development of modern information system has contributed to all—accuracy, timeliness, and reliability of data. In long-term, the project supports the GOR in building the NSSS that is capable to provide users with timely, accurate, and reliable data for evidence-based decision making (the latter was not part of the PDO at appraisal, however, at present the GOR initiated the discussions on importance of evidence-based decision making in policy matters). 16. This would not be attainable without investing in physical infrastructure which is fundamental for collection, production, and dissemination of reliable and timely data. While physical infrastructure is crucial, higher data quality and its three dimensions—accuracy, reliability, and timeliness cannot be achieved by installing additional hardware and software only. It requires continuous improvements in methodology for all areas of statistics in line with international standards, capacity building of data producers and users, and better coordination and strengthened cooperation among the NSSS key players to facilitate better use of data for designing better policies. 17. According to PAD, the planned STASYS 2 project interventions were structured under four components and contributed to four intermediate outcomes and PDO in coordination with national programs (PAD p.24-25). ICR made effort to reconstruct the TOC approach in its absence (see Figure 1): Page 9 of 159 The World Bank Second State Statistical System Development Project (P092429) Page 10 of 159 The World Bank Second State Statistical System Development Project (P092429) Project Development Objectives (PDOs) 18. The main objective of the project was to assist the Government of the Russian Federation to produce reliable, timely and accurate economic, financial, socio-demographic and other data for policy formulation and decision making in line with international good practice and frameworks as stated in the PAD and corresponding Loan Agreement (LA). 19. The main beneficiaries of the project were: (a) public sector entities; (b) private sector, business community, foreign investors; (c) civil society groups, academia, mass-media; and (d) international organizations and foreign governments. The strengthened capacity of data producers was expected to produce high quality, timely, and relevant data easily and readily accessible to users (PAD p.15-16). Key Expected Outcomes and Outcome Indicators 20. The achievement of the above-mentioned PDOs was to be assessed by the key outcome indicators: - Regular analysis of users’ satisfaction; - Overall average score from Report on the Observance of Standards and Codes (ROSC) style review; and - Share of statistical information provided by Federal Service of State Statistics of the Russian Federation (Rosstat) to the users with a break of established time schedule. Adequate policy, legal, regulatory and coordination frameworks as well as effective management quality assurance and human resources frameworks are in place - The Statistical Council is established (1-1) - Staff Satisfaction survey has been conducted (1-2) - Appropriate dynamic of Rosstat staff training is achieved (1-3) - Positive dynamics of the share of state budget expenditures for training purposes in the overall Rosstat budget is achieved (1-4) Statistical infrastructure enhanced in the area of classifications and standards, questionnaire design, sample selection, data collection and editing, imputation, analysis and dissemination procedures - Total number of classifications used throughout the NSSS that are compatible with the equivalent international classifications (2-1) - Mastering of international standards (filling of statistical questionnaires from international organization with harmonized data) (2-2) - Number of surveys that are conducted on a sample basis (2-3) - Reduction in respondent (enterprise) burden resulting from the introduction of administrative data to replace survey data (2-4) - Availability of metadata for all statistical data published (2-5) Core statistical programs have been enhanced to ensure that they meet international standards and national needs in terms of coverage, content, comparability over time and coherence - Number of new methods of data collection (3-1) Page 11 of 159 The World Bank Second State Statistical System Development Project (P092429) - Number of Millennium Development Goal (MDG) indicators produced by the NSSS (3-2) - Compliance with the International Conference of Labor Statisticians (ICLS) household income and expenditure methodology (3-3) - Availability of data on skilled labor, demand for skilled labor, and hidden earnings (3-4) The ICT infrastructure is sufficient to facilitate modern methods for collection, production and dissemination of data, and there are adequate funds and resources to maintain this infrastructure - Number of Rosstat website visits per annum (4-1) - Availability of data collected through remote/distance methods (4-2) - Computer equipment of Rosstat staff (4-3) - Availability of microdata bases for users in open access Household Budget Survey (HBS) and Labor Force Survey (LFS) (4-4) - Number of statistical tasks resolved on a new technological basis and forwarded to the United Foundation of algorithms and programs (4-5) Components 21. The project’s five components were designed to support the implementation of the government FTP and State program in statistics in a consistent and well-sequenced manner to reach the STASYS 2 PDO. - Component A: Modernization of economic statistics methodology (appraisal US$3.35 million; actual US$6.51 million) The component activities were to ensure (i) further development of the SNA harmonized with international standards, including the introduction of internationally compatible classifications, setting up institutional sector accounts, construction of assets and liabilities balance sheets for non-financial sector, development of methodological basis for compilation of Input-Output Tables (IOT); (ii) improvement of constant price estimates by enhancing CPI in compliance with European Union (EU) requirements; (iii) implementation/introduction of the new International Monetary Fund (IMF) Manual to improve producer price observations and apply modern methods of Producer Price Indexes (PPI) calculations; (iv) upgrade the methodology of international service trade statistics in compliance with United Nation’s Manual (2004); (v) improvement of quality, timeliness and analytical value of enterprise statistics. - Component B: Development of modern design and technology for statistical data collection, processing, and dissemination system (appraisal US$34.42 million; actual US$102.85 million) The component activities were to support Rosstat’s strategic IT goal of improving data survey development, data gathering, data analysis, and data dissemination processes using advanced ICT infrastructure. These were (i) improved access to data through electronic and web-enabled technologies; (ii) development of a modern system of the demographic data collection and processing; (iii) development of a system for data collection, analysis and dissemination of the household and population surveys data; (iv) improved integration of databases; (v) institutional strengthening of Rosstat IT policy framework and elaboration of proposals for strategic development of its IT-systems, their integration and consistency of new solutions; (vi) strengthened IT security planning and requirements; (vii) development of the corporate computer and information system (CIS); (viii) Page 12 of 159 The World Bank Second State Statistical System Development Project (P092429) development of the system of data collection and interagency interaction for monitoring of performance indicators of the budget institutions in social, financial spheres and real sector considering the international experience; and (ix) remote data collection for the purpose of social, economic statistics (labor, household budget, health and social security statistics). - Component C: Enhancement of social statistics (appraisal US$2.09 million; actual US$4.60 million) The component activities were to strengthen (i) statistics of living standards and poverty measurement, including population incomes, through development of new sample based surveys and incorporation of new models into existing ones, and elaboration of comprehensive statistics to evaluate social protection of the population based on the internationally accepted system of incomes and expenditures accounts such as European System of Integrated Social Protection Statistics (ESSPROS) or similar; (ii) labor statistics, including representative estimates of the key indicators of economic activity, employment and unemployment for all regions of the RF, wages and salaries statistics; (iii) demographic statistics. - Component D: Human resource development (appraisal US$6.46 million; actual US$6.78 million) The component activities were to assist establishment of a modern and sustainable system of continued professional education for statistician in data-providing agencies/institutions based on (i) analysis of current retraining system and elaboration of a strategy for its development; (ii) development of training programs and retraining materials; and (iii) conducting training courses, exchange of experience and knowledge sharing. - Component E: Project Management (appraisal (appraisal US$3.68 million; actual US$7.88 million) Component was to support the Project Implementation Unit (PIU) – Bureau of Economic Analysis Foundation (BEA) whose functions were limited to operational, procurement, financial and disbursement aspects of implementation. The component also included monitoring of project performance on the basis of target indicators, including the user satisfaction/trust surveys. 22. The project cost was US$50 million consisting of US$10 million of IBRD and US$40 million of borrower co- financing at appraisal. During project implementation, the borrower co-financing amount was increased threefold. At project closure, US$129 million out of total US$131 million (the disbursement rate of 98 percent) of project funds was disbursed and accounted for. Page 13 of 159 The World Bank Second State Statistical System Development Project (P092429) B. SIGNIFICANT CHANGES DURING IMPLEMENTATION (IF APPLICABLE) Revised PDOs and Outcome Targets 23. The development objectives and outcome targets were not revised during implementation. Revised PDO Indicators 24. There were few editorial adjustments of original PDO level indicators for clarification purposes. Table 1: Changes introduced to the PDO indicators PDO at appraisal Edited PDO indicators Comments PDO-1 Regular analysis of PDO-1 Index of users’ trust in The changes were introduced to provide users’ satisfaction official statistics clarity on the project’s content related impact and enhanced measurability based on the upgraded methodology. PDO-2 Overall average score PDO-2 Data quality assessment for The shift was introduced in 2018. These two from ROSC style review National Accounts Statistics (NAS) indicators are very similar from technical (as per DQAF for National perspective and their results remain Accounts Statistics as detailed in comparable for the entire period of the the IMF 2012 Guidelines), project implementation. PDO-3 Share of statistical PDO-3 Share of statistical It was edited to introduce positive information provided by information provided by Rosstat connotation. Rosstat to the users with a to the users per established time break of established time schedule schedule Revised Components 25. There were no changes in the goals and names of original project components. However, the project restructurings came along with significant increase in project scope due to the increased GOR co-financing. As a result, each of the components received additional financial resources for new activities at the subcomponent level. All new activities were aligned with original tasks under main four components and fit the Project Description (LA, Schedule 1, pages 5 and 6) and PDO. The implementation of additional activities not only strengthened the project results and outcomes but also broadened the areas of their application and impact. Page 14 of 159 The World Bank Second State Statistical System Development Project (P092429) 26. The proposed activities and indicators have been reviewed by WB Global Practices and determined to be relevant and in line with WB technical standards. See the new activities in Annex 6. Other Changes 27. The key change during implementation that introduced a substantial increase in counterpart co-financing was one-of-a kind among WB projects in statistics worldwide. The original cost of the project US$50 million of which US$10 million of IBRD loan amount remained unchanged over the course of the project life, while the borrower significantly increased its co-financing share from the original US$40 million to US$81 million by the end of the project, thus bringing the total project financing to US$131 million. 28. This change resulted in several project restructurings presented in table 2 below. STASYS 2 planned to proceed with Additional Financing (AF) however due to the restrictive geopolitical conditions that emerged after 2014, the decision was taken to replace AF with a “scheme of additional co-financing during ongoing implementation”. This approach maintained both financial and technical involvement of the WB in the important reform area while the allocation of GOR resources increased. See Annex 3 for the total project financing information. Table 2: Project restructurings Restructurings Dates Borrower co- Types of changes undertaken financing (US$ (Per Restructuring papers and million) Amendments to LA) Original (LA) June 28, 2007 40.0 Total amount of loan proceeds US$ 10 remains unchanged Restructuring #1 May 17, 2012 0.0 Change in Loan Closing Date Restructuring #2 February 6, 16.0 Change in Results Framework, Change in 2014 Loan Closing Date, Change to Financing Plan, Reallocation between Disbursement Categories, Change in Disbursement Estimates, Change to Components and Cost Restructuring #3 May 27, 2015 40.0 Change in Results Framework, Change in Loan Closing Date, Change to Financing Plan, Reallocation between Disbursement Categories, Change in Disbursement Estimates, Change to Components and Cost Page 15 of 159 The World Bank Second State Statistical System Development Project (P092429) Restructuring #4 April 4, 2018 25.0 Change in Results Framework, Change in Loan Closing Date, Change to Financing Plan, Reallocation between Disbursement Categories, Change in Disbursement Estimates, Change to Components and Cost, Change in Implementation Schedule Restructuring #5 September 0.0 Change in Loan Closing Date 17, 2020 Total in 121.0 borrower co- financing Actual May 23, 2022 118.7 disbursement of co-financing 29. The increase of STASYS 2 project budget under original components allowed for new activities to be added and augmented the end-values of initially established indicators. Accordingly, additional seven new intermediate indicators were introduced to monitor and evaluate these new activities. 30. The project was approved by the Board on June 28, 2007, with a planned implementation period of five years. The effective date of project commencement was April 22, 2008, with a closing date of August 25, 2012. There were total of five restructurings covering changes to extend the closing date, revise the financial plan, increase the cost of components, update the results framework, and to make reallocation between disbursement categories. At the project completion on December 31, 2021, STASYS 2 became a significant operation that attained its PDO with positive impact in place. At the same time, 14-year implementation period implies difficulties in assessing the project given accelerations in technological development, changes in operational environment and requirements, and shifts in priorities at the international, national, and institutional level. Rationale for Changes and Their Implications on the Original Theory of Change 31. The rationale for all project restructurings occurred during the implementation differs, and this ICR summarizes the reasons as follows: (i) reaction on changing environment both national and global; (ii) defining the approaches that would allow to achieve higher targets and strengthen PDO/outcomes in the longer run; and (iii) maintaining and possibly strengthening the dialog with the client Ministry of Finance (MOF), Ministry of Economic Development (MOED), and Rosstat. The detailed information on the project’s five restructurings provided in Annex 6. 32. The increases in borrower co-financing and accompanying restructurings were integrated into project developmental logic (TOC) through modifications and corresponding added activities fit the nature and scope of the original project design and PDO. Seven new indicators were introduced, and several of existing indicators were revised accordingly. Overall, the indicators deemed sufficient to measure production of good quality statistics, namely reliable, timely, and accurate data for decision making in line with international standards. Page 16 of 159 The World Bank Second State Statistical System Development Project (P092429) II. OUTCOME A. RELEVANCE OF PDOs Assessment of Relevance of PDOs and Rating 33. At the project onset, the objective of enhancing the state statistical system in Russia was consistent with the national Socio-Economic Program (2005-2008), SMP (2005-2010), FTP (2007-2011)3 and CPS (2006-2009)4 thus reflected the strategic alignment with the country and World Bank priorities. The development objective was relevant to the findings of the WB’s Systematic Country Diagnostic (SCD) throughout the project lifetime. 34. At the project closing, the relevance of STASYS 2 to the country’s higher-level objectives and priorities also remained strong. The current statistical production is guided by Rosstat’s strategy until 2024 according to which “the mission of NSS is timely supply of high-quality data to the users and minimal respondents’ burden. In essence, it is imperative to ensure the quality production of timely and inexpensive statistics. In addition, statistics as a public good need to be accessible in its full diversity”5. The latest directive, World Bank – Russian Federation Knowledge Exchange Program (KEP) 2021-20246, includes statistics activities in the cross-cutting area for strengthening of governance and institutions. This is seen as a confirmation of importance of statistics in WB and RF dialog at the time of December 2021, thus relevance of STASYS 2 PDO. 35. A discussion on how the data-based research and analysis can help and support the better governance in Russia took place in Moscow, in early December 2021. It has demonstrated a strong interest in shifting to evidence-based policy. Former Deputy Prime Minister and Minister of Finance, who is currently holding a position of the Head of Accounts Chamber, Mr. A. Kudrin highlighted the importance of modern ways to generate the data and tools to analyze it. The Head of Central Bank of Russia, Mrs. E. Nabiullina, stressed the significant value of availability of reliable and quality data, datasets, data rows, and disaggregated data7. GOR’s strong commitment to statistical development agenda remains strong. 36. Given the projects development objectives were relevant at appraisal and continued relevant to the country’s higher-level objectives and priorities, and consistent with the WB’s Systematic Country Diagnostic (2016) and latest KEP objectives, the relevance of the PDO was rated as High. Rating: High 3 See paragraphs 1, 8, and 10 4 See paragraph 2 5 https://rosstat.gov.ru/strategy 6 In Russia, the latest CPF was developed for 2012-2016. 7 https://www.econ.msu.ru/science/Article.20211123032431_9609/ Page 17 of 159 The World Bank Second State Statistical System Development Project (P092429) B. ACHIEVEMENT OF PDOs (EFFICACY) 37. The above-mentioned PDO can be divided into five outcomes to assist the GOR to produce: - reliable economic, financial, socio-demographic and other data - timely economic, financial, socio-demographic and other data - accurate economic, financial, socio-demographic and other data - reliable, timely and accurate economic, financial, socio-demographic and other data for policy formulation and decision making in line with international good practice and frameworks - reliable, timely and accurate economic, financial, socio-demographic and other data for policy formulation and decision making in line with international good practice and frameworks. 38. The project reached its PDO-level indicators’ targets in the course of implementation and maintained the level with exception of one that slightly dropped (1/100th point) by the project closure. The relevance and quality of the PDO indicators were good in terms of: (i) its impact on the systemwide performance not specific statistical product improvement per se; (ii) its complex and encompassing design in order to reflect different dimensions of data quality; and (iii) its methodology which is from the IMF user satisfaction and international standard such as SNA2008. See table 3 below. Table 3. Project Development Objective Indicators PDO Indicators Baseline / Plan / Actual PDO-1 Index of users’ trust in official statistics 1,02 / 1,04 / 1,03 PDO-2 Data quality assessment index for SNA statistics as per DQAF for SNA statistics as detailed 1 / 1,1 / 1,065 in the IMF 2012 Guidelines, gain PDO-3 Share of statistical information provided by 97,8 / 100 / 100 Rosstat to the users per established time schedule 39. All the intermediate indicators except one were met or exceeded, often to a significant degree. Project delivered a significant continuum of outputs towards improving the quality of statistical data along the defined lines such as accuracy, reliability, timeliness, and application of international standards. The evidence is presented in Annex 1. Results framework and Key outputs. See the detailed list in the Part B: Key outputs by components section. Assessment of Achievement of Each Objective/Outcome 40. It is important to note the significance of the word “assist” while assessing the achievements of PDO (see para 12) as STASYS 2 project was playing important but a subordinate role. In comparison to the project’s financial resources, Page 18 of 159 The World Bank Second State Statistical System Development Project (P092429) the FTP and State Program in statistics were more than four and seven times higher respectively. Moreover, the specificity of the project’s monitoring and evaluation (M&E) framework made it difficult to present its attributions to the results achieved in clear and indisputable ways. To a certain extent, the objectives reached were due to a combined impact of the GOR programs and STASYS 2 project. At the same time, this setup and coordination contributed to the sustainability that was embedded into the project design and strengthened during implementation thus likely to allow the development to continue further once the external support halts. See the M&E framework in Annex 5. 41. The ICR uses the PDO indicators and relevant sub-sets of the intermediate outcome indicators (IOI) in combination with additional data where relevant to assess the achievement of each of the unpacked PDO outcomes. 42. Data reliability. The following indicators were used to justify achievement of data reliability: (i) PDO-1 index of the user’s trust in official statistics; (ii) PDO-2 data quality assessment for National Accounts Statistics as per Data Quality Assessment Framework (DQAF) detailed in the International Monetary Fund’s (IMF) 2012 Guideline; (iii) IOI.2-1 total number of classifications used throughout the NSSS that are compatible with the equivalent international classifications; (iv) availability of metadata for all statistical data published; (v) IOI.3-1 number of new surveys and/or modules in the existing surveys; and (vi) IOI.N-7 number of statistical indicators based on electronic economic description. 43. The planned and actual numbers for PDO-1 Index of users’ trust in official statistics and respective scores used for calculations are given in Table 2 paragraph 13 of Annex 58. All planned targets for PDO-1 were achieved between 2008 and 2015, including its end-target value of 1.04 and reflected an increase in the level of users’ trust in official statistics. However, the planned numbers for 2017 and 2020 were short by 0.01 point and showed a slightly downward trend in users’ trust. ICR still sees it as an important achievement, considering overall low level of public confidence for any official information produced in Russia, in the specified time, coincides with global and regional turbulence that negatively impacted the society. ICR also notes that in many countries, the overall trust in official information was lower during crises, specially at the onset of unprecedented COVID-19 pandemic. Thus, it is considered not an indication of project underperformance but an invariable societal trend in 2020. Moreover, the actual numbers represented the robustness of methodology used to capture the changes and processes occurred. 2020 high inflation in Russia and the weight used for users’ trust to price information9 in the methodology also impacted the value. The next users’ trust survey was postponed due to COVID-19 restrictions. 44. The PDO-2 Data quality assessment (index) for National Accounts Statistics (per DQAF for National Accounts Statistics detailed in the IMF 2012 Guidelines) measures the development level of the statistical capacity of the NSSS. The DQAF questionnaire for quality assessment includes criteria such as methodological soundness (application of international/regional standards), accuracy, reliability, and accessibility of official statistics. All planned values of the PDO-2 indicator were achieved during the implementation and posted consistent increase for data quality score. It is the evidence of positive changes that took place in the NSSS from 2010 to 2020 due to contributions of various project activities (see Table 5 in Annex 5 paragraph 24). By 2015, the most important methodological task set in 2010 to close the gap in composition with the requirements of international standards was solved. The compilation process included previously unavailable accounts and increased the completeness of the set. A significant contribution in the data quality 8 Survey results are available with Rosstat and Analytics-Business-Consulting» phone.: +7(985) 222-82-41, http://anbico.ru/ 9 Together with inflation rate, money income, and unemployment rate. Page 19 of 159 The World Bank Second State Statistical System Development Project (P092429) was made through organizing data collection in electronic form, establishing an analytical system for the quality assessment, completeness, and reliability of reporting data, improving quality of the Statistical Register for observations, and supporting the system of electronic services for the interagency exchange and public services of the SSS. For more information, see Annex 5 paragraphs 31-34. As a result of FTP, the state program and project activities, the NSSS has reached a new level of development and became user-oriented system that can offer broader scope and higher quality of the statistical information. 45. IOI.2-1 Number of classifications used in the system of state statistics that are compatible with the equivalent international classifications measures the application of the classifications in the SSS. The use of classifications that comply with international standards in the system of state statistics also enables: (i) conducting statistical comparisons at the national and international levels; (ii) ensuring the compatibility of government information systems and information resources; (iii) ensuring interagency information exchange; (iv) ensuring definitive identification of objects of legal relations in legal acts in the socio-economic field; (v) creating conditions for developing a unified information space on the territory of the RF. The target value of 12 classifications was reached one year earlier than the planned date of 2016. The target value was maintained as a result of continued efforts to improve classifications, and about half of all classifications used by Rosstat now support production of statistical data in accordance with international standards. This contributes to the achievement of fifth PDO outcome. 46. IOI.3-1 Number of new surveys and/or modules of data collection reflects the extent to which new economic and social developments are presented in official statistics given the various users interest. To achieve this goal, new reporting forms and/or new sections were introduced into statistical forms regularly based on the dynamic assessment of respective improvements. During the project implementation period, three to 16 forms or new sections were introduced annually and total 159 changes were made to the forms. In 2021, Rosstat had 210 reporting forms and down from 268 used in 2008. By the end of the project, more than 75 percent of the forms were changed resulting in improved statistical tools and measurement of economic and social developments. For information, see Annex 5 paragraphs 67 and 68. 47. IOI.N-7 Number of statistical indicators based on electronic economic descriptions reflects the level of prevalence in electronic economic description use for presenting all statistical outputs, thus, avoiding potential errors, inconsistences, and duplications. It is defined as the share of statistical data generated as part of electronic economic descriptions (EED) work compared with the total number of statistics produced by Rosstat. Its base value in 2018 was 34 percent and the end-target 60 percent was successfully reached by the project closure. Currently two thirds of statistical data compiled by Rosstat are presented on the base of electronic economic descriptions. 48. Data timeliness. ICR used the following indicators to justify achievement of data timeliness: (i) PDO-3 share of statistical information provided by Rosstat to the users per established time schedule; (ii) IOI.2-4 number of sample surveys in different economic and social fields that are conducted on a census/full count basis; (iii) IOI.4-2 availability of data collected through remote/distanced methods; (iv) IOI.4-3 adequate modern computer equipment provided of Rosstat staff; and (v) IOI.N-3 share of official statistical reporting forms that could be submitted to National Statistical Office (NSO) electronically. Page 20 of 159 The World Bank Second State Statistical System Development Project (P092429) 49. PDO-3 Share of statistical information provided by Rosstat to the users per established time schedule reflects the punctuality of Rosstat when disseminating statistical data to users. The end-target value of 100 percent was successfully achieved in 2008. Since then, it has been met in all subsequent years as Rosstat fulfilled its responsibilities and prepared the statistical data and information on time in compliance with internationally accepted release schedule. 50. IOI.2-4 Number of sample surveys in different economic and social fields that are conducted on a census/full count basis shows the annual number of sample surveys conducted by Rosstat instead of census/full count surveys. In 2007, Rosstat conducted 54 sample surveys which was set as the baseline. During project implementation, the annual planned values were achieved except for years 2011, 2017, and 2020 when several Federal statistical surveys were cancelled. Over the project lifecycle, more than 700 sample surveys were conducted, including modules instead of full count surveys. It shows a sufficient improvement of methodological and organizational procedures for conducting surveys in Rosstat from 2008 to 2020 as well as significant reduction of time, costs, and response burden. 51. IOI.4-2 Availability of data collected through remote methods characterizes the level of implementation of modern technologies in statistical practice leading to time saving measures in data collection. There was no relevant practice at the onset of STASYS 2 project. Actual values for this indicator consistently surpassed the annual plans at rapid pace from the start. By 2015, the indicator reached 65 percent which (i) pointed to significant progress in the automation and digitalization of statistical processes, and (ii) ensured an increase in the efficiency of the collection of primary statistical data for the subsequent years. The indicator was used until 2016 and replaced by indicator N-3 “Share of official statistical reporting forms that could be submitted to state statistical organizations in electronic form”. New one aimed to capture the level of readiness of statistical bodies to collect data using electronic statistical reporting forms and processing data based on modern ICT technologies. The indicator reflects the share of official statistical reporting forms to be filled by businesses and return to the state statistical offices electronically. The target value of 100 percent for large enterprises was successfully achieved in 2018. 52. IOI.4-3 Modern computer equipment of Rosstat staff characterizes the level of equipping the personnel with modern computers and measured by the number per employee. Across the statistical system, there was a constant upgrading of hardware and software including the improvements funded by the project. All statisticians and staff were equipped with computer, had access to internet, and possessed modern tools and packages for the purpose of survey design, data collection, verification and analysis that speeds up the data production cycle. 53. In addition to the indicator IOI.N-3 Share of official statistical reporting forms that could be submitted to NSO electronically, the indicator IOI.N-4 Share of Regional Statistical Offices employees dealing with receiving and processing primary statistical data fully achieved its intended targets leading to significant savings in time for data collection. 54. Data accuracy: Indicators presented here also measure data accuracy. These were: (i) PDO-1 index of user’s trust in official statistics; (ii) IOI.2-1 total number of classifications used throughout the NSSS that are compatible with the equivalent international classifications; (iii) availability of metadata for all statistical data published; (iv) IOI.3-1 number of new number of new surveys and/or modules in the existing surveys; (v) IOI.N-4 regional Statistical Offices employees dealing with receiving and processing primary statistical data; (vi) IOI.N-7 number of statistical indicators based on electronic economic description. All these indicators met or overachieved their target values as described above. They confirm the achievement of higher level of data accuracy and data reliability simultaneously as described above. Page 21 of 159 The World Bank Second State Statistical System Development Project (P092429) 55. In addition, the project has made vast contributions to IT infrastructure and technical solutions required at Rosstat for modern statistical production needed towards accurate, timely and reliable statistics, established a platform for web-based sample survey and automated verification process for data collection. These improvements along with the platform supported by STASYS 2 project enabled the environment for conducting the first digital Population Census of Russia in 2021 under the restricted conditions of COVID-19 pandemic. 56. Conformity/correspondence to international standards: The compliance with international standards and comparability with other countries are measured by: (i) IOI.2-1 total number of classifications used throughout the NSSS that are compatible with the equivalent international classifications; (ii) IOI.2-2 mastering of international standards (filling of statistical questionnaires from international organization with harmonized data); (iii) IOI.3-3 compliance with the ICLS household income and expenditure methodology; (iv) IOI.N-1 number of methodological guidelines prepared for implementation for compiling satellite accounts in the social sphere recommended by Organization for Economic Cooperation and Development (OECD); (v) IOI.N-2 degree of implementation of 2008 SNA into Russian statistical practices; (vi) IOI.N-6 number of SDG indicators assessed by official statistical accounting bodies based on the international methodology. 57. IOI.2-2 Mastering of international standards (filling out the statistical questionnaires of international organizations with harmonized indicators) was used from 2008 to 2013, to monitor the application of international standards in Russian. The end-target value of 90 percent was planned for 2013 instead it was reached in 2010. The target was set below 100 percent, primarily due to the fact all indicators requested by international organizations were not developed at Rosstat. In addition, there was no proper correspondence between Russian and international methodologies in certain statistical domains. Due to the preparation for Russia’s accession to the OECD membership, the indicator was replaced with new one: IOI.2-3 Application of international statistical methodologies and standards (filling out questionnaires of international organizations listed in the Federal Plan for Statistical Work and official OECD questionnaires) reflecting the degree of harmonization the statistical methodology used in the RF with international standards. This indicator monitored the process of filling in additional 70 new questionaries recommended by OECD and set the end-target value at 77 percent which was reached in 2018. During the next years, it has reached 79 percent and maintained the level despite the process of RF accession to the OECD statistical standards was frozen. 58. IOI.3-3 Compliance with the methodology of the International Conference of Labor Statisticians (ICLS) reflects the results achieved in the implementation of the methodological recommendations of ICLS used in household income and expenditure survey. Initially, the survey program assumed collecting only indirect data on the level of income (based on spending and savings growth) and did not include variables on the composition of household income. The target value of the indicator “fully complies” was achieved in 2012 with the introduction of sample survey on household income and participation in social programs. The survey program and its composition of indicators met the standards of the 2003 Resolution of the International Labor Organization on statistics of household income and expenditure. 59. IOI.N-1 Number of methodological guidelines prepared for implementation for compiling satellite accounts in the social sphere recommended by OECD was introduced to monitor the methodological improvement of the economic and social statistics through the introduction of social satellite accounts. This is the only indicator in the project M&E Page 22 of 159 The World Bank Second State Statistical System Development Project (P092429) framework that failed to report any progress. The delay in developing the necessary approaches and study of relevant social processes made it difficult to introduce and implement the use of satellite accounts in social statistics. 60. IOI.N-2 Degree of implementation of 2008 SNA into Russian statistical practices characterizes the degree of implementation of the modern international standard of 2008 SNA into Russian statistical practice or the degree of compliance of Russian macroeconomic statistics with the latest international standards. The high value of this indicator represents the level of comparability of Russian macroeconomic indicators. The baseline value was set at 10 percent, and the end-target value 93 percent was achieved in 2012, progressing further with 95.5 percent in 2021. 61. IOI.N-6 Number of SDG indicators compiled by official statistical bodies and based on the international methodology and introduced into the Federal Plan of Statistical Work. Out of 90 indicators from the total list of SDG indicators, 54 indicators (60 percent) are within the competence of Rosstat, and 36 indicators (40 percent) are based on data produced by other ministries and agencies. At the project completion, there were 112 indicators compiled surpassing the target value of 108. 62. Data use for policy formulation and decision making: The PAD did not include separate indicators to reflect this process since it was not a requirement to unpack the PDO statement at appraisal. In this regard, the ICR is (i) noting the greater data accessibility and availability, and (ii) presenting relevant examples of government documents, orders, decrees, or reports that are prepared based on the respective statistical data and information. For example, IOI.3-4 availability of data on skilled labor, demand for skilled labor, hidden earnings, and IOI.4-4 availability of open access micro datasets (HBS and LFS) – are new information and tools due to project interventions that were authorized for release with Presidential Orders of 2012, 2018 and 2021. See the Table: STASYS-2 Outputs by components with their contribution to achieving the PDO in Annex 1 B. 63. The ICR also reviewed the IMF Article 4 statistical section from December 2020 and Annual Observance Reports of the Special Data Dissemination Standard (SDDS) of 2006 and 2020 for additional data and evidence outside the project’s results framework. According to these reports, statistics in the RF produces the data that are broadly adequate for surveillance. There are two dimensions - GDP and price statistics – the achievements in which can be attributed to STASYS 2 interventions: (i) Real GDP data rebased to 2016 prices are available from 2011. In April 2016, Rosstat released GDP estimates compiled according to the 2008 SNA. The main changes introduced in the revised series include improvements in the estimation of the imputed rental services of owner-occupied dwellings and the use of the market value of assets to estimate consumption of fixed capital. (ii) Aggregate price indices are compiled for each good and service item for all the regions and the RF as a whole. Detailed data on total annual sales by economic activity, which are used to develop weights for the PPI, are published on the Rosstat website. Further efforts to improve the treatment of seasonal items and a new household budget survey—which has been under consideration for some time—could significantly strengthen data quality. (iii) RF exceeded the SDDS timeliness requirements for Employment, Unemployment, Wages/Earnings, CPI, and PPI. (iv) RF met the SDDS metadata certification requirements for all categories. Page 23 of 159 The World Bank Second State Statistical System Development Project (P092429) (v) During 2020, RF provided available information on the methodology, sources, and reconciliation of data categories in the DQAF. 64. In 2021, World Bank released Statistical Performance Indicators (SPI), replacing the Statistical Capacity Indicator, with new methodology for assessing the maturity and performance of national statistical systems in five key areas. Data are available 2016 onwards. Although limited in terms of length and capturing the specific project contributions, the SPI shows the improvements and significant contributions the project made in data standards, methods, and IT infrastructure. Figure 2: Statistical Performance Indicators – Russian Federation, 2016 – 2019 Page 24 of 159 The World Bank Second State Statistical System Development Project (P092429) Justification of Overall Efficacy Rating 65. The project was instrumental in improving the quality of macroeconomic, microeconomic, and social data used by policy makers, business community and academia, as illustrated in the analysis of the users’ trust survey results. STASYS 2 project introduced a full-fledged user trust survey into statistical practice on biannual basis upon successful development. Two Councils, Scientific and Methodological Council with external representatives from universities and research centers and the Public Statistical Council, were established at the initial stage of project implementation. This was important for the successful and more efficient implementation of STASYS 2 project since Council meetings discussed issues related to data quality, methodological soundness, use of international practices, greater openness of the statistical production process, its planning, implementation, and dissemination of statistical products. 66. The STASYS 2 project facilitated integration of Russian statistics into international statistical network as a sound and qualified participant. Russia is part of international GDP comparison as the national statistics is now based on modern classifications and sampling surveys are broadly used as a method of statistical observations as per international standards. As a result, the efficiency of NSS has increased and provided opportunities for reliable, accurate and timely data provision. 67. As presented in Annex 5 table 7, main improvements of the Russian statistical products during 2013 - 2019 were recorded in the areas and the growth presented in parenthesis: - methodological soundness via application of the international standards (26.9 percent) Page 25 of 159 The World Bank Second State Statistical System Development Project (P092429) - efficiency of data exchange (20 percent) - reliability of data sources and intermediate and output statistics (20 percent) - reliability of administrative data (13.9 percent) - data accuracy and reliability (12.9 percent) and - data timeliness (2.9 percent). 68. On World Statistics Day on October 20, 2020, the Prime Minister of the Russian Federation Mr. Mishustin acknowledged Rosstat performance largely using the STASYS 2 PDO wording” Thanks to your professional attitude, coordinated and orderly work … government authorities are receiving reliable and timely information on different areas of society life. Based on this official statistical data we are making important government decision…”.10 69. The project has achieved its objectives to improve the data quality as the ICR verified in the indicators reflecting the progress made towards data accuracy, reliability, timeliness, introduction of internationally accepted methodologies, and data dissemination for use in analysis and decision-making process while preserving its development impact. Yet not all M&E indicators allow the results to be attributed to the project interventions in entirety. PDO-1 indicator demonstrated modest achievement reaching the end target value in 2015 but being slightly below it in 2020. Therefore, the overall efficacy is rated as Substantial. Efficacy (achievement of PDO) Rating: Substantial C. EFFICIENCY Assessment of Efficiency and Rating 70. As stated in the PAD, the project is not amenable to standard cost-benefit or economic rate of return analysis. Usually, NSOs are not involved in cost-recovery activities apart from marginal data publications that generate small amount of revenue. The financial returns from statistical projects will not be representative of the economic returns. Precise quantitative rates of return can be determined for this type of project and the ICR team did not find any examples or recommendations on how to do so. It is noted that statistical capacity-building operations seeking to transform NSSS tend to have high transaction costs and are slow in producing intended results. Often the results could be visible long after the projects have completed. 71. During project implementation, delays in procurement that are generally indicative of inefficiency occurred from time to time. In reality, the strict procurement requirement of having mandatory six shortlisted candidates for any assignment contradicted the availability of services offered, and very few providers existed in the Russia statistical consultancy market. As a result of this mismatch, it was not possible to fulfill the procurement requirement by default. This situation lengthened the selection procedure and delayed contract signing. The project team prepared regular explanations and proper justifications which were finally heard by procurement experts and the issue was softened but 10 Rosstat report on 2020 Results and New Tasks for 2021, p.93 Page 26 of 159 The World Bank Second State Statistical System Development Project (P092429) not fully resolved. Unpredictable and uncertain nature of COVID-19 pandemic also impacted the project implementation in the last two years. However, the project was able to be completed without cost overruns by its closing date of December 31, 2021. 72. The project achieved its targets through efficient use of resources and investments within the operation as highlighted in the aide-memoires. Most of the planned activities were completed as per established schedule. Delays in project implementation were effectively resolved through joint efforts of Rosstat and project team resulting in disbursement rate of more than 98 percent at project completion. Moreover, operational efficiency was achieved when BEA was selected as PIU and implemented two other WB projects in addition to STASYS 2, resulting in cost saving and economies of scale measures under the component E project management. 73. In broad terms, economic benefits of the project could be based on the following outcomes: (i) improved the efficiency of statistical operations of Rosstat contributed to the better-functioning economic sectors and layers of society by facilitating more coverage and higher quality data; (ii) enhanced the potential for evidence-based decision making at policy, program, and project levels through production of better-quality data; (iii) reduced the data collection cost significantly by improving IT systems and using electronic information and survey submission for respondents; (iv) reduced the costs for data processing and storage via significant upgrades the project made in the technical and information bases of Rosstat; and (v) shift from full count censuses to sample surveys11. The counterparts at Rosstat, MOF, and MOED mentioned that the project funds were utilized efficiently on the targeted activities with focus on “value for money” and strict spending controls. 74. Annual audits of Accounts Chamber also found that the “value for money” principle was carried out in STASYS 2 project implementation and observed this principle in place during site visits to Rosstat regional offices together with the BEA team. This was captured by the data for intermediate results indicator and showed a significant reduction in paper forms used for data processing at regional offices. In 2021, the use of paper forms in data processing was down to 35 percent compared with 100 percent paper-based period of 2015, surpassing the target 40 percent. 75. In addition, sample surveys were conducted instead of a census/full count method in different economic and social fields. Rosstat was able to achieve its annual plan for sample survey implementation with more than 700 sample surveys conducted over the course of project. This is possible only through sufficient improvement on the methodological, technological, and organizational procedures for conducting surveys in Rosstat which improved the data frequency and availability at significant reduction in cost. In particular, Component B activities had direct contributions in improving the efficiency and IT capacity of Rosstat. Examples can be found below. Table 3: Selected activities and their contributions to the overall efficiency of Rosstat and NSSS B.1.1 Developing corporate information A centralized solution (hardware and software) has been implemented and the solutions based on portal technologies Internet portals of Rosstat, including Territorial Bodies of Statistics (TBS) (development of Rosstat Internet-portal) for Internet portals, are hosted on the integrated platform. use of statistical web-applications and ensure It allowed users to receive high accessibility, integrity and reliability, user access to various information resources transparent information through the application, visualization, and other 11 See the Key outputs by components table in Annex 1, Part B. Page 27 of 159 The World Bank Second State Statistical System Development Project (P092429) and functions including maps interactive services. The capacity has improved, and search optimization conducted for end-to-end search incl. external search engines. Other improvements are capability to handle traffic from at least 10,000 users in page view mode; increased response time for users to access information from the portal in 2 seconds or less, ensured accessibility for people with disabilities; and reduction in the total cost of owning and maintaining the complex of hardware and software of all Rosstat Internet portals. B.7.7 Procurement of equipment and As a result of this work, a centralized multiservice network was built, covering software for the development of corporate the federal and regional levels of Rosstat (HQ, MIC, and Territorial Bodies of computer information system of Rosstat Statistics (TBS) of Rosstat). The multiservice network is met the growing needs of Rosstat with respect to emerging technologies (scalability) and allowing cost saving measures. Hardware and software were delivered to all Rosstat project sites, and the following subsystems were deployed: telecommunications infrastructure monitoring subsystem; subsystem of a unified catalog and basic network services; centralized e-mail subsystem; backup subsystem; information security subsystem; server virtualization subsystem. B.9.1 Development of remote data Testing of "Development of remote data collection methods for the purposes collection (organizing call-center to conduct of socio-economic statistics" was carried out during the sample labor market labor market survey) survey. The telephone data collection method was tested by setting up a pilot site at the Rosstat TBS in the Sverdlovsk oblast. The introduction of remote data collection mode/using telephone during monthly labor market surveys helped to increase the participation rate in large cities and improved the efficiency of data collection via cost savings. В.11 Developing IT-infrastructure of A multifaceted development of Rosstat IT infrastructure ensured reliable Rosstat to support centralized production of operation of the Rosstat information systems. The completion of this statistical data assignment allowed Rosstat to increase the degree of automation in production processes, to expand the scope of automated processes and to speed up their execution within the production cycle. Due to the unification and centralization of production processes, better results were achieved in terms of speed, quality, and scale in the utilization of statistical resources. (Savings) Advanced mechanisms for providing access to large volumes of data via telecommunication channels and related analysis enabled much wider utilization of statistical information in other sectors and provided the opportunity to generate statistical data for new requests from the Government without additional data collection from respondent. (Savings) In addition, the production of quality information and utilization of statistical data were improved by: - high consistency and continuity of statistical information thanks to improvement made in approaches to the compilation of regulatory and reference information that supports the data collection, processing, and establishment of integrated data warehouses - the possibility of generating additional statistical information based on the previously collected data - generating arbitrary user requests for information in storage and Page 28 of 159 The World Bank Second State Statistical System Development Project (P092429) production of electronic statistical publications - providing opportunities for statistical data analysis based on the use of OLAP technologies - providing web-access to statistical data repositories via telecommunication channels and the Internet - reliability and error-tolerance tested IT modules in Rosstat’s operations. The IT infrastructure of Rosstat meets the growing needs of programming solutions and large volume of data processing. B.13.3 Improving the tools for centralized Improvement of the centralized data processing tools within the CIS enabled data processing within Rosstat CIS Rosstat to not only reduce the time and cost but also improve the quality of statistical data. Completion of this contract allowed higher degree of automation in the production processes of Rosstat, expanded the scope and accelerated the execution of the production cycle. B.14 Increasing the degree of Formalization and automated parameterization in task description for the formalization and automated parameters in collection and processing of statistical data and higher degree of system task definition for data collection, processing, integration within Rosstat CIS made it possible to improve the quality of and increasing the degree of system statistical metadata and data significantly as well as reduce the cost of integration in Rosstat CIS statistical data collection and processing. В.15.1 Improving approaches to IT strategies Improvement of approaches to information policy in terms of system software in terms of system software allowed to: - define the standards for system software and use, - reduce the number of software packages currently in use (Savings) - reduce the expenses for purchases and technical support (Savings) В.16 Unification of collection, processing, The unification of methods for collecting, processing, and disseminating data and dissemination of data from sample within the framework of the sample surveys of the population and households surveys of population and households led to a reduction in the cost of conducting sample surveys. 76. While the extended period of project implementation increased the cost of supervision while the loan proceeds remained the same, the WB’s active engagement over this period helped maintain the two-way dialogue in an area that is critical to Russia’s development. Similar to other overage projects in the country portfolio, the WB’s partnership has culminated a renewed interest for its technical support and expertise which resulted in solid commitment and significant co-financing from GOR. The preparation of a RAS project and establishment of ECASTAT Trust Fund (TF) followed as evidence of this mutual commitment and interest. The new model of lending with smaller financial resources from the WB helped leverage larger country-financed program that likely enabled efficient development outcomes in statistics. In the process, Russian experience and knowledge has supported data capacity building in other parts of the world contributing to the global public goods agenda. 77. In conclusion, the project demonstrated high efficiency of resources and leveraged US$ 121 million government co-financing while reaching the PDO and often exceeding the targets set for intermediate results indicators. Therefore, the ICR team rates overall efficiency as Substantial. Page 29 of 159 The World Bank Second State Statistical System Development Project (P092429) Efficiency Rating: Substantial D. JUSTIFICATION OF OVERALL OUTCOME RATING 78. The project has achieved its objectives in a satisfactory manner, as it has left behind a stronger national statistical system that is capable to contribute to production of quality data at reduced costs and to ensure the monitoring of policy implementation and informed decision-making process. In terms of PDO relevance (High), achievements of the PDO Substantial (with modest achievements and caveats noted for PDO level indicator) and efficiency (Substantial), the project performed at a level consistent with Satisfactory rating. E. OTHER OUTCOMES AND IMPACTS (IF ANY) Gender 79. The program did not have a direct nor indirect impact on gender. Institutional Strengthening 80. In 2010, the Scientific and Methodological Council was established. The objective was to elaborate on development strategies for Russian statistics, to conduct wide range of research on relevant issues of the Russian economy, develop the expert opinion of Rosstat on general methodological problems, define recommendations for strategic issues, provide expert and advisory support for Rosstat and other institutions producing official statistics. 81. In 2011, a Public Statistical Council was established. The Public Statistical Council characterizes the extent to which the current and strategic activities of Rosstat are carried out by expert community and civil society, considering the various users in the statistical process. Since the establishment of the Public Council (comprised of 42 representatives from ROSSTAT, other relevant agencies and academia), 37 meetings were held and discussed the priority issues of developing national statistics including challenges and issues relevant to the successful implementation of the project. Mobilizing Private Sector Financing N/A Poverty Reduction and Shared Prosperity 82. Although STASYS 2 project did not specifically target the poverty alleviation objectives, it had a positive impact on improving socio-demographic, labor force, and living standard measurements by strengthening the capacity of Page 30 of 159 The World Bank Second State Statistical System Development Project (P092429) Rosstat. The project had improved the quality of relevant data through introduction of international standards in survey methodologies, strengthening the technological basis for data collection for monitoring of poverty and social changes, improving the communication and reducing the response burden12. Under STASYS 2, Rosstat has developed a national system of socio-demographic surveys based on international best practices and contributing to the development of social program design and policymaking through supplying the data and independent research as well as monitoring the processes in the areas of poverty reduction, welfare, social development, and macroeconomic issues. Other Unintended Outcomes and Impacts. 83. The project also introduced project management culture at Rosstat focusing on results monitoring and accountability as well as the delivery and progress. This important institutional change was supported by STASYS 2 implementation and enabled transfer of practice and experience for over a decade. III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME A. KEY FACTORS DURING PREPARATION 84. Overall, the project objectives were realistic. The design was simple with clearly defined components, appropriate time allocated for implementation and sequencing of tasks. 85. The Results Framework designed in 2007 was not perfect. The intermediate outcomes’ wording was rather broad though the PDO level objectives were clearly defined, and the right level of ambition was chosen. The project monitoring indicators in general were found adequate, but in many instances, they did not allow for direct attribution of anticipated project results to project activities. 86. The project implementation readiness was relatively high. The appropriate measures undertaken were: (i) selection of the PIU (BEA); (ii) preparation of comprehensive and detailed Project Operations Manual, with well- defined implementation arrangements and procedures; (iii) incorporation of lessons learned from first STASYS project implementation in Operations Manual; (iv) drafting of project Implementation Plan and Procurement Plan for the project initial five year period; (v) drafting of the Terms of Refences for key project activities – all the above took place prior to project effectiveness. 87. The factors that influenced the STASYS 2 project achievements are: - The preconditions of strong government ownership. The strong ownership was demonstrated by ongoing GOR efforts to improve key statistical products through FTP and State statistical programs and significant amount of co-financing from the onset (80 percent of total project financing). 12See description of project activities C.1.5 and C.1.15 (Annex 1 pages 97, 98 and 104) on how the STASYS 2 contributed to ROSSTAT updating and learning about poverty measurement. Page 31 of 159 The World Bank Second State Statistical System Development Project (P092429) - The project’s pragmatic design that ultimately permitted the flexibility to add support for improving data products when the opportunities emerged. The flexible approach allowed for extending the outcomes to other areas of data production and statistical analysis. - During preparation, the project kept the focus on development of the overall statistical system and strengthening the quality of essential macro-aggregates was the right decision and path, instead of the Public Administrative Reform which tend to shift the focus toward development of local indicators for decentralization and program budgeting. B. KEY FACTORS DURING IMPLEMENTATION 88. Factors under GOR control: - Delayed approvals of annual project budgets, implementation and procurement plans by the MOF at early years of project implementation further delayed the contracts signing. - Protracted mutual acceptance of the procurement plan and composition of Working Groups at project outset that led to delays in final approval of terms of references (TORs) drafted at STASYS 2 preparation (a year prior) and announcement of tenders under all four project components. - Changes in composition of ICC and frequent changes in representatives in other governing bodies (working groups and selection committees etc.) resulted in some delays. 89. Factors under WB control: - Specifics of scarce consultancy market in statistics in Russia along with WB strict procurement rules resulted in lengthier procurement process (selection of consultants took up to three months compared to 30-45 days standard)13. - Procurement of complex ICT services. The WB procurement was considering no-objection to the results of technical evaluation of bidders' proposals under «Development of recommendations for Rosstat IT Strategy» for six months. Selection of system integrator needed for developing solutions of numerous tasks under Component B, prepare TORs, and elaborate on technical requirements and specifications for follow up tenders were delayed for six months. Subsequently, the implementation of component activities (70 percent of project cost) was impacted. - BEA (PIU for WB projects) investigation by the WB integrity unit. 90. Factors “beyond either’s control”: - Financial crisis of 2008-2009 led to retrenchment of federal budget resources under statistical FTP and cutdown the planned activities, including those aligned with STASYS 2 project. Later, the FTP budget was fully backfilled and reinforced. STASYS 2 project funds remained the source for reforming the NSSS during this period. 13Scarce market of consultant services in statistics where majority of statistical service providers were state research institutions and state educational organizations. They were allowed to participate in bidding (Section III С.3 Appendix 2 to LA). However, according to new Procurement Guidance the requirement was to have six shortlisted candidates either commercial only or noncommercial. Compliance with this requirement was unrealistic. Page 32 of 159 The World Bank Second State Statistical System Development Project (P092429) - Unplanned termination OECD accession in 2015. - Unpredictable and uncertain nature of COVID-19 pandemic also impacted the project’s last two years. Onset of the pandemic, project implementation slowed down with lockdowns and procurement of goods and services faced severe supply chain issues near the end. Nonetheless, Rosstat and project team worked closely mitigating the challenges and meeting the deadline of December 31, 2021, without budget overrun. 91. The project was back on track based on: (i) absence of any violation under STASYS 2 project with respect to findings of the WB integrity unit’s investigation; (ii) close coordination of BEA and WB team with the MOF; and (iii) direct collaboration with WB procurement specialists. This period coincided with Russia’s preparation for OECD accession which did not take place, but its assessment of Russia’s statistics was captured in earlier results of STASYS 2 project. IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME A. QUALITY OF MONITORING AND EVALUATION (M&E) 92. The detailed assessment of M&E framework and achievement of targets are presented in Annex 5. The indicators selected for monitoring the achievements were adequate. M&E Design 93. All indicators were aligned with the FTP for development of state statistics in Russia and enabled progress monitoring in broad context of reforming the NSSS. However, this made it difficult to attribute the results and achievements to STASYS 2 project in entirety. The project results framework used quantitative (22) and qualitative (6) indicators. Quantitative indicators were designed to measure resources, costs, and statistical outputs (description of internal and external funding, labor resources and available equipment, publications, Internet, etc.). Qualitative indicators were intended for additional information and various factors that affect the development of NSSS and statistical processes, and to assess the produced statistics in terms of their compliance with users’ needs. Qualitative indicators reflected the presence or absence of the observed and reviewed phenomenon/processes. 94. The baseline was established for all 28 indicators. The target values were not identified for all indicators but defined as planned values and monitored annually. 95. The quantifiable indicators were adequate, and methodology was presented at the time of project design in the Results Framework. M&E Implementation 96. In accordance with the PAD and the Project Operations Manual (POM), an annual comprehensive regular assessment (monitoring) of the project implementation was set up, which was carried out by Rosstat and the Page 33 of 159 The World Bank Second State Statistical System Development Project (P092429) Interagency Coordination Council (ICC)—high governing body of the project—with technical assistance from the BEA Foundation. Information on the project monitoring results was provided on annual basis to the Russian MOF and MOED and WB. The results of the assessment were used to analyze the intermediate results of the project, as well as to adjust its activities to reflect new challenges and changes in the statistical system. 97. As the actual STASYS 2 implementation started one year after the PAD had been approved, M&E indicators were corrected/edited accordingly with two indicators dropped, while keeping the PDO level indicators intact. By the project effectiveness, the introduction of staff satisfaction survey (indicator 1-2) in Rosstat was not feasible due to (i) absence of any respective provisions in the set of Rules for Civil Service institutions and (ii) restriction to conduct such a survey even on a pilot basis within single executive body. Thus, this indicator was dropped before the project implementation started. Another intermediate outcome indicator “Availability of metadata for all statistical data published” (2-5) was dropped when Rosstat made it mandatory to publish the statistical information with metadata only, just before the project launch. 98. The project output indicators were refined and updated during WB missions over the implementation period 2008-2018 following: (i) improved assessment methods for the indicators (compatible with previous years for continuous comparison) and (ii) new agreed activities (seven new indicators were added to the initial M&E), as well as due to the extensions of the project implementation period. 99. Overall, the methodology was found to be sound, robust, and of good quality. The detailed description and assessment are presented in Annex 5. M&E Utilization 100. Incorporation of user satisfaction surveys, echoed PDO-1 indicator in project results framework, has become a part of Rosstat operation and since was conducted on a regular basis in accordance with international best practices and standards. Justification of Overall Rating of Quality of M&E 101. As noted earlier, the TOC was not part of the core design when the project started, but the Results Framework encompassed the concept behind it. The PDO level objectives were clearly specified, and overall, the indicators reflected the outcomes of the PDO statement. Intermediate results indicators were relatively adequate to capture the contribution of the activities under the components and outputs toward achieving PDOs. The indicators included in the M&E framework were specific, when relevant measurable (22 out of 28) with appropriate sampling and data collection methods, achievable, and time bound. At the time of appraisal not all baselines and targets were available. Following the WB missions, data updates were reported in the Implementation Supervision Reports (ISRs) accordingly (See Annex 5). Where relevant the M&E arrangements were embedded across the institution. Page 34 of 159 The World Bank Second State Statistical System Development Project (P092429) 102. There were several weaknesses in M&E design which were corrected early in the implementation. The data were reliable and of good quality because of sound methodology and the survey third/independent party provided additional quality control and accountability. M&E functions and processes are not likely to be sustained after project closing, however, Rosstat will continue to track some elements of STASYS 2 project M&E framework. M&E findings were communicated to the various stakeholders such as MOF, MOED, and Accounts Chamber on a regular basis. Although M&E information did not have direct negative influence on project implementation, and there were no shifts in the implementation direction of the operation, it positively supported all project restructurings. Most importantly M&E data could be used to provide evidence of outcome achievements. 103. The main M&E system weakness is that it doesn’t provide for project ATTRIBUTION to outcomes/PDO elements. The development objectives for STASYS 2 while appropriate for what was understood to be a long-term statistical reforming process, they may have been too broad for an individual operation. STASYS 2 project focused mainly on strengthening the system for producing good quality (several parameters such as accuracy, timeliness, reliability, and compliance with international standards) statistics along very specific lines—statistical infrastructure, systems IT modernization and staff knowledge, and improving the quality of selected statistical products. The objectives were and remain relevant to the overall development priorities for the Russian Federation. Rating: Modest (considering the acquired WB experience and requirements at present). B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE 104. Environmental and social compliance. The project did not envision works, land acquisition or resettlement. The STASYS 2 was classified as Environmental Category C. There was no adverse social impact perceived. During implementation, the project provided technical assistance with goods and services, and did not trigger any environmental safeguards or concerns. The environmental, social, and fiduciary risks were rated as Moderate during project implementation. 105. Fiduciary compliance. No major concerns were raised from fiduciary management (FM) perspective neither under project preparation nor during its implementation. The FM arrangements of STASYS 2, including procurement, financial management and disbursement were carried out by the Bureau of Economic Analysis Foundation (BEA), a noncommercial legal entity, competitively selected as the PIU by Rosstat, MOF, and MOED. Both the BEA financial and procurement capacities were assessed as adequate at preparation and continued to be sufficient for the entire implementation period. The Bank’s FM team regularly reviewed and rated the project FM as satisfactory. All quarterly unaudited interim reports had been submitted on time and were acceptable to the WB. No major control or accountability issues were identified during annual external audits - all had clean audit opinions and no issues were raised for attention in the accompanying management letters. Procurement performance of STASYS 2 project was rated satisfactory throughout the implementation cycle. Procurement of all goods and technical assistance services followed the WB Procurement Guidelines and Policies. Procurement staffing and planning were found sufficient and supported by the approved and regularly published Procurement Plans. The Bank’s Procurement team carried out regular reviews as part of the supervision missions and ratings were satisfactory. Post reviews were regularly conducted in due course on sample basis. Page 35 of 159 The World Bank Second State Statistical System Development Project (P092429) C. BANK PERFORMANCE Quality at Entry 106. The team correctly based its preparation of the project on the long-term and complex nature of reforming the Russian statistical system. Given the size of the country and economic transition, STASYS 2 project suggested a more focused reform actions for data collection and processing and recognized the need for careful prioritization and sequencing from the onset. For example, delays in implementing new Classification of Products would affect the pace of strengthening the compilation of National Accounts. 107. The preparation of STASYS 2 project was shaped by: - Analytical underpinning. The technical design of the project was based on a solid analysis of strengths and weaknesses of Russian NSSS through NSDS and SMP processes. It was further supplemented by preparation of third mid-term FTP (2007-2011). The Bank added to the preparation through findings of the first STASYS ICR, also ensuring strategic relevance of the proposed PDO, shaping the M&E framework, and providing a timely integrator role of these evaluations and recommendations to support the design of STASYS 2 project. For additional details, see paragraph 9. - Government commitment. Strong government commitment was present, and MOF co-financed US$40 million in the project. Rosstat has elaborated its own program to substantiate the project alignment with the GOR strategy and FTP through emphasizing the results and gains of STASYS 2 project implementation. - Ongoing WB dialogue with the Government of Russia. Although the WB investment portfolio in RF was shrinking the Country Management Unit (CMU) proactive engagement with the client in various new areas (direct engagement with regions, supporting Russia as a new development donor) created a supportive environment. STASYS 2 project was one of two projects in the entire pipeline signed by the GOR. - Previous collaboration in statistics. The WB had knowledge and experience in supporting the operations of Rosstat (previously Goskomstat) through TFSCB grants and STASYS project. These earlier engagements helped address GOR’s request for assistance in statistics in a timely manner and bring WB’s known satisfactory implementation arrangements in place. 108. The project preparation process took two years which at the time was in line with the average time to prepare projects in Russia due to coordination and commitment required at Rosstat and Government authorities. It is also on par with project team experience in similar operations, and the team received CMU’s continued support. 109. The decision to forego a formal Decision Review meeting was not in line with the standard practice for IPF preparation. To some extent, it was substituted by the STASYS ICR findings and recommendations. Overall, the PDO was clear and relevant to country’s strategy and conditions, key CAS pillars, and the need of the entire NSS and Rosstat. The PDO appeared to be largely reachable within the initially planned project life of five years with available funds almost fully disbursed by the original closing date. Page 36 of 159 The World Bank Second State Statistical System Development Project (P092429) 110. As noted in paragraph 107 and 108 with exception of M&E framework, the ICR did not find any significant drawbacks in the design of the project. The M&E framework was prepared based on the understanding and experience available to the project team at that time of preparation in 2007. Rating: Satisfactory Quality of Supervision 111. The project was implemented according to the original design but was restructured to increase the share of the borrower’s co-financing, increase the scope and add new activities to strengthen the achievement of the PDO, revise the M&E indicators, and extend of the closing date14. Notably, STASYS 2 was the first WB project in the Russia portfolio to explore the Additional Financing (AF) modality with the MOF. However, the AF did not materialize due to the geopolitical conditions that emerged in 2014. (No new lending was considered since then.) Similar to other projects in the active portfolio, the Government’s approach was to continue project implementation with additional borrower co- financing. This allowed the World Bank to maintain technical involvement in an important reform area, despite increases in government’s own resources. This paved a specific way forward for other projects in Russia portfolio to follow. 112. STASYS 2 never had been assigned “at Risk” status. The mid-term review (MTR) was not referenced in the Legal Agreement but was substituted by MTR report for the information to all parties. The interim ICR was suggested by the project team, and it was drafted but not completed. Both documents confirmed that implementation was on track reflecting early project results and provided useful recommendations. 113. As in many investment operations, STASYS 2 implementation faced start-up delays that were effectively addressed by Rosstat and the project team15. As implementation progressed after three years, Rosstat as well as government authorities, who were involved in the project, gained momentum, and maintained it up until project completion. 114. Supervision was sufficient given (i) the technical expertise concentrated in the World Bank’s Development Data Group (DECDG) as well as the World Bank’s Poverty Global Practice, and (ii) operation knowledge of the project team whose members were located both at the World Bank Headquarters in Washington DC and in the field, in Russia. Factual and quality of performance reporting (Aide-Memoires, Implementation Status Reports, MTR, etc.)16 were acceptable to the CMU, peers, and supervisors. All progress reports tracked the indicators in the results framework, emphasized key implementation challenges, and utilized to make any necessary course corrections. The fiduciary team joined most of the project missions and exercised close supervision of the BEA’s compliance with financial management, auditor’s performance, and procurement due diligence. 14 For more details, see para 27 and Annex 6 Rational for Restructuring. 15 For reasons, see para 93 and 94 Key factors affected implementation section. 16 Project documents and disclosable reports are available at: https://projects.worldbank.org/en/projects-operations/document- detail/P092429?type=projects Page 37 of 159 The World Bank Second State Statistical System Development Project (P092429) 115. The project operational environment. There were several potential risk factors that Rosstat and the project team faced: i. Changes in the leadership of the NSO. The transition process was handled well by both parties with closer supervision and did not affect the project implementation. ii. External factors that affected project implementation were handled proactively by the project team in collaboration with the client which brought Satisfactory ratings for both DO and IP. Those were candidly reflected in the ISRs. iii. Execution of this complex WB statistical project was coupled with regular work performed by Rosstat that includes large country-wide operations such as periodic censuses or surveys. Time to time, this delayed the project, but implementation pace was always back on track due to prior experience in managing same level of institutional development projects. Rosstat’s attention to project implementation, strong Project Implementation Unit (BEA), and thorough WB supervision helped overcome these potential risks. iv. As in many investment operations, STASYS 2 implementation faced start-up delays that were effectively addressed by Rosstat and the project team.17 As implementation progressed after three years, Rosstat as well as government authorities such as MOED, who were involved in the project, gained speed, and maintained it through project closure. 116. Certain steps have been taken to maintain project benefits. Almost all the project results have been institutionalized and could be sustained (see Annex 1 B, Table STASYS-2 Outputs). They show key deliverables by components and includes proof for institutionalization of each output and activity that were introduced into Rosstat’s regular work. The Government has ensured that recurrent and maintenance costs for the equipment acquired, and software programs developed under the project were adequately covered by the budget allocations. A new Rosstat Strategy (2024) was prepared before the project completion18. 117. As noted above, the initial Results Framework could have been enhanced (i.e., some indicators could have been reformulated) to greater extent during restructurings to maximize the use of the M&E system. Equally, the suggested new indicators for monitoring the progress and additional activities demonstrated the direct alignment of project outputs and outcomes. 118. The WB supervision performance was satisfactory. Close supervision at the onset of the project helped to resolve some of the early implementation challenges. The WB worked closely with government authorities to resolve several implementation issues already described in paragraphs 93-95. Financial management and procurement reviews have been undertaken with appropriate timeline and regularity. Close supervision, including field-based financial management and procurement processes, were important factors that helped to ensure the progress toward development outcomes. 17 Elimination of duplicative tasks from ICC and WG functions; establishment of additional WGs specialized on specific tasks under Components A and C; and separate the activities supervised by the MOED under Component B; and raising the ceiling/threshold of amounts for assignments to be considered by the ICC. These changes led to significant reduction of time for TOR elaboration and approvals, bidding processes, and selection procedures. 18 https://rosstat.gov.ru/strategy Page 38 of 159 The World Bank Second State Statistical System Development Project (P092429) Rating: Satisfactory Justification of Overall Rating of Bank Performance 119. The ICR rates the overall WB performance as satisfactory given satisfactory quality at entry and satisfactory implementation support. The team showed candor in project’s performance ratings early in the project cycle (MS for DO and MU for IP ratings) and combined efforts to resolve underlying implementation issues. The Aide-Memoires, ISRs and other project documents were informative, but not excessive, focusing on and resolving the issues. Despite its weakness, M&E framework helped capture the substantial results obtained through STASYS 2, achievement of its two project development objectives and many of the intermediate indicators that exceeded the targets set. The project team worked on expansion of the project impact, and thus has satisfactory overall outcome. Rating: Satisfactory D. RISK TO DEVELOPMENT OUTCOME 120. The sustainability of current achievements under STASYS 2 project are likely to remain. There is strong ownership by Rosstat including some ministries and agencies of the project outputs and processes implemented, particularly in IT infrastructure. Plans for further development of official statistics that allow for positive assessment and sustainability of STASYS 2 results are described in the Strategy for development of Rosstat and the state statistical system (2024)19. These are: - activities to increase data users’ satisfaction, including (i) establishment of a center to collect and analyze the users’ feedback, and satisfy the users’ needs in statistical data; (ii) conducting users’ satisfaction surveys with higher number of respondents and evolving survey programs (both PDO-1 Index of users’ trust to official statistics). - continuation of Public Council work (intermediate project indicator 1.1). - completion of introduction of SNA 2008 standards with a target of 97 percent to be reached in 2024 (intermediate project indicator N-2). - sequenced introduction of satellite accounts – to make three of them available in 2024 (intermediate project indicator N1). - implementation of several new programs for human resource development at Rosstat: «Rosstat Leaders», «Rosstat: development platform», «Lecture Hall” (intermediate indicator 1-2). - further activities to optimize and speed up data collection on electronic basis, using data flows from business, introduction of computerized platforms for households’ surveys. The plan to reach 100 percent in electronic submission of statistical forms and automate raw data collection up to 75 percent in 2024. The strategic tasks are set to reach 100 percent automation of data processing, 100 percent of automated data control, creation of single/integrated data warehouse (intermediate project indicators 4-2, 4-3, and 4-6 correspondingly). - further activities to fully exclude duplication in data collection by 2024, to introduce the united register of 19 https://rosstat.gov.ru/strategy Page 39 of 159 The World Bank Second State Statistical System Development Project (P092429) statistical forms (intermediate project indicator 2-5). - further activities to use the alternative data sources. During 2022-2024, there is a plan to launch Center for Big data, and conduct pilots in price, trade, tourism, agricultural, population statistics (continuation of project activity В.19.а). - development of integrated data dissemination BI platform to replace the UIIS system in late 2022 (continuation of project activity В.23). 121. Other factors in favor of greater sustainability: i. Rosstat’s focus on strategic actions to go well beyond better data collection, keeps issues of statistical system architecture and overall efficiency on the radar (Strategy 2024). ii. Interactive and consultative process with government agencies, academia, private sector and development partners supported by the project (STASYS 2 maintained this active approach and involvement of a wide range of data users and producers, their training, the strengthening of user-producer dialogue and data dissemination to stimulate a stronger demand for higher quality data). iii. Intensified international exposure stimulates interest to maintain quality of statistical system (Rosstat is a member of numerous international Working Groups and Cooperation with Eurasian Economic Commission (EEC) which is one of Rosstat focal areas of international activity). iv. Rosstat engagement in development partnerships will also stimulate focus on quality. As Russia is emerging as a source of technical expertise in statistics, there will be additional desire to maintain the quality of statistical products (as per statistical RAS project outline and activities). v. Several state programs of the RF that will support further development of official statistics (Strategy 2024). 122. At the same time, the long-term sustainability also depends on (i) strengthening coordination and management of the whole statistical system; (ii) continuously growing public confidence in Rosstat and official statistics in general; (iii) ability of the NSSS to respond to challenges of modern development agenda in statistics (Big data and digitalization etc.); and (iv) improvements in the monitoring and evaluation of investments in statistical system. 123. Additional risk, where Rosstat, like other ministries and agencies, is in dire need of personnel, including highly qualified IT personnel and data scientist and specialists. Staff retention became an issue following training on the use of new information technologies at some pilot sites. This needs careful management and planning. 124. Overall, and based on the latest data available (as of January 2022), the risk is considered negligible. The risk to the development outcome is rated negligible provided that Rosstat continues on its well-established path to continue strengthening the country’s statistics framework and conformity with best international practices in methodology, while maintaining high levels of transparency and accessibility to information. Rating: Negligible Page 40 of 159 The World Bank Second State Statistical System Development Project (P092429) V. LESSONS AND RECOMMENDATIONS Project level observations/lessons 125. Sustained and increased government co-funding for WB led operations could have a stronger impact on statistics sector development, foster improvements, and build institutional and human capacities providing for greater sustainability of project results and outcomes. In STASYS 2, the MOF provided significant amount of co-financing while Rosstat ensured strong leadership in subject matter and implementation issues. 126. Stability in the composition of government, implementing agency and the WB teams facilitated smoother project implementation and, therefore, the achievement of PDO. Due to the lengthy implementation period, STASYS 2 project underwent several rounds of changes in senior management of Rosstat, changes of middle management, and team composition of the MOED and MOF. The composition of the WB team also evolved. However, the project demonstrates exceptional continuity due to WB senior management support, consistency in policy directions on the GOR side, and commitment to stable implementation principles and client needs. 127. Embedding institutionalization of activities during implementation cycle is essential condition for sustainability of results in all statistical capacity building operations. STASYS 2 project followed three important principles to secure future sustainability: (i) agreement with ROSSTAT to formally introduce all project developed methodologies and procedures by internal executive orders; (ii) acceptance of all consultants’ work after piloting of results is conducted as part of regular ROSSTAT survey operations; (iii) strong coordination of the project supported staff training program with the planned methodology development and IT products. 128. Large, complex, and interrelated activities under statistical capacity building projects require closer monitoring and additional time for unforeseen delays. In STASYS 2, delays in the delivery of equipment, installation of networks, approvals for the remodeling of the server room resulted in 18-month lag between the timing of statistical applications design, delivery, and the time when actual pilot took place. 129. M&E framework for statistical capacity building projects with a long implementation cycle need additional consideration. This framework should be either revised at some point of implementation or additional flexibility should be embedded at M&E framework at design stage. 130. Operational requirements to align the project with broader national program aimed at producing better and higher quality results of the entire system may lead to complications in M&E system. First, the everlasting problem of assessing project attribution to these results is aggravated. Second, a new challenge of balancing the set of indicators to reflect system- and project- level impacts would need to be addressed. 131. The project implementation and restructuring were response to changing environments which could be seen as different phases of the project as opposed to changes to the original design to some extent. If this had been known in the beginning, another instrument such as Multi-Programmatic Approach might have been a better choice, in which each phase of the project could be reviewed and assessed on its own merit. Page 41 of 159 The World Bank Second State Statistical System Development Project (P092429) Country level observations 132. Long implementation period of STASYS 2 proved that: i. developments in statistics20 started to change faster compared with a decade ago. This may call for projects to allow or built enough flexibility in the content while complement them with concentrated supervision and implementation. ii. personnel turnover at NSOs common in bigger cities and metropolitan area remains an issue. iii. NSOs are running short of institutional memory and project’s early achievements are not well recognized at the end. iv. opportunity to observe sustainability of project’s earlier results exist. 133. Project teams responsible for statistical capacity building project implementation need to organize systemic engagement with broader range of counterparts on the Government side—WB team should not be working just with NSO but also engage with MOF and MOED counterparts as well as other core ministries. This requirement relates to the fact that MoF and MOED are named as primary contact agencies for the WB. In addition, these policy making bodies have strong interest in the quality of data and statistics in the country. This principle has been applied in full during STASYS 2 project implementation, and the project became an anchor for (i) maintaining policy dialogue with the GOR (Rosstat, Ministry of Finance, MOED); (ii) initiation/development of new modalities of WB – Russia cooperation as a middle-income country (preparation and implementation of statistical RAS); and (iii) accumulation of additional financial resources to be channeled in support of statistics regionally and globally through engaging Russia in its status of a new donor (preparation and execution of ECASTAT TF). Recommendations 134. Contingency time should be allocated for unforeseeable delays and risks related to the fulfillment of interrelated activities. This should be considered during project preparation, particularly (i) in the case of alignment with national programs/activities, and (ii) if acceptance of statistical products developed under the project requires piloting and/or testing within the regular NSO’s work to support further institutionalization. 135. It is important for project design to incorporate institutionalization of project outputs, e.g., formal confirmation of project results such as improved methodology, new methods, and international standards. This could be done through internal orders and instructions to be put in place before project completion and captured in the M&E indicators. 136. Special attention needs to be paid to complex phasing and timing of IT modernization processes, namely coordination of the multiple facets of IT modernization. E.g., equipment delivery, software testing, network installation, . establishment of computing centers, and relevant staff training. This is especially true for diverse and large countries. 20 Technological revolution, new approaches in data collection, digitalization of entire information cycle, and opportunities associated with big data etc. Page 42 of 159 The World Bank Second State Statistical System Development Project (P092429) ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS A. RESULTS INDICATORS A.1 PDO Indicators Objective/Outcome: Assist the government to produce reliable, timely and accurate data for policy formulation. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Regular analysis of users' Text 2007 - no user 1.04 1.03 satisfaction (Index of users' satisfaction surveys trust to official statistical data) 28-Jun-2007 15-Dec-2012 31-Dec-2021 Comments (achievements against targets): All planned targets for PDO-1 were actually achieved in the period from 2008 to 2015, which in general reflected an increase for the level of users’ trust in official statistical information during this time period. In 2017 and 2020, there was a slight lag (0.01) behind the target and a slightly downward trend by the project closure when global COVID-19 pandemic disrupted the last survey collection and update. But the survey became a part of Rosstat operation to be continued on a regular basis according to the international best practice and standards. Survey results are available at Rosstat and Analytics-Business- Consulting»Тел.: +7(985) 222-82-41, http://anbico.ru/ Formally Revised Indicator Name Unit of Measure Baseline Original Target Actual Achieved at Target Page 43 of 159 The World Bank Second State Statistical System Development Project (P092429) Completion Data quality assessment for Text 2007 - no self- 1.10 1.10 National Accounts Statistics assessments done. (NAS) (as per DQAF for First time in 2010 -1. National Accounts Statistics as detailed in the IMF 2012 28-Jun-2007 15-Dec-2012 31-Dec-2021 Guidelines) Comments (achievements against targets): Target reached. The planned value for PDO-2 indicator was achieved and maintained during the implementation of the project. This is due to the positive changes that took place in the national statistical system from 2010 to 2020 through the significant contribution of project activities. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Share of statistical Text 2006 - 97.8% 100% 100% information provided by Rosstat to the users 28-Jun-2007 15-Dec-2012 31-Dec-2021 according to established time schedule Comments (achievements against targets): Target reached. The target for PDO-3 was set at 100 percent and was reached in 2008. Since then it has been maintained during all subsequent years. Rosstat fulfilled all its obligations regarding the preparation of statistical information on time and in compliance with internationally accepted release schedules as well releasing the calendar is on it's website. Page 44 of 159 The World Bank Second State Statistical System Development Project (P092429) A.2 Intermediate Results Indicators Component: Component A. Modernization of economic statistics methodology Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Degree of implementation of Percentage 10.00 93.00 95.50 2008 SNA into Russian statistical practices (% of 27-May-2015 31-Dec-2018 31-Dec-2021 works fulfilled in accordance with "Action Plan to implement OECD recommendations for developing the SNA in Russian Federa Comments (achievements against targets): Target exceeded. The end target for IOI.N-2 indicator of 93 % was achieved and reached 95.5% in 2021, which shows a high level of implementation of the international SNA standards into statistical practice in Russia. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of methodological Number 0.00 2.00 0.00 Page 45 of 159 The World Bank Second State Statistical System Development Project (P092429) guidelines prepared for 27-May-2015 31-Dec-2018 31-Dec-2021 implementation for compiling satellite accounts in the social sphere recommended by OECD Comments (achievements against targets): Target not achieved. This IOI.N-1 indicator was introduced into the project results framework in 2015 and its values for 2015-2019 were set as "0". Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Total number of Text 2007 - 7 Annual target 12.00 classifications used throughout the SSS that are 28-Jun-2007 15-Dec-2012 31-Dec-2021 compatible with the equivalent international classifications Comments (achievements against targets): Target exceeded. The target value of 12 classifications for IOI.2-1 indicator was reached one year earlier than the planned date 2016. The target value for the intermediate indicator was maintained as a result of continued efforts to improve classifications. About half of all classifications used by Rosstat are now supporting the production of statistical data in accordance with international standards. Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Formally Revised Completion Page 46 of 159 The World Bank Second State Statistical System Development Project (P092429) Target Mastering of international Text 87% 90% End target value standards (filling out the reached in 2010 and statistical questionnaires of maintained until 2014. International Organizations Since then it has been with harmonized indicators) replaced by indicator "Application of internationally accepted statistical methodologies and standards (filling out questionnaires of international organizations listed in the Federal Statistical Action Plan and official OECD questionnaires), %" due to changes in the set of international questionnaires used for filling in and submission. 28-Jun-2007 15-Dec-2012 31-Dec-2021 Comments (achievements against targets): Target exceeded. The target value for IOI.2-2 indicator was met by 2010 – three years earlier and stayed at this level through 2014. Page 47 of 159 The World Bank Second State Statistical System Development Project (P092429) Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of sample surveys in Text 54.00 Annual target 44.00 different economic and social fields that are conducted on 28-Jun-2007 15-Dec-2012 31-Dec-2021 a census basis Comments (achievements against targets): Target reached. The name of this IOI.2-4 indicator added clarification in 2015. During the project implementation period, annual planned values were achieved with exception of years 2011, 2017 and 2020 when several planned Federal statistical surveys were cancelled. During this period, more than 700 sample surveys were conducted instead of full count surveys. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Reduction in respondent Text No Yes Yes (enterprise) burden 28-Jun-2007 15-Dec-2012 31-Dec-2021 Comments (achievements against targets): Target reached. Within IOI.2-5 indicator, every year there was a reduction in the number of unnecessary and duplicative forms, optimization of variables, increase in share of statistical reports submitted in electronic format, and growth in number of statistical forms converted into machine-readable format. This provided opportunities to introduce new reporting forms with the purpose to study those social and economic phenomena that were not previously covered by existing surveys and statistics. Page 48 of 159 The World Bank Second State Statistical System Development Project (P092429) Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of new surveys Text 0.00 Annual target 75.00 and/or modules of data collection 28-Jun-2007 15-Dec-2012 31-Dec-2021 Comments (achievements against targets): During the project implementation under IOI.3-1 indicator, there were 3 to 16 forms or new sections introduced with exception of 2021 when 75 changes were introduced following the revision of Federal Statistical Act of 2019 bringing the total to 159 respective changes. At the beginning of the project in 2008, Rosstat used 268 reporting forms which was reduced to 210 forms by the project closure. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of MDG indicators Text 33.00 34.00 34 - This indicator was produced by the SSS relevant until 2015. In September 2015 US General Assembly endorsed the Sustainable Development Goals (SDGs) for the period until 2030. 28-Jun-2007 15-Dec-2012 31-Dec-2021 Comments (achievements against targets): Page 49 of 159 The World Bank Second State Statistical System Development Project (P092429) Target reached. The annual planned value for this IOI.3-2 indicator was set to 34 in 2009 (based on the actual number in 2008) the same planned value was set and maintained during the relevant years between 2008-2015. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Application of internationally Percentage 70.00 77.00 79.00 accepted statistical methodologies and 06-Feb-2014 31-Dec-2015 31-Dec-2021 standards (questionnaires of international organizations listed in Federal Statistical Action Plan and official OECD questionnaire Comments (achievements against targets): Target exceeded. The target value for IOI.2-3 indicator of 77% was reached in 2018. Following years, the values were: 77% in 2019, 78% in 2020, and reached 79% in 2021. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Availability of national set of Text No Yes Yes, a set of national indicators for SDG SDG indicators has monitoring in the Russian been defined, data for calculations have been Page 50 of 159 The World Bank Second State Statistical System Development Project (P092429) Federation collected 04-Apr-2018 30-Jun-2021 31-Dec-2021 Comments (achievements against targets): Target reached. The target value for this IOI.N-5 indicator has been set starting from 2020 upon the review and endorsement of the national set of indicators for monitoring of SDGs by Presidential Administration and the Government of Russia. Thus the target value of the indicator was achieved in 2020. 2020 – “Yes, a set of national SDG indicators has been defined” 2021 – “Yes, a set of national SDG indicators has been defined, data for calculations have been collected”. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of SDG indicators Number 90.00 108.00 112.00 assessed by official statistical accounting bodies based on 04-Apr-2018 30-Jun-2021 31-Dec-2021 international methodology and introduced into Federal Statistical Action Plan Comments (achievements against targets): Target exceeded. At the project completion for IOI.N-6 indicator, there were 112 indicators compiled surpassing the target value of 108. Page 51 of 159 The World Bank Second State Statistical System Development Project (P092429) Component: Component B. Development of the modern design and technology for statistical data collection, processing and dissemination Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Share of official statistical Percentage 80.00 100.00 100.00 reporting forms that could be submitted to NSO 27-May-2015 31-Dec-2018 31-Dec-2021 electronically (as % of 2015) Comments (achievements against targets): Target reached successfully in 2018 for IOI.N-3 indicator. Statistical bodies are 100% ready for data collection through using electronic statistical reporting forms and processing the data using modern ICT technologies. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Regional Statistical Offices Percentage 100.00 40.00 35.00 employees dealing with receiving and processing 27-May-2015 31-Dec-2018 31-Dec-2021 primary statistical data (as % of 2015) Comments (achievements against targets): Page 52 of 159 The World Bank Second State Statistical System Development Project (P092429) Target exceeded. In 2021, the end value for this IOI.N-4 indicator reached 35% surpassing the target 40% which showed the reduction in paper forms the staff used to process at the regional offices (base year 2015) demonstrating a significant improvement of the statistical process and automation in the last years of project implementation. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of Rosstat website Text 2007 -500 1,000.00 3,000.00 visits per annum (thousands) 28-Jun-2007 15-Dec-2012 31-Dec-2021 Comments (achievements against targets): Target exceeded. Target value for IOI.4-1 indicator was achieved in 2009. Between 2010 and 2015, the value remained high reaching over 1.1 million visits per year. Between 2017-2018, the number of visits increased exponentially reaching 7-8 million when the Rosstat website became the main source of statistical information according to the user satisfaction survey. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Availability of data collected Percentage 0.00 60.00 65.00 through remote/distanced methods 28-Jun-2007 15-Dec-2012 04-Jan-2016 Comments (achievements against targets): Page 53 of 159 The World Bank Second State Statistical System Development Project (P092429) Target reached. Due to the significant progress in the automation and digitalization of statistical processes, actual values for this IOI.4-2 indicator consistently surpassed the annual plans at rapid pace from the start reaching 65% in 2015 when it was replaced with "Share of official statistical reporting forms that could be submitted to NSO electronically". Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Computer equipment of Text 2007 - 1 PC for 1 1 to 1 1 to 1 Rosstat staff (MODERN employee computer equipment) 28-Jun-2007 15-Dec-2012 31-Dec-2021 Comments (achievements against targets): Target reached. Planned baseline value was maintained and ensured for all subsequent years under IOI.4-3 indicator thanks to the regular maintenance of high level of technical equipment at Rosstat. At all levels of the statistical system, there were regular updates for hardware and software including that funded by the project. All staff were equipped with necessary equipment, had access to internet, and possessed modern tools and packages for survey design, data collection, verification and analysis purposes. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of statistical tasks Text 2007 - 34 313.00 313.00 resolved on a new technological basis and 28-Jun-2007 15-Dec-2012 31-Dec-2021 forwarded to the United Foundation of algorithms Page 54 of 159 The World Bank Second State Statistical System Development Project (P092429) and programs Comments (achievements against targets): Target reached. The target value of this IOI.4-5 indicator (moving tasks to new technology) was achieved in 2010 (just in two years) and has not changed. All statistical tasks and process are using the new technologies since 2011. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of systemic Number 0.00 45.00 146.00 statistical tasks moved to centralized primary statistical 06-Feb-2014 31-Dec-2018 31-Dec-2021 data processing Comments (achievements against targets): Between 2016 and 2018, the value of this IOI.4-6 indicator did not exceed 6 tasks which could not sufficiently affect the quality of the statistical process. A significant progress was made starting in 2019 and 2020, when the values reached 34 and 106 tasks respectively. In 2021, there were 146 statistical tasks have been transferred to centralized processing. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of statistical Percentage 34.00 60.00 60.00 indicators based on electronic economic 04-Apr-2018 30-Jun-2021 31-Dec-2021 description as percent of the Page 55 of 159 The World Bank Second State Statistical System Development Project (P092429) total number of statistical indicators assessed by Rosstat Comments (achievements against targets): Target reached. This IOI.N-7 indicator has been used in the project evaluation framework since 2018, its value in this (base) year was 34 %. The end-target value of 60 percent was successfully reached by the project closure. Component: Component C. Enhancement of social statistics Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Compliance with the ICLS Text Partial Fully complies Fully complies household income and expenditure methodology 28-Jun-2007 15-Dec-2012 31-Dec-2021 Comments (achievements against targets): Target reached. With the introduction of a sample survey of household income and participation in social programs in a regular basis, the target value of this IOI.3-3 indicator “fully complies” was achieved in 2012 and maintained since. This survey and the composition of indicators meet the standards of the Resolution of the International Labor Organization on statistics of household income and expenditure, adopted in 2003 on the 17th International Conference of Labor Statisticians. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Page 56 of 159 The World Bank Second State Statistical System Development Project (P092429) Availability of data on skilled Text Non available Yes Yes. Survey is labor, demand for skilled conducted every two labor years. 28-Jun-2007 15-Dec-2012 31-Dec-2021 Comments (achievements against targets): Target reached. The necessary information on the characteristics of workers of various professional groups was received in regular basis under this IOI.3-4 indicator during the enterprise surveys and could be used for making managerial decisions. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Availability of HBS and LFS Text No Yes Yes microdata bases for users in open access 28-Jun-2007 15-Dec-2012 31-Dec-2021 Comments (achievements against targets): Target reached. In 2008, Rosstat created the database “Microdata of the Household Sample Surveys” within this IOI.4-4 indicator which included the household budget survey databases for 2003–2004 and 2005–2007 and provided open access to all users through the Rosstat website. In 2016, the household income survey database was created and made available to the public. Component: Component D. Human resource development Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Page 57 of 159 The World Bank Second State Statistical System Development Project (P092429) Dynamics of Rosstat staff Text No Yes Yes training (number of trained staff) (persons) 28-Jun-2007 15-Dec-2012 31-Dec-2021 Comments (achievements against targets): During project implementation within this IOI.1-2 indicator, over 30 thousand employees and representatives have been trained attending multiple trainings over the years. The number of trained staff exceeded the annual plans during entire project period. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Dynamics of state budget Text No Yes Yes expenditures for training purposes (thousand rubles) 28-Jun-2007 15-Dec-2012 31-Dec-2021 Comments (achievements against targets): Within this IOI.1-3 indicator, annual plans were made to stabilize the budget for training purposes. During the project duration, Rosstat 's allocation for training budget exceeded STASYS2 training allocations with few exceptions in 2010, 2012, 2018, 2020 and 2021. Significant budget funds were allocated for training of Rosstat employees, together with STASYS 2 funding, all made it possible to provide necessary knowledge and skills to be obtained and provided opportunities to solve the complex methodological and structural tasks of the project. Component: Component E. Project management Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Page 58 of 159 The World Bank Second State Statistical System Development Project (P092429) The Statistical Council is Text No Statistical Council Yes Yes established 28-Jun-2007 15-Dec-2012 31-Dec-2021 Comments (achievements against targets): Target reached. For the IOI.1-1 indicator, there were two Statistical Councils, Scientific and Methodological Council and Public Statistical Council, established in 2010 and 2011 correspondingly - a year earlier than the planned date. At their meetings, most important issues for developing the national statistics were discussed along with challenges and issues relevant to the successful implementation of the project. Page 59 of 159 The World Bank Second State Statistical System Development Project (P092429) B. KEY OUTPUTS BY COMPONENTS The summary of key outputs that led to the project achieving its respective PDO elements is presented below by the main four components. Modernization of economic statistics methodology: - A methodology has been developed for analyzing and accounting enterprise groups while compiling industry accounts of the SNA, which makes it possible to introduce into Russian statistical practice methods of analyzing and statistical profiling groups of dependent legal entities, whose share in the total volume of production in the Russian Federation is quite high. Profiling of enterprise groups, when compiling the SNA production and generation of income accounts by sectors, allows one to get a true picture of the allocation of factors of production (labor and capital) and outcomes of production (output, value added) by kind of economic activity, and eventually to properly distribute production capacities across the regions of the country. - In the framework of introducing a compilation of national accounts for natural resources, and a compilation of an assets and liabilities balance sheet with the account for natural assets, within the project the methodology for constructing the System of environmental and economic accounting (SEEA) and the road map for implementing SEEA into the statistical practices of the Russian Federation were developed. This work was a high priority one in view of the recommendation of international organizations such as United Nations Statistical Division (UNSD), OECD, IMF, World Bank Group, and European Commission who adopted the implementation of the SEEA-2012 as a central framework and international standard in March 2012. - As part of the efforts aimed at the transition to the 2008 SNA standards, within the framework of the project, methodologies have been developed: for treatment of expenditures on military equipment as fixed capital formation; for treatment of intellectual property products as fixed capital, for assessment of the consumption of fixed capital as value added, etc., which were applied for obtaining annual and quarterly series of GDP and its components for the period from 2011 to 2016. - A system for seasonal adjustments of macroeconomic indicators was developed and implemented that enables constructing models for seasonal adjustments of main macroeconomic indicators. - A methodology for the anonymization of microdata for presentation to users for analytical purposes was developed and tested on the example of a structural survey of enterprises. Development of modern design and technology for statistical data collection, processing, and dissemination - A modernization of the computing and information system (CIS) of Rosstat was undertaken that made it possible to improve the system for collecting and processing statistical data (extension of the system for SNA indicators), to introduce elements of multi-dimensional data Page 60 of 159 The World Bank Second State Statistical System Development Project (P092429) analysis using integrated data storages with the purpose of creating current inquiries and carrying out analytical data processing (including arbitrary inquiries) to ensure a transition to the centralized statistical data processing and create efficient integrated system for protecting statistical resources from unauthorized access. - In the headquarters of Rosstat and the Main Interregional Computing Center of Rosstat, based on previously created information subsystems (including the Unified Agency Multiservice Network, the Unified Data Storage, the WEB-collection system, etc.), a modern information system has been created and is being supplemented with new components, which, in essence, are a production "conveyor" for collecting, processing and presenting official statistical information. Within the framework of this system, all the main tasks for maintaining and developing the IT infrastructure have been transferred to the Federal center, what ensures the optimization of resources. - A technological platform has been developed and put into operation to implement an integrated approach to organizing and conducting sample surveys of households (population), which helps to improve the quality of preparation and formalization of the methodology for processing household sample surveys; reduction of costs of material and time for preparing, conducting, processing and obtaining survey data; ensuring comprehensive measures for protecting personal data at the stage of survey data processing; preparation to use this platform as an integral part of the Digital Analytical Platform (DAP). - The concept of using Big Data in state statistics was developed taking into account international recommendations and good practices. Methodological approaches to the use of Big Data in certain statistical domains were presented through a roadmap and relevant activities were proposed covering the methodological, organizational, and regulatory aspects including activities to build data exchange and interactions with data sources and digital receipt holders. This enabled the data from digital and administrative sources to be used in consumer price statistics, retail, and tourism statistics, and in population statistics regular basis. Enhancement of social statistics - Methodological guidelines were developed for conducting comprehensive and specialized thematic surveys of the living conditions of the population and poverty monitoring. - The analysis was carried out and the methodology and techniques were improved for preparing microdata and aggregated data of household budget surveys and for providing access to microdata and aggregates for external users. - Recommendations were made to improve the empirical base and ensure the introduction into statistical practice integrated indices of well- being and quality of life and SDG indicators for the population as a whole and for certain population groups in accordance with the UN standards. - A methodology was developed for compiling statistics on social benefits, both monetary and in-kind, using administrative data available at Page 61 of 159 The World Bank Second State Statistical System Development Project (P092429) the Federal and regional levels. - Methodological recommendations were developed for estimating household income, expenditure and savings by various income groups and for coordinating the results with the estimates of the System of National Accounts 2008. - LFS program and techniques for calculating individual weights to obtain labor force survey statistics monthly for Russia as a whole and for the constituents’ entities of the Russian Federation were improved. - A set of works was carried out aimed at improving health statistics with regards to mortality statistics and demographic projections. Human resource development - Development of educational and methodological materials and application software for training Rosstat employees on seasonal adjustments of official socio-economic statistics, on constructing samples for statistical surveys, and on editing statistical data. - Organization and implementation of training programs at the national and international levels in various statistical domains, including: (i) use of modern information technologies in the production processes of Rosstat, (ii) quality management in official statistics; (iii) social, demographic, and health statistics; (iv) national accounts and supply and use tables; (v) price and finance statistics, (vi) labor, education, science and culture statistics, and (vii) seasonal adjustment of social and economic statistics. STASYS-2 Outputs Inputs/ Activities Outputs and their Introduction into statistical practices Contribution to PDO Component A. Modernization of economic statistics methodology A.1.21 Developing methodology for Guidelines have been developed for introducing environmental accounts into the Increase in data compiling environmental statistical practices of the Russian Federation in accordance with the standard of reliability. Introduction of economic accounts the system of environmental and economic accounting (SEEA). international standards Based on the guidelines, an action plan (“road map”) was prepared for the and good practices. implementation of priority accounts of the environmental and economic “Road map” is accounting system, which established the order of work for the federal executive considered for approval authorities on nine priority SEEA accounts and determined the timing of their in the Government introduction into statistical practices. office. The document went through several stages of finalization and coordination with Page 62 of 159 The World Bank Second State Statistical System Development Project (P092429) interested federal executive authorities, including a follow-up to the meeting of the Interagency working group organized under Rosstat. The Ministry of Economic Development of Russia sent the draft document to the Office of the Government of the Russian Federation, and it is undergoing final approval. Currently, the work is underway on the experimental compilation of time series for five SEEA accounts: - environmental protection expenditure account - environmental taxes and fees account - environmental subsidies and similar transfers account - water flow account - air emissions account Guidelines for the environmental taxes and fees account compilation have been developed (approved by Rosstat Order No. 872 dated December 31, 2020) and available at: https://gks.ru/bgd/free/meta_2010/IssWWW.exe/Stg/2020/met-ukaz- ekol.docx. The results of SEEA financial accounts compilation are posted on the website of Rosstat: https://rosstat.gov.ru/storage/mediabank/itogi_form-schet.pdf. A.1.3 Methodology for evaluation Evaluation of non-financial assets in fixed capital (by types) have been available Introduction of of non-financial fixed assets annually since 2015, included in the assets and liabilities balance sheet and capital international standards in current and constant formation accounts. The balance sheet is published on the Rosstat website. and good practices. market prices An official statistical methodology for calculating the current market value of fixed Increase in data assets by their types was prepared and approved by Rosstat Order No. 216 dated accuracy. April 16, 2021. А.1.5.а Developing methodological Improvement of index calculations in the methodology and practice of calculating Introduction of recommendations to production indices in line with the principles and standards accepted in international international standards account for changes in practice, ensuring harmonization and comparability of domestic indices with other and good practices. consumer properties, and countries. The optimal methods were chosen considering the cost, technology, and Increase in data accuracy individual, low-volume, and timeliness of production indices calculation in Russia. and reliability. irregular output while estimating output dynamics Page 63 of 159 The World Bank Second State Statistical System Development Project (P092429) А.1.5.b Developing methodological Based on the results of the work, the Official Statistical Methodology for Calculating recommendations to the Industrial Production Index was approved (Order No. 301 of Rosstat dated May account for products with 8, 2014, was adopted. long production cycles and New sections (4.1 - 4.3) were added on the accounting for changes in quality and for changes in consumer consumer properties of goods when calculating production indices, individual, properties while estimating small-scale, and irregular production, accounting for products with a long output dynamics; conducting production cycle. The Methodology went through a public discussion procedure pilot surveys with the participation of representatives of expert and academia circles at the meeting of the Scientific and Methodological Council of Rosstat. The methodology and techniques of calculating production indices has changed. The deflation method has been expanded, both within the “basket” of items (from 46 items in 2010 to 64 in 2014), and the index calculation method change from extrapolating volumes in physical units to deflating volumes in money terms, by kind of activity (by 2017 - 40 kinds of economic activity, including kinds of activities with a long production cycle). Accounting for irregular production has begun as part of seasonal adjustments of index series using modern DEMETRA 2.2 software. The gaps that existed in the methodology since 1964 were eliminated. These gaps were related to the accounting of products with a long production cycle, individual, small-scale, and irregular production, and changes in product quality. This has allowed an increase in the trust of users for the official statistical information. The outcome of this work used by Rosstat for a retrospective recalculation of production indices in 2014 when new base year switched to 2010. The results of the work and its subsequent application have significantly improved the quality and reliability of production indices for Russia as a whole and for its constituent entities. Harmonization and comparability of data on indices provided by the Russian Federation to the international organizations have been ensured. Rosstat did not receive negative feedback from international experts. Page 64 of 159 The World Bank Second State Statistical System Development Project (P092429) А 5.1 Developing methods for The possibilities of using statistical microdata for analytical purposes by a wide Microdata will become anonymization of microdata range of researchers have been expanded, while ensuring the confidentiality of the available to a wide range for provision to users for primary data of respondents. of researchers after the analytical purposes and their Methodological guidelines for compiling sets of anonymized microdata from implementation of the testing on the basis of annual structural survey collected with the federal statistical form No. 1- Digital Analytical structural enterprise survey enterprise "Basic information the enterprise activity" (Order of Rosstat dated April Platform and the 19, 2013, No. 165) were developed and approved. completion of the The guidelines include a description of various methods and algorithms for primary development of software data anonymization, rules, and methods for ensuring of direct and indirect data tools for anonymization confidentiality. The developed methods of anonymization use a unified approach of primary data. Lead to and do not lose their relevance, since they do not depend on the time availability of more characteristics of the microdata used. analytical papers to be Currently, the development of software for creating sets of anonymized microdata used for programs/ and is carried out within the framework of the Digital Analytical Platform (DAC). policy formulations. A.1.6. Developing methodological The methodology for calculating volume of production and production index for The generated indicators recommendations to agriculture in value terms has been improved with the account for the introduction are the indicators of the calculate volume of of new calcifications harmonized with the European Product Classification (CPA). state program for the agricultural production and It was harmonized with the European methodology and recommendations were development of volume index in the prepared for retrospective recalculation of agricultural production volume agriculture and framework of new timeseries in current and constant prices by the categories of agricultural producers regulating markets of classifications harmonized at the federal and regional levels, as well as the preparation of a list of goods and agricultural products, with the CPA (EEC) services used as inputs for producing crops and livestock based on input-output raw materials and food tables. products, and are also The following outputs were produced: included in the list of - algorithms for calculating volume and index of total agricultural production for indicators related to Russia and its constituent entities, as well as by some categories of agricultural ensuring the food producers, with indication of data sources; security of the Russian - experimental calculation of the total volume and index of agricultural Federation and are used production for Russia and individual subjects of the Russian Federation and by to make political some categories of agricultural producers; decisions for the Page 65 of 159 The World Bank Second State Statistical System Development Project (P092429) - procedures for calculating volume and index of agricultural production at the development of federal and regional levels, taking into account the results of the experimental agriculture. calculation; - algorithm for retrospectively recalculating time series of volumes and indices of agricultural production; - list of goods and services in Crop and Livestock at the 6-digit level to be used by the Department for Input-Output Tables Development and Enterprise Groups Statistics for constructing basic Input-Output Tables. Based on this work, Methodological Guidelines for calculating the volume and index of agricultural production were developed and approved by order of Rosstat dated 06.09.2011 No. 385. A.1.7 Developing methodology to Improving the accounting of production costs and production output by statistical Introduction of determine statistical units units defined closely to the standards of the 2008 SNA: international standards for compilation of the • in the software module for the harmonization of data on production, labor, and and good practices. production account and capital a data set was compiled for calculating SNA estimates based on local units of Increase in data reliability generation of income companies (local statistical units of production) account in compliance with • local units of the legal entities have been entered into the Statistical Register of the SNA concepts Rosstat, with a unique identifier for each unit. Methodological guidelines for registering and identification of economic entities in the statistical register were approved by the order of Rosstat dated December 12, 2016 No. 786 • for the first time in Russian statistics a term “Enterprise group” was introduced as a production accounting unit. Rosstat has established a new Department for Input- Output Tables Development and Enterprise Groups Statistics, which develops a methodology for statistical accounting of enterprise groups A.1.8 Development of An information base for calculating Gross Regional Product (GRP) has been Increase in data reliability Methodology for multistage established at the federal level, including algorithms for calculating output, and accuracy. assessment of regional intermediate consumption, and gross value added. GRP components are calculated macroeconomic indicators in a centralized way: gross value added by certain kinds of activities, consumption of fixed capital by constituent entities of the Russian Federation, some elements of the disposable income account (household final consumption expenditures, social Page 66 of 159 The World Bank Second State Statistical System Development Project (P092429) transfers in kind, household actual consumption) and elements of the generation of income account by constituent entities of the Russian Federation (wages, other taxes on production, gross profits of the economy, and gross mixed income). As a result of the gradual transition to centralized calculations of individual GRP components, the comparability of macroeconomic estimates at the regional and national levels has significantly improved. A.1.9 Elaboration of methodology Consumption of fixed capital was estimated in accordance with the provisions of Introduction of for measuring consumption the 2008 SNA, the estimate was included in the calculations of GDP components, international standards of capital in national starting from the calculations for 2011, and GRP calculations from the estimate for and good practices. accounts 2016. The official statistical methodology for calculating consumption of fixed capital was approved by Rosstat Order No. 21 dated January 22, 2021. A.1.10 Comprehensive evaluation of A thorough review and analysis of the statistical system was carried out, and a Sustainability of STASYS 2 Russian statistical system strategy for the development of the Russian statistical system for the period 2014- project results and and identification of its 2024 - Rosstat Strategy until 2024 was developed, including the next five year Plan impact. development priorities for the Development of the State Statistics System. Specific activities are proposed for each of the components of the development strategy. Respective provisions of the strategy and its activities were implemented within the framework of the government program and the STASYS 2 project in 2014-2021. A.1.11 Methodology for This work resulted in: Increase in macrodata constructing tables for • the implementation of a centralized model for maintaining the Statistical Register accuracy and reliability. calculation processes and (Order of Rosstat dated November 25, 2016 No. 744 “On approval of the Provisions recommendations for their of the Statistical Register”). Methodological guidelines for recording and implementation into identification of economic entities in the statistical register were approved by the statistical practices to order of Rosstat dated December 12, 2016 No. 786. estimate accuracy of macro- • the implementation of a centralized data processing system (Order of Rosstat economic indicators dated February 6, 2017 No. 83), which incorporates a tabular method for assessing the quality of the obtained statistical information Page 67 of 159 The World Bank Second State Statistical System Development Project (P092429) • construction of models for calculations of gross domestic product, including (i) - autonomous selection of adjustments to the statistical survey using the form 1- enterprise (large and medium), the results of full-count observation of small enterprises (including micro); (ii) adjusting data for economic transactions not observable by direct statistical methods; (iii) harmonization of calculated estimates for individual industries and sectors of the economy with the data from individual forms of statistical observation. • construction of tables for calculating national accounts estimates, including (i) sources and methods for obtaining data (surveys and censuses, administrative sources, integrated data, extrapolation and modeling, other sources); (ii) algorithms that enable estimation of the magnitude of underlying data adjustments (validation adjustments, conceptual adjustments, explicit adjustments for under-coverage, balancing adjustments). A.1.12 Methodology for accounting Outputs: Increase in macro data for intellectual property in • intellectual property products are included as one of the types of fixed assets with accuracy and reliability. the national accounts of the their further breakdown in the forms of federal statistical observation No. 11 and No. RF 11 (short); • a variable ‘investment into the intellectual property products’ with a breakdown by types of products is included in the form of federal statistical observation NП-2 (invest) • calculations of gross capital formation as regards R&D were done by commodity flow method on the basis of the Supply and Use Tables, and with the obtained data the value of R&D stocks was estimated using perpetual inventory method. Based on survey results and data from Supply and Use Tables, starting from 2011, intellectual property products are included in gross fixed capital formation. By applying the perpetual inventory method their current market value was estimated and included in the fixed assets, which are reflected in the balance sheet. In connection with the expansion of the boundaries of fixed assets in the formation of fixed capital by including the R&D products, the structure of GDP was defined more precisely. Page 68 of 159 The World Bank Second State Statistical System Development Project (P092429) Annual and quarterly estimates of GDP and its components, starting from 2011, were revised to account for R&D products and published on the official website of Rosstat. A.1.13 Methodology for treatment The results of the study have contributed to the GDP calculations: starting from Increase in macro data expenditures on military 2014: expenditures on long-life weapons systems have been included into capital accuracy and reliability. equipment as gross fixed assets and gross fixed capital formation. capital formation in the national accounts of the RF A.1.16 Development of The official statistical methodology for estimating the share of the oil and gas sector Increase in macro data methodology for measuring in the gross domestic product of the Russian Federation, calculations for 2017, 2018 accuracy and reliability. oil and gas sector GDP in the and 2019 was developed and approved by Rosstat Order No. 299 dated June 2, 2021 Russian Federation and published on the official website of Rosstat. The draft methodology was discussed at the section "Macroeconomic statistics and SNA" of the Scientific and Methodological Council of Rosstat on April 29.2021. A.1.18 Development of The developed algorithms are introduced into the calculation procedures for Increase in macro data methodology for centralized obtaining estimates for the government sector in the constituent entities of the accuracy and reliability. assessment of government Russian Federation. These components estimated within GRP, generation of income sector macroeconomic account, and actual final consumption by the constituent entities of the Russian indicators in the regions of Federation, starting from the calculations for 2016. Russia А.1.22 Developing methodology for An official statistical methodology for obtaining macroeconomic estimates of Increase in macro data estimating natural resources natural resources productivity and intensity of use has been developed, considered accuracy and reliability. productivity at the Scientific and Methodological Council of Rosstat on October 22, 2020, approved by Rosstat Order No. 737 dated November 27, 2020. Statistics have been published on the Rosstat website in the National Accounts section since July 2021/ А.1.23 Development of The developed methodological recommendations were considered at the section Increase in macro data methodological guidelines "Macroeconomic statistics and SNA" on 24.04.2018. accuracy and reliability. for the compilation of assets Page 69 of 159 The World Bank Second State Statistical System Development Project (P092429) accounts for fossil mineral Implementation of the recommendations will be carried out within the framework and energy resources within of the System of Environmental and Economic Accounting (SEEA). the environmental-economic The Ministry of Economic Development of Russia submitted to the Government of accounting framework the Russian Federation a draft order of the Government of the Russian Federation “On approval of the action plan (the road map) for the implementation of priority SEEA accounts” (letter No. 43488-RM/D05i dated December 19, 2020). А.1.24 Development of Outputs: Increase in data reliability methodology for the • Methodology for determining the composition of enterprises groups (legal and accuracy. treatment of enterprise entities) and developing their accounting in the economy of the Russian Federation groups while compiling SNA was developed and approved by the order of Rosstat No. 122 dated March 13, 2020; industry accounts • an algorithm was developed for calculating gross value added of multi- territorial enterprises at the place of VA creation, by industries (experimental calculations were done); • the information base for determining perimeters of enterprise groups was expanded through the use of administrative sources (agreements on information sharing were concluded with the Federal Antimonopoly Service and the Central Bank), as well as by using information on the structure of the charter capital of joint- stock companies obtained from the registrars of shareholder registers (Federal Law No. 174-FZ of 11.06.2021 was adopted :“On Amendments to the Federal Law “On Official Statistical Accounting and the System of State Statistics in the Russian Federation”) А.2.1 Development of Implementation of the project output: Increase in data accuracy methodological • in the Statistical Register of Rosstat, territorial divisions (local units) of legal and reliability. recommendations for entities were registered through using a unique identification for each local unit of assessment of output, a legal entity regardless of their locations. Methodological guidelines for intermediate consumption identification local units and entering then into the statistical register were approved and value added for multiple by the order of Rosstat dated December 12, 2016 No. 786 locations companies and • instructions were approved for filling out and submitting forms of federal production groups statistical observation, including No. P-1 and P-4, and obligations for multi-territorial Page 70 of 159 The World Bank Second State Statistical System Development Project (P092429) enterprises to provide statistical reports separately by each local unit were established (posted on the website of Rosstat in the heading "Information for respondents"). • the Centralized Data Processing System (Rosstat Order No. 83 dated February 6, 2017) was implemented, which makes it possible to obtain statistical data on multi- territorial enterprises at the federal level, validate and download them for calculating output, intermediate consumption and value added of multi-territory companies • a data set on the multi-territorial companies was compiled on the basis of the data from the Statistical Register, the Centralized Data Processing System, and electronic data sets in the program module for harmonizing data on production, labor and capital. Models for calculating GDP were designed within the program module, and output for local units was estimated. Data on multi-territorial companies and enterprise groups are updated in the program module on the annual basis А.3.1 Methodological approaches Statistical tools have been developed and approved to organize monitoring of the Increase in data accuracy to statistical observation in volume of work performed in "Construction" by main types of construction work: and reliability. the construction industry on federal statistical observation form No. UVR-S "Information on the main types of the basis of aggregated kinds construction work performed by the company on its own and instructions for of work and using volume reporting”. and price indices obtained by An algorithm for generating statistical information for calculating price indices and the method of construction physical volume indices of construction products using the component method has components been developed. In 2012, a pilot survey of construction organizations with more than 250 employees was conducted using these tools. A comprehensive analysis of the situation in the construction industry was carried out based on a comparison of the dynamics of the production of basic building materials, types of work, investments in fixed assets and the commissioning of fixed assets. Also, the information obtained on the cost of construction work made it possible to specify the cost structure for the production of construction products used in calculating the gross value added in construction.2011 Due to the fact the Federal Law “On Business Accounting” No. 402-FZ of December 6, 2011 which came into legal force on January 1, 2013, canceled the obligation to use primary accounting documents of a standard Page 71 of 159 The World Bank Second State Statistical System Development Project (P092429) form for recording economic activities of enterprises, starting from 2013, organizations do not have information on the types of work performed. This circumstance does not allow introducing observation in construction industry on the basis of main types of construction work into statistical practices. А.3.3 Development of Amendments were prepared and introduced into the Methodological guidelines for Increase in data accuracy methodology for assessment monitoring consumer prices for goods and services and calculating consumer price and reliability. of cohesive consumer price indices, approved by Rosstat Decree No. 110 dated December 30, 2005 (Rosstat indices at regional and Order No. 467 dated August 28, 2012). According to these changes, data on the federal levels structures of consumer expenditures of the population are generated at the federal and regional levels to be used for calculating total consumer price indices (CPI) on the basis of household consumer expenditures for two recent years shifted according to unified algorithms, taking into account updating coefficients for average annual expenses. The use of updating coefficients made it possible to ensure the comparability of basic indicators, since in the CPI calculations the prices of the base period are the prices of December of the year preceding the reference year, and the weights are the structure of consumer expenditures of the population estimated with the account for price changes at the end of the previous year. Component B. Development of modern design and technology for statistical data collection, processing, and dissemination B.1.1 Developing corporate The centralization of hardware and software solutions has been implemented. A The accessibility, information solutions on the centralized solution has been introduced, so that all Internet portals of Rosstat, reliability and timeliness basis of portal technologies including TBS Internet portals, are on a common hardware and software platform in of information provided (development functional the central node? to users improved. potential of Rosstat Internet- Achieved: high accessibility, integrity and reliability, transparency of information portal) allowing the use of resources provided to users through the application of various and popular forms of statistical web-applications information presentation and interactive services. ensuring an administered Capacity: search optimization, end-to-end search, incl. in external search engines; access for users to various simultaneous work of at least 10,000 users in page view mode; the response time information resources and for user access to statistical information of a unified Internet portal does not exceed 2 seconds, having reduced from 5 to 2 seconds; ensuring accessibility for people Page 72 of 159 The World Bank Second State Statistical System Development Project (P092429) services including mapping with disabilities; reduction in the total cost of owning and maintaining the entire data representation complex of hardware and software of Rosstat Internet portals. B.1.2 Enhancement of the internal A System has been created that allows minimizing delays in the dissemination and Data timeliness corporate intranet-portal of communication of official information to Rosstat employees, ensuring targeted increased. the state statistical system to delivery of information and newsletters. This System improves the efficiency of be integrated with the personnel management and creates conditions for supporting feedback from statistical data storage, Rosstat personnel. document management In addition, a universal tool has been created for convenient work with the subsystem, and HR information resources of Rosstat, and a mechanism has been developed for subsystem generating content in the System. It is planned to provide centralized access to the information resources of Rosstat, provide convenient and efficient opportunities for searching for information in the System. B.2.1 Development of a modern On the basis of modern software tools and information technologies, the software Data timeliness and system of the demographic for electronic processing vital and migration data was developed and introduced accuracy increased data collection and into commercial operation in 2011. processing B.3 Development of a system for Research in this area was carried out by Rosstat when implementing a system of Increase in data collection, analysis, and population sample surveys on social and demographic issues for the period up to reliability, accuracy and dissemination of the 2025. timeliness household and population The first stage of work on creating a platform for conducting sample surveys of the sample surveys data population and households was implemented and it has been further developed within project B.3.2 B.3.2 Introduction of an integrated Development and implementation of special software for automating the procedures Increase in data approach to organizing and for survey preparation and conducting (Collection System) and a centralized data accuracy, reliability and conducting household processing system of survey data at the Federal level (Processing System), which timeliness. sample surveys using a makes it possible to unify the methods of collection, processing and dissemination methodological and information in the framework of the system for the population and households technological platform sample surveys and is aimed at reducing the cost of conducting sample surveys. Page 73 of 159 The World Bank Second State Statistical System Development Project (P092429) A trial operation and training of Rosstat personnel how to operate the special software have been carried out, the system is being deployed and implemented within the Rosstat infrastructure B.4.1 Development of relevant A data base software was developed and implemented for calculating SNA totals. Increase in data software and creating This software product improves access to national accounts data, allows you to timeliness databases to produce generate various formats for presenting SNA indicators. summary national accounts The software package was finalized in 2020 in accordance with the contract B.4.10. indicators B.4.2 Developing hardware and Development of information base for calculating SNA estimates. An array of data has Higher quality data for software to produce been created to calculate macroeconomic indicators at the federal and regional levels macro analysis. including harmonized data on (GDP, GRP) for policy formulation production, labor and capital The algorithms and procedures incorporated in the software package enable at micro- and macro-levels comparing lists of observation units surveyed through using the reporting form: 1- enterprise, P-1, P-4, P5 (m), P-2 (invest), 11, 11-short, PM, MP (micro) and DB BOO, integrating data from these forms and calculating main macroeconomic indicators: output, turnover, accrued payroll fund, average number of employees by institutional sectors for the economy of the Russian Federation as a whole and by constituent entities of the Russian Federation. The results of calculations according to the given algorithms are the information basis for obtaining annual estimates of GDP and GRP and consolidated SNA accounts. The development of such product made it possible to harmonize data on the output and factors of production at the regional and national levels, which significantly increases the analytical value of national accounts. B.4.3 Developing an integrated The development and refinement of software tools is intended to improve the Data timeliness system of registers and usability of the Statistical Register in the Rosstat system, to improve its quality and providing integration with timeliness of the compilation of up-to-date lists (catalogues) of statistical the Rosstat CIS metadata observation units, to expand the possibilities of using the Statistical Register for subsystem Federal statistical observations. Page 74 of 159 The World Bank Second State Statistical System Development Project (P092429) B.4.4 Development of metadata The deployment of this subsystem enables generating necessary and sufficient Increase in data accuracy. subsystem of the Rosstat CIS reference and regulatory information in the process of preparing economic (1st stage - preparation of descriptions which is used in the statistical production process. economic descriptions) Duplication, inconsistency and redundancy of regulatory and reference data that are uploaded into the catalog of statistical indicators from economic descriptions of statistical works are eliminated. A function is provided for automatic generation of printed forms and XML templates of electronic versions of statistical reporting forms at the stage of preparing economic descriptions of statistical work for their timely posting on the Rosstat website. B.4.5 Develop software for multi- The implementation of the System made it the possible to increase the efficiency of Data timeliness increased dimensional data analysis Rosstat Departments in performing tasks of multidimensional analysis, generating of using integrated data arbitrary queries and the timely preparation of analytical reports on an arbitrary set storages with the purpose of of historic data and subject matter areas in the system of multi-dimensional data creating current enquiries analysis and data processing, including possibility of The development of a technology for consolidating data in a single measurement arbitrary enquiries space was provided, and tools for multidimensional data analysis was developed for the entire set of variables and historical periods in a common consistent space of reference. B.4.6 SuperSTAR The following microdata bases have been loaded into the SuperSTAR system: Establishing integrated - All-Russia population census of 2002; resource of primary - births and deaths in 1999-2008; statistical data on population - arrivals and departures in 2005-2008; and households using - Primary data from household budget surveys in 2003-2008; techniques to anonymize - Primary data of the labor market surveys in 1992-2009 individual data accessible via - Part of the data base for the All-Russia agricultural census of 2006 “Personal Internet (open access) with household plots and other individual farms” the help of SuperSTAR The developed procedures of privacy protection ensure protection of respondents’ software personal data when access to these data bases is allowed for Internet users. At present Internet users, with the help of SuperSTAR, can generate arbitrary queries to the microdata base of the Russia’s population census of 2002. Page 75 of 159 The World Bank Second State Statistical System Development Project (P092429) B.4.8 Developing IT solutions Development and implementation of a technology for exchanging statistical Conformation with based on international information between Rosstat and international organizations, as well as national international standards. standards to allow data and statistical agencies with the objective of using common open data and metadata metadata exchange at the dissemination standards and modern information technologies. international level Outputs: − introduction of technology for working with questionnaires to reduce manual data exchange between Rosstat employees (automated monitoring and process control, a standard interface for filling out various questionnaires); − compilation of an integrated information resource containing information on all requests of international organizations; − implementation of the technology for submitting Russian statistics by Rosstat at the request of international organizations, based on the SDMX 2.0 standard The were no negative comments regarding quality of the completed questionnaires, including those in SDMX standard. B.4.9 Upgrading technology for Outputs: Data accuracy and keeping and utilization of - improving the quality of constructing frames for conducting special statistical reliability increased. Statistical register based on surveys by Rosstat and other organizations producing official statistics; centralized information - improving the quality of information contained in the Statistical Register; resources - increasing the accessibility of information contained in the Statistical Register. B.4.10 Developing database In 2020, the IAS SNA software package (B.4.1) was finalized in order to use it for Increase in data software for calculating SNA maintaining a database of national accounts indicators, checking the consistency of timeliness. aggregates (IAS SNA) the entire system and preparing routine and special queries. The modified software of the IAS SNA is hosted on the internal server facilities of the Rosstat CIS, data on the SNA aggregates are being uploaded. Achievement: Further improvement of the access to national accounts data, better opportunities for providing SNA data in various formats at the request of users. Page 76 of 159 The World Bank Second State Statistical System Development Project (P092429) B.5 Institutional strengthening of The completion of this work leads to strengthening of the institutional role of Rosstat Rosstat IT policy framework in implementing information technologies for producing official statistics and developing a theoretical basis for improving information systems of other official statistics producing agencies B.6.1 Development of an efficient The implementation of the System enables an increase in the level of security in integrated system to protect accordance with the regulatory requirements in the field of information security of statistical data from the Rosstat CIS/ Information security has been ensured for the following main unauthorized access to automated procedures: ensure security of data in the - managing the information security process, CIS of Rosstat and in - collecting statistical data from actuators, transmission along the - preparing reports, communication channels - keeping statistics of incidents, etc.; - protecting communication channels; - managing the vulnerability of individual components of the Rosstat CIS; - secure storing key information of the certification authority. Delivery of equipment and licenses to TBS has been completed. Work has been carried out to upgrade adjacent LAN components involved in data transmission. B.7.1 Procurement of license for SuperSTAR licenses ensures data processing for analytical purposes, presentation and SuperSTAR Software dissemination of quantitative data at any required level of aggregation. In 2009, the B.7.3 Procurement of license for following software modules were purchased: SuperCROSS, SuperWEB, SuperWEB2, SuperSTAR Software (5-year) SuperTABLE, SuperCHANNEL, SuperSERVER, SuperVIEW. B.7.4 Extending license for SPSS These software packages are used for processing census data. software Currently, licenses are perpetual and are used for the needs of Rosstat and TBS in application programs to provide links between them. B.7.5 Supply and Installation of Equipment was supplied for installation in two data processing centers: in the Rosstat server and network Headquarters and in the Main Processing Center equipment B.7.6 Procurement of mobile The work contributes to an increase in the efficiency of the labor market survey Increase in data reliability devices with software to program and labor force surveys, ensuring high reliability of statistical data on and timeliness conduct household surveys Page 77 of 159 The World Bank Second State Statistical System Development Project (P092429) employment and unemployment; supports application of modern software and moving to paperless technology. B.7.7 Procurement of equipment As a result of this work, a centralized multiservice network was built, covering the and software for the federal and regional levels of Rosstat (HQ, MIC and TBS of Rosstat). The multiservice development of corporate network is adapted to the growing needs of Rosstat with the account for emerging computer information technologies (scalability). system of Rosstat Hardware and software were delivered to all Rosstat project sites, and the following subsystems were deployed: telecommunications infrastructure monitoring subsystem; subsystem of a unified catalog and basic network services; centralized e-mail subsystem; backup subsystem; information security subsystem; server virtualization subsystem. B.7.8 Additional procurement of Hardware and software were supplied. Increase in data equipment and software for timeliness the development of Due to a stronger and modernized corporate CIS, Rosstat managed to conduct corporate computer Population Census 2020 online for the first time in history. information system of Rosstat B.7.9 Technical equipment of the Delivery and installation of equipment. library B.7.10 Supply and installation of the The equipment has been delivered and installed. The efficiency of the labor market system for mobile data survey program and labor force surveys has been improved, and high reliability of collection statistical data on employment and unemployment has been ensured. A paperless technology has been introduced. B.7.11 Modernization of Unified Modernization of the Unified system for collection, processing, storage and System for Records presentation of statistical information Collection (USRC) aimed at The result of the work was an acceleration of the interaction between the components centralization of primary of the Federal and regional levels. The consistency of processing technology in the statistical data treatment regions was ensured. Timeliness was improved. The labor intensity of the USRC Page 78 of 159 The World Bank Second State Statistical System Development Project (P092429) support was reduced. In fact, a transition to a centralized collection of statistical records was achieved. B.8.1 Methodology for development and monitoring of performance indicators of Analysis and Major Government (ONDP) recommendations projects in spheres of elaborated were used by healthcare, social support of respective government families and socially agencies while preparing vulnerable group, pension the state programs and system and labor market ONDP projects in development corresponding arears. B.8.2 Methodology for Development of a system to monitor performance indicators of Major Government development and monitoring (ONDP) projects was included into scope of STASYS 2 activities due to the GOR move of performance indicators of to new governing methods – result based management and introduction of project Major Government (ONDP) modality for state programs. projects for development of manufacturing industry, ONDP (Main Directions of Government Activities) projects were developed with a agriculture, and transport purpose of creating necessary conditions to elevating the society standards of living. B.8.3 Methodology for In this regard, it was important to ensure coordination of main federal institutions’ development and monitoring performance, monitoring of implementation and achieving the results. Activities under of performance indicators of B.8.1-6 supported elaboration of sets of indicators to monitor the implementation Major Government (ONDP) and development of solid methodology for calculating the results indicators in: projects in the sphere of - healthcare, social support of families and socially vulnerable group, pension regional policy and use of system and labor market; natural resources - manufacturing industry, agriculture, and transport; B.8.4 Methodology for - regional policy and use of natural resources; development and monitoring - communication, telecommunication and computer technology; of performance indicators of - science and education, culture, youth policy; Major Government (ONDP) - public administration, financial and economic policy. projects in sphere of Page 79 of 159 The World Bank Second State Statistical System Development Project (P092429) communication, telecommunication and computer technology B.8.5 Methodology for development and monitoring of performance indicators of Major Government (ONDP) projects in spheres of science and education, culture, youth policy B.8.6 Methodology for development and monitoring of performance indicators of Major Government (ONDP) projects in the sphere of public administration, financial and economic policy B.7.12 Modernization of hardware Development of the multiservice network of the computing and information system Increase in data accuracy to provide for fault-free and reliability operation of USRC (CIS) of Rosstat made it possible to ensure the interaction of all CIS subsystems under increasing information loads. The result of the work was to support seamless operation of the Modernized USRC. Reducing the risks of malfunctioning of the Rosstat CIS. Improving the efficiency of computing resource utilization. B.9.1 Development of remote data Testing for the project "Development of remote data collection methods for the Increase in data collection (organizing call- purposes of socio-economic statistics" was carried out during a sample labor market timeliness center to conduct labor survey. The telephone data collection method was tested by setting up a test (pilot) market survey site of the call center in the Rosstat TBS in the Sverdlovsk oblast’. The introduction of remote data collection mode using telephone during monthly labor market surveys helps to increase the participation rate in large cities and improve the efficiency of data collection. Page 80 of 159 The World Bank Second State Statistical System Development Project (P092429) Recently, however, the efficiency of using call centers have been declining and this mode of data collection has been suspended due to increased traffic of fraudulent calls to the public, and consequently, the refusal of potential respondents to answer questions. B.10 Consulting services in the A thorough review of the current state and efficiency of the Rosstat CIS was done development of information using data from various surveys, as well as information collected by the Consultant and technological system of by the questionnaire method in the process of working on this assignment. Rosstat, ensuring the system integration, development of Concept for developing the Rosstat CIS in 2010-2012 and till 2017 was prepared as terms of reference and well as Terms of reference and special terms of reference for subsystems to be technical requirements worked on within the Concept for developing the Rosstat CIS in 2010-2013. Terms of reference and technical specifications for procurement of services and goods within STASYS-2 project were prepared. Coordinating activities aimed at developing the Rosstat CIS and ensuring the implementation of objectives stated in the project for the development of the Rosstat CIS in 2010-2013, В.11 Developing IT-infrastructure A multifaceted development of Rosstat IT infrastructure made it possible to ensure Increase in timeliness of Rosstat to support the reliable operation of Rosstat information systems. The fulfillment of this centralized production of assignment made it possible to increase the degree of automation of the production statistical data processes of Rosstat, to expand the scope of automated processes and to speed up their execution within the production cycle. Due to unification and centralization of production processes, significantly better results were achieved in terms of speed, quality and scale of utilization of statistical resources. Advanced mechanisms for providing access to large volumes of data via telecommunication channels and their analysis enable much wider utilization of statistical information in other sectors, and also provide the possibility to generate statistical data on newly received requests from the Government without additional collection of data from respondent. In addition, this makes it possible to reach higher quality in production and utilization of statistical data, namely: Page 81 of 159 The World Bank Second State Statistical System Development Project (P092429) - high consistency and continuity of statistical information thanks to improvement of approaches to the compilation of regulatory and reference information that supports the process of data collection and processing and establishing integrated data warehouses; - the possibility of generating additional statistical information based on the previously collected data; - generating arbitrary user requests for information in storage and production of electronic statistical publications; - providing opportunities for statistical data analysis based on the use of OLAP technologies - providing Web-access to statistical data repositories via telecommunication channels and via Internet; - -reliability and fault-tolerance of IT modules operations in Rosstat. The IT infrastructure of Rosstat meets the growing needs of programming solutions and increased volume of data processing. B.11.1 Implementing innovative The introduction of innovative mechanisms for managing the production activities of management methods in the Rosstat within the framework of a centralized IT system leads to an increase in the framework of Rosstat efficiency of the production activities of Rosstat, by the establishment of centers for centralized IT-system processing and storing information to improve approaches and tools for centralized planning, collecting and presenting statistical information in compliance with the developed methodology, as well as enabling the generation of totally new statistical products based on the previously collected data. B.11.2 Development of the The program interface for interaction with the SPEED, USRC and Regulatory and operational and Reference Information system (RRI) has been expanded, the functions of the administrative capacity of Centralized System for Data Processing (CSDP) have been widened to process new the centralized system for forms of statistical reporting; procedures for monitoring the completeness of data data processing of Rosstat collection have been developed; the monitoring and control of data processing at all levels have been mastered; capacity for data preparation for timely responses to Rosstat requests has been expanded; efficiency and convenience of using CSDP interface at all levels have improved; procedures for processing reports containing Page 82 of 159 The World Bank Second State Statistical System Development Project (P092429) special information have been enhanced; administrative tools for CSDP have been developed B.11.3 Development of hardware At present, the existing data storage systems (manufactured in 2015-2016) in the and telecommunication server premises of the central offices of Rosstat are almost totally obsolete and worn capacity of the Rosstat CIS out, while being used at 95% of their capacity. Organizational measures and disk space optimization, that have been undertaken since January 2021, have almost exhausted their efficiency. The supply of equipment and installation of software under this contract made it possible to significantly reduce the likelihood of failure of the Rosstat CIS caused by equipment wear and tear, and supported the migration of key information systems to new equipment. In addition, under the amendment to the contract one more Huawei OceanStore Dorado 5000V6 virtualization storage system No. 1 was purchased thanks to achieved savings. This does not completely solve the problems of Rosstat, but significantly reduces their severity. В.12 Developing multi-service The development of the multi-service network in Rosstat supported the interaction network of Rosstat of all CIS subsystems in the situation of growing information loads. В.13 Developing and As a result of the development of telecommunications infrastructure, technologies for implementing modern centralized data processing using large data processing centers based on high- information technologies for performance, fault-tolerant and energy-efficient solutions, using virtualization managing, collecting, technologies and cloud computing are becoming more and more widespread. producing, storing, and The creation of a unified centralized integrated system for collecting, processing, disseminating statistical data storing and disseminating statistical data provided the opportunity for automated configuration of parameters for collecting, processing and storing data based on meta-descriptions of statistical tasks using technological approaches to the previously created system for collecting, processing and storing data. The development of data storage systems has made it possible to provide efficient long-term storage and fast access not only to aggregated, but also to primary data, with the provision of multidimensional analysis mechanisms. The development and implementation of modern management information technologies made it possible to modernize the existing technologies and introduce new corporate information Page 83 of 159 The World Bank Second State Statistical System Development Project (P092429) solutions based on web-technologies that provide access to statistical information resources that have been modernized or newly developed as part of the centralization and unification of Rosstat production processes B.13.1 Developing electronic Development of the Rosstat CIS electronic document flow system made it possible to document flow system in expand the functionality of the agency electronic document management system Rosstat and increase the number of automated business processes to improve the efficiency and quality statistical production management in Rosstat. B.13.2 Improving data The improvement of the Rosstat CIS as regards data dissemination tools has led to a Increase in data dissemination tools within better quality of statistical products in terms of faster production of statistical data. timeliness. Rosstat CIS The methods and tools for information exchange of Rosstat with external organizations - producers and recipients of statistical information - have been improved following changes in the demands of society and the government for statistical data, including those in the open data format. New technological approaches have been introduced to the dissemination of statistical data, data exchange, and interaction with reporting entities, including open data format. B.13.3 Improving tools for The improvement of the centralized data processing tools within the Rosstat CIS has Timeliness and data centralized data processing made it possible to reduce the time and cost and to improve the quality of statistical quality increased. within Rosstat CIS data. Completion of this contract made it possible to increase the degree of automation of the production processes of Rosstat, expand the scope of automated processes and speed up their execution in the production cycle. B.14 Increasing the degree of Higher degree of formalization and automated parameterization in setting tasks for Timeliness and data formalization and automated the collection and processing of statistical data and higher degree of system quality increased parameterization for integration within Rosstat CIS made it possible to significantly improve the quality of problem definition for data statistical metadata and data, as well as reduce the cost of statistical data collection collection and processing, and processing increasing the degree of system integration in Rosstat CIS Page 84 of 159 The World Bank Second State Statistical System Development Project (P092429) B.14.1 Improving approaches to The key actions provisioned for by the Concept for developing the Rosstat CIS in 2011- maintaining metadata in the - 2017 were taken to follow those principles of centralizing data collection and statistical register subsystem processing, which would allow: of Rosstat CIS - a significant increase in the promptness of producing relevant statistical data, B.14.2 Improving approaches to - increase the flexibility and adaptability of Rosstat CIS in order to quickly respond maintaining metadata in to constantly changing external economic conditions, metadata in the RRI - take into account the individual needs of the regions more flexibly while subsystem of Rosstat CIS maintaining the uniformity of statistical methodology, B.14.3 Improving approaches to - to minimize the total cost of ownership of the system in the future. maintaining metadata in the The obtained solution made it possible to significantly increase the consistency of electronic economic statistical data in the Rosstat CIS and provide the possibility for disseminating descriptions subsystem of harmonized statistical information. the Rosstat CIS B.15 Improving IT policy-making in The creation of a methodological and regulatory framework for working out and terms of using general implementing the information technology policy of Rosstat made it possible to unify system software within the methods of collecting, processing and disseminating information within the Rosstat CIS framework of sample surveys of the population and households, which is necessary for developing a comprehensive approach for generating information resources following the principle of unification at all stages of planning, conducting surveys and publishing sample survey data on the basis of common concepts (principles of population data classification, common methodological, organizational and technological platform for collecting, processing and presenting to users statistical products that rely on modern communication technologies and meet the standards of data and metadata presentation). It allowed Rosstat to apply modern international standards for producing statistical data in the course of developing statistical production processes. В.15.1 Improving approaches to IT Improvement of approaches to information policy in terms of system software has strategies in terms of system made it possible to: software - define standards for the use of system software, - reduce the number of software packages currently in use - -reduce expenses for their purchase and technical support Page 85 of 159 The World Bank Second State Statistical System Development Project (P092429) В.16 Unification of collection, The unification of methods for collecting, processing and disseminating data within processing and the framework of the sample surveys of the population and households has led to a dissemination of data from reduction in the cost of conducting sample surveys. sample surveys of population and households В.17 Conducting a study of the As a result of this work, criteria and ways for improving the technological solutions needs of Rosstat in for the CSDP as well as other information systems and services of the Rosstat CIS, improving the automation of which support automated processes of collecting, processing and disseminating the processes of collecting, statistical data collected through forms of federal statistical observation, have been processing and presenting developed. data using the CSDP of the Further development of the CSDP is considered in the context of the overall Rosstat CIS development of the Rosstat CIS, including both development and modernization of a number of existing information systems and the introduction of new IT solutions to improve the performance of Rosstat divisions by increasing the scope of automation in collecting, processing and disseminating statistical data and eliminating manual functions (functions performed outside the information systems), shifting the focus of efforts from controlling the correctness of data provided by respondent to performing analytical functions, including forecasting in various areas of the economy. The results obtained will be used in the future in the preparation of technical requirements for the development of the Rosstat CIS components, which support the processes for collecting, processing and disseminating statistical data. В.19.а Development of the concept Methodological approaches to using alternative data sources in consumer price for using Big Data in official statistics and other branches of statistics are determined. statistics taking into account In consumer price statistics, the concept of using big data relies on combining several international sources of price data, while maintaining the currently approved algorithms for recommendations. calculating price aggregates at the high level, which will ensure comparability of data Development of with previous years. methodological approaches The main sources of price data, in addition to currently used price registration data, to the use of Big Data in are the following: some statistical domains - Cash register data (FTS of Russia); Page 86 of 159 The World Bank Second State Statistical System Development Project (P092429) - Data on goods subject to mandatory digital marking (Chestny ZNAK state track & trace system); - web-scraping for extracting data from websites. Also in the future, it is planned to use data from various service aggregators, such as air and railway tickets aggregators. Combining different sources also makes it possible to increase coverage of cities, various types of trade and service organizations, all types of observed goods (services), and hence significantly increase the scale of price observation, optimize and automate the process of data processing. B.19.b Development of an The implementation of the information system is aimed at improving the information system for completeness and quality of consumer price and retail trade statistics by collecting Big Data from supplementing primary data through automated monitoring of fiscal receipt data various sources for their use (cash register data) using artificial intelligence algorithms. Automated processing of in calculations of consumer cash register data on prices involves the identification and classification of goods and price index and statistics of services, data cleaning (eliminating incorrect data and data that do not comply with trade the methodological principles of price monitoring), ensuring comparability of goods and services over time, imputation of missing prices) Automation of this process will lead to a reduction in manual labor for collecting information on prices and will provide the possibility of transferring alternative data to external information systems within the customized samples. The system is supposed to be implemented in the first half of 2022 and achieve the following results: - improving the completeness and quality of consumer price statistics and retail trade statistics by enriching primary data through automated monitoring of fiscal receipt data using big data algorithms; - reducing the burden on respondents; - reducing manual price collection; - ensuring transfer of alternative data to external information systems within the customized samples. Page 87 of 159 The World Bank Second State Statistical System Development Project (P092429) B.22 Development of an analytical Outputs: subsystem for software - Expanding the capability for analyzing primary and aggregate data from a complex for calculating hardware/software complex for registering prices for goods and services with mobile prices devices and for calculating consumer price statistics using business intelligence methods and tools. − increasing the manageability and transparency of planning, preparing, conducting and monitoring the processes of consumer prices collection. − improving the speed of generating reports in the AIC RC and user requests for data. − optimizing the performance of the data storage of the APK RC. − providing the possibility of automated validation of statistics produced on the basis of data from alternative sources by comparing them with similar statistics calculated in the course of a survey of consumer prices and tariffs for goods and services in the APK RC in order to reduce the amount of manual labor and optimize costs in the future. Component C. Enhancement of social statistics С.1.1 Methodological 1. Methodological recommendations have been prepared for the implementation of HIS data are used: recommendations for a pilot household income survey, including: - when calculating the obtaining and statistical - wording of questions for respondents and instructions for interviewing (to obtain indicator "Reducing the evaluation of indicators of adequate and consistent information); poverty level by half functional and individual - guidelines for conducting a survey and for visual control of the collected data; compared to the level of income distribution based on - the content of the training programs for the survey personnel (program 2017" within the the data of household development, preparation of a training manual with attached achievement tests) framework of the income surveys, including - principles and procedure for selecting target groups of respondents (in view of national development conducting pilot surveys comparing data with external data sources in future) and procedures for their goal of the Russian selection; Federation "Preservation - instructions for interviewers on how to record data in case of impediments for of the population, health direct answers to the survey questions, so that records could be reviewed and and well-being of generalized later; people" in pursuance of - procedures for collecting and summarizing information about respondents who the Decree of the refused to participate in the survey President of the Russian Page 88 of 159 The World Bank Second State Statistical System Development Project (P092429) 2. A methodology has been developed to obtain statistical estimates of household Federation "On the income distribution by main social and economic groups and using this methodology national development to analyze actual data of the pilot survey: goals of the Russian - guidelines on obtaining statistical estimates of household income distribution on Federation for the period the basis of variables included in the pilot survey program; up to 2030" dated July - templates of statistical tables with survey data; 21, 2020 No. 474; - software to obtain indicators of household income distribution by main social and - when calculating economic groups of population and types of households; poverty rate to assess - organizational and methodological guidelines and tools for conducting a full-scale the effectiveness of the survey; activities of senior - a draft survey schedule with description of personnel responsibilities and data officials (heads of the collection and processing procedures; high executive bodies of - regulatory requirements for hiring survey personnel and controlling their the government) of the performance; constituent entities of - assessment of optimal workload for the personnel (with justification of labor the Russian Federation costs) and the activities of - training programs for the survey personnel; executive bodies of the - questionnaires and instructions for selecting respondents, maintaining the constituent entities of sample, and collecting data. the Russian Federation; - when making decisions Analytical and informational outputs. on additional measures 1. Organizational and technical arrangements for the preparatory stage of the of social support for pilot survey of household income, including: citizens: - development of the pilot survey work-plan and schedule; - Federal Law "On - preparation of household sample allocation by pilot regions, instructions for monthly payments to determining observation units and respondent network; families with children" - working out requirements for recruitment of interviewers and providing dated December 28, consultations for recruiting survey staff; 2017 No. 418-FZ; - supervision and consulting during personnel training in the territorial offices of - Decree of the President Rosstat; identifying and summarizing issues of the training programs which need of the Russian Federation to be improved No. 606 of 07.05.2012 "On demographic policy Page 89 of 159 The World Bank Second State Statistical System Development Project (P092429) - preparing action plan for the preparatory stage for household selection and PR of the Russian work among the population; Federation"; - preparation for processing data of the pilot survey (description of the rules for - Decree of the verification of primary data; description of table templates and calculation Government of the algorithms, description of data base creation and verification procedures, Russian Federation No. developing software for entering and verification primary data, creating data 1609 dated 07.12.2019 bases and constructing main statistical tables). "On Approval of the - Preparation of a report on the pilot survey, including Terms of the Far Eastern o main results, as regards coverage (participation/non-participation, reasons Mortgage Program"; for refusals, percentage of errors by questionnaire sections and - Decree of the President completeness of responses), of the Russian Federation o quality assessment of collected data (in accordance with sample plan) and No. 199 dated March 20, assessment of testing (from the point of view of finalization of tool to conduct 2020 “On additional a full-scale survey), and government support for o a summary of repeated interviews conducted during data verification families with children”; activities. - a list of instructions for 2. A database of survey microdata was created and a statistical analysis of the pilot the implementation of survey data was carried out with a description of the metadata. the Address of the President of the Russian 3. The survey data were entered, controlled and corrected. Federation to the Federal 4. The quality of information was assessed, data were edited and imputed using Assembly of the Russian standard statistical analysis tools. Federation dated April 5. An analysis was made of household income distribution and differentiation 21, 2021; (including comparison with data from external sources). - Decree of the President On a regular basis, functional and individual income distributions are evaluated of the Russian Federation using a new source of information – household income survey. of December 17, 2020 Currently, there are more than 8 thousand primary variables in the data base of each No. 797 "On a one-time round of the survey (in 2017 – there were 757); estimated variables - more than 9 payment to families with thousand (in 2017 - 1035). children"; - Federal Law No. 35-FZ of March 1, 2020 "On Amendments to Certain Page 90 of 159 The World Bank Second State Statistical System Development Project (P092429) Legislative Acts of the Russian Federation on Issues Related to the Disposition of Maternity (Family) Capital Funds" С.1.2 Methodological 1. Methodological and practical frameworks have been developed for estimating Accounted for when recommendations for value of goods and services provided to households for free on the basis of integrating analyzing the criteria for constructing the indicator for data from household budget survey and surveys of social institutions, with the assigning targeted social actual final consumption on purpose of subsequent estimation of actual final consumption of households, in assistance: the basis of HBS accordance with the 2003 ILO resolution. These frameworks contain: - Decree of the improvement according to - principles for estimating expenditures of institutions providing free services Government of the ILO recommendations (2003) to the population and providing free goods (medicines, means of rehabilitation for Russian Federation of on Household Income and the disabled, infant-feeding centers, fuel, etc.) per unit of actual consumption by March 31, 2020 N 384 Expenditure Statistics households of the services of these institutions (or goods provided free of charge); "On approval of the basic - a list of goods and services provided to households for free and their requirements fort and classification by types and sources, in order to generate questions in the HBS implementation of program on the volume of consumption of goods and services received by monthly cash payments households free of charge for children 3 to 7 years - forms for extracting from the reports of institutions providing free services old, an approximate list to the population information necessary to estimate the unit cost of the main types of documents of goods or services; (information) required - draft questionnaires with questions about the types of services and for the assignment of the frequency of their reception, which should be tested by conducting in depth specified monthly interview (focus-groups); payments, and a - the results of preliminary testing allowing to identify items that may be standard application unclear and suggest a list of control questions to obtain adequate information form for its assignment"; 2. Methods and procedures for registering household consumer expenditures while - Decree of the conducting HBS were improved, including Government of the - methodological recommendations for collecting data on the volume of Russian Federation of consumption of free goods and services by households as part of a regular survey June 28, 2021 N 1037 (as (list of characteristics for recording, proposed scheme for data collection, proposed amended on January 28, time frames for collecting data from households); 2022) "On approval of Page 91 of 159 The World Bank Second State Statistical System Development Project (P092429) - a questionnaire for surveying households on consumption of free goods and the rules for assigning services; and paying a monthly - recommendations for “incorporating” the new version of the questionnaire into allowance to a woman the HBS registered with a medical 3. The proposed survey program was tested (on the basis of one region covering up organization at the early to 100 respondents) and recommendations were prepared for its practical stages of pregnancy, and implementation, including a monthly allowance for - development of a plan and tools for testing, including the procedure for selecting a child aged 8 to 17 years respondents (households and institutions providing free services to the population); in the part not defined by - organizational and methodological support and testing: (printing forms and the Federal Law "On instructions, recruiting personnel for selecting and interviewing respondents, State Benefits for Citizens training of personnel, monitoring field work, and formal and logical control of the with Children", as well as collected data for their subsequent electronic processing); the list of documents - data entering and checking, constructing microdata base and tabulating data of (copies of documents, the test surveys; information) necessary - description of problems that have emerged during pilot survey and suggestions for the assignment of how to overcome them; these benefits, and - assessment of the feasibility of the data collection program application forms for 4. An algorithm has been developed for combining data on the frequency of receiving their assignment " free services by households and data on the cost of these services in order to clarify the composition of the actual final consumption of households. 5. Indicators for assessing household income taking into account in-kind receipts were introduced into statistical practices С.1.3 Methodological 1. Methodological recommendations have been developed for conducting four recommendations for specialized thematic surveys21 of the living conditions, in which: conducting integrated and - the needs for specific statistical indicators have been systematized, the frequency specific surveys of living of collecting data for each of the subject areas depending on the given survey topics has been substantiated; 21 Survey of the quality and accessibility of services in the field of education, health and social services, survey of the diet of the population, survey of the influence of behavioral factors on the state of health of the population, and survey of the use (budget) of time. Page 92 of 159 The World Bank Second State Statistical System Development Project (P092429) conditions and for poverty - a list of statistical indicators and their breakdown by main socio-demographic monitoring groups of the population (households) was prepared and agreed with the interested user groups; - recommendations were developed for variables to be included in the programs of each specialized survey; variables were divided into two groups: substantive and identification variables. The former is characteristic to a theme under study and the latter reflect the demographic and socio-economic aspects of the population and households; - there were identified compulsory (identification) variables for the population and households used in all surveys, and specific variables used in various “thematic” surveys; - detailed definitions of compulsory (identification) variables were given, and a sequence of questions that should be put forward to respondents in the course of the survey was suggested; - - recommendations were drafted for developing survey programs based on a list of variables related to the principal topic of each survey and the list of compulsory and additional identification variables; - proposals for the design of the survey program were drafted where variables were reasonably distributed by groups of respondents and along the logical sequence of questions; - - the selection of persons to be interviewed was substantiated to avoid duplication of information on certain groups of respondents; - wordings of questions were proposed, taking into account the characteristics of each group of respondents. 2. Methodological guidelines for integrated survey of living conditions and poverty monitoring were developed: - the list of variables to be incorporated in the integrated survey of living conditions was determined basing on the generalized list of variables for socio-economic surveys; - key variables for each of the survey themes were determined, and a group of variables for general description of living conditions and a group of identification Page 93 of 159 The World Bank Second State Statistical System Development Project (P092429) variables for population and households to reflect specific conditions of different population groups including socially vulnerable groups of population were indicated. - recommendations were given for the design and content of the program for the integrated survey including developing questionnaires for different topical modules (living conditions, health and medical care, participation in educational process, financial situation, etc.) and targeted at specific groups of respondents. 3. Recommendations were developed for the module structure of the survey in relation to the planned expansion of sample surveys to be conducted by Rosstat. 4. The possibility of integration of additional surveys into the existing infrastructure of population data collection used by Rosstat was assessed. 5. Recommendations were developed for sample sizes for all specialized surveys (for Russia as a whole and for the subjects of the RF) and for the survey design and program. 6. The experience of national statistical services was reviewed where social surveys are conducted in the form of a permanent (annual) module of the main survey and periodic supplementary topical modules on social issues. 7. Recommendations have been prepared aimed at optimizing the financial needs for the implementation of the integrated and thematic surveys of the living conditions. С.1.4 Methodology for establishing 1. Programs of additional statistical surveys have been developed and tested. These information base to obtain surveys are aimed at working out methods for estimating individual components of estimates of income from household income from work and property within the informal economy (with self-employment and from alternative sources of information), including: property, including - the development of a list of income components earned from self- conducting pilot surveys employment and from property and a list of respective data sources to be found in the existing programs of statistical surveys carried out within the framework of the Federal Plan of Statistical Work; Page 94 of 159 The World Bank Second State Statistical System Development Project (P092429) - the components of income, for the evaluation of which additional empirical data are needed (income from the provision of housing repair services, income from the provision of services for the maintenance and repair of vehicles and equipment, income from private tutoring, etc.) were indicated; - an assessment of various possibilities for obtaining information and substantiation of the most efficient methods for obtaining the necessary data for evaluating income from self-employment and from property; - the development of a work plan and tools for conducting pilot surveys/ including selection of respondents 2. Methods have been developed for integrating data from the experimental surveys into the system of macroeconomic estimates of income components at the regional level, including: ▪ methods for estimating volumes by income components: income from self- employment and from property, based on the results of pilot surveys; ▪ recommendations were drafted for choosing optimal programs for collecting data on income components using alternative data sources 3. Organizational and methodological support for experimental surveys was provided: printing of forms and survey materials, recruitment of personnel for the selecting and interviewing of respondents, training of personnel, monitoring field work, formal and logical control of the collected data for their subsequent electronic processing. 4. A pilot survey of 600 self-employed persons engaged with four kinds of services in four constituent entities of the Russian Federation was conducted. 5. Primary data were entered and checked; the base of microdata was created using the SPSS package; statistical tables with output data were constructed. 6. Estimates of the total volume of money income of the population have been obtained using data on certain income components obtained on basis of experimental surveys for the constituent entities of the Russian Federation in order to test the relevant methods estimating. Page 95 of 159 The World Bank Second State Statistical System Development Project (P092429) С.1.5 Methodology for the 1. Methodological guidelines have been developed including methods for calculating Introduction of development of database indicators of income distribution and poverty, in accordance with OECD standards, in international standards. and assessment of indicators relation to available sources of information, including: of “absolute” and “relative” - recommendations for the calculation of income distribution and poverty poverty based on the OECD rates in accordance with OECD standards; standards - suggestions for choosing information sources in the context of expanding the programs of household sample surveys; - a classified list of indicators used to calculate income distribution and poverty rates; - a detailed description of the methods for calculating income distribution and poverty rates from the OECD publications; - the procedure for applying different thresholds to determine relative poverty rates and the equivalence scales used in the calculation of “equivalized" income; - a list of output indicators for income distribution and poverty rates based on OECD standards, taking into account the specifics of the HBS program, for each year of the survey in the period 1997-2011; - algorithms for calculating income distribution and poverty rate on the basis of OECD standards 2. Recommendations have been developed for the generation of data sets for calculating income distribution and poverty rate based on the data obtained through HBS in the period of 1997-2011 and the HIS data for 2011, including the following: - a whole set of edited data (for each household); - data used for estimating income distribution and absolute and relative poverty rates; - data used for estimating income distribution and poverty rates (on the basis of the whole set of edited data); - microdata for estimating income distribution and poverty rates on the basis of HBS for the period of 1997-2011, in accordance with OECD standards 3. A procedure for automated microdata editing and imputation was developed and implemented on the basis of quarterly HBS for the period of 1997-2011. Page 96 of 159 The World Bank Second State Statistical System Development Project (P092429) 4. A description of all stages of the generation of the annual data array and aggregated indicators with respective algorithms for calculating income distribution and poverty rates in accordance with OECD standards (based on the results of the HBS) was prepared. 5. The following documents were prepared: - descriptions of the implemented methods and the possibilities of their further application in Rosstat; - information on the undertaken procedures of automated editing and imputation. 6. The system for calculating income distribution and poverty rates in accordance with OECD standards was tested, including: - development and implementation of algorithms for calculating these indicators for Russia as a whole and for the constituent entities of the Russian Federation for the period 1997-2011; - preparation of an economic description of the procedure for the obtaining output indicators of income distribution and poverty rate; - construction of tables with output indicators for income distribution and absolute and relative poverty rates on the basis of HBS for the period 1997-2011. for Russia (as a whole and for the constituent entities of the Russian Federation). 7. Recommendations have been drafted for incorporating the respective calculations into the existing procedures for obtaining and publishing data on socio- economic differentiation, including recommendations for: - publishing data on income distribution and poverty rate on the basis of OECD standards, including obtaining preliminary estimates of income distribution and poverty rates; - publishing a new series of data on of income distribution and poverty rates in addition to the existing system of indicators. 8. The results of comparative analyses are described as: - output estimates when using different equivalence scales and different thresholds to establish a relative poverty line - estimates for the period 1997-2011 of income distributions and poverty rates with similar estimates included in the current published data on socio-economic Page 97 of 159 The World Bank Second State Statistical System Development Project (P092429) differentiation (for Russia as a whole and for the constituent entities of the Russian Federation). Estimates of absolute and relative poverty based on OECD standards have been obtained. Starting with the data for 2016, 5 indicators from the OECD's "Income Distribution Questionnaire" have been published; for 2017, 37 more indicators from the OECD Income Distribution Questionnaire were obtained and submitted for placement in the EMISS https://www.fedstat.ru ; and further - all 42 indicators (for the population as a whole, without disaggregation) C.1.7 Upgrading the methodology Output: Increase in data accuracy and technology for 1. Preparation of a list of primary data and aggregates based on the results of the and reliability. processing microdata and HBS for the period 2000-2013, brought to a comparable methodological basis; results of household budget 2. Development of algorithms for calculating aggregates based on the results of the survey to enable access to HBS for the period 2000-2013, brought to a comparable methodological basis; microdata and aggregates for 3. Detailed description of the methodology for compiling additional derived HBS external users indicators for calculating non-standard (non-routine) HBS output data. 4. A detailed description of the technique for estimating the accuracy of the HBS indicators with indications of all its characteristics, and a report on testing the developed technique. 5. General methodological guidelines in relation to: - systematization of information resources for the possibility of prompt extraction of data, supporting working with data in accordance with a consistent methodology; - improving the quality of HBS microdata through improving the procedures for editing and imputing primary data; - working with HBS data using adapted standard data processing tools; - expanding the content of HBS data publications by introducing additional formalized information on the sample design effect. 6. The results of automatic editing and imputation were obtained, including a comparative analysis of the main final indicators obtained before and after editing and imputation. Page 98 of 159 The World Bank Second State Statistical System Development Project (P092429) 7. Detailed description of the requirements for user interfaces for calculating standard (mandatory) and non-standard (not mandatory) HBS aggregates, providing for the possibility of calculating specific tables for users, and reports on testing the above-mentioned user interfaces. 8. Training and consultations were provided to Rosstat employees on the developed methods and technologies. С.1.8 Development of the Introduction of 1. A comparison was fulfilled of the organizational and methodological guidelines international standards. methodology for forming the for HIS and the program of EU-SILC as regards key specifications (reporting periods; statistical base for the lists of indicators, methods used and procedures), ways of operationalization of the HIS data are used as in calculation of indicators of survey target variables, procedures of HIS data processing (data revision methods, C.1.1: income inequality and tracking codes of complex classifications, etc.) - when calculating the poverty, based on the 2. Proposals were prepared for the standardization of the data set obtained through indicator "Reducing the harmonization of the HIS on the basis of constructing equivalents with the EU-SILC, in terms of concepts, poverty level by half program of the sample survey classifications, algorithms and indicators used for compiling data sets for calculating compared to the level of of household income with the indicators of inequality and poverty 2017" within the program of pan-European 3. A description was prepared of the concepts, classifications, algorithms, and framework of the survey of EU-Statistics on indicators used for compiling dataset for calculating indicators of inequality and national development Income and Living Conditions poverty on the basis of HIS data with the purpose to ensure methodological goal of the Russian (EU-SILC) consistence with the EU-SILC. Federation "Preservation 4. Quality assessment of the HIS data was carried out in accordance with quality of the population, health criteria set by Eurostat for EU-SILC (relevance, accuracy, accessibility and clarity, and well-being of comparability with external sources) people" in pursuance of 5. Ways were determined to harmonize the HIS methodology with the purpose of the Decree of the compiling indicators of income inequality and poverty on the basis of a longitudinal President of the Russian module. Federation "On the 6. Recommendations were drafted for using a longitudinal module of HIS for national development obtaining income inequality and poverty indicators. goals of the Russian 7. A description was completed of the most reasonable and optimal conditions for Federation for the period the harmonization of the HIS dataset with the EU-SILC program; up to 2030" dated July 21, 2020 No. 474; Page 99 of 159 The World Bank Second State Statistical System Development Project (P092429) - when calculating 8. Methodological guidelines for compiling a dataset for calculating income poverty rates to assess inequality and poverty indicators on the basis of harmonization of the program for the effectiveness of the the sample survey of household income with the program of EU-SILC, including activities of senior descriptions of concepts, classifications, algorithms and indicators. officials (heads of the On a regular basis, primary data and aggregated indicators of income inequality and high executive bodies of poverty are generated, based on the use of a harmonized methodology with (EU- the government) of the SILC). constituent entities of New areas of statistics have been developed: non-monetary poverty, at risk poverty the Russian Federation and social exclusion rate (AROPE), compilation of indicators for the Sustainable and the activities of Development Goals (SDGs). executive bodies of the constituent entities of the Russian Federation; - when making decisions on additional measures of social support for citizens: - Federal Law "On monthly payments to families with children" dated December 28, 2017 No. 418-FZ; etc. С.1.10 Improving the empirical basis Output: Introduction of that ensures implementation 1. Inventory of the information resources available in Rosstat (databases and international standards. in the statistical practices microdata bases presented in statistical tables) collected through sample surveys on Increase in data accuracy indices for a comprehensive socio-demographic issues. and reliability. assessment of the well-being 2. On the basis of the inventory, a set of variables that can be used to calculate and quality of life in the indices for assessing the well-being and quality of life of the population (at the Federal general population and and regional levels) were determined and adapted. specific groups of the population on the basis of Page 100 of 159 The World Bank Second State Statistical System Development Project (P092429) international best practices, 3. Development of index calculation algorithms for assessing the well-being and including the approbation quality of life of the population based on available information sources in Russia as a whole and for the constituent entities of the Russian Federation. 4. Compilation of the list of missing variables which would allow a more comprehensive coverage of all indices for assessing the well-being and quality of life of the population (at the Federal and regional levels); clarification of the methodology for calculating indices for assessing the well-being and quality of life based on consultations with experts. 5. Operationalization on this basis of the missing variables in the form of specific questions for inclusion in the programs of sample surveys on socio-demographic problems, providing calculation of the missing variables. 6. Substantiation of the most reasonable and optimal allocation of the questions from the list to specific programs of sample surveys on socio-demographic problems, taking into account their topics, frequency, sample sizes and content. 7. Methodological recommendations for improving the empirical base that supports the introduction of indices into statistical practices to enable a comprehensive assessment of well-being and quality of life for the population as a whole and for individual groups of the population, based on international best practices 8. Calculation of indices to assess the well-being and quality of life of the population based on available information sources for Russia as a whole and for the constituent entities of the Russian Federation. 9. Results of a comparative analysis of indices calculated for Russia with similar indices for other countries (presented on the websites of international organizations). OECD, OECD STAT, Income Distribution Database - Poverty: https://stats.oecd.org/index.aspx?queryid=66598 С.1.11 Improving the empirical basis Output: that ensures implementation 1. Systematization of methodological materials for developing SDG indicators in the into statistical practices SDG field of inequality and poverty, social protection, undernourishment, healthy lifestyle and living conditions was accomplished. Page 101 of 159 The World Bank Second State Statistical System Development Project (P092429) indicators for the whole 2. The calculation procedures were suggested (adapted to the existing data population and its various sources). groups on the basis of UN 3. Test calculations of the SDG indicators were conducted (on the Federal and standards regional levels). 4. Additional variables needed for better coverage of all SDG indicators were proposed. 5. Operationalization of missing variables in the form of specific questions to be introduced in the programs of the social and demographic sample surveys was accomplished 6. Recommendations were given for obtaining estimates of informal sector to be used in SDG indicators calculations. 7. Substantiation of the optimal allocation of the questions from the list to specific programs of sample surveys on socio-demographic problems (taking into account their topics, frequency, sample sizes and content) 8. An analysis of statistical data on SDG indicators calculated on the basis of available empirical data sets was carried out (in comparison with other countries and with data for Russia) 9. An open access database for SDG indicators calculated on the basis of existing data sets was compiled (at the national and regional levels). 10. An assessment is made of the comparative performance of various sample surveys (used to obtain primary data for the calculation of SDG indicators) based on an example of the application of the relevant technique by the Indian Statistical Service. С.1.14 Methodology for compiling Elaboration of organizational issues in respect to using administrative sources for statistics on social benefits, producing statistical data on social payments to the population. both monetary and in-kind, using administrative data available at the Federal and regional levels Page 102 of 159 The World Bank Second State Statistical System Development Project (P092429) С.1.15 Development of methods for Development of methodology in respect to using alternative sources of information, generating poverty administrative data in first turn, to improve the statistical methods of assessment of indicators based on population incomes and poverty levels. statistical coordination of sample survey data on population incomes and participation in social programs and administrative data on the social benefits in Russia С.1.16 Developing methodological Approaches to integrate household income, expenses and savings microdata with recommendations for SNA statistics were developed. estimating household income, expenditure and savings by various income groups and coordinating the results with the 2008 SNA estimates С.2.1.a Methodological Output: recommendations for - a sample was created for conducting from Q4 2009 monthly labor force household sample design to surveys. conduct monthly labor force - Arrays containing expanded quarterly subsamples of primary sampling units surveys in Q4 2009 and in that should be divided within the constituent entities of the Russian Federation 2010-2011 (separately for urban and rural population) into monthly subsamples and among the territories and administrative units were sent to the territorial bodies of Rosstat; - “Methodological guidelines for conducting sample labor force surveys” (section 2 “Sampling of households”) were approved by the order of Rosstat of 21 December 2010 No. 452 - The results of the labor force survey for 2009 were published and since September 2009 contained monthly statistical data on the economically active Page 103 of 159 The World Bank Second State Statistical System Development Project (P092429) and economically inactive population aged 15-72 years (Занятость и безработица (gks.ru). С.2.1.b Methodological Output: recommendations for - recommendations were developed for improving methods for calculating improving calculation of individual weights to obtain statistical estimates for different periods on the basis individual weights to obtain of monthly labor force survey data for the Russian Federation and its constituents; statistical estimates of - the procedure was substantiated for calculating individual weights in case of a monthly labor force survey in lack of respondents in certain age groups. The recommendations were included in the Russian Federation and the economic description "Monthly labor force survey" (second stage - calculating the regions individual weights and obtaining derived indicators); - Methodological guidelines for conducting sample labor force surveys” (section 6 “Processing survey data”, 6.12 “Estimation of statistical characteristics and its accuracy”) were approved by the order of Rosstat of 21 December 2010 No. 452 - The results of the labor force survey for 2010 were published that contained monthly statistical data on the economically active and economically inactive population aged 15-72 years with indications of the accuracy of estimates. (Занятость и безработица (gks.ru) С.2.1.c Enhancement of labor force Output: survey program. Conducting - a pilot questionnaire "Questionnaire for a pilot survey of the quality of pilot surveys of employment employment and integration processes in the labor market" was developed in quality and integration addition to the Labor Force Survey Questionnaire; it contains questions in three processes in the labor thematic modules: voluntary work, participation in continuing education, income market from current employment; - a pilot sample survey of the quality of employment and integration processes in the labor market was conducted in the Republic of Tatarstan, Krasnodar Krai, Chelyabinsk oblast’ and the city of Moscow - data were processed and informational materials on the survey results were prepared. Page 104 of 159 The World Bank Second State Statistical System Development Project (P092429) Based on the results of the pilot survey, since 2016 additional questions on volunteer work have been included in the labor force sample survey program (Rosstat Order No. 467 dated October 9, 2015). Questions about continuing education were included in a separate modular survey (Order of Rosstat dated March 6, 2015 No. 91). С.3.1 Methodological Currently, the International Statistical Classification of Diseases and Related Health Introduction of modern recommendations on Problems, Tenth Revision (ICD-10) is in force on the territory of the Russian international standards. improvement of mortality Federation. By 2025, a complete transition of reporting in healthcare to the ICD-11 in statistics based on the WHO the territory of the Russian Federation is planned to ensure the reliability of statistical recommendations data in healthcare, methodological consistency, and international comparability of statistics on morbidity and mortality, to achieve healthcare development goals in the Russian Federation. As part of the work, a review was made of the problems that arose during the transition from ICD-9 to ICD-10. The experience of implementing the ICD-10 on the territory of the Russian Federation in 1990s showed that this process requires an integrated approach in terms of updating regulatory documents at various levels, ensuring consistent technical terminology with the account for the opinions of the leading scientific institutions of the Russian Federation and of the countries using the Russian language version. It is necessary to provide a legislative support to the introduction of the classification and organize training of healthcare specialists of the Russian Federation. The preparation of methodological guidelines for the transition from ICD-10 to ICD-11 has begun. The transition should be carried out in coordination with the Ministry of Health. С.3.2 Construction of improved Traditional methods of constructing mortality tables significantly underestimate Introduction of advanced mortality tables mortality at older ages, and, accordingly, significantly overestimate life expectancy for international standards. pensioners. Output: Page 105 of 159 The World Bank Second State Statistical System Development Project (P092429) - a new method and software have been developed for more accurate and correct calculation of mortality tables. The developed methodology and software make it possible to carry out calculations at the level of municipalities. - it will allow us to calculate full and short mortality tables in the most optimal way without using excessively cumbersome formulas, but without losing the accuracy of life expectancy estimates. The developed methodology will be included in the demographic forecasting model on the Digital Analytical Platform “Population” that is being developed at present (DAP “Population”) С.3.3 Ensuring comparability Recording of births and deaths has been improved to ensure comparability between Increase in data accuracy. between records of vital current records of vital events and population census data for the calculation of events and population demographic indicators. census data for calculating According to international recommendations, data in population censuses are demographic indicators recorded in accordance with respondents’ statements in places of their usual residence, which may not coincide with the address of their permanent legal registration. At the same time, the final data of population censuses are further used as a basis for calculating the population size and sex and age composition of the population by the cohort component method on the basis of administrative information on births, deaths, arrivals and departures. Thus, at present, survey data referred to the usual residence of persons are directly combined with data obtained from administrative sources, which mostly refer to the legal residence population. To eliminate this contradiction, a system of dual record of the population has been introduced: records of legal residence (according to administrative data) and records of usual residence (recorded during full count censuses by survey method).Hence both vital/migration events and persons should get additional characteristics, as they may be counted an least twice, on the one hand, as part of the legal population at the place of official registration, and, on the other hand, according to place of actual residence, information about which, under the current legislation on population statistics, can only be obtained in the course of a full-count population census. Page 106 of 159 The World Bank Second State Statistical System Development Project (P092429) Thus, we are talking about the creation of two parallel systems for accounting for the population and vital/migration events. This will be done within (DAP “Population”). С.3.4 Short-term demographic It has been proved that the existing population forecasting models of a complex forecasting in view of age- structure (Lee-Carter or Bongaarts) cannot be recommended for multi-regional specific changes in fertility demographic forecasting in Russia due to very high, by the standards of the Russian and mortality regions, demands for primary data and their possible inadequacy for the mortality trends in older ages. In this regard, a model has been developed that is a modification of the extrapolation method and the target modelling method, the advantage of which is sustainability to the irregularity of the data series used for estimating model parameters, as well as the ability to fully reflect the future changes in mortality at older ages. The developed model for population forecasting by age and sex was tested on real data on fertility, mortality and migration of the population and allows you to get more accurate results. The developed methodology will be incorporated into the demographic forecasting model in the framework of DAP “Population”. Component D. Human resource development D.2.1.b Training material "Manual Methodological material was prepared, which was used in the training of Rosstat "2008 System of National employees: Accounts " (for the system of - in 2012, a training course was held in the framework of the continuing state statistics)"on the basis professional education program: "Fundamentals of the System of National Accounts of the translation of the 2008 (2008 SNA)", state contract No. 97-PK-2012 / MESI-3; official UN publication - in 2013, training courses were held to improve the skills of federal civil servants of Rosstat under educational programs: “Methodological approaches to identifying institutional units in the Statistical Register by sectors of the economy according to the System of National Accounts 2008”, state contract No. 77-PK-2013 / MESI -4; Page 107 of 159 The World Bank Second State Statistical System Development Project (P092429) - "Fundamentals of the 2008 System of National Accounts (2008 SNA)”, state contract No. 77-PK-2013 / MESI-4; An Action Plan for the implementation of the OECD recommendations on the development of the system of national accounts of the Russian Federation was prepared, approved by the joint order of the Ministry of Economic Development of Russia, the Ministry of Finance of Russia and Rosstat dated 03.10.2014 No. 602/634/112n and agreed with the ministries and departments participating in its implementation. Large-scale work has been carried out to introduce the main methodological provisions of the 2008 SNA into Russian statistical practices • the capital assets boundary and gross fixed formation were extended to include output of research and development (R&D), as well as expenditure on weapons systems. In 2015, for the first time, annual and quarterly data on GDP and its components revised due to inclusion of R&D output were published on the official website of Rosstat, starting from the data for 2011. • the own-account production of housing services by owner occupiers was estimated. Methodological approaches for estimating own-account production of housing services by owner occupiers within the system of national accounts were approved by Rosstat on May 29, 2014, the official statistical methodology for estimating value of housing services produced by owner occupiers by the constituent entities of the Russian Federation was approved by order of Rosstat No. 138 dated March 12, 2021, • consumption of fixed capital was evaluated using current market asset value; • starting from December 2015, GDP estimates are published on the basis of the following methodological provisions - foreign economic transactions are accounted for in accordance with the methodology of the 6th edition of the IMF's Balance of Payments and International Investment Position Manual, - the assessment of hidden wages of employees and mixed incomes was refined based on the results of the development of the household financial account undertaken by the Bank of Russia, Page 108 of 159 The World Bank Second State Statistical System Development Project (P092429) - the work performed by domestic workers was evaluated, • a transition to annual compilation of Supply and Use tables was done, • annual and quarterly estimates of GDP and its components were updated starting from the data for 2011 on the basis of constructing basic Input-Output Tables for 2011, and Supply and Use Tables for goods and services for 2012 and 2013 • an assets and liabilities balance sheet was compiled and published on the official website of Rosstat D.2.1.c Development of training In 2018, training and methodological materials and a software package for distance materials and applicable learning on the topic " Sample design for statistical surveys " were developed. Since software "Sample design for 2019, they have been used to train specialists from the central office of Rosstat. In statistical surveys" the course of training, specialists from the Rosstat departments acquire the necessary knowledge of the sample statistical observation and develop skills of constructing samples and obtaining estimators with their accuracy characteristics. 20 Rosstat specialists were trained. D.2.1.d Development of training Training materials and software package have been developed and tested by materials and software specialists from the central office of Rosstat. Based on the results, an acceptance applications on statistical protocol was signed in December 2021. Follow up trainings will start in 2022. data editing Page 109 of 159 The World Bank Second State Statistical System Development Project (P092429) ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION A. TASK TEAM MEMBERS Name Role Preparation Supervision/ICR Buyant Erdene Khaltarkhuu Task Team Leader(s) Alexander Balakov Procurement Specialist(s) Galina S. Kuznetsova Financial Management Specialist Seda Pahlavooni Team Member Ronald N. Hoffer Social Specialist Olga A. Gubareva Procurement Team Gennady Pilch Counsel Olga Shabalina Team Member Mohammad Ilyas Butt Procurement Team Rahmoune Essalhi Procurement Team Arcadii Capcelea Environmental Specialist Igor Kecman Team Member Joseph Huntington La Cascia Team Member Page 110 of 159 The World Bank Second State Statistical System Development Project (P092429) B. STAFF TIME AND COST Staff Time and Cost Stage of Project Cycle No. of staff weeks US$ (including travel and consultant costs) Preparation FY06 12.988 80,309.27 FY07 23.650 98,023.04 Total 36.64 178,332.31 Supervision/ICR FY08 26.700 105,955.27 FY09 24.613 119,573.17 FY10 22.497 116,048.60 FY11 24.561 126,401.94 FY12 17.782 111,109.21 FY13 17.857 106,742.11 FY14 17.930 106,196.86 FY15 13.350 82,183.35 FY16 12.884 75,602.06 FY17 16.308 95,861.40 FY18 19.760 132,116.73 FY19 18.448 148,222.73 FY20 20.176 115,592.00 Total 252.87 1,441,605.43 Page 111 of 159 The World Bank Second State Statistical System Development Project (P092429) ANNEX 3. PROJECT COST BY COMPONENT Amount at Approval Actual at Project Actual Disbursement Components (US$M) Closing (US$M) (US$M) Component A. Modernization of economic statistics 3.35 6.99 6.51 methodology Component B. Development of the modern design and technology for statistical data 34.42 101.13 103.85 collection, processing and dissemination Component C. Enhancement 2.09 5.46 4.60 of social statistics Component D. Human 6.46 8.38 6.78 resource development Component E. Project 3.68 9.04 7.88 management Total 50.00 131.00 129.62 Page 112 of 159 The World Bank Second State Statistical System Development Project (P092429) ANNEX 4. ECONOMIC ANALYSIS Not applicable. Page 113 of 159 The World Bank Second State Statistical System Development Project (P092429) ANNEX 5. MONITORING AND EVALUATION FRAMEWORK General information 1. In accordance with the Project Appraisal Document and the Project Operations Manual (POM), an annual comprehensive regular assessment (monitoring) of the project implementation was set up, which was carried out by Rosstat and the Interagency Coordination Council (high governing body of the project) with technical assistance from the BEA Foundation. Information on the project monitoring results was provided on a regular annual basis to the Russian MOF and MOED, and the IBRD. The results of the assessment were used to analyze the intermediate results of the project, as well as to adjust its activities to reflect new challenges and ongoing changes in the statistical system. The Accounts Chamber of the RF – national controlling authority - considered the Monitoring and Evaluation (M&E) Framework as part of its annual reviews. 2. As regards STASYS 2 project M&E Framework, three project development objectives were determined and remained unchanged by essence but were slightly adjusted for the purpose of better measurability and to better accommodate the project impact. 25 intermediate outcome indicators were agreed at the beginning of the project and introduced during its implementation. Initially, PDO and intermediate outcome indicators were stated in Appendix No. 3 to the PAD; the IBRD Amendment Letter dated November 30, 2007, No. 2 to the package of documents prepared for signing of the Loan Agreement; and the POM for the STASYS 2 project. 3. Later on, during the project restructurings and Bank missions to provide support to the implementation of the project in the period 2008 - 2018, the project output indicators were refined and supplemented in a view of (i) improved assessment methods for the indicators (mainly compatible with previous ones for continuous comparison), (ii) new agreed activities, as well as due to the extensions of the project implementation period. Before project effectiveness it appeared impossible to introduce staff satisfaction survey (1-2 intermediate indicator) in Rosstat due to (i) absence of any respective provisions in the set of Rules for Civil Service institutions and (ii) impossibility to conduct such a survey even on a pilot basis in a single executive body. Thus, this indicator was dropped before actual project implementation. Another intermediate outcome indicator “Availability of meta data for all statistical data published” (2-5 intermediate indicator) was dropped too. By its internal order, Rosstat made it mandatory to publish only the statistical information with meta data available right before the actual project start up. 4. The project results framework uses quantitative (22) and qualitative (six) indicators. Quantitative indicators were designed to measure resources, costs and statistical output (description of internal and external funding, labor resources and available equipment, publications, Internet, etc.). Qualitative indicators reflected the presence or absence of the observed/reviewed phenomenon/processes. Qualitative indicators were intended to get an idea of the impact of various factors that affect the development of NSSS and statistical processes, and to assess the produced statistics in terms of their compliance with users’ needs. 5. Baseline was established for all 28 indicators. The target/final value was identified not for all indicators: for several of them the target value was defined as planned value to be met on an annual basis. Project Development Indicators (PDO) Page 114 of 159 The World Bank Second State Statistical System Development Project (P092429) 6. As of December 2021, three indicators were used as PDOs (they are all quantitative). The names of these indicators, their baseline, planned and actual values are given in Table 1. Table 1. Project Development Objectives Indicators PDO Indicators Baseline / Plan / Actual PDO-1 Index of users’ trust in official statistics 1,02 / 1,04 / 1,03 PDO-2 Data quality assessment index for SNA statistics as per DQAF for SNA statistics as detailed 1 / 1,1 / 1,065 in the IMF 2012 Guidelines, gain PDO-3 Share of statistical information provided by 97,8 / 100 / 100 Rosstat to the users per established time schedule 7. PDO-1 Index of user’s trust in official statistics. This indicator allows assessing the level and changes in the users’ trust in official statistics in general and by various categories of users. The indicator is constructed as the ratio of the overall trust index for the reference period to its value in the base period Qbp: Qrp / Qbp, where 2007 is chosen as the base period, and the value of the trust index for this year is taken for 1, with which further comparisons are made. There were no regular/systemic user satisfaction or user trust surveys introduced in 2007. The first two rudimentary surveys/pilots took place under first STASYS project. 8. To assess the overall trust index for the reference period (Qrp), an independent organization, using a specially designed questionnaire of 19 questions, regularly interviewed four target groups of respondents: representatives of authorities at various levels (federal executive authorities, executive authorities of the constituent entities of the Russian Federation, municipal authorities); business; research organizations and the media. The survey was conducted on a representative sample distributed over seven RF regions. 9. Based on the initial information collected through the survey, the index of users’ trust in official statistics was calculated. Calculations of the trust index were carried out for the whole sample of users and for various user groups. This index for a certain group of users was calculated as a simple average of the individual indices of the respondents included in the respective group. At the first step, the indices for individual respondents were calculated, and at the second step, they were averaged for the group. 10. The individual trust index was calculated on the basis of individual points Q4, Q11, Q12 according to the following formula Q = 0,25Q4 + 0,5 Q11 + 0,25Q12. where: Q4 – an average assessment of statistical information quality, Q11 – an average assessment of the trust in the key statistical data published by Rosstat, Q12 – average score of the overall assessment of the statistical organization. For each respondent individual scores were calculated as shown below: Q4 is an average score for answers to the Question 4 of the special questionnaire 6 where: Q 4 = Q i =1 4i /6), and Page 115 of 159 The World Bank Second State Statistical System Development Project (P092429) Q41 – points for the content of statistical data; Q42 – points for the reliability of statistical data; Q43 – points for the level of detail of statistical data; Q44 – points for the timeliness of statistical data; Q45 – points for the frequency of publication; Q46 – points for the time series consistency. Q11 is an average score for the trust in the most important statistical indicators published by Rosstat 5 (average score for answers to Question 11 is defined as: Q11 = Q i =1 11i /5), where: Q111 – points for trust in the published data on the population size; Q112 – points for trust in the published data on household money income; Q113 – points for trust in the published data on unemployment rate; Q114 – points for trust in the published data on GDP growth. Q115 – points for trust in the published data on consumer price index. Q12 average score of the overall assessment of the statistical organization (average score for the answers to Question 121): 4 Q12 = Q i =1 12i /4), WHERE: Q121 – points for the assessment of the statistical organization in terms of the quality of produced statistics (integral), Q122 – points for the assessment of the statistical organization in terms of reflecting most important public phenomena; Q123 – points for the assessment of the statistical organization in terms of the impartiality of the produced data; Q124 – points for the assessment of the statistical organization in terms of the regularity of rendering services to users. Each indicator in the questionnaire was evaluated in the range from 1 point (the lowest score) to 10 points (the highest score). Therefore, taking into account the normalization, the scores and their components ranged from 1 to 10. Along with the trust index, as a similar indicator for evaluating the Project development objectives, the index of user satisfaction with statistical information was used, which was constructed on the basis of a wider range of indicators as a simple average of eight components Q4, Q5, Q7, Q9, Q10, Q11, Q12, Q13 Q = (Q4+Q5+Q7 +Q9+Q10+Q11+Q12 +Q13)/ 8. Q5 –is the average assessment of the statistical data presentation quality (average score for the answers to question 5); Q7 – average assessment of the information placed on the statistical organization’s website (average score of the answers to Question 7), Page 116 of 159 The World Bank Second State Statistical System Development Project (P092429) Q9 – average assessment of servicing clients that receive information by e-mail and telephone (average score of the answers to question 9); Q10 – average assessment of the staff of the statistical organization (average score of the answers to question 10). 11. The set of indicators and corresponding questionnaire questions used to determine the users’ trust and satisfaction indices was determined at the initial stage of the project together with Rosstat specialists and agreed with all interested organizations. Later (in subsequent surveys), it did not change, so that it would be possible to compare the calculated index numbers at different points in time. 12. At the initial stage of the project until 2011, the trust index was measured every year, and then it was considered appropriate to measure once every two years. The measurement of the trust index was done with the involvement of an independent organization specializing in conducting population surveys. Individual points were obtained through interviewing various categories of users, and the integral score was obtained as a summary of individual points. More than 600 respondents took part in each of the relevant surveys during the project implementation period. 13. The planned and actual numbers for PDO-1 Index of users’ trust in official statistics and respective scores used for calculations are given in Table 2 below. Table 2. Indices of user trust and satisfaction in official statistical information (2007 = 1) (baseline) 2009 2007 2008 2010 2011 2013 2015 2017 2020 Index of users’ trust in official statistics (growth rate) − planned 1,02 1,025 1,025 1,03 1,04 1,04 1,04 1,04 − actual 1,0 1,02 1,029 1,025 1,03 1,035 1,039 1,034 1,03 Score of users’ trust (fact) 6,98 7,12 7,18 7,15 7,19 7,22 7,25 7,22 7,19 14. All planned targets for PDO-1 were actually achieved in the period from 2008 to 2015, which in general reflected an increase in the level of users’ trust in official statistical information in this time period, including its end-target value in 2015. In 2017 and 2020, there was a slight lag behind the planned numbers and a downward trend. The integral assessment of the user’ trust in official statistical information in 2020 was 7.19 points, or 72 percent of its maximum possible value of 10 points. It increased by 3 percent compared to 2008. The rather high level of the most recent assessment of the level of trust and its positive dynamics indicates in general an increase in trust in statistical information over the years of the project implementation, which should be recognized as one of the most important objectives of the project. 15. The increase in the users’ trust (Table 3) was mainly provided by such components as the quality of statistical information (7.6 points) and the overall assessment of the statistical organization (7.3 points) , while the user’s trust in the most important statistical indicators was noticeably lower (6.9 points) . Page 117 of 159 The World Bank Second State Statistical System Development Project (P092429) Table 3. Scores for users’ satisfaction in official statistics and its components, 2020 Indicator Score Overall average users’ trust index 7.2 Average score for the assessment for the quality of statistical data (4) 7.6 Average score for the assessment of trust in the most important statistical indicators 6.9 published by Rosstat (11) Average score for the overall assessment of the statistical organization (12) 7.3 Overall average user satisfaction index 7.3 Average score for the assessment of the quality of statistical data presentation (5) 7.2 Average score for the assessment of the information placed on the statistical 6.2 organization’s website (7) Average score for the assessment of servicing clients that receive information 7.8 by e-mail and telephone (9) Average score for the assessment of the staff of the statistical organization (10) 8.3 Average score for the assessment of trust in the most important statistical indicators 6.9 published by Rosstat (11) Average score for the overall assessment of the statistical organization (12) 7,3 Average score for assessing Rosstat as a whole (14) 7,1 The average index of user’s satisfaction is slightly higher that the user’s trust index, which is explained by higher scores of the components of the former. 16. It is important to note that while the overall assessment of trust in the most important statistical indicators published by Rosstat is 6.9, for some indicators the values are noticeably lower. For such indicators as inflation rate, money income, and unemployment rate the respective scores are 6.1, while the highest scores are found for the population size and GDP growth rate – 7.4 and 8.3, respectively. 17. The users’ trust index differs by target groups of respondents (Table 4). The highest scores are found for the groups of federal and regional authorities, and the lowest – for the groups of business and research communities. Page 118 of 159 The World Bank Second State Statistical System Development Project (P092429) Table 4. Scores of users’ trust index and its components for various groups of users of official statistics, 2020. Municipal autho- Regional autho- Federal autho- Mass Media institutes Research Business rities rities rities Total average users’ trust index 7.6 7.2 7.1 6.8 7.0 7.1 Average score for the assessment for the 8.0 7.6 7.3 7.3 7.5 7.5 quality of statistical data (4) Average score for the assessment of trust in the most important statistical indicators 7.3 7.1 7.0 6.5 6.6 6.8 published by Rosstat (11) Average score for the overall assessment of the statistical organization (12) 7.8 7.3 7.3 7.0 7.3 7.2 18. Thus, based on the collecting and analyzing the opinions of various categories of users, evidence- based assessments of users’ trust and satisfaction in official statistical data were obtained, and data quality, reliability and accessibility were assessed. In these surveys, a wide variety of information was collected, which was later used as a tool for developing a set of measures aimed at improving the efficiency of the Federal State Statistical Service, enhancing a wide range of statistical domains, expanding the possibilities for presenting statistical information to users, ensuring sustainable feedback from them and, in general, improving the reputation of Rosstat among general public and organizations. 19. PDO-2 Data quality assessment (index) for National Accounts Statistics (as per DQAF for National Accounts Statistics as detailed in the IMF 2012 Guidelines), gain/increase . This indicator measures the development level of the statistical capacity of the national statistical system. The higher this assessment, the more developed the statistical system is. To measure the level of national statistical capacity, the Data Quality Assessment Framework (DQAF) questionnaire is used in the 2012 version of the IMF manual. This data quality assessment system was developed by the International Monetary Fund (IMF). 20. The project outcome indicator Data quality assessment (index) for SNA statistics; growth is calculated in two steps: first, the integral data quality scores (in points) for the current year and for the base year are calculated, and then their growth rate is obtained as the ratio of the score values. Integral quality scores for the current and base years are calculated based on the criteria stated in the DAQF questionnaire. They can take values from 0 points (minimum value) to 4 points (maximum value). 21. The following criteria are used in DQAF questionnaire for quality assessment: − prerequisites of quality; − assurances of integrity; − methodological soundness (application of international/regional standards); − accuracy and reliability of official statistics; Page 119 of 159 The World Bank Second State Statistical System Development Project (P092429) − serviceability of official statistics to users; − accessibility of official statistics. 22. The integral score is calculated as the arithmetic mean of the partial scores corresponding to each criterion under consideration. Each partial score is also evaluated on a scale from 0 to 4 points. The change (growth) of integral estimates, as an outcome indicator, characterizes the general direction of changes occurring in the statistical system, and partial scores characterize how certain aspects change (methodological soundness, accuracy and reliability, serviceability, etc.) and reflect the respective changes in the institutional environment. 23. According to the adopted procedures, this indicator was measured within the framework of the project once every 2 years. An independent organization was involved in its evaluation. Partial scores were obtained on the basis of reviewing current documents of Rosstat and conducting interviews with its leading experts, and the integral score was obtained and as a combination of the partial scores. The indicator has been used in the project since 2013. Originally it was measured as the “Average score for Paris21 statistical capacity indicators based on DQAF”, and since 2018 has been changed to “Data quality assessment (index) for SNA Statistics (as per DQAF for national accounts statistics in the IMF 2012 Guidelines). There are only minor technical differences between these two versions of the estimate, so comparisons can be made between indicators for different years throughout the life of STASYS 2. 24. Planned and actual figures for PDO-2 “Data quality assessment (index) for National Accounts Statistics (as per DQAF for National Accounts Statistics as detailed in the IMF 2012 Guidelines), gain/increase” and respective scores used for the calculations are given in Table 5. Table 5. Data quality index for SNA statistics 2010 2013 2015 2017 2019-2020 Data quality index for SNA statistics, gain Plan 1 1.0 1.1 1.1 1.1 Actual 1.065 1.05 1.06 1.065 Score for data quality (actual) 3.1 3.12 3.26 3.28 3.30 25. All planned values of the PDO-2 indicator were achieved during the implementation of the project, which together with progression shown in score for data quality is evidence of positive changes that took place in the NSSS from 2010 to 2020, including those due to the significant contribution of various project activities. 26. The integral score for the development of the statistical capacity of Russia in 2019 was 3.3 points, or 82.5 percent of its maximum possible value of 4 points. It increased by 6.5 percent compared to 2010. The high level of the most recent assessment and its dynamics indicate that the national statistical system functioned effectively during the years of the project implementation and continues to develop on the basis of modern methodologies that reflect the best international statistical practice. Page 120 of 159 The World Bank Second State Statistical System Development Project (P092429) 27. While the integral index reflects the general development trend for the NSSS, the figures and dynamics of individual DQAF components are more informative. This detailed information is given in Table 6. Table 6. The integral index and its components, scores Indicators 2013 2015 2017 2019-2020 Data quality index for SNA statistics, 3.12 3.26 3.28 3.30 scores (actual) 0. Prerequisites of quality 3.1 3.1 3.1 3.1 1. Assurances of integrity 3.3 3.3 3.3 3.3 2. Methodological soundness 2.6 3.2 3.3 3.3 3. Accuracy and reliability 3.1 3.5 3.5 3.5 4. Serviceability 3.3 3.3 3.3 3.4 5. Accessibility 3.0 3.0 3.1 3.2 28. During the period of 2010 - 2013, there were no noticeable changes either in the integral index or in its components. The main changes in the various components of the statistical capacity began to occur in 2013-2015. The largest contribution was made by the work aimed at improving the methodology for compiling national accounts based on international standards (Section 2. Methodological soundness). The main efforts to develop Russian national accounts during this period were focused on the two areas. First of all, significant work has been accomplished to increase the compliance of Russian national accounts with international methodology. As a result, starting from the data for 2015, all components of Russian national accounts have been compiled in accordance with the 2008 SNA methodology. A retrospective calculation of the national accounts estimates for the period from 2011 to 2014 was also undertaken, which was important making various comparisons in the future. 29. The completeness of the accounts compiled was increased and previously not compiled accounts were included in the compilation process: financial account, revaluation account, other changes in the volume of assets account, balance of financial assets and liabilities. In addition, a methodological and information base was prepared for the transition to the regular compilation of annual assets and liabilities balance sheets, starting from the data for 2016, which made it possible to move on to compiling a complete set of SNA accounts in Russia on an annual basis. By 2015, the most important methodological task set in 2010 to close the gap in composition with the requirements of international standards was solved. The compilation process included previously unavailable accounts and increased the completeness of the accounts. 30. A significant contribution to the increase in the integral index of data quality was made by the work carried out in the following important areas: organizing data collection in electronic form; establishing an analytical system of the quality assessment and completeness and reliability of reporting data; improving quality of the Statistical Register which is the base of statistical observations; supporting the system of electronic services for the interagency interaction and public services of the SSS; other work related to ensuring high quality, accuracy and reliability of data (sections 0 and 3 of the respective assessments). Page 121 of 159 The World Bank Second State Statistical System Development Project (P092429) 31. Between 2016 and 2017, the increase in the average score of the statistical capacity assessment occurred mainly due to the improvement of the methodology for compiling national accounts (Section 2. Methodological soundness) and the accessibility of statistical data (Section 5. Accessibility). During this period, successful efforts continued to ensure transition of Russian national accounts to the 2008 SNA methodology. In 2017, Rosstat published annual and quarterly data on GDP and its components for 2011-2016 based on 2008 SNA concepts. This made it possible to close the gap in the time series of GDP data that has existed since 2011. In 2016, the annual assets and liabilities balance sheet was published for the first time using the 2008 SNA methodology, and later a series of balance sheets were prepared for the years 2011 - 2015. At the same time, Rosstat began developing methodologies for compiling production account and generation of income account taking into consideration the increase in the shares of enterprise groups and production globalization processes. In addition, the use of administrative data sources was expanded. 32. A certain contribution to the value of the integral index was made by the assessments of the outcomes in the following important areas: improvement of macroeconomic statistics, methods for constructing basic input-output tables, socio-demographic statistics, development of services for collecting data from respondents in electronic form; improving the quality of the frames for statistical observations, including those on the basis of Open Data and Statistical Register; development of United Inter-ministerial Information System (UIMIS), other work related to ensuring high quality, accuracy, reliability and accessibility of data. 33. In 2018-2019 the growth of the integral data quality index continued. It was achieved by improving the estimates of partial indices in the following sections: “Serviceability” (section 4) and “Accessibility of statistical data” (section 5). The increase in serviceability and accessibility of data was caused by such factors as better consulting work with users, the timeliness of data dissemination, the development of services for collecting data from respondents in electronic form; more extensive use of administrative data sources, improving the quality of the frames for statistical observations, developing Open Data, improving EMISS, carrying out other work related to ensuring high quality and efficiency of data dissemination, expanding the range of communication channels that facilitate the prompt communication by media representatives with Rosstat, conducting experimental works within the development of the digital analytical platform for the provision of statistical data. 34. As one can see from Table 7, the main positive changes in the activities of the Russian statistical products in the period from 2013 to 2019 were observed in the following areas: ensuring methodological soundness (growth by 26.9 %) through the adequate application of international and regional standards; increasing efficiency of data exchange (by 20 %), ensuring the reliability of data sources and intermediate and output statistic (by 20 %), ensuring the reliability of administrative data (by 13.9%) and the relevance of the source data (by 13.3 %), improving control over the response rate (by 11.4 %) and an increasing the efficiency of data dissemination (by 10 %). Table 7. Assessment of the state and development of the statistical capacity of the statistical system of Russia in 2013-2019. Time frame Indicators of statistical data quality Change 2013 2019 (%) Overall (average) score 3.12 3.30 +5.8% 0. Prerequisites of quality 3.1 3.1 0 Page 122 of 159 The World Bank Second State Statistical System Development Project (P092429) Time frame Indicators of statistical data quality Change 2013 2019 (%) Data collection and protection of data confidentiality 0.1 3.5 3.5 0 guaranteed by acting legislation 0.2 Efficiency of data exchange 2.0 2.4 +20.0% 0.3 Competence and relevance of human resources 3.0 3.0 0 0.4 Relevance of material and technical resources 3.0 3.0 0 0.5 Planning, control and assessment of activities 4.0 4.1 +2.5% 0.6 Focusing on quality issues 3.0 3.1 +3.3% 1. Assurances of integrity 3.3 3.3 0 1.1 Independence of the statistical process 3.0 3.0 0 1.2 Professional and ethical norms 3.5 3.5 0 2. Methodological soundness 2.6 3.3 +26.9% 2.1 Application of international/regional standards 2.6 3.3 +26.9% 3. Accuracy and reliability 3.1 3.5 +12.9% 3.1 Adequacy of initial data 3.0 3.4 13.3% 3.2 Control of the response rate 3.5 3.9 11.4% 3.3 Accuracy of administrative data 3.0 3.4 +13.9% Accuracy of data sources and intermediate and output 3.4 3.0 3.6 +20.0% statistical data 4. Serviceability 3.3 3.4 +3.0% 4.1 Rendering consultations to users 3.0 3.1 +3.3% 4.2 Timeliness of data dissemination 3.5 3.6 +2.9% 4.3 Periodicity of data dissemination 3.5 3.5 0 5. Accessibility 3.0 3.2 6.7% 5.1 Efficiency of data dissemination 3.0 3.3 10.0% 5.2 Updateability of metadata 3.0 3.1 3.3% Thus, as a result of the FTP, state program and project activities, the NSSS has reached a new level of development, becoming more and more user-oriented and significantly improving the scope and quality of the statistical information offered. 35. PDO-3 Share of statistical information provided by Rosstat to the users per established time schedule, in percent This indicator reflects the punctuality of Rosstat (compliance with taken obligations and internationally accepted release schedules) in disseminating statistical data to various users. This indicator Page 123 of 159 The World Bank Second State Statistical System Development Project (P092429) was replaced the originally proposed indicator "The share of statistical information provided by Rosstat to users with non-compliance of the established schedule”. 36. According to the Administrative Regulations of the Rosstat and its functions determined as “Providing official statistical information to the President of the Russian Federation, the Federal Assembly of the Russian Federation, the Government of the Russian Federation, other federal government bodies, government bodies of the constituent entities of the Russian Federation, local governments, courts, prosecutors, Bank of Russia, government non-budgetary funds, trade union associations and associations of employers, as well as international organizations” (approved by Rosstat Order No. 61 dated March 18, 2008). Annual basis, Rosstat’s work is organized in accordance with the Federal Plan of Statistical Work approved by order of the Government of the Russian Federation dated May 6, 2008 No. 671-p (as amended by order of the Government of the Russian Federation dated October 15, 2021 No. 2909-p) and disseminated by the central office of Rosstat and territorial bodies of Rosstat. The Head of Rosstat approves the Schedule for the preparation and placement of official statistical publications of Rosstat annually and publishes on the official site. 37. The base value of the PDO-3 was its value in 2007, that was 97.8%. The end-target value was set at 100 percent starting from 2008. This target was successfully achieved in 2008 and has been met in all subsequent years. Thus, from 2008 onward, Rosstat fulfilled all its obligations regarding the preparation of statistical information on time and in compliance with internationally accepted release schedule. This fact is important for assessing the activities of Rosstat, but along with the PDO-3 indicator, it would have been advisable to use more differentiated indicators in the course of project implementation. Using as a PDO indicator the one that was already achieved in 2008 and has not changed since then seems insufficiently informative to reflect the efforts undertaken by Rosstat to fulfill its obligations in various areas and impact of project interventions. Project Intermediate Outcome Indicators 38. 25 measures were used as indicators called Intermediate Outcome Indicators (some of these indicators were used only for certain periods of the project), of which 18 are quantitative indicators and 7 are qualitative indicators. The relevant indicators, their baseline, actual and target values are shown in Table 8. Table 8. Intermediate Outcome Indicators (*qualitative indicators) Intermediate Output indicators Baseline/ plan / actual* 1-1 The Statistical Council is established* no / yes / yes 1-2 Dynamics of Rosstat staff training (number of trained target value was not set; staff-persons: requested, actual, including STASYS-2) planned values set for each year 1-3 Dynamics of state budget expenditures for training target value was not set; purposes (million rubles) planned values set for each year 2.1. Total number of classifications used throughout the SSS that are compatible with the equivalent international 7 / 12 / 12 classifications Page 124 of 159 The World Bank Second State Statistical System Development Project (P092429) Intermediate Output indicators Baseline/ plan / actual* 2-2 Mastering of international standards (filling out the statistical questionnaires of international organizations 87/90/90 - replaced with harmonized indicators) 2-3 Application of internationally accepted statistical methodologies and standards (filling out questionnaires of international organizations listed in the Federal 70/77/79 Statistical Action Plan and official OECD questionnaires), % 2-4 Number of sample surveys in different economic and 54/ target value was not set; social fields that are conducted on a census basis planned values set for each year 2-5 Reduction in respondents (enterprise) burden* no/ yes / yes 3-1 Number of new surveys and/or modules of data 268/ target value was not set; collection 3-2 Number of MDG indicators produced by the NSSS 33 / target value was not set/34 - replaced partially complies/fully 3-3 Compliance with the ICLS household income and complies/ expenditure methodology* fully complies 3-4 Availability of statistical data on skilled and demand no/ yes / yes for skilled labor by professional groups* 4-1 Number of Rosstat website visits per year (thousand) 500/1000/3000 4-2 Availability of data collected through remote 0 / target value was not set / methods 65 - replaced 4-3 Modern computer equipment of Rosstat staff * 1-1/ target value was not set; planned values set for each year/1-1 4-4 Availability of microdata bases (HBS) and LFS in the no/ yes / yes open access for users 4-5 Number of statistical tasks resolved on a new technological basis and forwarded to the United 34 /313/313 - replaced Foundation of Algorithms and Programs 4-6 Number of systemic statistical tasks moved to the 1/45/146 centralized processing of primary statistical data Page 125 of 159 The World Bank Second State Statistical System Development Project (P092429) Intermediate Output indicators Baseline/ plan / actual* N-1 Number of methodological guidelines prepared for implementation for compiling satellite accounts in the 0/ 0 / 0 social sphere recommended by OECD N-2 Degree of implementation of 2008 SNA into Russian statistical practices (% of works fulfilled in accordance with the “Action Plan to implement the OECD 10/93/95.5 recommendations for developing the system of national accounts in the Russian Federation”) N-3 Share of official statistical reporting forms that could be submitted to state statistical organizations in 80/100/100 electronic form N-4 Number of TBS employees dealing with receiving and 100/40/35 processing primary statistical data N-5 Availability of national set of SDG indicators for the no / yes / yes Russian Federation* N-6 Number of SDG indicators compiled by official statistical bodies based on the international methodology 90/108/112 and introduced into the Federal Statistical Action Plan N-7 Number of statistical indicators based on electronic economic descriptions as percent of the total number of 34/60/60 statistical indicators compiled by Rosstat * –baseline value - the value of the indicator at the moment when it began to be used for assessment, plan and actual - the values of the indicator by the time of completion of the project; if the planned values of the indicator were not set or were set only for each year, and not by the time of project completion, then it is specially noted. 39. A number of the intermediate outcome indicators have been used from the start, others have been added or edited when the project was extended till 2021. Below is a description of the indicators used and the results of the analysis of the achievement of their target values. 40. Intermediate Outcome Indicator 1-1 The Statistical Council is established. At appraisal stage there were no specifications on the type of statistical council to be established. In this regard it would be useful to note that in 2010 the Scientific and Methodological Council was established according to the order of Rosstat dated March 3, 2010, No. 103 “On approval of the Provisions for the Scientific and Methodological Council of the Federal State Statistics Service”. Its purpose was to elaborate on development strategies for Russian statistics, to encourage the conduct of a wide range of research on relevant issues of the Russian economy, develop the experts’ opinion of Rosstat on general methodological problems, discuss and prepare recommendations for dealing with strategic development issues of Russian official statistics, provide expert Page 126 of 159 The World Bank Second State Statistical System Development Project (P092429) and advisory support for the developments of Rosstat and other institutions producing official statistics in the fields of statistical methodology and statistical education. 41. A year later a Public Statistical Council was established (https://rosstat.gov.ru/community/os). The Public Statistical Council characterizes the extent to which the current and strategic activities of Rosstat are carried out under the control of the expert community and civil society, and whether the opinions of various categories of users are taken into account when organizing the statistical process. Since the establishment of the Public Council, 37 meetings have been held, at which the most important issues of developing national statistics were discussed, including challenges and issues relevant for the successful implementation of the project. 42. The baseline value of this indicator was “no”, and the target value of “yes” (creation of Statistical Council was supposed to be reached in 2012 (the original date for completion of the project). However, the target has been reached in 2010 and 2011 respectively, when two Councils were established. 43. The establishment of both Councils at the initial stage of the 14-year project implementation period was important for its successful and more efficient implementation, since the meetings of the Councils discussed, among other issues related to the problems that arose during the implementation of STASYS 2. Also, it ensured greater openness of the statistical production process, its planning, implementation, and dissemination of statistical products. 44. Intermediate Outcome Indicator 1-2 Dynamics of Rosstat staff training (number of trained staff- persons: requested, actual, including STASYS-2). For the purpose of introduction of international standards, practical use of new methodological approaches and adequate operating of advanced IT tools and devices, the holistic approach requires to provide vast and targeted training activities. This indicator reflects the number of employees who have been trained and/or improved their skills in various theoretical and practical issues of statistics. The value of this indicator was presented in two ways: the total number of Rosstat employees who received training, and the number of Rosstat employees who received training funded by the project. The target value of the indicator was not set. The planned values of the indicator were determined separately for each year. 45. The main training activities for employees were carried out in the first years of the project. In 2008 it was planned to train 6.3 thousand people, in 2009 - 2.3 thousand people, and in the following five years, 2 thousand people per year. Thus, over the first half of the initially planned project implementation period, it was planned to train more than 20 thousand people, which was supposed to ensure a much higher qualification level of the statistical system personnel. Subsequently, when the project implementation period was extended, the overall training magnitude was significantly reduced, but the targeted training activities increased, as the process should have become more focused and goal oriented. In addition, the WB Reimbursable Advisory Service (RAS) project was prepared and launched in 2017. It was focused on skill and professional qualification development/upgrade of Rosstat and MOF personnel to a very large extent. 46. As one can see, all planned numbers were achieved and significantly exceeded. In 2009 and 2010 the actual figures were as planned, but during next years the targets were exceeded significantly, on average by 50%. In 2011 - 2,501 people were trained, in 2012 - 3,896 people, in 2013 - 3,496, in 2014 - 3,215, in 2015 - 1,853, in 2016 - 1,793, 2017 - 1,713, 2018 - 672, in 2019 - 882, in 2020 - 1,732, and in 2021 – 3,680 people were trained. Page 127 of 159 The World Bank Second State Statistical System Development Project (P092429) 47. From 2009 to 2012, the STASYS 2 funding supported training from 300 to more than 2,000 people on the programs related to the subject matter of the project components. Over two thousand people were trained under the project in 2010, the third year of its implementation. Such a big number of trainees was achieved through the introduction of distance learning courses. 48. During the entire period of the project implementation, more than 40 thousand people have been trained under all available sources (37 685 persons), including 4 918 out of STASYS 2 funds. The number of actually trained staff exceeded the planned targets during entire period of project implementation. Undoubtedly, such a significant number of employees who were trained in the timeframe of the project had a positive impact on the effectiveness of the implementation of various activities within the project. 49. Intermediate Outcome Indicator 1-3 Dynamics of the state budget expenditures for training purposes (in millions of RUB). This indicator reflects the amount of funds from the federal budget and the IBRD Loan proceeds, which were allocated for the training of Rosstat employees. The target value of this indicator was not set; its values were determined separately for each year. The idea behind was to monitor the process of moving the training activities from financing them out of STASYS 2 project resources to Rosstat own budget. It was important to ensure that the process of systemic upgrade of professional level of Rosstat employees is strongly embedded into its own budget formation and execution. 50. The main expenditures fell on the first half of the project implementation period. In 2008-2014, they amounted to over 14 million RUB a year. A significant share of expenditures in 2009-2012 was supported by project funds: over 7 million RUB in 2009, 13 million RUB in 2011, over 45 million RUB in 2010 and 35 million RUB in 2012. During the next years the training expenditures decreased (see para 45). Rosstat training budget was stable and exceeded the project allocation for training with few exceptions in 2010, 2012, 2018, 2020, and 2021. Significant budgetary funds allocated for the training of Rosstat employees together with STASYS 2 proceeds made it possible to improve their knowledge and skills and it provided an opportunity to solve the complex methodological and informational tasks of the project. 51. Intermediate Outcome Indicator 2-1 Total number of classifications used in the system of state statistics that are compatible with the equivalent international classifications. This indicator measures the application of the classifications in the system of state statistics that allow systematizing and structuring official statistical information in accordance with international standards for classifying socio-economic processes and phenomena. Harmonization of national classifications with international standards ensures the transparent presentation of statistical information to international organizations such as UN Statistical Commission, OECD, IMF, WB, and others in accordance with the requirements of international data reporting standards. 52. The use of classifications that comply with international standards in the system of state statistics also enables: (i) conducting statistical comparisons at the national and international levels; (ii) ensuring the compatibility of government information systems and information resources; (iii) ensuring interagency information exchange; (iv) ensuring definitive identification of objects of legal relations in legal acts in the socio-economic field; (v) creating conditions for developing a unified information space on the territory of the RF. 53. Total of 12 classifications were chosen as a target indicator by WB and Rosstat jointly based on the needs and probably of achieving this target. At the start of the project, Rosstat used 7 respective classifications. Rosstat has complied the list of classifications in consultation with other producers of official statistics while maximizing the coverage and impact from the implementation. Page 128 of 159 The World Bank Second State Statistical System Development Project (P092429) 54. The target value of 12 classifications was reached one year earlier than the planned date 2016. The target value for the intermediate indicator was maintained since then as a result of continued efforts to improve classifications. About half of all classifications used by Rosstat are now supporting the production of statistical data in accordance with international standards. There is a plan to extend and expand this list if the new international classification issued and requested from national statistical organizations. 55. Intermediate Outcome Indicator 2-2 Mastering of international standards (filling out the statistical questionnaires of international organizations with harmonized indicators). From 2008 to 2013, this intermediate indicator for application of international standards in Russian statistical system was used. The methodology “Methodological recommendations for improving the practice of filling out questionnaires of international organizations by Rosstat taking into account the development of international standards” was prepared in 2005 by Rosstat Research Institute of Statistics. 56. The baseline value of this indicator was 87% and it reflected the percentage of the completion of the international questionnaires at the start of the Project. The mid-term target value 90 % was achieved in 2010, three years before the planned date 2013. The target was set below 100%, primarily because not all indicators requested by international organizations have been developed in Rosstat. In addition, in certain statistical domains there was no proper correspondence between Russian and international methodologies. 57. In early 2009, the percentage of questionnaire completion was 89% already, and remained at 90 % through 2014 after reaching it in 2010. Later it was replaced with a new one - filling out questionnaires of international organizations listed in the Federal Plan for Statistical Work and official OECD questionnaires, starting from 2013. It was due to the preparations at that time for accession of the RF into the OECD membership and increased the number of requests from the international organizations for the provision of statistical information by Rosstat. In this regard, a new indicator 2-3 was added to the intermediate output indicators, and its assessment started from 2013. 58. Intermediate Outcome Indicator 2-3 Application of international statistical methodologies and standards (filling out questionnaires of international organizations listed in the Federal Plan for Statistical Work and official OECD questionnaires). This indicator reflects the degree of harmonization of the statistical methodology used in the Russian Federation with international standards, as it shows how complete the data provided by Rosstat to international organizations in accordance with international standards. The baseline value of the indicator in 2013 was set at 70%. The choice of this value was due to the fact that starting from 2013, the volume of requests for information received by Rosstat from the OECD sharply increased because of the communication between the Russian Federation and the OECD - about 70 questionnaires were recommended to be filled out. Rosstat preliminary analysis of the new OECD questionnaires showed that their completion rate in 2013 could be no higher than 70 %. The target value for this indicator was set at 77 %. 59. The intermediate indicator 2-3 is calculated by Rosstat in accordance with the Methodology for calculating the indicator "Application of international statistical methodologies and standards (filling out questionnaires of international organizations listed in the Federal Plan for Statistical Work and official OECD questionnaires.”, approved by order of Rosstat dated June 21, 2013. No. 229. 60. Between 2014 and 2018 the planned values of this indicator were achieved in full and amounted to: in 2014 - 72 %, in 2015 - 73 %, in 2016 - 75 %, in 2017 - 76 %, in 2018 - 77 %. By project closure, this indicator reached 79 % (during the extension of the project implementation period, the indicator was kept at the same Page 129 of 159 The World Bank Second State Statistical System Development Project (P092429) level in spite of the fact that the process of the accession of the Russian Federation to the OECD statistical standards was actually frozen). 61. Intermediate indicator 2-4 Number of sample surveys in different economic and social fields that are conducted on a census/full count basis. This indicator shows the number of sample surveys conducted by Rosstat in the respective year instead of census/full count surveys. The baseline value of the indicators was equal to 54. This was the actual number of sample surveys conducted by Rosstat in 2007. The target value of this indicator was not set; its planned values were determined for each year. 62. The actual values of this indicator were: in 2008 - 56 (plan - 56), in 2009 - 58 (plan - 58), in 2010 - 59 (plan - 58), in 2011 - 59 (plan – 60), in 2012 and 2013. - 61 (planned - 61), in 2014 - 60 (planned - 60), in 2015 - 58 (planned - 58), in 2016 - 59 (planned - 56), in 2017 - 55 (plan - 56), in 2018 - 57 (plan - 57), in 2019 - 58 (plan - 58), in 2020 - 44 (plan - 49) and in 2021 – 44 (plan - 42). 63. During the project implementation period, the planned values of indicators were achieved, except for years 2011, 2017 and 2020, which was caused by the cancellation of several planned Federal statistical surveys. In total, more than 700 sample surveys were conducted instead of full count surveys. It shows a sufficient improvement of the methodological and organizational procedures for conducting surveys in Rosstat from 2008 to 2021 and a significant reduction of time, costs, and response burden. 64. Intermediate Outcome Indicator 2-5 Reduction in respondents (enterprise) burden. This indicator characterizes changes in the response burden. It is obtained by expert estimates and shows the existence (absence) of the burden on respondents, judging by changes in the number of reporting forms (variables) they have to submit. This number includes those forms (variables) that were found irrelevant and excluded within the reference year. 65. According to the collected data, from 2008 to 2018, the burden on respondents (enterprises) was gradually decreased, the planned values for this intermediate indicator for all years of the project implementation were achieved. Every year, there was a reduction in the number of unnecessary and duplicative forms, optimization of variables, increase in share of statistical reports submitted in electronic format, and growth in number of statistical forms converted into machine-readable format. This provided opportunities to introduce new reporting forms with the purpose to study those social and economic phenomena that were not previously covered by existing surveys and statistics. 66. In addition to experts’ assessments three Tables A, B, and C below illustrate Rosstat efforts to reduce the reporting burden through optimization of statistical forms (source: Rosstat). They demonstrate the positive stable dynamic of respondent burden reduction from different angles: Table A. Total number of statistical forms at federal level approved per year (2022 includes a planned number) 2013 2014 2015 2016 2017 2018 2019 2020 2021 2022 253 254 243 233 223 222 225 217 210 237 Table B. Number of statistical forms converted into machine-readable format Page 130 of 159 The World Bank Second State Statistical System Development Project (P092429) 2013 2014 2015 2016 2017 2018 2019 2020 2021 2022 - - - 6 9 19 34 55 121 - Table C. Share of statistical reports submitted in electronic format (%) 2014 2015 2016 2017 2018 2019 2020 2021 2022 Big and medium 69 75 81 85 88 90 91 96 100 enterprises plan Small - - 44 50 54 62 75 86 enterprises Table D. Number of revised statistical forms with optimized set of indicators 2013 2014 2015 2016 2017 2018 2019 2020 2021 2022 248 241 243 134 141 109 172 155 187 67. Intermediate Outcome Indicator 3-1 Number of new surveys and/or modules in the existing surveys. This indicator reflects the extent to which new economic and social phenomena are presented in official statistics, taking into account the interests of various users. Therefore, to achieve this goal, new reporting forms (or new sections) are regularly introduced. Target and planned values for this indicator were not set. The assessment was based on the dynamic of respective improvements introduced into statistical forms. 68. During the project implementation period, from 3 to 16 forms or new sections were introduced each year, except for 2021 when 75 changes implemented following the revision of 2019 Federal Statistical Observation Act. In total, 159 respective changes were made to the form throughout the project cycle. Rosstat reduced the number of forms in use from 268 in 2008 to 210 in 2021. Therefore, by the end of the project, the respective changes were introduced in more than 75 percent of the forms, so one can say the statistical tools experienced significant changes and became more appropriate to the tasks of measuring economic and social changes. 69. Intermediate Outcome Indicator 3-2 Number of MDG indicators produced by the NSSS. The indicator is determined on the basis of the Millennium Development Goals (MDGs) formulated in the UN Millennium Declaration adopted by the UN General Assembly on September 08, 2000. In total, eight international development goals were formulated, which 189 countries agreed to achieve by 2015 (eradicate extreme poverty and hunger, reduce child mortality, combat HIV/AIDS, malaria and other diseases, etc.), and 58 indicators for monitoring the achievement of these goals were proposed by the 39th session of the Statistical Commission of the United Nations Economic and Social Council of February 26-29, 2008. 70. The 3-2 indicator reflects the availability of data in the RF to compile indicators from the official list set out in the UN document "Indicators for Monitoring progress in achieving the MDG from the Millennium Declaration” and reflected the level of socio-economic development in the country. Until 2009, the original version of this indicator was formulated as "The number of indicators of the Millennium Development Goals, compiled by the SSS". Page 131 of 159 The World Bank Second State Statistical System Development Project (P092429) 71. Initially, Rosstat together with ministries and agencies that collect data for monitoring the MDG in the Russian Federation decided that the number of indicators to be compiled will be equal to 33. The annual planned value for this indicator was set in 2009 (on the basis of actual number in 2008) and was equal to 34, the same value was set and reported for project duration during the period MDGs were relevant. 72. Starting from 2016, the intermediate indicator 3-2 has not been measured due to its replacement with the intermediate indicator N-6 Number of Sustainable Development Goals indicators compiled by official statistical bodies and based on the international methodology and introduced into the Federal Plan of Statistical Work (description of this indicators is given below). This was due to the fact that in September 2015 the United Nations General Assembly approved 17 Sustainable Development Goals (SDGs) for the period till 2030. 73. Intermediate indicator 3-3 Compliance with the methodology of the International Conference of Labor Statisticians (ICLS). This indicator reflects the results achieved (or not achieved) on the implementation of the recommendations of ICLS regarding the methodology of household income and expenditure survey in the system of official statistics. The indicator may take the following values: does not comply, partially complies, and fully complies; it is determined by expert opinion. The indication that the indicator 3-3 has reached the target value is the appropriateness of the statistical methodology and the statistical data produced on its basis for use at the international level. 74. Until 2012, the indicator was determined as “partially complies” with target value to be “fully complies”. Since the measurement of the level and compositions of household money income was based on the Household Income Survey (HIS) data, the survey program assumed collecting only indirect data on the level of income (based on spending and savings growth) and did not include variables on the composition of household income. 75. Since 2012, Rosstat has been implementing the HIS program in accordance with Decree of the GOR dated November 27, 2010, No. 946 “On launching in the Russian Federation a system of Federal socio- demographic statistical surveys and monitoring economic losses from mortality, morbidity and disability of the population”, and the Federal Plan of Statistical Work, which includes a study on the composition of household income. The rationale, including the description of the themes of the HIS, its goals and objectives, was agreed by Rosstat with the MOED, the Ministry of Labor of Russia, the MOF and approved by the GOR by adopting the Decree mentioned above. 76. The HIS program is being implemented by Rosstat taking into account the results obtained during the execution of the contract T2/2/C.1.8 “Improving statistics on household income, poverty and living standards, including modernizing the system of sample household surveys and improving administrative data” dated August 23, 2017 within the framework of Project Component C Enhancement of social statistics. Therefore, with the introduction on a regular basis of a sample survey of household income and participation in social programs (the program of this survey and the composition of the generated indicators meet the standards of the Resolution of the International Labor Organization on statistics of household income and expenditure, adopted in 2003 on the 17th International Conference of Labor Statisticians) the target value of the intermediate indicator “fully complies” was achieved. In the subsequent years this indicator was assigned the same value. 77. Intermediate indicator 3-4 Availability of statistical data on skilled and demand for skilled labor by professional groups. This indicator has only two values: “yes” – if data on labor demand by professional groups are available, and “no” – if data are missing. Page 132 of 159 The World Bank Second State Statistical System Development Project (P092429) 78. The indicator was measured on the basis of a sample survey of enterprises (without small businesses) every two years in October which allowed to generate statistical information on the number of workers and the labor needs of organizations by professional groups according to form No. 1-T (prof) “Information on the number of workers and on labor needs of organizations in workers by professional groups”. The actual value of the intermediate indicator was “yes” showing the necessary information on the needs of workers of various professional groups was regularly received and could be used for making managerial decisions. 79. Intermediate indicator 4-1 Number of Rosstat website visits per year (thousand). This indicator reflects the demand for accessible statistical data and the willingness of data users to receive Rosstat information services online. The baseline value of the indicator was set at 500 thousand visits per year, and one million visits per year was set as a target value for 2011 assuming 100% growth every year. 80. The target value was achieved in 2009. During several next years (2010-2015) this indicator remained at the level of 1.1 million visits per year, which exceeded the planned values. A sharp increase occurred in reaching to 7-8 million visits in 2017-2018. First, this was due to the fact that the website of Rosstat, according to the responses of the user satisfaction survey, became the main source of statistical information for users, far ahead of other available sources in popularity. Therefore, the rapid achievement of the target value of this indicator contributed to greater accessibility and openness of statistical data. 81. Intermediate indicator 4-2 Availability of data collected through remote methods. This indicator characterizes the level of implementation of modern technologies in statistical practice and leads to significant economy of time for data collection. There was no relevant practice at the time of the STASYS 2 project start. The target value of this indicator was not set, and its planned values were determined for each year. The indicator was used until 2016 and replaced with new indicator N-3 “Share of official statistical reporting forms that could be submitted to state statistical organizations in electronic form”. 82. The actual values of the indicator (always higher than the planned ones) occurred at a rather rapid pace from the start. It demonstrated the growth of about 10% between 2010-2012 and reached 21% in 2013. As a result, this number reached 61% in 2013, or more than half of the collected data, which pointed to significant progress in the automation and digitalization of statistical processes. By 2015, the indicator reached 65%, which ensured an increase in the efficiency of the collection of primary statistical data in subsequent years. 83. Intermediate indicator 4-3 Modern computer equipment of Rosstat staff. This indicator characterizes the level of equipping the personnel with modern computers and is measured by the number of computers per employee. 84. The baseline value was 1 to 1, that is one computer per one employee. The planned value was maintained. At all levels of the statistical system, there was a constant upgrading of hardware and software including that funded by the project. All statistical staff equipped with the ICT, has access to internet, and possessed modern tools and packages for the purpose of survey design, data collection, verification and analysis that speeds up all stages of data production cycle. 85. Intermediate indicator 4-4 Availability of microdata bases (HBS and LFS) in the open access. This qualitative indicator has two values: “yes” and “no” showing access to the primary data publicly available on the Rosstat website. For all years starting from 2008 and further, the “yes” value of this indicator was set. 86. In 2008, Rosstat created the database “Microdata of the Household Sample Surveys” which included the household budget survey databases for 2003–2004 and 2005–2007 and provided open access to all users Page 133 of 159 The World Bank Second State Statistical System Development Project (P092429) through the Rosstat website. In 2016, the household income survey database was created and made available to the public. In subsequent years, the access to income survey data for 2017, 2018, and 2019 was provided gradually. As a result, the planned values of the indicator were achieved in 2011 and maintained since. This led to a significant improvement of users’ capabilities to use official data for compiling and analyzing statistics. 87. Intermediate indicator 4-5 Number of statistical tasks resolved on a new technological basis and forwarded to the United Foundation of Algorithms and Programs (UFAP). This indicator reflects the level of automation of data processing within the statistical production process. Its baseline value was determined as 34 tasks, and the target value was set at 313 tasks, which means the total number of tasks that can be transferred to UFAP. In the future, this indicator did not increase, as it corresponded to the number of tasks in the annual plan of Rosstat, approved by the order of the Government of the Russian Federation dated May 06, 2008, No. 671-r. 88. The target value of the indicator (moving tasks to new technology) was achieved just in two years, by 2009, and has not changed. In 2015, the indicator was replaced by the indicator 4-6. 89. Indicator 4-6 Number of systemic statistical tasks moved to the centralized processing of primary statistical data. This indicator reflects the number of works with data processed in a centralized way (through the computing center of Rosstat). Centralized data processing improves the quality of the output statistical data and reduces the time of production, which generally has a positive effect on the efficiency of the statistical system. 90. The indicator has been used as an intermediate indicator since 2015 and its baseline value was 1. Between 2016 and 2018, the value of the indicator did not exceed 6 tasks which could not sufficiently affect the quality of the statistical process. A significant progress was made starting in 2019 and 2020, when the values reached 34 and 106 tasks respectively. In 2021, there were 146 statistical tasks have been transferred to centralized processing. 91. Intermediate indicator N-1 Number of methodological guidelines prepared for implementation for compiling satellite accounts in the social sphere recommended by OECD. This indicator reflects the level of progress in improving the methodology of economic and social statistics through the introduction of social satellite accounts. 92. The intermediate indicator N-1 was introduced into the project results framework in 2015 and its values for 2015-2019 were set as "0", and in 2020 it was supposed to start developing the respective methodology. However, these works were not started in 2020 either. The delay in developing these approaches generally makes it difficult to move towards a more active use of satellite accounts in the social sphere statistics, which impedes a more detailed study of relevant social processes and makes international comparisons more difficult. 93. Intermediate indicator N-2 Degree of implementation of 2008 SNA into Russian statistical practices (% of works fulfilled in accordance with the “Action Plan to implement the OECD recommendations for developing the system of national accounts in the Russian Federation). This indicator characterizes the degree of implementation of the modern international standard of 2008 SNA into Russian statistical practice or the degree of compliance of Russian macroeconomic statistics with the latest international standards. Page 134 of 159 The World Bank Second State Statistical System Development Project (P092429) The higher the value of this indicator is, the more adequate and comparable Russian macroeconomic indicators are with statistical data from other countries. 94. The value of this indicator is calculated by the Department of National Accounts of Rosstat according to the methodology approved by the orders of Rosstat dated July 21, 2016, No. 297 and June 21, 2013, No. 229 for calculating target indicators and indicators for the subprogram “Generation of official statistical information of the Government program of the Russian Federation” and “Economic development and innovative economy”. 95. According to this procedure, the indicator "The degree of implementation of the 2008 SNA standards into Russian statistical practice" includes: (1) the extent to which the methodological changes of 2008 SNA compared to the 93 SNA was introduced. It is estimated in percentage by experts based on an analysis of the actual implementation of the 2008 SNA standards; (2) introduction of the 2008 SNA accounts into the practice of compiling Russian national accounts, estimated in points; (3) non-introduction of the 2008 SNA accounts into the practice of compiling Russian national accounts, estimated in points; 4) partial introduction of the 2008 SNA accounts into the practice of compiling Russian national accounts, estimated in points. 96. This indicator has been measured since 2015. Its baseline value was set at 10 %, and the target value was set at 93 %. For all the years of the STASYS 2 implementation, the actual values exceeded the planned targets. In 2015, the actual value was - 63 % against the plan of 10 %; in 2016, the actual value was 70 % against the plan of 35 %; in 2017, the actual value was 75 % against the plan of 65 %; in 2018 – both values were 85%, in 2019 – both values were 90%, in 2020 – the actual was 95% against the plan of 92%, and the end target of 95.5 % was achieved in 2021 which show high level of implementation of the international SNA standards into the statistical practice in Russia. 97. Intermediate indicator N-3 Share of official statistical reporting forms that could be submitted to state statistical organizations electronically (as % of 2015). The intermediate indicator N-3 reflects the level of readiness of statistical bodies for data collection through using electronic statistical reporting forms and processing data on the basis of modern ICT technologies. The indicator reflects the share of official statistical reporting forms that could be filled out by different organizations and send to the state statistical offices in electronic form, rather than paper form. 98. The indicator used as an intermediate project outcome indicator since 2015, and its baseline value was set at 80 %. The target value of 100 % was successfully achieved in 2018. 99. Intermediate indicator N-4 Regional Statistical Office employees dealing with receiving and processing primary statistical data (as % of 2015). This indicator is measured as the number of employees of the regional offices who are involved in the processing of reports submitted to the territorial bodies in paper form. 100. The indicator has been used since 2015. The baseline value in 2015 was set at 100 percent, in subsequent years to gradually decrease it to 90% in 2016 and 80% in 2017 to be followed by annual decrease of 10 percentage until it reached the final target 40% in 2020. 101. The end target value was surpassed and reached 35%, i.e., more than half of the employees (from the base year of 2015) have ceased to be engaged in the acceptance and processing of paper forms, which demonstrates a significant improvement of the statistical process automation in the last years of project implementation. Page 135 of 159 The World Bank Second State Statistical System Development Project (P092429) 102. Intermediate indicator N-5 Availability of the national set of SDG indicators for the Russian Federation. This indicator was introduced into the system of project outcomes indicators following the recommendations of the UN General Assembly and the UN Statistical Commission to the UN member-states for establishing their own systems of indicators designed to monitor the achievement of the Sustainable Development Goals at the regional, national, and subnational levels, based on national priorities. The indicator can take two values: “yes” and “no”, depending on whether the set of SDG indicators is compiled at the national level or not. This indicator has been used since 2018 with baseline value stated as “no”. 103. The target values of the indicator have been set in 2020 as “Yes, a set of national SDG indicators has been defined” and 2021 – “Yes, a set of national SDG indicators has been defined, data for calculations have been collected”. In 2020, Russian set of national SDG indicators was defined, and the target value was achieved. 104. Intermediate indicator N-6 Number of SDG indicators compiled by official statistical bodies and based on the international methodology and introduced into the Federal Plan of Statistical Work. In 2015, the United Nations General Assembly approved 17 Sustainable Development Goals (SDGs) and 169 targets of the 2030 Agenda for Sustainable Development. These goals and targets are operationalized through SDG indicators. 105. In total, the list of global indicators of SDGs, agreed by the UN Statistical Commission, includes 232 indicators. They are based on international standards, definitions, and methodologies. The formulated indicators allow the UN to assess the level of socio-economic development of each specific country, analyze the effectiveness of state policy in the social sphere, and carry out cross-country comparisons. 106. By order of the GOR of September 23, 2017, No. 2033-r, subsection 2.8 was included in the Federal Plan for Statistical Work. "Indicators of achieving the goals of sustainable development of the Russian Federation". It included 90 SDG indicators and compiled by the organizations producing official statistics in accordance with international methodology. 107. The indicator N-6 has been used as a project indicator since 2018 with the base value of 90 indicators and the target value of 108 in 2021. Out of 90 indicators from the total list of SDG indicators, 54 indicators (60 percent) are within the competence of Rosstat, and 36 indicators (40 percent) are based on data produced by other ministries and agencies. These indicators were compiled in coordination with the federal government authorities and the expert community jointly working on the review, refinement, and agreement on the national set of indicators. 108. In 2019 and 2020, the actual values were 90 compared with plans of 96 in 2019 and 102 in 2020. At the project completion in 2021, there were 112 indicators compiled surpassing the target value of 108. Intermediate indicators N-5 and N-6 are included in the system of indicators for evaluating the event "Implementation of a set of organizational and methodological work to ensure the calculation of indicators according to an expanded list as part of comprehensive monitoring of the implementation of the SDGs". 109. Intermediate indicator N-7 Number of statistical indicators based on electronic economic descriptions as percent of the total number of statistical indicators compiled by Rosstat . This indicator reflects the level of prevalence of the use of electronic economic descriptions for the presentation of all statistical output data, thus avoiding potential errors, inconsistencies, and duplications. It is defined as the share of statistical data generated as part of the work on electronic economic descriptions (EED) of the total number of statistics produced by Rosstat. Page 136 of 159 The World Bank Second State Statistical System Development Project (P092429) 110. This indicator has been used in the project results framework since 2018 with baseline value of 34 percent and the end target set at 60 percent which was successfully reached by the project closure. Currently two thirds of statistical data compiled by Rosstat are presented on the base of electronic economic descriptions. Page 137 of 159 The World Bank Second State Statistical System Development Project (P092429) Page 138 of 159 The World Bank Second State Statistical System Development Project (P092429) Page 139 of 159 The World Bank Second State Statistical System Development Project (P092429) ANNEX 6. SUMMARY OF PROJECT RESTRUCTURING Rational for STASYS 2 project restructurings The STASYS 2 project supported the improvements in data collection, processing, dissemination and development of core statistical indicators at the federal and local levels of the Russian Federation. It catalyzed the adoption of advanced statistical methodologies for economic and social statistics and helped to bring international standards into the Rosstat operations. The project included essential upgrade of the ICT infrastructure at the central and regional Rosstat offices. It also provided training of statistical staff in the subject matter areas related to project interventions. Restructuring #1 - May 17, 2012 Financial crisis 2008-2009 led to retrenchment of federal budget resources under statistical FTP and cutdown of planned activities, including those aligned with STASYS 2 project. This was one of the reasons for slow project start up, at the same time leaving project funds as the main source for reforming the NSSS during this period. The WB supported the original closing date extension of STASYS 2 till February 28, 2014 to complete the originally planned project activities and to achieve the PDO. Restructuring #2 – February 6, 2014 The rational for project restructuring was threefold: a. extension of closing date till December 31, 2015 was to provide a suitable time period for the Borrower to explore submission of a request for Additional Financing to deepen the project's developmental impact; b. additional co-financing (equivalent to US$ 16 million); c. two of the above were to allow to scale up the currently achieved STASYS 2 project results through further development of System of National Accounts (SNA) to comply with the SNA-2008 and OECD requirements, including development of business statistics and integration of the system of environmental economic accounts; implementation of open data principles as applied to the household budget survey data; upgrading the quality of data in statistical register for greater information accessibility; modernization of Unified System for Records Collection aimed at centralization of primary statistical data treatment and its fault-free operation. Preparation of terms of references for priority statistical tasks, including social accounts and social accounting matrix recommended for introduction in Russia by the OECD, is also included into the plan aiming their possible implementation under the potential Additional Financing. Restructuring #3 – May 27, 2015 The rational for restructuring was twofold: Page 140 of 159 The World Bank Second State Statistical System Development Project (P092429) a. additional co-financing (equivalent to US$ 40 million). The GOR intended to allocate this amount under Additional Financing. Instead, it was suggested and agreed to increase the co-financing amount under on-going operation due to new 2014 geopolitical environment; b. extension of closing date till December 31, 2018 was to provide a needed time for completion of new activities envisaged to strengthen the project’s development impact. Restructuring #4 – April 4, 2018 The rational for this project restructuring was twofold. First, to support Rosstat in its effort to address new challenges—both national and global—that had a significant impact on the entire national statistical system and require further strengthening of Rosstat. These challenges included: - The Government of the Russian Federation (RF) has adopted several documents, including the "Strategy of the information society development in the Russian Federation (2017-2030) and the Program for the Digital Economy of the Russian Federation. These documents set new targets for Rosstat in the area of Rosstat Computer Information System (CIS) development. - In accordance with the United Nations (UN) recommendations, new tasks were assigned to Rosstat, namely: implementation of the activity plan for improving official statistics to produce necessary data for a comprehensive monitoring of the achievement of the Sustainable Development Goals (SDGs); coordination the compilation of the official statistical data on SDG indicators; and its submission to international organizations in accordance with the accepted statistical data exchange standards. Project activities support the Russia international commitments vis-à-vis SDGs that require further strengthening of the institution. Second, there was a strong development rationale for project extension, in particular: a. Project extension, addition of new selected activities and additional co-financing from the Borrower were allow to build on the strong results already achieved under both first STATSYS project and STASYS 2. The prospective project extension was to permit: (i) scale up and deepen the STASYS 2 project results to be achieved under current arrangements; (ii) allow for completion of the Rosstat data production with systemic strengthening and reorganization along the lines of centralized model; (iii) complete the centralization of Rosstat information capacities and creation of reliable and flexible integrated system for data production in the modern demanding environment; and (iv) strengthen the institutional and expert capacities (of Rosstat and other agencies) needed to address new challenges and take on new roles and responsibilities. On the domestic front, this included digitalization of the economy and a new coordination function and on the international front, the RF’s commitments to meet SDGs. This focus has been endorsed by the MOF and Ministry of Economic Development of the Russian Federation (MOED), which was supervising Rosstat at that time. b. The extension of STASYS 2 project was important from a broad institutional perspective and in the context of the strong and long-lasting dialogue between the Bank and the Russian authorities on statistics. Maintaining full engagement with Russia on statistics was likely to generate Page 141 of 159 The World Bank Second State Statistical System Development Project (P092429) additional regional (Europe and Central Asia—ECA) and global development gains, while strengthening economic policy. The extended project was complementing the Russia’s donor contributions of US$15 million to ECASTAT Trust Fund administered by the Bank and supporting technical assistance in statistics in ECA region, and US$4 million Reimbursable Advisory Service project to strengthen statistical expertise. The project extension was to preserve the synergy of the spectrum of operations and dialogue in statistics between the Bank and the Russian Federation. It was to strengthen its developmental impact on national statistical system, and maintain and broaden institutional relationship with MOF, MOED and Rosstat. The project closing date was extended till June 30, 2021. Additional co-financing was US$ 25milion. All newly proposed activities were built on the previously completed project tasks and assignments, and fell under the existing project structure. Restructuring #5– September 17, 2020 The rational for restructuring – extension of closing date till December 31, 2021 - was twofold: a. to provide an opportunity to link the ongoing project activities with the Strategy 2024 which has been approved by the Ministry of Economic Development of the Russian Federation (No. MO-104 of September 6, 2019) and identified key measures related to the digitalization of state statistics; and b. to allow time to implement and/or find an alternative solution to the execution of activities that have been delayed due to COVID-19 pandemic situation. The Strategy 2024 has introduced a new coordination procedure for all statistics producing agencies with regards to development of information systems and the digitalization of production processes. In order to comply with this new procedure, the execution of the activities planned under the Component B “Development of the modern design and technology for statistical data collection, processing and dissemination” required time beyond the closing date of June 30, 2021. Procurement packages B.19.a (to develop concept for using big data in the official statistics and determine the areas of technical and technological requirements in the state statistical system) and B.19.b (to develop an information system for using various sources in calculating the consumer price index and trade statistics) commenced in June 2020. Rosstat prepared the framework, readied itself to generate the information system, and had reached an agreement with the Federal Tax Services of Russia to receive source data. These were necessary foundations for successful implementation and long-term results. The COVID-19 pandemic and related measures had caused delays in the project implementation and procurement activities. The activity under the Component C “Enhancement of social statistics,” to support the monitoring and compilation of SDG indicators, involved expert missions from Germany. It had been postponed due to the travel ban. Delays occurred in signing the contracts for two activities when the consulting firms shifted to a remote operating mode in March, 2020. This further delayed the start of the 12-month implementation period. The work carried out under these contracts was directly related to the creation of the “social welfare” specified in the draft National Economic Recovery Plan for 2020-2021. In Page 142 of 159 The World Bank Second State Statistical System Development Project (P092429) addition, the training activities envisaged under the Component D “Human resources development” were postponed because of the restricted measures put in place due to the COVID-19. A face-to-face training on “Introduction of efficient data treatment under the implementation of modern IT systems” was cancelled until further notice. The client’s rationale for the proposed six-month extension was driven by the length of the lockdown issued by the government. After the restriction of movement in Moscow city had been lifted, the contracted local service providers and beneficiary agencies were back to regular operating mode and had started implementation. Five out of eight ongoing activities were expected to need additional implementation time. The contract to introduce an integrated approach to organizing and conducting household (population) sample surveys using a methodological and technological platform was expected to be completed by September, 2021. The training TORs were being restructured to allow development of online training and remote delivery of services by the German experts. Alternative method – online training activities were being discussed and planned for another three training courses. New activities added in the course of project implementation as a result of co-financing increase (by components) Component А. Modernization of economic statistics methodology A.1.11 Methodology for constructing tables for calculation processes and recommendations for their implementation into statistical practices to estimate accuracy of macroecomomic indicators A.1.12 Methodology for accounting for intellectual property in the national accounts of the RF A.1.13 Methodology for treatment expenditures on military equipment as fixed capital formation in the national accounts of the RF A.1.14 Development of the methodology for improvement of Russian business statistics A.1.16 Development of methodology for measuring oil and gas sector GDP of the Russian Federation A.1.18 Development of methodology for centralized assessment of government sector macroeconomic indicators for the regions of Russia A.1.21 Developing methodology for compiling environmental economic accounts А.1.22 Developing methodology for estimating natural resources productivity А.1.23 Development of methodological guidelines on compilation of assets accounts for fossil mineral and energy resources within the environmental-economic accounting framework А.1.24 Development of methodology for the treatment of enterprise groups while compiling SNA industry accounts Component В. Development of modern design and technology for statistical data collection, processing, and dissemination B.3.2 Introduction of an integrated approach to organizing and conducting household (population) sample surveys using a methodological and technological platform Page 143 of 159 The World Bank Second State Statistical System Development Project (P092429) B.4.9 Upgrading of Improvement of technology for Keeping and Utilization of Statistical register based on centralized information resources B.4.10 Developing database software for calculating SNA aggregates B.4.11 Improvement of the functionality of the Automated System for the general population of objects for official statistical observation B.7.11 Modernization of Unified System for Records Collection (USRC) aimed at centralization of primary statistical data treatment B.7.12 Modernization of hardware to provide for fault-free operation of USRC B.11.1 Implementing innovative management methods in the framework of Rosstat’s centralized IT-system B.11.2 Development of the operational and administrative capacity of the centralized system for data processing of Rosstat B.11.3 Development of hardware and telecommunication capacity of the Rosstat CIS В.12 Developing multi-service network of Rosstat B.13.1 Developing electronic document flow system in Rosstat B.13.2 Improving data dissemination tools within Rosstat CIS B.13.3 Improving centralized data processing within Rosstat CIS B.13.4 Development of a unified system for collecting and processing statistical information received in electronic form B.14.1 Improving approaches to the formation and maintenance of metadata in the statistical register subsystem of Rosstat CIS B.14.2 Improving approaches to the formation and maintenance of metadata in the RRI subsystem of Rosstat CIS B.14.3 Improving approaches to the formation and maintenance of metadata in the multi- dimensional data analysis subsystem (SMAD) and electronic economic descriptions subsystem of Rosstat CIS B.14.4 Development of the system for preparing electronic economic descriptions within Rosstat CIS B.15 Improving IT policy-making in terms of general system software В.16 Unification of collection, processing and dissemination of data from sample surveys of population and households В.17 Developing technical project for Integrated Computer Information Infrastructure for modernized state statistical system В.18 Developing algorithms for automated generation of statistical reports on the basis of primary data B.19.a Development of the concept of using "Big Data" (Big Data) in government statistics, taking into account international recommendations. Development of methodological approaches to the use of "Big Data" in selected sectors of statistics, including statistics of consumer prices, trade, tourism, services, employment, population Page 144 of 159 The World Bank Second State Statistical System Development Project (P092429) B.19.b Development of an information system of collecting "Big Data" (Big data) from various sources for their use in calculations of consumer price index and statistics of trade B.21 Developing software for demographic data processing, including those for demographic projections B.22 Development of an analytical subsystem for software complex for calculating prices B.23 Concept development and customization of dashboards for key socio-economic indicators on the basis of the Rosstat BI platform Component С. Enhancement of social statistics C.1.7 Upgrading the methodology and technology for processing micro data and results of household budget survey to enable access to micro data and aggregates for external users С.1.8 Development of the methodology of forming the statistical base for the calculation of indicators of income inequality and poverty, based on the harmonization of the program sample survey of household income with the program of pan-European survey of EU- Statistics on Income and Living Conditions (EU-SILC) С.1.10 Improving the empirical basis that ensures implementation in statistical practice indexes for a comprehensive assessment of the well-being and quality of life in the general population and specific groups of the population on the basis of international best practices, including the abrogation С.1.11 Improving the empirical basis that ensures implementation in statistical practice SDG indicators for the whole population and its various groups on the basis of UN standards C.1.13 Organizational and methodological work to support the compilation of a wide range of indicators for monitoring sustainable development goals (SDG) С.1.14 Methodology for compiling statistics on social benefits, both monetary and in-kind, using administrative data available at the federal and regional levels С.1.15 Development of methods for generating poverty indicators based on statistical coordination of sample survey data on population incomes and participation in social programs and administrative data on the poor С.1.16 Developing methodological recommendations for estimating household income, expenditure and savings by various income groups and coordinating the results with the 2008 SNA estimates С.3.2 Construction of improved mortality tables С.3.3 Ensuring comparability between records of vital events and population census data for calculating demographic indicators С.3.4 Short-term demographic forecasting in view of age-specific changes in fertility and mortality Component D. Human resource development Page 145 of 159 The World Bank Second State Statistical System Development Project (P092429) D.2.1.c Development of training materials and applying software "Sample design for statistical survey" D.2.1.d Development of training materials on statistical data editing D.3.2.25 Modern approaches to official statistical activities (training course for young employees of Rosstat) D.3.2.26 Training on sample survey methods D.3.2.27 Statistical Units for Production Analysis in the SNA D.3.2.28 Training course on European System of integrated social protection statistics (ESSPROS) D.3.2.29 Study visits of Rosstat employees in the field of SNA and macroeconomic statistics D.3.2.29.i Methodology for estimating intellectual property products in the national accounts D.3.2.30 Study visits of Rosstat employees by areas of statistics D.3.2.30.i Training course on using various data sources and modern methods in price statistics D.3.2.30.iii Statistical observations in the framework of the tourist sector surveys D.3.2.30.v Innovation approaches to collecting data the population and households during population sample surveys D.3.2.31 Training courses for Rosstat employees in the field of SNA and macroeconomic statistics D.3.2.32 Training courses for Rosstat employees in the field of modern methods of sector statistics (price statistics, enterprise statistics, social and ecological statistics) D.3.2.32.vi Creation of the subsidiary account of tourism D.3.2.33 Training course on increase in efficiency of work with statistical data in the conditions of implementation of modern information technologies (training in the Russian Federation) Page 146 of 159 The World Bank Second State Statistical System Development Project (P092429) D.3.2.34 Intensive training courses of Rosstat staff in creation of perspective model of the state statistics and to modern production technologies and dissemination of statistical data (training in the Russian Federation) Component E. Project Management E.5.3 Monitoring of the project performance efficiency on the basis of target indicators (Y2015) E.5.4 Monitoring of the project performance efficiency on the basis of target indicators (Y2017, Y2019, Y2020) E.5.4.i Monitoring of the project performance efficiency on the basis of target indicators (Y2017) E.5.4.ii Monitoring of the project performance efficiency on the basis of target indicators (Y2019) Accordingly, there were seven new indicators added to the original M&E framework to monitor the progress and impact of the new activities/inputs introduced as described above: N-1 Number of methodological guidelines prepared for implementation for compiling satellite accounts in the social sphere recommended by OECD; N-2 Degree of implementation of 2008 SNA into Russian statistical practices (% of works fulfilled in accordance with the “Action Plan to implement the OECD recommendations for developing the system of national accounts in the Russian Federation”); N-3 Share of official statistical reporting forms that could be submitted to NSO electronically (as % of 2015); N-4 Regional Statistical Offices employees dealing with receiving and processing primary statistical data (as % of 2015); N-5 Availability of national set of indicators for SDG monitoring in the Russian Federation; N-6 Number of SDG indicators assessed by official statistical accounting bodies based on international methodology and introduced into Federal Statistical Action Plan; N-7 Number of statistical indicators based on electronic economic description as percent of the total number of statistical indicators assessed by Rosstat. Page 147 of 159 The World Bank Second State Statistical System Development Project (P092429) ANNEX 7. BORROWER, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS From: Матюшев Илья Игоревич Sent: Tuesday, June 28, 2022 11:51 AM To: Buyant Erdene Khaltarkhuu Subject: World Bank Draft Implementation Completion Report for Second State Statistical Development Project [External] Dear Buyant, Having reiewed the Implementation Completion Report (ICR) on Second State Statistical Development (STASYS-2) Project submitted by your e-mail dated June 21, 2022, we would like to express our appreciation with the satisfactory rating assigned to the Project outcomes. We have a following comment to the current ICR: On page 33 under Factors beyond either’s control ICR claims that during the financial crisis of 2008-2009 STASYS-2 funds were the main source for reforming the national state statistical system while planned activities under statistical FTP were cut down. In fact, the overall funding of modernization activities under STASYS-2 project in 2008-2009 consisted R 62.8 million, while the overall expenditure under FTP in the same period exceeded R 1 billion. In fact, the contribution of the project in financial terms has slightly exceeded 5 per cent of expenditure under the FTP for 2007-2012. Therefore, we believe that appropriate changes should be introduced in paragraph 90 of the ICR. Kind regards, Ilya MATYUSHEV Deputy Director International Statistics Department Federal State Statistics Service (Rosstat) 39, Miasnitskaya St., bldg. 1, 107450, Moscow, Russia https://rosstat.gov.ru/ От: Buyant Erdene Khaltarkhuu Отправлено: 28 июня 2022 г. 20:29 Кому: Матюшев Илья Игоревич Тема: RE: World Bank Draft Implementation Completion Report for Second State Statistical Development Project Dear Ilya, Thank you for the review. Your comment is well noted and we would like to propose the following revision for the first bullet point under para 90. Page 148 of 159 The World Bank Second State Statistical System Development Project (P092429) “Financial crisis of 2008-2009 led to retrenchment of federal budget resources under statistical FTP and cutdown the planned activities, including those aligned with STASYS 2 project. Later, the FTP budget was fully backfilled and reinforced. STASYS 2 project funds as the source for reforming the NSSS during this period.” Please let us know if you agree. Best, Buyant From: Матюшев Илья Игоревич Sent: Tuesday, June 28, 2022 1:51 PM To: Buyant Erdene Khaltarkhuu Subject: HA: World Bank Draft Implementation Completion Report for Second State Statistical Development Project [External] Buyant, Thank you! This is to confirm that revision is fine with us. Kind regards, Ilya MATYUSHEV Deputy Director International Statistics Department Phone: +7 495 607 43 66 Fax: +7 495 607 49 27 Federal State Statistics Service (Rosstat) 39, Miasnitskaya St., bldg. 1, 107450, Moscow, Russia Page 149 of 159 The World Bank Second State Statistical System Development Project (P092429) Executive Summary of the Borrower's Implementation Completion Report Second State Statistical System Development Project22 Project Data Loan Agreement No. 4867-RU between the Russian Federation and the International Bank for Reconstruction and Development (IBRD) for financing the Second State Statistical System Development Project (Agreement) was signed on November 30, 2007 and became effective on April 22, 2008. The Agreement provided funds of the IBRD loan in a total amount of US$ 10 million for 5 years with maturity period through May 15, 2022 for financing the Second State Statistical System Development Project (Project). The initial closing date of the Project under the Agreement was established at August 25, 2012. During the implementation of the Project, amendments were made to the Loan Agreement regarding: (i) extension of the Project implementation period ( February 2014, May 2015, April 2018, September 2021); (ii) reallocation of the proceeds of the Loan between disbursement categories (May 2015); (iii) changes in the percentage of expenditures to be financed from the proceeds of the Loan (February 2014, May 2015, April 2018) and (iv) changes in the indicators of the Project results framework and monitoring (February 2014, April 2018). The Project extension in 2015 and 2018 was accompanied by an extension of the action plan agreed with the IBRD; supplementary funds has been provided in full from the Russian co-financing; the amount of the IBRD loan remained intact. The Federal Service for State Statistics (Rosstat) was responsible for project implementation in accordance with the Agreement and the regulation issued by the Government of of the Russian Federation. Bureau of Economic Analysis Foundation (BEA) was selected on a competitive basis as the Project Implementation Unit (PIU): BEA provided administrative and technical support to Rosstat in the project implementation. The Project was completed on December 31, 2021. Project Objectives and Components The objective of the Second State Statistical System Development Projects as described in the Loan Agreement was to assist the Government of the Russian Federation in producing reliable, timely and accurate economic, financial, socio-demographic and other data for policy formulation and decision- making in line with international good practice and frameworks through strengthening the capacity of the National Statistical System. The Project consisted of the following components: Component А. Modernization of economic statistics methodology Component В. Development of modern design and technology for statistical data collection, processing, and dissemination system 22 The full version of Borrower ICR is available in Russian and English. Page 150 of 159 The World Bank Second State Statistical System Development Project (P092429) Component C: Enhancement of social statistics Component D: Human resource development Component E: Project management Key Project Results The most substantial results of contracts achieved under Component А “Modernization of economic statistics methodology” include: − Development of methodology for the analysis and accounting for enterprise groups while compiling industry accounts of the System of National Accounts, which made it possible to introduce into Russian statistical practice methods of analyzing and statistical profiling of groups of dependent legal entities, whose share in the total volume of production in the Russian Federation is quite high. Profiling of enterprise groups, when compiling the SNA production and generation of income accounts by sectors, allows one to get a true picture of the allocation of factors of production (labor and capital) and outcomes of production (output, value added) by kind of economic activity, and eventually to properly distribute production capacities across the regions of the country; − in the framework of introducing a compilation of national accounts for natural resources, and a compilation of an assets and liabilities balance sheet with the account for natural assets, within the project the methodology for constructing the System of environmental and economic accounting (SEEA) and the road map for implementing SEEA into the statistical practices of the Russian Federation were developed. This work was a high priority one in view of the recommendation of international organizations (UNSD, OECD, IMF, World Bank Group, European Commission on the implementation of the SEEA-2012 Central Framework adopted as an international standard in March 2012); − as part of the efforts aimed at the transition to the 2008 SNA standards, within the framework of the Project, methodologies have been developed: for treatment of expenditures on military equipment as fixed capital formation; for treatment of intellectual property products as fixed capital, for assessment of the consumption of fixed capital as value added, etc,. which were applied for obtaining annual and quarterly series of GDP and its components for the period from 2011 to 2016; − a system for seasonal adjustments of macroeconomic indicators was developed and implemented that enabled Rosstat to construct models for seasonal adjustments of main macroeconomic indicators; − a methodology for the anonymization of microdata for presentation to users for analytical purposes was developed and further tested for structural survey of enterprises. The most substantial results of contracts achieved under Component В “Development of modern design and technology for statistical data collection, processing, and dissemination” include: − a modernization of the computing and information system (CIS) of Rosstat was undertaken that made it possible to improve the system for collecting and processing statistical data (extension of the system for SNA indicators), to introduce elements of multi-dimensional data analysis Page 151 of 159 The World Bank Second State Statistical System Development Project (P092429) using integrated data storages with the purpose of creating current enquiries and carrying out analytical data processing (including arbitrary enquiries) to ensure a transition to the centralized statistical data processing and create efficient integrated system for protecting statistical resources from unauthorized access; − in the headquarters of Rosstat and the Main Interregional Computing Center of Rosstat, on the basis of previously created information subsystems (including the Unified Agency Multiservice Network, the Unified Data Storage, the WEB-collection system, etc.), a modern information system has been created and is being supplemented with new components, which, in essence, are a production "conveyor" for collecting, processing and presenting official statistical information. Within the framework of this system, all the main tasks for maintaining and developing the IT infrastructure have been transferred to the Federal center, what ensures the optimization of resources; − a technological platform has been developed and put into operation to implement an integrated approach to organizing and conducting sample surveys of households (population), which helps to improve the quality of preparation and formalization of the methodology for processing household sample surveys; reduction of costs of material and time for preparing, conducting, processing and obtaining survey data; ensuring comprehensive measures for protecting personal data at the stage of survey data processing; preparation to use this platform as an integral part of the Digital Analytical Platform; − the concept of using Big Data in state statistics was worked out taking into account international recommendations; methodological approaches to the use of Big Data in certain statistical domains and a roadmap of activities and projects were proposed (methodological, organizational, regulatory, activities to build interaction with holders of digital sources) that are needed for using data from digital and administrative sources on a regular basis in consumer price statistics, retail trade and tourism statistics, and in population statistics. The most substantial results under Component С “Enhancement of social statistics” include: − methodological guidelines were developed for conducting comprehensive and specialized thematic surveys of the living conditions of the population and poverty monitoring; − the analysis was carried out and the methodology and techniques were improved for preparing microdata and aggregated data of household budget surveys and for providing access to microdata and aggregates for external users; − proposals were made to improve the empirical base that ensures the introduction into statistical practice integrated indices of well-being and quality of life and SDG indicators for the population as a whole and for certain population groups in accordance with the UN standards; − a methodology was developed for compiling statistics on social benefits, both monetary and in-kind, using administrative data available at the Federal and regional levels; − methodological recommendations were developed for estimating household income, expenditure and savings by various income groups and for coordinating the results with the estimates of the System of National Accounts 2008; Page 152 of 159 The World Bank Second State Statistical System Development Project (P092429) − the Labor Force Survey program and techniques for calculating individual weights to obtain labor force survey statistics on a monthly basis for Russia as a whole and for the constituents’ entities of the Russian Federation were improved; − a set of works was carried out aimed at improving health statistics as regards mortality statistics and demographic projections. The most substantial results under Component D “Human resource development” include: − development of educational and methodological materials and application software for training Rosstat employees on seasonal adjustments of official socio-economic statistics, on constructing samples for statistical surveys, and on editing statistical data; − Organization and implementation of training programs at the national and international levels in various statistical domains, including: (i) use of modern information technologies in the production processes of Rosstat, (ii) quality management in official statistics; (iii) social, demographic, and health statistics; (iv) national accounts and supply and use tables; (v) price and finance statistics, (vi) labor, education, science and culture statistics, and (vii) seasonal adjustment of social and economic statistics. Under Component Е “Project management” the Project Implementation Unit – the Bureau of Economic Analysis Foundation – provided assistance to Rosstat for the implementation of the Project in terms of procurement activities, financial management and monitoring. In addition to the activities of the PIU, this Component funded the services of short-terms consultants who assisted Rosstat and the PIU in specific project tasks’ performing as well as the project results framework and monitoring and evaluation of indicators. The tasks envisaged under project components have been completed as confirmed by achieved or exceeded project target indicators of project development objectives. Key Factors Affecting Project Implementation and Outcomes 1. The support of the Government of the Russian Federation and other federal executive authorities allowed the Federal Service for State Statistics to complete at the initial project stages all procedures required for signing the Agreement and fulfillment of the effectiveness conditions. 2. High level of project management. The clear project management system was used throughout the Project implementation period. The Interagency Coordination Council for Project implementation (ICC) was established to ensure control for the project implementation and coordination. The ICC comprised the representatives of Rosstat, Ministry of Finance and the Ministry of Economic Development, responsible for the implementation of the Project, and BEA. Besides, working groups were established for coordination and management of the separate Project components, which, when necessary, conducted meetings to discuss substantial issues of the Project implementation. In addition to Rosstat staff members, the working groups also comprised representatives of the Ministry of Finance, the Ministry of Economic Development and BEA. Page 153 of 159 The World Bank Second State Statistical System Development Project (P092429) Coordination of activities under the Project between Rosstat, the Ministry of Finance and BEA was performed under the Agency Agreement. All project-related financial and organizational issues were addressed by Rosstat together with BEA and in coordination with other agencies involved in the Project, within their competence, and were approved by the project management bodies. To ensure proper level of project management, Rosstat with BEA’s assistance, developed the Operational Manual. The Operational Manual included the key documents regulating daily project activities: project management rules and procedures, procurement procedures, financial management system, staffing policy, etc. 3. The special organization (Project Implementation Unit, PIU) was engaged to perform procedural and technical functions during project preparation and implementation, including procurement procedures, contracts signing and performance monitoring, opening and maintaining the Designated Account, preparing financial reports in accordance with the IBRD procedures, providing assistance to Rosstat in preparation of project reporting, and ensuring day-to-day contacts with the IBRD. At the same time, project management mechanisms used within Rosstat were coordinated and were given significant importance; implementation of project activities was monitored and coordinated with Rosstat’s activities on an ongoing basis. 4. The World Bank's extensive experience with similar projects implemented in other countries, expert support of the World Bank's specialists, assistance in dealing with problems emerging under the Project contributed to achieving high and sustainable project results. Besides, it should be mentioned that the World Bank has shown flexibility in project management considering specific aspects of country and national legislation, directly or indirectly affecting the Project. At all times during implementation of the Project, the World Bank was flexible in its response to ongoing changes, taking into account the Borrower’s suggestions, where possible. The flexible approach in review of project procurement and contract amendments contributed to implementing the activities required for Rosstat, on the one hand, and to effective project disbursement, on the other. Changes in the WB project team which occurred in the last years of project implementation did not entail the delays in expediting project funds. Sustainability of project results As part of the implementation of the Project, results were achieved that in the long term correspond to the priority areas of the socio-economic policy of the Russian Federation. The project has made a significant contribution to strengthening the national statistical system in the Russian Federation, and the results achieved during its implementation have become an integral part of the ongoing reforms in this area. Besides, new legal and normative acts were adopted and amendments to already existing legal and normative acts were made based on the proposals developed under the Project, including within the framework the System of National Accounts, as well as the development and improvement of the social and demographic statistics. The plans for the development of official statistics, which allow assessing the sustainability of the results of the Project, are defined in the document “Strategy 2.0” posted on the official website of Rosstat in 2021 – “Strategy for the development of Rosstat and the system of state statistics”. Page 154 of 159 The World Bank Second State Statistical System Development Project (P092429) Performance of the World Bank and the Borrower Performance of the World Bank The World Bank's involvement contributed significantly to successful implementation of the Project. The Bank's support, methodological and organizational assistance, readiness to maintain a constructive dialogue, understanding of specific aspects of Russian state statistical system development contributed to achieving successful implementation of the Project. The World Bank reviewed and approved implementation strategies, project implementation plans and procurement plans, bidding documents/RFPs at various stages of procurement procedures, advised BEA and Rosstat on the above issues and provided funds to finance the Project in accordance with the Loan Agreement. The IBRD also reviewed and approved amendments to the Loan Agreement proposed by the Borrower. The IBRD reviewed the BEA's reports, including financial reports and Project progress reports, as part of the supervision of the Project progress. The close monitoring over the Project implementation by IBRD helped ensure compliance with Loan Agreement No. 4867-RU. The IBRD also regularly organized field supervision missions for evaluating the Project implementation and outcomes and provided its recommendations to BEA and Rosstat upon completion of such missions. Overall, the communication between Rosstat, BEA and IBRD can be characterized as constructive and successful. The IBRD approved the key project implementation activities, bidding documents/RFPs and any other documents subject to its approval. The IBRD comments were generally constructive and justified and contributed to enhancing effectiveness of procurement procedures and the quality of bidding documents/RFPs. The IBRD independent supervision missions, comments and assistance of IBRD experts contributed to enhancing objectivity in evaluation of the Project progress and results achieved. One of the most important project success factors is that the Bank project team was highly skilled and knowledgeable about all details of the Project from the time the Project was started and had experience in similar projects. The performance of the World Bank during project preparation and supervision can be rated as highly satisfactory. Performance of the Borrower and the Project Implementation Unit Federal Service for State Statistics Based on the regulation of the Government of the Russian Federation #43 dated January 28, 2005 “On the work with projects, implemented by the Russian Federation with participation of international financial organizations” Rosstat has conducted the implementation of the project and was responsible for the attainment of established project indicators. Good preparation of the project coupled with effective implementation permitted to get excellent results. In planning and implementation of the project activities Rosstat has revealed clear understanding of tasks in enhancing the potential of the Russian statistical system, targeted at getting reliable, timely and accurate statistical data complying with the international standards and best international practice for further use by the Government in preparation and elaboration of policy measures. Page 155 of 159 The World Bank Second State Statistical System Development Project (P092429) Project implementation was in full compliance with procurement rules and financial management procedures of the IBRD. Deliveries of goods and services were undertaken in a timely fashion, appropriate budgetary appropriations for payments under contracts under the Project were provided in a timely fashion and in full. In spite of the problems encountered at an early stage, entailing delays in project active implementation, Rosstat has updated and agreed with the IBRD new procurement plan, terms of references and bidding packages were developed with an emphasis on key Rosstat tasks and vector of statistical system further development. Gradual progress in project implementation and disbursement and comprehensive control of the progress of project implementation have facilitated successful completion of planned activities and attainment of project objectives. Rosstat has efficiently implemented the project due to the fact that project development objectives and Government strategy in the field of statistics were consistent and aimed at better quality and reliability of state statistics system. Project development objectives were met, some of them had been attained at the project early stage. The performance of Rosstat as a Borrower can be rated as highly satisfactory. Ministry of Finance of the Russian Federation In the course of project implementation, the Ministry of Finance has been executing control over efficient and properly targeted appropriation of the Loan proceeds, on an annual basis provided in the federal budget allocations for co-financing of the project, agreed procurement plans and project implementation plans, project’s annual budgets. Representatives of the Ministry of Finance have participated in the work of Interagency Coordination Council, working groups and evaluation committees. Based on Rosstat proposals Ministry of Finance has granted support for the project extension and provided budgetary appropriations for the additional activities, targeted at enhancement of the project efficiency and attainment of project objectives. The performance of the Ministry of Finance of the Russian Federation related to the Project can be rated as highly satisfactory. Ministry of Economic Development of the Russian Federation Ministry of Economic Development in line with the Russian legislation has reviewed and agreed procurement plans and project implementation plans, project’s annual budgets. Representatives of the Ministry of Finance have participated in the work of Interagency Coordination Council, working groups and evaluation committees. In the course of review and approval of procurement plans, project implementation plans, project’s annual budgets and reports on implementation of these plans Ministry of Economic Development has exercised the control of targeted disbursement of project’s funds and overall compliance of the project with high priority measures of social and economic policy of the Russian Federation. Currently Ministry of Economic development is responsible for elaboration of official policy and legal regulation in the field of official statistics. The performance of the Ministry of Economic Development of the Russian Federation related to the Project can be rated as highly satisfactory. Project Implementation Unit Page 156 of 159 The World Bank Second State Statistical System Development Project (P092429) The Noncommercial Foundation “Bureau of Economic Analysis” actively participated in the Project preparation process and acted as the Project Implementation Unit after the effective date of the Loan Agreement throughout the period of implementation of the Project under Agency Agreement No. 01-01-06/04-42 dated March 21, 2008. Under the Agency Agreement the BEA Foundation was responsible for preparing bidding documents/RFPs, organizing and implementing procurement procedures for goods (services) and supporting contracts in accordance with the IBRD rules and project goals and objectives; opening and maintaining the Designated Account; recording funds of co-financing of the Project in accordance with the IBRD requirements and laws of the Russian Federation; providing project reports to the Rosstat, the Ministry of Finance, the Ministry of Economic Development and the IBRD in accordance with their requirements; engaging on a contractual basis the auditor acceptable to the IBRD to conduct project audit, etc. The project implementation materials and financial data were prepared by BEA and submitted to the Rosstat, the Ministry of Finance, the Ministry of Economic Development and the IBRD in a timely manner. As a result, all stakeholders were able to monitor the project progress. The Bureau of Economic Analysis Foundation properly performed its PIU functions. The BEA's performance of PIU functions can be rated as highly satisfactory. Advantages of the Project implementation with assistance of the International Bank for Reconstruction and Development With the World Bank's involvement in addressing the most important tasks of the Russian Federation, it is possible to attract additional financial resources into economic turnover on favorable conditions for a long period of time. The mechanisms for implementing such projects are regulated by strict and transparent procedures governing the activities of parties and providing proper control over the use of funds. Along with the funds provided, the project is supported by the World Bank highly skilled team that provides assistance to the executing agencies and the opportunity to apply the international experience (best practice) accumulated during implementation of similar projects around the world. Lengthy procurement procedures implemented in accordance with the IBRD rules had some impact on the Project duration, but it was eventually compensated by the quality of services provided and goods supplied. During Project preparation and implementation Rosstat’s project team, with the IBRD assistance, enhanced its experience in communication with the international financial organizations, procurement and implementation of contracts based on the World Bank rules and procedures. Conclusion Upon completion of the Project, we may confidently say that the key project objectives have been achieved. The total project disbursement amounted to 98% of the Project budget (the disbursement of the loan funds is amounted to almost 99.9%). The project activities significantly contributed to the reforms. The project has contributed to the implementation of the objectives of the Government of the Russian Federation and Rosstat in the field of provision of maximum quality (comprehensiveness, reliability, timeliness and consistency) and availability of official statistical information as well as its utility for the decision taking at all the levels of government Page 157 of 159 The World Bank Second State Statistical System Development Project (P092429) and management. Further development and amelioration of official statistics in Russia will be based on the “Strategy for the Development of Rosstat and official statistics system until 2024”. This document specifies the activities of Rosstat targeted at increased satisfaction of all groups of users of official statistics, implementation of these measures will enhance the output indicator of the project “Index of users’ trust in the official statistics”. Specifically, Rosstat plans to establish “Situation Center” for collection, analysis of official statistical data and meeting the requirements of users. User satisfaction surveys, organized in the Project’s framework will be continues, survey program and respondents’ sample will be extended. Stability of Project’s results will be ensured through: (i) completion of implementation of SNA- 2008 standards, envisaged by the Rosstat strategic plan by 2023; (ii) organization of electronic data collection, complete withdrawal of paper forms, streaming of business statistics produced by business entities, introduction of computerized platforms for household surveys; (iii) reduction of businesses “reporting burden” with full elimination by 2024 of duplicating reporting indicators; (iv) introduction of a unified register of forms of statistical data. Effective use of alternative sources of data is an integral part of an overall development strategy. In 2022-2024 the concept of use of big data in statistics, developed in the framework of the project, will be implemented. “Knowledge Center” on application of “Big Data” will be established, pilot projects in price statistics, statistics of retail trade, tourism, agriculture, population and other areas will be implemented. In the field of statistical information dissemination strategic plans of Rosstat envisage an establishment of a comprehensive BI platform of statistical data, designed to replace earlier systems starting in 2022. Thus, the Project demonstrated high efficiency in the use of available resources with the aim to achieve outcomes and had a long-term impact on the development of official statistical information on social, economic, demographic and environmental development of the country and satisfaction of the needs of the state and the society in statistical data fitting international standards. We assume that the Project can be rated as highly satisfactory considering the project relevance, effectiveness and efficiency, despite some implementation delays. Page 158 of 159 The World Bank Second State Statistical System Development Project (P092429) ANNEX 8. SUPPORTING DOCUMENTS (IF ANY) 1. Minutes of Project Concept Review 2. Aide Memoires, Preparation Missions 4. Project Appraisal Document, June 8, 2007 5. Agreed Minutes of Negotiations 6. Loan Agreement, November 30, 2007 7. OPCS Guidance: Implementation Completion and Results Report (ICR) for Investment Project Financing (IPF) Operations, March 2020 8. IEG Guidance: Draft Guidelines for Reviewing World Bank Implementation Completion and Results Reports: A Manual for IEG ICR Reviewers, July 2017 9. OPCS and IEG presentation on ICR for IPF Operations, 18 March 2021 10. Aide Memoires, Supervision Missions 11. Disclosed Project Implementation Status& Results Reports 12. Mid-Term Review Report 12. Project Restructuring Papers 13. OECD Assessment of the State Statistical System Report, 2015 14. Letters from Federal State Statistical Service of the Russian Federation (Rosstat) 15. Letters from Ministry of Finance of the Russian Federation 16. Letters from Ministry of Economic Development of the Russian Federation 17. Borrower’s ICR in Russian and English Page 159 of 159