Document of The World Bank Report No: ICR00003398 IMPLEMENTATION COMPLETION AND RESULTS REPORT (IDA-46070, TF93575, TF94696, TF98487) ON A IDA CREDIT IN THE AMOUNT OF SDR 1.4 MILLION (US$ 2 MILLION EQUIVALENT) TRUST FUND FOR STATISTICAL CAPACITY BUILDING (TFSCB) IN THE AMOUNT OF US$ 0.40 MILLION KOREA TRUST FUND (KTF) IN THE AMOUNT OF US$ 0.70 MILLION SECOND KOREA TRUST FUND (KTF) IN THE AMOUNT OF US$ 0.65 MILLION TO MONGOLIA FOR A MONSTAT: STRENGTHENING THE NATIONAL STATISTICAL SYSTEM OF MONGOLIA PROJECT IN SUPPORT OF THE MULTI COUNTRY STATISTICAL CAPACITY BUILDING PROGRAM June 26, 2015 Macro and Fiscal Management Global Practice EACCF East Asia and Pacific Region (EAP) CURRENCY EQUIVALENTS (Exchange Rate Effective June 17, 2015) Currency Unit = US$ 1.00 = MNT 1,878 US$ 0.71 = SDR 1 FISCAL YEAR January 1 – December 31 ABBREVIATIONS AND ACRONYMS BOM Bank of Mongolia CAS Country Assistance Strategy CITA Center for Information, Training and Advocacy CPI Consumer Price Index CPS Country Partnership Strategy CRC Civil Registration Center EU European Union GDDS General Data Dissemination System GDP Gross Domestic Products GIS Geographical Information System IBRD International Bank for Reconstruction and Development IDA International Development Association IFR Interim Financial Reports IMF International Monetary Fund IT Information Technology ISN Interim Strategy Note ISR Implementation Status & Results Report MAPS Marrakech Action Plan for Statistics MDG Millennium Development Goals MOF Ministry of Finance MNT Mongolian Tugriks MONSTAT Strengthening the National Statistics System of Mongolia MTR Mid-term review NDS National Development Strategy NSC National Statistical Council NSDS National Strategy for the Development of Statistics NSO National Statistics Office of Mongolia OECD Organization for Economic Cooperation and Developments PAD Project Appraisal Document PCT Project Coordination Team PDO Project Development Objectives i PPI Producer Price Index RF Results Framework SDDS Special Data Dissemination Standard SMP Statistical Master Plan SNA System of National Accounts STATCAP Statistical Capacity Building TFSCB Trust Fund for Statistical Capacity Building TOR Term of Reference UNESCAP United Nations Economic and Social Commission for Asia and the Pacific UNSC United Nations Statistical Commission Vice President: Axel van Trotsenburg Senior Global Practice Director: Marcelo Giugale Country Director: Bert Hofman Practice Manager: Mathew A. Verghis Project Team Leader: Altantsetseg Shiilegmaa ICR Author: Olga Shabalina ii MONGOLIA MONSTAT: Strengthening the National Statistical System of Mongolia CONTENTS Data Sheet A. Basic Information........................................................................................................ i B. Key Dates .................................................................................................................... i C. Ratings Summary ....................................................................................................... ii D. Sector and Theme Codes ........................................................................................... ii E. Bank Staff ................................................................................................................... ii F. Results Framework Analysis ..................................................................................... iii G. Ratings of Project Performance in ISRs ................................................................... vi H. Restructuring (if any) ................................................................................................ vi I. Disbursement Profile ................................................................................................ vii 1. Project Context, Development Objectives and Design ............................................... 1 2. Key Factors Affecting Implementation and Outcomes .............................................. 7 3. Assessment of Outcomes .......................................................................................... 12 4. Assessment of Risk to Development Outcome......................................................... 19 5. Assessment of Bank and Borrower Performance ..................................................... 20 6. Lessons Learned ....................................................................................................... 22 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners .......... 23 Annex 1. Project Costs and Financing .......................................................................... 25 Annex 2. Outputs by Component ................................................................................. 29 Annex 3. Economic and Financial Analysis ................................................................. 47 Annex 4. Bank Lending and Implementation Support/Supervision Processes ............ 48 Annex 5. Beneficiary Survey Results ........................................................................... 49 Annex 6. Stakeholder Workshop Report and Results................................................... 50 Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR ..................... 51 Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders ....................... 67 Annex 9. List of Supporting Documents ...................................................................... 68 iii MONGOLIA MONSTAT: Strengthening the National Statistical System of Mongolia DATA SHEET A. Basic Information MONSTAT: Strengthening the Country: Mongolia Project Name: National Statistical System of Mongolia IDA-46070,TF- Project ID: P113160 L/C/TF Number(s): 93575,TF-94696 ICR Date: 06/08/2015 ICR Type: Core ICR Lending Instrument: APL Borrower: MONGOLIA Original Total USD 2.00M Disbursed Amount: USD 2.15M Commitment: Revised Amount: USD 2.00M Environmental Category: C Implementing Agencies: National Statistical Office of Mongolia Cofinanciers and Other External Partners: Trust Fund for Statistical Capacity Building Korea Trust Fund Turkish International Cooperation Agency Federal Statistical Office of Germany (DESTATIS) Korean Statistical Information Service (KOSIS) B. Key Dates Revised / Actual Process Date Process Original Date Date(s) Concept Review: 09/24/2008 Effectiveness: 09/16/2009 06/26/2009 Appraisal: 01/23/2009 Restructuring(s): 12/20/2013 12/26/2013 Approval: 06/12/2009 Mid-term Review: 04/23/2012 06/12/2012 Closing: 12/31/2014 12/31/2014 i C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Satisfactory Risk to Development Outcome: Low or Negligible Bank Performance: Satisfactory Borrower Performance: Satisfactory C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings Quality at Entry: Satisfactory Government: Satisfactory Implementing Quality of Supervision: Satisfactory Satisfactory Agency/Agencies: Overall Bank Overall Borrower Satisfactory Satisfactory Performance: Performance: C.3 Quality at Entry and Implementation Performance Indicators Implementation QAG Assessments Indicators Rating Performance (if any) Potential Problem Quality at Entry Project at any time No None (QEA): (Yes/No): Problem Project at any Quality of No None time (Yes/No): Supervision (QSA): DO rating before Satisfactory Closing/Inactive status: D. Sector and Theme Codes Original Actual Sector Code (as % of total Bank financing) Central government administration 100 100 Theme Code (as % of total Bank financing) Administrative and civil service reform 5 5 Economic statistics, modeling and forecasting 20 20 Managing for development results 63 63 Other economic management 10 10 Other public sector governance 2 2 E. Bank Staff Positions At ICR At Approval Vice President: Axel van Trotsenburg James W. Adams ii Country Director: Bert Hofman David R. Dollar Practice Mathew A. Verghis Vikram Nehru Manager/Manager: Project Team Leader: Altantsetseg Shiilegmaa Mustafa Dinc ICR Team Leader: Olga Vadimovna Shabalina ICR Primary Author: Olga Vadimovna Shabalina F. Results Framework Analysis Project Development Objectives (from Project Appraisal Document) The main objective of the project is to improve efficiency and effectiveness of the national statistical system of Mongolia to enable it to provide relevant, timely and reliable data for evidence-based policy making in support of Government's MDG-based National Development Strategy (NDS). Revised Project Development Objectives (as approved by original approving authority) NA (a) PDO Indicator(s) Original Target Formally Actual Value Values (from Revised Achieved at Indicator Baseline Value approval Target Completion or documents) Values Target Years Indicator 1 : User satisfaction rates increased by at least 60 Value quantitative or 46.70 percent 60 percent 71.00 percent Qualitative) Date achieved 10/07/2010 12/31/2014 12/31/2014 Comments Overachieved. The results of regular Customers’ Satisfaction Surveys, its (incl. % methodology and recommendations are available on the NSO website. achievement) Targeted statistical products easily accessible in the media, with metadata and Indicator 2 : interpretation of findings. Statistical data dissemination Statistical data are 100% system developed Value accessible through Increased and operational. quantitative or website and printed accessibility Data, including Qualitative) publications in limited through the mass micro data, are types. media publically available through variety of channels. Date achieved 06/06/2010 12/31/2014 12/31/2014 iii Achieved. Data, including micro data are available through www.1212.mn, Comments smart phone, tablet applications (MONSTAT, EzSTAT), 19001212 hotline, (incl. % improved NSO website, regular publications. Statistical library was re- achievement) established. Trainings for media conducted. Internationally-accepted statistical techniques in collection, compilation and Indicator 3 : authenticity verification applied, and regular validations carried out for data sources and statistical products Development of new internationally accepted classifications and methodologies that Regularly enhanced coverage, Value Verification takes place, conducted check- accuracy of official quantitative or need to improve skills ups based on data. Development Qualitative) and methodology improved and adoption of methodology revision policy. Contribution to surveys conducted with quality check- ups. Date achieved 06/20/2010 12/31/2014 12/31/2014 Achieved. 14 surveys conducted with improved verification and quality check- Comments ups. Poverty measurement improved based on new sampling design and new (incl. % questionnaire for household surveys. Data quality policy developed based on EU achievement) quality management practices. Statistical outputs released in a timely manner in accordance with Indicator 4 : internationally-accepted frequency and timeframes. According to IMF assessment the Value Submission of NSO met all SDDS NSO meets the major quantitative or statistical data at technical requirements of GDDS Qualitative) SDDS requirements for timeliness and periodicity. Date achieved 06/20/2010 12/31/2014 12/31/2014 Achieved. According to IMF SDDS mission evaluation report (December 2014) Comments the NSO has met the SDDS requirements. The official submission for SDDS (incl. % depends upon implementation of the agreed action plan by the Ministry of achievement) Finance and Bank of Mongolia. (b) Intermediate Outcome Indicator(s) Original Target Actual Value Formally Values (from Achieved at Indicator Baseline Value Revised approval Completion or Target Values documents) Target Years Indicator 1 : Inter-agency coordination and user-producer dialogue iv NSO coordinates statistical activities, collection of official data. There was no Effective comprehensive evaluation interagency Value Effective of existing departmental coordination in (quantitative interagency and centralized statistical place. User- or Qualitative) coordination. reporting conducted in producer dialog terms of usefulness of enhanced. data collected, identification of non- needed info Date achieved 06/20/2010 12/31/2014 12/31/2014 Achieved. Department to work with other data producers set up. Agreements Comments signed with 9 ministries. Customer management system in place. Monthly press (incl. % conference with media agencies. Center for ITA operational. User satisfaction achievement) survey regularly conducted. Indicator 2 : Human resource capacity Capacity of the Local level staff is trained NSO management by central office staff. In and staff enhanced some cases, some at national and Optimization of personnel at both central subnational levels Value organizational and local levels were through (quantitative structure of NSO trained by international organizational or Qualitative) at the central and organizations. However, changes and local levels. there was no training comprehensive center and comprehensive training. NSO is training curriculum. recognized internationally. Date achieved 06/20/2010 12/31/2014 12/31/2014 Achieved. Department to strengthen local offices set up. Full-time statistician Comments position created at soum level. Training programs of 4 levels conducted. 4647 (incl. % staff trained locally, 406 in EU countries. NSO started hosting learning visits achievement) from abroad. Indicator 3 : Statistical Infrastructure Full scale use of register for Full scale use of surveys. 100% register for surveys. Register actualization is adaptation of 100% adaptation of weak, internationally internationally internationally Value accepted classifications accepted accepted (quantitative and standards not fully classifications, classifications, or Qualitative) applied, poor sampling standards. standards. Standard design and lack of Standard questionnaire for seasonal adjustment questionnaire for surveys developed surveys developed and applied. and applied. Date achieved 06/20/2014 12/31/2014 12/31/2014 v Achieved. New business register system and business register management Comments system developed. Online database system for population and household surveys (incl. % operational. 4 classifications, 11 methodologies developed based on international achievement) standards, and applied. Indicator 4 : Physical infrastructure Capacity for physical operations is The IT network adequate and well security system maintained. At fully installed and Most of the workstations least 75% of staff operational. All Value are outdated. Currently, at each unit at (100%) staff at (quantitative only 3 departments have NSO headquarters NSO at both central or Qualitative) access to appropriate have workstations and regional levels section of database. with direct access have workstations to databases for with direct access analysis and to databases. tabulation, and 60% of field staff have workstations. Date achieved 06/20/2010 12/31/2014 12/31/2014 Overachieved. The IT network security system was fully installed and Comments operational, data dissemination system developed and fully operational. Tablets (incl. % are vastly used for data collection which resulted in cost-efficiency of the NSO achievement) operations. G. Ratings of Project Performance in ISRs Actual Date ISR No. DO IP Disbursements Archived (USD millions) 1 12/15/2009 Satisfactory Satisfactory 0.10 2 01/25/2011 Satisfactory Satisfactory 0.10 3 04/05/2012 Satisfactory Satisfactory 0.65 4 03/27/2013 Satisfactory Satisfactory 1.39 5 11/29/2013 Satisfactory Satisfactory 1.61 6 06/24/2014 Satisfactory Satisfactory 1.99 7 01/28/2015 Satisfactory Satisfactory 2.15 H. Restructuring (if any) In December 2013 following the request from the NSO, the undisbursed funds under “Unallocated category” were reallocated to valid project expenditure categories under all project components to continue and complete the remaining tasks. vi I. Disbursement Profile vii 1. Project Context, Development Objectives and Design 1.1 Context at Appraisal 1. After 2008 crisis recovery, the economic reforms and country’s growth in Mongolia continued. Mongolia was strongly affected by the 2008 global financial crisis and extremely severe follow-up winter, both of which significantly stunted its economic growth. In 2009, real Gross Domestic Product (GDP) growth dropped from 8 percent to 2.7 percent per annum and exports shrank by 26 percent from US$2.5 billion to US$1.9 billion. Since then, the market has begun to recover. Having identified and learned from previous economic instabilities, the government introduced legislative reform and a new tightened fiscal policy to guide the country upwards. Mongolia was and continues to be the least densely (2 persons per sq. km) populated country in the world with 2.56 million people in 2009, one third of which were poor, though the overall poverty incidence seems to be declining. 2. The national statistical system (NSS) in Mongolia has undergone substantial transformation from a service structured to meet the needs of a planned economy to one reflecting the needs of the developing market economy and new society. It evolved from the system of net material product to the system of National Accounts. Data collection methods have changed from the mandatory complete reporting to sample-based surveys. The first Law on Statistics was adopted by Parliament in 1994, later amended in 1997, 1999, 2004 and 2008. A law on the Population and Housing Census was passed by the State Great Khural (Parliament) in early 2008. The statistical system of Mongolia remained centralized, with the National Statistical Office (NSO) in charge of managing official statistics and coordinating activities in the sector. The Bank of Mongolia functioned independently in the area of monetary and banking statistics, and the Ministry of Finance compiled budget statistics as part of its operations. Line ministries and other agencies produced two types of statistical information: (i) data collected for the NSO on the basis of methodologies and questionnaires that must be approved by NSO, and that appear in the annual Statistical Action Plan; and (ii) data collected for the needs of line ministries, using methodologies and questionnaires that must be reviewed and discussed with NSO. The latter did not appear in the Statistical Action Plan. The line ministries and agencies often published their own data summaries. 3. The NSO was reorganized in 1990. It became an independent agency under the authority of the State Great Hural/Parliament. It had branches in each the 21 aimags (provinces) and Ulaanbaatar and statistical divisions in all 9 districts of the Capital city, where about half of the country’s population resided. In 2001, Mongolia became a member of the International Monetary Fund (IMF)’s General Data Dissemination System (GDDS) with the NSO as lead agency. 4. At project appraisal, the country’s vision for development was outlined in the National Development Strategy (NDS). The NDS, approved by Parliament, included six main priorities for the period 2007-2021. They were: (i) achieving the Millennium 1 Development Goals (MDGs); (ii) developing an export-oriented, high tech, knowledge- based economy; (iii) exploiting strategic minerals to build savings, growth, and a modern processing industry; (iv) developing regions and infrastructure to reduce urban-rural disparities; (v) halting ecosystem imbalances; and (vi) consolidating political democracy and an accountable system that is free from corruption and red tape. To monitor progress in achieving these significant policy initiatives, comprehensive, high quality, and reliable statistics would be required. 5. The National Strategies for Development of Statistics (NSDS) were developed to guide the progress in statistics. The first Statistical Master Plan (SMP) was developed and approved by the Parliament in January 2006. The NSDS for 2006-2010 updated the SMP and was accompanied by a financing strategy for its implementation. It was prepared, discussed with stakeholders, and approved by the National Statistical Council in December 2007. The Strengthening the National Statistics System of Mongolia (MONSTAT) project contributed to implementation of this NSDS and the next one for 2011-2015, also approved by Parliament in 2011. A follow up Action Plan was approved by the National Statistical Council for implementation shortly thereafter. 6. The proposed project was in line with the Interim Strategy Note (ISN). The project was included in the ISN, May 21, 2009 (Report No. 48311-MN) to support the government in strengthening the NSS to produce reliable, timely and good quality data for monitoring key poverty and development indicators. The outputs of this project would help sustain achievements of the ISN pillars in the medium- to long-term. 7. The rationale for Bank support was to consolidate and build upon the progress made in the area of statistical capacity, for which the Bank had a comparative advantage. The Bank’s mandate to lead the statistical capacity initiative, particularly in International Development Association (IDA) countries, was reinforced in the IDA15 Results Measurement System. It specifically stated that Country Assistance or Partnership Strategies (CAS or CPS) were expected to include a review of the national statistical system and to indicate what is needed to strengthen capacity to generate and use statistical data. The Bank had already been a player in statistical development in Mongolia prior to project approval through a variety of technical assistance initiatives to enhance the capacity of the NSO and to support the preparation of the NSDS. To build on past achievements and to avoid duplication, the project team closely coordinated with other on-going operations in the country. 1.2 Original Project Development Objectives (PDO) and Key Indicators 8. As stated in the Project Appraisal Document (PAD) (II.C. and Technical Annex 3) and Financing Agreement (Schedule 1, page 5) the main objective of the project was to improve the efficiency and effectiveness of the national statistical system of Mongolia to enable it to provide relevant, timely and reliable data for evidence-based policy-making in support of the Government’s MDG-based National Development Strategy (NDS). 2 9. The progress towards the PDO was to be measured through the Results Framework outlined in the PAD (pages 5-6 and Annex 3). It encompassed the key indicators presented below: Table 1: Key indicators Objectives Hierarchy Key Indicators PDO Project Outcome Indicators The main objective of Access to information/dissemination the project is to improve 1. User satisfaction rates increased by at least 60% efficiency and 2. Targeted statistical products easily accessible in the media, with metadata and effectiveness of the interpretation of findings. national statistical system of Mongolia to Coverage/ Accuracy/ Reliability: enable it to provide 3. Internationally-accepted statistical techniques in collection, compilation and relevant, timely authenticity verification applied, and regular validations carried out for data and reliable data for sources and statistical products. evidence-based policy making in support of Timeliness: Government’s MDG- 4. Statistical outputs released in a timely manner in accordance with based National internationally-accepted frequency and timeframes. Development Strategy (NDS) Intermediate Outcomes Intermediate Outcome Indicators 1. Adequate policy and regulatory framework as 1.1 Mechanisms for effective interagency coordination established and well as effective operational. institutional framework, 1.2 Mechanisms for effective dialogue with data users and providers management and human operational. resources for statistics 1.3 National Statistics Training Center established. are in place 1.4 NSO central and local level structure optimized and staff trained, including staff from relevant ministries and agencies. 2.1 Registers compiled and maintained. 2.2 Internationally- accepted classifications, standards and methodologies 2. Statistical adapted and used in data production. infrastructure developed 2.3 Standard questionnaire for survey developed and applied. and operational 2.4 Targeted surveys conducted, thematic statistical data developed and results disseminated. 3. Investment in physical infrastructure and 3.1 Capacity for physical operations adequate and well maintained. equipment to facilitate 3.2 At least 75% of headquarter staff in each unit of NSO and 60% of field the production and staff equipped with workstations, with direct access to databases for analysis dissemination of data by and tabulation. statistical agencies is undertaken and adequately maintained 3 1.3 Revised PDO and Key Indicators, and reasons/justification 10. The PDO and key indicators were not revised. Information was available for all indicators defined in the PAD. In the ISRs several indicators were slightly re-worded and specified to reflect more accurately the high-level outcomes and intermediate results. 1.4 Main Beneficiaries 11. The main beneficiaries of the project were the NSO, the Government and the general public of Mongolia, as well as international development partners. 1.5 Original Components 12. The PDO was to be achieved through the implementation of four project components with a total allocation of US$3.1 million (IDA credit US$2 million; Trust Fund for Statistical Capacity Building (TFSCB) TF93575 grant US$0.4 million; first Korea TF TF94696 grant US$0.7 million) – see Annex 3. The components’ description is based on the PAD and the Financing Agreement. 13. Component A. Improving policy, regulatory, and institutional framework and the organizational structure of the statistical system (US$0.57 million) included five sub-components: - Review and improvement of the existing institutional framework to develop relevant legal environment and mechanisms for interaction of the NSS institutions and agencies for the production, use, exchange and disclosure of statistical information. - Improving the relationship between data users and producers to categorize and analyze the pool of users (government, industry, academia, media, international organizations, and the community at large) according to their functions and data needs. This would serve as a basis to define the demand from each category and inform each group about statistical programs and data that respond to their needs. The subcomponent would provide support to user needs and satisfaction surveys, more frequent conferences and workshops with the media, and a more active role for the National Statistical Council. - Improving data dissemination function and procedures to provide easier access to statistics to a larger range of users through wider dissemination of publications and an improved NSO website. - Strengthening human resource capacity to develop a human resource management framework with an incentive structure to retain and attract qualified professionals and managers in the national statistical system. The framework would include career development, staff assignment programs, and proactive and coordinated recruitment procedures; to support the design of a training strategy and the establishment of the National Statistical Training Center with a library containing relevant documentation in electronic and hard copy formats. - Review and improvement of the organizational structure of the national statistical system to support increased manpower in NSO divisions responsible for data collection 4 and production. It would also help to streamline the existing geographically-based organizational structure for data collection at the regional level. 14. Component B. Strengthening the statistical infrastructure (US$0.8 million) focused on enhancing these key elements – registers, classifications, standards and methods: - Improvement and updating of statistical registers (business, population, housing and agriculture). According to the national Statistical Law, the NSO has overall responsibility for establishing and maintaining these registers. Prior to the Law, the population register was maintained by the State Civil Registration Center (CRC), a state owned company, and the NSO had no access to its database. This subcomponent supported access to the CRC database and provided the necessary tools for establishing secure links. It also supported the development of the Housing (dwelling) Register and the creation of an agriculture register. The Business Register would be enhanced through the identification of inactive enterprises and improved coverage of large and complex enterprises. - Further development and improvement of classifications and other standards. Since a broad range of national classifications was available, the project focused on their revision, in particular the Classification of Economic Activities and the Classification of Products by Activity. In the absence of a comprehensive glossary of statistical terminology, a multilingual glossary was to be developed as a component of the metadata repository. - Further development and improvement of sampling methods to address the NSO’s lack of experience in designing samples and to introduce sampling to replace full coverage surveys, wherever feasible. - Improvement of skills in seasonal adjustment to develop indicators, questionnaires, and techniques for seasonal adjustments to address seasonal variance of production and prices. - Improvement of skills in seasonal adjustment. The subcomponent will develop the indicators, questionaries’ and techniques for seasonal adjustments to address seasonal variance of production and prices. - Improvement of skills in evaluating and improving data quality. To support the NSO in data quality issues, a working group, with strong management skills, was to be formed, to assist and facilitate the introduction of procedures for self- and peer review included in the “change management”. Training was to be provided to local NSO staff to separate statistical and administrative duties, and accompanying guidelines were to be developed. - Improving questionnaire design procedures and developing Master Questionnaire by establishing a questionnaire design unit to prepare the Master Questionnaire with modules for all relevant sectors and to ensure consistency. 15. Component C. Upgrading statistical operations, i.e. improving separate programs, methodologies and the existing data outputs (US$0.62 million) aimed at enhancing survey and administrative data collection through reengineering and/or continual improvement, taking advantage of new infrastructure and incorporating international standards and best practices. In addition, it was to support a thorough review and inspection of existing data outputs. 5 - Full implementation of the System of National Account (SNA) to produce national accounts that are timely, reliable and in line with internationally accepted methodologies. - Price statistics to improve calculation of the Consumer Price Index (CPI) and the Producer Price Index (PPI) by enlarging the basket of goods covered and upgrading the methodology; introducing a customized program for the calculation of CPI by location; and preparing the PPI for major sectors. - 2010 Census preparation to focus on the main preparatory tasks identified in the framework of the 2010 Census and to address the following tasks: the preparation of a census training plan and program, census mapping, training of trainers, training of managers, supervisors, and enumerators, provision of equipment and software. 16. During appraisal, the Bank team approached donors and other development partners to obtain additional funding. In case of availability of extra funds, the project was supposed to include additional activities linked to the improvement of the statistical operations subcomponent in such areas as: the non-observed economy; migration statistics; industrial statistics; investment and construction statistics; environmental statistics; agriculture statistics; information, communication and transportation statistics; demographic statistics; labor force statistics; domestic trade statistics on goods and services, external trade statistics on goods and services; justice and crime statistics; tourism statistics; social statistics; poverty and living conditions monitoring statistics. 17. Component D. IT Infrastructure and Equipment (US$0.93 million) was designed to enhance IT infrastructure and equipment needed to implement the reforms envisaged under the project. In addition, consultancy services were to support the preparation of bid documents to ensure effective implementation of business process changes introduced by the project: - Preparation of an IT plan to support the development of a detailed computerization plan for the national statistical system that would address a thorough upgrading of the IT solutions and IT competencies in all data producing agencies including the NSO (headquarters, and regional offices), statistical units in line ministries and statistical units in state-owned enterprises. - Establishment of an IT network for the national statistical system to upgrade existing IT and other equipment essential to carry out existing regular and project activities. 18. In case of availability of extra funds, the project planned to widen its support by extending the IT network to regional and local offices. 19. Component E. Project Management (US$0.1 million) aimed to ensure effective project implementation and day-to-day project administrative work, including monitoring and evaluation. A financial management specialist and a procurement specialist were to be hired by the NSO. 1.6 Revised Components 20. There were no major revisions of components during project implementation. 6 1.7 Other significant changes 21. There were no significant changes during project implementation. No significant changes took place in terms of scope, implementation arrangements, schedule, and initial funding allocations between expenditures categories. At the same time, it should be noted that additional financial resources were generated from a Korea Trust Fund TF98487 – second grant - in the amount of US$0.65 million. The grant was specifically earmarked for developing specific areas of economic and social statistics. The Turkish International Cooperation Agency provided in-kind support to the project in the form of printing equipment, worth over US$0.3 million. The equipment was installed and became operational as observed by ICR mission in October 2014. 2. Key Factors Affecting Implementation and Outcomes 2.1 Project Preparation, Design and Quality at Entry 22. The project emerged from a long-term engagement between the Bank and the Government of Mongolia, which focused on enhancing the operations of the NSS. Prior to the MONSTAT project, the Bank had provided Technical Assistance to the NSO under six separate projects with overall funding of nearly US$2 million. This commitment, along with other donors and development partners’ support, provided a basis for initial reforms, but gains achieved needed strengthening. Given its mandate to promote statistical capacity activities, particularly in IDA countries as confirmed during IDA14 and IDA15 replenishment negotiations, and its substantial experience in assisting countries in statistical improvement programs, the Bank was well positioned to support the development of Mongolia’s NSO and coordinate donor efforts in statistical development. Additional funding for project implementation was attracted from Korean Trust Fund. Moreover, the project was prepared in close coordination with the Governance Assistance Project through its component on Capacity Building for Data Transparency and with the Data Access for Poverty Monitoring, and Economic Capacity Building Project. 23. Overall, the Project Development Objective was clear. The PDO was clear and relevant to Mongolia country’s strategy and circumstances, key ISN pillars, and the needs of the entire NSS and the NSO. The PDO appeared to be reachable within the planned project life of five years with all available funds fully disbursed. 24. The project was designed under the umbrella of the Statistical Capacity Building program (STATCAP), whose key objective is to support a sector-wide approach to statistics as defined in a nationally-owned NSDS, coupled with strong government commitment to its implementation. In this regard, the project was generally well-designed and aligned with the key objective of the adopted NSDS, namely the upgrading of conceptual, methodological and analytical capabilities of the national statistical system, while meeting basic requirements for monitoring the macroeconomic situation, the MDG indicators, and the other national development objectives. To achieve this objective, the project focused on (a) improving the policy, regulatory, and institutional framework in 7 statistics, (b) strengthening statistical infrastructure, (c) upgrading statistical operations, and (d) investing in physical infrastructure and equipment of the NSO. 25. The project preparation team based project design on lessons learned from similar operations and experience gained in Mongolia and other countries. Specifically they were: a) Statistical capacity building requires integrated and coordinated support: project interventions adopted a systemic approach, as defined in the approved NSDS developed in a participatory way, rather than relying on fragmented and uncoordinated programs as was previously done; b) Ownership of key stakeholders is essential to improved implementation of statistical projects: the project supported the implementation of the NSDS, which was prepared through an interactive and consultative process with government, including line ministries, Parliament, academia, the private sector and development partners. In keeping with this approach, project implementation included active involvement of a wide range of users and producers, their training, the strengthening of user-producer dialogue and data dissemination to stimulate a stronger demand for high quality data; c) There is no need to create a stand-alone project implementation unit, when there is strong commitment by the implementing agency. The project used the NSO as its executing agency, establishing the internal coordination unit and hiring procurement and financial management consultants; d) Improved quality and wide dissemination of data has a positive impact on the use of data: the project supported improvements in data quality, such as improving inter- institutional coordination to produce data, promoting the adoption of standardized concepts, definitions and classifications, and developing skills for data analysis. It also envisaged the development of a dissemination strategy; e) Focus on institutional reform, going beyond data collection activities, has greater impact on the statistical system: the project supported a review of the legal framework and capacity building of statistical staff in technical and management skills to ensure greater sustainability; f) Twinning arrangement between NSO and a well-developed national statistical office (or a consortium of such offices) could be a better approach instead of hiring consulting firms or individual consultants. The project opted for a twinning arrangement to assist the NSO in project implementation, particularly capacity building activities. 26. In order to achieve the objectives, the following implementation and governance arrangements were proposed. Institutional arrangements were designed to integrate project implementation into the regular operations of the NSO, the project implementing agency. A Project Coordination Team (PCT), consisting of NSO staff, with assistance from financial management and procurement consultants, was formed to coordinate project activities, manage reporting and auditing activities, and ensure compliance with procurement, disbursement and financial management policies and procedures. Other technical activities were the responsibilities of NSO staff who managed them with the assistance of local and international consultants. As the project aimed at improving the national statistical system, the appropriate level of commitment and support 8 was ensured from various government ministries and agencies. The National Statistical Council (NSC) assumed the oversight and monitoring responsibility for project implementation. The Parliament appointed NSC members to (i) ensure efficient interdepartmental (inter-ministerial) interaction and resolve interdepartmental (inter- ministerial) issues; and (ii) exercise monitoring and evaluation of MONSTAT project progress on a semi-annual basis. 27. Twinning arrangements for project implementation became a novelty of statistical project design and a key instrument for implementation. The Marrakech Action Plan for Statistics (MAPS) Advisory Board has recommended greater use of statistical offices of developed countries through twinning mechanisms to reduce the implementation risks of capacity building projects constrained by the lack of expertise and technical assistance. Based on this recommendation, the project envisaged that a twinning relationship be established between the NSO and one (or a consortium) of well-established statistical offices from developed countries. The twining partner(s) was supposed to carry out most of the consulting and training activities that the project would finance based on detailed Terms of References (TORs) prepared by the NSO. Advantages of such an arrangement would be (i) a significant decrease in the number of contracts to be signed, (ii) a lower risk of any wrongdoing and improper management of project funds, (iii) efficient outreach to statistical expertise and sustained engagement with experts, and (iv) a reduction of transaction time and costs of implementation. 28. The main risks and mitigation measures were appropriately identified. The overall risk rating at project design was Moderate – high likelihood, low impact. The main risks identified and their mitigating measures are presented in Annex 4, based on the PAD (Implementation section, part E, page 16; and Annexes 7 and 8). 29. Project preparation processes was well-managed and as a result was exceptionally fast. The MONSTAT full cycle of design, appraisal and negotiation was completed within twelve months between September 2008, when the project concept review meeting was held, and September 2009, when the project was declared effective. This impressive achievement in World Bank investment lending operation was reached because of strong support, commitment and well-coordinated work of the NSO, government authorities, and the Bank project team. 2.2 Implementation 30. Overall, the project was successfully implemented according to its objectives, realistic design and agreed implementation arrangements. Despite challenging operational environment (see below) the project not only largely met its targets, but also completed several innovations in Mongolian statistics, such as the transfer from SNA 1993 to SNA 2008, the introduction of seasonal adjustments, and the development and launching of a dissemination policy. The initial IDA funds increased by the end of the project due to a favorable exchange rate dynamic. All resources were disbursed in full within the original closing date. 9 31. The project was implemented according to the original design and did not undergo major change. It was always well implemented and has never been assigned “at Risk” status. The project was restructured in December 2013 after completion of 80 percent of its key deliverables. Based on the request from the NSO the financing plan has been changed. Undisbursed funds under “Unallocated category” were used to continue and complete the remaining tasks under all project components and expenditure categories. The mid-term review (MTR) mission was conducted in a timely manner. A MTR report (June 2012) was prepared and discussed according to Bank procedures. Its recommendations were taken into account by the NSO. Among them such as: (i) to start looking beyond the time frame of the existing project to cover new areas of statistics and make the case for additional resources to secure the achievement of MONSTAT results; (ii) to conduct user satisfaction surveys (planned for October 2012) following the World Bank guidelines and adjusts the results of the 2010 survey so that they are as compatible as possible; and (iii) to consider extending the responsibilities of PCT to cover all development partner support to NSO. 32. Project implementation faced minor start-up delays that were effectively addressed by NSO and project team. As is the case in many investment operations, the early phases of project implementation included minor delays, mainly caused by factors beyond the control of the implementation team. Specifically, setting up the twinning arrangement required more time than originally anticipated. An invitation for proposals was sent to twinning bidders in December 2009 and the twinning agreement was signed in August 2010, with a Consortium comprising the national statistical office of Germany (DESTATIS) and Korea statistical office (KOSTAT). While this twinning partnership was under preparation, the project activities (e.g. support for the 2010 Population and Housing Census) were financed by the World Bank TFSCB and the first Korean Trust Fund. Other implementation issues were associated with a turnover of PCT staff at the NSO, and later on with a several months delay in the approval of the 2012 Procurement Plan. However, these issues were identified on time and quickly addressed by both the NSO and the Bank team, thus ensuring a stable implementation pace that resulted in actual disbursements exceeding the originally planned ones throughout the rest of project life. 33. The project was implemented under challenging operational environment. There were a number of challenging factors that NSO and project team were facing: (i) a heavy workload for NSO staff that was conducting three censuses at the beginning of project implementation (Population and Housing Census in 2010, the 2011 Establishment Census and the Agricultural Census in 2012); (ii) very remote and dispersed locations of aimags with weak statistical units; (iii) limited project management capacity at the NSO due to workload (see item (i)) and limited prior experience in managing institutional development projects; and (iv) limited tradition of in-depth utilization of data by potential users, thus the need for stronger advocacy and out-reach efforts within government and society by the NSO. 2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization 34. Overall, the Results Framework (RF) was adequate for monitoring progress. The high-level outcome indicators either had target values or were rather accurately 10 descriptive. The intermediate indicators covered the whole range of project activities and were adequate in number. Baseline information that was missing before project launch was provided shortly after the project started. Most importantly, the RF included synthetic quantitative indicator to measure project achievements, i.e. the user satisfaction rate. The level and trend of this indicator captured improvement in relevancy of data produced, access to data, its accuracy and reliability as the survey included questions on simplicity of access to data, its accuracy, and comparability. In addition, the efficiency of statistical system performance is indirectly measured by the description of achieved improvement in physical infrastructure that ensures broader use of modern techniques that makes data collection and processing faster, and less subject of error. Use of tablets with direct access to integrated data systems reduces the data collection and processing time, and lead to workload per unit decrease, hence, the improved efficiency of statistical system performance. 35. Tracking progress towards the desired outputs and outcomes was covered by monitoring and evaluation activities. These included semi-annual project progress reports prepared jointly by the Project Coordination Team and the Consortium; meeting of the MONSTAT Consortium Coordination Committee; Aide Memoires of supervision missions; the preparation of Interim Financial Reports (IFRs), audit reports and their submission to the respective committees and governing authorities, and the Bank. As recommended by the Mid-Term Review, the project team and the NSO increased the focus on project outcomes in Aide-Memoires, progress reports and particularly in the last Implementation Status Report. 36. The targets set out in the Results Framework were met. By the end of the project, the target value of one of the PDO level outcome indicators was exceeded, three were fully achieved. It should be noted that the NSO produces statistical data entirely in line with the SDDS requirements for timeliness and periodicity. This was confirmed by the IMF SDDS evaluation mission conducted in October 2014. At the same time, the official subscription to SDDS depends on other government agencies, namely the Ministry of Finance (MOF) and the Bank of Mongolia (BOM). 2.4 Safeguard and Fiduciary Compliance 37. There were no safeguard policy issues with this project, rated as a category C. The project focused on institutional reforms and capacity building without any civil works and, thus, no environmental impact. 38. Fiduciary compliance. The MONSTAT project complied with all applicable Bank policies and was entirely consistent with the relevant Operations Policy Guidelines. Lack of capacity in procurement and financial management was identified during project preparation with a risk rating of Moderate. At project onset, performance varied, but generally improved during project implementation and remained constantly satisfactory. No mis-procurement was identified. The latest and final IFR was conducted in August, 2014 and no unusual issues were noted. Due to a failure to upload the IFR into the system, it has been recorded as overdue. 11 39. All legal covenants were respected as agreed in the Financing Agreement. 2.5 Post-completion Operation/Next Phase 40. Certain steps have been taken to maintain project benefits. Some project achievements have been institutionalized and could be sustained (particularly results from components A, B and C). The government has ensured that recurrent costs, and maintenance costs for the equipment acquired under the project are sufficiently provided for in the national budget. The NSO efforts to retain its staff paid off: the five year trend demonstrates that the number of those leaving the statistical office goes constantly down while the number of application to enroll is increasing on a stable basis. The NSO established the interdisciplinary sampling team to maintain the improved samples which were designed under the project for Household and Expenditure, Labor force, Time use, Average salary, Copying with disasters, Social indicators and other surveys. An inter- departmental data quality team was created to focus on further development of data quality issues and consistent implementation of the agreed measures for its improvement. 41. As for the next steps, the NSO identified areas for consolidating lessons and building on project achievements. It began this forward-looking dialogue with the Bank even before project completion. This proactive approach led to a US$5 million component in a Smart Government Project that became effective in 2015. The positive implementation experience and achievements gained by the NSO during 2009-2014 will be further strengthened during the implementation of this new project. 42. NSO is pursuing new options and is generating additional resources for the development of the NSS. As Mongolia is moving to mid-income country status, it is crucial that the NSO develop a new strategy to attract resources for its development needs. In April 2015, the World Bank approved another TFSCB grant (TF0A0396) to enhance NSO capacity and to implement the priority areas of the new NSDS. The NSO has also applied to the Korea TF for a Statistical Data Literacy and Education Center Project. Thus, further cooperation with the Bank and other partners could not only help the Government continue institutional reforms and fill financing gaps, but also be beneficial for the NSS as a whole. 3. Assessment of Outcomes 3.1 Relevance of Objectives, Design and Implementation Relevance of Objectives Sub-Rating: Substantial 43. The PDO remains relevant and consistent with Mongolia’s current development priorities and the Bank’s Country Partnership Strategy (CPS). The CPS for the 2013-2017 period strongly supports more transparent and citizen-oriented public management through better data and e-government (p.29/23), although the CPS does not envisage self-standing statistical operations in the pipeline or a repeater project. The NSO is taking the lead for the Open Data initiative, the largest component in the Smart 12 Government Project. A reliable and trusted statistical system is crucial for a well- functioning market economy, and thus the objective of the MONSTAT project remains valid. Moreover, the outcomes of the project have laid the foundation for the achievement of CPS Pillar I - to enhance Mongolia’s capacity to manage the mining economy sustainably and transparently – by introducing a separate Outcome 1.2 that aims at supporting “the government in designing and implementing policies and systems for a more robust, equitable and transparent management of public revenues and expenditures”. In its capacity, significantly strengthened by the project, the NSO is better equipped to support the achievement of the CPS strategic objective of improving the quality of public services, strengthening accountability to service users, and targeted assistance to the poor and vulnerable. 44. Adapted Global Assessment of the National Statistical System of Mongolia emphasizes MONSTAT achievements and timeliness of its objectives. This comprehensive assessment was initiated in August 2014 at the request of the NSO, and jointly undertaken by the European Free Trade Association (EFTA), the United Nations Economic and Social Commission for Asia and the Pacific (UNESCAP) and the United Nations Economic Commission for Europe (UNECE). It acknowledges the positive impact of MONSTAT in several areas and provides in-depth recommendations on further improvements for the NSS and the NSO. It summarized that “the on-going process of building a modern NSS in Mongolia is on a fruitful path and that the NSO is playing a leading role in this encouraging development.” 45. New NSDS for 2015-2020 is being finalized. Currently the NSO is finalizing the third national statistical strategy before submission to the Parliament for approval. Relevance of Project Design and Implementation: Substantial 46. Project design was consistent with the PDO and targeted critical areas for achieving project objectives. The selection of sub-activities was notably relevant insofar as it was based on the NSDS diagnostic. Project design was informed and based on the long-term engagement and technical support provided by the Bank and other development partners. Technical assistance provided during project preparation addressed the most pressing needs and ensured consistency in the design of the project. 47. The actual implementation of the project was relevant to the achievement of the PDO. First, the decision to assign project implementation to the NSO contributed to building overall experience and capacity of the NSO, assuring project ownership, and strengthening the NSO internally, all of which contributed to achievement of the PDO. Second, the introduction of the twinning arrangement with an external Statistical Office was a positive initiative in attaining the PDO, in terms of knowledge sharing, direct access to relevant processes and procedures, and experience exchange. Third, the Bank’s decision to transfer the Task Team Leadership to the field led to stronger implementation and better client orientation of project supervision, ensuring PDO is fully supported. Overall Relevance Rating: Substantial 13 3.2 Achievement of Project Development Objectives Attribution and Results Measurement 48. To assess the extent to which the MONSTAT project achieved the PDO level results, the causal relationship between project inputs, outputs and outcomes were reviewed, (see Annex 2, Figure 2.1). It is difficult to attribute achieved results with full certainty, especially in institution strengthening and capacity building projects and small projects. Since at the time of project implementation there were no other major initiatives or significant donor-funded activities in statistics, and MONSTAT was designed to cover new domains of statistical developments compared to the previous projects, results assessed here can be most likely associated with the project. Still, in assessing results, the Implementation Completion and Results Report (ICR) team considered only those results that can be causally attributed to project activities. 49. The project has achieved its development objective. Annex 2 presents key project outputs in more details by sub-components. Project high level achievements and intermediate outcomes can be outlined as follows: - Data user satisfaction rate reached a level of 71 percent compared to target value of 60 percent, as documented by the final round of user satisfaction survey, conducted and completed in October, 2014. - Statistical products are easily accessible. Statistical data dissemination system substantially improved, it is newly developed and operational (data, including micro data, are available through www.1212.mn, smart phones and tablet applications such as MONSTAT and EzSTAT, thematic brochures, pocketbooks, monthly infographics, 19001212 hotline service for statistical inquiries, re-established Statistical library for users, NSO website with improved design and other regular publications). Training for media to raise awareness on statistics was conducted. - Internationally-accepted standards and classifications applied to most statistical areas, skills in authenticity verification and regular validations improved. The project introduced data quality management based on EU quality management practices, piloted and developed data quality reports for some statistical areas. Four international classifications 1 and 11 methodologies were developed based on international standards and applied, including ISIC 4.0 and SNA 2008. A master questionnaire was designed and is being used to update other surveys. Sampling design for household based surveys was improved. This resulted in better poverty measurement. Since 2012, NSO is calculating a poverty line which is consistent with international estimates. Business register system was 1 Classification of Assets that incorporates Classification of financial and non-financial assets”, 2013 (SNA 2008); “Central product classification”, 2011 (UN Central Product Classification (Version 2.0); Classification of the purposes of non-governmental organizations”, 2011 (UN Classification of the Purposes of Non-Profit Institutions Serving Households); ISIC 4.0 is used for GDP, IOT and SUT, and updated classification 3.0 to ISIC 4 for all economic sectors. 14 created and enriched. Consequently, the project contributed to conducting 14 surveys with improved verification and quality check-ups. Revision policy was adopted following the improved classification and standards. All these led to better data coverage, accuracy, reliability - overall, production of higher quality data. - Mongolia is joining the SDDS. An IMF mission concluded that the NSO has met almost all technical requirements to enter the SDDS. As stated in the IMF evaluation report regarding timeliness and periodicity, the NSO produces all data according to the SDDS requirements. Official subscription of Mongolia to the SDDS depends on data to be provided by other government agencies, such as MOF and BOM. IMF agreed with MOF and BOM on an action plan to fully comply with the SDDS. More specifically on disclosure of the central government debt data and on the development and improvement of the international reserves data. According to IMF, this could happen by the end of 2015. - Improved inter-agency coordination and strengthened user-producer dialogue. The NSO created a department for policy implementation and coordination, which is responsible for working with other producers of official statistics providing methodological guidance for the entire NSS. It also signed cooperation agreements with 9 ministries and agencies to improve coordination and data sharing 2. Training was conducted for 120 staff from these government agencies. Customer management system is in place. Monthly press conferences are held with key media agencies. Training for media on how to use statistical information was conducted. Statistical library for users was re-established in 2013. The hotline service – 19001212 became functional. Center for Information, Training and Advocacy (CITA) was established in 2013 and extended training to line ministries and other government agencies, and data users. Customers’ satisfaction survey is conducted once in two years with all reports placed on the NSO website. - Human resource capacity upgraded. A newly established NSO department for policy implementation and coordination was assigned a responsibility to strengthen the local offices. At local level, the NSO succeeded in establishing a position for a full-time civil servant with a mandate for data collection (treasury fund officer at soum level). CITA provides professional training programs of four levels to NSO staff. In total, under the project 4647 staff received in-country training, including 1578 staff from bags and khoroos, and 334 staff from regional offices; 406 staff (180 from regional offices) were trained in EU countries. The NSO is now recognized internationally by being elected as a member of UNSC and expert group of UNESCAP. NSO Mongolia is a co-chair of UB City Group for economies rich in natural resources. Currently, the NSO started hosting the knowledge sharing and best practice learning visits from the national statistical offices of other countries. - Statistical infrastructure developed. Comprehensive business register database was 2 Ministry of Population Development and Social Protection, General Authority of State Registration, Bank of Mongolia, Ministry of Finance, Ministry of Economic Development, Ministry of Environment and Green Development, Ministry of Culture, Sports and Tourism, Ministry of Health, and Customs Authority. 15 created, business register management system was developed and became operational as well as the online database system for population and household surveys. Four international classifications and 11 methodologies were developed and applied, including ISIC 4.0 and SNA 2008. The sampling design for household surveys and poverty assessment was improved. Standard master questionnaire was developed and used for surveys. 14 surveys (SNA, CPI, NOE, HSES, Agriculture census, Establishment census etc.) were conducted based on international standards, and results were disseminated. - Physical infrastructure substantially modernized. IT network security system was installed; it is fully operational. Data dissemination system developed and also fully operational. All staff at NSO and regional offices (compared with target value of 75 percent and 60 percent correspondingly) have workstations with direct access to databases for tabulation and analysis. Tablets are used for data collection, resulting in cost-efficiency of the agency operations. 50. Some specific project achievements could be highlighted as follows: - Data User survey (Customers satisfaction survey) conducted regularly; - User-friendliness increased; - The 2010 Population and Housing Census and 2011 Establishment Census data produced quickly, of good quality and linked to GIS (geographical information system); - Contribution to the first Agricultural Census conducted in 2012; - Compilation of Supply and Use table and publication of seasonally-adjusted, quarterly GDP at current and constant prices, estimated according to both production and expenditure approaches, since 2011; - Management issues studied and being introduced in NSO; and - Improved English-language proficiency. 51. Achievements in institutional development paved the way for a more successful implementation of the new NSDS. The project demonstrated important progress in strengthening the legal framework for statistics. Moreover, amendments to the law are being drafted in line with the recommendations of the MONSTAT project and the Adapted Global Assessment. 52. The project has raised awareness and interest in statistics through a number of important measures including: (i) an increased number of publications; (ii) a new dissemination policy; and (iii) modernizing of the NSO website, including a powerful data portal with micro data sets and etc. 53. What could not have happened without the project? While the counterfactual for the project cannot be easily constructed, there are a number of developments and processes that the client and team believe could not have happened if the project would not be in place. More specifically, during the ICR mission the team recorded the following 16 client views on MONSTAT project impact: (i) the pioneering/pilot nature of project work for NSO staff under all four project components that has eventually generated a catalytic effect and appetite to continue moving forward; (ii) twinning arrangements opened the door to international cooperation promoting the NSO to play more meaningful role in the international field; (iii) establishment of NSO-user relationship that has not been a regular practice before; (iv) accelerated transition to 2008 SNA; (v) moving towards full compliance with the SDDS; (vi) broad dissemination of statistical products – appropriate policy design, creation of information, Training and Advocacy Centers, and proper equipment of aimag level offices for this purpose; (vii) convening role of the project that included many Ministries in its activities and facilitated improved inter-agency coordination (viii) significant improvements in price statistics. Achievements of Objectives Rating: Substantial. 3.3 Efficiency Efficiency Rating: Substantial 54. As stated in the PAD, the project is not amenable to a cost-benefit or economic rate of return analysis. National statistical offices are not involved in any cost-recovery activities, apart from marginal data publications that generate minor amount of revenue. The financial returns from this project will not be representative of the economic returns. Precise quantitative rates of return cannot be determined for this type of project. The ICR team judges overall efficiency to be substantial. As highlighted by the MTR report, the project achieved its targets through efficient use of resources and investments. All planned activities were completed with no project extension. Early delays in project implementation were efficiently resolved with the joint effort of the NSO and project team, resulting in full disbursement in a timely manner. Broadly, the economic benefits from the project could be based on the following: (i) improved efficiency of statistical operations of the NSO, resulting in a broader coverage of and higher quality data that affect multiple sectors of society; (ii) better quality data will enhance the potential for evidence-based decision making, at policy, program, and project levels; and (iii) a significant reduction in the cost of data collection through improved systems as well as by moving from full count censuses to sample surveys. 55. As mentioned in the MTR report and supported by the ICR team: “the overwhelming impression, given by those interviewed, is that money is being used efficiently on targeted activities that are achieving good outcomes in terms of capacity built and statistics produced and disseminated; and that money has been used appropriately with consideration for value for money and strict spending controls”. 3.4 Justification of Overall Outcome Rating Rating: Satisfactory 56. The project has helped to improve efficiency and effectiveness of the national statistical system of Mongolia to enable it to provide relevant, timely and reliable data 17 for evidence-based policy. The project was a relevant and timely intervention. The improvement of the efficiency and effectiveness of the statistical system has been achieved as evidenced by the production and dissemination of better quality and timely data in all spheres of economic and social statistics, particularly in national accounts, industry, transport, construction and agricultural statistics. The training of statisticians and the adoption of new techniques and methods have improved data accuracy, timeliness and coherence. The statistical system has become better coordinated, stronger and more dynamic. Despite minor implementation issues at the early stages of the project, it triggered the long-term reform process, increased data production, and sped up several data collection processes on the ground. It also successfully overcame the minor issues related to financial management and procurement. In terms of PDO relevance (Substantial), achievements of the PDO (Substantial), efficiency (Substantial) the project performed at a level consistent with a satisfactory rating. 3.5 Overarching Themes, Other Outcomes and Impacts (a) Poverty Impacts, Gender Aspects, and Social Development 57. The project prioritized SNA, CPI and PPI statistics and did not specifically focus on social objectives, but had an overall positive impact on the latter by improving the overall capacity of the NSO and sub-national statistical bodies to produce relevant, timely data, based on sound methodologies. The project had only an indirect impact on poverty reduction by improving the poverty measurement methodology and introducing international standards. Since 2012 agreement was reached on a single measurement of poverty: the poverty line is calculated on internationally accepted method0logy. Local (Aimag – municipal) level poverty indicators are estimated since 2013. The project also assessed the existing gender indicators framework and provide recommendation for its development based on best international practice. By providing a steady supply of reliable and timely social and economic statistics, the NSO will contribute to policy making and monitoring processes in the country. (b) Institutional Change/Strengthening 58. The NSO has strengthened its institutional capacity. This could be supported by the following evidence: (i) harmonization of statistical legislation with international recommendations (UN Fundamental Principles of Official Statistics, EU Code of Practices), which provided the basis for the institutional change of the whole statistical system, including strengthened collaboration between the NSO and other relevant government agencies; (ii) modified organizational structure (by the end of the project and immediately after the NSO created a special division for Survey, Censuses and Sampling, and a department for Integrated Policy and Cooperation and Dissemination Policy unit, a Quality group); (iii) improved management capabilities; (iv) introduction of several internationally-accepted classifications; and (v) significant upgrade of staff expertise and skills. 18 (c) Other Unintended Outcomes and Impacts (positive or negative) 59. No negative unintended outcomes were observed. The project generated positive unanticipated impact for the private sector and population. Creation of integrated public database that comprised the comprehensive and updated household data and livestock information made the access to credit/loan resources easier and less time consuming. In cases of applications for loans and financial assistance the banks and other financial institutions can directly access the database, make assets assessment and make quick decisions on provision of loan funds to the applicants. 3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops 60. Overall project implementation and results assessments provided during the ICR missions were extremely positive. The final workshop of the MONSTAT project was organized in October 2014, with the participation of key stakeholders, including government, parliament, consortium, development partners, media, academia and other users. During its two visits to Mongolia, the ICR team met the direct beneficiaries of the project in Ulaanbaatar, and in regional offices (aimag and lower levels). The interviewed beneficiaries of the project considered that the project was timely and important in bringing the statistical system of Mongolia up-to-date with internationally-accepted standards; that it positively influenced their own professional development and skill sets which, in turn, built their confidence in their own capacity to operate in a more efficient way. The project helped increase the reliability, timeliness and quality of statistics produced in the country, and, hence, bolstered trust in the statistical system. Beneficiaries also stated that the capacity of their respective agencies to collect, process, disseminate data had significantly improved, and they would be able to sustain these gains in coming years. However, they also mentioned that there will be a need for future support to address new development needs and to strengthen expertise. 4. Assessment of Risk to Development Outcome Rating: Low or negligible 61. The prospects of maintaining the project’s development outcomes are strong, especially in the short to medium term. Some of the project’s achievements have already been institutionalized both at the center and aimag levels. The NSO leadership has shown a very strong commitment during the project implementation. The Chairman was unanimously reappointed to another six-year term. The objectives of the project remain relevant. There are post–operational steps that have already been taken (section 2.5). The NSO is actively engaged in the implementation of the statistical component (US$5 million) in Smart Government Project. 62. Factors in favor of greater sustainability: (i) NSO focus on institutional reform, not only data collection activities; this has greater impact on the statistical system (the project supported a review of the legal framework and capacity building of statistical staff in technical and management skills). 19 (ii) Interactive and consultative process with government, academia, the private sector and development partners: in keeping with this approach, project implementation aimed at an active involvement and training of a wide range of users and producers, the strengthening of a user-producer dialogue, and improved data dissemination to stimulate a stronger demand for high quality data, that in turn, contributes to sustainability once external support stops. (iii) International exposure: Mongolia has become an elected member of the UN Statistical Commission (one of only four countries representing Asia). Since 2012 the NSO is leading the “Ulaanbaatar city group on statistics for economies based on natural resources”. These are important initiatives for the NSO to assert its authority and influence internationally in developing relevant new approaches to measuring the contribution of natural resource-based activities and their impact on social sectors and the environment. (iv) Active NSO engagement in partnerships: with, for example, Finland, Germany, Korea, and Norway. 63. Some risks were identified that could potentially limit the prospects for the gains to be expanded over the medium term. Although the government and the Parliament have expressed a strong interest in developing the national statistical system, budgetary allocations for the NSO are stable for regular operations, and staff salaries are increasing, lack of funds for development needs remains an issue. Additional risks include: (i) staffing constraints - the small number, age and experience of NSO staff relative to the scope of activities being undertaken and planned, and the danger of losing staff; (ii) low level of knowledge absorption capacity; (iii) loss of access to international experts after project closure; (iv) reduction of grant financing options due to Mongolia’s recent middle income status. 64. Taken as a whole, the risk is judged to be low with a potential to expand to moderate in the absence of longer-term corrective actions. 5. Assessment of Bank and Borrower Performance 5.1 Bank Performance (a) Bank Performance in Ensuring Quality at Entry Rating: Satisfactory 65. Overall, the project team had an appropriate skill mix, substantial country experience and strong leadership. Technical aspects of the project were well covered, and a solid business relationship was established with the government and the NSO. Attention was given to involve partners and stakeholders in the preparation of the project. Lessons from earlier operations and projects, and complementarities with ongoing initiatives were taken into consideration. Project activities were fully in line with overall country needs and development objectives. Risks were identified in a timely manner and pro-active mitigation measures introduced. A decision to go for a twinning arrangement was made and implemented. 20 (b) Quality of Supervision Rating: Satisfactory 66. The composition of the Bank team remained stable and adequate throughout the project implementation. The responsible FM specialist, the Procurement Specialist resided in the Bank country office. After the mid-term review When the Task Team Leadership was moved from World Bank Head Quarter to the country office, project supervision was successfully carried out by a combination of co-leaders. This decision permitted regular project follow-up and the quicker response to address any necessary action, thus ensuring continuous monitoring and guidance. The Bank’s fiduciary and procurement team also provided constant and regular advice to the NSO team on grey areas, especially at the beginning of the project. This enabled the project team to identify potential errors in a timely manner, and/or address issues immediately. The Implementation Status Reports (ISRs) were prepared with sufficient detail and on a timely basis. The Bank’s performance did not have any shortcomings throughout the duration of the project. (c) Justification of Rating for Overall Bank Performance Rating: Satisfactory 67. In light of project outcomes, relevant project preparation and quality of intensive supervision, the overall performance is assessed as satisfactory. The team recognized the complications at early stage of implementation. Its intensive and continued supervision efforts and proactive actions allowed for achieving significant results, the completion of nearly all envisaged project activities, and full disbursement of project funds without any extension. 5.2 Borrower Performance (a) Government Performance Rating: Satisfactory 68. From the very beginning, the borrower’s ownership and commitment were consistent and strong. The NSO operates under authority of the Parliament, which approves the NSDSs and the NSO’s work plans. The Parliament recognized the results achieved through the implementation of the first NSDS for 2006-2010, whose goals were met with support from MONSTAT in 2009 and 2010. The second NSDS was approved by Parliament in December 2011. Government demonstrates continuing support as well through regular budgetary allocations. Other line ministries and agency collaborate closely with the NSO. (b) Implementing Agency or Agencies Performance Rating: Satisfactory 21 69. Despite its heavy workload to carry out the 2010 Population and Housing Census, the 2011 Establishment Census and the 2012 Agricultural Census, NSO staff managed to cope with project start-up delays and maintained a constant momentum throughout project. The NSO successfully managed the turnover of project staff that took place at the beginning of project by engaging a competent and knowledgeable Project Coordinator and experienced coordination team that successfully managed the procurement and financial management aspects of the project. The financial management system was run by the NSO (project accounting was separate from that of the NSO) and performed satisfactory during project implementation. It recorded all transactions and balances and allowed the preparation of regular financial statements that were submitted to the Bank on time. All procurement of goods and services were conducted in line with the provisions of the Financing Agreements and Project Procurement Plans, approved by the World Bank. No procurement compliance issue was raised during the project implementation. 70. Once the twinning partnership went on track, the NSO learned from best international practices and has benefited from a sustained engagement with experts and partners. Later on the decision was made to extend the responsibilities of PCT to cover all development partner support to the NSO. 71. The progress made suggests that the NSO is responsive to the advice received from consultants, e.g. on institutional and organizational issues and human resources, as well as in technical areas. This responsiveness is also evident in the NSO’s acceptance to implement recommendations from the MTR and the adapted Global assessment report, as well as to continue statistical reform in principle. (c) Justification of Rating for Overall Borrower Performance Rating: Satisfactory 72. Given achievements and results, the overall borrower performance is rated as satisfactory. 6. Lessons Learned 73. Assignment of implementation role to the NSO (rather than establishment of a separate Project Implementation Unit) was a strategically appropriate decision. The arrangement contributed to (i) project ownership by the key government agency, (ii) capacity development, and helped (iii) mainstream project lessons in the NSO’s regular operations. The team assisted the process by recruiting properly trained procurement and Financial Management specialists to meet implementation requirements. 74. Advantages of twinning partnership. Using the twinning arrangements under MONSTAT project proved to be very beneficial to overall project implementation. More important, the twining partnership ensured outreach to advanced statistical expertise and sustained engagement with international experts during the project life and after its 22 completion. On the operational side, the twinning arrangement allowed for: (a) increasing the number of activities by maximizing consolidation, while limiting the number of procurement packages; (b) simplifying implementation arrangements by ensuring high level oversight with a dedicated project team to manage the various components; (c) providing just in time implementation support for the procurement of complex IT systems; (d) designing a communication plan in advance in order to manage expectations, (e) lowering the risk of any wrongdoing and improper management of project funds, and (f) reducing implementation transaction time and costs. 75. Combination of good preparation planning, strong policy dialogue on the Bank side and project design fully reflecting the country needs resulted in strong commitment on the borrower side and exceptionally fast project preparation and movement from identification to effectiveness. 76. Bundling of TTL-ship at HQ and local WB office leads to effective project management. The decentralization of the TTL-ship to the field after mid-term review helped to address promptly the delays, and proactive overall project implementation. At the same time, presence of the co-TTL at HQ secured broader access to additional technical resources for project supervision and access to supplemental donor financing. Overall, this arrangement for project supervision facilitated successful project implementation (developmental - progress to PDO; and financial – full disbursement) within the original timespan with no extensions. 76. Starting from project design stage, a good balance between twinning arrangements, engagement of local expertise, and leading role of the NSO was found and maintained throughout project life. 77. Early consideration was given to organization of interpretation and translation services. Project implementation was facing a challenge on translation and interpretation side. Because of lack of knowledge of foreign language among most of project participants, particularly at local level, and broad involvement of foreign speakers in training, this was a constraint. Large volume of interpretation and translation services were delivered, ensuring proper communication of technical knowledge, building skills and establishment of strong rapport and understanding. 78. Successful implementation of the MONSTAT project highlighted that broadening of the concept of the National Statistical System goes far beyond the NSO per se and is essential for the development of the national statistical service. The project strongly supported a user-producer mechanism, comprising of the NSO, regional statisticians, and statisticians in line ministries and government agencies to discuss coordination, common statistical issues and challenges in the sector, through working groups and regular seminars. 79. Early engagement of procurement specialist had a positive impact on finding the best solutions to guiding project preparation and implementation. 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners 23 (a) Borrower/implementing agencies N/A (b) Co-financiers N/A (c) Other partners and stakeholders (e.g. NGOs/private sector/civil society) N/A 24 MONGOLIA MONSTAT: Strengthening the National Statistical System of Mongolia Annex 1. Project Costs and Financing (a) Project Cost by Component (in USD Million equivalent) Component Name IDA Korean TF TFSCB 2nd Korean TF Actual/ % of Actual/ % of % of Actual/ % of Appraisal Appraisal Appraisal Actual/Latest Appraisal Latest Appr Latest Appr Appr Latest Estimate Estimate Estimate Estimate Estimate Appraisal Estimate aisal Estimate aisal aisal Estimate A Improving policy, regulatory, and institutional framework and the organizational structure of the statistical system, A1 The review and improvement of existing $60,000 $172,352.62 287 $60,000 $75,777.15 126.3 $20,000 $7,253.82 36.3 institutional framework; A2 Improving the relationship between data users and $60,000 $169,773.90 283 $30,000 $103,659.93 345 $10,000 $10,338.72 103.4 producers; A3 Improving the data dissemination function and $80,000 $45,323.24 57 $20,000 $64,200.66 321 $20,000 $7,949.45 397 procedures; A4 Strengthening human $60,000 $86,651.18 144 $30,000 $114,263.39 330 $20,000 $73,410.90 367 resource capacity; A5 Review and improvement of organizational structure of $60,000 $240,051.83 400 $20,000 $101,897.42 509.5 $20,000 $34,558.41 172.8 the national statistical system. 25 Sub total $320,000 714,152.77 223 $160,000 459,798.55 287.4 $90,000 $133,511.30 148.4 B Strengthening the statistical infrastructure, B1 Improvement and updating $110,000 $100,475.13 91 $40,000 $24,625.62 61.6 $20,000 $21,588.14 108 of statistical registers B2 Further development and improvement of $60,000 $41,170.64 68.6 $50,000 $45,542.00 91.1 $10,000 $14,123.35 141 classifications and other standards B3 Further development and improvement of sampling $60,000 $32,294.22 54 $40,000 $0.00 0 $20,000 $17,931.51 70 methods B4 Improvement of skills in $60,000 $24,867.42 41.5 $40,000 $3,262.40 8.2 $20,000 0 0 seasonal adjustment B5 Improvement of skills in evaluating and improving $80,000 $57,421.28 72 $30,000 0.00 0 $20,000 0 0 data quality B6 Improving questionnaire design procedures and $80,000 $14,774.76 18.5 $40,000 0.00 0 $20,000 0 0 developing Master Questionnaire. Sub total $450,000 $271,003.45 60 $240,000 $73,430,02 30.6 $110,000 $53,643.00 48.8 C Upgrading statistical operations (improving individual subject matter programs and methodologies as well as the existing data outputs), C1 Full implementation of SNA $100,000 $338,936.33 339 $60,000 $29,326.49 48.9 $10,000 $50,465.77 505 C2 Price statistics (including PPI, CPI and other relevant $80,000 $64,104.16 80 $60,000 $21,091.69 35.2 $10,000 $56,453.67 565 indices) and BOP statistics C3 2010 Census preparation $150,000 $4,051.65 2.7 $60,000 $94,774.29 158 $90,000 $18,717.02 24 C4 Economic statistics $207,837 $214,666 33 C5 Social statistics $442,163 $435,334 67 Sub total $330,000 $407,092.14 123 $180,000 $145,192.47 80.7 $110,000 $125,636.46 114 $650,000 $650,000 100 26 D IT Infrastructure and Equipment D1 Preparation of an IT plan $70,000 $106,805.14 153 $20,000 21578.96 108 $10,000 $86,775.24 868 D2 Procurement of computers and other equipment and $650,000 $485,527.25 75 $100,000 0 0 $80,000 $434 0.54 establishment of an IT network Sub total $720,000 $592,332.39 82.3 $120,000 $21,578.96 18 $90,000 $87,209.24 96.9 E Project Management. E1 Consultants (Procurement and FM specialists, ad hoc $60,000 $143,554.34 239 advisory services, translation) E2 Audit Services $20,000 $13,449.68 67 E3 Goods (office equipment, $5,000 $1,987.44 40 software, furniture etc.) E4 Office Operating and $15,000 $9,400.14 63 Maintenance costs Sub total $100,000 $168,391.60 168 Unallocated $80,000 Total Project Cost $2,000,0 $2,152,972.3 108 $700,000 $700,000 100 $400,000 $400,000 100 $650,000 $650,000 100 00 5 The project cost in this table does not include in-kind contributions from TICA (estimated around $300,000). 27 (b) Financing Source of Funds Type of Cofinancing Appraisal Estimate Actual/Latest Percentage of (USD millions) Estimate Appraisal (USD millions) (%) Borrower 0.00 0.00 0.00 International Development Association (IDA) 2.00 0.00 100.00 Korea Trust Fund 0.70 0.00 100.00 Second Korea Trust Fund 0.65 0.00 100.00 Trust Fund for Statistical Capacity Building 0.40 0.00 100.00 Turkish International Cooperation Agency In kind* 0.30 0.00 100.00 *Installed and operational as observed by the ICR mission in October 2014. 28 MONGOLIA MONSTAT: Strengthening the National Statistical System of Mongolia Annex 2. Outputs by Component Table 2.1. Outputs by Component Component Project outputs (under the contract with Consortium) Project outputs and activities implemented by the NSO A. Improving policy, regulatory, and institutional framework and the organizational structure of the statistical system A.1 The review • Evaluation of the existing statistical institutional framework. • Intensive advanced level English training for selected and • Study tours to explore statistical institutional frameworks of employees from different departments. foreign countries (Germany). • Elaboration of strategies to introduce rotation of employees. improvement of • Development of a strategy to improve the statistical system in • Development of Concept of human resource development. existing • National strategies for development of statistics 2011-2015 framework of international standards and preparation of a general institutional overview of a sound statistical system; development of a statistical approved in 2011. Approval of Action plan to implement NSDS framework system model applicable in Mongolia in May 2012. • Development and strengthening human and technical resources • Memorandum of Understanding signed with ministries and and facilities that meet the requirements of timeliness and quality of agencies. primary data and provide consolidated management and methods • Joint working groups established and action plan developed. for the statistics staff at all levels. • User seminar in 2011 (30 participants). • Training for the staff of all levels statistical system organized in 2011 (38 participants). • Regional training (21 aimag, 329 soums and 9 villages, 334 statistical officers). • National Statistical Workshop in 2010 and 2012 with 418 staff from NSO, capital /aimag statistical department and divisions and district statistical divisions. • Statistical methodological guideline (2160) published. • Training for bag and khoroo statistical officers organized in 2012 (20 aimag, 1578 officers). • Training in administrative statistics. Training in official and administrative statistics and common understanding of statistical 29 Component Project outputs (under the contract with Consortium) Project outputs and activities implemented by the NSO methodology among government staff responsible for statistics. Statistical methodological guideline published (2160 pieces). • Adapted Global Assessment. • AGA in 2013-2014 that contribute to evaluation of level of conformity with accepted international and European standards, incorporating the UN Fundamental Principles of Official Statistics and European Statistics Code of Practice and defining priorities for future international cooperation programs. • Review of Law on Statistics to better incorporate UN Fundamental principles of official statistics and the Mongolian Code of Practice, reflect changes in the Mongolian society and better satisfy users’ needs. A.2 Improving • General user requirements analysis and use the results for the • User satisfaction surveys in 2012 and 2014. The reports the relationship improvement of the relevant subject matter areas for their program produced and disseminated. improvements. •Training for journalists/media people in 2010 (45 between data • Design and assistance in conducting a user satisfaction survey to representatives). users and get an impression of changes in user opinions. • User seminar in 2011 with 30 participants. producers • Organization of regular consultation workshops and seminars • Training for staff of ministries and agencies in 2011 (22), 2012 upon releasing statistical publications to explain the results and the (40) and 2013 (58). In total 120 representatives from ministries use of principal indicators. participated in various trainings. • Staff training in line ministries in data analysis (realized in a • Statistical library was re-established in 2013. separate bilateral activity, financed by the German Association for international co-operation (GIZ). Design of a questionnaire for a marketing survey of user needs A.3 Improving • Development of a comprehensive and integrated dissemination • Customer management system introduced at NSO. the data strategy and publications policy covering all printed reports and • Customer relations policy approved in 2014. web data that is in line with the international standards for data • Two seminars by Destatis experts in 2011 (45 staff from NSO). dissemination dissemination. • NSO introduced common corporate standards of design and function and • Dissemination of data that was not previously published through color for all products of NSO such as yearbook, bulletin, procedures new annual and monthly publications. brochures, census results reports, business cards, presentations and • Gradual development of a consistent style and layout for the website. various publications. • NSO‘s Data Dissemination Calendar includes a time of • Preparation and implementation of a marketing strategy, so that dissemination -11 am of the approved date of the month. 30 Component Project outputs (under the contract with Consortium) Project outputs and activities implemented by the NSO fees can be collected via the internet for access to electronic • Applications for Smart phone users developed (Monstat, Ezstat). publications and datasets. These applications have been installed to cell phones and • Workshop on data dissemination strategy with main focus on the computers of assistants of MP. Monstat application now has 206 publication of the census results to improve content and design of books and printed products. In the 1st half of 2014, 14 new printed the website and introduce programs and upgrade software materials have been added. applicable to national specifics. • In order to enter SDDS, the IMF was requested to carry out assessment. • Marketing and promotion made through website; TV, emails, newspapers and sending messages to the users of cell phone service of Mobicom (with the cell phone numbers starting 9911, 9909). 11,000 users of cell phone service of Mobicom company have obtained information on statistical products. • A number of promotions developed in the forms of TV advertisements, printed leaflets, and brochures and in infographic formats. • NSO website improved, with more features and news, websites of aimags are in progress of renewal reflecting a single corporate design. • Metadata, microdata from several surveys, electronic versions of published products such as yearbook and monthly bulletin placed on the NSO website. • www.1212.mn statistical dissemination website launched in 2012. www.1212.mn was updated with more data at aimag/capital city, soum/district and bag levels. • Online service to users is reaching the access of 187,157 calls in the first 6 months of 2014. • Offline version www.1212.mn (Devinfo) developed. • NSO approved the list of indicators to be produced at local levels (soum/district and bag/khoroo) in 2014-2016. Accordingly, NSO will produce 346 indicators (previously 315). A.4 • Assessment of the state-of-play and the needs of human resources • Independent subdivision of Civil Service Council established Strengthening development. specifically for recruitment of statistical officers. This new • Development of human resource management strategy that structure enables recruitment of qualified people with relevant includes performance management, career development, staff statistical education and experience. 31 Component Project outputs (under the contract with Consortium) Project outputs and activities implemented by the NSO human resource assignment and staff rotation. • Center for Information, Training, and Advocacy established in capacity • Study visit to Germany to experience practice in developed 2013. countries and get involved with on-the-job training. • Training needs evaluated. Center for Information, Training • Study visit to South Korea to develop the HRM strategy. and Advocacy designed a curriculum and 3-level training • Study visit to Germany for Project Planning and Project Financial modules based on this evaluation. Management Training. • In 2011-2014 the following trainings provided to build capacities in addition to trainings in specific statistical areas: • Foreign relations training in 2011 (6 staff of Department of International relations and cooperation). • Quality management training in 2011 (20 NSO staff). • Training for the staff of all levels of statistical system in 2011 (38 participants). • Management and leadership training in 2012 (6 people) • A seminar on the Action plan of official statistics development for 2011-2015 in 2012 (20 staff). • National Statistical Workshop in 2010 and 2012 involving 412 staff from all levels of statistical offices and divisions. • Regional training (21 aimag, 329 soums and 9 villages, 334 statistical officers) • Training for bag and khoroo statistical officers: 20 aimag, 1578 officers. • In 2013 NSO jointly with the National University of Mongolia organised the 2-level professional training for NSO staff and staff from aimag statistical offices on social and economic statistical methodology and technology applications. (35 staff from NSO and other ministries and agencies). • 5-level English training in 2011-2013 involving 312 learners (198 ponline and 114 classroom training). • Printed and disseminated professional statistical books and manuals (9 types). • The Monstat project resource centre developed, shared within NSO management and maintained. • Monstat project website developed. • Local trainings organized by WB and MOF contributed to a better understanding of relevant laws, regulations and guidelines, 32 Component Project outputs (under the contract with Consortium) Project outputs and activities implemented by the NSO improvement of financial and procurement operations, enhancement of professionalism of the project staff and sharing of experience among WB-funded project staff in Mongolia. A.5 Review and • Assessment of state-of-play and review of existing organisational • NSO set up teams of enumerators/researchers who will work improvement of structure and coordination of statistical activities. on a permanent basis. These enumerators carry out the household • Update of Program for Official Statistics Development and its socio-economic and labour force surveys. organizational Master Plan for approval. • National Strategies for Development of Statistics (NSDS) 2011 structure of the • Development of an organisational structure applicable to the – 2015 developed. Approved by the Parliament in 2011. The national national specifics. action plan was approved by the Statistical Council. statistical system • Study visit to Germany to study organisational structures of • Development of the NSDS (2016-2020). selected countries and develop recommendations on how to introduce appropriate structures applicable to national specifics. • Training course for producers of administrative statistics at ministries and governmental agencies. B. Strengthening the statistical infrastructure B.1 • Preparation of annual publication on register data. Publish data on • Updated and enriched indicators of BR database through Improvement the size distribution of local statistical units (and employment) by collaboration with GDT and GASR. Aimag and branch. Publish data as well on business “demography”, • Comprehensive business register database created by linking and updating of covering the births and deaths of enterprises and local units, broken agricultural self-employed and self-employed gold miners and statistical down by branch, size and Aimag. through collaboration with GDT and GASR. registers • Participation in the International Roundtable on Business Survey • Integrated farming business and Agricultural household Frames. registration system updated. • Creation of integrated database system linking the register to • Identified statistical units and developed BR management databases for obligatory enterprise reports, with technical assistance system that collects, renews, manages and services BR data. This to help develop specifications for the linkage. is a basis for business surveys. The current BR management • Work with the Ministry of Justice and the tax office to upgrade system composes of three parts (statistical unit database which the data for nongovernment organisations (NGOs). enables working on network, warehouse data base, automatic • Development of procedures for updating the activity status and checking & cleaning the data from external sources). Two of the ISIC code of the NGOs. three parts are already completed, and the remaining one is on the • Preparation of methodology to standardise terminology in process. questionnaires and reports. • Household and population registration database updated based • Development of more efficient procedures for register updating on population census and administrative data. 33 Component Project outputs (under the contract with Consortium) Project outputs and activities implemented by the NSO through better and closer cooperation with the Tax Administration. • Meetings of the Wiesbaden Group on Business Registers- • Improvement of the register by including turnover as an International Roundtable on Business Survey Frames 4 times additional characteristic. (last one held in Vienna, Austria in September 2014. • Investigating profiling procedures used by advanced NSO for determination of appropriate reporting procedures for big and complex enterprises. • Conducting profiling of big and complex enterprises using these procedures. • Improvement of Household register by creating a database and providing data confidentiality and security. • \development of database by creating a household-based registration system in agricultural statistics. • Conducting special censuses and surveys ato reflect its result in the business register. • Exploration of possibility of continuation of mechanism for enrichment and changes in the population database. • Connect the population database with the base registration database. • Study visit to Finland to improve methods for population and household register. • Study visit to Finland to develop a population and household registration booklet. • Study visit to Finland for business register methods improvement. B.2 Further • Review and revision of classifications to be in line with the latest • Approved Classification of Assets that incorporates development and versions of international standards and support survey areas in Classification of financial and non-financial assets”, 2013 (SNA implementing the revised versions. 2008). improvement of • Convert to ISIC 4. • Approved “Central product classification”, 2011 (UN Central classifications • Development of classification of the functions of government to Product Classification Version 2.0). and other be used in national accounts. • Classification of the purposes of non-governmental standards • Study visit to Germany for improvement of classifications organizations”, 2011 (UN Classification of the Purposes of Non- priority economic sector Profit Institutions Serving Households). • International standards of industrial classification ISIC 4.0 is used for GDP, IOT and SUT and updated classification from 3.0 to ISIC 4 for all economic sectors. 34 Component Project outputs (under the contract with Consortium) Project outputs and activities implemented by the NSO • In 2012 ISIC 4.0, 3-level trainings for the staff of ministries and agencies, local statistical officers and NSO staff. In total 71 people participated in the training. B.3 Further • Defining the required scope of sampling and sampling methods • Interdisciplinary sampling team estaclished(6 staff). development and and train the MNSO staff in relevant courses in particular steps. • TA visit from TurkStat. • Review of opportunities for sampling taking into account planned • Training of staff in sampling methodology jointly with improvement of improvements of the Statistical Register. UNSIAP in 2011 (25 staff). sampling • Training of staff (especially at the regional level) in sampling • Training in methodology of sample survey in South Korea and methods methods. the University of Michigan, the USA (4NSO staff). • Review the current suite of economic surveys to determine which • Sampling designed for HSES, Labor force, Time use, can be conducted on a sample basis without serious data loss to SUT/IOT, Average salary, Copying with disasters, Social users. indicators surveys. • Improvement of sampling in the household survey taking into • A General guideline for sampling design prepared in 2014. consideration the development of Household Register and the specific of particular groups of the population. • Study visit to Germany for Implementation of sampling methods and training applications using registers. B.4 • Seasonal adjustment technique (such as X-12 ARIMA) for the • Methodological guideline on seasonal adjustment developed Improvement of sub annual discrete estimates in all key sectors and activities (GDP, and applied based on UNECE Seasonal adjustment guideline output by activity, etc.). using Demetra+. skills in seasonal • Development of methodology for the use of seasonal adjustment • Seasonally-adjusted, quarterly GDP at current and constant adjustment for the estimation of GDP and CPI. prices disseminated (2011). • Recommendations on relevant software in seasonal adjustment for the estimation of GDP and CPI. • Study visit to Germany to improve skills in seasonal adjustment application beside national accounts. B.5 • Development of quality model comprising quality concepts and • Inter-departmental quality team established. Improvement of definitions, including the so-called output quality characteristics: • Core quality documents approved: Quality policy document, relevance, accuracy, timeliness, accessibility, interpretability and Code of practice and a Guideline for writing a user-oriented and skills in coherence. producer oriented quality report. evaluating and • Development of quality policy providing leadership, a framework • User oriented quality reports written for price survey and for definition of quality objectives, a commitment to satisfying user labour force survey. 35 Component Project outputs (under the contract with Consortium) Project outputs and activities implemented by the NSO improving data needs, continual improvement and reengineering where major • Workshop on quality management for NSO management and quality changes to existing processes are required. staff and specialists from ministries in charge of administrative • Development of the following quality tools: quality and statistics. performance indicators and measurement systems, quality control • NSO carried out the quality assessment of data produced by systems, self-administered quality checklists and a program of Ministry of Health and Ministry of Education and Science. quality reviews. • Implementation of quality policy including continual quality improvement and a quality evaluation program. • Training in quality awareness, principles and procedures • Development of quality manual documenting concepts, policies, procedures and responsibilities. • Study visit to Germany to improve concepts and procedures of data quality by the example of procedures in the host country Germany. B.6 • Development of questionnaire design guidelines for all types of • Questionnaire design team established Improvement of questionnaires (paper, electronic, face to face interviews and • Questionnaire “Household, population and social indicators computer assisted telephone interviews). report” developed. skills in • Training in the use of questionnaire design standards to survey • Guidelines for questionnaire design provided by Destatis evaluating and areas. translated. On this basis a standard questionnaire developed. improving data • Development of procedures of review for all existing Consequently, appropriate changes introduced into all quality questionnaires for conformance to the standards prior to their use. questionnaires used in official statistics nationally. • Development of Master Questionnaire and necessary guidelines for the use of it. • Cross-sectional coordination of questionnaire design, generating a master questionnaire and standards for social and economic surveys. C. Upgrading statistical operations (improving individual subject matter programs and methodologies as well as the existing data outputs) C.1 Full • Completion of 2010 Supply and Use Tables (SUT), investigate • Translation of the “SNA-2008” completed. implementation and analyse discrepancies in it, and document the research and • Regional training courses on the Establishment Census and findings involved in balancing the SUT. surveys to compile SUT and IOT organised in 21 aimags and the of SNA • Improvement of techniques involved in preparing QNA. capital. Establishment census and other related surveys carried • Training in SNA statistics (provide on the job training). out. 36 Component Project outputs (under the contract with Consortium) Project outputs and activities implemented by the NSO • Establish SUT yearly and disseminate it to users. • The following key methodologies approved by NSO: (i) • Improvement in estimation of gross fixed capital formation. methodology for estimation of annual & quarterly GDP by • Study the methods for estimating the size of the non-observed production and final expenditure approach, at constant and economy. current prices; (ii) concept to compile SUT and IOT for 2010 was • Preparation of improved estimates of the size of the non-observed developed and reviewed by the Methodological Standing Council economy in Mongolia. and the Board of NSO Chairman. The concept was approved by • Organization of study tours to learn about environmental an NSO Chairman Order of 15th February 2011 (No. 01/26); (iii) accounting (“Green GDP”) as practiced in other countries. methodology for institutional sector accounts; (iv) benchmarked • Improvement of coordination with balance of payments (BOP). and rebased of NAs based on SUT (2005); (v) Compiled SUT & • Improvement of assessment of gross national income (GNI), IOT, 2010 (9 times), including (a) SUT: 203 activities and 311 including labour migration and remittances. products, aggregated into 68x55, 68x48, 48x32, 32x20, 20x20, • Consultation for the calculation of consumption of fixed capital and (b) IOT: industry by industry matrices of 55, 48, 32 and 20; for the Government (vi) Annual & quarterly GDP by production and final • Supply and Use Tables (SUT) and Input-Output Tables (IOT) for expenditure approach, at constant and current prices have been 2010 estimated and disseminated; (vii) Seasonally adjusted QGDP has • GNI, labor migration and remittances been estimated and disseminated; (viii) Re-estimated FISIM, • SUT concept of taxes, subsidies and consumption incorporated FISIM into IOT and GDP estimates and • Estimation and improvement of non-observed economy disseminated; (ix) according to SNA 2008 NSO has compiled estimation SUT for 2011, 2012 and related matrices; (x) NA dynamics has • Compilation of Production matrix (SUT) been updated according to SUT 2010, rebased and benchmarked. • Estimation of gross fixed capital formation, classification of • Development of special release calendar for disseminating NA animals demand side and consumption of fixed capital estimations, including (a) user seminar on IOT and SUT was • Balancing of SUT-System and Improving the assessment of the organised in 2013 involving officials from ministries, scholars gross national income (GNI) and lecturers from universities and other interested users; and (b) • Improving economic statistic and practical questions of NA dynamics was disseminated in August 2014. environmental accounting. • Workshop on IOT in 30-31 October 2014. • Seasonal Adjustment method and application. • Improvement of sector accounts. • Study visits to Germany: Introduction of environmental accounting; Results of the implementation of Supply and Use tables and Input and Output tables; FISIM and Coordination with Balance of Payments; Compilation of Supply and Use Tables (SUT); Presentation of Results of implementation 1-18 and Improvement of institutional sector accounts, production side, capital and financial income and SUT compilation. 37 Component Project outputs (under the contract with Consortium) Project outputs and activities implemented by the NSO C.2 Price • Consumer Price Index (CPI) – with the aim of expanding its CPI: statistics scope: conduct additional research to update basket and weighs, and • National CPI estimated on the basis of weighted average for provide price statistics training for the local staff in charge of price 2010 HSES expenditure of 21 aimags and the Capital city. (including PPI, research. • The national aggregate indices compiled for main 12 groups CPI and other • Producer Price Index (PPI) – improve the calculation of industrial of COICOP and overall index. relevant indices) sector PPI, disseminate to the general public, improve the base • Integrated soum level consumer price into the National index and BOP period survey for the calculation of sectoral PPI and expand its of consumer prices. Furthermore, soum level consumer price was statistics scope integrated into aimag level CPI. • Review Price Statistics and develop a concept to improve Price • A booklet on PPI guidelines, definitions, methodology and Statistics. classifications. • Methods of constant price data collection and calculation. • Program for collecting prices of consumer goods and service • Study visit to Germany: by tablets developed. Data are directly sent to the server and used • Methods of constant price data collection and calculation (PPI) for estimation of CPI. • Improving price statistics PPI: • 2-stage surveys in industrial, retail trade and construction sectors carried out. • Monthly and quarterly PPI estimated (2010 base year, PPI for 2011 and 2012) • Brochure on definitions, concepts and estimation methodology and results • Industrial production index estimated and disseminated. Foreign trade index Export and import price index estimated and disseminated on a monthly basis. C.3 2010 Census It has been jointly agreed to cancel originally planned activities in the contract because of not being relevant. See paragraph 12 and preparation Annex 3 for the details. C.4 Improving • Assessment of current situation of results of surveys on informal Estimates of the size of t non-observed economy improved. Economic sector of estimation of drug trade and of estimation of intentional • Frame of NOE according to the OECD Manual defined. distortion of business accounting. •Concept of NOE survey and approved the methodology for Statistics • Discussions on estimation for the category and adjustments of estimation of indicators and questionnaire developed. national accounts. • Artisanal mining survey (20 aimags, 79 soums and 200 mining 38 Component Project outputs (under the contract with Consortium) Project outputs and activities implemented by the NSO • Integration in a survey and the estimation result in the GDP sites) estimation. • Survey on self-employed carried out in 2012. • Measurement of contribution of the non-observed economy to the national accounts (GDP) and its evolutions by economic sector by Investment statistics following direct and indirect methods. • Survey carried out (9,859 enterprises and organizations). • Identification of indicators and ratio to estimate VA of IS for Results disseminated. other years which will not conduct IS survey by economic sector. • Investment statistical data integrated into NA estimation. • Development of future concept of non-observed economy • Reports produced in English and Mongolian languages. estimation. Environmental statistics • Review of current status of statistical data in the field of Within the environmental statistics framework and measurement environmental accounting, evaluate and provide recommendations of green economy, the following activities have been carried out: for improvement. • NSO and the Ministry of Environment and Green Development • Provision of information on recent methodological developments (MEGD) signed the Memorandum of Understanding. in environmental accounts, share knowledge and international • NSO carried out self-assessment of current status of practices for introducing new methodologies and improving data environmental statistics. As a result, 175 indicators out of 492 are quality and provide recommendations on how to improve quality of currently produced. environmental statistical data in a case of Mongolia. • “System of environmental and economic accounts-2012" and • Recommendations and assistance in compiling physical flow thoroughly studied translated into Mongolian. Experimental accounts, monetary flow accounts, asset accounts and compilation of the Material flow account, which is the first basic classifications in natural recourse that will be used at national level. account of SEEA, underway. • Experimental estimation of the environmental economic accounts • 240 indicators to measure impacts of mining on economy and provide countries’ experience on the issue (in cooperation with (155), society (55) and environment (22) identified. the Ministry of Nature, Environment and Tourism). • Within the framework of estimating green economic indicators, 30 sets of indicators identified to evaluate process of transition to a green economy. • The OECD manual "Towards Green Growth: Monitoring Progress, 2011" translated into Mongolian language. C.5 Improving • Review of current status of official statistical data in the fields of Household Survey Social Statistics education, culture and sciences, evaluate and provide • Sampling size estimated recommendations for improvement. • HSES to identify poverty indicators on annual basis, to monitor • Improvement in knowledge on recent methodological implementation of the 1st objective of MDGs to reduce poverty developments in education, culture and sciences and on and hunger and extract necessary data and calculations to international practices for introducing new methodologies and produce poverty map (25,656 households in 9 districts of improving data quality. Ulaanbaatar and 22 aimags). Survey report produced. Poverty • Development of standards and classifications in education, indicators at aimag estimated using 2011 and 2012 household 39 Component Project outputs (under the contract with Consortium) Project outputs and activities implemented by the NSO culture and science that will be used at the national level. Train staff survey databases. in charge of education to acquire methodology to estimate lifelong learning indicators. Gender statistics • Publishing a special bulletin of education statistics. • Gender indicators’ framework approved. • Training to enhance capacity, knowledge and skills of the • Dynamics of statistical data on gender compiled. specialists responsible for wage and labour statistics. • The framework and design of gender statistical book finalized. • Matching the Mongolian labour and wage statistics with the • Training on the framework of indicators of gender statistics and international statistics. methodology of estimation for officers from government • Development of methodology and methods for estimation of agencies responsible for the gender issues and representatives of labour cost index and identify the relevant indicators that will be NGOs. used to estimate real GDP growth, labour market growth, and • Disseminated 50 indicators of gender statistics on the website labour productivity and produce data and information necessary for of NSO www.1212.mn. developing “National standards and classifications for labour cost”. • Training to improve knowledge on recent methodological Education statistics developments in international migration and on international • Classifications and methodology of education statistics practices of collecting, compiling and disseminating data on developed. international migration. • Classifications of education of Mongolia corresponds with • Review of legal documents of relevant Mongolian organisations UNESCO ICSED 2011. MNSO Questionnaire forms of labor and related to international migration and improve approaches to link vocational and technical education renewed (24 forms in total). data. • Identification of sources of data and develop indicators for International migration international migration. • MNSO approved principles of international migration statistics • Study data collection from border registration and its uses in and methodology for estimation of indicators of international official statistics, define questions, indicators and definitions for the migration. questionnaire of the Population Census 2011. • The Department of Population and Social Statistics (DPSS) is • Improve skill building in international migration statistics by working on development on inter-agency database based on preparing a guideline for developing methodology of some population registration database. indicators of gender inequality on ownership and possession of asset, on income allocation, on wage and salaries. Determine data Labor and wage statistics sources for the indicators and reflect Mongolian situation in gender • Pilot labor cost survey carried out. statistics. • Monthly average wage indicator and salaries of employees • Study visit to Germany on improving education statistics and from sample survey of enterprises 2000-2013 calculated. International migration statistics, and improving gender statistics. 7 questionnaires approved by the Chairman’s Order (31 Dec. • Study visit to Finland on Labor and Wage Statistics 2013, No 1/158). Average salary of employees estimated by using data from Social Insurance Fund (average salaries of 40 Component Project outputs (under the contract with Consortium) Project outputs and activities implemented by the NSO 554,500 employees for 34,000 establishments). D. IT Infrastructure and Equipment D.1 Preparation • Development of concept and structure of an information system • IT Plan developed with the experts’ input for further analysis of an IT plan concept and make a detailed planning for the establishment of basic by Mongolian expert. register databases and an integrated geographic information system. • Dissemination database developed and launched • Equipment and software needed identified. Costing of at www.1212.mn (MONSIS, Korean Government) implementation of the proposed system completed. Training and • IT networking, security equipment and related software specialisation needs of the IT staff to maintain the functioning of procured and installed. 151 equipment items installed o/w 27 at the proposed system identified. NSO and 124 at local statistical offices. • Establishment of three register data bases on population, • Increased speed of Internet of capital, aimag and district enterprises and agriculture units and train staff on database design. statistical offices to 1 MB and later on to 2-3 MB. • Development of a solution for a secure and safe integrated • Tablets used for data collection (livestock, price, Social network connection (e.g. VPN) that links between central, regional indicators survey) offices and agencies and determine the secure, reliable, continuous • 31 Ipads distributed to capital, aimag and district statistical activity and efficient utilisation of the network. Determine the offices. required hardware and software system to establish the planned • 330 3G modems distributed to soum statistical officers. network connection. • Flash of 2-4Gb provided to statistical officers. • Evaluation and improvement of technological infrastructure, the • Improved IT environment and work stations of capital, aimag server and network system in the Mongolian statistical system. and district statistical offices and better work environment. • Preparation of a development plan on information security and • 14 new programs developed and 9 programs renewed (3 confidentially. programs for tablets; 11 programs for online data entry). • Specifications of a DB structure which meet the standards for the • Android version of Ezstat and Monstat applications. newly planned statistical production system and implement • Further creation of national integrated online system of economic sector’s DB; population and households. • Objective of information standardisation and prepare guidelines for information system standardisation of the proposed system (data term, data domain, data code, data models). • Assistance in application software of link the geospatial information system with the Integrated Statistical Database. • Select the equipment of advanced technologies necessary for collecting statistical data to facilitate the work of census, survey enumerators and interviewers. 41 Component Project outputs (under the contract with Consortium) Project outputs and activities implemented by the NSO D. 2 Procurement of networking and security equipment. Procurement of Procurement of network security software. computers and other equipment Procurement of Network management system/server and management system tool. establishment of an IT network 42 Figure 2.1: Attribution and Results Measurement PDO Improve efficiency and effectiveness of the national statistical system of Mongolia to enable it to provide relevant, timely and reliable data for evidence-based policy making in support of Government’s Level MDG-based National Development Strategy. Access to information/dissemination 1. User satisfaction rates increased (71%, exceeding end target value of 60%) 2. Targeted statistical products easily accessible in the media, with metadata and interpretation of findings, including, but not limited to online statistical database 1212.mn newly created with metadata and disaggregated data of censuses and surveys; MONSTAT and EzSTAT applications for mobile phones and tablets are fully operational with uploaded statistical printed materials; corporate standard design is applied to all official publications; websites of central and local statistical offices improved Higher in more user-friendly manner. Level Coverage/ Accuracy/ Reliability: Outcome 3. Internationally-accepted statistical techniques in collection, compilation and authenticity verification applied, and regular validations carried out for data sources and statistical products (for example, 4 international classifications and 11 methodologies developed and applied including ISIC 4.0 and SNA 2008; data quality policy, quality report and code of practice were developed based on EU quality management practices). Timeliness: 4. Statistical outputs released in a timely manner in accordance with internationally-accepted frequency and timeliness (i.e. according to IMF evaluation the NSO met all technical requirements to enter SDDS according to IMF standards and assessment). Inter- 1. Adequate policy and regulatory framework as well as effective institutional framework, management and human resources for statistics are in place. mediate 2. Statistical infrastructure developed and operational. outcome 3. Investment in physical infrastructure and equipment to facilitate the production and dissemination of data by statistical agencies is undertaken and adequately maintained. 1. Mechanisms for effective interagency coordination established and operational; Mechanisms for effective dialogue with data users and providers operational; National Statistics Training Center (Center for Information, Training and Advocacy) established; NSO central and local level structure optimized and staff trained, including staff from relevant ministries and agencies. Key Outputs 2. Registers compiled and maintained; Internationally accepted classifications, standards and methodologies adapted and used in data production; Standard questionnaire for survey developed and applied; Targeted surveys conducted, thematic statistical data developed and results disseminated. 3. Capacity for physical operations adequate and well maintained; All staff at central office and regional offices have workstations with direct access to databases for analysis and tabulation. Provision of technical services (e.g. consultancies) Inputs Conducting training and workshops Procurement of ICT equipment and software 43 MONGOLIA MONSTAT: Strengthening the National Statistical System of Mongolia Annex 3. Results achieved through the implementation of the Trust Funds co- financed the MONSTAT project 1. TF093575 by Trust Fund for Statistical Capacity Building (TFSCB) in the amount of US$400,000 in period of August 2009 to October 2012 provided support to NSO for the 2010 Census preparation so the census would be prepared and conducted properly. The census was completed successfully in accordance with improved methodology, and NSO retained the capacity to use modern methodology for conducting the census for its ongoing and further activities. 2. TF094696 by Korea Trust Fund (KTF) in the amount of US$700,000 in period of September 2009 to December 2014 provided technical assistance for population census through revision of methodological approaches, census design and coverage. Coverage of household and business registries have been enriched due to technical assistance received under the grant. Mongolian NSO improved the sampling design for poverty estimates and started calculating poverty estimates at aimag level. 3. Second TF098487 by Korea Trust Fund (KTF) in the amount of US$650,000 in period of September 2010 to December 2014 supported enhancement economic and social and demographic statistics that were collected using surveys/censuses or administrative reporting through reviewing and re-engineering the existing practice. Content of data outputs and the corresponding data inputs in each selected subject matter area were reviewed, re-examined and rationalized by identifying and eliminating indicators that seemed inappropriate for a market economy and no longer needed, or that were not published. Consulting services and training activities benefited from new infrastructure already in place. The thematic areas covered were survey analysis in various domains such as poverty, social protection, health, and labor market. Details are in Table 2.1 Outputs by Components (C.4 and C.5), ICR pp.38-39. 44 MONGOLIA MONSTAT: Strengthening the National Statistical System of Mongolia Annex 4. Main Risks and Mitigation Measures Table 4.1: Main Risks and Mitigation Measures Rating of Rating of Risk factors Description of risk Mitigation measures residual risk risk II. Sector Governance, Policies and Institutions Sector Specific Lack of political support Substantial National funding has increased in Moderate Risks (General and commitment for a recent years, but this is insufficient public admin. modern statistical system, to upgrade performance while still sector) by not providing adequate maintaining all of the previous legal, administrative or statistical activities. Hence, a budgetary conditions. particular challenge will be to maintain the political support for better official statistics. One way of ensuring sustainability is to provide better outputs, which can be achieved by improving the dialogue between data producers and users. NSO and other main partners of the statistical system will need to improve communications with users, to build credibility and to focus on providing outputs that meet the needs of users. Further, linking the project closely with NDS and MDGs as well as with other ongoing and planned programs will create continuing demand for better data. Endorsement of the NSDS by the National Statistical Council and its adoption by the NSO is also an indicator that these potential risks could in fact be mitigated. III. Operation-specific Risks Technical Design Lack of external funding Substantial The task team has been Moderate and lack of effective communicating with EC and Japan coordination among for co-financing. Meanwhile the development partners team obtained co-financing grants from Korean Trust Fund and TFSCB. Signing a Memorandum of Understanding (MoU) among development partners and donors is also an option that clearly identifies relevant activities with proper sequencing and clear responsibilities to all involved parties will make sure effective and efficient coordination. 45 Implementation Resistance to change and Substantial The task team identified champions Low Capacity And lack of capacity in the of statistics within country and got Sustainability present statistical system, their support. The team will also due, for instance, to loss ensure involvement of all of competent staff stakeholders from preparation to the end of implementation and obtain Government’s continuing support. As the implementing agency, NSO has demonstrated that it has the capacity to carry out statistical capacity building projects and to take the lead in technical and methodological forums for statistical policy. Implementation of the NSDS will ensure the improvement of staff skills and availability of better data in the short-term by utilizing existing sources. These short term improvements will bring credibility to the system and ensure sustainability. Financial Overall Moderate See PAD, Annex 7 for details Moderate Management (changed from Substantial to Moderate by ICR team) Procurement Overall Moderate See PAD, Annex 8 for details Moderate (inserted by the ICR team) Social And N/A N/A Environmental Safeguards IV. Overall Risk (including Reputational Risks) Overall Risk No reputational risk is anticipated. Moderate Source: PAD (Implementation section, part E, page 16; and Annexes 7 and 8) 46 MONGOLIA MONSTAT: Strengthening the National Statistical System of Mongolia Annex 5. Economic and Financial Analysis (including assumptions in the analysis) N/A 47 MONGOLIA MONSTAT: Strengthening the National Statistical System of Mongolia Annex 6. Bank Lending and Implementation Support/Supervision Processes Task Team members Table 6.1: Task Team members Responsibility/ Names Title Unit Specialty Lending Mustafa Dinc Sr. Economist/Statistician/TTL GMFDR Gerelgua Tserendagva Procurement Specialist GGODR Ulziimaa Erdene Team Assistant EACMF Altantsetseg Shiilegmaa Economist/Team member GMFDR Supervision/ICR Altantsetseg Shiilegmaa Economist/TTL GMFDR Gerelgua Tserendagva Procurement Specialist GGODR Financial Management Dulguun Byambatsoo GGODR Specialist/Consultant Gantuya Paniga Senior Program Assistant EACMF Ulziimaa Erdene Team Assistant EACMF Olga Shabalina Sr. Statistician DECDG Adriana De Leva Consultant DECDG Christelle Signo Kouame Program Assistant DECDG Mustafa Dinc Sr. Economist/Statistician/Co-TTL DECDG 48 MONGOLIA MONSTAT: Strengthening the National Statistical System of Mongolia Annex 7. Beneficiary Survey Results 1. A formal beneficiary survey has not been conducted, but a World Bank mission visited Ulaanbaatar, Mongolia in late October 2014 and April 2015 as part of the ICR preparation process to participate in the concluding project workshop with key stakeholders, including government, parliament, consortium, development partners, media, academia and other users, and to review implementation progress, and to conduct interviews with beneficiaries. Both missions held meetings with relevant stakeholders and discussed the project results and outcomes. In addition, the first mission has visited the regional statistical office in Ovorkhangai aimag to observe the project results outside of the central office. 2. The interviews with the beneficiaries of the MONSTAT project showed that the results of the project were positive and confirmed the project outputs stated in Annex 2 and observations in paragraph 53. The interviewed respondents considered that the project was timely and important in bringing the statistical system of Mongolia up-to-date with the internationally accepted standards and classifications and positively influenced their own professional development and skill sets that provided them with a confidence in their own capacity to operate in a more efficient and effective way. 3. There have been noticeable changes in institutional structure of the state statistical system of Mongolia. The project helped increasing reliability, timeliness and quality of the statistics produced in the country, which, in turn, increased the trust and satisfaction in the statistical system outputs. The NSO management was considering the ways and means to sustain the gains and move forward further. It was mentioned that, nevertheless, there would be a need for future support. Areas for improvements were identified in the Global Assessment and elaborated further by the NSDS 2015-2020. 49 MONGOLIA MONSTAT: Strengthening the National Statistical System of Mongolia Annex 8. Stakeholder Workshop Report and Results (if any) NA 50 MONGOLIA MONSTAT: Strengthening the National Statistical System of Mongolia Annex 9. Summary of Borrower's ICR and/or Comments on Draft ICR Strengthening the National Statistical System of Mongolia Twinning Partnership MONGOLIA MONSTAT: P113160 “STRENGTHENING THE NATIONAL STATISTICAL SYSTEM OF MONGOLIA” PROJECT BORROWER IMPLEMENTATION AND COMPLETION REPORT Ulaanbaatar, Mongolia May 2015 51 CONTENTS I. KEY PROJECT DATA II. TWINNING PARTNERSHIP III. OUTPUTS AND OUTCOME OF THE PROJECT 3.1 Institutional development 3.2 Data Dissemination and User Relations 3.3 Human resource development IV. STATISTICAL INFRASTRUCTURE DEVELOPMENT 4.1 Statistical data quality improvement 4.2 Statistical Classifications 4.3 Sampling and standard questionnaire 4.4 Business register V. STRENGTHENING STATISTICAL SUBJECT MATTERS VI. IT INFRASTRUCTURE VII. CONCLUSION 52 I. KEY PROJECT DATA Duration June 2009-December 2014 Partners: • The World Bank • National Statistical Office of Mongolia (MNSO) • Federal Statistical Office of Germany (Destatis) • Statistics Korea (KOSTAT) Objective of MONSTAT project • To improve efficiency and effectiveness of the national statistical system of Mongolia to enable it to provide relevant, reliable, and timely data for evidence- based policy making in support of Government’s MDG-based National Development Strategy (NDS). Project Outcome Indicators • Access to information/ dissemination o User satisfaction rates have increase in at least 60% by the Closing Date o By the Closing Date, targeted statistical products are easily accessible in relevant media, with metadata and interpretation of findings. • Coverage/ Accuracy / Reliability o By the Closing Date, internationally accepted statistical techniques in collection, complication and authenticity verification are applied by the Recipient, and regular validations are carried out for data sources and statistical products. • Timeliness – By the Closing Date, statistical outputs are released in a timely manner in accordance with internationally accepted frequency and timeframes 53 Project components A. Improving policy, regulatory and institutional framework and the organizational structure of the statistical system • Institutional framework (e.g. statistical law) • Relations between data users and producers • Data dissemination • Human resources • Organisational structure of the system B. Improving Statistical Infrastructure • Registers • Classifications and standards • Sampling methods • Seasonal adjustment • Data quality • Questionnaire design C. Upgrading the Statistical Operations • National accounts • Price statistics • Census preparation • Social statistics • Economic Statistics D. IT Infrastructure and Equipment • IT Plan • IT Network 54 II. TWINNING PARTNERSHIP The Monstat project was implemented by the National Statistical Office of Mongolia through twinning partnership with the Consortium of the Federal Statistical Office of Germany (Destatis) and Statistics Korea (KoStat). The Twinning agreement was signed in August 2010 to provide expertise in a number of areas including institutional assessment, development and capacity building, statistical infrastructure, statisatical domains and informaton technology. There have been a number of advantages of implementing the project through Twinning partnership. Twinning partnership ensures easy access to expertise, sustainable relations between experts and long term collaboration between peer organisations after the end of the project (Box 1). Box 1. Advantages of Twinning Partnership for Project Implementation Twinning concept • The twinning is one of the principal tools that could be employed for the implementation of statistical capacity improvement projects in developing countries; • Twinning provides necessary technical expertise in a “one-stop-shop” for the development of modern and efficient statistical systems with the structures, human resources and management skills. Features of twinning • The project objective is clear; • Relevant project components to achieve the objective have been identified and results and monitoring framework is established; • Necessary ownership is in place and sufficient political support exists; • The twinning partner has required expertise with necessary human resource capacity and is willing to work with the recipient. Principles of twinning • The recipient country is in the leading role and selects the twinning partner; • The selected twinning partner helps transferring the requested expertise that is in line with the internationally accepted standards; • Twinning is a joint operation, in which each partner takes on responsibilities and commit themselves to achieving the agreed results; • The twinning partners prepare a detailed work plan that could be modified during the implementation; • Clear benchmarks are identified that allow for close monitoring of progress towards the final result; • The achievements of the twinning should be maintained by the recipient country after the end of the project implementation. • At the end of the project the new improved system must function under the sole responsibility and ownership of the recipient country. 55 The twinning partnership has enabled the Destatis and KoStat to significantly contribute to the project outcome and as a result the Consortium has provided best expertise not only from partners, but also from diverse countries including Norway, Finland and organizations such as Samsung, SK, Daewoo were involved in the areas of DB (KOSIS), ICT, GIS and Network. In total, 80 international experts provided consultancy in institutional assessment and development of strategies, human resources, user relations and data dissemination, business register, classifications, methodologies and information technology. During hte project period, 84 activities were conducted; 35 study visits and 49 expert missions. In total 406 participants from the NSO and regional offices were trained through twinning partnership. 180 participants from the MNSO and regional offices were trained abroad through twinnig partnership. III. OUTPUTS AND OUTCOME OF THE PROJECT 3.1 Institutional development Five areas were identified to strengthen institutional structure and management; a) improvement of legal framework; b) improvement of producer and user relations; c) improvement of data dissemination and access to data; d) strengthening human resources; and e) organisational structure. The project facilitated to achieve the following outcome: • A well-functioning national statistical system to contribute to Government’s decision-making process by providing timely and accurate data; • Assistance to the Government in monitoring progress towards national development goals, including the PRSP goals and the MDGs through improved data collection, production and dissemination; • Improved legal and institutional framework for sustainable statistical development including the capacities for strategic planning, human resources development and the application of Total Quality Management criteria and procedures throughout the institution; • Improved coordination and liaising between all stakeholders (government ministries, national and regional statistical offices, donor agencies, the media, research institutes) • Approval of the 2nd National Strategies for Development of Statistics (NSDS) 2011 – 2015 and action plan for implementation; • Improved inter-agency coordination and collaboration through capacity building trainings, workshops and signing Memorandum of Understanding with a number of ministries: o Ministry of Population Development and Social Protection o General Authority of State Registration o Bank of Mongolia o Ministry of Finance o Ministry of Economic Development o Ministry of Environment and Green Development 56 o Ministry of Culture, Sports and Tourism o Ministry of Health o Customs Authority • NSO has had the Adapted global assessment and is working on development of the next NSDS. 3.2 Data Dissemination and User Relations The baseline survey on user satisfaction was conducted in 2010 and showed the satisfaction rate of 47%. The following surveys were conducted in 2012 and 2014 and the customer satisfaction rates were increased to 61, 3% and 71% respectively. www.1212.mn statistical data dissemination website was launched in 2012. The website has metadata, raw data of some surveys, electronic versions of published products such as yearbook and monthly bulletin. In 2014 MNSO has enriched www.1212.mn website with more data at aimag/capital city, soum/district and bag levels. Applications for Smart phone users were developed (Monstat, Ezstat). These applications have been installed to cell phones and computers of assistants of MP. Monstat application now has 206 books and NSO has met all criteria of GDDS and is now making efforts to enter SDDS. The experts from IMF have conducted assessment for entering SDDS. The recommendations show that there are several legal provisions that need to be modified in order to enter SDDS. Also, the entrance to SDDS depends of the data release by the Ministry of Finance and Bank of Mongolia. In regard to NSO, it will achieve all criteria of SDDS by September 2015. Other outcome of the project in this area include increased data available for policy making and monitoring, increased awareness of importance of statistical data and increased use of data. Table 3.1 Access to Data Access stat. products 2012 2013 2014 I-X Printed products dissemination (pieces) 11 600 11 600 5150 Yearbook 2 000 2 000 2000 Monthly bulletin 4 800 4 800 3150 Monthly brochure 4 800 4 800 The number of users 2 820 4 123 2850 By visit 2200 923 1070 57 By letter of request 314 351 101 By email: information@nso.mn 306 341 277 1212 telephone inquiry 253 / X-XII/ 2 508 1403 Online service 118 413 203 372 300 992 Access of users to www.nso.mn 98 049 90 102 159585 Access of users to www.1212.mn 20364 /VIII- 113 270 141407 XII/ International organizations 14 14 14 3.3 Human resource development Monstat project contributed to staff capacity building tthrough trainings in statistical concepts and methods, establishment of Center for Information, Training and Advocacy, establishment of library, printing of books and manuals and creation of professional development programs in statistics. As a result, staff work environment improved, their satisfaction with work environment and career prospects was increased and staff retention rate has improved. More and more competent graduates are interested to work at national statistical system. However, staff workload has remained high and staff salary still low compared to salaries of private sector. During the project period nine types of professional books and manuals were developed and printed out. Table 3.2 Trainings by NSO provided through Monstat Subject areas 2010 2011 2012 2013 2014 TOTAL Institutional 529 196 456 1713 2894 framework Statistical - 116 - 116 infrastructure Statistical 1528 - 76 33 1637 operations TOTAL 2057 312 532 1746 4647 ( Mongolia) Foreign trainings 41 21 12 6 80 58 IV. STATISTICAL INFRASTRUCTURE DEVELOPMENT The main outcome of the statistical infrastructure development was the application of internationally accepted classifications, sampling methodology, standard questionnaire and data quality standards to all subject matter components of the project. 4.1 Statistical data quality improvement During the project implementation period NSO has developed three documents for data quality management which are based on EU Code of Practice and other relevant quality documents. The systematic introduction of quality management system in the NSO has served as the foundation for improving evidence based decision making processes of government institutions, businesses and citizens of Mongolia, ensuring transparency of the Government, civic engagement and inclusiveness in decision making process, enabling the Government to better monitor progress towards national development goals and increasing user satisfaction. The quality criteria introduced in these documents cover data relevance, accuracy, timeliness, punctuality, accessibility and clarity and comparability and coherence. The results of the introduction of the quality management system include increased knowledge, awareness and skills of NSO staff, ministries and other data providers on quality management and improved supply of timely and reliable data to users. Table 4.1 Quality team composition at NSO Quality team Team leader (Head of PICD) Secretary (Senior officer of PICD) PSSD (4) MESD (3) Deputy Deputy PAMD (3) PICD (3) director of director of PSSD, Head of statistician, МESD, senior DPTD (1) ITAC, head of officer-2, officer Head of PHCB, ESD, senior statistician-2 officer-1 officer-2 officer 59 Table 4.2 Quality management framework Core documents for quality management framework of Mongolia Statistical outputs Coordination of the NSS Statistical processes and service • Independent official • Sound methodology, • Relevance statistical activities • Appropriate statistical • Accuracy and • Mandate for data procedures reliability collection • Non excessive burden • Timeliness and • Adequacy of staff and on respondents punctuality financial resources • Cost effectiveness • Coherence and • Commitment to quality comparability • Statistical confidentiality • Accessibility and • Impartiality and clarity objectivity 4.2 Classifications During the project implementation period NSO has developed and applied the following classifications and Statistical infrastructure and operations • Classification of Assets that incorporates Classification of financial and non- financial assets”, 2013 (SNA 2008) • “Central product classification”, 2011 (UN Central Product Classification (Version 2.0) • Classification of the purposes of non-governmental organizations”, 2011 (UN Classification of the Purposes of Non-Profit Institutions Serving Households) • ISIC 4.0 is used for GDP, IOT and SUT • Updating classification 3.0 to ISIC 4 for all economic sectors. 4.3 Sampling and standard questionnaire NSO established the interdisciplinary sampling team and trained the staff in Germany, South Korea, Turkey and the USA. Jointly with the SIAP, NSO also organized sampling training to the staff in which 25 staff participated. As a result, sampling for surveys was designed; HSES, Labour force, Time use, SUT/IOT, Average salary, Copying with 60 disasters, Social indicators survey. In 2014 a general guideline for sampling design was prepared. A general guideline for questionnaire design was developed and used to make stepwise appropriate changes into all questionnaires used in official statistics nationally. 4.4 Business register NSO made significant progress to improve business register (BR). It has updated and enriched indicators of BR database through collaboration with General Department of Taxation (GDT) and General Authority for State Registration (GASR). As a part of work, it identified statistical units and developed BR management system that collects, renews, manages and services BR data, which is the basis of survey for businesses. The current BR management system composes of three parts (statistical unit database which enables working on network, warehouse data base, automatic checking & cleaning the data from external sources). Two of the three parts are already completed, and the remaining one is on the process. One of the signification progress made by NSO is updating of Household and population registration database V. STRENGTHENING STATISTICAL SUBJECT MATTERS The main outcome of the component was development and application of internationally accepted methodologies for economic and social statistics and increased outputs of reliable statistics by consolidating and strengthening existing surveys and administrative sources, and by designing and conducting new data-generating activities. The below statistical methodologies were developed and applied: SNA: • Methodologies for institutional accounts, GDP and QGDP; • Annual & quarterly GDP by production and final expenditure approach, at constant and current prices • The concept to compile SUT and IOT for 2010 • Benchmarked and rebased of NAs based on SUT (2005) • Compiled SUT & IOT, 2010 (9 times) • Seasonally adjusted QGDP • Re-estimated FISIM, incorporated FISIM into IOT and GDP estimates and disseminated. • Compiled SUT for 2011, 2012 and related matrices in line with SNA 2008. • NA dynamics has been updated according to SUT 2010, rebased and benchmarked. • NA dynamics was disseminated in August 2014. 61 CPI: • National CPI estimated on the basis of weighted average for 2010 HSES of 21 aimags and the Capital city. • The national aggregate indices compiled for main 12 groups of COICOP and overall index • Integrated soum level consumer price into the National index of consumer prices. PPI: • A booklet on PPI guidelines, definitions, methodology and classifications and results • 2-stage surveys in industrial, retail trade and construction sectors • Monthly and quarterly PPI estimated (2010 base year, PPI for 2011&2012) Other: • NOE according to the Organization for Economic Cooperation and Developments (OECD) Manual • Gender indicators’ framework • Classifications and methodology of education statistics in accordance with UNESCO ICSED • Principles of international migration statistics and methodology VI. IT INFRASTRUCTURE KoStat has provided consultancy service to develop IT plan and technical specifications of the network security equipment. The network security equipment was procured through open tender and the supplier was IT Zone, Mongolia. • 2013.12.25 – 2014.1.17 Equipment was installed at NSO; 10 equipment for sustainable security system and 17 equipment for network infrastructure. • 2014.1.20 –1.31 Jointly with engineers of “IT Zone Co. Ltd” installation was carried out at the capital city statistical office, statistical officers of 8 districts and 4 aimags and relevant connection was made. • 2014.2.4 –2.26 the installation and connection was carried out by NSO engineers at remaining 17 aimags. Of 151 equipment 27 were installed at NSO and 124 were installed at aimags, capital city and districts. As a result of installation of integrated IT security system the following benefits occurred: 62 • Improved infrastructure for future introduction of efficient, relevant and advanced technology at every stage of statistical operations; • Created an opportunity to transfer to data processing programs to online programs; • Enabled transfer of data with high level of data security and confidentiality in line with international standards; • Enabled real-time transfer and exchange of statistical data • Created supporting environment for e-governance, paperless technology; • Ensures possibility to access internal network and work from any place with Internet connection; and • Provide technical support to local statistical offices for computer and technical maintenance and service. Integrated IT system with high level of confidentiality and security enables the following benefits: • The system predicts data download, copy and transfer using mobile equipment • Anti-virus /attack system to prevent the system from outside attack • Carry out monitoring and reports using Solarwinds system. Figure 6.1 Integrated IT network system Created integrated IT network system and security system at NSS Mongolia 63 Figure 6.2 Internet speed increase of aimag statistical divisions The Internet speed of local statistical offices 64 VII. CONCLUSION A great progress has been made to modernize national statistical system of Mongolia. As a result, NSO has become internationally recognized modern statistical organization at global level. It is now the member of UNSD, serves in several Steering committee and technical experts’ committees of regional statistical organizations. Partnerships have been established with statistical offices of developed countries including the Destatis, KoStat, TurkStat, Norway Stat, Fin Stat and so on. It has grown as one of international consultant’s service providers. More and more staff from NSO participates in international conferences and makes presentations. NSO is a co-founder of Ulaanbaatar city group for economies based on natural resources. The sustainability of the project outcome lies with its stable leadership and management and staff capacities. The NSO is distinguished with longer term served management and staff among state organizations. Staff retention is high and an increasing number of graduates and job seekers apply to work in the statistical system. One of key drivers for sustainability is political support. NSO functions under the Parliament of Mongolia which ensures its leadership stability and support for strategy and program implementation. Despite progress, there are a number of areas for improvement. These areas have been reflected in the global assessment conducted in 2013-2014. It has been stated entrance into the SDDS has not achieved before closure of the project. It may be considered as a risk to the project. 65 Contacts National Statistical Office of Mongolia Mr. Sonomtseren Mendsaikhan Chairman Government Building III Baga toiruu -44, Ulaanbaatar -11 Mongolia Tel: +976 11 323118 E-Mail: monstat@nso.mn Website: www.nso.mn The World Bank Mongolia Ms. Altantsetseg Shiilegmaa Country Economist 5th Floor, MCS Plaza Building Seoul Street-4 210644 Ulaanbaatar Mongolia Tel: + (976) 70078214 ext. 214 Email: ashiilegmaa@worldbank.org Website: www.worldbank.org Federal Statistical Office of Germany Ms. Simone Koll Project Leader Gustav-Stresemann-Ring 11 65189 Wiesbaden Germany Tel: +49 611 75 4281 E-Mail: simone.koll@destatis.de Website: www.destatis.de 66 MONGOLIA MONSTAT: Strengthening the National Statistical System of Mongolia Annex 10. Comments of Cofinanciers and Other Partners/Stakeholders NA 67 MONGOLIA MONSTAT: Strengthening the National Statistical System of Mongolia Annex 11. List of Supporting Documents • Project Appraisal Document May 15, 2009 • Aide Memoires of Implementation Support Missions • Implementation Status and Results Reports December 2009, January 25 2011, April 5, 2012, March 27, 2013, November, 29, 2013, June 24, 2014, January 28, 2015 • MONSTAT project progress reports, prepared jointly by the NSO and Consortium • Mid-Term review of the implementation of MONSTAT June, 2015 • Grant Agreement dated June 26, 2009 • Mongolia Country Partnership Strategy 2013-2017 • Report on the Special Data Dissemination Standard Assessment Mission (October 20–31, 2014) prepared by Paul Austin, Noriaki Kinoshita, and Peter Bradbury. IMF Statistics Department, December 17, 2014 • Adapted Global Assessment Report August 2014 • NSO of Mongolia ICR May 2015 • MONSTAT project outcomes video (to be posted on Mongolia CMU website ) http://youtu.be/NMFi9wTdRZk • Bank Guidelines and Procedures (OPCS ICRR Guidelines, August 2006. Updated July 22, 2014, ICR presentations, including the IEG) 68 MONGOLIA MONSTAT: Strengthening the National Statistical System of Mongolia MAP 69