UKRAINE Public Investment Management Diagnostic Assessment Report The European Commission - World Bank Partnership Programm Part III for Europe and Central Asia Programmatic Single-Donor Trust Fund (TF0473423) - Public Finance Management Support Program for Ukraine (EUR4PFM) © 2022 International Bank for Reconstruction and Development / The World Bank 1818 H Street NW Washington DC 20433 Telephone: 202-473-1000 Internet: www.worldbank.org This work is a product of the staff of The World Bank with external contributions. The findings, interpretations, and conclusions expressed in this work do not necessarily reflect the views of The World Bank, its Board of Executive Directors, or the governments they represent. This publication was funded by the European Union. Its contents are the sole responsibility of the authors and do not necessarily reflect the views of the European Union. The World Bank does not guarantee the accuracy, completeness, or currency of the data included in this work and does not assume responsibility for any errors, omissions, or discrepancies in the information, or liability with respect to the use of or failure to use the information, methods, processes, or conclusions set forth. The boundaries, colors, denominations, and other information shown on any map in this work do not imply any judgment on the part of The World Bank concerning the legal status of any territory or the endorsement or acceptance of such boundaries. Nothing herein shall constitute or be construed or considered to be a limitation upon or waiver of the privileges and immunities of The World Bank, all of which are specifically reserved. Rights and Permissions The material in this work is subject to copyright. Because The World Bank encourages dissemination of its knowledge, this work may be reproduced, in whole or in part, for noncommercial purposes as long as full attribution to this work is given. Any queries on rights and licenses, including subsidiary rights, should be addressed to World Bank Publications, The World Bank Group, 1818 H Street NW, Washington, DC 20433, USA; fax: 202-522-2625; e-mail: pubrights@worldbank.org. Icons made by Good Ware and Flat Icons Design from Flaticon at www.flaticon.com Cover design: Kora Reichardt CONTENTS Acknowledgements  4 PIM Function 4. Selection and Budgeting  44 Abbreviations and Acronyms  5 4.A. BUDGET PREPARATION AND Overview 6 SELECTION  44 PIM-7. Project Selection and Budgeting  44 CHAPTER 1. BACKGROUND AND APPROACH TO THE PIM-8. Multi-year Budgeting  49 ASSESSMENT  12 PIM-9. Comprehensive Capital Budget  53 1.1. Context for PIM Assessment  12 PIM-10. Comprehensiveness and Degree 1.2. Background to the Assessment of Public/Parliamentary Access to and Objectives  15 Capital Budget Information  55 1.3. Methodology for the Assessment  15 4.B. BUDGET OUTTURN PERFORMANCE  62 PIM-11. Development and Capital Budget CHAPTER 2. PUBLIC INVESTMENT EXPENDITURE Execution Rates: Aggregate Expenditure AND INSTITUTIONAL MAPPING  18 Outturn Compared To Adjusted Original 2.1. Public Investment in Ukraine 18 Budget on a Commitment Basis  62 2.2. Institutional Mapping 20 PIM-12. Composition of Development and Capital Expenditure Outturn CHAPTER 3. ASSESSMENT BY INDICATOR FOR PIM Compared to Adjusted Original Budget FUNCTIONS  25 on a Commitment Basis  64 3.1. Details of the Assessment 25 PIM-13. Project Completion Time and Cost Variances for Completed Projects  65 PIM Function 1. Strategic Guidance and Screening 25 PIM-14. Stock and Monitoring of Capital Expenditure Arrears  66 PIM-1. Sector Analysis and Planning 25 PIM Function 5. Implementation  68 PIM-2. Strategic plans and investment guidance, project development and PIM-15. Procurement  68 preliminary screening 29 PIM-16. Project implementation PIM Function 2. Formal Appraisal  33 management  71 PIM-3. Formal Project Appraisal PIM-17. Control, Monitoring and Reporting: Procedures and Guidelines 33 Physical and Financial Milestones  74 PIM-4. Project Appraisal Capacity 38 PIM-18. Project Handover, Asset Registration, and Completion Review  78 PIM-5. Screening of Feasibility Studies 39 PIM Function 6. Adjustment  81 PIM Function 3. Appraisal Review  41 PIM-19. Project Adjustment  81 PIM-6. Independent Review of Appraisal 41 Contents 1 PIM Function 7. Service Delivery  85 CHAPTER 6. ASSESSMENT OF STATE-OWNED ENTERPRISES  133 PIM-20. Control, Monitoring and Reporting: Financial and Service 6.1. Public Investment and Institutional Delivery Performance  85 Mapping in Ukraine  133 PIM-21. Service Delivery  89 6.2. Assessment of State-Owned Enterprises  135 PIM Function 8. Evaluation  90 PIM-22. Scope, Nature and Follow-up of CHAPTER 7. CONCLUSIONS AND External Audit and Ex-Post Evaluation  90 STRATEGIC RECOMMENDATIONS  150 PIM-23. Legislative Scrutiny of 7.1. Conclusions and Strategic External Audit Reports  92 Recommendations Based on the Assessment  150 3.2. Summary of PIM Performance and Recommendations by Functions  94 7.2. Supplementary Discussions and Recommendations. PIM for CHAPTER 4. CLIMATE CHANGE-INFORMED PIM IN Post-War Reconstruction  156 UKRAINE  106 7.3. Recommendations Adapted 4.1. Introduction  106 to the Post-War Conditions  163 4.2. Methodological Approach  107 4.3. Existing Legal Framework  108 Annex 1. Tools with a Climate Change Focus for the Assessment Framework  168 4.4. Institutional Framework  109 Annex 2. Summary of Existing Laws, 4.5. Climate Change: Assessments Resolutions and Orders with a Bearing by PIM Function  109 on Climate Change and Disaster Risk Management  169 4.6. Conclusions and Recommendations  116 Annex 3. Details of Legislation CHAPTER 5. ASSESSMENT OF Regulating Capital Investment Streams 172 PUBLIC-PRIVATE PARTNERSHIPS  117 Annex 4. Scores of the 2021 PIM DA 5.1. Introduction and the Indicators  175 Current Situation  119 Annex 5. Calculations for PIM-12, 2018 5.2. Legal Framework  120 (UAH Thousand) 177 5.3. Institutional Arrangements  121 Annex 6. Calculations for PIM-12, 2019 (UAH Thousand)  179 5.4. PPP: Assessments by Indicators  125 Annex 7. Calculations for PIM-12, 2020 5.5 Recommendations Specific to PPP  132 (UAH Thousand)  181 Annex 8. Project Completion Time and Cost Variances of Completed Projects  183 2 Contents Tables Table 1. Structure of Public Sector Table 16. Deviation of Actual Development Capital Expenditures  19 Expenditure (excluding IFIs' projects) and Some Other Table 2. Mapping of Organizations and Roles in Expenditure Categories from Original Budget  63 Ukraine's PIM System – Central Government  21 Table 17. Development Expenditure through Table 3. Public Investments Spending by IFI-Financed Projects - Total Budget and Type of Procedure (UAH million)  24 Actual (UAH billion)  64 Table 4. Share of Non-SSIPs in Total Capital Table 18. Stock of Capital Expenditure Costs of Ongoing Investment Portfolio  24 Arrears (UAH million)  67 Table 5. Data on Indicators Provided by Table 19. Framework for the Assessment State Statistics Service  27 of PIM and CC  107 Table 6. Use of Sector Analysis and Table 20. National Strategic Documents Planning Results for SSIP  28 Addressing CC mitigation  110 Table 7. Strategic Documents by Key Sectors  30 Table 21. Examples of Strategic Justification for a Sample of Projects  111 Table 8. Key Stages and Dates of the Budget Calendar  46 Table 22. Contents of Guidelines  112 Table 9. Timeframe for Preparation and Table 23. Examples of Content of Sections Review of Budget Requests  48 about Environmental Impact and Protection in Project Proposals  112 Table 10. Projected and Approved Expenditures for 2019-2021  51 Table 24. Examples of Potential Uses of Monitoring Data  115 Table 11. Compliance of Mid-term Capital Expenditure Forecasts Prepared in 2017, 2018 and Table 25. Framework for the Assessment 2019 with the Established 'Good Practice' Criteria  52 of PIM and PPP  118 Table 12. Compliance with Criteria for Assessing Table 26. Institutional Mapping of PPP the Information Content of the Budget  57 Activities in Ukraine – Central Government  122 Table 13. Compliance with Criteria Relating Table 27. Summary of Indicator 5c  137 to Public Access to Budget Information  59 Table 28. Summary of Responsibility and Table 14. Key Dates for Review of the Timeline over Large SOEs' Financial and Draft State Budget by the VRU  61 Investment Plans  138 Table 15. Total Budget and Actual Table 29. Institutional Responsibilities over Development Expenditure106 (excluding SOE Capital Investments – Central Government  140 IFIs' projects) (UAH billion)  63 Table 30. PIM Reform Priorities  164 Figures Figure 1. Ranking of PIM Institutions by Figure 6. SOE Capital Investments by Scores in Design and Country Group  7 Source of Funding, UAH million  134 Figure 2. Gross Fixed Capital Figure 7. Capital Investments by Source of Funding Formation (% of Total)  19 with Breakdown by SOE size, UAH million  134 Figure 3. Structure of Consolidated Figure 8. State Guarantees Issued to SOEs Budget Expenditures (%)  19 by Sector, UAH million  135 Figure 4. The Structure of the Actual Capital Figure 9. Legislative Procedures for Investments and Transfers 2018 - 2020  20 SOE Capital Investments  139 Figure 5. Flow Chart of Project Preparation Figure 10. Schematic Approval Process and Selection Practices in Ukraine  23 for SOE Financial and Investment Plans  145 Contents 3 ACKNOWLEDGEMENTS This report was prepared by the World Bank team The report greatly benefited from the comments and based on the results of the Public Investment observations given by the reviewers – Mr. Oleksandr Management (PIM) Diagnostic Assessment of the Gryban (Deputy Minister of Economy of Ukraine), Ukrainian Central Government. The assessment was Ms. Ekaterina Yakovleva (Attaché, Sector manager conducted under the EU and World Bank Partnership for public finance management, European External Program Part III for Europe and Central Asia – Public Action Service (EEAS)), Mr. Oleksii Balabushko (Lead Finance Management Support Program for Ukraine – Governance Specialist, the World Bank), Mr. Bernard Component 2 Activities Trust Fund (TF073423). Myers (Senior Public Sector Management Specialist, the World Bank) and Mr. Ronald Myers (Public Finance The World Bank team consisted of Ms. Iryna and Governance Consultant). Shcherbyna (Senior Public Sector Specialist; the Team Leader), Ms. Natalia Manuilova (Senior The Bank team would like to acknowledge the efficient Financial Management Specialist); Mr. Jay-Hyung cooperation and assistance received from the Ministry Kim (Adviser), Mr. Ian Halvdan Ross Hawkesworth of Economy of Ukraine in coordinating this activity (Senior Governance Specialist), Mr. Dmytro Donets from the Ukrainian side, in particular from Ms. Julia (Procurement Specialist), Mr. Simon Groom, Mr. Skubak (Director of the Investment Department, Eduardo Aldunate and Mr. Martin Darcy (International Ministry of Economy of Ukraine), Ms. Inna Grygorenko Consultants), Ms. Irina Zapatrina, Ms. Inna (Deputy Director of the Investment Department - Samchynska and Ms. Tetiana Tavlui (Consultants). Mr. Head of the Division of State Investment Projects), Mr. William Gallagher (Consultant) provided professional Pavlo Onishchenko (Head of the Economic Strategy editorial review of the report. Ms. Vitalina Vergeles Department of the Department for Strategic Planning (Program Assistant) and Ms. Daria Gulei (Program and Macroeconomic Forecasting), Ms. Oksana Assistant) provided logistics and administrative Rosada (Deputy Director of the PPP Division of the support. Investments Department); Mr. Niko Gechechyvadze (Head of the Public-Private Partnership Support The assessment team is grateful for the valuable Agency), Ms. Tetiana Sinayeva (Deputy Director of advice of Mr. Daniel Boyce (Practice Manager, the the Financial Planning and Analysis of Public Sector World Bank) and Mr. Arup Banerji (Country Director Entities Division of the Department for Management ECCEE). of State Property Objects, Investments Department). The team is grateful to all government institutions involved for their constructive collaboration at the stages of assessment and report finalization. 4 Acknowledgements ABBREVIATIONS AND ACRONYMS ACU Accounting Chamber of Ukraine MoE Ministry of Economy of Ukraine BCU Budget Code of Ukraine MoF Ministry of Finance of Ukraine CBA Cost Benefit Analysis MoI Ministry of Infrastructure CC Climate Change Non-SSIP State Investment Projects included in the budget outside of the CEA Cost Efficiency Analysis regulated selection process CMU Cabinet of Ministers of Ukraine PEFA Public Expenditure and Financial CPI Consumer Price Index Accountability DIAM State inspection for Architecture PFM Public Finance Management and Urban Planning PIM Public Investment Management EIA Environmental Impact Assessment PPL Public Procurement Law EU European Union PPP Public Private Partnership FY Fiscal Year SEA Strategic Ecological Assessment GHG Greenhouse Gas SOE State-owned enterprise IA Internal Audit SSIP Selected State Investment Projects IFI International Financial Institution UAH Ukrainian Hryvnya MDA Ministries, Department, Agencies USPs Unsolicited proposals for PPP which are Key Spending Units projects M1 (WL) Weakest link method VRU Verkhovna Rada of Ukraine M2 (AV) Averaging method (Parliament) Ministry Ministry of Ecology and Natural WHO World Health Organization of Ecology Resources of Ukraine Currency and Indicative Exchange Rates À Local Currency Unit: Ukrainian Hryvnya (UAH) À 1 US$ = 26.33 UAH as of October 31, 2021 À 1 EUR = 30.50 UAH as of October 31, 2021 Fiscal Year À January 1 – December 31 Abbreviations and acronyms 5 OVERVIEW 1. The main purpose of the 2021 Public Investment resources available to finance reconstruction have Management (PIM) Diagnostic Assessment is plunged, with the country's economy expected to to provide the Government of Ukraine with shrink by an estimated 35 percent in 2022, depending an objective diagnostic of national-level PIM on the duration and intensity of the war.2 The performance in comparison to international good displacement of significant numbers of people within practice, with recommendations for improvement. the country further complicates the planning and This report identifies key strengths and weaknesses in prioritization of public investments. the design and performance of Ukraine's PIM system and contains a set of actionable recommendations 4. Post-war reconstruction inevitably introduces to correct the weaknesses over the short and long new challenges and exacerbates existing ones. term. The assessment also provides a baseline With this in mind, addressing the PIM pre-war against which to measure the success of future PIM drawbacks prior to reconstruction will yield great reforms. In addition to the core assessment, separate benefits. More than ever, an effective PIM system will assessments of climate change (CC) considerations in be required to ensure that scarce financial resources PIM and of PIM for public private partnerships (PPPs) are directed towards high quality, priority projects and state-owned enterprises (SOEs) were performed. that are then executed efficiently. Opportunities for corrupt practices can quickly emerge in emergencies 2. Although the assessment started in 2021, the when there is pressure to dilute robust project impact of the Russian invasion was considered in implementation procedures in the interests of quick formulating final recommendations. Because the results. Efficiency will be paramount as fast recovery Russian invasion radically changed the context and of infrastructure will be required once the war ends. focus of public investment near the conclusion of the core assessment, important issues emerging for the 5. This assessment has been performed using post-war reconstruction phase have been addressed the World Bank's most recent (2015) PIM in a dedicated chapter. assessment framework.3 This framework adopts the same principles as PEFA to score the dimensions 3. Russia's war against Ukraine has radically of the PIM system. There have been two previous changed the scale and nature of investment assessments of Ukraine's PIM system. The first, in requirements, while reinforcing the need for a 2012, was performed by the World Bank and the more robust, but agile PIM system to meet the second, in 2016,4 was performed by the IMF. Due to challenges of reconstruction. The direct damage methodological differences, direct comparison of caused to Ukraine's infrastructure during the war results from the three assessments is not possible. has reached a value of over USD110.4 billion, The CC assessment has been guided by two tools: i) with minimum recovery needs for replacement the PIM indicator of the Climate Responsive Public of destroyed assets set at USD188 billion,1 or 94 Financial Management Framework ((CRPFM-5) and percent of 2021 GDP. At the same time, the economic ii) Dimension 5 (Questions 1-3) of the World Bank's  1  Estimate by the Kyiv School of Economics, as of beginning of August 2022. https://kse.ua/about-the-school/news/direct- damage-caused-to-ukraine-s-infrastructure-during-the-war-has-reached-over-110-4-bln-minimum-recovery-needs-for-destroyed- assets-188-bln/  2  World Bank estimate: https://www.worldbank.org/en/country/ukraine/overview#3  3  'A New PIM Diagnostic Indicator: Strengthening the PIM Component of the PEFA Framework', J. Kim, G. Glenday & N. Biletska, World Bank 2015  4  The assessment was carried out in 2016, but not released until 2019: Ukraine Public Investment Management Assessment: Technical Report, IMF, June 2016. 6 Overview Infrastructure Governance Assessment Framework analysis presented in Figure 1, project appraisal and (InfraGov). When reviewing investments made by selection are consistently the weakest areas of PIM SOEs, indicator 5c of the IMF PIMA Framework, in most countries. Multiyear budgeting also scores alongside the indicators in the WB PIM DA framework, poorly, while national and sectoral planning tend to be were considered. weaker in emerging market countries. 6. Ukraine has made significant progress in 7. The Government's Public Finance Management improving PIM since 2015 and the level of its (PFM) Strategy for 2022-2025 envisages further performance is similar to comparator countries strengthening of the PIM system. Despite the across the world. The improvement began in 2015 mentioned improvements, Ukraine's PIM system still with amendments to the Budget Code, which clarified suffers from deficiencies – it remains fragmented, the roles and responsibilities of the main actors. In short of capacity to carry out adequate due diligence, 2015 the government also approved Resolution No. prone to corruption, and lacking in strategic 571 which established transparent appraisal and orientation. The large overhang of underfunded selection procedures, independent review, regular ongoing projects is one symptom of these continuing monitoring, and adjustment of state investment problems, as are the significant variances from projects. In 2016, the MoE approved basic procedures budget and schedule for completed projects. The PFM for the monitoring of public investment projects, Strategy Action Plan includes specific reform steps and followed by guidelines for elaboration of a state performance indicators aimed at addressing the PIM investment project in 2017. As illustrated by the IMF system's shortcomings. Figure 1. Ranking of PIM Institutions by Scores in Design and Country Group 1. Fiscal Rules 15. Monitoring of Assets 10 2. National & Sectoral Planning 14. Project Management 3. Central-Local Coordination 5 13. Transparency of Execution 4. Management of PPPs 0 12. Availability of Funding 5. Company Regulation 11. Protection of Investment 6. Multiyear Budgeting 10. Project Selection 7. Budget Comprehensiveness 9. Project Appraisal 8. Budget Unity Advanced Economies (n=1) Emerging Markets (n=15) Low-Income Developing Countries (n=14) Source: Public Investment Management Assessment – Review and Update, IMF, 2018 Overview 7 8. According to the assessment, clear roles and 9. The key recommendation of the report is responsibilities exist in PIM, but institutional to unify the PIM procedures, including the arrangements open up a "bypass" route that terminology applied to the PIM system, and to allows state budget funded investments to eliminate bypass rotes. A tight definition of an legitimately circumvent the PIM procedures. investment project is the key to closing the bypass This route is due to the non-mandatory nature of route and should be implemented in the short procedures caused by weaknesses in the terminology term through amendments to the Budget Code. and its application. The term "investment project" is Simultaneously, the MoF should be authorized, included in both the Budget Code and in the Law on through the Budget Code, to perform a gatekeeper Investment Activities, which allows for established function, allowing it to block projects that have not procedures to be circumvented in many investment been appraised or properly selected from being projects. Further, no organization is performing included in the budget on the initiative of government the "gatekeeper" role to prevent projects that entities and Parliament. Some international have not been positively appraised from getting experience in enforcing a unified entry point for funding. Different PIM streams exist with their own projects seeking state support is illustrated in Box 1. specific procedures: (1) direct budget financing (state capital investments); (2) state budget support 10. Combined with fundamental measures such (state guarantees, budget lending); (3) PPPs; and (4) as unification of PIM procedures, Ukraine can also corporate sector investments (SOEs' investments benefit from measures that respond to the various of own funds). These mechanisms are guided by constraints and urgency that the reconstruction different procedures between, and even within, each will present, such as using simplified appraisal means of financing, depending on the sources of methods. In the short term, these could be funding and the nature of the property in which the introduced across a range of project sizes, to facilitate investment is made. the infrastructure reconstruction phase. In the International Experience in Preventing the "By-Passing" of Established PIM Procedures BOX 1 Chile Croatia Chile's National Public Investment System is Croatia's Budget Act (Article 45) specifies backed by legislation and is the single entry that users of the state budget may assume point for projects to access budget funding. commitments under investment projects only Since the 1980s, all public bodies – ministries, after an expert evaluation, and feasibility and regional governments, municipalities, publicly efficiency assessment of any such investment owned companies, and public services – wishing project, are completed. These provisions apply to undertake an investment project or program, also to local and regional governments. Applicable must apply initially through the public investment assessment methods and approval procedures system for funding (Article 19bis of Law 1.263, are issued through a government directive. The Ley Orgánica de la Administración del Estado). same act (Article 124.10) authorizes financial Only initiatives that have been evaluated can penalties against legal persons for breaching be undertaken within the public sector. The these provisions. only exceptions (which are very rarely used) are defense projects and initiatives of the President. In 2010, the system was extended to concession projects where the private sector finances the investment. 8 Overview absence of full economic appraisal, metrics such as completion review of all reconstruction projects the number of people benefiting from the restoration and rapid analysis of results by central agencies to of services and the cost per user compared to similar feed lessons back into the continuing reconstruction projects (and considering that in some remote areas program will address those problems and minimize the cost of services will be higher, and the density of risks. A unified approach to PIM monitoring and people served will be lower) are useful. In addition, reporting will also streamline the provision of line ministries should be made responsible for information to Ukraine's development partners. presenting complete and coherent projects that do not involve slicing up larger projects to fit below the 12. Introduction of a government "center of value threshold for application of more advanced excellence" can resolve the problem of inadequate project appraisal procedures and methods. The MoE institutional capacity. The government PIM function should be given a right and responsibility to reject has been significantly understaffed (in number and "projects" that are obviously subsets of bigger projects skills) at each stage of the PIM cycle, from preparation to send them back to the ministries for proper to implementation. A unit at the center of government configuration. – a "center of excellence" – can provide additional capacity to line ministries with the largest needs. It is 11. A unified monitoring of the implementation important to note that this body's role is to support of the whole reconstruction program (regardless the successful delivery of large, complex, and risky of funding source) should be established. Without projects, including by outsourcing of required skills. a unified approach, fragmented monitoring and an However, to avoid conflicts of interest, such a center inconsistent approach to adjustment and re-appraisal cannot be granted rights to select and/or finance of investment projects (which have been identified public investment projects. The UK is a good example as problems in the assessment), will increase due to of the establishment of a center of excellence in accelerated preparation of reconstruction projects, project preparation and implementation called the potentially based on incomplete information. Basic Infrastructure and Project Authority (see Box 2). United Kingdom Infrastructure and Project Authority BOX 2 Reporting jointly to the Office of the Cabinet project to support decision-making and inform and the Treasury (the finance ministry) the approvals by the relevant ministry and the Infrastructure and Project Authority (IPA) has Treasury. the role of supporting the successful delivery of À Carrying out and supporting ad hoc large, complex, and risky projects. It has no direct delivery performance reviews - triggered by involvement in project implementation and no implementation performance issues - and approval or decision-making role. escalating issues that cannot be resolved to higher levels of authority (accounting officers The IPA's main functions are: and ministers). À Maintaining the system for monitoring À Working with ministries to build capability in government's major project portfolio, project management, including setting up the including: Major Projects Leadership Academy.  Preparing an annual report on progress in À Promoting transparency concerning major delivery of around 150 major projects projects through public information flows.  Running the "traffic-light" warning system for delivery confidence À Setting up and coordinating a system of The IPA has a staff of around 180, including integrated assurance and approval plans, analysts, policy advisors, commercial specialists, which is a schedule of quality assessments project delivery professionals, project finance over the planning and implementation of a professionals, and strategic delivery advisors. Overview 9 13. It will be important for sustainability of 16. Despite a successful upgrade of the PPP legal infrastructure to take the first steps towards framework, better harmonization with the Budget climate change-informed PIM immediately Code and strengthened transparency is needed during the reconstruction. This is in consideration in the mid-term. Among other improvements, of the significant role of infrastructure through two implementation of 'government-pays' PPPs should be relationships: (i) Climate change  infrastructure, allowed for in the Budget Code. Lack of 'government since public infrastructure is increasingly exposed pays' PPPs is a serious constraint on this commonly- to the risk of damage from climate-related disasters used funding framework. Although direct project and needs to be designed and operated in ways that financing for PPPs with state budget funds is reflect these risks; (ii) Infrastructure  climate change, allowed, there has been no case of the government since greenhouse gas (GHG) emissions are generated funding such a project to date. Despite the broad directly during construction and operation. requirements for transparency of information on PPP projects in the legislation (especially for 14. Although the existing legal framework concessions), there is a widespread problem with incorporates some formal elements of climate the practice of regularly applying a "confidential change considerations in PIM, it would be information" status. The confidential status is at beneficial to make these considerations more odds with international good practice, which restricts explicit, introducing budget tagging of CC related confidentiality to commercially sensitive information expenditures, and strengthening the role of the only. The absence of an independent appeals process Ministry of Ecology. The legal framework includes for private partners participating in PPP procurement/ regulations about public investments that have concessions can also act as a deterrent. impacts on CC variables and GHG emissions, although only applied after project selection. CC should be 17. The government has a detailed oversight considered in earlier PIM stages and evolve and adapt function of State-Owned Enterprises (SOEs), but to future CC-induced hazards. National strategies transparency concerning the overall scale of or projects must consider Ukraine's international budgetary support for SOEs' investment requires commitments. The Ministry of Ecology shall provide further improvement. The level of SOE capital advisory and supervision support for related projects, investments financed using budgetary lending and issue guidelines for CC consideration in project state guarantees exceeded 20 percent of total public designs, appraisal, and selection. Ukraine's budget capital spending during the evaluated period but no system has many important elements of performance- information on this support was included in central based program budgeting which could be useful government fiscal reports. Since SOEs use various in tagging CC expenditures. A definition of "climate sources of funding for their capital investments, change expenditure" should be included in the Budget they also follow different financing streams in each Code. Once legally established, such a definition would case, which may or may not be based on clear rules. be the starting point for the government to improve Depending on the relevant investment financing CC-informed strategic planning in the medium term, streams, SOEs are guided by twelve legislative acts to and to identify and present costed CC measures in receive state support for their investment projects. budget plans and reports. Legal conflicts are observed in these complex arrangements. Moreover, the legal framework does 15. The management of public-private partnership not prevent the "bypass route," in which SOEs can (PPP) projects has improved in some areas since also receive budget funds within a budget program the 2012 PIM assessment, notably in the legal requested by the line ministry. SOEs also do not have framework. There has been significant improvement clear internal rules for investment projects by own in the legislative framework for PPPs over the last five funds; only about a quarter of capital investment years. The adoption of the law on concessions and projects follow clear procedures. the law on PPP, as well as other legislative acts, has improved the PPP management system so that it now corresponds to good international practice. 10 Overview 18. The main high-level recommendation for 21. An integrated public investment management improving PIM for SOEs is that guidelines for information system (PIMIS) will improve internal investment management procedures be information sharing across the project cycle and developed in the post-war period. Even though should be developed over the medium term. The SOE investment projects follow different financing PIMIS should cover all business processes (the PIM streams with different requirements, the SOE itself stages) and all investment streams. To control the must have a comprehensive picture of the effective entry and flow of projects through their stages of implementation of its investments in the mid-term. design, appraisal, and approval, it is important that all This can be achieved through a unified approach for the key features of the projects and their development all SOE investment projects covering assessment, and approval status be entered. selection, and inclusion in a mid-term investment plan. 22. A comprehensive set of recommendations is 19. The introduction of a long-term national provided in Section 3.2 of Chapter 3, Chapters 4 infrastructure investment strategy as part of to 7, and in Table 30. The table provides a useful the country's regular strategic planning is vital breakdown between short-term/postwar priorities and for the effectiveness of public investments. those that may require more time to be implemented.   Implementation of such a strategy should then be detailed for the mid-term within a comprehensive public investment expenditure plan. Both instruments have a crucial role for steering financial resources towards investment in areas where creation or improvement of public services is most urgent, while at the same time considering sectoral development. Projecting forward to the post-war environment, the national reconstruction strategy may eventually evolve into a national infrastructure investment strategy. A realistic mid-term public investment expenditure planning should be introduced, covering all public investment streams involving budgetary funds or other state support (e.g., state loan guarantees and support to PPPs in any form). 20. The existing databases are created for different purposes and for different types of projects, making information sharing across the PIM system fragmented with consequences for transparency and portfolio management. There is no unified database that contains comprehensive information on all public investments at different stages in the project cycle. It is neither possible to track individual projects across the project cycle using a single information source, nor is it possible to obtain a full picture of the total number of projects (and their values) at different points in the project cycle. This fact makes it complicated to understand the progress of individual projects and to get comprehensive insights into the status of the project pipeline and implementation of the portfolio of ongoing projects. Overview 11 R 1 PTE CHA BACKGROUND AND APPROACH TO THE ASSESSMENT 1.1. Context for PIM 24. Moreover, the destructive impact of the war on infrastructure is immense and continues to Assessment mount. Although the assessment does not cover 2022, the current situation emphasizes the importance 23. While the situation has not been static, public of improvements to PIM, and many of the findings investment management is still recognized remain highly relevant as set forth in a supplementary as one of the weakest aspects of public chapter "PIM for Post-War Reconstruction." According financial management in Ukraine. PIM has been to the Ukraine Rapid Damage and Needs Assessment characterized by a high level of discretion at various – August 2022 report,7 as of June 1, 2022, direct stages of the PIM cycle. Inefficiencies in the PIM damage has reached over US$97 billion, with housing, system and a weak governance framework push the transport, and commerce and industry being the most cost of construction in Ukraine almost to EU and US affected sectors. Reconstruction and recovery needs, levels, despite lower labor costs. These problems are 5 as of June 1, are estimated at about US$349 billion, magnified by Ukraine's vast public investment needs – which is more than 1.6 times the GDP of Ukraine in an estimated USD100 billion and more up to 2030, as 6 2021. Integrated into these needs are critical steps estimated before the war. toward becoming a modern, low-carbon, disaster- and  5  The construction of 1 kilometer of the road in Ukraine costs about 90 million hryvnia ($ 3.2 million). According to the ARTBA (American Road and Transportation Builders Association), it costs about USD 2 to 3 million per mile in rural areas to build 1 mile of a new 2-lane unshared road and USD 3 to 5 million in urban areas; about USD 4 to 6 million costs to build 1 mile of a 4-lane road in rural areas and USD 8 to 10 million in urban areas (a mile is 1.6 kilometers). https://economy.24tv.ua/vartist-ukrayinskih-dorig-2020-chomu-novini-ekonomiki-ukrayini_n1477613  6  https://zakon.rada.gov.ua/laws/show/179- 2021 -%D0%BF#n25  7  The report was jointly prepared by the World Bank, the Government of Ukraine, and the European Commission: https://www. worldbank.org/en/news/press-release/2022/09/09/ukraine-recovery-and-reconstruction-needs-estimated-349-billion?fbclid=IwAR0 IXweVnvhcebI0kiIAI6NN_nccxSXqI0u5MDT7rDt5arg0OobWtyigNKc 12 Chapter 1. Background and Approach to the Assessment climate-resilient, and inclusive country that is more procedures for public investment projects. This closely aligned with European Union standards. To resolution identified the Ministry of Economy (MoE) as fund a whole National Recovery Plan to rebuild its the lead institution responsible for the management shattered infrastructure and revitalize the economy of public investments. In 2016, MoE developed basic after war with Russia the government estimates needs procedures for the monitoring of public investment of USD 750 billion. 8 projects. The key reforms of this period are summarized in Box 3. 25. Prior to 2015, appropriate mechanisms for transparent and efficient management of public 27. Despite the improvements implemented during investments were missing. The key principles and 2014-2016, the PIM system is still fragmented, criteria for allocation of public funds to investment lacking in strategic orientation, and prone to projects had not been defined. Consequently, corruption. The main issues arise at the procurement state budget funds were allocated to investment stage with significantly more attempts to exclude projects without requiring a feasibility assessment procurement from the scope of competitive tenders or adequate justification. Projects to be financed by in 2021, and weak public asset accounting, reporting, the state budget were selected based on discrete and monitoring (see Box 4). Areas of continued agreements rendering the PIM system inefficient and weakness include strategic prioritization, criteria- nontransparent. based selection, project database, investment portfolio management, and systematic monitoring 26. Following recommendations from the World of investment projects against implementation Bank, Ukraine adopted a series of PIM reforms milestones. The lack of formalized criteria for project during 2014-2016, beginning with amendments management arrangements, the absence of a cadre of to the Budget Code which clarified the roles professional managers of large-scale projects in public and responsibilities of the main actors. In 2015 agencies, and questionable independence of appraisal the government approved Resolution No. 5719 all add to the problematic state of PIM. establishing transparent appraisal and selection PIM Reforms of 2014-2016 BOX 3 The most important reforms during 2014-2016 c. Cost-benefit analysis methodology was were: adopted through a specific value of the social- discount rate. MoE applied the methodology a. Amendments to the Budget Code of Ukraine to selected projects. were adopted by the Parliament in December 2014, which identified the roles and d. An Inter-Agency Commission was created responsibilities of the key PIM stakeholders, comprising the key Ministers and members incorporated PIM into the budget process, of the Parliament's Budget Committee. The introduced transparent project selection, and Commission has been selecting projects and ensured the sustainability of financing. determining their budget allocations since the State Budget 2016. b. Resolution No. 571 introduced step-by- step guidelines for preparation of public e. Publication of project concept notes, project investment projects, appraisal and selection selection results, and planned budget based on the identified criteria. allocations on MoE's website was mandated.  8  Slide #8: https://uploads-ssl.webflow.com/621f88db25fbf24758792dd8/62c166751fcf41105380a733_NRC%20Ukraine%27s%20 Recovery%20Plan%20blueprint_ENG.pdf  9  Resolution No. 571 of 22.07.2015 of the Cabinet of Ministers. Chapter 1. Background and Approach to the Assessment 13 Findings of the Corruption Perception Index 202110 BOX 4 In the 2021 the Corruption Perception Index of the National Children's Specialized report, Ukraine scored 32 points out of 100 Hospital, Okhmatdyt"11 under the negotiation possible. Its score decreased by one point to procedure; prior year, and Ukraine ranks 122nd out of 180 À Excluded the construction of Kyiv Ring countries. Eswatini (formerly Swaziland) scores Road from the scope of the Law on Public the same. Zambia, Nepal, Egypt, the Philippines, Procurement;12 and Algeria are one point ahead. À Allowed the construction of the Dniester Introducing transparent and accountable PSPS and repairs to compressor stations for management of public assets and guaranteeing gas pipelines without competition, under the further development of the procurement sector negotiation procedure.13 were among core recommendations for Ukraine. According to the State Property Fund of Ukraine There were significantly more attempts to use (SPFU), as of July 1, 2021, more than 1 million non-competitive procurement in 2021 and objects of public property are registered in early 2022 than previously. For example, the Ukraine. However, only access to information on Parliament: assets that are subject to privatization or lease is À Allowed the purchase of everything necessary facilitated. Public property accounting and access for the Constitution Day and Independence to information therefore needs to be improved. Day, including the medical equipment, within Reflecting some progress, changes to legislation the public investment project "Construction and a new register were in development as of the of a modern medical and diagnostic complex end of 2021. 28. In response to the continuing challenges, 29. The capacity to develop project feasibility the government's Public Finance Management studies is improving but is still inadequate. Project (PFM) Strategies for 2017–2020 and for 2022- initiators often do not allocate appropriate time for 2025 envisaged further strengthening of the financial and procurement procedures, and other PIM system. Important measures remaining to circumstances that may impede timely and successful be completed are: (i) implementation of a mid- project implementation. The project disbursements term plan for priority public investments; (ii) the are, as a rule, lower than planned for a budget year. development of the capacities of line ministries to Because of the lack of funding, project initiators manage investment projects, including performing often elaborate the feasibility study on their own, cost-benefit analysis; (iii) consolidation and analysis of with consequences for quality, although the quality basic data on investment and operating costs based has been improving. According to the MoE's annual on recent representative projects ; (iv) introduction report, the tendency of investment projects admitted of a centralized monitoring system for large-scale by MoE to the selection procedure has also improved, public investment projects; and (v) development of showing the positive effect of moving up the learning comprehensive project management guidance on curve – 15.1 percent refused in 2020 compared to 27.5 investment projects financed from local budgets. percent in 2017.  10  https://ti-ukraine.org/en/research/corruption-perceptions-index-2021/  11  Resolution No. 712, dated July 14, 2021.  12  Law No. 1530, dated June 3, 2021.  13  Law No. 2009, dated January 26, 2022. 14 Chapter 1. Background and Approach to the Assessment 30. The portfolio of ongoing public investment À To identify short- and mid-term priorities for projects has consistently exceeded the country's future PIM reforms, feeding into the design of financial capacity, with a backlog of stalled, the updated PFM reform strategy for 2021-2024 unfinished projects. The estimated funding but also considering the dramatic infrastructure requirement to complete state investment projects damage caused by the Russian invasion. selected for 2021 amounted to UAH 79.2 billion, 14 À To provide a baseline against which to monitor the while relevant expenditure included in the budget success of new reforms. 2021 budget was UAH 4 billion, or 5.1 percent of the need. This fact means it will take 20 years to complete these projects at this rate. Despite a general understanding of the necessity to optimize the public investment portfolio, the total number of unfinished 1.3. Methodology for the public investment projects and their costs have not Assessment been fully identified. The lack of institutional capacity for asset management further complicates all the existing problems of the investment portfolio. The Core Assessment 33. There have been two previous assessments of Ukraine's public investment management (PIM) system. The first, in 2012, was performed 1.2. Background to the by the World Bank and the second, in 2016,15 was Assessment and Objectives performed by the IMF. The 2012 assessment used the World Bank's prototype PEFA-based framework, while the 2016 assessment used the IMF's 2015 PIMA 31. An assessment of Ukraine's PIM system was framework. requested by the Ministry of Economy. This assessment was carried out between July 2021 and 34. The current assessment has been performed May 2022 by a World Bank team working closely with using the World Bank's most recent (2015) PIM counterparts from MoE. The goal of the assessment is assessment framework which is an improvement improved targeting of public investments in line with over the 2012 version.16 Consisting of 23 indicators strategic policy priorities of the government. organized around the eight core "must have" PIM functions,17 the new framework is a PEFA-style tool 32. The main objectives of the current PIM embodying the structured objectivity of PEFA applied assessment are: to PIM. By choosing to use the 2015 framework, a À To provide the government with an objective direct comparison with the 2012 indicators is lost, and up-to-date diagnostic of national-level PIM but comparison at the level of the eight "must have" performance compared to international good functions is still possible. The new framework helps practice; to identify gaps in the current system more precisely than the previous one and, hence, helps to design À To monitor the results achieved through recent a detailed reform program. It will also create a solid PIM reforms; basis for benchmarking future progress. The cut-off date for the current assessment is October 31, 2021.  14  Based on the procedures set out in Resolution No. 571.  15  The assessment was carried out in 2016, but not released until 2019: Ukraine Public Investment Management Assessment: Technical Report, IMF, June 2016.  16  'A New PIM Diagnostic Indicator: Strengthening the PIM Component of the PEFA Framework', J. Kim, G. Glenday & N. Biletska, World Bank 2015  17  Chapter 2, 'Power of Public Investment Management, Transforming Resources into Assets for Growth', A. Rajaram, et al., World Bank, 2014 Chapter 1. Background and Approach to the Assessment 15 35. Like PEFA, the WB PIM assessment framework the share of annual investment expenditures and the is evidence-based and involves amalgamation of share of the value of the ongoing project portfolio. scores for different dimensions of an indicator to arrive at an overall score. Dimensions are rated A to D, as with PEFA, and an absence of evidence incurs the lowest rating of D. Dimension ratings Subsidiary Assessments: are amalgamated as appropriate using one of the Climate Change, PPPs, and two PEFA scoring methods, either weakest link (M1) or averaging (M2).18 The dimensions of individual SOEs indicators can assess either institutional/system 37. Subsidiary assessments have been carried out design – de jure characteristics - or the effectiveness to accompany the 2015 assessment framework of application of the design - de facto characteristics. in the important areas of climate change, Reflecting the fact that resources do not allow public-private partnerships, and investment by assessment of the entire population of projects, for state-owned enterprises (SOEs). The assessment some indicators, information on specific projects has framework does not include assessment of the been sampled from three sectors, healthcare, energy management of investment by SOEs nor does and ecology. The final output of the assessment is it consider the extent that climate change is a performance report, which includes a summary mainstreamed in the PIM system. These areas have of PIM performance and recommendations by been assessed in qualitative terms and are the subject functions (Section 3.2 of Chapter 3). Strategic of separate sections of the report. Likewise, the recommendations for PIM are also presented (see assessment of PPPs has been kept separate, reflecting Chapter 7) but these include recommendations the distinct institutional arrangements for PPPs in related to three subsidiary assessments and PIM for Ukraine. post-war reconstruction (see next section for further explanation). 38. It has become critical for PIM to consider climate change in terms of recognizing the 36. The assessment is complicated by the bypass positive and negative effects of a project on route (see Figure 5) that allows many projects climate change and determining optimal climate- to avoid the de jure preparation, selection, and proofing adaptations. Projects have impacts on monitoring processes set out in Resolution No. climate change and are, in turn, affected by climate 571 and supporting guidance. Despite the fact change. The assessment framework has been guided that the term "investment project" is included in by and is consistent with two tools with a climate both the Budget Code and in the Law on Investment change focus: (i) the PIM indicator (CRPFM-5) of the Activities, it has not prevented many investment Climate Responsive Public Financial Management projects from circumventing established procedures. Framework (see Tool I in Annex 1); and (ii) Dimension The principle applied during the assessment is that 5 (questions 1-3) of the World Bank's infrastructure effectiveness dimensions apply to the whole universe governance assessment framework (InfraGov) (see of projects, while institutional dimensions apply to Tool II in Annex 1). As shown in Annex 1, CRPFM-5 the quality of system design (even if not universally is more directly aligned with individual indicators in applied). This approach ensures that good system the World Bank's 2015 PIM assessment framework design is recognized, even if its application is uneven. than InfraGov, but both have provided useful insights. Section 2.2, Institutional Mapping, explains the Climate change has been addressed in qualitative relative importance of the bypass route in terms of terms, i.e., not formally scored.19  18  The weakest link method involves using the lowest scored dimension (the 'weakest link') as the basis for the indicator score. A '+' is added if other dimensions are scored higher. The averaging method involves averaging dimension scores to arrive at the indicator score. Tables for averaging are the same as those used for PEFA.  19  Scoring is not considered appropriate at this stage given the newness of PEFA climate and the attention to climate change in PFM in general. 16 Chapter 1. Background and Approach to the Assessment 39. The assessment of management arrangements Supplementary Discussions for public investment through public-private partnerships (PPPs) has been performed and Recommendations. PIM separately from the main assessment. Although for Post-War Reconstruction PPPs are just another modality for implementing and financing public investment projects, their distinct 41. The war has a significant impact on the place in Ukraine's PIM system warrants separate operation and future development of the PIM treatment. PPPs have been assessed on the basis system. Despite the post-war condition not being the of performance in relation to the 8 "must-have" subject of the main assessment, the supplementary functions. All assessments have been expressed in discussions and recommendations describe qualitative terms. important issues arising from the new situation. The supplementary section looks at the recommendations 40. The quality of investment by state-owned from the core PIM assessment and indicates which of enterprises is an important weakness in Ukraine these will be important to incorporate in PIM for post- and the assessment needed to be responsive war reconstruction and which recommendations can to this issue. To capture SOE investment in the be adopted for post-war needs. The section covers all assessment it was decided to apply Indicator 5c of eight PIM functions. the IMF PIMA framework, alongside the indicators in the World Bank's 2015 framework. This allows a more global assessment of SOE governance through the questions: Does the government oversee the investment plans of public corporations (PCs) and monitor their financial performance? The assessment looks into SOEs' investment plans and their implementation across all funding sources, including those that are budget funded as part of the regular PIM cycle, as well as those funded by loans and own funds of SOEs. Chapter 1. Background and Approach to the Assessment 17 R 2 PTE CHA PUBLIC INVESTMENT EXPENDITURE AND INSTITUTIONAL MAPPING 2.1. Public Investment in 18 percent in Bulgaria, 22 percent in Lithuania, 25 percent in Georgia, 25 percent in Romania, 26 percent Ukraine in Moldova, and 31 percent in Estonia.20 43. Capital expenditure makes up a relatively small 42. During the assessed period, total investment in share of total general government expenditure the economy was low by international standards and its importance declined in 2020. Recurrent and dominated by private investment. In 2018 expenditure dominates general government and 2019, total investment (gross fixed capital expenditure and, in 2018 and 2019, capital formation) stood at 17.6 percent of GDP, declining expenditure only represented around 11.5 percent of to 13 percent in 2020. The decline in 2020 can be the total. In 2020, this share fell further to 10.6 percent explained by the COVID-induced economic downturn (see Figure 3). and declining investor confidence. Over the period of the assessment, the importance of public investment 44. The largest share of public sector capital has grown in each year from 12.7 percent of total expenditure in Ukraine is financed through local investment in 2018 to 19.1 percent in 2020 (see budgets. Local budget capital spending represents Figure 2). While both public and private investment over 40 percent of the total (see Table 1). However, declined in 2020, public investment held up better local budgets finance a part of that spending by capital than private, explaining the significant increase in transfers from the state budget, which increases share. In comparison, gross fixed capital formation as annually. Thus, in 2018 such transfers amounted to percent of 2020 GDP stood at 17 percent in Poland, UAH 16 billion, or 17.7 percent of total local budgets  20  https://data.worldbank.org/indicator/NE.GDI.FTOT.ZS 18 Chapter 2. Public Investment Expenditure and Institutional Mapping Figure 2. Gross Fixed Capital Formation Figure 3. Structure of Consolidated Budget (% of Total) Expenditures (%) 100 100 90 90 80 80 70 70 60 80.9% 60 87.3% 85.4% 88.5% 88.6% 89.4% 50 50 40 40 30 30 20 20 10 19.1% 10 12.7% 14.6% 11.5% 11.4% 10.6% 0 0 2018 2019 2020 2018 2019 2020 Public Investments Private Investments Capital Expenditures Recurrent Expenditures Source: State Statistic Service, Bank Staff Source: Treasury, Bank Staff Calculations Calculations capital spending; in 2019 – UAH 19.4 billion, or 19.6 rose to 31.9 percent of the total and SOE spending percent; and in 2020 – UAH 24 billion, or 24.4 percent. fell to 24.0 percent. This significant shift in shares is, Capital expenditures by SOEs have normally been the like other changes, probably related to effects on SOE second largest source, representing around a third investment plans brought about by the COVID-induced of the total in 2018 and 2019, but this was reversed economic downturn. 2018 and 2019 are therefore in 2020 when central government capital spending probably more representative. Table 1. Structure of Public Sector Capital Expenditures21 UAH union, million Structure, percent Year State Local SOE State Local SOE Budget Budgets Investment Budget Budgets Investment 2018 53,793.1 90,386.2 72,152.7 24.9% 41.8% 33.3% 2019 56,773.6 99,199.7 82,063.0 23.8% 41.7% 34.5% 2020 70,922.3 98,204.0 53,309.3 31.9% 44.1% 24.0% Source: Treasury, MoE, World Bank Team Calculations  21  Transfers have been netted out to avoid double-counting. Chapter 2. Public Investment Expenditure and Institutional Mapping 19 45. State budget investment expenditure must development projects aimed at regional development be distinguished from capital expenditure. and meet the priorities defined in State Strategy for Capital expenditure, beside public investments, Regional Development and relevant development includes construction of housing, capital repairs and strategies of the regions. small acquisitions of equipment. Thus, investment expenditure and capital expenditure should not 47. Part of the public investments funded from the therefore be compared. state budget are implemented via SOEs, but the share of such investments is not significant. Budget 46. An important share of investment expenditure funding for capital investments implemented by SOEs through the state budget is made up of capital had been historically low – around 1-2 percent in 2016- transfers to local government and funding for 22 2018 – and growing to 8 percent in 2020. As evidenced the State Fund for Regional Development. In 2018 by the MoE reports and the list of priority investment and 2019, direct funding of investment projects projects approved by the CMU,23 the majority of SOE represented just under 60 percent of the total, rising investment projects are financed by their own funds to 70 percent in 2020 (see Figure 4). The State Fund and by external borrowings. Such capital investment for Regional Development is one of the main financial projects follow the specific legislation on SOE financial instruments for the implementation of state regional and investment plans, as reviewed and approved by policy. Financing from this fund is directed to the the government annually. implementation of investment programs and regional Figure 4. The Structure of the Actual Capital Investments and Transfers 2018 - 2020 2.2. Institutional Mapping 100 48. The Ministry of Economy takes the lead in 90 coordinating and overseeing the public investment 29.8% 80 40.4% management system, but there are many other 41.6% players within the central government. Table 2 70 maps the organizations involved in PIM against their 60 roles and major tasks. It focuses on central 50 government and excludes state-owned enterprises 40 and sub-national governments. 70.2% 30 58.4% 59.6% 49. While Ukraine has formally established project 20 preparation and selection practices, the system is frequently bypassed. The rules of the system 10 cover procedures for "State Investment Projects" and 0 the term "investment project" is included in both the 2018 2019 2020 Budget Code and in the Law on Investment Activities. Despite this fact, MDAs are not restricted from naming Public Investments Capital Transfers their projects in a way other than "State Investment (excluding Capital and State Fund Project"; thus, MDAs might name the project Transfers and State for Regional generically as a "reconstruction project" or a "building Fund for Regional Development project," causing the legislation to be inapplicable. By Development) using this approach, MDAs do not need to spend time on the project development, preliminary screening, Source: Treasury, Budget Program Passports, and appraisal stage, and do not need to compete data received from the MDAs, State Statistic against other investment projects. Service, Bank Staff Calculations  22  Mostly these are unconditional capital transfers: only rarely are transfers tied to a particular project.  23  List of priority investment projects https://www.kmu.gov.ua/storage/app/uploads/public/5fd/c80/ee6/5fdc80ee67197829472716.doc 20 Chapter 2. Public Investment Expenditure and Institutional Mapping Table 2. Mapping of Organizations and Roles in Ukraine's PIM System – Central Government Organization Role/Major Tasks Comments Line Ministries À To screen concept notes for projects proposed for selection Excludes SOEs* and (19), Central according to Resolution No. 571 (SSIPs) , which includes 24 SNG Government projects for roads of national significance not covered by the Bodies (42) State Road Fund, that directly affect their central functions [*Excludes capital and take decisions on feasibility study preparation investments by SOEs completed with own À To conduct state examination of feasibility studies for SSIPs funding and funding and submit SSIPs to the MoE raised via commercial À To design non-SSIPs,25 including budgeting banks, IFIs or any À To implement SSIPs that receive funding, including other direct financing contracting other than budget.] À To monitor SSIPs and prepare monitoring reports À To receive and administer capital requests from agencies and other entities under their control State Road À To design, implement, and monitor SSIPs for roads of national Guidelines for the Agency of significance covered by the State Road Fund (Order No. determination of Ukraine 573/1019) the overall socio- (Ukravtodor) economic efficiency of new construction, reconstruction and repair of public roads were adopted on December 14, 2021, after the cut-off date for the current assessment. Ministry of À To conduct independent review of SSIPs and take decisions Limited in capacity Economy on their admission to selection to cover all SSIPs by À To propose the total amount of expenditures on SSIPs for the independent review next three years to the Ministry of Finance and monitoring À To support the selection process for SSIPs À To publish a list of SSIPs on its official website À To monitor SSIPs based on the monitoring reports prepared by line ministries, central government bodies and submit relevant recommendations to the Inter-Agency Commission on state investments projects À To prepare and submit to the State Audit Service proposals for conducting state financial audit of SSIPs À Development and maintenance of an Excel projects database for SSIPs À Procurement regulatory function À PPPs / Concessions26  24  A Selected State Investment Project (SSIP) is an investment project implemented through state investment in public property objects and selected for funding in accordance with the legally defined procedures.  25  State Investment Projects included in the budget outside of the formal selection process. The difference between and SSIP and a non-SSIP is explained more thoroughly later in this chapter.  26  Please see Table 26 for further details related to PPPs / Concessions. Chapter 2. Public Investment Expenditure and Institutional Mapping 21 Organization Role/Major Tasks Comments Inter-Agency À To review and select SSIPs Consists of 50 percent Commission À To consider monitoring results for investment projects and of the ministries' on State take appropriate decisions representatives and Investment 50 percent of the À To prepare and submit to the MoE proposals and Projects members of the recommendations on conducting independent economic Verkhovna Rada appraisals or examinations of SSIPs; conducting a state Budget Committee financial audit of SSIPs Ministry of À To determine the total amount of expenditures on SSIPs for Finance the next three years and include it in the Budget Declaration À Direct negotiations with public entities regarding funding for non-SSIPs À To determine budget allocations Accounting As the supreme audit institution accountable to Parliament Also analyzes the Chamber À To carry out financial and performance audits, develop annual report on proposals and recommendations on measures to be taken for execution of the Law elimination and prevention of violations and deficiencies, and of Ukraine on the State develop recommendations about improvement of relevant Budget for a relevant legislation, including procurement. For procurement, the AC year submitted by the audits procurements financed by state budget funds at all government stages of procurement, including verification of legality and effectiveness of procurement, transparency and compliance with the prescribed procedures, assessment of the timeliness of receipt of goods, services, and work. Anti-Monopoly À To receive and administer complaints about public Commission procurement procedures State Audit À Analysis and verification of the legality and effectiveness of Under the Ministry of Service investment projects of institutions under the control, which Finance are implemented on the basis of state and local investment, state support, state and / or local guarantees, and the state of performance indicators achievements, management and use of investments (funds). Source: Bank Staff Compilation 50. A significant proportion of projects take but reflects the arbitrary nature of interpreting the bypass route, undermining the integrity of Resolution No. 571 by disregarding the Budget Code's the PIM system and requiring any assessment requirement concerning selection of state investment to differentiate between de jure and de facto projects by the Inter-Agency Commission.27 Projects practices. Figure 5 shows the established procedures that are subject to Resolution No. 571 are subject and the bypass route. The bypass route is a legitimate to rigid application of the regulations and there are route using the mainstream budget process no violations. It should be noted that investment  27  Article 33-1. 22 Chapter 2. Public Investment Expenditure and Institutional Mapping Figure 5. Flow Chart of Project Preparation and Selection Practices in Ukraine MINISTRIES, DEPARTMENTS, AGENCIES Screening of Project Concept Notes for SSIPs THE BYPASS ROUTE State examination of Project Proposals for Other Capital Projects not considered as SSIPs State Investment Projects (non-SSIPs) MINISTRY OF ECONOMY MINISTRY OF FINANCE Independent review Non-SSIP within List of SSIPs Admission of projects to the selection budget programs INTER-AGENCY COMMISSION THE DRAFT BUDGET Project Selection projects within the Road Fund can be selected both 52. Spending on non-SSIP projects has consistently following Decree 571 and following the joint order been more than one-third of total domestic budget of the Ministry of Infrastructure and the Ministry of investment expenditure (excluding investment Finance dated September 21, 2012 No. 573/1019 spending on regional development), and has risen "About approval of the methodology for determining to over 60 percent in recent year. Table 3 shows the amount of financing for the construction, annual spending by type of procedure. The scale reconstruction, repair and maintenance of automobile of the issue for regional development investment roads and the standards of expenses associated spending, which has its own formal procedures, is less with the maintenance of highways." The latter also significant, but still of concern. The share of non-SSIPs established selection criteria based on the results of in total investment spending for regional development socio-economic analysis. was around 40 percent in 2018 and 2019, rising significantly to 71.1 percent in 2021. Table 4 looks 51. Throughout this report, projects are at the investment through the non-SSIP route from differentiated based on whether or not they a portfolio perspective. It indicates that the share follow the formal selection process. Projects that of non-SSIPs in the total capital cost of the ongoing follow the formal appraisal and selection procedures portfolio (excluding IFI financed projects) fell from are referred to as selected state investment projects 45.0 percent to 22.6 percent during the assessment (SSIP);28 a project that bypasses the formal PIM system period, with an average of 33.6 percent. The share is referred to as a "non-SSIP." of non-SSIPs rose again in 2021 to 46.3 percent. As a share of the portfolio including IFI projects the share is considerably lower, averaging 19.3 percent for 2018- 2020.  28  State investment project is an investment project implemented through state investment in public property objects Chapter 2. Public Investment Expenditure and Institutional Mapping 23 Table 3. Public Investments Spending by Type of Procedure (UAH million) 2018 2019 2020 2021 IFI projects 5,911.2 9,564.1 8,691.5 8,724.5 Total state budget investments 9,191.7 11,419.8 15,643.2 24,809.0 SSIP 4,914.9 7,946.3 10,978.9 9,339.7 share in total state budget investments, % 53.5% 69.6% 70.2% 37.6% Non-SSIP 4,276.8 3,473.4 4,664.2 15,469.3 share in total state budget investments, % 46.5% 30.4% 29.8% 62.4% Total public investments in regional development 13,350.0 16,998.9 11,388.0 15,546.1 SSIP 8,050.0 9,920.0 7,686.4 4,500.0 share in total public investments in regions, % 60.3% 58.4% 67.5% 28.9% non-SSIP 5,300.0 7,078.9 3,701.6 11,046.1 share in total public investments in regions, % 39.7% 41.6% 32.5% 71.1% Total public investments 28,452.9 37,999.7 35,722.7 49,079.6 Source: Originally approved budgets for 2018-2021, budget program passports for 2018-2021, information received from relevant MDAs, World Bank Team estimates. Table 4. Share of Non-SSIPs in Total Capital Costs of Ongoing Investment Portfolio Average 2018 2019 2020 2021 2018-2020 Including Cost of IFI Projects 24.0% 19.0% 14.9% 19.3% 26.8% Excluding Costs of IFI Projects 45.0% 33.3% 22.6% 33.6% 46.3% Source: originally approved budgets for 2018-2021, budget program passports for 2018-2021, information received from relevant MDAs, World Bank Team estimates. 24 Chapter 2. Public Investment Expenditure and Institutional Mapping R 3 PTE CHA ASSESSMENT BY INDICATOR FOR PIM FUNCTIONS 53. This Chapter applies the World Bank's 2015 PEFA-style assessment framework for public 3.1. Details of the investment management in order to assess where Assessment Ukraine stands compared to good practice in different facets of its PIM system. For each of the 23 indicators, there is an introduction describing the PIM Function 1. Strategic rationale, followed by a summary table of the scores Guidance and Screening by dimension and the reasoning behind the scores. Each dimension of the indicator is then discussed in PIM-1. Sector Analysis and Planning detail to demonstrate the evidence supporting the score. Each dimension has been scored on a scale of RATIONALE A to D, with A being the highest score. The scores for each dimension are consolidated to give an overall 54. This indicator aims at analysis and assessment rating for the indicator (as shown in the summary of the capacities of MDAs to collect sector table). Dimension scores are amalgamated using specific data, to use it for analysis, and to apply either the critical link (M1) or averaging method (M2) it effectively in the planning of investments. The as defined by PEFA.29 Since some indicators have indicator focuses on the capacities of key sector five dimensions (not four, the maximum in PEFA), MDAs and analyzes available sets of data published where the M2 method is intended, the indicator rating by agencies. Moreover, it assesses the effectiveness has been estimated by extrapolation from the PEFA of MDAs in using available data and sector analysis averaging matrix.  29  See footnote 18 for an explanation of the differences between scoring methods. Chapter 3. Assessment by Indicator for PIM Functions 25 Summary of Scores and Performance Table PIM-1 Indicator/ Dimension Score Brief justification for score PIM-1. Sector analysis C Scoring Method M2 and planning 1.1. Sector analysis C All key sector MDAs collect and publish statistics and some analysis of and planning capacity their sectors. Moreover, the State Statistics Service collects and publishes data on each key sector. However, in very few cases are models to analyze, simulate and/or forecast sector supply and demand available. 1.2. Effective use of C Most MDAs use sector analysis and planning results for formulation sector analysis and of sector strategies and priorities, and for screening SSIP projects and planning results selecting them for financing. There is some evidence of using sector analysis and planning in estimating economic benefits from SSIP projects. in formulation of national and sector priorities and for sector analysis and forecasting. Moreover, the strategies, screening project documents entering MoE regularly prepares and publishes consensus the preparation and appraisal pipeline, estimation of forecasts of economic indicators, which can be used in economic benefits from sector projects, and selection forecasting.30 of projects and programs for inclusion in the mid-term and annual budgets in line with sector priorities. 56. All key sector MDAs collect and disclose statistical data on their websites. The healthcare sector, for example, is supported by the Center for 1.1. SECTOR ANALYSIS AND PLANNING Medical Statistics of the Ministry of Health of Ukraine, a CAPACITY public agency with the task of collecting and verifying a wide range of healthcare data and developing Performance level and evidence for scoring the methodologies for data collection. The center has dimension been collecting data since 1993 by region (oblast).31 The Ministry also has other agencies that specialize 55. Statistics and information on key sectors are in collection and analysis of data in particular areas collected and published by the State Statistics of healthcare. The Ministry of Ecology and Natural Service of Ukraine (SSSU) and international Resources collects and publishes data on various organizations, which forms a basis for sector aspects of environmental protection, such as waste analysis and planning. SSSU collects data on various management, forest management,32 and assessment sectors and economic indicators (see Table 5) of environmental pollution.33 The Ministry of Energy publishing them on its website. The data is available collects and discloses data on all subsectors of the to everyone, and it is possible to download it and use energy sector,34 publishing data on production, it for sector analysis, models, forecasts, and other consumption, and export of energy from various purposes within sector planning. Coupled with data sources. Moreover, it issues analytical materials published by international organizations (e.g., World on energy sources and markets, and on capital Bank, UN, IMF) line ministries have a solid background  30  Consensus forecast by Ministry of Economy (2020) https://me.gov.ua/Documents/Download?id=4bbc60fa-b4f7-4fb9-a335-b60a2ac0ac03  31  Data of the Center http://medstat.gov.ua/ukr/statdanMMXIX.html  32  Ministry of Ecology and Natural Resources' website https://mepr.gov.ua/content/vidkriti--dani.html  33  Ministry of Ecology and Natural Resources' website https://mepr.gov.ua/timeline/Zviti.html  34  Ministry of Energy's website http://mpe.kmu.gov.ua/minugol/control/publish/newscategory?cat_id=35081 26 Chapter 3. Assessment by Indicator for PIM Functions Table 5. Data on Indicators Provided by State Statistics Service Key sector Examples of data at SSSU Period Healthcare35,36 À Number of hospital beds 1990 - 2020 À Number of hospitals À Number of doctors À Number of patients by type of disease À Injuries at work À Number of hospitalized people À Number of days spent in hospital Ecology À Waste generation and processing 1995 – 2019 À Air pollution by CO2 1990 - 2020 À Capital investment in environmental protection Energy À Energy balance of Ukraine 2007 – 2019 À Supply and consumption of energy 2017 – 2020 À Final energy consumption 2007 – 2019 À Renewable energy consumption 2007 – 2019 À Prices for gas and electricity 2017 - 2020 Source: World Bank Mission Team based on the State Statistics Service of Ukraine investment in the sector. In addition to collecting The forecasts typically cover macroeconomic their own data, key sector MDAs use external sources indicators – GDP (including by sectors), industrial of data in their work, for example, databases of output index, consumer price index (CPI), producer international organizations (e.g., World Bank, WHO, price index (including by sectors), population, average EU, USAID). salary, export, import and other indicators - modeled under different scenarios for short-term and mid-term 57. While key sector MDAs have some sector perspectives.38 These forecasts could potentially be analysis and planning capacity, most lack used by key MDAs for forecasting the development of models that use available data to analyze, their sectors.39 simulate, and forecast sector supply and demand. The Department for Strategic Planning 58. The score for this dimension is C. and Macroeconomic Forecasting in MoE develops forecasts for economic and social development trends, for which sector MDAs must provide data.37  35  This information is published in a special section of SSSU in the form of a report with additional information on the sector, last report is dated 2020 https://ukrstat.gov.ua/druk/publicat/kat_u/2021/zb/11/Yearbook_2020.pdf  36  http://medstat.gov.ua/ukr/statdanMMXIX.html  37  Law on State forecasting and development of Programmes for Economic and Social development of Ukraine https://zakon.rada.gov.ua/laws/show/1602-14#Text  38  Forecast for economic and social development 2021 – 2023 https://www.me.gov.ua/Files/GetFile?lang=uk-UA&fileId=68afc88a-c642-4f3c-95c9-ed5a9c6546a6  39  Law on State forecasting and development of Programmes for Economic and Social development of Ukraine https://zakon.rada.gov.ua/laws/show/1602-14#Text Chapter 3. Assessment by Indicator for PIM Functions 27 1.2. EFFECTIVE USE OF SECTOR ANALYSIS some MDAs only screen projects on the basis of AND PLANNING RESULTS the urgency of the issue to be addressed without consideration of the sector context. Performance level and evidence for scoring the dimension 61. Most key sector MDAs use sector analysis and planning when estimating economic benefits from 59. While national and sector strategies show projects in their sectors. Estimation of economic evidence of using sector analysis and planning benefits is the responsibility of an implementing results in their formulation, how effectively such agency, which develops project concept notes and inputs are used is questionable. As a rule, sector feasibility studies. Concept notes and feasibility analysis and planning results are used to describe the studies contain evidence of using some sector current state of affairs in strategies and to suggest analysis, where relevant; however, in most cases, targets to be achieved. However, in many cases the there are no references to the sources of data and no targets are set with a positive bias and no justification justifications for assumptions. from forecasting methodologies. Only the latest National Economic Strategy 203040 explicitly states that 62. When selecting projects and programs for economic modeling was used to forecast the country's inclusion in the mid-term and annual budgets in development path and to set the targets. line with sector priorities, the responsible body, the Inter-Agency Commission (see PIM-3), must 60. Sector analysis and planning results must consider sector analysis and planning results. be used for screening projects entering the Sector analysis and planning results are provided preparation and appraisal pipeline. According to the members of the Commission, together with to Resolution No. 571, projects should be screened the feasibility studies of projects that are eligible for based on a concept note, which contains information selection. Moreover, representatives of sector MDAs on sector analysis and planning. Most key sector MDAs make presentations of their projects at the meetings use sector analysis and planning when screening of the Commission where they provide information on concept notes. Table 6 shows how sector analysis and sector analysis. planning results are used in three sectors. However, Table 6. Use of Sector Analysis and Planning Results for SSIP Areas of sectoral analysis use Healthcare Ecology Energy Formulating national and sector priorities and Yes Yes Yes strategies Screening project documents entering the Yes No Yes preparation and appraisal pipeline Estimating economic benefits from sector projects Yes Yes Not clear Selecting projects and programs for inclusion According to legislation, the Commission, when selecting in the mid-term and annual budgets in line with projects, must consider sector analysis and planning sector priorities results. According to MDAs, the Commission considers sector analysis and planning results, at least, at the meetings, where implementing agencies or line ministries make presentations of their projects. Source: World Bank Mission Team  40  https://www.nes2030.org.ua/ 28 Chapter 3. Assessment by Indicator for PIM Functions 63. There is no firm evidence of the application of PIM-2. Strategic plans and investment sector analysis in planning those investments that guidance, project development and do not follow Resolution No. 571.41 However, it is very unlikely that no consideration is given to sector preliminary screening plans when a project is proposed for budget funding outside the procedure established in Resolution No. RATIONALE 571; non-SSIPs are included in budget programs, 65. Strategic guidance and its role in public which in principle should be strategy-driven as investment management is assessed within this they are part of a performance-oriented budgeting indicator. The indicator analyzes availability of well approach. The rating for this dimension should elaborated strategic documents, which can be used therefore not be affected by non-SSIPs. for public investment planning. Submission of project profiles (known as 'project concept notes' in Ukraine) 64. The score for this dimension is C. and their screening by line ministries as an initial stage of investment planning is assessed based on the completeness of information provided by project initiators and the robustness of screening by line ministries. Assessment of the indicator covers the last three fiscal years (2018 – 2020). Summary of Scores and Performance Table PIM-2 Indicator/ Dimension Score Brief justification for score PIM-2. Strategic plans D+ Scoring Method M2 and investment guidance, project development and preliminary screening 2.1. Strategic guidance C Progress has been made in 2020 – 2021 with the development of the National Economic Strategy which will launch better strategic planning. But the period under analysis 2018 – 2020, does not show significant achievements in strategic guidance that might have had a positive impact on public investment planning. 2.2. Strength of D There is no investment strategy that serves as general guidance in Ukraine, strategic guidance although SSIPs selection decisions are taken with a strategic, mid-term perspective by the Inter-Agency Commission. 2.3. Submission of C Project implementing agencies usually submit project concept notes project profile (concept that contain most of the necessary information for review by the central note) agency. However, this is not the case for non-SSIPs. 2.4. Robust screening D In some cases, up to 50 percent of project concept notes are rejected. But of project profiles the number of concept notes submitted for central review is not large (for (concept notes) example, only four for Ministry of Ecology and Natural Resources in 2020). Causes of rejections are not known. Some may be due to formal aspects of the presentation and not to rejection of strategically or conceptually weak projects.  41  See Section 2.2 for an assessment of the relative importance of non-SSIPs compared to SSIPs Chapter 3. Assessment by Indicator for PIM Functions 29 2.1. STRATEGIC GUIDANCE (Action Plan) is developed annually, which informs ministries in developing their own annual action Performance level and evidence for scoring the plans. The Action Plan can contain specific investment dimension projects, but with no prioritization and specification of project costs. 66. There is no well-developed system for strategic planning at the national level nor is 67. In 2020, the list of priority public investment there continuous practice of applying a strategic projects was approved by the government,43 the approach to public investment. In the period under aim of which is to guide agencies in their decisions. analysis, the only strategic document that could be The list contains the names of projects and possible defined as a national strategy was the Sustainable sources of financing. It does not contain any additional Development Strategy up to 2020. The document 42 information concerning the reasons and justification contains broad goals, with a focus on the reform for inclusion of particular projects in the list. The agenda, but does not provide strategic guidance for list does not contain the data on project cost and public investment. However, when justifying public implementation period. investment projects, agencies could refer to this strategy showing the link between its goals and the 68. Sectoral strategic documents exist in Ukraine, project purpose. Another document that can be but not for all sectors, and planning processes do considered as strategic is the government's program, not follow the same approach. As a rule, strategies which is developed by the Cabinet of Ministers of contain broad goals, which are then specified in action Ukraine (CMU) for the period of its term. Government plans that can contain plans for implementation of programs typically contain priorities, tasks, and particular investment projects. Existing strategies performance indicators for each ministry. Since have been developed in different years and for 2018, there have been three different governments different periods, which makes it difficult to unify in Ukraine, meaning that the government program them (see Table 7). Except for the key sector strategies was redesigned three times, thus undermining the mentioned in the table below, there are a number strategic purpose of the document. Based on the of strategies in other sectors such as transport, government program, a government Action Plan education, information and communications Table 7. Strategic Documents by Key Sectors Type of Year of Sector Name of Document document approval Energy Strategy Energy Strategy of Ukraine 2035 2017 Healthcare Strategy National Strategy for Reforming the system of Healthcare in 2014 Ukraine 2015 – 2020 Ecology Strategy The Law of Ukraine on main principles (strategy) of state 2019 environmental policy to 2030 Ecology Action Plan National Action Plan on Environmental Protection to 2025 2021 Source: World Bank Mission Team  42  Sustainable development strategy till 2020 'Ukraine – 2020' https://zakon.rada.gov.ua/laws/show/5/2015%23Text  43  List of priority investment projects https://www.kmu.gov.ua/storage/app/uploads/public/5fd/c80/ee6/5fdc80ee67197829472716.doc 30 Chapter 3. Assessment by Indicator for PIM Functions technologies. For example, the National Transport This information is provided to MoF by MoE with the Strategy to 2030 was approved in 2018, the National aim of requesting the necessary amount of funding Strategy for the Development of Education in Ukraine for the forthcoming budget and to inform MoF of the to 2021 was approved in 2013, and the Strategy for forecast of investment funding needs, so that it can Digital Transformation of Social Sectors to 2023 was incorporate the data into the forecast of the budget approved in 2020. These strategies are overarching for the two years following the planned year. But the for the sectors, but there are also sub-sector specific MoE does not prepare a fiscally constrained mid-term strategies, for example, the Strategy for Improving public investment program. Road Safety in Ukraine to 2024, the Strategy for the Development of Seaports of Ukraine to 2038, the 72. The score for this dimension is D. Strategy for Energy Security, the National Strategy for Building a Safe and Healthy Educational Environment 2.3. SUBMISSION OF PROJECT PROFILE in the New Ukrainian School, and the Strategy for Treatment of Radioactive Waste in Ukraine. Performance level and evidence for scoring the dimension 69. The National Economic Strategy was developed in 2021 under the initiative of the Prime Minister. 73. Implementing agencies must submit fully The Strategy is considered as overarching for all specified project concept notes44 to line ministries. sectors; line ministries must adjust their plans According to Resolution No. 571 a project concept according to this document. The strategy does not note must contain detailed information on project provide clear prioritization of public investment purpose and its justification, economic preconditions and does not contain priority investment projects, for project implementation, and preliminary analysis but it stresses the necessity to develop a National of project efficiency (see Box 5). At this stage, Investment Plan for 5 years and to improve an implementing agency is required to analyze management of public investment projects. demand, examine alternative solutions, demonstrate compliance to strategic documents, specify costs, and 70. The score for this dimension is C. identify beneficiaries, risks, and impact. 74. Line ministries ensure the completeness of 2.2. STRENGTH OF STRATEGIC GUIDANCE project concept notes and discipline is generally Performance level and evidence for scoring the good. Implementing agencies submit concept notes dimension for public investment projects that usually contain most of the necessary information for central review. 71. Ukraine does not have a formal national If a concept note does not contain all required investment strategy, however some elements of information, the responsible line ministry rejects it strategic planning of investment are inherent to and requests improvement. Projects not following the budgetary management of public investment Resolution No. 57145 and seeking to find financing projects that follow Resolution No. 571. Project from the state budget are not required to prepare selection decisions are taken with a strategic, mid- project concept notes. They follow the usual budget term perspective by the Inter-Agency Commission (see process. PIM-3), considering financing for the next year and a preliminary plan of financing for the following 2 years. 75. The score for this dimension is C.  44  Project profiles are referred to as project concept notes in Ukraine.  45  See Section 2.2 for an assessment of the relative importance of non-SSIPs compared to SSIPs Chapter 3. Assessment by Indicator for PIM Functions 31 Content of a Project Concept Note BOX 5 À Project purpose and its justification:  Availability of a land parcel, titles related thereto and a list of measures that need to  Problems which shall be solved by be taken in order to execute titles thereto implementing the project; (when necessary); and  Results of analysis of demand on services  Estimated cost of maintenance of (goods), provision (production) of which the facility after implementation and must be assured following implementation justification of sources for maintenance of the investment project; [financing].  Results of preliminary analysis of possible À Results of preliminary analysis of investment alternative options of solving the problems; project implementation efficiency: and  Expected impact (environmental, social  Compliance of investment project purpose and economic benefits and impact from to state policy priorities. implementation of the project); À Economic preconditions for implementation  Recipients of project benefits; (financing) of the public investment project:  Preliminary assessment of risks of the  Calculated cost of the project, including project; the expenditures attributable to its development and implementation,  Duration and phases of implementation including costs incurred in previous of the project, opportunities for periods, excluding costs, including capital, implementation and management; and during the operational phase;  Justification of further activities on  Justification of the choice of possible implementation of the project (research, sources of financing; design). Source: Resolution No. 571 2.4. ROBUST SCREENING OF PROJECT feedback from civil society. If the decision is negative PROFILES or the project concept note needs to be refined, the MDA must inform an implementing agency by official Performance level and evidence for scoring the letter. An MDA may create a commission for taking dimension such decisions. Among key sector MDAs consulted, only the Ministry of Ecology and Natural Resources 76. Screening of public investment projects is has a commission, which consists of representatives of performed by line ministries, departments, various departments of the ministry. agencies (MDAs) for state budget funds by the Budget Code. An implementing agency submits 77. Available data shows that for Fys 2018 to 2020 the project concept note to the relevant MDA, which no project concept notes for public investment then considers it and takes a decision on whether to projects were rejected by five ministries and proceed with project development. If the decision is institutions.46 Only the Ministry of Energy reported positive, it must be published on the website of the that in FY 2021 it rejected 50 percent of project MDA for at least 15 days to receive comments and concept notes presented. The main reason for  46  Namely the State Management of Affairs, National Academy of Medical Sciences, Ministry of Health, Ministry of Energy and Ministry of Ecology and Natural Resources. 32 Chapter 3. Assessment by Indicator for PIM Functions rejection was noncompliance with the legislative wellbeing of the population. While cost-benefit requirements. In general, it can be said that screening analysis (CBA) or project appraisal can be complex and is largely compliance oriented and does not involve involve contentious judgments, it is generally accepted questioning of the project rationale. The percentage that decision-makers should be provided with of project concept notes rejected for Fys 2018 to 2020 technical advice on costs and benefits of alternative was 0 percent, and if rejections in 2021 by the Ministry projects, because public investment decisions involve of Energy are considered, the average rejection rate scarce resources and trade-offs between competing would still only be around 3 percent. interests. These costs and benefits should ideally be society-wide costs and benefits, rather than solely 78. The score for this dimension is D. fiscal or financial costs and benefits. 80. Dimensions to be assessed under PIM-3 are: PIM Function 2. Formal i. Clarity of roles and responsibilities in project appraisal for all participating MDAs; Appraisal ii. Availability of project appraisal technical guidance and support; PIM-3. Formal Project Appraisal Procedures and Guidelines iii. Comprehensiveness and quality of available technical guidance; RATIONALE iv. Proportionality of project appraisal, i.e., adequacy of appraisal procedures to project characteristics; 79. This indicator assesses the quality of project and preparation and appraisal procedures. Project v. Usage of feasibility studies to appraise large appraisal procedures are critical for assuring projects. public capital investment makes the best possible contribution to the growth of a country and to the Summary of Scores and Performance Table PIM-3 Indicator/ Dimension Score Brief justification for score PIM-3. Formal project C+ Scoring Method M2 appraisal procedures and guidelines 3.1. Clarity of appraisal A Roles and responsibilities for project appraisal are clear between and roles within line ministries and central agencies. The roles between agencies are regulated by the legislative acts, while the roles within agencies are defined either by organizational structure or separate internal orders of the agency. 3.2. Project appraisal B Guidelines are available and comprehensive from the point of view guidance and support of guidance for economic, financial and market analysis, and for environmental impact and risk assessment. Funding for project appraisal is not readily available but can be requested following standard budget procedures. 3.3. Technical content C Guidelines do not cover distributional analysis and sector specific issues. of guidelines Key economic parameters are not up to date. Chapter 3. Assessment by Indicator for PIM Functions 33 Indicator/ Dimension Score Brief justification for score 3.4. Proportionality of C The level of effort depends only on project cost and potential profitability. project appraisal This is rigorously controlled. Other parameters are not considered. The appraisal methodology is financial analysis for potentially self-sustaining/ profitable projects; CBA for unprofitable projects with a total capital cost over 30 million UAH and CEA for unprofitable projects under that value. Use of multi-criteria analysis is not specified. 3.5. Usage of feasibility C All projects following Resolution No. 571 must have a project proposal studies to appraise and this is enforced. Large projects financed by IFIs, or bilateral loans large projects have feasibility studies prepared following guidelines from the financier. However, 46.3 percent of large projects (by value and excluding IFIs) are included in the budget without following Resolution No. 571 procedures and these have no feasibility studies. EXISTING REGULATIONS REGARDING support from the World Bank, provision of assistance PROJECT APPRAISAL PROCEDURES AND and training for using the guidelines has been the GUIDELINES responsibility of MoE. MDAs have the right to contact MoE, when necessary, to receive explanations and 81. Resolution No. 571 adopted in July 2015 clarifications on the application of the guidelines. established the framework and procedures for project appraisal and selection. It created the Inter-Agency Commission for Public Investment 3.1. CLARITY OF APPRAISAL ROLES Projects as a collegiate body under the CMU and Performance level and evidence for scoring the assigned it the task of selection of public investment dimension projects. The resolution also defines procedures to be followed, institutional responsibilities and type of 83. Roles and responsibilities for project appraisal appraisal depending on total project investment cost. are clearly defined in Resolution No. 571. It states In appendices, it provides forms for presenting project what kind of agencies/departments can initiate and proposals47 and for the state expert review opinion. develop projects and what kind of agencies can submit them for selection. The key principle of project 82. Resolution No. 571 is complemented by the appraisal is a bottom-up approach, which means Guidelines for Elaboration of a Public Investment that a public agency of the lower level (implementing Project.48 The stated purpose of the guidelines is agency/responsible executor) elaborates a project to ensure a uniform approach to elaboration of a idea and a feasibility study, which must be reviewed public investment project, by explaining in detail and approved by a higher-level agency (line ministry/ how to prepare and appraise a project. Guidelines key spending unit for the state budget funds). The role were developed in 2016, with technical assistance of MoE is to perform independent review of appraisal from the World Bank, and approved and published and to screen projects against criteria, while the by MoE in 2017. Except for the discount rate, which Inter-Agency Commission takes a decision on project was recalculated in 2018, guidelines, methods and financing and implementation. The roles of all these key economic appraisal parameters have not been actors are clearly specified in legislation. updated over the last 3 years. Following initial  47  Project proposal is a document, in which the results of appraisal are presented. In case of an investment project following Resolution No. 571, the project proposal is Annex 1, which consists of Resume, Technical and Economic Analysis, and Plan of Implementation, Financing, and State of implementation of a project. Throughout the Resolution No. 571, project proposal is also referred to as "public investment project" In this report this document is called "project proposal".  48  Order of the Ministry of Economy No. 1865 34 Chapter 3. Assessment by Indicator for PIM Functions 84. The Project Proposal49 is the key document of the 3.3. TECHNICAL CONTENT OF GUIDELINES appraisal process setting out the appraisal findings. It includes information on the project purpose, its financial Performance level and evidence for scoring the and economic analysis, technical solutions, social dimension and environmental impact and other aspects specified in Resolution No. 571. Project proposals 88. Existing guidelines cover most key technical must be prepared by the implementing agency. The issues for financial and economic appraisal, but implementing agency has the right to use the services they do not cover distributional analysis, sector of external companies for development of a project specific issues, and climate change impact. The proposal if it does not have the necessary capacities. guidelines (see Box 6) are focused on explanation Some key sector MDAs have specialized enterprises of how to fill in the form for the project, financial that handle the development of feasibility studies in modeling, and cost-benefit analysis. They are their sectors, as is the case for the Ministry of Health. supposed to be general, suitable for all sectors. The guidelines do not include any requirement 85. The score for this dimension is A for preliminary technical design of projects. In this respect, the project proposal required under Resolution No. 571 falls short of what is usually 3.2. PROJECT APPRAISAL GUIDANCE AND required for a feasibility study. SUPPORT 89. To complement the guidelines the MoE has Performance level and evidence for scoring the a webpage dedicated to public investment on dimension its website, where it publishes international guidelines51 for cost-benefit analysis in some 86. Guidelines on project appraisal procedures and sectors,52 and includes additional learning methods are provided in the official legislative and explanatory materials.53 Few key economic acts and orders of the Ministry of Economy50 and parameters for appraisal are provided by MoE are readily available and used. The guidelines are (Investment Department, in particular), except for comprehensive and cover all topics required for the the social discount rate and appraisal horizons by implementation of Resolution No. 571. Any project project type. The discount rate was updated in 2018. aimed at creating or improving a state-owned asset, In its explanatory letter to MDAs, the MoE Investment including projects of SOEs, can be submitted for Department refers to its webpage for macroeconomic funding according to Resolution No. 571 and, in such indicators that can be used in project appraisal; cases, the guidelines must be (and are) followed, however, the summary of the forecast of indicators or the project will be rejected by MoE. If an agency (GDP, CPI, export, import, etc., for 2017 – 2019) has decides to search for financing from external sources, not been updated since 2016. It should be noted that the guidelines for project development issued by another department of MoE, the Department for the individual IFI or donor are applied. Funding for Strategic Planning and Macroeconomic Forecasting, the conduct of project preparation using external regularly publishes macroeconomic forecasts on its expertise is not always available. In fact, many public webpage. agencies that submit projects to MoE develop project proposals on their own, without external experts or consulting firms. 87. The score for this dimension is B.  49  Based on the content of the project proposal, it is equivalent to a prefeasibility appraisal in other countries, because technical aspects have not been developed to pre-design level.  50  Resolution No. 571 and MoE Order No. 1865 Methodological recommendation.  51  Link to website of the MoE with international and additional guidelines https://www.me.gov.ua/Documents/List?lang=uk- UA&id=f76f9c7a-7777-4419-a6d7-9faa1abec641&tag=MizhnarodniMetodichniRekomendatsii  52  Translated guidelines on CBA for culture and forestry sectors developed by governments of the UK and Canada for assisting corresponding agencies.  53  Additional materials at the MoE's website https://www.me.gov.ua/Documents/List?lang=uk-UA&id=24c6825e-1b25-4756-8f08- 835a903b106d&tag=DopomizhniNavchalniMateriali Chapter 3. Assessment by Indicator for PIM Functions 35 Technical Content of Guidelines BOX 6 The guidelines provide methodological guidance,  List of measures aimed at implementation including on technical and economic analysis, of the project including details of the in line with the required contents of the project funding schedule and time limits proposal as follows:  Staffing according to project life cycle stages À Goal of the project and its justification including:  Certificates, licenses and other permits needed for implementation of the project  A description of the problem or impediments whose resolution is to be  Investment expenses by years of project achieved by the project implementation, separately for stages of project elaboration and implementation  Qualitative and quantitative description of the results of implementation of  Operational expenses the project, describing and specifying À Project effectiveness analysis covering: the products or services; providing the information on the volume of production  The cost of final products (goods, works, and services provision, and specifying and services) expected capacity utilization  Calculation for cost-effectiveness indicators  Method of achievement of the end-result (financial analysis) based on the analysis of alternative  Projected social and environmental ways of achievement of the purpose and consequences arguments on advantages of the selected  Analysis and appraisal of project way implementation benefits and projected  Compliance of the project with directions economic effect (economic analysis) for for development of the state, as identified projects with a total cost over 30 million in strategic and programmatic documents UAH À Description of the project and projection of  Cost-effectiveness of implementation expenses including: and operation for project with a total cost  Technical and/ or technological analysis of under 30 million UAH the selected method of achievement of the  Assessment of impact of project end-result implementation on the budget  Need to secure land required to implement  Analysis of risks and possible ways to the project reduce them  Description of the existing infrastructure to À Information about organizational structure be used for implementation of the project and project management, including: and during operation of the facility  Information about the responsible  Environment protection measures executor/project implementing agency including reproduction and preservation including internal organizational structure of natural resources and mitigation of  The organizational chart for project environmental impacts implementation management  Need for development of design À Implementation and financing plan and its documentation and development phasing performance status. Source: World Bank Mission Team based on published guidelines 36 Chapter 3. Assessment by Indicator for PIM Functions 90. Guidelines contain worked examples of 3.4. PROPORTIONALITY OF PROJECT SWOT-analysis, financial modeling, cost-benefit APPRAISAL analysis (financial and economic), and other parts of project proposal that might be difficult Performance level and evidence for scoring the for implementing agencies to complete without dimension models. In addition to examples in the guidelines, MoE has uploaded Excel-based templates for financial 94. The method of project appraisal is determined modeling and CBA to its website, which can be used by project cost and potential profitability; no for analysis of projects. MoE provides support on the additional criteria such as sector, complexity, and use of the guidelines by answering questions from risk are considered. Financial analysis is applied to MDAs. MDAs interviewed considered guidelines to be self-sustaining/profitable projects, while both financial good but suggested that further clarifications of some and economic analysis is applied to unprofitable topics through focused capacity building would be projects with social and economic impact. Economic useful. analysis is differentiated by project cost: cost- efficiency analysis (CEA) is applied to projects with the 91. One important MDA has developed its own total investment cost below UAH 30 million, while cost- sectoral guidelines in line with MoE's general benefit analysis is applied to projects whose total cost recommendations. With the assistance of the EU, the is over UAH 30 million.55 Since cost-benefit analysis Ministry of Infrastructure (MoI) developed guidance is considered as more onerous than cost-efficiency for preparing investment proposals in the transport analysis, both in terms of its need for specialized sector and for assessing and selecting investment expertise and the greater transparency it sheds on projects (project proposals) in the sectors of transport, the quality of a project, there is a risk of projects being road infrastructure and postal services. These deliberately submitted with a cost of UAH 29 million.56 guidelines follow the approach required by Resolution In most cases the projects clearly comply with the No. 571 and the accompanying guidelines from MoE, rule of below or above UAH 30 million, but sometimes while providing sector specific recommendations misunderstandings appear with projects that almost on project development (especially CBA) and project reach the threshold.57 Multi-criteria analysis is not selection. There is no evidence of the existence of defined as a methodology for application for small or other sector specific guidance developed by sector repetitive projects. MDAs. 95. The score for this dimension is C. 92. MoE has published international guidelines on its website for use in project appraisal.54 In particular, guidelines for cost-benefit analysis in forestry and art & culture sectors, general guidelines for CBA and for social CBA. Some of the guidelines are translated into Ukrainian languages, while some are in English. There is no evidence that key sector MDAs use these guidelines. 93. The score for this dimension is C.  54  International methodological recommendations [MoE website] https://www.me.gov.ua/Documents/List?lang=uk- UA&id=f76f9c7a-7777-4419-a6d7-9faa1abec641&tag=MizhnarodniMetodichniRekomendatsii.  55  Approximately one million Euros which is a rather low limit. This can be compared for example with the case of Montenegro, where to be eligible for funding from the capital budget an investment proposal should include at least a feasibility study, and a cost-benefit analysis if the estimated value is above EUR 5 million. In the case of Albania procedures specify a threshold of 700 million LEK (EUR 5.68 million) for large projects to which a more rigorous appraisal is applied.  56  However, there are some projects with an estimated cost just below this limit which casts some doubt regarding ministries intentionally reducing estimated cost of projects to avoid the need to prepare appraisals using CBA.  57  For example, there is currently a project with a cost of 29 812.800 UAH in the healthcare sector. Another project spotted in the minutes of the meetings of the Commission is with a cost of 29 999 thousand UAH (health sector). Chapter 3. Assessment by Indicator for PIM Functions 37 3.5. USAGE OF FEASIBILITY STUDIES TO following the guidelines of the institutions providing APPRAISE LARGE PROJECTS the financing. Performance level and evidence for scoring the 97. The score for this dimension is C. dimension 96. All projects submitted to MoE and selected by the Inter-Agency Commission are subject to PIM-4. Project Appraisal Capacity complete appraisal studies ('project proposals') RATIONALE prepared following Resolution No. 571 and existing guidelines. When Resolution No. 571 was adopted 98. This indicator analyzes project appraisal and the procedure for public investment projects was capacities at central and key sector MDAs, established, it was expected that all projects would focusing on the quality of staff at agencies and undergo the same process to receive funding, but six availability of training for project appraisal. The years' experience has shown that it did not happen quality of staff is assessed as of the current moment, as expected. This is because there remain other ways while availability and frequency of training is analyzed to receive funding from the budget which are still in for the period of the past 3 years (2018 – 2020). The keeping with the legislation. This allows MDAs to make indicator seeks to determine whether central and key direct budget requests without addressing the MoE or sector MDAs have well trained staff for: maintaining the Commission to obtain funding for their investment and updating guidelines and methods; routinely projects. In this way, a significant share – on average re-estimating all key economic parameters used 29.7 percent (by value) of large capital investment in economic appraisal; and reviewing the quality projects for the period 2018-202058 - end up being of studies and methods for all large projects and funded as budgetary programs outside the scope selectively for other projects. This indicator requires of Resolution No. 571. Projects to be financed from identifying whether training at appropriate levels funds provided by IFIs or bilateral loans also have and frequency is provided to most MDAs to ensure a feasibility studies. However, such studies are prepared strong project appraisal capacity. Summary of Scores and Performance Table PIM-4 Indicator/ Dimension Score Brief justification for score PIM-4. Project D+ Scoring Method M2 appraisal capacity 4.1. Quality of staff C Central and key sector MDAs have limited capacities to provide quality to oversee project control and guidance to conduct project appraisal. Typically, agencies have appraisal methods and insufficient staff directly responsible for project appraisal methods and applications applications. The knowledge of most staff on project appraisal is limited to the content of the guidelines provided by MoE. 4.2. Scope and quality D No training has been provided for the last 3 years. Staff interviewed of project appraisal indicated additional training is required. training  58  The figure represents the share of non-SSIPs in the total capital cost of ongoing projects, excluding IFI financed projects. If IFI projects are included, the figure falls to 16.5 percent, because IFI projects follow formal procedures. The share has risen significantly to 51.8 percent in 2021, but this is outside the assessment period. 38 Chapter 3. Assessment by Indicator for PIM Functions 4.1. QUALITY OF STAFF TO OVERSEE 4.2. SCOPE AND QUALITY OF PROJECT PROJECT APPRAISAL METHODS AND APPRAISAL TRAINING APPLICATIONS Performance level and evidence for scoring the Performance level and evidence for scoring the dimension dimension 102. No training in project appraisal has been 99. MoE, as a central agency, has limited capacities provided to central and sector MDAs for the last 3 to provide quality control and guidance to the years. There were trainings in 2015 – 2016 following conduct of project appraisal. There are only four the establishment of the procedure for selection of people in the Unit for Public Investment Projects, public investment projects, but no training has been which currently handles the portfolio of 66 projects. provided since that time. However, MoE provides Every year, a minimum of 20 new projects are individual ad hoc advice to sector MDAs in case they submitted to the unit for review. In the past, only have questions regarding project appraisal procedures 10 days were provided for reviewing the project and methodologies. proposals, which was not enough considering the other responsibilities of the unit and its staff. 103. The score for this dimension is D. The period of project review was extended to a more appropriate 20 days in 2021. In addition to independent review of project proposals and PIM-5. Screening of Feasibility Studies provision of methodological support for project development, the unit deals with monitoring of public RATIONALE investment projects, provision of administrative and analytical support to the Inter-Agency Commission, 104. This indicator assesses three dimensions budget management and communication of the for screening out bad projects at early stages, Commissions' decisions to MoF. avoiding waste of effort on further design and appraisal. Project screening should be done stepwise. 100. Key sector MDAs have capacity only for Project concept notes must be reviewed to discard administration of projects and no capacity for bad projects before they gain momentum and before development of sector specific guidance or resources and effort have been invested preparing training to subordinate agencies. Typically, there are more detailed studies. Afterwards review of pre- only a few people responsible for public investment feasibility studies or feasibility studies prepared for projects management within key sector MDAs. For the pre-selected projects should allow weeding out example, at the Ministry of Ecology and Natural some projects, leaving only the better prepared and Resources, there is only one person managing public socio-economically more profitable candidates for investment projects, (not considering the head of inclusion in the budget. the relevant department);59 at the Ministry of Health, there are 2 people responsible for projects. They have 105. The dimensions examined cover screening enough capacity to review the project concept notes procedures, the existence of the public investment and project proposals for compliance with legislative program (PIP) database, and the extent to which requirements, but no capacity for development of screening has real impacts on what happens next sector specific guidelines60 or for advising on economic to a project. The first dimension looks at whether parameters to be used in appraisal. Key sector MDAs, there are appropriately sequenced and well-defined except the MOI which has its own guidance (see PIM- procedures for preparation and appraisal of projects, 3.3), rely on the guidance from MoE in this respect. and whether such studies are based on explicit criteria considering project type, size, sector, regulatory 101. The score for this dimension is C. requirements, and prior investment experience  59  The head of department estimated that only 10 percent of time was dedicated to public investment projects.  60  An option would be to get donor support, as the Ministry of Infrastructure (MoI) did for developing guidance for preparing investment proposals in transport sector and for assessing and selecting investment projects (project proposals) in the sectors of transport, road infrastructure and postal services. Chapter 3. Assessment by Indicator for PIM Functions 39 Summary of Scores and Performance Table PIM-5 Indicator/ Dimension Score Brief justification for score PIM-5. Screening of D+ Scoring Method M2 feasibility studies 5.1. Sequenced and C The general process is well-defined in Resolution No. 571 but about a third disciplined project of large projects by value (excluding IFI funded projects) are included in screening procedures the budget without following the prescribed process. 5.2. Prescribed public D A database in the form of an Excel spreadsheet exists, but it is not focused investment program on project screening and does not register projects before they have (PIP) database been reviewed by the Commission. It is regularly updated but information registered is limited. 5.3. Projects rejected D Very few projects are rejected following screening of feasibility studies. or required to be redesigned and reappraised with similar projects. The second looks at whether 5.1. SEQUENCED AND DISCIPLINED there is a legally sanctioned public investment PROJECT SCREENING PROCEDURES database recording all projects as they pass through pre-screening, preparation, and screening. Such Performance level and evidence for scoring the a database forms a project pipeline that allows dimension control over the flow of quality-controlled projects into the budget. The third dimension assesses the 107. The processes for project preparation, effectiveness of this part of the system through the appraisal and screening are well defined and measure of the share of projects rejected or required documented for investment projects. Existing to be redesigned and reappraised. regulations include: a. The Procedure for selecting public investment 106. The scoring of certain 'effectiveness' projects approved by Resolution of Cabinet of dimensions under this indicator is influenced by Ministers of Ukraine No. 571 of 22 July 2015; the large share of projects ('non-SSIPs') that avoid the formal procedures set out in Resolution No. b. The Procedure for preparing, implementing, 571. On average during the assessment period, 33.6 monitoring IFI-supported projects of economic percent of ongoing state budget-funded projects by and social development of Ukraine and total project value (see Section 2.2)61 were non-SSIPs. completing their implementation, approved by This figure rose to 46.3 percent in 2021, outside the Resolution of Cabinet of Ministers of Ukraine No. assessment period. For large projects, the average 70 of 27 January 2016. share for 2018-20 was 31.2 percent. For the sake of 62 brevity, these figures will not be repeated for each dimension.  61  The figure of 33.6 percent refers to domestically funded projects - SSIPs, non-SSIPs, regional development fund, and capital transfers, but excluding the road subvention, because of the difficulty in distinguishing between maintenance and capital investment. The figure excludes IFI financed projects. If IFI projects are included the figure falls to 19.3 percent, because IFI projects follow formal procedures.  62  16.8 percent including IFI projects. 40 Chapter 3. Assessment by Indicator for PIM Functions 108. There is also the document "Guidance 5.3. PROJECTS REJECTED OR REQUIRED on assessing and Selecting Projects (Project TO BE REDESIGNED AND REAPPRAISED Proposals) in the Sectors of Transport, Road Infrastructure and Postal Services". It presents Performance level and evidence for scoring the recommendations and criteria to be applied to assess dimension and select the investment projects that need support of the MoI, including public investment projects for 113. Most prefeasibility studies at the stage of which the MoI will be the main spending agency, and review by the line ministry are not rejected. One investment projects that need state aid. of the reasons for a low rejection rate may be the high rate of rejection at the earlier project concept stage, 109. The processes established in Resolution No. when the line ministry already takes a decision on 571 do not include sector specific aspects but whether or not to proceed with the project. But given do differentiate depending on the project size. that the focus of project concept note reviews (more Projects prepared by the implementing agencies are on compliance and strategic alignment) is different screened at several stages. The first screening occurs from the focus of feasibility study review (more at the stage of concept note, when the line ministry focusing on economic and environmental analyses), decides to proceed with further project development a low rejection rate at this stage is more likely a result or not. The second screening occurs at the stage of of overoptimism. Since line ministries are interested project proposal when the project is reviewed again by in selection and funding of their projects under the line ministry. The third screening happens when development, most of the conclusions of state expert the MoE reviews submitted projects, having the right reviews issued by line ministries are positive. However, to reject further development of the project. Formal not all project proposals pass the subsequent review procedures do not exist for the significant share of by MoE, despite having had a positive conclusion from projects that are funded as budgetary programs state expert review carried out on behalf of the line outside of Resolution No. 571. ministry. This suggests that sometimes state expert review is positively biased. Moreover, this rating of this 110. The score for this dimension is C. dimension is further negatively affected by the large number of projects not following Resolution No. 571, as detailed above. 5.2. PRESCRIBED PUBLIC INVESTMENT PROGRAM (PIP) DATABASE 114. The score for this dimension is D. Performance level and evidence for scoring the dimension PIM Function 3. Appraisal 111. The database of public investment projects exists and captures some information necessary Review for analyzing the project portfolio but does not include information on project screening and PIM-6. Independent Review of preparation. The database is focused on project Appraisal implementation. Once a project has been considered by the Commission for budgeting, regardless of the RATIONALE result of selection, it is included into the database. The database exists as an Excel spreadsheet. It is managed 115. This indicator assesses whether the and regularly updated and uploaded to the website documentation and findings of project appraisal of MoE by a dedicated unit. It includes information are subject to an impartial review by a party on projects disbursements, total cost, period of not involved in their preparation, with the aim implementation, and some information on project of validating quality and countering optimism implementation status. However, the database does bias. There is a tendency for project proposals to not include information about projects prior to the systematically over-estimate project benefits and funding decision. under-estimate costs. It has been observed all around the world and is especially serious for large 112. The score for this dimension is D. infrastructure projects like railways, motorways, Chapter 3. Assessment by Indicator for PIM Functions 41 Summary of Scores and Performance Table PIM-6 Indicator/ Dimension Score Brief justification for score PIM-6. Independent C Scoring Method M2 review of appraisal 6.1. Independent C According to Resolution No. 571, all projects are subject to a two-step review of appraisal independent review, first by the relevant line ministry that supervises the project proposer and then by the MoE; however, the independence of the line ministry and its commission (if created) can be questioned. Projects seeking other types of budget support have to go through state expert review according to Resolution No. 701. 6.2. Content of C Resolution No. 571 provides guidance regarding content of the independent review independent review. Methodology of the Order No. 243 provides details for state expert review of projects seeking other type of budget support. large bridges, and tunnels. Independent review is 6.1. INDEPENDENT REVIEW OF widely regarded as the best way to counter the usual APPRAISAL optimism bias of project promoters, by verifying the quality of the analysis, how realistic assumptions are, Performance level and evidence for scoring the and if estimations of demand and cost are reasonable. dimension It usually does not include revision of technical aspects of project pre-design. This review should ideally be 117. A two-stage independent review process is done at an early stage in project development, prior established in Resolution No. 571. The first step to the submission of a budget request and at least being a "state expert review", which is performed by when pre-feasibility or feasibility studies have been MDAs, which may create a commission to conduct completed. All projects, regardless of proposed it. As stipulated in Resolution No. 571, members of funding (donor funds, loan, own resources) and the commission should be entirely unconnected with implementation mechanisms (direct contracting or the project: "The members of such a commission PPP) should be subjected to such independent review. shall not be representatives of investment activity entity, who prepared a concept note or will be 116. The dimensions of the indicator cover responsible for public investment project elaboration existence of an independent review mechanism and representatives of responsible executor." If and the contents of independent review. The first such a commission is established in a ministry, as dimension looks at whether projects (larger and more it has been in the Ministry of Ecology and Natural complex projects) are submitted for independent Resources, it constitutes a first level of independent or quasi-independent review by an external agency review of project appraisal. Based on the findings of that is expert in the project appraisal methodology. the commission, or the review of the project proposal, Such a review could involve either (i) reviewing the the MDA issues an opinion on the status of project methods, parameters and assumptions used in a preparedness and recommended future steps. The pre-feasibility or feasibility study, or (ii) conducting state expert review must be conducted within a period a preliminary feasibility study of a project proposal. of 30 calendar days, which is in line with international Content of independent review focuses on the scope standards. If the decision of the state expert review of the independent review to assess if it covers all is positive, the MDA must officially appoint the dimensions of appraisal. responsible executor of the project, in most cases the agency that developed the concept note or project proposal. 42 Chapter 3. Assessment by Indicator for PIM Functions 118. The rarity of negative opinions following 121. The score for this dimension is C. state expert review suggests a lack of impartiality. Commissions are created by line ministries with an interest in advancing projects promoted by their 6.2. CONTENT OF INDEPENDENT REVIEW subordinate entities. Despite stipulations concerning Performance level and evidence for scoring the independence in the regulations, commissions may dimension be subject to the same optimism bias as project promoters, given the context within which they 122. Resolution No. 571 defines information operate. to be presented when a project is submitted to independent review and the content of the 119. After the mandatory state expert review of analysis to be done. Specifically, Article 12 indicates public investment project proposals, MDAs must that the state expert review should analyze: submit them for review at the MoE. MoE performs a second independent review of proposals based on a. The project's compliance with legislative the information submitted to verify correspondence requirements and its urgency and crucial nature; with the legislation and application of guidelines. The b. Environmental, economic, and social proposal must be submitted to MoE before March consequences; 1 each year. It must comply with the information c. The impact of results of implementation of the requirements specified in Resolution No. 571, must project on budget indicators; include findings of an independent state expert review of the investment project and may include d. Assumptions for the cost of the project and its expert review of construction designs. If there are no implementation plan; and reasons to reject a project, i.e., it complies with the e. Availability of appropriate staffing and physical requirements of Resolution No. 571 and guidelines infrastructure for implementation of the project. issued by MoE, projects are provided to the members of the Inter-Agency Commission for consideration and Annex 2 to Resolution No. 571 contains a detailed selection for financing. The large share of non-SSIP form for state expert review. It requires an MDA to projects bypassing the formal process (as detailed in assess the project's compliance with a list of criteria relation to PIM Indicator 5) undermines the impact that reflects mostly key analysis of the project of reviews on the general outcome of the review and proposal and adds detail to the list provided above. selection process. MDA must provide justification for its assessment. 120. Independent review of appraisal is also 123. Details on how to develop the state expert mandatory for other investment projects that review according to Resolution No. 701 are require funding form the state budget or other presented in the "Methodology for carrying out types of support from the state budget. According the state examination of investment projects to the Law of Ukraine On Investment Activities for issued by the MoE."64 An implementing agency, investment projects that require state support, which has developed a project proposal, submits it implemented with the involvement of budget with the supporting documents (financial reports, funds, funds of state enterprises, institutions and design documents) to the MDA. The MDA carries out organizations, as well as loans provided under the state expert review by checking compliance with state guarantees, state expert review is carried the requirements and in case of non-compliance out in accordance with the Resolution No. 701. 63 the project proposal is returned. If the information State expert review is performed by the MDA, to presented is complete, the methodology describes in which an implementing agency reports, or which is detail all aspects that must be analyzed which include: responsible for the sector in which the project will be a. The relevance and strategic alignment of the implemented. investment project;  63  Resolution of CMU No. 701 as of 09.05.2011 on the Procedure for carrying out the state examination of investment projects.  64  MoE Order No. 243 (13/03/2013) for carrying out the state examination of investment projects and the form of the conclusion on its results Chapter 3. Assessment by Indicator for PIM Functions 43 b. The presence of positive social effects; PIM Function 4. Selection and c. The effectiveness of the use of budget funds; Budgeting d. The reliability of technical and economic calculations; e. The validity of the scope and form of providing 4.A. BUDGET PREPARATION AND state support for project implementation; SELECTION f. The availability of appropriate staffing and logistics for project implementation, PIM-7. Project Selection and Budgeting g. The coherence of environmental, economic and RATIONALE social interests; and 126. This indicator assesses four dimensions h. Compliance of the investment project with the central to the selection of sound projects for requirements of the legislation. budget financing – the last point in the PIM system One important aspect missing is any reference to that can prevent the adoption of poor-quality project risks. Risks are not included in Annex 2 of the projects. Effective capital budgeting combines a Resolution No. 571 "Conclusion of State Expertise". top-down process in which each ministry is given an indicative ceiling for capital spending, and a bottom-up 124. The review done by the MoE concentrates process in which agencies prepare and submit sound on analysing the appraisal of the project. Line project proposals for budget consideration. This ministries must validate all data in project studies requires a well-designed budget calendar allowing and sign off on project proposals. MoE officials sufficient time for agencies to prepare their proposals, indicated in meetings held that they verify whether budget guidelines that ensure all project costs reflect calculations are correct, if indicators of economic expected price levels, and well prepared and carefully efficiency have been properly calculated, if there is appraised projects that meet transparent criteria for correct use of the discount rate, and compliance of selecting between competing projects. project implementation and operation period to the recommended one. They also check sources of data 127. The scoring of certain effectiveness and technical parameters of projects. The MoE review dimensions under this indicator is influenced by is quality and compliance focused, but it is not clear the large share of non-SSIP projects that avoid that it satisfies the need to test a project for optimism the formal procedures set out in Resolution No. bias by subjecting the basic assumptions used in the 571. On average during the assessment period, 33.6 estimation of costs and benefits to robust challenge. percent of ongoing state budget-funded projects by total project value65 were non-SSIPs (see Section 2.2). 125. The score for this dimension is C. This figure rose to 46.3 percent in 2021, outside the assessment period. For large projects, the average share for 2018-20 was 31.2 percent.66  65  The figure of 33.6 percent refers to domestically funded projects - SSIPs, non-SSIPs, regional development fund, and capital transfers, but excluding the road subvention, because of the difficulty in distinguishing between maintenance and capital investment. The figure excludes IFI financed projects. If IFI projects are included the figure falls to 19.3 percent, because IFI projects follow formal procedures.  66  16.8 percent including IFI projects. 44 Chapter 3. Assessment by Indicator for PIM Functions Summary of Scores and Performance Table PIM-7 Indicator/ Dimension Score Brief justification for score PIM-7. Project C Scoring Method M1 Selection and Budgeting 7.1. Clear and effective C The budget circular sets general limits on current and capital expenditures top-down budget by sector, but these limits have not identified expenditures on selected state investment projects (SSIPs). Despite the provision of the Budget Code that requires the inclusion in the budget of only projects evaluated and selected according to the established procedure (Resolution No. 571), the budget circular has not set restrictions on projects that did not follow the procedure, nor has it defined strategic priorities for such projects. Due to this, the volume of domestically funded projects included in the budget without application of the appraisal and selection procedures in Resolution No. 571 and regulations applying to SSIPs in the roads sector averaged 31.2 percent during 2018-2020. 7.2. Existence of C There is a well-designed budget calendar with integrated project planning and adherence to cycle for SSIPs, but it has not been adhered to during the last three years. project and budget MDAs had already submitted their detailed spending proposals when they preparation and received a list of their selected SSIPs with identified budget allocations. For approval calendar non-SSIPs, after receiving the budget circular, MDAs had 2 weeks in 2018 and 1 week in each of 2019-2020 to complete their spending proposals. 7.3. Budget guidelines C The legislation contains rules that require MDAs to adjust implementation and practice for and funding plans of new and ongoing SSIPs to budget year estimates. adjustment of project The legislation also contains rules according to which the MDAs adjust budgets to current implementation and funding plans of ongoing SSIPs arising from changing estimates economic conditions or project implementation experiences. The above rules do not apply to the significant share of investment projects that are included in the budget outside of the procedures defined in Resolution No. 571 and regulations applying to SSIPs in the roads sub-sector. 7.4. Transparent C There are many instances where the criteria for project selection are either criteria for project not identified, or not used. Clear criteria based on appraisal results are in selection including place but are not adhered to by the significant proportion of projects that adequate forward bypass the procedures in Resolution No. 571. There is a well-coordinated recurrent budget budget process, which ensures forward recurrent expenditures are met by forecast revenues after providing for implementation of projects under construction and/or operation. Chapter 3. Assessment by Indicator for PIM Functions 45 7.1. CLEAR AND EFFECTIVE TOP-DOWN 130. The Inter-Agency Commission has no strategic BUDGET priorities for selecting SSIPs since there is no investment strategy (PIM-2.2) and the budget Performance level and evidence for scoring the circular contains no specific strategic priorities. dimension The budget circular only contains a requirement for all expenditures to comply with the state policy 128. The budget circular sets total limits on current goals. The goals of state policy, in turn, correspond and capital expenditures by sector, but these to the government's Action Plan, the National limits currently exclude expenditures on SSIPs Sustainable Development Goals, and other forecasting (which follow Resolution No. 571) while including and program documents for economic and social expenditure for IFI projects through budget development. The budget circular has no specific programs and expenditures for SSIPs in the roads strategic criteria to guide the choice of proposals sub-sector.67 The intended procedure, as set out in for non-SSIPs or for roads-sphere SSIPs which are the Budget Code, is for the MoF to provide (through determined during the budget year. the Budget Declaration approved by the CMU) an indicative expenditure limit for investment to the 131. The score for this dimension is C. MoE, so that the Inter-Agency Commission can select SSIPs within this limit, prior to the preparation of the budget circular. The Commission's decisions would 7.2. EXISTENCE OF AND ADHERENCE TO then be reflected in the expenditure limits in the PROJECT AND BUDGET PREPARATION budget circular prepared by the MoF. However, this AND APPROVAL CALENDAR has not been happening and, in the assessed years, Performance level and evidence for scoring the the Commission selected SSIPs after the MoF issued dimension the budget circular, so the circular did not contain this information. 132. There is a well-designed budget calendar which accommodates project preparation for 129. In contrast to the Budget Code,68 the budget SSIPs, but it has not been adhered to during circular does not establish restrictions on the the last three years. The key stages and dates of inclusion in budget requests of project proposals the budget calendar are presented in Table 8. The that have not been selected according to the budget is approved no later than December 1 and, established process for SIPs. The budget circular in accordance with the legislation, contains budget neither prohibits non-SSIPs nor gives MDAs clear allocations for selected state investment projects alternative guidelines for including such projects in (SSIPs). The budget calendar provides sufficient time their budget requests. As a consequence, the budget for ministries to prepare SSIPs and for their review circular does not succeed in limiting the number of by the MoE before selecting them for inclusion in the project proposals from MDAs, allowing non-SSIPs draft budget. to 'jump the fence' and avoid the rigorous appraisal process and selection procedure set out in Resolution No. 571 (see Rationale for Indicator PIM-7 for figures). Table 8. Key Stages and Dates of the Budget Calendar Time period Budget calendar stage and project planning cycle January 21 MoF sends the letter to MDAs regarding the start of Budget declaration preparation By February 15 MDAs submit to the Ministry of Finance cost estimates of state policy goals and changes to the structure of expenditures under budget programs  67  SSIPs in the roads sub-sector follow the selection procedure established by joint Order of the Ministry Infrastructure and Ministry of Finance No. 573/1019 dated September 21, 2012, which take place during the budget year.  68  Part 7 of Article 33-1 of the Budget Code of Ukraine. 46 Chapter 3. Assessment by Indicator for PIM Functions Time period Budget calendar stage and project planning cycle By March 1 MDAs submit proposals to the MoE on state investment projects for selection for the mid-term period By April 6 The Ministry of Finance shall inform the MDAs on the instructions for preparing proposals for the budget declaration and approximate expenditure ceilings for the mid-term (3 years), without expenditures on SSIPs By May 1 The MoE reviews SSIPs, forms a list of projects allowed for selection, determines the total amount of expenditures on SSIPs for the next three years and sends them to the Ministry of Finance By May 15 The Ministry of Finance determines the projected amount of expenditures on SSIPs and includes them in the budget declaration submitted to the Cabinet of Ministers69 By May 25 The MoE prepares information on compliance of SSIPs with the selection criteria; development (implementation) of SSIPs in the budget period under planning and subsequent two budget periods, by area; total expenditures on SSIPs for the next three years and submits it to the Inter-Agency Commission By June 1 The Cabinet of Ministers approves a budget declaration including a defined total amount for expenditure on SSIPs70 Starting from the The Inter-Agency Commission reviews and selects SSIPs allowed for selection within beginning of June the limits of expenditures approved by the budget declaration By June 21 (within The Inter-Agency Commission completes the selection of state investment projects three weeks after the that are to be included in the draft state budget for the next year71 Cabinet of Ministers of Ukraine approves the Budget Declaration) By July 27 The Ministry of Finance shall issue a budget circular informing each MDA of its expenditure ceiling By August 10 MDAs submit budget requests to the Ministry of Finance By August 28 The Ministry of Finance holds bilateral meetings with the MDAs to reconcile requests By August 31 The Ministry of Finance submits a draft state budget to the Cabinet of Ministers By September 15 The Cabinet of Ministers submits a draft state budget to the Verkhovna Rada Until December 1 The Verkhovna Rada approves the state budget for the next year Source: Ministry of Finance 2020 Budget Calendar (Order of the Ministry of Finance of 30.01.2020 No. 32), Budget Code of Ukraine, Resolution of the Cabinet No. 571 dated 22 July 2015, minutes of the meeting of the Inter-Agency Commission on state investment projects dated 04.06.2020.  69  In 2018 the Ministry of Finance had to inform the Ministry of Economy on the total amount of state capital investments for the development and implementation of SIPs, also by May 15.  70  In 2018-2019 the Cabinet of Ministers of Ukraine had to approve the main directions of budget policy for the year under planning, with defined total amount of state capital investments for the implementation of SSIPs, also by June 1.  71  In 2018-2019, the deadline for completing the selection process was not set. Chapter 3. Assessment by Indicator for PIM Functions 47 133. For the period 2018-2020, MDAs had from 1 135. The time available to MDAs for preparing to 2 weeks to complete their spending proposals detailed budget requests for non-SSIPs is very after the budget circulars had been issued; short - two weeks in 2018 and one week in 2019 however, they were informed about selected and 2020. Such projects follow the same timetable as projects after submission of those proposals. The for other categories of non-capital expenditure funded Commission completed the selection of SSIPs during from the state budget. the 2018-2020 period after the budget proposals submission date (Table 9). 136. MoF does not analyze budget proposals for SSIPs given the country project selection 134. Large SSIPs have already been appraised procedure, which is managed by the MoE, and has, by MDAs, and independently reviewed, prior on average, less than 5 weeks to analyze capital to the budget circular. Appraisal (see PIM-3) and spending proposals for non-SSIPs. Table 9 shows state expert review (see PIM-5.3) is a prerequisite that the time available for the MoF to analyze budget of submissions made to MoE for March 1; and MoE proposals including non-SSIPs and brief ministers reviews proposals from March – April (see also prior to a final cabinet decision varied from 1 to 5 PIM-6.1). weeks, averaging 3 weeks. SSIPs are included in the state budget after the Cabinet has submitted annual draft budgets to the VRU, so there is no scope for MoF to analyze these submissions. Table 9. Timeframe for Preparation and Review of Budget Requests 2018 2019 2020 Date of SSIPs selections June 21 for 10.6 July 9 (in the amount of 40 November 16 percent more than percent more than provided indicative limit by the MoF) and November and finalized on 1-3 (within the established September 17 (by amount, which gave the 6.7 percent more MDAs the right to start filling than previous out budget requests) selection) Budget circular issue date July 27 September 3 August 11 (expenditures for SSIPs are not included) Budget requests submission date August 9 September 9 August 20 Time available for budget MDAs have already MDAs have already MDAs have already requests SSIPs completion completed their completed their detailed completed their detailed spending spending proposals detailed spending proposals proposals Time available for budget 2 weeks 1 week 1 week requests non-SSIPs completion Time available for MoF to review 5 weeks 1 week 3 weeks proposals and provide briefing to ministers (to September 15) Source: Minutes of the meetings of the Inter-Agency Commission on state investment projects dated 21.06.2018, 17.09.2018, 09.07.2019, 01.11.2019/03.11.2019, 16.11.2020; MoF's letters (budget circulars) dated 27.07.2018 No. 04110-09-9/20040, dated 03.09.2019 No. 04110-09-10/22087, dated 11.08.2020 No. 04110-08-2/24733. 48 Chapter 3. Assessment by Indicator for PIM Functions 137. The laws on the state budget for 2018-2020 The Inter-Agency Commission selects projects based were approved before the beginning of the on the results of the feasibility study in accordance fiscal year. The VRU approved the law on the state with the following criteria: the degree of project budget for 2018 on 7 December 2017; for 2019 on 23 implementation, positive economic effect or economic November 2018; and for 2020 on 14 November 2019. efficiency, positive social and environmental effects, justification for choosing the sources of funds 138. The score for this dimension is C. necessary for the operation of the facility, and savings in the cost of operating the facility compared to such costs before the project. 7.3. BUDGET GUIDELINES AND PRACTICE FOR ADJUSTMENT OF PROJECT 143. Projects with a political imperative can bypass BUDGETS TO CURRENT ESTIMATES the procedures for SSIPs and be included in the budget. A large share of investment projects in the Performance level and evidence for scoring the state budget do not follow the selection procedure for dimension SSIPs (see Rationale for Indicator PIM-7 for figures). 139. The legislation contains rules that require As part of the mainstream budget process, MDAs may MDAs to adjust implementation and funding include expenditures on the implementation of such plans of new and ongoing SSIPs to budget year projects in budget requests within their expenditure estimates.72 The rules establish that the MDAs must ceilings set by the MoF. As well as this route, the VRU make such adjustments within one month from the may require the government to include non-SSIPs, date of adoption of the law on the state budget for when finalizing the draft state budget for the second the corresponding year (or amendments to it) or the reading. corresponding decision of the CMU. Budget program 144. There is a well-coordinated budget process, passports73 can be formulated only based on the which ensures forward recurrent expenditures are adjusted information. met by forecast revenues. MDAs submit calculations 140. The above rules do not apply to non-SSIPs of new expenditures or an increase in existing and SSIPs following Order No. 573/1019. There is expenditures for the next three years to MoF in the no information about the availability of appraisal process of preparing a budget declaration. These estimates or findings for such projects, which calculations consider, among other things, the forward could form the starting point for making informed recurrent expenses for investment projects to be judgements on adjustments. completed in the current year and the next two years. MoF considers such an increase in expenditures when 141. The score for this dimension is C. calculating the expenditure ceilings for the relevant MDA for the next three years. 7.4. TRANSPARENT CRITERIA SELECTING 145. The score for this dimension is С. PROJECTS FOR BUDGET FUNDING, INCLUDING ADEQUATE FORWARD RECURRENT BUDGET PIM-8. Multi-year Budgeting Performance level and evidence for scoring the dimension RATIONALE 142. Clear criteria based on appraisal results 146. This indicator assesses two dimensions: are in place and adhered to for prioritization (i) multi-year fiscal forecasts and functional and budget selection of SSIPs. MDAs carry out allocations and (ii) multi-year public investment the feasibility study for all SSIPs, which includes an program (PIP) databases. A multi-year perspective analysis of a project's economic efficiency (see PIM-3). is particularly important to secure the funding of  72  Resolution of the Cabinet No. 571 dated 22 July 2015 'Some issues of public investment management': https://zakon.rada.gov.ua/laws/show/2059-19#Text.  73  Budget program passport is a document that defines the goal, objectives, directions for the use of budgetary funds, performance indicators and other characteristics of the budget program. MDAs approves their budget program passports after the MoF's review. Chapter 3. Assessment by Indicator for PIM Functions 49 Summary of Scores and Performance Table PIM-8 Indicator/ Dimension Score Brief justification for score PIM-8. Multi-year D+ Scoring Method M2 budgeting 8.1. Multi-year D Mid-term capital expenditure forecasts met one of the six requirements fiscal forecasts and and partially met another in two of the three assessed years. functional allocations 8.2. Multi-year C There is no comprehensive project database. Existing separate projects database(s) of projects databases have significant gaps and do not facilitate mid-term fiscal with approved funding management or management of the total cost of each project. MoE keeps records of SSIPs in Excel files containing a limited amount of information. The Ministry of Regional Development maintains a database of projects financed under the State Fund for Regional Development (SFRD), which also contains a limited list of information. MoF maintains a centralized financial database on the formulation and execution of all state budget expenditures, which contains separate, but limited, information on non- SSIPs, donor projects and SSIPs, along with other numerous information on non-investment expenditures. capital investment projects that span a number of of the current expenditure ceilings for MDAs, years, and to ensure the future affordability of their expenditures for the implementation of SSIPs and, as O&M impacts. Allocation of capital spending to the a separate line, forecasts of other capital expenditures most productive sectors and projects requires a (other than SSIPs). In 2018, the government submitted comprehensive, unified, and mid-term perspective to a forecast for 2020-2021 to the VRU, together with capital budgeting, as well as comprehensive databases the draft law on the state budget for 2019. This that are maintained for all projects to be monitored forecast contained the allocation of current and part and controlled from the stage of proposal and design, of capital expenditures by functions, the forecast of through appraisal, approval, budget selection, and expenditures for the implementation of SSIPs, and implementation up to project completion. as a separate line, unallocated by functions, part of other capital expenditures. Starting from 2019, the budget declaration replaced the Key Directions of 8.1. MULTI-YEAR FISCAL FORECASTS AND Budget Policy document. The budget declaration for FUNCTIONAL ALLOCATIONS 2020-202274 was supposed to provide the forecast of expenditures for 2021-2022 in the form of expenditure Performance level and evidence for scoring the ceilings for MDAs (without identification of capital dimension expenditures within them), and the total amount for 147. The government has not been consistent in SSIPs. However, the government has not approved it, preparing rolling mid-term budget estimates, so there is no forecast for these years. achieving this in only two out of the three years 148. On average only 25 percent of projected under consideration. In 2017, the government development expenditures for 2018-2020 are approved the forecast for 2019-2020 as part of the consistent with sector priorities set by MDAs in Key Directions of Budget Policy for 2018-2020. That their sector strategies (see item (a) of Table 11).75 forecast contained, among other things, a forecast  74  At the end of 2018, a reform of mid-term budget planning was carried out, which provides for the introduction of the Budget Declaration as a mid-term budget document.  75  Note that there were no mid-term projections for 2021-22, so it is not meaningful to verify strategic priorities. 50 Chapter 3. Assessment by Indicator for PIM Functions Based on the Budget Code76 all MDAs must prepare been selected,78 MoF has consistently included much their mid-term action plans (sector strategies). MDAs smaller amounts in its mid-term budget forecasts. should consider those mid-term action plans when For example, in the MoF forecast prepared in 2018, elaborating budget programs and while preparing its estimates were nearly five times less than the proposals for the budget declaration. Despite aggregated implementation plans for 2020 and three this requirement, in 2018 only 28.6 percent of times less than for 2021.79 MDAs prepared such plans with both financial and performance forecasts. The figures for 2019 and 2020 152. If a project is to be completed during the were 22.8 percent and 23.6 percent, respectively. forecast period, the forecast includes the recurrent expenditures required for the operation 149. Transfers to local budgets are reflected in the and maintenance of newly created facilities (item forecast as a lump sum, so it is difficult to assess (e) of Table 11). MDAs provide the Ministry of Finance their compliance with the strategic documents of with information on new expenditures and changes local authorities (see item (b) of Table 11). in the structure of expenditures for the coming three years at the beginning of the budget process. The 150. It is impossible to determine compliance with Ministry of Finance takes this information into account a notional "Golden Rule" because the necessary when making a forecast for the corresponding period. data is not available.77 The expenditure forecasts do not contain the total amount of capital expenditure 153. Approved capital expenditures, as a rule, from the state budget and, in 2018, the budget differed significantly from the forecast (Table 10), forecast also did not contain the amount of debt and explanations for deviations were not provided financing. So, at the stage of issuing the forecast, (item (f) of Table 11). The budget documentation for compliance with the Golden Rule (which has not been the corresponding years did not provide explanations formally adopted in any event), is not possible (see on the reasons for the deviation of capital item (c) of Table 11). expenditures from those that were projected in the previous year. 151. The mid-term expenditure forecasts do not take account of the implementation 154. Mid-term budget estimates covering 3 plans for SSIPs, and there is no indication years on a rolling basis partially contain capital that this is done for non-SSIPs either (item (d) expenditure plans and are consistent with 2 of the of Table 11). Whereas MoE routinely provides MoF 6 requirements (Table 11). with expenditure projections aggregated from the individual expenditure plans for the SSIPs that have 155. The score for this dimension is D. Table 10. Projected and Approved Expenditures for 2019-2021 2019 2020 2021 State capital investments Forecast for 2017, UAH billion 1.7 1.7 Approved by law, UAH billion 4.2 2.4 Deviation, % 250% 140%  76  Fifth part of the Article 22.  77  The 'Golden Rule' of government spending is a fiscal policy stating that a government should only increase borrowing over an economic cycle in order to invest in projects that will pay off in the future. This allows for borrowing to cover recurrent expenditure in a downturn, provided borrowing is lower in the upswing.  78  Paragraph 3 of Point 19 of the procedure for selecting state investment projects, approved by Resolution of the Cabinet of Ministers of Ukraine No. 571 of 22 July 2015.  79  Official data for other years is not available. Chapter 3. Assessment by Indicator for PIM Functions 51 2019 2020 2021 Forecast for 2018, UAH billion 2.1 2.1 Approved by law 2.4 4.0 Deviation, % 114% 190% Capital expenditures of the general fund Forecast for 2017, UAH billion 13.0 13.9 Approved by allocation, UAH billion 50.6 44.8 Deviation, % 390% 320% Forecast for 2018 (unallocated), UAH billion 8.5 36 Approved by allocation, UAH billion N/E* N/E* Deviation, % N/E* N/E* * N/E – Not existing – Information does not exist Source: The main directions of budget policy for 2018-2020; forecasts of the State Budget of Ukraine for 2018-2019 and for 2020-2021, submitted with the budget bills for 2017 and 2019 respectively; state budget laws for 2019 and 2020; state budget execution reports by economic classification for January 2019 and 2020. Table 11. Compliance of Mid-term Capital Expenditure Forecasts Prepared in 2017, 2018 and 2019 with the Established 'Good Practice' Criteria Forecasts prepared for Capital expenditure plans 2018-2020 2019-2021 2020-2022* (a) consistent with national and sector strategies and priorities; Partial Partial No (b) coherent with project and program policies and plans of local No No No government; (c) borrowing requirements consistent with Golden Rule over an No No No economic cycle; (d) provide for appropriation of funds for the length of the No No No construction, contract, concession or agreement period; 80 (e) appropriation of recurrent budget is provided for over the Yes Yes No period consistent with sustaining operations and maintenance of the capital or development expenditures; (f) annual budget capital expenditures are consistent with mid- No No No term estimates or, otherwise, significant differences are clearly explained. * The Cabinet of Ministers of Ukraine did not submit the forecast to the VRU  80  Ukraine has had no experience with 'government-pays' PPPs. 52 Chapter 3. Assessment by Indicator for PIM Functions 8.2. MULTI-YEAR DATABASE(S) OF 159. The MoF maintains a database on the PROJECTS WITH APPROVED FUNDING planning and execution of all state budget expenditures, which contains some information on Performance level and evidence for scoring the SSIPs, non-SSIPs and donor projects. This database dimension only has information on the amount of funding for three years (in budget requests) or for one year (in 156. There is no comprehensive database of all budget program passports and reports on their projects selected for funding through the state implementation) - and on performance indicators for budget. The lack of a comprehensive database project implementation, such as the area of premises prevents all projects from being monitored built, or the quantities of equipment purchased. The and controlled from budget selection through database also contains information on the total capital implementation and up to project completion, review, cost of SSIPs and some non-SSIPs. The database and registration. Each ministry maintains its own identifies SSIPs and IFIs projects separately, since database to manage selection and budgeting of they are classified under separate budget programs; projects within its authority. None of these databases however, there is no means of identifying non-SSIPS contains complete information about an investment because they usually are included in budget programs project at each stage of its implementation. with other expenditures. 157. MoE keeps partial records of SSIPs in 160. The score for this dimension is C. Excel format. The project data stored covers the implementation period, total capital cost as currently estimated, total amount funded in previous years, the planned amount for the current year and funds PIM-9. Comprehensive Capital Budget needed for completion, and funds allocation for three forecast years. The data does not provide descriptive RATIONALE details; the initial cost of the project, either as a whole 161. Capital expenditure requires comprehensive, or by year; the expected service delivery and / or accurate, and timely reporting to decision makers benefits; adjustments to costs and actual costs by during the year on actual spending compared year; or planned and achieved physical milestones. to budget and reporting to the legislature at the 158. The Ministry of Regional Development end of year. Comprehensive information on capital maintains a database of projects funded under spending, no matter how or where it is financed, is the State Fund for Regional Development (SFRD), required for a full appreciation of PIM activities. In which also contains a limited list of information. many countries, central government capital spending This database is available online81 and includes the is also financed by autonomous government agencies following information: the year of start and completion or extra-budget entities, such as Road Funds, of the investment component of the project, the scope Infrastructure Funds, Natural Resource Funds, or of its implementation, the total capital cost of the Privatization Funds. If this spending is not included project (allocated by sources of financing), the general in central government fiscal reports (other than as contractor/service provider, and information about a single line item for each entity) it is defined here the status of the project implementation. as unreported – even though it may be included in separate reports or financial statements of the entities themselves. Budget documentation should take a comprehensive view of all public investment activities; not just those directly financed through the general budget.  81  https://dfrr.minregion.gov.ua/Projects-list Chapter 3. Assessment by Indicator for PIM Functions 53 Summary of Scores and Performance Table PIM-9 Indicator/ Dimension Score Brief justification for score PIM-9. C+ Scoring Method M2 Comprehensive capital budget 9.1. Extra-budgetary D Capital expenditures of extra budgetary funds are not included in central capital expenditures government fiscal reports; however, there are no funds making major capital investments. The Treasury includes capital expenditures related to contingencies arising from natural disasters, etc., in budget execution reports. The level of SOE capital investments financed using budgetary lending and state guarantees exceeded 20 percent of total public capital spending in two of three assessed budget years. 9.2. Budget coverage A There is complete budget information and coverage for all projects and of donor funded programs funded with country programmable aid under government projects and programs control. All donor funds are included into the budget execution reports. 9.1. EXTRA-BUDGETARY CAPITAL 163. The Treasury includes capital expenditures EXPENDITURES related to contingencies arising from natural disasters, etc., in budget execution reports. Performance level and evidence for scoring the The contingency reserve fund is presented in the dimension original budget by a separate budget program. When contingencies need to be drawn, the expenditures 162. Ukraine has no extra-budgetary funds making are reallocated to the relevant budget program in significant capital investments, only the social expenditure reports. funds with little or no such expenditure.82 The Treasury does not include the capital expenditures of 164. The level of SOE capital investments at the extrabudgetary funds in the monthly, quarterly and expense of borrowing and state guarantee84 in two annual reports submitted to the government, but the out of the three assessed budget years exceeded main funds did not have capital investments during 20 percent of total public capital spending. SOEs the assessed period. Each of the extra-budgetary submit their financial statements to the state body to funds prepares quarterly and annual reports on its which they belong,85 the MoE or the MoF.86 Financial budget (estimates) execution, which include, inter alia, statements contain the total capital investments expenditures by economic classification, including at the expense of attracted funds, including state capital expenditures.83 The funds submit quarterly guarantees and crediting from the state budget reports to the Treasury and MoF, as well as to the VRU, within donor funded projects. The statements do not, the President of Ukraine, CMU, and the Accounting however, show separately capital expenditures at the Chamber of Ukraine (ACU). expense of borrowing and state guarantees. The MoE  82  The Pension Fund, the Social Insurance Fund, and the Unemployment Social Insurance Fund.  83  Based on the Procedure for preparation of financial and budgetary reporting by managers and recipients of budget funds, approved by the Order of the Ministry of Finance of Ukraine dated 24.01.2012 No. 44.  84  Since there is no information about actual capital investments of SOEs by state lending and guarantees, the calculation is based on all capital investments of SOEs by attracted credit funds.  85  Based on Decree of the Cabinet 'On approval of the Procedure for the financial statements submission' No. 419 dated 28 February 2000: https://zakon.rada.gov.ua/laws/show/419-2000-%D0%BF#Text.  86  Based on Order of MoE 'On approval of the Procedure for drawing up, approving and monitoring the implementation of the financial plan of a business entity of the public sector of the economy' No. 205 dated 2 March 2015: https://zakon.rada.gov.ua/laws/show/z0300-15#Text. 54 Chapter 3. Assessment by Indicator for PIM Functions consolidates this information and publishes on its matching expenditures. Accordingly, reporting on such website but does not submit it to the legislature. SOEs' expenditures is included in the reporting on the state capital investments at the expense of lending and budget execution on a monthly, quarterly and annual state guarantees accounted for 26.3 percent of total basis. public capital spending in 2018, 23.8 percent in 2019, and 18.0 percent in 2020. 167. The score for this dimension is A. 165. The score for this dimension is D. PIM-10. Comprehensiveness and 9.2. BUDGET COVERAGE OF DONOR Degree of Public/Parliamentary Access FUNDED PROJECTS AND PROGRAMS to Capital Budget Information Performance level and evidence for scoring the RATIONALE dimension 168. The annual budget presented by the executive 166. There is complete budget information and to the legislature, including the supporting coverage for all projects and programs funded documents submitted with the budget, should with country programmable aid.87 According to provide a complete picture of central government the legislation,88 loans from foreign countries, and capital expenditure and be easily accessible international financial institutions for investment to the public. This should include information on projects implementation are considered as capital transfers to sub-national governments, extra- government external borrowing. Annual state budget budgetary activities, and off-budget activities such as laws include expenditures for the implementation of quasi-fiscal activities and Public Private Partnerships such investment projects. The legislation prohibits the (PPPs). Information at the project level may be budgetary institutions from having extra-budgetary contained in annual plans or similar documents of funds. Thus, once a budget institution expects to MDAs, rather than in the budget documents, provided receive or receives international technical assistance these are submitted to the legislature either at the grants (which include grants for construction), they same time as the budget, or, if after the budget is include them in their budgets as own revenues, with submitted, prior to the start of the budget year. Summary of Scores and Performance Table PIM-10 Indicator/ Dimension Score Brief justification for score PIM-10. B Scoring Method M2 Comprehensiveness and degree of public / parliamentary access to capital budget information 10.1. Budget C Budget documentation shows full compliance with two elements and information content partially with three elements. 10.2. Public access to A The requirements are met for four elements out of five. budget information  87  'Programmable Aid' is an OECD-DAC term, and it includes lending by development banks, like WB and EBRD.  88  According to the ninth part of Article 13 of the Budget Code: https://zakon.rada.gov.ua/laws/show/2456-17#Text. Chapter 3. Assessment by Indicator for PIM Functions 55 Indicator/ Dimension Score Brief justification for score 10.3. Policy on A Budget Code is published on the legislature's website and requires publication of budget publication of all budget documents including their immediate availability information on the relevant websites. 10.4. Legislature's C The Parliament considers detailed capital expenditures of the state access to information budget, including projects funded by donors, as part of budget requests. and rule-bound The timetable and procedure for the legislature's review of the budget is procedures clearly set down in the legislation and gives two months for this review, which has been adhered to. The legislature adds investment project proposals to the budget, amounting to more than 10 percent of the total investment expenditures. Clear rules exist for in-year amendments to capital spending by the executive, but with strict limits on the extent to which capital expenditures may be expanded and re-allocated applying to SSIPs only. Retroactive approval by the legislature of budget amendments in relation to capital expenditures is not allowed. 10.1. BUDGET INFORMATION CONTENT 170. The budget documentation, which is all publicly available, contains summarized data on Performance level and evidence for scoring the central government development expenditures dimension for the budget year by MDA,90 but does not contain separate data on capital expenditures. 169. The government is required to submit a list Development expenditures include, in addition of SSIPs to the VRU with the draft budget,89 but to capital expenditures, certain current expenses, the project selection of SSIPs during the 2018-2020 notably, direct expenditure on research and period was completed after the budget submission development, and current transfers to enterprises date. This list should contain each project's total in the field of research, agriculture, fuel and energy cost, remaining budget expenditure needs for complex, transport, communications. The VRU project implementation and proposed budget publishes annual budget documentation on its allocations for the following year. But the government website.91 The capital expenditures for implementing has not included such information in the budget donor-funded projects are part of development documentation because the Inter-Agency Commission expenditure. was late with the project selection. Once the selection was completed, the government included the results 171. The requirements are met for two elements in the draft budget resubmitted to the VRU for the and partially met for three elements out of the second reading (with a separate budget program for required nine (see Table 12). each SSIP). 172. The score for this dimension is C.  89  According to paragraph 4-1 of Article 38 of the Budget Code of Ukraine.  90  According to Part 7 of Article 10 of the Budget Code of Ukraine.  91  The budget proposal for 2021 can be found at the following link: https://w1.c1.rada.gov.ua/pls/zweb2/webproc4_1?pf3511=69938. 56 Chapter 3. Assessment by Indicator for PIM Functions Table 12. Compliance with Criteria for Assessing the Information Content of the Budget Information content of budget Consistent Explanation documentation (Yes/No) 1. Summarized data on central No The budget documentation contains summarized data government capital expenditure on central government development expenditures for for the budget year, by MDAs (or the budget year by MDA but does not contain such data by sub-function or program). Data on capital expenditures. should also include budget and The annual report submitted by the CMU to the VRU estimated outturn for the current and published on the website of the Treasury contains year. actual indicators for the previous year in the same format as the indicators of the budget documentation (i.e., no summarized data on capital expenditures). Annexes to the approved State Budget Law for the current year, which can be found on the website of the VRU, contain approved indicators for the current year in the same format as the indicators of the budget documentation (i.e., no summarized data on capital expenditures). 2. The amount appropriated by Partially In the budget and, accordingly, the draft budget, each MDAs for the budget year for each large SSIP is presented as a separate budget program major central government budget- and funds with relevant appropriations by budget financed capital spending project. program for the planned budget period. However, large non-SSIPs are often included in other budget programs with diverse other expenditures; so, there is no specific project appropriation. 3. The total approved multi-year No The government did not include such information in the cost of each major capital project, budget documentation, although this is provided for in and the total estimated amount paragraph 4-1 of part one of Article 38 of the Budget spent up to the end of the current Code of Ukraine. year. 4. Details as in 2-3 above for all Partially For capital projects funded by donors, no capital central government capital projects expenditure figures are provided (as indicated in financed by donors. Paragraph 1). The draft budget contains information on donor-funded capital projects on loan terms in a separate annex. However, this annex contains only the total amount of loan for the project implementation and does not contain the total cost of the project, which may also include co-funding from the Ukrainian side. 5. A brief description of the multi- No The government does not prepare such information. year budgetary impact, and policy justification for all new major capital spending policy initiatives being introduced in the budget (including those financed by donors). Chapter 3. Assessment by Indicator for PIM Functions 57 Information content of budget Consistent Explanation documentation (Yes/No) 6. Details of capital grants or Yes Such information is contained in budget requests and transfers, and of lending (including is reflected in the directions of using budget funds and on-lending) by central government performance indicators. to state owned enterprises and/or sub-national governments that is predominantly intended to finance capital expenditure. 7. Details of contingent liabilities Yes Budget documentation contains information on fiscal of central government relating risks and their impact on the state budget indicators in to capital spending, such as the planned year,92 including details about fiscal risks guarantees for SOE or sub-national related to state debt and state guarantees, as well as to government borrowing to finance the activities of SOEs. capital projects. 8. The nature, rationale for, and Partially The budget documentation contains a list of quasi-fiscal estimated fiscal impacts of any transactions and their possible impact on the budget quasi-fiscal activities relating indicators for the planned year, but they do not cover to capital spending, where of public capital investments.93 The budget documentation fiscal significance; and of any tax also contains a list of all tax benefits with the calculation expenditures relating to capital of budget revenue losses in the current year and spending. planned year, including tax expenditures relating to capital spending.94 9. For any PPPs, disclosure of the No Information on fiscal risks in the budget documentation long-term stream of purchase for 2021 (paragraph 7 of this table) explains that the payments and receipts; details of assessment of fiscal risks related to the implementation contract provisions that give rise to of contracts concluded under the PPP, including contingent payments or receipts; concession agreements that may arise in the year being amount and terms of financing planned, was not conducted. provided by entities owned or controlled by government; and disclosure of how the projects affect the reported fiscal balance and public debt.  92  According to subparagraph 121 paragraph 1 of Article 38 of the Budget Code of Ukraine.  93  According to subparagraph 121 paragraph 1 of Article 38 of the Budget Code of Ukraine.  94  Examples of infrastructure quasi-fiscal transactions include lower service prices, lower consumer fees, or a formal or informal requirement for a private company to build a public asset, e.g., mining company shall build public road near the mine. 58 Chapter 3. Assessment by Indicator for PIM Functions 10.2. PUBLIC ACCESS TO BUDGET information on the execution of the state budget INFORMATION of Ukraine based on monthly, quarterly and annual results.96 ACU publishes on its website annual Performance level and evidence for scoring the reports on its activities, reports on state external dimension financial control (audit) activities, work plans, and its decisions.97 The State Audit Service publishes 173. Requirements for the disclosure of fiscal audit reports on its website.98 Procuring entities information are established for different public ensure that information about procurement is made authorities and are met in most respects. The public.99 Table 13 details public access to information VRU publishes on its website all registered draft according to the five specified requirements. The laws and supporting documents, including the draft requirements are met for four elements out of five. state budget and its supporting materials (budget documentation).95 MoF ensures the publication of 174. The score for this dimension is A. Table 13. Compliance with Criteria Relating to Public Access to Budget Information Consistent Elements of public access Explanation (Yes/No) 1. Details of all capital spending No Such information is not available to the public because in the annual budget, as defined it is not submitted to the legislature in sufficiently in element 1 of the Table 12, can disaggregated form: capital spending is included in be obtained by the public through the wider category of development spending (see appropriate means when it is explanation for item 1 of Table 12). submitted to the legislature. 2. In-year budget reports on Yes Monthly reports on the budget expenditures execution execution of capital spending contain information on the execution of total capital are routinely made available to expenditures. These reports are published on the the public though appropriate Treasury's website within one month of the end of the means within one month of their reporting month: completion.  https://www.treasury.gov.ua/ua/file-storage/ vikonannya-derzhavnogo-byudzhetu  3. Year-end reports on execution Yes Annual reports on the budget expenditures execution of capital spending are routinely also contain information on the execution of total made available to the public though capital expenditures. These reports are published on appropriate means within six the Treasury's website within three months of the end months of completed audit. of the reporting year:  https://www.treasury.gov.ua/ua/file-storage/ vikonannya-derzhavnogo-byudzhetu   95  Article 92 of the Law of Ukraine No. 1861 dated 10 February 2010 'On the Rules of Procedure of the Verkhovna Rada of Ukraine'.  96  Article 28 of the Budget Code of Ukraine.  97  Article 30 of Law of Ukraine No. 576 dated 2 July 2015 'On the Accounting Chamber'.  98  Article 15 of the Law of Ukraine No. 2939 dated 13 January 2011 'On Access to Information in Public Domain'.  99  According to paragraph 10 of part one of Article 10 Law of Ukraine No. 922 dated 25 December 2015 'On public procurement'. Chapter 3. Assessment by Indicator for PIM Functions 59 Consistent Elements of public access Explanation (Yes/No) 4. All external audit reports Yes All reports of the Accounting Chamber of Ukraine, which on central government capital is a Supreme Audit Institution, including reports on spending are made available to capital expenditures, are published on its the website: the public within six months of  https://rp.gov.ua/FinControl/FinReports/ . completed audit. All audit reports of the State Audit Service, which is the government audit service, including reports on capital expenditure, are also published on its website:  https://dasu.gov.ua/ua/plugins/userPages/54 . 5. All contract awards with value Yes Procuring entities publish the procurement contract above approx. US$100,000 and all its appendices in the publicly available electronic equivalent are published at least procurement system ( prozorro.gov.ua ) – within three quarterly through appropriate working days from the date of its conclusion. means. 10.3. POLICY ON PUBLICATION OF 28, MDAs are required to ensure the publication of BUDGET INFORMATION their budget requests on their websites, no later than three working days after the submission of the Performance level and evidence for scoring the draft State Budget Law to the VRU; budget program dimension passports and reports on their execution; reports on the implementation of SSIPs; and the results of 175. The Budget Code (published on the evaluations of the performance of budget programs. legislature's website) requires publication of all budget documents, including immediate 176. The score for this dimension is A. availability of all documents on the website. Article 28 of the Budget Code sets out requirements for availability of budget information. In accordance 10.4. LEGISLATURE'S ACCESS TO with these requirements, the MoF is required to INFORMATION AND RULE-BOUND ensure the disclosure of the following: (i) budget PROCEDURES declaration together with relevant information and Performance level and evidence for scoring the analytical materials; (ii) draft State Budget Law in the dimension newspaper "Uryadovy Courier" no later than seven days after its submission to the VRU; (iii) the voted 177. The VRU reviews detailed capital expenditures State Budget Law; (iv) information and analytical of the state budget, including projects funded materials on the state budget (on its official website by donors. Such information is included into MDAs in a form accessible to the public) within one month 'budget requests. MoF submits those requests to from the date of publication of the law; (v) information the Budget Committee of the VRU as part of budget on state budget execution according to the monthly, supplementary documents. Members of Parliament quarterly and annual results, including publication of also review budget requests within meetings of the the annual information in the newspaper "Uryadovy VRU's sectoral committees. Draft budgets of social Courier", no later than March 1 of the year following security funds included in budget documentation do the reporting year; and (vi) information on the not contain information on capital expenditures. The execution of the consolidated budget (state and local government does not submit information on PPP budgets). In addition, according to the same Article projects to the VRU. 60 Chapter 3. Assessment by Indicator for PIM Functions 178. The timetable and procedure for the projects that had not been appraised by the legislature's review of the budget is clearly set government in the draft state budget with a value down in the legislation and gives two months of 19.7 percent of the total public investments. The for this review, which has been adhered to. CMU figures for 2019 and 2020 were 13.3 percent and must submit the draft State Budget to the VRU by 10.3 percent, respectively. The VRU assigned the September 15 of the year preceding the year under government to take its proposals into account in planning. 100 The review procedure provides that the process of preparing the draft State Budget for national deputies and specialized committees then the second reading. These proposals in most cases send their proposals to the Budget Committee before concerned local development projects. October 1 for it to reach its conclusions.101 According to the schedule, the review and first reading of the 180. Clear rules exist for in-year amendments draft State Budget Law must be completed before to capital spending by the executive, with strict October 20, and no later than November 20 for the limits on the extent to which capital expenditures second reading.102 Review of the draft budget in the may be expanded and re-allocated applying first and second readings and of submitted proposals, to SSIPs only. The Inter-Agency Commission may the conclusions of which are prepared by the Budget propose amendments to SSIPs under certain Committee, are carried out at the VRU meeting. 103 conditions:104 a violation of the terms of development Table 14 shows the extent of adherence to the formal or implementation, an increase in costs by more timetable for the period 2018-2020. than 10 percent, or when a discrepancy between the object of procurement and the final result (which 179. The VRU has the right to include its own is planned to be achieved as a result of the project) projects and capital subventions in the budget has been identified. CMU may decide to reallocate and does so. In 2018, the VRU included investment expenditures on projects based on the proposals of Table 14. Key Dates for Review of the Draft State Budget by the VRU Formal 2018 2019 2020 Submitted by the government to By September 15 September 15 September 15 September 14 the VRU Adopted by the VRU in the first By October 20 October 10 October 10 November 5 reading Submitted by the GOU for the By November 11 November 19 November 5 November 27 second reading Adopted by the VRU in the second By November 20 November 23 November 14 December 15 reading Adopted as a whole By December 1 November 23 November 14 December 15 Source: Official Website of the Verkhovna Rada of Ukraine  100  Part three of Article 37 of the Budget Code of Ukraine.  101  Part one of Article 156 of the Law of Ukraine 'On the rules of procedure of the Verkhovna Rada of Ukraine' dated 10.02.2010 No. 1861.  102  Part four of Article 158 of the Law of Ukraine 'On the rules of procedure of the Verkhovna Rada of Ukraine' dated 10.02.2010 No. 1861.  103  Part four of Article 157 and part seven of Article 158 of the Law of Ukraine 'On the rules of procedure of the Verkhovna Rada of Ukraine' dated 10.02.2010 No. 1861.  104  Resolution of the Cabinet of Ministers of Ukraine No. 571 of 22.07.2015. Chapter 3. Assessment by Indicator for PIM Functions 61 the Inter-Agency Commission. The Budget Code only establishes a procedure, but no restrictions, regarding 4.B. BUDGET OUTTURN increases in development expenditures, 105 which can PERFORMANCE only be achieved reducing other expenditures through a CMU decision. There is no rule or restriction on the PIM-11. Development and Capital downward reallocation of development expenditure, Budget Execution Rates: Aggregate and such reallocations can lead to a significant reduction in development expenditure (PIM-11.1). Any Expenditure Outturn Compared increase that is not budget neutral requires approval To Adjusted Original Budget on a of the VRU. Commitment Basis 181. Retroactive approval by the legislature of RATIONALE budget amendments with respect to capital expenditures is not allowed. The Budget Code only 183. The capacity to implement the capital budget allows retroactive budget amendments in relation is an important indicator of PIM performance. to debt repayment and debt service. Specifically, Under-execution of capital spending seriously inhibits expenditures for servicing and repaying public debt governments' ability to translate public expenditure are executed in accordance with loan agreements, into improved public services. Under-execution even if the amount of such expenditures exceeds may reflect a variety of underlying problems in the amount approved by the law. All other forms of PIM processes, including poor project preparation, retroactive spending decisions are not allowed. underestimating costs or the time required to implement a project, over-budgeting relative to actual 182. The score for this dimension is C. costs, bottlenecks in implementation at procurement or construction stages, or delays in disbursements of budgeted funds. Alternatively, over-execution compared to the original budget may reflect the fact Summary of Scores and Performance Table PIM-11 Indicator/ Dimension Score Brief justification for score PIM-11. Development D Scoring Method M1 and capital budget execution rates: Aggregate expenditure outturn compared to original budget on a commitment basis 11.1. Budget execution D In two of the last three years the actual development expenditure rate (excluding donor funded projects) deviated from budgeted amount by more than 25 percent. 11.2. Donor budget D In all of the last three years actual donor funded project development execution rate expenditure deviated from estimated project expenditure by an amount equivalent to more than 25 percent of budgeted development expenditure.  105  Capital expenditures are part of the development expenditures approved by the state budget law. 62 Chapter 3. Assessment by Indicator for PIM Functions that the original budget did not properly incorporate projects) were 3.0 percent in 2018, -43.6 percent in project costs or expected inflation adjustments or 2019, and -44.5 percent in 2020 (Table 15). The low may simply be due to a policy decision to increase level of implementation of development expenditures capital spending during the year, e.g., by means of in 2019-2020 was influenced, in particular, by MDAs' a supplementary budget, in response to a change significant redistribution between development in circumstances such as an economic downturn, expenditures and consumption. or an unexpected surge in revenues. It is important that the factors that have contributed to the variance 185. By contrast to development expenditures, between the original budget for the project and consumption expenditures exceeded the budgeted the final outturn are identified and analyzed. The amounts in 2019 and 2020, by 3.2 percent and 18.6 variance across economic objects between budget percent, respectively. Notably, salary expenditures and outturn provides a starting point for more in- were exceeded by 1.6 percent and 3.9 percent in depth analysis as to why and how capital spending these two years. Utilities and energy expenditures was treated differently than other objects during were, in contrast, under-executed by 7.1 percent budget implementation, and the impacts of this on the and 25.2 percent (see Table 16). Total state budget efficiency and effectiveness of project implementation. expenditures were under-executed by 3.3 percent in 2019 and over-spent by 8.8 percent in 2020. 11.1. BUDGET EXECUTION RATE 186. The score for this dimension is D. Performance level and evidence for scoring the dimension Table 16. Deviation of Actual Development 184. In two of the last three years actual Expenditure (excluding IFIs' projects) and development expenditure (excluding IFIs' projects) Some Other Expenditure Categories from deviated by more than 25 percent from the Original Budget107 originally budgeted amount. The annual variations of total development expenditures (excluding donor FY 2018 FY 2019 FY 2020 Development 3.0% -43.6% -44.5% expenditure Table 15. Total Budget and Actual Consumption -0.6% 3.2% 18.6% Development Expenditure106 (excluding IFIs' expenditure, projects) (UAH billion) of which: FY 2018 FY 2019 FY 2020 Salary -0.8% 1.6% 3.9% expenditures Budget 130.8 123.5 153.9 Utilities 0.3% -7.1% -25.2% Actual 134.7 69.6 85.4 and energy Deviation, % 3.0% -43.6% -44.5% expenditures Source: State budget laws for 2018, 2019 and Source: State budget laws for 2018, 2019 and 2020; state budget execution reports in the form 2020; state budget execution reports in the form on annex 3 to the state budgets for 2018, 2019, on annex 3 to the state budgets for 2018, 2019, and 2020 and 2020  106  Based on the cash budget and outturns.  107  Based on the cash budget and outturns. Chapter 3. Assessment by Indicator for PIM Functions 63 11.2. DONOR BUDGET EXECUTION RATE À Delays in the development of design and estimate documentation; Performance level and evidence for scoring the À Slow mobilization of contractors; dimension À Long-term selection of subprojects and final 187. In all of the last three years, actual IFI's beneficiaries at the stage of implementation of project development expenditure fell short of framework projects, etc. the originally budgeted amount by more than 25 À Lengthy IFIs tendering procedures; and percent. The annual variations were -53.9 percent in 2018, -66.0 percent in 2019, and -52.8 percent in 2020 À Delays in the entry into force of international (see Table 17). According to the Ministry of Finance,108 agreements. the low level of implementation of donor funded project development expenditure was caused by the 188. The score for this dimension is D. following factors: À Weak disbursement planning by the responsible project executors; PIM-12. Composition of Development À Quarantine measures related to the COVID-19 and Capital Expenditure Outturn pandemic in 2020; Compared to Adjusted Original Budget on a Commitment Basis Table 17. Development Expenditure through IFI-Financed Projects - Total Budget and RATIONALE Actual (UAH billion) 189. This indicator measures the extent to which FY 2018 FY 2019 FY 2020 changes during budget implementation are due to factors beyond the changes resulting from the Budget 5.8 19.3 20.2 deviation of the aggregate capital spending from Actual 2.7 6.6 9.5 budget (PIM-11). It considers whether slippages in implementation impact evenly across sectors, or Deviation, % -53.9% -66.0% -52.8% whether priority sectors are relatively well protected. In addition to providing an additional, more micro Source: State Budget for 2018-2020, Treasury level indication of budget credibility, this indicator budget execution reports for 2018-2020 provides useful information for exploring the detailed Summary of Scores and Performance Table PIM-12 Indicator/ Dimension Score Brief justification for score PIM-12. Composition of development C Scoring Method M1 and capital expenditure outturn compared to adjusted original budget on a commitment basis 12.1. Composite variance of capital C In one of three years, the variance in the composition of expenditures across MDAs development expenditures across agencies exceeded the overall deviation in development expenditures by more than 25 percentage points. The deviation was not significant in the other two years.  108  Reports on the implementation of the Public Debt Management Program for 2020, 2019 and 2018.: https://mof.gov.ua/storage/files/ЗВІТ по Програмі удб 2020 рік.pdf https://mof.gov.ua/storage/files/ЗВІТ по ПРОГРАМІ УДБ 2019(1).pdf https://mof.gov.ua/storage/files/ЗВІТ по Програмі удб 2018_FINAL_.pdf 64 Chapter 3. Assessment by Indicator for PIM Functions functioning of the PIM system. For instance, the PIM-13. Project Completion Time and MDAs with the greatest and the lowest divergence Cost Variances for Completed Projects between budget and outturn provide possible entry points for exploring the underlying reasons for their RATIONALE performance. 192. Cost and time overruns are key indicators of the health of a PIM system. They can arise 12.1. COMPOSITE VARIANCE OF CAPITAL from: poor design and planning of project costs EXPENDITURES ACROSS MDAs and construction schedules; poor implementation Performance level and evidence for scoring the (management, procurement, etc.); delayed approvals dimension or budget releases; corruption; external factors, such as market price increases or shortages of key inputs; 190. In one of three years, the variance in the or some combination of all of these factors. Careful composition of development expenditures review of the capacities and performance in all the across agencies exceeded the overall deviation stages of project preparation and implementation in development expenditures by more than 25 as revealed by the other PIM indicators should help percent.109 The average annual variations across expose common causes. Estimates of cost and time MDA's were 29.9 percent in 2018; 52.2 percent in overruns should be found in project completion 2019; 50.8 percent in 2020. These figures deviated reports (as in PIM-18). from overall deviations in development expenditures (PIM-11) by 26.9 percentage points in 2018, 8.6 percentage points in 2019 and 6.3 percent points 13.1. PROJECT COMPLETION TIME AND in 2020 (see Annexes 5 – 7). The administrative COST VARIANCES FOR COMPLETED breakdown based on the cash budget and outturns PROJECTS was used to calculate the variance in budget Performance level and evidence for scoring the composition. There is no particular pattern visible dimension in the deviations: no agencies can be identified with systematic deviations from the overall deviation that 193. The three-year weighted average110 of persist from one year to the next. (inflation adjusted) cost overruns for SSIPs, at 30.8 percent, was higher than the 30 percent threshold 191. The score for this dimension is C. for this indicator. During the assessed period, four state investment projects were completed in 2018 and 2019 and none in 2020 (Annex 8). The cost of the Summary of Scores and Performance Table PIM-13 Indicator/ Dimension Score Brief justification for score PIM-13. Project completion time D Scoring Method M1 and cost variances for completed projects 13.1. Project completion time and cost D The three-year weighted average of the cost overruns was variances for completed projects higher than 30 percent. The three-year weighted average of the time overruns was lower than 100 percent.  109  Development expenditures include, among others, capital expenditures.  110  Average percentage variance weighted by total inflation-adjusted project implementation costs of each completed project. Chapter 3. Assessment by Indicator for PIM Functions 65 project 'Restoration and adaptation of the Mariinsky overruns, which is questionable for two multi-years Palace on the Hrushevskoho Str., 5a, Kyiv' amounted projects.112 Accurate data for these two projects would to UAH 922.8 million (equivalent to USD 35.8 million). significantly decrease the time overruns. The cost of other projects ranged from UAH 12.7 million (equivalent to USD 0.5 million) to UAH 122.3 195. The score for this dimension is D. million (equivalent to USD 4.8 million). The final cost of the largest completed SSIP was 29.3 percent lower than the approved cost adjusted for inflation. PIM-14. Stock and Monitoring of While the smallest of the four completed SSIPs was Capital Expenditure Arrears overspent compared to the planned projects cost. 194. The three-year weighted average111 of the RATIONALE time overruns, at 81.3 percent for SSIPs, 32.7 for 196. Overdue obligations to pay suppliers or SSIPs and non-SSIPs combined, was lower than contractors for capital expenditures could indicate the 100 percent threshold, however, there are a variety of underlying weaknesses in PIM or the doubts about the reliability of some figures. Of wider PFM system. Payment arrears could indicate the four completed SSIPs, one project ran over by poor integration of planned capital spending in 10 years; two projects ran over by one year; and one the budget, weak in-year cash forecasting, a lack of project was completed a year early. Funding for the commitment controls, or unforeseen lack of cash SSIP, 'Restoration and adaptation of the Mariinsky to meet the government's obligations. Supply and Palace at 5a Hrushevskoho Street', began before 2015, construction contracts normally establish a period when the preparation and selection system for state for verification and approval of payment after the investment projects was introduced. Accordingly, supplier/contractor submits the invoice, and a period there is no reliable data on the initial cost and timing within which the payment should be made. A default of this project. The assessment is therefore based on rule to decide when a payment is in arrears (if no the data contained in the project proposal submitted explicit due dates are set or recorded) should be that in 2015 under the new system, at which stage it a payment is in arrears 30 days after the procuring was already an ongoing project. There were nine agency authorizes the payment or 60 days after completed non-SSIPs which show almost no time receipt of the invoice. Summary of Scores and Performance Table PIM-14 Indicator/ Dimension Score Brief justification for score PIM-14. Stock and A Scoring Method M1 monitoring of capital expenditure arrears 14.1. Relative size and A Arrears ranged from 0.5 percent to 0.7 percent of total expenditures and growth in arrears have been growing no faster than the real economic growth rate over past two years. 14.2. Availability of A Data on expenditure arrears, including capital expenditures, is formed data on a monthly and annual basis. The Treasury draws up and submits such monthly reports to the Parliament, President, Accounting Chamber, Cabinet and Ministry of Finance (no later than the first day of the second month following the reporting period.  111  Average percentage variance weighted by implementation time of each completed project.  112  'Construction of mine No. 10 'Novovolynskaya" was started in 1989, while Ministry of Energy identified the original date for the project competition as 2020. Similarly, 'Construction (acquisition) of the territorial administration of the State Judicial Administration in the Chernivtsi oblast (Storozhynetsky district)' was started in 2008, while the State Judicial Administration identified the original date of project completion as 2019. 66 Chapter 3. Assessment by Indicator for PIM Functions 14.1. RELATIVE SIZE AND GROWTH IN submit reports, including the reports on expenditure ARREARS arrears, using an automated e-reporting system. The Treasury checks the data in the submitted reports for Performance level and evidence for scoring the consistency with similar data in the accounting records dimension of the Treasury. The authorization of the Treasury confirms that the spending units and recipients of 197. The stock of capital expenditure arrears is the budget funds compiled the budget reporting small compared to the size of the capital budget in accordance with relevant requirements, and the and has not been growing in an unsustainable reporting is subject to consolidation in the summary way. In 2018 and 2019, capital expenditure arrears reporting of the higher-level spending units. amounted to 1.1 percent of state budget capital expenditures and in 2020 to 0.7 percent (see 200. The Treasury draws up and submits monthly Table 18). Arrears are accounted for at the level of and annual reports on expenditure arrears. These individual spending units, budget programs and the reports go to the VRU, the President, the ACU, CMU economic classification of expenditure. Arrears have and MoF. These monthly reports must be submitted been growing no faster than the real economic growth no later than on the first day of the second month rate over the past two years. following the reporting period.114 198. The score for the dimension is A. 201. The reports allow the age of arrears to be tracked. The reporting presents the accounts payable as of the beginning of the reporting year, as of the end 14.2. AVAILABILITY OF DATA of the reporting period, in which the overdue arrears have been allocated, and the commitments awaiting Performance level and evidence for scoring the payment. In addition, the reporting shows the arrears dimension written off since the beginning of the reporting year. 199. Expenditure arrears are calculated monthly, 202. The score for the dimension is A. quarterly and annually113 by all budget institutions and recipients of budgetary funds. Spending units and recipients of budget funds prepare and Table 18. Stock of Capital Expenditure Arrears (UAH million) 2018 2019 2020 i) Stock of capital expenditures arrears 790.6 814.5 700.2 ii) Total actual capital expenditure 69,805.5 76,188.6 94,926.8 Ratio (i)/(ii), % 1.1% 1.1% 0.7% Change (i) compared to the previous year, % - 3.0% -14.0% Real economic growth rate over past two years, % - 3.2% -4.0% Source: Treasury budget execution reports for 2018-2020  113  In accordance with the Order of the Ministry of Finance of Ukraine No. 44 dated 24 January 2012.  114  Part two of Article 59 of the Budget Code. Chapter 3. Assessment by Indicator for PIM Functions 67 PIM Function 5. delivered in practice through open public access to full information on individual procurement exercises Implementation from beginning to end. Adequate recourse is assured by the existence and functioning of a complaints PIM-15. Procurement procedure that allows for submission and resolution of complaints in a fair, transparent, independent and RATIONALE timely manner, with the possibility of taking appeals to higher external authority. 203. Procurement is at the heart of PIM because of the widespread importance of procurement as a means of implementing capital spending projects, 15.1. TRANSPARENCY, but it is often a problematic area. Competition COMPREHENSIVENESS AND and comprehensive coverage are key to a public COMPETITION IN THE LEGAL AND procurement system that delivers economy in the use REGULATORY FRAMEWORK of public financial resources and should accordingly be embodied in the legal and regulatory framework. It is Performance level and evidence for scoring the one thing for the principle of competitive procurement dimension to be embodied in the legal framework, but competition must also be achieved in practice, making 205. The legal and regulatory framework non-competitive procurement the exception rather establishes a transparent, comprehensive, than the rule. Ideally, the principle of comprehensive and competitive public procurement system in coverage should also extend to donor projects which Ukraine. The MoE is the authorized body with the should use national public procurement procedures mandate to regulate and implement the state policy when these are of a high enough standard. in the field of procurement. Amendments to the 2015 Law on Public Procurement (PPL) that came 204. Transparency and recourse are also critical to into effect in April 2020 introduced a number of good procurement processes. Transparency must be significant changes, including lowering of thresholds embedded in the legal and regulatory framework and for mandatory simplified procurement (which is a Summary of Scores and Performance Table PIM-15 Indicator/ Dimension Score Brief justification for score PIM-15. Procurement B+ Scoring Method M2 15.1. Transparency, comprehensiveness, A The legal and regulatory framework provides the and competition in the legal and necessary basis for transparent, comprehensive, and regulatory framework competitive procurement. 15.2. Use of competitive procurement A All use of non-competitive procurement methods is methods fully justified within the law. 15.3. Share of programmable aid D Except in rare cases, donors do not use national funds subject to national procurement procurement procedures. procedures 15.4. Public access to complete, reliable A All procurement information is freely available to the and timely procurement information public through electronic means. 15.5. Existence of an independent A The appeals procedure is independent, transparent, administrative procurement complaints and efficient. institutions and mechanisms 68 Chapter 3. Assessment by Indicator for PIM Functions competitive method), introducing the possibility 207. Procurement of works resulted in successful for bidders to correct mistakes in submitted bids, completion of 212,762 procedures or slightly disqualification of abnormally low bids, Prozorro over 6 percent of the total number completed Market (e-catalogues), and the possibility returning procurements. Open competition, simplified the fee for filing a complaint to the Antimonopoly procurement, and reporting procedure for concluded Committee of Ukraine. The PPL supporting regulations contracts (publication of report) were the most have enshrined standards of public procurement widely used methods for works in 2020, for packages that are in line with those in the European Union both below and above the thresholds. The largest and satisfy all six of the assessment criteria for this number of bids received in 2020 was also observed dimension, meaning that the public procurement in procurement of civil works, reaching 38 bids for a system: single procedure. i. Is organized hierarchically, with clearly established 208. The score for this dimension is A. precedence; Is freely and easily accessible to the public; ii. 15.2. USE OF COMPETITIVE iii. Applies to all procurement undertaken using PROCUREMENT METHODS government funds; Performance level and evidence for scoring the iv. Embodies open competitive procurement as the dimension default method of procurement115 and includes clear definitions of the situations in which other methods 209. All contracts that are awarded by methods can be used and how this is to be justified; other than open competitive procurement are justified in accordance with the legal and v. Provides for public access to all of the following regulatory requirements. As indicated above, the procurement information: government law clearly stipulates the use of open competitive procurement plans; bidding opportunities, contract procedures and the conditions for exceptions. awards, and data on resolution of procurement Statistics from the e-procurement system for 2020 complaints; and indicate that non-competitive methods were used in vi. Provides for an independent administrative review 92 percent of procurements by number,116 but only process for handling procurement complaints. 32 percent by value.117 However, these figures are misleading because many of these procurements were 206. The introduction of e-procurement in 2016 small purchases below the threshold for competitive has significantly increased the ease of access to procurement that are defined in the law and did not public procurement. E-procurement has contributed relate to capital expenditure. In 2020 86 percent of to the transparency of the procurement process, procurements in the e-procurement system fell below expanded business opportunities for participation in the threshold. The degree of competition is indicated public procurement and allowed both regulators and by the number of tenders per procurement, which the public to monitor the public procurement process. has averaged 2.3 between 2016 and 2020. The trend Since the electronic procurement system, Prozorro, is upward and in 2020 the average number of bids for was launched, the number of purchases has increased all procurement methods increased by 25 percent. from 900 thousand in 2017 to almost 4 million The number of bids submitted in open competitions purchases in 2020 and, accordingly, the total value of averaged 2.8 in 2020. contracts signed through the system has risen from UAH 472 billion to UAH 676 billion in the same period.  115  All procurements with cost estimate equal or above UAH 50.000,00 UAH (Ukrainian hryvnia), approximately US$ 1,880, shall use competitive procedures by law. For purchases below the UAH threshold, the law still obliges the purchaser to either publish a report on concluded contract, or conduct a simplified (competitive) procedure, or purchase from e-catalogues administered by the Central Procurement Agency.  116  211,928 out of 3,545,097 - MoE data  117  UAH 492.0 billion out of UAH 728.8 billion Chapter 3. Assessment by Indicator for PIM Functions 69 210. Reporting on concluded contracts118 is the format and are carried out using the electronic most widely used non-competitive procurement procurement system. For procurements not using method for works. It totalled 77.3 percent by number the electronic procurement system, a report on the or 164,416 of the overall number of both below and procurement contract must be published in the above threshold procedures. Open competition (with/ electronic procurement system. This rule ensures without publication in English language) represented free access to information on procurement of all 13,834 cases or 6.5 percent, while simplified items, including those with a value that is below the procurement reached 11.3 percent, or 23,943 cases. established thresholds for competitive procurement. Another widespread procurement method for works In addition, the Ministry of Economy publishes annual was the negotiation procedure, held 2,493 times or 1.2 reports on its official website which include the public percent of cases in 2020. procurement system operations analysis. In relation to works contracts, the public can access information on 211. The score for this dimension is A. procurement plans (as published by individual public bodies), bidding opportunities, contract awards, and information on resolution of procurement complaints. 15.3. SHARE OF PROGRAMMABLE AID FUNDS SUBJECT TO NATIONAL 215. The score for this dimension is A. PROCUREMENT PROCEDURES Performance level and evidence for scoring the 15.5. EXISTENCE OF AN INDEPENDENT dimension ADMINISTRATIVE PROCUREMENT COMPLAINTS INSTITUTIONS AND 212. Most spending under IFI-supported projects is MECHANISMS done using the IFIs' procurement systems and in accordance with their rules and procedures. While Performance level and evidence for scoring the exact figures for the share of programmable aid funds dimension are hard to come by, the amounts involved are small and certainly below 40 percent of total programmable 216. Participants to procurement procedures can aid funds. In 2020, the e-procurement system shows submit a complaint to an independent appeal 39 IFI-funded procurement procedures with a total body, the Antimonopoly Committee of Ukraine. value of UAH 1.31 billion compared to a total figure of The Antimonopoly Committee is the complaint review UAH 728.8 billion managed in the system. authority exercising, among other things, control in the field of public procurement within the scope of 213. The score for this dimension is D. the powers vested in it by the Public Procurement Law (PPL) and the Constitution. The PPL was amended in 2020 to make it more efficient by reducing frivolous 15.4. PUBLIC ACCESS TO COMPLETE, appeals and unnecessary delays. The appeal must RELIABLE AND TIMELY PROCUREMENT be filed exclusively through the Prozorro electronic INFORMATION procurement system. The opportunity exists to appeal Performance level and evidence for scoring the the procurement procedure for individual lots. An dimension appellant may not withdraw his appeal if a registration card has been generated. A fee is charged for filing a 214. All information on all forms of public complaint; the fee is set in proportion to the expected procurement using public funds is stored in the value of the contract award (prior to 2020 a fixed fee procurement system and is publicly accessible. had been charged) and at a level that is not intended All purchases in value greater than or equal to UAH to dissuade serious appeals. The fee is refunded if the 50,000 have been transferred from paper to electronic appeal is successful or partially successful.  118  Reporting on concluded contracts is applied to the procurement of goods, works and services, with value not over UAH 50 thousand without using the electronic procurement system. 70 Chapter 3. Assessment by Indicator for PIM Functions 217. The appeals procedure has all the elements 221. New cost-estimating norms for construction required to ensure an independent, accessible, in Ukraine (also known as "DBN") were and efficient appeals process. The appeals approved by Order No. 281 (November 1, 2021) procedure: of the Ministry of Communities and Territorial Development. The order introduced two important i. Is comprised of experienced professionals, changes: i) wider use of external consultant familiar with the legal framework for procurement, engineering services to supervise public investment and includes members drawn from the private projects in the field of construction, bringing Ukraine sector, civil society, and government; closer to FIDIC standards; and ii) basic rules for the ii. Is not involved in any capacity in procurement application of estimation norms and regulations transactions or in the process leading to contract on pricing in construction for determining the award decisions; cost of new construction, reconstruction, capital iii. Does not charge fees that prohibit access by repair of buildings, buildings and structures of any concerned parties; purpose, their complexes and parts, linear objects of engineering and transport infrastructure, as well iv. Follows processes for submission and resolution as the restoration of monuments architecture and of complaints that are clearly defined and publicly urban planning. These requirements are mandatory available; for construction financed out of state funds, v. Exercises the authority to suspend the covering budget funds, funds of state and communal procurement process; enterprises, institutions, and organizations, and loans provided under state guarantees. The application vi. Issues decisions within the 10 business-day of the order in the construction of objects with the timeframe specified in the rules/regulations; involvement of other sources of financing is stipulated vii. Issues decisions that are binding on all parties, by the respective contract. while not precluding subsequent access to the courts to appeal a decision. PIM-16. Project implementation 218. According to the Antimonopoly Committee of Ukraine, in 2020, 12,675 appeals were received. management Of these appeals, 11,463 were accepted for consideration. As a result of consideration of appeals, RATIONALE 24,270 decisions were made leading to: 3,808 appeals 222. This indicator assesses project being denied, 6,834 being satisfied in full or in part, implementation management, in particular the and 1,145 being terminated. availability and content of guidelines, clarity of accountability, the availability and content of 219. The score for this dimension is A. implementation plans, and the system for total project cost management. The aspects of project Recent And Ongoing Reforms management listed above are factors in successful implementation of projects, within approved budgets 220. The ongoing war forces the government and timelines. Assessment covers the current situation to adopt quickly, and changes are necessarily and seeks to identify whether there are clear rules for adopted very often. For example, the Resolution of accountability for project management between and the Cabinet of Ministers of Ukraine dated February 28, within MDAs. In addition, the indicator stresses the 2022, No. 169 "On defense and public procurement of need for an effective use of a system for managing goods, works and services during the war Martial Law" total project costs. has been already amended six times. Chapter 3. Assessment by Indicator for PIM Functions 71 Summary of Scores and Performance Table PIM-16 Indicator/ Dimension Score Brief justification for score PIM-16. Project C Scoring Method M2 implementation management 16.1. Guidelines C The existing legislative framework provides a set of rules that must be on project followed during project implementation; however, they do not represent implementation consolidated guidelines and do not provide any advice on effective project implementation from the point of view of management. 16.2. Clear C Management accountability between agencies is clear and implementation accountability and plans are comprehensive. However, management accountability within implementation plans agencies is not completely clear, and there is no unified system for assignment of persons responsible for project implementation. 16.3. Total project cost C There is capacity to track total project costs and it is used actively to management system control costs of SSIPs; however, the system is manual and there is no dedicated electronic system for assuring thorough control of projects costs from beginning to end. There is no standardized mechanism to control the total project costs of non-SSIPs; however. It is possible to do this following the regulation for construction. 16.1. GUIDELINES ON PROJECT advice on effective management itself. Thus, there IMPLEMENTATION are two main issues in this area: lack of consolidated guidance and lack of management guidance in the Performance level and evidence for scoring the existing legislation. Good practice would involve a dimension unified 'handbook' that pulls together the rules and procedures set out in different places and provides 223. There are no consolidated guidelines for best practice advice on project management. project implementation management, either at the level of central government or key sector 224. The score for this dimension is C. MDAs. Currently project implementing agencies refer to a wide range of legislative acts that set the rules governing project implementation in different places. 16.2. CLEAR ACCOUNTABILITY AND The regulatory framework for public investment IMPLEMENTATION PLANS management sets out lines of accountability for management and reporting.119 Financial management Performance level and evidence for scoring the of projects is governed by the regulations for state dimension budget management, which require reporting on state budget funds to MoF.120 The legislative framework for 225. Management accountability within procurement121 and construction122 provides rules MDAs is not clear enough: some agencies on procedures and defines the roles of the agencies have departments responsible for project involved. Together, these legal instruments set out the implementation control and others do not. Partly rules and target compliance, but they do not provide reflecting the lack of unified guidance (see PIM-16.1),  119  Resolution No. 571, Order on Monitoring  120  Budget code, Regulation on passports of budget programs  121  The Law on procurement  122  Range of laws and legislative acts provide the rules for construction. 72 Chapter 3. Assessment by Indicator for PIM Functions there is no unified system for assignment of persons year following inclusion of the project in the state directly responsible for project implementation in budget, including roll forward of the monthly plan by line ministries and implementing agencies. Sector another year. MDAs may issue internal orders that establish responsibilities and some ad hoc rules for project 227. The score for this dimension is C. implementation, but the situation differs in each case. For example, the Ministry of Ecology and Natural 16.3. TOTAL PROJECT COST Resources has issued an internal order, which defines MANAGEMENT SYSTEM responsibilities for budget programs (including project-programs) and there is a designated unit for Performance level and evidence for scoring the coordination of projects and financing plans in which dimension one person is responsible for projects. In contrast, in the Ministry of Energy, each sector department 228. There is capacity to track total project costs is responsible for projects falling within its area of over time against the total approved budget, and competence, but there are no rules for establishing this is actively used to control expenditure of SSIPs responsibilities for project implementation at the level following Resolution No. 571. Total project costs are of positions within the organization. The Ministry of controlled by the MoE on behalf of the Inter-Agency Healthcare has a sector for assistance to investment Commission at least once a year using a paper-based projects implementation, but lines of accountability system. Implementing agencies prepare and line are defined by the public investment management ministries submit updated implementation plans and legislation, which say nothing about organizational monitoring reports, in which the information on the structures and management accountability within total project cost and amount financed is provided. organizations. Control of total project costs is performed primarily to comply with the requirement for a project review if 226. Project implementing agencies following increases in total costs exceed a specified threshold Resolution No. 571 have to prepare a (see PIM-19). Some key sector MDAs keep their own comprehensive implementation plan, but it records on total project costs, which allows them to does not contain anything on accountability control changes, but this is not systematic. and distribution of responsibilities, and there is no similar planning document for projects 229. While total costs of SSIPs are actively funded as budget programs outside this process. controlled, there is no mechanism to control The implementation plan forms part of the project total project costs of non-SSIPs. Since there is implementation and financing plan, which is Part III of no requirement to review non-SSIPs (which are not the project proposal. The plan contains information subject to Resolution No. 571), the incentives for on the project timeline, expenditure by aggregated monitoring their total project costs are weaker. The components, the procurement plan, and cash flow only instrument restricting changes in project costs forecast. The project activities and cost plan show is the requirement to review and get approvals the funds needed for implementation, for example, from supervising line ministries for the technical for construction or purchase of equipment, with construction design and plan in case of significant specification by year. Moreover, implementation plans changes. The design documents contain the cost contain information on the planned physical progress, sheet of the project, which needs to be re-approved, showing the components of the project, their cost, in case of changes. The amended project design must and the expected year of completion. For example, go through expertise by an organization authorized a project for the reconstruction of a hospital that by the Ministry for Communities and Territories includes works on several buildings will show the plan Development. This is more of an administrative for each separate building specifying construction exercise for construction objectives than a process for works and purchase of equipment. The plan is cost control. only detailed by month for the first year; financial and physical indicators are by year for subsequent 230. There is no dedicated electronic system for years the project implementation period. The tracking the costs of projects over their lifetimes implementation plan, as contained in the originally within MoE. There is a non-electronic database of approved project proposal, must be updated every projects that includes data on total project cost and Chapter 3. Assessment by Indicator for PIM Functions 73 cumulative disbursements to date. The database PIM-17. Control, Monitoring and includes only projects following Resolution No. 571 Reporting: Physical and Financial or being monitored by the Inter-Agency Commission and is updated at least once a year. At the time of Milestones assessment, the portfolio in the database consisted of only 67 projects, which means that it can be managed RATIONALE manually. However, every year the portfolio grows 232. This indicator covers issues of control, which points to the eventual necessity of introducing monitoring, and reporting on progress of public an automated system. Some key sector MDAs have investment projects. It focuses on content and their own databases allowing them to control changes frequency of financial and performance reporting, in total project cost. For example, at the Ministry of and analyzes the classification of expenditure and Health, a hand-written journal was introduced in 2015 milestones in the reports. Particular attention is for registering projects selected by the Inter-Agency paid to the quality of information provided in the Commission and recording funding received against monitoring and other reports that allow comparison total project costs. Similar records have been kept in of estimates and actual performance of project Excel format since 2020. implementation. Within this indicator the application of national monitoring and reporting procedures 231. The score for this dimension is C. to donor funded projects is analyzed as well. The assessment covers the period of the last completed year (2020). Summary of Scores and Performance Table PIM-17 Indicator/ Dimension Score Brief justification for score PIM-17. Control, С Scoring Method M2 monitoring and reporting: physical and financial milestones 17.1. Scope and C There is a well-designed expenditure and performance reporting system coverage of for SSIPs. Reporting allows direct comparison with budget estimates and expenditure and relates to all individual SSIPs in the budget while non-SSIPs are hidden in performance reporting bigger and complex budget programs. Expenditures are reported based on payments and not commitments. The focus of reporting is on annual performance rather than accumulated progress. 17.2. Timeliness of D Monitoring reports were prepared half yearly for the last fiscal year. In reports 2021, quarterly monitoring has been introduced with corresponding changes in Resolution No. 571. 17.3. Use of national A All donor-funded investment projects follow national procedures procedures on donor established specifically for donor/technical assistance and IFI projects. funded projects or programs 17.4. Quality of B The reports provide accurate data, but some drawbacks exist, for example, information reporting weak justification of discrepancies. 74 Chapter 3. Assessment by Indicator for PIM Functions 17.1. SCOPE AND COVERAGE OF as it comes after the end of the budget year. A copy EXPENDITURE AND PERFORMANCE of the report on Budget Passport Program execution REPORTING must be provided to MoE as an attachment to the annual monitoring report and provides verification of Performance level and evidence for scoring the the expenditure figures in the latter. dimension 235. Implementing agencies for SSIPs also report 233. There is a well-designed expenditure and on progress when updating their financial and performance reporting system with an annual physical implementation plans for the coming perspective for SSIPs. The Order on Monitoring123 budget year. The updated plan contains a section that issued within the framework of Resolution No. 571 shows project progress, including cash expenditures, established a reporting and monitoring regime for activities planned and performed with a timeline SSIPs. Reports cover cash expenditures against plan, presented in the same way as in the project proposal, physical progress against key milestones124 identified implementation plan, and monitoring reports. The in the implementation plan125 and procurement advantage of this reporting is that it has a whole-of- activity, all on an annual basis only. Commitments project perspective, but its timing, after the budget are not reported. Reporting is upwards from the is approved, is not helpful for informing high-level implementing agency to the MoE via the responsible budgetary decisions. MDA. The reporting requirements also apply to projects considered and rejected by the Inter-Agency 236. There is no systematic performance reporting Commission, but then implemented using another system for individual non-SSIPs. If a non-SSIP is funding source. There are other complementary assigned its own Budget Passport Program, then layers of reporting for SSIPs (discussed below), annual progress reports are prepared that relate which have different purposes and provide different directly to the project, in the same way as SSIPs. perspectives. However, many non-SSIPs will be included in broader Budget Passport Programs, which include recurrent 234. All SSIPs are assigned their own Budget spending and, potentially, expenditure on other Passport Program and their performance is projects. In these cases, there is not always discernible reported annually to MoF, as part of the budgetary performance reporting on individual projects: performance reporting.126 Since Budget Passport performance indicators might be for other activities Programs are components of the budget, there is a within the budget program, but not for non-SSIPs. In direct correspondence between financial reporting addition, the MoE is not informed in which budget through this system and the budget. Budget Passport programs it will find non-SSIPs. Programs include performance indicators, including outputs, which should give some measure of physical 237. Treasury prepares in-year and end-of-year progress against plan. Reports are required to budget expenditures execution reports which explain deviations from plan. The Budget Passport cover all projects and programs and can be Program, and the reports on its implementation, must directly compared with the approved budget.127 correspond with the approved implementation plan in These reports contain, among others, reports on the terms of both expenditures and indicators of physical execution of financial indicators as presented in the performance. The drawback is that the reporting annexes to the annual State Budget Law. Reports thus perspective is annual, rather than whole-of-project, allow comparison with each budget program and and is not useful as a basis for in-year adjustments project presented in the approved budget in terms  123  Order of MoE No. 1785 on Monitoring of development (implementation) of public investment projects  124  Examples of milestones include completion of land acquisition, launch of tenders, finalization of construction of separate components of an asset, and purchase of equipment.  125  The Plan of Implementation and Financing and State of Project Implementation forms Part III of the Project Proposal approved by the Inter-Agency Commission and is updated annually (see PIM 16) as part of securing budget funds for the next year of implementation.  126  Order of MoF No. 1098 on Passports of budget programs  127  Articles 60 and 61 of the Budget Code. Chapter 3. Assessment by Indicator for PIM Functions 75 of 'consumption' and 'development' expenditures,128 241. Reporting on the implementation plan and the salary, and utilities. Expenditures on SSIPs and non- Passport of Budget Program is linked to budgeting SSIPs that have been assigned their own Budget and is less frequent than reporting requirements Passport Program can be compared directly with under the Order on Monitoring. The updated plan approved budget allocations; for non-SSIPs integrated of implementation and financing and state of project into broader programs it is not possible to make a implementation must be provided to MoE not later direct comparison. than 15 days after the Law on the Budget of Ukraine enters into force. The frequency of such reporting 238. Treasury reports provide complete depends on the changes in the law on the state information on expenditures at the payment budget. If there are no changes in the budget, the stage, but only information on unpaid reporting has to be performed once a year. Reports commitments, not all commitments. Unpaid on execution of Passports of Budget Programs are commitments are shown in arrears reports only. Total prepared once a year, by March 1, together with commitments are not shown beside the approved and annual reporting on state budget execution. executed expenditures, as presented in the approved budget. 242. The Treasury prepares reports on the execution of financial indicators as presented in 239. The score for this dimension is C. the budget quarterly and annually. The Treasury submits such reports to the VRE, CMU, ACU, and MoF, and publishes them on its website. 17.2. TIMELINESS OF REPORTS 243. The score for this dimension is D. Performance level and evidence for scoring the dimension Recent And Ongoing Reforms 240. Implementing agencies prepared monitoring reports semi-annually and annually for the last 244. Quarterly reporting was introduced in 2021 completed fiscal year, in line with the Order on through amendments to Resolution No. 571, Monitoring by MoE.129 Semi-annual reports were however the annual reporting requirement provided to MoE by line ministries no later than the remains unchanged. Although regular quarterly fifteenth day of the month following the reporting reporting was not established by law before 2021, period and by March 1 for annual reports. Line the MoE or the Inter-Agency Commission could still ministries are required by the legislation in place to demand submission of monitoring reports upon perform monthly monitoring of projects under their request at any time during a year. Implementing responsibility or that of their subordinated agencies; agencies now submit reports to MDAs and, following however, in practice, monthly monitoring is not review of these submissions, the MDAs then report to performed by all MDAs as required because each has MoE no later than the 15th day of the month following its own system for controlling project implementation. the reporting period. Annual reporting requirements Line ministries are expected to publish information on remains as before. The corresponding change has monitoring on their websites.130 not yet been introduced to the Order on Monitoring, which still requires half yearly and annual reporting.131 Monthly monitoring by line ministries remains unchanged.  128  The budget does not present current and capital expenditures, but consumption and development expenditures. Last of them include some current expenditures beside the capital expenditures.  129  Order of the Ministry of Economy No. 1785 as of 25.19.2016 on Approval of the Procedure for monitoring of development (implementation) public investment projects  130  Example of information disclosure by the line ministry (Ministry of Health)  131  The article on quarterly monitoring in Resolution No. 571 appeared with the last amendments as of 07.04.2021 and seems that corresponding changes have not been done in Order of the Ministry of Economy No. 1785 as of 25.19.2016 on Approval of the Procedure for monitoring of development (implementation) public investment projects (Order No. 1785) 76 Chapter 3. Assessment by Indicator for PIM Functions 17.3. USE OF NATIONAL PROCEDURES 247. Donor-funded projects use national ON DONOR FUNDED PROJECTS OR procedures for financial and performance PROGRAMS reporting in part of the budget execution reports and budget programs passports execution reports. Performance level and evidence for scoring the MDAs include all received donor funds in their dimension budgets as own revenues and relevant expenditures. IFI-financed projects are included in the budget as 245. Donor-funded projects or programs are separate programs. Thus, these figures are included monitored using designated national procedures. in the Treasury reports and performance achieved is The procedures are not the same as those for projects included in the budget programs passports. funded from the state budget. There is one set of procedures for grant-funded 'technical assistance' 248. The score for this dimension is A. projects132 and one set of IFI projects.133 Although labelled as applicable to technical assistance, the procedures for grant-funded projects also apply 17.4. QUALITY OF INFORMATION to investment in fixed assets. Currently, in Ukraine REPORTING there are more than 500 international 'technical Performance level and evidence for scoring the assistance projects', one fifth of which is related dimension to construction and infrastructure. Monitoring of technical assistance projects is carried out by the 249. The quality of reporting of factual information Secretariat of the CMU together with the beneficiary on financial and physical progress is good and and authorized representatives of the development there are no substantive concerns. The sampled partner134 (by agreement).135 The procedures require line ministries136 did not identify any special concerns half yearly and annual reporting. The form of the with the quality of reporting from implementing report requires provision of data on expenditures by agencies. The MoF and MoE did not identify any categories and specification of discrepancies between specific problems with regarding to reporting by plan and actuals. The CMU prepares consolidated MDAs, nor did internal audit units or the State Audit annual reports on international technical assistance Service identify any systemic issues in the quality of to Ukraine. In addition to national procedures for reporting. monitoring of such projects, each donor establishes its own rules for monitoring and control. 250. Although based on small sample of reports for SSIPs, there are some concerns about the 246. Monitoring of IFI-financed projects is carried more analytical part of reporting. Explanations for out by the MoF, reporting to the CMU and Office of deviations from plan appear to be weak or missing the President. The responsible implementing agency and remedies are not well explained or missing (see reports on both physical and financial performance to Box 7). the MoF on a quarterly basis (by the 15th of the month following) and on an annual basis (by January 15 of the 251. The score for this dimension is B. next year). Reports on the status of loans and related transactions are required on a monthly basis. The MoF reports quarterly to the CMU on the status of loans and semi-annually on implementation progress. An annual performance report is also submitted.  132  Resolution No. 153 of the CMU on creation of a single system for attraction, use, and monitoring of international technical assistance.  133  Resolution No. 70 of the CMU preparation, implementation, carrying out of monitoring and completion of implementation of projects of economic and social development of Ukraine supported by the international financial organizations.  134  Development partner is a donor  135  Before 2020 this function was managed by MoE, the same department that dealt with public investment projects, but still procedure for PIP and donor projects were different  136  Ministries of Energy, Health and Ecology Chapter 3. Assessment by Indicator for PIM Functions 77 Examples of Analytical Weaknesses in Reporting BOX 7 Example 1: Children's Example 2: National Cancer Hospital137 Institute138 The agency implementing the OKHMATDYT While the structure of the monitoring report project (construction of a hospital for children) requires provision of reasons for discrepancies in its report for 2018 states the following and measures to eliminate them for each reason for discrepancies in all components of component of a plan, in many cases, this a procurement plan: 'The Passport of a budget requirement is disregarded and general reasons Program' for 2018 was approved by a joint Order for all components are provided (OKHMATDYT, of MoF and Ministry of Healthcare No. 508/373 National Cancer Institute (Report for 2018). as of 20.03.2018. On 27.11.2018 changes were made in the passport. This is an explanation of discrepancies for about 8 components, Example 3: Restoration of Lviv moreover, it does not provide clear reasons for University139 discrepancies. In the report on the project for Restoration of In the section for 'measures taken to eliminate National Lviv University, there are discrepancies, discrepancies' the agency just lists contracts but no explanations. concluded in 2018. PIM-18. Project Handover, Asset Registration, and Completion Review handover mechanisms, as well as asset registration. Internal and external completion reviews should be RATIONALE assessed within this indicator for the last three fiscal years. The indicator shows how the project handover 252. This indicator assesses the processes is arranged, how the agencies assure availability of necessary for the completion of a project and funding and personnel for facility operation, and what drawing conclusions on effectiveness of its kind of analysis they conduct for learning lessons for implementation. It analyzes project completion and future projects. Summary of Scores and Performance Table PIM-18 Indicator/ Dimension Score Brief justification for score PIM-18. Project handover, D+ Scoring Method M2 asset registration, and completion review 18.1. Formal project C A systematic project completion and handover mechanism exists, completion and handover but verification of the adequacy of funding and organizational mechanism arrangements for operations and maintenance is not performed.  137  https://moz.gov.ua/uploads/2/10686-info_20190101_ohmadet.pdf  138  https://moz.gov.ua/uploads/2/10687-info_20190101_2_nciukraine.pdf  139  https://mon.gov.ua/storage/app/media/innovatsii-transfer-tehnologiy/2021/02/11/Richnyy%20zvit%20DIP%20 Restavratsiya%20korpusu%20Lvivskoho%20universytetu%20Franka.pdf 78 Chapter 3. Assessment by Indicator for PIM Functions Indicator/ Dimension Score Brief justification for score 18.2. Asset registers B There are two registers, and the law obliges all assets to be registered in both. Most valuable information is recorded, but there is no information on the maintenance history and current condition. 18.3. Policy and guidance on D There is no internal completion review. internal completion review 18.4. Existence of external D There is no external completion review, but some elements of this completion review process exist for individual projects on an ad hoc basis. 18.5. Proportion of projects D No fully nationally funded large projects are subject to completion subject to completion reports reporting, while all donor funded large projects are. by country or donor partner 18.1. FORMAL PROJECT COMPLETION AND 254. To obtain a certificate for putting an asset into HANDOVER MECHANISM operation, the operating entity has to prepare the 'act' of the asset's preparedness for operation. The Performance level and evidence for scoring the act is prepared after full completion of construction dimension and includes a technical inventory together with the creation of a 'technical passport'. It must be signed 253. A formal project completion and handover by an implementing agency, general contractor, mechanism exists within the construction sector subcontractors, general designers, and technical system, but it is not imbedded in the PIM system. supervisors. The documents are submitted to DIAM The regulation for the construction sector is applicable (State inspection for Architecture and Urban Planning), a to all types of projects, regardless of the sources of new agency established in September 2021. DIAM verifies financing and appraisal procedures. Once a fixed asset documents for compliance with construction norms and is constructed, permission for putting it into operation standards and visits the site, taking photos and videos has to be issued. Different procedures for obtaining for the record. Earlier the function was performed by permission apply to different types of assets according the State Construction and Architectural Agency (DABI) to defined 'classes of consequences' resulting from which has now been dissolved. After verification of failure of the building or structure. The definition of documents, the agency takes a decision on issuance of 'consequences' extends to economic consequences, the certificate that gives the right to put the asset into so the focus of the process goes beyond safety. For operation, arrange ownership rights, and conclude simple assets where the impact of failure would be contracts for connecting municipal infrastructure. The relatively small and localized (1st class of consequences certificate must be issued within 10 days after the asset or CC1), it is enough to register the declaration of manager has submitted all the necessary documents. the completion of the construction by paper or electronic means, with no physical verification by an 255. The project completion and handover mechanism independent body. For more complicated structures described above does not include verification with wider consequences (2nd and 3rd classes of of the adequacy of funding and organizational consequences or CC2 and CC3, respectively), the asset capacities for operations and maintenance. This owner or manager of construction goes through an is the responsibility of implementing agencies and independent certification process which requires line ministries. They must assure operation and thorough checks on the newly created asset.140 Only maintenance by applying for funds from the state after registration of the declaration (CC1) or issuance budget and by launching a competition for hiring of the certificate (CC2 and CC3), can handover be personnel necessary for the asset operation. performed. Most public investment projects will need to follow the certification route, with the far easier self- 256. The score for this dimension is C. declaration applying only to minor works.  140  Resolution of CMU as of 13 April 2011 No. 461 on Issues of putting constructed assets into operation. Chapter 3. Assessment by Indicator for PIM Functions 79 18.2. ASSET REGISTERS SSIPs as required by the Order on Monitoring only has an annual perspective, and the final monitoring Performance level and evidence for scoring the report at the time of project completion focuses only dimension on the last year of implementation rather than the whole implementation period. None of the agencies 257. The asset register for the construction sector interviewed reported the existence of any form of includes information on assets at different stages basic completion review process, involving analysis of construction, from provision of permission of divergences from plan during implementation and for construction to putting into operation. The lesson learning. register is maintained and administered by DIAM. All construction assets, whether public or private, 261. The score for this dimension is D. are included in the construction asset register. The register includes information on the owner of the asset, construction company, design company, 18.4. EXISTENCE OF EXTERNAL documents for land usage, class of consequences of COMPLETION REVIEWS asset failure, and supervising consultant.141 Performance level and evidence for scoring the 258. There is a register of state property that dimension includes information on public legal entities and public assets. The register is managed by the Fund 262. There is no well-established procedure for of State Property. The information in the register is regular and systematic external completion updated quarterly and public agencies must provide reviews, except for IFI's projects. The State relevant information to the Fund. The Fund must Audit Service performs audits of IFI's investment maintain the register, including update of software projects, which are close to completion, analyzing and provision of the information from the register the results of their implementation and providing upon request.142 In the part about managing public recommendations. The same practice may be used for fixed assets, the register contains information on the completion review of SSIP, but it is not mandatory location of the asset, its owner, current book value, and no evidence of such reviews has been found. size/space, land plot, and the information on whether At the same time, bodies of the State Audit Service an asset is leased, concessioned, pledged, or not analyze efficiency of the use of assets created as a included in statutory capital of enterprises.143 There result of implementation of SSIP and non-SSIP projects is no information on asset age and condition. The within state financial control (inspection, state financial purpose of the register is to provide information for audit), but these audits are focused on operation only, monitoring the effective use of state-owned property not on the individual projects and analysis of results of and facilities and for ensuring the implementation of implementation. In some cases, a supervisory board of management decisions. trustees may be appointed, which can control project 259. The score for this dimension is B. implementation and assess the project completion results. Such a board functions for the ongoing SSIP 'construction of a hospital for children, OKHMATDYT'. 18.3. POLICY AND GUIDANCE ON The supervisory board of trustees is provided for in INTERNAL COMPLETION REVIEWS the articles of association, and it has authority to take Performance level and evidence for scoring the part in the monitoring and control of project progress, dimension including completion review. 260. There is no established procedure or 263. The score for this dimension is D. supporting guidance for regular and systematic internal completion reviews for public investment projects, either SSIP or non-SSIP. Monitoring of  141  Construction register: https://dabi.gov.ua/declarate/list.php?sort=num_re&order=ASC; https://data.gov.ua/dataset/e626418b-8403-4afa-bf9c-55581cf16f96  142  Resolution of CMU as of 14 April 2004 No. 467 on single register of state property  143  Structure of the Single register of state property 80 Chapter 3. Assessment by Indicator for PIM Functions 18.5. PROPORTION OF PROJECTS 265. The score for this dimension is D. SUBJECT TO COMPLETION REPORTS BY COUNTRY OR DONOR PARTNER Performance level and evidence for scoring the PIM Function 6. Adjustment dimension PIM-19. Project Adjustment 264. While there are no completion reviews of large, fully nationally funded projects, there is RATIONALE a strong system for completion review of donor projects. According to the legislative framework for 266. This indicator assesses project adjustment technical assistance projects (which includes major and budget reallocation mechanisms and their construction projects, as already stated), a completion practical application. It aims at identification monitoring report must be provided to the Secretariat of rules and regulations for project adjustment, of the CMU. This report requires provision of data on including availability of triggers for project review and planned and actual results of a project, assessment budget reallocation. The processes for adjustment of project results by beneficiary, and issues or of contracts and project design documents are proposals. 144 Projects financed by IFIs, in addition analyzed under this indicator as well. Assessment to being audited by the State Audit Service upon must show how monitoring contributes to efficient completion, must be analyzed after completion by the budget reallocation, what happens when there are line ministry or implementing agency.145 The project excessive deviations from plan, whether this area is completion report must be prepared within 2 months appropriately regulated, and what space for maneuver after all project payments have been completed. The project implementing agencies have during project report is submitted to the CMU for approval. After implementation. The indicator seeks to find out approval of the report, the implementing agency whether existing regulation for project review has informs VRU about the economic and/or socio- resulted in significant changes in public investment economic effects of the project. The report does not projects in the past 3 years. require ratification by the VRU. Summary of Scores and Performance Table PIM-19 Indicator/ Dimension Score Brief justification for score PIM-19. Project C+ Scoring Method M1 adjustment 19.1. Clear authority A Line ministries and implementing agencies can adjust contracts for managing and designs according to the legislative framework for the area of adjustments for procurement and the construction sector. While there is a transparent contracts and PPPs 146 provision of authority for adjustments to contracts, rules for adjustments to design documents are not specific enough. 19.2. Active monitoring B There is straightforward evidence of systematic and active oversight of of progress and SSIPs by senior management in implementing agencies, by supervising line budget reallocation ministries, and by central agencies. The Inter-Agency Commission typically procedures and reallocates funds from poor performing SSIPs to better performing ones actions within a year. This system is not, however, applicable to non-SSIPs.  144  Resolution No. 153 On creation of a unified system for international technical assistance https://zakon.rada.gov.ua/laws/show/153-2002-%D0%BF#n195  145  Resolution No. 70  146  PPPs are dealt with in a separate chapter. Chapter 3. Assessment by Indicator for PIM Functions 81 Indicator/ Dimension Score Brief justification for score 19.3. Mechanism С The policy provides the triggers according to which the original business to trigger review of case of SSIPs must be reviewed, but some of them are not fully explained project justification in the legislation and can be interpreted in diverse ways. The Inter-Agency Commission has the right to request business case review under a wide set of conditions, which are not defined as triggers. Reviews performed during the last three years, have not resulted in any problem projects being significantly re-designed, scaled down, or cancelled. There is no mechanism to trigger review of non-SSIPs. 19.1. CLEAR AUTHORITY FOR MANAGING /project implementing agency and contractor) and can be ADJUSTMENTS FOR CONTRACTS AND related to changes in the required quantity or quality of PPPs goods, changes in market prices or the macroeconomic environment, extension of the contract term or other Performance level and evidence for scoring the changes necessary for the full execution of the contract. dimension 269. The legislation sets out rules for adjustments 267. Line ministries and implementing agencies are to project construction plans and designs but provided with the authority to manage adjustments does not specify exact limits within which the for procurement contracts, project design, and implementing agency/line ministry can make construction plans within specified limits. The changes with no approvals. The project design legislative framework in the public procurement and consists of several stages150 that result in various types construction sectors provides rules for adjustments of documents151 that must pass through expertise that, within limits, do not require approvals by and approvals by various authorities.152 The final other agencies. The legislation for procurement and project design document, which is called 'working construction applies to all projects regardless of the documentation'153 and contains the construction plan process of selection or funding. In the case of SSIPs, with drawings, costs, and specification of equipment the information on adjustments must be provided to and materials, may be changed in the process of the Inter-Agency Commission, either for information construction with no expertise and approval by other or for re-consideration of project financing, in case agencies, except for the project design developer. specific triggers are breached. However, if the implementing agency initiates adjustments that will result in changes of any of 268. Adjustments for contracts for implementation the design documents preceding the final stage of of public investment projects can be made to the 'working documentation' (there are no specified extent the Law on Public Procurement147 allows.148 triggers), it will need to make formal adjustments, The key principle of allowed adjustments lies in the receive approvals, and go through expertise as if it possibility of changing prices and certain aspects of were a new project. goods and services that do not lead to an increase of the contract amount.149 Such changes in the contracts 270. The score for this dimension is A. can be made upon agreement of all parties (line ministry  147  The Law on Public Procurement, article 41 (part 4 – 6)  148  https://infobox.prozorro.org/articles/vnesennya-zmin-do-dogovoru-ta-publikaciya-zvitu-pro-vikonannya-dogovoru  149  Contract values typically include a contingency to small variations in cost due to unforeseen physical factors.  150  Except for assets with first class of consequences (CC1), where there I sonly one stage, but it is applied only to simple structures.  151  Order No. 45 of the Ministry for Territories and Communities Development of Ukraine on Procedure for Development of project documentation for construction  152  Resolution No. 560 of the Cabinet of Ministers of Ukraine as of May 11th, 2011, on the Procedure of approval of construction projects and their expertise  153  The working documentation provides detailed specification of drawings, technical solutions, and calculations that were approved at the previous stages of design. The preceding stages include drawings, technical solutions, and calculations, but with less details than in the working documentation. Preceding stages may result, for example, in design concept, technical and economic justification and other documents depending on the class of consequences of an asset. 82 Chapter 3. Assessment by Indicator for PIM Functions 19.2. ACTIVE MONITORING OF PROGRESS (see PIM-17.3), include proposals for resolving AND BUDGET REALLOCATION problematic issues. The consolidated semi-annual and PROCEDURES AND ACTIONS annual reports on grant-funded technical assistance projects (including construction projects) prepared Performance level and evidence for scoring the by the Secretariat to the CMU identify deviations dimension from the financial plan. The annual reports155 include conclusions, recommendations, and measures for 271. There are explicit rules and procedures on further use and attraction of technical assistance. monitoring progress and budget reallocation for SSIPs, however there is no such system 274. The Inter-Agency Commission has the right for non-SSIPs which are included in the State to take decisions on budget-neutral reallocations Budget without going through a formal selection between projects in the portfolio within a budget program. Active monitoring of projects approved year based on monitoring reports from MoE. If under Resolution No. 571 is performed by the MoE a monitoring report identifies poor progress in the based on the regular reporting regime described implementation of a particular project (e.g., increase under PIM-17. The MoE reports the findings of its of project cost by more than 10 percent, breaching of monitoring activities to the Inter-Agency Commission set deadlines, non-compliance of procurement result to inform its members on a project's progress. The with the plan) the Inter-Agency Commission may findings are presented to the members of the Inter- take a decision to reduce or stop financing for such a Agency Commission in the form of a table with the project and reallocate funds to other projects in the information on the project, its implementing agency, ongoing portfolio or to a new project in the pipeline line ministry, expenditures, and a brief description of within a budget year. If the decision on budget project progress, including comments on the deviation reallocation is taken, MoF is informed about it by MoE, from the plans and measures to be taken to close any while the actual transfer of funds is requested by the time/scope gaps. relevant line ministry. The MoE discloses information on monitoring and reallocation decisions taken by 272. The Inter-Agency Commission can also the Commission by publishing the minutes of the call for a mid-term audit of a project under meetings on its official website.156 A recent healthcare implementation if monitoring reveals problems. sector project, Reconstruction and Expansion of the The Inter-Agency Commission may take a decision National Cancer Institute, provides a good example: to request an audit of a partially or nearly completed UAH 150 million were approved for this project in public investment project, as well as projects on 2019, but execution did not go as planned. As a which fraud/corruption cases or nontargeted use of consequence, the Commission re-allocated almost the resources are suspected. Audits are carried out by the whole amount to other projects. State Audit Service. During the last three fiscal years, two public investment projects were audited,154 in the 275. At least on a monthly basis, senior area of responsibility of the Ministry of Culture. The management in implementing agencies and line audit reports include a performance focus, analyzing ministries are supposed to perform internal project implementation and identifying bottlenecks for monitoring. The form of this monitoring is successful completion. determined by the individual line ministry and is, as a consequence, variable in the degree of coverage 273. Donor-funded projects are actively and rigor across the public sector.157 The legislation monitored. The semi-annual consolidated reports allows (but does not oblige) line ministries to create on the implementation progress of IFI projects, commissions for internal monitoring of project which are submitted by the MoF to the government implementation and taking collective decisions.  154  Audit report of Comprehensive restoration and adaptation of the Zhovk castle of the State Historical and Architectural Reserve in Zhovkva.  155  Annual reports for 2019 and 2020 are available online at the government's web portal.  156  Minutes of the meetings of the Inter-Agency Commission from July 2016 to October 2020 [website], https://www.me.gov.ua/ Documents/List?lang=uk-UA&id=40253ac5-2a8c-447d-857d-1a34739d765a&tag=MonitoringStanuVikonanniaTaRealizatsiiDerzhavn ikhInvestitsiinikhProektiv, (accessed 15 June 2021)  157  Part II, Article 1 of the Order No. 1785 Chapter 3. Assessment by Indicator for PIM Functions 83 Although MoE does not control the internal 278. Independently of triggers leading to monitoring of projects, some discipline is imposed mandatory review, the Inter-Agency Commission through the requirement for line ministries to collect may take a decision on project cancellation or the data on progress from implementation agencies in necessity to review a project at any stage of its order to provide comprehensive quarterly and annual lifecycle if there are delays, cost overruns, change reports to MoE (as described under PIM-17). MoE in demand, and corruption cases. Specific criteria also has the right to request information on project give the right to the Inter-Agency Commission to implementation at any time.158 stop funding for project implementation or require project review under the following circumstances: 276. The score for this dimension is B. (i) if the project implementation time is extended by more than one year compared to the plan; (ii) if the cost has increased by more than 10 percent 19.3. MECHANISM TO TRIGGER REVIEW compared to estimates during procurement (however, OF PROJECT JUSTIFICATION this contradicts the 10 percent change in the cost Performance level and evidence for scoring the as a trigger for review, when the project must be dimension reviewed under such conditions without approval by the Commission); (iii) if the demand for the project 277. Triggers requiring an implementing agency services has significantly decreased; or iv) (if a fraud to review the business case for a SSIP focus on or corruption case related to project development cost overruns and change of design; reviews are or implementation is proved. Nevertheless, it should not triggered by demand changes or delays. Such be admitted that the Commission is not obliged to triggers are only applicable to SSIPs and there are cancel or initiate review of the project, but only given no specific rules applicable to non-SSIPs. Review of a right to do so.163 At the same time, the MoE or the business case may be triggered in the following any other agency can request a project review and circumstances: (i) changes that lead to increases reconsideration by the Inter-Agency Commission, in project costs of more than 10 percent, including if they become aware of delays, cost overruns, inflation; (ii) changes in technical solutions and/ or changes in procurement plans, but this rarely or the adding of extra expenses or additional happens.164 components159 not provided for by the approved project documentation; (iii) changes in the list or 279. The review of project justification typically purpose of equipment provided in the project results in changes to the project financing proposal; and (iv) other justified conditions that affect schedule and amount, as well as to the project results.160 The updated business case must implementation timeline, but not in cancellation go through examination by the line ministry161 and or rescoping. Review and subsequent adjustment be submitted to the Inter-Agency Commission for a are most often initiated because of (i) the inability decision on further project implementation.162 There of project implementing agencies to keep up with is some evidence that reviews are too easily triggered, the approved plan; (ii) project selection based on especially in relation to the cost trigger (which is unrealistic project cost estimates as a basis for relatively sensitive, given that it includes inflation), selection; and (iii) the effects of inflation on costs. leading to over-burdensome administration rather 280. The score for this dimension is C. than improved quality of investment decisions.  158  Part II, Article 3 of the Order No. 1785  159  Constituents or launch complexes of construction.  160  Article 7 of Resolution No. 571  161  Article 8 of Resolution No. 571  162  Article 7 of Resolution No. 571  163  Article 23 of Resolution No. 571  164  Article 30 of Resolution No. 571 84 Chapter 3. Assessment by Indicator for PIM Functions PIM Function 7. Service 282. The internal audit function for MDAs should meet international standards such as ISPPIA.165 Delivery Evidence of an effective internal audit function would include a focus on significant systemic issues and high- PIM-20. Control, Monitoring and risk areas, use by the SAI of the internal audit reports, Reporting: Financial and Service and action by management on internal audit findings. The internal audit function may be centralized in a Delivery Performance financial inspectorate with a mandate across entities of the central government, or by separate internal RATIONALE audit functions for individual government entities. The 281. The quality-of-service delivery should be combined effectiveness of all such audit organizations reported regularly to the sector agency and at is the basis for assessing this indicator. least quarterly to the line ministry and central ministries. Reporting and control of expenditure General observation commitments and the timely release of funds are critical to keeping service delivery on target. An 283. Separate budget programs cover the capital effective internal control system is one which is cost of each state investment project, while the relevant and comprehensive, is widely understood and capital cost of projects funded outside formal PIM complied with, and is circumvented only for genuine procedures are covered by budget programs that emergency reasons. Evidence of the effectiveness include operating costs of similar institutions/ of the internal control system should come from facilities. Once the state investment project is government financial controllers, and regular internal completed, the operating and maintenance costs are and external audits (PIM-22). The effectiveness of included in the relevant budget program, which covers expenditure commitment controls is singled out as the operations of the institution running the project. a separate dimension due to the importance of such controls to ensure that the government can always pay its bills (thereby avoiding expenditure arrears (PIM-14). Summary of Scores and Performance Table PIM-20 Indicator/ Dimension Score Brief justification for score PIM-20. Control, C+ Scoring Method M1 monitoring and reporting: financial and service delivery performance 20.1. Scope and C Treasury prepares quarterly and annual budget expenditures execution coverage of reports which cover all projects and programs, allowing direct comparison expenditure and of O&M expenditure with the approved budget. The reports provide performance reporting complete information on expenditures at the payment stage, but only unpaid commitments are given. MDAs establish service delivery targets for each budget program and report on their achievement.  165  International Standards for the Professional Practice in Internal Audit, issued by the Institute of Internal Auditors. Chapter 3. Assessment by Indicator for PIM Functions 85 Indicator/ Dimension Score Brief justification for score 20.2. Timeliness of C Budget execution reports which can be directly compared with the reports approved budget are prepared quarterly and issued within 6 weeks of end of period; and budget program passports execution reports (performance reports) are prepared only annually. 20.3. Quality of B There are some minor issues with the quality of the budget execution information reporting reports as identified by internal audit. The service delivery performance information is assured by the public availability of source information; conducting external and internal audits of the reliability of performance information and its sources; and using the electronic forms and information system to prepare and issue reports. 20.4. Coverage and B The most of state public authorities (90 percent) provide for the internal quality of the internal audit function and its overall effectiveness, and they cover 74 present of audit function total state budget expenditures. The Ukrainian Internal Audit Standards describe practical implementation of internal audit in the line ministries and public institutions. Internal audits are focused on systemic issues (more than 50 percent of staff time). The share of performance audits in the total number of audits increased to 30 percent in 2020 against 10 percent in 2018. 20.5. Extent of B IA units issued more than 14.1 thousand recommendations in 2020, and management response action was taken on about 85.5 percent of these recommendations, the to internal audit due date of which has come. findings 20.1. SCOPE AND COVERAGE OF 285. The reports provide complete information EXPENDITURE AND PERFORMANCE on expenditures at the payment stage, but only REPORTING. information on unpaid commitments. Unpaid commitments are shown within arrears reports Performance level and evidence for scoring the only. Total commitments are not shown beside the dimension approved and executed expenditures, as presented in the approved budget. 284. Treasury prepares quarterly and annual budget expenditures execution reports which 286. MDAs establish service delivery targets168 cover all projects and programs and can be directly for each budget program and report on their compared with the approved budget.166 According achievements. This information is contained to the law, quarterly and annual budget execution in the budget program passports and budget reports contain, among others, reports on the program passports execution reports.169 Budget execution of financial indicators as presented in the program passports execution reports contain actual annexes to the annual state budget law. Thus, reports performance results compared to all targets in the allow comparison with each budget program and budget programs passports. Each performance project presented in the approved budget in terms indicator in these documents covers all similar of 'consumption' and 'development' expenditures,167 institutions financed through the budget program. salary and utilities. The information comes from individual performance  166  Articles 60 and 61 of the Budget Code.  167  The budget does not present current and capital expenditures, but consumption and development expenditures. Last of them include some current expenditures beside the capital expenditures.  168  Performance indicators.  169  According to the MoF's Order No. 1098 dated 29 December 2002 'On budget program passports'. 86 Chapter 3. Assessment by Indicator for PIM Functions reports submitted to the MDA by each institution. for all budget spending units. Such a mechanism Generally, the monitored performance indicators track allows a thorough and regular monitoring and improvements in public service delivery (increased verification of financial information and cash flows volumes or better quality of services), including (in particular, crosschecking). The Treasury reports programs involving use of newly created assets/ include both approved expenditures (the law with facilities. all subsequent amendments) and actual figures, according to the cash accounting method. With regard 287. The score for this dimension is C. to expenditures and lending, the reports are compiled using all classification types (budgetary programs, functional and economic classification, and also the 20.2. TIMELINESS OF REPORTS administrative classification of the state budget).171 Performance level and evidence for scoring the 291. In support of Treasury reporting, effective dimension financial management information systems exist 288. Financial reporting is more regular and at the level of spending units. Each spending unit timelier than reporting on service delivery has its own system for automating the financial performance. Budget execution reports that can activities of public sector organizations. In this system, be directly compared with the approved budget are authorized persons with special access keep accounts, prepared quarterly and issued within 6 weeks of end sign payment orders, and generate reports. of period. 170 Budget program passports execution 292. There is not always a clear link between the reports (which deal with service delivery) are only generated performance information and the prepared annually, not quarterly. MDAs prepare the source data. Each performance indicator included in reports within three working days of submission of the execution reports for budget program passports the consolidated annual budget reports and publish is confirmed by either official state statistics, financial them on their official websites. These reports contain and other reporting, accounting data, statistical data planned and actual performance indicators, including or internal recording.172 When it comes to official service delivery indicators, and explanations for statistics, the connection is obvious. However, deviations. Spending units report on performance there is no information system linking performance more frequently, providing MDAs with actual indicators included in these execution reports and the performance indicators monthly and (or) quarterly, other source data used to confirm the performance depending on requirements established by the MDAs. indicators. There is also no full list of such documents 289. The score for this dimension is C. and the data which is contained in them. MDAs consolidate information about performance indicators from each spending units in Excel files and prepare 20.3. QUALITY OF INFORMATION the budget program passports and execution reports REPORTING using a centralized IT System. This system only contains the key figures on final planned / actual Performance level and evidence for scoring the performance and does not contain information on dimension their calculation. No significant concerns about the accuracy of performance information are highlighted 290. The use of the Treasury Single Account (TSA) in audit reports. ensures the accuracy of financial reporting. The Treasury uses the Treasury Single Account to service 293. The score for this dimension is B. all budget transactions (revenues and expenditures)  170  Treasury prepares quarterly reports no later than 35 days after the end of the reporting quarter (article 60 of the Budget Code)  171  2019 Public Expenditure and Financial Accountability (PEFA) Performance Assessment Report, Ukraine.  172  Based on MoF's Order No. 1536 dated 10 December 2010 'On the performance indicators of budget programs'. Chapter 3. Assessment by Indicator for PIM Functions 87 20.4. COVERAGE AND QUALITY OF THE MoF, together with justifications for the inclusion INTERNAL AUDIT FUNCTION or exclusion of audit objects and calculations of the workload of internal auditors. The MoF reviews Performance level and evidence for scoring the and analyzes the internal audits plans, in particular, dimension regarding the inclusion of audits focused on systematic issues in those plans. Based on the results 294. While the internal audit (IA) function is well of the analysis of the state bodies' operational internal established across government, its effectiveness audit plans for 2021, prepared by MoF, there are only is compromised by the failure to fully staff or very few cases of internal audits, which are focused on even partially staff IA units. State public authorities non-systemic issues. representing 90 percent of the total and representing 74 present of total state budget expenditures have 297. Internal audits focus on high-risk areas. MDAs internal audit departments. However, staffing is a with IA units approve annual internal audit plans serious problem: at the end of 2020, 18 public bodies using a risk-based approach, whereby each potential had more than 50 percent vacancies in their internal subject of an audit is scored according to risk factors audit departments and 11 bodies had not staffed their and only the riskiest subjects are selected. The report departments at all. These institutions constituted almost on the status of internal audit functioning in 2020 26 percent of overall state budget expenditures in 2020. shows that the share of performance audits in the While recruitment of qualified staff is undoubtedly a total number of audits increased to 30 percent in 2020 problem, there is also a suspicion that entities are not against 10 percent in 2018. The Accounting Chamber always pursuing recruitment seriously. and the State Audit Service are informed of the results of internal audits during the implementation of state 295. The Ukrainian Internal Audit Standards control measures, in particular with respect to findings describe the practical implementation of concerning internal control. internal audit in the line ministries and main public institutions of Ukraine. These Internal 298. The score for this dimension is B. Audit Standards also prescribe the professional and functional independence of internal auditors. Ukrainian standards are based on the Institute of 20.5. EXTENT OF MANAGEMENT Internal Auditors (IIA) but differ in some respects. RESPONSE TO INTERNAL AUDIT Methodical recommendations and guidelines on FINDINGS conducting internal audit procedures in line with Performance level and evidence for scoring the general accepted good practices were updated in dimension 2018. The Ministry of Finance has published five methodological guidelines for internal audit units 299. IA units issued more than 14.1 thousand during 2019-2020. These methodological guides cover recommendations in 2020, and action was taken issues of: (i) general competences, knowledge and on about 85.5 percent of these recommendations, skills; (ii) risk-based planning of activities; (iii) assessing the due date of which has come. According to the quality of internal audit; (iv) internal audit tools; the report on the results of IA units' performance and (v) methodological guidelines for internal audit. in 2020,173 almost 1.7 thousand audit reports were These methodological guidelines cover performance issued based on the results of internal audits. Almost audit as the assessment of the institution's activities 48.2 thousand shortcomings and problems were on: (i) the effectiveness of the internal control system, identified. The most shortcomings and problems (ii) the degree of achievement of goals, (iii) budget were identified in the following areas: performance of programs planning and execution, including budget control and supervisory functions; functioning of the program passports and relevant reports, and (iv) the internal control system; the correctness of accounting, quality of the administrative services delivery. the reliability of financial and budgetary reporting; budget programs planning and execution and their 296. Internal audits are focused on systemic issues performance. As a result of the implementation of (more than 50 percent of staff time). All public audit recommendations, the legal, regulatory and bodies with IA units send internal audit plans to the  173  Information on the functioning of the state internal financial control, 2020, prepared by the MoF. 88 Chapter 3. Assessment by Indicator for PIM Functions internal documents were revised and / or developed 302. The estimation of unit costs incurred in for the regulation and improvement of functions and service delivery is a key input to both evaluating processes. In particular, attention was directed at the the performance of a project and in implementing delivery of administrative services and paid services performance budgeting and management. Sector and the calculation of their associated costs, as well MDAs should estimate, at least annually, the costs of as methods of calculating performance indicators for delivering key services in key sectors. budget programs. 300. The score for this dimension is B. 21.1. EFFECTIVE SERVICE DELIVERY BY PROJECTS IN KEY SECTORS Performance level and evidence for scoring the PIM-21. Service Delivery dimension RATIONALE 303. MDAs are required to provide the MoE with information on the implementation status of 301. The ultimate purpose of public sector state investment projects during the three years projects and programs is to deliver services that after the completion of the investment stage and generate positive impacts. These are enhanced if commissioning of facilities.174 Reports should be the services can be delivered efficiently or at a low provided annually by March 1 of the year following the unit cost. Projects and programs are designed to reporting year. The template of the report contains, deliver targeted levels of services over an extended among other things, the information about service period. Once projects are operational, the services delivery, such as access to the service, improving the delivered should be monitored and variances from quality of services, compliance with standards, and the targeted amounts should be estimated, analyzed compliance with European standards. However, there and explained. Variances can arise because of poorly is no methodology for comparing actual services estimated targets, resulting from economic conditions delivered with targeted amounts for the specifically that differ from those forecasted, causing shifts in related project. Consequently, MDAs do not comply costs or demand for the service, or because the with these procedures and no such reports are project performance varies from expectations. prepared. Summary of Scores and Performance Table PIM-21 Indicator/ Dimension Score Brief justification for score PIM-21. Service D Scoring Method M2 delivery 21.1. Effective service D Spending units do not estimate, analyze, and explain variances in actual delivery by projects in service delivery indicators from the targeted amounts for the specifically key sectors related project. 21.2. Unit cost D Key spending units measure unit costs and report annually within the reporting and budget programs passports and the budget programs passports execution variances in key reports respectively using cash basis (not accrual) cost accounting systems. sectors  174  According to the MoE's Order No. 1785 dated 25 October 2016 'On approval of the Procedure for monitoring the state of development (implementation) of public investment projects'. Chapter 3. Assessment by Indicator for PIM Functions 89 304. Spending units do not estimate, analyze programs implementation during the fiscal year and explain variances in actual service delivery and as part of the annual execution reports for indicators from the targeted amounts for a budget program passports. Achieved performance specific project. Spending units, including those indicators are reported for all projects delivering key that use newly created assets/facilities, measure services within a relevant budget program. While and monitor service delivery by performance each institution keeps records on an accrual basis, indicators which are contained in the budget program performance indicators are reported using cash-based passports. These performance indicators include costs,176 with no allowance for expenses related to service delivery indicators, such as number of service consumption of fixed capital. recipients, service cost, service quality level, etc., but as performance indicators for the budget program 307. The score for this dimension is D. as whole, not for specific projects. Each spending unit has its own performance plan which it monitors during the fiscal year in accordance with the rules set by the MDA. There is no requirement or evidence to PIM Function 8. Evaluation show that the indicators in the plan are in line with the values targeted in the proposal for a specific project. PIM-22. Scope, Nature and Follow- up of External Audit and Ex-Post 305. The score for this dimension is D. Evaluation 21.2. UNIT COST REPORTING AND RATIONALE VARIANCES IN KEY SECTORS 308. This indicator aims at analysis and assessment Performance level and evidence for scoring the of external audit and ex-post evaluation of public dimension investment projects. It focuses on the procedure for external audit and timely submission of the audit 306. Spending units measure unit costs and reports. The indicator analyzes the existence and report on the achieved performance indicators practical application of ex-post evaluation of public on a cash basis. 175 They report to MDAs on achieved investment projects. The assessment covers the last performance indicators as part of the budget three completed fiscal years (2018 – 2020). Summary of Scores and Performance Table PIM-22 Indicator/ Dimension Score Brief justification for score PIM-22. Scope, nature D+ Scoring Method M2 and follow-up of external audit and ex-post evaluation 22.1. Timeliness of C Comprehensive annual budget execution reports are submitted to ACU by audit reports CMU in a timely way according to a legislated timetable. 22.2. Audit of service D Very few public investment projects are audited individually. Those delivery performance projects that are audited are at the stage of implementation, not and conduct of impact operation. studies  175  According to the MoF's Order No 1098 dated 29 December 2002 'On budget programs passports'.  176  According to the MoF's Order No. 1536 dated December 10, 2010, 'On the performance indicators of the budget program'. 90 Chapter 3. Assessment by Indicator for PIM Functions Indicator/ Dimension Score Brief justification for score 22.3. Ex-post evaluation D There is no formalized ex-post evaluation in Ukraine, but elements exist of large projects within procedures for monitoring at the operational stage for projects following Resolution No. 571. However, in practice, implementation of these procedures is not performing well due to their complexity and lack of capacities at MDAs. 22.1. TIMELINESS OF AUDIT REPORTS Execution reports do not include information about non-SSIPs (28.8 percent of total project costs). The Performance level and evidence for scoring the reports are published by the Treasury in the form of dimension Excel tables, and they do not contain any analytical data. Although the detailed information is not 309. The State Treasury provides annual budget available in the public sphere, ACU has indicated that execution reports to the Accounting Chamber of it receives the package of documents containing all the Ukraine and CMU. Annual reports are submitted information required by the Budget Code. to ACU by the CMU not later than April 1 of the year following the reporting period.177 MDAs do not 311. The score for this dimension is C. independently submit audited reports on capital expenditure to the ACU. 22.2. AUDIT OF SERVICE DELIVERY 310. The annual budget execution reports PERFORMANCE AND CONDUCT OF submitted to ACU for audit include specific IMPACT STUDIES information on public investment. According to the requirements of the Budget Code, as well as general Performance level and evidence for scoring the expenditure information, the reports must include dimension information on the implementation of SSIPs, indicating 312. The ACU does not systematically audit individual the total cost of each project, including information public investment projects, nor does it carry on the status of the project indicating the degree out audits of service delivery for such projects, of its readiness and the amount of state budget although it is allowed to do both. Special audits179 expenditures required for project completion.178 are typically focused on whole public agencies (not As well as information related to projects financed specifically on investment spending or projects) and from the state budget, the report must include on budget programs. Audits of IFI-financed projects are information on the implementation of investment also performed. Very few individual public investment projects implemented by the state with loans obtained projects have been audited by ACU during 2018 – from foreign countries, foreign financial institutions 2020, and none of those that were audited had been and international financial organizations. The total implemented as SSIPs according to Resolution No. cost of each project must be included, together with 571. As part of the audit of budget programs and of information on the status of the project, indicating the MDAs, performance audits of SSIPs may be performed degree of its readiness, volumes of relevant credits indirectly, but it may be difficult to disentangle the (loans) and state budget expenditures required for effects of projects from the wider elements of the project completion under budget programs, as well as program or the activities of an MDA. As an example, volumes of such credits (loans) and expenditures for in 2018 ACU audited the use of state budget funds the budget reporting period under budget programs. for maintaining the safe functioning of the Ukryttia In contrast to the requirements, the actual annual facility in the Chernobyl area, which is managed by the budget execution report only includes information Ministry of Ecology and Natural Resources.180 Within on the planned and actual costs within the year.  177  Budget code of Ukraine, Art. 61, https://zakon.rada.gov.ua/laws/show/2456-17#Text  178  Budget code of Ukraine, Art. 61  179  For the purposes of this assessment, special audits are distinguished from the standard audits of budget execution reports.  180  Audit report of use of budget funds for Ukryttia Program. Chapter 3. Assessment by Indicator for PIM Functions 91 this audit, three public investment projects that had financial and economic performance (economic NPV been selected by the Inter-Agency Commission are and benefit-cost ratio), social and environmental mentioned but are not subject to in-depth analysis indicators and indicators of budget efficiency. In of service delivery and impact. The report focuses on many respects, except for the conceptual link with compliance with budget processes and uses passports monitoring,181 the Order on Monitoring describes of budget programs as supporting documents. a de facto ex-post evaluation. However, in practice, according to the MoE, this requirement is not being 313. The score for this dimension is D. properly fulfilled as it presents technical difficulties for MDAs and there is no supporting methodological guidance and training. No such procedure exists for 22.3. EX-POST EVALUATION OF LARGE non-SSIPs, which are not subject to Resolution No. 571 PROJECTS and the related Order on Monitoring. Performance level and evidence for scoring the 315. The score for this dimension is D. dimension 314. No formal procedure or methodology for ex-post evaluation of public investment projects PIM-23. Legislative Scrutiny of exists. MoE recognizes this as one of the weakest External Audit Reports points in PIM system and sees a need for external assistance in this respect. Despite the lack of formal RATIONALE ex-post evaluation, some elements of it can be found in the existing procedures for monitoring SSIPs. 316. This indicator seeks to determine whether the According to the Order on Monitoring, 'monitoring' external audit reports are submitted in a timely has to be performed three years after an asset is put way and whether an adequate response to them into operation. Such 'monitoring' includes provision of is provided. The indicator assesses processes for actual data and discrepancies in financial, economic, external audit review, its content and timeliness. In social, and environmental indicators compared to addition, the indicator focuses on the response to what was forecasted in the project proposal and recommendations provided within audit and assesses implementation plan. The 'monitoring' report requires reaction and compliance to the recommendations by collection of data and calculation of indicators of MDAs. Summary of Scores and Performance Table PIM-23 Indicator/ Dimension Score Brief justification for score PIM-23. Legislative B+ Scoring Method M1 scrutiny of external audit reports 23.1. Timeliness of A The reports are provided in a timely way to VRU: annual audit reports on audit report reviews state budget execution are submitted to the VRU within 4 months of the end of the period in question. 23.2. Coverage of B Within the audit of annual budget, the results of all MDAs are analysed, legislative scrutiny including capital expenditures as part of development expenditure. Specific scrutiny of capital expenditures and public investment projects is fragmented, and there is no regulation that would require regular audit of investment as distinct from the rest of expenditures.  181  Implementation and performance monitoring are continuous processes; ex-post evaluation is a 'point-in-time' assessment. Many of the indicators listed in the Order on Monitoring, such as NPV, IRR, etc., are not monitoring indicators, as they cannot be calculated and examined on a continuous basis: they are 'point-in-time' indicators. 92 Chapter 3. Assessment by Indicator for PIM Functions Indicator/ Dimension Score Brief justification for score 23.3. Responsiveness A There is a clear legislative framework for providing responses to audit to audit scrutiny reports and audit recommendations and evidence that it works in practice. findings and recommendations 23.1. TIMELINESS OF AUDIT REPORT those implemented with IFI loans. Implementation of REVIEWS budget programs is also reported on in this document. The report analyzes budget execution from the point Performance level and evidence for scoring the of view of both financial flows and performance. dimension 320. When analyzing the report on execution of 317. The ACU informs the VRU of its audit findings the state budget, ACU provides a brief analysis in a timely way, including the facts relating to of state capital expenditures, which are also violations of budget legislation. The ACU submits mentioned in a fragmented way across its its conclusions and recommendations regarding the report. There is no specific section that focuses on annual report on execution of the State Budget to investment alone in the structure of the conclusions the VRU within 4 months of the end of the period in of the ACU on the results of its analysis of the annual question.182 The information is provided to VRU within report on execution of the Law on State Budget 15 days of approval by the ACU. A letter signed by the of Ukraine (the ACU's 'report on its conclusions ACU chairman is sent to VRU and includes copies of and recommendations'), which covers revenues, the conclusions and recommendation.183 expenditures, sources of financing and transfers only. There does not seem to be a systematic approach 318. The score for this dimension is A. to public investment analysis from year to year: for example, in the report for 2019 there is some analysis of public investment projects,184 while the same type 23.2. COVERAGE OF LEGISLATIVE of report for 2020 does not contain any information at SCRUTINY all on public investment projects (financed according Performance level and evidence for scoring the to Resolution No. 571), except for IFI projects and local dimension budgets. 319. The government's quarterly and annual 321. The ACU's conclusions and recommendations reports on state budget execution are audited by are scrutinized by the parliamentary Budget the ACU and the subject of legislative scrutiny. Committee, which has a public investment sub- There is no requirement to carry out annual audits committee, and by specialized sector committees of all the individual entities of central government, reporting to the VRU. The Budget Committee so parliamentary scrutiny focuses on the execution pays attention to capital investment expenditures of the state budget. The annual report on execution when reviewing the ACU's reports and the sector of the state budget submitted by the CMU, covers committees do so when reviewing special audits revenue, expenditure (including all capital and/or of relevance to their sectors. The responsibilities development projects) and assets/liabilities. It includes of the public investment sub-committee include information on public investment projects, including consideration of issues related to public investment  182  As verified by dates on submission letters from ACU to VRU and consistent with the findings of the 2019 Public Expenditure and Financial Accountability (PEFA) Performance Assessment Report, Ukraine (PI-30.1. Audit coverage and standards).  183  Chapter 40 of the Regulation for the Accounting Chamber https://rp.gov.ua/upload-files/About/RegulatoryDoc/arp_1.pdf  184  Report on execution of state budget, 2019 https://rp.gov.ua/upload-files/Activity/Collegium/2019/33-8_2019/Vysn_33-8_2019.pdf Chapter 3. Assessment by Indicator for PIM Functions 93 projects as these arise in any legislative instruments with the proposed improvements.186 If the audited or reports on implementation from the executive or agency does not respond as required, ACU informs ACU and of proposals to reallocate budget funding other superior bodies and the mass media.187 There for public investment projects between MDAs. The is evidence of audited agencies complying with the public investment sub-committee also participates ACU's recommendations and reporting on the results in the Inter-Agency Commission for State Investment of their implementation.188 Projects, which represents a potentially serious conflict of interest. 325. The score for this dimension is A. 322. Legislative scrutiny is concluded with the VRU preparing a resolution on the report on execution of the state budget.185 The opinion expressed in this report is grounded in the conclusions provided by ACU 3.2. Summary of PIM and the VRU Budget Committee. In contrast to the Performance and ACU's reports, significant attention is paid to public investment, including both SSIPs (funded according to Recommendations by the Resolution No. 571) and projects financed by IFIs. Functions The resolution for 2020, for example, advises CMU to take measures to ensure the efficient and effective use 326. There are many public institutions of state capital investments and compliance with the involved in PIM in Ukraine, each of them plays a deadlines for the implementation of public investment significant role at various stages of the system, projects. but overlapping roles and gaps exist. The MoE is responsible for development of legislation in 323. The score for this dimension is B. PIM and coordinating other MDAs. It is directly involved in the management of SSIPs by performing 23.3. RESPONSIVENESS TO reviews of project proposals, handling requests for AUDIT SCRUTINY FINDINGS AND capital investments to the MoF, arranging selection RECOMMENDATIONS of projects by the Inter-Agency Commission, and monitoring projects. Line ministries and central Performance level and evidence for scoring the government bodies are responsible for screening dimension project profiles, reviewing appraisals, arranging budget requests and financing, as well as closely 324. The ACU audit procedure includes monitoring controlling the progress of projects for which they are and assessing the implementation of the responsible. The Inter-Agency Commission's role is to audit recommendations. Audit reports provide select SSIPs for funding, monitor their implementation, recommendations to the agency/organization audited. and take decisions on budget re-allocations. The MoF The audited agency reports on implementation deals with budgeting of non-SSIPs funded through of recommendations. The monitoring reports are budget programs and takes decisions on the financing presented in the form of a table listing all of the cap for SSIPs. The Accounting Chamber's (ACU) recommendations (proposals) and indicating the responsibility is to perform financial and performance implementation status and timing against each audits of public expenditure, including expenditure one. This table should be filled in by the audited on public investment projects. The State Audit Service agency. Within one month after receiving the (SAS) also performs an external audit of investment recommendations, the audited agency must inform projects. The Anti-Monopoly Commission handles the the ACU of measures taken or planned to comply complaints related to public procurement.  185  Resolution of VRU on the Report on execution of the Law of Ukraine on the State budget of Ukraine for 2020  186  Guidelines on performance of Audit by the Accounting Chamber of Ukraine https://zakon.rada.gov.ua/rada/show/vr5-5150-15#Text  187  The Law of Ukraine on Accounting Chamber, Article 36 https://zakon.rada.gov.ua/rada/show/576-19#n330https://zakon.rada.gov.ua/rada/show/576-19#n330  188  Information on the implementation of the decision of the Accounting Chamber on the results of audit of the Program Ukryttia by Ministry of Ecology and Natural Resources. https://rp.gov.ua/FinControl/FinReactions/?id=1064 94 Chapter 3. Assessment by Indicator for PIM Functions 327. Despite the establishment of a regulated development strategy or vision, which comprises PIM system in which projects are appraised and straightforward and clear strategic objectives with selected for budgeting, a significant share of public specification of outputs and outcomes. Another investments still bypasses the established route important feature is a focus on alternative methods because of loopholes in the legal and regulatory of achieving of objectives, which should be already framework. The appraisal and selection of projects developed at the stage of project conceptualization. functions separately for the state and local budgets; Finally, an effective PIM system should involve a moreover, there is a separate procedure used by the formal review and approval of project ideas to assure Ministry of Infrastructure for transport sector projects, compliance with key national strategic goals and to which resembles the procedure for SSIPs. The current select the most promising alternatives for further legal and regulatory framework does not support development of project proposals. a unified PIM system for all projects receiving state budget funding. Main Findings by Indicator Main Cross-Cutting Challenge and PIM-1 [OVERALL SCORE C] Recommendation 329. Despite reasonable availability of statistical data, use of evidence-based sector analysis for Details are provided in Chapter 7. stronger strategic planning is not as strong as it should be. While relevant statistics are collected and Main cross-cutting challenge published by the State Statistics Service and sector MDAs, thorough analysis of the data and its use in The significant proportion of investment that appropriately sophisticated demand/supply models bypasses the formal appraisal and selection are lacking due to the lack of analytical and planning procedure, as well as project monitoring and capacities (Score C). Despite some evidence of the adjustments, as established in Resolution No. 571. use of analysis in strategy formulation, effective use of sector analysis, and planning results is not being Main cross-cutting recommendation achieved (Score C). To improve the terminology applied in the PIM system, so that it includes all public investments, PIM-2 [OVERALL SCORE D+] without exception. 330. Although high-level national strategic planning exists, it does not yet provide sufficient guidance to support identification and preliminary PIM Function 1: Strategic screening of project concepts. Strategic guidance, Guidance and Screening despite its availability, is not adjusted to the needs of a good PIM system and is not useful for project [Indicators 1-2] screening and prioritization (Score C). There is no investment strategy to guide the screening of public Main Features of a Good PIM System investment projects (Score D). Project profiles for screening and taking a decision on further 328. Good practice in strategic guidance and development of a project are only submitted to screening for public investment projects MDAs for SSIP projects (Score C). Screening of project has several features, the basis of which is profiles does not follow a standard approach with authoritative and evidence-based national each MDA following its own logic. The rejection rate of strategic investment planning. A good PIM project concept notes was 0 percent in 2018-2020 and system should be guided by a published national only around 3 percent in 2021 (Score D). Chapter 3. Assessment by Indicator for PIM Functions 95 Main Challenges and Recommendations Recommendation R-3 Relating to this Function The project concept stage and associated decision Challenge 1.1. to move to appraisal should be more firmly embedded in the legal and regulatory framework Lack of sophisticated methods for data analysis and then properly enforced by the MoE and the and forecast to support evidence-based strategic MoF with clear evidence of the project's rejection planning. or a request for it to be re-evaluated at a sufficient standard. Recommendation R-1 MDAs should improve their practices in sector analysis and planning with the introduction of PIM Function 2: Formal econometric modeling and application of methods Appraisal [Indicators 3-5] relevant to the sector. The analysis and results of modeling should be accessible to the public Main Features of a Good PIM System and available for using in sector forecasting or forecasting investment project impacts. The 331. Formal appraisal must be based on clear results of sector analysis performed by MDAs methodological guidance and the established should be used in strategic documents, making capacity of staff to implement it. To ensure it possible to set realistic goals for investment objective analysis and a unified approach to appraisal projects and assess their impacts on the sector. of public investment projects, governments should develop methodological guidance and make it Challenge 1.2. accessible to the actors in the PIM system. The methodological guidance on project appraisal should Insufficient strategic guidance to give direction to be transparent, leaving little room for interpretation. project identification and as a basis for screening The application of guidance should be disciplined but project concepts for strategic relevance. proportionate to project scale and risk,189 leading to a decision on economic and technical feasibility, and Recommendation R-2 financial, environmental, and social sustainability. Efficient training and deployment of staff is the Government should consider developing a cornerstone of good project appraisal and correct national infrastructure investment strategy (with application of guidance. a 10-year horizon, rolled forward every 3-5 years). The strategy should contain a clear outcome- oriented logic to guide MDAs, SOEs, and other actors of the PIM system. The adoption and Main Findings by Indicator updating of such strategy should be given a legal basis. PIM-3 [OVERALL SCORE C+] 332. Formal project appraisal procedures and Challenge 1.3. guidelines are in place for SSIPs, but this is not the case for non-SSIPs. Regulated by legislative acts Pre-appraisal screening of project concepts and by internal orders of MDAs, appraisal roles and for strategic relevance and rationale is applied responsibilities are clear between and within agencies unevenly. (Score A). Project appraisal guidance for SSIPs is available, is used by project developers, and covers  189  Sometime referred to as 'proportionality' or the 'proportionality principle'. 96 Chapter 3. Assessment by Indicator for PIM Functions the main topics of a typical feasibility study (Score Main Challenges and Recommendations B). While being universal and applicable to various Relating to this Function projects, guidelines lack sector-specific elements, distributional analysis, and up-to-date economic parameters (Score C). A methodological approach to Challenge 2.1. project appraisal depends on the project cost and the A significant share of the portfolio of public potential for making a profit only, while other risk- investment projects is not subject to formal based factors are not considered (Score C). Feasibility appraisal. studies are used to appraise large SSIP projects, including those of IFIs; however, 46 percent of projects (excluding IFI projects) are financed from the state Recommendation R-4 budget without an appraisal procedure that includes A strengthened legal framework – the Budget preparation of a feasibility study (Score C). Code, Resolution No. 571, and other related laws and legislative acts – should include provisions to PIM-4 [OVERALL SCORE D+] ensure that investment projects cannot be hidden in wider budget programs, thereby allowing them 333. Project appraisal capacity is not well to be funded without appraisal. developed due to limited experience and number of staff. Staff to oversee the application of project Challenge 2.2. appraisal methods are in short supply in MDAs, and in most cases the available project appraisal guidelines Inconsistencies in the application of appraisal are the only source of knowledge (Score C). Project tools resulting from the absence of detailed appraisal training has not been provided for the last methodological guidelines for practitioners and three years. The MDAs interviewed pointed out the poorly developed appraisal capacities. pressing need to arrange workshops to strengthen their knowledge and become familiar with up-to-date Recommendation R-5 appraisal methods (Score D). The MoE should improve project appraisal guidelines. The application and proportionality of PIM-5 [OVERALL SCORE D+] appraisal methods should be better justified and 334. Screening of feasibility studies is well defined; explained in the guidelines. non-SSIPs, however, avoid this stage and there is no systematic recording of the process, making it Recommendation R-6 hard to analyze its effectiveness. Sequenced and disciplined project screening procedures, as defined in MDAs should develop sector-specific guidelines Resolution No. 571, are applied to SSIPs, while non- (including sector-specific approaches to CBA). SSIPs do not follow the designated procedure (Score C). A prescribed public investment program (PIP) Recommendation R-7 database exists, but it does not cover the screening process. This absence of reliable recording makes it The MoE should arrange regular and continuous impossible to track the progress of projects (Score training for MDAs, project initiators and other D). The lack of good data makes it hard to assess the stakeholders in using existing (and improved) share of projects rejected or required to be redesigned guidelines and other sector specific methods of and reappraised. Based on the limited data provided appraisal. by key MDAs, in many cases no projects are rejected, and green lights are given for the development of projects that have received preliminary approval (Score D). Chapter 3. Assessment by Indicator for PIM Functions 97 of appraisal is functioning in the Ukraine PIM system, Recommendation R-8 however, its independence can be questioned, as the first level of independent review is performed by the Support MDAs in using-contracted outside MDA with an interest in project implementation (Score expertise when needed for large scale projects C). The content of an independent review is detailed where specialized skills would be essential for in the legislation, but there is no reference to project preparation. risks (Score C). Recommendation R-9 Investment-intensive MDAs and project initiators Main Challenges and Recommendations should establish project management teams or Relating to this Function units with adequate numbers of staff responsible for project appraisal. Challenge 3.1. Expert review, the first level of independent review, is performed by the MDA with PIM Function 3: Appraisal responsibility for the project, creating a conflict of Review [Indicator 6] interest. Main Features of a Good PIM System Challenge 3.2. 335. All project appraisals should be subject to Review of project proposals by MoE does not fulfil technical review or independent prefeasibility by the requirement of full independence. The MoE is an entity independent of the project sponsoring engaged in consulting MDAs and project initiators agency/ministry and with a high level of expertise during project development, which undermines in the conduct of project appraisal. Independent the independence of the review. review is especially important for all large projects above a defined total estimated cost or with new and Recommendation R-10 complex institutional arrangements. Independent review is widely regarded as the best way to counter The MoE should establish and lead a robust the usual optimism bias – over-estimation of benefits independent project review process, drawing and under-estimation of costs – of project promoters, on external expertise provided by parties by verifying the quality of the analysis, how realistic not involved in the project when needed assumptions are, and if estimates of demand and cost to supplement MoE capacities. The MoE are reasonable. It usually does not include revision of should establish a roster of external (to MoE) technical aspects of project pre-design. Independent independent reviewers from which it can draw review should follow a comprehensive set of criteria for specialist knowledge that it lacks. Members of covering all dimensions of appraisal, including realism the roster can be from other parts of the public of demand/benefits and cost estimates. sector or from the private sector. Projects should be reviewed by a team consisting of MoE staff and external experts. There should be a certification system for external reviewers who should not Main Findings by Indicator have any conflict of interest in relation to the project under review. The MoE should consider PIM-6 [OVERALL SCORE C] the opinion of the independent review team when deciding to permit the project to participate in 336. Independent review is well established in the the selection by the Inter-Agency Commission. At legislative framework; however, it is applied to the same time, the opinion should be provided to SSIPs only and the independence of the procedure members of the Commission. is not fully assured. A two-level independent review 98 Chapter 3. Assessment by Indicator for PIM Functions PIM Function 4: Selection and PIM-8 [OVERALL SCORE D+] Budgeting [Indicators 7-14] 339. There is a multi-year budgeting process, but it has not been functioning fully, which creates Main Features of a Good PIM System risks for multi-year projects. The practice of multi- year fiscal forecasts and functional allocations by the 337. Good selection and budgeting should include government falls short of good practice: only in two transparent criteria for selecting projects, a well- out of three years was such a forecast prepared, only structured budget system, effective gatekeeping, 25 per cent of projected development expenditures and adequate financing. After projects are are consistent with sector priorities, the mid-term appraised, they must be selected for financing from expenditure forecasts do not take into account the state budget. The selection must be based on implementation plans for either SSIPs or non-SSIPs, the transparent criteria allowing the most beneficial and approved capital expenditures differ significantly projects to be included in the budget. The budget from the forecasts (Score D). There are several preparation process should be structured in such separate databases of projects with approved funding, a way as to allow integration of investment and which cover various processes and are managed recurrent implications of projects into budgeting. by different MDAs; however, these have not been PIM and budget systems should include effective designed to support mid-term fiscal management and gatekeeping, which must ensure that only appraised management of total project costs. (Score C). and approved projects are included in the state budget. The budgeting process must ensure adequate PIM-9 [OVERALL SCORE C+] financing for selected projects, including recurrent costs after project completion. 340. The capital budget contains complete information on donor projects but is not comprehensive due to lack of reporting on extra- 4.A. BUDGET PREPARATION AND budgetary capital expenditures. There are no extra-budgetary capital expenditures captured by SELECTION [INDICATORS 7-10] the central government fiscal reports. SOE capital investment financed with budgetary lending and state Main Findings by Indicator guarantees exceeded 20 percent of total public capital spending in two of three assessed budget years (Score PIM-7 [OVERALL SCORE C] D). Donor funded projects and programs are covered by the budget execution reports, and all information 338. Project selection and budgeting is well on these projects and programs is available (Score A). organized for state investment projects, even if the schedule has not been adhered to and non- SSIPs avoided established procedures. The top- PIM-10 [OVERALL SCORE B] down budget process has not been working effectively because the budget circular does not contain limits 341. Despite high accessibility of the capital for expenditures for SSIPs, there are no restrictions budget information, the quality of the information on inclusion of non-SSIPs in the budget, and there is requires several improvements. The budget no strategic guidance for selection of projects (Score documentation contains neither summarized data on C). Adherence to the project and budget preparation capital expenditure by MDAs (or by sub-function or and approval calendar has been poor in the last three program), nor full information requirements for each years (Score C). Budget guidelines and practices for major capital project, including policy justification adjusting project budgets to current estimates are well and details for PPPs (Score C). Public access to budget arranged, but only SSIPs adhere to them (Score C). information is fully provided (Score A). Publication of Transparent criteria for project selection exist but are budget information is assured by the existing policy; applied to SSIPs only and adequate forward recurrent the actual publication is compliant with the law (Score budget is assured by the budget process (Score C). A). The legislature has full access to information on Chapter 3. Assessment by Indicator for PIM Functions 99 capital expenditure and rule-bound procedures are PIM-14 [OVERALL SCORE A] in place; however, the legislature added investment project proposals to the budget, amounting to more 345. The stock and growth rate of capital than 10 percent of the total investment expenditures expenditure arrears does not exceed the limits in the last three years (Score C). common for a good PIM system, and their monitoring is well organized. The size of arrears has been below 1 percent of total expenditures in the last three years and their growth rate was not faster than 4.B. BUDGET OUTTURN the real economic growth rate over the past two years PERFORMANCE [INDICATORS 11-14] (Score A). Reliable and complete data on the stock of arrears is generated monthly and annually (Score A). Main Findings by Indicator PIM-11 [OVERALL SCORE D] Main Challenges and Recommendations 342. Actual development budget expenditure, Relating to this Function including donor project expenditure, has significantly deviated from the originally budgeted Challenge 4.1. amount during the period under analysis. The deviation exceeded around 44 percent during two No one is performing the "gatekeeper" role and out of three years under analysis (Score D). Actual preventing projects that have not been positively donor project expenditure deviated from estimates by appraised from getting budget funding. almost 60 percent of budgeted development spending (Score D). Recommendation R-11 The MoF's gatekeeper function should be PIM-12 [OVERALL SCORE C] embedded in a strengthened legal framework (the Budget Code). The MoF, as a gatekeeper, must be 343. The variance in the composition of able to block un-appraised projects from entering development spending across agencies has the budget by the initiative of both government exceeded the overall deviation in development entities and Parliament. expenditure, but not by a consistently high proportion. The composite variance of development expenditures across MDAs exceeded 25 percentage Challenge 4.2. points in one of the last three years. The deviation was Budget documentation and financial reporting not significant in the other two years (Score C). are not sufficiently informative to serve as a background for adequate scrutiny of public PIM-13 [OVERALL SCORE D] investment projects and informed decision making. 344. Project completion time and cost variances for completed projects have breached the limits Recommendation R-12 acceptable for a good PIM system. In the last three years project completion time and cost variances for The budget should present the full list of state completed projects exceeded 30 percent (Score D). investment projects (as an annex) regardless of funding source or type, including the total approved multi-year cost, the total estimated amount spent up to the end of the current year, the amount for the budget year and the balance to complete, divided by year for the mid-term and in aggregate thereafter. 100 Chapter 3. Assessment by Indicator for PIM Functions Main Findings by Indicator Recommendation R-13 Ensure strict adherence to the rule of allocating at PIM-15 [OVERALL SCORE B+] least 70 percent of capital investments to ongoing 347. The procurement system is well elaborated projects in accordance with implementation plans. and established in Ukraine and could New projects should not be started if there is no potentially be used by donors. Transparency, fiscal space to implement them within mid-term comprehensiveness, and competition in procurement estimates. To implement this, appropriate mid- is assured by the legal and regulatory framework term analytical information should be added to (Score A). Use of competitive procurement methods project selection results. is the basis of the procurement system, while non- competitive methods are fully justified by law (Score Recommendation R-14 A). The share of programmable aid funds subject to national procurement procedures is non-existent, Financial reporting should be more as donors do not yet use the national procurement comprehensive. Reports should be extended to system (Score D). Public access to complete, reliable, contain SOE capital investments that are financed and timely procurement information is fully provided, with budgetary lending and state guarantees and all information can be found online (Score A). (by projects). Such information should include Independent administrative complaints institutions planned and actual indicators, the total approved and mechanisms exist within the procurement system multi-year cost, the total estimated amount spent (Score A). up to the end of the current year, and the balance to complete a project. PIM-16 [OVERALL SCORE C] 348. Projects are managed with reasonably PIM Function 5: clear accountability and control over total Implementation [Indicators costs, however, there are no guidelines uniting the principles of project management in a 15-18] widely applied system. Guidelines on project implementation do not exist, however, the regulations Main Features of a Good PIM System in various areas related to public investment and state budget process provide a set of rules that should 346. Good practice in project implementation be followed by all project implementers (Score C). stands on such pillars as good guidance, well Implementation plans are developed and followed. thought-out plans, clear accountability, efficient Accountability is clear between agencies, but there is a procurement, availability of funding, and lack of clarity within agencies, especially at the level of thorough reporting. The first requirement is for personal responsibility (Score C). A well-developed, but comprehensive guidelines on project implementation non-electronic, total project cost management system and management. Secondly, there should be clear is in place but applies to SSIPs only (Score C). organizational and management responsibilities and accountabilities for delivering on a detailed project implementation plan. Thirdly, procurement should be PIM-17 [OVERALL SCORE C] efficient, competitive and transparent. Fourthly, there 349. The processes of control, monitoring, and should be predictability in the availability of funds for reporting are established and function reasonably commitment and payment of project expenditures. well, but there is room for improvement in certain Finally, sufficiently regular and detailed reporting areas. Expenditure and performance reporting is should be in place allowing the control of annual and applied to SSIPs but is focused on annual expenditures total project costs. and does not cover commitments (Score C). Semi- annual and annual reports were provided during the period under analysis, while best practices require Chapter 3. Assessment by Indicator for PIM Functions 101 quarterly reporting (Score D). Donor funded projects or programs are monitored according to the national Challenge 5.2. legislation (Score A). Quality of information reporting Disjointed and incomplete reporting of is good enough but lacks detailed justification for implementation progress for whole projects. discrepancies (Score B). Recommendation R-17 PIM-18 [OVERALL SCORE D+] Improve the monitoring system to make it 350. Project handover and asset registration are more informative in terms of the total project in place; however, a formal, analytical completion progress, in both financial and physical terms. The review process does not exist in Ukraine's PIM monitoring reports should be focused not only system. A formal project completion and handover on in-year reporting – quarterly or annual results mechanism functions, but it does not examine against annual plans- but should clearly describe adequacy of funding and other preparatory works the situation with total project progress against for operation (Score C). There are two asset registers total cost, the planned implementation schedule, that contain information about all assets created, and implementation milestones. regardless of the type of project, but the information recorded is limited, missing information on the Challenge 5.3. maintenance history and current condition (Score B). Policy and guidance on internal completion review Good practice in total cost control does not extend does not exist (Score D). External completion review across all types of projects and the system used as a separate, integral process does not exist (Score would be difficult to scale up. D) and completion reports are mandatory for IFI and donor projects only (Score D). Recommendation R-18 Introduce automation of total project cost Main Challenges and Recommendations management (preferably using a software), which would alert the MoE and the MoF automatically Relating to this Function if the projected total project cost exceeds the approved amount of funding or established limits Challenge 5.1. for reassessment of a project. Too much informality in project implementation Challenge 5.4. arrangements. Lesson-learning from implementation experience Recommendation R-15 is not built into the system. The MoE should develop guidelines for project Recommendation R-19 management that would cover the end-to-end process of the project cycle. The guidelines should The strengthened legal framework should be useful for project initiators and MDAs, to which include provision for regular and systematic they report. internal and external completion reviews with analytical content. The results of reviews should Recommendation R-16 be presented to the Inter-Agency Commission and MoE for learning lessons. MDAs should transpose the guidelines to internal legislative acts/orders clearly stating the roles of actors involved in project implementation. 102 Chapter 3. Assessment by Indicator for PIM Functions PIM Function 6: Adjustment Recommendation R-20 [Indicator 19] Improve the mechanism for project reassessment by enforcing strict application of triggers and Main Features of a Good PIM System consequences for breaching them. Reassessment should lead to an explicit decision to continue, re- 351. Good process for project adjustment during scope, or cancel a project. Resolution No. 571 should implementation must be based on responsive be reviewed to avoid the possibility of ambiguous monitoring, judicious use of virement, and interpretation of triggers (see Recommendation fundamental review. The idea of responsive R-2). Consideration should also be given to raising monitoring is that the project progress should be the value trigger (currently a 10 percent cost actively followed to detect deviations from the plan increase) and expressing it in real terms. at an early stage in order to have an opportunity to resolve the issues. A good adjustment process is not possible without judicious use of virement, which PIM Function 7: Service should allow reallocation of budget between high and low performing projects. There should be a clear Delivery [Indicators 20-21] mechanism for triggering fundamental reviews of failing projects or those where baseline conditions Main Features of a Good PIM System have changed unfavorably. 353. Sustainable service delivery depends on adequate resourcing of operational expenditures, which is assured through good reporting and Main Findings by Indicator oversight of actual performance. In-year reporting and annual statements of operating and maintenance PIM-19 [OVERALL SCORE C+] expenditures and service delivery results are 352. A project adjustment process, based on active important for assuring quality service delivery. The monitoring and specific triggers exists but its service delivery should be monitored regularly by efficiency is open to question. Ukrainian legislation an MDA and the central agency responsible for in the construction and procurement sectors provide public investment policy. At the operational stage, rules for managing adjustments for contracts and the internal audit is essential for monitoring and PPPs (Score A). Active monitoring of progress and identification of risk areas. budget reallocation procedures are well established in legislation and applied in practice to SSIPs (Score B). A mechanism to trigger review of project justification Main Findings by Indicator is set up, however, it is not applied to non-SSIPs and there are some ambiguities in the legislation, PIM-20 [OVERALL SCORE C+] allowing various interpretations of triggers and their application (Score C). 354. Budget execution reports allow direct comparison of costs to those approved in the budget and internal audit and responses to its findings are functioning well. Expenditure Main Challenges and Recommendations and performance reporting allows comparison of Relating to this Function operational and maintenance costs with the approved budget, but not strictly on a commitment basis (only Challenge 6.1. unpaid commitments are covered). Service delivery targets are set for each budget program (Score C). Implementation of projects that are significantly Budget execution reports, which can be directly off track continues without an informed decision compared with the approved budget, are prepared on whether continuing with implementation still quarterly and issued within 6 weeks of end of a period makes sense. (Score C). There are no material issues in the quality of Chapter 3. Assessment by Indicator for PIM Functions 103 information provided by reporting (Score B). Internal PIM Function 8: Evaluation audit is operational for most, but not all, central government entities (90 percent) and it includes a [Indicators 22-23] focus on systemic issues (more than 50 percent of staff time) (Score B). Prompt and comprehensive Main Features of a Good PIM System action is taken by many MDAs in response to internal audit findings (Score B). 356. A good evaluation process must be based on ex-post studies and impact assessments. Policy and methodological guidance on ex-post evaluation must PIM-21 [OVERALL SCORE D] be available and applied to individual projects. Impact evaluations should also be undertaken for all major 355. Service delivery is not estimated and projects and a sample of smaller projects. monitored against the plans of specific projects. Service delivery by specific projects in key sectors is not measured and monitored (Score D). Unit cost reporting and variances in key sectors is performed Main Findings by Indicator within reporting on budget program passports using cash basis (not accrual) accounting systems (Score D). PIM-22 [OVERALL SCORE D+] 357. Elements of external audit and ex-post evaluation of public investment projects exist Main Challenges and Recommendations but are fragmented across the public finance Relating to this Function and investment system. Annual budget execution reports are submitted for the audit review by ACU in a Challenge 7.1. timely manner, but there is no practice of developing audit reports on individual projects (Score C). Audit of It is difficult to determine if individual projects are service delivery performance and conduct of impact delivering the volume and quality of services that studies is not performed for individual projects (Score were envisaged during planning D). Ex-post evaluation of large projects is not provided for in legislation, however, some elements of it exist Recommendation R-21 within operational monitoring (Score D). The MoE should review the regulation on PIM-23 [OVERALL SCORE B+] monitoring of projects during the operational stage to ensure measurement and reporting of 358. Legislative scrutiny of external audit reports service delivery performance against an ex-ante functions well, as stipulated by regulations. Annual project plan. audit reports on state budget execution are submitted to the VRU in a timely manner (Score A). Legislative Challenge 7.2. scrutiny covers aggregated capital expenditures of all MDAs, but it is not based on individual investment Not all MDAs have an internal audit function, projects (Score B). Responsiveness to audit scrutiny which affects project service delivery negatively findings and recommendations is provided for in legislation and works in practice (Score A). Recommendation R-22 MDAs should improve the internal audit function by ensuring that they have fully staffed internal audit departments and by strengthening control and enforcement of responses to internal audit findings. 104 Chapter 3. Assessment by Indicator for PIM Functions Main Challenges and Recommendations Relating to this Function Challenge 8.1. Concept and practice of ex-post evaluation is not institutionalized in Ukraine Recommendation R-23 Establish a formal ex-post evaluation process, covering efficiency, effectiveness, and impact: the legislative framework for SSIPs should be developed by adding by-laws and a methodology for ex-post evaluation of investment projects. Ideally ex-post evaluation must be performed not by MDAs or project implementers, but by independent third parties. However, additional funding for such services must be envisaged in the state budget. Challenge 8.2. There is no agency that would be clearly assigned to the external audit of public investment projects Recommendation R-24 SAS should coordinate with ACU more closely to minimize unnecessary overlap in audit, and more regularity in project audit should be introduced. Recommendation R-25 Perform external audit of individual major projects upon completion and on an annual basis for a sample of all projects using risk-based selection. ACU should be assigned the role of external auditor of public investment projects. The audit should be performed regularly. The results of audit should be communicated to the MDAs, project implementers, MoE, Inter-Agency Commission, and public. Chapter 3. Assessment by Indicator for PIM Functions 105 R 4 PTE CHA CLIMATE CHANGE- INFORMED PIM IN UKRAINE 4.1. Introduction Global Commission on The Economy and Climate190 the world will need to make significant investment in infrastructure over the next 15 years –around US$90 359. Climate change (CC) is a major threat trillion by 2030, and ensuring that this infrastructure to people and countries around the world. is sustainable will be a critical determinant of future Greenhouse gas concentrations are at their highest growth and prosperity. The report also highlights that levels in two million years and as a result, the earth is those investments can be recouped by transitioning now about 1.1°C warmer than it was in the late 1800s. to a green economy and unlocking new economic And because the Earth is a complex interrelated opportunities. Transitioning to a low-carbon, system, the increase in average temperatures has sustainable growth path could deliver a direct many profound consequences, including, intense economic gain of USD 26 trillion through to 2030 droughts, water scarcity, severe fires, rising sea levels, compared to business-as-usual. flooding, melting polar ice, catastrophic storms and declining biodiversity. Climate change will increase the 361. Which are the most important negative vulnerability of human and natural systems and the effects of climate change in each country depend economic costs of climate will be large. on its location, topography, socio economic conditions, and main economic activities. For 360. Financial resources and sound investments Ukraine climate-driven changes such as higher are needed to address climate change, to both temperatures may cause potential shifts in agricultural reduce emissions, promote adaptation to the zones that can compromise the country's food security impacts that are already occurring, and to build and economic growth. Urban population vulnerability resilience. According to the 2018 Report of The  190  https://newclimateeconomy.report/2018/wp-content/uploads/sites/6/2019/04/NCE_2018Report_Full_FINAL.pdf 106 Chapter 4. Climate Change-Informed PIM in Ukraine is magnified by infrastructure deficiencies including an reducing the use of fossil fuel powered motor vehicles aging and fragile housing stock, and limited potable thanks to improvements in public transportation by water supply. Increased incidence of strong floods in train. the last 20 years has affected nearly one-third of the population, especially in the Carpathian Mountains 364. Public infrastructure is also increasingly and their foothills. Droughts now occur on average exposed to the risk of damage from climate- once every three years. Other impacts of climate related disasters and needs to be designed change include: 191 a decline in surface water quality and operated in ways that reflect these risks. and quantity, erosion and flooding of coastal areas, More severe weather can have serious impact on damage to coal/gas infrastructure causing power infrastructure, as has been recently the case in supply interruptions, increased forest fire risks and Germany and China. Design standards must change in increase of some diseases. other ways to take account of the increased intensity and higher probability of the occurrence of heavy 362. Public infrastructure contributes to Climate rains, fires, fierce winds, or extreme heat or cold Change in diverse ways. Greenhouse Gas (GHG) and, consequently, increased resiliency to impacts emissions are generated directly during construction generated by climate change. and operation, for example from machines during construction or from the operation of a coal burning energy generation facility. Construction of a new road can increase deforestation, either by land clearing for construction or by facilitating access to forests 4.2. Methodological and fostering cutting of trees. There are also indirect Approach effects both during construction and operation, for example from the production of steel and cement to 365. To better address the aspects of PIM that be used for construction or from the increased use of are directly related to or affected by CC, the fossil fuel powered vehicles promoted by a new road. assessment of climate-informed PIM in Ukraine is organized around the eight 'must-have' functions 363. But provision of new public infrastructure of the World Bank PIM framework, rather than can also contribute to reducing GHG emissions, the more in-depth PIM indicators.192 For those for example by substituting electricity generated must-have functions that should include a CC aspect, by burning fossil fuels with solar or wind power Table 19 presents: (i) the climate responsive PIM generation, or by simply reducing use of energy by aspect; (ii) what is analysed; and (iii) the related PIM better insulation of public buildings and homes or by indicators. Table 19. Framework for the Assessment of PIM and CC Climate Must-have PIM responsive PIM What is analysed Related PIMs function aspect 1. Investment Climate- If CC impacts are incorporated into mid- PIM-1. Sector analysis Guidance informed and long-term strategic plans, if these and planning strategic plans serve as guidance for infrastructure PIM-2. Strategic plans guidance development planning, and if CC informed & investment guidance, strategies are being used as guidance for project development early screening of projects. and preliminary screening  191  Source: Climate Change Risk Profile Ukraine, USAID 2016  192  It will be remembered that the PIM indicators are already grouped according to the eight 'must-have' functions, which were originally expounded in Policy Research Working Paper 5397, A Diagnostic Framework for Assessing Public Investment Management, Anand Rajaram, Tuan Minh Le, Nataliya Biletska, Jim Brumby. WB. August 2010. Chapter 4. Climate Change-Informed PIM in Ukraine 107 Climate Must-have PIM responsive PIM What is analysed Related PIMs function aspect 2. Project Climate- If project appraisal studies incorporate PIM-3. Formal project Appraisal informed analysis of the impact of the project on appraisal procedures project CC and if project risks due to CC and and guidelines appraisal project resilience have been identified and incorporated into the project design. 4. Project Climate- If CC impacts of the project and risks due PIM-7. Project selection Selection and informed to CC are being used as criteria for project and budgeting Budgeting project selection and if there is clear identification selection of CC related projects in the budget (CC tagging). 5. Project Climate- If monitoring procedures during project PIM-17. Control, Implementation informed implementation include objectives, monitoring and monitoring indicators or activities relating to CC. reporting: physical and during financial milestones implementation 7. Project Climate risks If monitoring procedures during project PIM-20. Control, operation monitoring operation include objectives, indicators or monitoring and during activities relating to CC. reporting: financial operation and service delivery performance PIM-21. Service delivery Source: World Bank Mission Team 366. The assessment framework set out in Table 19 impacts on CC issues and factors climate impacts into is fully consistent with the dimensions of the PIM infrastructure design but is not specific to CC and indicator in the pilot PEFA Climate (CRPFM-5) and the adaption of infrastructure to CC-induced hazards. relevant questions of the climate dimension of the The legislation for disaster risk management is more World Bank's InfraGov assessment tool (questions 1-3 developed and detailed in respect of addressing the of InfraGov 5). hazards that might arise from CC. Despite the lack of specific CC provisions in the regulatory framework for PIM, the existing framework (see table in Annex 2) contains guidelines that can be used for assessment 4.3. Existing Legal of the risks of disasters. It also provides a basis for further development of PIM procedures that Framework incorporate CC. 368. The design standards for infrastructure 367. The existing legislative framework includes in Ukraine are aimed at both resilience of regulations about public investments that have buildings and mitigation of negative impacts on impacts on CC variables, as well as regulations the environment; however, they are based on about emissions of GHG. Most of the regulatory historical data that may not reflect CC impacts. framework concerns environmental impact, pollution, The existing design standards take account of climatic, and the resilience of assets to existing climatic geological, and seismological factors that may affect conditions. The legal framework therefore includes 108 Chapter 4. Climate Change-Informed PIM in Ukraine the project or asset. There are specific formulas that compliance with the UN Framework Convention define the characteristics of a facility, which should be on Climate Change, the Kyoto Protocol, and the resistant to snow, wind, and other weather conditions. Paris Agreement. Responsibilities also include According to the information provided by the Ministry policy formulation and control of compliance in for Communities and Territories Development, which environmental impact assessment of projects and is responsible for development and updating of the strategic ecological assessment. design standards, the future effects of the climate ii. The Ministry for Communities and Territories for on the asset to be constructed or reconstructed are Development is in charge of development and defined based on probability theory. The national approval of construction standards and policy standards for calculation of loads and impacts on formulation for compliance with construction the future assets advise using available statistics to standards, which include resilience to climate and generate probability distributions for the occurrence norms to increase energy efficiency. of hazards. However, these probabilities are based on past events and therefore do not consider changes in iii. The MoE, which is responsible for policy the probability of occurrence and magnitude of the formulation and implementation in the area of events due to CC. 193 compliance and selection of SSIPs based on CC related criteria. 369. While many design and construction iv. Line ministries (MDAs), which must ensure SSIP's standards have been updated during the last compliance to CC-related criteria. several years, some of them remain unchanged since the 1990s or 2000s. The Ministry for v. Agencies and SOEs which are responsible for Communities and Territories for Development preparing environmental impact assessments and plans to work until 2031 on updating design norms strategic ecological assessments. to create a unified and overarching regulatory environment in the construction sector harmonized with the requirements of the EU.194 This provides an opportunity to incorporate changes based on 4.5. Climate Change: forecasted impacts of CC in Ukraine. Assessments by PIM Function 4.4. Institutional Climate-informed Strategic Framework Guidance 370. The institutional framework in Ukraine 371. No strategic guidance pertaining specifically pertaining to CC issues includes a number of key to PIM and CC has been developed, but Ukraine players. The key institutions and their roles regarding has commitments under international agreements CC and PIM are: that are aimed at mitigation and adaptation to CC, and which have some influence on PIM. i. The Ministry of Ecology and Natural Resources These include the Association Agreement, the ensures the formulation and implementation United Nations Framework Convention on CC, and of state policy in the field of (among others) the Paris Agreement. Within scope of the Paris regulation of negative anthropogenic impact on Agreement, Ukraine's NDC target is not to exceed 60 climate change and adaptation to its changes and  193  In the national design and construction standard it is said 'the values of loads (the effects on assets) and coefficients may be set by probabilistic justification using available statistics. At the same time atmospheric effect may be set according to the data of the State Meteorological Service of Ukraine, as well as data of departmental meteorological services certified by the State Meteorological Service of Ukraine.'  194  Draft Order of the Cabinet of Ministers of Ukraine 'On approval of the Concept of development of the system of technical regulation and regulatory support in construction in Ukraine ' Chapter 4. Climate Change-Informed PIM in Ukraine 109 percent of 1990 GHG emission levels in 2030.195 In 372. There are no requirements for individual addition, Ukraine is committed to achievement of the projects to be compliant with international Sustainable Development Goals till 2030, including climate-related commitments. Ukraine must Goal 13 Climate Action, within which Ukraine aims to control and report on compliance with international reduce GHGs in economic activity. The second Kyoto environmental agreements and commitments, but Protocol obliged Ukraine to limit or reduce its GHG the contribution of each individual project is not emissions by 24 percent below 1990 levels, but SDGs assessed. If a project is likely to cause significant do not impose any particular targets on countries. impact on the environment, it will be assessed within The Low Emission Development Strategy to 2050 the full environmental impact assessment (EIA) for determines the national stakeholders' agreed vision construction projects, which typically happens after on decoupling further economic and social growth the project is screened, appraised, and selected for from growth in greenhouse gases emissions, but does financing. At the stage of preliminary screening, not address public investments and makes no specific developers of SSIPs must provide basic information in reference to resilience of infrastructure to CC. The the project concept note on the impact the project will national strategic documents concerning CC mitigation have on environment, but not specifically in relation to or having an impact on CC are shown in Table 20. CC.196 Table 20. National Strategic Documents Addressing CC mitigation Type of Year of Name of Document Status document approval The Concept/ The Concept on State Climate Policy Implementation till 2030197 2016 In force Strategy Action Plan The Action Plan to Implement the Concept on State Climate Policy198 2017 In force Strategy Low Emission Development Strategy to 2050 2017 In force Strategy The Law of Ukraine on main principles (strategy) of state 2019 In force environmental policy till 2030199 Action Plan National Action Plan on Environmental Protection till 2025200 2021 In force Action Plan 2020 National Renewable Energy Action Plan201 2014 Expired Strategy The Environmental Safety and Climate Adaptation Strategy by 2030 2021 Expected Action Plan Action Plan to the Strategy for Environmental Safety and Climate 2021 Expected Adaptation Strategy by 2030 Source: World Bank Mission Team  195  Updated Nationally Determined Contribution of Ukraine to the Paris Agreement (updated on 31.07.2021) https://www4.unfccc.int/sites/ndcstaging/PublishedDocuments/Ukraine%20First/Ukraine%20NDC_July%2031.pdf  196  Information required under Resolution No. 571 is less than what is normally required as a preliminary environmental impact assessment in a feasibility study as per international standards.  197  The Concept on State Climate Policy Implementation till 2030 in English https://mepr.gov.ua/files/docs/Zmina_klimaty/Concept%20of%20State%20Climate%20Change%20Policy.docx  198  The Action Plan to Implement the Concept on State Climate Policy https://zakon.rada.gov.ua/laws/show/878-2017-%D1%80#Text  199  The Law of Ukraine on main principles (strategy) of state environmental policy till 2030 https://zakon.rada.gov.ua/laws/show/2697-19#Text  200  National Action Plan on Environmental Protection till 2025 https://www.kmu.gov.ua/storage/app/uploads/ public/60a/222/2fb/60a2222fb2d25472902158.doc  201  2020 National Renewable Energy Action Plan https://zakon.rada.gov.ua/laws/show/902-2014-%D1%80#Text 110 Chapter 4. Climate Change-Informed PIM in Ukraine Table 21. Examples of Strategic Justification for a Sample of Projects Project name Strategic documents Creating a comprehensive 1. Law of Ukraine 'On the National Targeted Environmental Program for system for handling of Radioactive Waste Management' (dated 17 September 2008, No. 516-VI). radioactive materials, which 2. 'Strategies for radioactive waste management in Ukraine' (approved by are formed during stopping the order of the Cabinet of Ministers of Ukraine dated 19.08.2009 N 990-р). the operation of power units 3. Law of Ukraine 'On the National Program of Decommissioning of and reconstruction of Shelter the Chornobyl power plant and Transformation of the Shelter into an object. (Ukryttia, Chornobyl area Ecologically Safe System' (dated 15.01.2009 No. 886-VI). project) 4. Strategies for the transformation of the Shelter' (approved by the Decision of the Interdepartmental Commission for Comprehensive Resolution of the Chornobyl power plant of March 12, 2001, for No. 2). Geological study and industrial Law on approval of the National program of development of mineral development of the Avramivske resource base of Ukraine for the period till 2030 residual ilmenite deposit Source: World Bank Mission Team 373. According to Resolution No. 571, SSIPs must environmental impact'. However, there is no specific be aligned with the national and sectoral strategic mention to emissions of GHGs. documents, which by inference includes climate- related strategies, although there is no explicit 375. MoE has issued guidelines proposing the requirement. Project concepts and project proposals structure of the mentioned sections, but not should indicate the strategies to which the goal of specifying the methodology (Table 22). The the project is related; however, in most cases, the guidelines do not require evaluation of CC impact, strategies mentioned relate to direct project outputs climate adaptation or disaster risk management and outcomes, rather than cross-cutting issues like CC within public investment projects, but they do require (see examples in Table 21). Moreover, given that there environmental impact and risk assessment, including is no specific CC-informed public investment strategy, reduction of emissions, some of which can impact CC. screening for strategic alignment is difficult. 376. The information provided in project proposals submitted to the MoE does not include specific references to CC. Occasionally emissions of GHGs are Climate-informed Project mentioned, but there is no reference to CC impacts Appraisal of the project, risks to the project due to CC or the CC contribution of the project. As an example, the Climate-informed Appraisal of SSIPs project proposal to create a comprehensive system for handling radioactive materials contains information According to the Resolution No. 571 on forecast reductions in radioactive pollution, but nothing on GHG emissions. While another project 374. Resolution No. 571 requires that project proposal relating to mining (Geological study and proposals for SSIPs contain information on industrial development of the Avramivske residual emissions, pollution, energy saving, and mitigation ilmenite deposit) contains information on the volume of environmental impact risks. There are two of GHG emissions during the operational phase sections in a project proposal requiring forecasts (Table 23). These inconsistencies can be avoided by and analysis of environmental impact: 'measures for providing clear guidance to project proponents. environmental protection' and 'forecast of social and Chapter 4. Climate Change-Informed PIM in Ukraine 111 Table 22. Contents of Guidelines Section 17: Forecast of social and environmental Section 8: Measures for environmental protection impact Among the set of measures envisaged by the project, To make a forecast of environmental consequences, it is recommended to highlight and indicate in this the following structure is proposed: paragraph the following, which are aimed at: À analysis of the current state of the environment À reproduction and conservation of natural resources, in the territory that may be affected as a result À negative impact on the environment. of the project implementation (quantitative and qualitative characteristics), The information contained in this paragraph reveals À analysis of the impact of the project on the the environmental impacts of the project, therefore, environment in absolute and relative indicators: describing the measures, it is advisable to indicate improving air quality and reducing the level of its the expected measurable indicators (reduction of pollution; increasing the level of purity of water. emissions, increase in plantations, etc.). Source: World Bank Mission Team based on MoE guidelines Table 23. Examples of Content of Sections about Environmental Impact and Protection in Project Proposals Content of sections devoted to environmental protection in a project Project name proposal (sections 8 and 17) Creating a comprehensive 8. Measures for environmental protection. Brief information on resource system for handling of saving, protection of environment (water, for example), and monitoring radioactive materials, which of radioactive pollution is provided. There is reference to EIA. There is no are formed during stopping information on impacts of CC on the projects, neither on GHG emissions the operation of power units during the construction period. and reconstruction of Shelter 17. Forecast of social and environmental impact. This section focuses on object. (Ukryttia, Chornobyl impact and results related to reduction of risks of radioactive pollution. area project) Geological study and 8. Measures for environmental protection. The section includes information industrial development of on measures to prevent air pollution (from dust), water, and sound pollution. the Avramivske residual 17. Forecast of social and environmental impact. The section contains ilmenite deposit forecasts of GHG emissions during the operational stage of the project, as well as of water and soil pollution. It is stated that the emissions and pollution are within the norms of the Ukrainian legislation. Construction of water supply 8. Measures for environmental protection. Except for measures to protect networks in settlements environment, the section includes information on GHG emissions during of Lviv region, which use construction. GHG emissions from transportation of water before project transported water completion are also shown, these emissions will be avoided after putting the project into operation. 17. Forecast of social and environmental impact. Provides the same information as section 8. Source: World Bank Mission Team based on review of project proposals. 112 Chapter 4. Climate Change-Informed PIM in Ukraine 377. The legislative framework for PPP projects are approved by the Ministry of Ecology and Natural provides analysis of environmental impacts, Resources are then entered into the single register but it does not refer to CC. The project proposal of environmental impact assessments.205 However, for a PPP project contains two sections related doing an EIA with a CC orientation at the design stage to environment, but not directly referring to CC: is too late for including mitigation and (especially) 'environmental consequences of public-private adaptation measures into the project design, and for partnership, taking into account the possible negative considering technological, location and size options. impact on the environment' and 'socio-economic and Good practice would involve including assessment of environmental prospects after the expiration of the CC considerations in an earlier, preliminary 'EIA' that contract concluded in the framework of public-private could have an influence on project design at appraisal partnership.' 202 The guidelines for development stage. of a PPP project proposal contain information on environmental impact assessment, but it is focused 379. The report on the full EIA contains a section206 on analysis of the project's direct impact on the local that requires CC analysis; however, the existing environment and does not mention CC aspects or guidelines do not include a methodology for such GHG emissions. 203 analysis. The Ministry of Ecology has developed a general guideline for EIA and another specific guideline for the forestry sector. The general Climate-informed Appraisal of SSIP guidelines, which were approved in 2021, mention CC and GHG emissions, but do not specify the and Non-SSIP Projects According requirements of the analysis.207 The guidelines for the to the Legislative Framework on forestry sector mention CC, but again do not provide Construction any specific methodology on CC analysis.208 There are no other guidelines published by the Ministry of 378. A full EIA is only required for certain public Ecology. investment projects at the stage of project engineering design, that is, after the appraisal 380. Disaster risk management is considered in the decision and irrespective of whether they are full EIA, but only from the perspective of dangers SIPPs or non-SIPPs. This requirement is limited to from the project, rather than dangers to the State Budget funded projects or PPPs from certain project. The report on EIA must include information sectors only, as specified in the law. 204 The prescribed on the expected disasters the implementation of a sectors mostly relate to natural resources extraction, project could potentially cause; however, there is the oil and gas industry, agriculture, light industry or no specific requirement for identifying and studying other activities that might have significant impacts CC related disasters that may affect the project. on the environment. The report on the EIA has to be Measures to prevent or mitigate the impact of published for public consultation by the agency or SOE disasters on the environment must be described that plans to implement a project. The reports that in the report. The guidelines recommend applying  202  Resolution of CMU No. 384 as of 11.04.2011 on Some issues of organization of public-private partnership https://zakon.rada.gov.ua/laws/show/384-2011-%D0%BF#n274  203  Guidelines on the issues of public-private partnership https://www.me.gov.ua/Files/GetFile?lang=uk-UA&fileId=711e3245-1af5-47a1-b4e9-16ed2ae0225f  204  The Law on Urban Planning, Article 31 https://zakon.rada.gov.ua/laws/show/3038-17#n390  205  Resolution of the CMU No. 1026 as of 13/12/2017 On approval of the Procedure for submission of documentation for issuing an opinion on environmental impact assessment and financing of environmental impact assessment and the Procedure for maintaining the Unified Register of Environmental Impact Assessment https://zakon.rada.gov.ua/laws/show/1026-2017- %D0%BF#n79  206  The mentioned section requires description of environmental factors that are likely to be affected by the planned activity and its alternatives, including public health, fauna, flora, biodiversity, land (including land acquisition), soil, water, air, and climate- related factors (including CC and GHG emissions).  207  Order of the Ministry of Environmental Protection and Natural Resources of Ukraine No. 193 as of 15.03.2021 On approval of the General methodological recommendations on the content and procedure for compiling reports on environmental impact assessment https://mepr.gov.ua/documents/3342.html  208  Order No. 136 as of 02/03/2020 of the Ministry of Energy and Environmental Protection of Ukraine on approval of the Methodological recommendations for the development of the report on environmental impact assessment in the field of forestry https://mepr.gov.ua/documents/2749.html Chapter 4. Climate Change-Informed PIM in Ukraine 113 mathematical modeling, method of analogies and implementation. And, as has already been described, others to assess the risks of disasters caused by the specific consideration of CC in environmental impact project. The guidelines for the forestry sector provide and risk assessments done during project preparation some general recommendations on the application is not occurring, in practice, both for procedural and of disaster risk assessment in the sector but do not capacity reasons. provide any specific methodology. 384. In those cases where a full EIA is required at design stage,211 the implementing agency has to prepare an environmental monitoring plan. The Climate-informed Project general guidelines for EIA specify the details of how Selection and Budgeting the monitoring should be performed. The guidelines advise measuring all environmental impacts specified 381. Although one of the criteria established by in the EIA report and reporting this information, the MoE for selecting a new SSIP is a positive together with analysis, to the Ministry of Ecology and environmental effect, that criterion is not used in Natural Resources. The frequency of reporting is not practice due to a lack of necessary information. CC regulated by the general guidelines, but the guidelines assessment criteria and prioritization are not used to for the forestry sector advise providing monitoring select an SSIP for funding nor are they applied when reports once every two to three years until the selecting budget funded projects that do not follow negative impact on the environment fully disappears. the procedures in Resolution No. 571. The procedure for monitoring and its frequency must be specified in the monitoring plan at the stage 382. The fiscal planning and budgeting do not of EIA. In principle, this could be an instrument for allow tracking of climate change expenditures,209 monitoring CC impacts and risks, but this would (including those of investment projects related to require having identified the climate related impacts in climate policy implementation). Project assessment the original EIA, which is not currently happening. criteria and prioritization methods do not include consideration of a project's exposure to climate risks. 385. Monitoring as part of the Strategic Ecological There are no requirements for MDAs to prepare an Assessment (SEA) process is also required. Upon assessment of a project's exposure to climate risks. the completion of the SEA report, the monitoring Extreme weather and other disaster management plan is prepared and approved for controlling the are not considered during selection or during the impact the project may have on the environment. The preparation of project design documentation prior to monitoring must be performed by the agency that implementation. developed the planning document (a feasibility study, for example) once a year until the expiry of a planning document212 and a year after.213 The monitoring report Climate-informed Monitoring should be published on the website of the agency. during Implementation 383. Monitoring reports for SSIPs210 do not focus Climate Risks Monitoring on or require reporting related to CC or disaster during Operation risks. While the project proposal for an SSIP includes analysis of environmental impact and risks, there is no 386. Monitoring of environmental and CC risks requirement to report on this aspect during project during operation of non-SSIP is not conducted.  209  Climate change expenditures are any expenditures incurred to take measures to reduce risks, mitigate or address the impacts of climate change. Using the Technical Annex: Overviews of Climate Expenditure Tagging Frameworks. February 2021. World Bank.  210  Reports on passports of budget programs, monitoring reports on public investment projects, and updated Part III of feasibility study – see PIM 17.1 for a full discussion.  211  The Law of Ukraine on environmental impact assessment https://zakon.rada.gov.ua/laws/show/2059-19#n2  212  The legislation refers to the 'planning document', as strategic environmental assessment relates not only to investment project, but any government initiative that may have impact on environment.  213  Resolution No. 1272 of CMU as of 16.12.2020 On Approval of the Procedure for monitoring of consequences of execution of the document of the state planning for environment, including for health of the population https://zakon.rada.gov.ua/laws/show/1272-2020-%D0%BF#n10 114 Chapter 4. Climate Change-Informed PIM in Ukraine Budget reporting and monitoring cover operations 388. SOEs whose activities have an impact on the of budgetary institutions, without considering their environment, including those enterprises that direct impacts on the environment. Thus, when an have completed investment projects, monitor investment project in a budgetary institution (such such impacts and prepare an annual report on as healthcare or education institution) has been the findings. This is required by several regulations, completed, the respective Passports of Budget each of which addresses the reporting of a particular Programs and relevant reports contain performance type of environmental impact, notably: (1) generation indicators about healthcare and education only, and and management of waste,215 (2) water pollution,216 nothing about environmental consequences, including and (3) emissions of pollutants, including GHGs.217 in relation to CC. Table 24 presents how monitoring data required by existing legislation is useful for SOEs. The mitigation of 387. Monitoring of service delivery from SSIPs at the risks to the climate from an enterprise's activities, the operational stage should be carried out based including activity as a result of a project, might be on the indicators contained in the feasibility presented, for example, in the investment program study.214 However, despite this requirement, of the enterprise. At the same time SOEs, whose no reports are in fact prepared on the status of activities have an impact on the environment currently implementation of public investment projects implement investment projects with extensive after the completion of the investment stage and measures to reduce the negative impact on the commissioning of facilities. During such monitoring, environment. environmental indicators should be monitored only in terms of increasing / decreasing emissions of pollutants. A project's exposure to climate risks is not required to be monitored. Table 24. Examples of Potential Uses of Monitoring Data Type of Use of data Regulation monitoring data Water quality Scientifically substantiated recommendations Resolution of CMU No. 758 as of necessary for making management decisions 19.09.2018 On approval of the in the field of water use and protection and Procedure for state water monitoring. reproduction of water resources. Waste Determine and predict the impact of waste on The Law of Ukraine No. 187 as of generation and the environment, and the timely detection of 05.03.1998 On Waste, Article 29 management negative consequences and their prevention. Emissions Regulate emissions of pollutants into the Resolution of CMU No. 1655 as of atmosphere from stationary and mobile 13.12. 2001 On Approval of the sources, to reduce the degree of impact on the Procedure for maintaining state state of physical and biological factors of the air. accounting in the field of atmospheric air protection'. Source: World Bank Mission Team based on existing legislation.  214  Order of MoE No. 1785 as of 25.10.2016 On approval of the Procedure for monitoring the state of development (implementation) of public investment projects  215  Order of the State Statistics Service of Ukraine No. 164 as of 25.06.2021 On approval of the form of state statistical observation No. 1-waste (annual) Report on the formation and management of waste  216  Resolution of CMU No. 758 as of 19.09.2018 On approval of the Procedure for state water monitoring  217  Order of the State Statistics Service of Ukraine No. 405 as of 18.12.2019 On approval of the form of state statistical observation No. 2-TP (air) (annual), Report on emissions of pollutants and greenhouse gases into the atmosphere from stationary sources of emissions Chapter 4. Climate Change-Informed PIM in Ukraine 115 389. The monitoring of environmental and CC 392. When selecting public investment projects, risks has recently been enabled through a newly there is no screening based on criteria of climate adopted legislative framework, that will require policy alignment, climate impacts or climate- strengthening monitoring during operation and related risks. Indirect consideration is, however, given maintenance of public investment projects. In in the case of SSIPs because selection criteria include December 2019, the Ukrainian Parliament adopted environmental impact. And considering that project the Law on Monitoring, Reporting, and Verification of proposals include information on emissions, CC is to GHG emissions. 218 Together with supporting legislative some extent factored into the decision. acts,219 it is aimed at fulfilling Ukraine's obligations under international agreements. The idea of the 393. To make progress on CC-informed PIM, the newly introduced system is to monitor the emissions following improvements could be made: of facilities that fall under the list of sectors/activities a. Ensure that the national strategic documents of with high emissions and to verify the monitoring. The relevance to public investment address clearly the legislative framework covers only the industry and relevant international commitments and policies energy sectors, which are the biggest emitters of the and make specific reference to requirements for GHGs (CO2 and NO2) that are monitored. The Ministry resilience of infrastructure to CC. of Ecology and Natural Resources is the key body b. Update Resolution No. 571 and existing guidelines regulating this area. to require an assessment of the impact of a project on CC variables (GHG emissions), strictly requiring that individual projects be compliant 4.6. Conclusions and with international climate-related commitments Recommendations and policies, and that there be an analysis of resilience of the project to CC impacts. 390. Mainstreaming of CC into PIM is at very initial c. Support CC informed appraisal of projects, stage in Ukraine and improvements to the existing strengthen the CC criteria in project appraisal framework are required. There is no specific legal methodologies and EIA methodologies, and framework requiring consideration of CC impacts begin to generate forward-looking information on projects or of the effects of specific projects on about variables important for project design and CC. There are also no guidelines on how to consider appraisal which will be affected by CC (such as these issues when appraising and selecting public increased strengths of winds, greater intensity investment projects. of rains, changes in rain patterns, forecasted frequency of forest fires, etc.) 391. Fiscal risk statements do not identify d. Develop guidelines and procedures for tagging CC climate change related risks and climate-related related projects in the budget, and for monitoring expenditures are not identified in the budget and CC variables during project implementation and consequently not tracked. The only exception is the operation. This could be implemented through budget program 'State support for measures aimed the budget program classification: the third at reducing greenhouse gas emissions (increasing register of the budget program classification220 absorption), including insulation of social security with the number "9" could identify and tag climate facilities, development of international cooperation on related budget programs and the last digit could climate change.' The key spending unit of this program differentiate mitigation and adaptation measures is the Ministry of Ecology and Natural Resources. of such a program. However, climate change expenditures of other ministries and agencies, and subnational governments are not tagged in the state budget.  218  The Law of Ukraine on Monitoring, Reporting, and Verification of GHG emissions https://zakon.rada.gov.ua/laws/show/377-20#Text  219  Resolution of the CMU No. 880 as of 23.09.2020 On approval of the list of activities, greenhouse gas emissions of which are subject to monitoring, reporting and verification https://zakon.rada.gov.ua/laws/show/880-2020-%D0%BF#n8 and Resolution of CMU No. 960 as of 23.09.2020 On approval of the Procedure for monitoring and reporting on greenhouse gas emissions https://zakon.rada.gov.ua/laws/show/960-2020-%D0%BF#n9  220  The classification consists of seven digits grouped in four registers (MoF's Order of January 14, 2011 No. 11 "On budget classification".). 116 Chapter 4. Climate Change-Informed PIM in Ukraine R 5 PTE CHA ASSESSMENT OF PUBLIC-PRIVATE PARTNERSHIPS 394. PPPs are public investment projects may require budget funds or create fiscal obligations implemented with private financing and or fiscal risks for the state, leading to the inefficient management and should therefore share common use of limited state resources if management is approaches to their identification and preparation. inadequate. This problem can undermine the fiscal In common with conventionally financed projects, and fiduciary obligations of the government. In view PPPs support the creation of infrastructure and of this, it is considered both desirable and effective provide services that are a significant component of to use a unified PIM system incorporating PPP in the public good, or address market failures that would the framework of infrastructure planning and public prevent or impede the provision of these services financial management.221 This chapter considers PPPs directly by the private sector. Additionally, both forms separately from more conventional infrastructure of implementation involve financial obligations for procurement and financing modalities to reflect the public sector, including direct and / or contingent current practice in Ukraine. liabilities. 396. The assessment of the current arrangements 395. PPPs should be appraised for basic need and for organizing PPP in Ukraine was carried out using public value prior to decisions being taken about the same methodology as the main assessment the form of financing. Uncoordinated management considering the idiosyncrasies of PPP. Due to these of these processes negatively affects the proper features, and due to there being no current possibility identification and selection of new infrastructure of providing long-term fiscal commitments on PPP projects and can lead to the inefficient allocation of projects, indicators characterizing the budget outturn resources. The implementation of all public projects performance were excluded from the PPP assessment.  221  See Chapter 10 of the World Bank's Public Investment Management Reference Guide, 2020. Chapter 5. Assessment of Public-Private Partnerships 117 Additionally, given the different procedures compared to the implementation of conventional concerning their implementation and adjustment, this public investment projects and the lack of information chapter did not assess those indicators (see Table 25). on their practical use, the assessment of the selected Given the limited experience of PPP in Ukraine indicators was carried out at a qualitative level. Table 25. Framework for the Assessment of PIM and PPP Indicators excluded from evaluation Reason 4. Selection & budgeting 4.A. Budget preparation & selection PIM-9. Comprehensive capital budget In PPP projects, the capital is provided by the private sector rather than the government. Accordingly, the information on PIM-10. Comprehensiveness and degree capital budget submitted for consideration to the legislature of public/parliamentary access to capital does not include any capital commitments for PPP projects. budget information 4.B. Budget outturn performance PIM-11. Development and capital budget execution rates: Aggregate expenditure outturn compared to adjusted original budget on a commitment basis The issue of budget outturn performance in relation to PPP PIM-12. Composition of development and projects can be considered only in the context of providing capital expenditure outturn compared to government support to PPPs, including those implemented in adjusted original budget on a commitment the form of a concession. This is not possible today as result of basis existing limitations in the Budget Code of Ukraine. Hence, the corresponding indicators are excluded from the assessment PIM-13. Project completion time and cost variances for completed projects PIM-14. Stock and monitoring of capital expenditure arrears 5. Implementation PIM-17. Control, monitoring & reporting: This indicator does not consider the specifics of PPP and is physical and financial milestones relevant only for conventional procurement. Management of total costs during a PPP project lifecycle is carried out by a private partner / concessionaire. The public partner/ grantor does not interfere with the private partner's operational activities and only controls the achievement of the performance indicators by the private partner / concessionaire as stipulated by the PPP / concession agreement. PIM-18. Project handover, asset registration This indicator does not consider the specifics of PPP and and completion review is relevant only for conventional procurement. For PPP projects, including those that are implemented in the form of a concession, the handover of management responsibility for future operation / maintenance and adequate budget to maintain and operate the assets is to the private sector, and occurs when the PPP / concession agreement is signed. 118 Chapter 5. Assessment of Public-Private Partnerships Indicators excluded from evaluation Reason 7. Service delivery PIM-21. Service delivery This indicator does not consider the specifics of PPP and is relevant only for conventional projects. In PPP projects, the reimbursement of investments by a private partner / concessionaire depends on his achievement of the performance indicators specified in the relevant agreement. Most of these performance indicators (or even all) relate to the services provided under the agreement. 8. Evaluation PIM-23. Legislative scrutiny of external audit This indicator does not consider the specifics of PPP and there reports are no specific requirements covering PPP external audit reports Source: World Bank Mission Team 5.1. Introduction and the UAH 18.27 million and UAH 157.8 million, respectively; one for a thermal power plant of UAH 51.5 million; Current Situation and one in tourism. Two of these agreements are in the process of early termination. The remaining two 397. The Government of Ukraine recognizes agreements, for Kherson and Olvia port projects, the importance of PPPs for improving public with investments of USD 53 million and USD 654 infrastructure for service delivery and includes in million respectively, were signed by the Ministry the National Sustainable Development Goals, the of Infrastructure (MoI) in 2020. At the time of the initial target of 45 new PPP projects by 2030. Later, assessment (October 31, 2021), these projects were in 2019 the GoU revised this indicator: new targets are still in the transition period (pre-implementation). 205 PPP contracts in 2020; 216 PPP contracts in 2025; Over the same period, an unknown number of other and 240 PPP contracts in 2030. 222 The importance projects were signed but subsequently cancelled. of accelerating infrastructure development through 399. Most public authorities do not actively private sector participation is constantly emphasized initiate infrastructure projects on PPP terms. in government strategies and programs, including Currently, only MoI is preparing PPP proposals for the National Economic Strategy until 2030. However, projects in the seaport sector and in roads and railway Ukraine still has limited and not particularly successful station reconstruction. Reflecting this activity, at the experience in the practical implementation of PPP time of the assessment, six concept notes on the projects. implementation of road projects in the form of non- 398. Only six concession projects have been concession PPPs have been adopted in accordance implemented in relation to state assets up to with the requirements of the legislation. No other line 2021. Four concession agreements were concluded ministries currently consider exploring the potential during 2004 – 2012 for relatively small projects in the for PPP implementation. energy sector: two projects for wind power plants of  222  Information in accordance with the letter of the MoE Nr.3032-04/10 dated 13.01.2022 and the Report of the State Statistic Agency of Ukraine for achievement of SDG 17 in 2020 - http://www.ukrstat.gov.ua/. As of January 26, 2022, the National Report "Sustainable Development Goals" is posted on the official website of the Ministry of Economy, which does not reflect these changes https://www.me.gov.ua/Documents/List?lang=uk-UA&id=938d9df1-5e8d-48cc-a007-be5bc60123b8&tag=TSiliStalogoRozvitku Chapter 5. Assessment of Public-Private Partnerships 119 400. Ukraine has had a number of unsolicited (see Box 8). MoI has applied this to all concept notes proposals ('USPs') submitted for consideration and feasibility studies of PPP projects, including over the past four years. The number of USPs is even the most basic information. This is at odds with higher than the number of PPP proposals initiated by international good practice which would restrict public authorities over the same period. Three USPs confidentiality to commercially sensitive information. were submitted for airport infrastructure projects, one for a seaport and one for energy infrastructure. One of these proposals has been rejected, all others are still at different stages of consideration; so far, no agreements have been signed. The rules for USPs 5.2. Legal Framework remain the same as for proposals initiated by the 402. In recent years (2016-2021) Ukraine has government in accordance with good practice. significantly improved its legislation in the field of 401. It has been impossible to assess the quality of PPP/concessions. A new law on concessions has been completed concept notes and feasibility studies adopted; significant improvements have been made to of PPP projects due to their being classified by the the Law on PPP and other relevant laws and sub-laws authorities as "confidential." Despite the broad of Ukraine. The legal framework in the field of PPP requirements for transparency of information on PPP has been updated with the support of the EBRD and projects in the legislation, especially for concessions, the World Bank. As a result, the legislation specifically there is a widespread problem with the practice of related to PPP is now generally in line with good regularly applying a 'confidential information' status international standards and allows the use of modern innovative approaches to structuring PPPs. Access to Public Information BOX 8 According to the Law of Ukraine "On Access to dissemination, and not the entire document Public Information": (Part 8 of Article 6 of the Law). À Confidential information refers to public À The law also defines that information on information with a limited access. It is the possession, use or disposal of state and considered to be information to which the municipal property, as well as information access is restricted by an individual or legal on budget commitments or the disposal entity, except for subject of government of budget funds in any other way are the authorities, and which may be disseminated information to which access cannot be in the manner prescribed by them at their restricted. (Part 5 of Article 6 of the Law). request in accordance with the conditions provided for by them. That means, that the The concept notes have been prepared by the law clearly stipulates that subject of power Ministry of Infrastructure, which is the subject of (public authorities) may restrict access to government authority; they contain information information received by them confidentially, on the possession, use or disposal of state or if the dissemination of this information is not communal property, access to which may not be agreed by those persons (individuals or legal restricted in accordance with the law. The same entities) who granted it such status (Articles 6 – applies to the feasibility studies developed by the 7 of the Law); Ministry of Infrastructure (subject of authority). À It is the information that can be restricted, In addition, documents as a whole (concept notes not the document. Therefore, if the document and feasibility studies) are currently classed contains information with limited access, as confidential, and not individual information only the information that has been granted contained in them: this is expressly prohibited by confidential status is not a subject to law. 120 Chapter 5. Assessment of Public-Private Partnerships 403. At the same time, a serious obstacle to the 406. PPP legislation now permits modern development of 'government-pays' PPPs and some innovative mechanisms for structuring PPP concession projects remains. As the Budget Code projects, allowing an implementation model that does not currently provide for the possibility of balances public interest with private business making long-term commitments on PPP (concession) expectations. These mechanisms include various projects, most of the forms of government support for forms of government support which can be used PPP implementation provided for in the PPP Law and for both concessional and non-concession PPPs; Concession Law cannot be applied in practice. 223 This Step-in-Rights for lenders; and a competitive means that projects that rely on such arrangements, dialogue procedure.225 However, all this potential typically in the social sectors, are not bankable. This has not been realized due to a combination of the is the main reason that to date no 'government-pay' current constraints of the Budget Code and the low PPP contracts have been implemented in Ukraine: institutional capacity of public authorities. only commercially attractive concession projects have been prepared. A new draft law amending the Budget Code is currently in the parliamentary process and, if approved in the form required, would solve this issue. 5.3. Institutional 404. The current PPP specific legislation clearly Arrangements defines the conditions for using PPP for the implementation of infrastructure projects. It 407. Ukrainian legislation clearly defines the establishes a unified procedure for PPP project institutional roles and responsibilities of various initiation, regardless of the form of PPP (concession public entities through all phases of the PPP / 'user pays' or 'government pays') or the initiator project cycle (see Table 26). Responsibility for state of a PPP proposal (public authority or private policy in the field of PPP is vested with the MoE which business). The process of appraising PPP proposals is legally mandated with developing, overseeing is clearly established in the legislation and clarified and monitoring PPP policy in Ukraine. MoE is the in comprehensive guidance. It meets international 'gatekeeper' in PPP project implementation for state principles in this area and can be considered to assets. MoE approval of conclusions concerning the represent good practice. results of efficiency analysis of PPP implementation is mandatory when adopting a decision to use PPP 405. In 2021, a detailed PPP manual was developed implementation for state-owned assets. with World Bank support.224 This manual provides comprehensive guidance on the application of current 408. The 'Agency for Public-Private Partnership legislation to initiate, prepare, appraise, procure Support' (PPP Agency) assists the MoE in providing and manage PPP projects, including methodological methodological support to public authorities approaches to project prioritization and the during the preparation and implementation of formation of pipeline projects. It includes analysis and PPP projects and in boosting the PPP pipeline. experience of their use in other countries. It provides Established by MoE in 2019, the PPP Agency is a basis for increasing the institutional capacity of responsible for: identifying PPP projects; assisting public authorities and for accelerating the process public partners in the preparation of identification of preparing and implementing PPP projects for the reports, concept notes, feasibility studies and tender development of sustainable and resilient privately documents; capacity building; and seeking investors financed infrastructure.  223  In accordance with the Budget Code of Ukraine, state and municipal bodies can make budget commitments at the expense of the appropriate budget only within a mid-term period (3 years). Long-term liabilities are only possible for energy service contracts, following lobbying on behalf of that sector.  224  Project 'Strengthening the Use of Public Private Partnerships (PPPs) for Better Public Capital Investment Management Projects'  225  Can be used only for procurement in concession projects Chapter 5. Assessment of Public-Private Partnerships 121 for PPP projects. IFC is providing active support to bias and ensure that decisions are made on an the development of the PPP Agency's institutional objective basis. Most governments handle this by capacity. However, it has been impossible to assess keeping the two functions entirely separate. this capacity, as all documents prepared by the PPP Agency, including the project identification reports, 410. The Ministry of Finance (MoF) has the have been classified as 'confidential'. responsibility to assess the fiscal costs and risks that can arise from PPP projects, as well as the 409. The dual role of MoE as 'gatekeeper' and feasibility of providing governmental support. project promoter through the PPP Agency creates MoF also analyzes the impact of PPP/concession a conflict of interest. The motivations of a promoter project risks on the overall level of fiscal risk to the are to boost the attraction of a project to investors public finances. However, at the present time, MoF's and lenders, and this can lead to 'optimism bias'. 226 capacity in the field of assessing and controlling key Often, there are also strong political pressures to fiscal risks arising from PPP projects is insufficient to conclude transactions with a focus on speed, rather exercise these powers. It does not have a specialized than the quality of the transaction. The motivations of unit responsible for PPP projects and lacks the a 'gatekeeper' though are entirely different in that it is methodological principles for assessing fiscal costs tasked with quality checking claims and assumptions and risks from PPP contracts, which still need to be made by project promoters so as to rein in optimism developed. Table 26. Institutional Mapping of PPP Activities in Ukraine – Central Government Role/Major Tasks Comments Line Ministries (19), Central Government Bodies (42) À To prepare and screen concept notes for projects in relation to proposals under Limited in capacity and their management and decide on feasibility study preparation experience in PPP À To prepare PPP based proposals consisting of the feasibility studies and financial models À To conduct an analysis of efficiency of PPP proposals (including unsolicited proposals) and submit the conclusion on their results to the MoE À To take decision on PPP implementation (if approved by the MoE) À To establish a tender commission for PPP procurement À To engage advisors or independent experts for assisting in preparation of tender documentation and support of the tender commission's activity (if needed) À To organize the tender and select the winner À To negotiate a PPP/concession agreement with the winner À To sign the PPP/concession agreement as the public partner/grantor À To sign the direct agreement (if applicable) À To monitor the implementation of the PPP/concession agreement in which they are public partners/grantors  226  'Optimism bias' is the systematic tendency for project promoters and planners to over-estimate benefits and under-estimate costs of proposals. This is a behavioral characteristic that can be compounded by strategic misrepresentation. 122 Chapter 5. Assessment of Public-Private Partnerships Role/Major Tasks Comments Ministry of Infrastructure À Performs all the tasks outlined above for line ministries and central government Experience of bodies in the field of seaports and river ports, railways, airports, roads, and the preparation and postal service concluding two concession agreements in seaports and the most extensive portfolio of potential PPP projects Ministry of Economy À To take part in the assessment of PPP concept notes Inadequate capacity À To conduct jointly with other line-ministries the independent review of PPP and limited experience proposals, reach conclusions and approve (or refer) them with PPP À To monitor the implementation of the PPP/concession agreements and report on their results to the Ministry of Finance À To prepare and screen PPP concept notes for projects in relation to objects for which there is no state body designated by the Law to perform the necessary PPP management functions, and to take decision on the feasibility study preparation À To prepare PPP proposals consisting of feasibility studies and financial models for projects in relation to objects for which there is no state body designated by the Law to perform the necessary PPP management functions À To conduct an analysis of efficiency of PPP proposals (including unsolicited proposals) for projects in relation to objects for which there is no state body designated by the Law to perform necessary PPP management functions, and to submit the conclusion on their results to the Cabinet of Ministers À To delegate its representatives as members of PPP/concession tender commissions À To publish the results of PPP implementation, including the assessment and monitoring of the general level of risks for the public partner in PPP agreements À To monitor the effectiveness of public authorities in the field of PPP, to protect the legal rights and interests of public and private partners in PPP agreements Cabinet of Ministers of Ukraine À To decide on PPP implementation (or otherwise) for projects in relation to objects for which there is no state body designated by the Law to perform the necessary PPP functions À To decide on the implementation and further operation of public roads of state importance on the terms of the concession À To sign the concession agreement as the grantor or to define the public partner/ grantor for projects in relation to objects for which there is no state body designated by the Law to perform the necessary PPP functions À To take decision on the expediency of the concession and direct negotiations with the lessee of state property À To approve the conditions of concession tenders for projects of construction and further operation of public roads of state importance À To take decisions on government support for PPP/concession project implementation À To determine a body that conducts the final evaluation of the implementation of the PPP / concession Chapter 5. Assessment of Public-Private Partnerships 123 Role/Major Tasks Comments Ministry of Finance À To take part in approval of PPP concept notes Limited in capacity À To assess the fiscal consequences that may arise during the implementation of and experience in PPP/concession as a result of direct and indirect obligations of the public partner/ PPP; absence of grantor, the feasibility and the possibility of providing government support and to internal coordination approve the conclusion on the results of analysis of efficiency in the review of PPP À To delegate its representatives as members of PPP/concession tender commissions feasibility studies À To conduct a general assessment of fiscal risks, including PPP fiscal risk, based on the appropriate reports of the MoE PPP Agency À To identify PPP project ideas State enterprise under À To assist public partners in the preparation of identification reports, concept management of the notes, feasibility studies and tender documents MoE À To provide for capacity building À To seek potential investors and lenders for PPP/concession projects Project Office SPILNO À To support the activities of the Ministry of Infrastructure in preparation of Located in the Ministry concept notes and PPP proposals for seaports and roads and in providing of Infrastructure, monitoring for two seaport concessions in Kherson and Olvia seaports funded by donors Project Office 'Reform Support Team' À To support the activities of the Ministry of Infrastructure in preparation of Located in the Ministry concept notes and PPP proposals for seaports and railway station buildings of Infrastructure, funded by donors Anti-Monopoly Committee À To give consent to the signing of a PPP/concession agreement in cases provided for by law À To provide for analysis of compliance with the requirements of the legislation on protection of economic competition and legislation in the field of state aid in the process of approval of conclusions on results of analysis of efficiency of PPP implementation 411. Line ministries retain ownership and selecting private partners/concessionaires for the responsibility for PPP projects. They are responsible implementation of relevant projects. They also act as for: preparing and analyzing PPP proposals, the public partners/grantors under PPP/concession including USPs; approving decisions on PPP contracts.227 Ultimately though, it is the government as implementation; organizing and conducting tenders; a whole that bears the risk should things go wrong.  227  In accordance with the legislation, other public entities ( the Cabinet of Ministers of Ukraine, the National Academy of Sciences of Ukraine, national branch academies of sciences, or in case of a concession agreement and the CMU decision approval - state enterprises, institutions, organizations, companies, 100 percent of shares of those belong to the state or another business company, 100 percent of the shares (shares) of which belong to the state) also can participate in the PPP/concession projects as public partners/grantors under some conditions established by the Law. 124 Chapter 5. Assessment of Public-Private Partnerships 412. The most experienced ministry in PPP 415. The government's goals for PPP development is the MoI, which implements state implementation are not realistic or credible. The policy on seaports and river ports, railways, NSDGs set a target of concluding 45 PPP projects airports, roads, and the postal service. Currently, before 2030 (SDG 17), but from 2017 – 2021 only two this ministry has the most extensive portfolio of concession agreements were concluded. Additionally, potential PPP projects. The activity of MoI in the field in accordance with the previously mentioned of PPP is strongly supported by two donor-funded Cabinet Decree,231 46 PPP projects should have been Project Offices, SPILNO 228 and the Reform Support implemented by 2023, but to date PPP concept notes Team. On August 31, 2021, the MoI itself had a PPP have only been prepared for 6 of the 46. & Privatization Division consisting of three persons; however, this small team does not have the technical 416. A systematic approach to identifying priorities capacity to review, assess, give opinions on, direct and with PPP potential at an early stage is still missing. manage PPP projects in the manner set out in the legal Principles and a methodology for the selection of framework. priority investment projects identified by a Cabinet Decree in 2020,232 as well as for possible sources of their financing or funding (including PPP), are unknown. A methodology for selecting those to be implemented by PPP still has not been developed. 5.4. PPP: Assessments by Indicators 417. There is no national PPP project pipeline in key market-facing sectors such as health, energy and environment, making it difficult for 413. The following section gives the qualitative national and international investors to view assessment by selected indicator organized future opportunities. The first steps to applying a around the main PIM functions. program approach to PPP were made by the State Road Agency of Ukraine (Ukravtodor) at the end of 2020. Ukravtodor, together with the Ministry PIM Function 1. Guidance & of Infrastructure and the IFC, has prepared a PPP pipeline for the road sector and has begun its Screening implementation. This initiative should provide a beacon of good practice for other sectors to follow. PIM PPP-2. Strategic Plans & Investment Guidance, Project 418. PPP proposals must be screened for their compliance with the NSDGs and the strategic Development and Preliminary priorities of the country and industry. Potential PPP Screening projects must be screened using criteria which include the existence of suitable options, other than PPP, for 414. Sector analysis, forecasting and planning for solving problems; suitability for implementation as a the PPP implementation of infrastructure projects PPP; and (as with other investment proposals) socio- are still not practiced in MoE, MoF and in line economic and environmental consequences of project ministries of Ukraine. Strategic documents, such implementation. Additionally, demand for the services as the National Report 'Sustainable Development provided under the project and risks - potential cost Goals: Ukraine'229 and the National Economic Strategy and distribution between partners, including fiscal until 2030230 contain separate goals related to PPP/ risks – should be analyzed, as well as the project's concessions. At the same time, these goals are not efficiency compared with implementation without the accompanied by results from any analysis which involvement of a private partner. This requirement justifies the projected use of PPP implementation.  228  SPILNO' in Ukrainian means 'TOGETHER'  229  https://ukraine.un.org/uk/49413-2017-nacionalna-dopovid-cili-stalogo-rozvitku-ukraina  230  Cabinet Decree 'On approval of the National Economic Strategy for the period up to 2030' No. 179 dated 3 March 2021  231  Ibid  232  Cabinet Decree 'On adopting the list of priority investment projects for the state until 2023' No. 1581-r dated 16.12.2020 Chapter 5. Assessment of Public-Private Partnerships 125 is established through a Cabinet decree.233 For the none of the projects proposed for implementation period 2017 – 2021, only nine concept notes (three for as a PPP were subsequently recommended for seaports in 2017 and six for highways in 2021) and two implementation as a state investment project. feasibility studies relating to concessions for certain Additionally, only one of the projects submitted for state assets (ports) were prepared and approved. All consideration as a state investment project in 2019 these documents have been prepared by the MoI, with was subsequently recommended for implementation strong donor support (IFC and EBRD). However, the as PPP. This project was submitted by the Institute quality of this screening cannot be assessed since all of Neurosurgery of the National Academy of Medical of them have been defined by the MoI as confidential. Sciences of Ukraine. No further steps were taken to explore this possibility, and it has more recently been 419. PPP projects are not initially identified based re-submitted for implementation as a state investment on the results of an analysis of their potential project. The ability to switch implementation effectiveness234 nor is there any legislative methodologies, based on emerging evidence and provision for doing so.235 The only ministry that is assessment, is an important feature of a good PIM actively involved in initiation/implementation of PPP system and may need to be strengthened in Ukraine. projects, MoI, selects projects for the preparation of concept notes for PPP implementation mainly based on initiatives of the Ministry's Project Offices (RST and SPILNO). As reported by MoI, this is done by PIM Function 2. Formal considering a project's commercial attractiveness and Appraisal ability to attract financial resources for the preparation of a PPP proposal. PIM PPP-3. Formal Project Appraisal 420. Project profiles (concept notes) are required Procedures and Guidelines for projects proposed for PPP implementation, but screening does not appear to be rigorous. Despite 422. The roles and responsibilities of MoE, MoF, similarities in the format, the process is not integrated line ministries and central agencies in PPP projects with that of state investment projects.236 In practice, are clear (see Table 26). The appraisal process at none of the PPP concept notes prepared during all stages – concept note evaluation, preparation the last five years was rejected,237 including when a of conclusions on the result of the assessment of project was lacking strategic relevance or a convincing feasibility studies and financial models to analyze rationale for the project's sustainability. This efficiency, and approval of these conclusions by MoE undermines the primary purpose of pre-screening. and MoF – is also clear. Roles and responsibilities are defined in legislation238 and explained in the PPP 421. Preparation of concept notes leaves room Manual.239 for the possibility of changing the project implementation mechanism but in practice this 423. The PPP Manual provides a detailed rarely happens. Evaluation of concept notes provides methodology for the preparation of a feasibility an opportunity for considering the implementation study and financial model, as well as for their modality, even if processes differ for state investment appraisal.240 This manual covers financial appraisal, projects and PPPs. Notwithstanding this possibility, economic analysis – using social cost benefit analysis  233  Cabinet's Decree 'Some issues of organization of public-private partnership' Nr. 384 dated 11.04.11 with amendments  234  As an example of a prioritization approach, the World Bank's PPP Manual describes a methodological approach to prioritize PPP projects based on country / territory strategic development priorities and sector priorities. https://www.me.gov.ua/ Documents/Detail?lang=uk-UA&id=2bc79196-a3b2-41a9-86e2-f55f9a8f5c9c&title=MetodichniRekomendatsii  235  Cabinet's Decree 'Some issues of organization of public-private partnership' Nr. 384 dated 11.04.11 with amendments  236  As defined in Resolution No. 571.  237  In accordance with the legislation before taking decision on a PPP feasibility study preparation a concept note for the relevant project (for state assets) should be approved by the public authority-project initiator and MoE and MoF.  238  PPP Law and LoC  239  https://www.me.gov.ua/Documents/Detail?lang=uk-UA&id=2bc79196-a3b2-41a9-86e2-f55f9a8f5c9c&title=MetodichniReko mendatsii  240  See Part 2 and Annex to the PPP Manual https://www.me.gov.ua/Documents/Detail?lang=uk-UA&id=2bc79196-a3b2-41a9- 86e2-f55f9a8f5c9c&title=MetodichniRekomendatsii 126 Chapter 5. Assessment of Public-Private Partnerships or cost-effectiveness analysis – market analysis and for improving projects through consultation and risk assessment for different sectors, and includes discussion with interested (and often knowledgeable) the evaluation of environmental and social impacts, parties are being missed. and other potential approaches for project proposal preparation and evaluation. Particular attention is paid to further evaluation of the public value of the PIM PPP-4. Project Appraisal Capacity project, first identified in a concept note, as well as project risk assessment. In the concept note, only a 427. The quantity and quality of staff in central qualitative assessment of project risks is undertaken, agencies and key sector MDAs for overseeing and whilst in a feasibility study, all risks with significant conducting PPP project preparation and appraisal fiscal consequences should be assessed quantitatively. is not adequate for them to fulfill the roles assigned to them. Substantial capacity development 424. The depth and scale of the preparation and will be required before this is the case. There is a appraisal of PPP projects is not based on the size possibility of future support from a dedicated PPP or risk involved in the project. The legal framework agency created by MoE but, again, its capacity will provides for no specific approach to allow for reduced need to be strengthened before this becomes fully appraisal requirements for smaller projects: all PPP effective. The PPP division of MoE has positions for proposals should be assessed following the same only five people, of which two positions are currently procedure. The minor exception is for unsolicited vacant. There are also capacity issues in organizing proposals, which are exempt from the preparation the appraisal process for PPP proposals within MoF, of a concept note only, but in all other respects the as well as in line ministries and state agencies (except same procedures apply. As Ukraine does not yet have MoI and the State Agency on Energy Efficiency – as enough experience in PPP project preparation and explained below). implementation, it may be too early to think about differentiating, for example, by small projects or 428. MoF has the responsibility to assess the repeat projects (of which, there have been none, so fiscal consequences from PPP project proposals far), and this need not be seen as a serious weakness in relation to their fiscal risks and the feasibility at this stage. of providing any governmental financial support for such proposals.242 It also has the responsibility 425. The World Bank, IFC and EBRD are actively of analyzing the impact of PPP/concession project involved in the process of PPP development in risks on the overall level of fiscal risks which could Ukraine. They aid in preparation and evaluation of impact the public finances. However, at the present PPP projects as well as supporting the development of time, MoF has limited understanding of the subject PPP legislation, capacity building, and institutionalizing matter related to PPPs; no experience or capacity to the PPP process. However, donors work mostly with fulfill its role; and there are no internal procedures for MoE, MoF, MoI and the State Road Agency on potential reviewing PPP proposals within the ministry. 'user-pays' PPPs. PPP projects in health, education, energy, heating, environment, water supply and 429. MoF has no division responsible for the sanitation are not among their current priorities in analysis of PPP project proposals or for monitoring Ukraine. This is due to current legislative challenges in and control of implementation. Instead, the conducting so-called 'government-pays' PPPs.241 analysis of PPP project proposals is carried out by its specialized divisions according to a project's technical 426. The legislation does not require stakeholder area. The department of MoF in charge of monitoring consultation, except with the public authorities fiscal risks is not involved in the review of PPP concept involved in the appraisal process. Contrary to notes nor in the analysis of draft PPP / concession good practice, communication and engagement agreements. Only two introductory training sessions with civil society on a PPP project implementation on the PPP Manual have been conducted for the is not provided for. This means that opportunities ministry (by the World Bank in 2020-2021).  241  As opposed to 'user-pays' PPPs.  242  Cabinet's Decree 'On approval of the Methodology for assessing fiscal risk' Nr. 351 dated 24.04.2019 'and 'Some issues of organization of public-private partnership' Nr. 384 dated 11.04.11 with amendments Chapter 5. Assessment of Public-Private Partnerships 127 430. MoI, despite being the most competent 432. Training on PPP issues is fragmented, ministry on PPP issues, has a privatization and PPP uncoordinated and infrequent. Public authorities division consisting of only 3 people. Only the head at the central and local level need comprehensive of the division has been trained on PPP issues. Almost and holistic training programs on PPP issues. During all the tasks facing the Ministry in the field of PPP are 2020 and 2021 the World Bank and IFC organized carried out by the Project Offices, SPILNO and RST. training on PPP, mostly for the MoE, the MoF and the RST has 10 staff involved in PPP related activities and PPP Agency. However, training for high-level decision SPILNO has five. Projects prepared and supported makers in the identification, preparation and selection by these offices are allocated to them on a subjective of projects for PPP implementation is not frequent basis. The staff of these offices are regularly trained on enough to achieve a critical mass of skills at the PPP issues and supported by donors. They have been required level. involved in the preparation of two seaport concession projects and have a fairly good working knowledge in PPPs. In other ministries (except for the State PIM PPP-5. Screening of Feasibility Agency on Energy Efficiency), there are no divisions responsible for PPP issues; or with competencies in Studies this area. 433. Well-defined and documented processes 431. The Agency for Public-Private Partnership for feasibility study preparation, appraisal and Support (PPP Agency) was created in 2018 by screening exist, but the stringency of screening the MoE with the purpose of providing support raises questions. These processes are prescribed to public partners during the preparation and in the legislation and clarified in the PPP Manual. All implementation of PPP projects. Its main function concept notes and PPP proposals (feasibility studies) is to implement measures for boosting high-quality should undergo in-depth appraisal with formal and investor-attractive PPP projects in various decisions at key stages, including a decision to proceed sectors. The PPP Agency supports line ministries, with a feasibility study preparation. Since none of the agencies and local authorities in PPP proposal feasibility studies (2) prepared by public authorities in preparation and appraisal. The staff of this agency, the last five years was required to undergo redesign consisting approximately of 10 persons, are trained and/or reappraisal this may raise concerns about the and institutionally supported by IFC. The PPP Agency stringency of the screening. Among five unsolicited actively participates in the activities of the MoI proposals (USPs) submitted during the last five years, and local self-government bodies in the process of only one was rejected by the MoI; and decisions on identifying and preparing PPP project proposals, PPP implementation for four others were adopted, providing them with advice. The Agency prepares but with only one being eventually successful.244 All project identification reports on behalf of these PPP feasibility studies (USPs and those prepared by organizations, and actively cooperates with MoI's the public side) are confidential and cannot therefore project offices. 243 It is not possible to assess the quality be evaluated in this assessment, making it difficult to of the documents prepared by the PPP Agency's comment on their quality. employees, since all documents, including the project identification reports, are classified by the Agency as confidential. This lack of transparency is unusual compared to international comparators and could not be considered as good practice.  243  According to information provided by the PPP Agency, in 2019 it prepared 2 identification reports on airports; in 2020 - 62 identification reports on roads (12 of these projects were selected for further consideration) and 11 identification reports for projects in the city of Lviv; in 2021, until August - it prepared 11 identification reports for state and municipal property. It also assisted in the preparation of concept notes for 3 airports, 7 railroad stations, 6 roads and 2 seaports.  244  The winner was adopted in August 2019, but the PPP agreement still was not signed. 128 Chapter 5. Assessment of Public-Private Partnerships 434. There is no computer database that can PIM Function 4. Selection and be used by public authorities for managing and analyzing the data on PPP projects, including on Budgeting fiscal risks issues. Very limited information on PPP projects through their stages of design, appraisal, PIM PPP-7. Project Selection and approval and implementation, including on the Budgeting; and PIM PPP-8. Multi-year pipeline of project proposals, is openly available, Budgeting including on official sites of public authorities. 437. The PPP Law and Law on Concessions (LoC) provide the possibility of receiving government PIM Function 3. Appraisal financial support for implementation of PPP/ concession projects. This can be in the form of Review providing state guarantees and local guarantees; through funding from state or local budgets, or PIM PPP-6. Independent Review of other sources in accordance with national and Appraisal local programs; or by payments to the private partner as provided for in a PPP agreement, in 435. PPP legislation has clear requirements for the particular payments for operational readiness, or preparation of a PPP proposal and its subsequent for construction.245 However, Ukraine has so far had independent review. Following the appraisal of no experience of applying government support for a PPP proposal, a written conclusion should be PPP projects. Funding through budget programs is prepared by the proposer. A review of a PPP proposal not currently provided to line ministries to support and its conclusions by the MoE, the MoF and other PPP projects; neither is there any methodology or relevant line ministries is obligatory for PPP projects guidance on providing government support in the that involve state assets, even if they do not need framework of PPPs. government support at the expense of the state budget. Approval of the conclusion by the MoE is a 438. It is not presently possible to make necessary condition for adopting a decision on PPP 'government-pays' PPP projects bankable in project implementation. Ukraine. A draft law on amendments to the Budget Code of Ukraine provides for the possibility of long- 436. Although independent reviews do take place, term obligations in the framework of PPP contracts. due to capacity constraints, these reviews may Currently under consideration by the Parliament,246 not be effective in identifying issues that need the draft law would also establish limits on the scale to be addressed in projects before they proceed. of obligations and government support for PPP The current PPP/concession legislation does not projects. If the draft Law is adopted, it would be a provide for the possibility of independent technical significant move towards making socially orientated reviews by any entity external to the responsible projects bankable. Procedures for applying and public authorities. The quality of any review therefore approving government financial support should then relies entirely on the skills of the central oversight be developed, as well as an appropriate guideline. A entities, MoE and MoF, which have been shown to be computer database, with a long-term perspective under-resourced in terms of human capacities. These for budgeting of PPP projects, would also become capacity weaknesses strongly suggest that the review important in the practical application of the new system may not be as strong in practice as it is on funding regime. paper.  245  In the form of new construction, reconstruction, restoration, overhaul or technical re-equipment.  246  Recently adopted in the first reading. Chapter 5. Assessment of Public-Private Partnerships 129 PIM Function 5. personalized); scanned copies of the concluded concession agreements; applications of applicants Implementation for participation in the preliminary selection; and requirement for all tender proposals of bidders to be PIM PPP-15. Procurement published in the e-system for all concessions.249 This condition does not apply to non-concession PPPs. 439. Selection of private partners/concessionaires does not come within the scope of the Public Procurement Law. Instead, there are separate PIM PPP-16. Project Implementation procurement rules defined by the LoC for concessions and by the PPP Law and the relevant sub-law247 for Management non-concession PPPs. Competitive tender procedures 441. There is comprehensive guidance for apply to all procurements of private partners/ operational staff in MDAs on PPP project concessionaires, including those for unsolicited implementation;250 however, good practice in proposals which, as already indicated, are treated its application remains to be achieved. Only six the same as government-initiated proposals. These concession projects have been implemented to date. procedures correspond to good international practice Four of these were concluded before 2010 and two in all respects. They are clearly established and of these are out of operation and in the process of easily accessible to the public. Open competitive termination. The two other concession agreements, procurement is the principal procurement method for the seaports of Kherson and Olvia, were concluded for non-concession PPPs. For concessions, it is also more recently, in 2020, and are still in the transition possible to apply a competitive dialogue procedure, period. The management systems for these two but this has not yet happened in Ukraine. It is concessions are still in the formative stage and there obligatory to contract independent, experienced is no clear responsibility for their management within advisors for tender documentation preparation if the the MoI, as yet. Each of these projects is managed cost of the project is more than UAH 250 million. One by two large working groups each consisting of drawback of the legislation is that an independent representatives of the public and private partners.251 complaints procedure for handling PPP/concession procurement and for clearing appeals has not yet 442. Slow progress to date on the currently live been defined. concessions indicates that these arrangements may not be very efficient. Implementation plans for 440. The LoC also permits the e-procurement of these projects, including a detailed timeline, critical concessionaires for projects with investment path and key milestones linked to funding tranches, value less than UAH 250 million if the tender is are still absent nearly a year after the signing of the conducted without a pre-qualification stage.248 The agreements. International practice would indicate that CMU Decree on the use of electronic procurement all of these issues would already have been agreed for conducting concession tenders also established in principle as part of an implementation plan prior the comprehensive procedures under the LoC, to concluding the contracts. Delays such as these are covering: decisions and conclusions on admission / damaging to the investor climate and are very likely non-admission of bidders to participate in the tender; to deter further investment through concessions minutes of the meetings of the tender commissions; and PPP, unless they can be resolved in good time, requests for clarifications and elimination of consistent with international practice. violations in the process of the competition (not  247  Cabinet's Decree 'Some issues of organization of public-private partnership' Nr. 384 dated 11.04.11 with amendments.  248  Cabinet's Decree 'Some issues of using the electronic trading system for concession tenders' – Nr.1210 from 9.12.2020  249  Except for the information determined by the applicant as confidential and / or secret in accordance with the Law of Ukraine 'On Access to Public Information'  250  Manual on PPP, Phase 4  251  These Working Groups can consist of between 20 and 30 people 130 Chapter 5. Assessment of Public-Private Partnerships 443. Monitoring reports sent to MoE display are no requirements for submitting requests or a number of areas requiring significant seeking approval of proposed changes to the improvement. Those reviewed have no information relevant agreements by the MoE and the MoF concerning: performance indicators for the projects; before amendments are signed. The Concession the financial obligations of the concessionaires; and Law additionally has the requirement to publish analysis of the risks assigned to the concessionaires amendments to the concession agreement within in the framework of monitoring. Taking performance five days from the date of their introduction in the indicators as an example, monitoring becomes Electronic Trading System.255 impossible when the desired levels of performance, against which actual performance should be compared, are not provided. PIM Function 7. Service Delivery PIM Function 6. Adjustment PIM PPP-20. Control, Monitoring PIM PPP-19. Project Adjustment and Reporting: Financial and Service Delivery Performance 444. According to the legislation,252 the procedure for amending a PPP / concession agreement, is one 446. In 2020, MoE established a comprehensive of the essential components of an agreement. 253 procedure for submission of annual reports on the A PPP / concession agreement may be amended by operational implementation of PPP/concession its parties or at the request of one of the parties on agreements by the public partners (grantors). the basis of a court or arbitration decision in case of This procedure, the development of which has been significant breach by the other party of obligations supported by the World Bank, required detailed under such agreement, or in case of significant change reports on PPP/concession agreements covering all of circumstances, which guided the parties in the actual performance results.256 The first reports in this process of concluding a PPP agreement. Additionally, procedure have demonstrated a need to strengthen the Concession Law establishes a prohibition for the the conceptual and practical understanding of how parties to change any of the terms of a concession such reports should be prepared. The reports are agreement which were included in the bid and high-level and formalistic in their approach to the became the basis for determining the bidder as the provision of information. As a result, the information winner. – which covered neither service delivery performance indicators nor evaluation of authority risks – is of 445. The same legislation obliges contracting limited usefulness for understanding the operational authorities to inform the MoE about amendments performance of PPPs. of a PPP / concession agreement within a year after such changes.254 However, there  252  Concession Law and CMU Decree 'Some issues of organization of public-private partnership' Nr.384 dated 11.04.2011 with amendments  253  Including in accordance with the Step-in-Rights procedure if a direct agreement is concluded for the implementation of a PPP / concession project,  254  In case of amendments to a PPP / concession agreement during the reporting period, the state partner / grantor must provide for to the MoE certified copies of additional agreements with such amendments together with the annual report (MoE Order 'About the statement of the Procedure for submission by the state partners (grantors) of the annual report on performance of an agreement concluded in the framework of the public-private partnership, including a concession agreement' No. 986 dated 26.05.2020).  255  At the moment the Electronic Trade System is not put into operation. The 'Electronic Trading System' is an e-procurement system that should be developed for organizing e-procurement for concessions in accordance with the Concession Law (this term defined in the Concession Law). It is not the same as the general e-procurement system, PROZORRO.  256  MoE Order 'On approval of the procedure for submission by the state partners (grantors) of the annual report on the implementation of the agreement concluded in the framework of the public-private partnership, including the concession agreement' – No. 986 from 26.05.2020 Chapter 5. Assessment of Public-Private Partnerships 131 PIM Function 8. Evaluation 3. The MoF needs to develop regulations, good practices and capacity to assess fiscal risks arising from PPP contracts ex-ante, and to monitor, report PIM PPP-22. Scope, Nature and and disclose them ex-post. Follow-up of External Audit and 4. The MoE should strengthen regulation concerning Ex-Post Evaluation the amendment of PPP / concession agreements, so that proposed adjustments to PPP/concession 447. The PPP Law and LoC both require an ex-post agreements become the subject of oversight evaluation for all PPP/concession agreements and approval by MoE and MoF; and extend the regardless of their size. However, due to inadequate requirement of the Concession Law to publish capacity for PPP in Ukraine no such evaluations have amendments to all PPP agreements within 5 days been provided so far. There is no methodology or after their approval for non-concession PPPs. guidance on how to conduct ex-post evaluations. 5. In the event that the draft Budget Code is 448. The Accounting Chamber of Ukraine (ACU) has amended to allow multi-year commitments, the not conducted any audits on PPPs to date. This is MoE should develop a small pipeline of pilot due to the absence of any projects that draw directly projects of 'government-pays' projects to test on the budget. If the draft law on amendments to the them on the investor / lender market and to learn Budget Code of Ukraine is passed, it would provide critical lessons in preparing and procuring them. for the possibility of long-term obligations in PPP 6. Transparency in the process of developing and contracts. This would likely lead to future PPP projects appraising the PPP option should be dramatically coming under the remit of the ACU. increased with the 'confidential' status being reserved only for the commercially sensitive information of bidders. 7. A coherent and endorsed strategy for the use of 5.5 Recommendations private finance in the delivery of public service Specific to PPP (PPP in the broadest definition) should be developed. All public investment projects, including PPPs, 1. should be appraised first for basic need and public value to the nation prior to decisions being taken about the form of financing. 2. The MoE should decide whether to retain the role of promoter or regulator in order to avoid the current conflict of interest. 132 Chapter 5. Assessment of Public-Private Partnerships R 6 PTE CHA ASSESSMENT OF STATE-OWNED ENTERPRISES 6.1. Public Investment and 450. Capital investments implemented by SOEs were on the rise until 2019 and reached UAH Institutional Mapping in 82 billion or 2 percent of GDP, with the largest source of funding from SOEs' own funds and Ukraine credit facilities. SOEs invest in core infrastructure projects such as energy networks, railways, seaports, Public Investments and defense systems. As demonstrated in Figure 6, the largest source of funding for capital investment Implemented by SOEs comes from their own funds, i.e., accumulated profits, funding between 64 percent and 82 percent of all 449. Infrastructure sectors in Ukraine are their capital investments. The second largest source dominated by state-owned enterprises (SOEs) is borrowings—commercial banks, securities market which historically implement significant capital placements, IFIs—financing between 7-15 percent of investments in public infrastructure. SOEs that all capital investments. The share of budget funding operate as natural monopolies are typically found in for such investments has been low – around 1-2 the energy and transport sectors but are also present percent in 2016-2018, increasing to 8 percent in 2020. in other sectors of the economy. SOEs with market shares exceeding 50 percent are identified in at least 15 of 28 subsectors and markets in which they operate.257  257  WB (2019) Reducing Market Distortions for a More Prosperous Ukraine. Chapter 6. Assessment of State-Owned Enterprises 133 Figure 6. SOE Capital Investments by Source of Funding, UAH million 90,000 80,000 Total: 82,063 9,927 70,000 3,825 4,233 Total: 72,153 Other sources 7,496 9,005 60,000 Budget financing Total: 53,309 50,000 4,437 4,775 Total: 47,711 6,443 Attracted funds 40,000 3,281 Total: 40,714 8,732 30,000 Own funds 5,499 59,185 58,848 20,000 38,653 40,448 26,240 10,000 0 2016 2017 2018 2019 2020 Source: Treasury, WB Staff Calculations 451. Large SOEs do not rely on budget funding for accumulated profits and various types of borrowings. their capital investments. If analyzed by SOE size, Smaller SOEs receive budget funding for their capital natural monopolies and SOEs with profits exceeding investments; this trend increased from 2019-2020 UAH 50 million fund their capital investments via (Figure 7). Figure 7. Capital Investments by Source of Funding with Breakdown by SOE size, UAH million 100 1,883 3,411 4,699 9,266 4,431 90 8,577 4,704 All Other SOEs: Other sources 4,230 4,212 3,811 80 7,387 4,499 All Other SOEs: Budget financing 3,115 3,278 6,440 70 9,005 All Other SOEs: Attracted funds 5,323 60 All Other SOEs: Own funds 50 Large SOEs: Other sources 40 34,842 54,481 54,349 36,236 Large SOEs: Budget financing 30 22,962 Large SOEs: Attracted funds 20 10 Large SOEs: Own funds 0 2016 2017 2018 2019 2020 Source: Treasury, WB Staff Calculations 134 Chapter 6. Assessment of State-Owned Enterprises Figure 8. State Guarantees Issued to SOEs by Sector, UAH million 20,000 18,000 16,000 18,923 2016 14,000 15,127 2017 12,000 2018 10,000 8,000 2019 6,000 6,965 2020 4,000 5,035 1,020 2,980 2,928 2,000 43 — — — — — — — 0 Energy Sector SOEs Transport SOEs Defence SOEs Source: Treasury, WB Staff Calculations 452. SOEs' investment potential largely depends billion (2020), with the energy sector dominating for on the amount of profits they can retain, with both internal and external guarantees. However, only a balanced and predictable dividend policy part of these state guarantees is directed towards forming a foundation for proper planning and capital investments. For example, SOEs' capital implementation of their investment plans. SOEs investment projects guaranteed by the state in 2020 in Ukraine have complicated and non-formalized accounted for only UAH 7.9 billion of the total UAH relationships with the central budget, which 18.1 billion (Figure 8), whereas UAH 10.2 billion were undermine the companies' operational efficiency. provided to PJSC Ukrenergo to cover its arrears to When fiscal pressures are low, SOEs are held suppliers. less accountable for their financial inefficiency; but during times of fiscal stress, they are usually requested to transfer more profits to the budget. This approach impacts SOEs' ability to properly plan their investments and sustain their investment 6.2. Assessment of State- commitments, leading to productivity losses. Owned Enterprises 453. Borrowed funds are the second largest source for SOEs capital investments, funding between 7 Introduction and the Current percent and 15 percent in 2016-2020, with some of Situation these loans being guaranteed by the government. SOEs borrow funds from local commercial banks, they 454. SOEs implement a large share of public sector issue bonds locally and internationally, and they also capital expenditure in Ukraine – above 30 percent take long-term loans from the IFIs – directly or via in 2018-2019 and 24 percent in 2020. SOEs have on-lending through the state budget. Some of these normally been the second largest source of public borrowings are provided directly to SOEs at their own sector capital expenditure, representing around credit risk, and sometimes the government provides a third of the total in 2018 and 2019, but this was state guarantees. During 2016-2020, total state reversed in 2020 when central government capital guarantees issued to SOEs and state-owned banks expenditure rose to 31.9 per cent of the total and SOE fluctuated between UAH 5 billion (2019) and UAH 23.8 spending fell to 24.0 per cent. This significant shift is Chapter 6. Assessment of State-Owned Enterprises 135 most likely related to effects of COVID-19 measures procurement legislation, which provides a coherent and the initiation of the Great Construction project,258 framework for goods and services procured during both of which were funded from the central budget in the implementation process, except for projects 2020. financed by the IFIs which follow their own dedicated procedures. 455. SOEs fund their capital investments from several sources, including their own funds, borrowings, and to a lesser extent, from budget funding. Budget funding for capital expenditure of Methodological Approach projects implemented by SOEs had been historically low – around 1-2 percent in 2016-2018 – but grew 457. The assessment of SOEs' capital investment to 8 percent in 2020. Most of such projects are practices in this report applies elements of the implemented by medium size SOEs, while large SOEs IMF PIMA Framework's Indicator 5c, alongside the that are natural monopolies and those with profits indicators in the World Bank's 2015 framework. exceeding UAH 50 million, rely on their own funds and Indicator 5c allows a broad assessment of SOE borrowings. SOEs' capital and retained earnings are governance through a single question: Does the the main sources of funding for capital expenditure, government oversee the investment plans of public providing between 64 percent to 82 percent of corporations and monitor their financial performance? all funding needs. The second largest source is To answer this question, the assessment investigates borrowings—commercial banks, securities market SOEs' investment plans and their implementation placements, IFIs—financing between 7-15 percent of across all funding sources, including those that are all capital expenditure. funded from the state budget, as well as those funded by loans and own funds of SOEs. The summarized 456. Depending on the source of funding for capital answer to this question is presented in Table 27. investments, different legislative procedures and approval routes are applied. Budget-funded capital 458. Extending this analysis, this chapter follows expenditure follows the SSIP cycle, as described in a similar approach to the main assessment, detail earlier in this assessment report; however, the considering the various aspects of budget funded share of such funding is insignificant. In addition to the capital investments and the specifics of SOE SSIP cycle, SOEs can be financed by means of direct investments made outside the budget. It analyzes funding from the state budget within various budget the legislative framework for the financial and programs, thus bypassing the SSIP cycle. Capital investment planning of SOEs and its actual application, expenditure funded from SOEs' own and borrowed the effectiveness of the institutional setup, covering resources fall under different legislation and approval the roles and responsibilities of various agencies and procedures, including separate sets of requirements SOE management; implementation and monitoring for IFI funding,259 and borrowing with state of investment projects; evaluation of projects; guarantees.260 The variety of these cycles correspond and central government oversight of the financial to the specific requirements of the funding source and performance of SOEs. The analysis investigates the are based on a robust approval process. All of them similarities and differences of these metrics, looking at are covered in this chapter. Importantly, all capital how SOEs manage investment projects regardless of expenditure implemented by SOEs—own and budget funding source. funded—fall under the requirements of the public  258  Great Construction is the national project initiated by President Volodymyr Zelensky in autumn 2019 with the main purpose to build and renovate the necessary social, transport infrastructure throughout Ukraine.  259  Resolution of the CMU No. 70 as of 27.01.2016 On preparation, implementation, carrying out of monitoring and completion of implementation of projects of economic and social development of Ukraine supported by the international financial organizations.  260  Resolution of CMU No. 835 as of 13.11.2013 on the Procedure on Selection of investment projects, for implementation of which state support is provided and Resolution of CMU No. 809 as of 15.06.2011 On approval of the Procedure for approval of attraction of loans, providing guarantees or sureties for such obligations by state-owned enterprises, including business associations (except banks), in the authorized capital of which 50 percent or more shares are owned by the state. 136 Chapter 6. Assessment of State-Owned Enterprises Table 27. Summary of Indicator 5c Indicator/ Dimension Brief Description PIM-SOE. Central There is a legislative framework in place for the government to review and approve Government oversight financial plans of SOEs, which also include capital investment plans. This assessment of state-owned concludes that in principle the government follows this procedure, although in enterprise (SOE) practice very few SOEs include capital investment plans in their financial plans. investment plans and Hence, the government cannot review and approve the entirety of SOEs' investment financial performance plans financed from all sources: state budget, own, or borrowed funds. The government does not prepare or publish an aggregate report on SOE performance. This practice ceased to exist with the last report published for 2018. At the same, consolidated information on SOEs' capital investment plans is made publicly available by the MoE annually. This published information covers all funding sources for capital investment by SOEs, including the state budget, own funds, and borrowings. Legal Framework investment projects following Resolution No. 571 procedures become subject to a rigid application of the SSIP procedures and regulations, and there 459. The legislative framework for capital are no exceptions noted in this process. However, investment by SOEs varies depending on the the assessment could not confirm the same for the funding source. The main legislation for the capital investment projects implemented by SOEs and budget-funded investment is the Law on Investment financed under the budget programs. Activities,261 Resolution No. 835262 on provision of state aid for investment projects, and Resolution No. 571263 461. For capital investments funded by borrowed that establishes guidelines for all public investment funds, SOEs are required to follow specific projects, including their appraisal and selection legislation depending on the source. If an SOE procedures. In addition, SOEs follow their sectoral would like to borrow funds, it must follow: (i) legislation that sets out the strategic priorities for Resolution No. 70 for IFI financing; (ii) Resolution No. the sector, forms the basis for all capital investment 835 for loans from the state budget or loans from projects, regardless of their funding source, and other creditors (non-IFI's), or those to be covered outlines possible sources of funding. Energy sector by state guarantees; or (ii) Resolution No. 809 for SOEs, for example, follow the Law on the Energy approval of state guarantees.265 Market,264 as the main strategy setting document. 462. Capital investments implemented using 460. Public investments implemented under the SOEs' own funds follow Order No. 205 on financial SSIP framework under Resolution No. 571 follow planning for SOEs.266 Order No. 205 establishes the the robust practices, but this is not the case detailed process for financial planning, prioritization for capital investments implemented by SOEs of expenses and the approval procedure, including under broader budget programs. The current for investment plans, among others. Order No. 205 assessment confirmed that budget-funded public  261  Law on Investment Activities (1991) with changes and amendments.  262  Resolution of CMU No. 835 as of 13.11.2013 On approval of Procedure for selection of investment projects, for implementation of which state aid is provided.  263  Resolution of the CMU No. 571 as of 22.07.2015 on Some issues of public investment management  264  Law on Energy Market (2017), with changes and amendments.  265  In case of attraction of loans with state guarantees following Resolution No. 835 or Resolution No. 70 with state guarantees of MoF, Resolution No. 809 must be applied as well for approval of provision of guarantees.  266  Order of MoE No. 205 as of 02.03.2015 On approval of the Order on preparation, approval, and control of execution of the financial plan of an entity of public sector of economy. Chapter 6. Assessment of State-Owned Enterprises 137 also offers a pro-forma for financial plans but lacks preceding the implementation year, except for an example of a pro-forma investment plan. If an SOE SOEs regulated by the National Commission for decides to use own funds (i.e., accumulated profits) State Regulation of Energy and Utilities, the plans to fund capital investments, these must be reflected for such SOEs must be approved by December 31st; in the financial plan and approved in accordance with and the established procedure. The approval process for À Once approved, MDAs/ownership agencies financial plans is clear and established as follows: submit consolidated data on SOEs' financial (and À Based on the mid-term sectoral strategy and the investment) plans to the MoE, the MoF, and the SOE's own investment plan (if available), annual State Tax Service. financial (and investment) plans are drafted by the senior management with a detailed list of projects 463. For large SOEs – natural monopolies and to be funded and their proposed funding sources; those with profits exceeding UAH 50 million per À For those SOEs that have an established year – the procedure differs, as their financial (and Supervisory Board and respective provision in their investment) plans must be signed off by the CMU. charter, the financial (and investment) plan has All the above referenced procedures apply, however, to be approved by the Supervisory Board before in addition, to their ownership agency or ministry, MoF proceeding further;267 and MoE must approve and send for further approval by CMU. Moreover, specific procedure applies to JSC À A draft annual financial (and investment) plan is Ukrzaliznytsia (JSC Ukrainian Railways), whose plans submitted to the ownership agency or responsible must also be approved by MoI. All these central level ministry for approval by June 1st of the year ministries – MoE, MoF, MoI – must analyse SOEs' preceding the implementation year; draft financial (and investment) plans within 10 days À Ownership agencies collect and submit and decide on approval or rejection. Following their consolidated indicators of SOEs' financial plans decision, the CMU takes its own decision on approval to the MoF by July 15 , so they can be considered th of a financial (investment) plan. Once approved by during the state budget preparation; the CMU, the ownership agency or SOE sends the approved financial (investment) plan to MoE, MoF, and À Final approval of the financial (and investment) State Tax Service. Responsibility and timeline for this plans must be completed by the ownership process is summarized in Table 28. agency or ministry by September 1st of the year Table 28. Summary of Responsibility and Timeline over Large SOEs' Financial and Investment Plans Type of SOE Financial Plan Approved by Approval Timeline Natural monopolies with profits exceeding UAH 50 MoE and MoF July 1st million per year Owned directly by and reporting to the CMU MoE June 1st Operating in the power (electricity) sector as natural MoE and MoF November 1st monopolies, with expected profits exceeding UAH 50 million per year, whose activities are regulated by the National Commission for State Regulation of Energy and Utilities  267  Establishment of a Supervisory Board is only obligatory for certain types of SOEs, according to criteria defined by GoU (Resolution No. 142 of CMU). Even for those SOEs, where their financial plans have to be approved by their Supervisory Boards, this practice is limited to those enterprises that have legislated such a body in their organisational structure. In practice, this norm is yet to be implemented at most SOEs (beyond the existing nine), as its implementation depends on the establishment of a board. 138 Chapter 6. Assessment of State-Owned Enterprises Type of SOE Financial Plan Approved by Approval Timeline Operating in the power (electricity) sector, owned by MoE November 1st and reporting to CMU, whose activities are regulated by the National Commission for State Regulation of Energy and Utilities Special Case: JSC Ukrzaliznytsia (JSC Ukrainian Railways) MoI June 1st Source: WB staff compilation of the legislation framework 464. Capital investments implemented via approve the proposed project on the basis of detailed borrowed funds must follow the same procedures information, including a draft of the loan agreement. for financial plans as established under Order No. Once the project is approved, with the loan received 205, but with additional requirements of the IFIs. or bonds issued, the SOE must provide a copy of the In addition to the procedures described above, SOEs signed agreement to the MoF for further registration. must adhere to Resolution No. 809 268 in cases where Any changes to loan agreements or conditions of they are looking to attract mid- and long-term loans a bond placement must be approved by the MoF. or place bonds (with or without state guarantees), to Resolution No. 835 sets out the rules for competitive Resolution No. 70269 in the case where they attract selection of investment projects with the involvement financing from IFIs, or Resolution No. 835 in case of of MoE and MoF for analysis of project economic seeking loans from the state budget, compensation of efficiency and fiscal risks. The main legislative routes interest rates, or state guarantees. In cases SOE seeks to approve SOE capital investments are presented funding according to Resolution No. 809 or No. 70, below in Figure 9. the MoF must be notified in advance, and receive and Figure 9. Legislative Procedures for SOE Capital Investments FUNDING BUDGET FUNDS SOE OWN FUNDS BORROWED FUNDS SOURCE À Law on Investment In addition to procedures Activities À Sectoral Laws and for own funds: LEGISLATIVE À Sectoral Laws and Strategies PROCEDURES À Resolution No. 809 Strategies À Order No. 205 À Resolution No. 70 (IFIs) À Resolution No. 571 À SOE Charters À Resolution No. 835 À Resolution No. 835 Source: World Bank mission interviews, author's presentation  268  Resolution of CMU No. 809 as of 15.06.2011 On approval of the Procedure for approval of attraction of loans, providing guarantees or sureties for such obligations by state-owned enterprises, including business associations (except banks), in the authorized capital of which 50 percent or more shares are owned by the state.  269  Resolution of the CMU No. 70 as of 27.01.2016 On preparation, implementation, carrying out of monitoring and completion of implementation of projects of economic and social development of Ukraine supported by the international financial organizations. Chapter 6. Assessment of State-Owned Enterprises 139 Institutional Arrangements 466. Overlapping functions and limited capacity are the main constraints in both monitoring of capital investments implemented by SOEs, 465. As is the case for all public investment and oversight of their financial performance, projects, capital investments by SOEs are subject including implementation of investment plans. to a multi-layer institutional approval process. The development of capital investment projects, their Some of the institutional functions overlap, causing processing, and approval by different routes—while delays in the approval and implementation process complex—is clearly laid out in the existing legislation. (as also noted in 2019 IMF PIMA assessment), and The only exception is the capital investments some are not followed up consistently in practice. implemented by SOEs under special budget Institutional responsibilities are presented in Table 29. programs, as detailed above and discussed in the recommendations section. Table 29. Institutional Responsibilities over SOE Capital Investments – Central Government Role/Major Tasks Comments SOEs À Prepare annual investment plans based on long-term sectoral strategies (10 Varies by sector and by years) or own mid-term investment strategies (3-5 years) entity À Finalize financial and investment plans (responsibility: senior management) À Approve, if established and empowered by charter (responsibility: supervisory boards) À Monitor the implementation of the approved projects (quality and financial dimensions) À Report on project implementation status to their ownership agencies/ ministries and MoE Ownership agencies / ministries À Develop and approve sectoral strategies and authorize mid-terms investment Limited resources plans for SOEs under their mandate to allow for timely approval of financial / À Prioritize and authorize individual capital investment projects for SOEs under investment plans and their mandate quality implementation À Authorize annual financial plans and investment plans, including individual monitoring projects, for SOEs under their mandate Limited resources to À Comment, revise, correct SOEs' financial plans and investment plans, as pro-actively engage necessary, to authorize for further processing on monitoring and À Monitor the implementation of the approved projects and investment plans for assessment of SOEs' SOEs under their mandate (quality and financial dimensions) financial performance À Monitor financial performance of SOEs under their mandate Sector regulators À Authorize investment plans for SOEs operating in regulated sectors (i.e., energy) N/A 140 Chapter 6. Assessment of State-Owned Enterprises Role/Major Tasks Comments MoF À Take part in approval of SOE financial and investment plans – annual plans and Limited connectivity individual projects with SOE ownership À Assess fiscal implications of proposed SOE investment projects agencies/ ministries À Authorize SOE capital investment projects that involve budget funding, during the annual borrowings and / or state guarantees assessment À Monitor the implementation of approved projects for all SOEs (financial dimension only) À Conduct a general assessment of fiscal risks, emanating from operations of SOEs À Identifies and assesses risks for projects following Resolution No. 835 MoE À Take part in the approval of SOEs' capital investment projects for entities under Insufficient capacity their mandate and limited resources À Monitor the implementation of the approved projects for all SOEs (financial to pro-actively engage dimension only) on monitoring and À Prepare and publish the annual aggregated results of implementation progress assessment of financial of financial and investment plans of SOEs performance of SOEs À Collect and publish financial performance data for SOEs on the public web The MoE no longer portal (ProZvit) produces an aggregated À Assesses economic efficiency of projects following Resolution No. 835 report on SOE À Perform independent review of appraisal of projects following Resolution performance (last one No. 571 dated 2018) Cabinet of Ministers of Ukraine À Approve financial (and investment) plans for SOEs under their direct mandate, N/A as well as those operating in regulated or monopoly sectors, for which profits exceed UAH 50 million State Audit Service À Assess the effectiveness, legality, targeting, and efficient use of budget funds, N/A with the aim of achieving budget savings Accounting Chamber of Ukraine À Perform audit of the use of state budget funds by SOEs (this includes N/A investment projects financed by the state budget and by IFIs) À Perform audit of procurement financed by the state budget funds Antimonopoly Committee of Ukraine À Takes decisions on whether state aid (in particular, in the form of state N/A guarantees) is admissible and issues relevant conclusions. Source: World Bank mission interviews, author's presentation Chapter 6. Assessment of State-Owned Enterprises 141 SOEs: Assessments by PIM 469. All sectoral SOEs develop their own mid- and long-term strategic development plans based Must-Have Functions on their respective sectoral strategies. Each SOE must develop mid- and long-term strategic 467. The assessment below of capital investments development plans and approve them with their processes and procedures for SOEs follows the ownership agency or respective line ministry. Such structure of the main assessment. To provide plans must be based on sectoral strategies and a certain degree of comparability with other PIM can be updated annually on a rolling basis. The dimensions in this report, the assessment is organized government has developed respective methodological around the eight "must have" functions of the WB recommendations271 to help SOEs develop their own PIM framework, rather than the more in-depth 23 PIM strategic development plans. These plans should indicators.270 contain general information about the SOE, current analysis of its operational activities, strategic mission and objectives, expected results, and conclusions. The application of SWOT analysis is advised for the existing PIM Function 1. Guidance and and planned activities implemented by SOEs. Such Screening recommendations are optional and are a helpful tool to guide SOEs in developing a comprehensive multi- 468. Ukrainian SOEs follow the applicable sectoral year strategic planning document. strategies and develop their own mid and long- term strategic plans of capital investments. 470. Based on the sectoral strategies and their Sectoral strategies are developed and approved by own mid- and long-term strategic development respective line ministries in their sectors of operation. plans, SOEs prepare annual financial plans, which All main sectors have such strategies in place – for also contain investment plans. Such annual plans example, the Energy Strategy of Ukraine through must be directly linked to their own investment plans 2035, the 10-year Development Plan for the Electric (typically for 3-5 years). This assessment could not, Power Transmission Grid, and the National Transport however, locate and verify such mid-term investment Strategy through 2030. plans as published by the SOEs. As there is no legislative requirement for such publication, SOEs refrain from making this information public. PIM Must-Have Functions: PIM Function 1. Guidance and Screening Must-have PIM function Good policy treatment272 What is analysed 1. Investment Guidance Regulated sectoral SOEs: Treat as part of sector- The basis for the planning specific planning and strategy. Justify public of investment projects by support if used. SOEs, linkages to sectoral strategies, mid- and long- Non-regulated SOEs – own funds: SOE develops term plans, and the use own strategy and guidance. of such strategies and Non-regulated SOEs – budget funds or state plans as guidance for guarantees: Treat as part of sector strategies. project development and Justify budget funding/ subsidy/ guarantee. screening.  270  It will be remembered that the PIM indicators are already grouped according to the eight "must-have' functions, which were originally described in Policy Research Working Paper 5397, A Diagnostic Framework for Assessing Public Investment Management, Anand Rajaram, Tuan Minh Le, Nataliya Biletska, Jim Brumby. WB. August 2010.  271  Ministry of the Economic Development and Trade of Ukraine, Order No. 971 dates August 14, 2013 (Methodological Recommendations).  272  World Bank (2021): Assessing Public Investment Management Functions and Institutional Arrangements for State-Owned Enterprises. A Diagnostic Framework. 142 Chapter 6. Assessment of State-Owned Enterprises 471. Overall, it may be concluded that the or borrowed funds will be included in the financial existing system of SOE investment guidance and and investment plan in consultation and agreement preliminary screening are structured properly in with their ownership agency/line ministry. A project Ukraine. All SOEs—regulated and non-regulated— appraisal typically covers the main metrics of the must follow their sectoral strategies, develop their proposed investment, such as the payback period, own mid- and long-term strategic development plans net present value, internal rate of return, and other that are approved internally, then by sector regulators referenced indicators. Such appraisal is carried out (for regulated sectors), and after by ownership by the SOE itself or with the engagement of external agencies or line ministries. Potential investment consultants, as needed. projects included in such mid- and long-term strategic development plans undergo preliminary screening 473. If an SOE seeks budget funding for a capital and prioritization at all levels of this process, ready investment project, it must adhere to public to be substantiated and transposed into the detailed investment procedures established by Resolution investment plans. There seems to be a reasonable No. 571 or Resolution No. 835. Resolution No. connection between high level sectoral objectives and 571 envisages development of a project concept entity level investment plans. note and a project proposal. In such cases, a line ministry plays an important role in taking the project concept through the approval process ending in the selection stage by the Inter-Agency Commission. More PIM Functions 2 and 3. Project details on this procedure can be found in Chapter 3 Appraisal and Independent Review of this report (PIMs 1 – 6, 16 – 23). SOEs can also receive budget funding for implementation of public 472. The procedures for project assessment vary investment projects by applying for state aid following according to the source of funding. Financial and Resolution No. 835. To receive such funding, SOEs investment plans prepared by SOEs (as discussed must develop a project proposal, which is like the one above under Legal Framework) must contain under Resolution No. 571, but does not require CBA or information on the source of funding. Capital CEA.273 investment projects to be implemented with own PIM Must-Have Functions: PIM Function 2 and 3. Project Appraisal and Independent Review Must-have PIM function Good policy treatment What is analysed 2. Project Appraisal All SOEs – own funds: Conduct financial and risk How the SOE capital analysis. Additional requirements may apply in investment projects are case of major environmental or social risks, or for appraised and analysed. mega projects. All SOEs – budget funds or state guarantees: Conduct full integrated analysis. 3. Independent Review of Listed SOEs: Comply with market disclosure If and how the SOE the Appraisal requirements applicable to listed entities. capital investments are independently reviewed. All SOEs: All mega projects to be approved by line ministries, MoF, MoE and, ideally, by agencies with independent experts.  273  Order of Ministry of Economy No. 724 as of 19.06.2012 On approval of the form of the project (investment) proposal, based on which the investment project is prepared, for the development of which state support may be provided, the Procedure for development and the form of the investment project for the implementation of which state support may be provided. Chapter 6. Assessment of State-Owned Enterprises 143 474. Independent review of appraisal is required with their financial plan, not all SOEs submit it for for public investments financed from the state government approval and monitoring. The MoE, which budget, with SOEs own funds and loans with state is engaged in reviewing all financial plans from SOEs, guarantees. Independent review is required by observes that approximately half of the submitted Resolution No. 571, Resolution No. 835, and the Law financial plans do not include an investment plan on Investment Activities. 274 As described in Chapter 3, component. There is no pro-forma template, for the Resolution No. 571 requires "state expert review" investment plans, so each SOE prepares one based to be overseen by MDAs, which may create a special on their understanding and capacity. As a result, commission for this purpose. Members of such some investment plans remain general, containing commission should be entirely independent of the only basic information on planned investments, project under review. The findings of this commission while others include complete specification per form the basis for the MDA's opinion on the project investment projects, covering detailed calculations preparedness and recommendations for next steps. and projections. This variation creates a certain degree Resolution No. 835 and the Law on Investment of complication for approvers and becomes one of the Activities require the state expert review by MDAs as reasons for delayed approval of the plans. well, and it has a similar form and methodology as the state expert review according to Resolution No. 571.275 476. Approval of SOE financial (and investment) For capital investment projects implemented with plans must be completed by September of borrowed financing without state guarantees, the the planning year and is sometimes delayed, current assessment has not identified any legislative undermining implementation of ongoing and requirements for an independent review of appraisals planned investment projects. The ownership of capital investment projects. entity is required to analyze financial plans from SOEs within 10 days of receipt, compare them with previous periods, and approve or return them to the SOE for improvement, if necessary. The final approval PIM Function 4. Project Selection and of financial plans by the ownership entity has to be Budgeting completed by September 1st or by December 31st for electric power enterprises licensed by the National 475. This assessment has identified that, in practice, Commission for State Regulation in the Fields of some SOEs do not submit their investment Energy and Utilities. In case of delays in approval by plans as part of the financial planning process, any agency in this complex process, SOEs cannot as summarized in Figure 10 (and detailed above implement their investments or other significant under Legal Framework). Despite the legislative transactions. In practice, such delays can be significant requirement to provide an investment plan together – for example, the range of delays identified by the PIM Must-Have Functions: PIM Function 4. Project Selection and Budgeting Must-have PIM function Good policy treatment What is analysed 4. Project Selection and Regulated sectoral SOEs: Require SOE What approval processes are Budgeting Board approval, sector regulator and established for SOEs capital government approval. investments and their selection, depending on the source of funding. Non-regulated SOEs – own funds: Require SOE Board approval. How multi-year implementation is addressed. Non-regulated SOEs – public funds or state guarantees: Require SOE Board Access to capital budget information. approval, and government approval.  274  Article 15.  275  Resolution of CMU No. 701 on approval of the procedure for state expert review of investment projects. 144 Chapter 6. Assessment of State-Owned Enterprises Figure 10. Schematic Approval Process for SOE Financial and Investment Plans OWNERSHIP AGENCY/ SECTOR MOF AND MOE CMU LINE MINISTRY REGULATOR For SOEs under their For SOEs direct mandate,or operating in For all SOEs For all SOEs opeating as natural regulated monopoly, or have profits sectors above UAH 50 mln Source: Authors' depiction of local legislation Accounting Chamber can range from a few days to 1.4 allocations, with no guarantee of continuity of years. 276 This delay in the approval process presents funding from one year to another. The investment significant risk of implementation delays, preventing planning process for SOEs is also complicated by the SOEs from reliably planning their investments and fact that the government does not have predictable sustaining their investment commitments to ongoing dividend policies: the size of dividend payments is projects. set by the government annually through the budget law. These two factors undermine SOEs' ability to 477. SOEs' supervisory boards play no formal role do mid- to longer-term capital investment planning. in the capital investment planning process, which To implement large capital investments—the does not adhere to the recommended practices. implementation of which spans several years—SOEs Supervisory boards are charged with strategy setting, have been relying on long-term funding from IFIs and oversight over performance indicators, and ensuring other lenders as sustainable sources of long-term proper risk assessment at SOEs and should be taking financing. a more active role in capital investment planning. Presently, SOE boards' role in the process is not well 479. There are no specific criteria for selection of established and complicated by two factors: (i) only projects financed with SOE's own funds or those nine SOEs in Ukraine have established supervisory financed from the state budget under budget boards; and (ii) SOE boards are only involved on a programs. Selection of projects funded via SSIP under discretional basis in capital investment planning, and Resolution No. 571 is well regulated and established only at those SOEs where such authority is assigned to and is performed by the Inter-Agency Commission. them in the entity charter. For example, Ukrzaliznytsia Selection of projects financed under broader state (Ukrainian Railways) approved their 2021 investment budget programs following Resolution No. 835 and plan at the senior management level, as the board was Order No. 205 is performed by the Commission not fully appointed and thus not operational during established by an MDA/ownership agency. While in all the investment planning period. In the current setting, cases profitability, economic impact, environmental, the final decision on all capital investments by SOEs and social impacts are considered when taking rests with the government, leaving no tools for SOE decisions, the selection criteria is not specific for boards to weigh in on strategic priorities or sources of projects funded by SOEs' own funds or under the funding for such investments. budget programs (Resolution No. 835). 478. Mid- and long-term investment planning 480. Access to information on capital investment remains an issue for Ukrainian SOEs. The current by SOEs remains limited. During this assessment, time horizon for allocating public funds remains only a few SOEs disclosed their entity-level investment mainly short-term, as it depends on annual budget plans.  276  Accounting Chamber report on the results of the effectiveness audit of the revenue collection from SOEs reporting to the Ministry of Economic Development, Trade and Agriculture of Ukraine (22.06.2021). Chapter 6. Assessment of State-Owned Enterprises 145 PIM Function 5. Project updated on both qualitative and financial progress of projects. Implementation 482. All capital investments implemented by 481. SOEs remain in full charge of the SOEs—own and budget funded—fall under the implementation of their capital investment requirements of the PPL. This legislation provides projects and report regularly on the a coherent framework for the goods and services implementation progress to the government. procured during the implementation process. Implementation reporting is structured similarly to Capital investments projects financed by the IFIs project planning documentation. Progress reports follow their dedicated procedures to be in line with are submitted quarterly to the ownership agencies the IFI procurement requirements, as legislated by or the line ministries responsible for monitoring Resolution No. 70.277 There are no exceptions to these implementation of approved financial and investment two routes of capital investments execution. plans. The ownership agencies or line ministries are PIM Must-Have Functions: PIM Function 5. Project Implementation Must-have PIM function Good policy treatment What is analysed 5. Project All SOEs: Apply public procurement How the procurement rules are Implementation rules.278 Active project management, applied. monitoring, and project progress What are the regular progress updates updates. and monitoring during project Listed SOEs: In addition to the above, implementation. comply with relevant disclosure requirements. PIM Functions 6, 7 and 8. Project 484. SOEs are required to prepare and submit their quarterly and annual financial statements to their Adjustment / Project Operation / ownership agencies and are subject to multiple Project Evaluation regulations in accounting and financial reporting. SOEs follow general reporting and disclosure 483. Project adjustments must undergo the same requirements for all Ukrainian entities – private or procedure as an approval process, which is done state owned – such as the Accounting Law (1999 with via regular reporting to the government and changes and amendments), and the Commercial annual financial planning. Using the annual financial Code of Ukraine (2003).279 In addition to these general and investment planning process, SOEs can introduce requirements, SOEs are also subject to some specific adjustment to ongoing projects if they follow the same reporting requirements that may apply to them procedures as apply to a regular financial planning depending on their legal form or sector, such as cycle. This approach keeps all parties up to date outlined in the CMU Resolution No. 1067.280 Moreover, with the implementation progress and ensures that SOEs in the form of joint stock companies are also all necessary approvals are obtained through due subject to reporting and disclosure requirements process.  277  Resolution of the CMU No. 70 as of 27.01.2016 On preparation, implementation, carrying out of monitoring and completion of implementation of projects of economic and social development of Ukraine supported by the international financial organizations.  278  Public procurement or alternative competitive procurement rules.  279  Commercial Code of Ukraine (2003), Article 73.  280  Resolution No. 1067 dated November 9, 2016, "On approval of the Procedure for disclosure of information on the activities of state unitary enterprises and business companies in the authorized capital of which more than 50 percent of shares belongs to the state. Available online at: https://zakon.rada.gov.ua/laws/show/1067-2016-%D0%BF#Text. 146 Chapter 6. Assessment of State-Owned Enterprises PIM Must-Have Functions: PIM Functions 6, 7 and 8. Project Adjustment / Project Operation / Project Evaluation Must-have PIM function Good policy treatment What is analysed 6. Project Adjustment Regulated sectoral SOEs: Require SOE Board What are project approval, sector regulator and government approval. adjustment procedures for SOEs capital Non-regulated SOEs – own funds: Require SOE investments. Board approval. Non-regulated SOEs – public funds or state guarantees: Require SOE Board and government approval. 7. Project operation All SOEs: Provide quarterly and annual financial What are the financial statements. reporting practices. All SOEs – public funds or state guarantees: How is the aggregated Report regularly on SOE performance, social or policy reporting over SOE activities: quantities achieved, cost incurred, and financial performance budget support received. established. 8. Project Evaluation All SOEs: Timely complete, publish, and ensure What are the review of all external audit reports by line ministries completion procedures and legislature. for SOE capital investment projects. All SOEs – own funds: Conduct financial and risk analysis of the projects, except in case of major What is the process for environmental or social risks, or mega projects. ex-post evaluation of projects. All SOEs – public funds or state guarantees: Complete full integrated analysis for all large projects. Source: World Bank authors, based on the World Bank World Bank (2021): Assessing Public Investment Management Functions and Institutional Arrangements for State-Owned Enterprises. A Diagnostic Framework. under the Law on Joint Stock Companies (2008), performance oversight, collecting and processing the Law on Securities and Stock Market (2006), and financial information from the largest SOEs and those regulations issued by the National Securities and Stock that send their financial information via ownership Market Commission (NSSMC).281 entities. The MoE manages the SOE portal (ProZvit), which contains information on about half of SOEs 485. Although financial reporting and disclosure (1,800 SOEs out of about 3,600 existing). While this regulations are largely in place, the SOE oversight information is available at an entity level, regular and functions are currently divided between the consistent aggregated reporting of SOEs' financial different branches of the government. SOEs performance is not available. The MoE used to publish report to their ownership agencies, including line an aggregate TOP-100 SOEs report (2014–2018), which ministries, the State Statistics Service, and MoF, which provided a comprehensive snapshot of the financial complicates the monitoring and oversight function. performance of the largest SOE and their indicators, The MoE plays a coordinating role in SOE financial but this practice was unfortunately discontinued.  281  NSSMC Decision No. 2826, dated June 19, 2013, "Provision on the disclosure of information by securities issuers". Available online at: https://zakon.rada.gov.ua/laws/show/z2180-13#Text. Chapter 6. Assessment of State-Owned Enterprises 147 486. SOEs are subject to external audit only for to demonstrate effect, the government should projects that are funded from the state budget. remain on the path of strengthening the corporate The ACU can conduct financial and effectiveness governance practices of SOEs by appointing audits of such projects at their own initiative in supervisory boards with a majority of independent line with the Law on the Accounting Chamber, or members for at least the top 50 SOEs. at the request of the central or local governments, 4. Approval of SOE financial and investment or members of the Parliament. Such audits are plans should be gradually transferred to SOE conducted for the capital investments implemented supervisory boards in line with good international under Resolution No. 571. Further details are outlined practice and recommendations from the OECD. in Chapter 3. This step would allow SOEs to manage their cash flows in a more effective manner, prioritizing critical infrastructure investments with multi-year Proposed Recommendations implementation horizons. 5. Delays in approval of financial and investment 487. While the framework for SOEs' capital plans should be minimized to the extent possible investment is in place and largely followed, and stay within the legislated time periods. The investment project approval, implementation, and authorities in charge of clearing and approving the funding allocation can be significantly improved. plans are recommended to follow the legislated 1. Investment plans envisaged under Order No. 205 time frames to minimize delays in implementing should be consistently enforced by all levels of the the capital investment projects. government – ownership entities, line ministries, 6. Multi-year capital investment funding should be MoE, MoF, and CMU. This fundamental document introduced. As it may be difficult to introduce must become an integral part of the financial multi-year commitments for budget-funded planning process and submitted as legislated. capital investment, SOEs should at least be The MoE may consider developing a detailed allowed to plan their own funds allocation for a methodology and a template for investment plans mid-term implementation period. that can be applied by SOEs. 7. SOEs should be allowed to retain a stable and 2. Project proposal and appraisal methods healthy portion of their earnings to re-invest these under Resolutions No. 571 and No. 835 should funds into building the critical infrastructure and be aligned for a more consistent approach. growing Ukraine's public asset value. This is now Resolution No. 571 ruling the SSIP process should complicated by the annual determination of the prevail and be applicable to all capital investments dividend to be paid, which varies significantly from funded by the budget funds, as implemented year to year and precludes SOEs from sustainably by SOEs. This will help align the methodologies, planning their investment strategy. streamline the implementation and enforcement, 8. Transparency of SOE capital investments and and reduce room for the misallocation of public access to information could be significantly funds. improved if SOEs were to disclose their mid-term 3. SOE boards should be empowered to take a investment strategies and annual investment more active role in capital investment planning plans on their own websites. This would allow for and approval. For those SOEs where boards are a more transparent reporting and increased public established their role in SOE capital investment scrutiny over the capital investments projects planning should be clearly stipulated in the implemented in the public interest. Currently, company charters, allowing the boards to weigh this practice is not widely followed by SOEs not in on strategic priorities and sources of funding disclosing their mid- or longer-term investment for SOEs investments. For this recommendation plans. 148 Chapter 6. Assessment of State-Owned Enterprises 9. SOE financial oversight needs to be streamlined with the institutional responsibilities over SOE performance monitoring centralized at the MoE. This will enable better coordination and information flow among multiple SOE ownership agencies and ministries and improve timely decision making. 10. SOE aggregate reporting should be reinstated following the successful practice of 2014-2018. The government would significantly improve its commitment to SOE transparency, including to transparency of SOEs' capital investments, if it reinstates its regular aggregate TOP-100 SOEs report (2014–2018) to provide a comprehensive snapshot of the financial performance of the largest SOE and their main indicators. This would also respond to the recommendations of the IMF, OECD, and the World Bank, previously shared with the government. 11. Staff capacity should be significantly improved. Capacity building, training, and expert support all need to be considered. Donor support may be requested if needed. Chapter 6. Assessment of State-Owned Enterprises 149 R 7 PTE CHA CONCLUSIONS AND STRATEGIC RECOMMENDATIONS 7.1. Conclusions among regional development projects (which are not the focus of this assessment), varied from 33 percent and Strategic to 71 percent of total investments from 2018 to 2021. Recommendations Based The main regulatory recommendation is on the Assessment that public investment should be clearly defined in the legal and regulatory framework and the terminology will apply to all 488. The main reason for the lack of a kinds of investment projects, so that there is no breakthrough in improving the PIM system is ambiguity concerning the scope of the PIM system. the significant proportion of investments that Ambiguity could be resolved through amendment of bypasses the formal appraisal and selection Art. 2, point 26 of the Budget Code by following criteria procedure established in Resolution No. 571. of public investments: The issue arises due to the non-mandatory nature of procedures, compounded by weaknesses in the À Construction, expansion, acquisition, renovation, terminology and its application, allowing projects or replacement of existing facilities (including the to circumvent the system. Despite the fact that the cost of land, engineering, architectural planning, term "investment project" is included in both the and contractual services) that require a total Budget Code and in the Law on Investment Activities, expenditure of at least [financial value] over the life it has not prevented many investment projects from of the project, or circumventing established procedures. The average À Rehabilitation of existing facilities with a total share of non-SSIPs in the total public investments expenditure of [financial value] and an economic was 19.3 percent over the period of 2018 – 2020, life of at least [number of years] years. increasing to 26.8 percent in 2021, 33.6 percent in À Major equipment with a cost of [financial value] 2019, and 46.3 percent in 2020 (excluding IFI projects). and a useful life of [number of years] years, when it Out of total public investments, the share of non-SSIPs is not a part of a construction project. 150 Chapter 7. Conclusions and Strategic Recommendations 489. Limited progress in strategic planning 490. Ukraine has 15 types of capital investment has been made in 2020 – 2021 on the National streams but still lacks a unified PIM system. Economic Strategy, 282 and there has been no effort Public investment is implemented through different to develop a specific mid-term public investment mechanisms according to the means of financing: (1) plan. Although it represented a step forward, the direct budget financing (state capital investments); National Economic Strategy still does not contain (2) state budget support (state guarantees, comprehensive guidelines, or the specification budget lending); (3) PPPs; and (4) corporate sector of outcomes and outputs needed for investment investments (SOEs' investments of own funds). planning within different sectors. The continued Annex 3 sets forth the different responsible agencies lack of legislative requirements concerning the and the legal/regulatory instruments that apply to 15 approval of a mid-term plan and the procedure for types of capital investment streams. It shows that very its development and approval will further exacerbate similar projects can be subject to different procedures delays and cost overruns in project implementation, depending on the financing source, while other slowing down the achievement of qualitative and capital investments lack specific rules and procedures quantitative improvements in the provision of public governing critical components of a functioning PIM services. system. Since SOEs use various sources of funding for their investments, they follow different laws in each The main strategic recommendation case, except for the legislation on financial planning is that long-term planning should which is mandatory for all but does not provide be improved to include a national clear rules for public investment management. infrastructure investment strategy and a realistic Environmental impact is partially considered within mid-term public investment expenditure plan some investment streams. should be introduced, which covers all public investment streams involving budgetary funds or 491. Rules and procedures cover the critical PIM other state support (e.g., state loan guarantees system components only for state investment and support to PPPs in whatever form). A long-term projects, PPP, the regional development fund, IFI strategy, which is sufficient for effective investment funding, credit funds with state guarantees, and planning, could be presented as a consolidated group infrastructure projects.283 Under these rules and of sectoral strategies with an analysis of the main procedures, the MoE, the Ministry for Communities cross-sectoral links, as well as having an investment and Territories Development, the MoF, and sectoral strategy component. The mid-term public investment ministries are in charge of overseeing, supporting or plan should present: (i) the intended mid-term goals controlling project appraisal and selection, monitoring, and outcomes of the strategy; (ii) estimation of total evaluation, and initiating audits within relevant funding and allocation of funds between investment streams. Other rules and procedures, relating to other streams and sectors; (iii) cross-cutting strategic capital investment streams only partially cover the objectives for investments, such as energy saving, critical PIM system components. In particular, they accessibility for people with disabilities, overcoming usually cover project selection without any guidelines the consequences of climate change, etc.; and (iv) on project appraisal. Monitoring of such projects is general requirements for project selection within each carried out based on construction legislation, under of the streams. Any relevant public investments would which the results are not formalized, submitted to the be required to follow the mid-term investment plan. implementing agency, or published. Given the uncertainties concerning total funding for investment (made worse in the context of the war), the 492. Some identical streams follow different base estimates may need to be supplemented by low- procedures, potentially leading to ineffective case and high-case scenarios and related allocations project selection and undermining PIM as a for outer years of the plan. whole. For example, projects in the newly established  282  It should be mentioned, such recent progress did not affect the score since the period under analysis is 2018 – 2020.  283  Infrastructure projects is a new category of state investment projects that has come into existence after the period covered by the assessment. Chapter 7. Conclusions and Strategic Recommendations 151 stream of "infrastructure bonds" (row 14 in Annex 3) 493. Local governments receive state budget funds are the same as state investment projects (row 1 in for their investments through different capital Annex 3); however, they follow different regulations investment streams, and not all of them are and have different responsible agencies, adding to covered by clear rules. The regional development the undesirable fragmentation of the PIM system. fund (row 7 in Annex 3) and local road subvention Similarly, projects in the road sector can be selected (row 4 in Annex 3) are covered by established rules, under different streams, e.g., the state road fund while other subventions (row 10 and row 12 in Annex (row 3 in Annex 3) or state investment projects 3) are not. Beside this, the regional development (row 1 in Annex 3). The same situation exists in the fund is not a subvention but direct financing of local environmental protection sphere – similar projects can investments through the state budget, while the be selected under environmental protection measures cash flows are going to local government institutions (row 5 in Annex 3) or as state investment projects (row that are implementing projects that should be done 1 in Annex 3). The existence of dual tracks for certain through the relevant local budget. types of projects leads to a lack of transparency and inconsistencies in competition for limited resources. The recommendations are that all investments in the local infrastructure The recommendation is that each type must follow a unified approach and local of investment project must follow a governments must also follow the comprehensive unified and clear procedure. To do this, at rules covering all of the PIM stages. Article the legislative level, the types (spheres) of investment 105 of the Budget Code on "subventions for the projects that require state financing (support) must investment projects implementation" which covers be clearly linked with the relevant streams of their all subventions of an investment nature, presents financing and not interfere in other streams. Relevant the basics of good practice, and should be properly procedures should cover all PIM stages, including applied. Improvements to this article could also completion review and post-completion evaluation. be made, for example, by adding the population In addition, strong strategic planning and budgeting density as a criterion for project selection. The systems, are required, including the aforementioned regional development fund's financing should also mid-term investment plan, in which the mid-term be subvention to local government budgets and not limits for each stream are set. So as not to overstretch direct financing of projects, so that it does not subvert limited capacities and create bottlenecks and to keep the budget process at the local level. Regarding PIM effort proportionate to the scale of expenditure and/ rules for local governments, advisory methodological or risk, less onerous procedures and methods should guidelines, which local self-governments may apply, be maintained for lower value/lower risk projects, as should be established at the national level. Ongoing is done currently through the use of a value threshold. training of local staff should also be available, The value threshold should be kept under review including through a special online training platform. and adjusted as necessary to balance supply and If applied adequately, such rules would support demand for the scarce analytical and managerial skills improvements in the effectiveness and efficiency of required. Bearing in mind that a project should include public investments in local infrastructure. all activities necessary to deliver sustainable benefits to beneficiaries, line ministries should be made 494. At the same time state budget programs for responsible for presenting complete and coherent subventions do not follow performance program projects that do not involve slicing up larger projects budgeting, despite the fact that relevant local to fit below the threshold. All related project profiles budget programs reflect performance in their (when these exist) that are integral to achievement of spending. Performance program budgeting is actively the same purpose should be merged into one project used now at the central and local government levels at the screening stage, when the MDA management with clear outputs and, partially, outcomes and KPIs. considers concept notes and takes a decision on Excluding state budget programs for subventions whether to proceed with project development. The from PPB leads to a lack of strategic focus in local MoE should be given the right and responsibility to investments and weak effectiveness at the national reject "projects" that are obviously subsets of bigger level. Performance presented in the relevant local projects and send them back to the ministries for budget programs is usually limited to an investment proper configuration. project and the national investment goal is ignored. 152 Chapter 7. Conclusions and Strategic Recommendations The recommendation is that all The institutional recommendation in investments from the state budget in relation to CC-PIM is that the Ministry local infrastructure must be based on of Ecology should be designated as performance program budgeting. Each budget supervisor for the consideration of climate change program for investment subvention (including the impacts and effects on PIM. Public investment regional development fund) from the state budget to projects (possibly over an established threshold) must local budgets must have a high-level mid-term goal, be analyzed from climate change adaptation and outputs, and outcomes. This will ensure that selected mitigation points of view prior to project selection; the projects will contribute to the achievement of strategic analysis results must be used in the project selection goals. and implementation stages. The Ministry of Ecology should provide guidelines for CC to be incorporated in 495. Legislative improvements with some appraisals and should perform an independent review relevance to climate change have occurred since of appraisals to ensure that CC has been adequately the 2012 PIM assessment, which did not itself addressed. address climate change (CC). Resolution No. 571 requires environmental impact assessment as part 497. The management of PPP projects has of public investment project appraisal and Order No. improved in some areas since the 2012 PIM 1865 (dated December 22, 2017) of the MoE gives assessment, notably in terms of the legal details of environmental impact assessment as part framework, but transparency remains an issue. of public investment project appraisal. The law of The budget documentation still does not disclose Ukraine on Strategic Environmental Assessment, PPP-related fiscal risks or include any information on adopted in 2018, requires strategic environmental PPPs. However, on the positive side, there has been assessment of draft public planning documents, significant improvement in the legislative framework complementing the procedures for the preparation for PPPs over the last five years. The adoption of and adoption of such documents. When used the law on concessions and the law on PPP, as well correctly, the latter should improve strategic guidance as other legislative acts, has improved the PPP for public investments planning in relation to CC. management system so that it now corresponds to good international practice. In addition, a manual 496. Despite some improvements to the legal for PPP projects covering the end-to-end project framework, the evidence suggests that CC management process was developed in 2021 (outside considerations are not well embedded in the the assessment period). Despite successful upgrading Ukraine PIM system. There are some elements of of the legal and regulatory framework for PPP, there CC analysis related to public investment projects, remain some inconsistencies between the new but these elements do not form a comprehensive framework and the Budget Code, which does not allow approach to incorporating assessment of CC into implementation of 'government-pays' PPPs. This is a appraisal and selection of projects, either from the serious constraint on this commonly used funding perspective of a project's impact on CC or from the framework for PPPs. Direct project financing for PPPs perspective of the potential effects of CC on the with state budget funds is allowed, however, there has project. The Ministry of Ecology and Natural Resources to date been no case of the government giving direct (Ministry of Ecology) is not involved in the CC analysis financial support to a PPP. 'Government-pays' PPPs during the feasibility study, but only after all selection do not work due to the lack of an instrument such procedures have been completed and then only for as a long-term obligation in the framework of PPP projects with significant impacts on the environment. contracts. The legislation to allow financial support There are different procedures managed by the to 'government-pays' PPPs has now been passed by Ministry of Ecology or the regulatory bodies in the the Parliament (but outside the assessment period); construction sector which require environmental however, it only concerns special funds of the budget, impact analysis, but only in very rare cases is CC such as the Road Fund and Inland Waterways Fund mentioned. Even if environmental impact analysis (for state assets) and allows commitments for projects involves such indicators as greenhouse gas emissions relating to the creation or improvement of municipal or energy efficiency indicators, there is no evidence or assets. requirement to assess the sustainability and resilience of infrastructure to the potential effects of CC. Chapter 7. Conclusions and Strategic Recommendations 153 498. Weaknesses in some processes lead to 500. Since SOEs use various sources of funding reduced transparency in relation to PIM for PPPs. for their capital investments, they also follow Despite the broad requirements for transparency different streams in each case which may or may of information on PPP projects in the legislation, not be based on clear rules and procedures (see especially for concessions, there is a widespread Annex 3). Depending on the relevant investment problem with the practice of regularly applying a streams SOEs are guided by twelve legislative acts to "confidential information" status. The confidential receive state support for their investment projects. status is at odds with international good practice One of those streams (row 11 which concerns which restricts confidentiality to commercially environmental protection measures) is not covered at sensitive information only. Also, central government the project appraisal and expertise stages and three authorities have not been following the requirements of them (rows 6, 11, and 12) are not covered at the of the PPP legislation in a timely way, which has project appraisal, expertise, and selection stages. Legal contributed to undermining confidence among private conflicts are observed in these complex arrangements. partners. The absence of an independent appeals For example, Resolution No. 835 regulates project processes for private partners participating in PPP selection with support from state guarantees, but in procurement/concessions is also a deterrent. practice it does not apply to guarantees for IFI loans. At the same time, the decree does not contain relevant Regarding PPP projects, institutional exceptions and legally it must be applied for all state capacity strengthening, legislative guarantees. Moreover, the legal framework does changes, and the development of not prevent the "bypass route", whereby SOEs can mechanisms for the application of budget funding also receive budget funds within a budget program in these projects are required. MoF capacity needs requested by the line ministry (the "non-SSIP" route), to be strengthened via the creation of a specialized thus bypassing the requirements for Resolution unit or team responsible for PPP projects and the No. 571. SOEs also do not have clear internal rules development of the methodological principles for for investment projects by own funds; only about a assessing fiscal costs and risks of PPP contracts. Also, quarter of capital investment projects follows clear comprehensive changes to the Budget Code must be procedures. introduced to allow financing of PPP projects from the general fund of the budget. 501. While the government's oversight function appears to be detailed (though not transparent 499. Transparency concerning the scale of enough) it works in a one-sided manner; budgetary support to SOEs remains poor. The level procedures for preparation of financial plans by of SOE capital investments financed using budgetary SOEs284 do not extend to financial planning for lending and state guarantees exceeded 20 percent capital investments due to the lack of enforcement of total public capital spending, but no information of relevant legislation, so the oversight function on this support was included in central government does not significantly contribute to the fiscal reports. This is the same finding as in the 2012 effectiveness of SOE capital investments. The PIM assessment. Some changes were observed at the main actor and supervising body for SOEs in terms of institutional level during this period. The share of the public investment is the ownership agency/ministry. public sector in the economy decreased in 2020 to 8 It approves financial and investment plans of SOEs, percent (by 1.8 percentage points), compared to 2013. develops strategic documents to be followed by SOEs, At the same time, the share of capital investments of prioritizes and authorizes capital investment, controls public sector entities decreased to 7.7 percent (by 1.3 implementation of projects, and assesses overall percentage points), and the share of the number of performance of enterprises. The MoF is involved in economic entities in the public sector decreased to 0.5 approval of financial and investment plans, assesses percent (by 0.2 percentage points). the fiscal risks of projects, and performs financial  284  Procedures were introduced in 2015 by the MoE Order No. 205. 154 Chapter 7. Conclusions and Strategic Recommendations monitoring of investment projects. The CMU is an are created for different purposes and for different ownership institution for some SOEs, for which it types of projects. It is neither possible to track approves financial plans. It also deals with approval individual projects across the project cycle using a of plans of natural monopolies and enterprises single information source, nor is it possible to obtain with the net profit exceeding UAH 50 million. The a full picture of the total number of projects (and their State Audit Service and the Accounting Chamber of values) at different points in the project cycle. This Ukraine perform external audits of SOEs within their makes it complicated to understand the progress mandate. While the high-level arrangements are in of individual projects and to get comprehensive place, basic tools – comprehensive requirements, insights into the status of the project pipeline and guidelines, instructions, forms – for SOE investment implementation of the portfolio of ongoing projects. management are missing, and the legislation is not being followed. The result is that there is no complete The main recommendation for and consistent portfolio of investment projects, and strengthening staff capacity is that no comprehensive vision of whether the investment a sustainable system of training for plans are linked with the strategic priorities. These employees involved in all PIM stages should be set weaknesses call into question the efficiency of SOE up. Skills development should become embedded in investments as a whole. work practices. To do this it should be a mandatory part of the induction of new employees involved in The recommendation regarding SOE PIM and should be repeated periodically through investments is that guidelines for the refresher courses for existing staff. Participation and internal investment management success in PIM training should also become part of procedures should be developed for SOEs. Even staff performance evaluation and there should be a though SOE investment projects follow different credible certification process. Developing the training streams with different requirements, the SOE itself through an on-line platform will ensure consistency must have a comprehensive picture of the effective and quality, while offering flexibility for participants implementation of its investments in the mid-term. in timing and location of the training. Given the Therefore, a unified procedure for all SOE investment weaknesses in upstream processes, capacity building projects covering assessment, selection, and inclusion in appraisal will be important, but downstream in a mid-term investment plan should be developed. processes, like project management and contract This plan, which should follow a well-defined management, should not be neglected. Because it format, should be subject to review by the oversight takes time to build capacity, the government may want ministry and the MoF and the MoE. Introduction to think practically about pooling skills and creating of improvements in the structure and content of centers of expertise that can be used broadly across investment planning will strengthen the government's institutional lines to support priority needs. oversight function. Capacity building should also not take place in a 502. PIM is weakened by the shortage of qualified void and must be preceded by other recommended staff and the absence of process automation. reforms, such as anchoring the definition of a public Project appraisal capacity is not well developed due investment project in law (and then enforcing it) and to little experience and limited number of staff. issuing standardized project management guidelines. Project appraisal training has not been provided for The main recommendation in the information the last three years.285 Information sharing across technology sphere is that a comprehensive public the PIM system is fragmented with consequences investment information management system for transparency and portfolio management. There (PIMIS) be created, covering all business processes is no unified database that contains comprehensive (the PIM stages) and all investment streams. To information on all public investments at different control the entry and flow of projects through their stages in the project cycle. The existing databases  285  To address this issue, the project under which the current assessment falls is also establishing an online training platform for certification of staff in the required skills. Chapter 7. Conclusions and Strategic Recommendations 155 stages of design, appraisal and approval it is important procedures will inevitably have to be made. But that all the key features of the projects and their care will be required so as not to undermine the development and approval status be entered and progress that has already been made, for example, monitored in a centralized information system for all in public procurement. With this in mind, the current projects. This forms a controlled pipeline of projects section looks at the recommendations from the under preparation and approval that feeds into the core PIM assessment and indicates which of these mid-term public investment plan. This PIMIS should be will be important to incorporate in PIM for post-war mandated by law or regulation so that projects cannot reconstruction and which recommendations can be be parachuted into a budget without entering through adopted for post-war needs. the formal channels of the public investment pipeline. The PIMIS is likely to be a long-term development, 504. PIM for post-war reconstruction cannot be so it could be useful to identify the core module or divorced from the changed political context, functionality that would be most critical to start with, which will likely lead to a reorientation of public like controlling the project pipeline and monitoring investment. This reorientation may in some cases linked to financing. The MoE should be responsible be strongly influenced by political imperatives rather for development, introduction, and maintenance of than by pure efficiency-related criteria. Ukraine can such a system. Other actors in PIM should have rights be expected to look westward toward the EU more for making changes in the system within their area of than ever; this will affect public investment priorities responsibility. The data from the system should be as the country seeks closer integration politically publicly available. and economically. This will impact the demand for infrastructure services as Ukraine endeavors to become more embedded in European supply chains. 505. Reconstruction will be hugely symbolic, and 7.2. Supplementary the part it could play in restoring confidence Discussions and should not be underestimated. The return of refugees and internally displaced people will Recommendations. PIM for require confidence-building measures, including Post-War Reconstruction reconstruction. The reconstruction should therefore be people-driven aiming to provide housing and jobs, so that returnees have basic services and livelihoods. Introduction Nevertheless, financial and human resources will be scarce and rational decision-making and efficient 503. PIM for post-war reconstruction needs to be implementation will be required to ensure that these streamlined but the fundamental functions of PIM scarce resources are used to best effect. This is the will still need to be performed. If Ukraine's pre-war goal of PIM for post-war reconstruction. PIM system had been in line with international good practice, the country would have been better prepared for the demands of developing and implementing an urgent post-war reconstruction program. The Legal and Institutional assessment findings and recommendations of the Framework core PIM assessment indicated that Ukraine's PIM system falls short of good practice in many respects. 506. Strengthening the legal and institutional The weaknesses identified cannot be corrected in framework for PIM is a precondition for a the short term and yet the post-war reconstruction successful reconstruction effort. The core PIM of the country will be a matter of extreme urgency. assessment has described a highly fragmented PIM A pragmatic approach will therefore be required. system differentiated by various factors including Fast-tracking infrastructure reconstruction will be vital source of finance and/or implementation modality. and some compromises in terms of administrative The existing institutional framework involves 156 Chapter 7. Conclusions and Strategic Recommendations diverse legal instruments, various non-standard Strategic Guidance and procedures, procedural gaps, and too many oversight organizations. The recommendation is for a more Screening unified PIM system, involving an end to multiple entry-points, introduction of unified procedures and A Short- to Mid-Term National methods, and establishment of more consolidated Reconstruction Strategy management responsibilities. 509. A strategic approach to post-war 507. An essential early reform should be reconstruction is required. The PIM assessment the establishment by law, and subsequent has highlighted shortcomings in the information enforcement, of a universally applicable and requirements and analytical capacities for strategic unambiguous definition of a public investment planning, and the absence of an overarching long-term project. The scope of the public investment strategic plan for public investment. Improvements system should be delineated so as to avoid any in these areas will require structured reforms and room for misinterpretation; this is as important take time to achieve. Post-war reconstruction cannot for reconstruction projects as it is for development wait for these developments but will nevertheless projects. This will ensure that all budget-funded still require a strategic approach, meaning a clear projects are subject to the same procedures, without identification of the problem and its scale, setting exception. of objectives, determining the constraints, and establishing what needs to be done to achieve 508. Management of post-war reconstruction will the objectives within the constraints. A strategic be critically handicapped if similar fragmentation approach also entails assessing the availability of is reproduced. Unified procedures for planning, resources, both financial and human, for achieving management, and decision-making will be essential the objectives and setting forth criteria for prioritizing to ensure that reconstruction priorities are outputs within these constraints. A basis for assessing identified and addressed in a consistent way without whether the objectives have been achieved is also duplication or gaps. Institutional fragmentation risks required, meaning easily measurable key performance creating an unfocused reconstruction program, indicators and baseline figures for these indicators. with damaging competition for scarce resources, inconsistent decision-making, and unnecessary 510. A short- to mid-term strategy for delays in reestablishment of critical public services, reconstruction of damaged or destroyed public all of which must be avoided. A unified approach to infrastructure is essential. Such a strategy is reconstruction is therefore essential and could also be required to guide choices and ensure that scarce a precursor to a more homogeneous PIM system in financial resources are directed toward the highest the post-reconstruction phase. Capacity was strained priorities in terms of re-establishing basic services before the war and will most likely become much and kick-starting the economy. The strategy may also more of a problem in the reconstruction phase, when extend to construction of entirely new facilities, where some further capacity will have been lost, needs and it is clear that displaced populations will not be able to workloads will be higher, and on-time completion return to their regions of origin. The strategy should will become even more essential. A combination of cover: leveraging donor funds to get short term capacity, building capacity internally, and considering the À Overview of the reconstruction task: a summary establishment of a government "center of excellence" assessment of the damage and its impact on to provide capacity to line ministries with the largest access to basic services and livelihoods needs will be a good options to solve the capacity À Analysis of changes in the demand for problem. infrastructure services as a result of the movement of people and businesses within the country caused by the war Chapter 7. Conclusions and Strategic Recommendations 157 À Objectives for re-establishment of access to basic  Pre-war reforms, in education and health for services and livelihoods example, might have envisaged the phasing out or downsizing of certain facilities. It would À Reconstruction needs to deliver on objectives not make sense to rebuild (or build back to the À Estimation of costs of reconstruction and sources same scale) such facilities if they were to have of funding been closed anyway. The same logic applies À Criteria for prioritizing reconstruction projects to previously planned expansion of service provision when projects should consider the creation of new public facilities. 511. The country's newly prepared Comprehensive Recovery Plan (Plan) cannot be static.286 Costing  Reconstruction of some facilities may not an estimated USD 750 billion, and presented on be compatible with climate change policies. July 4-5, 2022, the Plan focuses on the profound Concepts may therefore need to be modified transformation of the country into a new European to include mitigation and adaptation measures member based on the principles of the green that were not part of the original asset. economy. The Plan will need to be frequently updated À Will reconstruction of the facility be consistent as new information becomes available on the extent with the post-war economic geography and of the war damage, on implementation progress demography of the country? and problems, and on availability of resources for implementation.  Depending on the nature of any peace settlement, post-war trade patterns might 512. The national reconstruction strategy may be radically altered, and it would not make eventually evolve into a national infrastructure sense to build back (or build back to the same development strategy. The evolution of a post-war scale) transport infrastructure, storage, or reconstruction strategy into a long-term development other trade-related facilities that serve former strategy may be some years away and it will not patterns of trade. Sometimes construction of necessarily an easy transition. Nevertheless, some new facilities in new locations may need to capabilities developed for the reconstruction strategy take precedence over general reconstruction if can be expected to be transferable to longer-term exports depend on it. investment planning.  Even after allowing for the return of IDPs and refugees, the distribution of the population across the country may be changed by the war Preliminary Screening of with implications for the scale of facilities in Reconstruction Project Concepts certain areas. À Is the project consistent with the post-war 513. While restoration of pre-war infrastructure defensive posture and vulnerabilities of the and public services will be the main priority, country? longer-term strategic considerations should not  Reconstruction in at-risk zones may have lower be ignored when screening initial proposals for priority when there are equally valid competing reconstruction projects. The following questions claims on scarce financial resources in stable should therefore be considered at an early stage of parts of the country. project development before a decision to prepare a reconstruction project is reached:  Based on the lessons of the war, concepts for reconstruction projects may need to consider À Is the reconstruction of the facility compatible adaptations for the better withstanding of any with pre-war sector development/reform future conflict and the protection of citizens. strategies and cross-cutting issues, like climate change?  286  https://www.urc2022.com/urc2022-recovery-plan 158 Chapter 7. Conclusions and Strategic Recommendations Mostly the answer to these three questions will be 516. Working in favor of a more straightforward in the affirmative but for some it will not. Raising approach is the fact that decisions to rebuild are the questions early will help ensure that very scarce less contentious because the demand for the reconstruction resources are not wasted. services has already been demonstrated. But choices will still need to be made in a systematic 514. Reconstruction may demand a broader way. Stepwise assessment, decision-making, and perspective on how a project is defined to include objective external review should be retained, even "area-wide" group projects. Reconstruction will if the procedures and methods are simplified and often be about rebuilding communities through the fast-tracked. This means beginning with a preliminary infrastructure that serves them. The project may screening (see above) and an initial decision on not necessarily be concerned with the creation of whether more in-depth project preparation is an individual sector-specific asset, but rather the worthwhile, followed by a decision based on more reconstruction of a group of assets that deliver thorough preparatory work, especially in relation the basic services needed to allow a community to to cost estimates and the demand for the restored function again. Related projects, such as housing services. Simple metrics like reconstruction cost per (family-type group homes for children, including user may be used to prioritize more cost-effective those orphaned by the conflict), schools, clinics, options and to set maximum allowable unit costs police stations, and commercial facilities can be thresholds.287 Speed of completion may also need gathered into one project with realistic mid-term to be considered: higher cost projects that can implementation plans (note: the first step is often start delivering services sooner may sometimes be housing). preferable when users have no alternatives. 517. While a technical feasibility study will always be required for an appraisal decision, a full Appraisal and Independent economic cost-benefit analysis may be replaced by Review simplified qualitative analysis for reconstruction projects. Such an approach may involve the use of 515. Robust appraisal of reconstruction projects multi-criteria analysis to give structure to qualitative will be important, even if the full set of appraisal assessments. Cost-effectiveness analysis may also procedures and methods is not applied in the be performed where this can help in choosing post-war emergency. Stronger and universally between alternatives and benchmarking against applicable appraisal procedures and methods similar projects to ensure that unit costs of services coupled with continuous capacity building have delivery are not excessive. In effect, methods usually been recommended. A more stringent independent reserved for smaller projects can be applied to large review process, with greater objectivity and depth projects because of the urgency and because of the has also been recommended. Implementation of greater confidence in the demand for services from these recommendations will need to be a longer-term reconstructed facilities. The latter point suggests the aspiration which is only possible when the country is need for more sophisticated analysis when a project back in development mode; in reconstruction mode, involves new construction to meet the demands of simpler procedures and methods will be needed to permanently displaced people and businesses. Box 9 ensure swift, but robust decision-making. gives an idea of the kind of criteria that should be considered when assessing and reaching a decision on a reconstruction project at the appraisal stage when a full social cost-benefit analysis is not performed.  287  Rules of thumb to keep costs under control by specifying maximum unit costs, above which reconstruction will not be allowed or must be subject to special approval. Chapter 7. Conclusions and Strategic Recommendations 159 Possible Appraisal Criteria for Reconstruction Projects BOX 9 À Is the proposed reconstruction consistent with  Was pre-war demand for the facility's the direction of pre-war government policy services growing, static, or falling? and with strategic post-war priorities in the  Is the return of pre-war demand contingent sector concerned? on the return of displaced people? If so, À Will the facility provide vital services for the is restoration of public services a critical public and businesses? factor in the return of these people, or are there other more important factors?  Are services critical to basic needs of the population, to restoring livelihoods, or À Are the costs of reconstruction reasonable? restarting businesses?  Are the reconstruction costs per user (unit  Are services currently not provided by of demand) in line with those for other alternative nearby facilities? reconstruction projects of the same type or with other realistic benchmarks, allowing À Is the success of the project contingent on the for differences in the cost of services completion of other reconstruction projects between more remote areas and urban providing complementary services? areas?  Are complementary projects ongoing or  Does cost-effectiveness analysis of planned to be completed in time to realize alternative reconstruction options (where the benefits of the proposed projects? these exist) indicate that the preferred  Should the project be planned as a option is superior? If not, are there other component of an integrated reconstruction factors to be taken account of in reaching a project? decision on the preferred option? À Was there strong demand for the services of À Did the facility have adequate O&M financing the facility before the war and is that demand pre-war and is this funding expected to be expected to return quickly to pre-war levels? restored when the facility is reconstructed? [Demand needs to be quantified and evidence If not, is there an alternative proposal for provided] meeting O&M costs?  Did the facility have spare capacity before the war or was it used at, or above, planned capacity? 518. It will be important to consider alternatives À Simple repairs of nearby existing facilities as part of appraisal even if a full economic cost- providing the same services to reinstate capacities. benefits analysis is not used to decide between À Extensions to nearby existing facilities providing them. A broad perspective of alternatives must the same services to increase capacities. be taken that goes beyond technical variations in the reconstruction design. The attractiveness of À Completely new facilities on new sites in the alternatives will depend on the extent of damage vicinity – this alternative should not be dismissed to an existing facility and the level of reconstruction too early as reconstruction of existing facilities required (partial or total). Alternatives may not be as may be complex and costly, depending on the convenient in terms of accessibility as reconstruction nature of the damage in the immediate area, and of the damaged/destroyed facilities, but the cost abandonment and new construction may be more savings involved may be seen to outweigh these cost-effective.288 disadvantages. Possible alternatives may include:  288  In this case a cost-effectiveness analysis should ideally be performed, looking at lifecycle cost streams of the alternatives. 160 Chapter 7. Conclusions and Strategic Recommendations 519. An agile approach to appraisal decision 523. Once a reconstruction project is agreed, with room for reversals and modification will it must be fully funded from beginning to end be important. Data for decision-making may also (unless the costs change radically because of new be unreliable and subject to revision, especially on information) even if it takes more than a budget the cost side, so room for reversal or revision of year to implement. This means that decisions on decisions in light of better information will need to new reconstruction projects should be made only be built into processes. In a post-war emergency, after provision is made for commitments to ongoing the reconstruction program cannot be allowed to projects and should be programmed within a rolling be handicapped by inertia in decision-making. The mid-term perspective thereby providing the possibility full extent of damage and hence costs may only be of greater scope for new projects in out years as ongoing revealed once reconstruction work has begun and projects are completed. Thus, the basic requirements for structures are exposed, thus requiring dynamic sound budgeting of investment projects apply equally planning and potential review of previous decisions. to post-war reconstruction, where such an approach relies also on reliable and timely information on financial 520. The need for strong independent review of and physical progress reaching those responsible appraisals will be as important as ever during the for programming, so that the fiscal space available post-war reconstruction. A strengthened independent for new reconstruction projects can be accurately review process was one of the main recommendations estimated. It also relies on reliable and comprehensive of the core PIM diagnostic assessment (see Chapter 3). information on financing from all sources. The issues that give rise to the need for independent review – issues with the quality of appraisal, optimism 524. An increase in donor funding for the bias, and deliberate misrepresentation – are just as reconstruction effort is expected and must be likely to be present during post-war reconstruction as managed carefully to maximize its effectiveness. in normal times. Third-party review of the appraisal of Donors have already pledged additional financial reconstruction projects will therefore be important for resources and more funding is likely to be forthcoming. the quality of decision-making. Processes need not be These financial resources need to be carefully as elaborate as for new public investment projects, but managed so that they are channeled to national an objective eye on appraisal findings and decisions priorities as identified in the Comprehensive Recovery will still be necessary. Plan and coordinated with fully nationally funded initiatives. Duplication and service provision "deserts" 521. Retaining public confidence in the fairness can arise when reconstruction efforts from different of the reconstruction program will be important. funding sources are not properly coordinated. It is The criteria for appraising reconstruction projects therefore important that one central body develops and the appraisal results should therefore be publicly and maintains a comprehensive view of the whole available. When decisions are altered to reflect reconstruction program regardless of funding source. changes in understanding of the project and its costs (see discussion of an agile approach above), this must 525. There will be pressure during budgeting to be done in a transparent way. spread financial resources thinly in the interests of fairness and to start as many reconstruction projects as possible in order to be seen to be "doing something" for everyone. It will be a Budgeting mistake for budget decisions to be influenced by these pressures. The focus should be on delivering 522. Reconstruction projects must be programmed results – measured in terms of number of citizens within a realistic mid-term financial framework. The accessing restored services in each sector – rather mid-term investment plan linking the objectives of a than on managing appearances by distributing long-term investment strategy and the funding realities financial means evenly. In this respect, the oversight and providing predictable project financing beyond organizations should be ruthless in withdrawing the current year is a core recommendation of the core budget funding from non-performing reconstruction PIM assessment. Such a plan is even more important projects that are demonstrably failing to deliver. This in the reconstruction phase when predictable requires comprehensive and active monitoring of the financing will be critical to restoring services. reconstruction program (see also next section). Chapter 7. Conclusions and Strategic Recommendations 161 526. Project preparedness will be extremely Accountability on Project Outcomes important in deciding to allocate funds to a reconstruction project. The readiness of a project to 528. External auditors, the anti-corruption agency, be implemented (i.e., preparedness) is a core decision and civil society should be important voices of criterion for budgeting (and programming over a accountability on project outcomes. MDAs must mid-term framework); this is even more so in post- be in no doubt that any corrupt actions within the war reconstruction where there is desperate need procurement process for post-war infrastructure to re-establish public services as quickly as possible. recovery will be exposed. Since the most serious As a result, in some circumstances, readiness may consequences of corruption result from procurement override other factors. Many of the usual issues like on investment projects, the ACU should plan to audit land ownership, site access, and permission to carry each project under implementation. While respecting out new construction, will not apply to reconstruction. their distinct mandates and independence, there But there will be new issues like the safety of the may be some benefit in coordinating project audits site for workers (because of unexploded munitions with the State Audit Service to avoid duplication or mines). Availability of utilities such as water and to the degree possible. The work of the National electricity will remain important but may depend on Agency of Corruption Prevention in developing the timing of complementary projects. This notion recommendations for a damage assessment suggests the possible use of a broader definition of a methodology will be important. This should ensure reconstruction project (see also the discussion above that there will be no corrupt practices when letting on area-wide group projects). investment contracts. The government should promote civil society's involvement through enhanced transparency. Implementation Procurement Project Management 527. Ukraine has made noteworthy progress 529. Strengthened project management in public procurement, as recorded in the core procedures and methods (a recommendation PIM assessment, and the gains should not be from the core PIM assessment) could be one sacrificed in the implementation of the post-war of the most urgent areas of PIM reform for the reconstruction program. Opportunities for corrupt post-war reconstruction program. The core PIM practices can quickly emerge in emergencies when assessment has exposed informality and lack of there is pressure to dilute robust procedures in the clear lines of accountability in project management, interest of quick results. Lack of competition due together with the absence of formal guidelines on to severe damage to some construction companies project management procedures. The successful may also be a problem. As much as possible, implementation of a results-oriented emergency Ukraine should resist adverse pressures and protect reconstruction program, with tight deadlines for the advances it has made in public procurement. re-establishing public services for devastated Further, implementation should not be allowed to be communities, will necessitate effective project held up unduly by over-burdensome procurement management if reconstruction projects are to be procedures; some streamlining will therefore delivered on time and budget. Preparation of a probably be unavoidable, but under no circumstances manual of project management procedures for the should transparency be allowed to erode. This may reconstruction program should be done as early as be a difficult balancing act, with some necessary possible. The manual should be updated regularly compromises in both directions. based on experience (see the discussion below of project completion reviews). Likely shortages of project management skills could be addressed by pooling together scarce human resources in a central government unit with the aim of then directing them towards the highest priority projects. Thus, implementing agencies could "borrow" skilled project managers (or other downstream professionals) or seek ad hoc specialist inputs or advice, when required. 162 Chapter 7. Conclusions and Strategic Recommendations Monitoring and Adjustment Budgeting above), but such funding needs then need to be budgeted, which is best done within a mid-term 530. Unified monitoring of the implementation financial perspective. That completed projects are of the reconstruction program and timely delivering services on a sustainable basis should be adjustment of projects in the event of setbacks verified as part of the ex-post completion review (see will be important. Fragmented monitoring and next paragraph). an inconsistent approach to adjustment and re- appraisal of investment projects have been identified as problems in the core PIM assessment. These are areas where early improvements would be of Ex-Post Completion Review significant benefit to the smooth implementation of 532. Rapid learning and feedback into design and the post-war reconstruction program. Accelerated implementation of the reconstruction program preparation of reconstruction projects, potentially will be important for its success. The core PIM based on incomplete information, makes sound assessment recommended a more systematic monitoring of implementation even more important approach to post-completion project review, both so that adjustments can be made rapidly when internally and externally. The post-war reconstruction updated information comes to hand. The main program should have a structured review process oversight body must be ready to reassess the case for such that lessons can be learned quickly and fed reconstruction projects in light of current information back into the design and implementation of the rest during implementation and be prepared to cancel or of the program. Reviews without the organizational postpone those projects that are no longer likely to arrangements for compiling and disseminating be completed within a reasonable budget. The need findings will be of limited use. Organizational for a unified approach to the post-war reconstruction arrangements should therefore be put in place to program has already been discussed and this should ensure that findings are assimilated. Post- completion be reflected in a unified monitoring system, rather reviews should include confirmation that projects than the diverse systems that have been identified in are delivering services of the specified quality to the the core PIM assessment. Mobilizing internal auditors expected number of users. As with other aspects to verify monitoring information might be one way of post- war PIM, transparency will be important in of improving discipline in terms of accuracy and maintaining public confidence and the findings from timeliness. reviews should be publicly available. Operations 7.3. Recommendations 531. When financial resources are extremely tight, as they will be post-war, it is vital that the Adapted to the Post-War reconstruction effort is directed towards facilities Conditions that will not be prevented from functioning effectively because of shortage of budget for 533. The recommendations provided within operations and maintenance expenditures. The discussions and recommendations around the sustainability of operations and maintenance is PIM for post-war reconstruction are consistent important in peace time when investment projects with those from the core PIM assessment but may augment or improve the quality of public services, are adapted to the post-war conditions. Table 30 but it becomes more important in the post-war summarizes the longer-term recommendations context when communities depend on the restoration arising from the core PIM assessment alongside the of basic services, not improvements at the margin. main recommendations for PIM for reconstruction. This is not an area where significant problems were A critical task for the next stage will be to consider identified in the core PIM assessment, but problems how to create the capacities needed to implement could easily emerge in the post-war environment. each recommendation, and how to sequence the The existence of sustainable funding streams for implementation of recommendations until capacities operations and maintenance should be one of the are scaled up. criteria for selecting reconstruction projects for (see Chapter 7. Conclusions and Strategic Recommendations 163 Table 30. PIM Reform Priorities Prerequisites for Short term and Post-War PIM Reform Core Mid and Long-Term PIM infrastructure recovery Priorities Reform Priorities Institutional Framework À Introduce in the Budget À Implement a unified approach for project À Build sectoral MDAs Code a tight definition of an appraisal and selection covering all internal capacity for the investment project, as the investment project types and funding implementation of the unified key to closing the bypass sources. PIM system. route. À Establish methodological PIM guidelines À Establish a comprehensive À Consider a government for local self-governments. Such guidelines public investment "center of excellence" to should cover all PIM stages, including information management provide capacity to line pre-feasibility and screening, appraisal system (PIMIS) covering the ministries with the largest and independent review, project selection, whole project cycle. needs in the implementation monitoring and adjustment, operation and À Set up a sustainable system of the unified approach to completion review. of training for employees reconstruction. À Authorize the Ministry of Ecology and involved in all PIM stages at Natural Resources to be the core adviser the national and sub-national for the consideration of CC impacts and level. effects on PIM. À Strengthen capacity and À Establish a definition of "climate change improve MoF procedures on expenditure" in the Budget Code and fiscal cost and risks of PPP implement a tagging mechanism for such projects. expenditures through budget programs. À Authorize SOEs' supervisory À Develop guidelines for internal SOE's boards to approve SOE investment management procedures financial and investment covering assessment, selection, and plans gradually. inclusion in a mid-term investment plan. À Establish a database(s) in the most accessible form covering the project cycle until the end of implementation. Strategic Guidance and Screening À Provide MDAs with clear mid-term strategic À Establish a long-term national post-war priorities based on results of infrastructure investment the damage assessment to guide the strategy which considers CC submission of any capital investment commitments. project proposals. À Create a mid-term public À Develop and apply rigorous and systematic investment plan. screening of reconstruction proposals À Institutionalize screening of using criteria that prioritize restoration of all project concepts at the essential public services to communities gateway to appraisal. and businesses while not contradicting long-term strategic development goals. À Monitor screening to ensure clear evidence of projects being rejected or requested for re-evaluation at a sufficient standard. 164 Chapter 7. Conclusions and Strategic Recommendations Prerequisites for Short term and Post-War PIM Reform Core Mid and Long-Term PIM infrastructure recovery Priorities Reform Priorities Appraisal and Independent Review À Introduce simplified À Strengthen the CC criteria in project Institutionalize formal appraisal appraisal methods for any appraisal methodologies and EIA for all project proposals by: type of post-war capital methodologies and begin to generate À Extending current procedures investment projects, even forward-looking information about to all types of projects and for major reconstruction variables important for project design and funding sources projects, focusing on appraisal which will be affected by CC. ensuring that demand for À Developing methodological services from the project guidelines - national and is urgent, unmet by other sectoral means, and likely to return À Instituting continuous to pre-war levels. Verify that capacity development opportunities to "build back better" have been exploited, especially in relation to climate change, both in terms of climate change mitigation and adaptation. À Make line ministries responsible for not slicing up projects to fit below the threshold. All related project profiles should be merged in one project at their screening stage, when MDA considers concept notes and takes a decision on whether to proceed with project development. À Give to the MoE the right and responsibility to reject "projects" that are obviously subsets of bigger projects and send them back to the ministries for proper configuration. Chapter 7. Conclusions and Strategic Recommendations 165 Prerequisites for Short term and Post-War PIM Reform Core Mid and Long-Term PIM infrastructure recovery Priorities Reform Priorities Budgeting À Authorize the MoF to À Allow only projects appraised according to À Strengthen mid-term perform a gatekeeper established rules to be selected. financial programming of function to block projects À Introduce clear and transparent selection public investment projects from entering the budget criteria scaled to the type and complexity within available fiscal space. by the initiative of both of the project that give a heavy weighting À Improve the content of government entities and to readiness for implementation, alignment budgetary information Parliament (through the with strategic post-war priorities, and available to decision-makers Budget Code) if they have the availability of financial means for and the public on all relevant not been properly appraised sustainable operations and maintenance. capital investment projects. and selected. The capacity of the project owner to À Extend financial reports manage the project and the procurement by including SOE capital strategy should also be given a heavy investments that are funded weighting. with budgetary lending À Strengthen realistic fiscal forecasting with and state guarantees (by a mid-term perspective for reconstruction projects). spending and ensure that only projects that can be fully funded from beginning to end are selected for funding. Implementation and Adjustment À Introduce agile procurement À Introduce strengthened project À Introduce uniform project for reconstruction while management procedures and management procedures and maintaining adequate methods, with an emphasis on defining methods across government. discipline, and consider clear accountability to ensure that À Strengthen active monitoring available institutional reconstruction projects are delivered on with a focus on controlling mechanisms to decrease time and budget. costs and identifying and corruption risks for the À Establish unified monitoring of rectifying poorly performing procurement process. the implementation of the whole projects. À Develop an approach to reconstruction program (regardless À Improve the mechanism for pooling and prioritizing of funding source) and ensure timely project reassessment by access to scarce project adjustment of projects in the event enforcing strict application of management skills. of setbacks by the authority which is triggers and consequences responsible for the supervision. for breaching them. À Introduce a mandatory external audit of the implementation of all major investment projects with publication of the results. 166 Chapter 7. Conclusions and Strategic Recommendations Prerequisites for Short term and Post-War PIM Reform Core Mid and Long-Term PIM infrastructure recovery Priorities Reform Priorities Operations À Ensure sustainable funding of operations À Strengthen monitoring for reconstructed facilities so that benefits of service delivery and are realized. reporting on service delivery performance from newly completed projects. À Modernize asset management. Completion Review À Establish basic completion review of À Extend obligatory completion all reconstruction projects and rapid reviews - with an analytical analysis of results by central agencies component – to all projects. to feed lessons back into the continuing À Introduce selective ex-post reconstruction program. evaluation of project effectiveness and impact. Chapter 7. Conclusions and Strategic Recommendations 167 EX TOOLS WITH A CLIMATE 1 CHANGE FOCUS FOR THE ANN ASSESSMENT FRAMEWORK Tool I: Dimensions of PEFA Climate Indicator CRPFM-5 – Climate Responsive Public Investment Management Dimension Comment CRPFM–5.1 Climate related provisions in regulatory framework for public investment management CRPFM–5.2 Climate related project selection Relates directly to PIM-7 CRPFM–5.3 Climate related provisions for project appraisal Relates directly to PIM-3 CRPFM–5.4 Reporting from entities in charge of implementation Relates directly to PIM-17 Tool II: InfraGov Dimension 5 Incorporating resilience to climate change, natural disasters, and public health risks is important for infrastructure outcomes. InfraGov guiding questions for Dimension 5: Question Comment 1. Are infrastructure projects aligned with national strategies and international Relates to PIM-2 commitments on climate change (e.g., on transitioning to long-term, low-emissions strategies)? 2. Are there mechanisms in place to monitor and mitigate environmental and climate Relates to PIM-20 change risks throughout operation and maintenance, and possible disposal? and PIM-21 3. What is the status of incorporation of disaster risk management and climate change Relates to PIM-1 to adaptation in national public investment systems? PIM-7 4. Are there well-designed disaster risk finance and insurance mechanisms in place to Outside scope of help incentivize resilient infrastructure through the financing of preventive measures? the PIM assessment 168 Annex 1. Tools with a Climate Change Focus for the Assessment Framework EX SUMMARY OF EXISTING 2 LAWS, RESOLUTIONS AND ANN ORDERS WITH A BEARING ON CLIMATE CHANGE AND DISASTER RISK MANAGEMENT Name of the Law or legislative act Content Regulations directly related to CC 1 The Law of Ukraine on monitoring, reporting, and Sets rules for monitoring and reporting of verification of GHG emissions GHG emissions of operating facilities 2 Resolution of the CMU No. 880 as of 23.09.2020 On Specifies the Law on Monitoring, approval of the list of activities, GHG emissions of which Reporting, and Verification of GHG are subject to monitoring, reporting and verification emissions 3 Resolution of CMU No. 960 as of 23.09.2020 On approval Specifies the Law on Monitoring, of the procedure for monitoring and reporting on Reporting, and Verification of GHG greenhouse gas emissions emissions Regulations on disaster management 4 Order No. 637 of the Ministry of Labor and Social Policy Provides methodology for identification of of Ukraine as of 04.12.2002 on approval of methodology disaster risks for definition of risks and their acceptable levels for declaration of safety of objects of increased danger 5 Order No. 98 of the Ministry for Emergencies and Provides methodology for identification of Protection of the Population from the Consequences of disaster risks the Chornobyl Disaster (reorganized to State Emergency Service of Ukraine) as of 23.02.2006 on Approval of the Methodology for identification of potentially dangerous objects (facilities/assets) Regulations on environmental impact 6 The Law of Ukraine on Strategic Environmental Regulates strategic environmental Assessment, adopted in 2018 assessment of draft public planning documents and complements the procedures for the preparation and adoption of public planning documents ensuring the strategic environmental assessment thereof. Annex 2. Summary of Existing Laws, Resolutions and Orders with a Bearing on Climate Change and Disaster Risk Management 169 Name of the Law or legislative act Content 7 Resolution No. 1272 of CMU as of 16.12.2020 on approval Sets rules for monitoring according to the of the procedure for monitoring of consequences of Law of Ukraine on Strategic Environmental execution of the document of the state planning for Assessment environment, including for health of the population 8 The Law of Ukraine on regulation of construction activity Requires environmental impact (adopted in 2011) assessment of certain type of construction projects 9 Order No. 45 of the Ministry for Development of Requires EIA of certain type of Territories and Communities on development of the construction projects design documentation for construction 10 Resolution of the CMU No. 1026 as of 13/12/2017 Sets the rules for EIA of construction On approval of the procedure for submission of projects documentation for issuing an opinion on environmental impact assessment and financing of environmental impact assessment and the procedure for maintaining the Unified Register of Environmental Impact Assessment 11 Order of the Ministry of Environmental Protection and Guidelines for EIA required by the Natural Resources of Ukraine No. 193 as of 15.03.2021 Resolution No. 1026 On approval of the general methodological recommendations on the content and procedure for compiling reports on environmental impact assessment 12 Order No. 136 as of 02/03/2020 of the Ministry of Energy Guidelines for EIA in forestry sector and Environmental Protection of Ukraine required by the Resolution No. 1026 on approval of the methodological recommendations for the development of the report on environmental impact assessment in the field of forestry 13 Resolution of CMU No. 571 as of 22.07.2015 on some Requires environmental impact issues of public investment management assessment within public investment project appraisal 14 Order of MoE No. 1865 as of 22.12.2017 on approval Details environmental impact assessment of methodical recommendations for preparation and within public investment project appraisal appraisal of public investment project Regulations on design of infrastructure 15 Composition and content of materials for impact Approved in 2003, last changed in 2010 assessment on environment at design and construction of  https://www.minregion.gov.ua/wp- enterprises, buildings and other structures. content/uploads/2017/12/4.1.-DBN-A.2.2- 1-2003.-Sklad-i-zmist-materialiv-otsinki.pdf  16 Basic requirements to buildings and structures, Approved in 2008, last changed in 2020 mechanical resistance and resilience.  https://www.minregion.gov.ua/ wp-content/uploads/2017/03/ DBN-V.1.2-6-2008.pdf  170 Annex 2. Summary of Existing Laws, Resolutions and Orders with a Bearing on Climate Change and Disaster Risk Management Name of the Law or legislative act Content 17 Basic requirements to buildings and structures, energy Approved in 2008 saving.  https://www.minregion.gov.ua/ wp-content/uploads/2017/03/ DBN-V.1.2-11-2008.pdf  18 System for ensuring the reliability and safety of Approved in 2009, last changed in 2018. construction sites. General principles of ensuring the  https://www.minregion.gov.ua/ reliability and structural safety of buildings and structures. wp-content/uploads/2017/04/ DBN-V.1.2-14-2009.pdf  19 System for ensuring the reliability and safety of  https://www.minregion.gov.ua/wp- construction sites. Loads and impacts. content/uploads/2017/12/37.1.-DBN-V.1.2- 22006.-SNBB.-Navantazhennya-i-vplivi.- Nor.pdf  Source: World Bank Mission Team Annex 2. Summary of Existing Laws, Resolutions and Orders with a Bearing on Climate Change and Disaster Risk Management 171 EX DETAILS OF LEGISLATION 3 REGULATING CAPITAL ANN INVESTMENT STREAMS Capital Agency Critical components of a functioning PIM system Applied to Investment Responsible Stream for Stream Project Appraisal Project Selection Project Monitoring SOEs Supervision and Expertise 1 State Ministry of 1. Resolution No. 1. Resolution No. 1. Resolution No. Investment Economy 571 dated July 22, 571 dated July 22, 571 dated July 22,  Projects 2015 (including 2015 (including 2015 (including environmental positive ecological environmental impact) effect) impact) 2. Order of MoE 2. Order of MoE No. 1865 dated No. 1785 dated December 22, December 22, 2017 2017 3. Resolution No. 903 dated July 11, 2007 (construction legislation) 2 PPP Ministry of 1. Law on 1. Law on 1. Law on Economy concessions No. concessions No. 155 concessions No. 155  155 dated October dated October 3, dated October 3, 3, 2019 2019 2019 2. Law on PPP No. 2. Law on PPP No. 2. Law on PPP No. 2404 dated July 1, 2404 dated July 1, 2404 dated July 1, 2010 2010 2010 3. PPP manual 3. PPP 3. PPP manual (including announcement (including environmental about the environmental impact assessment) competition performance 4. PPP manual indicators) (including criteria 4. Resolution No. of Environmental 903 dated July 11, Compliance and 2007 (construction Environmental Impact legislation) Assessments) 172 Annex 3. Details of Legislation Regulating Capital Investment Streams Capital Agency Critical components of a functioning PIM system Applied to Investment Responsible Stream for Stream Project Appraisal Project Selection Project Monitoring SOEs Supervision and Expertise 3 Capital State Road No rules 1. Order of Ministry 1. Resolution No. investments Agency of infrastructure and 903 dated July 11,  within the State MoF No. 573/1019 2007 (construction Road Fund (the dated September legislation) national level) 21, 2012 (including minimal negative impact of the project on the environment) 4 Subvention State Road No rules 1. Oblast level rules 1. Resolution No. from the central Agency 903 dated July 11,  budget to local 2007 (construction budgets within legislation) the State Road Fund 5 Environmental Ministry No rules 1. Order of Ministry 1. Resolution No. protection of Ecology of Ecology and 903 dated July 11,  measures and Natural Natural Resources 2007 (construction Resources No. 194 dated June legislation) 12, 2015 6 Water Ministry No rules No rules 1. Resolution No. Management of Ecology 903 dated July 11,  Development and Natural 2007 (construction Fund Resources legislation) 7 Regional Ministry for 1. Resolution No. 1. Resolution No. 1. Resolution No. 196 Development Communities 196 dated March 196 dated March 18, dated March 18, 2015  Fund and 18, 2015 2015 2. Resolution No. Territories 903 dated July 11, Development 2007 (construction legislation) 8 IFI funding and Ministry of 1. Resolution No. 1. Resolution No. 70 1. Resolution No. 70 credit funds Finance 70 dated January dated January 27, 2. Resolution No.  through the 27, 2016 2016 903 dated July 11, budget 2007 (construction legislation) 9 IFI credit Ministry of 1. Resolution No. 1. Resolution No. 70 1. Resolution No. 70 funds by state Finance 70 dated January dated January 27, 2. Resolution No.  guarantees 27, 2016 2016 903 dated July 11, 2007 (construction legislation) 10 Subvention on Ministry of No rules No rules 1. Resolution No. socio-economic Finance 903 dated July 11,  development of 2007 (construction territories legislation) Annex 3. Details of Legislation Regulating Capital Investment Streams 173 Capital Agency Critical components of a functioning PIM system Applied to Investment Responsible Stream for Stream Project Appraisal Project Selection Project Monitoring SOEs Supervision and Expertise 11 Other capital Ministry of No rules No rules 1. Resolution No. investments Finance 903 dated July 11,  which are 2007 (construction not identified legislation) as "State Investment Project" 12 Other Sectoral No rules No rules 1. Resolution No. subventions ministries 903 dated July 11,  with capital 2007 (construction investment legislation) nature 13 Internal credit Sectoral 1. Order of MoE 1. Resolution No. 1. Resolution No. funds by state ministries, No. 724 dated June 835 dated November 903 dated July 11,  guarantees Ministry of 19, 2012 13, 2013 2007 (construction Finance 2. Order of MoE No. legislation) 1279 dated November 13, 2012 3. Resolution No. 701 dated June 9, 2011 4. Order of MoE No. 243 dated March 13, 2013 5. Resolution No. 131 dated February 23, 2011 14 Infrastructure Sectoral 1. Resolution 1. Resolution No. 1. Resolution No. bonds289 ministries, No. 1245 dated 1245 November 17, 1245 November 17,  Ministry of November 17, 2021 2021 Economy, 2021 (Including conclusion 2. Resolution No. Interagency 2. Order of MoE on environmental 903 dated July 11, commission No. 724 dated June impact assessment 2007 (construction 19, 2012 in relation to projects legislation) 3. Order of MoE that may have a No. 243 dated significant impact on March 13, 2013 the environment) 15 SOEs own funds Sectoral No rules No rules 1. Resolution No. ministries, 903 dated July 11,  Ministry of 2007 (construction Finance legislation)  289  New stream established after the period covered by the assessment: amendments to law on Economy and Commodity Markets (No. 738 of June 19, 2020). 174 Annex 3. Details of Legislation Regulating Capital Investment Streams EX SCORES OF THE 2021 PIM 4 ANN DA INDICATORS PIM Indicator Scoring Dimension score Overall method score i ii iii iv v PIM Function 1. Guidance & screening PIM-1. Sector analysis and planning M2 C C C PIM-2. Strategic plans & investment guidance, project M2 C D C D D+ development and preliminary screening PIM Function 2. Formal appraisal PIM-3. Formal project appraisal procedures and guideline M2 A B C C C C+ PIM-4. Project appraisal capacity M2 C D D+ PIM-5. Screening and selection of feasibility studies M2 C D D D+ PIM Function 3. Appraisal review PIM-6. Independent review of appraisal M2 C C C PIM Function 4. Selection & budgeting 4.A. Budget preparation & selection PIM-7. Project selection & budgeting M1 C C C C C PIM-8. Multi-year budgeting M2 D C D+ PIM-9. Comprehensive capital budget M2 D A C+ PIM-10. Comprehensiveness and degree of public/ M2 C A A C B parliamentary access to capital budget information 4.B. Budget outturn performance PIM-11. Development and capital budget execution rates: M1 D D D Aggregate expenditure outturn compared to adjusted original budget on a commitment basis PIM-12. Composition of development and capital expenditure M1 C C outturn compared to adjusted original budget on a commitment basis Annex 4. Scores of the 2021 PIM DA Indicators 175 PIM Indicator Scoring Dimension score Overall method score i ii iii iv v PIM-13. Project completion time and cost variances for M1 D D completed projects PIM-14. Stock and monitoring of capital expenditure arrears M1 A A A PIM Function 5. Implementation PIM-15. Procurement M2 A A D A A B+ PIM-16. Project implementation management M2 C C C C PIM-17. Control, monitoring & reporting: physical and financial M2 C D A B C milestones PIM-18. Project handover, asset registration and completion M2 C B D D D D+ review PIM Function 6. Adjustment PIM-19. Project adjustment M1 A B C C+ PIM Function 7. Service delivery PIM-20. Control, monitoring and reporting: financial and M1 C С B B B С+ service delivery performance PIM-21. Service delivery M2 D D D PIM Function 8. Evaluation PIM-22. Scope, nature and follow-up of external audit and M2 C D D D+ ex-post evaluation PIM-23. Legislative scrutiny of external audit reports M1 A B A B+ 176 Annex 4. Scores of the 2021 PIM DA Indicators EX CALCULATIONS FOR PIM-12, 2018 5 ANN (UAH THOUSAND) Originally Originally approved Absolute Deviation, # Name of the ministry (department) approved Actual Deviation adjusted deviation % budget budget 1 Ministry of Defense of Ukraine 25,464,406.4 33,067,448.2 25,599,662.4 7,467,785.7 7,467,785.7 29.2% 2 State Road Agency of Ukraine (including nationwide expenditures and crediting) 33,777,103.6 24,022,757.0 33,956,513.1 -9,933,756.1 9,933,756.1 29.3% 3 Ministry of Regional Development, Construction, Housing and Municipal 13,518,972.2 9,366,699.8 13,590,779.2 -4,224,079.4 4,224,079.4 31.1% Services of Ukraine (including nationwide expenditures and crediting) 4 Ministry of Internal Affairs of Ukraine 7,916,955.4 12,071,974.9 7,959,006.9 4,112,968.0 4,112,968.0 51.7% 5 Ministry of Agrarian Policy and Food of Ukraine 7,030,743.1 5,019,601.2 7,068,087.4 -2,048,486.2 2,048,486.2 29.0% 6 Ministry of Finance of Ukraine (including nationwide expenditures and crediting) 8,725,544.0 7,685,473.1 8,771,890.4 -1,086,417.2 1,086,417.2 12.4% 7 National Academy of Sciences of Ukraine 4,303,385.6 4,415,322.5 4,326,243.3 89,079.1 89,079.1 2.1% 8 Ministry of Energy and Coal Industry of Ukraine 4,246,437.6 3,259,910.1 4,268,992.9 -1,009,082.8 1,009,082.8 23.6% 9 Ministry of Economic Development and Trade of Ukraine 4,129,109.7 5,835,947.9 4,151,041.8 1,684,906.1 1,684,906.1 40.6% 10 Ministry of Education and Science of Ukraine (including nationwide expenditures 4,280,770.5 5,047,583.6 4,303,508.1 744,075.5 744,075.5 17.3% and crediting) 11 Ministry of Ecology and Natural Resources of Ukraine 3,012,708.1 3,784,890.4 3,028,710.3 756,180.1 756,180.1 25.0% 12 Ministry of Infrastructure of Ukraine 2,629,848.2 970,245.4 2,643,816.8 -1,673,571.4 1,673,571.4 63.3% 13 State Space Agency of Ukraine 2,173,551.8 2,072,011.8 2,185,096.8 -113,085.0 113,085.0 5.2% Annex 5. Calculations for PIM-12, 2018 (UAH Thousand) 177 178 Annex 5. Calculations for PIM-12, 2018 (UAH Thousand) Originally Originally approved Absolute Deviation, # Name of the ministry (department) approved Actual Deviation adjusted deviation % budget budget 14 Ministry of Health of Ukraine (including nationwide expenditures and crediting) 3,776,553.1 3,397,795.3 3,796,612.5 -398,817.2 398,817.2 10.5% 15 State Judicial Administration of Ukraine 1,611,506.7 1,818,036.3 1,620,066.3 197,970.0 197,970.0 12.2% 16 Ministry of Social Policy of Ukraine (including nationwide expenditures and 1,533,120.5 1,838,620.0 1,541,263.8 297,356.3 297,356.3 19.3% crediting) 17 National Academy of Agrarian Sciences of Ukraine 1,196,565.0 1,154,637.6 1,202,920.6 -48,283.0 48,283.0 4.0% 18 Chief Directorate of Intelligence of the Ministry of Defense of Ukraine 930,923.0 1,005,141.5 935,867.7 69,273.9 69,273.9 7.4% 19 Ministry of Culture of Ukraine 861,164.3 971,011.5 865,738.4 105,273.1 105,273.1 12.2% 20 State Service of Special Communications and Information Protection of Ukraine 745,433.5 829,267.9 749,392.9 79,875.0 79,875.0 10.7% Other Ministries (departments) 4,796,512.6 9,752,825.1 4,821,989.6 4,930,835.5 4,930,835.5 102.3% share of other MDAs in total development expenditures 3.5 7.1 3.5 х х х Total composition of development expenditures across agencies 136,661,314.9 137,387,201.2 137,387,201.2 x 41,071,156.5 29.9% The absolute overall deviation in development expenditures (PIM-11) 3.0% Variance 26.9% EX CALCULATIONS FOR PIM-12, 2019 6 ANN (UAH THOUSAND) Originally Originally approved Absolute Deviation, # Name of the ministry (department) approved Actual Deviation adjusted deviation % budget budget 1 State Road Agency of Ukraine (including nationwide expenditures and crediting) 29,650,173.3 11,952,775.6 15,823,011.5 -3,870,236.0 3,870,236.0 24.5% 2 Ministry of Defense of Ukraine 24,718,308.4 5,675,264.7 13,191,089.1 -7,515,824.4 7,515,824.4 57.0% 3 Ministry of regional development, construction, housing and municipal services 14,452,755.7 10,463,075.6 7,712,808.9 2,750,266.7 2,750,266.7 35.7% of Ukraine (including nationwide expenditures and crediting) 4 Ministry of Internal Affairs of Ukraine 11,687,592.9 13,884,341.3 6,237,161.4 7,647,179.9 7,647,179.9 122.6% 5 Ministry of Economic Development, Trade and Agriculture of Ukraine 13,278,097.5 5,098,096.0 7,085,944.8 -1,987,848.7 1,987,848.7 28.1% 6 Ministry of Finance of Ukraine (including nationwide expenditures and crediting) 7,546,040.9 4,284,836.5 4,026,994.8 257,841.7 257,841.7 6.4% 7 Ministry of Education and Science of Ukraine (including nationwide expenditures 9,656,856.9 7,183,076.1 5,153,445.7 2,029,630.4 2,029,630.4 39.4% and crediting) 8 National Academy of Sciences of Ukraine 4,775,080.7 603,584.9 2,548,253.5 -1,944,668.6 1,944,668.6 76.3% 9 Ministry of Energy and Environmental Protection of Ukraine (including 7,540,392.3 2,309,735.3 4,023,980.3 -1,714,245.1 1,714,245.1 42.6% nationwide expenditures and crediting) 10 Ministry of Health of Ukraine (including nationwide expenditures and crediting) 4,426,391.7 3,551,863.3 2,362,173.3 1,189,690.0 1,189,690.0 50.4% 11 Ministry of Infrastructure of Ukraine 2,368,337.7 876,660.5 1,263,879.1 -387,218.6 387,218.6 30.6% 12 State Space Agency of Ukraine 1,806,196.7 37,056.7 963,888.8 -926,832.2 926,832.2 96.2% Annex 6. Calculations for PIM-12, 2019 (UAH Thousand) 179 180 Annex 6. Calculations for PIM-12, 2019 (UAH Thousand) Originally Originally approved Absolute Deviation, # Name of the ministry (department) approved Actual Deviation adjusted deviation % budget budget 13 Ministry of Social Policy of Ukraine (including nationwide expenditures and 2,019,442.0 1,852,635.0 1,077,688.6 774,946.4 774,946.4 71.9% crediting) 14 State Judicial Administration of Ukraine 1,358,196.7 1,718,698.4 724,810.7 993,887.8 993,887.8 137.1% 15 National Academy of Agrarian Sciences of Ukraine 1,221,012.4 41,592.9 651,601.4 -610,008.5 610,008.5 93.6% 16 National Academy of Medical Sciences of Ukraine 1,015,527.0 466,269.7 541,942.7 -75,673.0 75,673.0 14.0% 17 Ministry of Culture of Ukraine 4,500.0 840.0 2,401.5 -1,561.5 1,561.5 65.0% 18 Chief Directorate of Intelligence of the Ministry of Defense of Ukraine 887,865.5 162,482.7 473,815.3 -311,332.6 311,332.6 65.7% 19 State Service of Special Communications and Information Protection of Ukraine 886,165.1 75,270.1 472,907.9 -397,637.8 397,637.8 84.1% 20 Foreign Intelligence Service of Ukraine 626,002.7 189,086.9 334,070.5 -144,983.6 144,983.6 43.4% Other Ministries (departments) 2,842,210.9 5,761,392.4 1,516,764.7 4,244,627.7 4,244,627.7 279.8% share of other MDAs in total development expenditures 2.0 7.6 2.0 х Total composition of development expenditures across agencies 142,767,147.0 76,188,634.5 76,188,634.5 x 39,776,141.2 52.2% The absolute overall deviation in development expenditures (PIM-11) 43.6% Variance 8.6% EX CALCULATIONS FOR PIM-12, 2020 7 ANN (UAH THOUSAND) Originally Originally approved Absolute Deviation, # Name of the ministry (department) approved Actual Deviation adjusted deviation % budget budget 1 State Road Agency of Ukraine (including nationwide expenditures and crediting) 57,732,622.8 38,096,940.6 31,126,979.8 6,969,960.8 6,969,960.8 22.4% 2 Ministry of Defense of Ukraine 32,163,176.9 5,368,555.0 17,341,019.8 -11,972,464.8 11,972,464.8 69.0% 3 Ministry for Communities and Territories Development of Ukraine (including 14,843,134.9 9,158,672.6 8,002,788.3 1,155,884.3 1,155,884.3 14.4% nationwide expenditures and crediting) 4 Ministry of Internal Affairs of Ukraine 11,687,592.9 10,544,541.3 6,301,454.0 4,243,087.3 4,243,087.3 67.3% 5 Ministry of Energy and Environmental Protection of Ukraine (including 8,945,222.3 1,079,931.5 4,822,884.2 -3,742,952.7 3,742,952.7 77.6% nationwide expenditures and crediting) 6 Ministry of Economic Development, Trade and Agriculture of Ukraine 8,704,465.3 2,984,088.9 4,693,078.2 -1,708,989.3 1,708,989.3 36.4% 7 Ministry of Education and Science of Ukraine (including nationwide 9,717,961.9 3,927,308.7 5,239,512.6 -1,312,203.9 1,312,203.9 25.0% expenditures and crediting) 8 National Academy of Sciences of Ukraine 4,779,737.4 267,266.2 2,577,031.5 -2,309,765.3 2,309,765.3 89.6% 9 Ministry of Infrastructure of Ukraine 4,213,266.3 2,195,259.8 2,271,614.4 -76,354.5 76,354.5 3.4% 10 Ministry of Finance of Ukraine (including nationwide expenditures and 4,608,835.5 5,153,327.9 2,484,888.5 2,668,439.4 2,668,439.4 107.4% crediting) 11 Ministry of Health of Ukraine (including nationwide expenditures and crediting) 4,395,605.9 8,996,118.8 2,369,924.1 6,626,194.7 6,626,194.7 279.6% Annex 7. Calculations for PIM-12, 2020 (UAH Thousand) 181 182 Annex 7. Calculations for PIM-12, 2020 (UAH Thousand) Originally Originally approved Absolute Deviation, # Name of the ministry (department) approved Actual Deviation adjusted deviation % budget budget 12 Ministry of Culture, Youths and Sports of Ukraine (including nationwide 2,630,350.8 293,608.8 1,418,173.5 -1,124,564.7 1,124,564.7 79.3% expenditures and crediting) 13 National Academy of Agrarian Sciences of Ukraine 1,438,744.7 44,595.8 775,710.1 -731,114.3 731,114.3 94.3% 14 Ministry of Veterans' Affairs, Temporarily Occupied Territories and Internally 1,323,938.7 658,067.1 713,811.6 -55,744.4 55,744.4 7.8% Displaced Persons of Ukraine (including nationwide expenditures and crediting) 15 Main Directorate of Intelligence of the Ministry of Defense of Ukraine 1,196,270.9 184,333.6 644,978.5 -460,644.9 460,644.9 71.4% 16 Ministry of Social Policy of Ukraine (including nationwide expenditures and 1,897,239.9 825,866.0 1,022,911.2 -197,045.3 197,045.3 19.3% crediting) 17 National Academy of Medical Sciences of Ukraine 743,695.9 447,053.7 400,969.3 46,084.4 46,084.4 11.5% 18 Foreign Intelligence Service of Ukraine 673,321.0 260,226.9 363,026.1 -102,799.2 102,799.2 28.3% 19 State Service of Special Communications and Information Protection of Ukraine 645,739.5 86,101.9 348,155.3 -262,053.5 262,053.5 75.3% 20 Security Service of Ukraine 617,085.2 593,042.2 332,706.1 260,336.0 260,336.0 78.2% Other Ministries (departments) 3,106,969.1 3,861,610.6 1,675,145.9 2,186,464.8 2,186,464.8 130.5% share of other MDAs in total development expenditures 1.8 4.1 1.8 х х х Total composition of development expenditures across agencies 176,064,977.8 94,926,763.0 94,926,763.0 x 48,213,148.4 50.8% The absolute overall deviation in development expenditures (PIM-11) 44.5% Variance 6.3% EX PROJECT COMPLETION TIME AND COST 8 ANN VARIANCES OF COMPLETED PROJECTS The total capital cost of the project Project completion time the ratio of the project the ratio final (actual) inflation- final original, final, year of the final to the project original, final, Name of the project original, UAH adjusted (actual), year of the of the number of inflation- number of number thousand construction UAH beginning beginning years to adjusted years of years cost, UAH thousand and the end and the end the initial, construction, thousand percent percent Weighted average overruns X X X 130.8% X X X X 132.7% Restoration and adaptation of the Mariinsky Palace at 5a 1,305,536.0 1,305,536.0 922,807.8 70.7% 2002-2018 2002-2018 17 17 100.0% Hrushevskoho Street, Kyiv Restoration with redevelopment of attics in the building No. 5. on 4,802.8 4,945.8 12,686.4 256.5% 2015-2018 2015-2019 4 5 125.0% the Nyzhankivsky Street for the educational premises of the Lviv National Music Academy named MV Lysenko within the volume of the existing attic Reconstruction with re-profiling of the unfinished construction of 39,757.1 122,324.7 122,324.7 100.0% 2007-2010 2007-2019 3 13 433.3% the House of Culture in Ostroh for the academic building of the National University 'Ostroh Academy' Reconstruction of the educational building No. 4 of Cherkasy 60,130.5 60,130.5 57,788.4 96.1% 2018-2020 2018-2019 3 2 66.7% National University maned Bohdan Khmelnytsky - architectural monuments of local significance at the address: Cherkasy, Ostafy Dashkovych Street, 24 State support for the construction of mine No. 10 N/A 4,457,239.5 N/A N/A 1989-2020 1989-2020 31 31 100.0% 'Novovolynskaya' Construction and reconstruction of public highways in the N/A 109,992.0 N/A N/A 2018-2019 2018-2019 2 2 100.0% Khmelnytsky region Annex 8. Project Completion Time and Cost Variances of Completed Projects 183 184 Annex 8. Project Completion Time and Cost Variances of Completed Projects The total capital cost of the project Project completion time the ratio of the project the ratio final (actual) inflation- final original, final, year of the final to the project original, final, Name of the project original, UAH adjusted (actual), year of the of the number of inflation- number of number thousand construction UAH beginning beginning years to adjusted years of years cost, UAH thousand and the end and the end the initial, construction, thousand percent percent Construction and reconstruction of public highways in the N/A 204,612.4 N/A N/A 2018-2018 2018-2018 1 1 100.0% Cherkasy region Subvention from the state budget to the oblast budget of the N/A 19,000.0 N/A N/A 2018-2018 2018-2018 1 1 100.0% Kherson oblast for the construction of an overpass on ave. Admiral Senyavin - St. Zalaegerseg in the city of Kherson Construction of real estate for the needs of foreign diplomatic N/A 3,360.0 N/A N/A 2018-2018 2018-2019 1 2 200.0% institutions of Ukraine Construction (acquisition) of the territorial administration N/A 74,601.0 N/A N/A 2008-2019 2008-2019 11 11 100.0% of the State Judicial Administration in the Chernivtsi oblast (Storozhynetsky district) Reconstruction of existing non-residential buildings No. 1, 2, 11 of N/A 199,905.5 N/A N/A 2017-2019 2017-2019 3 3 100.0% a military town with the placement of the Odessa Administrative Court of Appeal at the address: Odessa, Primorsky District, Gagarina Avenue, 19-21 (adjustment for the completion of construction) Reconstruction of the stadium 'Trudovye Rezervy,' N/A 213,720.1 N/A N/A 2017-2020 2017-2020 4 4 100.0% Dnepropetrovsk. Indoor sports demonstration facility for sports games. Adjustment Construction of the Tatarbunarsky group water pipeline in the N/A 19,943,8 N/A N/A 2018-2020 2018-2020 3 3 100.0% Odessa region and the turn. Platforms for water towers near the village. Glubokoe and near the village. Borisovka, Tatarbunarsky district, Odessa region 186 Annex 8. Project Completion Time and Cost Variances of Completed Projects