The World Bank Group in Ukraine, 2012–20 Country Program Evaluation © 2022 International Bank for Reconstruction and Development / The World Bank 1818 H Street NW Washington, DC 20433 Telephone: 202-473-1000 Internet: www.worldbank.org ATTRIBUTION Please cite the report as: World Bank. 2022. The World Bank Group in Ukraine, 2012–20. Country Program Evaluation. Independent Evaluation Group. Washington, DC: World Bank. COVER PHOTO Adobe Stock / maksym EDITING AND PRODUCTION Amanda O’Brien GRAPHIC DESIGN Luísa Ulhoa This work is a product of the staff of The World Bank with external contributions. The findings, interpretations, and conclusions expressed in this work do not necessarily reflect the views of The World Bank, its Board of Executive Directors, or the governments they represent. The World Bank does not guarantee the accuracy of the data included in this work. The bound- aries, colors, denominations, and other information shown on any map in this work do not imply any judgment on the part of The World Bank concerning the legal status of any territory or the endorsement or acceptance of such boundaries. RIGHTS AND PERMISSIONS The material in this work is subject to copyright. Because The World Bank encourages dissem- ination of its knowledge, this work may be reproduced, in whole or in part, for noncommercial purposes as long as full attribution to this work is given. Any queries on rights and licenses, including subsidiary rights, should be addressed to World Bank Publications, The World Bank Group, 1818 H Street NW, Washington, DC 20433, USA; fax: 202-522-2625; e-mail: pubrights@worldbank.org. The World Bank Group in Ukraine, 2012–20 Country Program Evaluation November 8, 2022 Contents Abbreviations v Acknowledgments vi Overview vii Огляд xiii Bibliography 46 1. Background���������������������������������������������������������������������������������������������������������������1 Country Context 1 World Bank Group–Supported Program 4 2. Governance and Anticorruption�����������������������������������������������������������������������������12 Anticorruption Institutions 14 Public Financial Management 21 Crisis Response and Macroeconomic Resilience�������������������������������������������������25 3.  Enhancing Macroeconomic Resilience 26 Reducing Financial Sector Vulnerabilities 29 Energy Security and Efficiency������������������������������������������������������������������������������34 4.  Sector Governance and Transparency 35 Energy Efficiency 37 Energy Security 39 5. Findings and Lessons���������������������������������������������������������������������������������������������41 Findings 41 Lessons 44 ii Boxes World Bank Group Support for Enhancing the Capacity of the National Box 3.1.  Bank of Ukraine and the Deposit Guarantee Fund 31 Figures Figure 1.1. Major Political Milestones 3 World Bank Commitments to Ukraine by Financing Instrument, Figure 1.2.  Fiscal Years 2012–20 5 Figure 1.3. Ukraine Commitments by Sector, Fiscal Years 2012–21 6 Ukraine Advisory Services and Analytics Delivery by Sector, | Figure 1.4.  Fiscal Years 2012–21 7 Evolution of the World Bank Group Strategy, Fiscal Years 2012–21 Figure 1.5.  9 Figure 2.1. Control of Corruption 15 Tables Table 1.1. Key Economic and Social Indicators 2 Independent Evaluation Group Performance Ratings for Ukraine, Table 1.2.  Fiscal Years 2012–19 7 Table 2.1. Ukraine: Select Governance Indicators, 2010–19 13 Table 2.2. Summary of Anticorruption Reforms in Sector Programs 19 Table 4.1. Ukraine Energy Supply and Transit 38 iii Appendixes Appendix A. World Bank Group Support in Other Areas 58 Appendix B. Methodology 67  esults of Surveys of Public Perceptions of Service Delivery Appendix C. R and Online Media Analysis 70 World Bank Group Support to Address Fiscal and Financial Appendix D.  Sector Vulnerabilities in Ukraine 74 Appendix E. World Bank Group–Supported Program in Ukraine 87 iv Abbreviations ASA advisory services and analytics CPE Country Program Evaluation CPF Country Partnership Framework CPS Country Partnership Strategy CSO civil society organization DGF Deposit Guarantee Fund DPL development policy loan EU European Union FY fiscal year GAC governance and anticorruption GDP gross domestic product HUS household utility subsidy IEG Independent Evaluation Group IFC International Finance Corporation IMF International Monetary Fund NACP National Agency on Corruption Prevention NPL nonperforming loan PEFA Public Expenditure and Financial Accountability PFM public financial management PIM public investment management Independent Evaluation Group World Bank Group    v All dollar amounts are US dollars unless otherwise indicated. Acknowledgments The report was prepared by an Independent Evaluation Group team led by Konstantin Atanesyan (senior evaluation officer) and Yumeka Hirano (econ- omist), under the guidance and supervision of Jeff Chelsky (manager) and Oscar Calvo-Gonzalez (director) and the overall direction of Alison Evans (Director-General, Evaluation, and vice president). The evaluation team included Independent Evaluation Group staff and consultants Yar Batoh, Corky De Asis, Lev Freinkman, Oksana Gryshkevych, Ruben Lamdany, Franz M. Loyola, Mariia Kriuchok, George Polenakis, Judyth Twigg, and Salman Zaheer. Harsh Anuj worked with the team on data collec- tion and analysis of online news media articles, and Victor Andres Beltran Vela supported the team on preparing, conducting, and analyzing the survey data. Dung Thi Kim Chu and Anastasia Soltis provided administrative sup- port to the evaluation team in Washington, DC, and Kyiv, respectively. Peer reviewers were Sergei Guriev (professor of economics at SciencesPo, Paris, and former chief economist at the European Bank for Reconstruction and Development), Johannes F. Linn (senior fellow at the Brookings Insti- The World Bank Group in Ukraine, 2012–20  Acknowledgments tution, former World Bank vice president for the Europe and Central Asia Region), and Ivanna Vladkova Hollar (International Monetary Fund mission chief for Ukraine). Independent Evaluation Group is grateful to the numerous representatives of the government of Ukraine, private sector entities, international and bilateral development partners, and representatives of nongovernmental organizations and academia who provided valuable insights into the World Bank Group’s support to Ukraine. The team is also thankful to Bank Group management and staff who provided valuable information and feedback to the evaluation team. vi Overview Highlights This evaluation assesses the performance of the World Bank Group partnership with Ukraine in 2012–20, focusing on support for gover- nance and anticorruption, crisis response and economic resilience, and energy security and efficiency. The Bank Group contributed to the establishment of apex anti- corruption institutions, promoting anticorruption reforms in some sectors (health, energy, social protection) and strengthening public financial management. At the same time, lack of direct engage- ment in justice sector and public administration reform diminished the impact of Bank Group support across the portfolio. The Bank Group was part of an international coalition that helped stabilize the Ukrainian economy after the 2014–15 crisis, making a significant contribution to restoring the health and stability of the banking system and enhancing the technical and institutional capacity of the National Bank of Ukraine and the Deposit Guaran- tee Fund. The International Finance Corporation was an important contributor to corporate governance and risk management in the financial sector. However, despite substantial strengthening of macroeconomic fundamentals, Ukraine’s economy still faces a variety of risks, aggravated by the COVID-19 pandemic. The Bank Group helped improve energy sector governance and put in place institutional arrangements to promote energy efficien- cy, including by playing a critical role in unbundling the natural gas monopoly, leading the dialogue on tariff and subsidy reforms, and helping to diversify energy supply. At the same time, reforms sup- ported by the World Bank have not led to significant private invest- ment in energy infrastructure modernization or tangible improve- ments in service to consumers. vii  Main lessons from this evaluation include the following: (i) sustained engagement, even when demand for reform was weak, and invest- ment in country knowledge and analytics positioned the Bank Group to respond quickly when a political window of opportunity presented itself; (ii) a lack of engagement on aspects of justice reform under- mined the impact of reforms in other areas, including the financial sector and anticorruption; in anticipation of significant engagement in Ukraine once military aggression against it ceases, the World Bank would be well advised to invest more in deepening its understand- ing of the links between specific weaknesses in the justice system and Ukraine’s ability to make progress on specific development objectives; (iii) public outreach and engagement to explain the rea- sons for reform and the costs of inaction are critical to sustain re- forms; and (iv) institutional reforms that involve painful adjustments to households, such as tariff increases, need to be accompanied by improvements in service quality, including in infrastructure. viii   This Country Program Evaluation (CPE) reviews the effectiveness of the World Bank Group partnership with Ukraine during 2012–20. It discusses (i) the extent to which Bank Group engagement was relevant to the country’s main development challenges, (ii) the contributions of the Bank Group–supported program to development outcomes, and (iii) whether the Bank Group effec- tively collaborated with development partners. The CPE also draws lessons for future engagement. This CPE contains a special focus on three of the main challenges faced by Ukraine during the evaluation period and in which the Bank Group played a significant role: governance and anticorruption, crisis response and economic resilience, and energy security and efficiency. Main Challenges and Outcomes Ukraine has significant economic potential, but its economic and social de- velopment over the past three decades has been slow and highly volatile be- cause of internal factors, exogenous shocks, slow adoption of market-based principles of economic management, and weak institutions. Many of the key development challenges, such as corruption, weak governance, lack of energy security, and ineffective public services, have persisted in the face of stop-and-go reform efforts. This evaluation finds that between 2012 and 2020, the Bank Group helped Ukraine undertake several important reforms and, in some areas, estab- lish foundations to break the cycle of reform and reversion. Before 2014, Independent Evaluation Group World Bank Group    ix and despite low government interest in reform, the Bank Group invested in building country knowledge and local partnerships and was well prepared to respond when a political opening for reform presented itself. Since 2014, after the change of government and subsequent economic and political cri- ses, the Bank Group has leveraged opportunities to significantly ramp up its activities and influence the trajectory of reform in Ukraine. The Bank Group has been able to provide effective support to the government to stabilize the economy and the financial sector, begin to tackle endemic corruption, reform the health and pension systems, and enhance energy security. At the same time, many challenges remain unresolved. In a highly volatile political environment such as in Ukraine, it is difficult to gauge how well the policy and structural changes facilitated by the Bank Group will withstand geopolitical risks, pressures from powerful vested interests, and shifts in the preferences of Ukrainian people and their leaders. Ukraine’s economy remains vulnerable to macroeconomic and political risks, aggravated by the uncertainties of the COVID-19 pandemic. Vested interests are still well organized, and populist pressures for policy reversal have increased, slow- ing progress and drawing into question the credibility of the government’s commitment to anticorruption and the sustainability of many Bank Group– supported reforms. Some of the important reforms supported by the Bank Group have yet to produce tangible results for the population. For example, the establishment of new high-level anticorruption institutions has not translated into higher rates of prosecution or improved public perceptions of the pervasiveness of corruption. Similarly, significant institutional and structural reforms in the energy and social sectors (for example, health and pensions), while import- ant for resolving the fiscal crisis and reducing opportunities for corruption, did not result in more private sector investments in infrastructure or im- proved services. Lack of attention to important enabling areas with systemic impacts, such as justice sector and public administration reform, has also undermined the impact of progress in several areas. World Bank Group Contributions The World Bank Group in Ukraine, 2012–20  Overview Responding to the different political situation after 2014, the World Bank broadened its engagement on governance and helped the government establish legal and institutional foundations for improving transparency and fighting corruption. The World Bank also supported sector reform programs (health, energy, banking) to advance the anticorruption agenda, including through tariff and subsidy reforms in the gas sector, modernization of bank supervision, and a deregulation effort to reduce administrative barriers for small and medium enterprises. At the same time, the effectiveness and sustainability of World Bank–supported reforms on governance and anticorruption continue to be undermined by weaknesses in the overall quality of public administration and lack of progress on reforming the judiciary. The Bank Group was a key partner in helping Ukraine manage the severe economic crisis of 2014–15 caused by a triple shock from the disruptive x change in government, the conflict in Eastern Ukraine, and a weak external environment. The Bank Group joined an international coalition to assist Ukraine by supporting the policy reforms needed to stabilize and resume economic growth. Substantial International Bank for Reconstruction and Development lending (alongside considerably larger International Monetary Fund and European Union rescue packages) helped the authorities reduce sizable fiscal and balance of payments deficits. The Bank Group focused on expenditure rationalization, particularly in social protection and pensions, energy tariffs, and subsidy reform. The Bank Group’s continued engagement in the financial sector provided effective and timely support during the crisis and helped improve the stability of the banking system. Nevertheless, de- spite substantial strengthening of macroeconomic fundamentals, Ukraine’s economy is vulnerable to macroeconomic shocks, which are aggravated by the COVID-19 pandemic. The energy sector was a major contributor to the fiscal crisis through large subsidies for gas and the losses of the state-owned gas monopoly. The main obstacles to reform included market capture, weak sector governance, under- investment, and heavy dependence on gas transit. The Bank Group provided assistance to enable tariff and subsidy reform, unbundle the gas monopoly, strengthen institutional arrangements for promoting energy efficiency, and build regulatory capacity. However, improvements in the quality and reliabil- ity of services and the credibility of regulatory mechanisms and institutions still lag. Reforms were not sufficient to attract private investment for sector modernization and to enhance customer satisfaction and choice. Independent Evaluation Group World Bank Group    xi Lessons This CPE offers the following lessons: 1. Continuity of engagement during periods of weak demand for reform posi- tioned the Bank Group to respond quickly when a window of opportunity presented itself. When there was little appetite on the part of the govern- ment for significant policy reform (2012–13), the World Bank invested heavily in analysis and partnerships at the technical level of government. These efforts helped the World Bank respond rapidly after the change in political leadership. 2. Greater attention was needed in the justice sector given its importance to the efficacy of reforms across a range of other sectors. This lack of attention was particularly relevant to the effectiveness of anticorruption reforms and to reforms in the energy and banking sectors. Entrenched interests often used the justice system to neutralize the impact of reforms in other sectors, thereby undermining the credibility of the broader reform effort and commitment to change. 3. Effective communication by the World Bank through outreach and en- gagement with civil society organizations is important to help the public understand the reason for reforms and the costs of not reforming. Al- though Bank Group strategies envisaged broad engagement with civil society and the private sector, implementation was uneven across sectors, with communication on banking sector reform particularly lacking. 4. Institutional reforms that impose a burden on citizens need to compen- sate by making progress in service delivery. Despite many accomplish- ments, Ukrainians remain deeply skeptical about the overall progress and impact of reforms on their daily life. Institutional reforms that impose painful adjustments on the citizenry (such as tariff increases) need to be paired with improvements in service quality, including in infrastructure. The World Bank Group in Ukraine, 2012–20  Overview xii Огляд Цей переклад надано лише для зручності, англійська версія є офіційною версією звіту. Основні моменти У рамках цього оцінювання проводиться оцінка результатів співробітництва між Групою Світового банку та Україною у період з 2012 до 2020 року з акцентом на підтримку, що надавалася у сфері врядування та боротьби з корупцією; реагуванні на кризи та забезпеченні стійкості економіки; а також енергетичної безпеки та енергоефективності. Група Банку сприяла створенню провідних антикорупційних інституцій, просуваючи антикорупційні реформи в певних секторах (охорона здоров’я, енергетика, соціальний захист) і зміцнюючи управління державними фінансами. Водночас, відсутність безпосереднього залучення до реформування судової системи та державного управління зменшила вплив підтримки Групи Банку в рамках всього портфеля. Група Банку була частиною міжнародної коаліції, яка допомогла стабілізувати українську економіку після кризи 2014-15 років, зробивши значний внесок у відновлення життєздатності і стабільності банківської системи та зміцнення технічного та інституційного потенціалу Національного банку України та Фонду гарантування вкладів. Міжнародна фінансова корпорація зробила важливий внесок у розвиток корпоративного управління та управління ризиками у фінансовому секторі. Однак, незважаючи на істотне зміцнення макроекономічних основ, економіка України, як і раніше, стикається з цілою низкою ризиків, посилених пандемією коронавірусу (COVID-19). Група Банку допомогла поліпшити управління енергетичним сектором і створити інституційні механізми для підвищення енергоефективності, в тому числі зігравши важливу роль у xiii  поділі монополії на природний газ, відіграючи провідну роль у діалозі щодо реформування тарифів і субсидій і допомагаючи диверсифікувати джерела енергопостачання. У той же час реформи, підтримувані Світовим банком, не призвели до значних приватних інвестицій в модернізацію енергетичної інфраструктури або відчутного поліпшення обслуговування споживачів. Основні уроки, визначені за результатами цього оцінювання, включають (i) постійне залучення, навіть у часи слабкої зацікавленості у реформах, а також розуміння та проведення аналітики інвестиційної ситуації в країні дозволили Групі Банку швидко долучитися до надання підтримки, коли з’являлося політичне вікно можливостей; (ii) відсутність безпосереднього залучення до реформування судової системи послабило вплив реформ в інших сферах, включаючи фінансовий сектор і боротьбу з корупцією. В очікуванні значного залучення до діяльності в Україні, відразу після припинення військової агресії, Банку доцільно було б інвестувати більше коштів у поглиблення свого розуміння зв’язків між конкретними недоліками в системі правосуддя і здатністю України домагатися прогресу в досягненні конкретних цілей в області розвитку; (iii) інформування громадськості та залучення до роз’яснення причин необхідності реформ й ціни бездіяльності мають вирішальне значення для продовження реформ; і (iv) інституційні реформи, які передбачають болючі зміни для домашніх господарств, як-от підвищення тарифів, повинні супроводжуватися поліпшенням якості послуг, у тому числі інфраструктури. xiv   У цьому Оцінюванні програми для країни (CPE) розглядаються результати співробітництва між Групою Світового банку та Україною в період 2012 -2020 років. У ньому обговорюється (i) ступінь, до якого залучення Групи Банку відповідало основним проблемам розвитку країни; (ii) внесок підтримуваної Групою Банку програми в результати розвитку; і (iii) ефективність співпраці Групи Банку з партнерами з розвитку. В CPE також розглядаються уроки, отримані протягом минулого періоду, для покращення взаємодії у майбутньому. У цьому CPE особлива увага приділяється трьом основним проблемним питанням, з якими зіткнулася Україна протягом періоду за який проводилося оцінювання, і в яких Група Банку відіграла значну роль: врядування і боротьба з корупцією, реагування на кризу й забезпечення стійкості економіки, а також енергетична безпека і енергоефективність. Основні проблеми та результати Україна має значний економічний потенціал, але її економічний і соціальний розвиток за останні три десятиліття був повільним і вкрай нестійким через внутрішні чинники, зовнішні потрясіння, повільне впровадження ринкових принципів управління економікою і слабкі інститути. Багато з ключових проблем розвитку, як-от корупція, слабке врядування, відсутність енергетичної безпеки та неефективні державні Independent Evaluation Group World Bank Group    xv послуги, зберігаються, зважаючи на те, що реформи проводилися за принципом «стоп-вперед». Це оцінювання демонструє, що в період з 2012 по 2020 рік Група Банку допомогла Україні провести кілька важливих реформ і, в деяких сферах, закласти основи для забезпечення тривалого впливу таких реформ. До 2014 року, незважаючи на низький інтерес уряду до реформ, Група Банку інвестувала в розвиток знань про країну і місцевих партнерств, і була належним чином підготовлена до активного залучення, коли з’явилася політична можливість для реформ. З 2014 року, після зміни уряду і подальших економічних і політичних криз, Група Банку використала можливості для значного розширення своєї діяльності і впливу на траєкторію реформ в Україні. Група Банку змогла надати ефективну підтримку уряду в стабілізації економіки і фінансового сектора, почати боротьбу з повсюдною корупцією, реформування системи охорони здоров›я та пенсійного забезпечення, а також підвищити енергетичну безпеку. У той же час, багато проблем залишаються невирішеними. У вкрай нестабільній політичній обстановці, такій як в Україні, важко оцінити, наскільки добре політика і структурні зміни, що проводяться за сприяння Групи Банку, допоможуть у протистоянні геополітичним ризикам, тиску з боку впливових кіл і змінам у виборі українського народу і його лідерів. Економіка України залишається вразливою до макроекономічних і політичних ризиків, що посилюються невизначеністю, пов›язаною з пандемією COVID-19. Інтереси впливових кіл, як і раніше, доволі потужні, а популістський тиск з метою зміни політики посилився, що уповільнює прогрес і ставить під сумнів довіру до прихильності уряду боротьбі з корупцією і сталість багатьох реформ, підтримуваних Групою Банку. Деякі з важливих реформ, підтримуваних Групою Банку, ще не принесли відчутних результатів для населення. Наприклад, створення нових антикорупційних установ на високому рівні не призвело до збільшення числа судових переслідувань або зміни ставлення суспільства до повсюдної корупції. Аналогічним чином, значні The World Bank Group in Ukraine, 2012–20  Overview інституційні та структурні реформи в енергетичному та соціальному секторах (наприклад, охорона здоров›я та пенсійне забезпечення), хоча і були важливі для врегулювання податково-бюджетної кризи та скорочення можливостей для корупції, не призвели до збільшення приватних інвестицій в інфраструктуру або поліпшення послуг. Недостатня увага до важливих стимулюючих напрямків, які матимуть системний вплив, як-от судова реформа та реформа державного управління, також підірвала вплив прогресу в декількох сферах. Внесок Групи Світового банку Реагуючи на політичну ситуацію, що змінилася після 2014 року, Світовий банк розширив своє залучення до реформи врядування і допоміг уряду створити правові та інституційні основи для xvi підвищення прозорості та боротьби з корупцією. Світовий банк також підтримав секторальні програми реформ (охорона здоров›я, енергетика, банківська діяльність) для просування антикорупційного порядку денного, в тому числі за допомогою реформи тарифів і субсидій в газовому секторі, модернізації банківського нагляду і зусиль з дерегулювання для зниження адміністративних бар›єрів для малих і середніх підприємств. У той же час, ефективність і сталість підтримуваних Світовим банком реформ у сфері врядування й боротьби з корупцією, як і раніше, послаблюються недоліками загальної якості державного управління і відсутністю прогресу в реформуванні судової системи. Група Банку була ключовим партнером у наданні допомоги Україні в подоланні важкої економічної кризи 2014-15 років, спричиненої потрійним шоком від революційної зміни уряду, конфлікту на сході України і слабкого зовнішнього середовища. Група Банку приєдналася до міжнародної коаліції з надання допомоги Україні шляхом підтримки політичних реформ, необхідних для стабілізації та відновлення економічного зростання. Значні обсяги кредитування Міжнародного банку з реконструкції та розвитку (поряд зі значно більшими пакетами допомоги Міжнародного валютного фонду та Європейського Союзу) допомогли владі скоротити значний дефіцит бюджету та платіжного балансу. Група Банку зосередилася на раціоналізації витрат, особливо Independent Evaluation Group World Bank Group    xvii в галузі соціального захисту та пенсійного забезпечення, тарифів на енергоносії та реформуванні субсидій. Постійне залучення Групи Банку до реформування фінансового сектору забезпечило ефективну і своєчасну підтримку під час кризи і допомогло покращити стабільність банківської системи. Проте, незважаючи на істотне зміцнення макроекономічних основ, економіка України вразлива до макроекономічних потрясінь, які посилюються пандемією коронавірусу (COVID-19). Енергетичний сектор став основним чинником фінансової кризи через великий розмір субсидій на газ і збитки державної газової монополії. Основними перешкодами на шляху реформ були захоплення ринку, слабке управління сектором, недостатні інвестиції і значна залежність від транзиту газу. Група Банку надала допомогу в проведенні реформи системи тарифів і субсидій, поділі газової монополії, зміцненні інституційних механізмів сприяння енергоефективності та нарощуванні потенціалу регулювання. Однак, підвищення якості і надійності послуг, а також збільшення довіри до механізмів регулювання й інституцій, як і раніше, є недостатніми. Реформ було недостатньо для залучення приватних інвестицій на модернізацію сектора і підвищення задоволеності споживачів і розширення їхнього вибору. Отримані уроки За результатами CPE визначені наступні уроки: 1. Безперервність взаємодії в періоди низького рівня зацікавленості в реформах дозволила Групі Банку швидко залучитися до діяльності, коли з›явилася можливість. Коли в уряду не було особливого бажання проводити значні політичні реформи (2012-13 роки), Світовий банк вклав значні кошти в аналіз і побудову партнерських відносин на технічному рівні уряду. Ці зусилля допомогли Світовому банку швидко відреагувати на зміну політичного керівництва. 2. Необхідно приділяти більше уваги судовій владі, враховуючи її важливість для ефективності реформ у цілому ряді інших секторів. Цей брак уваги був особливо актуальним для ефективності The World Bank Group in Ukraine, 2012–20  Overview антикорупційних реформ та реформ в енергетичному та банківському секторах. Вкорінені інтереси часто використовували систему правосуддя для нейтралізації впливу реформ в інших секторах, тим самим підриваючи довіру до більш широких зусиль з реформування і підтримки змін. 3. Ефективна комунікація Світового банку за допомогою інформаційно- роз›яснювальної роботи та взаємодії з організаціями громадянського суспільства важлива для того, щоб допомогти громадськості зрозуміти необхідність реформ та ціну відмови від реформ. Хоча стратегії Групи банку передбачали широку взаємодію з громадянським суспільством і приватним сектором, їх реалізація була нерівномірною по секторах, при цьому особливо бракувало xviii інформації про реформу банківського сектора. 4. Інституційні реформи, які створюють додатковий тягар для громадян, повинні компенсуватися прогресом у наданні послуг. Незважаючи численні досягнення, українці, як і раніше, дуже скептично ставляться до загального прогресу і впливу реформ на їхнє повсякденне життя. Інституційні реформи, які спричиняють болючі зміни для громадян (як-от підвищення тарифів), повинні супроводжуватися поліпшенням якості послуг, у тому числі інфраструктури. Independent Evaluation Group World Bank Group    xix 1 | Background This Country Program Evaluation (CPE) reviews the relevance and ef- fectiveness of the World Bank Group partnership with Ukraine between 2012 and 2020. It responds to the following evaluation questions: (i) To what extent were the Bank Group’s strategic positioning and engagement relevant to and aligned with the country’s main development challenges and evolving political economy? (ii) To what extent did Bank Group engagement contribute to development results in Ukraine, including helping to deal with crises and their aftermath? (iii) How effectively did the Bank Group leverage internal synergies and collaborate with major development partners? The CPE also draws lessons for future engagement in Ukraine. Country Context Ukraine has significant economic potential, but economic and social de- velopments have been slow and highly volatile over the past decade. A lower-middle-income country with a population of 43 million and a gross national income per capita of $3,540 in 2020 (Atlas method), Ukraine is endowed with a well-educated and entrepreneurial population, vast areas of fertile land, sizable energy and other natural resources, and a geographic location at the crossroads of Europe and Asia. However, Ukraine’s gross na- tional income per capita remains far below that of its neighbors and com- parator countries.1 Although some indicators of human development have returned to pretransition levels, life expectancy at birth (72 years in 2019) has changed little over the past 20 years and lags the European Union (EU) and Organisation for Economic Co-operation and Development averages by more than 10 years. The poverty rate (at the international poverty rate of $5.50 a day, 2011 purchasing power parity) increased from 3.4 percent in 2012 to a peak of 6.3 percent in 2015 and declined to 3.4 percent in 2018. During the same period, inequality increased slightly.2 Ukraine’s uneven economic performance was a result of both internal factors and exogenous shocks. The country was hit hard by the 2008–09 global eco- 1 nomic and financial crisis, with gross domestic product (GDP) shrinking by almost 15 percent in 2009. After a period of recovery, the country faced two shocks in 2014 and 2015: an armed conflict in Eastern Ukraine and a drop in global commodity prices (particularly for metals and agricultural goods). As a result, real GDP contracted by 6.6 percent in 2014 and 9.8 percent in 2015 (table 1.1). The national currency (hryvnia) depreciated 47 percent in 2014 and a further 33 percent in 2015, while the consolidated fiscal deficit reached 10.1 percent of GDP in 2014 and the public debt burden more than doubled to 70 percent of GDP in 2015 (World Bank 2017f).3 Many of the country’s de- velopment challenges have persisted over the past three decades in the face of stop-and-go reform efforts. These challenges include corruption and weak governance; energy inefficiency, affordability, and supply insecurity; ineffec- tive public services and poorly targeted social assistance; and conflicts and shocks (World Bank 2017a, 2017c, and 2018d). Table 1.1. Key Economic and Social Indicators Ukraine, by Year Average during 2012–19 Indicators  2012 2015 2019 Ukraine ECA World GDP growth (annual %) 0.2 –9.8 3.2 –0.6 1.7 2.8 GNI per capita (current dollars) 3,500 2,700 3,370 3,060 24,714 10,834 Inflation, consumer prices (annual %) 0.6 48.7 7.9 13.5 1.4 2.3 Life expectancy at birth, total (years) 70.9 71.2 71.8 71.3 77.3 71.9 The World Bank Group in Ukraine, 2012–20  Chapter 1 Infant mortality (per 1,000 live births) 9.2 8.1 7.2 8.1 8.2 31.2 Sources: International Monetary Fund World Economic Outlook database; World Bank World Develop- ment Indicators database. Note: ECA = Europe and Central Asia; GDP = gross domestic product; GNI = gross national income. The period covered by this CPE can be divided into three distinct subperi- ods, defined by political developments and corresponding adjustments in Bank Group engagement: (i) 2012–13 was a period of stagnant engagement with the government of President Yanukovych amid low demand for reform; (ii) 2014–19, after the so-called Euromaidan revolution (also known as the “Revolution of Dignity”), included a major economic and financial crisis, during which the Bank Group focused on stabilizing the economy and sup- porting institutional reforms; and (iii) the election of President Zelensky in April 2019 (with an unprecedented majority)4 led to closer dialogue in areas 2 previously considered too politically sensitive (such as land reform), amid continued opposition from vested interests (figure 1.1). This CPE also covers the beginning of the negative impact of the COVID-19 pandemic. Figure 1.1. Major Political Milestones Beginning of public protests, Maidan Election of President revolution (“Revolution of President Yanukovich Yanukovich Dignity”) flees the country February 2010 November 2013 February 2014 Period 1: Before and during the Maidan Annexation of Crimea by the Russian Federation; Ukraine signs the beginning of phase Association of active war in Minsk II ceasefire Agreement with eastern Ukraine Election of accord; end of the EU, including (Donetsk and President large-scale the DCFA Luhansk oblasts) Poroshenko hostilities March 2014 March–April 2014 June 2014 February 2015 Period 2: After the Maidan Independent Evaluation Group World Bank Group    3 President Zelensky’s party (“Servant of the People”) wins majority in parliamentary Election of President Zelensky with 73% of (National Rada) elections, with 254 out of the vote 424 seats April 2019 July 2019 Period 3: After the election of President Zelensky Source: Independent Evaluation Group. Note: DCFTA = Deep and Comprehensive Free Trade Agreement; EU = European Union. World Bank Group–Supported Program During the period under review, the Bank Group engagement with Ukraine was guided by two strategies, with the latter informed by the 2017 Sys- tematic Country Diagnostic (figure 1.5). The fiscal year (FY)12–16 Country Partnership Strategy (CPS; World Bank 2012b) covered the period before and after the 2014 events and was based on two pillars: (i) state capacity for service delivery and government accountability, and (ii) growth and compet- itiveness through a better regulatory and investment climate and improved business infrastructure. The FY17–21 Country Partnership Framework (CPF; World Bank 2017c) placed greater emphasis on governance-related issues and citizen engagement and focused on four areas: (i) better governance, anticorruption, and citizen engagement; (ii) making markets work; (iii) fiscal and financial sustainability; and (iv) efficient, effective, and inclusive ser- vice delivery. Because of the COVID-19 pandemic, the CPF was extended by one year until June 30, 2022.5 The new CPF for FY23–27 has been postponed indefinitely given the ongoing armed conflict (since February 2022).6 Bank Group support evolved in response to crises and political develop- ments. Before 2014, engagement was characterized by slow-moving in- vestment projects (mainly in energy and infrastructure), no budget support operations, and limited advisory services and analytics (ASA). Beginning in 2014, and in response to crises and political changes, the Bank Group dras- The World Bank Group in Ukraine, 2012–20  Chapter 1 tically ramped up financial support to the government (figure 1.2), with the majority of financing in the form of development policy loans (DPLs) linked to policy reforms related to crisis response and better governance (fig- ure 1.3). In addition, the Bank Group continued to support the infrastructure sector through new projects in energy (2016 Power Transmission Project, $330 million, and 2021 Power System Resilience project, $177 million) and transport (2016 Road Sector Development Project, $560 million). Analytical work continued to focus on institution building, including support for anti- corruption reforms (figure 1.4). In response to the COVID-19 pandemic and conflict in Eastern Ukraine, the Bank Group approved over $1.1 billion of support in 2020–21. This support included additional financing for a health project ($135 million), two additional financings for the Social Safety Nets Modernization Project 4 ($150 million and $300 million), a public health and vaccines project ($90 million), and a budget support operation to assist with economic recovery ($350 million).7 In November 2020, the World Bank approved a $100 million loan to support conflict-affected areas in the east (“Eastern Ukraine: Reconnect, Recover, Revitalize”) and support government efforts to promote the recovery, reintegration, and inclusion of the conflict-affected population.  orld Bank Commitments to Ukraine by Financing Figure 1.2. W Instrument, Fiscal Years 2012–20 1800 1600 1400 World Bank commitment 1200 (US$, millions) 1000 800 600 400 200 0 2012 2013 2014 2015 2016 2017 2019 2020 Fiscal Year Guarantee IPF DPL PforR Independent Evaluation Group World Bank Group    5 Source: World Bank Business Intelligence (January 29, 2021). Note: DPL = development policy loan; IPF = investment project financing; PforR = Program-for-Results. The Bank Group supported private sector development (most notably agri- business) mainly through International Finance Corporation (IFC) invest- ments and Bank lending operations. IFC committed $751 million through 38 investments, the bulk of which (21 investments totaling $494 million) were in the agribusiness sector. A 2019 Program-for-Results operation (“Acceler- ating Private Investment in Agriculture”) aimed to eliminate constraints on private sector participation in the agriculture input and output markets. A programmatic development policy financing series included prior actions to strengthen the business regulatory framework and address distortions in the agricultural land market. The Multilateral Investment Guarantee Agency had six project exposures in Ukraine, amounting to $204 million. Table 1.2 pres- ents outcome and Bank performance ratings for projects in Ukraine that closed during the evaluation period, and appendix D provides additional details on the Bank Group portfolio. Building on the assessment of private sector devel- opment constraints contained in the 2017 Systematic Country Diagnostic, the Bank Group produced in 2020 a Country Private Sector Diagnostic (IFC 2020) that examined ways to unleash private sector potential. These included im- proving agricultural productivity, integration into manufacturing global value chains, and health care. The report identified persistent cross-cutting con- straints, such as weak competition landscape, limited access to finance, inade- quate infrastructure, and energy market distortions. Figure 1.3. Ukraine Commitments by Sector, Fiscal Years 2012–21 Public Admin Financial Sector Transportation Energy and Extractives Sector Social Protection The World Bank Group in Ukraine, 2012–20  Chapter 1 Health Water / Sanitation / Waste Education Agriculture 0 500 1,000 1,500 2,000 Ukraine commitment (US$, millions) IBRD commitment Source: World Bank Business Intelligence (December 14, 2021). Note: IBRD = International Bank for Reconstruction and Development. 6  kraine Advisory Services and Analytics Delivery by Sector, Figure 1.4. U Fiscal Years 2012–21 Public Admin Social Protection Energy and Extractives Financial Sector Transportation Sector Agriculture, Fishing, Forestry Health Industry, Trade and Services Education Water / Sanitation / Waste 0 5 10 15 20 25 30 Share of ASA delivery (%) Source: World Bank Business Intelligence (December 14, 2021). Note: ASA = advisory services and analytics. ndependent Evaluation Group Performance Ratings for Table 1.2. I Ukraine, Fiscal Years 2012–19 Independent Evaluation Group World Bank Group    7 Amount Bank (US$,  IEG Outcome Performance Exit FY Project Name millions) Rating Rating 2012 State Tax Service Mod- 40 Moderately Moderately ernization Project satisfactory satisfactory 2013 Rural Land Titling and 82 Moderately Moderately Cadastre Development satisfactory satisfactory Project 2013 Social Assistance 93 Unsatisfactory Moderately System Modernization unsatisfactory Project 2014 Development of State 41 Moderately Moderately Statistics System satisfactory satisfactory (continued) Amount Bank (US$,  IEG Outcome Performance Exit FY Project Name millions) Rating Rating 2015 Public Finance Modern- 4 Unsatisfactory Unsatisfactory ization Project 2015 Second Export Devel- 305 Satisfactory Satisfactory opment Project 2015 Urban Infrastructure 137 Moderately Moderately satisfactory satisfactory 2015 Roads and Safety Im- 380 Moderately Moderately provement satisfactory satisfactory 2015 DPL 1 750 Satisfactory Satisfactory 2015 Programmatic Financial 500 Satisfactory Satisfactory Sector DPL 1 2016 Hydropower Rehabili- 138 Moderately Moderately tation satisfactory satisfactory 2016 Power Transmission 194 Moderately Moderately unsatisfactory satisfactory 2016 DPL 2 500 Satisfactory Satisfactory 2016 Programmatic Financial 500 Satisfactory Satisfactory Sector DPL 2 2017 Energy Efficiency 200 Satisfactory Satisfactory Source: World Bank Business Intelligence (January 31, 2021). Note: DPL = development policy loan; FY = fiscal year; IEG = Independent Evaluation Group. The World Bank Group in Ukraine, 2012–20  Chapter 1 8  volution of the World Bank Group Strategy, Fiscal Years 2012–21 Figure 1.5. E Country Partnership Strategy FY12–16 Systematic Country Diagnostic 2017 Country Partnership Framework FY17–21 Focus area 1: Improving public services and Pathway 1: Macroeconomic stability Focus area 1: Making markets work public finances: Support to building relations with citizens Addressing the largest sources of fiscal and 1.1. Improving infrastructure services, particularly financial sector vulnerability in energy and transport 1.1. Improved governance of public finances Pathway 2: Private sector productivity 1.2. Creating a level playing field in the private 1.2. Improved efficiency of social expenditures sector (health, education, and social safety net) Strengthening infrastructure investment 1.3. Land reform 1.3. Improved efficiency, quality, and governance Creating a level playing field in the private of municipal infrastructure services sector Focus area 2: Fiscal and financial stability Focus area 2: Improving policy effectiveness Reforming land markets 2.1. Mitigating the largest medium-term fiscal and economic competitiveness: Support to risks building relations with businesses Tapping trade opportunities 2.2. Strengthening financial sector stability 2.1. Improving business regulatory environment Pathway 3: Effective services and targeted for a more competitive and diversified economy assistance Focus area 3: Efficiency and inclusiveness of social services delivery 2.2. Improving infrastructure for business Optimizing the financing and improving the activities quality of health and education 3.1. Increasing the efficiency of health services 2.3. Improving productivity and competitiveness Further improving targeting of social assistance 3.2. Increasing the targeting of social assistance in agriculture Providing effective support to conflict-affected Cross-cutting focus area: Use of GAC filter to people Cross-cutting focus area: Better governance examine both GAC-related implementation (strengthened public financial management to risks and the potential to positively influence Cross-cutting pathway: Building institutions improve transparency), anticorruption, and country GAC outcomes of better GAC citizen engagement Source: Independent Evaluation Group. Note: FY = fiscal year; GAC = governance and anticorruption. Independent Evaluation Group World Bank Group    9 Country Program Evaluation Scope and Coverage The CPE includes three special themes that reflect important challenges Ukraine faced during the evaluation period. These themes were identified in discussions with the Bank Group Ukraine country team and the Ukrainian authorities during the early stages of this evaluation and through review of the country strategy and analytic documents. » Governance and anticorruption. Weak governance and high levels of cor- ruption have been perennial challenges for Ukraine, permeating most sectors of the economy. Anticorruption was an overarching theme in all Bank Group programs and government of Ukraine strategies. The CPE examines the rel- evance and efficacy of Bank Group support to achieving better transparency, accountability, and associated institutional reforms (chapter 2). » Crisis response and economic resilience. Ukraine went through several se- vere political and economic crises during the evaluation period, and the Bank Group was a key partner in supporting economic stabilization and in helping to address underlying fiscal and financial sector vulnerabilities. The CPE assesses how well the Bank Group contributed to stabilizing the economy and building the foundation for economic resilience (chapter 3). » Energy security and efficiency. A well-functioning energy sector, with a secure supply, is vital for Ukraine’s economic competitiveness and fiscal sus- The World Bank Group in Ukraine, 2012–20  Chapter 1 tainability and the well-being of its people. Although the country is endowed with sizable energy resources and has an advantageous geographic location, it is also one of the most energy-intensive economies in the world, with high energy consumption perpetuated by aging and inefficient infrastructure and high and poorly targeted subsidies. The development of Ukraine’s energy sector has been challenged by geopolitical contestation and market capture. These factors have made effective management of the sector extremely chal- lenging, contributing to frequent political and economic crises and the high incidence of corruption. The CPE takes an in-depth look at the effectiveness of Bank Group support in helping the country manage these complex chal- lenges (chapter 4). 10 1  $15,000 for Poland, $12,000 for Romania, and $9,000 for Turkey (World Bank Data, GNI [gross national income] per capita, Atlas method [current US$]; see https://data.worldbank. org/indicator/NY.GNP.PCAP.CD). 2  See https://data.worldbank.org. 3  A detailed discussion of the series of crises and efforts to address Ukraine’s fiscal and finan- cial sector vulnerabilities can be found in appendix D. 4  President Zelensky was elected in April 2019 with 73 percent of the vote. In July 2019, his party (“Servant of the People”) won 60 percent of seats in parliament (the Verkhovna Rada). 5  Some areas, such as education and subnational service delivery, were assigned a criticality rating of “medium” and downscaled. 6  At the time of the Country Program Evaluation submission (April 2022), the World Bank Group management was in close dialogue with the Ukrainian authorities to provide immedi- ate support and be ready to move toward reconstruction when conditions allowed. See also the Joint International Monetary Fund–World Bank Group Statement on the War in Ukraine (March 1, 2022): https://www.worldbank.org/en/news/statement/2022/03/01/joint-imf-wbg- statement-on-the-war-in-ukraine. 7  An additional financing for a health project ($135 million, approved in April 2020), two addi- tional financings for the Social Safety Nets Modernization Project ($150 million, approved in April 2020, and $300 million, approved in November 2020), and a public health and vaccines project ($90 million, approved in May 2021). Independent Evaluation Group World Bank Group    11 2 | Governance and Anticorruption Highlights Governance and anticorruption gained importance in World Bank support to the government’s reform agenda after 2014 and re- ceived another boost after the change in administration in 2019. The World Bank made important contributions to advance gov- ernance and anticorruption reforms, including through help in establishing new apex anticorruption institutions. It effectively used opportunities to advance the governance and anticorruption agenda within sectoral reform programs, including public finan- cial management, energy, health, and social protection. The World Bank also supported a successful communications campaign to build public support for several aspects of the reform agenda. A lack of attention to the justice sector and minimal progress on public administration reform continue to undermine the impact of broader reforms. Anticorruption institutions have obtained few convictions for corruption-related offences, changes to bankrupt- cy regimes have not made a meaningful dent in the high levels of nonperforming loans, and public perception of the level of corrup- tion has not improved significantly. 12   Ukraine’s governance shortcomings have been well-documented (table 2.1). The country has been continually ranked well below its Eastern European neighbors on most international corruption indexes (figure 2.1), reflecting the fact that corruption and state capture have been deeply entrenched. Lack of political commitment to reform and opposition from vested interests delayed policy reforms until 2014, when anticorruption gained importance in the government’s policy agenda in response to pressure from civil society and international partners. Reform momentum received a boost after the change in administration in 2019. Table 2.1. Ukraine: Select Governance Indicators, 2010–19 Indicators  2010 2014 2017 2018 2019 2020 Corruption Perceptions Index — 26 30 32 30 33 score (TI)a Open Budget Index score (IBP)b 62 46 54 — 63 — Control of corruption (WGI) c –1.03 –0.99 –0.78 –0.87 –0.71 — Government effectiveness (WGI) –0.78 –0.41 –0.46 –0.42 –0.30 — Regulatory quality (WGI) –0.52 –0.63 –0.32 –0.22 –0.26 — Rule of law (WGI) –0.81 –0.79 –0.71 –0.72 –0.70 — Voice and accountability (WGI) –0.08 –0.14 0.01 –0.01 0.06 — Political stability and absence of 0.01 –2.02 –1.87 –1.83 –1.52 — violence (WGI) Sources: International Budget Partnership; Transparency International; World Bank Worldwide Gover- Independent Evaluation Group World Bank Group    13 nance Indicators database. Note: Corruption Perceptions Index scores are not comparable between 2010 and 2012–20; data for IBP are available for 2010, 2015, 2017, and 2019. IBP = International Budget Partnership; TI = Transparency International; WGI = Worldwide Governance Indicators; — = not available. a. Corruption Perceptions Index scores relate to the degree to which corruption is perceived to exist among public officials and politicians by businesspeople and country analysts. Scores range between 0 (highly corrupt) and 100 (highly clean). See www.transparency.org. b. Open Budget Index scores are a measure of budget transparency. The index uses individual indicators that assess whether the central country government makes key budget documents available to the public in a timely manner and whether the data contained in these documents are comprehensive and useful. Score ranges between 0 (highly nontransparent) and 100 (highly transparent). See the International Budget Partnership Open Budget Survey rankings, https:/ / internationalbudget.org/open-budget-survey/rankings. c. The WGI relate to the strength of governance performance along six dimensions. Scores range from –2.5 (weak performance) to 2.5 (strong performance). The WGI are a research data set summarizing the views of the quality of governance provided by a large number of enterprise, citizen, and expert survey respondents in industrial and developing countries. These data are gathered from a number of survey institutes, think tanks, nongovernmental organizations, international organizations, and private sector firms. The WGI do not reflect the official views of the World Bank, its Executive Directors, or the countries they represent. The WGI are not used by the Bank Group to allocate resources. See http:/ /info. worldbank.org/governance/wgi/index.aspx#reports. This chapter assesses the Bank Group’s contributions to the governance and anticorruption (GAC) agenda in Ukraine through its support for the establishment and strengthening of anticorruption institutions and public financial management (PFM) reform. It also summarizes the Bank Group’s contributions to anticorruption in individual sectors (energy, health, social protection, and so on), with more details in chapters 3 and 4 and appendix A. Anticorruption Institutions Anticorruption was a key theme in both Bank Group–supported strategies (FY12–16 CPS and FY17–21 CPF), but concrete actions were limited before 2014. The 2012 CPS contained a frank assessment of the situation in the country and identified corruption and state capture as dominant imped- iments to sustained economic growth and shared prosperity. At the same time, the CPS did not prioritize core governance activities given the lack of government commitment. Instead, the strategy provided selective interven- tions on PFM (as discussed later in this chapter), deregulation, and strength- ening the governance of municipal infrastructure services. The CPS also signaled a pause in budget support operations until there was, among other things, consistent progress on governance. Bank Group engagement on anticorruption received a boost after 2014, when a comprehensive package of anticorruption laws established a set The World Bank Group in Ukraine, 2012–20  Chapter 2 of new specialized apex anticorruption institutions.1 Responding to the new political situation, and capitalizing on broad international support for reform efforts in Ukraine, the World Bank strengthened its engagement on GAC and used a programmatic series of DPLs to advance several core policy reforms. Prior actions included measures to advance the adoption of the new asset-declaration system, strengthen verification arrangements for asset declarations, and improve external budget audits. The FY17–21 CPF was more ambitious in its GAC agenda. It advocated a two- pronged strategy to address Ukraine’s GAC challenges. The first prong sought to build core governance institutions to systematically enhance the effec- tiveness of public sector operations. The second prong included support for sector-level reforms. The World Bank produced several influential ASA prod- 14 ucts, such as the 2018 state capture study (Balabushko et al. 2018), which introduced a new methodology for identifying captured sectors of Ukraine’s economy and measuring the economy-wide impact of capture. According to Independent Evaluation Group (IEG) interviews, the study generated broad public interest and was widely used by development partners and civil soci- ety organizations (CSOs).2 Figure 2.1. Control of Corruption Lithuania Georgia Poland Slovak Republic Country Croatia Romania Bulgaria Kazakhstan Ukraine 0 20 40 60 80 Independent Evaluation Group World Bank Group    15 Percentile rank 2012 2019 Source: Worldwide Governance Indicators database. Note: Percentile rank indicates a country’s rank among all countries covered by the Worldwide Gover- nance Indicators (WGI): 0 corresponds to the lowest rank, and 100 corresponds to the highest rank. WGI summarize views on the quality of governance provided by many enterprise, citizen, and expert survey respondents in industrial and developing countries. These data are gathered from several survey insti- tutes, think tanks, nongovernmental organizations, international organizations, and private sector firms. The WGI do not reflect the official views of the World Bank, its Executive Directors, or the countries they represent. The WGI are not used by the Bank Group to allocate resources. The black bars indicate lower and upper bounds of 90 percent confidence interval. The World Bank relied significantly on development partners (especially the EU and International Monetary Fund [IMF]) to provide capacity-building assistance to implement DPL-supported governance reforms. Between 2008 and 2020, there were no new World Bank project loans to facilitate the modernization of government systems and processes (for example, in PFM, public administration, e-governance, tax administration, decentralization, or the justice sector). IEG interviews with development partners indicated that additional project support from the World Bank would have been welcomed and could have helped close implementation gaps. The establishment of legal and institutional foundations for fighting corruption was a central aspect of Bank Group support for GAC in Ukraine. Public sector transparency was enhanced by expanding access to budget and procurement information, introducing the electronic asset disclosure system, implementing e-procurement, opening various public registries, and making several data sets (including court decisions) publicly available. The National Agency on Corruption Prevention (NACP), created with technical support from the World Bank, operates a comprehensive and publicly accessible e-declaration system for the personal assets of all high- and mid- level officials. Since 2018, the NACP has successfully collected, processed, and made public about 800,000 e-declarations per year. This information has been used by other anticorruption institutions (such as the National Anti-Corruption Bureau of Ukraine) and by the civil society. The World Bank, jointly with other donors and Ukrainian CSOs, supported NACP staff training and advocacy to assist with full implementation of the system. IEG interviews with local stakeholders, including anticorruption advocates and development partners, indicated that asset disclosure has influenced the behavior of specific groups of officials (such as judges). Other Bank Group The World Bank Group in Ukraine, 2012–20  Chapter 2 contributions included support for anticorruption enforcement institutions through the Stolen Asset Recovery Initiative and advisory assistance to local CSOs such as the Anti-Corruption Action Center. However, the impact of anticorruption legislation is hampered by a lack of reform of the judiciary (both courts and the prosecutor’s office). Although the legal framework for the judiciary was revised in 2015 through a package of constitutional amendments and new laws, implementation has been slow and partial, deepening public skepticism over the government’s commitment to reform. Judicial reforms have been mostly cosmetic, having been successfully neutralized by entrenched interests (Dubrovskiy et al. 2020). Enforcement of anticorruption legislation, particularly when high-level officials are involved, remains weak, fueling public cynicism and undermining the effectiveness of a 16 range of anticorruption laws and initiatives. The courts have regularly blocked the investigation and sentencing of corrupt officials (Center for Insights in Survey Research 2021). As of July 2021, National Anti-Corruption Bureau of Ukraine has investigated 879 cases, made 584 formal accusations, and brought 325 cases to courts, but has secured only 56 convictions.3 The launch of operations of the High Anti-Corruption Court provides some reason for optimism. The World Bank, together with international and local partners, was instrumental in protecting the High Anti-Corruption Court law against attempts to water down its core clauses (such as the independence of the Court and selection of the judges), including by temporarily withholding financial support. In 2019, its first year of operation, the High Anti-Corruption Court accelerated the resolution of corruption-related cases (Savin, Mykolaychuk, and Center for Combating Corruption 2020). If sustained, this work would strengthen incentives within the law-enforcement community and civil society to investigate and publicize evidence of corruption. Bank Group engagement in justice sector reform throughout the review period was limited to modest diagnostic work. The 2017 Systematic Country Diagnostic emphasized the importance of justice sector reform and noted that “building better anticorruption, justice, and public administration institutions [is] critically important for Ukraine and would have far-reaching ramifications for progress along each of the other development pathways” and “justice reform is an important component of strengthening governance in Ukraine and facilitating its aspirations of joining the [EU]” (World Bank Independent Evaluation Group World Bank Group    17 2017h, 44 and 48). Although organizations such as the National Anti- Corruption Bureau of Ukraine, for example, can expose corruption cases, only a reformed judiciary can reduce systemic corruption (Oxenstierna and Hedenskog 2017). At the same time, the 2017 CPF stated explicitly that in justice sector reform (as well as in civil service reform and decentralization), the Bank Group would rely on other development partners (the EU and bilateral agencies) that were better placed to provide support, according to the CPF (World Bank 2017c). However, IEG interviews for this CPE revealed little evidence of effective coordination and complementarity with development partners in this area. For example, it was unclear whether the aspects of justice reform needed to make Bank Group–supported reforms effective (for example, for nonperforming loan [NPL] resolution in the banking sector) were to be prioritized (or even covered) by other partners. Where the World Bank was active on the GAC agenda, there was coordination with donor activities and the World Bank was a source of relevant analysis. Donor coordination on the introduction of the new asset-declaration system at the NACP was particularly successful—the system was funded by the United Nations Development Programme, the World Bank provided technical assistance, the IMF supplied policy leverage, the EU and the United Kingdom’s Department for International Development provided support for system development.4 The World Bank mobilized additional grant funding to augment its ASA for governance and PFM reforms. The Department for International Development’s Good Governance Trust Fund allowed for the expansion of World Bank support in PFM and anticorruption reform. The World Bank also leveraged EU, German Agency for International Cooperation, and United States Agency for International Development resources to carry out Public Expenditure and Financial Accountability (PEFA) assessments and disseminate their findings. The Bank Group drew on the knowledge and experience of Ukrainian think tanks and CSOs in the design and implementation of its strategy. Histori- cally, CSOs have played an important role in promoting the GAC agenda in Ukraine, and their successful cooperation with international donors was rec- ognized as an effective “sandwich model” for advancing reforms.5 IEG iden- tified a few instances of cooperation between the World Bank and Ukrainian CSOs on ASA and of integration of local knowledge providers into moni- The World Bank Group in Ukraine, 2012–20  Chapter 2 toring of reform implementation: the state capture study (Balabushko et al. 2018) with the Institute for Economic Research and Policy Consulting, the Extractive Industries Transparency Initiative with DiXi Group, and anticor- ruption advocacy and monitoring with Anti-Corruption Action Center. Overall, Bank Group support to advance GAC reforms contributed to sev- eral positive outcomes. The Bank Group played an important role in the broad anticorruption coalition of development partners, civil society, and reform-oriented segments of government; was a key participant in a joint communications campaign (with other donors) to accelerate the adoption of key laws and regulations; and provided critical technical assistance to build capacity in new apex anticorruption institutions. The World Bank used sectoral reform programs (for example, in energy, health, and banking) to 18 advance the GAC agenda. Table 2.2 summarizes anticorruption progress at the sector level with the help of the World Bank. More details on specific sector engagement are given in chapters 3 and 4 and appendix A. However, the concrete impact of Bank Group–supported GAC reforms in Ukraine has been modest. The establishment of many high-level anticor- ruption institutions was a step in the right direction and held the promise to create “islands of integrity” in the public sector. However, these institutions have yet to make a systemic impact. The effectiveness of Bank Group–sup- ported reforms on GAC is undermined by weaknesses in the overall quality of public administration and by a lack of progress on reforming the judiciary. The World Bank’s decision not to engage actively in the justice sector under- estimated the severity of the development constraints imposed by a dys- functional court system, including in addressing specific sector constraints such as NPL and debt resolution, tax collection, and state-owned enterprise reform. Political opposition to anticorruption policies remains strong, which is illustrated by the fact that the core anticorruption legislation has faced multiple challenges in courts. Table 2.2. Summary of Anticorruption Reforms in Sector Programs World Bank Group Sector Main Achievements Contributions Unfinished Agenda Banking Reform of the legal The World Bank Group Governance arrange- and institutional frame- provided support through ments for state- works in the sector DPLs, ASA, and technical owned commercial Independent Evaluation Group World Bank Group    19 reduced corruption assistance to develop banks remain weak. risks in supervision and adopt the legal and and opportunities for institutional frameworks money laundering and improve institutional and ensured greater capacity of the National transparency in banks’ Bank of Ukraine and the ownership. Deposit Guarantee Fund. (continued) World Bank Group Sector Main Achievements Contributions Unfinished Agenda Gas Reforming governance Policy and institutional re- Regulation of supply and structures in the sector forms through DPLs (prior regional gas distrib- distribution led to greater transpar- actions promoting institu- utors needs to be ency (for example, in tional and tariff reforms, strengthened. the allocation of new including restructuring gas licenses) and re- of key sector entities) ductions in corruption provided advisory and opportunities and illicit technical support for un- practices. bundling the state-owned gas monopoly (Naftogaz), improving capacity of the regulator, and reforming the tariff and subsidy system. Other Implementation of The Bank Group provided Weaknesses in reg- energy the EITI and improved technical assistance for ulation of electricity and utilities payment and contract EITI implementation and and coal markets disclosure led to in- financed development (which affects creased transparency and implementation of transparency of in the sector, includ- better payment and con- pricing); considerable ing in the operations tract disclosure systems. variation in transpar- of several municipal ency across munic- utilities (district heating, ipal utilities; need to water), reducing op- impose and enforce portunities for corrupt stricter across-the- practices. board performance standards by the regulator. The World Bank Group in Ukraine, 2012–20  Chapter 2 Health Pharmaceutical The Bank Group support- Extending new procurement reform, ed the pharmaceutical provider payment including creation of procurement reform and rules beyond primary a national purchasing the establishment of the care to hospitals; agency, the NHSU, NHSU through a health solidifying the posi- and new provider pay- sector investment project, tion of transparent ment rules replaced “Serving People, Improv- contracting and drug a system conducive ing Health,” and accom- procurement mech- to corruption through panying ASA (for example, anisms so that they opaque procedures social accountability are not bypassed by for the allocation of tools for CSOs monitoring COVID-19 emergen- budget funding. municipal services and cy arrangements. procurement) in Ukraine, providing technical assis- tance and financing. (continued) 20 World Bank Group Sector Main Achievements Contributions Unfinished Agenda Social Reform measures Bank Group projects and Sustaining momen- protection (introduction of means ASA supported capac- tum for improved testing, monetization ity-building measures targeting of benefits of some benefits, within the administration and expanding the and enhanced cen- and ongoing advisory and guaranteed mini- tral control over local technical assistance. (For mum income. governments’ compli- example, Bank Group ance with the estab- contributions included lished rules of benefit analytical, modeling, administration) led to and simulation work on reduced opportunities parametric changes to the for corruption in the household utility subsidy.) social benefit adminis- tration. Source: Independent Evaluation Group. Note: ASA = advisory services and analytics; CSO = civil society organization; DPL = development policy loan; EITI = Extractive Industries Transparency Initiative; NHSU = National Health Service of Ukraine. Public Financial Management The World Bank was an effective contributor to strengthening PFM and in- creasing the transparency of public finance in Ukraine, especially since 2015, within a broad coalition of international partners. The World Bank played a key advisory role and used DPLs to advance several key pieces of legislation, Independent Evaluation Group World Bank Group    21 including amendments to the budget code and a new law on the Accounting Chamber in 2015, which laid the foundation for the Chamber to strengthen its independence and expand its mandate. Although the World Bank’s role in the delivery of day-to-day technical assistance was relatively modest, the Bank Group was an influential adviser to the Ministry of Finance and made a substantial contribution to the design of the government’s PFM reform strategy for 2017–20. PEFA assessments in 2012 and 2019 provided a comprehensive picture of the progress made by Ukraine in PFM (World Bank 2012c, 2019c).6 Ukraine demonstrated good progress over the evaluation period in transparency and accountability arrangements within the PFM system, such as public access to information, transparency in public procurement, predictability of transfers to local budgets, and the quality of internal controls and external audits. Overall, 21 of the 31 PEFA indicators improved. Importantly, the score for internal controls on nonsalary expenditures increased from B to B+. Although Ukraine lags most of its neighbors on many governance indicators, its Open Budget Index score (a consolidated measure of budget transparency) increased from 54 (out of 100) in 2012 to 63 in 2019 and is either better than or similar to the scores of many EU member countries in Eastern Europe.7 PEFA 2019 (World Bank 2019c) laid the basis for the current government’s 2021–24 PFM reform strategy. Progress in PFM contributed to a rationalization of public spending in pensions, social protection benefits, and energy subsidies. The World Bank supported the government in strengthening key building blocks of the PFM system, including the automation of budget planning and the establishment of the integrated Human Resources (and Payroll) Management Information System through the Strengthening Public Resource Management Project (FY17), funded by an EU grant. However, a more comprehensive rationaliza- tion of public spending has been hampered by slow implementation of the Human Resources (and Payroll) Management Information System. Major steps were taken to make Ukraine’s public procurement system more efficient and transparent and less susceptible to corruption. This was achieved through the implementation of the e-procurement system (ProZor- The World Bank Group in Ukraine, 2012–20  Chapter 2 ro),8 enabled by the new public procurement law. This law, updated with tech- nical assistance from the World Bank and adopted in December 2015, aligns national procurement regulations with EU directives and regulates the appli- cation of e-procurement. The general public now has easy access to informa- tion on public procurement. The number of business entities registered with ProZorro as actual and potential bidders increased from 64,000 in 2016 to 240,000 in 2020, reflecting strong expansion in small and medium enterprise interest in participation. The share of competitive public procurement almost doubled by 2018 (from 35 percent in 2013), exceeding the first DPL (2014) tar- get of 55 percent (Smits et al. 2019; World Bank 2019c). Annual savings from more competitive procurement were estimated by the Ministry of Finance at 0.7 percent of GDP in 2019 (Ministry of Finance of Ukraine 2019). 22 Some progress was made in public investment management (PIM) through establishing, and ensuring compliance with, more transparent project- selection procedures. Most central government public investment projects since 2016 have been appraised and selected through a transparent process (tracked through a results indicator in the first and second DPLs and supported by World Bank technical assistance, including a PIM assessment and PEFA update). According to IEG interviews with government partners, the World Bank was one of the few international partners that maintained a long-term institutional engagement on PIM. The annual budget execution rate for public investment projects increased from 70 percent in 2010 to 91 percent in 2019, exceeding the CPS target of 80 percent. However, according to PEFA 2019, PIM is still one of the weakest elements in the country’s PFM system, lacking strategic and transparent allocation of overall resources. It was rated C+ in 2019, up from D+ in 2015, with the score for investment project costing decreasing to D in 2019 from C in 2015 (World Bank 2019c, 47). Investment spending remains fragmented because many ad hoc investment projects are still not subject to the established competitive selection procedures. The total costs of such projects are nearly double the value of properly selected investments (World Bank 2019c). Independent Evaluation Group World Bank Group    23 1  These institutions included the National Anti-Corruption Bureau with investigative func- tions, the Specialized Anti-Corruption Prosecutor’s Office to prosecute high-level corruption crimes, and the National Agency on Corruption Prevention, which is responsible for verifying the asset declarations of public officials and implementing conflict-of-interest provisions. 2  Other influential advisory services and analytics products included regular policy notes, pub- lic finance reviews (World Bank 2017g, 2018d), and a growth study (Smits et al. 2019). 3  See https://nabu.gov.ua/en. 4  In September 2020, the Department for International Development was replaced by the Foreign, Commonwealth & Development Office. 5  “Sandwich model” describes a partnership in which domestic civil society elaborates on policy ideas and implementation while the international community presses the political elite into adopting reforms (Nitsova, Pop-Eleches, and Robertson 2018, 7). 6  Public Expenditure and Financial Accountability assessments measure the ability of public financial management processes and institutions to contribute to desirable budget outcomes, aggregate fiscal discipline, strategic allocation of resources, and efficient service delivery. They include 31 key components in seven broad areas, each measured by a four-grade scale on which D is the lowest and A the highest score. 7  In 2019, Ukraine scored 63 on the International Budget Partnership’s Open Budget Index (0–100), compared with 64 for Romania, 60 for Poland and the Slovak Republic, and 59 for The World Bank Group in Ukraine, 2012–20  Chapter 2 Czechia (International Budget Partnership 2020). 8  ProZorro is a locally developed, low-cost information technology solution that was launched in 2015 with support from civil society. It provides comprehensive procurement coverage of the public sector, including subnational governments and major nonbudget entities. During 2016–18, the annual number of procurement tenders quadrupled, and their value tripled. The system is fully sustainable as it is funded by users’ (bidders’) fees. 24 3 | Crisis Response and Macroeconomic Resilience Highlights After the 2014–15 economic and financial crises, the World Bank Group joined an international coalition to help stabilize the Ukrainian economy. The Bank Group was an important contributor to restoring the health and stability of the banking system and enhancing the ca- pacity of core financial sector institutions. The International Finance Corporation contributed to improving corporate governance and risk management in the financial sector. Efforts to reform the banking sector’s regulatory framework fo- cused on legislative changes supported through prior actions in development policy loans. The majority of these changes became bogged down in the legislative process for a prolonged period by vested interests in the Rada. Although most legislation has sub- sequently passed, implementation remains weak in the face of an unreformed judiciary. The Bank Group made an important contribution to fiscal consol- idation through its support for pension reform and the reform of utility subsidies and health services. This support helped the gov- ernment bring household energy prices to market levels without social unrest. 25  Ukraine went through several economic crises during the evaluation period, and the Bank Group was a key partner in helping to manage them while supporting efforts to build greater economic resilience. This section assesses the relevance and effectiveness of the Bank Group’s engagement to restore macroeconomic stability, enhance resilience, and reduce macroeco- nomic and financial sector vulnerabilities. Enhancing Macroeconomic Resilience Fiscal and financial sector vulnerabilities were critical constraints to devel- opment in Ukraine. They became most visible during the global financial crisis of 2008–09, but a lack of political will and opposition from vested interests resulted in major policy reforms being delayed until the 2014–15 macroeconomic and financial crisis. In 2014, the exchange rate depreciated by 47 percent, inflation accelerated to 24 percent, the fiscal deficit exceeded 10 percent of GDP, and public debt (including guarantees) spiked from 36 (in 2011) to 70 percent of GDP (World Bank 2019c). Energy tariffs, house- hold utility subsidies, pensions, and health and social protection systems were major fiscal burdens. By 2014, subsidies for underpriced gas combined with other losses of Naftogaz (the state-owned gas monopoly) accounted for 7 percent of GDP and for about 3.3 percent of GDP in direct budget subsidies. Throughout the evaluation period, the Bank Group maintained a good un- The World Bank Group in Ukraine, 2012–20  Chapter 3 derstanding of Ukraine’s macroeconomic, fiscal, and financial sector vulner- abilities and provided relevant financial and technical support. World Bank support ($2.2 billion) was large relative to International Bank for Recon- struction and Development borrowing limits, but modest compared with the resources provided by the IMF ($17.5 billion) and the EU (€11 billion) through parallel arrangements. Budget support operations (DPLs) beginning in 2014 sought to incentivize the policy reforms needed to regain macroeco- nomic and financial sector stability and strengthen institutions for economic management. A significant body of World Bank ASA underpinned the identi- fication and design of reforms supported by the DPLs. By 2017–18, macroeconomic stability had been reestablished, with the World Bank supporting many of the underlying reforms, including improving fiscal 26 consolidation; reforming energy tariffs to reduce subsidies and the quasi-fiscal deficit; strengthening the social safety net to cushion the impact of higher energy prices on poor people; and stabilizing the banking sector. The overall fiscal deficit declined to just over 2 percent of GDP in both 2017, down from 10 percent of GDP in 2014. Public and publicly guaranteed debt declined from 85 percent of GDP in 2014 to 61 percent in 2018 and further to 50 percent by the end of 2019. The World Bank made an important contribution to fiscal consolidation through its support for pension reform and reform of utility subsidies and health services. This support (through policy dialogue, technical analysis, DPL prior actions, and targeted ASA and technical assistance) helped the government bring household energy prices to market levels without so- cial unrest. Spending on the subsidy declined from a peak of 2.1 percent of GDP in 2017 to 1.4 percent in 2019. Scaling down the subsidy contributed to an overall reduction in social spending from 4.4 percent of GDP in 2017 to 3.0 percent in 2019. The government, with World Bank support, slowly consolidated the existing categorical benefit programs and expanded the means-tested guaranteed minimum income program. The primary reform promoted by the World Bank in social protection was a dramatic expansion of the household utility subsidy (HUS) in 2014–15 to protect the population and maintain support for energy reform. The rapid expansion of the HUS (from fewer than 5 percent of households in 2014 to 55 percent by mid-2015) was underpinned by World Bank analytical and Independent Evaluation Group World Bank Group    27 simulation work on the distributional effects of increased energy prices, mit- igation measures, and the distributional incidence of subsidies. DPLs in 2014 and 2015 contained prior actions related to reforming Ukraine’s inefficient and inequitable gas and heating subsidies while protecting poor people. These actions were also based on World Bank analytical work showing that if the energy sector moved rapidly to full cost recovery, household subsidies would need to be quickly increased in conjunction with major simplifica- tion of the benefits application process. Bank Group support in these areas facilitated a peaceful, rapid alignment of energy tariffs, taking advantage of earlier investments in benefits processing and administrative capacity. However, the expanded HUS was not fiscally sustainable, costing 1.8 percent of GDP at its peak in 2016. The second DPL took a longer-term perspective, requiring measures to shrink and apply a means test to the HUS benefit and, in parallel, to gradually replace the universal child benefit with the income- based guaranteed minimum income. The IMF relied heavily on World Bank expertise when implementing the 2014–15 energy price reform, and there was close coordination with the IMF when designing policy lending on pensions and social protection. Importantly, within the World Bank, easy access to the Energy Sector Management Assistance Program facility (see chapter 4) enabled cross-sectoral work, including with energy and social protection sector specialists. The World Bank’s support for pensions and social protection focused on effi- ciency, fiscal sustainability, and equity. It involved extensive analytical work and policy dialogue, as well as pension-related prior actions in two DPLs. The pension system was a major fiscal burden due to rapid population aging, labor emigration, and incentives to hide income from taxation. The social protection system was complex and expensive, and the bulk of its spending did not reach poor people. The $750 million FY18 policy-based guarantee contained prior actions to enhance the adequacy and sustainability of old- age pension benefits and included the enactment (in 2017) of the pension reform law aimed at improving the program’s fiscal sustainability, adequacy of the benefits, and incentives to contribute. The underlying reforms im- posed stricter eligibility requirements, phased out early retirement schemes, adopted clear indexation rules, and introduced a flexible retirement age The World Bank Group in Ukraine, 2012–20  Chapter 3 corridor that effectively increased the retirement age. Pension expenditures were reduced from about 18 percent of GDP in 2010 to less than 11 percent in 2016 and stayed relatively stable through 2019. The FY20 $350 million First Economic Recovery DPL (World Bank 2020c) included a prior action on pension indexation intended to boost the purchasing power of benefits and improve their predictability and sustainability. Ukraine’s response to the COVID-19 crisis has demonstrated improved eco- nomic management and a more resilient economy. Although overall mac- roeconomic vulnerabilities increased during 2020, driven by both the direct impact of the pandemic (including a 4.4 percent decline in GDP) and strong pressures from vested interests to ease fiscal and monetary policies, to date, the deterioration in macroeconomic indicators and financial sector stability 28 has been manageable. Reducing Financial Sector Vulnerabilities Ukraine’s financial sector was highly vulnerable in the wake of the global financial crisis of 2008–09. Compounding weak sector governance, the reces- sion and depreciation caused a major banking crisis in the country leading to deposit outflows, rising number of NPLs, and many bank failures. The share of NPLs reached 55 percent of total loans in 2017, up from 17 percent in 2012, among the highest in the world at the time (World Bank 2021). The relevance and ambition of the Bank Group–supported program in the fi- nancial sector grew during the period under review. Under the CPS FY12–16, the Bank Group’s objectives were quite modest (World Bank 2012b). Reflect- ing the fact that authorities had shown little interest in fundamental reforms of the financial sector, the strategy included only one subobjective (out of 20)—on the stability of the financial system—to be pursued by strengthen- ing banking sector regulation and supervision. Despite the lack of appetite for reform, the World Bank—through policy dialogue, analytical work, and technical assistance—continued to advocate for reforms, accumulated sector knowledge, and built relationships with technical counterparts in the gov- ernment. This proved to be a sound strategy, given that, with no active IMF program in Ukraine at that time and relatively modest IMF engagement in the financial sector, the World Bank was well positioned to quickly respond with support when the opportunity presented itself. Independent Evaluation Group World Bank Group    29 After 2014, support for financial sector reform became a top priority for the Bank Group. The primary instrument used by the World Bank to advance reforms was the financial sector DPL series (two $500 million loans approved in 2014 and 2015). The financial sector DPL series supported reforms to (i) improve the legal and institutional framework to strengthen the resiliency of the banking system, (ii) improve the solvency of the banking system through bank recapitalization, and (iii) strengthen the financial, operational, and regulatory capacity of the Deposit Guarantee Fund (DGF) for the resolution of insolvent banks. The program was boosted by a well-funded technical assistance program. In addition, the 2018 policy-based guarantee contained prior actions to improve the governance of state-owned commercial banks and facilitate the resolution of NPLs. The World Bank played a prominent role in supporting the stabilization and cleanup of the banking sector in 2014–16, and since 2017, has focused on reforms in state-owned commercial banks to address the NPL overhang. Initially World Bank technical assistance focused on building capacity in the DGF (established with direct support from the World Bank in 2010), which, as a new institution, had not been captured by vested interests (box 3.1). The DGF led efforts to close failed banks and enable asset recovery. The DGF’s financial capacity to reimburse insured depositors in failed banks was strengthened, and longer-term funding arrangements were established. These steps helped prevent financial panic and large-scale withdrawal of bank deposits, although some challenges remained for the DGF to achieve financial sustainability.1 In December 2016, on the advice of the IMF and World Bank, the largest private bank in the country (PrivatBank) was de- clared insolvent and was nationalized. This was an unpopular but necessary decision to advance the cleanup of the banking system—PrivatBank alone was responsible for 43 percent of total NPLs (National Bank of Ukraine 2019, 39). By 2018, 97 banks had been closed (more than half of the total number of operational banks in 2014). In the latter half of the evaluation period, the World Bank contributed to the effort to restore the credibility and in- dependence of the National Bank of Ukraine through technical assistance (box 3.1).2 With Bank Group support, the banking sector regulatory framework was up- The World Bank Group in Ukraine, 2012–20  Chapter 3 graded, aligned with the EU regulations, and made consistent with the Basel III principles. As of the end of July 2018, all banks met the minimum regula- tory capital of 7 percent of risk-weighted assets. The banks’ core equity cap- ital ratio increased from 8.3 percent in late 2015 to 11.2 percent by the end of July 2018 and to 13.5 percent at the end of 2019. Related party exposures substantially declined, whereas liquidity ratios improved (World Bank 2021). IFC helped identify and address vulnerabilities in risk management and corporate governance. IFC helped establish a market for distressed assets and improved risk management in the financial sector through its Financial Market Crisis Response advisory services project, benefiting six banks. These efforts targeted institutional gaps in asset markets, which held back the cleanup of bank balance sheets, and contributed to the strengthening of cor- 30 porate and risk management in the banking sector. IFC interventions were complementary to World Bank efforts and included pre-privatization invest- ment, investment in privatized state-owned enterprises, stronger corporate governance, and short-term trade and agribusiness financing.  orld Bank Group Support for Enhancing the Capacity of the Box 3.1. W National Bank of Ukraine and the Deposit Guarantee Fund The World Bank Group’s advisory support to the National Bank of Ukraine and the Deposit Guarantee Fund (as part of programmatic financial sector technical assistance) covered bank supervision, bank resolution, and deposit insurance. The Bank Group helped deliver two Bank Quality Asset Reviews in 2015–16 that assessed the scale of the banking crisis and identified steps to clean up the system. Just-in-time advice included assistance to the National Bank of Ukraine to develop recapitalization plans and to commence the disclosure of the banks’ ultimate beneficiary owners. The Bank Group was also a major source of technical assistance for the Deposit Guar- antee Fund with a focus on strengthening its institutional capacity to carry out effective bank resolution and liquidation, as well as securing sufficient funding. This support helped advance reforms at a time when National Bank of Ukraine was perceived to be captured by vested interests and therefore unable to fulfill its mandate. The World Bank helped the authorities modernize key financial sector laws and advised them on preparing the new Insolvency Code, a critical component for improving the nonper- forming loan resolution framework. The International Finance Corporation provided Independent Evaluation Group World Bank Group    31 vital inputs to the design of the financial sector development policy loan series. The substance of the sector reform program and associated analytics were presented in several Bank Group reports, including policy notes (fiscal year 2015) and the sources of growth study (fiscal year 2019). Source: Independent Evaluation Group. Despite these achievements, Ukraine’s banking sector remains vulnerable. The pace of reform slowed considerably after 2016 as the risk of the banking sector’s collapse diminished. Passage of key pieces of World Bank– supported legislation was impeded by vested interests in the Rada. It was not until after the 2019 election that legislation was passed, although implementation challenges remained. Indeed, several past decisions on bank closures have been challenged in courts, drawing into question the real impact of changes to the legislative and regulatory frameworks. In addition, although the share of NPLs declined gradually in 2019–20, it remained high at 41 percent, with state-owned commercial banks accounting for 73 percent of the total. Largely due to the burden of NPLs and weak de facto protection of creditor rights, commercial bank lending to the private sector has remained depressed and annual credit growth has been negative in real terms since 2015, failing to meet the CPF target of 10 percent. The absence of meaningful judicial reform continues to undermine efforts to reduce the large stock of NPLs burdening the banking sector. Public support for financial sector reform was weak throughout much of the review period. Reasons for the large number of bank closures were poorly understood by the public, who came to associate this aspect of the financial sector reform agenda with external players such as the World Bank and IMF. Although dissemination and outreach were evident in several aspects of the GAC agenda, this was not the case with respect to financial sector reform. As stressed in IEG’s evaluation of Bank Group support for addressing country-level fiscal and financial sector vulnerabilities (World Bank 2021), with a relatively low level of financial literacy among the general public in Ukraine, a concerted effort was needed to foster a better understanding of the importance of banking sector reform and the costs of not reforming. The World Bank Group in Ukraine, 2012–20  Chapter 3 32 1  Facing sizable repayments to depositors in 2015–17, the Deposit Guarantee Fund borrowed long term from the government and the National Bank of Ukraine. The specific repayment arrangements for these loans remain unclear. 2  In 2019, the National Bank of Ukraine won the Global Central Banking Transparency award from Central Banking magazine. Independent Evaluation Group World Bank Group    33 4 | Energy Security and Efficiency Highlights Main energy sector challenges in Ukraine included high energy intensity, weak governance, outdated infrastructure, and high de- pendence on a gas supply from a single source. Energy subsidies were a major fiscal burden. After 2014, the World Bank actively helped Ukraine reform the en- ergy sector, focusing on governance, efficiency, and security. The main areas of progress with World Bank support were energy tariff reforms, unbundling of the natural gas monopoly, diversification of the gas supply, and institutional arrangements for promoting energy efficiency. At the same time, despite World Bank Group support, progress on reforming electricity markets and strengthening regulatory capac- ity has been slow. Institutional reforms and the virtual elimination of subsidies have yet to translate into more private investment in sector infrastructure and visible improvements in energy services for consumers. 34   Energy supply, services, and transit have played critical roles in Ukraine’s development. Ukraine’s energy sector has been shaped by multiple forces stemming from ideological (market-based or subsidized) and geopolitical contestation. These factors have fueled strong incentives for market capture and control by a few private players, weakened sector governance, and con- tributed to underinvestment in aging and inefficient infrastructure. The high energy intensity of the economy—a legacy of the industrial base, housing stock, and Soviet Union–era pricing policies—has made tariff reforms deeply unpopular and difficult in the face of limited progress in enhancing service quality and affordability. This chapter discusses Bank Group contributions to developments in the core areas of its energy sector support: (i) sector gover- nance and transparency (including fiscal implications), (ii) energy efficiency, and (iii) energy security. Sector Governance and Transparency At the beginning of the review period, the Ukrainian energy sector was characterized by weak governance, outdated infrastructure, high dependence on gas imports from a single source, and large fiscal costs. The government resisted reforms to promote cost-recovery tariffs, competition, and indepen- dent regulation. The practice of “socializing” high energy costs through the budget had strained the country’s fiscal condition. By 2014, direct subsidies for underpriced gas, combined with other losses of Naftogaz (the state- Independent Evaluation Group World Bank Group    35 owned gas monopoly), accounted for 7 percent of GDP (including 3.3 percent of GDP in direct subsidies). Before 2014, Bank Group support to the ener- gy sector was dominated by investment lending (with technical assistance components), and meaningful institutional reforms did not begin until 2014. This step coincided with far-reaching tariff reforms and the suspension of gas imports from the Russian Federation, with the energy sector serving as a major contributor to the macroeconomic crisis. Taking advantage of the political opening after 2014, the Bank Group helped establish a framework for competition, private investment, and independent regulation. In parallel with IMF and EU support, the World Bank increased its work on tariff and subsidy reform through DPLs and technical assistance, which (twinned with a better-targeted HUS program) virtually eliminated budget subsidies and generated enhanced transparency in the gas sector. The World Bank effectively leveraged donor funds to provide just-in-time and technical assistance and helped implement socially sensitive tariff and subsidy reforms across different ministries, development partners, and the energy regulatory commission.1 Bank Group and IFC advisory services and technical assistance supported aggressive “catch-up” tariff and social protection reforms and pushed for further unbundling and competition (the fiscal impact of energy tariff reform is covered in chapter 3). Other notable results included the strengthening of institutional arrangements for promoting energy efficiency by establishing the Energy Efficiency Fund;2 the strengthening of financial intermediaries, especially the state-owned UkrGasBank (through IFC technical assistance to develop capacity and governance); and the establishment and strengthening of the technical capacity of the National Energy and Utilities Regulatory Commission. Power sector investment projects included technical assistance for strengthening regulatory capacity and independence and developing a new model for the wholesale electricity market.3 Specific support was provided by the Bank Group for the government’s plan for implementation of gas sector reform (adopted in 2015) to comprehensively restructure the gas sector, including developing and implementing key legislation.4 The World Bank was the government’s main partner in the design and imple- mentation of the unbundling of Naftogaz. Ukraine made this commitment The World Bank Group in Ukraine, 2012–20  Chapter 4 as part of its 2014 Association Agreement with the EU. DPL prior actions, investment lending, and technical assistance supported the restructuring of Naftogaz and distribution companies, improvements in payment discipline, debt resolution in the energy sector, audits of Naftogaz and its subsidiaries, progress in transparent privatization, and partial channeling of gas-transit fees to the budget through a treasury single account. IMF and EU budget support and European Investment Bank investment and liquidity support were contingent on unbundling. The unbundling continued in 2019, with Naftogaz’s transmission and transit functions being separated from the rest of the company, in line with recommendations from the Bank Group study of unbundling options. Cooperation with development partners was an important aspect of the 36 Bank Group’s support for the energy sector. Joint work under the aegis of the Brussels declaration allowed donors to agree on an international financial institution–EU list of key structural reforms to enable investment in the gas sector. Trust funds played a vital role in supporting Bank Group analytical work on such topics as tariff and social protection reform and Naftogaz unbundling. After the 2014 crisis, the Energy Sector Management Assistance Program provided timely funding for a multisector World Bank team and used entry points from different sectors to forge consensus among counterparts and development partners to design and implement tariff and subsidy reforms.5 However, reforms have not been followed by commensurate private invest- ment in the sector and did not enhance customer satisfaction and choice. Municipal-level ownership and coordination of reforms and the national gov- ernment’s commitment to modernize services remain weak. Ukraine’s energy sector needs to attract substantial investments for modernization and effi- ciency improvements to provide the level of services necessary to sustain the reform effort, including accessing the EU single market for gas and electricity, enhancing the competitiveness of its industrial base, and managing green- house gas emissions in accordance with international commitments. There has been little progress in enhancing competition in the electricity sector, which is still dominated by a few powerful private players. The energy and competition commissions have been unsuccessful in curtailing the market power of major private players or preventing them from padding supply costs, which are then passed on to the market (or ultimately borne by taxpayers). Independent Evaluation Group World Bank Group    37 Debts to energy suppliers have been increasing because of unaffordable tariff incentives offered for renewable energy-based power generation, and the market power of a few players to manipulate costs remains unchecked, under- mining the sustainability of earlier achievements in this area. Energy Efficiency Bank Group policy advice, lending, and support to institutional and financing mechanisms has helped put Ukraine on a more energy-efficient path. Bank Group–supported tariff and subsidy reforms, together with investments to improve the reliability and efficiency of aging and outdated supply infrastructure, have begun to reduce energy demand and create incentives for greater efficiency on both the supply and demand sides (table 4.1). Progress on competition in the gas sector, facilitated by the unbundling of Naftogaz (see the Sector Governance and Transparency section in this chapter), and steadily improving metering and management of district heating are expected to lead to improved services, enhanced accountability, and reduced corruption. Gas demand has dropped from 50.7 billion cubic meters in 2012 to 25.0 billion cubic meters in 2019 (of which about 20 billion cubic meters were produced from domestic resources). Total primary energy demand, energy intensity, and carbon dioxide emissions also dropped during the evaluation period by 37, 25, and 33 percent, respectively. Correspondingly, the monetary value of the domestic gas market decreased from $14.5 billion equivalent in 2012 to $7.7 billion in 2019. Table 4.1. Ukraine Energy Supply and Transit Energy Type 2010 2013 2015 2017 2019 2020 Total energy (ktoe) 73,950 70,087 50,856 49,912 — — Natural gas supply (bcm) — 46.0 29.7 27.4 25.0 — Natural gas transit (bcm) 98.6 86.1 67.1 93.5 89.6 55.8 Electricity (GWh) 130.1 124.1 117.1 104.9 119.9 — Portion from renewable energy 0.0 0.7 1.0 1.2 3.6 7.3 (nonhydro, %) Coal and gas (%) 41.5 44.7 35.2 35.9 36.2 35.2 Sources: International Energy Agency—Data and Statistics; Ministry of Energy of the Russian Federation; The World Bank Group in Ukraine, 2012–20  Chapter 4 Naftogaz; National Energy and Utilities Regulatory Commission. Note: bcm = billion cubic meter; GWh = gigawatt hour; ktoe = kiloton of oil equivalent; — = not available. Elements of a strategy for the recovery of district heating have been put in place, but actual improvements in energy services have not been seen at the level of households. These elements include energy tariff reform, with targeted social protection; financing and technical support to businesses, including small and medium enterprises, supported by IFC technical assistance for energy-efficient investments; financing and technical support to households through an Energy Efficiency Fund (supported by the World Bank); joint IFC–Bank Group design and operationalization of an EU energy efficiency grant for residential buildings; and support to municipalities for energy-efficient district heating (the World Bank’s District Heating Energy Efficiency Project). At the same time, actual improvements in energy 38 services (electricity, gas, and district heating) are not being seen at the level of households or experienced by consumers, with recent surveys indicating little net change in the perception of respondents about the quality of services (see appendix C). IFC has played an important role in promoting energy efficiency. IFC developed energy efficiency–oriented financial products (investment loans and targeted credit lines for energy efficiency improvements through financial intermediaries). It partnered with the EU to jump-start the energy-efficient renovations of multifamily buildings, provided advisory services on cleaner production and residential energy efficiency, and helped structure public- private partnership transactions and transparent mechanisms to attract private sector financing.6 IFC also partnered with the state-owned UkrGasBank (which it would not typically do, given its private sector mandate) to increase lending to businesses to make them more energy efficient and competitive. Energy Security Bank Group support contributed to breaking the country’s dependence on gas imports from a single source. This outcome was achieved through dif- ficult but effective tariff reforms resulting in a significant reduction in gas demand and a diversification of supply to European suppliers. The $500 mil- lion gas-supply guarantee (approved in 2016) provided by the Bank Group Independent Evaluation Group World Bank Group    39 in partnership with the European Investment Bank promoted competition among gas suppliers by enabling a financially weak Naftogaz to procure gas competitively from European suppliers, breaking its dependence on Rus- sian suppliers in an affordable manner. Furthermore, as a result of Naftogaz unbundling, the newly created Gas Transmission System Operator, unen- cumbered by domestic gas production and other nontransmission functions, was able to leverage Ukraine’s locational advantage to negotiate a favorable gas-transit agreement with the Russian Federation’s Gazprom for 2020–24. Gas-transit volumes dropped from approximately 86–88 billion cubic meters in 2012–13 to 62 billion cubic meters in 2014 (the year of the crisis) before rising to approximately 90 billion cubic meters in 2019, yielding a transit revenue of $2.7 billion (table 4.1). However, this revenue is unlikely to be sustained, given recent and ongoing developments in gas-transit routes. 1  World Bank Group–supported outputs included “Advancing Energy Tariff and Subsidy Re- forms” (2018); “Facilitating Electricity and Gas Market Reforms in Ukraine” (2018); “Advice to the Design and Creation of Energy Efficiency Fund” (2017); “Ukraine: Moving Forward Energy Tariff Reforms” (2017); “Social Safety Nets and Energy Reform” (2016); “Mitigating the Impact of Gas and Heating Tariff Increases through Targeted Cash Transfers and Energy Efficiency Measures” (2015); “Advisory Services and Technical Assistance to Naftogaz and the Government of Ukraine on the Reform of the Natural Gas Sector” (2015); and “Analysis of the Restructuring Options of NJSC [National Joint Stock Company] Naftogaz, Part 1: Unbundling Options for Gas Transmission; Part 2: Unbundling Options for Gas Storage” (2016, jointly with European Commission). 2  The Energy Efficiency Fund is a grant facility to jump-start the energy-efficient renovation of multifamily buildings, including funding grants for residential building retrofitting. 3  “Power Transmission” (2008–16), “Second Power Transmission Project” (2015–20), “Hydro- power Rehabilitation” (2005–16), and “District Heating Energy Efficiency Project” (2014–21). 4  Natural Gas Market Law (2015), Law on National Energy and Utilities Regulation (2016), Elec- tricity Market Law (2017), and Law Establishing the (Residential) Energy Efficiency Fund (2017). 5  The Energy Sector Management Assistance Program is a global knowledge and technical assistance program, established in 1983 and administered by the World Bank. Its mission is to assist low- and middle-income countries to increase skills and institutional capacity to achieve environmentally sustainable energy solutions for poverty reduction and economic growth. The World Bank Group in Ukraine, 2012–20  Chapter 4 6  Public-private partnerships with Kyiv District Heating company and Centrenergo, the last remaining state-owned thermal generation company. 40 5 | Findings and Lessons Findings During the period covered by this evaluation, the Bank Group helped Ukraine undertake several important reforms. Alongside development partners, the Bank Group helped the government stabilize the economy and the banking sector, improve the fiscal situation, establish the institutional underpinnings to improve governance, reform the health and pension systems, and enhance energy security. At the same time, a lack of attention to important enabling areas with systemic impact, such as justice sector and public administration reform, undermined the impact of reforms in several areas. The Bank Group was well positioned to leverage opportunities arising from the events of 2014–15. Throughout the evaluation period, the Bank Group had a clear understanding of the challenges and vulnerabilities and present- ed a well-prioritized picture of what needed to be done. The Bank Group was able to leverage long-standing partnerships with diverse counterparts and a strong presence in the field. In the early part of the evaluation period, in the face of weak demand for reforms, the World Bank invested in building country knowledge and local partnerships and was well prepared to adjust its strategy and expand all aspects of its engagement when a political opening presented itself. After 2014, the World Bank made a fundamental adjustment to its original strategy, reallocating funds from slow-moving investment projects and supporting critical reforms through budget support operations. These achievements are not irreversible. In Ukraine’s volatile political envi- ronment, it is difficult to gauge how well the policy and structural changes facilitated by the Bank Group will withstand pressures from powerful vested interests and shifts in public opinion. Ukraine’s economy remains vulnerable to a variety of macroeconomic and political risks, aggravated by the uncer- tainties of the COVID-19 pandemic. Vested interests remain well organized, and populist pressures for policy reversal and risky macroeconomic initia- tives have been increasing. Other vulnerabilities include weak domestic ownership of some reforms, a lack of a robust national proreform consensus, 41 and inadequate attention to institutional building in critical enabling areas (judiciary, public administration, decentralization, and regulatory capacity). Some important reforms supported by the Bank Group have not produced tangible improvements in the daily lives of the general population. The establishment of high-level anticorruption institutions has not translated into a sustained increase in prosecutions in cases of corruption or improved public perceptions of the pervasiveness of corruption. Similarly, significant institutional and structural reforms in energy and social sectors (health, pensions), while important for resolving the fiscal crisis and reducing oppor- tunities for corruption, have not yet resulted in higher levels of private in- frastructure investment or improved service standards. A lack of attention to justice sector and public administration reform has undermined the impact of reforms in several areas. Partnerships were an important aspect of the Bank Group’s engagement during the review period and helped mobilize and leverage significant bilat- eral funding. The World Bank team helped coordinate various donor activities in several areas that were central to its own strategy and provided consid- erable analytical input to increase the overall effectiveness of multidonor programs. Bank Group partnerships were especially effective and well-coordi- nated in the context of macroeconomic stabilization. Development partners drew extensively on the World Bank’s country knowledge and high-quality, The World Bank Group in Ukraine, 2012–20  Chapter 5 just-in-time technical assistance in key sectors (such as energy, health, and social protection). The strength of the obstacles presented by vested interests was obvious in the lead-up to and throughout the evaluation period. A clearer initial sense of the results chain leading to effective enactment and implementation of key legislation might have highlighted potential points of stress and risks to the achievement of results earlier in the process. This understanding would have pointed to the need for more deliberate attention to judicial and parlia- mentary constraints and the need to build demand for better governance and commitment to follow up in operations with the financial sector. Even though other development partners “led” on judicial reform, given the importance of the court system to building effective financial sector crisis preparedness, the Bank Group might have sought to play a bigger role in this area. 42 Bank Group engagement with the nongovernmental sector contributed to the success of reforms in several areas. Engaging more directly and frequent- ly with civil society was an effective way to foster public understanding for reforms and build demand for better governance. A considerable effort was made by the World Bank to include CSOs in consultations on various aspects of the World Bank–supported reform agenda. There were also instances of effectively integrating local knowledge providers into the delivery of ana- lytical work and the monitoring of reform implementation progress at the local level. The FY12–16 CPS Completion and Learning Review (World Bank 2017f) concluded that, in Ukraine, Bank Group efforts had the biggest impact when the dissemination of Bank Group analytical work provided a platform for public debate, since it is a valuable tool to help civil society confront vested interests. In hindsight, the Bank Group could have invested more ef- fort in explaining to the public the need for the reforms it supported (partic- ularly in the financial sector). Governance and anticorruption: Bank Group assistance raised the profile of the GAC agenda in Ukraine and helped mobilize and coordinate large amounts of bilateral funding to support reforms. In addition to support for establishing and strengthening a set of new apex anticorruption institutions, several sectoral engagements generated important anticorruption benefits, including through tariff and subsidy reforms in the gas sector, the modern- ization of bank supervision, and a deregulation to reduce administrative Independent Evaluation Group World Bank Group    43 barriers for small and medium enterprises. Stronger engagement on justice sector reform could have contributed to progress in other areas, including in the financial sector. Crisis response and macroeconomic resilience: The Bank Group made significant contributions to the international effort to help Ukraine respond to economic crises. On fiscal policy, the main achievements included the elimination of quasi-fiscal deficits in the energy sector and enhanced sustainability of the pension fund. The main outcomes in the financial sector included the restored health and stability of the banking system and enhanced supervisory capacity of core sector institutions and regulators. However, many critical fiscal and financial sector vulnerabilities remain, including weaknesses in tax performance and a largely unreformed state-owned enterprise sector (outside of energy). The pace of reform in the banking sector has slowed considerably as the risk to stability has diminished; NPLs are still high, negatively affecting private sector credit growth. In response to the COVID-19 pandemic, the World Bank was quick to restructure existing operations and prepare additional financing to provide immediate financial support. Energy: The Bank Group partnership on energy in Ukraine was highly rel- evant to the country’s needs and made important contributions, especially in implementing tariff and subsidy reforms, informing market design and legislation, and seeding financing and institutional arrangements for invest- ments to promote energy efficiency. However, there have been few tangible improvements in service delivery to consumers. The expectations embed- ded in reforms advocated by the World Bank since the 1990s—namely that unbundling, independent regulation, and privatization will mobilize private investments, depoliticize the sector, and enhance customer satisfaction— have yet to materialize. Lessons This CPE offers the following lessons: 1. Continuity of engagement during a period of weak demand for reform, including through strategically targeted analytical work, facilitated in- creased ownership within the government and positioned the Bank Group The World Bank Group in Ukraine, 2012–20  Chapter 5 to respond quickly when a window of opportunity presented itself. During 2012–13, when there was little appetite on the part of the government for significant policy reform, the Bank Group invested heavily in analysis and in establishing partnerships at the technical level of government. These efforts helped the Bank Group respond rapidly after the change in political leadership. For example, sustained World Bank engagement enabled the Ministry of Finance, after a long period of limited interest in reform, to design and implement a comprehensive PFM reform. 2. Greater World Bank attention was needed in the justice sector, given its importance to the efficacy of reforms across a range of other sectors. En- trenched interests often used the justice system to neutralize the impact of reforms in other sectors, thereby undermining the credibility of the broader reform effort and commitment to change. Lack of direct World 44 Bank engagement in the justice sector adversely affected the effective- ness of reforms in specific areas, particularly anticorruption reforms and reforms in the energy and financial sectors. In anticipation of significant engagement in Ukraine once military aggression against it ceases, the World Bank would be well advised to invest more in deepening its under- standing of the links between specific weaknesses in the justice system and Ukraine’s ability to make progress on specific development objectives. 3. Effective outreach and engagement by the World Bank with civil society is important to help the public understand the reasons for reforms and the costs of not reforming. Although Bank Group strategies envisaged broad engagement with civil society and the private sector, implementation was uneven across sectors, with a communications strategy on financial sector reforms particularly lacking. 4. Institutional reforms that impose a burden on citizens need to compen- sate by making progress in service delivery. Despite many institutional and policy reforms, Ukrainians remain deeply skeptical about the overall progress and impact of reforms on their daily life. General public support for reform-oriented processes, such as EU integration, may help sustain some momentum on institutional reform. However, institutional reforms that impose painful adjustments on the citizenry (such as tariff increas- es) need to be paired with improvements in service quality, including in infrastructure. Independent Evaluation Group World Bank Group    45 Bibliography American Chamber of Commerce in Ukraine. 2019. Ukraine Business Climate Survey Results. Kyiv: American Chamber of Commerce in Ukraine. Balabanyan, Ani, and Debabrata Chattopadhyay. 2017. “Concept Note: Facilitating Electricity and Gas Market Reforms in Ukraine.” World Bank, Washington, DC. Balabushko, Oleksii, Oleksandra Betliy, Veronika Movchan, Ruslan Piontkivsky, and Mykola Ryzhenkov. 2018. “Crony Capitalism in Ukraine: Relationship between Political Connectedness and Firms’ Performance.” Policy Research Working Paper 8471, World Bank Group, Washington, DC. Bertholet, Fabrice Karl. 2018. “Concept Note: Advancing Energy Tariff and Subsidy Reforms.” World Bank, Washington, DC. Bussolo, Maurizio, María E. Dávalos, Vito Peragine, and Ramya Sundaram. 2018. Toward a New Social Contract: Taking on Distributional Tensions in Europe and Central Asia. Washington, DC: World Bank Group. Krasnolutska, Daryna, and Volodymyr Verbyany. 2020. “Ukraine’s Leader Is Being Broken by the System He Vowed to Crush.” Bloomberg, December 17, 2020. https://www.bloomberg.com/news/articles/2020-12-17/ukraine-s-leader-is-be- The World Bank Group in Ukraine, 2012–20  Bibliography ing-broken-by-the-system-he-vowed-to-crush. Caprio, Gerald, Jr. 2018. “Assessing the FSAP: Quality, Relevance, and Value Added.” Independent Evaluation Office Background Paper BP/18-02/02, International Monetary Fund, Washington, DC. Center for Insights in Survey Research. 2021. “Public Opinion Survey of Residents of Ukraine, March 13–21, 2021.” International Republican Institute, Washington, DC. https://www.iri.org/sites/default/files/wysiwyg/public_-_03.2021_nation- al_eng-_public.pdf. Centre for Economic Strategy. 2017. Critical Thinking Against Populism. Kyiv: Centre for Economic Strategy. Delcour, Laure, Antoaneta Dimitrova, Klaudijus Maniokas, and Kataryna Wolczuk. 46 2019. “State Capacity, State Capture and Their Effects on the Implementation of Association Agreements in the Eastern Neighbourhood.” EU-STRAT Working Paper 19, EU and Eastern Partnership Countries—An Inside-Out Analysis and Strategic Assessment, Berlin. Democratic Initiatives Foundation. 2018. National Anticorruption Study. Kyiv: Demo- cratic Initiatives Foundation. Dubrovskiy, Vladimir, Kálmán Mizsei, Kateryna Ivashchenko-Stadnik, and Mychailo Wynnyckij. 2020. Six Years of the Revolution of Dignity: What Has Changed? Kyiv: CASE Ukraine. Eastern Partnership. 2020. Joint Communication to the European Parliament, the European Council, the Council, the European Economic and Social Committee, and the Committee of the Regions: Eastern Partnership Policy Beyond 2020: Reinforcing Resilience—An Eastern Partnership That Delivers for All. Brussels: Eastern Partnership. EBRD (European Bank for Reconstruction and Development). 2020. Ukraine Road Corridors. London: EBRD. IFC (International Finance Corporation). 2019. Ukraine Agri Aggregator Advisory Services Completion Report. Washington, DC: IFC. https://disclosures.ifc.org/ project-detail/AS/592348/ukraine-agri-aggregator. IFC (International Finance Corporation). 2020. Creating Markets in Ukraine: Country Private Sector Diagnostic. Washington, DC: IFC. IMF (International Monetary Fund). 2014a. “Ukraine: Ex Post Evaluation of Excep- Independent Evaluation Group World Bank Group    47 tional Access under the 2010 Stand-by Arrangement.” Country Report 14/146, IMF, Washington, DC. IMF (International Monetary Fund). 2014b. “Ukraine: Request for a Stand-by Ar- rangement.” Country Report 14/106, IMF, Washington, DC. IMF (International Monetary Fund). 2017. “2016 Ukraine Article IV Consultation.” Country Report 17/83, IMF, Washington, DC. IMF (International Monetary Fund). 2019. “Ukraine: Request for Stand-by Arrange- ment and Cancellation of Arrangement under the Extended Fund Facility—Press Release; Staff Report; and Statement by the Executive Director for Ukraine.” Country Report 19/3, IMF, Washington, DC. IMF (International Monetary Fund). 2020a. “Ukraine: Evaluation of the Exceptional Access to the 2015 Extended Arrangement—Press Release and Report.” Country Report, 20/2004, IMF, Washington, DC. IMF (International Monetary Fund). 2020b. “Ukraine: Request for Stand-by Arrange- ment and Staff Report.” Country Report 20/197, IMF, Washington, DC. Institute for Economic Research and Policy Consulting. 2020. Municipal Competitive- ness Index 2019/2020. Kyiv: Institute for Economic Research and Policy Consult- ing. International Budget Partnership. 2020. Open Budget Survey 2019. Washington, DC: International Budget Partnership. International Labour Organization. 2019. Future of the Ukrainian Pension System: Adequacy, Coverage and Sustainability. Decent Work Technical Support Team and Country Office for Central and Eastern Europe. Budapest: International Labour Organization. Kohli, Sandeep. 2020. “Concept Project Information Document—Ukraine: Facilitating Power System Integration with Europe Project.” World Bank, Washington, DC. Kyiv International Institute of Sociology. 2015. “Corruption in Ukraine: Comparative Analysis of National Surveys: 2007, 2009, 2011, and 2015.” Kyiv: Kyiv Interna- tional Institute of Sociology. The World Bank Group in Ukraine, 2012–20  Bibliography Liu, Feng. 2017. “Completion Summary: Ukraine Advice to the Design and Creation of Energy Efficiency Fund.” World Bank, Washington, DC. Lough, John, and Vladimir Dubrovskiy. 2018. “Are Ukraine’s Anti-Corruption Re- forms Working?” Chatham House Russia and Eurasia Programme, London. Manroth, Astrid. 2015. “Completion Summary: Ukraine: UA-Municipal Energy Effi- ciency Financing.” World Bank, Washington, DC. Menon, Rekha, and Bianca Frogner. 2010. “What Underlies Ukraine’s Mortality Cri- sis?” Human Development Sector Unit, Europe and Central Asia, World Bank, Washington, DC. Ministry of Agrarian Policy and Food. 2018. Concept of Development of Farms and Agricultural Co-operation 2018–20. Kyiv: Ministry of Agrarian Policy and Food. 48 Ministry of Economic Development and Trade of Ukraine. 2017. Export Strategy of Ukraine: Strategic Trade Development Road Map 2017–21. Geneva: The Interna- tional Trade Center. Ministry of Finance of Ukraine. 2019. Report on Public Procurement. Kyiv: Ministry of Finance of Ukraine. Ministry of Health of Ukraine. 2015. National Health Reform Strategy for Ukraine 2015–20. Kyiv: Ministry of Health of Ukraine. National Bank of Ukraine. 2019. Financial Stability Report. Kyiv: National Bank of Ukraine. National Health Service of Ukraine. 2020. Report: The National Health Service of Ukraine 2019. Kyiv: National Health Service of Ukraine. National Investment Council of Ukraine. 2018. Agricultural Sector of Ukraine: Secur- ing the Global Food Supply. Kyiv: National Investment Council of Ukraine. Nitsova, Silviya, Grigore Pop-Eleches, and Graeme Robertson. 2018. Revolution and Reform in Ukraine: Evaluating Four Years of Reform. Washington, DC: PONARS Eurasia, Institute for European, Russian and Eurasian Studies, Elliott School of International Affairs at the George Washington University. OECD (Organisation for Economic Co-operation and Development). 2017. Anti- Corruption Reforms in Ukraine: Fourth Round of Monitoring of the Istanbul Anti- Independent Evaluation Group World Bank Group    49 Corruption Action Plan. Paris: OECD. https://www.oecd.org/corruption/acn/ OECD-ACN-4th-Round-Report-Ukraine-ENG.pdf. OECD (Organisation for Economic Co-operation and Development). 2020a. Monitoring the Implementation of Ukraine’s SME Development Strategy 2017–20. Paris: OECD. https://www.oecd.org/eurasia/competitiveness-programme/ eastern-partners/Monitoring-the-Implementation-of-Ukraine’s-SME- Development-Strategy-2017-2020-ENG.pdf. OECD (Organisation for Economic Co-operation and Development). 2020b. OECD Monitoring of the Energy Strategy of Ukraine until 2035. Paris: OECD. https:// www.oecd.org/eurasia/competitiveness-programme/eastern-partners/ Monitoring-the-energy-strategy-Ukraine-2035-EN-.pdf. OECD (Organisation for Economic Co-operation and Development), European Union, European Training Foundation, and European Bank for Reconstruction and Development. 2020. SME Policy Index: Eastern Partner Countries 2020— Assessing the Implementation of the Small Business Act for Europe. Paris and Brussels: OECD and European Union. https://www.oecd.org/development/sme- policy-index-eastern-partner-countries-2020-8b45614b-en.htm. Olszański, Tadeusz A. 2017. “A Quarter-Century of Independent Ukraine: Dimensions of Transformation.” Number 64, Centre for Eastern Studies (OSW), Warsaw. Orange Health Consultants. 2018. “Health Care in Ukraine.” Netherlands Enterprise Agency, Rotterdam. Oxenstierna, Susanne, and Jakob Hedenskog. 2017. “Ukraine’s Economic Reforms: Prospects of Sustainability.” Report FOI-R-4472-SE, Swedish Defense Research Agency, Stockholm. Pavlenko, Olena. 2020. “Playing the Long Game: How Civil Society Can Lead Change.” DiXi Group, Kyiv. Pirani, Simon, Jack Sharples, Katja Yafimava, and Vitaly Yermakov. 2020. “Implica- tions of the Russia-Ukraine Gas Transit Deal for Alternative Pipeline Routes and the Ukrainian and European Markets.” Energy Insight Research Paper 65, Oxford Institute for Energy Studies, Oxford. The World Bank Group in Ukraine, 2012–20  Bibliography Romaniuk, Piotr, and Tetyana Semigina. 2018. “Ukrainian Health Care System and Its Chances for Successful Transition from Soviet Legacies.” Globalization and Health 14 (1): 116. Savin, Andriy, Bohdan Mykolaychuk, and Center for Combating Corruption. 2020. Історичні рішення, вироки та атаки. Чим запам’ятався перший рік Антикорупційного суду. Pravda, September 7, 2020. https://www.pravda.com. ua/articles/2020/09/7/7265567. Semikolenova, Yadviga Viktorivna. 2015. “Completion Summary: Ukraine—Miti- gating the Impact of Gas and Heating Tariff Increases through Targeted Cash Transfers and Energy Efficiency Measures.” World Bank, Washington, DC. Semikolenova, Yadviga Viktorivna. 2016. “Completion Summary: Ukraine—Moving Forward Energy Tariff Reforms.” World Bank, Washington, DC. 50 Smits, Karlis, Edgardo M. Favaro, Anastasia Golovach, Faruk Khan, Donald F. Lar- son, Kateryna Goychuk Schroeder, Gabriela Schmidt, Oleg Nivievskyi, Eugeniu Osmochescu, Hanna Ponomarenko, Jesus Crespo Cuaresma, Harald Oberhofer, Yevhen Hrebeniuk, and Colin Christopher Ek. 2019. Ukraine Growth Study Final Document: Faster, Lasting and Kinder. Washington, DC: World Bank. http://docu- ments.worldbank.org/curated/en/543041554211825812/Ukraine-Growth-Study- Final-Document-Faster-Lasting-and-Kinder. Socher, Richard, Alex Perelygin, Jean Wu, Jason Chuang, Christopher Manning, Andrew Ng, and Christopher Potts. 2013. “Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank.” Paper presented at the Confer- ence on Empirical Methods in Natural Language Processing, Seattle, October 18–21. Tax Administration Diagnostic Assessment Tool Secretariat. 2018. Performance Assessment Report for Ukraine. Washington, DC: Tax Administration Diagnostic Assessment Tool Secretariat. Ukraine Now. 2019. “Reforms in Ukraine: Progress and Priorities.” Paper presented at the Ukraine Reform Conference, Toronto, July 2–4. USAID (United States Agency for International Development). 2018. “Key Issues in Pension System Reform in Ukraine.” USAID Financial Sector Transformation Project, Kyiv. Independent Evaluation Group World Bank Group    51 WHO (World Health Organization). 2019. Ukraine Review of Health Financing Reforms 2016–2019. WHO–World Bank Joint Report. Copenhagen: WHO Regional Office for Europe. WHO (World Health Organization). 2020. “Budgetary Space for Health in Ukraine: Policy Document to Support Budget Preparation Dialogue for 2021.” Health Pol- icy Paper Series No. 20/01, WHO Regional Office for Europe, Copenhagen. World Bank. 2005. “Ukraine—Social Assistance System Modernization Project.” Proj- ect Appraisal Document 32719-UA, World Bank, Washington, DC. World Bank. 2007. Ukraine—Country Partnership Strategy for the Period FY08–FY11. Washington, DC: World Bank. World Bank. 2008. Ukraine—Country Assistance Evaluation. Washington, DC: World Bank. World Bank. 2009. “Combating Ukraine’s Health Crisis: Lessons from Europe.” Knowledge Brief, Volume 17, World Bank, Washington, DC. World Bank. 2010. Ukraine: Strategic Choices to Accelerate and Sustain Growth. Coun- try Economic Memorandum. Washington, DC: World Bank. World Bank. 2012a. “Ukraine—Completion and Learning Review: IEG Review FY08– 11.” Independent Evaluation Group, World Bank, Washington, DC. World Bank. 2012b. Ukraine—Country Partnership Strategy for the Period FY12–FY16. Washington, DC: World Bank. World Bank. 2012c. Ukraine: Public Financial Management Performance Report. Washington, DC: World Bank. World Bank. 2013a. Ukraine—Demand for Good Governance in the Water Sector. Washington, DC: World Bank. World Bank. 2013b. “Ukraine—Social Assistance System Modernization Project.” Im- plementation Completion and Results Report ICR2672, Independent Evaluation Group, World Bank, Washington, DC. World Bank. 2014a. “Ukraine—First Development Policy Loan.” Program Document The World Bank Group in Ukraine, 2012–20  Bibliography 86951-US, World Bank, Washington, DC. World Bank. 2014b. “Ukraine Social Safety Nets Modernization Project.” Project Ap- praisal Document 84672-UA, World Bank, Washington, DC. World Bank. 2014c. “Ukraine Special Focus: A Roadmap for Urgent Macroeconomic and Structural Reforms.” Europe and Central Asia Region, World Bank, Washington, DC. World Bank. 2015a. “Road Sector Quality Assurance, Management and Development: Key Policy Options.” World Bank, Washington, DC. World Bank. 2015b. “Ukraine—Second Development Policy Loan.” Program Docu- ment 95279-UA, World Bank, Washington, DC. 52 World Bank. 2015c. “Ukraine—Serving People, Improving Health Project.” Project Appraisal Document PAD939, World Bank, Washington, DC. World Bank. 2016a. Energy Efficiency Transformation in Cities: Policy Note on Munici- pal Energy Efficiency Financing. Washington, DC: World Bank. World Bank. 2016b. Sustainable Urban Transport for Kyiv: Towards a Sustainable and Competitive City Built upon the Legacy System and Innovations. Washington, DC: World Bank. World Bank. 2016c. “Ukraine Health Policy Dialogue—Summary of Activities and Results Achieved.” World Bank, Washington, DC. World Bank. 2017a. Irrigation and Drainage Strategy of Ukraine. Washington, DC: World Bank Group. World Bank. 2017b. “Ukraine—Completion and Learning Review for Country Part- nership Framework for the Period FY12–FY16: IEG Review.” Independent Evalu- ation Group, World Bank, Washington, DC. World Bank. 2017c. Ukraine—Country Partnership Framework for the Period FY17– FY21. Washington, DC: World Bank. World Bank. 2017d. “Ukraine: Development of the State Statistics System for Mon- itoring the Social and Economic Transformation (DEVSTAT).” Project Perfor- mance Assessment Report 113588, Independent Evaluation Group, World Bank, Independent Evaluation Group World Bank Group    53 Washington, DC. World Bank. 2017e. “Ukraine—Energy Efficiency Project (P129704).” Implementation Completion and Results Report ICR4191, World Bank, Washington, DC. World Bank. 2017f. “Ukraine—Programmatic Development Policy Loan Series.” Implementation Completion and Results Report ICR4070, World Bank, Washington, DC. World Bank. 2017g. Ukraine—Public Finance Review. Washington, DC: World Bank. World Bank. 2017h. Ukraine—Toward Sustainable Recovery and Shared Prosperity. Systematic Country Diagnostic. Washington, DC: World Bank. World Bank. 2018a. Strategy for Prioritization of Investments, Funding and Moderniza- tion of Ukraine’s Road Sector. Washington, DC: World Bank. World Bank. 2018b. “Ukraine—Economic Growth and Fiscally Sustainable Services Policy Based Guarantee.” Program Document 132120-UA, World Bank Group, Washington, DC. World Bank. 2018c. “Ukraine—Energy Efficiency Project (P129704).” Implementation Completion and Results Report Review ICRR0020956, Independent Evaluation Group, World Bank, Washington, DC. World Bank. 2018d. Ukraine: Public Finance Review 2—Towards a More Effective and Fiscally Sustainable Public Sector for Ukraine. Washington, DC: World Bank. World Bank. 2019a. “Challenges and Options for Reforming the Social Safety Net System in Ukraine: A Reform Proposal.” World Bank, Washington, DC. World Bank. 2019b. “Ukraine—First and Second Programmatic Financial Sector Development Policy Loan.” Project Performance Assessment Report 144601, Independent Evaluation Group, World Bank, Washington, DC. World Bank. 2019c. Ukraine—Public Expenditure and Financial Accountability (PEFA) Performance Assessment Report. Washington, DC: World Bank. World Bank. 2019d. “Ukraine: Serving People, Improving Health Project: Project Restructuring.” Restructuring Paper RES31959, World Bank, Washington, DC. World Bank. 2020a. Building Effective, Accountable, and Inclusive Institutions in Europe The World Bank Group in Ukraine, 2012–20  Bibliography and Central Asia: Lessons from the Region. Washington, DC: World Bank. World Bank. 2020b. Enhancing Government Effectiveness and Transparency: The Fight Against Corruption. Washington, DC: World Bank. World Bank. 2020c. “Ukraine—First Economic Recovery Development Policy Loan.” Program Document 148886-UA, World Bank, Washington, DC. World Bank. 2021. Addressing Country-Level Fiscal and Financial Sector Vulnerabilities: An Evaluation of the World Bank Group’s Contributions. Independent Evaluation Group. Washington, DC: World Bank. World Bank Group. 2021. Creating Markets in Ukraine—Doubling Down on Reform: Building Ukraine’s New Economy. Country Private Sector Diagnostic. Washington, DC: World Bank Group. https://www.ifc.org/wps/wcm/connect/fbb356dd-8363- 42c6-9d72-f609c4f9b9c0/CPSD-Ukraine.pdf?MOD=AJPERES&CVID=ns.UNH8. 54 APPENDIXES Independent Evaluation Group The World Bank Group in Ukraine, 2012–20 Appendix A. World Bank Group Support in Other Areas Agribusiness Agribusiness is a major contributor to the Ukrainian economy, generating more than 9 percent of gross domestic product and about 40 percent of total exports earnings (2019) and employing 15 percent of the active population. Ukraine has 41.5 million hectares of agricultural land (70 percent of the territory) and 30 percent of the world’s extremely fertile black soil. Ukraine is the third-largest agricultural exporter to the European Union (EU). Gross output in the sector increased by about a quarter during 2012–19, but it was heavily concentrated in large holdings, while production in smallhold- ers stagnated. This gap between potential and actual production levels has been attributed to the land sale moratorium (since 1992) that has hampered access to finance and discouraged investment. World Bank Group–Supported Program and Contributions The World Bank Group in Ukraine, 2012–20  Appendix A The World Bank Group contributed to the improvement of export oppor- tunities for larger agricultural holdings and had a much smaller impact on smallholder farms or in enhancing productivity in agricultural small and me- dium enterprises (SMEs). Bank Group support for the development of agri- business, including improvement in investment climate and deregulation, was the centerpiece of its program of support for private sector development in Ukraine. The program consisted mainly of International Finance Corpora- tion (IFC) investments ($750 million) and advisory services (30 projects) in support of innovation and modernization, higher productivity, better corpo- rate governance, access to finance for farmers, opening of new markets, and cleaner production methods. IFC projects covered the entire value chain to boost innovation and productivity and help producers enter new markets. IFC advisory services supported access to finance for farmers, the opening of new markets, modernization of the supply chains, improved logistics, better 58 corporate governance, and cleaner production methods. The World Bank supported regulatory reform aimed at agribusinesses but refrained from investing in agriculture while the land moratorium was in place. Regulatory reform: The deregulation reform program, launched in 2014 with the help of the Bank Group and international partners, resulted in a reduc- tion of the regulatory burden on the private sector, including agribusinesses. The regulatory improvements included lower cost of business entry, reduced cost of tax compliance, streamlined asset (including land) registration, and improved SME access to public procurement. Bank Group support for regulatory reform was relevant and timely, benefiting from extensive diagnostic and analytical work. The 2014 joint World Bank–IFC study (IFC 2014) identified excessive red tape, inadequate implementation of business regulations, and weak public sector governance as major obstacles for private sector development. It recommended a set of specific measures such as wholesale reduction of the number of permits and licenses, regulatory impact assessment of new legislation, risk-based system of inspections, adoption of EU regulations (food safety), and introduction of e-government and information and communication technology solutions. Reform measures supported by development policy loan prior actions (promoting a legal framework for certification, accreditation, and conformation to international standards; land reform and registry) were backed by technical assistance that focused on setting up a deregulation Independent Evaluation Group World Bank Group    59 framework, easing licensing and permit requirements, harmonizing safety standards, simplifying export procedures, reducing the cost of trade logistics, and leveling competition. IFC contributions included assistance with simplifying grain-quality certification, implementing phytosanitary testing, registering export contracts, and operationalizing modern food safety controls consistent with EU and global standards to promote Ukrainian exports (box A.1).  kraine’s Integration into the European Union Market and Box A.1. U World Bank Group Support The 2014 deregulation program, launched to achieve compliance and convergence with European Union (EU) standards and global best practices, positively impacted the business environment in key sectors, including agriculture. The World Bank Group supported this process through prior actions in two development policy loans (establishing a legal framework for a system of certification, accreditation, and confirmation of correspondence to international standards; creating a single national agency for accreditation; enacting legislation on copyright; conducting a public awareness campaign for Ukraine’s EU accession; enacting a law on “the key principles of public control of entrepreneurial activity” [Inspections Law], easing business registration and property registration; reducing the number of permits; establishing a deregulation framework; easing licensing and permit requirements; and harmonizing food safety standards and procedures, technical regulations, and conformity assessments with EU requirements). Previous analytical work (for example, IFC 2014) was the basis for selecting development policy loan prior actions on private sector development. Other Bank Group support included Bank Group and International Finance Corporation technical assistance on streamlining regulations and improving food safety standards to comply with EU requirements. Source: Independent Evaluation Group. The World Bank Group in Ukraine, 2012–20  Appendix A Agribusiness and land reform: IFC support made a substantial contribution to enhancing the sustainability and growth prospects of agribusiness in Ukraine through investments and advisory services, although the productivity gap between Ukraine and other European countries remains wide. Overall, farmers’ access to inputs, services, and financing improved, leading to an expansion (by almost 25 percent) in agricultural exports—from less than $18 billion in 2012 to more than $22 billion in 2019.1 Upgrading of production techniques (including climate-smart methods) resulted in yield increases of up to 25 percent for selected crops in more than 4,000 SME farms (IFC 2019). The application of new financing arrangements made lending more affordable and generated an additional $356 million in 60 credit for farmers. IFC’s advisory services helped provide better access to quality inputs, extension services, and agricultural finance by introducing best international practices, such as preharvest financing with crop receipts, insurance products for strategic crops, and agriculture-specific credit-risk assessment tools for banks that resulted in improved quality (reduction in nonperforming loans by 50 percent) and expansion of the agricultural credit portfolio. The Crop Receipts Project (2013–20) helped establish crop receipts as a tradeable financial instrument, enhancing access to finance for SME farmers. According to IFC, the Crop Receipts Project helped more than 2,000 small farmers gain access to financing (IFC 2020b). IFC also helped remove export impediments in several subsectors (poultry and dairy) and supported larger integrators to become international operators. For example, the Agri Aggregator Project (2013–19) improved the marketing of inputs, farmers’ capacity for on-farm management, access to high-quality extension services, and financing terms for the purchase of inputs and services. The project worked with small- to medium-scale farmers and leveraged the IFC and Bayer Crop Science financing program, via Raiffeisen Bank Aval and Crédit Agricole, to offer affordable short-term financing solutions. The Dairy Support Project (2018–20) improved the operational efficiency and productivity of dairy producers and helped them access export markets. The Country Private Sector Diagnostic, jointly prepared by the World Bank and IFC (IFC 2020a), further examined sectors where Bank Group interventions could unleash private sector potential in Ukraine, including climate-smart agriculture technologies. Independent Evaluation Group World Bank Group    61 The World Bank supported land reform through the 2014–15 development policy loans and advisory services and analytics (ASA). Development policy loan prior actions aimed at transforming the ownership/organizational structure in agriculture and included the establishment of a unified system of registration for legal rights to land and real estate within one agency, the review of land-title issuances, cumulative issuance of 3.4 million land titles, the implementation of the new Land Code, and the initiation of an information campaign to inform farmers about their rights and opportunities as land holders. After the government committed (in 2019) to gradually open the land market, the World Bank approved a $200 million Program- for-Results operation (2020–25) to support land reform and increase agricultural sector competitiveness and diversification; it was supported by ongoing (EU-funded) ASA on land governance, smallholder development, geographic information system monitoring, and registry unification. The gradual opening of the land market will let farmers use land as collateral and could facilitate private investment in agribusiness. The World Bank also contributed (through ASA and technical assistance) to streamlining land registration in an effort to reduce corrupt practices, strengthen protection for property rights, and improve the monitoring of land use. Health With the COVID-19 pandemic strongly affecting Ukraine (with over 2.4 mil- lion confirmed cases and about 56,000 deaths as of October 2021),2 improv- ing the efficiency of the health sector has become ever more important. At the beginning of the evaluation period, government health spending was allocated based on inputs rather than services provided, distorting incentives and contributing to a low quality of care. Most health resources were spent on inpatient and specialized outpatient care, and a relatively small amount was spent on primary and preventive care. Provider payments and pharma- ceutical procurement were often captured by corrupt actors who diverted public resources for private gain. World Bank Group Program and Contributions Bank Group diagnostics guided health sector reforms launched in 2017. The The World Bank Group in Ukraine, 2012–20  Appendix A investment project (“Serving People, Improving Health,” approved in 2015, with additional financing approved in 2020) and technical assistance helped strengthen primary care at the oblast level and advanced the restructuring of provider payment mechanisms, laying the groundwork for hospital restruc- turing (World Bank 2015). Health reforms (supported by investment projects and a policy-based guarantee prior action) changed the formulas for paying providers, creating incentives for more efficient spending and promoting fiscally sustainable and effective services. The average length of a hospital stay declined from 11.7 days in 2013 to 10.0 days at the end of 2020, and the number of hospital beds per 100,000 members of the population was reduced from 880 in 2013 to 600 in May 2021. The share of health spending on primary health care increased from 9 percent of total public expenditures on health in 2015 to 14.6 percent in 2019 (although it remained below the 62 Country Partnership Strategy target of 24 percent). Ukraine’s response to COVID-19 built on the health sector reforms made during the evaluation period with the help of the World Bank. Bank Group support was instrumental in enhancing the efficiency of health services and supporting vital clinical services. The first phase of the reform centered on primary care (and the next phase—hospital optimization) has been politically controversial. It formally took effect in April 2020, but its implementation has been delayed by the COVID-19 pandemic. Key institutions in the health sector have been strengthened, and the chang- es have been sustained even in the face of the pandemic and ongoing po- litical opposition. Measures have been taken to remake the system’s rules to constrain backsliding and contain the fiscal burden. The World Bank– supported health project helped set up National Health Service of Ukraine business processes, including a hotline for questions or complaints about the new health financing modality. The National Health Service of Ukraine hotline was used as the model for a successful COVID-19 hotline that was established early in the pandemic (also with the World Bank’s assistance). Independent Evaluation Group interviews with sector stakeholders indicated a broad consensus that the health sector reforms could not have proceeded as rapidly and effectively, if at all, in the absence of the World Bank’s analyti- cal work and policy dialogue. Conflict in Eastern Ukraine Independent Evaluation Group World Bank Group    63 The unresolved conflict in Eastern Ukraine severely complicated Ukraine’s development challenges.3 By 2020, over 14,000 people had been killed and 2.7 million were displaced, including over 1.3 million internally displaced. The conflict has severely impacted socioeconomic activity in Ukraine’s east- ern industrial heartland of Donetsk and Luhansk oblasts, which accounted for 12.5 percent of Ukraine’s population, 15.7 percent of the country’s gross domestic product, and a quarter of its exports (World Bank 2020). In addition to major social, economic, and political stress, the conflict generated addi- tional budget pressures and raised credit and investment risks. Since 2014, the security situation in Eastern Ukraine remained volatile, with daily hostil- ities occurring along the contact line. The World Bank partnered with the Ministry for Reintegration of the Temporarily Occupied Territories to provide a range of support. ASA has been the primary instrument used to help the government mitigate the impact of the conflict, including an assessment of peacebuilding and recovery needs, a proposal on capacity building within the government to coordinate and implement recovery interventions, and the design of pilot programs. Several nonlending interventions were funded under a World Bank–administered Multi-Partner Trust Fund on Peacebuilding and Recovery, established in 2017. Bank Group recommendations contributed to the establishment of the national strategy on recovery in the conflict-affected areas, informed international partner programs for Eastern Ukraine, and helped enhance Ministry for Reintegration capacity. The World Bank also supported the setup and operation of the Online Portal on Peacebuilding and Recovery, which has been used as a community engagement and monitoring tool to help build citizen trust and confidence in the government reintegration strategy. The World Bank team undertook a study on the economic recovery of the Donbas, which assessed how the region could move from its industrial past to discover new drivers of growth and development. As a result of extensive diagnostic work and dialogue, in November 2020 the World Bank approved a $100 million investment project (“Eastern Ukraine: Reconnect, Recover, Revitalize”) that supports the government’s The World Bank Group in Ukraine, 2012–20  Appendix A efforts to promote the recovery, reintegration, and inclusion of the conflict-affected population. References IFC (International Finance Corporation). 2014. Ukraine: Opportunities and Challenges for Private Sector Development. Washington, DC: World Bank. IFC (International Finance Corporation). 2019. Ukraine Agri Aggregator Advisory Services Completion Report. Washington, DC: IFC. https://disclosures.ifc.org/ project-detail/AS/592348/ukraine-agri-aggregator. IFC (International Finance Corporation). 2020a. Creating Markets in Ukraine: Country Private Sector Diagnostic. Washington, DC: IFC. 64 IFC (International Finance Corporation). 2020b. “IFC Helps Small Farmers in Ukraine Get Access to $1 billion of Working Capital Financing.” Press Release, April 23, 2020. https://pressroom.ifc.org/all/pages/PressDetail.aspx?ID=18558. World Bank. 2015. “Ukraine—Serving People, Improving Health Project.” Project Appraisal Document PAD939, World Bank, Washington, DC. World Bank. 2020. “Eastern Ukraine: Reconnect, Recover, Revitalize (3R) Project.” Washington, DC: World Bank. Independent Evaluation Group World Bank Group    65 1  Reflecting significant losses in manufacturing exports, related to the conflict in Eastern Ukraine, the share of agricultural exports in the country’s total goods exports expanded from 26 percent to 44 percent in 2012–19. See https://www.trade.gov/knowledge-product/ ukraine-agricultural-sector. 2  See https://covid19.who.int/region/euro/country/ua. 3  This report was finalized in 2021, prior to military aggression against Ukraine begun in February 2022, and does not reflect the consequent impact on the country. The World Bank Group in Ukraine, 2012–20  Appendix A 66 Appendix B. Methodology Country Program Evaluations assess World Bank Group performance and the contribution of Bank Group assistance programs to development outcomes. The Ukraine Country Program Evaluation used a mixed methods approach including desk reviews, analyses of macro- and socioeconomic indicators and portfolios, semistructured stakeholder interviews, two per- ception surveys, and an analysis of online news media (appendix C). Country Program Evaluations normally include intensive fieldwork, but in the cir- cumstances of the COVID-19 pandemic and related restrictions, the Inde- pendent Evaluation Group replaced a country visit with virtual interviews with key stakeholders, including Bank Group staff, government officials, de- velopment partners, civil society, and the private sector from August 2020 to January 2021. The Independent Evaluation Group undertook two surveys and an online media analysis to gauge the in-country views (general public and expert or informed stakeholder levels) and perceptions of the Bank Group, its support to Ukraine, and the key reforms it supported. Public Opinion Survey The survey was conducted by the Kyiv International Institute of Sociology in November and December 2020, based on a questionnaire developed by the Independent Evaluation Group World Bank Group    67 Independent Evaluation Group. The survey used the computer-assisted tele- phone interview method based on a random sample of mobile phone numbers. The survey was conducted as a part of a regular omnibus survey. The sample is representative of the adult population (18 years and older) of Ukraine. Informed Stakeholder (Expert) Online Survey The survey was conducted in December 2020 to obtain the views of key stakeholders in government, donor agencies, private sector, civil society organizations, academia, and media, who were either directly involved in or observed the implementation of Bank Group activities in Ukraine. Data were collected through an online survey using the SurveyMonkey tool. Respon- dents were recruited via personal email communication, based on the list used by the Bank Group Country Opinion Surveys (281 individuals). The sur- vey included multiple-choice and open-ended questions. The total response rate was 22 percent, with almost half of the respondents representing gov- ernment institutions, 14 percent media, 19 percent donor agencies, 22 per- cent civil society organizations, and 8 percent private sector. Analysis of Online News Media The original data set of relevant articles covered January 2016 to November 2020, mostly in the Ukrainian and Russian languages (figure B.1). Figure B.1. Analytical Pipeline for Online News Media Analysis The World Bank Group in Ukraine, 2012–20  Appendix B Source: Independent Evaluation Group. Note: N = number. In the data set of translated articles, mentions of the Bank Group and topics of interest (see table C.1 for a list of these topics) were identified using string searches. Time-series plots of these data sets presented in figure B.2 show that there was an increasing trend in both. 68 Figure B.2. Trends in News Article Data Sets Source: Independent Evaluation Group. Note: The break in 2020 is likely due to the COVID-19 pandemic causing issues in the source data. Independent Evaluation Group World Bank Group    69 Appendix C. Results of Surveys of Public Perceptions of Service Delivery and Online Media Analysis Public Opinion Survey The survey aimed to gauge the perceptions of progress in improving public service delivery, one of the key areas of World Bank Group support. Only a third of the respondents were familiar with Bank Group activities in Ukraine, with mostly neutral views. The COVID-19 response led the known areas of Bank Group engagement (31 percent), followed by anticorruption (27 per- cent) and health-care reforms (23 percent). In terms of the quality of service delivery in areas of active Bank Group involvement, such as district heat- ing and natural gas supply, many respondents indicated either no chang- es (42 percent for district heating and 29 percent for gas) or worsening of service (28 percent for district heating and 36 percent for gas) after the tariff and subsidy reforms. On health services, 56 percent considered them to have worsened over time, whereas only 17 percent thought that health services had improved. On pension reform, over 50 percent of working respondents were dissatisfied with pension benefits, considering them to be insufficient The World Bank Group in Ukraine, 2012–20  Appendix C for retirement. On social protection reforms, such as the household utility subsidies and guaranteed minimum income programs, only 15 percent of respondents agreed that benefits had been distributed more fairly, with close to 40 percent holding the opposite view. Informed Stakeholder (Expert) Online Survey The survey aimed to gauge stakeholder views on Bank Group engagement relevant to the country’s main development challenges. Over 80 percent of respondents partially or fully agreed that Bank Group programs were rele- vant to addressing Ukraine’s critical development challenges, broadly ef- fective, and adapted appropriately to changes in the political and economic situation. Most respondents (72 percent) thought that the Bank Group col- laborated effectively with other development partners (the European Union, 70 the International Monetary Fund, and the European Bank for Reconstruction and Development). A slight majority of respondents had a favorable impres- sion of the Bank Group’s ability to identify, build, and use local capacity and expertise in program design and implementation (56 percent). The majority of respondents felt that Bank Group assistance was most effective in the fi- nancial sector (51 percent), health and social protection (40 percent), and re- sponse to macroeconomic and financial crises (38 percent). The respondents mentioned a broad spectrum of areas needing additional Bank Group sup- port, including anticorruption, education, health care, energy, the financial sector, climate change, small and medium enterprise finance, land reform, judicial system, infrastructure, statistics, and innovations and information technology. The respondents also provided suggestions on general directions of support, the strengthening of partnerships (especially with local nongov- ernment counterparts), the importance of long-term institutional support, and a better understanding of local context, culture, and political economy. Analysis of Online News Media The Independent Evaluation Group conducted an analysis of Ukrainian online news reports to gauge the public perception of reforms supported by the Bank Group as viewed by the general public and covered in local media. Keywords extracted from the text in the context of the Bank Group (after re- moving stopwords, lemmatization, and part-of-speech tagging to only retain nouns) revealed that among the most frequent were “land,” “corruption,” Independent Evaluation Group World Bank Group    71 and “energy,” in that order. To further understand the topical composition of Bank Group mentions, a search taxonomy was developed to calculate the frequencies of the words and phrases (table C.1). Sentiment analysis of the references to the Bank Group and the above topics was conducted using two pretrained sentiment-classification algorithms. A particular text extract (that is, a sentence quintet) was classified as having a positive or negative sentiment if both models agreed; otherwise, it was clas- sified as being neutral. Overall, the sentiment in the Bank Group mentions was found to be neutral, with an average 16 percent of the mentions being negative, 12 percent positive, and the rest neutral. These proportions did not vary based on the type of news source. However, substantial variations were found in the sentiment scores for Bank Group mentions based on the region of the news articles. Map C.1, panel a, shows the proportion of Bank Group mentions with negative sentiment, and map C.1, panel b, shows the propor- tion of Bank Group references with positive sentiment. There was a cluster- ing of positive sentiment in the western regions and a clustering of negative sentiment in the eastern regions.  requency of Keywords and Phrases (Topics) in World Bank Table C.1. F Group Context (number) Keyword or Phrase Sentence Quintets land 27,402 corruption 15,321 energy 13,746 health 13,015 pension 10,401 crisis 5,007 Donbas 4,165 tariff 3,256 electricity 3,032 social protection 1,515 Total 96,860 Source: Independent Evaluation Group.  roportion of Negative and Positive Sentiment World Bank Map C.1. P The World Bank Group in Ukraine, 2012–20  Appendix C Mentions by Region Source: Independent Evaluation Group. Note: Shades are based on quartiles, with darker shades representing higher values and lighter ones representing lower values. Map C.2 shows the regional distribution of sentiment based on specific 72 topics in Bank Group context. For example, the negative sentiment regarding corruption-related issues is concentrated in the central regions, whereas negative sentiment regarding land reforms is more clustered in the eastern regions.  istribution of Sentiment Scores for Selected Topics by Regions Map C.2. D in World Bank Context Independent Evaluation Group World Bank Group    73 Source: Independent Evaluation Group. Note: Shades are based on quartiles, with darker shades representing higher values and lighter ones representing lower values. Appendix D. World Bank Group Support to Address Fiscal and Financial Sector Vulnerabilities in Ukraine Key Messages This appendix assesses the extent to which the World Bank Group helped the government of Ukraine identify and reduce fiscal and financial sector vulner- abilities in the decade after the global financial crisis.1 The main findings are the following. » Identification of vulnerabilities: Throughout the evaluation period, the Bank Group had a clear and accurate understanding of fiscal and financial sector vulnerabilities and a generally well-prioritized sense of what needed to be addressed. This understanding was well reflected in major country diagnos- tics such as Country Economic Memorandums (CEMs), country strategies, operations, and advisory services and analytics. » Effectiveness and impact of Bank Group contributions: The fiscal position, The World Bank Group in Ukraine, 2012–20  Appendix D excise tax (especially tobacco) revenues, and debt position of the government improved significantly after 2014. The pension system was reformed toward long-term sustainability. With Bank Group support, the government closed many illiquid and insolvent banks, significantly reducing related fiscal risks and strengthening supervision and regulation, bank resolution frameworks, and the Deposit Guarantee Fund (DGF). Progress was most significant in the first two years after the new government came to power in 2014. However, af- ter 2016, vested interests effectively delayed or blocked the implementation of many agreed reforms. » The government’s fiscal and financial sector stabilization efforts, to which the Bank Group contributed, were largely effective, but crisis preparedness– related institutional reforms were less successful in the face of strong vested 74 interests. » Although progress was made in enhancing the oversight and regulation of private banks, efforts fell short in addressing the oversight of the state- owned commercial banks (SOCBs) that dominate Ukraine’s banking sector. Although the Bank Group supported the introduction of critical legislation into Parliament, vested interests largely succeeded in holding up the enact- ment of key reforms until 2021. The effectiveness of the newly established independent supervisory boards for SOCBs was being undermined by vested interests through the imposition of unduly onerous asset-declaration re- quirements for foreign nationals. Moreover, the absence of meaningful judi- cial reform undermined efforts to reduce the large stock of nonperforming loans (NPLs) burdening the banking sector. » With hindsight, what could have been done better? The strength of the obstacles presented by vested interests was obvious in the lead-up to and throughout the evaluation period. A clearer initial sense of the results chain leading to effective enactment and implementation of key legislation might have highlighted potential points of stress and risks to the achievement of results earlier in the process. This consideration would have pointed to the need for more deliberate attention to judicial and parliamentary constraints and a need to build demand for better governance and commitment to follow up in financial sector operations. Even though other development partners “led” on judicial reform, given the importance of the court system to build- ing effective financial sector crisis preparedness, the Bank Group might have Independent Evaluation Group World Bank Group    75 sought to play a bigger role in this area, beyond its support for the creation of the National Agency on Corruption Prevention in 2015. Country Context and Bank Group Strategy The global financial crisis hit Ukraine hard, with real gross domestic product (GDP) dropping almost 15 percent in 2009. The government was able to recapitalize banks and stabilize the system; deposit outflow was reversed, and the DGF was created. However, rising public debt and NPLs reflected continued fiscal and financial sector weaknesses. Government institutions continued to face mistrust because of widespread corruption and state capture by vested interests. During 2010–14, attempts at fiscal consolidation were insufficient, and public debt continued to increase. Inadequate bank supervision and regula- tion, connected party lending, and the failure of many banks contributed to a general loss of confidence and a massive currency, fiscal, and social crisis. The Bank Group’s 2012–16 Country Partnership Strategy (CPS) was set in re- lation to two strategic priorities: (i) strengthening the relationship between citizens and the state, including through improving public services and pub- lic finances, and (ii) strengthening the relationship between businesses and the state by improving policy effectiveness and economic competitiveness (World Bank 2012). The 2012–16 strategy continued the focus on pension reform, recognizing that Ukraine’s public finances could not be set on a sus- tainable footing without reforming a pension system characterized by one of the world’s highest public expenditures as a share of GDP (17.5 percent of GDP in 2012, compared with an 8 percent average for the Europe and Central Asia Region and 7.5 percent average for Organisation for Economic Co-oper- ation and Development member countries). Given the country’s track record of slow reform, lack of government owner- ship, and strong vested interests, the baseline scenario in the CPS envisaged limited operations. New lending was limited to infrastructure projects and one social assistance project. The CPS anticipated potential increases in lending primarily in the area of banking sector rehabilitation, if and when The World Bank Group in Ukraine, 2012–20  Appendix D the government credibly intended to accelerate policy reform. In February 2014, amid the economic and security crisis, the exchange rate depreciated sharply, triggering major losses in banks’ highly dollarized balance sheets and a full-blown financial panic. The banking system lost almost one-third of hryvnia deposits and more than half of foreign exchange–denominated retail deposits between the end of December 2013 and the end of June 2015. This exacerbated preexisting problems related to the large stock of NPLs (particularly in SOCBs). More than one-quarter of the 181 banks operating at the end of 2013 failed within 18 months and were transferred to the DGF for resolution. Failure to reduce NPLs, combined with weak creditor rights and corruption in the judiciary, continues to constrain lending to the private sector. 76 Identifying Fiscal and Financial Sector Vulnerabilities Identifying Fiscal Vulnerabilities The 2010 CEM correctly recognized that the external environment had changed dramatically and that the major shocks mandated adjustment. It argued that “the current fiscal model ha[d] proven unsustainable and fiscal reform [became] the most urgent priority” (World Bank 2010, 63). The World Bank warned that fiscal policies were exacerbating broader macroeconomic risks and that they could have dire macroeconomic consequences and result in a loss of confidence. The World Bank recommended that the government undertake strong corrective fiscal measures to create fiscal space for produc- tive public expenditures, including public investment, which could support growth and job creation. At the same time, the World Bank recommended reforming the pension system and energy tariffs and resolving the problem of large value-added tax refund arrears (box D.1).  iscal Reforms Identified in the 2010 Country Economic Box D.1. F Memorandum Infrastructure investments: Increase energy and infrastructure investment by 1 per- Independent Evaluation Group World Bank Group    77 cent of gross domestic product in 2010 and 2012 (focus on transport and energy). Pension reform: Cap pension payments for working pensioners; freeze special and privileged pension benefits transferred from the budget; gradually increase retirement age for women from 55 years to 60 years; lengthen years of service required for new pensioners to receive a minimum pension from 20 to 30 for women and 25 to 35 for men; introduce a stable inflation indexation rule for pensions. Utility tariffs and targeting social assistance to the most vulnerable households: Increase threshold for extreme low income in the guaranteed minimum income from 50 percent of the subsistence minimum; increase eligibility threshold in the housing and utility subsidies program to 30 percent; introduce eligibility threshold for housing and utility privileges at two or three subsistence minimums; increase gas tariffs for households, heating enterprises (the tariffs of these two consumers should converge), (continued)  iscal Reforms Identified in the 2010 Country Economic Box D.1. F Memorandum (cont.) and budget institutions; change thresholds of gas tariff differentiation for households to better target subsidy; increase gas tariffs for industry and increase heating tariffs for households; increase electricity tariffs for residential and nonresidential consumers during 2010–12. Tax policy and administration reform: Implement a program to deal with value-added tax refund arrears; eliminate value-added tax privilege for agriculture and value-added tax exemptions for medicines, publishing houses, and printed materials and excise exemption on bio components of fuel. Public financial management reform: Implement legislation on public procurement in line with good international practice and harmonized with European Union direc- tives; strengthen regulatory framework for project evaluation; corporatize large (natural monopoly) state-owned enterprises to enable them to attract private capital and joint ventures; improve financial reporting of state-owned enterprises. Source: World Bank 2010. Identifying Financial Sector Vulnerabilities The World Bank Group in Ukraine, 2012–20  Appendix D The Bank Group had a good grasp of financial sector vulnerabilities, reflect- ed in a wide range of Bank Group analyses including the 2010 CEM, which noted that “mounting vulnerabilities in the banking sector, generated by lax credit analysis in the context of fast credit growth fueled by external borrow- ing, were accentuated by the [global financial] crisis. [Before the 2008 crisis], regulation and supervision were unable to catch up with the growth of the sector, and thus currency and maturity risks increased, coupled with severe under-provisioning for potential problem loans” (World Bank 2010, 66). The CEM argued that there was a need for “renewed effort to tackle the legacies of the past and strengthen regulation and transparency to regain trust” and noted that “supervision and regulation [were] not ready to prevent another crisis” (World Bank 2010, 67). 78 The 2010 CEM contained a comprehensive list of banking sector reform pri- orities, which demonstrates a good awareness of policy priorities, including those associated with risks to the integrity of the banking sector. The CEM referred to the nonbank financial sector in an annex, noting only its slow growth relative to that of the banking sector (box D.2).  anking Sector Measures Recommended by the World Bank Box D.2. B in 2010 Regulation and supervision: Enact amendments to legislative framework to facilitate restructuring of nonperforming loans; enact amendments to legislative framework to facilitate bank mergers and reorganization; enact a legislative framework to enforce disclosure of ultimate ownership of banks and nonbank financial institutions that would significantly enhance the definition of beneficiary ownership and include related legal entities, family, and other associated persons at all levels; ensure that capitalization plans in all banks are fulfilled; legislate and enforce consolidated supervision of finan- cial conglomerates; enact legislative and regulatory framework to introduce updated Basel regulations; increase capital requirements to open a bank and strengthen stan- dards and enforcement of “fit and proper” criteria; revise capital requirement and provi- sioning rules to ensure solvency and avoid excessively procyclical banking lending. Bank resolution and restructuring of state-owned commercial banks: Further im- prove the set of resolution tools available to the National Bank of Ukraine; complete Independent Evaluation Group World Bank Group    79 diligence in all banks recapitalized with public funding and improve corporate gover- nance arrangements; transfer bank resolution powers to the Deposit Guarantee Fund; divest nationalized banks that are commercially viable and can be sold at a fair value. Transparency and governance: Enact legislation to make individual bank data transpar- ent and publicly available in line with predominant international practice; avoid conflict of interest in appointment of members of the Council of the National Bank of Ukraine. Source: World Bank 2010. The CPS for fiscal years 2012–16 (World Bank 2012) noted that NPLs in the state-dominated banking system remained high and commercial banks taken over by the state remained unresolved. Banking sector vulnerabilities were exacerbated by volatility in international financial markets and deleveraging by European banks (which held about one-third of Ukraine’s banking system assets). In light of these risks, the CPS argued that safeguarding the sound- ness of the poorly regulated and supervised banking sector and rationalizing the state’s role and ownership in the financial system were the most pressing financial sector challenges for the medium term. The CPS contained a frank assessment of progress in reducing corruption in Ukraine. The main text of the CPS was relatively understated, but annex 4 was more direct, citing the results of a survey of public perceptions of cor- ruption, in which 92.2 percent of Ukrainians believed that corruption among government officials was a key problem in the country and only 17 percent believed in the government’s commitment to fight corruption (World Bank 2012). The appendix noted that the new anticorruption law would accom- plish very little unless it was properly implemented and accompanied by other anticorruption legislation. Although the financial sector was not cited explicitly, these observations are clearly relevant to it. External audits undertaken in early 2010 revealed that 61 banks (represent- ing 59 percent of banking sector assets) needed capital amounting to about 4 percent of GDP (IMF 2014). The financial sector was saddled with a large stock of NPLs, heavily concentrated in SOCBs, which had been engaging in extensive connected lending. By the end of 2010, 18 banks were under liqui- The World Bank Group in Ukraine, 2012–20  Appendix D dation and another 4 were under temporary administration. An incomplete and ineffective restructuring process and weaknesses in the legal, tax, and judicial systems were obstacles to dealing with NPL resolution. Legislation governing disclosure of ultimate controllers of banks and amend- ing the central bank law to strengthen its independence had been enacted before 2010, but implementation and enforcement were weak. The only tool available to the National Bank of Ukraine and DGF to resolve problem banks was liquidation. However, resolution efforts were slow and, in some cases, the National Bank of Ukraine did not exercise proper control of the liqui- dation process, resulting in large losses to the DGF, the banking sector, and taxpayers.2 Tax treatment of loans continued to hinder restructuring. 80 Addressing Fiscal and Financial Sector Vulnerabilities Addressing Fiscal Vulnerabilities The World Bank addressed fiscal vulnerabilities primarily through analytical work and policy dialogue (including on energy sector reform), followed by a major scale-up of lending. This work was coordinated with the International Monetary Fund’s Extended Fund Facility and European Union budget support. Increased tariffs resulted in a decline in unsustainable energy subsidies from 7 percent to 2 percent of GDP, and budgetary subsidies evolved from regressive producer subsidies to targeted consumer subsidies. Moving from universal to targeted subsidies required considerable technical assistance work, to which the World Bank contributed significantly. The World Bank also supported pub- lic investment management reforms through technical assistance. A large part of the development policy operation agenda supported fiscal consolidation through reform of energy subsidies. In 2014, energy producer subsidies to the state-owned enterprise Naftogaz were about 7 percent of GDP. Reform sought to increase energy tariffs, eliminate producer subsidies, and create targeted consumer subsidies to support poor households. Three loans focused on the energy sector and included attention to tariff and sub- sidy reforms, which helped significantly reduce fiscal vulnerabilities. Addressing Financial Sector Vulnerabilities Independent Evaluation Group World Bank Group    81 Bank Group staff were proactive in tracking financial sector vulnerabilities. Progress was greatest with respect to restoring financial stability and weak- est in building institutional capacity (where reform required confronting vested interests). The 2012 CPS recognized the lack of political willingness to confront financial sector weaknesses, and lending plans were curtailed accordingly. Reflecting the severity of the problem in the financial sector, an extensive financial sector monitoring project was undertaken with support from the Financial Sector Reform and Strengthening Initiative, the Swedish International Development Cooperation Agency, and Switzerland’s State Secretariat for Economic Affairs. This work permitted an accelerated re- sponse when the government changed in 2014. The World Bank approved a programmatic series of two financial sector development policy operations, whose development objectives included (i) strengthening the operational, financial, and regulatory capacity of the DGF for the resolution of insolvent banks; (ii) improving the solvency of the banking system through the implementation of bank recapitalization and restructuring plans and timely enforcement actions; and (iii) strengthening the legal and institutional framework to improve resiliency and efficiency of the banking system. In November 2018, the World Bank approved a policy-based guarantee on economic growth and fiscally sustainable services. It supported reforms to improve governance (of SOCBs, in particular). Prior actions included (i) the enactment of a law to establish independent supervisory boards, strengthen risk management at SOCBs, and adopt a strategy for SOCBs, which lays out a road map for gradual divesture, and (ii) the amendment of the tax code to re- move the 25 percent ceiling for provisioning expenses, issue orders on the tax treatment of transactions involving NPLs, and enact a law to increase creditor rights. The operation highlighted substantial risks to implementation. Effectiveness Effectiveness of Support to Reduce Fiscal Vulnerabilities The World Bank Group in Ukraine, 2012–20  Appendix D The elimination of energy subsidies that formed a key element of World Bank–supported reform, coupled with other expenditure and tax measures, led to a substantial improvement in the fiscal position and debt dynamics by 2016. This improvement was characterized by primary fiscal surpluses, declining public debt, significantly reduced subsidies, and near-full provi- sioning of NPLs. By 2019, these fiscal vulnerabilities had been substantially reduced. But despite these gains, the fiscal situation remained vulnerable to pressure from vested interests, changes in political will, external shocks, weaknesses in public financial management, and corruption. Effectiveness of Support to Reduce Financial Sector Vulnerabilities With support from development partners, Ukraine made progress during 82 the evaluation period in reducing financial sector vulnerabilities. About half of the country’s 180 banks were declared insolvent and sent for resolution. Developments have been modest but positive in dealing with the challenges in the SOCBs. Provisioning of NPLs is deemed adequate, in spite of the high NPL ratio of 55 percent of total loans at the end of 2018. Although supervisory boards are in place at the three largest SOCBs, the quality of at least some of the independent supervisory board members draws into question the extent of their real commitment to reform. The key achievements of the financial sector development policy loan series were to stabilize the banking sector, strengthen the National Bank of Ukraine, establish the Financial Stability Committee, and conduct the related party review exercise. The operational, financial, and regulatory capacity of the DGF was strengthened, including giving it resolution authority. The Law on Deposit Insurance System in Ukraine was enacted in March 2012, assigning a bank res- olution mandate to the DGF as of September 2012. The International Finance Corporation’s interventions to strengthen risk management in the financial sector show positive outcomes, but with limited impacts because of the small share of banks participating in the International Finance Corporation program (less than 2.5 percent of financial sector assets and total loans at the time of the Independent Evaluation Group’s evaluation in 2013). Ukraine’s financial sector remains highly vulnerable because of the high level of NPLs in SOCBs and weak regulation of the nonbank financial sector. Vested interests operating through members of the Rada have slowed or out- Independent Evaluation Group World Bank Group    83 right prevented the approval and enactment of critical legislation (including efforts to strengthen supervision of the fast-growing insurance sector).3 As a result, Ukraine performed poorly in dealing with its large stock of NPLs. An International Finance Corporation Financial Markets Crisis Response Project was unable to generate sufficient demand for NPLs. In Doing Business 2019, Ukraine scored 31.7 out of 100 in resolving insolvency, giving it a ranking of 145 out of 190 economies, well below most regional comparators (World Bank Group 2019). The courts remain a major constraint to the efficacy of efforts to reduce financial sector vulnerabilities by preventing the effective implementation of existing laws, such as asset recovery from failed banks, NPL resolution, and the insolvency law approved in October 2018. As of May 2019, no cases of high-level corruption had been adjudicated. The November 2018 policy- based guarantee describes the “inefficient judicial system” as a “major impedimen[t] to NPL resolution in Ukraine” (World Bank 2018, 13). In April 2019, the court decision to reverse the nationalization of PrivatBank (which is being appealed) was a reminder that the judiciary cannot be relied on to implement legislation in a predictable manner. Although dissemination and outreach were evident in other aspects of the reform agenda, this was not the case for financial sector reform. With a rel- atively low level of financial literacy among the general public, a concerted effort was needed to foster a better understanding among civil society of the importance of, and implications for not pursuing, financial sector reforms, in- cluding improvements in bank supervision. This, however, was not an explicit element of the Country Partnership Framework (World Bank 2017a, 2017b).4 References IMF (International Monetary Fund). 2014. “Ukraine: Ex Post Evaluation of Excep- tional Access under the 2010 Stand-by Arrangement.” Country Report 14/146, IMF, Washington, DC. IMF (International Monetary Fund). 2019. “Ukraine: Request for Stand-by Arrange- ment and Cancellation of Arrangement under the Extended Fund Facility—Press The World Bank Group in Ukraine, 2012–20  Appendix D Release; Staff Report; and Statement by the Executive Director for Ukraine.” Country Report 19/3, IMF, Washington, DC. World Bank. 2010. Ukraine: Strategic Choices to Accelerate and Sustain Growth. Coun- try Economic Memorandum. Washington, DC: World Bank. World Bank. 2012. Ukraine—Country Partnership Strategy for the Period FY12–FY16. Washington, DC: World Bank. World Bank. 2017a. “Ukraine—Completion and Learning Review for Country Partner- ship Framework for the Period FY12–FY16: IEG Review.” Independent Evalua- tion Group, World Bank, Washington, DC. World Bank. 2017b. Ukraine—Country Partnership Framework for the Period FY17– FY21. Washington, DC: World Bank. 84 World Bank. 2018. “Ukraine—Economic Growth and Fiscally Sustainable Services Policy Based Guarantee.” Program Document 132120-UA, World Bank Group, Washington, DC. World Bank Group. 2012. Doing Business 2012: Doing Business in a More Transparent World. Washington, DC: World Bank Group. World Bank Group. 2019. Doing Business 2019: Training for Reform. Washington, DC: World Bank Group. Independent Evaluation Group World Bank Group    85 1  This appendix was prepared by Željko Bogetić, Jeffrey Allen Chelsky, and Melvin Vaz. Interviews with senior officials in the government of Ukraine, senior officers of commercial banks and other financial institutions, and Kyiv-based staff of the World Bank Group, the International Monetary Fund, and the European Commission were conducted during a visit to Kyiv in May 2019, augmented by interviews with Bank Group and International Monetary Fund staff at headquarters in Washington, DC. 2  In Doing Business 2012, Ukraine ranked 156 in terms of resolving insolvency (World Bank Group 2012). 3  Shortcomings in the implementation of reforms to strengthen governance and oversight of the financial sector are also reflected in the fact that the International Monetary Fund was able to complete only three of the envisaged 15 reviews under the Extended Fund Facility ar- rangement by the time it expired in March 2019, with the banking system’s failure to address the burden of legacy nonperforming loans listed alongside delayed anticorruption measures as the reasons for weak performance (IMF 2019). 4  Annex H of the Country Partnership Framework for fiscal years 2017–21 identifies a list of sectors that would require “raising civil society awareness and creating opportunities for di- alogue on key institutional reforms” (World Bank 2017b, 87). Although the list is not exhaus- tive, the financial sector is not listed among other priority reform areas. The World Bank Group in Ukraine, 2012–20  Appendix D 86 Supported Program in Ukraine Appendix E. World Bank Group– Table E.1. Ukraine Lending Portfolio Active during Fiscal Years 2012–21 IBRD Lending Commitment Approval Exit Project Instrument Amount Project Name FY FY GP Status Type (US$, millions) Ukraine Higher Education Project 2021 2027 EDU Active Investment 200 Emergency COVID-19 vaccine 2021 2023 HNP Active Investment 90 Access to Long-Term Finance Additional Financing 2021 2023 FCI Active Investment 100 Power System Resilience 2021 2027 EAE Active Investment 177 Additional Financing 2 for SSNMP 2021 2021 SPJ Active Investment 300 Eastern Ukraine: 3R Project 2021 2026 SSI Active Investment 100 Economic Recovery DPL 2020 2021 MTI Closed DPF 350 Additional Financing to Serving People, Improving 2020 2021 HNP Active Investment 135 Health Project Additional Financing for Social Safety Nets Modern- 2020 2021 SPJ Active Investment 150 ization Accelerating Private Investment in Agriculture 2019 2025 AGR Active PforR 200 Access to Long-Term Finance 2017 2022 FCI Active Investment 150 DPL 2 2016 2016 MTI Closed DPF 500 Programmatic Financial Sector DPL 2 2016 2016 FCI Closed DPF 500 Road Sector Development Project 2016 2022 TDD Active Investment 560 Programmatic Financial Sector DPL 1 2015 2015 FCI Closed DPF 500 Second Power Transmission Project 2015 2020 EAE Active Investment 330 Serving People, Improving Health Project 2015 2021 HNP Active Investment 215 Social Safety Nets Modernization Project 2015 2021 SPJ Active Investment 300 DPL 1 2014 2015 MTI Closed DPF 750 (continued) Independent Evaluation Group World Bank Group    87 88 The World Bank Group in Ukraine, 2012–20  Appendix E IBRD Lending Commitment Approval Exit Project Instrument Amount Project Name FY FY GP Status Type (US$, millions) Second Urban Infrastructure Project 2014 2021 WAT Active Investment 300 District Heating Energy Efficiency Project 2014 2021 EAE Active Investment 332 DEVSTAT Additional Financing 2013 2014 POV Closed Investment 10 Second Road and Safety Improvement Project 2013 2020 TDD Active Investment 450 Ukraine Export Development II Additional Financing 2012 2015 FCI Closed Investment 150 Total n.a. n.a. n.a. n.a. 6,849 Ongoing Projects during FY12–20 Ukraine—Energy Efficiency 2011 2017 EAE Closed Investment 200 Hydropower—Additional Financing 2010 2016 EAE Closed Investment 60 Roads and Safety Improvement 2009 2015 TDD Closed Investment 400 Public Finance Modernization Project 2008 2015 GOV Closed Investment 50 Power Transmission 2008 2016 EAE Closed Investment 200 Urban Infrastructure 2008 2015 WAT Closed Investment 140 Second Export Development Project 2007 2015 FCI Closed Investment 155 Social Assistance System Modernization Project 2006 2013 SPL Closed Investment 99 Hydropower Rehabilitation 2005 2016 EAE Closed Investment 106 Development of State Statistics System for Monitoring 2004 2014 POV Closed Investment 32 Socioeconomic Transformation Project Rural Land Titling and Cadastre Development Project 2003 2013 AGR Closed Investment 195 State Tax Service Modernization Project (APL 1) 2003 2012 GOV Closed Investment 40 Total n.a. n.a. n.a. n.a. 1,677 Source: World Bank Business Intelligence (December 15, 2021). Note: 3R = Reconnect, Recover, Revitalize; AGR = Agriculture; APL = adaptable program loan; DEVSTAT = Development of the State Statistics System for Monitoring the Social and Economic Transformation; DPF = development policy financing; DPL = development policy loan; EAE = Energy and Extractives; EDU = Education; FCI = Finance, Competitiveness, and Innovation; FY = fiscal year; GOV = Governance; GP = Global Practice; HNP = Health, Nutrition, and Population; IBRD = International Bank for Reconstruction and Development; MTI = Macroeconomics, Trade, and Investment; n.a. = not applicable; PforR = Program-for-Results; POV = Poverty; SPJ = Social Protection and Jobs; SPL = Social Protection and Labor; SSNMP = Social Safety Nets Modernization Project; TDD = Transport and Digital Development; WAT = Water. Table E.2. Guarantees Approved during Fiscal Years 2012–20 Lending IBRD Project Approval Project Latest Instrument Commitment ID Project Name FY Exit FY Status DO Type (US$, millions) P164414 Ukraine Policy-Based 2019 2020 Active — Adjustment 750 Guarantee P155111 Ukraine Gas Supply 2017 2021 Active S Investment 500 Security Facility Source: World Bank Business Intelligence (January 30, 2021). Note: DO = development outcome; FY = fiscal year; IBRD = International Bank for Reconstruction and Development; S = satisfactory; — = not rated. Independent Evaluation Group World Bank Group    89  dvisory Services and Analytics for Ukraine, Fiscal Years Table E.3. A 2012–20 Total Lifetime Total BB Total BETF Expenditure Global FY (US$,  (US$,  (US$,  Task Name Practice Delivered thousands) thousands) thousands) Ukraine Finance FCI 2021 198,609 3,860,704 4,059,313 and Enterprise Recovery Ukraine PMR EAE 2021 0 3,156,303 3,156,303 Support to PAR GOV 2021 283 2,026,035 2,026,319 and PFM Re- forms Ukraine FS Moni- FCI 2021 1,245,004 0 1,245,004 toring TA Ukraine EE Fund EAE 2021 88,526 610,083 698,610 Development TA Advancing EAE 2021 215,283 409,412 624,695 Energy Subsidy Reform Ukraine Re- FCI 2021 0 412,368 412,368 mittances and Payments The World Bank Group in Ukraine, 2012–20  Appendix E Government of SPJ 2021 0 249,289 249,289 Ukraine Leader- ship Peacebuild- ing-Recovery Ukraine Conflict SPJ 2021 54,947 0 54,947 Recovery PFR-3 MTI 2021 177,377 0 177,377 Program Model TDD 2021 0 579,390 579,390 for Pilot Road Concessions Eastern Ukraine MTI 2021 11,988 796,780 808,767 Socioeconomic Growth Study Policy Note on WAT 2021 31,013 0 31,013 the WSS Sector Ukraine Health HNP 2021 40,919 0 40,919 Financing Re- forms 90 (continued) Total Lifetime Total BB Total BETF Expenditure Global FY (US$,  (US$,  (US$,  Task Name Practice Delivered thousands) thousands) thousands) Ukraine- FCI 2020 215,465 494,855 710,320 Supporting Anti- Corruption Laws and Institutions Ukraine: Rural HNP 2020 105,463 105,463 Health and Tele- medicine Ukraine Country ENV 2020 68,821 123,459 192,280 Forest Note Social Safety SPJ 2020 114,518 114,518 Nets Rational- ization including HUS Pension Reform SP 2020 72,973 72,973 Assessment and New Initiatives Psychosocial SPJ 2020 — 399,875 399,875 Support to Conflict-Affected Population Ukraine #C013 FCI 2019 — 201,262 201,262 Supporting the Deposit Guaran- Independent Evaluation Group World Bank Group    91 tee Fund Ukraine Sourc- MTI 2019 378,842 — 378,842 es of Growth Analysis Ukraine Pro- MTI 2019 81,042 — 81,042 grammatic Public Finance Review 2 Integrated Ap- SPL 2019 77,000 — 77,000 proach toward Modernization of SP Ukraine Com- FCI 2019 61,104 — 61,104 petitiveness and Innovation for Growth (continued) Total Lifetime Total BB Total BETF Expenditure Global FY (US$,  (US$,  (US$,  Task Name Practice Delivered thousands) thousands) thousands) Ukraine’s SPL 2019 59,893 — 59,893 Pension Policy Dialogue II Ukrzaliznytsia TDD 2019 140,571 — 140,571 Modernization Strategy Review of Edu- EDU 2019 445,096 — 445,096 cation Sector of Ukraine Ukraine: GOV 2019 47,715 — 47,715 E-Government Technical Assistance Port Sector Re- TDD 2019 30,935 139,288 170,223 form for Attract- ing Private Sector Participation Ukraine Land Other 2019 697,099 — 697,099 Reform Support Ukraine— EAE 2019 7,715 119,914 127,629 Supporting the District Heating The World Bank Group in Ukraine, 2012–20  Appendix E Sector Market Assess- EAE 2019 — 149,640 149,640 ment of Small Hydro Rehabilita- tion in Ukraine Ukraine: Recov- URS 2018 251,144 — 251,144 ery and Peace- building Support Labor and Skills SPL 2018 — 299,306 299,306 Advancing EAE 2018 3,750 905,643 909,393 Energy Tariff and Subsidy Reforms Government of URS 2018 33,670 241,512 275,182 Ukraine Capac- ity Building for Peacebuilding and Recovery (continued) 92 Total Lifetime Total BB Total BETF Expenditure Global FY (US$,  (US$,  (US$,  Task Name Practice Delivered thousands) thousands) thousands) Facilitating Elec- EAE 2018 152,706 402,016 554,722 tricity and Gas Market Reforms in Ukraine Ukraine Agricul- WAT 2018 93,814 400,834 494,648 ture, Irrigation, and Land TA Facilitating Forest ENV 2018 — 146,385 146,385 Sector Reform in Ukraine Toward Greener TDD 2018 23,813 1,067,324 1,091,137 and More Effi- cient Logistics in Ukraine Ukraine’s SPL 2018 226,442 — 226,442 Pension Policy Dialogue Ukraine Compe- FCI 2018 250,365 — 250,365 tition Policy Sustainable Mo- TDD 2018 — 172,442 172,442 bility for Odessa Strategy for TDD 2018 238,400 — 238,400 Prioritization of Independent Evaluation Group World Bank Group    93 Investments, Funding, and Modernization of Ukraine’s Road Sector Ukraine— URS 2018 237,188 — 237,188 Technical Assistance to Land Policy Ukraine: Admin- GOV 2018 89,854 — 89,854 istrative Services and EGovern- ment Sustainable TDD 2017 17,526 474,988 492,514 Urban Transport for the City of Kyiv (continued) Total Lifetime Total BB Total BETF Expenditure Global FY (US$,  (US$,  (US$,  Task Name Practice Delivered thousands) thousands) thousands) Improving Tax GOV 2017 — 267,996 267,996 Administration Increasing Fiscal GOV 2017 — 597,273 597,273 Transparency and Account- ability Financial Disclo- GOV 2017 9,946 561,139 571,084 sures and Pre- ventive Anticor- ruption Agency Improving HNP 2017 15,864 494,516 510,380 Effectiveness in Human Develop- ment Improving Trans- TDD 2017 — 348,513 348,513 parency and Governance of Infrastructure Ukraine Pro- MTI 2017 — 249,991 249,991 grammatic Public Finance Review 1 Ukraine Health HNP 2017 104,937 — 104,937 The World Bank Group in Ukraine, 2012–20  Appendix E Policy Dialogue Systematic Other 2017 362,447 — 362,447 Country Diag- nostic Ukraine Trade FCI 2017 — 73,848 73,848 Finance Advice to the EAE 2017 — 29,972 29,972 Design and Cre- ation of Energy Efficiency Fund Rapid Actions to GOV 2017 — 300,739 300,739 Strengthen Court Management Lower Dnieper TDD 2017 — 382,173 382,173 River Waterway and Port PPP (continued) 94 Total Lifetime Total BB Total BETF Expenditure Global FY (US$,  (US$,  (US$,  Task Name Practice Delivered thousands) thousands) thousands) Socioeconom- URS 2017 396,386 — 396,386 ic Impacts of Conflict and Displacement Social Account- GOV 2016 77,156 — 77,156 ability Tools for CSOs Monitoring Public Invest- GOV 2016 47,150 229,564 276,715 ment System Skills for Em- SPL 2016 149,423 — 149,423 ployment and Productivity Ukraine— AGR 2016 46,522 149,860 196,382 Agricultural Trade, Transport, and Logistic Ukraine Urban- URS 2016 40,965 149,159 190,124 ization Review Effective HNP 2016 281 381,419 381,699 Response to HIV/AIDS and TB Epidemics Ukraine: Ener- URS 2016 125,138 637,169 762,307 Independent Evaluation Group World Bank Group    95 gy Efficiency Transformation in Cities Ukraine Country ENV 2016 97,366 — 97,366 Environmental Analysis Ukraine: Moving EAE 2016 11,541 1,097,182 1,108,723 Forward Energy Tariff Reforms Social Safety SPL 2016 154,451 — 154,451 Nets and Energy Reform Modernization of TDD 2016 137,696 145,996 283,692 Ukrzaliznytsia Ukraine Shared POV 2016 59,566 — 59,566 Prosperity FY16 (continued) Total Lifetime Total BB Total BETF Expenditure Global FY (US$,  (US$,  (US$,  Task Name Practice Delivered thousands) thousands) thousands) Ukraine Agricul- AGR 2016 163,977 199,071 363,048 ture, Irrigation, and Land TA Ukraine Justice GOV 2016 82,413 –13,091 69,322 Policy Dialogue Conflict and URS 2016 42,022 — 42,022 Displacement Mainstreaming Ukraine—Munic- EAE 2015 11,270 127,161 138,431 ipal Energy Effi- ciency Financing Mitigating the EAE 2015 — — — Impact of Gas and Heating Tariff Increases Ukraine Techni- FCI 2015 — 70,625 70,625 cal Assistance on Consumer Ukraine—Pro- FCI 2015 458,432 — 458,432 grammatic FS Monitoring TA Ukraine—Road TDD 2015 42,519 — 42,519 The World Bank Group in Ukraine, 2012–20  Appendix E Sector Policy Dialogue Agriculture and AGR 2015 74,521 31,587 106,108 Land Monitor- ing II Ukraine #A025 FCI 2015 — 382,611 382,611 Building Capacity at Deposit Guar- antee Fund Ukraine Policy MTI 2015 35,184 — 35,184 Notes Ukraine Shared POV 2015 77,827 — 77,827 Prosperity Note Fiscal, Structural, MTI 2014 104,675 825,855 930,529 and Governance TA (continued) 96 Total Lifetime Total BB Total BETF Expenditure Global FY (US$,  (US$,  (US$,  Task Name Practice Delivered thousands) thousands) thousands) Effective Re- HNP 2014 449,457 — 449,457 sponse to AIDS Epidemic Agriculture and AGR 2014 289,814 228,499 518,313 Land Monitoring Ukraine-Dissem- FCI 2014 — 48,065 48,065 ination-Personal Insolvency Municipal SWM URS 2014 100,732 — 100,732 Sector Review Ukraine Pro- MTI 2014 — 70,489 70,489 grammatic PSD TA TA in Legal Re- FCI 2014 77,446 — 77,446 form Relative to Business TA on Regulatory FCI 2014 109,182 — 109,182 Reform Ukraine Educa- EDU 2013 241,245 — 241,245 tion Measuring GOV 2013 1,006 159,440 160,446 Governance in Health and Edu- Independent Evaluation Group World Bank Group    97 cation Public Invest- GOV 2013 102,362 43,943 146,305 ment Manage- ment Assess- ment Ukraine: Munici- URS 2013 162,247 — 162,247 pal Demand-Side Governance TA on Health HNP 2013 — 33,929 33,929 Sector Reform in Ukraine PEFA Dissemi- GOV 2013 330,917 — 330,917 nation Ukraine—Financial FCI 2013 140,376 — 140,376 Sector TA (continued) Total Lifetime Total BB Total BETF Expenditure Global FY (US$,  (US$,  (US$,  Task Name Practice Delivered thousands) thousands) thousands) TA on Health HNP 2013 90,273 — 90,273 Sector Reform 2 Ukraine Analyti- FCI 2013 — 119,602 119,602 cal Note on PSD Ukraine #10323 FCI 2013 327,300 — 327,300 Planning and Implementing PFM TA MTI 2012 89,573 79,996 169,569 Ukraine Financial FCI 2012 137,126 — 137,126 Sector TA Labor Mobility SPL 2012 3,327 — 3,327 CA-Ukraine: URL 2012 — — — Supporting Kyiv in City Vision and Strategy PEFA Update GOV 2012 — — — Creditworthiness EAE 2012 — — — Improvements— Ukraine Utilities Ukraine Trust FCI 2012 — — — Fund Seed The World Bank Group in Ukraine, 2012–20  Appendix E Money Total n.a. n.a. 11,868,931 26,343,398 38,212,328 Source: World Bank Business Intelligence and Standard Reports (December 15, 2021). Note: AGR = Agriculture; BB = Bank budget; BETF = Bank-executed trust funds; CA = Cities Alliance; CSO = civil society organization; EAE = Energy and Extractives; EDU = Education; EE = Energy Efficiency; ENV = Environment and Natural Resources; FCI = Finance, Competitiveness, and Innovation; FS = financial sector; FY = fiscal year; GOV = Governance; HNP = Health, Nutrition, and Population; HUS = house- hold utility subsidy; MTI = Macroeconomics, Trade, and Investment; n.a. = not applicable; PAR = Public Administration Reform; PEFA = Public Expenditure and Financial Accountability; PFM = public financial management; PFR = Program-for-Results; PMR = Partnership for Market Readiness; POV = Poverty; PPP = public-private partnership; PSD = private sector development; SP = Social Protection; SPJ = Social Protection and Jobs; SPL = Social Protection and Labor; SWM = solid waste management; TA = technical assistance; TB = tuberculosis; TDD = Transport and Digital Development; URS = Social, Urban, Rural, and Resilience; URL = Urban, Disaster Risk Management, Resilience, and Land; WAT = Water; WSS = Water Services and Sanitation; — = not available. 98 nternational Finance Corporation Investment Commitments, Fiscal Years 2012–20 (US$, thousands) Table E.4. I Original Net Total Net Commitment Project Project Commitment Commitment Commitment FY Status Primary Sector Size (IFC Balance)a (LN) (LN + EQ)b 2020 Active Finance and Insurance 50,000 20,000 20,000 20,000 2020 Active Chemicals 70,000 35,000 35,000 35,000 2020 Active Agriculture and Forestry 20,000 20,000 20,000 20,000 2019 Active Public Administration 14,315 14,359 14,359 14,359 2019 Closed Plastics and Rubber 6,510 6,452 288 288 2018 Closed Plastics and Rubber 29,590 11,836 11,836 11,836 2018 Active Agriculture and Forestry 12,500 12,500 12,500 12,500 2018 Active Agriculture and Forestry 40,000 0 0 0 2018 Active Agriculture and Forestry 151,500 30,000 30,000 30,000 2018 Active Agriculture and Forestry 17,000 17,000 17,000 17,000 2017 Active Transportation and Warehousing 37,000 37,000 28,549 28,549 2017 Active Collective Investment Vehicles 15,000 15,000 0 15,000 2017 Active Agriculture and Forestry 147,500 60,000 40,000 40,000 2016 Active Chemicals 15,000 15,000 7,500 7,500 2016 Closed Agriculture and Forestry 35,000 15,000 10,000 10,000 2016 Closed Agriculture and Forestry 33,327 3,184 2,790 2,790 2016 Closed Agriculture and Forestry 10,000 10,000 10,000 10,000 2016 Closed Agriculture and Forestry 25,000 10,000 10,000 10,000 2015 Active Agriculture and Forestry 25,000 25,000 15,000 15,000 2015 Active Finance and Insurance 457 454 0 454 2014 Active Agriculture and Forestry 50,000 30,000 30,000 30,000 (continued) Independent Evaluation Group World Bank Group    99 100 The World Bank Group in Ukraine, 2012–20  Appendix E Original Net Total Net Commitment Project Project Commitment Commitment Commitment FY Status Primary Sector Size (IFC Balance)a (LN) (LN + EQ)b 2014 Closed Finance and Insurance 5,000 5,000 5,000 5,000 2014 Active Chemicals 100,000 15,000 9,583 9,583 2014 Closed Agriculture and Forestry 47,079 47,043 47,043 47,043 2014 Active Agriculture and Forestry 250,000 100,000 100,000 100,000 2014 Closed Agriculture and Forestry 65,000 65,000 64,157 64,157 2013 Closed Wholesale and Retail Trade 30,600 10,000 0 0.00 2013 Closed Agriculture and Forestry 50,000 40,000 40,000 40,000 2013 Active Chemicals 85,000 30,000 21,825 21,825 2013 Closed Agriculture and Forestry 50,000 30,000 0 0 2013 Active Agriculture and Forestry 110,886 21,087 0 21,085 2013 Closed Agriculture and Forestry 16,000 16,000 16,000 16,000 2013 Active Wholesale and Retail Trade 75,000 45,088 45,088 45,088 2013 Closed Agriculture and Forestry 50,000 50,000 20,000 20,000 2012 Closed Transportation and Warehousing 87,600 32,000 22,000 22,000 2012 Closed Agriculture and Forestry 22,500 5,000 4,000 4,000 2012 Closed Agriculture and Forestry 5,000 5,000 5,000 5,000 2012 Closed Finance and Insurance 15,000 15,000 0 0 Total n.a. n.a. 1,869,364 919,003 714,518 751,057 Source: International Finance Corporation management information system (January 10, 2020). Note: EQ = equity; FY = fiscal year; IFC = International Finance Corporation; LN = loan; n.a. = not applicable. a. IFC investment commitments exclude short-term finance. b. Net commitment = original commitment less loan and equity cancellation. nternational Finance Corporation Advisory Services in Ukraine, Fiscal Years 2012–20 Table E.5. I Primary Total Funds Project Implementation Implementation Project Project Business Managed by IFC ID Project Name Start FY End FY Stage Status Area (US$, millions) Financial sector 604205 IFC Digilab Finance ECA 2020 2023 Portfolio Active FIG 0.95 603603 Ukraine Financial Inclusion—Credit 2020 2024 Portfolio Active EFI 1.90 Reporting 603667 Ukraine Financial Inclusion— 2020 2024 Portfolio Active EFI 1.80 Consumer Protection 602198 Green Finance Project: Phase 1 2019 2023 Portfolio Active FIG 1.99 597667 ECA Financial Markets Risk Man- 2013 2019 Portfolio Active FIG 3.25 agement Project Energy sector 602817 Ukraine Energy Efficiency Fund 2019 2023 Portfolio Active EFI 17.27 603138 ECA Cities Platform II 2019 2023 Portfolio Active INR 4.00 601079 Kyivenergo District Heating 2016 2018 Com- Closed CPC-PPP 0.62 pleted Agriculture 599789 Ukraine Crop Receipts Project 2015 2020 Portfolio Active EFI 4.87 600339 Europe and Central Asia Agri- 2015 2021 Portfolio Active EFI 5.36 Finance Project 592348 Ukraine Agri Aggregator 2013 2019 Com- Closed MAS 6.67 pleted 599215 Agribusiness Standards Advisory 2013 2017 Com- Closed MAS 4.56 in ECA pleted (continued) Independent Evaluation Group World Bank Group    101 102 The World Bank Group in Ukraine, 2012–20  Appendix E Primary Total Funds Project Implementation Implementation Project Project Business Managed by IFC ID Project Name Start FY End FY Stage Status Area (US$, millions) Business environment 602031 Ukraine Dairy Supply Chain Devel- 2018 2023 Portfolio Active MAS 4.75 opment Project 600668 ECA FM Public Outreach 2016 2019 Portfolio Active EFI 1.08 600664 Ukraine Investment Climate Reform 2016 2019 Portfolio Active EFI 3.33 Project 584508 Ukraine Investment Climate: Agri- 2012 2016 Com- Closed TAC 2.86 business pleted 592347 ECA SME Crisis Resilience Program 2013 2017 Com- Closed FIG 2.36 pleted 599538 Direct Client Work—ECA Corporate 2013 2019 Portfolio Active ESG-CG 1.00 Governance PPP 602109 Kyiv Hospital Consolidation 2018 2021 Portfolio Hold CPC-PPP 1.42 602196 Olvia Port PPP 2018 2020 Portfolio Active CPC-PPP 0.51 603032 Kherson Port PPP Ukraine 2018 2020 Portfolio Active CPC-PPP 0.32 Total n.a. n.a. n.a. n.a. n.a. 70.87 Source: International Finance Corporation management information system (January 10, 2020). Note: CG = corporate governance; CPC = Corporate Portfolio Committee; ECA = Europe and Central Asia; EFI = Equitable Growth, Finance, and Institutions; ESG = Envi- ronmental, Social, and Governance; FIG = Financial Institutions Group; FM = Financial Markets; FY = fiscal year; IFC = International Finance Corporation; INR = Infrastruc- ture; MAS = Manufacturing, Agribusiness, and Services; n.a. = not applicable; PPP = public-private partnership; SME = small and medium enterprise; TAC = Trade and Competitiveness.  ultilateral Investment Guarantee Agency Projects Active in Table E.6. M Ukraine, Fiscal Years 2012–21 Gross Exposure Project Title FY Issued Sector (US$) Insurance of Mandatory Reserve at the 2012 Financial 3,969,784 Central Bank Whirlpool Ukraine 2012 Manufac- 6,566,957 turing Raiffeisen Bank Aval—Joint Stock 2013 Financial 142,500,000 Company Porsche Mobility—Porsche Leasing 2014 Financial 23,940,000 Lantmännen Axa 2017 Manufac- 6,350,472 turing Aluminum Beverage Can Factory 2019 Manufac- 20,700,000 turing ProCredit Mandatory Reserves 2021 Financial 20,937,240 Total n.a. n.a. 224,964,453 Source: Business Intelligence reporting database (December 15, 2021). Note: FY = fiscal year; n.a. = not applicable. Independent Evaluation Group World Bank Group    103 The World Bank 1818 H Street NW Washington, DC 20433