FOR OFFICIAL USE ONLY Report No: PAD3955 INTERNATIONAL DEVELOPMENT ASSOCIATION PROJECT APPRAISAL DOCUMENT ON A PROPOSED CREDIT IN THE AMOUNT OF US$50 MILLION TO THE REPUBLIC OF UZBEKISTAN FOR A MODERNIZING UZBEKISTAN NATIONAL INNOVATION SYSTEM (MUNIS) PROJECT OCTOBER 5, 2020 Finance, Competitiveness and Innovation Global Practice Europe and Central Asia Region This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed without World Bank authorization. CURRENCY EQUIVALENTS (Exchange Rate Effective September 18, 2020) Currency Unit = Uzbek som 10,219 som = US$1 FISCAL YEAR January 1 - December 31 Regional Vice President: Anna Bjerde Country Director: Lilia Burunciuc Regional Director: Lalita M. Moorty Practice Manager: Ilias Skamnelos Task Team Leader(s): Yeraly Beksultan ABBREVIATIONS AND ACRONYMS ADPI Agency for Development of the Pharmaceutical Industry BCR Benefit-to-Cost Ratio BIPM International Bureau of Weights and Measures BIRD Business Investment in Research and Development BSL Biosafety Levels CPF Country Partnership Framework CPSD Country Private Sector Diagnostic ECA Europe and Central Asia ERC European Research Council ERIC European Research Infrastructure Consortium ESF Environment and Social Factor Risk ESMF Environment and Social Management Framework FEZ Free Economic Zone FTE Full-Time Equivalent GDP Gross Domestic Product GERD Gross Expenditure on Research and Development GFDRR Global Facility for Disaster Reduction and Recovery GFM Grant Fiduciary Manual GLP Good Laboratory Practice GMP Good Manufacturing Practices GOU Government of Uzbekistan GRS Grievance Redress Service HEI Higher Education Institution HEMIS Higher Education Management Information System ICP Innovation Capabilities Program ICR Implementation Completion Report ICT Information and Communications Technology IEB International Expert Board IEG Independent Evaluation Group IFR Interim Financial Report ILAC International Laboratory Accreditation Cooperation IMP Institutional Modernization Plan IP Intellectual Property IPF Investment Project Financing IPSAS International Public Sector Accounting Standards ISCED International Standard Classification of Education ISR Implementation Status and Results KPI Key Performance Indicator MID Ministry of Innovative Development MOF Ministry of Finance MSTQ Metrology, Standardization, Testing and Quality NGO Non-Government Organization NIS National Innovation System NPV Net Present Value NQI National Quality Infrastructure OPCS Operations Policy and Country Services PCT Patent Cooperation Treaty PDA Project Designated Account PDO Project Development Objective PMU Project Management Unit POM Project Operation Manual PPA Project Preparation Advance PPE Personal Protective Equipment PPP Public-Private Partnership PPSD Project Procurement Strategy for Development P-RAMS Procurement Risk Assessment and Management System PRIM Public Research Institutes Modernization PRO Public Research Organization PSC Project Steering Committee RAS Reimbursable Advisory Services RCP Research Commercialization Program R&D Research and Development REP Research Excellence Program RI Research Institute SAMA State Asset Management Agency SDG Sustainable Development Goal SDI Supplier Development Initiative SDR Social Discount Rate SME Small and Medium Enterprise SOE State-Owned Enterprise SORT Systemic Operational Risk Rating Tool STEP Systemic Tracking of Exchanges in Procurement STI Science, Technology and Innovation TA Technical Assistance TOR Terms of Reference TRIPS Trade-Related Aspects of Intellectual Property Rights TTU Technology Transfer Unit UZSTANDART Uzbek Agency for Standardization, Metrology and Certification WBG World Bank Group WHO World Health Organization WIPO World Intellectual Property Organization The World Bank Modernizing Uzbekistan National Innovation System (MUNIS) Project (P170206) TABLE OF CONTENTS DATASHEET............................................................................................................................. 1 I. STRATEGIC CONTEXT ................................................................................................................... 6 A. Country Context................................................................................................................................ 6 B. Sectoral and Institutional Context .................................................................................................... 7 C. Relevance to Higher Level Objectives............................................................................................. 15 II. PROJECT DESCRIPTION .............................................................................................................. 17 A. Project Development Objective (PDO) ........................................................................................... 17 B. Project Components ....................................................................................................................... 17 C. Project Beneficiaries ....................................................................................................................... 25 D. Results Chain .................................................................................................................................. 25 E. Rationale for the World Bank’s Involvement and Role of Partners................................................ 26 F. Lessons Learned and Reflected in the Project Design .................................................................... 26 III. IMPLEMENTATION ARRANGEMENTS ......................................................................................... 28 A. Institutional and Implementation Arrangements .......................................................................... 28 B. Results Monitoring and Evaluation Arrangements......................................................................... 29 C. Sustainability................................................................................................................................... 30 IV. PROJECT APPRAISAL SUMMARY ................................................................................................ 30 A. Technical, Economic and Financial Analysis ................................................................................... 30 B. Fiduciary.......................................................................................................................................... 31 C. Legal Operational Policies ............................................................................................................... 34 D. Environmental and Social ............................................................................................................... 34 V. GRIEVANCE REDRESS SERVICES ................................................................................................. 36 VI. KEY RISKS ................................................................................................................................. 36 VII. RESULTS FRAMEWORK AND MONITORING ................................................................................ 39 Annex 1: Summary of Strategy for Innovative Development (2019-2021) ............................................ 51 Annex 2: Women’s Participation in Science in Uzbekistan .................................................................. 54 Annex 3: Economic and Financial Analysis ........................................................................................ 55 Annex 4: Visual of implementation arrangements ............................................................................. 64 The World Bank Modernizing Uzbekistan National Innovation System (MUNIS) Project (P170206) DATASHEET BASIC INFORMATION BASIC INFO TABLE Country(ies) Project Name Uzbekistan Modernizing Uzbekistan National Innovation System Project Project ID Financing Instrument Environmental and Social Risk Classification Investment Project P170206 Moderate Financing Financing & Implementation Modalities [ ] Multiphase Programmatic Approach (MPA) [ ] Contingent Emergency Response Component (CERC) [ ] Series of Projects (SOP) [ ] Fragile State(s) [ ] Performance-Based Conditions (PBCs) [ ] Small State(s) [ ] Financial Intermediaries (FI) [ ] Fragile within a non-fragile Country [ ] Project-Based Guarantee [ ] Conflict [ ] Deferred Drawdown [ ] Responding to Natural or Man-made Disaster [ ] Alternate Procurement Arrangements (APA) [ ] Hands-on Enhanced Implementation Support (HEIS) Expected Approval Date Expected Closing Date 28-Oct-2020 30-Mar-2026 Bank/IFC Collaboration No Proposed Development Objective(s) The PDO is to enable the development of a market-oriented National Innovation System in Uzbekistan. Components Component Name Cost (US$, millions) Improving research excellence and commercialization 20.00 Page 1 of 60 The World Bank Modernizing Uzbekistan National Innovation System (MUNIS) Project (P170206) Promoting innovation in the private sector 20.00 Supporting STI policy-making and reforms 5.00 Project management and monitoring and evaluation 5.00 Organizations Borrower: Republic of Uzbekistan Implementing Agency: Ministry of Innovative Development PROJECT FINANCING DATA (US$, Millions) SUMMARY -NewFin1 Total Project Cost 50.00 Total Financing 50.00 of which IBRD/IDA 50.00 Financing Gap 0.00 DETAILS -NewFinEnh1 World Bank Group Financing International Development Association (IDA) 50.00 IDA Credit 50.00 IDA Resources (in US$, Millions) Credit Amount Grant Amount Guarantee Amount Total Amount Uzbekistan 50.00 0.00 0.00 50.00 National PBA 50.00 0.00 0.00 50.00 Total 50.00 0.00 0.00 50.00 Expected Disbursements (in US$, Millions) WB Fiscal Year 2021 2022 2023 2024 2025 2026 Page 2 of 60 The World Bank Modernizing Uzbekistan National Innovation System (MUNIS) Project (P170206) Annual 3.11 8.10 15.70 12.80 8.99 1.30 Cumulative 3.11 11.21 26.91 39.71 48.70 50.00 INSTITUTIONAL DATA Practice Area (Lead) Contributing Practice Areas Finance, Competitiveness and Innovation Climate Change and Disaster Screening This operation has been screened for short and long-term climate change and disaster risks SYSTEMATIC OPERATIONS RISK-RATING TOOL (SORT) Risk Category Rating 1. Political and Governance  Moderate 2. Macroeconomic  Moderate 3. Sector Strategies and Policies  Moderate 4. Technical Design of Project or Program  Moderate 5. Institutional Capacity for Implementation and Sustainability  Moderate 6. Fiduciary  Moderate 7. Environment and Social  Moderate 8. Stakeholders  Moderate 9. Other  Moderate 10. Overall  Moderate COMPLIANCE Policy Does the project depart from the CPF in content or in other significant respects? [ ] Yes [✓] No Page 3 of 60 The World Bank Modernizing Uzbekistan National Innovation System (MUNIS) Project (P170206) Does the project require any waivers of Bank policies? [ ] Yes [✓] No Environmental and Social Standards Relevance Given its Context at the Time of Appraisal E & S Standards Relevance Assessment and Management of Environmental and Social Risks and Impacts Relevant Stakeholder Engagement and Information Disclosure Relevant Labor and Working Conditions Relevant Resource Efficiency and Pollution Prevention and Management Relevant Community Health and Safety Relevant Land Acquisition, Restrictions on Land Use and Involuntary Resettlement Not Currently Relevant Biodiversity Conservation and Sustainable Management of Living Natural Not Currently Relevant Resources Indigenous Peoples/Sub-Saharan African Historically Underserved Traditional Not Currently Relevant Local Communities Cultural Heritage Not Currently Relevant Financial Intermediaries Not Currently Relevant NOTE: For further information regarding the World Bank’s due diligence assessment of the Project’s potential environmental and social risks and impacts, please refer to the Project’s Appraisal Environmental and Social Review Summary (ESRS). Legal Covenants Sections and Description Not later than sixty (60) days from the Effective Date, the PMU shall recruit an environmental protection specialist and a social protection specialist with terms of reference and qualifications acceptable to the Association. Sections and Description Not later than the ninety (90) days from the Effective Date the Recipient shall establish a Project Steering Committee with composition and terms of reference acceptable to the Association, to be chaired by a representative of the MID, which role shall be to ensure smooth coordination between various public entities engaged in the Project implementation; Page 4 of 60 The World Bank Modernizing Uzbekistan National Innovation System (MUNIS) Project (P170206) Sections and Description Not later than the ninety (90) days from the Effective Date the PMU shall establish and further throughout the Project implementation period sustain a fully functional accounting software and the accounting and financial controls based on the accounting software. Sections and Description Not later than the one hundred and twenty (120) days from the Effective Date, the Recipient, through the MID, shall establish an International Expert Board with a composition and terms of reference acceptable to the Association, to advise MID on (i) overall Project implementation; and (ii) selection of the Grants, based on the Project Operational Manual’s, respective Program Manual’s and the Grant Fiduciary Manual’s rules and criteria. Conditions Type Description Effectiveness the Recipient, through MID, has adopted a Project Operation Manual acceptable to the Association. Type Description Disbursement the Recipient, through MID, has adopted the Grant Fiduciary Manual. Page 5 of 60 The World Bank Modernizing Uzbekistan National Innovation System (MUNIS) Project (P170206) I. STRATEGIC CONTEXT A. Country Context 1. Over the past decades, Uzbekistan achieved high rates of growth, a rapid decline in poverty, and sizeable equity gains.1 According to official estimates, the annual Gross Domestic Product (GDP) growth averaged 7.2 percent between 2000 and 2016, and the gross national income per capita rose from US$560 in 2001 to US$2,020 in 2019, raising Uzbekistan to the group of low middle-income countries. Social progress was also robust, and poverty declined from 27.5 percent in 2001 to 11.4 percent in 2018. According to the World Bank’s calculations, extreme poverty is currently almost non-existent in Uzbekistan. Those impressive results were driven by an inward-oriented strategy and the dominance of large, often monopolist, state-owned enterprises (SOEs) that were financed by an abundant supply of foreign currency due to improved terms of trade. 2. Since mid-2016, the country embarked upon a complex economic transition. A wide range of reforms, with a clear shift towards a more open, export-oriented, and private sector-driven economy, have been implemented. The objective of the reforms has been to reinvigorate Uzbekistan’s growth prospects, strengthening the foundations of sustained social progress. The economic reforms were part of a more comprehensive vision for the country established by the 2017–2021 National Development Strategy, issued in February 2017, and subsequently detailed by the Reforms Roadmap 2019-2021 in January 2019. 2 Reforms included the unification of exchange rate markets, the reduction of import tariffs for most products, the removal of price controls in many sectors, and the simplification of business regulation – including a significant simplification of the tax system. 3. Significant obstacles to the development of the enterprise sector have been removed or substantively reduced. Structural reforms have tackled significant obstacles to firm growth, as illustrated by the recent tax reform that is already enabling firms to formalize. 3 Early results are reflected in the improvement of business environment indicators, such as those from the 2019 World Bank’s Enterprise Survey rankings. For example, the country was listed among the top 20 global reformers in 2018 and ranked 76th among 190 economies a year later, achieving an improvement of 78 positions compared to 2012. It currently ranks 69th globally, 8th best in starting a business, and 22nd in enforcing contracts. 4. However, the private sector in Uzbekistan is still in its nascent stages. Until 2016, approximately 20 percent of total GDP was produced by fully (100 percent) State Owned Enterprises (SOEs), while purely private firms produced about the same volume.4 Since 1996, economic activity has shifted towards sectors with a lower export propensity and labor productivity, often with a large share of informal firms – as indicated by the share of jobs moving from manufacturing and mining (with above-average productivity) to services (“other services” and “trade and catering”) until 2016. 5 SOE reform has emphasized the introduction of global principles of corporate governance, the reduction of subsidies, and the imposition of hard budget constraints, while major high-productivity sectors have yet to be privatized. 5. The country has also initiated a significant public sector overhaul to improve transparency, accountability, and performance. The Ministry of Finance (MoF) is revising its public financial management strategy. The State Asset Management Agency (SAMA) was established and tasked with the function of enhancing ownership and control over SOEs, including through better corporate governance. A Public-Private Partnership (PPP) Development Agency was created as part of the MoF to house a dedicated cadre of experts in project-finance, improving the quality of PPP deals. A new high- 1 World Bank. Report No. 106454-UZ. Systematic Country Diagnostic for Uzbekistan. May 20, 2016. 2 The Strategy encompassed five priority policy areas: enhance state and public institutions, secure the rule of law and reform the judicial system, promote economic development, foster social development, and ensure personal and public security. 3 See EBRD (2018) for a detailed discussion on the key bottlenecks for private sector development in Uzbekistan. 4 Based on State Committee for Statistics. 5 Uzbekistan Growth and Jobs Report, World Bank (2018). Page 6 of 60 The World Bank Modernizing Uzbekistan National Innovation System (MUNIS) Project (P170206) level Economic Council was also created to advise the President and the Cabinet on the design and sequence of reforms. These measures will improve policy-making, policy coordination, and help create new markets. 6. Uzbekistan also committed to achieving sixteen Sustainable Development Goals (SDGs) by 2030, including for climate change and urban sustainability. Climate change poses a significant risk to Uzbekistan’s economic and social achievements. Anticipated climate impacts include increases in monthly maximum temperatures across Uzbekistan, high variability of rainfall across different agro-ecological and climatic zones, and increased glacier melting with implications for water availability and river flow. The country is also vulnerable to natural disasters, especially earthquakes. A risk assessment developed by the World Bank and the Global Facility for Disaster Reduction and Recovery (GFDRR) Innovation Lab showed that the annual average population affected by earthquakes in Uzbekistan is about 1 million and the annual average affected GDP is US$2 billion. The impact of the COVID-19 pandemic 7. The recent outbreak of COVID-19 exposed the fragility of the country’s achievements. After two decades of sustained poverty reduction, the pandemic could force up to a million people into poverty this year. Based on data from April 2020, the share of households with at least one working member fell by 40%, and employment is well below both 2019 levels and the pre-COVID trend. The self-employed have been the most affected: the share reporting any self- employment fell by 67% in April. Based on Bank projections, the poverty rate could rise from the pre-crisis estimate of 7.4% to between 8.7 to 10%. A sharp decline in migrant remittances (20% lower than 2019) will push those already poor into deeper hardship. 8. Since late-January 2020, the authorities have systematically implemented strong anti-crisis measures to slow COVID transmission, protect livelihoods, and sustain the reform momentum. An early lockdown phase had a significant impact in slowing community transmission, however, the health system remains strained by a second wave of infections in July and August 2020. The Government announced a $1.5 bn anti-crisis fund to achieve three short-term outcomes: (a) saving lives; (b) safeguarding the economy; and (c) sustaining the structural reforms. The Government has significantly boosted health spending, expanded safety nets, and provided time-limited and targeted tax, debt, and cash flow relief to the most affected businesses and sectors of the economy. The authorities have also adjusted their reform plans: measures with a positive social and poverty impact have been accelerated, while those potentially amplifying suffering for the poor have been deferred. Structural reforms without immediate poverty impacts – such as financial sector and SOE reforms – continue to be implemented with assistance from the Bank. 9. Full economic recovery from the COVID-19 pandemic will likely require massive vaccination, but Uzbekistan may not be ready to benefit from a vaccine once it becomes available. Vaccination will be important to enable the return of social interactions and to restore production and consumption patterns without risking a second wave of the disease. Given constraints regarding production capacity and global demand for the vaccine, it is unclear when Uzbekistan would be able to aquire the vaccine in the quantities needed. B. Sectoral and Institutional Context 10. In parallel to the structural and public sector reforms, the Government of Uzbekistan (GOU) has started to revamp the country’s scientific and innovative capacity. 6 In September 2018, the GOU announced a comprehensive strategy to transform the country into one of the top-50 global innovators in the next ten years -- the Strategy for Innovative Development for 2019-2021. 7 The Strategy is led by the Ministry of Innovative Development (MID) and foresees seven intermediate goals, including: (i) improving research excellence; (ii) strengthening the links between education, science, and industry; and, (iii) increasing the quantity and the effectiveness of public investments. As a core result, gross 6 Based on the Background Note “An Assessment of Uzbekistan NIS”, prepared by Iwona M. Borowik, Paulo Correa and Alena Sakhonchick. 7 Presidential Decree 5544 of September 21, 2018. Page 7 of 60 The World Bank Modernizing Uzbekistan National Innovation System (MUNIS) Project (P170206) expenditures in research and development (GERD) are expected to quadruple from 0.2 to 0.8 percent of GDP between 2018- 2021. The Strategy has the strong support of the presidency, and 2020 was declared as the “Year of Development of Science, Education and the Digital Economy (see Annex 1 for further details). These and the other priorities of the Strategy for Innovative Development have become even more important since the start of the COVID-19 pandemic, which necessitates a National Innovation System that is better equipped to potentially produce a vaccine or treatment, and improve preparedness for future pandemics. Moreover, the pandemic has brought to light the need to urgently address other perennial societal challenges, such as vulnerability to climate change and earthquakes. 11. Many initiatives under the Strategy for Innovative Development have already begun. Most noticeably, the responsibility for most research and development (R&D) funds was concentrated in the MID, and a dedicated fund to finance R&D and innovation projects based on competitive selection processes was formed. Technology parks, business incubators, and accelerators programs were created. The State Fund for Support of Business Development was established to provide guarantees on loans of commercial banks to small business entities. In the area of higher education, in addition to starting broad reforms to improve quality, the GOU also adopted measures to improve the relevance to the private sector. For example, initiatives include the ‘One Million Programmers’ project, information technology (IT) training centers, and a dedicated innovation fund to channel competitive grants to Higher Education Institutions (HEI). Uzbekistan’s NIS does not perform according to the requirements of a market-oriented economy 12. Nevertheless, the challenges faced by Uzbekistan’s national innovation system (NIS) are complex and rooted in the legacy of an unfinished transition to a market-oriented model. The term “national innovation system” has existed for more than 30 years and has been widespread among policymakers. 8 To achieve impact, policies depend on a variety of interactions among institutions, markets, and individuals, comprising a “system”. A simplified version of a NIS can be described by three ‘stakeholders’ (government, public research organizations and the enterprise sector), one critical linkage (collaboration science-industry), and the country’s framework conditions. These elements and interactions aim to initiate, import, modify and diffuse new technologies and economically useful knowledge rooted inside the borders of a nation state. The shift towards a market-based economy had a profound impact on Uzbekistan’s NIS. Economic reforms were followed by a period in which conditions for the performance of high-quality research deteriorated. The mechanisms through which science and industry cooperated, based on a close relationship between large SOEs and sector research institutes (RIs), virtually disappeared, as subsidies to SOEs declined and profit-orientation changed SOE’s priorities. Core institutions and functions, including policy-making capacity, weakened (following the general trend of the public sector in the country). Uzbekistan is now dealing with the legacies of the soviet model while it is starting to build the foundations of a modern NIS. Like other institutions in the country, the NIS is at the beginning of a challenging transition, as evidenced by poor results of core performance metrics. 13. Research performance is weak by international standards, but some pockets of excellence are emerging. Both the volume and quality of research performance are low by international standards. Between 1996-2018, Uzbekistan produced 11,328 scientific documents, equivalent to 0.02 percent of the world's scientific research output, of which 11,037 were citable documents. 9 The average citations per paper (6.52) is about half the world’s median (13.25). 10 Yet, the country seems to have made a remarkable improvement in terms of international collaboration, a critical factor to improve research quality. 11 The most notable performances are in the areas of physics and astronomy, biochemistry, 8 The term ‘national innovation system’ is originally defined in Freeman (1987) and Lundvall (1992). 9 These results are above the world’s median for both the number of documents and of citable documents (4,445 and 4,199) but much lower than Kazakhstan’s levels of both variables (23,849 and 23,031). Based on the SJR database. 10 The H-index is an author-level metric, measuring the citation impact and productivity of the publications of a scientific researcher. The index is based on the set of the scientist's most cited papers and the number of citations that they have received in other publications. The country’s H-index, in turn, was 86, close to the world’s median (91) but far from the maximum (USA, 2,222) or the minimum value of the top 50 best performers (Estonia, 255). 11 During the same period, international collaboration increases from about 20 to more than 50 percent of the documents produced. Page 8 of 60 The World Bank Modernizing Uzbekistan National Innovation System (MUNIS) Project (P170206) genetics and molecular biology, medicine, agriculture, and biology. In some categories, the evidence suggests that Uzbekistan may be among the regional leaders, as in the case of mathematical physics for which levels of citations per document in 1996-2018 were only second to Taiwan (see Figure 1). 12 14. Patent activity by Uzbekistan’s residents, an intermediate step towards research commercialization, has been historically low, despite improvements in recent years. Between 2010-18, domestic patent applications by residents increased from 238 to 470, while applications by non-residents remained stable. Patents granted to residents also increased to an average of about 147 per year in the 2017-18 period, up from 115 in 2009-10. Yet, results for international patenting activity, a more robust evidence of patent quality, remained substantively unchanged, with less than five annual Patent Cooperation Treaty (PCT) patent applications between 2010-18 (except for 2014). 13 Patents are neither necessary nor sufficient for innovation but are an indication of the potential, including for research commercialization through licensing and spinoff companies. 15. The share of innovative firms in Uzbekistan is not large, especially among small and medium enterprises (SMEs). The lack of innovative firms hampers the impact of research by reducing the demand for knowledge produced by the system. The lack of innovativeness of existing firms is also closely related to the small share of knowledge-intensive startups and business investments in R&D. The share of firms declaring to have introduced products or services new to their primary market in 2019 (67.2 percent) is consistent with both ECA and global averages (64.7 and 65.1, respectively). However, the share of firms introducing new processes (14.4 percent) is lower than ECA and about half the global average (19.5 and 30.4 percent respectively). Moreover, only 23.2 percent of firms declared to have introduced a new product or service in 2019, compared to 30.8 in ECA and 36.6 percent for the world. The gap is driven by small (5-19 employees) and medium (20-99 employees) firms, as it virtually disappears for large firms. 14 Research funding and the number of researchers do not explain underperformance 16. Research funding has been structurally low, when considering Uzbekistan’s development level, but does not fully explain the low research and innovation performance. On the one hand, Uzbekistan’s performance is better than most countries with similar GERD. On the other hand, some countries, such as Armenia and Azerbaijan, achieved higher citation levels in the 1996-2018 period. So, while data suggests that additional investment in R&D is needed to improve Uzbekistan’s scientific performance, it also indicates that the country could achieve better results at the current investment levels (see Figure 2). In the past two decades, the absolute value of R&D spending in Uzbekistan remained stable in real terms, but the levels of R&D intensity declined from 0.35 in 2000 to 0.13 percent of GDP in 2018. 15 Systematically low R&D spending affects not only current research but also the country’s research capacity over time, as the stock of research capital deteriorates. In fact, GERD per researcher, a proxy for the capital per worker in the sector, varied between US$15-20 thousand over the past decade. 16 Overall, Uzbekistan seems to be on the side of lower-middle- income countries that spend very little on R&D. 17. The quantity of research personnel does not seem to be a significant obstacle for Uzbekistan’s performance. The number of R&D personnel in Uzbekistan slightly increased between 2012-17 to reach 36.7 per thousand (headcount criteria), corresponding to approximately 2.6 per thousand of the labor force – similar to Kazakhstan, but much less than in Turkey (8.5), a moderate innovator. Yet, the estimated number of full-time equivalent (FTE) personnel in 2017 was 44 percent lower, indicating that researchers spend, on average, almost half of their time with other activities – compared 12 Preliminary evidence also suggests that Uzbekistan is among the top 5 performers in Asia in the subject category of Virology 13 In the same period, Kazakhstan PCT patent applications varied from a minimum of 18 to 27 applications in 2019. Based on WIPO database. 14 Based on 2019 WBES. 15 For comparison, lower middle-income countries were spending between 0.43 and 0.47 percent of GDP in the same period. 16 Meanwhile, GERD per researcher varied between US$40 -53 thousand in Kazakhstan and US$95 - 100 thousand in lower-middle income countries. UNESCO Database. R&D per capita, per researchers FTE, in PPP 2005 constant. Page 9 of 60 The World Bank Modernizing Uzbekistan National Innovation System (MUNIS) Project (P170206) to less than 30 percent in Denmark, a leading innovator. 17 Women’s participation has been stable and comparatively large. The share of female researchers was equal to or higher than 40 percent of the total headcount (See Annex 2 for further details). Figure 1: Citations per subject area: Mathematical Physics Figure 2: Citations Index (1996-2018) Source: Staff elaboration based on Scimago data Source: Staff elaboration based on Scimago data and WDI Notes: 1996-2018 for H index and 2017 for GDP; excludes Africa. 18. While the supply of human capital may represent an obstacle over the medium term, there are improvements in the age distribution of researchers. 18 Researchers between the ages of 35-44 and 45-54, considered the most productive, corresponded to almost half the total headcount in Uzbekistan in 2017 (49.2 percent), not too far from Italy (58.7 percent) that has a more mature innovation system. The formal qualifications of researchers are less favorable. Less than one-third of researchers in Uzbekistan (31.9 percent) had a Doctor of Philosophy (Ph.D.) or equivalent level of education in 2017 (International Standard Classification of Education, ISCED, level 8), while the vast majority (61 percent) had a Master of Science (MSc) or equivalent (ISCED level 7). In terms of career development, only 11 percent of researchers are at the highest possible grade/ post of seniority (Category A) in their careers (compared to 36 and 38 percent in Georgia and Egypt, respectively), indicating an opportunity for younger cohorts of researchers. 19 The lack of effective ownership and control in public research organizations (PROs) discourages research excellence 19. PROs are still the most dominant players in Uzbekistan’s research and innovation landscape. There are 28 RIs originally under the Academy of Sciences and 21 under sectoral ministries and committees, including 11 under the Ministry of Agriculture and Water and six under the Ministry of Health. Most of the research funding, about 59 percent of total R&D spending in 2018, was concentrated in PROs, with RIs accounting for the largest share – a core characteristic of the previous research system.20 Researchers are also predominantly employed in PROs, but HEIs account for the largest share 17 Most of the R&D personal is dedicated to research, with staff supporting ‘other activities’ corresponding to about 5 percent the total headcount (10 percent of FTE), better than Chile (14.3) and Serbia (15.6). 18 In 2017, the share of researchers 65 years old or older decreased by more than half to reach 6.4 percent, while the share of young researchers (25- 34 years old) raised to 27 percent of total headcount. 19 ISCED is the UNESCO’s International Standard Classification of Education. 20 PRI corresponds to ‘GOV’ in UNESCO classification. In modern NIS, not only is about 2/3 of research performed by the business sector, but most of the research spending is also performed by the higher education institutions (HEI). Page 10 of 60 The World Bank Modernizing Uzbekistan National Innovation System (MUNIS) Project (P170206) (about 78 percent of total researchers based on headcount criteria), while RIs employ about 18 percent. 21 This mismatch between the employment of researchers and the allocation of funding is symptomatic of the governance challenges faced by the science, technology, and innovation (STI) policy in Uzbekistan. 20. The primary source of research funding for RIs is the competitive grant programs managed by the MID. Until recently, most of the funding for RIs had been allocated through competitive research grants: small, often insufficient amounts of budgetary funds to cover fixed costs such as basic research salaries, maintenance, and overhead. In 2018, with the appointment of the MID as a main stakeholder responsible for the development of innovation, the research funding mechanism changed. The core funding of research institutes is provided by the state budget through the MoF to cover the institutes’ operational cost and salaries, while the main source of funding of research activity is though competitive funds managed by the MID. 22 RIs may also obtain additional streams of funding, for example, from contract research or international programs. A right balance between grant and budgetary funding for RIs is essential because while the first instrument is needed to promote competition and productivity among researchers, the other is necessary to allow organizations to plan for more extensive research infrastructure, such as laboratory facilities. 23 21. Systematically low R&D spending has had significant effects on research infrastructure, especially laboratory facilities. One particularly relevant situation is the absence of a Biosafety Level 3 (BSL-3) laboratory in the country – which limits dramatically the type of research that can be done on COVID-19 and other diseases, including basic steps necessary to identify if there is a subtype of the virus in the population. 24 On a more fundamental level, the Virology Institute has urgent needs in terms of isolation of contaminated areas and maintenance of its ventilation system. Also, the level of collaboration between the Institute of Plant Chemistry and several other RIs active in plant substances and natural plant compounds and the private sector is being limited by the lack of a good laboratory practice facility (GLP), which costs about US$5-6 million. To address those limitations, the MID is considering developing an infrastructure research strategy for Uzbekistan, including better regulation of access to research facilities by the broad community of researchers, such as that of the European Research Infrastructure Consortium (ERIC) regulation. 22. Despite its importance, the exercise of ownership and control by the government is weak. This partly reflects the overall situation of the public sector’s governance, but it is further compounded by the relatively large number of RIs. The disproportionally large number of PROs may represent an opportunity for consolidation – with benefits in terms of lower overhead costs and better exercise of ownership and control, as in the case of Germany. The RIs, originally affiliated to the Academy of Sciences, currently report to the MID. Nevertheless, the Academy still appoints the director-general of the institutes. Those institutes must provide an annual report to the MID. However, the standard instruments for encouraging adjustment or performance improvements are not available (such as performance-based contracts with the institutes or results-based contracts with top managers). 23. Proper tools for effective management of RIs are missing. Directors lack the empowerment necessary to manage for results, such as fixed mandates and administrative autonomy. In addition, the GOU lacks the mechanisms to hold the RIs and their directors accountable. Results-orientation is precarious as research programs are predominantly decided on 21 These results change slightly when FTE is considered, with HEI’s share decreasing to 49 percent of total and research institutes’ share rising to 26 percent of the total, possibly reflecting the HEI’s researcher dedication to teaching activity. 22 The MID has begun reforming the science funding system. Now it is a competitive selection of R&D proposals on a rolling basis (with calls for proposals announced online every two months) and based on the priorities and needs of the national economy. The Ministry has stepped up funding (the average size of grants tripled and reached US$80,000) and at least half of the allocated funds are now used by beneficiaries to purchase R&D equipment. 23 The government has also launched short-term (3-month) foreign internships for researchers. The Ministry has also launched calls for proposals for joint research in cooperation with German, Chinese and Turkish research institutes and universities.. 24 Biosafety Level 3 (BSL-3) corresponds to the level of biosafety necessary when working with risk group level 3 microorganisms, according to the regulations of the US Centers for Disease Control and Prevention. Viral isolation and initial characterization of viral agents found in cultures of SARS- CoV-2 infected specimens should only be contained in BSL-3 facilities. Page 11 of 60 The World Bank Modernizing Uzbekistan National Innovation System (MUNIS) Project (P170206) a ‘curiosity basis’ by scientific committees. Research performed by HEIs shows the same structural limitations, with the topic being selected mainly according to the individual interest. The link between career progress and research excellence is not strong enough – at best in the format of the number of citations (quantity) without properly weighting the impact factor of the publication (quality) in some HEIs or reduced to a financial bonus in some institutes. The use of patents as a proxy for publications (a metric for research achievement), a practice prevalent in countries with robust NIS, is not used. Intellectual property (IP) laws and other regulations do not sufficiently encourage science-industry collaboration 24. Promoting research commercialization is among the most challenging yet relevant reforms to align incentives to the desired outcomes. 25 Research commercialization is not an automatic process. Transitioning from research results to a patent or a spillover is a process that faces many market and institutional failures. Funding for early stages of research commercialization (such as the preparation of proof of concept or prototyping) tends to be underprovided by the private sector. Scientific career development rules often discourage the allocation of researchers’ time towards commercialization efforts. Uncertainty over the ownership of the IP reduces the incentives for researchers to allocate time for research commercialization and increases the risks of early-stage investors. Finally, the expertise needed for this type of activity is still evolving and hardly available. 26 25. A weak IP regime and understanding of international IP laws lead to poor commercialization of outcomes. It may also prevent the country from benefitting from the promise of the Agreement on Trade-Related Aspects of Intellectual Property Rights (TRIPS) provisions for COVID-19. Practitioners point to nebulous and inconsistent IP laws as one of the reasons why IP ownership is often uncertain. Furthermore, IP provisions in various PROs are not uniform, and there is no country-wide policy on IP rules for commercialization and contract research. This leads to PROs not fully capturing the economic gains and proliferating their research through effective licensing. An IP landscape analysis to inform research – a practice that helps researchers prevent duplication of efforts and identify potential collaborators and licensees – is not undertaken. Moreover, there appears to be little understanding of TRIPS provisions for public health, which can be utilized for addressing the COVID-19 pandemic. 26. Presidential Decree N. 3855 27 that regulates the commercialization of publicly funded R&D and other measures go in the right direction, but not enough. For example, it is unclear whether the Decree will create the proper incentive regime for researchers, given existing rules for career advancement; or will eliminate all uncertainty related to IP ownership to enable private investment. Moreover, commercialization in practice is hampered by the lack of expertise in IP valuation, for example, for the case of determining the equity share of a RI in a joint venture with specialized organizations, such as technology transfer offices, and funds. The innovation grants, while specifically aiming at commercialization, represent the smallest share in total grant funding (about two percent of the total amount of publicly funded grants). Some RIs consider that innovation grants should be significantly higher to cover the existing technological gap and additional costs, such as those related to marketing or IP protection. With the private sector in its infant stages, firm capabilities are still evolving 27. Basic firm capabilities are still evolving, reflecting the private sector’s early stages of development. The private sector is young and predominant in industries that did not benefit from the process of capital accumulation in past decades, through which large SOEs in the manufacturing and mining sector often modernized in combination with the 25 Research commercialization is the process through which academic knowledge (and more specifically research results) is transferred from PROs to the enterprise sector. It is a potential source of ideas for innovation from existing firms or the startup of knowledge-based enterprises. 26 Technology transfer offices are one common attempt to address this last challenge, but without the right incentives and appropriate access to funding and expertise may easily become bureaucratic organizations. 27 Presidential Decree N. 3855 of July 2018 regulates the commercialization of publicly funded R&D results. Specifically, two ways of commercialization are described: i) the sale of goods or services produced as a result of R&D commercialization, and ii) the creation of commercial firms with the participation of R&D organizations in their equity by means of their intellectual property. In addition, starting October 1, 2018, additional one-time bonus payments were introduced for authors of discoveries patented outside Uzbekistan in the amount of 10 minimum monthly wages.. Page 12 of 60 The World Bank Modernizing Uzbekistan National Innovation System (MUNIS) Project (P170206) inflow of foreign capital. Uzbekistan’s firms are, on average, six years younger than companies in ECA, and their top managers have five years of experience in the sector, compared to 15.4 in ECA. Very few firms have financial statements reviewed by external auditors (11 percent, compared to 31.0 and 47.6 percent for ECA and the world, respectively), which not only is an obstacle for obtaining a bank loan, but also an indicator of how well the firm can measure its performance. 28. Uzbekistan’s private sector seems to be comparatively better with regards to the subset of technological capabilities.28 Domestic firms of all firm sizes are more likely to license foreign technology (19.35 percent) than their peers in ECA and the world (14 and 12.9 percent, respectively). The share of domestic firms investing in R&D is slightly below the ECA average but significantly above the world average (33, 36.5, and 22.9 percent, respectively), while firms with foreign participation outperform both comparators. The share of small firms (5-19 employees) investing in R&D is inferior to that of ECA, but the proportion of medium (20-99) and large (+100 employees) size firms is larger. 29. Nevertheless, evidence on firms’ capacity to conform to international standards is surprisingly unfavorable. The adoption of quality certification, an indicator of firm capacity to comply with international standards, is remarkably low: 8.3 percent of firms in Uzbekistan report having quality certification, much lower than both ECA and world averages (21.6 and 15.3 percent, respectively). The gap is driven predominantly by domestic firms and SMEs. This reflects, in part, Uzbekistan’s low integration in global value chains, where demand for quality standards tends to be higher (only 5 percent of firms export more than 10 percent of their sales, compared to 14.4 percent in ECA). 30. A key factor for the low levels of quality certification is the inadequacy of existing metrology, standardization, testing, and quality services (MSTQ). The MSTQ services in Uzbekistan are led by the Uzbek Agency for Standardization, Metrology and Certification (Uzstandard). Uzstandard has recently initiated a series of actions to converge to international best practices—although the results are uneven. The goal was to reform obsolete regulations, improve the institutional structure, and strengthen the technical capacity of the main organizations. 29 While some of the elements of a modern regulatory framework were introduced, such as the adoption of technical regulations or the creation of the accreditation agency, these elements are not always adequately designed or implemented, undermining the sustainability and overall impact of reforms. The transition towards a modern, market-friendly national quality infrastructure framework is still at the early stages. 31. Uzbekistan’s authorities have also been taking a series of measures to revamp the local pharmaceutical industry. In 2017, Uzfarmsanoat, the holding controlling 70 companies in the pharmaceutical industry (including research laboratories and plants dedicated to vaccine production), was liquidated and replaced by the Agency for Development of the Pharmaceutical Industry (ADPI) under the Ministry of Health. In parallel, several tax-relief measures were adopted to promote local production and encourage foreign companies to locate in free economic zones (FEZs) in selected regions. In April 2019, Presidential Decree 5707 adopted new measures to accelerate the development of the sector, including favoring local firms in public procurement. 30 Strengthening the capacity of the local industry could expand the government’s options for procuring a COVID-19 vaccine in the future, given the growing uncertainty about the global supply. Policies to transition to a market-oriented NIS 32. The transition to a market-oriented NIS will be long and complex. Transitioning to a market-based NIS implies not only strengthening the contribution of research to generating potential solutions to societal challenges but also 28 The source of firm-level data used in this report is the World Bank Enterprise Survey, unless mentioned otherwise. 29 Reforms include the development of new laws on standards, metrology, technical regulations, conformity assessment and certification. Both the accreditation agency and the National Institute of Metrology are working to obtain international recognition. 30 From July 2019 to December 31, 2020, all applications for the supply of import medicines in the public procurement process will be rejected if there are two or more local participants that produce similar products. Other measures include subsidizing 50 percent the cost of registration of products in foreign markets and creating a state fund to support infrastructure investments in research institutes and state laboratories, covering expenses of local pharmaceutical producers for pre-clinical and clinical research and other purposes Page 13 of 60 The World Bank Modernizing Uzbekistan National Innovation System (MUNIS) Project (P170206) fostering the impact of public investments in STI on economic growth by unclogging the channels through which it happens: science-industry collaboration and private sector innovation. Effective science-industry collaboration is vital because it improves research relevance and creates the right conditions for research commercialization – the way through which research translates into innovation and productivity growth. To achieve those outcomes, governments need to combine well-designed policies, including on public funding, with an institutional environment conducive to the right actions by the NIS stakeholders. In sum, under the appropriate framework conditions, STI policies and reforms that correct for market and institutional failures intrinsic to research and innovation activities (uncertainty, externalities, and coordination) will contribute to economic growth and the country’s societal challenges (see Figure 3). 31 33. Building a market-oriented NIS in Uzbekistan will require the development of new policies, funding, and institutional reforms. Choosing the country’s ‘right’ policy mix, calibrating for complementarity, consistency, and overcoming political obstacles can be challenging. 32 The proposed project will help the GOU address some of those challenges, thereby contributing to the development of a market-oriented NIS in the country. The following are seen as some of the core policy/reform priorities: (i) improving research excellence and commercialization to increase the impact of R&D funding on solutions to societal challenges and economic growth; (ii) Promoting private sector innovation with a policy mix that reflects the scope of firms’ capability needs, from support to SME’s incremental innovation (quality) to mentorship services for early-stage projects and startups; and, (iii) Supporting STI policy-making and reforms, particularly about the adoption of international best practices for the design and implementation of meritocratic, transparent and accountable funding schemes for research and innovation; the establishment of a dedicated and autonomous organization to manage those funds; and the development of instruments to better management of STI policies, including a monitoring and evaluation system (M&E) that allows for systematic learning and adaptation by policy-makers. 34. The renewed emphasis on STI policies is consistent with the priorities of a lower-middle-income country. STI policies come as a complement to a strong reform program. Current levels of R&D in Uzbekistan correspond roughly to the average investments of countries with a similar per capita income, but less than a quarter of the investments performed by China when the country was at a similar development level. Other resource-intensive countries, including those with a large share of exports from agriculture, such as Australia and Chile, also show significantly higher levels of investment in R&D. Countries transitioning from a middle-income level often require a gradual increase in public investment in STI. Naturally, this requires balancing the gradual development of Uzbekistan’s STI capacity with the needs of the country in its current development stage. 35. The renewed emphasis on science can also address some of Uzbekistan’s pressing needs. Proper investment in science may help the country address many of its societal challenges, such as climate change, natural disasters, and global diseases. The effects of climate change are better understood when global knowledge is complemented by country- specific studies – leading to tailored adaptation or mitigation measures. A research program on geology, seismology, and new materials can unfold new, incremental ways to make buildings more resistant to earthquakes. Uzbekistan can benefit significantly from strengthening its knowledge of the local dynamics of viral diseases. Such investments can contribute more broadly to the achievement of the national SDGs, as acknowledged in the United Nations Agenda 2030. 33 Research 31 On the positive side, the broad country’s framework conditions have been gradually improving as structural reforms advance. In fact, some initial results are already reflected in improvements of business environment indicators, those from the 2019 World Bank Enterprise Survey rankings. 32 Cirera, Xavier; Maloney, William F.. 2017. The innovation paradox: developing-country capabilities and the unrealized promise of technological catch-up (English). Washington, D.C : World Bank Group 33 In fact, the United Nations Agenda 2030, unanimously adopted at the United Nations Sustainable Development Summit in September 2015, positioned S&T policies as key means of implementation of the SDGs.1 UN Agenda document Page 14 of 60 The World Bank Modernizing Uzbekistan National Innovation System (MUNIS) Project (P170206) is also an indispensable complement to higher education activities, another goal of the current National Development Strategy. 34 Figure 3: The impact of public investment in R&D through a market-oriented NIS – a simplified view Source: Staff elaboration C. Relevance to Higher Level Objectives 36. The proposed project is fully aligned with the GOU’s National Development Strategy for 2017-21. The National Development Strategy aims to achieve high and sustained rates of inclusive economic growth and job creation through economic and public sector reforms. On the economic side, the Strategy’s immediate goals are to increase international competitiveness, private sector and export-oriented growth, and industrial diversification. Public sector reform aims to reshape the whole of the state, reducing its role as producer of goods and services in favor of policymaking, regulation, and provision of public services to the population. The proposed project will directly contribute to both the economic and public sector reforms. 37. The project is aligned with the World Bank Group’s (WBG) twin goals as well as with the objectives of maximizing finance for development and creating markets. By contributing to sustained growth and job creation, the project is consistent with the WBG’s dual goals of shared prosperity and eradicating extreme poverty. By supporting STI policies and reforms, the project will also contribute to maximizing finance for development as it will leverage private 34 As increased emphasis on science and research go hand in hand with efforts to enhance higher education’s quality and relevance in the country. Higher Education Note: Improving Higher Education’s Quality and Relevance. Policy Note for the Economic Council. World Bank 2019. Mimeo. Page 15 of 60 The World Bank Modernizing Uzbekistan National Innovation System (MUNIS) Project (P170206) sector investments in innovation. It is also consistent with creating markets – directly, as innovation is the creation of a new market, or indirectly, as innovation policies and institutions enable the development of such new markets. 38. The project is fully aligned with the WBG COVID-19 Crisis Response Approach Paper “Saving Lives, Scaling-up Impact and Getting Back on Track”. The project directly supports Pillar 4 of the Approach Paper, Strengthening Policies, Institutions and Investments for Rebuilding Better, by supporting the development of a National Innovation System that is transparent, equitable, sustainable, and has the potential to produce solutions to achieve several of the SDGs. The project will also contribute to Pillar 3, Ensuring Sustainable Business Growth and Job Creation, through its direct support to the private sector. Finally, the project’s flexibility to support the local production of a COVID-19 vaccine, should the need arise, relates to Pillar 1, Saving Lives. 39. In response to the COVID-19 pandemic, the Bank’s priority in FY21-22 is to sustain Uzbekistan’s ambitious reforms in the context of a new post-COVID normal. New and existing operations are being aligned to the WBG COVID- 19 Approach Paper: on-going investment projects have been or are being adjusted to strengthen resilience and support recovery; two loans have been reduced/partially canceled to release funds to be reprogrammed towards recovery; front- loading up to $650 mln under IDA19 is being considered to support the anti-crisis response; and more than a quarter of 15 Advisory Services and Analytics (ASA) to be delivered in FY21 are focusing on supporting the GOU in shaping relevant policies in the post COVID period, including a comprehensive poverty reduction strategy. 40. To save lives and protect the poor and vulnerable, in April 2020, the Bank approved a $95 mln IDA emergency COVID project (P173827) augmented with a $4.3 mln Pandemic Emergency Facility grant to shore up health and safety net systems, as well as a $200 mln supplementary Development Policy Operation (P173948) to overcome a short-term cash crunch following unprecedented health and social expenditure demands. 41. To save livelihoods, preserve jobs, and ensure more sustainable business growth and job creation, the Bank is working with the government to accelerate implementation of the new Agriculture Modernization (P158372) and Ferghana Valley Rural Development (P166305) projects, as these offer immediate opportunities to expand rural access to credit and financing amidst COVID-19 financial disruptions. EU-funded grants of $7.6 mln under the Horticultural Development Project (P164226) are being repurposed to target farmers affected by the crisis. $92 mln in the Regional Roads (P146334) project will be used to boost temporary jobs for unemployed workers, and adjustments to the Modernizing Higher Education Project (P128516) supported emergency purchases of distance learning modules to maintain tertiary instructional delivery. 42. To strengthen policies, institutions and investments for resilient, inclusive, and sustainable growth, the Bank will deliver an electricity transmission and modernization project ($300 mln), and three projects to transform financial sector ($15 mln), tax ($60 mln), and statistical ($50 mln) institutions. The Bank is also preparing a standalone DPO (tentatively $500 mln IDA/IBRD) to support ambitious reforms that would aim to remove factor market constraints, reform and privatize SOEs, and accelerate measures to transform agriculture. The DPO will also focus on “building back better” through reforms to the social safety net system and measures to build a cleaner and greener economy. The IFC is also working on investments and advisory services in renewable energy generation, chemicals and fertilizers, agrifinance, cotton production, and banking. 43. Partnerships and coordination of COVID response. To respond to the COVID-19 crisis, sustain the reform momentum, and fill the domestic financing gap in 2020, a socio-economic task force, led by the World Bank and UNDP, was created at the request of the government to coordinate partner assistance in shoring up the government’s health, social, and economic policy response to the crisis. Through this and other coordination platforms, the Bank ensured that the FY21 pipeline remains focused on reform areas of strong government demand for Bank’s knowledge and high relevance for economic recovery and transition. In addition, and despite turbulent global financial sector conditions, Uzbekistan remains an area of active interest to private sector investors. This is especially the case in the energy sector, Page 16 of 60 The World Bank Modernizing Uzbekistan National Innovation System (MUNIS) Project (P170206) where the World Bank Group plays a leading role along with ADB and EBRD in supporting increased private sector participation through demonstrative projects such as this proposed operation. 44. The project is also aligned with the revised Country Partnership Framework (CPF) FY2016-2020 Report No. 105771-UZ and complements the current portfolio, including the Uzbekistan Emergency COVID-19 Response Project. 35 Support to private sector innovation aligns the project with Area 1 “Sustainable transformation towards a market economy,” as described in the Performance and Learning Review of the CPF for FY19-21. By strengthening research and innovation policies and institutions, the project is also aligned with Focus Area 2 “Reform of selected state institutions and citizen engagement.” While responding to the CPF’s higher-level goals to support the growth of the private sector and the creation of markets, the project complements other institution building and reform support projects – such as the series of development policy operations, the Institutional Capacity Building, and the Agricultural Modernization projects. Finally, the project complements the measures supported by the Uzbekistan Emergency COVID-19 Response Project that focuses on health policies to prevent, detect, and respond to the threat posed by COVID-19. The proposed project will enable a better contribution of the local private sector by supporting innovation that could be used in the local Personal Protective Equipment (PPE) industry or, eventually, to produce COVID-19 vaccines locally if needed. II. PROJECT DESCRIPTION A. Project Development Objective (PDO) PDO Statement 45. The PDO is to enable the development of a market-oriented national innovation system in Uzbekistan. PDO Level Indicators 46. The PDO level indicators are: (i) Papers published in internationally peer-reviewed journals, based on research funded by the project (target: 50 citable documents); (ii) Investment in research commercialization and innovation, mobilized by the project (target: US$ 24 million); (iii) SMEs with internationally recognized certification, funded by the project (target: 90 SMEs); (iv) Regulations disciplining the allocation of competitive funds for research and innovation and reflecting international best practice, adopted by MID. B. Project Components 47. The project comprises three main components. Component 1 will focus on improving research excellence and commercialization, Component 2 will help promote innovation in the private sector, and Component 3 will support policy- making and reforms and strengthen the capabilities of the public sector in the area of STI policies. Components 1 and 2 will finance pilot programs designed and implemented in accordance with international best practices, building policy- making capacity, and generating ‘quick wins’ to demonstrate the potential economic and social value of a market-oriented NIS. Technical assistance (TA) activities in Component 3 will provide some of the necessary means for the introduction of 35 The CPF was revised following a Performance and Learning Review in June 2018. Page 17 of 60 The World Bank Modernizing Uzbekistan National Innovation System (MUNIS) Project (P170206) better policies and the implementation of institutional reforms required for the development of the NIS. Component 4 corresponds to the activities related to project management and monitoring. 48. The project addresses a core challenge faced by developing countries in building modern NIS: managing complex policies and reforms with scarce public resources. In this sense, the project is ‘foundational’ as it aims to help form and strengthen the main innovation policy pillars necessary to build a market-oriented NIS. Given this foundational nature, the project pays special attention to sustainability issues. Component 3 TA activities will facilitate the ‘graduation’ of project pilot programs into full government budget-funded programs and the transformation of temporary organizational structures into permanent organizations of the NIS. 36 Notably, the project design reflects a delicate balance between the necessary details for project implementation and the flexibility required for adjustments to an intrinsically uncertain reform environment. 49. The project’s design incorporates robust governance of the funding mechanism pilots, a minimum level of comprehensiveness to achieve impact, and adjustments to reflect the country’s current stage of development. Meritocracy, accountability, and transparency form the basis of modern research and innovation funding schemes. The decision-making processes will be based on clear and objective criteria reflecting scientific excellence or innovative potential. They will follow a rigorous assessment by independent technical experts, protected from undue political interference. Some degree of centralization is recommended to enhance the consistency of program decisions. 37 To ensure the autonomy of decision-making, an International Expert Board (IEB) will be responsible for the decisions on funding allocation under those programs (see Component 4 and Section III.A), 38 and a transparent grievance redress mechanism will be established by the PMU. The governance attributes are or will be detailed in draft program manuals. 39 50. In addition to better governance, the project design also reflects the fact that the impact of research and innovation programs often depends on several market and institutional conditions that may be absent. Component 3 will help address those missing conditions with TA activities. Notably, to overcome the ‘valley of death’ inherent to research commercialization, Subcomponent 1.2 is dedicated to the initial stages of research commercialization, and Subcomponent 2.2 is dedicated to later stages. Finally, given the current conditions of Uzbekistan’s NIS and overall development stage, the project emphasizes mission-oriented research and includes a component on firm capabilities. Overall, the project design reflects closely the experience of other STI operations in ECA in the past decade, as well as the recommendations of the recent World Bank flagship studies (see Section II.F). 40 Component 1: Improving research excellence and commercialization (US$20 million). 51. The objectives of this component are to improve the capacity of public research to achieve international standards of research quality and enable the commercialization of research results. It comprises three subcomponents. 36 For example, subcomponents 1.1 and 1.2 are expected to be eventually merged into a national science funding organization following international good-practice, while subcomponent 2 could be transformed into a national innovation agency. This transition is envisaged to be completed at least one year before the project’s closing date. 37 For further discussions on this topic see Morgan Jones, Molly, Louise Lepetit, Joachim Krapels, Catherine A. Lichten, Anton Spisak, and Catriona Manville, Organising for excellence: An international review of good practice in organizational design and governance of research funding bodies. Santa Monica, CA: RAND Corporation, 2018. 38 Those decisions are supposed to be final. In exceptional cases, the MID will be able to modify those decisions. This will require the presentation of a written justification to the IEB and its express concurrence. The justification and IEB decision will be publicly available. 39 Those program manuals complement standard project operational documents, including the Project Operational Manual (POM), the Financial Management Handbook, and the PPSD. 40 Cirera, Xavier; Frias, Jaime; Hill, Justin; Li, Yanchao. 2020. A Practitioner's Guide to Innovation Policy : Instruments to Build Firm Capabilities and Accelerate Technological Catch-Up in Developing Countries. World Bank, Washington, DC. World Bank.” Page 18 of 60 The World Bank Modernizing Uzbekistan National Innovation System (MUNIS) Project (P170206) 52. Subcomponent 1.1. Research Excellence Program (REP, US$10 million). The REP will finance research sub- projects submitted by researchers from RIs or local universities. The REP will emphasize joint research proposals from local and international researchers (including Uzbek scientists abroad), 41 joint research proposals from researchers and the private sector, and proposals from young (junior) researchers (aiming to strengthen the country’s future research capability). Selection will be based on the two core criteria: (i) scientific merit; and, (ii) social and/ or economic relevance, for example, the potential contribution to solving some societal challenge or on its commercial application. In particular, the REP will encourage proposals in the fields of renewable energy, energy efficiency, and low-carbon technologies. It is also envisaged that one of the initial project calls will target COVID-19-related research. 53. Operational procedures. The REP will be guided by the REP Manual and implemented by the PMU, which will follow strict procedures of meritocracy adopted internationally — for example, those followed by the European Research Council (ERC), including international peer review. During the development of the program, open consultation forums 42 in RIs and HEIs will be established to engage local and international researchers, junior researchers and develop detailed understanding of necessary changes to improve the effectiveness of transparency and meritocracy in funding allocation (e.g. size of the grants, fields of the proposals). Grants for junior and senior researchers would go up to US$200,000 and US$600,000, respectively. 43 Each call will have the flexibility to establish lower maximum values to encourage more participation and better demonstration effect. Eligible candidates must belong to a RI or a HEI. Implementation time is expected to be up to 24 months. Collaborative research will be supported by larger funding and extended periods. The program may also target preliminary research required for applications to international research funds for which the country qualifies (such as for the Framework Programs of the European Research Area). When complete, the project will support activities to obtain feedback on the REP and establish a vehicle, such as a committee, for deliberation over the results. All procedures will be set out in an approved REP Manual. The REP is expected to be the starting point of a National Science Foundation. 54. Subcomponent 1.2. Research Commercialization Program (RCP, US$4 million). 44 The RCP will (i) finance research commercialization sub-projects on the development of proof-of-concept and small-scale prototyping, and (ii) provide mentorship to sub-projects with commercialization potential. The RCP is expected to attract and support research commercialization sub-projects from RIs and universities. Mentorship activities will offer individualized training and coaching, as well as facilitate the matching between potential candidates and prospective enterprises/ investors. Component 3 will support complementary regulatory/ legal reforms necessary for effective research commercialization. 55. Mentorship to sub-projects: MID will establish a Technology Transfer Unit that will provide tailored capacity- building for RCP sub-projects on varying themes, including market sizing exercises, developing commercialization concept, sales and marketing, team building, investment and finance, presentational and pitching skills. In view of the nascent market for investment readiness services in the country, a reputable international contractor will be competitively hired to help build capacity of the TTU and thus build a strong cadre of technology commercialization specialists in the country. The contractor’s key tasks include (i) development of capacity of the TTU staff, including training and supervising the TTU commercialization managers to ensure regular knowledge transfer; (ii) drafting RCP manual based on analysis of gaps in 41 International collaboration calls will also target the scientific Diaspora, following the experience of Croatia’s Unity Through Knowledge Fund. 42 Calls for young researchers will be informed by the Young Researchers Grant Program experience of the ERC. 44 This subcomponent will help to transfer research results to the market, a process that is currently impeded by many market and institutional failures. For further discussions see Zuniga, P. and Correa, P. (2013). Technology Transfer from Public Research Organizations: Concepts, Markets and Institutional Failures. Innovation Policy Platform Policy Brief. Page 19 of 60 The World Bank Modernizing Uzbekistan National Innovation System (MUNIS) Project (P170206) existing commercialization practices; (iii) support TTU in implementing RCP (selecting RCP beneficiaries, providing peer review, help developing monitoring mechanisms etc.); (iv) provide mentoring to RCP beneficiaries. 56. Operational procedures. 45 (i) Research Commercialization Grants: funding will be in the format of small grants varying from US$75,000 to US$150,000 for a period no longer than 24 months. Within those boundaries, each call will have the flexibility to adjust its parameters depending on the stage of research commercialization, the field of science, and related factors. 46 Research projects will be selected on a competitive basis, according to their scientific and commercial potential, with a possible target of reaching at least 35 percent female beneficiaries. (ii) Mentorship to sub-projects: The subcomponent will also finance the establishment of a Technology Transfer Unit (TTU) under the MID to oversee the implementation of the RCP. A specialized entity (e.g., the Larta Institute, IC2, or other) will be selected47 for, inter alia, (i) building capacity of the TTU in implementing RCP, (ii) drafting the RCP Manual, (iii) supporting selecting the beneficiaries, and (iv) providing mentorship to Research Commercialization Grants Program beneficiaries, (all under the Project Management Unit’s -PMU- supervision). 48 Decisions on the selected projects will be submitted to the IEB for concurrence. Jointly with the hired entity, the TTU will work directly with the researchers (possibly through online feedback processes) to explore the potential for their commercialization. It will also coordinate the provision of training for existing technology transfer offices or similar organizations of RIs and universities, as well as other TA activities in the field of research commercialization. The contractor will also provide detailed recommendations on sustainably maintaining the RCP after the completion of the engagement. Inputs from the targeted beneficiaries (researchers, enterprises, investors) of the RCP will be sought through an online consultation platform to better understand needs for subgrants (amounts and fields), the type of mentorship, and services to be offered by the TTU. 57. Subcomponent 1.3. Public Research Institutes Modernization (PRIM, US$6 million). This subcomponent will finance investment in the modernization of infrastructure of selected public research institutes. The selection of research institutes (RIs) will be undertaken by the MID after full assessment of the IMP of all RIs candidates. 49 The design of the IMP will be supported by TA based on growing international experience of improved governance of RIs, as in the cases of Germany, Italy, and New Zealand, as well as open consultation and engagement forums with the researchers and other stakeholders of the NIS 50 Access to the infrastructure research funding will be conditional on the acceptance of an IMP by the MID. Participation will be voluntary, and any RI is eligible. 58. Operational procedures. IMPs are not expected to be comprehensive but should be robust enough to generate a substantial improvement in research excellence or research commercialization. Proposed measures should be conceived as part of a long-term reform strategy but be commensurate with the resources available for the activity and completed 45 This funding scheme follows closely the Small Business Technology Transfer (STTR) scheme provided under the US Small Business Innovation Research (SBIR) program. 46 For example, proof-of-concept calls often require lower co-funding amounts and offer a lower amount of total funding as compared to prototyping. In addition, the amounts and the time required for the development of proof-of-concept in the software industry tends to be lower than in the biotech sector. 47 The selection of the entity to implement the RCP will be competitive, based on the costs of service, the quality of the mentorship program, and demonstrated results (see Section III.A). 48 The Project Management Unit will be responsible for fiduciary responsibilities and administrative tasks. 49 The research infrastructure funded by the program will be consistent with a broader research infrastructure strategy to be developed under Component 3. It will be subject to the regulation of access norms established internationally, such as those of the European Research Infrastructure Consortium (ERIC). Lastly, it will contain an assessment of maintenance costs and budgetary implications acceptable to the MID. 50 See OECD (2014). STI Policy Profiles: Universities and Public Research Institutes. OECD, Paris; and OECD (2011), Pubic Research Institutions: Mapping Sector Trends. OECD, Paris. The sub-component will also follow the experience under the Serbia Innovation Project (SIP). Page 20 of 60 The World Bank Modernizing Uzbekistan National Innovation System (MUNIS) Project (P170206) within a short period (e.g., 6-12 months). Research infrastructure projects will be accompanied by feasibility studies acceptable to the World Bank. Where relevant, IMPs may be requested to include plans to diversify the gender composition of the management and administration. Component 3 will support the hiring of a consultant to advise the MID on the design and the review/ acceptance of the IMP. The IMP framework may include a guideline to target the enhancement of participating RI’s climate change mitigation or adaptation research capacity, as part of institutional modernization plan. 59. During the pre-appraisal mission, the project team assessed some of the RIs based on the (i) quality/ relevance of their research; (ii) preparedness to adopt administrative reforms; and, (iii) the availability of clear proposals for the modernization of their respective research infrastructures. 51 Among several potential candidates, the team undertook a preliminary assessment of three potential candidates for pilot programs, namely the Institute of Virology, the Institute of Seismology, and the Institute of Plant Chemistry. In particular, the project envisages the possibility of financing the acquisition of a mobile BSL-3 laboratory for COVID-19-related research (which would also be necessary for research on other viral diseases). 52 Component 2: Promoting innovation in the private sector (US$20 million). 60. The objective of this component is to increase private sector innovation in Uzbekistan. It comprises two subcomponents reflecting the dual needs of Uzbekistan at the current stage of development, namely enabling technology adoption by firms and diffusion and supporting the initial growth of R&D-driven innovation, as well as the expansion of a knowledge-based startup sector in Uzbekistan. 61. Subcomponent 2.1. Innovation Capabilities Program (ICP, US$10 million). The objective is to enable SMEs to develop new or improved products, services, and processes, enabling their expansion in domestic, regional, and global markets. The ICP will finance (i) supplier development matching grants for the adoption of quality standards by domestic SMEs, 53 and (ii) targeted technical assistance and purchase of equipment to support modernization of selected parts of 51 Based on the Background Note “Review of Potential Candidates to the Public Research Institutes Modernization Program”, prepared by Ravi Gupta. 52 The BSL-3 laboratory corresponds to the minimum level of biosafety recommended by the Centers for Disease Control and Prevention for COVID- 19 when performing scientific research, including viral isolation and initial characterization of viral agents found in cultures of SARS-CoV-2 infected specimens as those needed in Uzbekistan. The total (construction and equipment) costs of a 30 m2 modular BSL-3 laboratory was estimated at US$ 400.000 in 2009, with annual maintenance budget at about 10-15 percent of the facility construction costs. Compared to a standard construction, the modular option could be built in a relatively short period of time and cost less to build and maintain. See Ssengooba W, Gelderbloem SJ, Mboowa G, Wajja A, Namaganda C, et al. (2015) Feasibility of establishing a biosafety level 3 tuberculosis culture laboratory of acceptable quality standards in a resource-limited setting: An experience from Uganda. Health Research Policy and Systems 13:4; and Heckert RA, Reed JC, Gmuender FK, Ellis M, Tonui W (2011) International biosafety and biosecurity challenges: suggestions for developing sustainable capacity in low-resource countries. Applied Biosafety 16: 223-230. Bridges, D. J. , James Colborn, Adeline S. T. Chan, Anna M. Winters, Dereje Dengala, Christen M. Fornadel, Barry Koslof Modular Laboratories—Cost-Effective and Sustainable Infrastructure for Resource-Limited Settings. The American Journal of Tropical Medicine and Hygiene, Volume 91, Issue 6, 3 Dec 2014, p. 1074 - 1078 . 53 Supplier development initiatives or programs aim to stimulate linkages between local SMEs and large local and foreign companies. Such programs pilot a group of targeted and coordinated interventions to support local SMEs to increase their sophistication, compete more effectively and integrate into the global and regional supply chains of the large companies. International experience suggests that SDPs can make a substantial impact in generating increased domestic value added. The Czech and Serbian experience showed that having SDP in place is a win-win for all parties involved in the process (Government, foreign investors and local companies). Countries that attract Foreign Direct Investments (FDI) but do not develop linkages do not fully benefit from FDI; SDPs are instrumental in ensuring spillovers and attracting additional FDI. For a review of supplier development programs see World Bank. 2008. International experience in supplier development (English). Washington, DC: World Bank and on the experience of North Macedonia, as documented in Metz, Melissa; Popovik Friedman, Gordana; Adamon, Karamath Djivede Sybille; Vrboski, Perica; Bilotserkivska, Iryna. 2017. FYR Macedonia supplier development pilot program toolkit : materials and lessons learned (English). Washington, D.C. : World Bank Group. Page 21 of 60 The World Bank Modernizing Uzbekistan National Innovation System (MUNIS) Project (P170206) the national quality infrastructure 54. Modernization efforts would include streamlining and implementing procedures for certification, accreditation and conformity, upgrading of laboratory facilities for metrology, calibration and testing. Activities aiming at the modernization of the national quality infrastructure (NQI) will be selected according to the needs of the sectors in which the SD grants are implemented. Priorities will be identified through various consultation mechanisms such as an open forum or an online platform The ICP will be piloted in one to three segments of the economy. Possible candidates include medical devices and personal protective equipment, as well as the pharmaceutical and biotechnology industries, while product and service selection criteria will encourage climate consideration through energy efficiency or saving. 62. The SD matching grants will be implemented jointly with at least one large firm as a co-sponsor to reduce moral hazard and adverse selection risks, as well as to improve the chances of market access by the beneficiaries. 55 Co-sponsor(s) of the SD matching grants will be selected competitively based on the (i) amount of resources offered for co-funding; (ii) quality of the supply development program proposed; (iii) market size and sophistication; and, (iv) degree of commitment to procure from selected beneficiaries. The project funding is expected to be equal to or less than the contribution of the co-sponsor(s). The co-sponsor will provide the supplier development requirements to be targeted. 63. Operational procedures. The SD matching grants will be guided by the SD Matching Grants Manual. SMEs pre- selected by the co-sponsor(s) will be eligible—with women-led or managed firms especially encouraged to apply. Selection will be based on the implementation plan submitted by the prospective candidate. The most competitive implementation plans will be those considered more likely to achieve the results established in the sponsor(s)’ development plan. The project’s contribution is capped at US$50,000 per firm but could be up to 10 percent higher for women-led or managed firms. The matching funds provided by the prospective beneficiary are expected to be not less than 50 percent of the total cost of the proposed plan. The costs incurred in the preparation of the proposal could be counted as part of the matching. The duration of the program is expected to be 6-12 months, with a possible extension of up to two months for women- led or managed firms. Selection will be made by a panel composed predominantly of the private sector, submitted to the IEB for concurrence. Beneficiary feedback will be collected from stakeholder SMEs periodically to improve implementation. 64. Subcomponent 2.2. Business Investments in R&D Program (BIRD program, US$10 million). This activity will finance matching-grants for sub-projects to (i) support later stages of research commercialization, (ii) R&D for SMEs, and (iii) early stages of knowledge-based startups. In addition to SMEs, potential beneficiaries will include projects that graduate from the RCP and require, for example, funding for large-scale prototyping, small-scale production for market testing; and, independent inventors with knowledge-intensive ideas that require funding to develop proof-of-concept and prototypes. 65. The BIRD program will encourage the submission of joint R&D projects from SMEs and between SMEs and PROs. It is expected to emphasize applied R&D projects and the development of proofs of concept and prototypes. The BIRD program will encourage applications (including from early stage enterprises) that have environment- and climate-friendly innovation potential, namely applications to achieve climate change mitigation or adaptation results as well as other areas 54 A National Quality Infrastructure (NQI) is the ecosystem that supports the development, adoption and compliance of standards to promote product quality. It is comprised of the institutions, norms, and regulations, metrology and testing infrastructure and public and private stakeholders that develop, supervise or adopt quality standards 55 For the importance of demand size and sophistication in product upgrade see for example Atkin D, A Khandelwal and A Osman (2014). “Exporting and Firm Performance: Evidence from a Randomized Trial”, CEPR Discussion Paper 10276. Page 22 of 60 The World Bank Modernizing Uzbekistan National Innovation System (MUNIS) Project (P170206) related to existing societal challenges. This component may lead to forming a joint-stock company dedicated to the production of COVID-19 vaccines. Component 3 will provide TA for this decision by the GOU.56 66. The BIRD program will also offer mentorship assistance in the form of ‘investment readiness’ activities to startup firms that graduate from the program. These activities provide individualized training, mentoring, and coaching, as well as facilitate the matching between potential candidates and prospective investors, helping entrepreneurs to overcome standard obstacles in obtaining external funding and know-how for business growth. 57 Component 3 will support the design of a public co-investment fund to crowd-in this type of investment from the private sector. The implementation of this instrument, however, will depend on the existing demand throughout the project lifetime. 58 67. Operational procedures. The BIRD program will be guided by the BIRD Program Manual. Co-finance by the awardee will vary between 30 and 70 percent of the total cost of the project (up to 10 percent more for women-led and or managed firms), with a maximum amount of US$300,000 per individual project or US$600,000 for joint projects, for a period no longer than 24 months. 59 Within those boundaries, each call will adjust its parameters to the stage of the project (vs. proof-of-concept or prototype) and the sector of activity (e.g., software vs. biotech), among other factors. Eligible firms are SMEs (including startups) formally registered in Uzbekistan, with women-led and managed firms especially encouraged to apply. Final decisions will be taken by the IEB. The provider of investment readiness assistance activities will be selected competitively by the PMU under the guidance of the IEB, taking the into account the growingly available international evidence. 60 Component 3 will fund TA required to inform the selection process. Beneficiary feedback will be collected from stakeholders periodically to improve implementation. Component 3: Supporting STI policy-making and reforms (US$5 million). 68. This TA component aims to address the lack of adequate information about good policies and the weak capacity of policy-making organizations. 61 Importantly, in addition to the support of enabling reforms for project activities, it will also support coordination and consensus building for the implementation of reforms aiming at the development and implementation of new Science Technology and Innovation Strategy of Uzbekistan. The component will finance (i) advisory work for the preparation of studies or draft regulations and policies; (ii) training (e.g., mini-courses) and study tours; and, (iii) consensus-building workshops. MID will encourage women’s participation in these events. The MID will be encouraged 56 The decision to proceed with the investment will be based on a feasibility study, co-substantiated by the opinion of at least three international experts on vaccine production and virology. A preliminary framework was shared with the MID as part of the preparation of this project. See Background Note “A Proposed Approach for the Decision on Producing SARS-CoV-2 Vaccines in Uzbekistan”, prepared by Danica Ramljak, Paulo Correa and Yeraly Beksultan. 57 Prospective entrepreneurs, especially researchers, often lack knowledge about the availability of external sources of finance, the key factors that investors look for in making investment decisions, and the presentational skills necessary for successfully ‘pitching’ of business propositions. Prospective investors are also not easily accessible to individual entrepreneurs. Mason, Colin and Jennifer Kwok (2010) “Investment Readiness Programmes and Access to Finance: A Critical Review of Design Issues”, Local Economy 25(4): 269-92. 58 The BIRD program will provide a potential ‘exit strategy’ to projects, expected to be the private equity industry. 59 Each call will have flexibility to establish lower maximum values that will encourage more participation and better demonstration effect. 60 One example is the experience of the Western Balkans Investment Readiness Project. A randomized experiment worked with 346 firms and delivered an investment readiness program to half of the firms, with the control group receiving an inexpensive online program instead. A pitch event was held for these firms to pitch their ideas to independent judges. The investment readiness program resulted in a 0.3 standard deviation increase in the investment readiness score, with this increase occurring throughout the distribution. Two follow-up surveys show that the judges' scores predicted investment readiness and outcomes over the subsequent two years. Treated firms attained significantly more media attention and were 5 percentage points more likely to have made a deal with an outside investor, although this increase is not statistically significant. See McKenzie, D., Cusolito, A. P., Dautovic, E. (2018). ' Can Government Intervention make firms more investment-ready? A randomized experiment in the Western Balkans'. World Bank Policy Research Working Paper." 61 Cirera, Xavier; Frias, Jaime; Hill, Justin; Li, Yanchao. 2020. A Practitioner's Guide to Innovation Policy : Instruments to Build Firm Capabilities and Accelerate Technological Catch-Up in Developing Countries. World Bank, Washington, DC Page 23 of 60 The World Bank Modernizing Uzbekistan National Innovation System (MUNIS) Project (P170206) to monitor the participation of women in the activities of Component 3 and based on the results, consider measures to improve their participation. The component will be implemented along thematic areas, and each of the three activities complementing each other to maximize the likelihood of impact on the policy or reform to be implemented. Processes that ensure the engagement of targeted beneficiaries will be established in all stages of the process to ensure the TA meets their needs, and maximizes consensus. 62 69. TA will be provided towards project implementation as indicated in Components 1 and 2, including (i) advisory work for the design of the process of selection and evaluation of IMPs under Subcomponent 1.3; (ii) the assessment of the NQI system gaps that will guide investments and reforms under Subcomponent 2.1, 63 and, (iii) development of citizen engagement processes, and (iv) the design of a potential co-investment fund to crowd-in private investments to the BIRD program (Subcomponent 2.2). In addition, the Component will finance studies to fill knowledge gaps of the GOU related to the implementation of the current Innovation Strategy, including the consolidation of research and innovation funding and the development of a research infrastructure strategy. 70. Training activities will include formal on-the-job training and study tours for the staff of the MID, RIs and other institutions, and the Uzbek scientific community. Potential beneficiaries include the Patent Office and the Center for Scientific and Technical Information. 64 The capacity-building programs will also include relevant educative training and climate and science-related awareness sessions. 71. Consensus-building activities will target the enhanced coordination of key stakeholders and the development and implementation of a communications strategy. Coordination of innovation policy and sector reforms is a significant challenge due to the systemic nature of the innovation process and the number of different stakeholders in any NIS. In addition, there has been insufficient public discussion about the ‘destination’ of the economic transformation and the long-term changes that it will bring about, particularly in what relates to the role of a market-oriented NIS. The component will support the MID to articulate a narrative around the economic transformation of Uzbekistan and the role of a market- oriented NIS. TA will help consolidate and strengthen the functioning of coordination bodies, as well as the development of a communications strategy to get feedback, build consensus, and improve implementation. Component 4: Project management, and monitoring and evaluation (US$5 million). 72. This component will support project implementation activities. It will finance the operational costs of the PMU and other related operational costs, including those incurred in the administration of the programs. The PMU will be responsible for the day-to-day implementation of project activities, and for fiduciary activities (including procurement, financial management, environmental and social compliance, reporting citizen engagement, project audits, and other functions). This component will also finance the establishment and operation of the IEB that will guide project 62 For example, the project will support: (i) a study of the current needs of national R&D statistics, a strategy for improvement, and a consensus- building workshop among key stakeholders; (ii) the preparation of a robust M&E framework for better management of STI policies, on-the-job training on its use, and consensus-building about lessons learned and reforms needed; (iii) a broad consultation process of proposals towards the design of the country’s equivalents to a national science foundation and an innovation agency, as well as the sustainable institutionalization of the REP and the RCP into the policy arsenal of this agency; and, (iv) studies of the existing bottlenecks to research commercialization and proposals of reforms (e.g., drafting of legal reforms, adjustment to career development plans, and other), as well as consultation with key stakeholders (RIs and universities). Finally, another theme covered across the three subcomponents relates to the decision to produce vaccines locally, and Component 3 will fund the studies, training, and consensus-building activities for effective decision-making. 63 The modernization of the NQI will be supported by studies related to the gaps with international good practice, an assessment of demand for NQI services in selected value chains, and the preparation of reform proposals. 64 Since 2018, the Patent Office is a unit of the Ministry of Justice. It has collaborated closely with World Intellectual Property Organization (WIPO) and is involved in the World Trade Organization accession process. The Center for Scientific and Technical Information was created in 2019 under MID with the main goal to improve the state system of scientific and technical information. Page 24 of 60 The World Bank Modernizing Uzbekistan National Innovation System (MUNIS) Project (P170206) implementation, including final decision-making on funding schemes, the selection of proposals to be supported by the REP, and the BIRD Program. In addition, the component will finance the operation of a Project Steering Committee (PSC) to be created to improve coordination among beneficiaries and key NIS stakeholders. The allocation of US$5 million is commensurate with the the additional administrative and coordination burden on the PMU. Section III provides further detail on the project implementation arrangements. C. Project Beneficiaries 73. Direct project beneficiaries include: (i) individual senior and junior researchers, particularly women, who receive grants for developing eligible R&D ideas; (ii) Uzbek RIs, design bureaus, and scientific and engineering laboratories that receive project funds to upgrade their laboratories or modernize their management systems, including women working or potentially hired in those; (iii) SMEs, particularly women-owned and managed, that receive financial and technical support through Component 2; (iv) staff of the MID, particularly women working therein or potentially hired, and other entities involved in policy formulation and execution; (v) key stakeholders of the NIS, including the MID and other relevant authorities and ministries of Uzbekistan. It is not possible to estimate the number of direct beneficiaries (people) under other components because of the multitude of entities that will be involved in the activities. D. Results Chain 74. Figure 4 illustrates the chain of results and development impacts flowing from the project activities. To achieve the PDO and envisaged development impacts, Uzbekistan needs a market-oriented NIS that: (i) generates better and more relevant research; (ii) enables effective science-industry collaboration; (iii) promotes private sector innovation; and, (iv) counts on the capacity of the public sector to design and implement effective policies and reforms. The outputs of Components 1 and 2 (research funded through competitive selection, financing of proof-of-concept and prototypes, internationally-certified SMEs, and funds allocated to innovative firms and startups) will generate demonstration effects, leading to improved quality and relevance of policies, programs and reforms adopted thereafter. The activities will also build the capacity of the government and related beneficiaries. Component 3, in turn, will finance the production of the information, the formal training, and the consensus-building activities needed for the GoU to approve complementary reforms and transform successful pilots into government programs and policies. Components 1, 2 and 3 will enable Outcome level 2, the adoption by the GoU of core reforms and policies regarding (i) the design and implementation of funding schemes for research and innovation (including the creation of autonomous organizations to manage those funds); (ii) the management of research commercialization (including the TTU ); and (iii) other policies and reforms to improve the quality of public spending on R&D (including a strategy to modernize public research organizations). Page 25 of 60 The World Bank Modernizing Uzbekistan National Innovation System (MUNIS) Project (P170206) Figure 4: Results Framework Source: Staff elaboration E. Rationale for the World Bank’s Involvement and Role of Partners 75. The rationale for the World Bank’s involvement is multi-faceted. First, the World Bank has ample experience in implementing similar projects worldwide (for example, Serbia, Croatia, and Vietnam), including in Central Asia. Second, projects by the World Bank help to quickly pilot and test new best-practice policy tools to demonstrate success, build trust, and potentially transform the way research is financed, conducted, and linked to industry. This transformation includes placing a greater emphasis on competitive financing mechanisms, international peer-review processes, inter- disciplinary scientific teams, and a closer connection between research and industry. Third, there is a strong rationale for the World Bank to provide fiduciary oversight for a project that attempts to create the right environment and incentives for the science and technology (S&T) and R&D sectors that are prone to interest group capture (even in more advanced economies but especially in post-transition economies). Finally, the project has synergies with the Higher Education Modernization Project in terms of improving the quality and relevance of university research. F. Lessons Learned and Reflected in the Project Design 76. The project reflects lessons learned from a team review of more than 20 World Bank activities in the field of competitiveness and innovation that were implemented over the past ten years in the ECA region. The review included TA, reimbursable advisory services (RAS), the 2013 Independent Evaluation Group (IEG) Report on the ‘World Bank Group Page 26 of 60 The World Bank Modernizing Uzbekistan National Innovation System (MUNIS) Project (P170206) Support to Innovation and Entrepreneurship’, and other World Bank studies, including two flagship studies in the field of innovation policy. 65 77. The proposed project aims to support the development of the institutional foundations of a market-oriented NIS. This is in contrast to directly financing large programs or infrastructure investments to boost science and innovation outcomes. The project directly reflects the recommendations of the two flagship reports to resolve that (i) innovation is hampered by market and systemic failures that cannot be addressed by partial, isolated interventions in developing countries; and, (ii) the necessary policy capabilities to diagnose problems, identify policy solutions, and implement them is often very weak. The project design also tries to address the difficulties of many World Bank projects with the sustainability of their interventions once the project ends. 78. The World Bank’s involvement in formal consensus-building activities, like those under Component 3, is critical for successful implementation, as in the case of the Serbia Innovation and Competitiveness Project. Building and reforming Uzbekistan’s NIS will be a long-term process. The pace of reforms of the NIS is unique to each country, and its timing is not entirely under the control of the beneficiary. Even when reforms are within the boundaries of project beneficiaries, project funds are rarely large enough to encourage all stakeholders to act. Using reform milestones to condition disbursement increases the risk of severe implementation delays. 79. The demonstration effects of well-designed pilots, like those funded by Components 1 and 2, are a common feature among successful projects. While often limited in scope (i.e., not systemic), the most durable institutional changes attributable to World Bank projects were associated with a long process of learning derived from practical experience. For example, most researchers from Croatia’s Rudjer Boskovic Institute initially opposed research commercialization activities but changed their minds once the compatibility between research excellence and commercialization became evident. ‘Quick wins’ support broader advocacy for reforms, while creating a direct constituency of beneficiaries. Pilot initiatives are also effective instruments to test demand, improve program design, build institutional capacity, as well as sensitize policymakers. 80. Components 1 and 2 incorporate flexibility in the selection among funding instruments, reflecting lessons learned from Armenia, Montenegro, and Serbia. This allows the project to adjust to opportunities and challenges as circumstances evolve, such as the inadequacies of legal structures that are often revealed only during implementation. In Serbia, for example, problems in the existing IP regime were revealed only through informal discussions with professionals during the implementation of a pilot program. 66 Also, as Cirera and Maloney (2017) warn, developing countries need to avoid the common mistake of directly importing policies and institutions from advanced economies, leaving flexibility to adjust these policies to fit their institutional reality. Moreover, as the experiences of Armenia, Georgia, and Montenegro’s projects suggest, developing countries’ institutional and policy environments will change more often than expected – leading to sometimes costly project restructuring. 67 81. The operational experience and extensive reviews of innovation policies directly informed the funding mechanisms and programs under Components 1 and 2. The choice of the innovation programs was closely informed by Cirera, Frias, Hill, and Li (2020) ‘Practitioner’s Guide to Innovation Policy Instruments’. The guide describes the menu of instruments available to policy-makers, the existing evidence about their impact, and the institutional and contextual factors that determine their effective design and implementation. Measures to support research commercialization (Component 1) have been funded by most World Bank’s projects, and a sense of what constitutes a good practice has 65 Cirera, Xavier; Maloney, William F. 2017. The Innovation Paradox: Developing-Country Capabilities and the Unrealized Promise of Technological Catch-Up. Washington, DC: World Bank. Cirera, Xavier; Frias, Jaime; Hill, Justin; Li, Yanchao. 2020. A Practitioner's Guide to Innovation Policy: Instruments to Build Firm Capabilities and Accelerate Technological Catch-Up in Developing Countries. World Bank, Washington, DC. 66 Serbia Research, Innovation and Technology Transfer Project. 67 Armenia E-Society and Innovation for Competitiveness Project; Georgia National Innovation Ecosystem (GENIE); and Montenegro Higher Education, Research and Innovation for Competitiveness (HERIC) Project. Page 27 of 60 The World Bank Modernizing Uzbekistan National Innovation System (MUNIS) Project (P170206) emerged. Grant operational manuals, draft regulations, and the project components of the Croatia, Serbia, and Kazakhstan projects were reviewed in preparation of this project. Moreover, some of the proposed interventions have been rigorously tested, such as the proposed Investment Readiness Program, where a randomized experiment tested whether those interventions improved the supply of ‘investable’ projects to angel investors in the Western Balkans.68 82. The transparency and credibility of the funding mechanisms, as incorporated in the project, are critical because they will inevitably create winners and losers. Episodes of detailed public scrutiny and, sometimes, undue criticism have often occurred. It is crucial to preserve the integrity of the GOU, the World Bank, and the reform process. This project proposes an arms-length IEB, along the lines of similar boards in Croatia, Serbia, and Kazakhstan. Such vetting and decision mechanisms that are far removed from local politics not only instill faith that decisions to award grants are made based on merit and not nepotism, but also expand international collaboration and knowledge inflow to the country. 83. The project also invests in broad ownership among stakeholders and officials to ensure project continuity regardless of government changes. A critical lesson learned is that while the creation of steering committees and boards for the governance of entities (such as innovation funds) may bring additional complexity, it also promotes continuity and ownership. In projects aiming to reform innovation ecosystems, it is vital to organize a critical mass of individuals, with significant social and political capital, into a technocratic network, clearly seeing its benefits from the envisaged changes. III. IMPLEMENTATION ARRANGEMENTS A. Institutional and Implementation Arrangements 84. The project will be implemented by the MID through the PMU. The PMU will perform two types of activities: (i) those related to ‘standard’ project management functions; and, (ii) those associated with the implementation of the funding schemes. 85. The PMU will perform day-to-day project administration activities and will retain the ultimate responsibility for enforcement of the World Bank’s fiduciary (financial management and procurement) rules, including (i) planning, budgeting, procurement, financial reporting, accounting, disbursement, and arranging audit over the project’s funds utilization; (ii) carrying out project M&E activities; and, (iii) building project awareness and handling grievances. 86. In addition, the PMU will be responsible for the administration of the funding schemes (REP; RCP; BIRD; and SDI). The administration of the funding schemes will follow the procedures established in the respective program manuals; the common fiduciary and procurement rules established in the Grants’ Fiduciary Manual (GFM) and the Procurement Guidelines; and, the overall regulations described in the Project Operational Manual (POM). Those functions include: (i) Supporting organization of competitions to select research projects, startups, RIs and other entities selected under this project; (ii) Overseeing procurement of goods and services required by the selected research projects, and other entities to be established under the project to ensure that the procurement procedures comply with all applicable World Bank’s rules and regulations; and, (iii) Ensuring that individual research project activities comply with all the applicable World Bank’s fiduciary, environmental and social requirements. 87. Core PMU staff. To start, the PMU will be composed of the following positions: PMU Director; Coordinator for Science Activities (Component 1); Coordinator for Innovation Activities (Component 2); Coordinator for Technical Assistance (Component 3); Financial Management Specialists (2); Procurement Specialists (2); and, a Monitoring and Evaluation Specialist. The coordinator will be responsible for the implementation of each respective project component. Two financial management and procurement specialists are envisaged from the beginning to reflect the two main functions of the PMU. In addition, one resident advisor on STI policy is expected to be hired. The initial number of total 68Cusolito, Ana Paula; Dautovic, Ernest; Mckenzie, David J., 2018. Can government intervention make firms more investment-ready? A randomized experiment in the Western Balkans (English). Policy Research working paper; no. WPS 8541; Impact Evaluation series. Washington, D.C.: World Bank Group. Page 28 of 60 The World Bank Modernizing Uzbekistan National Innovation System (MUNIS) Project (P170206) staff (nine) is on the higher side, but it is consistent with the World Bank’s experience in implementing similar projects in the region. Staff will be hired by the MID subject to the World Bank’s clearance of the terms of reference (TOR) and the selected candidates. 69 88. As the project evolves, the hiring of an Environmental and Social Protection Specialist and an IT Specialist will be considered. Particularly, it is envisaged that the implementation of Components 1 and 2 may need additional staff to manage the selected research projects. Experience suggests that an additional 6-9 staff, between program managers, financial managers, and procurement specialists may be necessary. The options of relying on government staff to perform those tasks (as on-the-job training) or contracting specialized organizations will be considered. In all cases, the management of project funds and fiduciary responsibilities will remain with the PMU. 89. International Expert Board (IEB). The IEB will be composed of a pool of five distinguished foreign scientists (in disciplines of strategic importance for Uzbekistan), experienced venture capitalists/entrepreneurs, as well as former STI policy-makers. The primary functions of the IEB are: (i) selection of research projects to be financed by the REP and the BIRD program; (ii) supervision of the RCP and the SDI; and, (iii) advice to the MID on STI policies and reforms. The IEB will also provide guidance and supervise the implementation of the REP and the BIRD program on activities such as the design and implementation of the call for proposals, and the monitoring of progress and achievements of selected research projects. The IEB members will be hired competitively and approved by the World Bank. The IEB is expected to meet monthly. Most of the work will be carried out virtually, with local meetings taking place bi-annually. The details of the IEB’s operating model, rules, and responsibilities will be defined in the IEB’s Statute. Support to the functioning of the IEB, including the preparatory work for the monthly meetings, will be performed by the PMU. 90. Project Steering Committee (PSC). The PSC will be chaired by the Minister and composed by the representatives of the main beneficiaries and key stakeholders. The primary roles of the PSC are to ensure coordination of activities and facilitate the identification of implementation bottlenecks and potential solutions to be presented to the PMU. The scope of work of the PSC will include the regular review of the project implementation and M&E data, and the provision of advice on project implementation – especially on issues related to the activities under Component 3 vis-a-vis the progress in the implementation of the GOU’s Innovation Strategy and overall development of the NIS. The operational costs for and administrative support to the PSC will be provided by the PMU. TOR satisfactory to the World Bank will detail the composition and the rules for the functioning of the PSC. B. Results Monitoring and Evaluation Arrangements 91. The MID, through the PMU, will be responsible for conducting the overall M&E of the project. The PMU will report on implementation progress, results achieved, and issues that impede progress and results to the MID, PSC, and the World Bank. The PMU will have a dedicated M&E Specialist responsible for the design and implementation of necessary processes and ensuring quality. The M&E Specialist will work in close collaboration with technical component leads. Uzbek citizens will be engaged in monitoring the project as well as in providing feedback on how activities could be pursued more effectively through activities indicated in the Stakeholder Engagement Plan. 92. For M&E results, the project’s results framework (Section VI) will be utilized, and progress will be disclosed to the public through the Implementation Status and Results (ISR) Report on the World Bank’s external website. The MID will be encouraged to publish the project status on its website. In addition to the outcomes identified in the results framework, the project’s progress will be validated at mid-term and against external indicators that are critical to increase investor confidence (such as improvements in World Governance Indicators scores, for which project interventions are expected to contribute but do not fully influence). 69 The Technology Transfer Unit is not part of the PMU and will be created as part of the MID. Page 29 of 60 The World Bank Modernizing Uzbekistan National Innovation System (MUNIS) Project (P170206) 93. For the M&E framework, the PMU will collect data with input from the MID and the IEB. Data collection is not expected to be costly or complex. Data is obtained directly from the project beneficiaries as part of grant and payment conditions and regular reporting. An annual survey of scientists and entrepreneurs will be carried out to complement this data. The survey will be distributed yearly to a representative group of respondents and will be publicly available on the project website. The PMU will produce regular project progress and grant monitoring reports, a mid-term report for the World Bank and key stakeholders, and input to the results and a completion report at the end of the project. The PMU will organize semiannual events chaired by the Vice Minister of the MID to disseminate project results. C. Sustainability 94. The GOU is committed to an ambitious strategy of fostering innovation as a driver of economic growth. The authorities set a goal of entering the top 50 in the Global Innovation Index by 2030 and plan to quadruple R&D spending, improve scientific excellence, and strengthen links between science, education, and industry. Project sustainability will depend, ultimately, on continued GOU’s determination to (i) maintain a stable macro-economic environment; (ii) maintain a business environment conducive to enterprise growth and firm-level innovation; and, (iii) continue to finance the R&D and technology commercialization programs developed under the auspices of this project. Furthermore, capacity building, which is an integral feature of the project design, will help to ensure that, after the pilot, Uzbek nationals will have acquired the skills to design and implement the programs developed under the auspices of the project. Finally, engagement with the beneficiaries of each activity, ensuring their inputs are reflected in the designs, implementation and success of each one, will ensure that the activities meet the needs of the Uzbek beneficiaries and can therefore be sustained and replicated locally after project closure. IV. PROJECT APPRAISAL SUMMARY A. Technical, Economic and Financial Analysis 95. The economic analysis of this project follows the World Bank’s guidance note on economic analysis of investment project financing (IPF). It uses an ex-ante cost-benefit analysis framework to assess and monetize costs and benefits of implementation of the proposed project and associated externalities. In particular, it looks into direct (financial) costs associated with the implementation of the project and indirect costs associated with deadweight losses due to the collection of taxes and possible environmental damages. On the benefits side, the analysis covers financial and economic revenues generated by innovative activities facilitated by the project. The analysis arrives at a Net Present Value (NPV) of US$6.2 million in the baseline case. This figure was derived from quantifiable benefits of Components 1 and 2, taking into account the ‘no-project’ counterfactual and time horizon till 2030. For further details, see Annex 3. 96. The principal benefit of the project comes from the sustainable development of innovative research and production in Uzbekistan through the following vehicles: (i) accumulation of scientific infrastructure and human resources - prerequisites for cutting-edge R&D and accommodation of scientific knowledge from abroad; (ii) establishment of strong industry-R&D links and developing the science commercialization ecosystem; (iii) addressing existing market failures and information asymmetries; (iv) strengthening institutional frameworks to support commercially viable R&D activities; (v) increasing attractiveness of scientific careers among younger generations, thus contributing to a stable supply of human capital to the NIS; and, (vi) diversifying the local economy and increasing share in GDP of production with high value- added. 97. The assessment of the costs of the project is straightforward. In broad terms, it is based on two types: (i) direct financial outflows under the project components; and, (ii) indirect cost of public funds needed to finance the project. 98. A cost-benefit analysis based on a series of assumptions, available statistical data, and relevant literature indicates the project’s NPV of US$6.2 million and ERR of 28.8 percent. This figure is derived from quantifiable benefits of Components 1 and 2, taking into account the ‘no-project’ counterfactual and time horizon till 2030. To test the Page 30 of 60 The World Bank Modernizing Uzbekistan National Innovation System (MUNIS) Project (P170206) sensitivity of the result to changes in key variables, the NPV was recalculated using a 13 percent discount rate (vs. 11.6 percent in the baseline case) and lowering the projected social return rate by 20 percentage points to 40 percent (vs. 60 percent in the baseline case). The application of a higher discount rate had a net negative effect while lowering the project’s expected impact on the economy reduced benefit streams and did not impact its costs. Table 1. Comparison of NPV and benefit-to-cost ratio (BCR) in baseline and alternative scenarios Scenario NPV* BCR* Baseline US$6.2 million 1.4 Pessimistic US$3.1 million 1.2 *These results do not include contributions from the private sector (i.e., through matching grants) 99. Consideration of economic analysis in the project design. The cost-benefit analysis presented in this analysis supports the implementation of the project considering positive net benefits. As an alternative to the project, the continuation/ expansion of the MID’s existing science commercialization mechanisms could be considered. These mechanisms heavily focus on the supply side of R&D, lacking measures to capture the demand and to increase enterprise sector participation. Although, in this case, there would be lower initial costs and time savings, this avoided cost is much smaller compared to the foregone benefits. These foregone benefits include, among others, economies of scale and combination of supply and demand approaches through matching grants and the TTU’s activities. 100. Use of economic analysis during project implementation. The PMU will collect information on disbursements by component/ subcomponent, cost structure of R&D projects, and achievement of result indicators, among other data. This information will be used to determine at the mid-term of project implementation whether the economic analysis needs an update. A full economic analysis will be carried out at the end of the project to compare against appraisal. 101. Project impact on the GOU’s fiscal situation. The project poses no risk to the monetary or fiscal stability of Uzbekistan. A total of US$50 million to be spent in a span of five years comprises less than 0.01 percent of the country’s GDP in 2019 and about 0.4 percent of the planned total budget spending in 2020. It should not result in inflationary processes or cause a budget deficit. B. Fiduciary (i) Financial management (FM) 102. The FM assessment of the FM arrangements of the MID capacity to implement the proposed project were found to be generally acceptable provided risk mitigation measures are duly addressed by the MID (please see Table below). The overall risk for implementation of this Project was assessed to be Moderate at this stage due to the following: i) this is the first WB project for the MID, ii) the MID, being the state agency, lacks the related experience in managing IFI projects; iii) the consistency and adequacy of FM arrangements will be challenged by the number of grants, scholarships, etc. (representing close to 70 percent of the project funds). Table 2. Actions to be addressed by the MID Action description Responsibility Due Date Adopt the Project Operation Manual (POM) acceptable to the Bank which shall include a MID Effectiveness chapter on the Project’s Financial Management arrangements (i.e. detailed guidance and condition procedures on project budgeting and planning, accounting, internal controls, funds flow, financial reporting, auditing and disbursement, etc.). Adopt a Grant Fiduciary Manual (GFM) acceptable to the Bank that shall state the MID Disbursement Page 31 of 60 The World Bank Modernizing Uzbekistan National Innovation System (MUNIS) Project (P170206) detailed fiduciary arrangements and financing mechanism (disbursement, monitoring, condition reporting, audit and etc.) for grant activities stated in Component 1 and 2. Install a fully functional accounting software and the needed accounting and financial MID Legal covenant controls based on the accounting software and sustain those throughout the project (90 days from implementation period. effectiveness) Not later than the one hundred and twenty (120) days from the Effective Date, the MID Legal covenant Recipient, through the MID, shall establish an International Expert Board with a (120 days from composition and terms of reference acceptable to the Association, to advise MID on (i) effectiveness) overall Project implementation; and (ii) selection of the Grants, based on the Project Operational Manual’s, respective Program Manual’s and the Grant Fiduciary Manual’s rules and criteria. Recruit an environmental protection specialist and a social protection specialist with Legal covenant terms of reference and qualifications acceptable to the Association. (60 days from effectiveness) Establish a Project Steering Committee with composition and terms of reference Legal covenant acceptable to the Association, to be chaired by a representative of the MID, which role (90 days from shall be to ensure smooth coordination between various public entities engaged in the effectiveness) Project implementation; 103. The project accounting will be conducted as per International Public Sector Accounting Standards (IPSAS, cash basis), and the accounting records will provide the necessary details on each transaction made from the Project funds. The PMU under the MID will be in charge of keeping accounting for this Project, and it will maintain and keep the accounting records in the accounting system that would be acquired and installed by project effectiveness. The project accounting records will be maintained in accordance with the Cash Basis International Public Sector Accounting Standards. At the same time, the PMU will apply accrual-basis accounting to reporting with respect to state agencies (Ministry of Finance, Tax Committee, Statistics Committee, etc.). This system would allow fully automated accounting and reporting, including automatic generation of Statements of Expenditure, Interim Un-audited Financial Report (IFR), and other reports required by national legislation. The system will have built-in controls to ensure data security, integrity, and reliability. 104. The PMU will prepare Interim un-audited financial reports (IFRs) starting with the quarter in which the first disbursement occurs. The IFRs will be submitted to the World Bank on a quarterly basis within 45 days after the end of each calendar quarter and will include information on the sources and uses of funds, detailed use of funds by each activity, the budget category, grantees, as well as movements and balances in the DA and Transit accounts. The format of the IFRs will be agreed with the World Bank and will be part of the POM. It will include (a) project sources and uses of funds, (b) uses of funds by project activities, (c) project balance sheet, (d) DA statement, and (e) statement of expenditure withdrawal schedule. IFRs will be automatically generated by the project accounting software. 105. The FM arrangements related to the grants (Components 1 and 2) will be defined in the GFM. The process funds flows, financial and progress reporting, monitoring over grantees, funds utilization, audit, etc. will be specified in detail in the GFM. The adoption of the GFM will be a disbursement condition. The Borrower, including the MID as an implementing agency, will have the right to access the data and records of the Grantees and request review or independent audit of the Grantees over funds usage. The Grantees will be required to ensure a minimum level of internal controls, which will include opening and maintaining a dedicated bank account, maintaining simplified accounting and financial reporting over expenditures and transactions related to funds usage, and maintaining proper data and records management related to the funds received. The Grantees will be subject to monitoring visits by the MID-assigned staff and by the World Bank’s staff or the World Bank’s assigned experts. 106. The MID will be responsible for arranging the independent annual audit of Project Financial Statements. The Project Financial Statements audit will be conducted by an independent private auditor firm acceptable to the Bank, on terms of reference acceptable to the Bank, and according to the International Standards on Auditing issued by the Page 32 of 60 The World Bank Modernizing Uzbekistan National Innovation System (MUNIS) Project (P170206) International Auditing and Assurance Standards Board of the International Federation of Accountants. The project audit will include (i) the audit of Project financial statements; (ii) the review of the financial performance of the Grantees; and (iii) review of the internal controls of the PMU with special attention to the compliance with requirements established in the Loan Agreement and the World Bank’s guidelines and procedures and also the local legislation requirements. No entity audit will be required. 107. The Projects audited financial statements and the audit report will be furnished to the Bank within six months of the end of each fiscal year and also at the closing of the project. The audited project financial statements will be disclosed to the public in a manner acceptable to the World Bank. Following the World Bank’s formal receipt of these statements from the Borrower, the World Bank will make them available to the public in accordance with the World Bank’s Policy on Access to Information. Audit of annual project financial statements will be financed from credit proceeds. 108. The Audit arrangements of the Grantees, scholars, and other beneficiaries receiving funding under Components 1 and 2 of the project will be defined in the GFM. 109. The disbursement arrangements will follow the traditional disbursement mechanism, including direct payments, replenishments of the designated account, reimbursements, and special commitments. The minimum application size and designated account ceiling will be specified in the Project disbursement letter. The PMU will open the project’s Designated Account in USD in the financial institution acceptable to the World Bank and a transit account in Uzbek soums as may be necessary, and both accounts will be used only for the inflow of the Project funds and payment of eligible expenditures. Eligible project expenditures, consisting of regular goods, consulting services, non-consulting services, training and operating costs, would be documented to the World Bank using Statements of Expenditure (SoE) and full documentation. For all expenditures disbursed on the basis of SOEs, full documentation in support of the SOEs will be retained in the PMU, as per local regulation requirements, but not less than for five years after the project closing date. This information will be available for review by Bank missions during project supervision and by the project audits. The PMU will be in charge of fiduciary responsibility over the project (procurement, financial management and reporting, disbursements and payments, contracting, budgeting and budget monitoring and controlling, financial reporting and arranging for audits, etc.). The detailed guidance on the designated accounts, DA maintenance and ceilings, costs allocations and etc. will be specified in the Disbursement and Financial Information letter. (ii) Procurement 110. Applicable procurement framework. All procurement of goods, non-consulting and consulting services contracts will be conducted through the procedures as specified in the World Bank’s Procurement Regulations for IPF Recipients - Procurement in Investment Project Financing Goods, Works, Non-Consulting and Consulting Services, dated July 2016, revised November 2017 and August 2018. The project will be subject to the World Bank’s Anticorruption Guidelines, dated October 15, 2006, revised in January 2011, and as of July 1, 2016. 111. Procurement planned. The major planned procurement is expected to include the consultancy assignments for scientific performance and economic relevance of applied research and development (R&D), stimulate enterprise-level innovation to achieve development of new or improved products, technologies and processes, and create new institutions, such as an International Expert Board (IEB), a Technology Transfer Unit (TTU.) In addition, the project will include TA and capacity building to support implementation of the mentioned activities. The PPA fund will be used to support the MID to conduct a needs assessment for the proposed Project’s technology commercialization support activities, including review of existing research grant programs, funding mechanisms, science audit and identification of needs in technical expertise and creation of a technical experts’ database. 112. Grant Program. A Research Excellence Program (REP): Grants for junior and senior scientists to be distributed through several selection rounds. The POM and GFM will include the specific operational details on how the Grant Program will be managed. Page 33 of 60 The World Bank Modernizing Uzbekistan National Innovation System (MUNIS) Project (P170206) 113. Summary of Project Procurement Strategy for Development (PPSD) and Procurement Plan (PP). The PPSD is under preparation by the MID and it will identify the appropriate procurement approaches under the project. Based on the PPSD findings, the MID will prepare the PP for the first 18 months of project implementation, to be agreed at the negotiations. The PP, including its updates, shall include: (i) a brief description of the activities/contracts; (ii) the selection methods to be applied; (iii) cost estimates; (iv) time schedules; (v) the World Bank’s review requirements; and (vi) any other relevant procurement information. The PP will be updated at least annually or as required during project implementation to reflect any substantial changes in procurement approaches and methods to meet the actual project implementation needs and market fluctuations and improvements in institutional capacity. The updated PP, along with the revised PPSD (if required), will be subject to World Bank’s prior review and no objection. The PPSD will include detailed assessments of the markets for goods and services required for the project implementation, procurement approaches and procurement risks analysis along with corresponding proposed risk mitigation measures. 114. Procurement risk assessment. The World Bank is processing a procurement capacity assessment using the Procurement Risk Assessment and Management System (P-RAMS). Based on the assessment and taking note of the existing capacity within the MID procuring the relatively large consultancy activities will be challenging. 115. Use of national procurement procedures. All contracts for goods, works and services to be procured in line with the national market approach shall follow the procedures set out in the Public Procurement Law (PPL) of the Republic of Uzbekistan dated April 2018. The PPL and related regulations have been assessed and found broadly consistent with the World Bank’s Procurement Regulations Section V – Para 5.4: National Procurement Procedures with some specific conditions that are spelled out in the PPSD and the textual part of the PP that were approved at negotiations. However, the GOU envisages enacting an amendment of the current PPL and related regulations. Upon enactment of this amendment, the National Procurement Procedure will be reassessed accordingly along with the associated conditions that will be revised in the textual part of the project PP. 116. Procurement supervision and ex-post review. Routine procurement reviews and support will be provided by the procurement team based in the Tashkent World Bank’s country office. In addition, at least two project implementation support missions are expected to take place each year, during which procurement ex-post reviews will be conducted for the contracts that are not subject to the World Bank’s prior review on a sample basis (covering 20 percent of contracts). One procurement ex-post review report will be prepared per fiscal year, including the findings of physical inspections of not less than 10 percent of the contracts awarded during the review period. 117. Systematic Tracking of Exchanges in Procurement (STEP). All procurement transactions under the project must be recorded in or processed through the World Bank’s planning and tracking system, STEP. This ensures that comprehensive information on the procurement and implementation of all contracts for goods, non-consulting services, and consulting services awarded under the whole project are automatically available. This tool will be used to manage the exchange of information (such as bidding documents, bid evaluation reports, no objections, etc.) between the MID and the World Bank. .C. Legal Operational Policies . Triggered? Projects on International Waterways OP 7.50 No Projects in Disputed Areas OP 7.60 No . D. Environmental and Social 118. Aiming to support the development of a market-oriented NIS, including policies and institutions, the project does not pose any major environmental and social challenges. However, the project may have a positive direct effect on climate-related issues by supporting research and innovation in this area. Key risks relate to the project's potential for Page 34 of 60 The World Bank Modernizing Uzbekistan National Innovation System (MUNIS) Project (P170206) handling and the need to dispose hazardous materials properly. The client will prepare and disclose the ESMF since the project is financing a broad range of small and medium-scale grant financed activities, most of which will not be identified until implementation begins. The ESMF will cover applicable ESF Standards and the WBG’s Environmental Health and Safety Guidelines. An SEP and LMP will also be prepared. E. Gender 119. The introduction of autonomous, meritocratic and transparent mechanisms for the allocation of research and innovation funds, the experience of other Bank project shows, contributes to the reduction of the gender gap in the sector (as the proportion of female beneficiaries tend to be more consistent with the proportion of female applicants). In fact, data on gender and innovation in Uzbekistan suggest that there is opportunity for improvement for women’s representation, to which this project can contribute. The number of female R&D personnel in Uzbekistan has slightly increased between 2012-2017 to reach 15.1 thousand (headcount criteria), which corresponds to around 0.24 percent of total female labor force in the country. Of all R&D personnel (full-time equivalent), most of the female R&D personnel are concentrated in the private non-profit field of activities (around 50 percent), followed by government (41 percent), higher education and business enterprises (40 percent). Distribution of female researchers per age cohort seems to indicate a funneling effect with age, with more than a third of all female researchers (when headcount is considered) being concentrated in the 35-44 age cohort, followed by 27 percent in the 25-34 age group, generally the most productive age cohorts. Only 22 percent of female researchers are in the 45-54 age cohort, while the least represented are the female researchers in the age cohort of 65 and older, and younger than 25 (around 8 percent of total female researchers.) In order to address these gaps, the project includes incentives in the grant programs and government technical assistance support to encourage an increase in women’s participation. These efforts will be monitored with gender disaggregated indicators in the M&E framework, as well as informal tracking by the PMU. F. Citizen Engagement 120. No potential effects to the lives of ordinary citizens are envisaged, and the potential benefits of citizens’ participation (when compared to community-driven development projects) are naturally minor. Yet, systematic consultation with project beneficiaries and feedback mechanisms are envisaged in order to strengthen project implementation. The project will thus support to the extent possible: (i) open consultation forums and focus group discussions/roundtables in RIs and HIEs, and with SMEs for the REP, RCP, PRIM, and RCI activities; and (ii) a report card approach to beneficiary feedback which deploys an online survey but also engages an independent organization to assess and facilitate dialogue around results with each beneficiary group (researchers, private sector and government). The MID, based on the advice of the PMU, will decide on the adoption of stakeholders’ recommendations. 121. Given the focus of the project on the governance of funding mechanisms, there is a particular emphasis on grievance redress mechanisms. This will be emphasized in the implementation of Subcomponents 1.1, 1.2, 2.1 and 2.2 (with funding schemes), including through an online platform to be established by the PMU and widely advertised through the appropriate media channels. The project will monitor the complaints and their resolution, and the PMU will post the results online semi-annually. Decisions will be taken by the IEB. 122. Through Component 3, stakeholders will not only be consulted but empowered to voice their preferences about proposed policies and reforms. The goal of the proposed engagement mechanisms is to ensure that all potential beneficiaries have feedback opportunities through the subgrant cycle and that the project is able to respond to the feedback in a timely manner. In addition to active feedback from project stakeholders during project implementation, however, Component 3 will be dedicated to the promotion of consultation and consensus-building about reforms and policies to be introduced to improve the market orientation of the NIS. Moreover, to ensure that the participatory mechanisms are robust and account for any possible constraints in the COVID-19 context, the design of these processes will be adapted through civic technology innovations (digitalized and facilitated). The results framework also includes an indicator that measures the effectiveness of these citizen engagement efforts. This will be collected alongside feedback Page 35 of 60 The World Bank Modernizing Uzbekistan National Innovation System (MUNIS) Project (P170206) on project outcomes that will be deployed by the PMU to project beneficiaries. This information will also be disaggregated by gender to identify gender-specific concerns, if any. V. GRIEVANCE REDRESS SERVICES 123. Communities and individuals who believe that they are adversely affected by the World Bank’s supported project may submit complaints to existing project-level grievance redress mechanisms or the World Bank’s Grievance Redress Service (GRS). The GRS ensures that complaints received are promptly reviewed in order to address project- related concerns. Project affected communities and individuals may submit their complaint to the World Bank’s independent Inspection Panel which determines whether harm occurred, or could occur, as a result of the World Bank’s non-compliance with its policies and procedures. Complaints may be submitted at any time after concerns have been brought directly to the World Bank's attention, and the World Bank’s Management has been given an opportunity to respond. For information on how to submit complaints to the World Bank’s GRS, please visit http://www.worldbank.org/en/projects-operations/products-and-services/grievance-redress-service. For information on how to submit complaints to the World Bank Inspection Panel, please visit www.inspectionpanel.org. VI. KEY RISKS Table 3: SORT ratings Political and Governance Moderate Macroeconomic Moderate Sector Policies and Strategies Moderate Technical Design Moderate Institutional Capacity for Implementation and Sustainability Moderate Fiduciary Moderate Environmental and Social Moderate Stakeholders Moderate Overall Moderate . 124. Political and governance. The PDO, helping to build a market-oriented NIS, is consistent with the market-oriented reforms underway that have received strong political support. 70 In addition, the modernization of the science and innovation policy of Uzbekistan, including the objective of making them more relevant for social progress and economic growth, has been backed by the senior leadership in the GOU, including the President. The project’s goal and activities follow the process of modernization already started by the MID, without significant political challenges. Coordination challenges are likely to appear as the process of modernization advances, but most of the reforms and policies supported by the project are under the domain of MID. Rating: Moderate. 125. Macroeconomic. Until the COVID-19 outbreak, Uzbekistan’s growth prospects and the fiscal situation were relatively comfortable. Growth is now projected to be sharply lower in 2020 at 1.5 percent. 71 The government adopted a billion-dollar package to increase health and social spending, and ease tax, debt, and cash-flow constraints on businesses. 70 The effect of the COVID-19 outbreak on the drivers of poverty reduction, such as income growth and remittances, is likely to increase poverty levels in 2020, affecting the political support to the market-oriented reforms. 71 Additional Development Policy Financing: Sustaining Market Reforms in Uzbekistan (P173948), Project Information Document, April 2020 Page 36 of 60 The World Bank Modernizing Uzbekistan National Innovation System (MUNIS) Project (P170206) Lower tax revenues and additional crisis spending are expected to increase the fiscal deficit to 5.6 percent of GDP in 2020 and 4.7 percent of GDP in 2021. The current account deficit is expected to widen to around 8.5 percent of GDP in 2020, but a drawdown on reserves and development partner assistance is expected to finance the deficit. In the absence of a second wave of the pandemic, Uzbekistan’s macroeconomic situation is likely to be sustainable. Nevertheless, a possible deterioration of the macroeconomic situation can directly affect this development outcome by reducing the fiscal space available for disbursements. Indirectly, the macroeconomic situation may affect the achievement of the development outcome by raising costs through inflation, focusing the attention of the GOU away from the project, while a widening gap between the balance of payments and an exchange rate devaluation may cause a delay in imports, such as for the lab equipment envisaged in the project. Rating: Moderate. 126. Sector policies and strategies. Innovation policy is nascent and fragmented in Uzbekistan, and the project activities will help shape those policies by supporting the development of a new Science Technology Innovation Strategy. One potential risk refers to reminiscent practices that do not fully favor autonomous, transparent, and accountable methods of funding distribution. The project addresses this issue upfront, placing the adoption of research and innovation funding schemes compatible with international best practices at the core of its objectives. Most decisions related to the allocation of the project’s funding schemes will be taken by an expert board selected internationally on a competitive basis. The GOU’s dissent is possible but will require a written justification, board concurrence, and will be made publicly available. Detailed program manuals will embed internationally accepted meritocratic selection criteria and transparent selection processes. The involvement of private investors, when possible, will help to address potential selection biases. Lastly, Component 3 of the project directly supports studies, training, and consensus-building needed for the adoption of all those rules by the MID – including the creation of autonomous organizations for managing research and innovation funding. Rating: Moderate. 127. Technical design. The project is fundamentally a capacity-building project based on two core activities, piloting funding schemes for research and innovation and providing TA for reforms and the sustainability of pilots. One level of complexity is that the implementation of the funding schemes is to be complemented by TA activities to enable reforms and adoption of new policies. Yet, the project does not commit to the implementation of many reforms; rather, the emphasis is on the adoption of policies and regulations under the domain of the MID. This approach is based on the experience of several science and innovation projects implemented in the region in the past decade. The three components are independent of each other and based on reasonably available and well-documented knowledge (i.e., international best practices). Lack of prior experience and low implementation capacity remain a significant challenge despite light project design. Rating: Moderate. 128. Institutional capacity for implementation and sustainability. Given the implementation capacity challenges, the project design is highly dependent on the PMU’s capacity to manage funding schemes under Components 1 and 2, in addition to the TA activities. Managing funding schemes is a time consuming and relatively complex activity that adds substantial work to the intrinsic administrative and fiduciary obligations posed by the World Bank’s TA project. To address this, the project allocates 10 percent of the loan to the PMU’s operation (Component 4), including resources for hiring managers for the different funding schemes, training those managers, and covering operational expenditures related to their implementation (e.g., peer-reviewing costs). Also, resources from a Project Preparation Advance (PPA) is expected to start addressing these risks before project effectiveness. Moreover, project components are designed to provide the amount of content necessary for project implementation, embedding, at the same time, the flexibility required to adjust to evolving political circumstances. Sustainability of the piloted policy instruments will be comprehensively supported through dedicated TA that will include the elaboration of specific reform measures aiming institutionalization of the new programs during the project life. Rating: Moderate. 129. Fiduciary risk. The fact that the beneficiaries of the funding schemes – researchers in RIs and universities, as well as firms – have heterogeneous fiduciary discipline and oversight capacity is a plausible risk to the project. In addition, the Page 37 of 60 The World Bank Modernizing Uzbekistan National Innovation System (MUNIS) Project (P170206) lack of familiarity of the PMU with the World Bank’s procedures, common to any project start, will be compounded by the requirements related to the funding schemes. The project addresses this challenge in three ways: (i) as described before, making resources available for the PMU to hire the number of staff that is commensurate to the task at hand; (ii) by adopting a single financial management manual that is common to all funding schemes and, thus applicable to all project beneficiaries; and (iii) centralizing in the PMU the primary responsibility for project financial management. Rating: Moderate. 130. Environmental and social risks. The project is not expected to have any large, significant, or irreversible environmental impacts as it comprises primarily TA and matching grants for improving national capacity in R&D. The project financed activities, including all grants, will be screened based on the ESMF, which outlines the guiding principles of environmental screening, assessment, review, management, and monitoring procedures for two each type of activity financed by the project under Components 1 and 2. Additionally, the ESMF describes roles and responsibilities in carrying out environmental work during the project implementation. The environmental due diligence procedures identified in the ESMF comply both with the Uzbek national laws and regulations, as well as the World Bank’s Environment and Social Framework. Given the capacity-building nature of the project, neither the funding schemes nor the TA activities are expected to have significant negative social impacts. The project needs to pay special attention to ensure that the funding schemes are equally accessible to all qualified applicants without discrimination of people with disabilities, ethnic and linguistic minorities, and persons from various regions, gender, ages, and socio-economic background. Rating: Moderate. 131. Stakeholder. The MID is new and has not yet established effective coordination and collaboration with other line ministries and other stakeholders in the system. Therefore, the project aims to help address this challenge by strengthening the MID’s leadership role through early success stories and TA and capacity building in developing and coordinating quality reforms. In addition, as part of consensus-building activities, the project will support the development of a broader stakeholder’s consultation process as well as an effective communication strategy. Rating: Moderate. 132. Overall project risk is Moderate following all of the SORT ratings. These ratings reflect the new Guiding Principles in Risk Assessment and the Application of SORT issued by the Operations Policy and Country Services (OPCS) in April 2020. Page 38 of 60 The World Bank Modernizing Uzbekistan National Innovation System Project (P170206) VII. RESULTS FRAMEWORK AND MONITORING Results Framework COUNTRY: Uzbekistan Modernizing Uzbekistan National Innovation System Project Project Development Objectives(s) The PDO is to enable the development of a market-oriented National Innovation System in Uzbekistan. Project Development Objective Indicators RESULT_FRAME_TBL_ PD O Indicator Name PBC Baseline End Target Market-oriented national innovation system in Uzbekistan Papers published in internationally peer-reviewed journals, 0.00 50.00 based on research funded by the project (Number) Investment in research commercialization and innovation 0.00 24,000,000.00 mobilized by the project (Amount(USD)) SMEs with internationally recognized certification funded by the 0.00 90.00 project (Number) Regulations disciplining the allocation of competitive funds for research and innovation reflecting the best practices adopted by No Yes MID (Yes/No) PDO Table SPACE Page 39 of 60 The World Bank Modernizing Uzbekistan National Innovation System Project (P170206) Intermediate Results Indicators by Components RESULT_FRAME_TBL_ IO Indicator Name PBC Baseline End Target Component 1: Improving research excellence and commercialization R&D projects implemented with support of the project (Number) 0.00 50.00 Papers submitted for publication in peer-reviewed journals (Number) 0.00 70.00 of which co-authored with international authors (Number) 0.00 20.00 of which co-authored with or authored by young researchers 0.00 20.00 (Number) Institutional modernization plans completed (IMP) (Number) 0.00 3.00 Value of investment in research infrastructure (Amount(USD)) 0.00 4,500,000.00 Research commercialization proposals implemented with 0.00 20.00 support of the project (Number) of which implemented by female beneficiaries (Number) 0.00 7.00 Component 2: Promoting innovation in the private sector Enterprise R&D projects funded (Number) 0.00 30.00 Investments in knowledge-based start-ups mobilized by the 0.00 4,000,000.00 project (Amount(USD)) Enterprise-industry research contracts concluded (Number) 0.00 15.00 SMEs that introduced a new product/service (Number) 0.00 500.00 Co-sponsor of the Supplier Development Initiative selected by No Yes MID (Yes/No) Grievance mechanism for all grants programs is established and No Yes implemented (Yes/No) Satisfaction rate of grant applicants on the application process (reflecting transparency and meritocracy) (Percentage) 0.00 75.00 Page 40 of 60 The World Bank Modernizing Uzbekistan National Innovation System Project (P170206) RESULT_FRAME_TBL_ IO Indicator Name PBC Baseline End Target Component 3 – Supporting STI policy-making and reforms Workplan for the establishment of the organizations managing research and innovation funding agreed between MID and core No Yes NIS stakeholders (Yes/No) Regulations for the management of research and innovation funding reflecting international best practices agreed between No Yes MID and NIS stakeholders (Yes/No) Technology Transfer Unit (TTU) created; staff trained and knowledge and strategy for research commercialization and No Yes policies adopted by MID (Yes/No) IO Table SPACE UL Table SPACE Monitoring & Evaluation Plan: PDO Indicators Methodology for Data Responsibility for Data Indicator Name Definition/Description Frequency Datasource Collection Collection Number of citable documents that are funded by the Research Excellence MID, PIU, Program of the project. Papers published in internationally peer- Academy of Scimago Journal defines annually interviews PMU reviewed journals, based on research Sciences, citable documents as funded by the project universities exclusively articles, reviews and conference papers (usually called country’s scientific output) that are Page 41 of 60 The World Bank Modernizing Uzbekistan National Innovation System Project (P170206) published during the selected year by a journal. Other equivalent definitions from equivalent or similar organizations can be considered. The total amount of funds dedicated to research commercialization and innovation that are mobilized by the Research Commercialization Programs (RCP) and Business Investments in R&D Program (BIRD) of the project. Investment in research commercialization bi-annually statements interviews PMU and innovation mobilized by the project Research commercialization includes the value of funds allocated for proof of concept, prototyping, initial batch of products for testing, and market patenting amongst others, including pure start-up funding. Number of SMEs – firms PIU, SMEs with internationally recognized with less than 100 bi-annually uzstandard, interviews PMU certification funded by the project employees – with smes internationally recognized Page 42 of 60 The World Bank Modernizing Uzbekistan National Innovation System Project (P170206) quality certification, including ISO 9000 or 14000, or HACCP. Internationally recognized quality certification refers exclusively to internationally recognized certifications: ISO (International Organization for Standardization) for manufacturing and services, HACCP (Hazard Analysis and Critical Control Point) for food (especially, but not exclusively, for seafood and juices), and AATCC (American Association of Textiles Chemists and Colorists) for textiles. Certificates granted only nationally and not recognized in international markets are excluded. Certifications provided by the multinational companies are also included for the purpose. Regulations disciplining the allocation of The indicator captures the bi-annually legislation legislation monitoring PMU competitive funds for research and adoption of regulations by Page 43 of 60 The World Bank Modernizing Uzbekistan National Innovation System Project (P170206) innovation reflecting the best practices MID that establish the adopted by MID guiding principles and rules of the allocation of competitive grants to support resesarchers and firms in line with the best international practices for allocation of competitive funds. Best practices require the following attirbutes: (1) meritocracy - the minimum international peer review and autonomy of decision- making body; (2) transparency - the clarity on the selection criteria and the predictability (advance notice) of proposal calls and procedures on the calls; (3) accountability - a clear responsibility attributed to decision-makers and decision-making process is well documented and the documentation is publicly available to the extent possible on the process and accountability outlining the responsibilities of the Page 44 of 60 The World Bank Modernizing Uzbekistan National Innovation System Project (P170206) parties involved. ME PDO Table SPACE Monitoring & Evaluation Plan: Intermediate Results Indicators Methodology for Data Responsibility for Data Indicator Name Definition/Description Frequency Datasource Collection Collection Total number of implemented R&D projects R&D projects implemented with support financed by Research bi-annually PMU, IEB financial statements PMU of the project Excellence Program (REP – US$ 10 million) of the project. Number of papers that are Academy of submitted for publication in Sciences, Papers submitted for publication in peer- peer-reviewed journals. bi-annually interviews, surveys PMU SRIs, reviewed journals Submissions do not universities necessarily mean publication selection. of which co-authored with international authors Young researchers include of which co-authored with or junior associates between authored by young researchers 25 and 34 years of age. Number of institutional modernization plans that Institutional modernization plans have been completed with Annually PMU approved IMP PMU completed (IMP) the support of Public Research Institutes Modernization (PRIM -- US$ Page 45 of 60 The World Bank Modernizing Uzbekistan National Innovation System Project (P170206) 6 million). Number of research infrastructure projects Value of investment in research supported through Public quarterly PMU, PROs PRO reports PMU infrastructure Research Institutes Modernization (PRIM, US$ 6 million). Number of research commercialization proposals that are funded by Research Commercialization Program (RCP – US$ 4 Million) of the project. PMU Research commercialization Research commercialization proposals quarterly financial financial statements PMU includes the value of funds implemented with support of the project statements allocated for proof of concept, prototyping, initial batch of products for testing, and market patenting amongst others, including pure start-up funding. of which implemented by female beneficiaries Number of enterprise R&D financial projects that are funded by quarterly review of statements PMU Enterprise R&D projects funded statements the Business Investments in R&D Program (BIRD Page 46 of 60 The World Bank Modernizing Uzbekistan National Innovation System Project (P170206) program -- US$ 10 Million) of the project. Knowledge-based start-up is a company as defined by international organizations (e.g. UNIDO, OECD) whose purpose is to bring PMU, Investments in knowledge-based start-ups technology products or quarterly participant interviews PMU mobilized by the project services to market. These firm companies deliver new technology products or services or deliver existing technology products or services in new ways. Number of contractual firms, Enterprise-industry research contracts research agreements monthly interviews, surveys PMU accelerators concluded concluded between enterprises and industries. Number of firms that introduced (a) products or services that are new to the market and (b) products or WBES services that have significant Note: firm-level survey improvements in participant data is weighted to SMEs that introduced a new bi-annually interviews, SME surveys capabilities, user SMEs obtain the product/service friendliness, components or representativeness for sub-systems. Significant the country improvements include new or significantly improved capabilities or other functions; technical Page 47 of 60 The World Bank Modernizing Uzbekistan National Innovation System Project (P170206) specifications; components and materials; incorporated software; user friendliness. This does not include: i) minor changes; ii) regular seasonal changes; iii) routine upgrades, iv) purely aesthetic design changes that do not affect functionality; v) the resale of a good purchased from other enterprises The indicator aims to capture MID’s selection of the co-sponsor of the Co-sponsor of the Supplier Development Supplier Development quarterly PMU, MID monitoring PMU Initiative selected by MID Initiative (SDI) financed by the Innovation Capabilities Program (ICP, US$10 Million) of the project. A grievance redress mechanism will be established for each grant program to allow for Grievance mechanism for all grants PMU PMU appeals, complaints and programs is established and implemented feedback and ensure that the programs remain transparent, meritocratic and responsive to their Page 48 of 60 The World Bank Modernizing Uzbekistan National Innovation System Project (P170206) targeted beneficiaries. The PMU will be responsible for administering an online survey after each call for proposals for the grants programs, aimed at the targeted beneficiaries Following (including those who had each successful applications, completed Satisfaction rate of grant applicants on rejected applications, or call for relevant grant Online survey Online survey the application process (reflecting who did not apply) to proposals administering body transparency and meritocracy) receive feedback on their for each perceptions of the grant meritocracy and program. transparency of the process. The feedback will be reviewed by the grant administrating body to improve the subsequent calls for proposals. The indicator aims to capture the agreed between MID and the core NIS Workplan for the establishment of the stakeholders workplan on MID, NIS organizations managing research and the establishment of the bi-annually interviews PMU stakeholders innovation funding agreed between MID organizations managing and core NIS stakeholders research and innovation funding, such as National Science Foundation and Innovation Agency. Page 49 of 60 The World Bank Modernizing Uzbekistan National Innovation System Project (P170206) The indicator aims to capture the agreement between MID and NIS stakeholders on the regulations for the Regulations for the management of management of research research and innovation funding MID, NIS and innovation funding quarterly interviews PMU reflecting international best practices stakeholders reflecting international best agreed between MID and NIS practices. stakeholders The criteria of best practices are defined under PDO indicators. The indicator aims to capture the establishment of the TTU; provision of Technology Transfer Unit (TTU) created; trainings to staff; and the PMU, TTU, staff trained and knowledge and strategy annually interviews, surveys PMU adoption by MID of MID for research commercialization and knowledge and strategy for policies adopted by MID research commercialization (as defined in PDO indicators) and policies ME IO Table SPACE Page 50 of 60 The World Bank Modernizing Uzbekistan National Innovation System Project (P170206) Annex 1: Summary of Strategy for Innovative Development (2019-2021) Table 1: Summary of Strategy for Innovative Development STRATEGY FOR INNOVATIVE DEVELOPMENT OF THE REPUBLIC OF UZBEKISTAN FOR 2019 – 2021 Goal: Development of human capital as the main factor determining the level of competitiveness of a country on the world stage and its innovative progress. Objectives: • Securing a position among the top 50 countries in the Global Innovation Index by 2030; • Improving the quality and coverage of education at all levels; • Strengthening scientific potential and integration of education, science and entrepreneurship; • Increasing public and private investment in innovation, R&D, and facilitating access to finance for innovation and R&D; • Increasing the efficiency of public authorities; • Ensuring IPR protection, facilitating doing business and PPP; • Creating a sustainable social and economic infrastructure. Reform areas, undertaken actions and exemplar target indicators Reform Areas Undertaken actions Target Indicators for 2030 (examples) 1. Development of science, innovation and technology transfer 1. Attracting young people to science ∙ The “Fund for Supporting Innovative ∙ Patent applications by 2. Creating centers for the collective use of Development and Innovative Ideas” was origin: 26.8 scientific equipment. established under MID and funds ∙ PCT international 3. Improving the prestige of scientific competitive innovation grants. applications by origin: activity. ∙ 20 Technology and Innovation Support 10 4. Creation of database of scientific and Centers (TISCs) were established in ∙ Utility model technical information. universities and research institutes applications by origin: 5. Improving mechanisms for IP evaluation. under a project implemented with the 32,0 6. Creation of national/regional technology WIPO till December 2019. ∙ Scientific and technical transfer offices. ∙ Center for Scientific and Technical publications: 45.0 7. Creation of a technology parks, free Information under MID was created in ∙ Citable documents: 16 economic zones, free industrial zones. 2019 ∙ H-index:16.0 8. Formation of joint organizations with ∙ The Center for Advanced Technologies foreign partners for the production of ∙ ISO 9001 quality (CAT) established. It implements a CAT certificates: 17 high-tech products. Science Accelerator in Life Science. ∙ High-tech Export: 24 ∙ Regulatory sandbox in the development ∙ Foreign direct of blockchain technology and the investment net sphere of crypto assets circulation was outflows: 49.0 launched. ∙ New business ∙ Business incubators and accelerators density:13.5 programs were set up. ∙ ISO 14001 ∙ The Technopark ‘Yashnabad’ was environmental formed in 2017 equated to the certificates: 33 innovation zone. It hosts 33 tenant ∙ High-tech companies (as of May 2020) and mid -tech products: ∙ Agency for Space Research and 38 Technology (‘Uzbekcosmos’) was created in 2019. ∙ The Presidential Fund for the Commercialisation of the Results of Scientific and Scientific-Technical Activities was set up. Yet, there are Page 51 of 65 The World Bank Modernizing Uzbekistan National Innovation System Project (P170206) difficulties in implementation of its projects. 2. Improving the system of financing innovation 1. Increase of GERD. ∙ A cooperation platform was developed ∙ GERD 0.8 percent of 2. Access to finance for innovation by MID that contains ready-to- GDP by 2021 (innovative funds, innovative banks, implement innovative projects mainly ∙ Ease of getting credit: venture capital funds). for private sector. 70 3. Simplification of procedures for bank ∙ In 2017, the State Fund for Support of ∙ Microfinance loans for innovation, improving the Business Development was created institutions' gross loan microfinance system. under the Cabinet of Ministers. The portfolio: 12 4. Stimulating the participation of the private fund provides guarantees to small ∙ Market capitalization: sector in innovation development. business entities on loans of 24 5. Creation of a unified database of commercial banks. ∙ Venture capital deals: innovative projects and potential 30 investors. ∙ Joint venture / strategic 6. Simplification of procedures and alliance deals: 12.5 transparency of grant programs. 3. Development of infrastructure and ICT 1. Improving the regulatory system of the ∙ The IT-Park was created in Tashkent to ∙ ICT Access: 100 percent telecommunications sector, including by support start-ups and ICT export. ∙ ICT use: 45 percent expanding the participation of business ∙ ‘Learning and technology centers’ were ∙ E- Government Services: entities on the basis of public-private set up to train young people in ICT and 63 partnerships. help them develop business ideas. The ∙ E- participation index: 2. Increasing the share of export of ICT program is coordinated by the IT Park. 62 services to 4 percent by 2021. ∙ «1 million programmers» project was ∙ ISO 14001 3. Liberalization of access to international launched in 2019. Free distance learning environmental telecommunication networks. is provided for young population in IT certificates: 33 4. Stimulating population to use ICT. specialties. 5. Development of the domestic software industry through the creation of an industrial park for start-up projects. 4. Improving the education system and developing human capital 1. Improving quality of training in HEIs, ∙ New education law has been drafted. ∙ Public spending on introduction of distance learning. ∙ Distance vocational learning platform education 2. Development of inclusive education. has been launched in May 2020. ( percent of GDP): 46 3. Quality assurance. ∙ Academic Innovation Fund was created ∙ Assessment in reading, 4. Increasing the coverage of higher with a budget of about USD 4million math and science: 480 education. and 32 collaborative grant projects are ∙ Tertiary enrolment: 60 5. An increase in the proportion of students under implementation targeted at ∙ Graduates in science in the areas of natural and technical improving the relevance and quality of and engineering: 50 sciences. education (Project funded by the World ∙ Researchers: 24 6. Strengthening the research component of Bank). HEIs. ∙ A Decree was approved in April 2020 7. Creation of Foresight centers in leading indicating that: 30 percent of the total universities. admission quotas for state for master's programs and advanced training courses will be allocated to women annually; separate grants for women at the universities will be allocated. Page 52 of 65 The World Bank Modernizing Uzbekistan National Innovation System Project (P170206) ∙ Higher Education Management Information System (HEMIS) is under implementation (Project funded by the World Bank). ∙ Internal and External Quality Assurance system for HEIs is under implementation (Project funded by the World Bank). 5. Development of competition and reduction of administrative barriers 1. Development of modern corporate law. ∙ Changes in Tax Code were introduced ∙ Regulatory quality: 70 2. Formation of a competitive market starting from 2020. Number of taxes ∙ Rule of law: 65 stimulating transition to economy based were reduced. ∙ Control of corruption: on innovation. ∙ The Law “On Investments and 77 3. Improving antitrust policy, ensuring non- Investment Activities” was adopted ∙ Ease of creating a discriminatory access to goods, works and increasing investment attractiveness for business: 95 services of natural monopolies for all legal foreign Investors. ∙ Ease of protecting entities and individuals, introducing ∙ A public sector innovation ‘mGov minority investors: 80 effective pricing mechanisms for products Award’ was launched. ∙ Creation of a unified of monopolistic enterprises. electronic register of 4. Creating equal conditions for carrying out real estate. business activities and preventing monopolization of the domestic market. ∙ Publication of master 5. Ensuring digitalization of the conduct of plans for territories. business and simplification of the ∙ Introduction of registration of property. advanced land 1. 6. Increase in the share of electricity allocation mechanisms. production using renewable and ∙ Increase of renewable alternative energy sources by more than and alternative energy 20 percent by 2025. sources by more than 20 percent by 2025. Page 53 of 65 The World Bank Modernizing Uzbekistan National Innovation System Project (P170206) Annex 2: Women’s Participation in Science in Uzbekistan 1. The number of female R&D personnel in Uzbekistan has slightly increased between 2012-2017 to reach 15.1 thousand (headcount criteria), which corresponds to around 0.24 percent of total female labor force in the country. 72 Of all R&D personnel (full-time equivalent), most of the female R&D personnel is concentrated in private non-profit field of activities (around 50 percent), followed by government (41 percent), higher education and business enterprises (40 percent). 2. UNESCO Institute of Statistics estimates that less than 30 percent of the world’s researchers are women. 73 Yet, female researchers have been performing relatively well in Uzbekistan. The share of female researchers was stable in the period between 2012 and 2017 at about 42 percent of total headcount, which is on average lower by ten percentage points than in such peer economies as Kazakhstan, North Macedonia, Georgia, and Armenia. 74 This is also around four percentage lower than the average in Central Asia. Of the total female researcher headcount, the estimated number of full-time equivalent (FTE) is less than half (48 percent), indicating that female researchers spend on average more than half of their time with other activities – compared to only 11 percent in Serbia. 3. When both headcount and full-time equivalent are considered, of all researchers, more than half (around 55 percent) of female researchers are concentrated in medical and health sciences, followed by social sciences and humanities (close to 50 percent) and natural sciences and engineering (around 35 percent). Comparatively, economies with the highest representation of female researchers in natural sciences and engineering are Myanmar (76 percent) and Mongolia (61 percent). Yet, the sectors where female researchers are the least represented (26 percent) are in the field of agriculture and veterinary services, which is the opposite case with more than 50 percent representation in such economies as Belarus, Ukraine, Serbia, Moldova, and Algeria. 4. Distribution of female researchers per age cohort seems promising, with more than a third of all female researchers (when headcount is considered) being concentrated in the 35-44 age cohort, followed by 27 percent in the 25-34 age group, - generally the most productive age cohorts. Only 22 percent of female researchers are in the 45-54 age cohort, while the least represented are the female researchers in the age cohort of 65 and more and less than 25 (around 8 percent of total female researchers). Representation is heterogeneous across different fields of activities. When headcount is considered, predominant share of all female researchers (more than 80 percent) conservatively remain in the field of higher education, followed by government (11 percent), business entities (7 percent) and private non-profits (0.4 percent). In contrast, some of the economies with the largest concentration of female researchers in business entities are the Republic of Korea (more than 55 percent) and Russian Federation (more than 45 percent). 5. Room for improvement exists in formal qualification of female researchers where the achievement is yet limited. Of all female researchers, only 25 percent had a PhD or equivalent level of education (ISCED level 8) in 2017, which is below Egypt (51 percent) and Turkey (36 percent), and yet above such frontrunners as the Republic of Korea (22 percent) and Japan (20 percent). The vast majority of female researchers in Uzbekistan (68 percent) had a MSc or equivalent (ISCED level 7) in 2017. In terms of career development, only 9 percent of female researchers are at the highest possible grade/post of seniority (Category A) in their careers, which is 5 percentage points lower than, for example, in Egypt and Georgia, indicating a potential for progress of younger female researchers. 72 Based on WDI, total female labor force – 6305110, total labor force in Uzbekistan – 15,453,701. 73 UNESCO, “Women in Science – 2019”. 74 UNESCO database. Page 54 of 65 The World Bank Modernizing Uzbekistan National Innovation System Project (P170206) Annex 3: Economic and Financial Analysis Overview 1. This economic analysis follows the World Bank guidance note on economic analysis of investment project financing and uses ex-ante cost-benefit analysis framework to assess and monetize costs and benefits of implementation of the proposed project (MUNIS) and associated externalities. In particular, it looks into direct (financial) costs associated with implementation of MUNIS and indirect costs associated with deadweight losses due to collection of taxes and possible environmental damages. On the benefits side, the analysis covers financial and economic revenues generated by innovative activities facilitated by the project. The analysis arrives at Net Present Value of US$ 6.2 million in the baseline case. 75 2. Public rationale. Public intervention in R&D commercialization in Uzbekistan is justified by two well-known features of R&D investments: 1) information asymmetries and coordination failures that prevent efficient private investments and 2) economies of scope and other positive externalities of establishing strong industry-R&D links and innovation consortia that cannot be captured in form of private returns and thus require public investments. 3. World Bank’s value added. World Bank actively participated in designing and implementing development projects in the country (e.g., the Modernizing Higher Education Project) and is seen as long-term strategic partner of Uzbekistan. The World Bank will be able to draw from the lessons learned of the existing projects in Uzbekistan and similar science commercialization projects implemented worldwide. 4. Standing. The analysis is carried out from the national perspective of Uzbekistan. The parties with standing are: (a) Uzbek taxpayers since reimbursement of the World Bank’s loan allocated to the MUNIS would be financed from the state budget. Uzbek taxpayers are thus affected in two ways: increased direct tax burden and deadweight loss due to distorted decisions caused by taxes. At the same time, this group will benefit from increased innovative production, increased exports of high-value-added products and enhanced capacity for innovative activities. (b) Research institutes, universities, R&D groups and laboratories will benefit from direct financial inflows and indirect externalities in form of innovation-enabling environment, improved opportunities for investment attraction, diversification of sources of financing S&T/R&D activities. (c) Uzbekistan Ministry of Innovative Development and line ministries will bear indirect expenditures in the form of increased workload of the personnel and/or direct expenditures to hire new employees to maintain activities in the frame of the MUNIS. (d) Uzbek universities and technology firms that would receive increased funding or equity investments/technological upgrade. (e) Uzbek small and medium enterprises will benefit from increased access to finance to undertake new initiatives, and/or broaden the product or services range, and/or improve the existing product quality and production efficiency. (f) Uzbek companies and individuals involved in design and implementation of the project would receive benefits 75 The technical note is based on the Ramsey equation of social discount rate (Ramsey, 1928). The equation assumes a particular iso-elastic functional form for utility, constant population, no inequality within society and perfect certainty. The note suggests setting the value of β in a way that reflects utility discount rate. In our analysis, a good proxy for this rate is the real rate of interest yield of saving accounts (nominal yield minus inflation rate) denominated in domestic currency (7.2% average in 2013-2019). The expected growth rate of per capita consumption (σ) in Uzbekistan is set at 4.4%, equal to average projected GDP growth (a suitable proxy for σ) in 2020-2021 as per IMF’s latest World Economic Outlook. Plugging these parameters into the Ramsey formula yields a social discount rate of 11.6% (7.2+1*4.4). Page 55 of 65 The World Bank Modernizing Uzbekistan National Innovation System Project (P170206) in form of rents and salaries. 5. Assumptions. The economic analysis will consider costs and benefits of the MUNIS based on following assumptions: (a) The project lifetime is 2020-2025 (i.e. no project extension); (b) The most of funds allocated to all components of the project will be disbursed in the period of 2021-2025; (c) Ex-ante calculation of NPV under the base case uses a 11.6 percent social discount rate (SDR) that has been derived in the frame of the technical guidance note jointly issued by the Bank’s DEC and OPCS units on January 20th, 2016[1]. However, the discount rate assumption will be altered to calculate net benefits in the pessimistic case scenario for sensitivity analysis. (d) The time horizon covered by the cost-benefit analysis is the period from 2020 till 2030. The 11-year timeline is dictated by the assumption that major quantifiable benefits of project activities will be incurred between year 3 and year 9 after the project launch. 6. Benefits calculated in this analysis do not include taxes paid by beneficiaries as well as goods and services purchased on competitive domestic markets (e.g. furniture, cleaning services, office rent) as these are considered transfers. For example, a benefit to state budget in the form of taxes generated by the MUNIS PIU, startups or consortia is a loss to the taxpaying party (that is, the PIU or a grantee). Similarly, revenue received in competitive markets by third- party suppliers as a result of MUNIS activities (e.g. a landlord renting out the PIU office) are also considered transfers since they are equal to the opportunity cost of suppliers operating on a competitive market. 7. Other key assumptions made while estimating costs and benefits (reported later in this Annex). All present values presented in the analysis are in terms of 2020 US dollars, rounded to nearest decimal, unless noted otherwise. All sums in national currency were translated into US dollar equivalent using official exchange rate of National Bank of Uzbekistan as of May 2020. 8. Counterfactual. In general, construction of a counterfactual scenario is a complicated process since it needs information about an activity that has not occurred. Hence it is intuitive to use information on baseline trends from the period before the project. For the purpose of this analysis, the task team used the “no project” counterfactual – i.e. assuming the government of Uzbekistan continues existing science commercialization programs. Based on available data and discussions during project preparation and appraisal missions, there are significant differences in the type and scale of support of science commercialization projects that are summarized below: (a) The average annual amount of funding of startups, applied and targeted R&D is US$ 4.5 million 76 (i.e. US$ 22.5 million between 2021 and 2025) (b) The average amount of grant financing is slightly less than UZS 560 million (US$ 55,400) per research group77. (c) The selection process in a counterfactual scenario is cumbersome and may take up to one year (mostly due to dependence on the budget cycle). (d) Selection and commercialization support of subprojects are carried out by institutions that existed before MUNIS launch, and the counterfactual assumes no establishment of the IEB, TTO and limited involvement of international consultants. 9. Therefore, these differences between the MUNIS and the counterfactual translate into significant differences in 76 The total amount of “innovation”, “practical”, “startup” and “targeted” grants (as per MID classification) in 2019 amounted to US$ 4.52 million as per data shared by MID. 77 The average size of “innovation”, “practical”, “startup” and “targeted” grants (as per MID classification) in 2019 amounted to UZS 197.1 million (US$ 20,000) as per data shared by MID. Of these grants, the majority have a 2-year term. Page 56 of 65 The World Bank Modernizing Uzbekistan National Innovation System Project (P170206) cost structure and flows of benefits. In particular, it is assumed that in the counterfactual case, (a) beneficiaries are given innovation grants of smaller size; (b) application and compliance costs to beneficiaries would be higher due to bureaucratic reporting and eligible expenditure constraints; and (c) smaller size of grants, less flexible design, and absence of commercialization support would limit results of grant financing to a lower level compared to the MUNIS. Calculation of Costs and Benefits 10. The potential costs and benefits of the MUNIS project can be divided into two groups: 1) direct costs and benefits associated with launch and administration of the project and 2) side effects (externalities) of the project. These are presented in Table 1 below. Table 1. List of potential impacts of MUNIS COSTS Group with standing Component Description Uzbekistan taxpayers Component 1 Direct financial outflows on grants to research teams and PhD scholars. (individuals and Component 2 Direct financial costs of proof-of-concept/prototype development grants. firms) Component 3 Operating costs of strengthening institutional capacity; remuneration expenses of members of the International Expert Board etc. Direct financial outflows on establishment and operation of Technology Transfer Unit. Component 4 Direct financial outflows related to project implementation. All components Distortionary cost of taxation. Components 1,2 Environmental impact of upgrading existing laboratories and implementation of innovative projects in frame of MUNIS. Research institutes, Components 1, 2 Administrative expenses incurred by R&D groups during application for universities and R&D grants, preparing business plans and performance reports, expenses groups related to compliance with formal requirements. Ministry of All components Indirect expenditures in form of increased workload of their personnel Innovative and/or direct expenditures in form of hiring new employees to maintain Development, other activities in frame of MUNIS. state agencies BENEFITS Uzbekistan taxpayers All components Increased access to innovative goods and services, increased employment in industries with high value-added, establishment of innovation-enabling environment Ministry of All components Increased institutional capacity for science commercialization. Innovative Development, other state agencies Uzbek residents Component 1 Increased income in form of wages in underemployed R&D market obtaining PhD training in frame of MUNIS Uzbek enterprises Component 2 Increased market capitalization due to investments into capital stock, human resources and intangibles. Page 57 of 65 The World Bank Modernizing Uzbekistan National Innovation System Project (P170206) Calculation of Costs 11. General and administrative costs. The current analysis considers three groups of administrative costs to be incurred by MUNIS stakeholders: (a) up-front administrative costs during the stage of design, public awareness raising etc.; (b) one-off administrative costs during selection process (grant program applicants); and (c) recurrent compliance costs incurred by MUNIS grantees. The first group of administrative costs comes in the form of increased workload and/or recruiting of new employees by state agencies engaged in design and launch of the project, negotiations over the IBRD Loan Agreement, and obtaining clearance from line ministries and parliament. Available data and experience from similar projects 78 suggest that 6,240 person-hours are spent by public offices at this stage (see Table 2). Applying relevant nominal average wages as of Q1 2020, the present value of the estimated administrative costs of group (a) amounts to US$ 20,000. As these are costs related to the launch of MUNIS, these are not included in calculation of the “no-project” counterfactual case costs. Table 2. Estimated Number of Public Employees Engaged and Estimated Time Spent Number of Employees Time Spent by each Employee Total Person-hours per Year (hours) per Year Spent per Year MID 10 400 4,000 MoF 5 400 2,000 Other (MoIFT, Academy of 3 80 240 Sciences) “No-project” counterfactual 0 0 0 12. Calculations of the one-off administrative costs of applying for grants and recurrent compliance costs in the scope of Components 1 and 2 are based on the following data. Table 3. Applicants Engaged in Selection Process and Estimated Time Spent Number of Number of Application costs Recurrent compliance costs applications selected projects (person-hours) (person-hours) per year Component 1 (REP grants) 300 44 2,400 88,000 Component 1 (RCP grants) 200 27 1,600 60,000 Component 1 (PRIM grants) 5 3 40 18,000 Component 2 (SDI grants) 200 100 1,600 600,000 Component 2 (BIRD grants) 100 32 800 128,000 Counterfactual 680 226 6,800 565,000 13. In the first stage of selection in the scope of grant programs of the MUNIS, applications are sent online and do not require significant investments on the applicants’ side. It is thus assumed that each applicant spends approximately 8 person-hours to apply for a MUNIS grant. Once selected, research teams will incur cost of incorporating themselves as companies. Available data suggests that establishing a company in Uzbekistan will take 3 working days and about US$ 22 78 For instance, Technology Commercialization Project in Kazakhstan. Page 58 of 65 The World Bank Modernizing Uzbekistan National Innovation System Project (P170206) in registration fees. 79 Based on average cost of labor and registration fees, cost of establishing a company is around US$ 40. Assuming that 44 research groups and 27 proof-of-concept applicants will receive grants, cost of establishing 71 companies would total to about US$ 3,000. 14. Yet, once they pass the selection stage, grantees (R&D research grants in Component 1.1, proof-of-concept grants in Component 1.2, SMEs and startups under ICP and BIRD programs) incur larger recurrent compliance costs, which, on average, result in costs equivalent to hiring 1 full-time employee (usually a person in charge of financial management and procurement) throughout the project lifetime. Considering the planned duration of the MUNIS grant programs, relevant nominal monthly salaries, the present value of these two groups of administrative costs amounts to US$ 1.6 million. 15. At the same time, application and compliance costs in the counterfactual scenario are assumed to be higher by 25 percent per grantee due to absence of online application channels and the larger number of participants in the application and reporting procedures. In particular, application for grants would require engagement of university/research institute management (as grant funds are accrued to budgets of universities/ research institutes before research groups receive them). As a result, the expected value of these two groups of administrative costs amounts to US$ 0.6 million in the counterfactual case. 16. Direct financial outflows. Apart from the indirect administrative expenses mentioned above, another significant group of costs are direct financial outflows in the form of research and commercialization grants and consultancy contracts, as in Table 4. Table 4. Direct Financial Outflows Component/ US$ million % Counted toward Economic Absolute Value Counted toward subcomponent Cost Economic Cost (US$ million) Component 1 grants 20 96 24 Component 2 grants 20 100 2 Project implementation 2 91 1.82 Counterfactual 22.5 96 21.6 17. Component 1 – Improving research excellence and commercialization. The component will finance three types of grants for eligible R&D ideas: (a) Grants for teams of researchers up to US$ 0.15-0.3 million each. The grant program would finance laboratory equipment, workshops, visiting scholars, etc. In all, US$ 20 million. would be allocated on research grants. (b) Research commercialization grants of up to US$0.075-0.15 million each. (c) Also, a total of US$ 6 million is allotted to Public Research Institute Modernization grant program. The average grant size is expected to be around US$ 2 million. 18. Here it is assumed that grant programs will be launched in the 2nd year of the project, after establishment of selection board, disbursement mechanisms and awareness raising among Uzbek S&T community. Thus, expenditures under Component 1 are relatively evenly disbursed in 2-3 rounds of calls for proposals between years 2 and 5 of MUNIS. 19. The primary costs of this component are associated with direct financial outflows in form of grants to research teams and PRIs modernization. The preliminary cost break of R&D projects obtained during the appraisal mission and data from similar projects indicate that the bulk of funding (circa 54 percent) would be spent on procurement of imported Page 59 of 65 The World Bank Modernizing Uzbekistan National Innovation System Project (P170206) equipment and reagents. The team considers those as cost since these goods (and accompanying installation/maintenance services) are imported and do not generate employment or rents domestically. The remaining 46 percent of funds would be spent on remuneration of researchers, knowledge exchange and operational expenditures. From financial costs and benefits point of view, this sum is counted towards cost. From economic point of view, this is a transfer of funds to residents and firms of Uzbekistan, since the amount of such expenditures is small relative to whole economy (and thus do not have distortionary impact on labor and goods markets) and if assumed that the new jobs were created and services were procured on competitive markets. What is counted towards economic cost is the opportunity cost of residents and firms that had to forgo salaries and rents they would earn elsewhere. In a full-employment economy this opportunity cost would be 100 percent of the observed salaries and rents, yet in practice (given 9% unemployment rate in 2019 and shadow markets) there is economic benefit associated with new employment generated by MUNIS subprojects. Here we use the shadow conversion factor of 91 percent 80, and assume that economic opportunity cost of services procured locally is equal to 91 percent of financial outflows. Based on this, 96 percent of costs of the research grant program are counted as cost. 20. Taking into account the above and given 11.6percent social discount rate (SDR), present value (PV) of total costs of Component 1 is US$ 15.5 million. 21. Component 2 – Promoting innovation in the private sector (US$ 20 million). This component will promote science- enterprise links through matching grants (between US$ 0.05 million and US$ 0.3 million) to companies for quality certification and small-scale production. It is known that expenditures made under this component will primarily be allocated on upgrading winning laboratories to international standards, while private co-financing will cover taxes, headcount and operational costs. At the same time, it is assumed that the equipment required to upgrade laboratories would be imported with no transfer of funds to local firms and residents. Taking into account above assumptions and SDR of 11.6 percent, PV of costs of Component 2 amount to US$ 16.1million. 22. Component 4 – Project management and M&E (US$ 2 million). This component will support project implementation activities, including the following: (a) Operations of a Project Implementation Unit (PIU) responsible for the day-to-day implementation of project activities, ensuring the disbursement of Loan proceeds in compliance with applicable World Bank rules, and rigorous monitoring of project indicators. (b) Maintenance and update of the project web-portal that will include systems for online application for grant programs, feedback collection and grievance management, etc. (c) Undiscounted direct financial outflows on project management, monitoring and evaluation is estimated at US$ 2 million. As mentioned above, what is counted towards economic cost of this sum is the opportunity cost of residents and firms that had to forgo salaries and rents they would earn elsewhere. Applying the shadow conversion factor of 91 percent and the SDR of 11.6% - PV of costs of Component 4 amounts to US$ 1.4 million. 23. Negative environmental and social externalities. The MUNIS project may also cause negative impact on environment due to possible emergencies at R&D sites, wastes generated by laboratories, field trials of new products and substances, etc. However, it is not possible to know beforehand precisely what research disciplines will be supported under the auspices of the project and the precise, detailed environmental compliance issues will emerge during the course of project implementation. It is expected that safeguard measures in frame of the project would comply with World Bank safeguard policies and procedures and with best international practices for laboratory safety, waste disposal and operating 80 One might use the employment rate (1 – the unemployment rate) as a proxy for the shadow conversion. Page 60 of 65 The World Bank Modernizing Uzbekistan National Innovation System Project (P170206) procedures. Therefore, in addition to helping to rebuild research and scientific capacity, the project will help to introduce world class safety and environmental procedures. Thus, potential environmental impact of the project is expected to be negligible and is not quantified in this analysis. 24. A considerable negative externality of the project is the deadweight loss resulting from distorted decisions of economic agents due to collection of taxes to finance the project. Existing literature on distortionary impact of taxes is extensive and estimates of this negative externality range from US$0.025 to US$0.30 per dollar of public funds raised through taxation. 81 The current analysis takes the median of this range at about 16 tiyins per 1 som of tax revenue. Given these assumptions, the present value of distortionary impact of taxes amounts to US$ 1.5 million. This is compared to the counterfactual case: R&D projects are financed from taxes collected in the preceding year, and the PV of such “cost of public funding” is estimated at US$ 2.9 million. Calculation of Benefits 25. The principal benefit of the project comes from sustainable development of innovative research and production in Uzbekistan through the following vehicles: (a) Accumulation of scientific infrastructure and human resources – prerequisites for cutting-edge R&D and accommodation of scientific knowledge from abroad; (b) Establishment of strong industry-R&D links and developing science commercialization ecosystem; (c) Addressing existing market failures and information asymmetries; (d) Strengthening institutional framework to support commercially-viable R&D activities; (e) Increasing attractiveness of scientific career among young generation, thus contributing to stable supply of human capital to NIS of the country; and (f) Diversifying local economy and increasing share in GDP of production with high value-added. 26. This is an incomplete list of important positive externalities of the proposed project, though barely quantifiable. This is mainly due to uncertainties related to the scope and scale of expected innovation activities, e.g. commercialization of R&D ideas in vast array of sectors ranging from development of new crops and varieties to mining and IT will impact different groups of population with differing scale at different time horizons. Through creation of new Such activities are expected to reduce extreme poverty and its perennial satellites - extremism and political instability. Given lack of relevant literature and data, monetization of these benefits will also require a large vector of assumptions and simplifications. Thus, monetary value of possible benefits put at micro level is doomed to be highly imprecise. Instead, current analysis tries to analyze macro-level impact of the proposed project based on available literature. 27. The well-known cross-country comparative study conducted by Lederman and Maloney (2004), shows that the social return rate of R&D in such lower-middle income country as Uzbekistan approaches 60 percent on average. That is each dollar spent on R&D in Uzbekistan is expected to generate gross gain of 1.60 dollars. Meanwhile time lag with which R&D expenditures impact economy is also important for cost-benefit analysis. Goto and Suzuki (1989) estimated the private and social return rates in various industrial sectors in Japan, and found that not only is there a lag in the impact of R&D activities but that the lags also vary depending on the industry. Specifically, they showed that the impact takes an average of two years in the case of electrical machinery, electronic and communication equipment parts, and mechanical 81 Feldstein, Martin. 1999. “Tax Avoidance and the Deadweight Loss of the Income Tax.” Review of Economics and Statistics 81 (4): 674. Ballard, Charles L., John B. Shoven, and John Whalley. “The Welfare Cost of Distortions in the United States Tax System: A General Equilibrium Approach.” Working Paper 1043, National Bureau of Economic Research. 81 Goto, Akira; Suzuki, Kazuyuki. 1989. R&D Capital, Rate of Return on R&D Investment and Spillover of R&D in Japanese Manufacturing Industries. The Review of Economics and Statistics, 71, (4), 555-64. Page 61 of 65 The World Bank Modernizing Uzbekistan National Innovation System Project (P170206) machines. This is a substantially different period than that for drugs and medicines whose lag period exceeds five years. The authors suggest that this heterogeneity is explained by technological differences between sectors or by different employee skill levels and competition. 82 28. At the same time, despite such high social return on innovation, developing countries in fact have relatively lower level of R&D expenditure. This “innovation paradox” 83 stems from the lack of complementary factors, including the poor firm capabilities and institutional capacity. The project is designed to try to address the issue of accumulating such capabilities (through (i) the demonstration effect of grant programs, selection and monitoring procedures novel to Uzbekistan; (ii) institutional capacity building; and (iii) building a pipeline of innovation-focused firms). This is what differentiates the project from the “no-project” counterfactual, whereby government would focus heavily on supply- driven support of NIS, whereby best practices are simply imported and imposed on existing actors and institutions with weak capabilities. 29. Following conservative approach of estimation, current analysis assumes time lag between expenditures and impact to be 4 years for R&D projects under Component 1 and 3 years for PhD grants. In contrast to grants under Component 1, it is assumed that grantees under Component 2 will already have their research results produced outside MUNIS. Hence Component 2 grants will focus solely on reaching commercialization targets (e.g. no publication requirements) and will have much shorter “time to market” (1-2 years after winning grant). 30. Other unquantified principal benefits of the project come from development of innovative research and production in Uzbekistan through the following vehicles: (a) Increased knowledge production. Contributions to knowledge from MUNIS subprojects usually measured through contributions to scientific publications, accumulation of human capacity. (b) Increased research capacity. Building research capacity that benefits future research activity. This includes the use of research information to improve targeting of future research, individual and group development of research skills and research capacity, development of the capability to use existing national or international research. (c) Building up commercialization capacity. Benefits to the NIS from training and developing the commercialization team of the TTO. (d) Building up institutional capacity to support commercially viable R&D activities. Accumulation of institutional knowledge that help take informed decisions by policymakers (including database of R&D, proof-of-concept applications); strengthening linkages between various agencies involved in the project; and benefits stemming from streamlining administrative procedures in the MID. (e) Increasing attractiveness of scientific career among the younger generation, thus contributing to stable supply of human capital to the NIS of Uzbekistan. (f) Diversifying local economy and increasing share in gross domestic product with high value-added. Calculation of NPV of the Project 31. Table 5 presents the NPV of estimated costs and benefits of the MUNIS from the perspective of Uzbekistan from 2020 to year-end 2030. All benefits and costs have been taken to their present value. According to the results shown in the table, NPV of the MUNIS and associated activities, when compared against the counterfactual, is US$ 3.2 million. 83 Cirera, Xavier; Maloney, William F.. 2017. The Innovation Paradox: Developing-Country Capabilities and the Unrealized Promise of Technological Catch-Up. Washington, DC: World Bank. Page 62 of 65 The World Bank Modernizing Uzbekistan National Innovation System Project (P170206) Sensitivity Analysis 32. Although the base case calculations show about US$ 6.2 million of net benefit, this figure may change in following scenarios: a higher SDR is applied; and, expected impact of the project on economy is in lower end of estimated range. 33. To test viability of the project against these arguments, the analysis recalculates present values of costs and benefits by using 13 percent SDR (vs. 11.6 percent in base case) and lowering projected social return rate by 20 percentage points. Application of a higher SDR has net negative effect as all costs of the project occur within first 5 years of the analysis’ timeline, while the bulk of benefits associated with growth of economy occur at a later stage. Meanwhile, lowering expected impact on the economy reduces benefits from the project and do not impact its costs. 34. Results of the base case calculation and alternative scenario are presented in Table 6 below. Table 6: Comparison of net benefits in base case and alternative scenarios (US$ million) Base case (11.6% SDR, 60% social rate of Pessimistic case (13% SDR, 40% social return) rate of return) Net benefits 6.2 3.2 Benefit-to-cost ratio 1.3 1.1 35. Other considerations. Due to time considerations and presence of large number of contingencies this cost-benefit analysis did not analyze in detail the “butterfly effect” of science commercialization in Uzbekistan on long-run employment and diversification of the economy beyond year 2030. The analysis neither included possible negative impact of the project on environment. Calculation of impacts on micro-level like rents to landlords, construction companies and service providers were also forgone as payments for their services and goods are expected to be negligible and equal to their opportunity cost. 36. In summary, the ex-ante cost-benefit analysis of the MUNIS shows positive net benefits amounting to US$ 6.2 million. At the same time, it should be noted that due to the data limitations, it does not capture (a) majority of economic benefits that will be realized beyond 2030 (i.e. 5 years after the project closure; and (b) significant positive externalities generated by the project (links between science and industry, increased capacity of state agencies and research groups) that could not be captured by imperfect or partial result indicators (for example, scientific publications, partnerships between researchers and enterprise sector). Page 63 of 65 The World Bank Modernizing Uzbekistan National Innovation System Project (P170206) Annex 4: Visual of implementation arrangements Figure 1.1 Implementation arrangements Page 64 of 65 The World Bank Modernizing Uzbekistan National Innovation System Project (P170206) Figure 1.2 Financial and procurement flows Page 65 of 65