The World Bank Enabling Digital Governance Project (P164824) REPORT NO.: RES56328 RESTRUCTURING PAPER ON A PROPOSED PROJECT RESTRUCTURING OF ENABLING DIGITAL GOVERNANCE PROJECT APPROVED ON APRIL 24, 2019 TO THE REPUBLIC OF SERBIA GOVERNANCE EUROPE AND CENTRAL ASIA Regional Vice President: Antonella Bassani Country Director: Xiaoqing Yu Regional Director: Asad Alam Practice Manager/Manager: Fabian Seiderer Task Team Leader(s): Shiho Nagaki, Srdjan Svircev, Tiago Carneiro Peixoto The World Bank Enabling Digital Governance Project (P164824) List of Abbreviations CFU Central Fiduciary Unit DRDC Disaster Recovery Data Center EDGE Enabling Digital Governance GOS Government of Serbia G2C Government to Citizens ICT Information and Communication Technology MC Management Committee MIT Ministry of Information and Telecommunications MoF Ministry of Finance MPALSG Ministry of Public Administration and Local Self-Government MTTT Ministry of Trade, Tourism, and Telecommunications OAMA Office Administration Management Application OITeG Office for Information Technologies and e-Government PIU Project Implementation Unit PDO Project Development Objective PPS Public Policy Secretariat SC Steering Committee VAT Value Added Tax The World Bank Enabling Digital Governance Project (P164824) BASIC DATA Product Information Project ID Financing Instrument P164824 Investment Project Financing Original EA Category Current EA Category Partial Assessment (B) Partial Assessment (B) Approval Date Current Closing Date 24-Apr-2019 30-Jun-2024 Organizations Borrower Responsible Agency The Republic of Serbia Project Development Objective (PDO) Original PDO To improve access, quality, and efficiency of selected administrative e-Government services. OPS_TABLE_PDO_CURRENTPDO Summary Status of Financing (US$, Millions) Net Ln/Cr/Tf Approval Signing Effectiveness Closing Commitment Disbursed Undisbursed IBRD-89470 24-Apr-2019 07-May-2019 25-Jun-2019 30-Jun-2024 50.00 21.82 26.86 Policy Waiver(s) Does this restructuring trigger the need for any policy waiver(s)? No I. PROJECT STATUS AND RATIONALE FOR RESTRUCTURING A. Project Status The World Bank Enabling Digital Governance Project (P164824) 1. Progress on the Enabling Digital Governance (EDGE) Project has been uneven due to implementation challenges related to procurement and the COVID-19 pandemic. The project was approved by the Board of Directors for EUR 43.8 million (US$55 million) on April 24, 2019 and became effective on June 25, 2019. Its project development objective (PDO) is “to improve access, quality, and efficiency of selected administrative e-Government services”. It has three components: (a) Foundations for Digital Service Delivery (US$26.7 million); (b) Transformation of Services (US$22.6 million); and (c) Digital Skills Development, Institutional Strengthening and Change Management (US$5.7 million). The project has undergone complex procurement activities since the effectiveness, affected by a conflict-of-interest issue and perverse market behaviors, which led to the repeated cancellation of procurement processes and relaunching them. In addition, during the COVID-19 pandemic, the Office for Information Technologies and e-Government (OITeG) had to shift its priorities to deal with the pandemic and provide related online services. The project performance ratings for the prospect of achieving the PDO (DO) and Implementation Progress (IP) were downgraded to Moderately Unsatisfactory (MU) in May 2021 given the slow implementation and repeated procurement problems. However, the pandemic and lockdown experience emphasized the importance of digitalizing public administration and services. The Government team led by the OITeG has been resilient and made continued efforts for due diligence in the procurement process and active project management. 2. The implementation and disbursement pace picked up in mid-2022 after all pending procurement activities were completed meriting an upgrade in the project performance ratings (both DO and IP) to Moderately Satisfactory (MS) in June 2022. A rapid implementation team between the World Bank, OITeG, and the Project Implementation Unit (PIU) was established in 2021 and biweekly meetings have been organized to proactively address implementation bottlenecks. This proactive management successfully led to the relaunch of the procurement of high-priority activities. The World Bank team has also cooperated with the Integrity Vice Presidency (INT) to manage procurement risks. In early 2022, OITeG completed all outstanding procurement activities with a total contracted amount of EUR 26.35 million (60 percent of the loan). Disbursement picked up by mid-2022 after more than one year of zero disbursement between 2021 and 2022. The accelerated implementation, particularly in Component 1, led to 39 percent of disbursement rate in September 2023 and more than 20 percent of the project funds disbursed in fiscal year 2023 alone. This led to the implementation progress (IP) being upgraded to Satisfactory in September 2023. 3. Progress toward achieving PDO remains MS and the PDO will unlikely be achieved by the current closing date of June 30, 2024. As of September 2023, two out of the four PDO-level indicators (number of citizens and businesss accessing e-services and average increase in user satisfaction with selected e-services) were assessed to be on track. However, the PDO-level indicator for the number of services digitalized and available online targeted at businesses and citizens have not changed from baseline since no services have been digitalized to date. During the implementation, it was found that the PDO-level indicator on measuring efficiency of services – amount of time it takes for citizens and businesses to obtain selected digitalized services – was methodologically impossible to measure. Therefore, it requires changing the indicator to a measurable one. 4. Slow progress in digitalizing services, weak engagement of citizens in this process, and missing synergies among various components/activities require a different approach to overcome siloed implementation. While the project intended to deeply involve citizens and accompany several sectoral ministries in transforming the administration through citizen engagement and change management activities, these activities were implemented in isolation. Perception surveys to measure the satisfaction and needs of citizens were conducted independently from identifying priority services for digitalization. The timing of two activities was not well coordinated to reflect the survey results in the selection process. The change management activity has mostly focused on theoretical training rather than accompanying the sectoral ministries for actual changes, for instance, in the business process reengineering to generate consensus and work toward a common goal for digitalization. This is partly due to the project design with weak articulation between Components 2 The World Bank Enabling Digital Governance Project (P164824) and 3 and parallel contracts with several consulting firms. Given the weak capacity of the OITeG for coordinating several stakeholders and tasks, this design needs to change to integrate all related activities into fewer ones to avoid parallel tasks without synergies. The institutional capacity of the OITeG should also be strengthened, particularly in coordination and project management, to better manage complex processes. 5. Component 1 is almost complete. This component aims to establish the necessary cross-cutting foundations to support the use of information and communication technology (ICT) to provide public services to citizens and business, including standards, procedures, and digital infrastructure. Around 24 percent of the funds allocated to this component are disbursed and 56 percent are committed. Five out of six major activities (network operations center, central backup systems, connectivity for e-Government, data center equipment, and security operations center) have been completed. The activity—interoperability and creation of meta-registry—will also be completed before June 30, 2024, the current closing date. However, two planned activities in the original design: Active-Active Data Solution System1 (USD 3 million) and Data Center Security Solutions2 (USD 2.5 million) have yet to be contracted. 6. Component 2 is lagging behind. The objective of this component is to improve back-office processes to reduce administrative burdens and increase efficiency of service delivery. Around 19 percent of the funds allocated to this component are disbursed and around 43 percent are committed. Two out of three main activities (analysis of transforming services for citizens and e-Archiving) have been completed. The development of the Office Administration Management Application (OAMA) is expected to be completed in October 2024, after the current project closing date. The most critical activity under this component which is the digitalization of services has just started. 7. The analysis of transforming services for citizens was finalized and endorsed by the OITeG in February 2023. The project first created a comprehensive Administrative Service Inventory of over 1,000 administrative procedures identified within the Government. The Project then developed the proposal for development of 150 “as-is” process maps for digitalization of Government to Citizen (G2C) services from the inventory based on the 17 criteria3. Based on the results of analysis under as-is maps production, optimization measures for 38 administrative procedures have been approved by the associated ministries and the Public Policy Secretariat (PPS). Of these, 30 administrative procedures have been nominated for further digitalization process by the OITeG. The digitalization of these procedures is under preparation including preparation of technical specifications and required changes in terms of normative instruments (i.e. regulations, rulebooks) in some identified services. 8. Component 3 has mixed progress. This component aims to strengthen the capacities of citizens, businesses, and the Government to achieve project results. In terms of achievements, (a) more than 7,800 government officials have received training in digital skills and institutional setup and (b) baseline of the citizen satisfaction with digitalized services has been established, with an operational framework for the ongoing communication activities. However, the change management activity has been significantly delayed. The approach taken by the consulting firm is heavily focused on 1 Active-Active replication functionality is an activity to ensure high-level availability of critical systems and services. Active-Active replication solution includes the installation of all relevant hardware and system software. 2 This activity aims to improve the data center security, including the installation of security hardware and purpose-built applications, which will provide a high level of IT security and insights into events. 3 Volume and implementation timeframe within the service life, Existing of necessary back-end infrastructure, Well defined and functioning business processes, Frequency of use, Urgency and relevance, Outreach, Existence of key enablers, Back office readiness, Level of complexity, Time and money savings, Improvement in accountability and transparency gains, Legal and regulatory framework, Leadership and political will, User readiness, Sustainability and External factors (EU compliance, support of other donors, etc.) Development of monitoring, evaluation and reporting systems to measure the impacts of service digitalization. The World Bank Enabling Digital Governance Project (P164824) training and does not include handholding institutions to manage actual changes. Given the lack of progress, interest, and ownership in project beneficiaries and benefits for them, the contract was canceled in September 2023. The format of this activity will need to change. 9. The project has uncommitted funds of US$20.2 million, including some savings, after procuring most planned activities in the original design. The uncommitted funds result from savings from the procurement of main activities (original sub-components 1.1, 1.2, 2.2 and 2.3) and the cancellation of a few activities, which were funded by the Government’s own resources. As of September 2023, 60 percent of project funds are committed. A further 2 percent of project funds will be committed by the current closing date, June 30, 2024. The remaining 38 percent of project funds (US$20.2 million) needs to be utilized for originally planned activities (especially digitalization of services) that have yet to be initiated, reallocated to existing or new activities to fully utilize project funds. B. Rationale for Changes 10. To achieve the PDO, the Project needs to deliver digital services to citizens and businesses that operate efficiently. An extension of the project closing date by 23 months would be necessary to allow for digitalization of services that are yet to start. In addition, two originally planned activities under Component 1: Active-Active Data Solution System and Data Center Security Solutions would also need to be implemented during the extension period. 11. Successful digitalization requires adjusting the original project approach, ensuring synergies across components and activities and strengthening the implementation arrangement. The identified top 30 priority services for digitalization mostly aim to facilitate the management of important life events for citizens and led by the Ministries of Interior, Health, Education and Defense and some others. The services include the issuance of passports, identity cards (ID), drivers’ license, residence registration and certificates, reimbursement of health-related costs, awarding of scholarships and loans for students, and military service certificates among others (Table 1 and Annex). Each registry will be managed in the meta-registry funded under Component 1 and connected to the national e-ID system as part of the digitalization process. The total number of beneficiaries is estimated at about 3 million. Additionally, the project identified the need for upgrading existing services that serve a large number of people such as vehicle inspection, e-inspector, and e-baby to make them faster, easier and simplified as well as upgrading government websites to provide timely and quality information to citizens. As some of them are operationally complex and require legislative changes, the sequencing of these actions is important to provide faster and more visible outcomes to citizens. The project also needs to integrate the digitalization process, change management, and citizen engagement activities to manage significant changes, which facilitate the provision of citizen-oriented services. While the OITeG has already extensive experience in digitalizing services, it faces a challenge in the lack of manpower. The implementation arrangements need to be improved, through a partnership between OITEG and the Public Policy Secretariat (PPS), the establishment of a joint and dedicated task force in the OITeG service delivery unit, that will be composed of staff fully dedicated to the digitalization of services. It also needs to strengthen the capacities and ownership of sectoral ministries for project implementation. Table 1. Top 5 large services for digitalization and top 3 large existing services for upgrading Responsible Digitalization of services # of Beneficiaries Ministry Ministry of Interior Issuing passports to citizens 853,123 Ministry of Interior Issuing identity cards to citizens 682,484 Ministry of Interior Issuing driver's license 376,695 The World Bank Enabling Digital Governance Project (P164824) Health Insurance Reimbursement of transportation costs 306,115 Fund related to healthcare services Ministry of Interior Registration of residence 172,833 Responsible # of Upgrading existing online services Ministry Users/Transactions Ministry of Interior Vehicles’ Technical Inspection 1,200,000 Multiple e-Inspector Information system 55,000 Multiple E-Baby application 65,000 Source: OITeG 12. Since the project preparation, the need for digitalization has evolved because of COVID-19 and needs to take into account severe security threats in the region. In addition to COVID-19, which lasted more than two years, the geopolitical and security threats in the region due to Russia’s invasion of Ukraine raised serious concerns about increasing risks for data protection and cybersecurity of the government systems. The OITeG expressed the vital need for strengthening data protection and cybersecurity further and to provide support through the EDGE Project. There is an opportunity to adapt the project, which is taking a whole-of-government approach, to these changing circumstances and respond to current needs and priorities in the public policies of the Government of Serbia (GOS). 13. To facilitate smooth implementation, the implementation arrangements need to be simplified and made more practical. The chair of the Steering Committee will be shifted from the Prime Minister to the Minister of Information and Telecommunications (MIT) to enable the SC to meet more regularly to provide guidance and oversight of the project, which was identified as key bottleneck. The disbursement arrangements need to be updated to allow for 100 percent financing of eligible expenditures as the timely availability of counterpart funds was another source of bottlenecks and delays. Over the course of the project, the OITeG has repeatedly expressed its concerns about challenges in mobilizing the government counterpart funds and its lengthy approval processes, which have delayed contract payment. Moreover, in early 2023, it was revealed that while the loan is exempted from value-added tax (VAT), the part of contracts financed by the government contribution is subject to the VAT payment, as stipulated in the national legislation. The restructuring therefore proposes to eliminate the requirement for counterpart funds and for the project to fully fund eligible expenditures. 14. As part of the Public Administration Reform Strategy, the GOS adopted the new e-Government development program for 2023–20254 and the EDGE Project continues to play a key part. Based on the previous program for 2020– 2022 and its implementation progress, the objective is to develop an efficient and user-oriented administration in the digital environment. The program specifically aims to develop infrastructure for e-Government, ensuring interoperability, improving legal security in the use of e-Government, and increasing accessibility of e-Government to citizens and the economy through the improvement of user services and open data in public administration. In addition, the program supports implementation of the strategy for the development of information society and information security for 2021– 2026, in line with the European Union Network and Information Security Directive. The EDGE Project finances about half of the planned investment in the total program cost of EUR 107 million and continues to play a key part in the Government program. 4Art. 10 and 38 paragraph 1 of the Law on the Planning System of the Republic of Serbia (Official Gazette of the Republic of Serbia, number 30/18). The World Bank Enabling Digital Governance Project (P164824) 15. The OITeG has requested to abolish 9 percent of government counterpart funds due to implementation issues. Over the entire course of the Project, the OITEG has repeatedly expressed its concerns about challenges in mobilizing the government counterpart funds within the Government and their lengthy approval processes, which have caused delays in contract payment. Moreover, in early 2023, it was revealed that while the loan is exempted from Value-Added Tax (VAT), the part of contracts financed by the government contribution is subject to the VAT payment, as stipulated in the national legislation. Because of these complications, the government has requested to cancel the counterpart funds. 16. In this context, the Project savings are proposed to be utilized to finance new activities and reallocated across components. Under Component 1, proposed new activities are aimed at strengthening the security and reliability of service delivery channels, and enhancing data governance (moved from Component 2). Under component 2, the restructuring limits the focus to citizen-centric digital services while proposing expansion of the scope beyond just digitalization of new services to serve a wider user base by including enhancement of existing services and inception of new “greenfield services”, enhancing the communication outreach and introducing new activities focused on institutional strengthening and change management (moved from Component 3) to ensure synergy. Under component 3, additional support to project management is proposed. The new activities would also require revisions to the results framework. 17. The Ministry of Finance sent a formal letter dated October 31st, 2023, requesting the following changes to the project: (a) extend the project closing date by two years, (b) reallocate resources according to the emerging needs, (c) revise the project implementation arrangements, and (d) abolish the government counterpart fund. II. DESCRIPTION OF PROPOSED CHANGES 18. This restructuring proposal is in response to the request from the GOS, received on October 31st, 2031. The proposed changes are (a) extension of the closing date; (b) revision of project components, scope, and cost; (c) revision of implementation/institutional arrangements; (d) revision of disbursement arrangements; and (e) revision to the Results Framework to reflect the updated scope of the project. These changes will also necessitate revisions in the implementation schedule and disbursement estimates. The PDO and fiduciary management will remain the same. a. Extension of the Closing Date 19. The closing date will be extended by 23 months – from June 30, 2024 to May 31, 2026 - to complete the ongoing activities and initiate new activities with the reallocation of uncommitted funds/savings. The new timeline for each component was carefully assessed during the restructuring process. As a result, Component 1 will be completed by January 2026, and Component 2, by May 2026. Component 3 will be completed by June 2024, except for support for project management until the new closing date. b. Adjustment of the Components and Reallocation of Resources Table 2. Summary of Changes and Reallocation of Funds (US$ million) Original New Description of Original Components Uncommitted New Components Allocation Allocation Changes Component 1: Foundations for Digital Service Delivery Subcomponent 1.1: Subcomponent 1.1: Implementation of the Implementation of the 7.50 3.95 4.0 Amount revised Interoperability Platform Interoperability Platform and Meta-Registry The World Bank Enabling Digital Governance Project (P164824) Original New Description of Original Components Uncommitted New Components Allocation Allocation Changes Subcomponent 1.2: Included in Establishment of the 1.70 1.44 0.0 Subcomponent 1.1 Government Meta-Register Subcomponent 1.3: Subcomponent 1.2: Implementation of the Implementation of the 13.65 2.60 14.5 Amount revised Disaster Recovery Data Disaster Recovery Data Center Center Subcomponent 1.3: Subcomponent 1.4: Amount revised for Strengthening Strengthening additional activities Cybersecurity, Data Cybersecurity, Data 3.85 1.91 14.3 Original governance, Protection Protection, and Regulatory Subcomponent 2.1 and Management and Environment moves here. Regulatory Environment Component 1 Total 26.70 9.90 32.8 Component 2: Citizen-Centric Digital Services Subcomponent 2.1: Data Moved to new and Document 9.35 1.00 0.0 Subcomponent 1.4 Management Subcomponent 2.1: Subcomponent 2.2: Administrative Service 0.75 0.57 2.6 Amount revised Administrative Service Inventory and Business Process Reengineering Subcomponent 2.3: Merged into Business Process 7.50 4.30 0.0 Subcomponent 2.2 Reengineering Subcomponent 2.4: Subcomponent 2.2: Digitalization of Public 5.00 4.24 5.9 Amount revised Digitalization of Services Services Digital caravan and Subcomponent 2.3: institutional Citizen Engagement and 5.2 strengthening support Institutional added. Subcomponent Strengthening 3.2 moved here Component 2 Total 22.6 10.11 13.7 Component 3: Digital Skills Development and Project Management Subcomponent 3.1: Subcomponent 3.1: Strengthening Digital Strengthening Digital Skills 1.8 0.05 2.1 Amount revised Skills and Knowledge and Knowledge Sharing Sharing Subcomponent 3.2: Change Management, Moved to 1.4 0.52 0.0 Communication, and Subcomponent 2.3 Citizen Feedback Subcomponent 3.3: Project Subcomponent 3.2: 2.5 0.69 3.5 Amount revised management Project management Component 3 Total 5.7 0.22 5.6 0 Government Contribution -2.9 Cancelled Funds The World Bank Enabling Digital Governance Project (P164824) Original New Description of Original Components Uncommitted New Components Allocation Allocation Changes Total 55 20.23 52.1 Component 1. Foundations for Digital Service Delivery 20. The project will expand the capacity of existing platforms and further strengthen the security and reliability of service delivery channels and enhance data governance in the public administration. The following changes, including reallocation of resources, are introduced to complement ongoing data center improvements, capacity-building program, and data management activities. Subcomponent 1.1: Implementation of the Interoperability Platform and Establishment of the Government Meta-Register (Description and Allocation revised) 21. Two subcomponents 1.1 and 1.2 are merged into one as two interlinked activities have been conducted as one activity. The amount was also revised based on the actual commitments. Subcomponent 1.2: Implementation of the Disaster Recovery Data Center (Allocation revised) 22. There are no additional activities. Two planned activities in the original design—Active-Active Data Solution System (US$3 million) and Data Center Security Solutions6 (US$2.5 million)—will be implemented during the period of 5 extension. The amount is also revised. Subcomponent 1.3: Strengthening Cybersecurity, Data Protection and Management and Regulatory Environment (Description and Allocation Revised) 23. This subcomponent merges the original subcomponents 1.4 and 2.1 to ensure synergies. One planned activity in the original subcomponent 1.4 — Cybersecurity and Data Protection Laboratory (US$1 million)—will be implemented during the period of extension. The amount is also revised. The development of OAMA and the implementation of e- archiving software will be moved from Subcomponent 2.1 to Subcomponent 1.3, since these shared platforms are critical components of the foundational ICT infrastructure. This component aims to strengthen data governance and make authorities more informed decisions by increasing its data analytics capabilities through better data collection, storage, management, and use of data and electronic documents, including, inter alia, the implementation of a data and document management system. 24. Furthermore, the following activities were added: 25. Equipment for cloud data center (US$2 million). One of the requirements for government cloud architecture is compliance with Data Privacy and Information Security Laws to protect the private data collected from citizens in Serbia. The private government cloud model implies the deployment of underlying hardware infrastructure, to support government bodies’ requests for availability and performance of delivered services. The request for the hardware is based on the demands and current workload of the existing environments, considering yearly growth and the government strategy to expose more services to citizens and the private and public sectors. 5 Active-Active replication functionality is an activity to ensure high-level availability of critical systems and services. Active-Active replication solution includes the installation of all relevant hardware and system software. 6 This activity aims to improve the data center security, including the installation of security hardware and purpose-built applications, which will provide a high level of IT security and insights into events. The World Bank Enabling Digital Governance Project (P164824) 26. Help Desk for e-Government transformation process (US$0.5 million). This activity aims to establish a centralized help desk (service desk) for improving the usage of Government ICT services and applications and develop a dedicated team and a knowledge database about the usage of common Government ICT applications. The project will develop the IT Service Management Tool (ITSMT) to manage the issues from citizens or internal users regarding ICT network, security, or applications. Component 2. Citizen-Centric Digital Services (Description and Allocation Revised) 27. The project will take a different approach to address the three dimensions of project objectives—access, quality, and efficiency of e-services—in a more impactful manner. It will (a) integrate change management and citizen engagement activities into this digital transformation process; (b) strengthen the institutional capacity of the OITeG and other key stakeholders for leading and coordinating intergovernmental activities in an agile, user-centric way; and (c) streamline the implementation arrangements through a partnership between OITeG and PPS of the Prime Minister’s office. Given its extensive experience in digitalization of services and capacities for coordinating and motivating stakeholders, the PPS will partner with the OITeG on this component (see details in Section D). Subcomponent 2.1: Administrative Service and Business Process Reengineering (Merged and Allocation Revised) 28. The original subcomponents 2.1 and 2.2 were merged into one as they were conducted as one activity as a prerequisite for digitalization of services, the two subcomponents were merged into one. The allocation was revised based on the actual spending (completed). Subcomponent 2.2: Digitalization of Services (Description and Allocation Revised) 29. The project is set to broaden its scope beyond digitalization of new services to serve a wider user base. The initial focus was on transitioning non-digitized administrative services to digital platforms. This effort will remain; however, a newfound emphasis will be placed on enhancing existing digital services and the inception of greenfield services. This recalibrated focus should significantly elevate the overall user experience by enhancing the mobile responsiveness, usability, and accessibility of widely used websites.  Digitalization of new services. The prioritization of 30 administrative services or procedures for digitalization has been completed. The identified services are mostly led by the Ministries of Interior, Health, Education and Defense and connected to the ID system to facilitate the life management of citizens. The sequencing of digitalization was carefully assessed during the first phase of the project and determined based on the usage of services (uptake) and the complexity of the digitalization process. The list in Table 1 presents the list of services with high impacts, and the list in Annex 1 presents the full list of services with the implementation order based on the hybrid criteria of priority, operational complexity and readiness. The list may be reviewed and revised in view of required changes that may affect or delay the digitalization process and/or arising new needs. In this respect, the list should be considered a flexible instrument that may accept the replacement of listed services with others having equal or superior importance according to the criteria established. Going forward, the project will support preparing technical specifications, and provide legal and implementation support for required legislative, regulatory and administrative changes and digitalization process. The whole process is expected to be completed by May 2026.  Enhancement of existing digital services. Recognizing the substantial user base of existing digital services, efforts will be channeled toward improving their performance and user experience. Three large critical The World Bank Enabling Digital Governance Project (P164824) services have been identified for enhancement (for example, vehicle registration, e-inspector, and e-baby), and additional services may be digitalized based on their feasibility and impact. In a similar vein, given that two-thirds of Serbians access government information via mobile phones, at least 20 websites providing informational content of public utility will be updated, making them mobile responsive and compliant with international accessibility guidelines.  Inception of new ‘greenfield’ services. Greenfield services refer to new services that are not dependent on administrative procedures or legacy systems and can be launched within a limited time span (for example, 2 months). Identification of greenfield services will be through sectoral engagement, which will be enhanced through the institutional strengthening subcomponent. 30. To ensure the timely and effective implementation of the digitalization process, the OITeG will set up a Joint Task Force, composed of OITeG and PPS staff, and implementation teams in sectoral ministries will be strengthened. To avoid the delays observed in past activities, the task force will be led by a member of OITeG senior management with full decision-making capacity and composed of project managers for the different digitalization project. This task force will be placed in the service delivery unit of the OITeG to benefit from their extensive experiences in digitalization of services. This task force will include teams with standard skills for the development of digital services (e.g., user research, product management), and will be staffed by OITEG personnel as well as additional consultancies on an as-needed basis. The task force will work with the project implementation team in each sector. The implementation teams will also be supported to include necessary technical experts who ensure provision of better services with digital solutions. Services will only benefit from digitalization once their design are jointly validated by the dedicated teams and sectoral counterparts, and once prototypes have been validated through user testing. This setup aims to address and overcome the challenges faced in previous attempts at digitalization, providing a focused and efficient approach toward achieving the project’s goals. New Subcomponent 2.3: Citizen Engagement and Institutional Strengthening (New, Moved from Component 3, Description and Allocation Revised) 31. The planned communication and citizen feedback activities under this subcomponent will continue. The original objective of this subcomponent was to strengthen engagement with government stakeholders and citizens. The existing activity on communication will be expanded to enhance both internal and external communication regarding the digitalization of services, interoperability platform, and meta-register. The project will cover the costs of communication and production of communication materials. The current surveys on service users’ satisfaction will be complemented by user research and testing with users and non-users of digital services. 32. Digital Caravan (US$1 million): To further supplement citizen engagement activities, a digital caravan will be included as an outreach mechanism. The digital caravan will build on the Government’s ‘Digital Expedition’ Caravan, which engaged citizens on issues of digital literacy, digital skills, and safety. Additionally, the caravan will serve as a conduit for user research and testing. This approach will allow for an iterative mechanism where the digitalization and enhancement of services are continually informed by data gathered from user interactions. The project will cover the operational costs of the digital caravan. The project will support the following aspects for the digital caravan: (a) training and capacity building to equip both teams with the requisite knowledge, skills, and tools to effectively perform their duties; (b) research and testing equipment and software such as eye-tracking devices, touchscreen monitors, and audio-recording devices to facilitate accurate user research and testing; (c) streamlining of the operational logistics to ensure seamless coordination between the two teams, effective coverage of the targeted regions, and efficient handling of on-site logistics; and (d) robust publicity campaigns to inform citizens about the caravan’s schedule, objectives, and the benefits of engaging with the digital services showcased. 33. Institutional Strengthening and Change Management (US$1 million). The project will provide additional support for strengthening institutional capacities and change management in the digitalization process. The project will finance The World Bank Enabling Digital Governance Project (P164824) necessary consultants for the new task force, which will be comprised of OITeG and PPS staff and experts. The required expertise may include the Product Manager, Product Owner, User Researcher, User Experience Designer, Technical Architect, Content Designer, and Performance Analyst. The project will also support ministries, departments, and agencies benefitting from the project and hire consultants for project management. Positions on an as-needed basis will be financed subject to an approved staffing plan and justifications of need and demonstrable evidence that the position will significantly enhance the digitalization process supported by the project. The allocation of consultants’ time to activities other than their approved terms of reference will be considered ineligible expenditures and lead to the suspension of contracts. 34. Under this component, the project will also finance agile change management interventions to support sectoral engagement and follow-through. For the successful digitalization of services, increased buy-in, digital capacities and behavior change are needed. As the lead agency for this agenda, the OITeG will need to engage with its stakeholders much earlier in the design process. Both OITeG and line ministries will strengthen their digital and project management capabilities with agile and user-centric approaches. Finally, the new business processes underpinning the digitalization of services will necessitate changes in how line ministries manage staff and data. Consequently, change management will finance four types of activities for both the OITeG and line ministries. First, training on agile management and approaches for greater ministerial ownership and buy-in. Second, the development of integrated change management implementation charters. These plans will be developed with the respective line ministries and the vendor digitalizing services. The plans will also integrate any existing internal and external communication plans. Third, a knowledge and learning workshop series for the task force and a wider group of stakeholders to build awareness and improve strategic decision-making. Fourth, as needed, individual change management consultants can be hired to support the frontline uptake of new procedures, applications, and ways of working. Component 3. Digital Skills Development and Project Management (Description and Allocation Revised) 35. Component 3 will be simplified to reflect the integration of change management, communication, and citizen feedback into Component 2. It focuses only on digital skills development and project management. An additional US$ 1 million will be allocated to project management until the new closing date. c. Adjustment in the Implementation Arrangement 36. The implementation arrangement is simplified but strengthened for the digitalization of services. In the original implementation arrangement, the Steering Committee (SC) was expected to oversee the project and meet quarterly while the Management Committee (MC) was expected to meet monthly for project supervision. However, the two distinct mechanisms have not been functioning as designed and only three SC meetings were organized, and the MC has been organized in an ad hoc manner over the past three years. The implementation arrangement will be revised to make it more practical: change the chair of the SC from the Prime Minister to the Minister of IT and Telecommunication, abolish the MC, and introduce the Joint Task Force (JTF). The main objective of the JTF is to support the digitalization of services and overall support to the project implementation. The JTF will be created within the OITeG and led by the Deputy Director of OITeG. It will consist of OITeG and PPS staff as well as additional project managers and technical experts for each digitalization project. The JTF will work closely with the Implementation Teams in the line ministries that own the services to be digitalized. The implementation teams will also be strengthened, where needed, with experts to be financed by Sub- Component 2.3. In addition, the biweekly meetings established in 2021 between the World Bank team and the PIU to discuss and solve implementation issues will continue, with the participation of the OITeG senior management. The revised implementation arrangement is presented in Figure 1. The World Bank Enabling Digital Governance Project (P164824) Figure 1. Implementation Arrangement Original Changed Steering Committee Steering Committee Chair: PM Chair: MiT and OITeG Members: PM, OITeG, MoF, MTTT, MPALSG, Members: PM, OITeG, MoF, MPALSG, MoJ, MoJ, MoI, and other agencies MoI, PPS, and other agencies Frequency of meetings: quarterly or more Frequency of meetings: biannual Role: Ensures inter-agency coordination, helps Role: Ensures inter-agency coordination, helps to overcome project Implementation Management Committee to overcome project Implementation bottlenecks Chair: OITeG Director bottlenecksOITEG - Project Director Members: Core members - OITeG, MoF as Joint Task Force on Digitalization with PPS needed MPALSG, MoI, MoJ, MTTT and representatives from other line ministries Frequency of meetings: monthly Role: Oversite of the project implementation OITeG (Project Director) Implementat CFU in CFU in MoF PIU ion teams for MOF PIU each activity Note: CFU = Central Fiduciary Unit; MiT = Ministry of Information and Telecommunications; MoF = Ministry of Finance; MoI = Ministry of Interior; MoJ = Ministry of Justice; MPALSG = Ministry of Public Administration and Local Self-Government; MTTT = Ministry of Trade, Tourism, and Telecommunications; PM = Prime Minister. d. Changes to Disbursement Arrangements 37. The nine percent counterpart financing requirement will be eliminated, and the percentage of loan financing will be increased to 100 percent. In response to the government request, the government counterpart funds will be eliminated. Accordingly, Section IV of Schedule 2 will be deleted to apply this cancellation retroactively as of the signature date. Because of the significant amount of savings in the project, this cancellation will not impact on financing the necessary activities to achieve the PDO. e. Adjustment in the Results Framework 38. As a result of changes in the project scope and approach, the Results Framework will need to be adjusted. During the project implementation, it was found that the definition of several PDO indicators and its measurements are unclear. Also, given additional resources in cybersecurity and data security, the following changes will be made in the Results Framework:  PDO indicator 1 “Number of citizens and businesses accessing e-services disaggregated by gender and size of businesses” will change to “Number of citizens and business accessing e-services”. The disaggregation by gender will be measured by an intermediate indicator. The disaggregation by size of businesses will be dropped as e-portal does not capture this dimension. The target is revised upward, as the actual status is already close to the target.  PDO indicator 2 “Number of services digitalized and available online targeted at businesses and citizens” will be adjusted to measure “Number of services targeted at businesses and citizens digitalized and upgraded, and available online.” It will measure not only newly digitalized services but also upgraded existing services. The World Bank Enabling Digital Governance Project (P164824)  PDO indicator 3 “Average increase in user satisfaction with selected e-services provided disaggregated by gender and size of businesses (Citizens Engagement)” will be revised to measure “the user satisfaction with the newly digitalized e-services (citizen engagement)” to clarify the definition of this indicator. Disaggregation will be dropped, as gender dimension is captured in one intermediate indicator (see below) and the size of business is not available in e-portal. The target was revised upward, as the actual target is deemed to be low.  PDO indicator 4 “Amount of time it takes for citizens and businesses to obtain selected digitalized services” will be changed to measure “estimated annual savings from newly digitalized services”. The current indicator to measure the time reduction is not methodologically possible, and the measurement of this indicator is unclear. Instead, annual savings for citizens will be calculated by an already available tool, PPS calculator to measure efficiency (https://rsjp.gov.rs/sr/kalkulator/).  Intermediate indicator “Number of IT services secured by the Security Information and Event Management System (SIEM)” will be added to measure the strengthened security system.  Intermediate indicator “Number of services available through the e-Gov Portal” will be dropped as it overlaps with PDO indicator 2.  Intermediate indicator “Number of citizens accessing e-services disaggregated by gender” will be added to measure the gender dimension. f. Alignment with the Paris Agreement 39. The Project is aligned with the goals of the Paris Agreement on both mitigation and adaptation. The Project is consistent with Serbia's climate commitments and contributes towards the country's low carbon and climate resilient development trajectory. 40. Assessment and reduction of adaptation risks. The Project is consistent with achieving Serbia's climate adaptation targets and reducing its vulnerability to observed and anticipated climate risks towards floods and extreme heat. Furthermore, the Project activities do not pose a material risk from current and future climatic conditions. The overarching program takes into consideration the climate and disaster risks such as flooding, droughts, and extreme temperatures identified using the Climate and Disaster Risk Screening Tool. Specifically, the implementation of the Disaster Recovery Data Center will incorporate resilient design towards floods and extreme heat to ensure the existing data infrastructure is resilient towards these climate risks. Additionally, the continued provision of digital services for citizens living in areas prone to flood and extreme heat as well as the continuation of business operations will be ensured during times of climate hazards. 41. Assessment and reduction of mitigation risks. The proposed project will contribute to and is consistent with achieving the Republic of Serbia's Nationally Determined Contribution and the country's priorities stated in the Second National Communication of the Republic of Serbia under the UNFCCC (SNCUNFCCC). The activities to be integrated for the existing data centers include the consolidation of multiple data centers into the government’s main data center and the Disaster Recover Data Center which will be energy efficient and follow standard guidelines for green data center design, thus reducing GHG emission. The World Bank Enabling Digital Governance Project (P164824) III. SUMMARY OF CHANGES Changed Not Changed Results Framework ✔ Components and Cost ✔ Loan Closing Date(s) ✔ Reallocation between Disbursement Categories ✔ Disbursement Estimates ✔ Institutional Arrangements ✔ Implementation Schedule ✔ Implementing Agency ✔ DDO Status ✔ Project's Development Objectives ✔ PBCs ✔ Cancellations Proposed ✔ Disbursements Arrangements ✔ Overall Risk Rating ✔ Safeguard Policies Triggered ✔ EA category ✔ Legal Covenants ✔ Financial Management ✔ Procurement ✔ Other Change(s) ✔ Economic and Financial Analysis ✔ Technical Analysis ✔ Social Analysis ✔ Environmental Analysis ✔ IV. DETAILED CHANGE(S) The World Bank Enabling Digital Governance Project (P164824) OPS_DETAILEDCHANGES_COMPONENTS_TABLE COMPONENTS Current Current Proposed Proposed Cost Action Component Name Component Name Cost (US$M) (US$M) Foundations for Digital Service Foundations for Digital Service 26.70 Revised 32.80 Delivery Delivery Transforming Services for Citizens, Businesses, and 22.60 Revised Citizen-Centric Digital Services 13.70 Government Digital Skills Development, Institutional Strengthening and 5.70 Revised Digital Skills Development 5.60 Change Management TOTAL 55.00 52.10 OPS_DETAILEDCHANGES_LOANCLOSING_TABLE LOAN CLOSING DATE(S) Original Revised Proposed Proposed Deadline Ln/Cr/Tf Status Closing Closing(s) Closing for Withdrawal Applications IBRD-89470 Effective 30-Jun-2024 31-May-2026 30-Sep-2026 OPS_DETAILEDCHANGES_REALLOCATION _TABLE REALLOCATION BETWEEN DISBURSEMENT CATEGORIES Financing % Current Allocation Actuals + Committed Proposed Allocation (Type Total) Current Proposed IBRD-89470-001 | Currency: EUR iLap Category Sequence No: 1 Current Expenditure Category: G, W, nonCS, CS, TR, OC 43,690,500.00 16,190,240.51 43,690,500.00 91.00 100 iLap Category Sequence No: FEF Current Expenditure Category: FRONT END FEE 109,500.00 109,500.00 109,500.00 Total 43,800,000.00 16,299,740.51 43,800,000.00 The World Bank Enabling Digital Governance Project (P164824) OPS_DETAILEDCHANGES_DISBURSEMENT_TABLE DISBURSEMENT ESTIMATES Change in Disbursement Estimates Yes Year Current Proposed 2019 0.00 4,300,000.00 2020 3,636,800.00 0.00 2021 5,249,950.00 0.00 2022 8,877,500.00 6,375,000.00 2023 13,734,700.00 18,500,000.00 2024 18,501,050.00 20,825,000.00 . The World Bank Enabling Digital Governance Project (P164824) . Results framework COUNTRY: Serbia Enabling Digital Governance Project Project Development Objectives(s) To improve access, quality, and efficiency of selected administrative e-Government services. Project Development Objective Indicators by Objectives/ Outcomes RESULT_FRAME_TBL_PDO Indicator Name PBC Baseline End Target Access to e-Services Number of citizens and business accessing e-services (Number) 1,270,000.00 2,500,000.00 Rationale: Drop the disaggregation by gender and move it to the intermediate indicator. Revise the target upwards given the current Action: This indicator has been Revised trend. Access to e-Services Number of services targeted at businesses and citizens digitalized 0.00 30.00 or upgraded, and available online (Number) Action: This indicator has been Revised Quality of e-Services User satisfaction with selected e-services (Citizens Engagement) 0.00 65.00 (Percentage) Action: This indicator has been Revised Efficiency of e-Services The World Bank Enabling Digital Governance Project (P164824) RESULT_FRAME_TBL_PDO Indicator Name PBC Baseline End Target Estimated annual savings stemming from newly digitalized 0.00 20% of the new sub-components 1.2 and 2.2 cost services (Text) Action: This indicator has been Revised PDO Table SPACE Intermediate Results Indicators by Components RESULT_FRAME_TBL_IO Indicator Name PBC Baseline End Target 1. Foundations for Digital Service Delivery Number of entities connnected to the Interoperability Platform 8.00 18.00 (Number) Establishment of GOS Meta Register (Yes/No) No Yes Number of IT services secured by the Security Information and 3.00 6.00 Event Management System (Number) Action: This indicator is New 2.Transforming Services for Citizens, Business and Government Development of an Administrative Service Inventory (Yes/No) No Yes Number of "to be" process maps generated for selected services 5.00 50.00 (Number) Number of services available online through the e-Gov Portal 0.00 30.00 (Number) Rationale: Action: This indicator has been Marked for Deletion This indicator is the duplication of PDO indicator 2. The World Bank Enabling Digital Governance Project (P164824) RESULT_FRAME_TBL_IO Indicator Name PBC Baseline End Target Number of citizens accessing e-services disaggregated by gender 0.00 1,250,000.00 (Number) Action: This indicator is New of which women (Number) 0.00 625,000.00 Action: This indicator is New 3. Digital Skills Development, Change Management and Institutional Strengthening Number of Citizen Satisfaction Surveys completed (Number) 0.00 3.00 Number of Government Officials trained in digital literacy 0.00 10.00 (Number (Thousand)) IO Table SPACE The World Bank Enabling Digital Governance Project (P164824) Annex 1: List of Priority Procedures for Digitalization as of October 2023 Complexity of Number of Priority implementation (1) Order of Institution Name of the procedure Beneficiaries level the simplest - (4) implementation the most complex Ministry of Issuing a driver's license 376,695 1 2 1 Interior Confirmation of handover of weapons Ministry of to the ownership of the Republic of 10,671 1 2 2 Interior Serbia Certificate of approved temporary Ministry of residence or permanent residence of a 8,243 1 2 3 Interior foreigner in the Republic of Serbia Ministry of Certificate of residence 8,107 1 2 4 Interior Republic Fund Request for reimbursement of for Health transportation costs related to the use 306,115 2 1 5 Insurance of health care Ministry of War participation certificate 22,847 3 1 6 Defense Ministry of Certificate of regular military service 15,588 3 1 7 Defense Ministry of Certificate of completed military 9,000 3 2 8 Defense service Ministry of A certificate that the person is kept in 2,130 3 1 9 Defense the military records Declaring invalid public documents of JP "Official primary and secondary schools and 26,312 2 2 10 Gazette" higher education institutions Ministry of Awarding of student scholarships and 17,000 2 2 11 Education student loans Commissariat Confirmation that the person is in the for Refugees and 2,710 3 2 12 register of internally displaced persons Migration Entry in the voter list of the Ministry of information that the person will vote 2,603 3 2 13 Foreign Affairs abroad Ministry of Deregistration of residence 2,064 1 3 14 Interior Ministry of Registration of residence 19,143 1 3 15 Interior Ministry of Application for a temporary stay 3,986 1 3 16 Interior abroad longer than 90 days Ministry of Application for return to the Republic 1,856 1 3 17 Interior of Serbia after a long stay abroad Ministry of Registration of residence 172,833 1 3 18 Interior The World Bank Enabling Digital Governance Project (P164824) Ministry of Issuing an identity card to a citizen of 682,484 1 4 19 Interior the Republic of Serbia Ministry of Family Care and Parental allowance of the mother 62,000 2 - 20 Demography Traffic Safety Driver tachograph card 19,829 4 1 21 Agency (Issue/Renewal/Replacement) Issuance of a certificate of Traffic Safety professional competence and a 4,728 4 2 22 Agency qualification card for a professional driver Registration for the exam for Ministry of obtaining the title of licensed 72 3 - 23 Finance appraiser Reporting the loss of an ID card and Ministry of advertising the missing ID card in the 26,706 1 2 24 Interior country Reporting the disappearance and Ministry of declaring the passport invalid in the 11,718 1 2 25 Interior country Ministry of Issuance of a test driver's license 70,579 1 3 26 Interior Ministry of Passport 10,125 3 3 27 Foreign Affairs Ministry of Issuance of residence certificate 131,120 1 2 28 Interior Ministry of Issuance of a certificate of passing the Tourism and 92 2 1 29 professional exam for a tourist guide Youth Ministry of Issuance of identification for a tourist Tourism and 146 2 3 30 guide Youth Ministry of Certificate of time spent working 525 3 1 31 Foreign Affairs continuously abroad Issuance of authorization for the Ministry of acquisition of weapons for natural 13,929 1 3 32 Interior persons The procedure for recognizing a Agency for foreign school document of primary 797 2 3 33 Qualifications education The procedure for recognizing a Agency for foreign school document of secondary 2,073 2 3 34 Qualifications education Ministry of Issuing passports to citizens of the 853,123 1 4 35 Interior Republic of Serbia Ministry of Issuance of a weapon list 12,444 1 4 36 Interior Ministry of Reporting the loss of an identity card N/A 1 1 37 Interior for a foreigner The World Bank Enabling Digital Governance Project (P164824) Republic Fund for Health Referral for treatment abroad 689 2 3 38 Insurance Directorate for License to perform activities of an Prevention of authorized person in the field of 514 3 3 39 Money prevention and detection of money Laundering laundering and financing of terrorism Import permit for sporting and Ministry of hunting weapons, ammunition, Internal and 34 2 4 40 explosives and other materials for Foreign Trade production in this area Ministry of Construction, Issuance of a maritime certificate 241 4 3 41 Transport and Infrastructure Ministry of Construction, Issuance of shipping book 174 4 3 42 Transport and Infrastructure Ministry of Certificate of professional competence Construction, of the person responsible for the 48 4 3 43 Transport and management of passenger Infrastructure transportation in road traffic