Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review RCIP 5 - Uganda (P130871) Report Number: ICRR0023448 1. Project Data Project ID Project Name P130871 RCIP 5 - Uganda Country Practice Area(Lead) Eastern and Southern Africa Digital Development L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IDA-56350 28-Feb-2022 75,764,731.67 Bank Approval Date Closing Date (Actual) 22-May-2015 31-Aug-2022 IBRD/IDA (USD) Grants (USD) Original Commitment 75,000,000.00 0.00 Revised Commitment 75,000,000.00 0.00 Actual 75,765,648.67 0.00 Prepared by Reviewed by ICR Review Coordinator Group Ranga Rajan Ihsan Kaler Hurcan Avjeet Singh IEGSD (Unit 4) Krishnamani 2. Project Objectives and Components DEVOBJ_TBL a. Objectives The Project Development Objectives (PDOs) as stated in the Financing Agreement (Schedule 1, page 5) and in the Project Appraisal Document (PAD, page 9) is to support the Republic of Uganda's (hereafter Uganda) efforts to: (i) lower prices for international capacity and extend the geographic reach of broadband networks; and Page 1 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review RCIP 5 - Uganda (P130871) (ii) improve the Government's efficiency and transparency through e-Government applications. b. Were the project objectives/key associated outcome targets revised during implementation? No c. Will a split evaluation be undertaken? No d. Components This project in Uganda was the fifth phase in the Series of Projects (SoP) under the Regional Communications Infrastructure Program (RCIP). The first phase was in Kenya, Burundi, and Madagascar, the second in Rwanda, the third in Malawi, Mozambique, and Tanzania and the fourth in Comoros. The RCIP aimed to: (i) accelerate the roll-out of the terrestrial backbone infrastructure to ensure access to submarine cables across the East and South Africa (E&SA) region and to extend access to broadband services within countries; (ii) finance the purchase of broadband capacity (domestic, regional and international) for use by Governments and other targeted users; and (iii) finance related activities such as implementing e-government applications and extending access to Information and Communication Technologies (ICT) services in the rural areas of the participating countries. There were four components (PAD, pages 10 - 14). 1. Enabling Environment. The estimated cost at appraisal and the actual cost was US$3.00 million. This component aimed to build the capacity of the Ministry of Information, Communication and Technology (MICT) and the National Information Technology Authority, Uganda (NITA -U) to review, develop, and implement relevant ICT policies, strategies, laws, and technical regulatory frameworks. There were two subcomponents: (i) strengthening the existing policy, strategy, and regulatory environment for the sector and (ii) capacity building of MICT and NITA - U through an ICT skills development program and institutional strengthening to develop a strategy to sustain the ICT function within Government 2. Connectivity. The estimated cost at appraisal and the actual cost was US$36.00 million. When this project was under preparation, the Government had already embarked on a three-phase program to build a National Backbone Infrastructure (NBI) to connect Government and public sector offices and facilities. However, the program did not have full coverage and was missing key regions (the northern part of the country) and with limited ministries, departments, and agencies (MDAs) connected. This component included the following sub-components: (i) pre-purchase of international capacity for the Government and priority sectors; (ii) constructing new links to connect underserved areas that were not connected to the NBI; and (iii) extending the Government Network (GovNet) for providing broadband connectivity to MDAs, local governments, schools, hospitals, universities, and non-governmental organizations (NGOs). This component also financed technical assistance (TA) activities related to the design and implementation of the above-mentioned sub-components, a feasibility study for further extension of NBI in the northern part of Uganda (noted above) and an analysis of potential environmental and social impacts of the project and support for implementing the safeguard instruments (other than the Resettlement Action Plan). Page 2 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review RCIP 5 - Uganda (P130871) 3. E-Government. The estimated cost at appraisal and the actual cost was US$40.00 million. The actual cost was lower than the appraisal estimates due to a procurement issue relating to Government e- procurement during implementation. An original contract for developing the e-procurement system was terminated. The Government decided to proceed with developing this system outside the project using government financing. The financing allocated to this activity was utilized for further expanding connectivity activities (component two activities). This component planned to finance activities aimed at improving the Government's ability to deliver public services via digital platforms for citizens and laying the foundational platforms for e-Government. Activities in this component were as the following: (i) developing the necessary standards and frameworks for e- Government, including the Government Enterprise Architecture (GEA); (ii) developing Government cloud- based infrastructure in the existing national data center; (iii) establishing a shared platform for improving the government's ability to deploy e-services; (iv) implementing a cybersecurity program and a National Cybersecurity Strategy; (v) establishing a Government data integration and sharing program; (vi) implementing a Government unified communication system; (vii) implementing an e-procurement system; (viii) developing citizen-centered e-Services; and (ix) developing the Digital Authentication and e- Signatures Platform. 4. Project Management. The estimated cost at appraisal and the actual cost was US$6.00 million. This component provided project management support such as undertaking procurement, financial management, monitoring and evaluation, environmental and social safeguards management, institutional strengthening of the NITA – U, Ministry of ICT, and PPDA in project implementation and implementation of Resettlement Action Plans. e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project cost. The estimated cost at appraisal was US$85.00 million. The actual cost was US$77.66 million. The difference between the appraisal estimates and actual cost was mainly due to reduced recipient contribution (discussed below). Project financing. The project was financed by an International Development Association (IDA) credit of US$75.00 million. The amount disbursed was US$75.76 million. The difference between the IDA credit and disbursements was due to exchange rate changes during implementation. Recipient contribution. Recipient contribution of US$10.00 million was planned at appraisal. Total funding from the Government during implementation was US$1.9 million. The recipient contribution was allocated to operational expenses rather than capital investment. The ICR does not explain why the recipient contribution was less than the planned amount at appraisal. Dates. The project was approved on May 22, 2015, only became effective a year later on May 26, 2016, and was scheduled to close on February 28, 2022. The ICR does not provide details on the reasons for the delays between project approval and project effectiveness. The project closed six months behind schedule on August 31, 2022. Page 3 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review RCIP 5 - Uganda (P130871) Other changes. The closing date was extended by six months through a Level 2 restructuring on February 19, 2022, for completing the final monitoring and evaluation report and for completing a few smaller project activities. There were no other project restructurings. 3. Relevance of Objectives Rationale Country and sector context. As a landlocked country, Uganda depends on infrastructure investments and the competitiveness of the communication sectors of its coastal neighbors. Although access to mobile telephony and mobile-based services grew in Uganda before appraisal (from 780,000 in 2003 to 21.6 million in December 2013), access to broadband services was limited, with poor quality and high internet service prices. Entry level mobile broadband service was about 19% of the average Ugandan's monthly income. The high costs led to low broadband penetration rates (1.4% as compared to the regional average of 4.3%). The high costs and low penetration were due to factors such as Uganda's landlocked position that required overland access to submarine cables landing in neighboring countries, technical problems of the national network and perceived limited market opportunity by private operators in most rural areas of Uganda. Therefore, the PDOs of lowering the monthly cost of basic broadband services, boosting internet penetration and improving e-government services was important to the government strategy. Government strategy. The Government had implemented major policy reforms in the ICT sector before appraisal, such as (i) establishing the Ministry of ICT (MICT) for providing strategic and technical leadership on matters of policy, laws, regulations, and strategy; (ii) establishing an independent regulatory body, the Uganda Communications Commission (UCC); (iii) fully liberalized the telecommunications market; and (iv) invested in developing National Backbone infrastructure (NBI) with "open access principles" for bringing reliable, high quality, low-cost connectivity to all regions of the country and for use by both the Government and the private sector. However, even with these developments, large portions of the northern parts of the country remained without internet coverage. The PDOs aligned with the Government's Vision 2040 document and the National Development Plan (NDP) for 2015 - 2020. The NDP identified promoting the ICT sector as one of its eight primary objectives and specified the need to enhance ICT usage in business and government service delivery. Bank country strategy. The PDOs were fully aligned with the Bank strategy at appraisal. The Country Assistance Strategy (CAS) for 2011 - 2015 specifically articulated the need to deploy ICT infrastructure and improve Government efficiency and transparency through e-Government applications. The PDOs were highly relevant to the current Bank's Country Partnership Framework (CPF) 2016 - 2020 objectives of (i) enhancing the resilience of the poor and vulnerable, (ii) improving access to urban services, and (iii) improving the delivery of social services. Bank regional strategy. The project was fully aligned with the Bank's 2008 Regional Integration Assistance Strategy (RIAS) for Sub-Saharan Africa and the 2011 RIAS Strategic Update " Partnering for Africa's Regional Integration." The RIAS recognized the key role that ICT can play in regional integration and increasing competitiveness of African economies. Page 4 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review RCIP 5 - Uganda (P130871) Previous Bank experience. The Bank had financed four phases of the RCIP aimed to address the connectivity constraints by leveraging ICTs in countries in East and Southern Africa. This project in Uganda had ICT infrastructure investments and enabling environment reforms adapted to the country context but set within a coordinated regional framework. The regional approach was appropriate as this could be expected to help ensure seamless options for international connectivity. This was especially important for landlocked Uganda, which had to rely on transiting traffic through neighboring countries to gain access to submarine cables. Overall, the relevance of the PDO to the Government strategy and the Bank strategies for Uganda and the region is rated as High. Rating Relevance TBL Rating High 4. Achievement of Objectives (Efficacy) EFFICACY_TBL OBJECTIVE 1 Objective Lower prices for international capacity and extend the geographic reach of broadband networks (the connectivity development objective). Rationale Theory of change. Capacity building of the MICT and the NITA -U were likely to help in providing an enabling environment for developing the sector. Infrastructure investments such as pre-purchase of international internet bandwidth, constructing missing links for the NBI to serve underserved regions, extending GovNet to connect MDAs along with the outputs of the TA activities related to the design and implementation of these sub-components, a feasibility study for further extension of the NBI in the northern part of Uganda, and an analysis of the potential environmental and social project impacts and support for implementing the safeguard instruments were aimed at extending international bandwidth capacity in the country. These activities were expected to result in the intended outcomes of lowering the prices of international capacity and extending the geographic reach of broadband networks. The causal links between project activities, outputs, and outcomes were logical, and the intended outcomes were monitorable. Outputs. (ICR, pages 11 - 12).  10 Giga Byte (GB) of international internet bandwidth was pre-purchased for a fixed 10-year period (known as an Indefeasible Right of Use or IRU) for boosting total capacity in Uganda and lowering the overall wholesale bandwidth price. There was no baseline, and targets were not specified for this activity. Page 5 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review RCIP 5 - Uganda (P130871)  A total of 1606 kilometers (km) of broadband fiber were deployed, significantly exceeding the target of 670 km. This included (i) 842 km of NBI for connecting the underserved remote northeastern parts of Uganda and (ii) 764 km of last-mile connectivity for connecting MDAs, local Governments, schools, hospitals, and NGOs.  The availability of NBI increased from a baseline of 97.00% to 99.84%, slightly exceeding the target of 99.80%.  The 4G mobile network coverage increased from 10% of the population in 2016 to 79% by the end of 2021; 3G coverage in non-metropolitan areas rose from 63% in 2016 to 93% by the end of 2021. There were no targets for these indicators.  The Government Network (GovNet) was extended to 400 connections as targeted to provide broadband connectivity to MDAs, local Governments, schools, hospitals, and NGOs. Outcomes. (ICR, pages 11 - 12 and pages 23 - 24). The outputs described above were expected to lead to the intended outcomes of (i) increasing the volume of international traffic, (ii) increasing the volume of national traffic, (iii) lowering the wholesale average price of international communications, and (iv) increasing the number of project beneficiaries (including females).  The international internet bandwidth (a proxy for the volume of international traffic) increased from 25,679 megabits per second (Mbps) at the baseline on December 31, 2014, to 249 277 Mbps on August 31, 2022) far exceeding the target of 50,000 Mbps. The ICR does not provide any reasons for the targets being vastly exceeded but acknowledges that following the Mid-Term Review (MTR), the targets could have been modified to reflect the higher level of progress made since project preparation.  There were two proxies for the volume of national traffic: (i) the percentage of people with access to internet services and (iii) the number of people with access to telephone services (fixed mainlines plus cellular phones per 100 people). The percentage of people with access to internet services increased from 16.20% at the baseline to 55.10% on August 31, 2022, exceeding the target of 35%. The number of people with access to telephone services increased from 51.90 (fixed mainlines plus cellular phones per 100 people) to 73, slightly short of the target of 75.  The price of wholesale international capacity (per megabit per second (Mbit/s) per month) dropped from US$97.00 on December 31, 2014, to US$2.60 on August 31, 2022, for the period of the IRU, far exceeding the target of US$40.00. The ICR does not provide reasons for why the outcomes were significantly higher than the target. The retail price of a 500 megabyte (MB) mobile broadband pre-pay package fell from US$7.73 to US$2.04 when the project closed. Although not all of this retail price reductions were directly attributable to the project, given that other external factors could have contributed to this reduction, it is reasonable to assume that the significant reduction in the wholesale price had a flow-on effect on the retail market.  3.9 million people directly benefitted from project activities, short of the target of 5.06 million. 39.94% of the beneficiaries were females, slightly short of the target of 40%. The ICR (para 19) notes that this indicator was a point-in-time measurement, which reflected the user numbers for the month when the measurement was taken, rather than a measurement that cumulatively documented the number of beneficiaries over the project lifetime. The ICR (para 18) notes that the highest total seen for this indicator during the project showed the number of beneficiaries to be 4.8 million in August 2021. The number of project beneficiaries was defined as the total of (i) users of e-services, (ii) the number of ICT services subscribers in geographic areas covered by the "missing links"; (iii) several students in Page 6 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review RCIP 5 - Uganda (P130871) universities or schools with connectivity infrastructure; (iv) the number of civil servants in MDAs with improved internet connectivity; and (v) number of email accounts provided to civil servants in MDAs. Given that the targets for most of the indicators were realized, the efficacy of this PDO is rated as substantial. Rating Substantial OBJECTIVE 2 Objective Improve the Government's efficiency and transparency through e-Government applications (the transparency development objective). Rationale Theory of change. The project activities aimed at increasing the Government’s efficiency (defined as savings in time and money due to streamlined administrative process and reducing paperwork) and transparency (defined as better information to the public, which in turn can help in improving government accountability) in government’s delivery of public services through e-government applications. The outputs of activities such as developing standards and frameworks for e-government, developing Government cloud-based infrastructure in the National Datacenter, establishing a shared platform for improving the Government's ability to deploy e- services, implementing a cybersecurity program, establishing a Government data integration and sharing program, implementing Government unified communication and e-procurement systems, and developing citizen-centric e-services and Digital Authentication and e-signatures platform, were aimed at increasing the number of public digital platforms and services in Uganda. The outputs were expected to increase the efficiency and transparency of delivering public services. The causal links activities, outputs, and outcomes were logical, and the intended outcomes were monitorable. Outputs (ICR, pages 30 -31). The following activities were completed as targeted.  The cloud-based infrastructure in the National Datacenter, the interoperability platform, the Cybersecurity program across the Government, the Government unified communication system, and the Government data integration and sharing program.  55.97% of the MDAs/local governments were using the shared public service delivery program, slightly exceeding the target of 50%.  24 e-services were implemented, exceeding the target of 10. 38.31% of e-services were implemented with a gender perspective, exceeding the target of 30%. Outcomes (ICR, pages 12 and 25). Page 7 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review RCIP 5 - Uganda (P130871) The outputs described above were to result in two outcomes. (i) Increase the number of transactions per year using the shared public service delivery program (a proxy for government efficiency), and (ii) Increase the percentage of the population who were satisfied with the quality of e-services (a proxy for transparency).  There were 87,858,587 transactions per year utilizing the shared public service delivery platform, exceeding the target of 15,000,000. The ICR does not explain why the targets were significantly higher than the target.  77.30% of the beneficiaries were satisfied with the quality of e-Government services supported by the project when the project closed, exceeding the target of 50%.  While the original design prioritized five sectors for e-services development (health, education, agriculture, justice, and trade), the actual prioritization of the type of e-services developed under the project was determined by the implementation agency NITA-U, in coordination with the MICT rather than defined at project preparation. This flexible approach allowed the Government to respond to the needs of the current environment, such as developing critical national response services for contact tracing, health monitoring, and designing emergency relief to respond to the COVID-19 pandemic. Given that the outcomes were realized, the efficacy of this PDO is rated as substantial. Rating Substantial OVERALL EFF TBL OBJ_TBL OVERALL EFFICACY Rationale Efficacy of the project in achieving the first objective – lower the prices for international capacity and extend the geographic reach of broadband services (the connectivity objective) - is rated as substantial since the intended outcomes were realized for the most part. Efficacy of the project in achieving the second objective – improve the Government’s efficiency and transparency through e-Government applications (the transparency objective) - is rated substantial since the intended outcomes were realized. Therefore, overall efficacy is rated as substantial. Overall Efficacy Rating Substantial 5. Efficiency Economic analysis. A “with the project” and “without the project” cost-benefit analysis was conducted at appraisal and closure for activities associated with connectivity infrastructure and e-Government activities. These activities accounted for 89% of the actual cost. At appraisal, the expected benefits from expanding the Page 8 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review RCIP 5 - Uganda (P130871) ICT services were assumed to come from growing internet users, more intensive use of information services with lower unit prices and greater choice, and the development of locally relevant content and applications. The quantitative benefits were to come from savings in time and cost by substituting travel by telephone calls or mobile application usage (especially important for people living in rural and remote areas) and productivity increase (such as in agriculture because of easy and immediate access to market information that would increase Gross Domestic Product (GDP) per capita. The e-Government applications were expected to transform the delivery of public services by leveraging advances in ICT. However, the PAD (page 74) acknowledges the challenges of quantifying the economic and financial impacts of e-Government applications due to data constraints in Uganda as in other countries. The costs included in the economic analyses were the project cost and industry benchmark annual maintenance of 7%. The calculations at appraisal resulted in an ex-ante economic rate of return (EIRR) of 17%. The Net Present Value (NPV) was not calculated at appraisal. At project closure, a similar analysis was conducted that used secondary data and assumptions. Some additional benefits were included for the ex-post analysis. Based on econometric modeling, it was assumed that for every 10% increase in mobile broadband penetration, there would be a 2.46% increase in GDP per capital growth. However, the ex-ante analysis assumed that a 1% increase in mobile penetration would increase growth in GDP per capita by 0.06% percentage points. Furthermore, the analysis adopted data from Moldova for quantifying the savings from digitization of service delivery. The analysis assumed that digitization would result in a 79% decrease in the unitary cost of public services (that is, due to the savings associated with handling, storing, and processing documents). The analysis also included the savings in cost and time due to cybersecurity savings, which were not included in the ex-ante analysis. The period was taken as nine years, which is a realistic amortization period for this type of infrastructure investment (ICR, page 38). The ex-post EIRR was 37.25%, and the NPV at closure was US$69.12 million at a 10% discount rate. However, there were differences in the assumptions underlying the ex-ante and ex-post analysis (such as the inclusion of cybersecurity benefits and the impact of the project on GDP per capita in the ex-post analysis). Further, the economic analysis did not quantify the benefits such as improved access to good quality, social cohesion (supporting family relations, especially in poor households when transport costs are high and connecting with relatives living abroad), and improved business environment. Overall, while there were data limitations and differences in the assumptions underlying the ex-ante and ex-post analysis, the calculations showed that the project was economically viable both before and after the project. Administrative and operational issues during implementation. The project's progress, which became effective a year after approval, was slow in the first year. This was mainly due to the slow turnaround of submitted procurement deliverables by the World Bank task team, which slowed the ability of the implementing agency to launch key procurements. This was rectified, and all the activities were completed, albeit with a six- month extension of the project closing date. The project demonstrated good value for money. Despite the lower than planned counterpart funding at appraisal, all the envisioned activities had been completed with the Bank financing at the project closure. Overall, the efficiency of the project in achieving the project activities is rated as substantial. Efficiency Rating Substantial Page 9 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review RCIP 5 - Uganda (P130871) a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 89.00 Appraisal  17.00  Not Applicable 89.00 ICR Estimate  37.25  Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome The relevance of the PDOs to the Government and the Bank strategy is High. Overall efficacy and efficiency of the project are rated as substantial. This review concludes that there were only minor shortcomings in the project's achievement of its objectives (such as some targets were not realized) and in its efficiency (such as delays in effectiveness and slow progress in the initial year). The project's overall outcome is rated as satisfactory. a. Outcome Rating Satisfactory 7. Risk to Development Outcome The risks to the sustainability of development outcome are low to moderate. The ICR (para 65) notes that the extension of the NBI and last-mile fiber deployments to Government MDA is likely to be operated under the NITA-U's operational budgets. Operational budgets for other relevant sub-components, such as the data center, will likely be funded under separate Government line items in the long term. The ICR (para 67) notes that NITA –U agency’s operational structure, which operates and maintains the NBI, the GovNet fiber network, the national data center, and the e-services platform and infrastructure implemented by the project, is currently under review by the Government and subject to changes. While this includes the potential for it to be integrated with the Ministry of ICT and National Guidance, these changes in institutional arrangements need to ensure that operational budgets and responsibilities for these components are maintained so that the development outcome continues to be delivered in the long run. There is a risk that the availability of the NBI might not be maintained above the current 99.8% target to ensure operational sustainability and meet service-level agreement (SLA) contractual targets with private sector customers. The ICR (para 66) notes that this target was achieved at project closure, and availability levels were lower than the target for the project duration. The ICR also notes that once availability levels are Page 10 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review RCIP 5 - Uganda (P130871) fully stabilized, it will be important to consider raising this target in the future to above 99.9% to match global commercial expectations. There is a risk that new e-services might not be developed, and the data integration and sharing platform might not be fully utilized in the long term (ICR, para 65). However, the ICR noted that this is not expected to pose a high risk to development outcomes as the project financed TA activities to develop the enabling environment. The ICR (para 66) notes that private sector involvement is required for the long-term sustainability of the NBI. The ICR notes that ensuring availability levels of the NBI are maintained above 99.8% when the project closed is critical to ensuring operational sustainability and meeting service-level agreement (SLA) contracts with the private sector. 8. Assessment of Bank Performance a. Quality-at-Entry The Bank prepared this project in Uganda - the fifth phase of the RCIP - based on the experiences from RCIP phases from 1 to 4. Lessons incorporated in design included: (i) Given that multiple stakeholders were involved, the Bank prepared this project that was directly in line with the Government's vision, which saw the ICT sector as an enabler for key sectors of the economy; and (ii) Recognizing that the role of the Government is not necessarily to fund and build new infrastructure, but rather to establish the regulatory environment, the project design incorporated TA activities aimed at institutional strengthening of MICT and NITA-U. The technical aspects of the project design were similar to those of the previous four phases financed by the Bank. However, they were appropriately adapted to the country context but set within a coordinated regional framework. At appraisal, the Bank conducted an analysis to assess the project's economic viability. The implementation arrangements at appraisal were appropriate. A project management team established within the NITA-U was responsible for day-to-day implementation. The preparation team identified several risks at appraisal, including inadequate experience of NITA-U and procurement delays. The preparation team incorporated several mitigation measures. These measures were appropriate as the project performance was not undermined by the risks identified at appraisal. The arrangements for monitoring and evaluation and environmental and fiduciary compliance were appropriate (discussed in sections 9 and 10). Quality-at-Entry Rating Satisfactory Page 11 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review RCIP 5 - Uganda (P130871) b. Quality of supervision Ten Implementation Status and Results Reports (ISRs) were filed over the implementation period, implying twice a year supervision mission. The supervision team established close working relationship with the implementing agency (with regular fortnightly meeting with the client for the last three years of implementation). These meetings enabled the team to supervise the project during the COVID - 19 pandemic, when in person missions were halted. There were minor shortcomings. The project delays in the first year was mainly due to the slow turnaround of submitted procurement deliverables by the World Bank task team. This was however rectified when more robust arrangements were put in place. The ICR provides no information on the continuity of leadership during implementation. Overall, the Bank performance is rated as satisfactory. Quality of Supervision Rating Satisfactory Overall Bank Performance Rating Satisfactory 9. M&E Design, Implementation, & Utilization a. M&E Design The M&E indicators were based on the design of the other phases of the RCIP and used similar methodology and indicators. This allowed for an integrated approach to M&E across the broader RCIP program and also enabled better comparison of progress that could be used in future RCIP projects. The indicator baselines and targets of the M&E framework were then tailored for the specific Ugandan context (ICR para 47). While the indicators were appropriate, the project did not adequately encompass all outcomes of the PDO statement such as improving government transparency. One moderate shortcoming in M&E design was the choice of PDO indicators used to monitor the extended reach of broadband networks deployed by the project. The indicators used were "access to internet service" and "access to telephone services". The two indicators did not correlate to an increase in coverage. The ICR (para 48) observed that the design could have used other indicators such as the percentage of population with internet coverage, or geographic coverage of the country (such as derived from regulator mapping). b. M&E Implementation The ICR (para 50) observed that the M&E framework was effectively implemented during the project lifetime. The NITA-U provided timely updates to project indicators. Further, having industry-standard indicators precluded the need for additional data beyond those monitored by the regulator - the Uganda Page 12 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review RCIP 5 - Uganda (P130871) Communications Commission (UCC) and the NITA - U. The ICR (para 51) observed that the PDO indicator on direct project beneficiaries was captured as a point-in-time indicator. Using a cumulative indicator (over the project lifetime) would have been more appropriate to demonstrate how many beneficiaries were impacted by the project. c. M&E Utilization The ICR (para 52) noted that M&E data was actively used in project management and World Bank supervision activities and included in Implementation Supervision Reports (ISRs). This was particularly important given the two years of remote supervision during the COVID-19 pandemic. The ICR (para 53) observed that the targets set for several key indicators were significantly exceeded by project close, so these indicators would have been less useful in tracking progress during the project's lifetime. A re- evaluation of these indicators at the project mid-point could have helped in measuring progress during the second half of the project lifespan. While there were moderate shortcomings in the M&E design, implementation, and utilization, the M&E system was sufficient to assess the achievement of the project objectives and test the links in the results chain. Therefore, the M&E quality is rated as substantial. M&E Quality Rating Substantial 10. Other Issues a. Safeguards The project was classified as a Category B (partial assessment) under the World Bank safeguard policies. Six safeguard policies were triggered at appraisal: Environmental Assessment (OP/BP 4.01); Natural Habitats (OP/BP 4.04); Forests (OP/BP 4.36); Physical Cultural Resources (OP/BP 4.11); Indigenous Peoples (OP/BP 4.10); and Involuntary Resettlement (OPBP 4.12) (PAD, page x). Environmental Assessment, Natural Habitats, Forests, and Physical Cultural Resources. The environmental safeguards were triggered, as the project entailed civil works components. The environmental impacts were, however, expected to be site-specific and manageable. The Natural Habitats safeguards were triggered as the project activities could cross natural habitats such as wildlife conservation areas. The Forest safeguards were triggered, as there were forest reserves along the highways where the optical fiber cables were to be laid. The Physical Cultural Resources safeguards were triggered for addressing chance finds (PAD, page 65). As the specific project sites were unknown at appraisal, an Environment and Social Management Framework was prepared and publicly disclosed to address the safeguards (PAD, para 66). However, the ICR does not explain how these safeguard policies were implemented. Indigenous Peoples (IPs). The safeguard on IPs was triggered, as project activities could be implemented in northeastern and southwestern parts of Uganda inhabited by marginalized groups of people (including the Batwa IPs in Kisoro, Kasese, and the Kanungu districts and the Ik and Tepeth IPs in Karamojo). An IP Page 13 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review RCIP 5 - Uganda (P130871) framework was prepared and publicly disclosed at appraisal to address IP issues (PAD, page 65). However, the ICR does not explain how this safeguard policy was implemented. Involuntary Resettlement. This safeguard was triggered, as the project involved constructing base stations, which could entail land acquisition and involuntary displacement. A Resettlement Policy Framework (RPF) was publicly disclosed at appraisal to address these issues (PAD, page 65). However, the ICR does not explain how this safeguard policy was implemented. The ICR (para 55) noted that the project did not experience any environmental or social issues during implementation. However, reporting requirements for environmental and social issues were not followed according to the terms laid out in the agreed Environmental and Social Safety Plan. This resulted in non- reporting in 2021 (due partly to the impact of the COVID- 19). As subsequent reporting showed no issues, the overall rating for compliance with environmental and social safeguards was moderately satisfactory (ICR, para 55). b. Fiduciary Compliance Financial management. The Bank assessed the financial management arrangements of NITA -U at appraisal (PAD, page 22). With mitigation measures, financial management risk was rated as substantial at appraisal. There was one financial management issue during implementation. This was due to an ineligible expense relating to the procurement of the e-Government procurement system. The client refunded this. Procurement. The Bank conducted a procurement assessment of NITA-U at appraisal (PAD, page 23). The Bank identified the following risks: (i) inadequate experience of NITA staff and inadequate staff of NITA-U to address procurement issues. With mitigation measures, procurement risk was rated as substantial at appraisal. There were two procurement issues during implementation. The first related to procuring the Government e-procurement system under component three activities. Due to implementation challenges, the original contract for the development of the system had to be terminated. The Government decided to develop the system outside of the project. The second related to a case of mis-procurement of 90 laptops due to an incorrect interpretation of the required procurement process. The client refunded these costs. c. Unintended impacts (Positive or Negative) The ICR (para 41) noted that the project deployed a national Government data center facility for securely hosting existing Government information systems and new e-services implemented under the project. However, the NITA-U also utilized the excess capacity at the facility to offer paid hosting services to other market entities, such as the banking sector. At appraisal, a private sector data hosting market did not exist, and there were no other alternatives to a publicly owned and operated facility. When the project closed, multiple private datacenter operators offered hosting services in the Ugandan market. Page 14 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review RCIP 5 - Uganda (P130871) d. Other None. 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment Outcome Satisfactory Satisfactory Bank Performance Satisfactory Satisfactory Quality of M&E Substantial Substantial Quality of ICR --- Substantial 12. Lessons The ICR Review draws the following three main lessons from the experience of implementing this project, with some adaptation of language. 1. Drawing lessons from regional projects can help design and implement ICT projects. The preparation of this project benefitted from being part of the broader Regional Communication Infrastructure Program (RCIP). The inclusion of Uganda as the fifth phase of the Regional Communication Infrastructure Program (RCIP) enabled greater efficiencies in the design of the project, enabling a common set of indicators, standards, and scope to be used both in the design of the project and the structuring of implementation arrangements. However, drawing from the implementation approach of the previous RCIP projects, the project could have improved the connectivity outcomes for this project through greater involvement of the private sector in deploying the National Backbone Infrastructure (NBI) through Public-Private Partnership (PPP) or direct investment models. 2. Using clear, industry-standard indicators can help in the M&E of ICT projects. In this project, the indicators were industry standard indicators that were also used in other phases of the RCIP. Hence, no additional data was required for monitoring the project activities. 3. Deeper engagement with the private sector can improve the efficiency and sustainability of outcomes in ICT projects. When this project was approved in 2015, the digital ecosystem in Uganda was in its infancy. Since then, the ICT sector has matured, with new private sector players in new market segments. Ensuring future interventions such as this would help create a more conducive environment for the private sector. 13. Assessment Recommended? Page 15 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review RCIP 5 - Uganda (P130871) No 14. Comments on Quality of ICR The ICR is well-written and concise, and the main body of the text adheres to the recommended length of around 15 pages. The theory of change articulated in the text clearly articulates the causal links between project activities, outputs, and outcomes and provides adequate evidence and analysis to assess project performance. The ICR derives suitable lessons from the experience of implementing this project. However, the ICR is inconsistent with the Bank's guidance in that it does not provide sufficient discussion of implementing the safeguard policies. The ICR does not provide adequate information on why the project became effective a year after approval and the reasons for the lower than planned recipient contribution to the project. The ICR could have also better explained the reasons for substantially exceeding some of the targets. There is a discrepancy between the rating for overall Bank performance in the datasheet and the main body of the ICR text. While the datasheet shows the rating for Bank performance as Satisfactory, the main body of the text (ICR, para 63) shows the rating as Highly Satisfactory. a. Quality of ICR Rating Substantial Page 16 of 16