Document of The World Bank Report No: ICR00003451 IMPLEMENTATION COMPLETION AND RESULTS REPORT (IDA-H5530) ON A GRANT IN THE AMOUNT OF SDR 9.7 MILLION (US$15 MILLION EQUIVALENT) TO THE REPUBLIC OF BENIN FOR A e-BENIN PROJECT March 16, 2015 Transport and Information Communication Technologies Global Practice Africa Region -i- CURRENCY EQUIVALENTS (Exchange Rate Effective June 30, 2015) Currency Unit = SDR 1.00 = US$ 1.41 US$ 1.00 = 590.43 CFAF FISCAL YEAR ABBREVIATIONS AND ACRONYMS AGeNTIC Agence de Gestion des Nouvelles Technologies de I 'Information et de la Communication (New Information and Communication Technologies Management Agency) ABETIC Agence Béninoise des TIC ATRPT Autorité Transitoire de Régulation des Communications Électroniques et des Postes du Benin (Temporary Regulatory Authority for Posts and Telecommunications) ARCEP Benin Autorité de Régulation des Communications Électroniques et des Postes du Benin (Benin Regulatory Authority of Electronic Communications and Post) BCEAO Central Bank of West African States BP Bank Procedures BTSA Benin Telecom SA CAA Caisse Autonome d 'Amortissement (Autonomous Reserve Fund) CAS Country Assistance Strategy CDMA Code Division Multiple Access CPIA Country Policy and Institutional Assessment DFIs Development Financial Institutions DNTIC Direction Nationale des TIC (National Department of ICT) DP Direct payment FDI Foreign Direct Investment FDUTIC Fonds de Developpement Universel des TIC (ICT Universal Development Fund) GDP Gross Domestic Product GSM Global System for Mobile ICT Information and Communication Technology IDA International Development Association IFC International Finance Corporation ISP Internet Service Provider IXP Internet Exchange Point MCC Millennium Challenge Corporation -i- MCTIC Ministry of Communications and ICT MDG Millennium Development Goal M&E Monitoring and Evaluation ODOP Operational Directives Operational Manual PAD Project Appraisal Document PCU Project Coordination Unit PNGE Plan National sur la Gouvernance Electronique (National Program for Electronic Governance) PPF Project Preparation Facility PPP Public-Private Partnership PRSP Poverty Reduction Strategy Paper UNPAN United Nations Public Administration Network US$ United States Dollars WA Withdrawal application WBG World Bank Group Senior Global Practice Director: Pierre Guislain Sector Manager: Boutheina Guermazi Project Team Leader: Isabelle Huynh ICR Team Leader: Jacqueline Dubow - ii - Data Sheet A. Basic Information B. Key Dates C. Ratings Summary D. Sector and Theme Codes E. Bank Staff F. Results Framework Analysis G. Ratings of Project Performance in ISRs H. Restructuring I. Disbursement Graph 1. Project Context, Development Objectives and Design ............................................... 1 2. Key Factors Affecting Implementation and Outcomes .............................................. 4 3. Assessment of Outcomes ............................................................................................ 9 4. Assessment of Risk to Development Outcome......................................................... 14 5. Assessment of Bank and Borrower Performance ..................................................... 15 6. Lessons Learned ....................................................................................................... 17 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners .......... 18 Annex 1. Project Costs and Financing .......................................................................... 19 Annex 2. Outputs by Component ................................................................................. 20 Annex 3. Economic and Financial Analysis ................................................................. 24 Annex 4. Bank Lending and Implementation Support/Supervision Processes ............ 25 Annex 5. Beneficiary Survey Results ........................................................................... 26 Annex 6. Summary of Borrower's ICR and/or Comments on Draft ICR ..................... 32 Annex 7. List of Supporting Documents ...................................................................... 34 MAP - iii - A. Basic Information Country: Benin Project Name: eBenin Project Project ID: P113370 L/C/TF Number(s): IDA-H5530 ICR Date: 04/08/2016 ICR Type: Core ICR GOVERNMENT OF Lending Instrument: TAL Borrower: BENIN Original Total XDR 9.70M Disbursed Amount: XDR 9.58M Commitment: Revised Amount: XDR 9.58M Environmental Category: C Implementing Agencies: La Poste du Benin Cofinanciers and Other External Partners: B. Key Dates Revised / Actual Process Date Process Original Date Date(s) Concept Review: 02/12/2009 Effectiveness: Appraisal: 02/01/2010 Restructuring(s): 05/23/2014 Approval: 03/25/2010 Mid-term Review: 09/16/2013 10/28/2013 Closing: 06/30/2015 06/30/2015 C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Moderately Satisfactory Risk to Development Outcome: Moderate Bank Performance: Moderately Satisfactory Borrower Performance: Moderately Satisfactory C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings Quality at Entry: Moderately Satisfactory Government: Moderately Satisfactory Implementing Quality of Supervision: Moderately Satisfactory Moderately Satisfactory Agency/Agencies: Overall Bank Overall Borrower Moderately Satisfactory Moderately Satisfactory Performance: Performance: - iv - C.3 Quality at Entry and Implementation Performance Indicators Implementation QAG Assessments (if Indicators Rating Performance any) Potential Problem Project Quality at Entry No None at any time (Yes/No): (QEA): Problem Project at any time Quality of Supervision No None (Yes/No): (QSA): DO rating before Moderately Closing/Inactive status: Satisfactory D. Sector and Theme Codes Original Actual Sector Code (as % of total Bank financing) General information and communications sector 4 10 Information technology 4 5 Other industry 12 5 Public administration- Information and communications 40 50 Telecommunications 40 30 Theme Code (as % of total Bank financing) Infrastructure services for private sector development 30 10 Legal institutions for a market economy 10 10 Micro, Small and Medium Enterprise support 10 30 Other public sector governance 30 10 Regulation and competition policy 20 40 E. Bank Staff Positions At ICR At Approval Vice President: Makhtar Diop Obiageli Katryn Ezekwesili Country Director: Pierre Frank Laporte Madani M. Tall Practice Manager/Manager: Boutheina Guermazi Philippe Dongier Project Team Leader: Isabelle Huynh Boutheina Guermazi ICR Team Leader: Isabelle Huynh ICR Primary Author: Jacqueline J. Dubow -v- F. Results Framework Analysis Project Development Objectives (from Project Appraisal Document) The Project's development objectives are: (a) to improve access to lower cost and better quality ICT services; and (b) to enable the development of e-applications, in the Recipient's territory. Revised Project Development Objectives (as approved by original approving authority) The PDOs remained unchanged. (a) PDO Indicator(s) Original Target Actual Value Formally Values (from Achieved at Indicator Baseline Value Revised approval Completion or Target Values documents) Target Years Indicator 1 : Improvement of the ICT sector as a result of the technical assistance Value quantitative or 1 3 4 Qualitative) Date achieved 02/26/2010 06/30/2015 06/30/2015 Comments Target exceeded. This core indicator measures the impact of the enabling legal and (incl. % regulatory framework on access,quality and price of ICT services. Score 1-5 percent of achievement) achievement: 100 percent. Indicator 2 : Retail price of Internet services (including broadband) Value quantitative or $153 $80 $50 Qualitative) Date achieved 06/26/2010 06/30/2015 06/30/2015 Comments Target exceeded. The decrease in retail price of Internet can be indirectly attributed to (incl. % new legal and regulatoryframework promoting level playing field competition and achievement) enhanced regulatory oversight. . Indicator 3 : Increased percentage of public services using electronic transactions Value quantitative or 0 30 percent 3 pilots 0 pilots Qualitative) Date achieved 02/26/2010 06/30/2015 10/16/2013 06/30/2015 Comments Original indicator was replaced at MTR with "Number of e-gov services in a pilot (incl. % phase". Original indicator didn'tcapture properly the contribution of the project, limited achievement) to pilot projects (and not addressing the mainstreaming of eGov services.) Indicator 4 : Increased percentage of ICT SME Value quantitative or 18 30 99 Qualitative) Date achieved 06/26/2010 06/30/2015 06/30/2015 Comments This target measures the impact of improved enabling environment on the private sector (incl. % and ICT entrepreneurship - vi - achievement) (b) Intermediate Outcome Indicator(s) Original Target Actual Value Formally Values (from Achieved at Indicator Baseline Value Revised Target approval Completion or Values documents) Target Years Indicator 1 : Rate (%) of satisfaction of regulated entities with the improved capacity of the regulator Value (quantitative 0 60 66.7 or Qualitative) Date achieved 02/26/2010 06/30/2015 06/30/2015 Comments (incl. % Results from surveyed entities that are regulated by the Regulatory Authority achievement) Indicator 2 : Access to telephone and Internet services (# per 100 people), (%) Value Phone services 37 +20% +282% (104.33) (quantitative Internet services 2.14 +20% +888% (18.99) or Qualitative) Date achieved 06/26/2010 06/30/2015 06/30/2015 Comments The Project contributed indirectly to these outcomes with the new telecom law that (incl. % promotes competition, supportUniversal Access, and enables enhanced regulation. The achievement) annual 'Week of the Internet' offered opportunities to spur Internet use Indicator 3 : Access to broadband Internet services (% of population) Value (quantitative 2 +20% (2.4) 2.91 or Qualitative) Date achieved 06/26/2010 06/30/2015 06/30/2015 Comments (incl. % Access to broadband Internet services (% of population) achievement) Number of direct and indirect beneficiaries of component 1 (of which % of women) - Indicator 4 : Dropped at MTR Value (quantitative 3 million 4 million N/A or Qualitative) Date achieved 02/26/2010 06/30/2015 06/30/2015 Comments This indicator was dropped at MTR as there was no available baseline and no agency (incl. % able to track it (as wereIndicators 12 and 16.) achievement) Indicator 5 : User perception of quality of public services targeted by the project (%) Value (quantitative 25 50 0 or Qualitative) Date achieved 02/26/2010 06/30/2015 06/30/2015 Comments This indicator was adjusted at MTR to cover the three pilots envisaged under the (incl. % Project. The beta versionsstarted only in May 2015, a month before the Project's - vii - achievement) closing, giving no time for a citizen feedback. Indicator 6 : Number of e-Government services - Dropped at MTR Value (quantitative 17 25 N/A or Qualitative) Date achieved 02/26/2010 06/30/2015 06/30/2015 Comments (incl. % This target was dropped at MTR consistently with the modification of Indicator 3. achievement) Indicator 7 : Number of manpower trained under the project (of which % of women) Value (quantitative 0 500 250 259 or Qualitative) Date achieved 02/26/2010 06/30/2015 01/15/2014 06/30/2015 Comments (incl. % 23 percent of the trained workers were women. achievement) Number of direct and indirect beneficiaries of component 1 (of which %age of women) Indicator 8 : - Dropped at MTR Value (quantitative 0 1000 dropped dropped or Qualitative) Date achieved 02/26/2010 06/30/2015 06/30/2015 06/30/2015 Comments This indicator was dropped at MTR as there was no available baseline and no agency (incl. % able to track it (as wereIndicators 8 and 16.) achievement) Percentage of of applications financed through the matching grant that are available on Indicator 9 : the market. Value (quantitative 0 5 50 or Qualitative) Date achieved 02/26/2010 06/30/2015 06/30/2015 Comments This indicator measures the effectiveness of the matching grant mechanism in (incl. % identifying and supporting highperformers amongst ICT entrepreneurs. achievement) Difference (%) btwn the proportion of start-ups helped by AGENTIC that remain in Indicator 10 : activity after 2 yrs, with thosethat did not benefit from AGENTIC support. Revised at MTR to: “% of start-ups helped by AGENTIC that remain in activity after 2yrs" Value (quantitative 0 20 20 or Qualitative) Date achieved 02/26/2010 06/30/2015 06/30/2015 Comments The indicator shows the value of new services offered by AGENTIC (coaching, (incl. % incubating start-up SMEs in ICTsector). To ease tracking, it was modified at MTR. achievement) Indicator 11 : Percentage of firms qualifying for innovation grants that compete for the grant. Value 0 30 61.55 (quantitative - viii - or Qualitative) Date achieved 02/26/2010 06/30/2015 06/30/2015 Comments This indicator measures the quality of applicants (that is their qualifiability), which (incl. % reflects the potential ofyoung ICT entrepreneurs in Benin. achievement) Number of direct and indirect beneficiaries of component 2.2 (of which %age of Indicator 12 : women) - Dropped at MTR Value (quantitative 0 1,000 dropped dropped or Qualitative) Date achieved 02/26/2010 06/30/2015 06/30/2015 06/30/2015 Comments This indicator was dropped at MTR as there was no available baseline and no agency (incl. % able to track it achievement) G. Ratings of Project Performance in ISRs Date ISR Actual Disbursements No. DO IP Archived (USD millions) 1 10/26/2010 Satisfactory Satisfactory 1.07 2 05/20/2011 Satisfactory Satisfactory 1.44 3 12/17/2011 Satisfactory Satisfactory 2.16 4 05/23/2012 Moderately Satisfactory Satisfactory 2.87 5 01/01/2013 Moderately Satisfactory Satisfactory 4.92 6 07/10/2013 Satisfactory Satisfactory 7.22 7 01/11/2014 Moderately Satisfactory Moderately Satisfactory 9.33 8 07/01/2014 Moderately Satisfactory Moderately Satisfactory 10.37 9 12/21/2014 Moderately Satisfactory Moderately Satisfactory 12.71 10 06/15/2015 Moderately Satisfactory Moderately Satisfactory 13.81 H. Restructuring (if any) ISR Ratings at Amount Board Restructuring Restructuring Disbursed at Reason for Restructuring & Key Approved PDO Date(s) Restructuring Changes Made Change DO IP in USD millions A Level 2 restructuring was approved on May 23, 2014. The restructuring involved: (1) reallocation betweencategories of expenditure to allocate unallocated 05/23/2014 N MS MS 10.20 funds; and, (2) modifications to the Results Framework to better capture thecausality between project activities and results, and adjust some targets accordingly. - ix - ISR Ratings at Amount Board Restructuring Restructuring Disbursed at Reason for Restructuring & Key Approved PDO Date(s) Restructuring Changes Made Change DO IP in USD millions PDO indicator 3 was modified because it wasdisconnected from the project realm. A new indicator was added to capture the result on La Poste du Benin, which became a majorbeneficiary of the project. I. Disbursement Profile -x- 1. Project Context, Development Objectives and Design 1.1 Context at Appraisal 1. Country background. Benin is a small country located in the Gulf of Guinea, next to Nigeria, with a population of 8.7 million and a per capita annual income of $540.1 Following an economic collapse and a political crisis in 1989, the country moved to a pluralist democracy and free market economy. This resulted in three peaceful democratic transitions in government and enhanced political and social stability throughout 1996-2008. In addition Benin has undertaken important structural reforms that have helped establish fiscal discipline, opened the economy, privatized some public enterprises, and improved incentives for the private sector. Most social indicators improved steadily throughout the 1990s, including those for education, health, water supply, rural sanitation, and access to modem energy services. By providing technical assistance to help the Government create an enabling environment for the information communication technologies (ICT) sector, the e-Benin Project expected to support the Government achieve its vision and leverage the sector to be a key driver of growth, competitiveness, and improved governance. 2. Sector background. The Government of Benin has placed reform of the ICT sector as well as improved access to ICT at the center of its development agenda, as reflected in the 2008 Sector Development Policy. The Government’s vision to convert Benin into the "Digital Capital of Africa," taking advantage of the country's privileged location and its access to international connectivity. This vision is an ambitious one for Benin given current constraints in the sector, such as uneven access to communications infrastructure, high prices for services, a nascent legal regulatory and institutional framework, a shortage of skilled human resources in regulation and in ICT-specific disciplines, and limited diffusion of ICT across the Government, as well as within the business community. The sector remains under developed and is still unable to contribute fully to Benin’s economic and social development. 3. Benin ranked relatively low on the competiveness index (100 out of 134 economies) as a result of a number of ICT constraints, as summarized in Table 1. Table 1. Benin’s ICT Constraints Score Rank Availability of latest technologies 3.8/4.2 107/113 Firm level technology absorption 4.4/4.2 88/114 Laws relating to ICT 3.4 88 FDI and technology transfer 4.2 107 Mobile telephone subscribers 85.3 102 Internet users 1.4/3.4 120/116 1 All dollars are in United States dollars unless otherwise indicated. -1- Personal computers 0.6 122 Broadband Internet subscribers 0.0/0.0 113/128 Source: Africa Competitiveness Report (Benin) 2013 1.2 Original Project Development Objectives (PDO) and Key Indicators (as approved) 4. The PDOs as described in the Financing Agreement are: (a) to improve access to lower cost and better quality ICT services, and (b) to enable the development of e- applications, in the Recipient’s territory. 5. The core sector indicators are: (a) improvement of the ICT sector as a result of the technical assistance, (b) retail price of Internet services -including broadband Internet-, (c) increase percentage of public services using electronic transactions, and (d) increase percentage of ICT small- and medium-enterprises (SMEs). 1.3 Revised PDO (as approved by original approving authority) and Key Indicators, and reasons/justification 6. The PDO remained unchanged during project implementation. 1.4 Main Beneficiaries, 7. The main beneficiaries fall into three categories: (a) the people of Benin who are benefiting from improved access and lower prices; (b) government agencies, and their employees; and (c) stakeholders in the ICT sector of Benin, including the private sector and entrepreneurs. The PAD broadly identifies beneficiaries as the private sector, ICT firms, and telecommunications operators. The aide-memoires provide more specifics and indicate the main beneficiaries of the Project to be the Ministry of Communication and ICT (MCTIC), the Agence de Gestion des Nouvelles Technologies de I 'Information et de la Communication (AGeNTIC) and La Poste du Bénin (La Poste). 1.5 Original Components (as approved) 8. The Project’s components as described in the PAD are. Component 1: Enabling Environment ($5.6 M) Strengthening legal, regulatory and institutional framework. Implement the program to develop a legal, regulatory and institutional framework that will foster competition in the ICT sector, such program to include: (a) the development of complementary legislation and regulatory instruments in the ICT sector, and (b) strengthening of regulatory capacity of ARTP and sustaining policy making capacity of the MCTIC. Provision of affordable high-speed Internet connectivity and universal services. Implement a program to promote affordable high-speed connectivity and universal access in the Recipient’s territory, such program to consist of: (a) develop a broadband strategy and ways to tackle bottlenecks in the supply of broadband services, (b) develop regulatory tools to ensure open and improved access to bandwidth and lower prices of -2- connectivity, (c) develop a universal access regime, including identification of areas for universal access; and (d) study and design the national and regional exchange points. Component 2: Enabling e-Applications and Facilitating Uptake of e-Business ($7.0 million) e-Government Applications. Conduct studies to develop the legal framework, enterprise architecture and interoperability standards for e-Government applications, and implementation of several e-application pilot projects. e-Business. Strengthen AGeNTIC’s capacity to develop and implement programs targeting ICT entrepreneurs with some incubation-type of services and support, to support greater awareness of ICT and e-applications. Provide matching grants to eligible public or private entities to support specific development projects to develop, rollout and to scale-up private sector ICT applications. Component 3: Project Implementation; Monitoring &Evaluation ($1.8 million) Conduct a program of activities to strengthen the Recipient’s institutional capacity, at the Project Implementation Unit (PIU) level, for Project implementation, coordination, financial management, monitoring and evaluation, including establishment and operation of a results-based monitoring and evaluation system, maintenance and management of the management information system and provision of associated training, and carrying out of implementation-related studies, assessments, surveys and communication strategies which are aimed at raising public awareness regarding reform in the ICT sector. 1.6 Revised Components 9. The components remained unchanged. 1.7 Other significant changes 10. At the midterm review (MTR) in October 2013, some indicators were adjusted or dropped as they were either not measurable, disconnected, or overreaching. Details on the indicators can be found under section F of the Data Sheet. 11. In May 23, 2014, based on the findings of the MTR, and at the request of the Government, a Level 2 restructuring was approved by the Country Director to reallocate resources and modify the Results Framework. The restructuring involved: (a) reallocation between categories of expenditure to allocate the refund of preparation advance and unallocated funds; and (b) modifications to the Results Framework to better capture the causality between project activities and results, and adjust some targets. The details of project proceeds reallocated can be found in Annex 1, Section c. 2. Key Factors Affecting Implementation and Outcomes 2.1 Project Preparation, Design and Quality at Entry 12. The project was prepared over 13 months, from the PCN review to the Board Approval (February 2009 to March 2010). The project became effective five months after Board Approval (August 2010). Project preparation and design were appropriately responsive -3- to the government’s reform initiatives in the telecom sector. The design was forward-looking, particularly concerning the AGeNTIC’s component. By complementing the e-Government transformation with the development of the private sector in anticipation of the greater ICT outsourcing needs of the Government, the Project adequately addressed the self-sustainability of the new government services delivery model that it promoted. 13. The background analysis conducted to prepare the project was sound. The technical analysis of the sector was thorough and the Project activities were aligned with the National Policy and fully owned by the Government. In November 2006, the Office of the President of the Republic released an interim roadmap for improving, rehabilitating, and revitalizing the ICT sector (ICT Plan - Benin). With World Bank support, the Government adopted a new sector that policy (December 2008) that was consistent with the (then proposed) Project, and conformed to international best practices while reflecting Benin’s context. The sector policy laid out the strategic objectives and the actions to be taken to facilitate the uptake of the ICT sector. In addition, the Government developed a National Program for Electronic Governance with a clear vision to use ICT as a tool to improve governance and transparency, and to strengthen the administration. 14. The Project draws on lessons learned from previous and ongoing World Bank financed projects in ICT (such as e-Rwanda, P098926, or e-Sri Lanka additional financing, P128702), as well as the 2008 ESW report on the ICT sector in Benin. One of the key lessons learned is the need for an integrated approach to set the basis for the transition to a digital government (connectivity infrastructure/supply, applications/demand, and capacity building/enabling environment). 15. The assessment of risks and mitigation measures prepared by the team at appraisal was adequate. The overall risk of the project was rated as substantial at entry (PAD pages 14-15), and included eight issues (Table 2). All identified risks materialized. One risk was successfully turned into an opportunity (access to international capacity). For the rest, mitigation measures allowed the Project to progress, albeit at a slower pace than what was expected, and consequently with delay in impact. -4- Table 2. Risks and Mitigation Measures Risks Mitigation Measures Delays in enactment of The new law was enacted only in August 2014 with significant delay (some enabling legal framework say that the first version had been drafted back in 2004). As a consequence, the updated institutional framework (set-up of the Regulator, creation of the universal access agency, and a number of regulatory tools aiming at insuring a more level playing field) materialized. There was little that the Task Team and the Beneficiary could do, other than finance a couple of events with the Parliamentary commission in charge of law, but to little effect. The Slow transition from the interim During most of the project implementation, and because of the delayed regulator to the new regulatory adoption of the law, the interim regulator was in place with limited powers. authority and a lack of During that time however, the project funded some important studies confidence of private players in (broadband strategy, universal access strategy) that involved the interim the competence of the regulator; capacity building was also funded. It is only when the law was new regulator enacted that the Regulator was able to implement new regulatory tools in the areas of interconnection tariffs and quality of service in particular. Impediments to gaining access to As the e-Benin Project implementation started, it became clear to the international capacity at competitive Government and the World Bank that a direct access to international capacity prices through a connection to a submarine cable, was necessary to take Benin to the next level of the digital transition. Based on this vision, the West Africa Regional Communications Infrastructure Project, Benin was swiftly prepared and is currently under implementation. Benin Telecom SA (BTSA) The Government attempted privatization failed, and BTSA has remained an remains a public entity in a underperforming public operator that maintained relatively high prices of access to private sector-led industry its infrastructure. Leadership, implementation The Project suffered a lack of strong leadership, with four ministers that capacity, and sustainability of the e- succeeded at the head of ministry in charge of ICT. Only the last but one government agenda contributed to a tangible progress with the enactment of the new sector law. Uncertainty over e- business AGeNTIC took a long time to effectively start implementation of its activities. potential As a result, this component outcome is not satisfactory, and the sustainability of the Government’s support to innovation and entrepreneurship in the ICT sector remains questionable. Resistance to institutional In spite of awareness-building and training, the change management was not changes in business processes successful and resistance to change affected the project outcomes, notably for the enterprise architecture implementation, as well as for the three eGov pilot projects. Risks of a change in The frequent changes in government contributed to the delays in the law government/transition to a newly adoption as well as effective change management. elected administration and continuation of policy and reform agenda 16. The e-Benin Project is the World Bank’s first ICT project in Benin. The design responded to the immediate needs of the country at the time. Nonetheless, the project objectives of improving access through lower cost and better quality ICT services and the expected outcomes were ambitious compared to the institutional context, the limited access to ICT infrastructures and the weak capacities of the country. The Project title as well as its -5- objectives created the disproportionate expectation that Benin would transition swiftly to the digital government. In fact, the Project was only laying the foundations (policy, institutional, legal and regulatory, and capacity building) for the future e-Government era. 17. At Appraisal, the Project had valid reasons to justify the approach adopted due to: (a) the lack of enabling legal and regulatory framework, and (b) the absence of alternative support from other partners to support sectoral reforms. 2.2 Implementation 18. The IDA Grant (H553-BJ) was approved on March 25, 2010 and became effective on August 25, 2010; the original closing date of June 30, 2015, was not extended. 19. Project implementation started slowly due to procurement delays and to MCTIC’s weak capacity. The time for obtaining opinions from the National Directorate for Public Procurement Control (DNCMP) varied from four to 12 weeks. The lapses delayed implementation as most procurement activities fell under the DNCMP’s jurisdiction. The due diligence processes were slow, despite the TTL’s requests to accelerate these processes. The PIU team ended up preparing most of the terms of reference and procurement plans to make up for the limited capacity of focal points in implementing agencies. The consolidation of several activities to optimize procurement activities, and the definition of a clearer division of responsibilities among the different focal points were corrective actions initiated by the PIU, which allowed disbursements to accelerate. 20. There were long delays in the enactment of the new law. The new law was part of the regional harmonization process 2 , and the Project’s cornerstone 3 . The lower (than at preparation phase) government’s commitment to reform, the administrative inertia against change management, and the complexity of the law process in the country were identified risks that fully materialized. The law was enacted in August 2014, four years after the Project’s inception. It was a major milestone in the project’s life, directly attributable to the constant pull and push exercised by the Minister, the PIU and the World Bank (task team and CMU). By Project closing, it is too early to assess the effectiveness of the new institutional entities that were enabled by the Project, but they appear in the Beneficiary’s ICR to play a growing role in the sector. 21. Leadership and implementation capacity of the e-Government agenda were weak. By Project closing, the flagship pilot applications to be supported by MCTIC, and then after by the newly created Benin ICT Agency, ABETIC (as of March 2015), were still at an early 2 The Economic Community of West Africa States, and the West African Economic and Monetary Union (two regional organizations) had pushed for harmonizing telecom laws in member countries. 3 The law would enable: (a) the appointment of a full-fledged regulator (the interim entity, active for 12 years, had nearly no executive powers); (b) the provision effective regulatory tools to enhance transparency and greater competition in the sector, and (c) the creation the Information and Communication Technologies Management Agency (ABSU-CEP) to increase Internet access. -6- concept stage, and the pilot projects (civil registry in three municipalities, on-line statistics and information in 12 schools, and e-visa) had a soft launch in June 2015. 22. Component 2, supporting e-Business was well designed but experienced slow implementation and its impact will likely not be long lasting. AGeNTIC had been created as a public-private partnership and had been identified as an adequate entity to implement the component. However, AGeNTIC had a slow start launching the procurement process for the incubation center and beginning the training activities. In fact, it took two years for AGeNTIC to operationalize its incubator arm, with several difficulties in procuring IT equipment, setting up the matching grant mechanism, and reporting to the PIU. In an effort to strengthen AGeNTIC, implemented the suggestion of the Task Team and, hired two additional individual consultants, one IT engineer and one marketing and development engineer—this additional staffing contributed to accelerating this component’s implementation, albeit with delays. 23. The MTR was conducted in October 2013, at which time 55 percent of the activities had been implemented (with a disbursement ratio at 53 percent and commitments at 74 percent), but delays were noted on the telecom law, the e-Business component, and La Poste’s activities. The postal activities had been refined after Project Effectiveness to take advantage of the postal network as a platform of governmental service delivery. The Government’s MTR report identified the following factors behind the delays in delivering activities: (a) a project not well understood by some parts of the administration and civil society that expected the realization of concrete infrastructure; (b) the lack of sharing of information between the different agencies and ministries; (c) a lack of human resource capacity (telecommunication engineers and technical staff) within the MCTIC to operationalize the studies recommendations; (d) the resistance to change from some stakeholders; (e) the delays in adopting the new law, the e-Business component implementation, and the establishment of a full-fledged regulatory authority (ARCEP); (f) the high turn-over of staff within the MCICT; and (g) the slow procurement evaluation processes. 24. The MTR rated Components 1 and 2 ‘Moderately Satisfactory’ and Component 3 ‘Satisfactory.’ The main take-away was that these ratings could worsen quickly if: (a) the new communication law was not approved, and (b) the ARCEP and the Delegation Nationale des TIC were not established promptly to improve the e-Business climate. As a result of the MTR, the project underwent a Level 2 Restructuring to reallocate some resources and modified the Results Framework (see details section 1.6). After the MTR implementation continued with a major milestone in July 2014 (the new law), but there were growing delays and the uncertain sustainability of the e-Business activities, as well as of the three e- Government pilot projects. -7- 2.3 Monitoring and Evaluation (M&E) Design, Implementation and Use 25. The PDO illustrated the Project’s objectives adequately. The indicators provided a quantitative baseline and clear target indicators against which progress towards objectives were assessed. Nonetheless, there were some intermediary indicators to which the Project was contributing only indirectly, such as “access to telephone and Internet,” (the new law was to enhance market efficiency, and therefore increased access), or not at all, such as “increase in percentage of public services using electronic transactions” (this indicator was dropped at MTR). One of the PIU team members was a monitoring and evaluation (M&E) specialist. At project design, information and data were to be provided by ARCEP-Benin, MCTIC, and existing operators where relevant, and national entities in charge of collecting economic and demographic data. Over the course of the project, ARCEP-Benin, AGeNTIC, and MCTIC developed their capacity to collect and analyze information. 26. During implementation the PIU team collected data regularly, despite some difficulties in getting all data from the different focal points units, and provided regular semi-annual reviews. Project reporting was detailed and was key to managing the Project, as well as to helping the Bank be more effective in supervision. At the MTR, the Results Framework was adjusted to better reflect what could or not be attributed to the Project: for instance, “increase in percentage of public services using electronic transactions” was subsequently replaced by “number of e-Government services in a pilot phase.” 27. The data was used mainly for the purpose of monitoring the project implementation. It helped bringing up corrective action when needed. For instance the slow uptake of the PDO indicator 4 (increase in percentage of ICT SMEs) indicated that the e-business subcomponent was experimenting delays. After MTR, an increased effort to supervise this subcomponent and provide support to AGeNTIC started. After project closing, some of the data are still collected, in particular those pertaining directly to the telecommunications sector, such as PDO indicator 2 on the retail price of internet services. 2.4 Safeguard and Fiduciary Compliance 28. Financial management was rated satisfactory throughout Project implementation. PIU staffing remained adequate and proper books of accounts and supporting documents were kept in respect of all expenditures. The auditor’s opinions on the annual financial statements were submitted on time and were unqualified. 29. Procurement rated satisfactory throughout Project implementation. Apart from one procurement activity that kept pending for several months at AGeNTIC (due to equipment that did not comply with the technical specifications) all components were implemented following the Bank’s procurement guidelines for works, goods, and services. The PIU was staffed with a Procurement Specialist proficient in Bank procedures. 30. The Project was classified as Category C because of its low environmental impact. The Project involved technical assistance and building capacity to create an environment to -8- enhance ICT access and applications. No civil works were financed; and a (limited) rehabilitation of the IXP building, and the AGeNTIC office. 2.5 Post-completion Operation/Next Phase 31. The Project has contributed directly to the creation of the West Africa Regional Communications Infrastructure Project (WARCIP), Benin (active until June 2017), which finances the connection to the undersea cable ACE to bring massive international capacity to Benin (the connection was effective early 2015). In that sense, the Project contributed to setting the conditions of long-term development of the ICT sector in Benin. Beyond the supply side of the sector, the Project set a clear course of action to roll-out e-Government services throughout the Benin administration. The strategic and financial recovery plan of La Poste can also be considered a success: La Poste can now play an active role in enhancing the government service delivery through different channel (physical through post offices, digital through the Internet, and financial transactions). 32. Efforts are still required to complete the regulatory framework with the specific laws on information society (cybercrime, security of electronic transactions, electronic signature, encryption, etc.), the introduction of regulations with standards on domain names, online content development, consumer protection, interoperability standards for e-Government, domestic backbone connectivity (reaching out to rural and remote areas, etc.). These laws and regulations are key to the sector’s sustained development and the protection and security of all its stakeholders. By MTR the Government expressed interest in preparing an ‘e-Benin 2 Project’ to build on the original Project and operationalize the 17 studies completed during its implementation. However, in view of the slow progress on the law (at the time), on the e- Government services pilot projects and on the e-Business component, the Task Team conditioned the preparation of a Phase 2 to improvements in these weak areas. By Project closing, the pilot projects were prioritized to insure they would be launched on time. The possibility of a next phase shifted to the WARCIP Project through which the dialogue on policy continues. 3. Assessment of Outcomes 3.1 Relevance of Objectives, Design and Implementation Overall Relevance Rating: Substantial 33. Objectives. The PDOs were built around the country needs, as well as the Bank’s country and sectoral assistance. The Project supported two key pillars of the Bank's Country Assistance Strategy (CAS) for FY09-12: (a) strengthening competitiveness and accelerating private sector-led growth (CAS outcome 1); and (b) promoting better governance and strengthening institutional capacities (CAS outcome 3). 34. Design. The Project activities were relevant to the objectives and the components were adequately designed to achieve Project outcomes. The activities encompassed the necessary building blocks for raising Benin’s capacity in ICTs starting with its administration. -9- The Project catered to the enabling environment for administration’s digitization, the incentives to develop and pilot e-Government services, and to support services to the ICT SMEs so they can work with and help the administrations to transform more efficiently. Two dimensions could have been better framed at the onset. In component 2, AGeNTIC was designated to implement the e-Business activities and act as an incubator for young entrepreneurs, but its location in an exclusive area of Cotonou was a barrier to attracting the targeted audience. Secondly, the PDO stated that the project would contribute towards improving access to lower cost and better quality ICT services. This may have contributed to the false perception that the Project was financing infrastructure or rolling-out a full-fledged e-Government program at a national scale. 35. Relevance of implementation. Appropriate adjustments were made during implementation that maintained the Project’s relevance. For instance, Component 1 emphasized strengthening La Poste as a potential platform of service delivery and responded to the Government’s decision to refinance the state-owned enterprise (SOE) debt. Component 2’s greater focus on AGeNTIC started after MTR when it became clear that the agency faced difficulties in making progress in implementing its activities. To help with this, an international advisor and two local consultants were hired to provide timely and tailored support. 3.2 Achievement of Project Development Objectives Rating: Substantial 36. The evidence for the achievement of the PDO needs to show that: (a) there has been an improved access to lower cost and better quality ICT services; and, (b) e-applications have been enabled. The Results Framework shows that three out of four of the project PDO indicators’ targets were met. Substantial achievements were realized in the policy, legal and regulatory enabling environment, as well as increased access and lowered prices. Regarding the activities related to the e-Government applications and the e-Business subcomponent, all but three (out of 11) intermediate outcome indicators were met. Creation of an enabling legal and regulatory framework for the sector 37. The ICT sector in Benin has undergone some important changes as a result of the e-Benin Project. In addition to the new law, the regulatory framework was strengthened with more than 20 pieces of secondary legislation adopted or in final draft. The institutional landscape evolved, with the transformation of the Temporary Regulatory Authority for Posts and Telecommunications into the Agence de Régulation des Communications Électroniques et de la Poste (ARCEP). The Project supported the set-up of ABSU-CEP in charge of the universal service access, as well as the Benin Agency for ICT (ABETIC) in charge of the ICT development in the public and private segments of the economy. These three agencies benefited from technical assistance meant to accelerate their operationalization. For instance, ARCEP’s capacity was strengthened through a number of studies and training on topics such as interconnection catalog, number portability, ‘.bj’ management, cybersecurity, and postal regulation. Finally, the Project also funded the feasibility studies for an Internet exchange point (to be financed under the WARCIP project), and a national data center which constitute important - 10 - infrastructure for economies transitioning to the digital economy. 38. The largest technical assistance activity supported a report on enterprise architecture, norms and standards, and interoperability framework, which represent the technical foundation of all e-Government endeavors. This technical assistance is a prerequisite to any sustainable effort to transform and modernize a public administration and enable its transition towards the digital economy. The output of this technical assistance was of high quality, and was endorsed by a cross-ministerial committee in which IT Directors of 12-15 ministries participated. By Project closing, ABETIC was trusted with the leadership of the e-Government program. Being an autonomous entity helps it work constructively with all governmental agencies, setting aside potential turf battles, and progressively bringing them to adopt the new enterprise architecture framework and underlying interoperability standards and norms. 39. The retail price of the Internet dropped significantly from $325 in 2010 to $50 in 2015. The Project contributed to increase the demand for Internet, which contributed to lower prices for service providers. The annual events ‘Week of the Internet’ funded by the Project also contributed to the democratization of the Internet. 40. The SOE La Poste benefited from the Project through technical assistance and enhanced dialogue with the Autonomous Reserve Fund ( Caisse Autonome d’Amortissement) which agreed to a large debt refinancing to clean La Poste’s balance sheet. The Project funded inter alia an updated corporate strategy, a design for new management information systems (MIS), and a semi-residential strategic advisor, which, together with in-house efforts to redress the profit-and-loss balance, and organizing the auditing of nine fiscal years, led to a full turn-around of the enterprise. By successfully strengthening La Poste, the Project allowed the SOE to become a key player in the delivery of new e-Government services. The intermediate outcome indicator relative to La Poste, which dealt with the acquisition and operationalization of a new MIS and its outcome (ability to produce timely six-month profit-and-loss accounts to better manage the organization), was not reached. The MIS was procured and put into operation, but the semiannual profit and loss was not produced by the time the Project closed, although the beneficiary explained that the new MIS contributed greatly to improved managerial capacity. Enabling e-Government applications and e-Business 41. Three online pilot government services ("e-services") were chosen to materialize the recommendation of the study on e-Government validated in July 2014 -see paragraph 38-. This was a small subcomponent by its amount (about $160,000) but important in the sense that these pilots were one of the few tangible changes that citizens could observe and benefit from. The first pilot, on civil registry, aimed at testing electronic management of civil forms in three cities (Abomey-Calavi, Banté, and Bassila). The second pilot, on education, aimed at managing electronically classroom activities, teachers’ attendance and their class performance, students’ grades and their attendance, parents’ communications, finances as well as collecting statistical data in 11 schools (in cities of Akassato, Aklampa, and Djougou,). The third pilot dealt with online visa application with the Department of Immigration and Emigration. There was a concerted effort, with AGeNTIC, to reach out to local ICT firms and - 11 - entrepreneurs so they participate to the competitive selection for the development of the e- Government solutions at the pilot phase. One of the pilots was awarded to an SME that had benefited of AGeNTIC’s support services.” 42. Because these pilots were launched with delay, it is too early to assess their success. The beta version for each pilot application (education, civil registry and visa) were delivered about three months before the Project closing (instead of one year), and an official launch with the Minister took place in June 2015. The delayed adoption of the e- Government study (four-month delay) was increased by municipal elections in early 2015 which led to a change of mayors in most cities, and temporary suspension of the civil registry pilot implementation. The outsourcing of the e-Government pilot applications to local firms was a successful process, proving that talented and capable ICT-related small enterprises could support Benin’s e-Government services deployment. As of January 2016, the three pilots were not mainstreamed. The PIU has reported that attempts to mainstream the pilots have been delayed by the electoral calendar (presidential elections scheduled in March 2016). 43. AGeNTIC supported 24 start-ups of young ICT entrepreneurs and provided matching grant benefiting 16 SMEs investing in technology, therefore, stimulating the private sector. 24 project leaders have benefited from the incubator support and their online applications are available today to the market and five of them have created companies; 16 companies received co-financing and management support. Over 800 people (including 29 percent women) have received training provided by the incubator. The results are positive and comparable to the first year of the CTIC Dakar, a major regional incubator. 3.3 Efficiency Rating: Substantial 44. The efficiency in carrying out Project activities was satisfactory; and the Project’s objectives were achieved with the resources budgeted initially. From preparation to completion, the costs in achieving project objectives were reasonable in comparison with both the benefits achieved and with comparable projects. The PIU and the implementing agencies implemented their activities efficiently with no budget overrun. The quasi 100 percent disbursement rate with no Project extension nor additional financing testifies to a well-controlled activities implementation. The delays didn’t generate extra costs, but deferred benefits for the citizens who are expecting improved interactions with the administration through the mainstreaming of e-Government services. 3.4 Justification of Overall Outcome Rating Rating: Moderately Satisfactory 45. Based on the ratings for the relevance, achievement of objectives, and efficiency, the overall outcome is rated as Moderately Satisfactory. 3.5 Overarching Themes, Other Outcomes and Impacts (a) Poverty Impacts, Gender Aspects, and Social Development - 12 - 46. The strengthening of the regulatory framework is directly attributable to the Project, and the drop in Internet prices allowed consumers in Benin to benefit from lower costs. As time and mobility constraints are more important for women than men, enhanced access to Internet usually translate into improved opportunities for women. The Results Framework also captures some outcomes of the Benin’s women’s population: 23 percent of the persons who benefited from training provided under the Project were women (indicator 10) and 29 percent of the persons who benefited from the support of AGeNTIC-led activities were women. (b) Institutional Change/Strengthening 47. The Project focused on creating a predictable legal and policy environment by strengthening the body of legal and regulatory texts, and building regulatory capacity. It funded twining arrangements and opportunities for peer-to-peer learning from more advanced regulators (Côte d’Ivoire, France, and Senegal). It contributed to building national capacity on e-Government and e-Business through the creation of several agencies the ARCEP, the ABSU-CEP, the ABETIC, the recovery of La Poste agency, positioning the agency as a strong point of entry for e-Government services, is a strong outcome. Overall, an important long-term impact can be expected to carry over into the WARCIP Benin Project. (c) Other Unintended Outcomes and Impacts (positive or negative) 48. The Project triggered the realization that the “supply” side of the Internet economy was not in place to allow blossoming of the e-Government and e-Business. This led to the preparation of a complementary project increasing international connectivity of the country, WARCIP Benin, and late in the Project life, the procuring of a portion of the national backbone in the northern part of the country, through an EximBank of China loan. 3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops 49. A beneficiary survey was conducted in June 2015, at project closing. It targeted a group of 60 persons, including 40 staff working in the ministries or agencies supported by the project, and 20 people randomly chosen with no affiliation with the Project. This survey reused the same questions used during the MTR. Subject MTR (10/2013) (%) Project Completion (%)(07/2015) Project pertinence 85 93 Capacity building 78 69 satisfaction level Internet connection quality 15 (good) 25 (medium) 60 (bad) 15 (good) 25 (medium) 60 (bad) Internet connection cost 90 (too expensive) 60 (too expensive) Strength of the e-Benin project: - Public and private sector targeting - Successful negotiation of the WARCIP project - Small PIU - Capacity building - Flexibility of the project - 13 - - Political to support the project Weaknesses of the e-Benin project: - Infrastructure component absent from the project - High level of sophistication of the studies lessen their knowledge impact with many recipients - Weak understanding of the project goal within the general population 4. Assessment of Risk to Development Outcome Rating: Moderate 50. The risk to development outcome relative to enhanced access to ICT services is low. The new legal and regulatory framework is the cornerstone of the Project outcome sustainability and, while it will carry long term development, it calls for a new generation of texts on the digital economy (electronic signature, cybercrime and cyber security, data protection, etc.) for which the Project provided resources to help prepare the first drafts. The changes brought by the Project at the institutional level are also expected to last (ARTPT, ABETIC, ABSU-CEP). Together these changes have contributed to increasing the access to lower cost and better quality ICT services. 51. The risk to development outcome relative to e-services is moderate. Two main risks are foreseeable: (a) with ABETIC taking over the e-Government services development and the spurring of the ecosystem (including the private sector development of ICT SMEs), AGeNTIC’s future may be at odds which would negate the Project’s investment to build the agency’s capacity; and (b) the high quality set of reports on interoperability, enterprise architecture, and norms and standards may not be put to good use, particularly since the e- Government team of MPTIC transferred its mandate to ABETIC. 5. Assessment of Bank and Borrower Performance 5.1 Bank Performance (a) Bank Performance in Ensuring Quality at Entry Rating: Moderately Satisfactory 52. The Project design at entry was appropriate. Project identification built on previous technical assistance activities relative to sector policy and reform. The Project scope was therefore well aligned with Government’s strategic priorities, while also falling under the Country Assistance Strategy’s framework. The Project scope and objectives were well proportioned to the size of the grant amount. The Project preparation was conducted with responsive and proactive engagement by the Task Team, who benefited from good quality dialogue with the Government’s experienced sector experts and officials. The fiduciary aspects of the Project at preparation were well addressed, the Task Team benefited from good support from the country office and regional fiduciary staff. The risks were well captured, but underestimated the slow parliamentary process of law adoption, as well as the level of resistance to change. Finally, the Results Framework proposed two PDO indicators that were - 14 - remotely attributable to the Project (decrease in price of Internet, and increase in percentage of public services using electronic transactions), which contributed to the unclear perception of e-Benin being a public service transformation project, whereas it was primarily laying the foundation to enable such transformation. 53. A more realistic appraisal of AGeNTIC’s capacity, and recognition in the Risk Framework that the agency’s physical location (in an exclusive part of town) was likely to deter a portion of its stakeholders to visit the premises and benefit from the services and access to technology, would have helped to concentrate more attention on this subcomponent from early in implementation. (b) Quality of Supervision Rating: Moderately Satisfactory 54. The Bank Task Team ensured moderately satisfactory supervision of the Project with biannual missions that allowed comprehensive monitoring of project implementation, with high-quality aide-memoires and timely ISRs. After the MTR these were complemented by regular conference calls with a focus on the e-Business subcomponent and on the e- Government pilot projects. On the fiduciary level, country-based procurement and financial management specialists provided excellent support with timely and substantive support to the PIU and implementing agencies. Overall, the Task Team performed with professionalism and with good support from management. 55. The Task Team overcame the challenges inherent to project implementation albeit with some time lap. For instance, the Level 2 restructuring of the Project that occurred after MTR allowed reallocation of disbursement categories needed to insure optimal use of funds while remaining compliant with the legal agreement. It also adjusted the Results Framework to enhance the attribution of Project outcomes, but dropped indicators that are today mandatory in IPF results framework (project beneficiaries). When activities or subcomponents experienced delays or difficulties, the Task Team channeled more attention and, to the extent possible, additional resources to overcome them. For instance, La Poste and AGeNTIC benefited from having help from semi-residential strategic advisors to accelerate implementation of activities. For the e-Government services pilot projects, a specialist (consultant) in innovation and government transformation was hired to provide support in the last semester of the Project. The resistance to change was the main break that prevented the pilots from becoming operational A clearer path for sustainability could have been discussed with the PIU under the WARCIP Benin project. (c) Justification of Rating for Overall Bank Performance Rating: Moderately Satisfactory 56. Based on the justification herein, the overall Bank performance is rated Moderately Satisfactory. 5.2 Borrower Performance (a) Government Performance Rating: Moderately Satisfactory - 15 - 57. The Project funds were fully disbursed and completed in time. At the highest level there was strong endorsement of the project that however varied with ministries (high turnover during project life). The passing of the new law was in itself a significant milestone towards which the Minister worked really hard to achieve. 58. DGCEP was the operational arm of the Project for the MPTIC. While it was effective in coordinating the work on the legal and regulatory framework, it suffered some weaknesses including the high turnover of leadership (reflecting the frequent changes of ministers) and human resources that were insufficient in numbers and skills. For a structure supposed to pull together the rest of the ministries to adopt a new enterprise architecture and e-Government framework, it lacked the right mix of experience and skills, and was short of ICT engineers. It recognized it and called for a new governance framework of the e-Government program in which it would primarily focus on its core strength which is planning and setting up financing mechanisms for connectivity infrastructure. DGCEP was not proactive enough in leading the pilot projects on e-Government projects, but claimed that the newly ABETIC should have been clearly designated for this task. (b) Implementing Agency or Agencies Performance Rating: Moderately Satisfactory 59. The agencies’ commitment to achieving development objectives was strong throughout the Project, although some of their staff lacks capacity and/or the right skills-set and experience. La Poste performed largely above expectation under a dedicated Director General, a Board President who was deeply involved in turning-around the enterprise, and staff who followed the leadership of their newly appointed Director General. AGeNTIC has a mixed record of accomplishment: slow at project start, difficult relation with the PIU with regard to fiduciary activities, a strong General Director but maybe not the right profile for an incubation-type organization. AGeNTIC managed to catch-up on disbursements and implementation, both for the incubation and the matching grant mechanisms 60. The PIU performed extremely well, as reported in all aide-memoires. The team had a strong cohesion, was able to provide timely support to implementing agencies while responding to the requests of the Bank Task Team. By MTR, a sector specialist was hired to help the PIU strengthen its understanding of the sectoral issues. The PIU was in charge of both projects; and it built a strong understanding of the sectoral issues and at times could inform the Bank Task Team (in HQ), about the political economy in the sector. Therefore in addition to its strong handling of the traditional fiduciary dimensions of project implementation, the PIU helped stakeholders and the World Bank to resolve issues pertaining to project implementation, making it smoother and more efficient. (c) Justification of Rating for Overall Borrower Performance Rating: Moderately Satisfactory 61. Based on the ratings above, the overall Borrower’s rating is Moderately Satisfactory. - 16 - 6. Lessons Learned a. Government transformation from paper-based to digital represents a massive undertaking in resources and time. Such endeavor need to be championed over many years, empowered with sufficient resources and the authority to defeat resistance to change. b. Three drivers for a successful e-Government program are: leadership, governance and skills. Placing the leadership of the e-Government program in the Ministry in charge of ICT may not be the best governance arrangement to insure that changes are spread across administrations; the Prime Minister or President’s level is more appropriate for that purpose. The number of skilled employees (who can enable the transformation) and their location throughout the governmental agencies (central and local) constitutes a key success factor, and should be considered as preliminary stage to any e-Government transformation program. c. Resistance to change can derail an administration modernization program that embarks in e- Government. Even when the decision-making levels are championing e-Government, the change has to happen at every level of the administration organizations. Individuals can sense that they would “lose” in a process of modernization (which brings better governance) and therefore passively resist it. Mitigation measures need to be adequately included in project design, and adequate resources be accounted. d. International experience shows the importance of incubation-type of entities to boost entrepreneurship and innovation in countries. Successful models demonstrate that State subsidies are needed for an extended period of time; in that regard, the experience of Benin is in line with international dominant model. Successful models also show the benefits of flexible business model with developers and marketing experts mix their experience to invent and reinvent constantly based on the new technologies available. In that regard, AGeNTIC management team didn’t have the right skill mix. e. Efforts to closely link capacity building activities with the studies brought a greater coherence to the Project. For example, as part of the study for e-Government (enterprise architecture, data center, standards and standard) three training sessions (on enterprise architecture, interoperability framework, and data center) were organized by the same consulting firm: This training raised awareness on the reforms to come and prepared stakeholders to changes. By the same token, since the study didn’t effectively lead to the effective implementation of the program, one can argue that the capacity built has not borne fruit. f. Learning from practitioners in other countries (through field trips and follow-up exchanges of information) contributed to strengthen capacity and to the Project’s implementation. For example, technical information shared on the management and information systems of La Poste of Côte d’Ivoire helped La Poste (of Benin) to prepare technical specifications for their own system. g. On the importance of the Project’s name, ‘e-Benin,’ raised the expectation of tangible changes in the public and private sectors. A more fitting name would have been, for instance, ‘Enabling Public Sector Modernization in Benin. - 17 - 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners (a) Borrower/implementing agencies N/A (b) Cofinanciers (c) Other partners and stakeholders (e.g. NGOs/private sector/civil society) N/A - 18 - Project Cost and Financing (a) Project Cost by Component (in $ million equivalent) Appraisal Estimate Actual/Latest Percentage of Components ($ millions) Estimate ($ millions) Appraisal Component 1 5.6 4.6 81.2 Component 2 7.0 7.0 100.0 Component 3 1.52 3.2 213.0 Total Project Costs 14.2 Unallocated 0.8 Total Financing Required 15.00 14.9 99.1 (b) Financing Appraisal Actual/Latest Type of Percentage of Source of Funds Estimate Estimate Cofinancing Appraisal (%) ($ millions) ($ millions) Borrower 0.00 0.00 .00 IDA Grant 15.00 0.00 .00 Category of Expenditure Allocation Percentage of Financing Current Current Revised Current Revised (1) Goods, consultants’ services, 6,800,000 8,350,000 70.1 86.1 training and operating costs for Parts A, B1, and C (2) Goods, consultants’ services, 900,000 1,050,000 9.3 10.8 training and operating costs for Parts A, B1, and B2 (3) Goods and consultants’ 400,000 300,000 4.1 3.1 services financed by matching grants under Part B2 (B) (4) Refund of preparation advance 1,050,000 0 10.8 0 Unallocated 550,000 0 5.7 0 Total amount 9,700,000 9,700,000 - 19 - Outputs by Component 1) The Project delivered most of the expected outputs. The following table includes the activities carried out during the project per component linked to the output attained: Components/ Subcomponents Output Outcome Component 1: Creation of an enabling environment for improved access to ICT services (a) Strengthening the legal, Support to develop secondary New law (July 2014) and draft secondary regulatory and institutional legislation (decrees of legislation framework applications) Support to develop key New strategies/instrument in place and regulatory strategies and enforced: instruments. - information security and cybercrime strategic framework - Center of excellence to train civil servants in e-Government services Support to the creation of ARCEP created and operational ARCEP-Benin and capacity building Government portal set-up Enhanced access to government information for citizens Audit, recommendation and La Poste’s strengthened management and acquisition of an up-to-date marketing capacity management information system (La Poste) Corporate strategy and semi- La Poste increased revenues and breaks even residential international advisor in CY2014 (La Poste) (b) Provision of affordable Legal studies to create the special SPV created and increased capacity can flow high-speed Internet purpose vehicle (SPV) in charge in Benin connectivity and universal of managing international service connectivity (preparation to WARCIP Benin) Universal service strategy Increased access to telephone and Internet Broadband strategy Action plan to increase infrastructure access and investment by Eximbank of China Support to the creation of ABSU- ABSU-CEP created and operational CEP (universal service) Feasibility study of an IXP IXP planned with a view to lower Internet - 20 - prices Component 2: Enabling of e-applications and promotion of e-Business (a) Enabling e-Government Technical assistance to finalize E-government program updated Applications the Politique National sur le Gouvernement Electronique Setting up the Delegation ABETIC created and operational Nationale des TIC/ABETIC Enterprise architecture, norms Increased readiness to roll out the new and standards for interoperability, enterprise architecture, foundation to the e- feasibility of a datacenter Government program e-Government pilot projects Beta version delivered but pilot not rolled out designed (b) Facilitating the uptake of Empower AGeNTIC through AGeNTIC was empowered by the project; eBusiness technical and capacity-building over 800 people received training. It now needs to seek a new positioning vis-à-vis ABETIC. Launch of an incubation 24 entrepreneurs benefited from technical and program under AGeNTIC consulting services Launch of a matching grant 16 companies benefit from the matching grant program managed by AGeNTIC 2) e-Business and AGeNTIC At project design phase, the WB had limited experience with supporting incubating entities. In Benin, there was a strong demand from the Government, as well as a pre-existing dedicated agency, and its managing director was a recognized strong professional. It was a para-public entity and resources provided by the e-Benin project were to constitute the majority of its operational budget. AGeNTIC concentrated on the local market and didn’t sign any strategic partnership with the private sector in Benin or neighboring countries. At implementation, this sub-component experienced several delays which make it hard to assess, at project closure, the performance and sustainability. What is tangible is that, both on entrepreneurship support and matching grant, most of the resources were appropriately disbursed. The task team met in several occasions the “clients” of AGeNTIC, and gathered that some impulse was effectively given to innovation and youth entrepreneurship. Whether this impulse is sustainable is difficult to assess: 10 months after project closing, AGeNTIC repositioned itself as a consulting body with a specialty in “maîtrise d’ouvrage déléguée” for large ICT projects. It is uncertain whether it relies primarily on the incubated SMEs it once supported, but it certainly maintains a repertory of ICT SMEs in Benin which it can pull together in consortium of bid on contracts. - 21 - Trainings were provided in partnership with local or international partners (Cisco, Microsoft) and tacked a large variety of ICT-related topic, from basic skills acquisition to more sophisticated software development or business plan topics. AGeNTIC also offered work space and free access to broadband for a local Django community of developers. 3) e-Government pilot projects In component 2, the e-Benin project financed 3 pilots pertaining to online government services (e-gov) : education, visa, civil registry (‘état civil’). The budget allocated to this activity was 100 million F CFA and the targeted population around 30,000 (see table below). These pilots were identified based on the e-gov study carried out by TTI and focused on the pre- requisite for the e-government (enterprise architecture, norms and standards, datacenter), and followed the identification of the public administrations that were the most interested in implementing these pilots: education (schools in Abomey Calavi, Djougou, Bassila), local governments (towns of Abomey Calavi, Bassila and Savalou), and the visa administration. Overall, these pilots, which relies on public-private partnerships involving Moov, MTN, and computer companies, faced implementation challenges, particularly regarding change management and the lack of buy-in from staff on the ground: the review of processes, the transparency allowed by ICT (vs. informal interactions between citizens and civil servants), and the modification of financial flows, notably the ones regarding civil registration, explain the reluctance of local staff. To compensate, the highest authorities should reiterate that the need for simplification of citizen-state interactions and the improvement of governance, constitute priority objectives for the Government. The underlying economic models must be better explained to stakeholders, and mobile payments (m-payments), a key element of "simplification", must be generalized to ensure citizens’ buy-in.  Civil Registry Designed to improve the accessibility of civil registration services to citizens, the civil registry application aims to first set up an e-services platform enabling citizens to make their demand for official documents online, and second to gradually establish a database of vital records of citizens. Status: The pilot has been implemented in three towns namely Abomey Calavi, Bassila and Savalou. Issues: - Mobile payment contract between all stakeholders - Implementation and extension at national level (equipment and involvement of new centers for civil registry, a total of 546 contracts with the platform operator for the license and training) - Fees amount, administrative costs - Not implemented with MTN Situation at project closing: The owner of the solution was contacted to extend the pilot to central districts of 4 additional cities (Cotonou, Parakou, Porto-Novo, Ze and Bohicon) by the e- gov Project of the DGCEP at MPTIC. - 22 -  e-education The application on school statistics was to set up a unique database (registry) of students of primary, secondary and higher education in Benin, and to have educational statistics in "real time" on the Beninese education system. Several other features are available to teachers, parents, administration, etc. Status: The application is deployed in three towns (Abomey Calavi, Djougou, and Bassila) and 9 schools, of which 2 private schools. Issues: - Mobile payment contract between all stakeholders - Implementation and extension at national level (equipment and enrollment of new schools, and contract with the platform operator for the license and training) estimated at about 10 000 US dollars per school - Not implemented with MTN Situation at project closing: The owner of the solution was contacted by the e-gov Project at DGCEP, to extend the pilot to 4 additional schools.  e-visa The e-visa application aims first to set up an e-service platform that allows visitors to submit and track their visa applications, and secondly to gradually build a database for visa applications. Status: Application is completed technically but lack of contact has not been deployed. Issues: The ‘Direction de l’émigration et l’immigration’ is responsible for the implementation, the organizational change management and training, the development of electronic payment, has been little involved. Delivery of visas is currently operated by a service provider for which the e- Visa project is perceived as a competitor; this undoubtedly negatively affected the ownership of the Directorate. Table: Budgets for e-government pilots Initial amount allocated to pilots 100 000 000 F CFA Final amount financed by the e-Benin project (excluding the TTI contract) 82 859 800 F CFA Amounts disaggregated by pilot for equipment and subscription Civil Registry 21 000 000 F CFA Education 16 000 000 F CFA Visa 10 000 000 F CFA IT equipment 26 490 000 F CFA Internet connectivity 9 369 800 F CFA Total 82 859 800 F CFA - 23 - - 24 - Annex 3. Economic and Financial Analysis 1. The PAD acknowledged that due to the technical assistance nature of this Project, calculation of a financial rate of return and economic rate of return is not applicable. This is all the more accurate since this Project was mostly laying the foundations of the e-Government framework, with limited actual on-line services piloted (and none mainstreamed.) 2. Most e-Government projects refer to this qualitative analysis of cost and benefits of transforming an administration from a paper-based organization to a partial or fully-digitalized one, for the government and for the end-user (source: European Commission, Study on e- Government and the reduction of administration burden, 2014). - 25 - Annex 4. Bank Lending and Implementation Support/Supervision Processes (a) Task Team members Names Title Unit Lending Hugues Agossou Senior Financial Management Specialist Mavis Ampah Lead ICT Policy Specialist GTI11 Jerome Bezzina Senior Regulatory Economist GTI11 Daniel Kwabena Boakye Economist GMF01 Yann Burtin Senior Underwriter MIGEA Doyle Gallegos Lead ICT Policy Specialist GTI11 Boutheina Guermazi Practice Manager GTI11 Isabelle Huynh Senior Operational Officer GTI11 Kaoru Kimura ICT Policy Specialist GTI11 Marc Jean Yves Lixi Senior Operations Officer GTI11 Pepita Hortense Olympio Program Assistant AFMBJ Michele Ralisoa Noro Operations Analyst GTI11 Sandra Sargent Senior Operations Officer GTI09 Supervision Mavis Ampah Lead ICT Policy Specialist GTI11 Jerome Bezzina Senior Regulatory Economist GTI11 Daniel Kwabena Boakye Economist GMF01 Yann Burtin Senior Underwriter MIGEA Jacqueline Dubow Consultant GTI11 Doyle Gallegos Lead ICT Policy Specialist GTI11 Elena Gasol Ramos Senior ICT Policy Specialist GTI09 Boutheina Guermazi Practice Manager GTI11 Isabelle Huynh Senior Operational Officer GTI11 Kaoru Kimura ICT Policy Specialist GTI11 Marc Jean Yves Lixi Senior Operations Officer GTI11 Pepita Hortense Olympio Program Assistant AFMBJ Enapha P.E. Sandrine Ouensou Consultant GTIDR Michele Ralisoa Noro Operations Analyst GTI11 Sandra Sargent Senior Operations Officer GTI09 (b) Staff Time and Cost Staff Time and Cost (Bank Budget Only) Stage of Project Cycle $ Thousands (including travel No. of staff weeks and consultant costs) Lending Total: 33.13 209.5 Supervision/ICR 79.3 515.1 Total: 112.43 724.6 - 26 - Annex 5. Beneficiary Survey Results (June 2015) The satisfaction survey of beneficiaries of the e-Benin project could not be a standard large-scale survey of the Benin population. Indeed, the direct beneficiaries of the project are the staff of MCTIC, especially those working in structures created or supported by the project, and those involved in the studies or targeted by training sessions. Companies supported by the AGeNTIC are also direct beneficiaries of the e-Benin project. But even though the Benin citizens cannot be considered a primary target of a project mainly focused on feasibility studies and capacity building for administrative staff project management, it is useful or strategic, to probe the opinion and perception of some of Benin citizens on the project. Thus, a sample of 60 people was surveyed. This group consists of 40 agents from MCTIC services targeted by the project, 10 member companies supported by the AGeNTIC, and 20 random people in the cities of Cotonou Calavi and in cyber cafés in Cotonou. Given the small size of the sample it is not possible to make reliable extrapolations on Benin's population. However, the survey can be used to triangulate the information from interviews, and gives interesting insights into the views of Beninese, information that could be used for more accurate future investigations. In a range of possible questions, the survey focused on the relevance of the project concept, the usefulness of the project, the level of satisfaction following the services rendered, the connectivity quality, the cost of connectivity, and finally the strengths and weaknesses of e- Benin. - 27 - Relevance of the project concept Graph #1: Relevance of the project concept Yes, [VALUE] No, [VALUE] Among those surveyed, 93% think that the idea of e-Benin project is relevant, against 85% during the mid-term evaluation. Those not seeing the relevance of the project include citizens who do not understand the project and some customers of the AGeNTIC’s incubator who are not sure they can fly on their own after the incubation period is over. But mostly, the concept of e- Benin was welcomed, especially within the Ministry, where there are still some people questioning the rationale for allocating the bulk of project resources to studies when the Government cannot afford to implement the studies recommendations, and when external financing is not certain. Usefulness of e-Benin project Overall, and in the same proportions as at the mid-term evaluation, the sampled group estimated that the project is helpful. Among commonly cited reasons are: - The launch of major projects in a sector (ICT) usually neglected by the authorities; - Capacity building in a domain where there are not enough specialists; - The expected contribution of the project to lower Internet access costs; - The ability to boost youth employment with the e-business component. For the majority of citizens of Benin, the usefulness of the project should be considered from the perspective of ICT access in remote villages. Therefore, they wondered about Internet access in non-electrified villages. Some of them proposed to couple the e-Benin project with a rural electrification large-scale program. - 28 - Satisfaction survey Graphic #2: Satisfaction rate among people trained by the project MU, [PERCENT MS, 20% AGE] S, [VALUE], 69% (against 78% at mid-term) of trained respondents are satisfied with the training sessions. To substantiate their responses, they provide two main arguments: - The diversity and originality of the training topics whether sessions was organized by the PCU or those organized by the AGeNTIC; - The very small number of participants per session which promotes a good level of acquisition of the themes. People who are not fully satisfied mentioned: - Some sessions were too short; - The lack of follow-up (they were not promised any career opportunities after). The slight decrease in the rate of satisfaction since the mid-term evaluation is explained by the fact that people already trained still hoped for other training sessions through study tours with per diem. - 29 - Quality of the Internet connection in Benin Graphic #3: Quality of the Internet connection in Benin Good, [VALUE] Average, Bad, 60% [VALUE] From the mid-term review to the ICR, the perception of the quality of the connection remained identical. Only 15% of 60 recipients interviewed believed that the quality of the connection was good. It is challenging to analyze this result against one of e-Benin objectives: to increase quality of Internet access. The project activities could contribute to achieving this objective, but cannot affect it directly. Indeed, e-Benin does not fund infrastructure installation for universal access. On the other hand the WARCIP project which followed e-Benin and implemented by the same PCU, included the landing of the second submarine cable and thus could create redundancy, increase internet speed and so the quality of the connection. - 30 - Cost of Internet connectivity in Benin Graphic #4: Cost of Internet connection in Benin [CATEGORY NAME], [VALUE] High, [VALUE] 65% of respondents (against 90% at mid-term) estimated that the cost of internet access was high. But respondents noted a slight decline in costs. Citizens cannot explain the causes of the decline and vaguely attributed it to e-Benin. In reality, it is rather the announcement of the arrival of the ACE cable that forced BTSA to lower its costs, encouraging private operators to get international capacity from the national operator instead of Nigeria, hence lowering the costs. Strengths and Weaknesses of e-Benin project: As positive forces, respondents highlighted: - Targeting both the state administration and the private sector - The large scope of the project - The successful negotiations of the WARCIP project - The small project team, who could work without internal conflicts - The strengthening of civil servants capacity - The flexibility of the e-Benin Project - The government's willingness to ensure project success. As weaknesses, respondents emphasized primarily: - The lack of funding for infrastructure - The sophisticated character of the studies that could not be understood by everyone - Poor understanding of the project by Benin population - 31 - Table 1 : swot analysis SWOT ANALYSIS OF THE E-BENIN PROJECT STRENGTHS WEAKNESSES -Stakeholder buy-in -Delays in e-Business component -Dynamic and supportive project team -Delays in some responses from CAA -Good pace of implementation of tasks -Low flow of information between the -Reports generally produced in time difference services of MCTIC -Administrative, financial and accounting -Late start of the pilot projects procedures respected -Synergies study - training OPPORTUNITIES THREATS - Consistent political will to promote ICT -Mobility of MCTIC staff -Approval of the Warcip project, which will -Turnover within the PCU implement some of the emergencies -Regular change of Ministers identified by e-Benin -Interest fthe project shown by other donors (ADB, KfW, European Union, etc.) One of the main weaknesses of the project remains the late launch of the pilot project. - 32 - Annex 6. Summary of Borrower's ICR and/or Comments on Draft ICR The objective of e-Benin Project is to improve access to the Information and Communication technologies (ICT) and to promote the development of e-applications in the country. The project was implemented by an independent Coordination Unit under the Ministry in charge of Communication and ICT. The project played a decisive role in strengthening the institutional, legal and regulatory ICT and telecommunications sector in Benin. He induced the commitment of parliamentarians to vote the law on electronic communication and post and supported the development of application drafts of the Law on electronic communications and postal and development fronts specific draft laws on the Information Society (laws on cybercrime, electronic signatures and certification services, cryptology, electronic transactions, secure electronic transactions). The passage of the law allowed the creation of ARCEP, the ABETIC and ABSU-CEP. The project has conducted important studies in the field of broadband connectivity and the promotion of e-Government. This helped among others, to provide the country with a broadband strategy, a strategy and action plan for universal access to ICTs, a general framework for interoperability and a national data center design. Support was provided to “La Poste du Bénin” in defining its development strategy and the audit of its information and management system. This helped to boost the Post sector facing many difficulties. The feasibility study of an Internet exchange point (IXP) has been finalized and this exchange point, which was not expected was installed. Further to the study on e-Government, three pilot projects have been implemented. These are e-education projects, e-Visa and e-Civil-Registry project. The e-Business component of the project supported ICT skills and create new business opportunities for local companies operating in the ICT field. Project support for capacity building is appreciated as a total of 255 people MCTIC were trained in 40 modules in 7 different countries. The project is relevant as the degree of achievement of results. All planned activities have been carried out excepted e-Government applications whose late implementation failed to have the expected statistics. The physical implementation rate is 99.53. The review of the implementation of the indicators is generally satisfactory. Two indicators were not able to be properly informed while they are on a positive trend. The first indicator relating to the provision of financial statements certified for “La Poste du Bénin” and the indicator on the level of use of pilot projects applications. The project efficiency was assessed in connection with the implementation capacity but also resource mobilization and the proper use of these resources. Most project expenditures were incurred, indicating a readiness to carry out the activities and willingness to complete these activities. Note here the dynamism and diligence of financial services: to avoid delays, expenses are incurred without waiting for the availability of funds to pay them. 106 refunds of funds (DRF) were sent to the Bank and since its start, the project has no cash flow problem. During one of its mission the World Bank estimated that "the agreed action plans are respected, the targets will be achieved." Indeed, the financial implementation rate is 99.04%. That is a relevant indicator of efficiency when it binds to the physical implementation rate of 99.53%. - 33 - As for the impact of the project, we note the beginnings of behavior change among focal unit’s members that have gradually acquired the reflexes of diligent management, orderly and efficient while some of them had never previously written ToR for studies. One of the most visible effects of the project is the development of new institutions in new or renovated buildings, equipped and functional (ARCEP, ABETIC, ABSU-CEP). For example, the ABSU-CEP, benefit contributions paid by operators for universal access. Another effect is already noticeable recovery of the Post of Benin. While she was in an alarming state of indebtedness, the Post benefited from the recovery plan built thanks the study funded by e-Benin to gradually regain some financial health. He recently won a prize in Paris as a reward to its recovery efforts. The results of the project include many situations that favored sustainability. For example, the institutional anchoring of the project within the Ministry in charge of ICT. This develop skills in performance and diligence for public agents. So they have opportunity to reinvest their knowledge in the same project or other projects. The project hold several training sessions with topics related or connected to ICT. This for the benefit of the ministry agents. The involvement of focal points and agents of the various services of MCTIC in the preparation, conduct and validate studies. Creating public structures (ABETIC, ABSU-CEP), which already took over in the promotion of ICT / telecoms in the country. Some weaknesses are noted: delays in e-Business component, low flow of information between the services of MCTIC, late start of pilot projects, mobility of MCTIC staff, regular change of Ministers. Several lessons learned: - The personal commitment is crucial in achieving results. - Realism as a condition of success in project design - Anticipation as good project management brand - The involvement of all relevant institutions as a guarantee of success - Supervision of public-private partnership as a condition to produce conclusive results This document features a critical mass of information to guide future actions in the area of ICT and electronic communications in Benin. - 34 - Annex 7. List of Supporting Documents 1. Aide Memoires, Back-to-Office Reports, and Implementation Status Reports. 2. Project Appraisal Document 3. Loan Agreement and the Amendment 4. Restructuring Report Report No: 88493 5. Midterm Review, October 2013 6. Rapport Financier interimaire Oct. 30, 2015 7. Studies - 35 - - 36 -