Document of The World Bank FOR OFFICIAL USE ONLY Report No: IMPLEMENTATION COMPLETION AND RESULTS REPORT ON A MULTI-DONOR TRUST FUND WITH THE GOVERNMENT OF CANADA AS SOLE DONOR IN THE AMOUNT OF CDN 19.2 MILLION (US$ 19.5 MILLION EQUIVALENT) TO TWELVE CARIBBEAN COUNTRIES (ANTIGUA AND BARBUDA, BARBADOS, BELIZE, DOMINICA, GRENADA, GUYANA, JAMAICA, ST. KITTS AND NEVIS, ST. LUCIA, ST. VINCENT AND THE GRENADINES, SURINAME, AND TRINIDAD AND TOBAGO) FOR A/THE SUPPORTING ECONOMIC MANAGEMENT IN THE CARIBBEAN PROGRAM December 8, 2017 Governance Global Practice Latin America and Caribbean Region CURRENCY EQUIVALENTS (Exchange Rates Effective December 8, 2017) Canadian Dollar Currency Unit = (CDN) CDN 1.29 = US$ 1.00 US$ 0.77 = CDN 1.00 FISCAL YEAR Antigua and Barbuda: 1 April - 31 March Barbados: 1 April - 31 March Belize: 1 April - 31 March Dominica: 1 July - 30 June Grenada: calendar year Guyana: calendar year Jamaica: 1 April - 31 March St Kitts and Nevis: calendar year St Lucia: 1 April - 31 March St Vincent and the Grenadines: calendar year Suriname: calendar year Trinidad and Tobago: 1 October - 30 September Regional Vice President: Jorge Familiar Calderon Country Director: Tahseen Sayed Khan Senior Global Practice Director: Deborah L. Wetzel Practice Director: James A. Brumby Practice Manager: Arturo Herrera Gutierrez Task Team Leader(s): Ruxandra Burdescu ICR Main Contributors: Eduardo Estrada/May Olalia ABBREVIATIONS AND ACRONYMS AGD Accountant General’s Department ASYCUDA Automated System for Customs Data CARIBTAX Caribbean Business Taxation Reform Project CARTAC Caribbean Regional Technical Assistance Centre CATT Customs Assessment Trade Toolkit CDN Canadian Dollar CIDA Canadian International Development Agency (replaced by DFATD) COFOG Classification of the Functions of Government COUSINS Comprehensive Underlying Systems for SEMCAR Countries Software CPF Country Partnership Framework CPS Country Partnership Strategy DB Doing Business DFATD Department of Foreign Affairs, Trade and Development (replaced by GAC) ECCB Eastern Caribbean Central Bank EGRIP Electronic Government for Regional Integration Project GAC Global Affairs Canada GDP Gross Domestic Product GFS Government Finance Statistics GFSM2001 Government Finance Statistics Manual 2001 GRM Grant Reporting and Monitoring (GRM) ICR Implementation Completion and Results Report ICT Information and Communication Technology IFC International Finance Corporation IFMIS Integrated Financial Management Information System IMF International Monetary Fund IPSAS International Public Sector Accounting Standards IRD Inland Revenue Department ISR Implementation Status Report JAMTAX Jamaica Business Taxation Reform Project MDTF Multi-Donor Trust Fund MTR Mid-Term Review M&E Monitoring and Evaluation OECS Organisation of Eastern Caribbean States PDO Project Development Objective PEFA Public Expenditure and Financial Accountability PFM Public Financial Management PMF Performance Measurement Framework PWC PricewaterhouseCoopers RETF Recipient-Executed Trust Fund RFP Request for Proposal RMF Risk Management Framework RPS Regional Partnership Strategy SC Steering Committee SEMCAR Supporting Economic Management in the Caribbean SIGTAS Standard Integrated Government Tax Administration System SMF Strategic Management Framework SOE State-Owned Enterprise TA Technical Assistance TF Trust Fund UNCTAD United Nations Conference on Trade and Development US$ United States Dollar UWI University of the West Indies UWIOC University of the West Indies – Open Campus VAT Value Added Tax WB World Bank WBG World Bank Group WDI World Development Indicators TABLE OF CONTENTS DATA SHEET .......................................................................................................................... 6 I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES ..................................................... 11 A. CONTEXT AT APPRAISAL ....................................................................................................... 11 B. SIGNIFICANT CHANGES DURING IMPLEMENTATION (IF APPLICABLE) ..................................... 17 II. OUTCOME .................................................................................................................... 20 A. RELEVANCE OF PDOs ............................................................................................................ 20 B. ACHIEVEMENT OF PDOs (EFFICACY) ...................................................................................... 22 C. EFFICIENCY ........................................................................................................................... 29 D. JUSTIFICATION OF OVERALL OUTCOME RATING .................................................................... 30 E. OTHER OUTCOMES AND IMPACTS (IF ANY) ............................................................................ 30 III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME ................................ 32 A. KEY FACTORS DURING PREPARATION ................................................................................... 32 B. KEY FACTORS DURING IMPLEMENTATION ............................................................................. 32 IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME .. 35 A. QUALITY OF MONITORING AND EVALUATION (M&E) ............................................................ 35 B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE ..................................................... 37 C. BANK PERFORMANCE ........................................................................................................... 37 D. RISK TO DEVELOPMENT OUTCOME ....................................................................................... 38 V. LESSONS AND RECOMMENDATIONS ............................................................................. 39 ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS ........................................................... 41 ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION ......................... 59 ANNEX 3. PROGRAM COST BY COMPONENT ........................................................................ 63 ANNEX 4. EFFICIENCY ANALYSIS ........................................................................................... 66 ANNEX 5. BORROWER, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS ... 67 ANNEX 6. SUPPORTING DOCUMENTS (IF ANY) ..................................................................... 68 ANNEX 7. DETAILED DISCUSSION ON ACHIEVED RESULTS (BEYOND THE INDICATORS) .......... 69 ANNEX 8. COMPARISON OF PROPOSED SEMCAR ACTIVITIES AND DELIVERED ...................... 76 ANNEX 9. STRUCTURE OF SEMCAR TRUST FUND AND DIVISION OF LABOR WITH THE IMF .... 80 ANNEX 10. SUMMARY OF RATINGS ..................................................................................... 82 The World Bank Caribbean - Supporting Economic Management in the Caribbean Program (P123665) DATA SHEET BASIC INFORMATION Product Information Project ID Project Name P123665 (Global Programs and Partnerships) MDTF for the Support for Economic Management in the Caribbean Region Phase 1 (P123665) Country Financing Instrument Caribbean Trust Fund (Trustee TF071579) Original EA Category Revised EA Category Not Required (C) Not required (C) Note: See Annex 9 for the full list of trust funds corresponding to each project ID. Project ID Project Name P128874 (Technical assistance) Programmatic TA for the Support for Economic Management in the Caribbean Region Phase 1 (P128874) Country Financing Instrument Caribbean Trust Fund Original EA Category Revised EA Category Not Required (C) Not required (C) Project ID Project Name P131197 (Recipient-Executed) SEMCAR SmartStream Enterprise Agreement Negotiation (P131197) Country Financing Instrument Caribbean Trust Fund Original EA Category Revised EA Category Not Required (C) Not required (C) Page 6 of 82 The World Bank Caribbean - Supporting Economic Management in the Caribbean Program (P123665) Project ID Project Name P158239 (Recipient-Executed) SEMCAR Budget and Revenue Systems Enhancement Project (P158239) Country Financing Instrument Caribbean Trust Fund Original EA Category Revised EA Category Not Required (C) Not required (C) Organizations Beneficiary Implementing Agency The World Bank International Monetary Fund Caribbean Government of Antigua and Barbuda Government of St. Kitts and Nevis Government of St. Lucia Project Development Objective (PDO) Original PDO The higher level strategic objective is to improve economic management, regional integration and competitiveness in up to 12 Caribbean countries, through more efficient, effective and accountable public institutions, policies, processes and Information and Communication Technology (ICT) systems. Within this strategic context, the Program objective is to contribute to more efficient, effective, accountable, and regionally integrated Tax, Customs and Public Financial Management (PFM) institutions, policies, processes and ICT systems. This will be achieved through the provision of policy advice, technical assistance and selected ICT investments. Page 7 of 82 The World Bank Caribbean - Supporting Economic Management in the Caribbean Program (P123665) FINANCING Original Amount (US$) Revised Amount (US$) Actual Disbursed (US$) World Bank Financing TF 18,239,000 19,483,875 17,067,619 Total 18,239,000 19,483,875 17,067,619 Non-World Bank Financing Borrower 0 0 0 Total 0 0 0 Total Project Cost 18,239,000 19,483,875 17,067,619 Note: Includes financing for World Bank, International Monetary Fund, and recipient-executed activities. Difference between original amount and revised amount due to foreign exchange gains. KEY DATES Approval Effectiveness MTR Review Original Closing Actual Closing 03-Sep-2010 02-Feb-2011 12-Jul-2014 31-Dec-2013 31-Aug-2017 RESTRUCTURING AND/OR ADDITIONAL FINANCING Date(s) Key Revisions 27 September 2013 Closing date first extension of Trustee Account to Feb. 2015 11 July 2014 Closing date second extension of Trustee Account to Dec. 2015 26 October 2015 Closing date third extension of Trustee Account to Dec. 2016 9 August 2016 Closing date fourth extension of Trustee Account to Aug. 2017 KEY RATINGS Outcome Bank Performance M&E Quality Moderately Satisfactory Moderately Satisfactory Substantial Page 8 of 82 The World Bank Caribbean - Supporting Economic Management in the Caribbean Program (P123665) SECTORS AND THEMES Sectors Major Sector/Sector (%) Public Administration 64 Other Public Administration 64 Information and Communications Technologies 36 Public Administration - Information and 9 Communications Technologies ICT Services 18 Other Information and Communications Technologies 9 Themes Major Theme/ Theme (Level 2)/ Theme (Level 3) (%) Economic Policy 25 Fiscal Policy 25 Tax Policy 25 Public Sector Management 75 Public Finance Management 50 Public Expenditure Management 25 Domestic Revenue Mobilization 25 Public Administration 25 Transparency, Accountability and Good 25 Governance ADM STAFF Role At Approval At ICR Regional Vice President: Pamela Cox Jorge Familiar Calderon Country Director: Yvonne M. Tsikata Tahseen Sayed Khan Page 9 of 82 The World Bank Caribbean - Supporting Economic Management in the Caribbean Program (P123665) Senior Global Practice Director: Deborah L. Wetzel Practice Director: James A. Brumby Practice Manager: Arturo Herrera Gutierrez Arturo Herrera Gutierrez Task Team Leader(s): Kathy Lalazarian Ruxandra Burdescu ICR Contributing Author: Eduardo Estrada/May Olalia Page 10 of 82 The World Bank Caribbean - Supporting Economic Management in the Caribbean Program (P123665) I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES A. CONTEXT AT APPRAISAL Context Regional Context 1. The impact of the 2007-2008 global financial crisis severely affected the Caribbean region. The global financial crisis exacerbated economic problems in the region by increasing fiscal imbalances and already elevated public sector debt (see Table 1). Moreover, World Bank Working Paper No. 7544 notes that the impact of the crisis on the Caribbean region vis-à-vis the United States, which was at the center of the crisis, indicates that the region’s economy contracted more and exhibited no signs of recovery by 2010. 1 As such, at the time of approval of the Supporting Economic Management in the Caribbean (SEMCAR)2 Program in July 2010, the key challenge for economic management in the Caribbean was how to reinvigorate and sustain growth alongside reducing high unemployment and poverty rates, restoring fiscal and debt sustainability, diversifying the economy, and securing a more sustainable external position. Table 1. Selected Economic Indicators of SEMCAR Countries Countries GDP Growth Rate (%) Central Government Debt as % of Trade as % of GDP GDP3 2005- 2008- 2010 2016 2005- 2008- 2010 2014 2005- 2008- 2010 2016 2007 2009 2007 2009 2007 2009 Antigua & Barbuda 9.5 -6.0 -7.0 4.4 81.1 76.6 74.5 83.0 118.7 110.1 104.8 90.1 Barbados 3.8 -1.8 0.3 1.6 64.1 79.2 97.5 128.9 94.7 92.1 96.6 N/A Belize 2.8 2.0 3.3 -0.8 88.2 80.4 80.2 76.5 120.8 119.9 115.7 125.1 Dominica 3.1 3.0 0.7 0.9 73.1 57.1 56.7 64.3 89.0 90.6 89.1 96.5 Grenada 5.1 -2.8 -0.5 1.9 75.9 76.0 85.5 88.5 81.5 75.5 73.1 65.6 Guyana 3.4 2.6 4.4 3.3 59.9 63.2 65.3 51.2 153.7 131.8 131.3 105.7 Jamaica 1.7 -2.6 -1.5 1.4 117.2 120.4 131.7 132.7 97.5 100.2 80.9 78.4 St. Kitts & Nevis 3.5 1.6 -2.2 3.6 92.7 102.6 115.8 64.8 88.9 83.1 82.2 104.7 St. Lucia 2.4 1.9 -1.7 0.7 57.0 51.0 54.7 72.9 110.1 107.6 112.4 100.9 St. Vincent & the Grenadines 2.9 2.3 -3.4 3.0 N/A N/A 63.6 77.6 90.5 89.1 84.0 76.8 Suriname 4.5 3.6 5.2 -10.4 17.6 15.7 18.6 26.6 96.7 98.5 91.0 94.3 Trinidad and Tobago 8.1 -0.5 3.3 -5.1 16.4 22.0 35.3 40.9 108.8 95.8 85.8 98.8 Sources: World Bank WDI; figures for central government debt as % of GDP from Bank of Guyana, Bank of Jamaica, Office of the Prime Minister of St. Vincent & the Grenadines, Suriname Debt Management Office, and Central Bank of Trinidad and Tobago when not available through WDI. Sector Context 2. At the time of approval of the SEMCAR Program, countries in the region still lacked many of the features of modern tax, customs and public financial management (PFM) processes, systems and institutions needed to raise sufficient resources and implement sound expenditure policies effectively, efficiently and transparently. 1 Auguste Kouamé and M. Ivanova Reyes. Before and After the Global Financial Crisis – Evaluating the Caribbean’s Synchronization with Global Engines of Growth. World Bank Policy Research Working Paper No. 7544 (January 2016). 2 The program is primarily a Bank-executed technical assistance intervention. As such, concept approval is being used as the reference point instead of appraisal. 3 Figure presented for Barbados on central government debt as % of GDP in 2014 corresponds to 2013. Page 11 of 82 The World Bank Caribbean - Supporting Economic Management in the Caribbean Program (P123665) 3. Tax Administration Challenges. In the tax area, while efforts were underway across the region to implement Value Added Tax (VAT), with success in many cases, issues had emerged such as the need for more systemic changes in order to make the overall tax departments more effective. In addition, in line with growing Caribbean interest in revenue authorities which combine tax and customs administrations (and, if outside Inland Revenue Departments (IRD), property tax) a number of administrations were considering such structures, and the Eastern Caribbean Central Bank (ECCB) had commissioned the preparation of an implementation guide by the Caribbean Regional Technical Assistance Centre (CARTAC) – that was completed in April 2010 which provided a roadmap of the issues to be considered when implementing a revenue authority. 4. The tax administration IT system, Standard Integrated Government Tax Administration System (SIGTAS)—while a major step forward several years prior to the program—had critical shortcomings in functionality and had become therefore inadequate in supporting modernized procedures. It did not manage tax files based on taxpayer identification, though this was an international practice. Therefore, there were risks of fraud or administrative errors. The system needed to be enhanced to provide information to effectively support tax collection and compliance programs. SIGTAS software source code had been modified uniquely in each country, making standardized support for the software difficult and expensive. These challenges required developing more comprehensive agendas for reforming tax administration institutions, including organizational reform, business processes, and ICT. 5. Customs Administration Challenges. Challenges in the customs area were multi-dimensional. Customs departments faced operational and technical constraints in areas such as clearance procedures, valuation of goods, and import procedures. These deficiencies had a major impact on the competitiveness of firms within the countries and their ability to easily import and export. Many customs departments required more fundamental institutional reforms to effectively implement the new procedures and processes. 6. With respect to customs systems, seven of the countries to be included in the Program were using versions of United Nations Conference in Trade and Development’s (UNCTAD) Automated System for Customs Data (ASYCUDA) system. A few countries were upgrading to the latest web-enabled version (ASYCUDA World), with their own resources, and others were considering upgrading. There were also some countries not using ASYCUDA.4 CARTAC’s 2009 IT study recommended that countries continue to upgrade their versions of ASYCUDA and that the system becomes the regional standard for all countries unable to procure or develop (and maintain) alternatives. 7. The development challenge in the customs area was to ensure that business processes were re- engineered to increase administrative efficiency, reduce compliance costs, and take full advantage of the functionality offered by the most up-to-date technology as well as future inter-operability between customs’ systems and the system to replace SIGTAS. 8. PFM Challenges. Finally, the region faced challenges in the PFM area. Some countries had systemic weaknesses in their basic budget processes—as basic as the ability to transfer funds among 4 For example, St. Kitts and Nevis used Crown Agents’ TRIPS system, and Jamaica used its home-grown CASE system. Page 12 of 82 The World Bank Caribbean - Supporting Economic Management in the Caribbean Program (P123665) accounts. Expenditure controls were not in place in many cases. Capital and recurrent budgets were handled separately. Issues were noted in both debt and cash management. On the other hand, while many countries still required technical support in fundamental areas of PFM, some were beginning to focus on second level reforms such as program budgeting, accrual accounting and medium term budget frameworks. These types of reforms were meant to support governments in developing and implementing more inclusive policies and accountable systems. Their implementation also required changes at a range of levels. 9. Both advanced reforms and more basic processes required improved financial information systems to accurately track cash flow, control expenditures and produce meaningful financial reporting. Again, challenges were seen on the PFM software side as well. SmartStream had been widely implemented for expenditure management and budget control. The use of SmartStream improved financial management and budget control in the countries where it had been implemented. However, implementation was uneven and some of the countries were frustrated and underserved. The business processes were not modified to take advantage of the process models that these systems automated. Thus, it reduced the effectiveness of IT systems and business operations. 10. For most countries studied, there was little integration of information within financial systems and between financial systems and revenue systems. Microsoft Excel and Access tools were being used in some countries to collect and, in few cases, transfer the relevant information. The revenue systems lacked revenue accounting functionality meaning aggregation of the revenue information was subject to errors. These PFM challenges required a number of interventions to be resolved, ranging from changing basic laws to restructuring departments and updating the supporting ICT tool. Rationale for Bank Support at Time of Approval 11. The World Bank was approached by the Government of Canada (through the then Canadian International Development Agency, CIDA) to implement the proposed SEMCAR Program and work along with the International Monetary Fund (IMF). In 2009, CIDA introduced its new strategy for the Caribbean and wanted to consolidate and build synergies with other donors around an intervention that would address the abovementioned challenges. In this regard, the World Bank had country strategies in place for Belize, Guyana, Jamaica and Organisation of Eastern Caribbean States (OECS) that highlighted the need to support public sector modernization including improving fiscal management. In addition, at the time of approval, the World Bank-financed Organization of Eastern Caribbean States E-Government for Regional Integration Project (EGRIP) that was aimed at delivering regionally integrated e-government application that apply economies of scale was in the early stages of implementation. On the other hand, the IMF through CARTAC had been providing technical assistance and training virtually to all Caribbean countries on macroeconomic policy, PFM, financial sector supervision, financial stability, macroeconomic statistics, and tax and customs policy and administration. Thus, sharing the responsibility for providing technical assistance in the abovementioned areas between the World Bank and IMF was a logical continuation of their existing work program. Moreover, in the areas of supporting and deploying ICT reforms along with mainstreaming concerns on gender and environmental considerations, the World Bank had the instruments for engagement including establishing recipient-executed grants which was envisioned in implementing the Regional ICT Learning Center. As such, the World Bank was requested to be the administrator of the Canada contributions which were later established under a Multi-Donor Trust Fund (MDTF) arrangement to allow other donors to participate in the future. To this end, the World Bank Page 13 of 82 The World Bank Caribbean - Supporting Economic Management in the Caribbean Program (P123665) had three responsibilities under the Program: (i) trust fund administration; (ii) execution of technical assistance; and (iii) supervision of recipient-executed activities. Theory of Change (Results Chain) 12. Shown below is the logic model of the SEMCAR Program. Given the large number of activities and outputs, the details are presented in Annex 1 rather than in the figure below. Figure 1. SEMCAR Program Logic Model Activities/Outputs Immediate Outcomes Intermediate Outcomes Ultimate Outcome (Impact) • Scoping assessments to • 100 - Increased ability of tax • More efficient, effective, • Improved economic determine the status of tax administration institutions and accountable and regionally management, regional administration, customs staff to manage processes and integrated Tax, Customs and integration and administration, and PFM, ICT systems in at least five PFM institutions, policies, competitiveness in up to 12 related ICT developments & countries processes and ICT systems Caribbean countries, through commitment to SEMCAR’s • 200 - Increased ability of more efficient, effective and approach Customs administration accountable public institutions, • Technical assistance to institutions and staff to manage policies, processes and ICT modernize institutions, processes and ICT systems in at systems processes, procedures, and least four countries • Note: Emphasis added to legislation in the • 300 - Increased ability of highlight the portion of the abovementioned areas Ministries of Finance, line sentence that corresponds to • Technical analyses to ministries and their staff to impact investigate feasible options for manage PFM processes, environmental taxation & tools including gender responsive to enable voluntary tax budgeting and ICT systems in at compliance least five countries • Development of a regional PFM • 400 - Improved access to and revenue systems regional training and ICT modernization strategy with technical support for Tax, accompanying country specific Customs and PFM roadmaps administrations • Note: For full list of activities and outputs, please refer to Performance Measurement Framework Matrix that appears in Annex 1 Project Development Objectives (PDOs) 13. According to the Trust Fund Administration Agreement with Canada, the objective of the Program is “to improve economic management, regional integration and competitiveness in the Caribbean sub- region by facilitating the emergence of efficient, effective, accountable and regionally-integrated institutions, policies and processes as well as information and communication technology (ICT) in tax, customs and public financial management systems, through the provision of policy advice, technical assistance and selected investments in information and communication technology.� 14. On the other hand, the SEMCAR Program Document stipulates that the Program objective is to contribute to more efficient, effective, accountable, and regionally integrated Tax, Customs and PFM institutions, policies, processes and ICT systems. 5 Against, SEMCAR’s logic model and the formulation in 5 According to the Program Document: “The higher level strategic objective of the Program is to improve economic management, regional integration and competitiveness in up to 12 Caribbean countries, through more efficient, effective and accountable public institutions, policies, processes and ICT systems. Within this strategic context, the Program objective is to contribute to Page 14 of 82 The World Bank Caribbean - Supporting Economic Management in the Caribbean Program (P123665) the Administration Agreement, the Program Document’s formulation of the objective is at the level of the intermediate outcome and not impact - which is at the higher strategic level usually considered within the Bank as beyond those of one single project/program. However, as per the Bank’s Implementation Completion Report Guidelines, the assessment should be based on what is legally binding. As such, for purposes of this completion report and following Bank guidelines, the formulation of the Administration Agreement will be used. Key Expected Outcomes and Outcome Indicators 15. Key expected outcomes are presented in the Logic Model. Table 2. SEMCAR Program Key Results Indicators Expected Results Indicators Ultimate Outcome ICT regional integration plan for Tax, Customs and PFM systems Improved economic management, regional integration and competitiveness in up to 12 Caribbean countries, through more efficient, effective and accountable public institutions, policies, processes and ICT systems. Intermediate Outcomes TAX: The Regional Strategic Management Model endorsed at the More efficient, effective, accountable and regionally SEMCAR workshop in October 2011 is institutionalized in at least 3 integrated Tax, Customs and PFM institutions, countries policies, processes and ICT systems. Audit case selection is risk-based in at least 3 countries. CUSTOMS: Support to development of a preclearance framework for air and maritime cargo in at least four countries PFM: Adequate legal, regulatory and organizational structures verified or established in at least 5 countries TRAINING & ICT: Enabling environment and inputs for large-scale implementation of integrated ICT systems completed 16. In terms of indicators, the Program Document discussed during the concept review meeting for SEMCAR Program only provided the logic model with desired results but not how these would be measured in terms of indicators. As such, for purposes of this completion report, the Steering Committee6 endorsed and Canada agreed Performance Measurement Framework (PMF) matrix, as provided for in the SEMCAR Second Year Progress Report, was adopted and is the basis for the assessment. 7 In this regard, more efficient, effective, accountable, and regionally integrated Tax, Customs and PFM institutions, policies, processes and ICT systems. This will be achieved through the provision of policy advice, technical assistance and selected ICT investments.� 6 In terms of governance arrangements, SEMCAR was designed with a Project Management Team, a Donor Committee, and a Steering Committee. The Project Management Team was responsible for (i) preparing annual work plans for endorsement by the Steering Committee and approval by the Donor Committee; (ii) appraising, supervising, and reporting on the progress of recipient-executed projects; (iii) reporting on the progress of program activities; and (iv) monitoring and evaluating the Trust Fund. The Donor Committee was responsible for (i) making decisions about allocations of proceeds from the Trust Fund as set forth in the annual work plan; (ii) providing general guidance for operations of the program; and (iii) setting work plan priorities, among other activities. The Steering Committee was responsible for endorsing the progress reports and the annual work plans, in view of priority activities considering the collective commitment to the program by the beneficiary countries. The Steering Committee was comprised of high-level representatives of each of the beneficiary countries, a representative of the contributing donors, the IMF, and the World Bank (as the Trustee) 7 Per the SEMCAR Second-Year Progress Report, the Steering Committee of July 2011 discussed the possibility to adjust the results matrix in order to be more compatible with activities being implemented or planned. And in March 2013, during meetings with CIDA, the PMF matrix was revised and accepted by CIDA. Page 15 of 82 The World Bank Caribbean - Supporting Economic Management in the Caribbean Program (P123665) the indicators for ultimate and intermediate outcomes are provided in the table above while the complete PMF matrix is provided in Annex 1. 17. The primary beneficiaries of the SEMCAR Program were the government officials of the 12 countries working in the areas of tax, customs, PFM, and ICT. The Program was to build their capacities and provide them assistance to be more efficient and effective in their day-to-day responsibilities. Secondarily, the beneficiary countries and the people living in these countries were to reap the rewards of having a more efficient and effective public resource management operating in the countries with having better capacities to address the challenges mentioned above. Components 18. As approved by Bank management, SEMCAR was designed to be implemented in two phases. The SEMCAR Program or Phase 1, to be funded under TF071579, would focus on technical assistance for urgent reforms and ICT investments in tax administration, customs administration, and PFM. Phase 2 or a follow-up operation would then focus on investment in the upgrading of ICT systems and support for associated policy reforms to improve the efficiency of the public sector. Technical assistance activities and in-depth assessments conducted in Phase 1 would lay the foundation for large ICT investments to be made in Phase 2. 19. The SEMCAR Program was divided into five components (see Annex 1 for details regarding the activities and outputs for each component): i. Component 1 – Enhanced regional tax administrations: The objective of this component was to support the implementation of tax administration reforms through capacity building and technical assistance for the implementation of institutional modernization and restructuring of tax administration procedures. Regional risk and strategic management frameworks were to be completed and models were to be implemented in a selected number of countries. In addition, user business requirements and detailed requests for proposals for an enhanced regional tax administration system were to be developed, using state of the art ICTs. ii. Component 2 – Enhanced regional customs administrations: The main goal of this component was to increase the ability of Customs administration institutions and staff to better manage processes and ICT systems. The Customs Assessment Trade Toolkit (CATT) was selected as the main tool to evaluate the relative strengths and weaknesses of Customs administrations, compare them to good practices, and produce action plans to overcome the main weaknesses identified by the CATT assessments. A regional risk model was to be developed and implemented in a selected number of countries. iii. Component 3 – Enhanced regional public financial management: The aim of this component was to reform PFM processes and institutions to ensure that future ICT systems are geared to sound business processes. For most countries, there was ample scope to strengthen existing budget processes. Therefore, on-the-ground technical assistance for modernizing and restructuring of PFM processes including budget formulation, budget execution, treasury management, cash accounting standards, fiscal reporting and internal audit were to be delivered. In addition, business requirements for Integrated Financial Management Information Systems (IFMIS) upgrades were to be prepared and a data warehousing system Page 16 of 82 The World Bank Caribbean - Supporting Economic Management in the Caribbean Program (P123665) and interoperability modules across the three main systems were to be implemented in a selected number of countries. iv. Component 4 – Regional training and ICT support center: This component was designed to develop the design of the business model for the Regional Training and ICT Support Center, along with actual delivery of regional training programs. The intention of the regional training center was to provide development programs and seminars for tax and customs administrations as well as PFM staff. Training activities were to be linked to Components 1, 2 and 3, in order to create synergies between ICT and business-related support activities. v. Component 5 – Project management: Included project management, quality assurance, and monitoring and evaluation. 20. These activities were to be jointly executed by the IMF and the World Bank with some activities to be pursued by the Bank under the Recipient-Executed Trust Fund (RETF) arrangements. B. SIGNIFICANT CHANGES DURING IMPLEMENTATION (IF APPLICABLE) Revised PDOs and Outcome Targets 21. The development objective of the SEMCAR Program as stipulated in the Administration Agreement was not revised. Likewise, the outcome targets/results areas as presented in the logic model and in the Program Document were not revised. Revised PDO Indicators 22. With respect to the PDO (ultimate, intermediate and immediate outcome) indicators, there were no significant changes made after the SEMCAR Second Year Progress Report version provided for the complete set of indicators in the PMF, except on a customs indicator related to the post clearance audit program. This was dropped in the SEMCAR Progress Report covering the period April 1, 2015 to March 31, 2016 since CARTAC was already providing support in this area. Revised Components 23. With the Steering Committee responsible for reviewing and endorsing the SEMCAR work plan for the donors’ approval and Canada being the sole donor who is represented in Steering Committee Meetings, the Meetings became the venue for agreeing on and approving the work plan. In this regard, it is in these Steering Committee Meetings that specific activities under each component and their prioritization were discussed and approved. As such, given the evolving priorities and business needs of the SEMCAR countries, it is not surprising that specific activities/outputs under each component were revised to ensure that the program was responsive to SEMCAR countries’ needs and thus ensure ownership and sustainability. Moreover, from the onset, the SEMCAR countries cautioned on the ambition of the Program and communicated that some planned outputs were more appropriate for a follow-up operation. Accordingly, some activities under each of the components were dropped, revised and/or approached differently (see Annex 8 for comparison of planned and executed activities). Page 17 of 82 The World Bank Caribbean - Supporting Economic Management in the Caribbean Program (P123665) 24. Under the Tax component, there were three notable changes that were introduced to recalibrate against what is feasible. These are reflected in the SEMCAR Second Progress Report and all relate to ICT systems. First, the activity to have a regional maintenance contract for tax administration systems negotiated or signed was dropped. It was not possible to negotiate a regional maintenance contract since most countries were operating different versions/features of SIGTAS, with some having modified the source code. Second, the activity to have regional tax administration system upgrades implemented in at least five countries was scaled-down to only the completion of detailed request for proposals (RFP) to acquire or design a tax software for at least five countries. Third, the activity to complete detailed user requirements to acquire or design an integrated tax administration system to replace SIGTAS system for five countries was revised to drop reference to the replacement of SIGTAS and clarify that the conceptual design was for a regional tax administration system not just for five countries in keeping with the Program’s regional approach. Moreover, at the 2015 Steering Committee Meeting, it was decided that absent a new tax administration system and prevailing fiscal challenges, it was critical that SIGTAS be stabilized in the countries (i.e., Antigua and Barbuda, Belize, Dominica, Grenada, St. Kitts and Nevis, St. Lucia and St. Vincent and the Grenadines) currently using it and its functionalities be improved building on the positive experience of St. Kitts and Nevis. 25. Under the Customs component, four activities related to ICT were dropped or revised and a new activity to improve communications outreach was added. Two ICT-related activities were considered to be unrealistic to implement given that countries were implementing ASYCUDA World and the maintenance contract was not a priority for them. These were the activities on: (i) regional contract for custom management systems being negotiated and signed; and (ii) regional customs management system maintenance implemented in at least four countries because current systems in place in each country differs which will complicate establishing a regional maintenance contract.8 A third activity on defining user business requirements for an automated exchange of Customs declarations was dropped, as it was planned to be subsumed under another activity (i.e., the Comprehensive Underlying Systems Integration, COUSINS), which was also later dropped.9 26. On the other hand, the activity with respect to development of user requirements for regional upgrade to customs management system including linkage to the new tax IT system was recalibrated to reflect what can realistically be achieved. To this end, the activity focused on providing technical assistance to support integration between the tax and customs administrations, including the exchange of information on revenue and taxpayers and declaration of customs cargo in at least four countries. Moreover, in response to SEMCAR countries demand, a new activity was included to establish effective feedback mechanisms between Customs and critical stakeholders to improve Customs performance. 27. On the PFM component, two activities were revised and one new activity on state-owned enterprises (SOE) was added. The activity to develop user business requirements for IFMIS upgrades including linkage to budget formulation module for at least five countries was replaced with the development of a common framework for functional requirement of core and non-core IFMIS components. Second, the development of tools to track gender-specific expenditures and their integration into IFMIS modules to produce gender disaggregated reports and its piloting in at least one 8 The Customs Assessment Trade Toolkit (CATT) exercises conducted under SEMCAR have noted that even those on ASYCUDA have implemented the systems differently. 9 COUSINS was not related exclusively to customs. Page 18 of 82 The World Bank Caribbean - Supporting Economic Management in the Caribbean Program (P123665) country was scaled-down to simply the creation of awareness for improved management and delivery of gender-sensitive budgetary planning. Likewise, during the Eighth Steering Committee Meeting in December 2014, the countries requested for technical assistance in strengthening corporate governance and fiscal risk framework for SOEs. 28. Under Component 4, there were no revisions in activities. Other Changes 29. Extensions. Implementation schedule changed from an original three-year period to six-and-a- half years. The Trust Fund (TF071579) became effective on February 2, 2011 with the original closing date of December 31, 2013. The closing date was extended four times to its current closing date of August 31, 2017 primarily to complete activities endorsed by the Steering Committee.10 30. Change in Approaches. During implementation, some approaches on executing SEMCAR activities had to be revisited. For example, at the Fifth Steering Committee Meeting of May 2013, the idea of establishing COUSINS11 was agreed to become the single comprehensive undertaking for interoperability frameworks, common framework for core and non-core IFMIS, reporting and data warehousing, functional requirements under SEMCAR’s PFM component and related activity under the customs component. However, COUSINS was initially postponed (decision from the Sixth Steering Committee Meeting of November 2013) as the countries wanted to focus on more urgent matters (e.g., stabilization of SIGTAS, optimizing SmartStream) and then later dropped given the varying levels of ICT development and priorities in each country. The aspects of establishing reporting and data warehousing systems and interoperability modules were then incorporated through the conduct of regional ICT assessments but this only allowed for the identification of specifications and not the implementation of systems per se. 31. Another example is with respect to the feasibility study, institutional design and business plan for the establishment of a regional training and ICT support center. Provision of ICT support was separated from regional training in 2015. ICT support was provided to the countries through an ICT Advisor based in Barbados. With respect to the regional training center, the Program pursued an approach that was mindful of messages from SEMCAR countries from the onset that have communicated the need “to use existing structures for supporting regional integration to guarantee country ownership, sustainability and 10 The four extensions were: (i) February 28, 2015 – This first extension was necessary to allow sufficient time for the work program endorsed by the Steering Committee to be pursued. This extension also allowed for the Letter of Understanding (LOU) between the International Monetary Fund (IMF) and Bank to be extended. (ii) December 15, 2015 – This second extension was necessary to accommodate an extension of the closing date of the LOU between the IMF and the Bank that would allow the IMF to complete their PFM activities under SEMCAR. (iii) December 31, 2016 – This third extension was necessary to allow the program to receive potential trust funds contribution from the European Union and to allow the recipients to complete activities that were under their execution. (iv) August 31, 2017 – This fourth extension was necessary to allow the recipients to complete complex procurement activities under the recipient executed grants that were intended to strengthen SIGTAS Tax Administration System, to upgrade equipment for the ASYCUDA World Customs Administration and to strengthen SmartStream Financial Management System in selected SEMCAR countries. 11 COUSINS was to support the integration and interoperability between Banking, Customs, Inland Revenue and Treasury systems and would prepare the countries for expansion into E-Government. Page 19 of 82 The World Bank Caribbean - Supporting Economic Management in the Caribbean Program (P123665) development impact�.12 A feasibility study comparing the viability of a physical center against a virtual center was conducted and resulted in the endorsement by the Steering Committee of a hybrid approach. To this end, the Program partnered with the University of the West Indies and completed piloting of a hybrid model. 32. With respect to negotiating a regional maintenance contract for IFMIS, SEMCAR initially provided assistance through a recipient-executed grant executed by St. Lucia for the purposes of negotiating a regional maintenance contract with SmartStream. Negotiations were completed in 2013. However, due to delay in decisions among the countries on how the implementation of the maintenance contract would be funded, the contract was never signed. By the time, the countries requested for SEMCAR financing of the maintenance contract, the validity of the negotiated contract already expired. Nevertheless, in 2016, a new recipient-executed grant implemented by St. Lucia became effective that allowed for the provision of support to Belize, Dominica, Grenada, St. Lucia and St. Vincent and the Grenadines to upgrade selected SmartStream components to improve their PFM operations and access to critical financial data. 33. Funding. With the appreciation of the Canadian Dollar against the United States Dollar, the Multi- Donor Trust Fund had foreign exchange gains of about US$ 1.2 million (approximately 7%). This allowed the IMF and the World Bank to pursue additional activities such as those related to SOE, supervision of the recipient-executed contemplated for the upgrade of SIGTAS and additional PFM short term missions.13 Moreover, Canada remained the sole-donor for the SEMCAR Program.14 While no additional donors participated in the end, the Program’s logic model and PMF remained intact as it was fully costed against Canada’s contribution. Rationale for Changes and Their Implication on the Original Theory of Change 34. Some of the activities were revised or dropped with new ones added during implementation of the program to better respond to demands from SEMCAR countries. However, these changes did not compromise the Logic Model of the program. II. OUTCOME A. RELEVANCE OF PDOs Assessment of Relevance of PDOs and Rating 35. The SEMCAR program is consistent with the following core areas of the World Bank’s Latin America and the Caribbean Regional Strategy of 201715: spurring economic growth and transparency and accountability. In terms of spurring economic growth, efforts to improve tax and customs administration are key to strengthening the business environment. The SEMCAR program also contributes to the Bank’s objective of fostering more effective and transparent governance in the region. 12 SEMCAR First Steering Committee Meeting Minutes. 13 SEMCAR Annual Report (May 2015). 14 The program was designed to accommodate potential contributions from other donors hence it was established as an MDTF. However, while there was interest expressed from the European Union in 2016, this did not materialize due to timing considerations and priorities. 15 World Bank LCR Regional Update 2017. Page 20 of 82 The World Bank Caribbean - Supporting Economic Management in the Caribbean Program (P123665) 36. The SEMCAR program is aligned with the 2015-2019 OECS Regional Partnership Strategy (RPS). It is positioned to deliver on the RPS’s second pillar that seeks to support public sector modernization through supporting OECS governments to building more efficient, transparent and accountable public administrations, more robust institutional capacity and stronger frameworks for partnership with the private sector. 37. Similarly, the SEMCAR program is aligned with the following areas of engagement highlighted in the 2014-2017 Jamaica Country Partnership Strategy (CPS): (i) public sector modernization; and (ii) enabling environment for private sector growth. The CPS states that Jamaica has been afflicted by weaknesses in the budget process that affect the planning and execution of government spending. Jamaica also faces challenges associated with tax and customs administration that undermine the investment climate. The CPS highlights that Jamaica lags behind regional competitors in Doing Business (DB) indicators such as ease of paying taxes and trading across borders. In that regard, the program is tightly aligned with Bank efforts to improve fiscal management and public administration performance, as well as the business environment. 38. The SEMCAR program is aligned with the following area of engagement of the 2015-2019 Suriname CPS: (i) creating a conducive environment for private sector development. Within this area of engagement, one of the objectives of the Bank is to enhance the business environment through activities aimed at improving standards for financial reporting in the public sector in line with international standards, and improved customs procedures. The Bank expects to contribute to the development of a Public Financial Performance Report during the Country Partnership Framework (CPF) period, and strengthen the accounting and auditing framework for SOEs. In particular, the SEMCAR program is consistent with Bank efforts to improve the operations of Suriname’s Customs Office and reduce customs clearance times. 39. The 2018-2022 Belize CPF will focus on two main areas: building climate resilience and promoting financial inclusion and social resilience. The SEMCAR program is not aligned with any of these areas, although strengthening public financial management is still a country priority. The CPF emphasizes that the World Bank Group (WBG) will maintain regular dialogue with the government and will remain open to engage in the cross-cutting area of fiscal sustainability, should there be demand. Engagement in this area could include support with public financial management reforms. 40. As for Barbados, Guyana, and Trinidad and Tobago, the Bank does not have a CPF or CPS, thus the relevance of the PDOs to Bank strategies could not be assessed for these countries. However, the program’s objectives and activities remain relevant to their respective national development plans. Moreover, the Bank intends to develop a CPF for Guyana in FY19. Out of the SEMCAR member countries with a CPF or CPS, the only country where there are moderate shortcomings in the relevance of the program’s development objectives to the current CPF is Belize. Nevertheless, it is aligned with Belize’s National Development Plan for 2010-2030 in particular with respect to their pursuit of the plan’s pillar 1 (democratic governance for effective public administration and sustainable development) and pillar 3 (economic resilience: generating resources for long-term development). 41. With respect to the donor, the SEMCAR program is fully aligned with Canada’s international assistance priorities that consider integral the concepts of advancing gender equality, supporting Page 21 of 82 The World Bank Caribbean - Supporting Economic Management in the Caribbean Program (P123665) economic sustainability, and helping to strengthen governance institutions and practices in its assistance programs, policies and projects.16 42. Based on the above, the rating for relevance of PDOs is Substantial. B. ACHIEVEMENT OF PDOs (EFFICACY) Assessment of Achievement of Each Objective/Outcome 43. The SEMCAR Program has substantially achieved its development objective and desired outcomes (ultimate, intermediate and immediate) as per the PMF agreed by Canada (see Table 3). Table 3. SEMCAR – Summary Table on Number of Indicators Achieved Outcome No. of Exceeded Achieved/Largely Partially Total Level Indicators Achieved Achieved Ultimate 1 1 1 Intermediate 5 1 2 2 5 Immediate 16 2 10 4 16 Total 22 4 12 6 22 Achievement of the Ultimate Outcome Indicator 44. The SEMCAR Program achieved its ultimate outcome indicator target with the ICT regional integration plan for tax, customs and PFM systems in place at the regional and country levels that would benefit all twelve SEMCAR countries. The plan is focused on simplifying and improving quality and efficiency of operations and systems in the three areas. Moreover, the SEMCAR Steering Committee has prioritized the enhancement of regional tax administration and financial management information systems and their interoperability with public revenue administration systems, as well as harmonizing public sector policies, process and systems for the next step. These plans were further enhanced with the definition of business and technical requirements for the Regional Tax System and the Integration of the Customs, PFM and Tax Systems which significantly advances readiness of any follow-up investment operation to implement the systems. As originally envisaged, the Program delivered on its ultimate indicator and laid the foundation for establishing regionally integrated tax, customs and PFM systems. Achievement of Immediate & Intermediate Outcomes 45. The desired intermediate outcome per the SEMCAR Logic Model was more efficient, effective and accountable and regionally integrated tax, customs, policies, processes and ICT systems. Achievement was to be measured through five indicators which were in general largely achieved and exceeded in one case. First, a Regional Strategic Management Model with respect to tax administration was to be institutionalized in three countries and this was over-achieved with six countries (St. Kitts and Nevis, 16Results-Based Management for International Assistance Programming at Global Affairs Canada: A How to Guide (Second Edition, 2016). Page 22 of 82 The World Bank Caribbean - Supporting Economic Management in the Caribbean Program (P123665) Antigua and Barbuda, Grenada, Trinidad and Tobago, Dominica and St. Lucia) having institutionalized the model by early 2016. Second, audit case selection in both customs and tax was to be risk-based in three countries. This target value has been largely achieved with risk-based selection used in Jamaica (tax and customs), Dominica (customs), Grenada (tax) and adopted in St. Kitts and Nevis and St. Lucia. Third, the development of a pre-clearance customs framework for air and maritime cargo in at least four countries was partially achieved. SEMCAR completed a draft framework and pre-clearance readiness assessment for Dominica, St. Kitts and Nevis, Jamaica and St. Lucia, but no further support was provided to implement nor pilot it absent country demand. Fourth, adequate legal, regulatory and organizational structures were to be verified or established in at least five countries. This has been largely achieved. A PFM law was enacted and promulgated in Grenada and the 2017 budget was prepared in keeping with the requirements of the 2015 PFM Act.17 Belize and St. Lucia have draft PFM laws but have yet to be enacted with St. Lucia currently working on advancing it in parliament. Likewise, the SEMCAR Progress Report of 2016 notes that there have been improvements in the organizational performance of the Accountant General’s Departments (AGD) of Belize, St. Kitts and Nevis, St. Lucia and Suriname. Fifth, the indicator of establishing an enabling environment and inputs for large-scale implementation of an integrated ICT system with all countries having access to the ICT training center was partially achieved. Through the RETF grants, improvements in tax administration and IFMIS environments were supported in some countries. Likewise, a hybrid learning course on the basics of SOE was piloted and made available to all countries. 46. With respect to the immediate outcomes, underpinning the achievements of the intermediate and immediate outcome indicators are SEMCAR’s success in strengthening capacities of tax, customs and PFM staff and institutions in the SEMCAR beneficiary countries making them better contributors to overall economic management (see Annex 1 for details). 47. Beyond the indicators, discussed below are outcomes to which the SEMCAR Program contributed to improving economic management18 (increased efficiency, effectiveness and accountability in tax and customs administration and PFM), regional integration (knowledge sharing, consensus building, ICT/regional tax administration system) and competitiveness (facilitating trade) in most of the beneficiary countries. The Program being demand driven, its outcome contributions are manifested in the countries that requested for its support. Achievements are in varying degrees across countries given differences in starting points, capacities and priorities. Likewise, the Program contributed to increasing awareness on environmental and gender considerations in tax, customs and PFM areas. Moreover, and probably the most transformational contribution of SEMCAR, it adopted a regional approach that is anchoring all these achievements. Within this approach, based on diagnostics conducted and good practices identified in one SEMCAR country, a common framework for addressing a specific tax, customs and PFM challenge was developed and endorsed for all the beneficiary countries to use as key reference and which each country can then customize depending on where it is at, its capabilities and priorities making it more relevant and realistically implementable. Contribution to Improved Economic Management 17Grenada. Letter of Intent to the IMF (May 2, 2017). 18Defined as “the management of the resources, finances, income, and expenditure of a community, business enterprises, etc.� (Collins English Dictionary on line). Page 23 of 82 The World Bank Caribbean - Supporting Economic Management in the Caribbean Program (P123665) 48. Tax Administration. SEMCAR facilitated the emergence of an effective, efficient and accountable tax administration. Some of the results evidencing these are: 49. In St. Kitts and Nevis, informed by the Strategic Management Framework (SMF)19 that was developed by SEMCAR, its Inland Revenue Department (IRD) enhanced its SIGTAS tax administration system functionality with business intelligence that allowed the IRD to have better and more accurate data, consistent and timely information and to deliver the right information to the right people at the right time. This not only put in place evidence-based and improved decision-making at IRD but as well as contributed to savings in time, money and other resources in performing its functions.20 In addition, with strong leadership, St. Kitts and Nevis IRD adopted the most critical elements of the SMF – a dedicated planning unit has been established, operational plans are now prepared annually and are the basis of the budget estimates submitted to the Ministry of Finance as well as a complete monthly monitoring of performance – making staff more accountable. These improvements could have indirectly contributed to slight improvements in the taxes on income and overall tax revenue to GDP ratios from 2011-2015.21 Moreover, with support from SEMCAR, it put in place recently an E-IRD solution which is expected to enhance monitoring and supervision capabilities, minimizing unnecessary manual operations and optimizing the management of clients. 50. Following the success in St. Kitts and Nevis, Belize with support from a SEMCAR also recently enhanced its SIGTAS with business intelligence functionality which is expected to reap similar benefits as St. Kitts and Nevis. 51. In Grenada, under the strong leadership of the Comptroller and the Planning Unit manager, the IRD institutionalized the SMF – transforming it into a performance-based entity. It now has strategic and operational plans in place which are updated annually, planning unit effectively functioning despite being understaffed, performance is being assessed on a monthly, quarterly and annual basis.22 These contributed to overall improvements to tax administration that has been noted to have improved revenue performance. As noted in the Grenada’s Letter of Intent to the IMF dated May 2, 2017,23 their revenue performance is attributable not only to strong economic growth in 2016 but as well as in improvements in tax administration. As noted in the Letter, IRD rolled out Corporate Strategic Business Plan to Staff and is progressing with its operational plans. IRD also implemented a compliance strategy in 2016 which resulted into an improved average compliance rate of 61%. Likewise, a payment installment plan for taxpayers was introduced that allowed payments over the year and helped cash flow regularization for the government. 19 SMF (i) provided a roadmap to the tax administrations to achieve their articulated objectives and strategies, (ii) focused their resources on priority programs and activities that delivered the desired outcomes, (iii) provided a logical reference to develop credible budgets, (iv) established an accountability system within the organization and developed better internal coordination of programs and activities across the organization. 20 Minutes of the 9th SEMCAR Steering Committee Meeting (June 2015). 21 Table 2B in the IMF Article Consultation (2017) presents tax revenue to GDP ratio of 20.1% in 2011 and 21.4% in 2015 and taxes on income to GDP ratio of 4.3% in 2011 to 5.4% in 2015. 22 Maureen Tracy, SEMCAR: Institutionalizing SMF in SEMCAR Recipient Countries – Journey, Achievements and Lessons Learned (May 2016). 23 Grenada: Letter of Intent, Memorandum of Economic and Financial Policies, and Technical Memorandum of Understanding (submitted to the IMF and dated May 2, 2017). Page 24 of 82 The World Bank Caribbean - Supporting Economic Management in the Caribbean Program (P123665) 52. For its part, Dominica - using the Risk Management Framework (RMF) developed under SEMCAR which was based on Jamaica’s experience – reported an increase in its revenue by $1.3 Million for its financial year 2014/15 by simply adopting a risk—based approach to the selection of cases for audit. This increase resulted from auditing only 30% of cases instead of its previous practice of 100%.24 53. Customs Administration. SEMCAR’s primary contribution in the area of Customs has been in increasing awareness and capacities of the countries on the importance of and tools for strengthening strategic and risk management frameworks. Their adoption of these are for the most part reflected in the modules that they have in place in their customs ICT systems which are all now operating on ASYCUDA World platforms albeit not financed under SEMCAR. Moreover, it appeared that in customs, the countries were most keen in improving accountability and sought SEMCAR support on client outreach. 54. As a result, Jamaica has improved its web-based feedback mechanism between customs, general public and stakeholders. Likewise, the Jamaica Customs Agency adopted an “Agency Risk Management Policy� (ARMP) in 2014 that allows it to target control efforts and resources to transactions that present the highest risk; and to this is end, pre-clearance, intelligence and post clearance controls are given due emphasis.25 This contributed to trade facilitation. It is noted that Jamaica’s Doing Business Trading Borders indicators improved from more than 20 days in average to export or import in DB2010 to less than 3 days in DB2017. Moreover, Program interventions were consistent with the overall direction of customs reform in the beneficiary countries which in the area of improving doing business have translated in a significant reduction in the average days to export or import in almost all the 12 countries based on data from DB2010 to 2017.26 55. For its part, the Barbados Revenue Authority which was established in 2014 independently of SEMCAR benefited from technical advice from SEMCAR on how to streamline operations, business processes and systems integration given its mandate as the single entity for revenue administration. As of April 2017, the excise section of the customs department has been integrated into the tax administration27 and full integration is still in process. 56. Public Financial Management. SEMCAR facilitated the emergence of efficient, effective and accountable PFM institutions and operations among selected SEMCAR countries through its contributions in improving legislative framework, organizational performance, budget process, reporting and systems. Some of the results are evidenced below. 57. Legislative Framework. In Grenada, SEMCAR’s focus has been in improving the legislative framework for PFM in Grenada. The enactment of the 2015 PFM Act has been noted in the 2015 PEFA to be critical in strengthening fiscal discipline and providing sustainability to improvements introduced such as in the budget preparation process as these are now mandated under the Act. Similar improvements are expected in St. Lucia and Belize which have followed Grenada’s model in drafting new comprehensive PFM acts with support from SEMCAR. 24 Dominica IRD presentation to the SEMCAR Regional Workshop – A Practical Approach to Risk Assessment in the Caribbean Region in October 2016. 25 Jamaica Customs Agency Website. 26 Barbados was maintained its average of 5.9 days from DB2010 to DB2017. 27 Stephane Schlotterbeck. Tax Administration Reforms in the Caribbean “Challenges, Achievements and Next Steps�. IMF Working Paper WP/17/88 (April 2017). Page 25 of 82 The World Bank Caribbean - Supporting Economic Management in the Caribbean Program (P123665) 58. Organizational Performance. In St. Lucia and Nevis, SEMCAR contributed to improving the organizational performance of their Accountant General Departments (AGDs). For St. Lucia, an effective cash management function was established in the areas of cash flow forecasting and the establishment of the TSA. The TSA operations allowed for daily consolidation which led to significant reduction in penalties payable to the Bank of St. Lucia. The bank reconciliation tools were also shared by St. Lucia with the other SEMCAR beneficiary countries. In Nevis, with the adoption of SEMCAR recommendations, the AGD achieved improvements in: efficient budget execution by processing all expenditure payments including budget control in the IFMIS, the adoption of direct deposit method for vendor payments reducing significantly the number of checks issued, and bringing bank reconciliation up to date. 59. Budget Process. In Belize, the country achieved significant advances in its planning and budget processes. It now uses a multi-year approach to budget formulation using rolling forward estimates for the budget year plus two outer years. Likewise, it has put in a place a program budgeting structure. 60. Reporting. Belize and St. Lucia are now producing International Public Sector Accounting Standards (IPSAS) (cash basis) compliant financial statements that have improved in-year reporting. For Belize, this also allowed compliance to its Fiscal Responsibility Regulations. St. Lucia, St. Vincent and Grenadines and Suriname now have new Chart of Accounts structure that are reporting against the IMF’s General Financial Statistics (GFS) and the UN Classification of the Functions of Government (COFOG) with St. Kitts and Nevis in advance stage of adoption. In Antigua and Barbuda, compilation of economic and financial statistics improved with the adoption of GFMS2001 and COFOG that was adopted since their 2014 budget preparation process. 61. SOE Management. Through the support provided by the SEMCAR Program (diagnostics, SOE Academy and on-line course, benchmarking study on water utilities and study tour to Malaysia), increased the knowledge of SEMCAR countries on corporate governance and the fiscal risk from SOEs and how to manage them based on good international practices. It also built consensus among in-country stakeholders (e.g., ministries of finance, SOE management) on common challenges (e.g., reporting). As such, it helped set the stage for undertaking further steps in SOE reform and provided just in time guidance to those countries currently undertaking the reforms (e.g., Jamaica and Grenada). Contribution to Improved Regional & Systems Integration 62. SEMCAR’s success in improving regional integration are evident in three key areas: sharing knowledge that surfaced good practices from within the sub-region and promoted learning from each other, building consensus for regional approaches, and promoting regional system solutions that are customizable to country specific needs. 63. Knowledge Sharing & Identifying Good Practices from Within. With respect to regional integration, beyond the ICT integration aspects, the Program facilitated a more intensive and solutions- oriented knowledge sharing across countries. Moreover, it was transformational in facilitating the identification of good practices and experiences among the SEMCAR countries that were actively shared and used as reference in developing regional solutions/approaches to inform other beneficiary countries’ reform path (e.g., St. Kitts and Nevis Business Intelligence enhancement to SIGTAS, Grenada new PFM Act, St. Lucia’s bank reconciliation tools, Dominica and Jamaica’s risk management, etc.). Page 26 of 82 The World Bank Caribbean - Supporting Economic Management in the Caribbean Program (P123665) 64. Further, an approach towards a sustainable regional learning solution was also piloted. Cognizant of Steering Committee preference to use existing institutions over creating new ones, SEMCAR piloted with the University of the West Indies (UWI) a hybrid approach in providing training. In this regard, a one- week SOE Academy was conducted face-to-face and the materials were later converted to an online course that could be used as introduction or refresher course on the basics of SOEs. However, due to the current structure of UWI particularly the UWI Open Campus (UWIOC), it is unable to host the course for free for an indefinite period. As such, the course is currently available upon request on the World Bank’s Open Learning Campus.28 65. Building Consensus. Improved regional integration was also manifested in the consensus building for regional frameworks and approaches to tackle common challenges. For example, this is manifested in the endorsement by the Steering Committee of the Regional Strategic and Risk Management Frameworks that have informed the reform trajectory in tax and customs administrations of the beneficiary countries. Another powerful manifestation of consensus building is exemplified by the 12 countries admirably agreeing to endorse further support on stabilizing SIGTAS despite the fact that it will not directly benefit all of them. 66. Promoting Regional System Solutions. At the core of SEMCAR is the development of regional system solutions to harness economies of scale. With respect to ICT systems for tax administrations, the SEMCAR Program succeeded in building consensus for a regional approach towards adopting a tax administration system. It developed the business requirements including conceptual design with functional and technical specifications for a comprehensive regional tax solution system for the seven29 countries currently using SIGTAS. This system can be customized to fit the specific needs of each country. The specifications would serve as inputs to request for proposals when investing in the establishment of a system which is not designed to be funded under the Program. If successful, this would be the first time a single tax system has been put in place for several sovereign countries. In the meantime, SEMCAR stabilized the existing SIGTAS functionalities in St. Lucia and St. Vincent and Grenadines and enhanced those of St. Kitts and Nevis as well as Belize. 67. On IFMIS, the Program developed a detailed data exchange design and requirements to integrate Customs Management System with Public Financial Management System and Tax ICT System that would serve as inputs into the RFP for investing in the system – which is not covered by this Program. This included specifications for the technical requirements (including common specifications and specific requirements for each information system if needed), security requirements, messaging definition, Service Level Agreement, software warranty requirements and post-implementation and maintenance service requirements. To this end, Trinidad and Tobago issued an RFP for its IFMIS in December 2016 guided by the specifications developed under the SEMCAR Program potentially saving it about 10% of the cost of the actual system (i.e., as per industry standard) had it commissioned consultants to develop its own RFP. At the same time, Jamaica is using these as well in its consideration to deploy an IFMIS. Moreover, the Program supported the stabilization of current SmartStream through a Recipient-Executed 28https://wbg.sabacloud.com/Saba/Web_spf/NA1PRD0002/common/leclassview/dowbt000000000004453. 29These seven SIGTAS users are: Antigua and Barbuda, Belize, Dominica, Grenada, St. Kitts and Nevis, St. Lucia, and St. Vincent and Grenadines. Page 27 of 82 The World Bank Caribbean - Supporting Economic Management in the Caribbean Program (P123665) Grant executed by St. Lucia on behalf of Grenada, Belize, Dominica, St. Vincent and the Grenadines and itself. This enabled Grenada to acquire an enhanced SmartStream budget module which was a critical requirement for implementing multi-year budgeting which is expected to be utilized for the upcoming budget cycle. Contribution to Improved Competitiveness 68. By design, the Customs component was expected to contribute to improving competitiveness. In this regard, SEMCAR’s interventions were aligned with overall initiatives in the area of customs administration in the countries and together have contributed to improvements in the DB trading across borders indicators – particularly with respect to average days to export or import that facilitates trade and can thereafter improve competitiveness. Contribution to Mainstreaming Environmental & Gender Considerations in Tax, Customs, and PFM 69. Environment. The SEMCAR Program worked on increasing awareness on how environmental considerations could be incorporated in the areas of tax, customs and PFM. The assessments of the state of environmental consideration in each country was conducted followed by a study on viable options and examples of good practices and then pilot implementation of the options were explored with volunteer countries. In this regard, Jamaica considered the options and recommendations resulting from the study in the review of their environmental levy. As a consequence, the SEMCAR Program contributed in the broadening of the environmental levy introduced in 2008 that was charged only to imported products with the introduction of 0.5% levy on the domestic side in FY2015/16. 70. Gender. The SEMCAR Program also worked on increasing awareness on how gender considerations could be mainstreamed in the areas of tax, customs and PFM. Similar to the approach on environment, assessments of the state of gender considerations in each country was conducted followed by a study on feasible options and piloting of recommended options including a process-mapping to identify how to incorporate gender in tax administration. In this regard, Barbados volunteered to pilot the implementation of a gender-sensitive voluntary compliance exercise on tax. As such, a complete set of tools for conducting the exercise was developed and training were conducted. However, the pilot could not proceed due to the unavailability of data. That said, tools are readily available for any country interested to conduct the exercise. In the area of PFM, a workshop was conducted jointly with CARTAC on gender-sensitive budgeting and seven SEMCAR countries benefited from the workshop deliberations. Justification of Overall Efficacy Rating 71. Overall, the SEMCAR Program contributed to improvements in economic management as evidenced by improved performance in tax and customs administrations and PFM in relevant SEMCAR countries who are now able to benefit for example from a new PFM law (e.g., Grenada) and enhanced institutions that: (i) are performance-oriented/accountable (e.g., St. Kitts and Nevis, Grenada on tax, Jamaica on customs), (ii) have adopted risk-based policy approach in their operations (e.g., Dominica on tax, Jamaica on customs), and (iii) have improved their budget execution processes and treasury operations (e.g., St. Lucia, Nevis). The Program also successfully contributed to regional integration by identifying good practices among SEMCAR countries, facilitating intensive knowledge sharing, and more importantly by building consensus across SEMCAR countries on adopting regional approaches to address Page 28 of 82 The World Bank Caribbean - Supporting Economic Management in the Caribbean Program (P123665) common operational challenges in tax, customs and PFM administration and in establishing ICT systems. With respect to contributing to improved competitiveness, interventions in customs operations have facilitated trade. Given these outcomes and the level of achievement against the results indicators while operating in a complex context, the overall efficacy is rated Substantial. C. EFFICIENCY Assessment of Efficiency and Rating 72. Given that the SEMCAR program is primarily a technical assistance intervention, traditional measures of efficiency cannot be easily applied. Accordingly, the economic rate of return was not calculated and presented at concept; thus, no baseline numbers could be used to judge the efficiency of the program. However, there are aspects of design and implementation that contributed to the efficient use of resources, as described below. Most importantly, improved capacity of government officials through training and advisory services, reinforced by knowledge sharing and capacity development at the regional level, and the upgrading or introduction of ICT systems for tax, customs, and PFM, should result in improvements in performance in the beneficiary countries and maximize the efficient use of public resources. That said, these benefits accumulate over time and quantifying them is extremely difficult and do not easily allow for comparison with the costs involved in achieving the program objectives. Moreover, it should also be noted that the institutional and legal arrangement with the IMF does not authorize the Bank to demand financial reports from the IMF which did not allow for determining the total costs of specific interventions and comparing them against the quantifiable benefits. 73. Quantifiable Benefits on the Part of Beneficiaries. As an example of improvements in efficiency achieved through the SEMCAR program, the bank reconciliation system introduced in St. Lucia eliminated cumbersome manual processes, with reported reductions in overdraft penalties to the Bank of St. Lucia. Likewise, Dominica’s implementation of a risk-based audit allowed it to collect more revenues. Moreover, by using SEMCAR’s outputs as inputs into their IFMIS RFP, Trinidad and Tobago was able to avoid the cost of commissioning its own consultants to develop an RFP independently, which is typically equivalent to 10% of the cost of investment in the systems. 74. Efficiency Gains from Program Management. In terms of program management, several aspects are worth highlighting that point to the fact that cost efficiency aspects were considered during implementation. One of the Task Team Leaders of the program was based in Kingston, Jamaica rather than Washington, DC. A Regional ICT Advisor was placed in Barbados with the dual objective of being more responsive to client needs, and reducing the need of staff/short term consultants traveling from Washington, DC. Regional advisors for PFM and Tax were also placed at CARTAC. Meetings between program staff and clients were commonly held via videoconference and missions were combined whenever possible. Regional resources were leveraged for knowledge exchange between SEMCAR countries in areas where a country was more advanced than the others. The Steering Committee Meetings also served a dual purpose of defining the evolution of the program, and as a venue to share country experiences. 75. Cost efficiency considerations were also central to the design and execution of high value contracts. Significant savings were achieved by the task team in the negotiation of a contract to gather requirements for a new tax system, one of the main activities of the Program. It was estimated at concept Page 29 of 82 The World Bank Caribbean - Supporting Economic Management in the Caribbean Program (P123665) that this activity would cost US$1.2 million. The task team negotiated a contract for US$0.8 million, which included requirements gathering for a new tax system, but also for the integration of PFM and revenue administration in ten countries.30 76. Avoided Costs by Anchoring on Diagnostics. By anchoring the activities on diagnostics, it helped ensure that SEMCAR Program funds were properly utilized. For example, by conducting current state assessments on ICT before moving forward with pursuing ICT solutions, the Program was able to avoid wrongly investing in systems integrating current budget execution systems with the tax administration systems as these were later found to be outdated. 77. Avoided Costs from Applying a Regional approach. Adoption of a regional approach yielded significant savings with respect to the time and cost associated with introducing reforms. For example, a substantial amount of time was devoted to developing new PFM laws in Grenada. Yet, it took substantially less time to draft similar legislation for St. Lucia and Belize. This is largely because knowledge from Grenada’s experience was utilized to reproduce legislation across countries. Likewise, for reforms in revenue administration, while it took years for Jamaica to develop and implement strategic and risk management frameworks on its own, it took only a few months to develop generic regional frameworks which were then adapted and applied in several other countries. 78. Despite the absence of robust mechanisms to measure the efficiency of SEMCAR, the technical assistance provided through the program has set the countries in a trajectory that will allow them to reap efficiency gains in key areas related to public sector management. Given the above discussions, the efficiency rating is deemed to be Modest. D. JUSTIFICATION OF OVERALL OUTCOME RATING 79. The overall outcome rating is rated Moderately Satisfactory. Relevance and efficacy are rated Substantial. However, efficiency is rated Modest. E. OTHER OUTCOMES AND IMPACTS (IF ANY) Gender 80. As highlighted in the PMF matrix, capacity building on gender responsive budgeting was included as one of the activities of the SEMCAR program. In that regard, a workshop on budgeting was jointly delivered by SEMCAR and CARTAC (April 2-4, 2014) with sessions aimed at raising awareness on gender- responsive budgeting. Seventeen participants from seven SEMCAR countries benefited from workshop deliberations. A proposal to conduct a gender focused study on tax compliance was proposed by Barbados. However, the study could not be conducted due to basic data challenges (for example, difficulty obtaining accurate information regarding the amount owed by taxpayers). 30 This translates to a marginal cost per country of $47,000 in producing the business requirements for a new tax system which is a significant savings when compared to what Jamaica spent (up to US$300,000 in requirements gathering for a similar system) Page 30 of 82 The World Bank Caribbean - Supporting Economic Management in the Caribbean Program (P123665) Institutional Strengthening 81. Institutional strengthening was one of the main objectives of the SEMCAR program. SEMCAR was to build the capacities of government officials working in the areas of tax, customs, and PFM and provide them assistance to be more efficient and effective in their day-to-day responsibilities. Mobilizing Private Sector Financing 82. The program was not designed to mobilize private sector financing. Poverty Reduction and Shared Prosperity 83. The program was not designed to have direct impacts on poverty reduction and shared prosperity. That said, having a more efficient and effective public resource management is a fundamental ingredient to sustain economic growth, reduce high unemployment and poverty rates, restore fiscal and debt sustainability, diversify the economy, and secure a more sustainable external position, which were key challenges for economic management in the Caribbean when the program was designed. Other Unintended Outcomes and Impacts 84. The governance structure of the SEMCAR program promoted regional ownership and alignment with countries’ priorities. As noted earlier, the Trust Fund was designed to have a Steering Committee comprised of representatives from each of the beneficiary countries and international partners (Canada as the donor, and the IMF and the World Bank). Annual Work Plans had to be agreed by consensus, which meant considering individual country priorities, but collectively agreeing on which activities to support as a program. This arrangement led to the strategic allocation of resources, as well as the provision of resources to support emerging priorities. 85. Another positive outcome of the SEMCAR program was improved coordination, not only among beneficiary countries, but also among international organizations and regional actors. The Steering Committee meetings served as a convening space for beneficiary countries, the Government of Canada, the IMF, and the World Bank to review progress on stated objectives, discuss priorities, and share knowledge and country experiences. Representatives from the ECCB, European Union, OECS Secretariat, and the University of the West Indies (UWI) were invited to some of the Steering Committee meetings as observers and participated actively during deliberations. Ten Steering Committee meetings and one High- Level meeting were held during the length of the program. 86. Activities undertaken under the program also had positive spillover effects in other World Bank Group operations in Jamaica and the Caribbean. In the case of Jamaica, the institutionalization of the risk management framework in customs complemented World Bank Group efforts to support trade facilitation reforms. Achievements in risk management are also being leveraged under the Trade Logistics in the Caribbean project. Through the Caribbean Growth Forum, which was launched in 2012, tax reforms supported by the SEMCAR Program were prioritized in Jamaica. The Jamaica Business Taxation Reform Project (JAMTAX) and the Caribbean Business Taxation Reform Project (CARIBTAX) are other operations, Page 31 of 82 The World Bank Caribbean - Supporting Economic Management in the Caribbean Program (P123665) funded by Trust Funds from the International Finance Corporation (IFC), that also had synergies with SEMCAR activities. III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME A. KEY FACTORS DURING PREPARATION 87. Complexity. The Program as designed was to operate within a very complex environment that could undermine its performance and value-added: (i) addressing multiple reform areas (tax, customs, PFM, and ICT) that by themselves are complex; (ii) targeting 12 beneficiary countries with absorptive and implementation capacity challenges and varying priorities and at different stages of reforms; and (iii) working with multiple stakeholders/agencies within the countries. To address these challenges, the Program - guided by the concept review meeting and in consultation with the donor and beneficiary countries - targeted its specific interventions to selected countries (not all 12) based on demand to avoid spreading itself too thinly. And in terms of the nature of technical support, the concept review meeting mandated the team to focus the Program mainly on providing the requisite TA to prepare countries to modernize their tax, customs, and PFM systems and that this will be achieved through business process re-engineering and institutional strengthening/capacity building and not implementing of ICT investments and to manage expectations of donors and beneficiary countries in this regard. 88. Ambition. Against the complexity discussed above, the Program articulated an ambitious and broad development objective and at concept promised to deliver within a three year time frame. The ambition in the development objective was tempered through the PMF with more realistic results indicators. On the other hand, the Program could not be delivered in the original plan of three years – actual implementation period was double the projected implementation period and necessitated four extensions. Underpinning this actual longer implementation was the time it took to build consensus for regional integration/approaches which was at the core of SEMCAR. As such, on top of rigorous diagnostics, continuous consensus building became a critical responsibility for the Program during implementation. B. KEY FACTORS DURING IMPLEMENTATION 89. Regional Approach Mitigated Potential Risk to Establishing Regional Synergies. The primary risks identified at design were the lack of political will among beneficiary countries to tackle a focused regional agenda on tax, customs, and public financial management issues as well as the difficulty in coordinating reforms and system improvements across countries and line ministries to reap regional synergies. Cognizant of its complex operating environment, SEMCAR approached establishing regional synergies systematically. First, it conducted diagnostics and assessments that helped identify good practices in SEMCAR countries. Second, it used these good practices as reference in developing regional frameworks/approaches to solve common challenges among the SEMCAR countries. Third, the regional frameworks were then presented to the Steering Committee to be endorsed as common frameworks which are customizable for each country to ensure their relevance at their level. Fourth, countries then volunteered to be the pilots and as success and lessons learned surface, adoption by others was facilitated. This led for example to the adoption of common/regional approaches on strategic and risk management for tax and customs administration and solution systems for tax administration and IFMIS. Page 32 of 82 The World Bank Caribbean - Supporting Economic Management in the Caribbean Program (P123665) This approach facilitated increased knowledge sharing and strengthened consensus building among SEMCAR countries. However, this process takes time to reap the benefits. For example, while the regional strategic management framework was endorsed in October 2011 and recognized as a critical element for pursuing tax and customs administration reforms, the process of institutionalization remains a work in progress for the countries (i.e., as of 2016, six started institutionalization against a target of three under SEMCAR) and is being continued under CARTAC with the closure of the SEMCAR Program. 90. Demand Driven Activities May Seem to Diffuse Results & Focus but They Help Ensure Success. SEMCAR was a programmatic engagement with articulated and approved general parameters, but its work program was approved and determined annually by the Steering Committee based on beneficiary countries’ priorities. As such, activities were subject to change and may seem to lack focus or diffuse results as the interventions may differ significantly from country to country depending on their priorities, capacities and starting points and thereby reaping different results (e.g., support to AGD in Nevis resulted in more efficient budget execution while that in St. Lucia resulted in improvements in cash management). However, they are guided overall by the three levels of outcomes in the logic model. On the other hand, while it was not identified as an original activity, technical support for improving corporate governance and fiscal risk management of State-Owned Enterprises was identified as a priority for support by the beneficiary countries. And as a manifestation of their commitment, they actively participated in all the activities pursued in this area. In this regard, COUSINS not being an organic request from the beneficiary countries could have contributed to its demise. The introduction of COUSINS may have been a good idea but it took some attention away from priority concerns of the beneficiary countries. As such, after being endorsed in the May 2013 Steering Committee Meeting, it was put on hold by the Steering Committee in November 2013 as they were more concerned with stabilizing their current systems. 91. Complex Administration and Program Management Arrangements Was a Challenge. As described in the Components section, program activities were jointly executed by the IMF and the World Bank with some activities pursued by the Bank under the Recipient-Executed Trust Fund arrangements (see Annex 9 for detailed information on the Trust Fund structure and division of labor between the IMF and the World Bank). Administration of the Trust Fund necessitated transfer of funds under a Financial Intermediary Arrangement whereby the IMF is not obligated to report to the World Bank. This structure resulted in this completion report’s inability to better reflect on the Project’s efficiency as no financial reports are provided to the Bank as per the legal arrangement with the IMF. Moreover, coordination issues surfaced at the beginning as is common in projects that are executed by multiple agencies that have their own institutional practices. However, this was mitigated early thanks to the collaborative approach taken by the IMF and World Bank team. Collaboration measures were adopted including: working on a six-month work plan, centralizing communication with the beneficiary countries through the SEMCAR e- mail account, carrying out periodic meetings between the two organizations to plan and coordinate activities, conducting joint missions and rotating responsibilities for leading assessments and technical missions, and providing systematic inputs to the Progress Reports based on the PMF. These established a positive relationship between the two organizations that enabled successful joint delivery of the Program. Within the Bank, initially activities where being executed by two teams - the Public Sector with respect to the technical areas of tax, customs and PFM while the ICT team was focused on ICT solutions. As noted in SEMCAR’s Mid-Term Review (MTR), the bifurcation of program management in the World Bank contributed directly to delays and aborted initiatives in ICT and training. This was later addressed by consolidating program management and leadership in the Public Sector team which then drew expertise from the ICT team as necessary. Page 33 of 82 The World Bank Caribbean - Supporting Economic Management in the Caribbean Program (P123665) 92. A related challenge is personnel turn-over within the World Bank and the IMF. The WB and the IMF faced several personnel turnovers since the beginning of the program design. From the World Bank side alone, task team leadership for the overall program changed four times since program design. While turnover of personnel is a common constraint in almost all projects in any multilateral organizations, it was necessary to strengthen the institutional capacity of the project management and prevent key activities from becoming dependent on one person. In this regard, measures were taken by documenting critical processes to facilitate knowledge transfer, carrying out periodic meetings, and delegating managerial activities to different members of the SEMCAR team. 93. Intensive Supervision is Critical to Support Implementation Capacity Constraints. While there has always been a recognition of limited capacities of potential recipients/implementing agencies of RETF grants, this could be mitigated by adequate supervision on the part of the Bank. In this regard, the first RETF which was to result into a maintenance contract for SmartStream could have been successfully executed had there been more proactive implementation support on the part of the Bank. While there were issues on the part of the beneficiary countries that may have prevented them from entering into a contract (e.g., using different versions of SmartStream and uncertainty of funding for the contract), it is noted that in the Grant Reporting and Monitoring (GRM) report for the RETF that for the most part the Bank’s implementation support was limited to reviewing reports submitted. On the other hand, benefiting from intensive preparation and implementation support, the second RETF was successfully implemented in less than a year with 82 percent disbursement in spite of involving complex procurement packages. 94. Challenge of Sequencing Technical Assistance and ICT. According to the World Development Report on Digital Dividends, maximum impact could be gained from digital technologies if the institutions are strong but it does not mean that digitally enabled reforms need to wait for country’s institutions to improve as digital technologies can also strengthen institutions. It is however critical to understand the entry points given the varying interactions and heterogeneity of institutions and digital technology. In this regard, while the mandate from the concept review meeting was to lay the foundations through TA before pursuing ICT, the implementation revealed that TA needed to have more information and this can only be obtained if a certain level of ICT systems are put in place. As per a Steering Committee representative: “Some of the benefits of implementation of strategic management framework has gotten impeded by the poor IT systems that the countries have. Getting information from IT systems is a challenge. You need data to make decisions.� 95. Mid-Term Review. In May 2013, the SEMCAR Steering Committee requested that a Mid-Term Review be conducted of the SEMCAR Program’s performance and results. The Review judged the program overall as Moderately Satisfactory. The Program remained highly relevant to clients. Advances made in several countries in terms of changes in public financial management policies and procedures, plus improvements in risk and strategic management introduced in tax administration and customs appeared to be real and sustainable. The quality at entry of the SEMCAR program was judged in the Review to be Moderately Unsatisfactory, reflecting shortfalls in risks assessments, program preparation funding (a major factor), articulation of performance targets, program financing levels, and program management structure. These deficiencies posed significant challenges to the World Bank and IMF teams during implementation, requiring remedial actions. The bifurcation of program management in the World Bank contributed directly to delays and aborted initiatives in ICT and training. World Bank performance was Page 34 of 82 The World Bank Caribbean - Supporting Economic Management in the Caribbean Program (P123665) judged Moderately Satisfactory. IMF performance was judged as Satisfactory. A beneficiary survey was undertaken as part of the Review and ratings were highly positive with 98% of respondents indicating that the donor was getting value for money. 96. The Review provided key recommendations for strengthening SEMCAR, which included among others, maintaining the SEMCAR Program Manager in the region; appointing a Regional ICT Advisor as soon as possible; proactively using the recently formed Regional ICT Implementation Team; appointing a Training Coordinator to oversee the training component development; strengthening communication and coordination between IMF and World Bank teams; piloting e-learning events in high priority areas such as SIGTAS; and designating an IMF Program Manager to oversee its SEMCAR activities. The Review indicated that while the startup costs of the innovative SEMCAR Program had been real (requiring a learning curve for the nearly unprecedented initiative in World Bank/IMF collaboration), mid-course corrections, remaining available time and resources, and continuing strong commitments from clients held out the likelihood of substantial additional gains. Some of the recommendations from the Mid-Term Review were adopted by the World Bank and IMF teams.31 IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME A. QUALITY OF MONITORING AND EVALUATION (M&E) M&E Design 97. At design, SEMCAR’s theory of change/logic model was clearly articulated with its development objective stipulated in the Administration Agreement. The Program articulated a very ambitious development objective against a very complex operating context: (i) addressing multiple reform areas (tax, customs, PFM, ICT) that by themselves are complex; (ii) targeting 12 beneficiary countries with absorptive and implementation capacity challenges and varying priorities and at different stages of reforms; and (iii) working with multiple stakeholders/agencies within the countries. This challenge was recognized at the outset. To manage expectations against an ambitious development objective that was formulated broadly, the Program Document acknowledged at design that its result-based M&E framework is indicative and is designed to overcome a number of challenges in: (i) providing flexibility given that indicators may not all be applicable to all countries that are at different in their reform programs; (ii) selecting indicators that capture expected sequencing and phasing of activities within and between countries and at regional levels; (iii) identifying indicators that are sustainable, measurable and informative for all countries; and (iv) providing specificity of indicators given that interventions under the Program will inform specific activities to be implemented in each country and therefore would have 31While a SEMCAR Program Manager was not maintained in the region, a Regional ICT Advisor was appointed and served from Barbados. A Training Coordinator was never appointed, but capacity building remained a key aspect of the program until closing. A one-week workshop and e-learning course on SOE was piloted with the University of the West Indies. An IMF Program Manager to oversee SEMCAR activities was not appointed, given the role of CARTAC. The IMF’s Fiscal Affairs Department was in charge of overseeing both CARTAC and SEMCAR activities pertaining to the IMF. Coordination with the IMF improved with the development of a coordination mechanism, as well as quarterly meetings to review activities and avoid overlaps in thematic areas. To address delays in ICT investments highlighted in the Mid-Term Review, PWC was hired to develop a Regional ICT Strategy, which included in-depth assessments of ICT capabilities and platforms for each of the beneficiary countries and the development of a roadmap towards regional ICT standards. Page 35 of 82 The World Bank Caribbean - Supporting Economic Management in the Caribbean Program (P123665) different results. Against this context, the Program Document only presented the logic model and the desired outcome areas sans indicators. It intended to revise/develop the M&E framework with indicators once specific interventions have been identified and with a proposal to evaluate once activities are completed to better articulate the results. M&E Implementation 98. As per instruction during the concept review and as provided for in the Program Document, the results/PMF framework for the program with indicators was developed in consultation with the Steering Committee, agreed with CIDA in March 2013 and finalized in the Second Year Progress Report. Against the perceived broadness of the development objective, following the Bank’s ICR guidelines and consistent with Canada’s Results-Based Management Guidelines, the intended objectives were clarified and translated into expected measurable outcomes in the PMF and were inferred based on the intentions of the Program vis-à-vis the sector challenges and the intervention areas. Once the PMF was adopted, it was then regularly and closely monitored, and reported in the annual progress report and presented to the Steering Committee for review. 99. In addition, the indicators selected in the PMF were deliberately output focused (e.g., number of countries that institutionalized the SMF vs. number of countries with performance-oriented tax administrations) and with less number of beneficiary countries for each outcome area to manage the ambition vis-à-vis complexity as discussed above. As a consequence, while reporting on the outputs as per the PMF was satisfactory, collection of evidence on how these outputs/PMF indicators are leading to results on the ground was less systematic and was also complicated given the different starting points and priorities of the countries that were targeted. In this regard, the SEMCAR’s MTR could have been the opportunity to implement the proposal in the Program Document to revisit the indicators and articulate results better based on activities being implemented and/or completed and the countries targeted. Nevertheless, during Steering Committee Meetings, presentations on results being achieved and lessons learned by each country following interventions from SEMCAR are being shared and later reported in the Progress Reports which allowed for assessment of achievement of objectives in this completion report. M&E Utilization 100. Reporting on progress against the PMF was systematically and thoroughly reviewed during Steering Committee Meetings and has been used to inform changes in approaches and/or where intensive focus is necessary to ensure outcomes are achieved. For example, the long delay in the achieving the targets for the indicators related to the regional learning and ICT support center led to changing the approaches (from physically establishing a learning center) towards considering other solutions (e.g, virtual) and eventually to the piloting of a hybrid solution that are tapped on existing regional models/institutions (e.g., University of the West Indies). The SEMCAR indicators were also constantly being reviewed for relevance against all other similar initiatives in the regions and countries to avoid wasting limited resources in SEMCAR. This led to dropping the immediate outcome indicator related to a post clearance audit program and specific support activities planned in this regard. Justification of Overall Rating of Quality of M&E Page 36 of 82 The World Bank Caribbean - Supporting Economic Management in the Caribbean Program (P123665) 101. For the reasons outlined in the previous paragraphs, the overall rating for quality of M&E is Substantial as the shortcomings of an ambitious and broad development objective were proactively addressed following Bank and Canada guidelines and enabled adequate progress reporting and capture of evidence of results achieved within a complex operating environment. B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE 102. Environmental and social safeguards. The program is primarily executed by the Bank and the IMF and as such did not trigger safeguard issues. Nevertheless, within the program efforts were made to consider environmental considerations on taxation. 103. Financial management. There was a component that was recipient-executed which had challenges on financial management, but these were overcome in the end. 104. Procurement. There was a component that was recipient-executed which had challenges on procurement. Except for an instance of a failed procurement process as part of the grant given to Antigua and Barbuda, these were overcome in the end. 105. To address challenges with fiduciary arrangements during the second recipient-executed operation under the Trust Fund, the World Bank team organized a regional knowledge sharing workshop in Washington, DC with the three Project Implementation Units, which took place February 27-March 1, 2017. The sessions focused on fiduciary and legal arrangements, as well as and technical and operational aspects. Hands-on support was provided for on-going procurement processes. C. BANK PERFORMANCE Quality at Entry 106. The Bank’s preparation was well-founded on diagnostics. The design of the SEMCAR program was informed by a study financed by CIDA that highlighted the need for ICT investments and policy reforms to address deficiencies that were limiting the efficient mobilization and administration of public resources in the Caribbean countries. In this regard, the Bank management mandated the SEMCAR to focus the Program on laying the foundations for ICT investments that management emphasized will need to be financed by follow-up operation and that the Program be driven by country demands. This informed the development of the results/PMF matrix and the set of activities that were pursued and led to the achievements of the outcomes discussed above. 107. However, on hindsight, the Bank could have done better at entry. The most significant shortcomings on the part of the Bank at design stage has been in under-estimating the implementation period and arrangements for a complex operation that was promoting regional cooperation and not pursuing a more focused (i.e., it could have pursued one instead of all three technical areas). While the Program’s ambitiousness and complexity were noted at concept review meeting and the team addressed these during implementation, it led to delayed implementation necessitating several extensions. Moreover, while the Program substantially achieved its development objective, there is perceived diffusion of results achieved given the differing starting points and priorities of the various countries that Page 37 of 82 The World Bank Caribbean - Supporting Economic Management in the Caribbean Program (P123665) requested for support; whereas if it focused on one technical area, it could have demonstrated a more concentrated, rich and obvious set of results (e.g., deeper advances in tax administration which remains a high priority for the countries and is at the core of their request for a follow-up operation). Furthermore, collaboration protocols could have been discussed and developed upfront with the IMF given that legally and institutionally the arrangements with the IMF would not allow the Bank to oversee its work. Quality of Supervision32 108. At implementation, the Bank together with the Steering Committee and Canada adopted the PMF to clarify the expectations on results under the Program to manage expectations against the ambitious and broad formulation of the development objective. While the Bank team remained responsive to changes to circumstances during implementation, it was always focused on monitoring progress against the articulated expected results in the PMF and safeguarded guidance from Bank management with respect to the extent of intervention available under the Program (i.e., no ICT investments). With respect to the challenge on implementation arrangements, it established collaboration protocols with the IMF and hand-over and knowledge management protocols within the Bank teams to ensure continuity given the extensions against movements of staff as is natural in the institution. The Bank was agile and flexible such that shortcomings at design were for the most part addressed at supervision/implementation stage to ensure that the achievement of desired results are not compromised. 109. With respect to the Bank role of providing implementation support to RETF grants, the experience has been mixed but compared to resources involved was satisfactory. As discussed in the section on Key Factors Affecting Implementation, the supervision of the first RETF (involving US$120K) was less than proactive that led to an unsuccessful grant while the intensive preparation and implementation support provided under the second RETF (involving US$2.95M) enabled substantial achievements within less than a year of implementation and exiting with an 82% disbursement rate. Justification of Overall Rating of Bank Performance 110. For the reasons discussed in the previous paragraphs, the overall rating for Bank performance is Moderately Satisfactory. D. RISK TO DEVELOPMENT OUTCOME 111. At program completion, the risk to development outcome is Moderate. Reforms undertaken with support from the SEMCAR Program continue to be relevant. Achievements during the program are also feeding into other initiatives led by the countries themselves or with the support from other donors. In particular, CARTAC is continuing to provide support in the implementation of strategic and performance management systems33 that were identified in SEMCAR’s regional strategic management framework. The United Kingdom’s Department for International Development and the European Union are also providing further support on PFM reforms picking up from SEMCAR. In addition, a new operation that builds on the 32Under SEMCAR, the role of the Bank is primarily as an implementing entity and secondarily as provider of implementation support to RETFs. As such, its performance is being assessed here for both roles. 33 CARTAC Annual Report, July 2016. Page 38 of 82 The World Bank Caribbean - Supporting Economic Management in the Caribbean Program (P123665) results of the SEMCAR Program is under discussion with the Government of Canada through Global Affairs Canada (GAC) with a focus on improving tax administration systems and with strong interest from beneficiary countries.34 SEMCAR laid a strong foundation for public sector management in the Caribbean. The outcomes of this intervention are fundamental to inform the next steps. However, beneficiary countries might be sidelined by emergencies as a result of natural disasters (as is presently the case of Dominica and Antigua and Barbuda), which are common in the Caribbean, but the structural issues remain relevant for their own development goals. In this regard, the Bank with funding from Canada, will also be pursuing through an Externally Funded Output technical assistance on mainstreaming climate change and disaster resilience in PFM. V. LESSONS AND RECOMMENDATIONS 112. The following lessons learned and recommendations from the program are intended to inform future World Bank Group operations. 113. Success Factor for a Regional Solution: Thinking Regionally and Delivering Customized Solutions. Pursuing regional solutions is possible but it takes time and a systematic approach is required. SEMCAR’s “magic� formula involves the following sequence: (i) conducting assessments and identifying good practices in the region; (ii) transforming these good practices into regional frameworks in approaching common challenges; (iii) consensus building to secure endorsement by the country leaders/Steering Committee for the regional framework; (iv) piloting of the frameworks that allow for customization depending on country circumstances; (v) capturing successes and lessons learned from pilots; and (vi) replicating based on successes and lessons learned from pilots but always allowing for customization to ensure country relevance and ownership. In this regard, while there are common issues in tax, customs, and PFM across the region, the fact that each country is at different stages of reform, with specific assistance needs in different aspects of tax, customs, and PFM administration, SEMCAR’s lessons learned stresses the need to develop and implement specific, customized solutions to address the issues but anchored on common principles/framework. There is a strong case for continuing to promote economies of scale particularly in the design and procurement of specialized ICT solutions in these areas. At the same time, these solutions, and the roadmap for implementing them, will have to be flexible enough to accommodate the different stages of progress and capacity in each beneficiary country. 114. Manage Complexity with Less Ambition, Focus, Realism and Adequate Preparation Funding. Against a complex operating environment, to demonstrate a clearer and richer result (and avoid perception of diffused results or worse less than satisfactory value-added), SEMCAR should have articulated a narrower development objective that can be clearly attributed to the actual interventions and focused on one area of public resource management challenge. Likewise, to manage expectations on the Program, SEMCAR should have been more realistic and addressed at-entry the challenges of the implementation timeline and arrangements to ensure efficiency which could have been addressed by adequate preparation funding – a shortcoming noted in the MTR. 34As an example of country demand, the Government of Saint Lucia submitted a letter to the World Bank Regional Director for the Caribbean in February 2017 expressing appreciation for the support received during the SEMCAR program and indicating the government’s willingness to participate in a follow-up operation. Page 39 of 82 The World Bank Caribbean - Supporting Economic Management in the Caribbean Program (P123665) 115. Recipient’s Limited Capacity Always Demands More from the Bank. The Recipient Executed Grants’ preparation and implementation were delayed by limited capacity on the ground to handle financial management and procurement requirements under World Bank rules and procedures. Such situations always demand greater proactivity and intensive preparation (including conduct of a launch workshop that provides training on applicable Bank policies and procedures) implementation support from the Bank. For example, the second RETF required increased and more timely communication, trust building, and intense follow up by the team with all the different counterparts which paid off facilitating a disbursement rate of eighty-two percent. Page 40 of 82 The World Bank Caribbean - Supporting Economic Management in the Caribbean Program (P123665) ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS This Annex presents the Performance Measurement Framework Matrix for SEMCAR with information on results achieved by program completion (August 31, 2017). Results indicators following the new ICR template are provided for the Recipient Executed Trust Fund only. A. PERFORMANCE MEASUREMENT FRAMEWORK MATRIX ICR Cumulative Progress at Expected Results Indicators Baseline Targets Comment/Explanation Closing Ultimate Outcome ICT regional End of Phase 1: Target achieved – ICT Business and Technical requirements were defined to integration plan for ICT Integration regional integration plan for facilitate the adoption of a Regional Tax System in 7 Tax, Customs and Plan Tax, Customs and PFM countries (AB, BEL, DOM, GRE, SLU, SKN, and SVG). PFM systems delivered In addition, requirements were gathered to facilitate the Integration of Customs, Public Financial Management and Tax Systems in all 12 beneficiary countries. To this end, the following outputs were satisfactorily delivered: Ultimate Outcome (i) Business and Technical Requirements for both a New Tax System and a Common Tax System, and the Improved economic Integration of the Customs, PFM and Tax Systems; management, regional integration and (ii) Conceptual Report for the Integration of PFM, competitiveness in up to 0 Customs and Tax administration information systems; 12 Caribbean countries, through more efficient, (iii) Sustainability Strategy for the proposed ICT solutions; effective and accountable public institutions, (iv) Recommendations informing the development of policies, processes and Terms of References, Evaluation Processes, and ICT systems. Selection Criteria for evaluating RFPs to integrate PFM, Customs and Tax administration systems in the beneficiary countries; and (v) Recommendations informing the development of Terms of References and Evaluation Criteria for the Implementation of a Common Tax System in the targeted SEMCAR Countries Intermediate Outcomes Page 41 of 82 The World Bank Caribbean - Supporting Economic Management in the Caribbean Program (P123665) ICR Cumulative Progress at Expected Results Indicators Baseline Targets Comment/Explanation Closing TAX: The Regional End of Phase 1: At Target exceeded – Six (6) Utilizing a SEMCAR developed SMF assessment tool, Strategic Management least 3 countries countries institutionalized the countries that achieved at least a minimum score of 65 Model endorsed at the SMF (SKN, A&B, GRE, T&T, on a scale of 100 were considered to have SEMCAR workshop in DOM, and SLU), with some institutionalized the SMF. As of early 2016, six countries October 2011 is instances using talent from scored at least 65 percent (SKN, A&B, GRE, T&T, DOM, 0 institutionalized in at regional successes. and SLU). However, as demonstrated by a case study least 3 countries looking at the successes and lessons of SMF implementation, with the exception of GRE, interviewed tax departments experienced difficulties maintaining a passing score. Audit case selection is End of Phase 1: At Target largely achieved – Risk-based selection is increasingly being used in JAM risk-based in at least 3 least 3 countries Two (2) countries (DOM and (tax and customs); DOM (customs). with the approach countries. JAM) are fully utilizing risk also being adopted in SKN, SLU, and GRE. 0 based selection in both Tax and Customs departments. Intermediate Outcomes CUSTOMS: Support to End of Phase 1: At Target partially achieved – The development of a regional pre-clearance framework development of a least 4 countries Stage 1 of a 3-stage pre- was completed through visits to DOM, SKN, JAM, and More efficient, effective, preclearance clearance regional framework SLU. The readiness of selected countries to implement accountable and framework for air and 0 was completed with field visits pre-clearance of goods in Customs in the short to regionally integrated Tax, maritime cargo in at carried out in DOM, SKN, medium term was assessed. Customs and PFM least four countries JAM, and SLU. institutions, policies, PFM: Adequate legal, End of Phase 1: At Target largely achieved – GRE: A new PFM Act strengthening fiscal discipline was processes and ICT regulatory and least 5 countries New legislation drafted and prepared, passed (in August 2014), and implemented. systems. organizational enacted with varying degrees Related regulations to implement the new Act were also structures verified or in 3 countries. Extensive developed. A supplementary Fiscal Responsibility Bill established in at least assistance provided to was also developed and passed; it became effective 5 countries improve organizational January 2017. structures in 4 countries. 0 BEL: New PFM law was drafted. SLU: A similar Finance Act was drafted and is currently undergoing review. Improved organizational performance for the Accountant General Departments in SLU, BEL, SKN, and SUR. TRAINING & ICT: End of Phase 1– All Target partially achieved – The target of establishing a regional training center was Enabling environment SEMCAR Countries ICT environment was refocused. The originally envisaged ICT support center and inputs for large- 0 have access to the assessed and supporting aspect was dropped. The program instead focused on scale implementation ICT Training Center action plans were provided to piloting a hybrid training model offering face-to-face Page 42 of 82 The World Bank Caribbean - Supporting Economic Management in the Caribbean Program (P123665) ICR Cumulative Progress at Expected Results Indicators Baseline Targets Comment/Explanation Closing of integrated ICT (physical or virtual)address identified workshops with complementary virtual online classes for systems completed weaknesses. Countries were selected trainings. provided access to a pilot virtual training center via the University of the West Indies Immediate Outcomes Component 1: Tax Administration Updated End of Phase 1: At Target achieved - Six (6) Ten (10) of the 12 member countries have SMF implementation “action least 3 countries countries have presented implementation action plans, of which 6 achieved plans� in place to SMF implementation action significant progress in institutionalization by achieving the support and guide the plans and achieved the minimum implementation score of 65 percent. 0 institutionalization of minimum SMF assessment Beneficiaries include: SKN, A&B, GRE, DOM, SLU, and the regional strategic score of 65 percent. T&T management model in at least 3 countries. Risk based criteria for End of Phase 1: At Target largely achieved – A regional Risk Management Framework model was audit case selection least 3 countries DOM and JAM fully utilizing developed and implemented in five (5) countries. Risk established in at least 0 risk based selection in both Based Selection Criteria for audit case selection was for 3 countries. Tax and Customs tax and customs in: JAM, DOM, SKN, SLU, and GRE. 100 – Increased ability of departments. tax administration Tax compliance End of Phase 1: At Target partially achieved - DOM and JAM were projected to be fully utilizing risk institutions and staff to programs implemented least 3 countries Five (5) countries initiated the based selection and profiling in both Tax and Customs manage processes and 0 based on risk profiling implementation of upgraded departments. This approach was adopted with varying ICT systems in at least in at least 3 countries. tax compliance programs degrees of success in GRE, SKN, and SLU. five countries. Comprehensive End of Phase 1: At Target partially achieved - Based on the Risk Management Maturity Assessment service, education and least 3 countries Five (5) countries advanced Tool developed by SEMCAR, DOM and JAM were outreach programs implementation of predicted to have attained a ‘proficient’ rating in that developed based on 0 comprehensive service, category and GRE a competent rating. Beneficiaries of risk assessment education and outreach this activity include: JAM (tax and customs); DOM outcomes in at least 3 programs based on risk (customs); approach adopted in SKN (Tax), SLU and countries. assessment outcomes GRE (Tax and Customs) Institutionalization of End of Phase 1: Target achieved - Six (6) SEMCAR developed a SMF assessment tool to measure the SMF initiated in at Adoption of SMF in countries initiated utilization of the SMF. SKN, A&B, GRE, DOM, SLU, and least 3 countries. 0 at least 3 countries institutionalization of the SMF. T&T all achieved the minimum SMF assessment On a scale of 100 utilizing a threshold score of 65 percent in the process to SEMCAR institutionalize the SMF. Page 43 of 82 The World Bank Caribbean - Supporting Economic Management in the Caribbean Program (P123665) ICR Cumulative Progress at Expected Results Indicators Baseline Targets Comment/Explanation Closing Business requirements End of Phase 1: 12 Target achieved – Business The gathering of Business Requirements commenced for an enhanced tax countries and Technical Requirements initially using Grenada as a pilot to model a regional tax administration system for a regional Tax System system. However, this approach was later revised. In- developed. were developed; depth country gap assessments were carried out for each Recommendations to inform of the 7 countries using the SIGTAS system. Action plans 0 the development of Terms of (regional and national) were developed with References and Evaluation recommendations for convergence. The approach was Criteria for the endorsed by the Steering Committee and requirements Implementation of the system for a regional tax system were thereafter later defined. was also delivered Immediate Outcomes Component 2: Customs Management To support the End of Phase 1: At Target achieved – RMF DOM, SLU, SKN, and GRE were assessed to have development of an least 4 countries piloted in four (4) countries implemented an importer profiling framework. DOM importer profiling (DOM, SLU, SKN, and GRE) advanced on the implementation of the RMF which was 0 framework in at least 4 as basis for importer profiling necessary to develop the importer profiling framework. countries. framework. Options were presented to SLU, SKN, and GRE consideration. Post clearance audit End of Phase 1: At Target dropped - CARTAC Since this activity was being pursued by CARTAC in 200 – Increased ability of program supported in least 4 countries supported this initiative. several countries, it was dropped to avoid duplication. Customs administration at least 4 countries Instead, the SEMCAR team coordinated with CARTAC to institutions and staff to 0 follow-up on progress made to incorporate post- manage processes and clearance as appropriate into RMFs. CARTAC supported ICT systems in at least this initiative in SLU and SVG. four countries. Electronic manifest End of Phase 1: At Target achieved – Based in module in customs least 4 countries part on CATT outcomes, systems implemented SLU, JAM, SVG, T&T, and in at least 4 countries 0 potentially SUR have (or will) upgrade to ASYCUDA World to include use of the electronic manifest module. Immediate Outcomes Component 3: Public Financial Management 300 – Increased ability of Development of a End of Phase 1: Target achieved – SEMCAR identified technological and functional Ministries of Finance, line common framework for Common Framework and functional limitations preventing the integration of core and non- ministries and their staff functional framework requirements to integrate core core IFMIS components. A regional IT strategy and to manage PFM requirements of core developed for and non-core IFMIS country specific action plans were prepared to rectify 0 processes, including and non-core IFMIS functional components were fully issues. As a follow-up to these activities, Business and gender responsive components. requirements of delivered as May 2017. Technical Requirements for the Integration of the budgeting and ICT core and non-core Customs, PFM and Tax Systems were delivered along systems in at least five IFMIS components with recommendations informing the development of Page 44 of 82 The World Bank Caribbean - Supporting Economic Management in the Caribbean Program (P123665) ICR Cumulative Progress at Expected Results Indicators Baseline Targets Comment/Explanation Closing countries Terms of References, Evaluation Processes, and Selection Criteria for evaluating associated RFPs. Support for gender End of Phase 1: Target achieved – Training A regional workshop was jointly conducted by SEMCAR responsive budgeting. Number of people delivered on gender- and CARTAC (April 2014) on Gender-Responsive trained about responsive budgeting. Budgeting to build awareness. The workshop benefitted gender budgeting 28 participants representing 13 Caribbean countries (this (TBD) includes 17 participants from 7 SEMCAR countries). Common framework End of Phase 1: Target achieved – A The generic internal audit framework (which includes a for effective internal Common common framework was draft internal audit legislation, charter, and manual) was audit in an IFMIS framework developed for effective developed based on select country studies and environment designed developed for internal audit. Training was discussed with key stakeholders in a joint in at least 5 countries effective internal also delivered to 29 SEMCAR/CARTAC workshop attended by 65 audit in an IFMIS participants from 8 SEMCAR participants from 20 Caribbean countries from April 24- 0 environment. countries on various internal 26, 2013. In addition, the SEMCAR LTX participated in a Advisory and audit topics. CARTAC Internal Audit Workshop held in Barbados in technical services September 2014 that attracted 39 active participants from provided. 14 countries (including 29 participants from 8 SEMCAR countries) who (i) discussed current internal audit topics, (ii) engaged in training activities, and (iii) developed individual country action plans for internal audit. At least 5 countries End of Phase 1: At Target exceeded – Nine (9) A CARTAC workshop, which was accompanied by follow with the capability of least 5 countries SEMCAR countries (SLU, up TA in 2012, addressed much of the need in this area. producing functional SVG, SUR, SKN, BEL, A&B, A second practical workshop focusing largely on (COFOG and GFS) DOM, GRE, and JAM) have SEMCAR countries (5 of the 7 countries attending were view of budget and received direct support in SEMCAR) was convened jointly by SEMCAR/CARTAC in financial statement. relation to mapping to early March 2014. Each country made progress in GFSM2001 and COFOG. redeveloping its CoA and also developed an action plan SEMCAR provided direct for further reform and TA. SEMCAR provided targeted in- support to four countries country assistance to SUR, SLU, SVG, and SKN. SUR 0 (SLU, SUR, SVG, and SKN) and SLU implemented the revised CoAs. Work on new and the other 5 jointly with CoA advanced considerably in SKN and SVG advanced CARTAC and IMF. at the stage where there were plans for implementation in FY2016 budget. CARTAC supplemented this effort by providing CoA advice in BEL. New CoA structure which supports reporting against GFS and COFOG has been introduced in SLU, SVG, and SUR. Page 45 of 82 The World Bank Caribbean - Supporting Economic Management in the Caribbean Program (P123665) ICR Cumulative Progress at Expected Results Indicators Baseline Targets Comment/Explanation Closing As for SKN, a new CoA structure compliant with GFS and COFOG has been designed, populated, and tested in the budgeting, accounting and payroll systems for implementation in FY2017 budget. Support to producing End of Phase 1: At Target exceeded – Eight (8) A practical workshop on preparation of IPSAS compliant timely and least 5 countries countries (A&B, BEL, DOM, financial statements was convened in early March 2014 comprehensive in-year JAM, SKN, SLU, SVG, and jointly by SEMCAR/CARTAC. Some 21 representatives budget execution T&T) were supported to from 8 SEMCAR countries attended (A&B, BEL, DOM, reports in at least 5 produce IPSAS compliant JAM, SKN, SLU, SVG, and T&T). Each country made SEMCAR countries financial statements as a way progress towards producing financial statements to improve in-year reporting. compliant with the cash based IPSAS and also produced an action plan for further reform and TA. Direct support 0 and extensive TA were also provided to BEL, SLU, SKN, and SVG. In addition, a three-day workshop on Cash basis IPSAS was delivered in SKN in September 2015 at which 26 government staff attended. A two-day workshop on PEFA was also delivered to 32 government staff in SKN in August 2015. Immediate Outcomes Component 4: Regional Training and ICT Support Center Target partially achieved - A The target of establishing a regional training center was pilot of the regional learning refocused. The ICT support center aspect was dropped 400 - Improved access to End of Phase 1: At center was carried out. and a hybrid training model offering face-to-face and regional training and ICT Content and services least 12 courses SEMCAR collaborated with virtual online classes was pursued instead. SEMCAR technical support for Tax, provided by regional 0 developed by the the University of the West assessed the current state of learning facilities in the Customs and PFM training center. center Indies to develop, organize, region, which informed the pursuit and piloting of a hybrid administrations and deliver a regional face-to- training model. SEMCAR collaborated with the University face training (the "SOE of the West Indies and the Strengthening Distance Page 46 of 82 The World Bank Caribbean - Supporting Economic Management in the Caribbean Program (P123665) ICR Cumulative Progress at Expected Results Indicators Baseline Targets Comment/Explanation Closing Academy�) along with an e- Education in the Caribbean (SDEC) program to deliver learning course on the regional workshop and e-learning course on SOEs to "Strengthening Oversight promote knowledge exchange and share experiences Capacity and Corporate across and within countries. Governance of State-Owned Enterprises". Target partially achieved - As much as 56 officials from 10 SEMCAR countries Training was successfully participated in the face-to-face SOE Academy. End of Phase 1: At Actual training delivery delivered to 70 participants Furthermore, The University of the West Indies enrolled 0 least 100 people by the center. (56 face-to-face and 14 forty-four (44) participants to the online course. Of those trained virtually) by the University of 44, 21 logged into the course, and 14 successfully the West Indies achieved a Certificate of Participation. Target achieved - ICT environment across the Consensus reached on region was assessed and adopting regional ICT End of Phase 1: At roadmaps to achieve technical standards 0 least 4 countries regionally consistent and interoperability standards was presented to framework. and endorsed by the Steering Committee. Outputs Component 1: Tax Administration 110 – Virtual scoping Number of scoping Year 1: 12 Target achieved – 12 Assessment missions deployed detailed questionnaires assessments to assessments countries scoping assessments carried covering the areas of Tax, Customs, PFM and ICT to the determine the current out. 12 SEMCAR countries and followed up with virtual status of tax scoping assessments. The findings were utilized to clarify administration and ICT 0 the main needs of the beneficiary countries and to operations and level of develop the program’s annual work plans. commitment to the SEMCAR regional approach. Number of follow-up Year 1: One (1) Target achieved – LTA missions country carried out 17 follow up 120 – Follow-up assessments in 7 countries: Year 2: Four (4) assessments for tax GRE, SKN, GUY, SLU, T&T, 0 countries administration including A&B, and DOM ICT in at least 5 countries. Year 3: Five (5) countries follow- up 130 – On-the-ground 1. Regional advisor Target partially achieved: One (1) Regional Advisor was installed installed 0 technical assistance for Year 1: 1 Year 1: 1 Page 47 of 82 The World Bank Caribbean - Supporting Economic Management in the Caribbean Program (P123665) ICR Cumulative Progress at Expected Results Indicators Baseline Targets Comment/Explanation Closing the implementation of Year 2: 1 Year 2: 1 institutional modernization Year 3: 1 and restructuring of tax 2. Number of short- Target achieved – There SMF: (i) A High Level Meeting in 2012 targeted at administration (SMF and term expert (STX) TA were 30 short-term (STX) TA Ministers of Finance and other senior Finance officials RMF) procedures assignments planned assignments in total to was held to Secure commitment from the regional delivered; institutional and to institutionalize the institutionalize the SMF and leadership, to provide adequate additional resources, as legal framework SMF and RMF RMF necessary, to support a time-bound implementation of enhanced, including Year 1: 5 STX visits Year 1: 4 visits the SMF system; (ii) A regional SMF implementation incorporation of a gender Year 2: 7 STX visits Year 2: 9 visits review workshop to highlight progress across the perspective, in at least countries and “lessons learned� was held in July 2013; five countries. (iii) A Regional Change Management Workshop was held in January 2013; (iii) Strategic Management workshop 0 Year 3: 7 STX visits Year 3: 8 visits held in May 2014 for 12 countries to share experiences and “lessons learned� across the Region in as implementation of the SMF model continues. Approaches and achievements were shared among recipient countries with institutionalizing the SMF with particular focus on “Benefits of the SMECAR Regional SMF Model – country experiences. Year 4: 9 STX visits RMF: 8 STX visits conducted: Jamaica (2 – one of which was a follow up), Dominica (3—two of which were follow- ups), St. Kitts & Nevis, St. Lucia and Grenada. A regional RMF review workshop held in May 2014. 140 – User (internal and Business requirements Target achieved – Business external) business and technical requirements and technical requirements including a specifications specifications completed for detailed conceptual and completed seven (7) countries 0 functional design for a regional tax administration system completed. 150 – Detailed Request RFP’s are completed End of Phase 1: Target achieved – ICT for Proposals, including Complete environment assessed and technical specifications, to consensus building implementation roadmap 0 acquire or design a tax and launch RFP produced. Detailed inputs for software completed for at RFP drafted (Specifications, least 5 countries TOR, evaluation criteria, etc.) Page 48 of 82 The World Bank Caribbean - Supporting Economic Management in the Caribbean Program (P123665) ICR Cumulative Progress at Expected Results Indicators Baseline Targets Comment/Explanation Closing Options for End of Phase 1: Target achieved – Study on Information on environmental taxation has been gathered 160 – Study on feasible environmental taxation Proposals for feasible options for and analyzed. Best practices and options for options for environmental presented to policy environmental tax Environmental taxation was environmental taxation have been presented to the 0 taxation presented to makers presented to policy finalized in early 2015. Steering Committee. policy makers. makers Options were presented to SC in June 2015 Tools to facilitate End of Phase 1: Target achieved – Analytical 170 – Tools to enable voluntary compliance Service delivery work was conducted on voluntary compliance with particularly for women options to facilitate gender and taxation. Policy tax administration in small business, voluntary options were presented to requirements, with developed compliance by SEMCAR SC in December of 0 particular regard to the women in small 2014. A pilot execution of the needs of women in small business developed recommendation was businesses, developed in attempted in Barbados but at least five countries could not be completed because of data limitations. Outputs Component 2: Customs Management 210 – Virtual scoping Number of scoping Year 1: 12 Target achieved – 12 Assessment missions deployed detailed questionnaires assessments to assessments countries scoping assessments carried covering the areas of Tax, Customs, PFM and ICT to the determine the current out. 12 SEMCAR countries and followed up with virtual status of customs scoping assessments. The findings were utilized to clarify administration, ICT 0 the main needs of the beneficiary countries and to operations and level of develop the program’s annual work plans. commitment to the SEMCAR regional approach. Number of in-depth Target achieved – 7 Seven (7) in-depth assessments through the CATT were assessments and countries received in-depth conducted (BAR, SLU, JAM, SVG, SUR, T&T, and GUY). 0 Year 1: 1 country action plans and assessments, action plans Follow up TA was carried to address weaknesses follow-up missions and follow-up missions identified by the CATT assessments. 220 – In-depth assessments and follow- Year 2: 4 countries Risk Management workshop were held for 12 countries up action plans for to develop a regional risk management framework for customs administration both tax and customs administrations. Also, risk and ICT frameworks in at management framework (tax and customs) and model least 4 countries based Year 3: 5 follow- were developed; and simulated/tested in four countries— on CATT findings. up countries JAM, DOM, SKN, and GRE. Results of simulations using existing taxpayer databases (tax and customs) proved the relevance and need to embed risk-based approaches in day-to-day taxpayer/trader compliance management Page 49 of 82 The World Bank Caribbean - Supporting Economic Management in the Caribbean Program (P123665) ICR Cumulative Progress at Expected Results Indicators Baseline Targets Comment/Explanation Closing 230 – On-the-ground Short-term expert TA Target achieved - 14 STX RMF: 8 STX visits conducted: JAM, DOM (x2—one of technical assistance for missions to support 0 Year 1: 2 STX visits which was follow-up), SKN, and GRE. visits completed the implementation of implementation of risk institutional modernization management Year 2: 3 STX visits SMF: One (1) STX visit to JAM and 1 STX visit to SLU. through organizational programs. structures, legislation and Pre-clearance: Stage 1 of the pre-clearance regional procedures especially on framework completed. One (1) STX to DOM, SKN, JAM, risk management and SLU each. systems, and HR Year 3: 3 STX visits management including incorporation of a gender perspective, delivered in at least 4 countries. Information exchange End of Phase 1: Target partially achieved – Heightened interest was generated in five (5) countries to solution between tax Exchange of Regional framework in place adopt the regional framework (BAR, SLU, A&B, JAM, and 240 – Technical and customs information solution to promote integration SUR). assistance to support developed developed in-line between Customs and Tax integration between the with regional Administrations. DOM advanced implementation of the RMF to the extent tax and customs solution where they are exchanging information between tax and administrations, including customs based on the Taxpayer Identification Number. the exchange of 0 An MOU was drafted to facilitate information sharing information on revenue between tax and Customs. and taxpayers and declaration of customs Separately, an STX in April 2015 to Barbados provide TA cargo, delivered in at to assist the newly instituted Revenue Authority least 4 countries. streamline revenue administration, business processes and systems integration. Interactive tools (i.e. End of Phase 1: At Target partially achieved – Field visit to GRE Customs Department was conducted 250 – Effective feedback Web pages) are fully least 4 countries Framework for feedback to draft a regional approach to improve the feedback mechanisms between operational and useful. mechanism was delivered. mechanism between customs and stakeholders. The Customs and critical 0 GRE, JAM, and SLU made framework was delivered to GRE in December 2014. stakeholders to improve noteworthy progress. JAM improved its web page allowing feedback from Customs performance. general public and stakeholders. SLU upgraded its web page as well. 260 – Study on feasible Feasible options for End of Phase 1: Target achieved – Regional options for integrating integrating measures Measures on import study on feasible options for environmental measures to support clearance system integrating environmental 0 into customs environmental to support measures into customs administration completed preservation environmental administration was and presented to policy developed, and preservation completed. Measures to Page 50 of 82 The World Bank Caribbean - Supporting Economic Management in the Caribbean Program (P123665) ICR Cumulative Progress at Expected Results Indicators Baseline Targets Comment/Explanation Closing makers. presented to policy presented for policy support environmental makers. decision preservation were presented to SC in December 2014. Outputs Component 3: Public Financial Management 310 – Virtual scoping Number of scoping Year 1: 12 Target achieved – 12 Assessment missions deployed detailed questionnaires assessments to assessments scoping assessments carried covering the areas of Tax, Customs, PFM and ICT to the determine the current out. 12 SEMCAR countries and followed up with virtual status of PFM and ICT scoping assessments. The findings were utilized to clarify 0 reforms and level of the main needs of the beneficiary countries and to commitment to the develop the program’s annual work plans. SEMCAR regional approach. Number of in-depth Year 1: 3 in-depth Target exceeded - A total of Year 1: 3 in-depth (SVG, SLU, and BEL) + 1 PEFA 0 assessment and five (17) in-depth and three assessment (SVG). follow–up missions Year 2: 2 in-depth (17) follow-up assessments Year 2: 2 in-depth (SUR and SKN). In addition, 3 follow- were conducted. ups were conducted in Year 2 (SLU, BEL, and SVG). 320 – In-depth Year 2: 3 follow-up Year 3: 1 follow-up (SLU) and 2 HQ-led missions (SUR assessments of PFM and and GRE). ICT framework in at least Year 4: 1 follow-up mission to SLU 5 countries. In addition, in-depth ICT gap assessments with action plans were conducted for 11 countries between November 2014 and December 2015. Follow-up assessments were conducted for 10 of the countries to gather business and technical requirements. Resident advisor Target achieved Resident Advisor was in place for 2 years initially, from 330 – On-the-ground installed and tasks Year 1: 1 Year 1: 1 August 2011 to August 2013. A new Resident Advisor technical assistance for identified in the was installed on June 16, 2014 for one additional year. Year 2: 1 Year 2: 1 modernizing and scoping and 0 Part-time support undertaken as an interim measure restructuring of PFM Year 3: 1 Year 3: 1 assessment activities Year 4: 1 between August 2013 and May 2014. processes including of the program budget formulation, undertakes budget execution, Number of expert Target exceeded – At least accounting and internal assignments 83 expert missions completed audit delivered; undertaken as a result Year 1: 10 expert Year 1:10 institutions and legal of the tasks identified 0 frameworks modernized missions in the scoping and Year 2: 39 expert Year 2: 21 in at least 5 countries. assessment activities missions Page 51 of 82 The World Bank Caribbean - Supporting Economic Management in the Caribbean Program (P123665) ICR Cumulative Progress at Expected Results Indicators Baseline Targets Comment/Explanation Closing of the program Year 3: 32 Year 3: 17 expert Year 4: 20 missions IFMIS regional End of Phase 1: Target partially achieved – A follow-up grant was provided to St. Lucia for execution maintenance contract IFMIS regional IFMIS regional maintenance on the behalf of 5 member countries to acquiring 340 – Regional signed maintenance contract negotiated but was upgrades, licenses and hardware equipment to enhance maintenance contract for 0 contract signed not signed. SmartStream. Under the grant, St. Lucia implemented all IFMIS negotiated. envisioned project activities, with a disbursement rate of over 96% (USD 1.5m) Functional End of Phase 1: Target achieved - 350 – Common requirements for Delivery of Framework and functional framework for functional integration of core and functional requirements to integrate core requirements of core and non-core IFMIS 0 requirements for and non-core IFMIS non-core IFMIS components. integration of core components were fully components. and non-core IFMIS delivered as May 2017. In components addition, Reporting and data- End of Phase 1: Target not achieved – warehousing system Implement Specification for data and across the three main reporting and data storage systems were systems is warehousing gathered and documented implemented system in at least 5 with recommendations during countries the regional ICT assessment. Although Antigua &Barbuda 360 – Reporting and data- was provided with a grant to warehousing system procure Infrastructure across the three main 0 Equipment in Support of systems is implemented ASYCUDA World installation, in at least 5 countries. it was unable to complete the major ICB procurement processes to support the Project’s objective. This resulted in Project disbursement’s equaling 38% of total grant resources (USD 600k). Interfaces and End of Phase 1: Target not achieved – An STX visit to Barbados in April 2016 assisted the newly 370 – Interfaces and interoperability Implement Specification for interfaces instituted Revenue Authority to streamline revenue interoperability modules modules between the interfaces and and interoperability modules administration, business processes and systems between the three main 0 three main systems interoperability were gathered and integration. systems are implemented are implemented modules between documented with in at least 5 countries. the three main recommendations during the Page 52 of 82 The World Bank Caribbean - Supporting Economic Management in the Caribbean Program (P123665) ICR Cumulative Progress at Expected Results Indicators Baseline Targets Comment/Explanation Closing systems in at least regional ICT assessment. 5 countries Workshop and training End of Phase 1: Target achieved – Workshop Workshop on budgeting delivered jointly by SEMCAR 380 – Awareness created delivered to sensitize Number of people on gender-responsive and CARTAC (April 2-4, 2014), with sessions aimed at for improved policy makers on trained about budgeting delivered in April awareness building on gender-responsive budgeting. management and delivery 0 gender responsive gender sensitive 2014. Training delivered to 17 Seventeen (17) participants from seven (7) SEMCAR of gender-sensitive budgeting concepts budgetary planning officials from 7 SEMCAR countries benefitted from the workshop deliberations. budgetary planning. and tools (TBD) countries Chart of accounts End of Project: At Target exceeded – Nine (9) In addition, recipient-executed grants implemented by St. harmonized (as least 5 countries SEMCAR countries (SLU, Kitts & Nevis and St Lucia supported seven countries in needed) with UN SVG, SUR, SKN, BEL, A&B, the improvement of their Business Intelligence systems COFOG classification DOM, GRE, and JAM) have (i.e.Qlikview) or PFM systems (i.e. SmartSteam) and IMF GFS received direct support in economic classification relation to mapping to These critical updates facilitated the efficient GFSM2001 and COFOG. management of the CoA and other core PFM functions, SEMCAR provided direct as well as the production of requirements specification support to four countries documentation and work plans, definition of processes; (SLU, SUR, SVG, and SKN) development of templates; configuration of software; and the other 5 jointly with production of the environment set; testing of the systems CARTAC and IMF. and training of officers for both the BI solution and E-IRD 390 - Chart of accounts platform. harmonized (as needed) with UN COFOG 0 The critical updates for these systems included the classification and IMF purchasing of: (i) SmartStream Builder v8.0.04 Software GFS economic and Migration Licenses, upgrading of the SmartStream classification. Financials and HR/Payroll applications from SmartStream release 7.0.03 using Sybase Adaptive Server to SmartStream release 8.0.04using MS SQL Server. SmartStream Application Review and CPM Budget Implementation; (ii) Sybase PowerBuilder Enterprise 12.6; (iii) Microsoft SQL Server certification classes; (iv) Firewalls, Switches and other Infrastructure; (v) Servers; (vi) Oracle Licenses; (vii) software Licenses including Windows 2012 R2 Standard, Windows Server 2012, MS Office Professional 2013, SQl Server 2012, MS RDP Cal's, Windows Server CAL's; and (viii) Qlikview Licenses for BI implementation. Outputs Component 4: Regional Training and ICT Support Center Page 53 of 82 The World Bank Caribbean - Supporting Economic Management in the Caribbean Program (P123665) ICR Cumulative Progress at Expected Results Indicators Baseline Targets Comment/Explanation Closing 410 – Feasibility studies undertaken, and the institutional design and Regional feasibility Target partially achieved - A initial business plan study, institutional End of Phase 1: resident ICT Advisor was developed, for the design and initial 0 Selected ICT hired and based in Barbados establishment of a business plan Advisor hired. from October 2014 until close. sustainable regional completed training and ICT support center. Target achieved - Feasibility study was completed in End of Phase 1: September 2013. The Regional feasibility University of the West Indies study, institutional subsequently presented to design and initial the SC proposals for a hybrid business plan learning center. The UWI was presented to contacted to design and policymakers for develop materials from a pilot decision. face-to-face training (the SOE Academy) and e-learning course. 420 – Technical Core elements for End of Phase 1: Target largely achieved – The Regional ICT Strategy was successfully delivered, assistance for the interoperability Interoperability Core elements for with suggestions on how to improve interoperability development of regional frameworks for PFM, frameworks interoperability frameworks between Tax, Customs and PFM systems. Detailed tax and customs 0 delivered to developed using data for 10 technical requirements for a common approach towards ICT technical standards and interoperability delivered to decision- decision-makers in countries integrating the three systems in all member countries frameworks delivered. makers in 12 countries 12 countries. were also delivered. Number of people At least one per Target exceeded – Annual Trainings were provided annually to the benefit of all trained (including year workshops have been SEMCAR countries and more than 300 public officials. male/female ratio provided to the benefit of all ICT workshop on tax, customs, and PFM systems tracked and by At least one per member countries and more delivered in April, 2012 in Antigua and Barbuda - 50 component) component than 300 public officials. people from 11 countries attended the workshop, of 430 - Regional workshops Number of workshops / which about 46% were women. The learning facility being and training in tax, training events piloted was intended to document these workshops and customs and PFM 0 delivered trainings and then convert them to e-learning modules to systems delivered to Number of curricula improve retention and sustainability. Examples of recent users. developed trainings include: (Incremental targets (i) Project Coordinators’ from A&B, SKN, and SLU, along set for women in with key staff and Government officials representing the training and workshops Steering Committee attended the three-day regional based on % F/M staff) knowledge-sharing event focused on fiduciary training Page 54 of 82 The World Bank Caribbean - Supporting Economic Management in the Caribbean Program (P123665) ICR Cumulative Progress at Expected Results Indicators Baseline Targets Comment/Explanation Closing and Bank policies. (ii) Building on a broad set of activities supporting SOE reforms under the SEMCAR Program, which initiated in May 2015, the SEMCAR core team in collaboration with the World Bank (WB) Global Knowledge and Research Hub in Malaysia and the Government of Malaysia organized a knowledge exchange to support strengthening the capacity of Caribbean countries to assess and more effectively manage their State-Owned Enterprises (SOEs). The objectives of the Study Tour were: (i) to increase awareness of good practices in SOE management through a knowledge exchange designed to identify practical solutions and foster a dialogue between practitioners regarding on-going reform programs across participating countries, and (ii) to share lessons learned and notable innovations in Malaysia’s public sector management, including the delivery unit approach (PEMANDU) and the Malaysian model of one-stop shops. (iii) Sixteen officials from ten participating SEMCAR countries had an opportunity to learn from Malaysia’s PS modernization experience. Delegations from A&B, BAR, GRE, JAM, SVG, SUR, T&T, SLU, SKN, and DOM had an opportunity to learn from Malaysia’s National Transformation Program (NTP) that included both the government and the economic transformation, as well as strategic reform initiatives that cut across these relationships, such as the role of Khazana and the case of the one-stop shop UTC. Malaysia’s good regulatory practices that simplified a vast number of regulations and re-engineered business processes that made it easier for firms to interact with the state, focus on implementation, broad consultative processes and strong leadership to coordinate interactions between government agencies for better policy implementation by the Prime Minister’s Performance Management and Delivery Unit (PEMANDU). The delegations indicated their interest to continue and learn lessons in public sector modernization from other regions as well. Page 55 of 82 The World Bank Caribbean - Supporting Economic Management in the Caribbean Program (P123665) ICR Cumulative Progress at Expected Results Indicators Baseline Targets Comment/Explanation Closing ASYCUDA user group End of Phase 1: Target not achieved A Regional ICT Implementation team has been established ASYCUDA user established. This team included users of ASYCUDA, group established SmartStream and SIGTAS, and was used to coordinate the SIGTAS Enhancement Project, a SEMCAR Grant 0 that was implemented by SKN on behalf of up to 7 member countries that currently use SIGTAS. Trainings on Oracle database and a business intelligence tool for SIGTAS were completed. 440 - Assistance and Number of in-depth TBD Same as 430 support for ASYCUDA, technical SIGTAS and IFMIS workshops/training for administrator groups 0 system administrators provided. delivered across the three components Number of people At least one per Same as 430 (male/female ratio) component trained in technical 0 workshops for system administrators across the three components B. RESULTS INDICATORS B.1 PDO Indicators Note: The PDO Indicators that appear below correspond to the SEMCAR Budget and Revenue Systems Enhancement Project (P158239), which corresponds to the recipient-executed portion of the Trust Fund. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Countries with upgraded or Number 1.00 4.00 4.00 4.00 Page 56 of 82 The World Bank Caribbean - Supporting Economic Management in the Caribbean Program (P123665) modernized IT systems to 02-Feb-2016 31-Aug-2016 30-Apr-2017 enhance Tax Administration, Customs Management or PFM Comments (achievements against targets): Target Achieved. B.2 Intermediate Results Indicators Note: The Intermediate Results Indicators that appear below correspond to the SEMCAR Budget and Revenue Systems Enhancement Project (P158239), which corresponds to the recipient-executed portion of the Trust Fund. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Countries with tax systems Number 1.00 2.00 2.00 2.00 that can perform business analytics and generate 02-Feb-2016 31-Aug-2016 30-Apr-2017 performance management reports Comments (achievements against targets): Target Achieved. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Infrastructure equipment is Yes/No N Y Y Y Page 57 of 82 The World Bank Caribbean - Supporting Economic Management in the Caribbean Program (P123665) acquired to support 02-Feb-2016 31-Aug-2016 30-Apr-2017 implementation of the ASYCUDA World Customs Administration System Comments (achievements against targets): Target Partially Achieved. Some, but not all the infrastructure equipment was purchased before project closing. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Countries with updated FMIS Number 0.00 1.00 1.00 4.00 02-Feb-2016 31-Aug-2016 30-Apr-2017 Comments (achievements against targets): Target Achieved and Surpassed. C. KEY OUTPUTS BY COMPONENT Please refer to section A, Performance Measurement Framework Matrix of this Annex. Page 58 of 82 The World Bank Caribbean - Supporting Economic Management in the Caribbean Program (P123665) ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION A. TASK TEAM MEMBERS (WORLD BANK) Name Role Preparation Supervision/ICR Ruxandra Burdescu Task Team Leader (at closing) Davide Zucchini Task Team Leader (June 2014-May 2016) Kathy Lalazarian Task Team Leader (at approval) Juan Navas-Sabater Task Team Leader (design/at approval) Roberto Panzardi Task Team Leader (design) Sonia Cristina Rodrigues Da Fonseca Procurement Specialist John Oliver Moss Procurement Specialist Plamen Stoyanov Kirov Procurement Specialist David I Financial Management Specialist Eduardo Franca De Souza Financial Management Specialist Ximena Rosio Herbas Ramirez Environmental Safeguards Specialist M. Yaa Pokua Afriyie Opong Social Safeguards Specialist Adam Shayne Counsel Nightingale Rukuba-Ngaiza Counsel May Cabilas Olalia Team Member/ICR Contributing Author Fanny Weiner Team Member Kerry Natelege Crawford Team Member Richard Anthony Sutherland Team Member Francisco Lazzaro Team Member Montserrat Urquiola Ralero Team Member Diana Carolina Annandsingh Rattia Team Member Angela Nieves Marques Porto Team Member Eduardo Andres Estrada ICR Contributing Author Page 59 of 82 The World Bank Caribbean - Supporting Economic Management in the Caribbean Program (P123665) Deepak Bhatia Team Member Arturo Herrera Gutierrez Team Member/Practice Manager Svetlana Proskurovska Team Member Rohan Longmore Team Member Lord E. Walker Team Member Anat Lewin Team Member Murat Vardal Team Member Marcelo Buitron Team Member Onur Erdem Team Member Enrique Fanta Ivanovic Team Member Jose Eduardo Gutierrez Osorio Team Member Ana Mie Horigoshi Team Member Mather B. Pfeiffenberger Team Member Marcio Carcel Team Member Jenni Amanda Pajunen Team Member David Santos Ruano Team Member Jonna Maria Lundvall Team Member Elizaveta Perova Team Member Laura Eliza Aguirre Tellez Team Member Oscar E. Alvarado Team Member Francisco Galrao Carneiro Team Member/Program Leader Roland N. Clarke Team Member Cem Dener Team Member Gary J. Fine Team Member Svetlana Klimenko Team Member Alina Koenig Team Member Uttandaraman Pugalumperumal Team Member Josef S. Trommer Team Member Marinus Verhoeven Team Member Page 60 of 82 The World Bank Caribbean - Supporting Economic Management in the Caribbean Program (P123665) Lorena Vinuela Team Member The Bastian Wibowo Team Member Gabriel Yorio Gonzalez Team Member Jaime Zaldivar Team Member Marta Riveira Cazorla Team Member Leah April Team Member Henri Fortin Team Member Adrienne Hathaway Team Member Laura de Castro Zoratto Team Member B. TASK TEAM MEMBERS (INTERNATIONAL MONETARY FUND) Name Role Vinette Keene Senior Economist, Revenue Administration Sandeep Saxena Senior Economist, Public Financial Management Justin Zake Customs Component Team Lead Desmond Wyllis IMF Regional Tax Advisor for SEMCAR Julie Cooper Former Senior Economist, Public Financial Management Chris Isles Former Senior Economist, Public Financial Management Holger van Eden Former Deputy Division Chief, Public Financial Management/IMF Task Team Leader for SEMCAR Note: Attorneys from the IMF’s Legal Department drafted and reviewed the Letter of Understanding (LOU) with the World Bank to guide the rules for joint execution of the SEMCAR Program. C. STAFF TIME AND COST Staff Time and Cost Stage of Project Cycle No. of staff weeks US$ (including travel and consultant costs) Preparation FY10 0.00 0.00 Page 61 of 82 The World Bank Caribbean - Supporting Economic Management in the Caribbean Program (P123665) FY11 0.00 0.00 Total 0.00 0.00 Supervision/ICR FY11 24.49 71,297.29 FY12 117.99 363,116.36 FY13 150.38 662,175.66 FY14 180.72 736,532.12 FY15 131.58 438,113.48 FY16 103.70 415,493.01 FY17 131.13 587,417.57 FY18 67.19 336,034.92 Total 907.18 3,610,180.41 Note: Includes staff time and cost for the World Bank only. Page 62 of 82 The World Bank Caribbean - Supporting Economic Management in the Caribbean Program (P123665) ANNEX 3. PROGRAM COST BY COMPONENT A. SEMCAR BUDGET AS PRESENTED IN PROGRAM DOCUMENT Components Amount at Approval Amount at Approval (CDN$M) (US$M) Enhanced Regional Tax Administration 5.32 5.05 Enhanced Regional Customs Management 1.66 1.58 Enhancing Regional Public Financial Management 6.45 6.13 Establishment of a Regional Training and ICT Support Center 1.81 1.72 Project Management, Quality Assurance and Monitoring & Evaluation 3.96 3.76 Subtotal 19.2 18.24 Foreign Exchange Gains 1.24 Total 19.48 Note: The budget for each component was presented in the Program Document in Canadian Dollars. The equivalent amount in US Dollars is provided for reference purposes. The Revised Budget for the program, as presented in the Data Sheet, is equivalent to the amount at approval (US$18.24M) plus foreign exchange gains (US$1.24M), for a total of US$19.48M. Page 63 of 82 The World Bank Caribbean - Supporting Economic Management in the Caribbean Program (P123665) B. DISBURSEMENTS – WORLD BANK EXECUTED (BE) Components Disbursements at Program Closing (US$M) Enhanced Regional Tax Administration 0.550 Enhanced Regional Customs Management 0.783 Enhancing Regional Public Financial Management 2.159 Establishment of a Regional Training and ICT Support Center 2.784 Project Management, Quality Assurance and Monitoring & 1.284 Evaluation (includes Regional Project Coordination) Program Management 1.140 Total 8.700 C. DISBURSEMENTS – INTERNATIONAL MONETARY FUND EXECUTED (FE) Components Allocated Amount for FE Disbursements at Disbursements as a Executed Activities Program Closing Percentage of Allocated (US$M) (US$M) Amount (%) All Program Components 5.936 5.936 100 Total 5.936 5.936 100 Page 64 of 82 The World Bank Caribbean - Supporting Economic Management in the Caribbean Program (P123665) D. DISBURSEMENTS – RECIPIENT EXECUTED (RE) Components Allocated Amount for Disbursements at Disbursements as a RE Executed Activities Program Closing Percentage of Allocated (US$M) (US$M) Amount (%) Enhanced Regional Tax 0.620 0.598 96 Administration Enhanced Regional Customs 0.700 0.265 38 Management Enhancing Regional Public 1.625 1.568 96 Financial Management Total 2.945 2.431 83 Note: Enhancing Regional Public Financial Management includes allocations and disbursements under two Recipient-Executed operations. E. DISBURSEMENTS – ALL PROGRAM COMPONENTS (BE+FE+RE) Components Allocated Amount for Disbursements at Disbursements as a Program Activities Program Closing Percentage of Allocated (US$M) (US$M) Amount (%) All Program Components 19.484 17.067 88 Total 19.484 17.067 88 Note: Includes disbursements for all program components: Enhanced Regional Tax Administration, Enhanced Regional Customs Management, Enhancing Regional Public Financial Management, Establishment of a Regional Training and ICT Support Center, Project Management, Quality Assurance and Monitoring & Evaluation (includes Regional Project Coordination), and Program Management. Page 65 of 82 The World Bank Caribbean - Supporting Economic Management in the Caribbean Program (P123665) ANNEX 4. EFFICIENCY ANALYSIS For a discussion on efficiency analysis, please refer to section C on Efficiency. Page 66 of 82 The World Bank Caribbean - Supporting Economic Management in the Caribbean Program (P123665) ANNEX 5. BORROWER, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS Comments from the International Monetary Fund on the draft ICR were incorporated during the World Bank’s internal review process. The draft ICR was shared with beneficiary countries for comments on November 21, 2017, requesting feedback by December 1, 2017. No comments were received from beneficiary countries. However, country perspectives were considered during the preparation of the ICR through select interviews with government officials and a review of the completion reports from implementing agencies (Governments of Antigua and Barbuda, St. Kitts and Nevis, and St. Lucia). Page 67 of 82 The World Bank Caribbean - Supporting Economic Management in the Caribbean Program (P123665) ANNEX 6. SUPPORTING DOCUMENTS (IF ANY) The following is a list of supporting documents of the SEMCAR Program: • Program Document: Joint World Bank (WB) – International Monetary Fund (IMF) Proposal for Canadian International Development Agency (CIDA) Regional Development Project “Supporting Economic Management in the Caribbean� (SEMCAR) • Legal Agreements: o Administration Arrangement between the World Bank and the Canadian International Development Agency and amendments o Letter of Understanding between the World Bank and the International Monetary Fund and amendments o Grant agreements: ▪ Saint Lucia: TF012374 and TF0A2237 ▪ Antigua and Barbuda: TF0A2238 ▪ St. Kitts and Nevis: TF0A2236 • Annual Progress Reports: o Six-Month Work Progress Report 2011/2012, August 1, 2011 – March 31, 2012 o Second Year Progress Report, April 1, 2012 – March 31, 2013 o Third Year Progress Report, April 1, 2013 – March 31, 2014 o Annual Report, April 1, 2014 – March 31, 2015 o Annual Report, April 1, 2015 – March 31, 2016 • Minutes of Steering Committee Meetings: Minutes from ten Steering Committee Meetings and one High-Level Meeting • Mid-Term Review: Mid-Term Review of the Supporting Economic Management in the Caribbean Program (SEMCAR) • Project Initiation Note: Project Initiation Note on a Proposed Grant in the Amount of US$4.80 Million to the Governments of Antigua and Barbuda, Belize, Dominica, Grenada, St. Kitts and Nevis, St. Lucia, and St. Vincent and the Grenadines for a SEMCAR Budget and Revenue Systems Enhancement Project (P158239) • Implementation Status Report (ISR): ISR dated 30-Aug-2016 as part of the SEMCAR Budget and Revenue Systems Enhancement Project (P158239) • Grant Reporting and Monitoring (GRM) Report: GRM for TF012374 as part of SEMCAR SmartStream Enterprise Agreement Negotiation (P131197) • Implementation Completion and Results Reports (ICR) from Implementing Agencies: Completion reports from Antigua and Barbuda (TF0A2238), St. Kitts and Nevis (TF0A2236), and St. Lucia (TF0A2237) as part of the SEMCAR Budget and Revenue Systems Enhancement Project (P158239) Page 68 of 82 The World Bank Caribbean - Supporting Economic Management in the Caribbean Program (P123665) ANNEX 7. DETAILED DISCUSSION ON ACHIEVED RESULTS (BEYOND THE INDICATORS) 1. The SEMCAR Program contributed to improving economic management (increased efficiency, effectiveness and accountability in tax and customs administration and PFM), regional integration (knowledge sharing, consensus building, ICT/regional tax administration system) and competitiveness (facilitating trade) in most of the beneficiary countries. The Program being demand driven, its outcome contributions are manifested in the countries that requested for its support. Achievements are in varying degrees in across countries given differences in starting points, capacities and priorities. Likewise, the Program contributed to increasing awareness on environmental and gender considerations in tax, customs and PFM areas. In addition, in the process of implementing the recommendations from the Program, capacity was built among the staff of the relevant countries and agencies supported. Contribution to Improved Tax Administration 2. Strategic Management. Informed by the scoping assessments that identified among others the absence of a framework for strategic management in the beneficiary countries, the Program developed a Strategic Management Framework (SMF) that: (i) provided a roadmap to the tax administrations to achieve their articulated objectives and strategies, (ii) focused their resources on priority programs and activities that delivered the desired outcomes, (iii) provided a logical reference to develop credible budgets, (iv) established an accountability system within the organization and developed better internal coordination of programs and activities across the organization.35 It involved the articulation of both strategic and operational plans, among others. The Regional Strategic Management Model was endorsed by the SEMCAR countries as their common framework and was institutionalized in six countries – St. Kitts and Nevis, Antigua and Barbuda, Grenada, Trinidad and Tobago, Dominica and St. Lucia. 3. Some of the results evidencing increased effectiveness, efficiency and accountability in tax administration are presented below: 4. The most successful institutionalization of the SMF has been in Grenada. According to Tracy (2016),36 Grenada was considered to have done an excellent job with a score of 81 (out of 100 that represents full implementation of the SMF elements). Under the strong leadership of the Comptroller and the Planning Unit manager, the IRD has strategic and operational plans in place which are updated annually, planning unit effectively functioning despite being understaffed, performance is being assessed on a monthly, quarterly and annual basis. Grenada continues to be committed to the SMF. As noted in the Grenada’s Letter of Intent to the IMF dated May 2, 2017,37 their revenue performance is attributable not only to strong economic growth in 2016 but as well as in improvements in tax 35 Vinette Keene, SEMCAR Setting the Stage for the Strategic Management Framework- Presentation (undated). 36 Maureen Tracy, SEMCAR Institutionalizing SMF in SEMCAR Recipient Countries – Journey, Achievements and Lessons Learned (May 2016). 37 Grenada: Letter of Intent, Memorandum of Economic and Financial Policies, and Technical Memorandum of Understanding (submitted to the IMF and dated May 2, 2017). Page 69 of 82 The World Bank Caribbean - Supporting Economic Management in the Caribbean Program (P123665) administration. As noted in the Letter, IRD rolled out Corporate Strategic Business Plan to Staff and is progressing with its operational plans. IRD also implemented a compliance strategy in 2016 which resulted into an improved average compliance rate of 61%. Likewise, a payment installment plan for taxpayers was introduced that allowed payments over the year and helped cash flow regularization for the government. 5. According to the assessment by Tracy (2016), in St. Kitts and Nevis with a score of 68, SMF have been successful in putting in place the most critical elements – operational plans are now prepared annually and are the basis of the budget estimates submitted to the Ministry of Finance as well as a complete monthly monitoring of performance. There is also strong leadership at the top and a dedicated planning unit established. Likewise, with the SMF, St. Kitts and Nevis realized that the limitation of its existing tax administration system (SIGTAS) in its inability to generate data and reports that are required for effective management. This influenced their deployment of a business intelligence tool – without funding from the SEMCAR Program - to enhance their existing SIGTAS. As reported in the Ninth Steering Committee Meeting Minutes of June 2015, the BI tool revealed that with a clean database and proper utilization of appropriate business analytic tools, revenue departments can make better use of the existing SIGTAS and associated data to provide policy makers and technicians alike with more useful and meaningful information to facilitate timely and informed decisions. Moreover, the Internal Revenue Department (IRD) adopted Key Performance Indicators (KPIs) transforming it into a results-oriented entity. 6. In terms of sustainability and impact, the CARTAC Annual Report of 2016 states “Through the targeted support of SEMCAR and continuing support from CARTAC, more countries recognize the need to implement strategic and performance management systems.�38 From the beneficiary perspective, a Steering Committee representative noted that SEMCAR “has enhanced the capacities of tax administration in the region, especially in terms of strategic management. This has allowed tax administrations to appreciate and understand much better the core issues facing the tax administrations and the importance of having a strategic approach to addressing those core issues and functional areas.� 7. A subset of the SMF is the development of a Risk Management Framework (RMF) which was adopted by SEMCAR countries and introduced in varying degrees in Jamaica, Dominica, Grenada, St. Kitts and Nevis and St. Lucia. Leveraging on its experience, Jamaica led the SEMCAR working group in finalizing the RMF and the implementation of a harmonized risk management policy framework. To assist in the implementation, the Standard Reference Model for Risk Management (SRMRM) tool was developed to among others assist in identifying risk profiles of taxpayers and customs clients. 8. In terms of results, among the key example is in Dominica wherein its adoption of the SEMCAR RMF Model, contributed to increasing its revenue by $1.3 Million in its financial year 2014/15 from the 38 CARTAC Annual Report, July 2016. Page 70 of 82 The World Bank Caribbean - Supporting Economic Management in the Caribbean Program (P123665) number of cases it selected for audit (which was estimated to be 30% compared to its previous practice of 100%).39 9. ICT for Tax. The SEMCAR Program succeeded in building consensus for a regional approach towards adopting a tax administration system. It developed the business requirements including conceptual design with functional and technical specifications for a comprehensive regional tax solution system for the seven countries currently using SIGTAS. This system can be customized to fit the needs of the specific each country. The specifications would serve as inputs to request for proposals when investing in the establishment of a system which is not designed to be funded under the Program. If successful, this would be the first time a single tax system has been put in place for several sovereign countries. In the meantime, to stabilize the existing SIGTAS and enhance its capabilities, the SEMCAR completed a Recipient-Executed grant which became effective in June 2016 that was executed by St. Kitts and Nevis for the benefit of Belize, Grenada, St. Lucia and itself in the strengthening of SIGTAS to support effective operational management and decision making by improving data quality and enabling the use of business intelligence building on the experience of St. Kitts and Nevis. This is currently enabling the selected beneficiary countries to perform business analytics and produce more accurate performance management reports. Contribution to Improved Customs Administration 10. Strategic Management. In the area of customs, informed by the deployment of several assessments including the Customs Assessment Trade Toolkit (CATT), the Program supported SMF application in Jamaica and St. Lucia while RMF was supported in Jamaica, Dominica, St. Kitts and Nevis and Grenada. Workload analysis of Jamaica customs was also conducted. Likewise, the Program supported the operationalization of the Barbados Revenue Authority which consolidated tax and customs administration into a single entity. 11. As a result, Jamaica has improved its web-based feedback mechanism between customs, general public and stakeholders. Likewise, the Jamaica Customs Agency adopted an “Agency Risk Management Policy� (ARMP) in 2014 that allows it to target control efforts and resources to transactions that present the highest risk; and to this is end, pre-clearance, intelligence and post clearance controls are given due emphasis.40 This contributed to trade facilitation. 12. For its part, the Barbados Revenue Authority benefited from guidance on streamlining revenue administration, business processes and systems integration. 13. It is also noted that Program interventions were consistent with the overall direction of customs reform in the beneficiary countries which in the area of improving doing business have translated in significant reduction in the average days to export or import. 39 Dominica IRD presentation to the SEMCAR Regional Workshop – A Practical Approach to Risk Assessment in the Caribbean Region in October 2016. 40 Jamaica Customs Agency Website. Page 71 of 82 The World Bank Caribbean - Supporting Economic Management in the Caribbean Program (P123665) Contribution to Improved Public Financial Management 14. The SEMCAR Program supported various countries in the area of PFM in varying degrees and on various areas (see Table 4). Table 4. SEMCAR – Summary of PFM Technical Assistance Delivered PFM Areas Belize Grenada St. Lucia St. Kitts St. Vincent & Suriname & Nevis Grenadines 1 GFS Compliant Chart of X X X X Accounts 2 Transition to IPSAS X X X 3 Strategic Budgeting X X 4 Improving Business Processes X X X X and Procedures 5 Improving Cash Management X X and TSA 6 Financial Legislation X X X 7 Generic Internal Audit Regional workshops – jointly with CARTAC Framework 8 Building Awareness of Gender Responsive Budgeting Source: SEMCAR Fourth Progress Report, May 2015. 15. Among the results evidencing improved PFM framework and practices, are the following: i. New comprehensive PFM acts were drafted for Grenada, Belize and St. Lucia with Belize and St. Lucia informed by that of Grenada which has now been enacted and promulgated while those of Belize and St. Lucia remain as draft to date. ii. Improved organizational performance of AGDs in St. Lucia, Belize. In Nevis, for example, with the adoption of SEMCAR recommendations, the AGD achieved improvements in a number of areas: efficient budget execution by processing all expenditure payments including budget control in the IFMIS, the adoption of direct deposit method for vendor payments by which a number of checks issued was reduced significantly, bring bank reconciliation up to date. For St. Lucia, an effective cash management function was established in the areas of cash flow forecasting and the establishment of the TSA. The TSA operations allowed for daily consolidation which led to significant reduction in penalties payable to the Bank of St. Lucia. The bank reconciliation tools were also shared by St. Lucia with the other beneficiary countries. iii. Belize and St. Lucia are now producing IPSAS (cash basis) compliant financial statements. For Belize, this allowed compliance to Fiscal Responsibility Regulations. iv. St. Lucia, St. Vincent and Grenadines and Suriname now have new Chart of Accounts structure that are reporting against the IMF’s General Financial Statistics (GFS) and the UN Classification of the Functions of Government with St. Kitts and Nevis in advance stage of adoption. Page 72 of 82 The World Bank Caribbean - Supporting Economic Management in the Caribbean Program (P123665) v. Belize also significantly advances on in the planning and budget processes. It now uses a multi-year approach to budget formulation using rolling forward estimates for the budget year plus two outer years. Likewise, it has put in a place a program budgeting structure. 16. SOE Management. Through the support provided by the SEMCAR Program (diagnostics, SOE Academy and on-line course, benchmarking study on water utilities and study tour to Malaysia), increased the knowledge of SEMCAR countries on corporate governance and the fiscal risk from SOEs and how to manage them based on good international practices and built consensus among in-country stakeholders (e.g., ministries of finance, SOE management) on common challenges (e.g., reporting). As such, it helped set the stage for undertaking further steps in SOE reform and provided just in time guidance to those countries currently undertaking the reforms (e.g., Jamaica and Grenada). In this regard, one beneficiary country representative indicated that the South-South Knowledge Exchange to Malaysia has provided them with a new vision on how to pursue their own SOE reform. As for the SOE Academy, a Steering Committee representative noted that Program was receptive to the needs of the beneficiary countries in terms of the provision of support in the area of SOEs. The SOE Academy allowed participants to learn from each other’s experience. Learning in a regional setting has its advantages, she emphasized. There is “some comfort that even when we are dealing with issues, we are not alone. An open discussion with the presence of [international] experts makes a difference,� she said. 17. ICT for PFM/IFMIS. The SEMCAR Program developed a detailed data exchange design and requirements to integrate Customs Management System with Public Financial Management System and Tax ICT System that would inform as inputs into the RFP for investing in the system – which is not covered by this operation. This included specifications for the technical requirements (including common specifications and specific requirements for each information system if is needed), security requirements, messaging definition, Service Level Agreement, software warranty requirements and post-implementation and maintenance service requirements. To this end, Trinidad and Tobago issued an RFP for its IFMIS in December 2016 guided by the specifications developed under the SEMCAR Program. At the same time, Jamaica is using these as well in its consideration to deploy an IFMIS. Moreover, the Program supported the stabilization of current budget systems through a Recipient- Executed Grant executed by St. Lucia on behalf of Grenada, Belize, Dominica, St. Vincent and the Grenadines and itself. This enabled Grenada to acquire an enhanced SmartStream budget module which was a critical requirement for implementing multi-year budgeting which is expected to be utilized for the upcoming budget cycle. Contributing to Improved Regional Integration 18. Regional Integration. The SEMCAR Program achieved its ultimate outcome indicator target with the ICT regional integration plan for tax, customs and PFM systems in place at the regional and country levels. These plans are focused on simplifying and improving quality and efficiency of operations and systems in the three areas. Moreover, the SEMCAR Steering Committee has prioritized the enhancement of regional tax administration and financial management information systems and their interoperability with public revenue administration systems, as well as harmonizing public sector policies, process and systems for the next step. These plans were further enhanced with the definition Page 73 of 82 The World Bank Caribbean - Supporting Economic Management in the Caribbean Program (P123665) of business and technical requirements for the Regional Tax System and the Integration of the Customs, PFM and Tax Systems which significantly advances readiness of any follow investment operation to implement the systems. 19. Knowledge Sharing and Consensus Building. With respect to regional integration, beyond the ICT integration aspects, the program also facilitated a more intensive and solutions-oriented knowledge sharing across countries with countries good experiences actively shared to inform other beneficiary countries reform path. Improved regional integration was also manifested in the consensus building where for example the twelve countries admirably agreed to endorse further support on stabilizing SIGTAS despite the fact that it will not directly benefit all of them. 20. Regional Training Center. Cognizant of Steering Committee preference to use existing institutions over creating new ones, SEMCAR piloted with the University of the West Indies (UWI) a hybrid approach in providing training. In this regard, a one-week SOE Academy was conducted face-to- face and the materials were later converted to an online course that could be used as introduction or refresher course on the basics of SOEs. However, due to the current structure of UWI particularly the UWI Open Campus, it is unable to host the course for free for an indefinite period. As such, the course is currently available upon request on the World Bank’s Open Learning Campus.41 Mainstreaming Environmental Considerations in Tax, Customs, and PFM 21. The SEMCAR Program worked on increasing awareness on how environmental considerations could be incorporated in the areas of tax, customs and the PFM. The assessments of the state of environmental consideration in each country was conducted followed by a study on viable options and examples of good practices and then pilot implementation of the options were explored with volunteer countries. In this regard, Jamaica considered the options and recommendations resulting from the study in the review of their environmental levy. As a consequence, the SEMCAR Program contributed in the broadening of the environmental levy introduced in 2008 that was charged only to imported products with the introduction of 0.5% levy on the domestic side in FY2015/16. Mainstreaming Gender Considerations in Tax, Customs, and PFM 22. The SEMCAR Program also worked on increasing awareness on how gender considerations could be mainstreamed in the areas of tax, customs and PFM. Similar to the approach on environment, assessments of the state of gender considerations in each country was conducted followed by a study on feasible options and piloting of recommended options including a process-mapping to identify how to incorporate gender in tax administration. In this regard, Barbados volunteered to pilot the implementation of a gender-sensitive voluntary compliance exercise on tax. As such, a complete set of tools for conducting the exercise was developed and training were conducted. However, the pilot could not proceed due to the unavailability of data. That said, tools are readily available for any country 41 https://wbg.sabacloud.com/Saba/Web_spf/NA1PRD0002/common/leclassview/dowbt000000000004453. Page 74 of 82 The World Bank Caribbean - Supporting Economic Management in the Caribbean Program (P123665) interested to conduct the exercise. In the area of PFM, a workshop was conducted jointly with CARTAC on gender-sensitive budgeting and seven SEMCAR countries benefited from the workshop deliberations. Page 75 of 82 The World Bank Caribbean - Supporting Economic Management in the Caribbean Program (P123665) ANNEX 8. COMPARISON OF PROPOSED SEMCAR ACTIVITIES AND DELIVERED Table 5. Comparison of Proposed SEMCAR Activities and Delivered Planned Activities Delivered/Revisions (from Program Document) (from Final Progress Report) Component 1 - Tax High level scoping assessments to determine the Done current status of tax administration and related ICT developments and the level of commitment to the SEMCAR regional approach. Selective in-depth assessments of tax administration Done and ICT framework in at least 5 countries On-the-ground technical assistance for the Done implementation of institutional modernization though improved organizational structures, legislation and procedures, including incorporation of a gender perspective, delivered in at least five countries. Regional maintenance contract for tax administration Dropped systems negotiated and signed. Detailed user requirements, including technical Revised to clarify that the conceptual design specifications, to acquire or design an integrated tax is for a regional tax administration solution administration system to replace the SIGTAS system system for all countries and not just to replace completed for at least 5 countries SIGTAS Regional tax administration system upgrades Recalibrated to focus on producing a detailed implemented in at least 5 countries. request for proposal; RETF 2 stabilized SIGTAS in St. Lucia and St. Vincent and the Grenadines), introduced business intelligence functionality in Belize and developed an E-IRD solution for St. Kitts and Nevis. Study on feasible options for environmental taxation Done presented to policy makers Tools to enable voluntary compliance with tax Done – additionally, a pilot of a tool was administration requirements, with particular regard to attempted in Barbados the needs of women in small businesses, developed in at least five countries Component 2 - Customs High level scoping assessments using the CATT Done methodology to determine the current status of Page 76 of 82 The World Bank Caribbean - Supporting Economic Management in the Caribbean Program (P123665) Planned Activities Delivered/Revisions (from Program Document) (from Final Progress Report) customs administration and related ICT developments and the level of commitment to the SEMCAR regional approach. Selective in-depth action plans of customs Done administration and ICT framework in at least 4 countries based on CATT findings On-the-ground technical assistance for the Done implementation of institutional modernization through organizational structures, legislation and procedures especially on risk management systems, and HR management including incorporation of a gender perspective, delivered in at least 4 countries. Regional contract for customs management systems Dropped negotiated and signed. User requirements for regional upgrade to customs Revised to focus on TA supporting integration management system including linkage to new tax IT between tax and customs system, developed in at least 4 countries Regional customs management system maintenance Dropped implemented in at least 4 countries User (internal and external) business requirements Dropped including a detailed conceptual and functional design for an automated exchange of Customs declarations including cargo for information-exchange at regional level. Study on feasible options for integrating Done environmental measures into customs administration completed and presented to policy makers. Added new activity on feedback mechanisms between and critical stakeholders to improve customs performance Component 3 - PFM High-level scoping assessments to determine the Done current status of PFM and ICT reforms and level of commitment to the SEMCAR regional approach In -depth assessments of PFM and ICT framework in Done at least 5 countries Page 77 of 82 The World Bank Caribbean - Supporting Economic Management in the Caribbean Program (P123665) Planned Activities Delivered/Revisions (from Program Document) (from Final Progress Report) On-the-ground technical assistance for modernizing Done and restructuring of PFM processes delivered; including budget formulation debt management and internal and external audit institutions and legal frameworks modernized in at least 5 countries Regional maintenance contract for IFMIS negotiated Negotiations completed but not implemented. RETF2 covered some upgrades to Smartstream for Belize, Dominica, Grenada, St. Lucia and St. Vincent. User (internal and external) business requirements for Replaced with the development of a common IFMIS upgrades, including linkage to budget framework for functional requirement of core formulation module prepared for at least 5 countries and non-core IFMIS components Reporting and data-warehousing mechanisms across Initiated through COUSINs which was the three main systems is implemented in at least 5 dropped at request of SEMCAR Steering countries. committee Interfaces and interoperability modules between the Initiated through COUSINs which was three main systems are implemented in at least 5 dropped at request of SEMCAR Steering countries. committee Tools to track gender-specific expenditures are Revised to create awareness evaluated and developed, and integrated into IFMIS modules to produce gender disaggregated reports piloted in at least one country, with results presented to policy makers. Chart of accounts harmonized (as needed) with UN Done COFOG classification and IMF GFS economic classification, allowing governments to better monitor environmental interventions. Added TA on Strengthening corporate governance and fiscal framework for SOEs Component 4 – Regional Training and ICT Support Center Feasibility studies undertaken, and the institutional Feasibility study conducted and piloted design and initial business plan developed, for the hybrid approach of virtual and face-to-face establishment of a sustainable regional training and training ICT support centre Technical assistance for the development of regional Done Page 78 of 82 The World Bank Caribbean - Supporting Economic Management in the Caribbean Program (P123665) Planned Activities Delivered/Revisions (from Program Document) (from Final Progress Report) ICT technical standards and interoperability frameworks delivered, ensuring they fit within regional ICT and e-Government strategies and policies Regional workshops and training curriculum in Done support of user technical assistance and upgraded ICT systems delivered for tax, customs and PFM administrations Assistance in the establishment of a customs system No user group established user group delivered and the existing tax system and IFMIS user groups with workshops and training supported Page 79 of 82 The World Bank Caribbean - Supporting Economic Management in the Caribbean Program (P123665) ANNEX 9. STRUCTURE OF SEMCAR TRUST FUND AND DIVISION OF LABOR WITH THE IMF Table 6. Structure of Trustee TF071579 – Multi Donor Trust Fund Supporting Economic Management in the Caribbean Fund Fund Name Project Name and ID TF071579 Multi Donor Trust Fund Supporting MDTF for the Support for Economic Economic Management in the Caribbean Management in the Caribbean Region TF010167 Conduct of High-Level Scoping Phase 1 (P123665) – Bank & IMF Executed Assessments TF099369 SEMCAR Program Management TF099282 Supporting Economic Management in the Caribbean [SEMCAR IMF LOU] Customs Modernization Technical Programmatic TA for the Support for TF010825 Assistance Economic Management in the Caribbean TF011625 Tax Administration Technical Assistance Region Phase 1 (P128874) – Bank SEMCAR Public Financial Management Executed TF011668 Technical Assistance TF013001 Regional Project Coordination TF018199 SEMCAR ICT Advisory Services SEMCAR Budget and Revenue Systems TF0A3531 Enhancement Project TF012374 Saint Lucia - Supporting Economic SEMCAR SmartStream Enterprise Management in the Caribbean - SEMCAR Agreement Negotiation (P131197) – Recipient Executed SEMCAR Budget and Revenue Systems SEMCAR Budget and Revenue Systems TF0A2236 Enhancement Project Enhancement Project (P158239) – SEMCAR Budget and Revenue Systems Recipient Executed TF0A2237 Enhancement Project SEMCAR Budget and Revenue Systems TF0A2238 Enhancement Project TF081607 Bank executed component Window 1 TF081608 IMF Executed component Window 3 TF081609 Recipient executed component Window 2 Page 80 of 82 The World Bank Caribbean - Supporting Economic Management in the Caribbean Program (P123665) Table 7. SEMCAR – International Monetary Fund and World Bank Division of Labor Components International Monetary Fund World Bank Tax Scoping ICT (including supervision of In-Depth Technical Assessments RETF grant) On the Ground TA (SMF/RMF) Gender Mainstreaming Environment Mainstreaming Training Simplification of business processes Customs Same as above In-depth assessments through CATT ICT (including supervision of RETF grant) Pre-clearance Gender Mainstreaming Environment Mainstreaming Workload Analysis (Jamaica) PFM Same as above ICT (including supervision of RETF grant) SOE Gender Mainstreaming Environment Mainstreaming Regional Training and ICT ICT Support Center Regional Training Center Program Management Trust Fund Administration Steering Committee Meetings Page 81 of 82 The World Bank Caribbean - Supporting Economic Management in the Caribbean Program (P123665) ANNEX 10. SUMMARY OF RATINGS Table 8. Overall Outcome Rating Sub-ratings Overall Outcome Rating Relevance Efficacy Efficiency Substantial Substantial Modest Moderately Satisfactory Table 9. Key Ratings Outcome Bank Performance M&E Quality Moderately Satisfactory Moderately Satisfactory Substantial Page 82 of 82