Document of The World Bank FOR OFFICIAL USE ONLY Report No: ICR00005583 IMPLEMENTATION COMPLETION AND RESULTS REPORT (Credit Number: IDA 55840) ON A CREDIT IN THE AMOUNT OF SDR 33.9 MILLION (US$50 MILLION EQUIVALENT) TO THE ISLAMIC REPUBLIC OF PAKISTAN FOR THE PAKISTAN SINDH PUBLIC SECTOR MANAGEMENT REFORM PROJECT November 30, 2021 Governance Global Practice South Asia Region CURRENCY EQUIVALENTS (Exchange Rate Effective October 31, 2021) Currency Unit = PKR US$1 = PKR 170.95 US$1 = SDR 0.71 FISCAL YEAR July 1 - June 30 Regional Vice President: Hartwig Schafer Country Director: Najy Benhassine Regional Director: Zoubida Kherous Allaoua Practice Manager: Hisham Ahmed Waly Task Team Leaders: Charles Victor Blanco, Jose Eduardo Gutierrez Ossio ICR Main Contributor: K. Migara O. De Silva ABBREVIATIONS AND ACRONYMS ADP Annual Development Plan BI Business Intelligence BoR Board of Revenue BPS Basic Pay Scale CGA Controller General of Accounts CPS Country Partnership Strategy DLIs Disbursement-Linked Indicators DLRs Disbursement-Linked Results EEP Eligible Expenditure Program ERP Enterprise Resource Planning ERU Economic Reform Unit ET&NCD Excise Taxation and Narcotics Control Department FA Financing Agreement FBR Federal Bureau of Revenue FD Finance Department FMIS Financial Management Information System FS Finance Secretary FY Fiscal Year GDP Gross Domestic Product GoS Government of Sindh GSTS General Sales Tax on Services HR Human Resources IA Internal Audit ICR Implementation Completion and Results Report ICT Information and Communication Technologies IDA International Development Association IR Intermediate Results IRI Intermediate Results Indicators ISR Implementation Status and Results Report IT Information Technology M&E Monitoring and Evaluation MEC Monitoring and Evaluation Cell MIS Management Information System MS Moderately Satisfactory MTR Mid-Term Review MU Moderately Unsatisfactory NADRA National Database Registration Authority OPCS Operations Policy and Country Services PAD Project Appraisal Document PAO Principal Accounting Officer P&DD Planning and Development Department PDO Project Development Objective PDOI Project Development Objective Indicator PEFA Public Expenditure and Financial Accountability PFM Public Financial Management PIU Project Implementation Unit PKR Pakistani Rupee PMIS Procurement Management Information System PPMS Public Procurement Management System PV Present Value RF Results Framework SDR Special Drawing Right SPPRA Sindh Public Procurement Regulatory Authority SPSMRP Sindh Public Sector Management Reform Project SRB Sindh Revenue Board STRMP Sindh Tax Revenue Mobilization Plan STRMRP Sindh Tax Revenue Mobilization Reform Plan STS Sales Tax on Services TA Technical Assistance ToC Theory of Change TTL Task Team Leader TPV Third-Party Verification YOY Year-on-Year TABLE OF CONTENTS DATA SHEET .......................................................................................................................... 1 I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES ....................................................... 5 A. CONTEXT AT APPRAISAL .........................................................................................................5 B. SIGNIFICANT CHANGES DURING IMPLEMENTATION (IF APPLICABLE) .......................................8 II. OUTCOME .................................................................................................................... 11 A. RELEVANCE OF PDOs ............................................................................................................ 11 B. ACHIEVEMENT OF PDOs (EFFICACY) ...................................................................................... 12 C. EFFICIENCY ........................................................................................................................... 18 D. JUSTIFICATION OF OVERALL OUTCOME RATING .................................................................... 20 III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME ................................ 21 A. KEY FACTORS DURING PREPARATION ................................................................................... 21 B. KEY FACTORS DURING IMPLEMENTATION ............................................................................. 21 IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME .. 22 A. QUALITY OF MONITORING AND EVALUATION (M&E) ............................................................ 22 B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE ..................................................... 23 C. BANK PERFORMANCE ........................................................................................................... 25 D. RISK TO DEVELOPMENT OUTCOME ....................................................................................... 26 V. LESSONS AND RECOMMENDATIONS ............................................................................. 26 ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS ........................................................... 28 ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION ......................... 42 ANNEX 3. PROJECT COST BY COMPONENT ........................................................................... 44 ANNEX 4. EFFICIENCY ANALYSIS ........................................................................................... 45 ANNEX 5. BORROWER, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS ... 47 ANNEX 6. SUPPORTING DOCUMENTS .................................................................................. 48 ANNEX 7: PROVINCIAL GENERAL SALES TAX ON SERVICES (GSTS) (IN PKRS, BILLIONS) ......... 49 ANNEX 8: PFM PERFORMANCE INDICATORS ....................................................................... 51 The World Bank PK Sindh Public Sector Management Reform Project (P145617) DATA SHEET BASIC INFORMATION Product Information Project ID Project Name P145617 PK Sindh Public Sector Management Reform Project Country Financing Instrument Pakistan Investment Project Financing Original EA Category Revised EA Category Not Required (C) Not Required (C) Organizations Borrower Implementing Agency Sindh Revenue Board, Economic Reform Unit, Finance Islamic Republic of Pakistan Department Project Development Objective (PDO) Original PDO The development objective of the Project is to strengthen public sector performance in the Province of Sindh through improved revenue generation and expenditure management. Page 1 The World Bank PK Sindh Public Sector Management Reform Project (P145617) FINANCING Original Amount (US$) Revised Amount (US$) Actual Disbursed (US$) World Bank Financing 50,000,000 50,000,000 47,381,512 IDA-55840 Total 50,000,000 50,000,000 47,381,512 Non-World Bank Financing 0 0 0 Borrower/Recipient 220,000,000 0 0 EC: European Commission 12,000,000 0 0 Total 232,000,000 0 0 Total Project Cost 282,000,000 50,000,000 47,381,512 KEY DATES FIN_TABLE_DAT Approval Effectiveness MTR Review Original Closing Actual Closing 28-Jan-2015 28-Apr-2015 13-Nov-2017 31-Aug-2020 28-Feb-2021 RESTRUCTURING AND/OR ADDITIONAL FINANCING Date(s) Amount Disbursed (US$M) Key Revisions 16-Jun-2018 30.08 Change in Results Framework Change in Implementation Schedule 21-May-2020 47.38 Change in Loan Closing Date(s) KEY RATINGS Outcome Bank Performance M&E Quality Satisfactory Satisfactory Substantial Page 2 The World Bank PK Sindh Public Sector Management Reform Project (P145617) RATINGS OF PROJECT PERFORMANCE IN ISRs Actual No. Date ISR Archived DO Rating IP Rating Disbursements (US$M) 01 07-May-2015 Satisfactory Satisfactory 0 02 13-Nov-2015 Moderately Satisfactory Moderately Satisfactory 12.34 03 05-May-2016 Moderately Satisfactory Moderately Satisfactory 13.85 Moderately 04 28-Dec-2016 Moderately Unsatisfactory 19.06 Unsatisfactory Moderately 05 12-Jun-2017 Moderately Unsatisfactory 19.06 Unsatisfactory Moderately 06 01-Feb-2018 Moderately Unsatisfactory 22.53 Unsatisfactory 07 29-Jun-2018 Moderately Satisfactory Moderately Satisfactory 30.08 08 21-Dec-2018 Moderately Satisfactory Moderately Satisfactory 30.08 09 25-Jun-2019 Moderately Satisfactory Moderately Satisfactory 34.88 10 23-Dec-2019 Moderately Satisfactory Moderately Satisfactory 34.88 11 15-Jun-2020 Moderately Satisfactory Moderately Satisfactory 47.38 12 23-Dec-2020 Moderately Satisfactory Moderately Satisfactory 47.38 SECTORS AND THEMES Sectors Major Sector/Sector (%) Public Administration 100 Sub-National Government 50 Other Public Administration 50 Themes Major Theme/ Theme (Level 2)/ Theme (Level 3) (%) Page 3 The World Bank PK Sindh Public Sector Management Reform Project (P145617) Economic Policy 20 Fiscal Policy 20 Tax policy 20 Public Sector Management 80 Public Finance Management 40 Public Expenditure Management 20 Domestic Revenue Administration 20 Public Administration 40 Transparency, Accountability and Good 40 Governance ADM STAFF Role At Approval At ICR Regional Vice President: Annette Dixon Hartwig Schafer Country Director: Rachid Benmessaoud Najy Benhassine Director: Mario Marcel Cullell Zoubida Kherous Allaoua Practice Manager: Alexandre Arrobbio Hisham Ahmed Waly Zubair Khurshid Bhatti, Paul Charles Victor Blanco, Jose Task Team Leader(s): Welton Eduardo Gutierrez Ossio ICR Contributing Author: K. Migara O. De Silva Page 4 The World Bank PK Sindh Public Sector Management Reform Project (P145617) I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES A. CONTEXT AT APPRAISAL Context 1. This Implementation Completion and Results Report (ICR) focuses on the Sindh Public Sector Management Reform Project (SPSMRP) of the Province of Sindh in the Islamic Republic of Pakistan. The ICR reviews the initiation, design, implementation processes and challenges, and the achievement and sustainability of the PDO and compendium outcomes, including the disbursement-linked indicators (DLIs) and disbursement-linked results (DLRs). It seeks to document what worked, what did not work so well, and why. Almost all Bank-funded projects encounter routine implementation challenges, which could be considered a given throughout their implementation cycles. What is significant though is the response to the challenges that ultimately influence the achievement of the PDO and the overall outcome rating. In the case of the SPSMRP, the evidence confirms that representatives of the Government of Sindh Province (GoS) and the World Bank project implementation and supervision teams worked in a collaborative manner to put the project back on track after implementation challenges were encountered. The Outcome section of this ICR provides evidence (in a tabular form) to support this assertion, which ultimately influenced the overall outcome rating of the project. 2. The main objectives of the SPSMRP were to improve revenue mobilization and to enhance expenditure management. The twin objectives were aligned with the priorities of the GoS, with an emphasis on boosting the collection of sales taxes1 and ensuring better financial management2. This was to be achieved by improving budgetary processes, building robust systems such as e-procurement, a financial management information system, and a tax management information system. Of the four provinces in Pakistan, Sindh Province was the most industrialized province at the time when the SPSMRP was prepared. It is also the second largest province, with an estimated 47.9 million inhabitants3. Despite its apparent wealth, it faced challenges, such as erratic economic growth and declining incomes at the time the SPSMRP was initiated. Additionally, the delivery of key public services was largely unsatisfactory and revenue collection was abysmal and static. The available financial resources were poorly managed. Capacity was weak vis-à-vis revenue generation. Although three independent agencies4 were responsible for various aspects of revenue collection, the heavy reliance on federal transfers, which were erratic, undermined the ability of the GoS to finance the province’s development needs. In addition, the weak public expenditure management further eroded the fiscal space to carry out critical public sector reforms, thereby providing the justification for the initiation of the project. Thus, the SPSMRP was intended to support the GoS in overcoming perennial shortfalls in revenue, which severely constrained economic growth and the efficient delivery of critical public services to the citizens of the province. 1 This was evidenced by the Sindh Tax Revenue Mobilization Plan formulated in February 2014 and subsequently approved by the Chief Minister (see: Project Appraisal Document [PAD], page 15). 2 The GoS introduced several PFM measures prior to the SPSMRP, but challenges remained in achieving their goals, as indicated in the PEFA Assessments conducted in 2009 and 2013. See also the summary of ratings in Table A6.1 in the PAD Annex 6 (page 82). 3 At the time the SPSMRP was prepared, the province had a total population of 42 million. During the preparation of the ICR, the population had increased to 47.9 million. 4 For example, the Board of Revenue (BoR), the oldest and the largest, was in charge of rural land management and taxes. The Excise and Taxation Department (E&TD) was in charge of collecting property professional, infrastructure cess, MVT. The Sindh Revenue Board (SRB), created in 2010, was in charge of collection of Sales Taxes on Services (STS). The Finance Department (FD) acts as the facilitating and coordinating organization to the three tax-collecting agencies. Page 5 The World Bank PK Sindh Public Sector Management Reform Project (P145617) 3. Beyond the financial challenges, there were severe institutional and structural deficiencies that needed to be corrected. For example, the mandate to collect the provincial tax was split among three independent tax collection agencies (as indicated in footnote 4), which generally increased the cost of tax collection and administration. The establishment of coordination among them, which was the responsibility of the Finance Department (FD), remained a challenge. Despite the fact that revenue collection responsibilities were spread across these three agencies, only nine taxes accounted for almost 97 percent at the time of project preparation. As such, it became necessary to advocate for tax administration reforms.5 4. The initiation of the project was timely, as it was aligned with the launch of the comprehensive Sindh Tax Revenue Mobilization Plan (STRMP). The STRMP was approved by the Chief Minister in February 2014. The SPSMRP was designed to meet the goals outlined in the STRMP. Additionally, the SPSMRP supported financial management reforms aimed at stopping the deterioration in public financial performance, which had seriously constrained the economic and social development in the province. A reformed public financial management (PFM) system was established to improve public financial management, while also enhancing budget credibility — which is one of the critical pillars of public expenditure management. These reforms were introduced to ensure transparency in financial flows and to provide a template for better procurement practices. 5. The SPSMRP was aligned with the Pakistan Country Partnership Strategy (CPS) of 2015-2019. Key aspects of the CPS included: (i) increasing revenues to fund the delivery of public services; (ii) improving public financial management; and (iii) improving development expenditures.6 Both of these objectives are addressed by the project components. Theory of Change (Results Chain) 6. In the face of weak revenue mobilization capacity and poor public expenditure management, it was imperative that critical pathways be found to resolve these twin challenges. An important assumption that underpinned the SPSMRP was that building appropriate institutional and human resource capacity, applying technological solutions in the form of management information systems, and developing policy guidelines could lead to better solutions to these challenges. The successful resolution of these challenges would in turn lead to achieving the project development objective (PDO). The Theory of Change (ToC, Figure 1), which was prepared for this ICR,7 takes into account the PDO and the changes made during project restructuring. As such, it maps out the key SPSMRP objectives; the activities that were implemented; the outputs that were generated by these activities; and the short- and medium-term outcomes that were expected to contribute toward achieving the PDO. Critical assumptions that could impede or facilitate project implementation and results were also anticipated. 5 See the Project Appraisal Document of December 30, 2014, p. 15. 6 World Bank, PAD, December 2014, p. 20, paragraph 28. 7 Please note that the PAD did not have a ToC, as it predated the requirement to include a ToC in all PADs. Page 6 The World Bank PK Sindh Public Sector Management Reform Project (P145617) Figure 1: Theory of Change (ToC): Mapping of the SPSMRP Page 7 The World Bank PK Sindh Public Sector Management Reform Project (P145617) Project Development Objectives (PDOs) 7. The Project Development Objective is “to strengthen public sector performance in the Province of Sindh through improved revenue generation and expenditure management”. It was anticipated that full implementation of the project interventions would result in “improvement in revenue collection, public financial management and procurement systems”. Key Expected Outcomes and Outcome Indicators 8. The achievement of the PDO was to be determined by two key outcome indicators, namely: “improved collection of Sales Tax on Services, improved credibility of budget execution, and increased timelines of contract execution”8. 9. The following indicators were defined to measure the success of the PDO: (a) improved collection of Sales Tax on Services; (b) improved credibility of budget execution; and (c) reduced time taken to process procurement contracts. The PDO Indicators (PDOIs) and the Intermediate Results Indicators (IRIs) are shown in the Results Framework (Annex 1). Components 10. The project consisted of two main components. These were Component 1: Results-based financing, which provided support to the GoS to finance eligible public sector management reforms. The GoS subsequently sought reimbursement once the agreed-upon eligible expenditure programs (EEPs) were achieved through a Third-Party Verification (TPV) of the DLIs. Payment was made to the GoS once the achievement of the DLIs was confirmed through the TPV. A sum of US$40 million was allocated to this component. Component 2 provided technical assistance (TA) to support a selected number of activities, which were important for achieving the DLIs. A sum of US$10 million was allocated to this component. The total project funding was US$50 million. B. SIGNIFICANT CHANGES DURING IMPLEMENTATION (IF APPLICABLE) Revised PDOs and Outcome Targets 11. In response to the request made by the GoS on March 28, 2018, the Bank agreed to carry out a level-2 restructuring of the project. Although the PDO remained the same, two of the three PDO indicators were removed/revised. For example, PDO Indicator 2, Improved Credibility of the Budget, was replaced with the DLI #6, and PDO Indicator 3, Reduced time taken to process procurement contracts, was dropped (see Table 1). 12. At the time of project restructuring, the SPSMRP was rated Moderately Unsatisfactory (MU) due to the non- achievement of several targets. Both ratings on “Progress towards achieving PDO” and “Implementation Progress” were rated MU. These ratings were largely due to a lack of progress in strengthening public procurement, as well as several areas related to improving budget credibility. After nearly four years since it had become effective, project disbursement at the time was approximately 60 percent of the total US$50 million. The project had 11 legal covenants. Before the Mid- term Review (MTR), the GoS was in compliance with nine of the 11 legal covenants, with two non-complied with and partially complied.9 To address these implementation challenges, a level-2 restructuring was completed with a 8 Ibid, p. 21. 9 See ISR of February 1, 2018. Page 8 The World Bank PK Sindh Public Sector Management Reform Project (P145617) comprehensive revision of the project’s Results Framework and the DLIs, thus setting more clearly defined and realistic targets (as indicated in Table 1). During the completion of the project, the government was in compliance with all legal covenants.10 13. The DLIs focused on the achievement of specific deliverables or outputs. These included increasing tax revenue mobilization; enhancing the performance of PFM systems; improving management of the development portfolio; and strengthening the public procurement system. 14. The TA supported technical consultancies and initiating activities. These activities included training seminars, conferences, and the purchase of equipment for the implementation of the reform activities of the Sindh Tax Revenue Mobilization Plan (STRMP)11. 15. Following the MTR, which took place from November 13-17, 2017, the World Bank advised the GoS to initiate several changes. These included changes to the PDO indicators (PDOIs), some intermediate results indicators (IRIs), and the disbursement-linked indicators (DLIs). The GoS accepted the recommendations of the Bank to restructure the project by focusing on defining clear targets for project activities that had experienced delays and were overly ambitious. As noted, project restructuring provided an opportunity to devise a more realistic alignment of the expected deliverables with the existing implementation capacity. Revised PDO Indicators 16. Since it was a level-2 restructuring, the original PDO was left unchanged. However, of the three PDO-level results indicators, one remained, namely PDOI 1 concerning “improved collection of sales taxes on services (STS). It remained on track at the time of the MTR; as such, it was not changed. PDOI 2 concerning “improved credibility of the budget” was dropped and replaced with DLI 6 pertaining to “transparency in budget formulation, allocation and execution” 12. The MTR report attributed a large, unexpected variation in the FY16 budget along with a problematic definition of the baseline (that is, PEFA 2.2) to the immeasurability of the indicator. PDOI 3 concerning “reduced time taken to process procurement contracts” was also removed due to the three-year delay in the implementation of the procurement management information system (PMIS)13. The restructuring led to changes to the Results Framework, which are reflected in Table 1. Table 1: Summary of Changes to the Results Indicators after Restructuring Indicator DLI Type of Revision Rationale for Change PDO 1: Improved collection of N/A No change. Indicator on track. No changes. Sales Tax on Services (STS) PDO 2: Improved credibility of The target was unrealistic and impacted by a large, the budget unexpected variation in the FY16 budget, along with an unclear definition of the baseline (t that is, PEFA 2.2). N/A Dropped (Replaced). Therefore, the indicator was replaced with indicator DLI 6, which was more clearly defined and more straightforward to measure. PDO 3: Reduced time taken to The indicator was unrealistic. Due to significant delays with N/A Dropped. process procurement contracts the implementation of the Procurement Management 10 See the last project ISR of December 23, 2020. 11 See World Bank, Project Restructuring Paper, Report Number RES 314191, p 4. 12 The previous indicator concerning “improved credibility of the budget” was dropped because it was not in line with 2.2 Table/data in the PEFA of 2013. 13 See World Bank, MTR Report, November 13-17, 2017, p. 1. Page 9 The World Bank PK Sindh Public Sector Management Reform Project (P145617) Information System (PMFIS), the indicator was unlikely to be achievable by the project closing date. Therefore, it was dropped. IRI #1: Approval and initial DLI No change.3 No changes, as the one-time target was met. implementation of the STRMP. #1 IRI #2: Enhanced the human Targets for this indicator were revised considering the reform DLI resources capacity for Targets revised. context, while expected achievements were more clearly #2 administration at the SRB defined. IRI #3: Increased automation of The indicator and target remain unchanged in substance, with No change in SRB systems some clarifications to the verification protocol. The funds DLI substance. allocated to the DLI were increased by US$1.5 million, as #3 Increase of DLI automation will be a critical factor for increased revenue amount. collection. IRI #4: Increased tax base N/A No change. Indicator on track. No changes. IRI #5: Risk-based audits of The newly created tax audit had limited capacity, and IRI #5 taxpayers implemented Targets revised. was replaced with more realistic targets. The SRB Tax Audit DLI Decrease in DLI Unit was to be expanded, strengthened, and be made more #4 amount. autonomous. Given the adjusted / lower targets, the DLI amount was reduced by US$1 million. IRI #6: Improved SRB N/A No change. Indicator remains relevant. No changes. transparency IRI #7: Debt management The targets and verification protocol of the indicator were N/A Targets revised. systems instituted adjusted and clarified. IRI #8: Increased extra-budgetary This indicator was impacted by its imprecise definition and N/A Dropped. releases captured in the FMIS was accordingly dropped. IRI #9: Establishment of internal Converted from DLI The DLI was converted into an IRI. Targets were adjusted and DLI audit to an IRI. set at more realistic levels. Overall, the DLI amount was #5 Targets revised. reduced by US$2 million compared to the original allocation. IRI #10: Transparency in budget Converted from an The indicator was converted into a PDO indicator, replacing formulation, allocation, and IRI to a PDO PDO indicator 2. Targets were maintained. Given that the new DLI execution indicator. PDO indicator was critical to the overall achievement of the #6 No other changes in PDO, its DLI allocation was increased by US$1.5 million. substance. IRI #11: Contract data entered Given the complexity of procurement reforms and delays in into the procurement MIS establishing the procurement management information N/A Dropped. system, the achievement of this indicator was unrealistic. Hence, it was dropped. IRI #12: Contracts processed The achievement of this indicator was unrealistic within the through the procurement N/A Dropped. timeframe of the project and was therefore dropped. modules IRI #13: Procurement officials DLI The indicator was maintained, but the targets were and Targets revised. certified #7 clarified, thus setting more realistic expectations. IRI #14: Quarterly departmental The indicator was maintained, but the targets were revised DLI development plan monitoring Targets revised. and clarified, thus setting a more realistic expectation. #8 reports prepared and published IRI #15: Departments where geotagging of development N/A No change. The indicator was maintained. schemes is implemented, and information published IRI #16: Departments with proactive feedback mechanisms N/A Targets revised. The indicator was maintained with clarified targets. established Source: World Bank Restructuring Paper Note: DLI= disbursement-linked indicator; FMIS= Financial Management Information System; IRI= intermediate results indicator; N/A= Not Applicable; PDO= Project Development Objective; PEFA= Public Expenditure and Financial Accountability; SRB= Sindh Revenue Board; STRMP= Sindh Tax Revenue Mobilization Plan. Page 10 The World Bank PK Sindh Public Sector Management Reform Project (P145617) Revised Components 17. Neither of the two components were revised. Other Changes 18. No other changes to the components and subcomponents were planned, other than the formal restructuring as noted. Rationale for Changes and Their Implication on the Original Theory of Change 19. Since the restructuring did not affect the original components of the project, there were no significant implications on the original theory of change for the project. II. OUTCOME A. RELEVANCE OF PDOs Assessment of Relevance of PDOs and Rating 20. As noted, the project closing date was extended from August 31, 2020 to February 28, 2021. The COVID-19 pandemic, which resulted in a long period of lockdown, was the main reason for the project extension. The lockdown restricted the supply of ICT equipment and the closure of the offices and firms delayed the consultancy assignments. The additional implementation period provided a boost to the project, as it gave the Project Implementation Unit (PIU) ample time to implement outstanding activities to the benefit of the project. This helped to cover existing implementation gaps and challenges, particularly the remaining procurement activities related to the Economic Reform Unit (ERU) and the SRB data centers, as well as the SRB’s Enterprise Recourse Planning (ERP) and business intelligence (BI) units. 21. The PDO was highly relevant. Section A, paragraph 1 of this ICR (under the Context and Appraisal sections) provides evidence to support this assertion. Prior to the initiation of the project, the Sindh Province had experienced a broad range of development challenges, most notably the lack of enough revenue to support its development activities. It also lacked the capacity to improve its expenditure management. Thus, these challenges weakened the province’s ability to implement important economic development initiatives. The initiation of the SPSMRP was timely, as it was intended to support the GoS in overcoming the deficit in revenue generation and the weaknesses in its public expenditure management. Several years after the initiation and implementation of the project, the PDO still remains relevant. 22. The problem statement and the critical paths outlined in the Theory of Change in Section A of this report were realistic, allowing the GoS to resolve revenue shortfalls and poor expenditure management. As the ToC indicates, the assumption underlying the PDO was that if the revenue generation capacity of the GoS improved and if expenditure management was enhanced across the provincial ministries and other provincial public sector agencies, the Government would be able to significantly increase its domestic revenue mobilization to generate enough financial resources. These resources could in turn be utilized to finance the province’s development needs. This would also contribute to promoting growth and enhancing the livelihoods of its citizens. Having identified the problem, the next step was to define the critical Page 11 The World Bank PK Sindh Public Sector Management Reform Project (P145617) paths to achieve the PDO, as illustrated in Figure 1. As noted, the critical paths mapped out in the ToC of this report provide further justification of the relevance of the PDO and the outcomes it sought to achieve. 23. The achievement of the PDO depended on building or adopting innovative technologies and systems. In addition, it depended on strengthening the human resource (HR) capacity of the staff of the SRB, which was the principal implementation agency. The restructuring of the project after the MTR further reinforced the capacity to achieve original expected project outcomes, namely improved collection of the STS and expenditure management through enhanced transparency of the budgetary process. 24. Although the PDO was relevant, the outcomes and related PDO-level indicators to measure success could have been better designed. For example, PDOIs (2) and (3) were largely qualitative and posed measurement challenges. However, the restructuring initiated after the MTR addressed these shortcomings, as outlined in Table 1. Therefore, the ICR proposes that the project restructuring strengthen the capacity of the SRB and the associated implementing entities to achieve the recommended changes, which in turn largely removed the constraints that initially slowed project implementation and the achievement of the targeted project outcomes. Hence, the PDO relevance is rated as Substantial. B. ACHIEVEMENT OF PDOs (EFFICACY) Assessment of Achievement of Each Objective/Outcome 25. Some of the IRIs and the DLIs had been achieved prior to the restructuring of the SPSMRP in June 2018. The changes resulting from the restructuring further improved the capacity of the project to achieve the remaining IRIs and the DLIs. By the end of the June 2020 and November 2020 implementation support and supervision missions, respectively, key project ratings had improved from Moderately Unsatisfactory (MU) to Moderately Satisfactory (MS) 14. 26. One of the outcomes of the PDO is to improve revenue generation. The achievement of the target set for the General Sales Tax on Services (GSTS) was significant. The province has 15 taxes and other minor levies in the portfolio of its tax options. Close to 98 percent of its total tax collection is contributed by seven taxes, which include the GSTS, Sindh development maintenance of infrastructure tax, the stamp duty, the property tax, the excise tax, the capital value tax, and the motor vehicle tax. As illustrated in Table 2, if direct and indirect taxes are considered, the share of the GSTS of the total revenues collected during FY20 was 57 percent. Table 2: Total Tax Revenue (Direct and Indirect, Pakistani Rupees, billions) FY19 FY20 RE Percentage Share of (Revised Total Tax Estimate) Total tax revenues (1+2) 177.7 235.0 1. Direct taxes 7.3 14.84 Agriculture income taxes 0.6 1.2 1% Property taxes 2.8 8.0 3% Land revenues 0.2 0.1 0% Taxes on professions, trade and 0.4 1.0 0% 14 World Bank, Implementation Status Report and Results Report (ISR), November 23, 2020 and modified on December 14, 2020. Page 12 The World Bank PK Sindh Public Sector Management Reform Project (P145617) callings Capital value tax on immoveable 2% 3.3 4.5 property 2. Indirect taxes 170.4 220.2 General Sales Tax on Services (GSTS) 93.2 135.0 57% Excise 5.1 6.0 3% Stamp duties 9.9 13.0 6% Motor vehicle taxes 7.3 7.8 3% Other indirect taxes 54.9 58.4 Hotel taxes 0.0 0.0 0% Entertainment taxes 0.1 0.1 0% Electricity taxes 0.7 1.1 0% Sindh Development 24% 53.9 56.9 Maintenance of Infrastructure Cotton fees 0.2 0.2 0% Others 0.1 0.1 0% Source: Sindh Government statistics, verified by the World Bank team (May 2021). 27. Another outcome of the PDO was improved expenditure management. More specifically, the expected outcome was to enhance “transparency in budget formulation, allocation and execution”. These targets were consistently achieved each successive fiscal year, starting from FY2014-15 to FY2019-20. The Implementation Status and Results Report (ISR) of November 23, 2020 confirms that the Finance Department (FD) had submitted the Budget Strategy Paper for FY2019-20 to the Cabinet. The FD also submitted all four budget execution reports for FY2018-19 to the Provincial Assembly and published them on its website.15 (See also see Annex 8 for additional information concerning the Province’s PFM performance). It is important to note that the Government of Sindh has continued to submit both the Budget Strategy Paper and Budget Execution Quarterly Reports for FY 2020-21 following the closure of the SPSMRP 16. 28. Regarding the PDO results indicators, Table 3 confirms that they have been achieved. Table 3: PDO Level Results Indicators PDO-Level Results End of Project Actual Target(s) Achieved Baseline Comments/Variance Indicators Target(s) (during ICR) Indicator 1: Based on the cumulative Improved revenue collection, the collection of Sales 3.50 3.50 7.00 target was achieved and Tax on Service surpassed by 50 percent. (STS) (Percentage) Indicator 2:  Budget The Finance All targets have been This indicator was changed Transparency in Strategy Department (FD) has: achieved. to the PDO indicator. budget Paper was The FD has: 15World Bank ISR of November 23, 2020. 16See the attached links as evidence: https://www.finance.gos.pk/Budget/BudgetStrategyPaperhttps://www.finance.gos.pk/Budget/BudgetQuarterReportshttps://www.finance.gos.pk/ Home/Download?path=Budget%2FQuarterReports%2F2020-21-Q4.pdf Page 13 The World Bank PK Sindh Public Sector Management Reform Project (P145617) formulation, not  Submitted to the  Submitted the Budget The target was achieved and allocation, and prepared. cabinet the Strategy Paper for verified by a TPV. execution (DLI)  One Budget Strategy FY2019/20 to the quarterly Paper for Cabinet. report was FY2019/20.  Submitted all four submitted  All four budget budget execution to the execution reports for Provincial reports for FY2018/2019 to the Assembly. FY2018/19 were Provincial Assembly; submitted to the they were also Provincial published on the FD’s Assembly; they website. were published on the FD’s website. Source: Results Framework Performance Table. Note: DLI= Disbursement-linked Indicator; PDO= Project Development Objective; TPV=Third-Party Verification. 29. As noted, the project had design challenges in the form of overly ambitious targets and measurement difficulties related to PDOI (2) and PDOI (3). However, the willingness of the SRB to embrace and implement the recommendations of the restructuring paper greatly contributed to the achievement of the PDOIs, the IRIs and the DLIs, as highlighted in Tables 2 and 3. 30. Of the original 16 IRIs, three were dropped. All of the remaining 13 IRIs and DLIs were achieved, as follows: Approval and initial implementation of the Sindh Revenue Mobilization Reform Plan (DLI); (ii) Enhanced SRB resource capacity for administration (DLI); (iii) Increased automation of SRBs (DLI); (iv) Increased tax base (IRI); (v) Risk-based audits of taxpayers implemented (DLI); (vi) Improved SRB performance transparency (IRI); (vii) Debt management system instituted (IRI); (viii) Establishment of internal audit (IRI); (xi) Details of internal audit for FY2017-18 and 2018-2019; (xii) Procurement officials certified (DLI); (xiii) Quarterly departmental and plan, monitoring reports prepared and published (DLI); (xiv) Departments where geo-tagging of development schemes is implemented, and information published (DLI)17; and (xv) Department with proactive feedback mechanism established (IRI). 31. Table 3 provides a snapshot of information about achievements, especially the column concerning “actual targets achieved” and the “column on variances”. Table 4: IRI and DLI Performance Table Intermediate Results (IR) Baseline End of Project Target(s) Actual Target(s) Achieved Comments/Variance Indicators 17IRI 12: Six of 44 have been achieved, which was the target. The Monitoring and Evaluation Cell (MEC) is working to guarantee the sustainability of this initiative, and the current number of departments on the dashboard is 39. The MEC has also added a few attached offices of these 39 departments to the dashboard. Page 14 The World Bank PK Sindh Public Sector Management Reform Project (P145617) IR Indicator 1: Approval and Tax Reform Plan The FD has publicly disclosed initial approved in the Sindh Tax Revenue implementation February 2014; Mobilization Reform Plan. The Sindh Tax Revenue of Sindh Tax 2012-13 STS Mobilization Plan was Target achieved. Revenue collection is The SRB’s collection of STSs disclosed. FY16-17. Mobilization Pakistani Rupee for FY2013/14 reached PKR Reform Plan. (PKR) 34 billion. 39.43 billion. (DLI 1) Partially achieved. IR Indicator 2: Enhanced the The SRB has approved a The Board of the SRB Targets for this Staffing and SRB’s human revised staffing plan in approved the revised indicator were revised training plan is resource accordance with the staffing plan in accordance during the 2018 project under capacity for recommendation of the third- with the recommendation restructuring. development. administration. party review. of the third-party review. (DLI 2) The revised target was achieved. All targets have been Zero automated achieved. IR Indicator 3: notices. Unique The SRB has issued notices to Increased taxpayer at least 90 percent of non- The SRB has issued notices automation of identification compliant registered to at least 90 percent of Target achieved18. the SRB’s system does not taxpayers in FY18/19 through non-compliant registered systems. (DLI 3) exist. No third- automated systems. taxpayers in FY18/19 party linkages. through automated systems. Target achieved and exceeded. The indicator was measured as a cumulative percentage increase from a baseline of 4,800 IR Indicator 4: taxpayers in FY13/14. Increased tax The original target 4,800.00 25.00 307.00 base estimated was a 25 (percentage) percent increase with 9,563 taxpayers. According to the TPV report, in FY18-19, the number of registered taxpayers was 31,144, which represented a 307 percent increase. 18 The third-party linkages are now established with the National Database Registration Authority (NADRA), the Federal Board of Revenue (FBR), and the Controller General of Accounts (CGA) Pakistan. Unique Taxpayer Identification has been implemented for the SRB’s taxpayers, as the National Tax Numbers are being used for the taxpayers’ ID with the letter “S” (for Sindh Province). This letter is inserted at the beginning of the ID. Page 15 The World Bank PK Sindh Public Sector Management Reform Project (P145617) All targets were achieved. The SRB has: The SRB has:  Issued and  Issued and communicated audit communicated audit observations to the observations to the concerned taxpayers. concerned taxpayers.  Finalized the audits  Finalized the audits initiated in FY2017/18. initiated in FY2017/18. IR Indicator 5:  Initiated assessment  Initiated an assessment Targets for this Risk-based (by the SRB’s (by the SRB’s Operations indicator were revised audits of Operations Wing) System absent. Wing) under the during the 2018 project taxpayers under the relevant relevant provisions of restructuring and implemented. provisions of the law. the law. achieved. (DLI 4)  Selected audit cases  Selected audit cases according to revised according to revised risk- risk-based criteria, based criteria, which which were updated were updated according according to the to the feedback feedback obtained obtained from the from the completed completed audit cases audit cases initiated initiated in FY2017/18. in FY2017/18. Annual reports All targets have been IR Indicator 6: irregular: Four quarterly reports and achieved. Improved SRB quarterly one annual report were Four quarterly reports and Target achieved. performance reports are not published. one annual report were transparency. published. published.  FD develop a Debt database and debt IR Indicator 7: All the targets were Targets for this Debt manual, Publishes debt Debt achieved but with the indicator were revised management bulletins, Sensitivity management adjustment of few during the June 2018 systems do not analysis report and debt systems verification protocols. project restructuring exist. sustainability analysis instituted. and achieved. document  Four government  Four government departments, including departments, the Finance Department, including the Finance have conducted internal Department, have audits and submitted conducted internal IR Indicator 8: internal audit reports to audits and submitted Establishment Internal audit their Principal internal audit reports of an internal function does Accounting Officers Target achieved. to their PAOs. audit function. not exist. (PAOs).  The remaining three (DLI 5)  The remaining three PAOs have undertaken PAOs have undertaken actions in response to actions in response to the forgoing internal the forgoing internal audit reports within a audit reports within a month of such month of such reports. reports. Page 16 The World Bank PK Sindh Public Sector Management Reform Project (P145617) The FD has: The FD has:  Submitted the Budget Budget Strategy  Submitted the Budget Strategy Paper for FY IR Indicator 9: Paper not Strategy Paper for FY 2019/20 to the Transparency in prepared. 2019/20 to the Cabinet. Cabinet. budget  Submitted all four  Submitted all four formulation, One quarterly Target achieved. budget execution budget execution allocation, and report reports for FY2018/19 to reports for FY2018/19 execution (DLI submitted to the Provincial Assembly to the Provincial 6). the Provincial and published them on Assembly and Assembly. the FD’s website. published them on the FD’s website. The SPPPRA has: Targets have been achieved and surpassed. IR Indicator 10:  Certified at least five  Six hundred and No certification Procurement hundred officers of ninety-six officers mechanism Target achieved. officials grade Basic Pay Scale certified as of exists. certified. (DLI 7) (BPS)-16 or above. November 25, 2020.  Conducted a third- party  The TPV confirmed review of the the achievement of certification process. the target. The Planning and All targets have been Development Department achieved. (P&DD) has:  Quarterly progress IR Indicator 11:  Prepared and published reports for six Quarterly at least twelve quarterly departments (that is, development reports concerning the 17 reports in total) Target achieved. plan monitoring implementation of - prepared and reports development schemes published. prepared and for at least four  The TPV report was published. (DLI government conducted and 8) departments confirmed the collectively. achievement of the  Conducted a third-party targets. review. IR Indicator 12: Departments where geo- tagging of Target achieved. development 0.00 6.00 6.00 schemes is implemented, and information published. All targets were achieved. Collection of community Monitoring and Evaluation IR Indicator 13: feedback for at least 10 Cell (MEC) Departments percent of provincial Annual collected community with proactive - Development Plan (ADP) feedback for at least 10 Target achieved. feedback schemes costing over PKR percent of provincial ADP mechanism 1billion in at least four schemes costing over PKR 1 established. government departments. billion in at least five government departments. Page 17 The World Bank PK Sindh Public Sector Management Reform Project (P145617) Justification of Overall Efficacy Rating 32. This section of the ICR provides evidence to confirm that good progress has been made toward the achievement of the PDO. The restructuring of the project strengthened the capacity of the SRB and the associated implementing entities to achieve the revised PDOIs and related targets. Of the 16 IRIs, 3 were dropped and the remaining 13 were achieved, representing an 82 percent achievement in comparison to the original project design. The overall achievements are captured in Tables (2), (3) and (4) of this report. The analysis of the DLIs, IRIs and the PDOIs clearly shows this progress. Initial implementation challenges were overcome through project restructuring, which brought the project to a successful closure. 33. Under Component 1, the project achieved the following during FY2014-15 to FY2018-19: (a) developed the Sindh Tax Revenue Mobilization Plan (2014-2019) for tax collecting agencies; (b) improved sales taxes for service collection (see also the paragraph below concerning revenue collection); (c) developed the Budget Strategy Paper and submitted Quarterly Execution Reports to the Provincial Assembly of Sindh (see footnote 16); (d) enhanced the SRB’s human resource capacity for administration; (e) increased the automation of the SRB systems; (f) increased the tax base in the SRB; (g) implemented the risk-based taxpayer audits; (h) improved SRB transparency on performance by issuing Quarterly and Annual Performance Reports, which are then published on the SRB website; (i) instituted a debt management system in the Province; (j) established an Internal Audit (IA) function in four departments, namely the departments of finance, health, home, and school education; (k) nearly 700 employees of the GoS from the BS-16 level and above procurement certified; and (l) developed the ADP Dashboard with geo-tagging of development schemes and community feedback. 34. Under Component 2, the project achieved the following during FY2014/15 to FY2018/19: (a) provided IT support to the SPPRA, the P&DD and the Excise Taxation and Narcotics Control Department (ET&NCD); (b) conducted a survey to support the Excise Department in handling property and professional taxes in the Sukkur district (FY2017 to 2019); (c) carried out an Actuarial Valuation of Employee Benefit Schemes to ascertain the unfunded liabilities; (d) developed Standard Operating Procedures (SoPs) for the FD; (e) conducted a review of development statistics of the Sindh Bureau of Statistics, and shared recommendations with the Pⅅ (f) established a state-of-the-art Data Center Facility in the FD, which went live on January 29, 2021. It could help line departments in housing their data; and (g) made several reforms in the Sindh Government Servants Benevolent Fund. These project achievements were made by utilizing a good mix of performance-based incentives and technical assistance. As such, they may provide a useful roadmap for future projects. Based on these achievements, the PDO (efficacy) was achieved, and the overall efficacy is rated as High. C. EFFICIENCY Assessment of Efficiency and Rating 35. The efficiency gains from the project interventions were considerable relative to the costs incurred during the implementation of the SPSMRP. The SPSMRP resulted in significant achievements as listed in section 2.2 of this ICR (see Tables 2, 3 and 4). These achievements included, among others, the development of robust support systems and innovations that should serve the Province of Sindh and the citizens who reside in it, as well as future generations. As a result of the SPSMRP, the province can now boast of improved revenue generation, especially the collection of sales taxes, as well as the enhanced management of public finances and improvement of the core competencies of the SRB staff. Additionally, there were efficiency gains in procurement. 36. The SPSMRP has improved the capacity of the province to collect more revenues. The collection of sales taxes has also systematically improved, rising from as low as 49 percent of total taxes in 2013-2014 to as high as 57 percent in Page 18 The World Bank PK Sindh Public Sector Management Reform Project (P145617) 2019-2020, as shown in Table 5. Overall, the project has exceeded its target of 3.5 percent annual collection growth (minus inflation) for the General Sales Tax on Services (GSTS). Table 5 shows the annual growth in GSTS in nominal and real terms. Table 5: General Sales Taxes on Services 2013-14 2014-15 2015-16 2016-17 2017-18 2018-19 2019-20 RE General Sales Tax on Services (GSTS) - Billion Pakistan Rupees 39 49 61 76 92 93 135 (PKR) GSTS share of total tax revenue 49% 51% 49% 53% 53% 52% 57% GSTS growth (year-on-year [YOY]) - Nominal 26% 24% 25% 21% 1% 45% GSTS growth (YOY) - Real 20.80% 21.40% 21% 17% -4% 34% Source: Punjab and Sindh civil accounts and budget documents, KPRA, and Balochistan Revenue Authority. RE is revised estimates. Note: The Sindh average growth rate is 24 percent. Table 6: Annual Growth in General Sales Taxes on Services (GSTS, PKR) Estimated Estimated Revenues Actual Present PV of Revenues with Years without Project Revenues at Difference Value (PV) of Difference at Project (PAD - Intervention ICR Difference PAD High Scenario) (PAD) 2013-14 394 394 394 (Baseline) 412 495 83 79 416 3.9 2014-15 430 616 186 169 438 7.5 2015-16 450 724 274 237 463 11.2 2016-17 470 876 406 334 488 15 2017-18 491 769 278 217 515 18.8 2018-19 1.036 56.4 Source: World Bank Staff calculations 37. The increased revenue collection and savings derived from public spending through better financial and procurement practices made funds available for the development of the province . In the PAD, the section on the economic analysis of the project estimated that the net present value of the next five years was approximately US$13.9 million under the low side scenario; US$28 million under the moderate scenario; and US$56.3 million under the high scenario. For the next 25 years, the projected figures were US$101.9 million under the low scenario; US$204.8 million Page 19 The World Bank PK Sindh Public Sector Management Reform Project (P145617) under the moderate scenario; and US$413.3 million under the high scenario. In short, the increase in tax revenues was deemed possible because the cost of the implementation of the project far outweighs the cost of interventions under the SPSMRP.19 38. The project has exceeded its overall target of 3.5 percent growth in annual collection, minus inflation, on the General Sales Tax on Services (GSTS). Table 6 shows the annual growth in the GSTS in nominal and real terms. The total number of registered taxpayers has also increased by 307 percent from a baseline of 4,800 as of June 30, 2014, that is, it has increased to 31,144 as of the end of June 30, 2019. The present value using actual revenue data for five years and the methodology used in the PAD is US$1.036 billion, which has substantially exceeded the PAD estimates at the high scenario. At the PAD stage, the revenue growth rate for the high scenario was estimated at 5.5 percent compared to the double-digit actual growth rate in revenues achieved during this period. The benefits of increased tax collection will go beyond these years because the increase in tax base and the improved capacity of the tax department will contribute to higher tax collection beyond the project period. The detailed analyses are presented in Annex 4. 39. Hence, the increase in revenues alone covers the cost of the intervention in all proposed areas. Moreover, the automation of the tax system, human resource management reforms in the tax machinery, and the institutionalization of evidence-led tax policy will further improve revenue collection. The implementation of risk-based audits of taxpayers will result in improved compliance and less evasion. 40. Although budget execution is not easily quantifiable, the SPSMRP has enhanced the transparency in initiating and executing budgets. The benefits of the project will also flow from improved budget credibility; strengthened budget execution; better internal controls; enhanced oversight; and increased transparency in the management of public resources. The project has also supported the development of sustainable human resource capacity. especially in PFM. Thus, the achievable benefits of this project have more than offset the cost of implementing the project. As noted, there were also efficiency gains in procurement. One of the expected outcomes of the project was to enhance “transparency in budget formulation, allocation and execution”. These targets were consistently achieved each successive fiscal year, starting from FY2014-15 to FY2019-20. Thus, the efficiency of the SPSMRP is rated as Substantial. D. JUSTIFICATION OF OVERALL OUTCOME RATING 41. The SPSMRP experienced some initial problems in the first three years of implementation. The timely intervention of the World Bank implementation support teams following the findings from the MTR and the willingness of the GoS to proceed with the project restructuring in June 2018 (as emphasized in the previous sections of this ICR), contributed to removing implementation bottlenecks. This resulted in the achievement of all the IRIs, the DLIs and above all, the PDO. The ICR did not apply a split rating due to the following reasons: (i) project restructuring did not result in changes to the PDO; (ii) the key project outcome areas of improved revenue and expenditure management continued to remain highly relevant and unaltered; and (iii) the modifications introduced to the PDO level indicators did not in any way compromise the level of ambition or achievement of key project outcomes. In view of these achievements and despite the initial implementation challenges, the overall outcome rating is Satisfactory. 19 World Bank, Project Appraisal Document (PAD), paragraph 55, page 31. Page 20 The World Bank PK Sindh Public Sector Management Reform Project (P145617) III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME A. KEY FACTORS DURING PREPARATION 42. The outcome section highlighted the key achievements of the SPSMRP, despite the initial implementation bottlenecks. Apart from the favorable overall outcome rating, it is important to draw lessons from the key factors that affected the implementation and outcome of the project. 43. Several factors influenced the preparation of the SPSMRP. The project context was very well captured in the PAD, which articulates the rationale underlying the initiation of the project. The context was characterized by inadequate revenue generation, and poor expenditure management, that slowed the development of the Province of Sindh. The twin problems of weak revenue mobilization and poor public expenditure management are clearly reflected in the PDO statement. There were also extensive consultations among the World Bank project preparation team and representatives of the national and provincial officials of Pakistan who were drawn from the economic sector. The two teams worked in tandem during project preparation. Workshops were held to clarify issues and to agree on the PDO statement and definitions. These were related to the PDO level results indicators (PDOIs); intermediate-level results indicators (IRIs), and the disbursement-linked indicators (DLIs). The PDO statement was realistic. 44. However, as previously mentioned in this ICR, two of the PDO-level results indicators were not easily measurable, and some of the targets were overly ambitious. Subsequently, PDOIs (2) and (3) were dropped and several targets were revised during the project restructuring. The project components of results-based financing and technical assistance to support the achievement of the DLIs were clear, and the activities were well delineated. The Bank team held extensive consultations with government counterparts prior to finalizing the PAD for Board approval on June 28, 2015. 45. The risk and mitigation measures outlined were adequate; however, the institutional capacity to implement the SPSMRP should have received greater scrutiny. The Sindh Revenue Board, which served as the main implementation agency of the project, initially experienced some capacity constraints. It was supported by the DLI (“Enhanced SRB Human Resource Capacity for Administration”) and was able to address these constraints. As a result, the SRB was able to hire and train 150 Assistant Commissioners, Sales Tax Officers, and Auditors. Additionally, the timelines for the launching of activities may have been too optimistic. The initial delays encountered in the implementation of project activities could have been largely avoided if the implementation schedule had received closer scrutiny given the above-mentioned capacity constraints. The Financing Agreement (FA) of the project was signed on March 11, 2015, and the SPSMRP became effective on April 28, 2015. B. KEY FACTORS DURING IMPLEMENTATION 46. Prior to the restructuring of the SPSMRP in June 2018, some of the activities related to enhancing PFM and debt management could not be implemented due to design issues with the indicators. For example, the PDO indicator associated with improving the credibility of the budget and the indicator related to measuring extra budgetary releases were deemed to have not been properly defined. The compendium targets were also poorly set. Moreover, there were difficulties in achieving some of the targets, such as debt management, internal audit, preparation of the quarterly departmental development plans, and monitoring reports, these factors also contributed to the implementation challenges. Page 21 The World Bank PK Sindh Public Sector Management Reform Project (P145617) 47. There were significant delays in the implementation related to strengthening procurement management and practices, which were largely attributable to poor design. The complexity of implementing public procurement reforms across the board were underplayed or poorly understood during the preparation of the project. This lack of understanding of the public sector environment invariably led to the setting of targets that were deemed unrealistic; as such, they were the least likely to be achieved. However, it should be noted that after considerable delays, the public procurement management information system (PMIS) — the establishment of which was a pre-condition for the achievement of other indicators/DLIs — was successfully launched. 48. The advent of the COVID-19 pandemic may have affected the implementation of activities prior to project completion. The impact of the pandemic on the project was somewhat captured in the last two ISRs of June and November 2020. However, a detailed assessment of the impact that the increasing severity of the pandemic had on some of the project areas could have provided a more comprehensive picture of the project’s implementation path. 49. At the time of project restructuring in June 2018, the project’s overall outcome rating was Moderately Unsatisfactory (MU). However, once the implementation bottlenecks were addressed through the modified project design, the revision of the Results Framework (RF), and the revamping some of the project activities (as highlighted above), the overall project outcome rating improved to Moderately Satisfactory (MS). IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME A. QUALITY OF MONITORING AND EVALUATION (M&E) M&E Design 50. The Sindh Revenue Board has a strong sales tax collection system. It was tasked with the responsibility of systematically tracking and reporting on key performance indicators as defined in the project’s results framework. The DLIs were tracked through third-party verification missions. In addition, the Economic Reform Unit in the Finance Department had the overall responsibility for tracking and reporting on results. M&E Implementation 51. Systematic monitoring of the SPSMRP was conducted by both the client and the Bank through implementation support and supervision missions (throughout the life cycle of the SPSMRP). The tracking and reporting in the ISRs focused on the PDOIs, the IRIs and the DLIs. During the restructuring, some PDOIs were dropped, and three of the 16 IRIs were also dropped, as indicated in the previous sections of the ICR. 52. On the client side, a M&E Cell was established within the Planning and Development Department (P&DD). The P&DD had overall responsibility for tracking project activities and assessing the achievement of the PDOIs and the IRIs. The assessment of the DLIs was outsourced to an independent, Third-Party Verification (TPV) team to assure greater transparency. 53. The P&DD established a dashboard for investment interventions that enabled tracking and reporting of implementation progress and deliverables on a systematic basis. The P&DD also had responsibility for monitoring the Annual Development Program (ADP). Overall, the M&E Cell was strong on the supply side in terms of the preparation of Page 22 The World Bank PK Sindh Public Sector Management Reform Project (P145617) reports, but weak on the demand side of the M&E equation. There is not enough evidence to assess how effectively the P&DD utilized the available data/information during project management/implementation. 54. On the Bank side, implementation support and supervision missions were regularly conducted. The MTR mission was completed in November 2018, and its findings and recommendations resulted in the restructuring of the SPSMRP. 55. Following each mission, Aide Memoires and Implementation Status Report and Results were prepared, and key issues were flagged to the attention of the client and the Bank’s management. These regular missions and the attendant observations that were shared enabled the Bank team and the client to make the necessary corrective measures and improve project implementation and performance toward achieving the PDO. The removal of bottlenecks was the result of the openness with which the Bank engaged with the client, as well as the willingness of the client to accept recommendations resulting from the various implementation support missions and the findings and recommendations of the MTR. The implementation of MTR recommendations and clarification of design issues and revisions to selected targets greatly improved the achievements of the PDOIs, IRIs, and the DLIs. As a result, project ratings that were previously downgraded to Moderately Unsatisfactory (MU) changed to Moderately Satisfactory (MS) following the restructuring. It should also be noted that progress toward the achievement of the PDO was rated as Moderately Satisfactory (MS) in the last ISR (November 2020), and the overall implementation progress (IP) was rated as Moderately Satisfactory (MS). The performance of the SPSMRP systematically improved after project restructuring in June 2018. 20 M&E Utilization 56. Information gathered through the routine quarterly and annual reports was used by the PIU to improve project implementation and management. Findings and conclusions emanating from Aide Memoires and ISRs were publicly shared. Bank staff and management made frequent use of the information contained in these documents such as through regular portfolio reviews. Justification of Overall Rating of Quality of M&E 57. The overall M&E rating is Substantial, and this in in line with the rating provided in the last ISR of December 23, 2020, which rated M&E as Satisfactory. B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE Environmental and Social Safeguards 58. The project did not involve any construction and/or physical works of any nature, and its activities were largely related to softer interventions. Moreover, it focused on reforms, such as strengthening public sector performance through improved revenue generation and expenditure management; training; enhancing human resource capacity; and system automation. These activities do not have any environmental and social implications/issues. The project was classified as Category C.21 Hence, no safeguard policy was triggered. 20 See page 2 of the ISR, December 23, 2020. 21 A proposed project is classified as Category C if it is likely to have minimal or no adverse environmental impacts. Beyond screening, no further EA (Environmental Assessment) action is required for a Category C project (Ref: OP 4.01 - Environmental Assessment 1999 (Revised 2013); Section 8 (c). Page 23 The World Bank PK Sindh Public Sector Management Reform Project (P145617) Procurement 59. The project’s procurement plan was updated during the restructuring. For example, the PDO indicator relating to the Procurement Management Information System (PPMS) was dropped. The dropped component of the PDO was the timeline of the procurement cycle related to the period from advertisement until the award of the contract. This process was linked with the PPMS so that Sindh Public Procurement Regulatory Agency (SPPRA) was planning to obtain this data from the PPMS rather than by collecting the data through a firm. When the PPMS eventually became functional, this timeline data was reported sooner than had been done in the past. Currently, all procurement data is posted in the system. The SPPRA is the only procurement agency in Pakistan that has this system, with the availability of real-time data. This year, the SPPRA worked to convert the database to open contracting in data format. 60. A new procurement plan was prepared to accelerate the allocation of funds and implementation of activities under the technical assistance component. The new plan better clarified the timelines and procedures. Under the revised plan, the SRB awarded two IT contracts and one individual contract. The total contracts awarded by the SRB were estimated at US$3.2 million, with the largest estimated cost for a single contract of US$1.1 million for the purchase of computer software. The Economic Reform Unit (ERU) awarded nine consultancy contracts, three IT contracts, and two contracts for the purchase of goods. The ERU and the SRB reviewed their procurement needs to ensure the remaining activities were fully funded and implemented before project closure. The readiness of the SRB to utilize the TA funds was a challenge throughout project implementation. However, the ERU was tasked with identifying the needs of the Finance Department, as well as other departments; the latter required more effort in terms of establishing timely coordination. 61. According to the Restructuring Paper, the project’s expected results in public procurement were adjusted to more realistic levels because of the complexity of the proposed reform, as well as the limited results achieved prior to the restructuring.22 The DLIs related to the certification of procurement officials were clarified during the restructuring. As a result, about 696 officers were certified in November 2020.23 This certification program makes Sindh the only province in the country to require mandatory procurement certification. To date, the Sindh Public Procurement Regulatory Authority (SPPRA) also remains the only procurement authority in the country where all procurements are captured and entered in a database. Given these achievements, the ICR rates procurement as Satisfactory. Financial Management 62. Financial management arrangements remained adequate during the life of the project, which mainly supported the provincial government expenditures through its recurrent budgeting. Disbursements were made upon the achievement of the DLIs, which were verified by the TPV firm. All DLIs were met within the required timeframe, and related funds were disbursed. The project had two small TA components with each of the implementing entity where complete utilization of funds was achieved only by the ERU. The SRB component was not able to fully utilize the allocated funds, and unutilized funds are to be returned. This is currently taking place and is pending with the Government of Sindh. The project prepared regular financial reports with the support provided by the Bank team. 63. The Bank received the project’s audited Annual Financial Statement within six months of each financial year. The external audit was conducted by the Office of the Auditor General of Pakistan (AGP), the Supreme Audit Institution of Pakistan which was acceptable to the Bank. The external audit was conducted in accordance with the International Standards of Supreme Audit Institutions, and the AGP expressed an unqualified opinion in the audit reports, which were 22See page 8 of the Restructuring Paper. 23This is the number of certified employees compared to the total number of staff in grades BS-16 and above (the latter number was difficult to determine). This was a new reform, and the SPPRA was still able to achieve these numbers, thus exceeding the target. The firm that served as the Third-Party Verification (TVP) also did not compare the number with any other criteria which were not part of the target. Page 24 The World Bank PK Sindh Public Sector Management Reform Project (P145617) submitted during each financial year. The project did not perform any internal audits during its entire lifecycle, although the Bank had repeatedly informed the borrower of the importance of doing so. As noted, financial management issues identified in the early phase of implementation were identified and addressed in time. Although the SRB’s Financial Management Team had been slow, overall fiduciary aspects were satisfactorily supervised. Thus, the financial management of the project is rated as Moderately Satisfactory. Disbursements 64. The Bank approved an amount of US$50 million for the SPSMRP. As shown in Table 7, US$40 million of this amount was allocated to Component1 (results-based financing) and US$10 million to Component 2 (TA). The project had a 100 percent disbursement in Special Drawing Right (SDR) terms, and the difference between the original approved amount of US$50 million and the final amount of US$47.38 is due to a change in the exchange rate (SDR) during the project life. Table 7: Total Disbursements as of Project Closure Amount Allocated Percentage Disbursed (US$ million) (%) Component 1 40.00 Component 2 10.00 IDA (Source of Funds) 50.00 Total Amount Disbursed (Last ISR, November 2020) 47.38 100 C. BANK PERFORMANCE Quality at Entry 65. The quality at entry is rated Satisfactory. The Project benefitted from greater due diligence and scrutiny before the Board approval, and the Bank team extensively consulted the objective and scope of the Project with all relevant agencies. This engagement cumulated in the preparation of the PAD and the thorough consultations engendered a greater trust between the Bank and the Government with regards to the proposed Project design. Quality of Supervision 66. Adequacy of supervision inputs and processes. Over the life of the project, the Bank devoted nearly 200 staff weeks and US1.2 million for project preparation, supervision and extended implementation support. Twelve ISRs were filed in the course of project implementation and made publicly available. Implementation support missions were undertaken regularly with frequent technical discussion in-between to discuss progress ratings and bring key issues to the attention of management. The Bank's multidisciplinary teams (including skills in FM, procurement, social and environmental safeguards) reviewed relevant documents before, during, and after the implementation support missions, including the project restructuring and MTRs. The ISRs, Aide Memoires (AMs) and Management Letters (MLs) reflected well and candidly the project implementation progress. Page 25 The World Bank PK Sindh Public Sector Management Reform Project (P145617) 67. There was a high turnover of Task Team Leaders (TTLs) and Co-TTLs. At the last count, the SPSMRP had had eight TTLs and Co-TTLs who supervised the project. However, this did not cause any negative impact on the project implementation pace, given the readiness of the Bank staff to provide technical advice and extend its support to the PIU on critical implementation matters. Justification of Overall Rating of Bank Performance 68. Bank staff consistently provided quality advice to the client. This led to the removal of bottlenecks and enhancement of project performance. However, the TTLs who managed the project prior to the MTR could have been more proactive in providing evidence updates to the counterparts/implementing agency concerning the slow implementation. Still, the project restructuring initiated after the MTR paved the way for the much-needed changes that led to the project moving from an unsatisfactory rating to a satisfactory rating. Overall, the Bank performance is rated as Satisfactory. D. RISK TO DEVELOPMENT OUTCOME 69. As noted, the project ran into severe bottlenecks which slowed the implementation of activities. However, once these lapses were corrected through a level-2 restructuring, the project was back on track. Under the project, over 1,300 staff have been trained and more sales taxes have been collected. Public expenditure management has also been enhanced through the development of a management information system; the design of the procurement certification course and the establishment of a transparent budget management system — all of which have reinforced the sustainability of the project. Therefore, any risk to development outcomes after the Bank’s exit is likely to remain low. Thus, the ICR concludes that the reforms achieved under the SPSMRP will most likely be sustained in the coming years. V. LESSONS AND RECOMMENDATIONS 70. The political context needs to be taken into consideration at the design stage to identify all conceivable risks and potential mitigation measures. Although it may not be possible to foresee all impending challenges, particularly during implementation, the Bank team could still have spent more time in assessing the regional government’s mandate, implementation capacity, commitment to adopting the proposed reforms, as well as the suitability of the Bank’s (project) instrument. This is particularly the case when designing the set of PDOIs and IRIs, some of which were dropped during the restructuring. The Bank team and the client need to take quick actions by proactively finding solutions to the challenges when they arise. The SPSMRP benefited, albeit with some delays, from Bank advice and demonstrated flexibility in adopting the recommendations resulting from the MTR and the Restructuring Paper. 71. Project restructuring could be effectively used as an important tool for turning around an initially non- performing project. Project restructuring, when initiated in a timely manner, could provide an early opportunity to put the project on a corrective path. Thus, potential implementation delays could be avoided. Although Bank teams usually wait until the MTR to make a comprehensive assessment concerning project performance. However, at times, it will be beneficial to implement a restructuring prior to the MTR after reassessing the relevance and the efficacy of some of the results proposed during the design, especially concerning how best to include the right mix of performance-based incentives and technical assistance to incentivize implementation success. As the current project achievements show, training and IT support could be effectively utilized as key incentives to sequence other reforms. Thus, it set an ambitious level during project design and restructuring. These are some of the interim ‘milestones’ that the Bank supervision missions could continue to assess, as well as reporting on the progress of specific project targets. Page 26 The World Bank PK Sindh Public Sector Management Reform Project (P145617) 72. A prior political economy analysis of the demand for reforms before launching the operation would have averted most of the implementation delays. It should not be assumed that once some of the reforms are introduced, a demand for such reforms within all project counterparts, or a champion to spearhead these reforms throughout the implementation cycle will automatically arise to make these reforms sustainable. Hence, a priori, an analysis of the potential risks and challenges with an associated road map for risk mitigation and implementation support strategy could have minimized some of the project implementation delays encountered prior to the MTR mission and project restructuring. 73. Each key policy reform area needs to be supported by sufficient complementary assistance to guarantee better reform implementation. For instance, when the Development Policy Operation (DPO) focuses on adopting measures to enhance tax revenue mobilization, the performance of PFM systems, the management of the development portfolio, and the strengthening of the public procurement system, it is important to consider the need to provide corresponding support. Such support will be needed through a well-coordinated TA to operationalize such reforms. For instance, reforms in tax administration alone rarely led to increased revenue collection unless there is a deliberate effort to train staff to acquire the requisite core competencies in tax and financial management. The project’s eventual success is, in part, due to the TA which supported training seminars, conferences, and the purchase of equipment for the implementation of the reform activities of the Sindh Tax Revenue Mobilization Plan (STRMP). 74. High turnover of both World Bank and Project staff invariably affects project implementation. As noted above, the high turnover of Bank staff who were directly involved in the supervision of the project was not significant in terms of its impact on project implementation. Rather, it was the readiness of the staff to provide technical advice and support to the PIU that made the difference. The willingness of the implementing agency and the client to reach out to the World Bank for necessary technical support before and during the implementation, as evidenced in this particular project, could go a long way to overcoming potential implementation delays, thereby enhancing the project’s efficacy. High project staff turnovers are particularly challenging at times when several exogenous factors, such as the current pandemic, affect project implementation. This potential, unpredictable event could be taken into consideration at the design stage of future projects so that the Bank team and the client could think through a series of mitigation measures to be adopted when required, including by ensuring consistency in the sector-specific and fiduciary specialists on the team, especially in the Country Offices where frequency of interactions with the client are highest. . Page 27 The World Bank PK Sindh Public Sector Management Reform Project (P145617) ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS A. RESULTS INDICATORS A.1 PDO Indicators Objective/Outcome: Strengthen public sector performance through improved revenue generation and expenditure management Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Improved collection of Sales Percentage 3.50 3.50 7.00 Tax on Services (STS) 30-Jun-2014 30-Jun-2019 04-Jun-2020 Comments (achievements against targets): Based on the cumulative revenue collection, the target was achieved and surpassed by 50 percent. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Transparency in budget Text a) Budget Strategy The Finance All targets have been formulation, allocation and Paper not prepared Department (FD) has: achieved. execution (DLI) b) One quarterly • Submitted to FD has: (i) submitted report submitted to the cabinet the to the Cabinet the the Provincial Budget Strategy Paper Budget Strategy Paper for FY2019/20. • for FY2019/20; and (ii) Page 28 The World Bank PK Sindh Public Sector Management Reform Project (P145617) Assembly All four budget submitted all four (4) execution reports for budget execution FY2018/19 were reports for FY2018/19 submitted to the to the Provincial Provincial Assembly; Assembly and they were published published them on on the FD’s website. FD's website. 30-Jun-2014 30-Jun-2019 04-Jun-2020 Comments (achievements against targets): This indicator was changed from an intermediate to a PDO level indicator. The target was achieved and verified by TPV (Third Party Verification). A.2 Intermediate Results Indicators Component: Component 1 and 2 Intermediate Results Indicators Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Approval and initial Text Tax Reform Plan • The FD has (i) FD has publicly Target is already met. implementation of Sindh Tax approved in Feb 2014; publicly disclosed the disclosed the Sindh (i) FD has publicly Revenue Mobilization 2012-13 STS collection Sindh Tax Revenue Tax Revenue disclosed the Sindh Reform Plan (DLI) is PKR 34 billion. Mobilization Reform Mobilization Reform Tax Revenue Plan. • The Plan. (ii) SRB’s Mobilization Reform SRB’s collection of collection of STS for Plan FY16-17. STSs for FY2013/14 FY2013/14 reached reached PKR 39.43 PKR 39.43 billion. (ii) SRB’s collection of STS for FY2013/14 Page 29 The World Bank PK Sindh Public Sector Management Reform Project (P145617) billion. reached PKR 39.43 billion. 30-Jun-2014 30-Jun-2015 31-Aug-2020 04-Jun-2020 Comments (achievements against targets): Target achieved. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Enhanced SRB human Text Staffing and training SRB has approved a All targets have been resources capacity for plan is under revised staffing plan achieved. The Board administration (DLI) development as per the of SRB has approved recommendation of the revised staffing the third party review. plan as per the recommendation of the third-party review. 30-Jun-2014 30-Jun-2019 04-Jun-2020 Comments (achievements against targets): Targets for this indicator was revised during the 2018 restructuring. The revised target was achieved. Indicator Name Unit of Measure Baseline Original Target Formally Revised Actual Achieved at Page 30 The World Bank PK Sindh Public Sector Management Reform Project (P145617) Target Completion Increased automation of SRB Text Zero automated SRB has issued all All targets have been systems (DLI) notices; Unique notices to at least 90% achieved taxpayer identification of non-compliant system does not exist; registered taxpayers No third party in FY2018/19 through SRB has issued notices linkages. automated systems. to at least 90% of non- compliant registered taxpayers through automated systems. 30-Jun-2014 30-Jun-2019 04-Jun-2020 Comments (achievements against targets): Target achieved. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Increased tax base Percentage 4,800.00 25.00 307.00 30-Jun-2014 30-Jun-2019 04-Jun-2020 Comments (achievements against targets): Target achieved and exceeded. The indicator was measured as a cumulative percentage increase from a baseline of 4,800 taxpayers in FY13/14. Page 31 The World Bank PK Sindh Public Sector Management Reform Project (P145617) The original target estimated was 25 percent increase with 9,563 taxpayers. As per the TPV (Third Party Verification) report, in FY18-19, the number of registered taxpayers was 31,144 which was a 307 percent increase. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Risk based audits of Text System Absent SRB has: (i) carried out SRB has: (i) issued and All targets have been taxpayers implemented (DLI) a strategic review to communicated audit achieved. SRB has: (i) enhanced riskbased observations to the issued and criteria objectivity; concerned taxpayers; communicated audit and (ii) approved a (ii) finalized the audits observations to the compliance risk initiated in FY2017/18; concerned taxpayers; management strategy, (iii) initiated (ii) finalized the audits in order to increase assessment (by SRB initiated in FY2017/18; the audits’ hit rate. Operations Wing) (iii) initiated under the relevant assessment (by SRB provisions of the law; Operations Wing) (iv) selected audit under the relevant cases according to provisions of the law; revised risk-based (iv) selected audit criteria updated cases according to according to the revised risk-based feedback obtained criteria updated from the completed according to the audit cases initiated in feedback obtained FY2017/18. from the completed audit cases initiated in FY2017/18 Page 32 The World Bank PK Sindh Public Sector Management Reform Project (P145617) 30-Jun-2014 30-Jun-2019 30-Jun-2019 04-Jun-2020 Comments (achievements against targets): Targets for this indicator was revised during the 2018 restructuring and achieved. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Improved SRB performance Text Annual report Four quarterly reports All targets have been transparency irregular, quarterly and one annual report achieved. reports not published published Four quarterly reports and one annual report are published 30-Jun-2014 30-Jun-2019 04-Jun-2020 Comments (achievements against targets): Target achieved. Indicator Name Unit of Measure Baseline Original Target Formally Revised Actual Achieved at Page 33 The World Bank PK Sindh Public Sector Management Reform Project (P145617) Target Completion Debt management systems Text Debt Management a) Complete loan by a) FD conducts a debt All targets have been instituted Systems do not exist. loan debt stock sustainability analysis. achieved. (opening and closing) b) Publication of the Debt sustainability and flow details above on the FD analysis conducted included in budget website approved by and published on FD’s documents for FY FS. website with the 2017/18. b) approval of Finance Publication of annual Secretary on 26 June debt statistics bulletin 2019. in-line with international best practices having a lag of not more than three months. 30-Jun-2014 30-Jun-2019 30-Jun-2019 04-Jun-2020 Comments (achievements against targets): Targets for this indicator was revised during the 2018 restructuring and the revised target was achieved. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Establishment of internal Text Internal audit function (i) At least seven (7) (i) Four (4) All targets for have audit (DLI until Restructuring, does not exist. Government Government been achieved. IRI thereafter) Departments (in Departments including (i) Four (4) addition to the FD) Finance Department Government Page 34 The World Bank PK Sindh Public Sector Management Reform Project (P145617) have: (A) conducted have conducted Departments including internal audits; and internal audits and Finance Department (B) submitted their submitted internal have conducted internal audits reports audit reports to their internal audits and to their PAOs. (ii) At PAOs. (ii) Remaining submitted internal least five (5) PAOs three (3) PAOs have audit reports to their have undertaken undertaken actions in PAOs. actions in response to response to the the foregoing internal forgoing internal audit audit reports, within a reports, within a (ii) Remaining three month of receipt of month of such report. (3) PAOs have such reports. undertaken actions in response to the forgoing internal audit reports, within a month of such report 30-Jun-2014 30-Jun-2019 30-Jun-2019 26-Nov-2020 Comments (achievements against targets): Target achieved. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Page 35 The World Bank PK Sindh Public Sector Management Reform Project (P145617) Transparency in budget Text a) Budget Strategy FD has: (i) submitted All targets have been formulation, allocation and Paper not prepared to the Cabinet the achieved. execution (DLI) Budget Strategy Paper FD has: (i) submitted for FY2019/20; and (ii) to the Cabinet the b) One quarterly submitted all four (4) report submitted to Budget Strategy Paper budget execution for FY19/20; and (ii) the provincial reports for FY 2018/19 assembly submitted all four (4) to the Provincial budget execution Assembly and reports for FY18/19 to published them on the Provincial FD’s website. Assembly and published them on FD's website 30-Jun-2014 30-Jun-2019 04-Jun-2020 Comments (achievements against targets): According to TPV reports, the target was achieved. Note: The indicator was upgraded to a PDO indicator during the Restructuring. There are no changes in substance. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Procurement officials Text No certification SPPRA has certified at SPPRA has: (i) certified Targets have been certified (DLI) mechanism exists. least five hundred at least five hundred achieved and (500) officers of grade (500) officers of grade surpassed. (i) 696 BPS-16 or above BPS-16 or above; and officers certified as of Page 36 The World Bank PK Sindh Public Sector Management Reform Project (P145617) (ii) carried out a third November 25th, 2020. party review of the (ii) TPV has been certification process. carried out and previously confirmed the achievement of the target. 30-Jun-2014 30-Jun-2019 30-Jun-2019 25-Nov-2020 Comments (achievements against targets): Target achieved. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Quarterly developmental Text Zero P&DD has: (i) All targets have been development plan prepared and achieved. monitoring reports prepared published at least (i) Quarterly progress and published (DLI) twelve (12) quarterly reports for six (6) reports on departments (i.e. 17 implementation of reports in total) development schemes prepared and for at least four (4) published. (ii) TPV Government report was conducted Departments and confirmed the collectively; and (ii) achievement the carried out a third targets. party review. Page 37 The World Bank PK Sindh Public Sector Management Reform Project (P145617) 30-Jun-2014 30-Jun-2019 04-Jun-2020 Comments (achievements against targets): Fully achieved. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Departments where geo- Number 0.00 6.00 6.00 tagging of development schemes is implemented and 30-Jun-2014 30-Jun-2019 04-Jun-2020 information published. Comments (achievements against targets): Target achieved. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Departments with proactive Text Zero 7.00 Collection of All targets have been feedback mechanisms community feedback achieved. established for at least 10% of MEC has collected provincial ADP community feedback schemes costing over for at least 10% of Page 38 The World Bank PK Sindh Public Sector Management Reform Project (P145617) PKR 1 billion in at least provincial ADP four (4) Government (costing over PKR 1 Departments. billion) from five (5) departments 30-Jun-2014 30-Jun-2019 30-Jun-2019 04-Jun-2020 Comments (achievements against targets): Target achieved. Page 39 The World Bank PK Sindh Public Sector Management Reform Project (P145617) B. KEY OUTPUTS BY COMPONENT Objective/Outcome 1 Outcome Indicators Improved collection of Sales Tax on Service (STS) (Percentage) 1. Approval and initial implementation of Sindh Tax Revenue Mobilization Reform Plan 2. Enhanced the SRB’s human resource capacity for administration Intermediate Results Indicators 3. Increased automation of the SRB’s systems 4. Increased tax base 5. Risk-based audits of taxpayers implemented 6. Improved SRB performance transparency (a) Developed the Sindh Tax Revenue Mobilization Plan (2014-2019) for tax collecting agencies; (b) improved sales taxes for service collection (see also the paragraph below concerning revenue collection); (c) developed the Budget Strategy Paper and submitted Quarterly Execution Reports to the Provincial Assembly of Sindh (see footnote 16); (d) enhanced the SRB’s human resource capacity for administration; (e) increased the automation of the SRB systems; (f) increased the tax base in the SRB; (g) implemented the risk-based Key Outputs by Component taxpayer audits; (h) improved SRB transparency on performance by (linked to the achievement of the Objective/Outcome 1) issuing Quarterly and Annual Performance Reports, which are then published on the SRB website; (i) instituted a debt management system in the Province; (j) established an Internal Audit (IA) function in four departments, namely the departments of finance, health, home, and school education; (k) nearly 700 employees of the GoS from the BS-16 level and above procurement certified; and (l) developed the ADP Dashboard with geo-tagging of development schemes and community feedback. Page 40 The World Bank PK Sindh Public Sector Management Reform Project (P145617) Objective/Outcome 2 Transparency in budget formulation, allocation and execution (DLI) Outcome Indicators (Text) 1. Debt management systems instituted 2. Establishment of an internal audit function 3. Transparency in budget formulation, allocation, and execution 4. Procurement officials certified Intermediate Results Indicators 5. Quarterly development plan monitoring reports prepared and published 6. Departments where geo-tagging of development schemes is implemented, and information published 7. Departments with proactive feedback mechanism established (a) Provided IT support to the SPPRA, the P&DD and the Excise Taxation and Narcotics Control Department (ET&NCD); (b) conducted a survey to support the Excise Department in handling property and professional taxes in the Sukkur district (FY2017 to 2019); (c) carried out an Actuarial Valuation of Employee Benefit Schemes to ascertain the unfunded liabilities; (d) developed Standard Operating Procedures (SoPs) for the FD; (e) conducted a review of development statistics of Key Outputs by Component the Sindh Bureau of Statistics, and shared recommendations with the (linked to the achievement of the Objective/Outcome 2) Pⅅ (f) established a state-of-the-art Data Center Facility in the FD, which went live on January 29, 2021. It could help line departments in housing their data; and (g) made several rreforms in the Sindh Government Servants Benevolent Fund. These project achievements were made by utilizing a good mix of performance-based incentives and technical assistance. As such, they may provide a useful roadmap for future projects. Page 41 The World Bank PK Sindh Public Sector Management Reform Project (P145617) ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION A. TASK TEAM MEMBERS Name Role Preparation Zubair Khurshid Bhatti, Paul Welton Task Team Leader(s) Uzma Sadaf Procurement Specialist(s) Syed Waseem Abbas Kazmi Financial Management Specialist Afzal Mahmood Team Member Martin M. Serrano Counsel Miki Terasawa Social Specialist Javaid Afzal Environmental Specialist Sher Shah Khan Team Member Immanuel Frank Steinhilper Team Member Supervision/ICR Charles Victor Blanco, Jose Eduardo Gutierrez Ossio Task Team Leader(s) Uzma Sadaf Procurement Specialist(s) Noaman Ali Financial Management Specialist Anwar Ali Bhatti Team Member Nasreen Shah Kazmi Team Member Mishka Zaman Social Specialist Javaid Afzal Environmental Specialist Rahat Jabeen Environmental Specialist Yesica Gabriela Morales Zazueta Team Member Page 42 The World Bank PK Sindh Public Sector Management Reform Project (P145617) B. STAFF TIME AND COST Staff Time and Cost Stage of Project Cycle No. of staff weeks US$ (including travel and consultant costs) Preparation FY14 41.974 431,899.48 FY15 57.375 246,081.95 FY16 0 0.00 Total 99.35 677,981.43 Supervision/ICR FY14 0 821.91 FY15 0 3,623.88 FY16 19.987 75,053.04 FY17 10.748 53,556.92 FY18 22.060 116,887.85 FY19 22.424 114,383.22 FY20 19.985 94,248.27 Total 95.20 458,575.09 Page 43 The World Bank PK Sindh Public Sector Management Reform Project (P145617) ANNEX 3. PROJECT COST BY COMPONENT Amount at Approval Actual at Project Percentage of Approval Components (US$M) Closing (US$M) (%) Component 1: Results-based financing to provide an 40.00 40.00 100 incentive for achieving eligible PSM reforms. Component 2: Technical Assistance (TA) to support capacity building and 10.00 10.00 100 institutional strengthening associated with achievement of eligible PSM reforms. Total 50.00 47.3824 100 24 The difference between the original approved amount of US$50 million and the final amount of US$47.38 is due to a change in the exchange rate (SDR) during the project life. Page 44 The World Bank PK Sindh Public Sector Management Reform Project (P145617) ANNEX 4. EFFICIENCY ANALYSIS The efficiency gains from the project interventions were considerable relative to the costs incurred during the implementation of the SPSMRP. The overall benefits of the project are difficult to estimate, except for the intervention in tax collection. Therefore, the present value of benefits is estimated only on the basis of increased tax collection from the GST on services. The same methodology and assumptions employed at the time of appraisal have been used for the current analysis with actual data. The proposed intervention will increase the revenues from services, as well as impact the overall performance of the tax department. Key Assumptions: 1. The Project Appraisal Document (PAD) had considered three possible project scenarios for the increase in tax revenues. Under the conservative scenario, it was assumed that tax revenues would increase at a rate that is 0.25 percent higher than the growth rate of gross domestic product (GDP), whereas under the moderate and (high) scenarios, tax revenues were estimated to increase at a rate that is 0.5 percent (1 percent) higher than the growth rate of GDP. The historical average growth rate of real GDP in Pakistan at the time was 4.5 percent. The calculations in the ICR have used the actual revenue data for five years. 2. The future values of the difference in revenues (with and without the project) are discounted, assuming a 5 percent discount rate. The actual growth rate in revenues for the five-year period substantially exceeded the estimates under the high scenario in the PAD. The present value (PV) using actual revenue data for five years is US$1,036 million compared to US$56.3 million estimated in the PAD under the high scenario (Table 11). The benefits of increased tax collection will however go beyond these years because the increase in the tax base and improved capacity of the tax department will contribute to higher tax collection beyond the project period. Table 8: Expected Benefits in Terms of Increased Revenues (US$ million) and Growth in the GSTS (with and without project intervention) Estimated Estimated Revenues Actual PV (Present PV of Revenues with Years without Project Revenues at Difference Value) of Difference at Project (PAD - Intervention ICR Difference PAD High Scenario) (PAD) 2013-14 394 394 394 (Baseline) 412 495 83 79 416 3.9 2014-15 430 616 186 169 438 7.5 2015-16 450 724 274 237 463 11.2 2016-17 470 876 406 334 488 15 2017-18 Page 45 The World Bank PK Sindh Public Sector Management Reform Project (P145617) 491 769 278 217 515 18.8 2018-19 1,036 56.4 Figure 2: Growth in GSTS (with and without project intervention) 1000 900 GSTS - Actual at Completion 800 Amount in US$ 700 600 GSTS - Growth estimated in PAD 500 GSTS - Without Project Intervension 400 2013-14 2014-15 2015-16 2016-17 2017-18 2018-19 (Baseline) Page 46 The World Bank PK Sindh Public Sector Management Reform Project (P145617) ANNEX 5. BORROWER, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS Sindh Revenue Board Response Improved revenue generation was one of the Project Development Objectives (PDOs) of the Sindh Public Sector Management Reform (SPSMR) Project. The Sindh Revenue Board is one of the premier tax-collecting agencies of the province. As such, it was selected as one of the major implementing agencies for the project. The SPSMR Project proved to be a catalyst for change for the Sindh Revenue Board (SRB). The project has brought significant reforms in the human resource, automation, and taxpayer audit functions of the SRB. The reform agenda started with this project. However, it does not stop at the closure of the project. Thus, these reforms have put the SRB on a trajectory for change. With the help of the DLI, namely, the Enhanced Human Resource Capacity for Administration, the SRB was able to hire and train 150 Assistant Commissioners, Sales Tax Officers and Auditors. The quality tax training was also provided to existing SRB officers and officials through the International Bureau of Fiscal Documentation (IBFD) in Amsterdam, the Netherlands. The World Bank has also arranged for capacity-building workshops for the auditors hired for the newly established taxpayers audit wing at the SRB. Theses workshops were conducted by technical experts arranged by the World Bank. Through increased automation of the SRB systems (DLI), the IT wing of the SRB prepared and implemented the automation plan. It also established third-party linkages and issued e-notices to non- compliant taxpayers. The establishment of the new data center and the implementation of the ERP and the BI solution through TA financing also proved to be very beneficial. The establishment of the taxpayer’s audit wing at the SRB was one of the initiatives at the SRB, and it was propelled by the SPSMR Project. The DLI regarding the risk-based audit of taxpayers inspired the SRB to form this wing at the SRB. Initially, the SRB faced difficulties in achieving the target values for this DLI. However, the restructuring of the project in June 2018 helped the SRB to achieve the revised target values of this DLI. As noted, the SPSMR Project helped the SRB to bring reforms to key functions. Currently, the SRB is ensuring the sustainability of the reform agenda initiated through the SPSMR Project. The SRB would also like to work with the World Bank on more such initiatives/projects in the future. Page 47 The World Bank PK Sindh Public Sector Management Reform Project (P145617) ANNEX 6. SUPPORTING DOCUMENTS Implementation Status Results and Reports (ISR) Reviewed: Pakistan (PK) Sindh Public Sector Management Reform Project (P145617) • December 23, 2020 • June 15, 2020 • December 23, 2019 • June 25, 2019 • December 21, 2018 • June 29, 2018 • February 1, 2018 • June 12, 2017 • December 28, 2016 • May 5, 2016 • November 13, 2015 • May 7, 2015 Project Appraisal Documents for the Sindh Public Sector Management Reform Project (P145617) Restructuring Paper on a Proposed Project Restructuring of the PK Sindh Public Sector Management Reform Project approved on January 28, 2015 to the Islamic Republic of Pakistan. Aide-Memoire: Pakistan Sindh Public Sector Management Reform Project (Loan IDA-55840). Aide-Memoire: Mid-Term Review Mission. November 13-17, 2017. Technical Mission – June 11, 2018 International Development Association: Project Appraisal Document on a Proposed Credit in the Amount of SDR 33.9 million (US$ 50 million equivalent) to the Islamic Republic of Pakistan for the Sindh Public Sector Management Reform Project (P145617). December 30, 2014. Implementation Support Mission for Sindh Public Sector Management Reform Project (IDA Credit No. 5584-PK). November 24, 2020. Economic Reform Unit, Finance Department, Government of Sindh. Letters (Subjects): • Re: Notice of Impending Closing Date and Disbursement Deadline Date Sindh Public Sector Management Reform Project (Credit No. 5584-PK). February 1, 2021. • Re: Sindh Public Sector Management Reform Project (Credit No. 5584-PK) Implementation Supervision Mission. June 12, 2019. • Re: Islamic Republic of Pakistan: Credit Number 5584-PK (Sindh Public Sector Management Reform Project) Amendment to the Financing Agreement. June 21, 2018. • Restructuring of Sindh Public Sector Management Reform Project (SPSMRP), IDA Credit No. 5584-PK. March 28, 2018. • Re: Mid-Term Review mission for the Sindh Public Sector Management Reform Project (P145617, Loan IDA-5584 PK), November 13-17, 2017. December 20, 2017. • Change in Task Leadership for Sindh Public Sector Reform Project (P145617). August 28, 2017. Pakistan: Credit No. 5584-PK (Sindh Public Sector Management Reform Project) Closing. March 31,2015. PK-Sindh Public Financial Management and Accountability Assessment Report, November 2013 PK-Sindh Public Expenditure and Financial Accountability (PEFA) Report, April 13, 2020. Third Party Validation (TPV) for FY 2018-19, January 2020. Page 48 The World Bank PK Sindh Public Sector Management Reform Project (P145617) ANNEX 7: PROVINCIAL GENERAL SALES TAX ON SERVICES (GSTS) (IN PKRS, BILLIONS) Table 9: General Sales Taxes on Services 2019-20 2013-14 2014-15 2015-16 2016-17 2017-18 2018-19 (RE) General Sales Tax on Services (GSTS) 39 49 61 76 92 93 135 GSTS share of total tax revenues (%)25 49% 51% 49% 53% 53% 52% 57% GSTS growth (year-on- year [YOY]) – Nominal (%) 26% 24% 25% 21% 1% 45% GSTS growth (YOY) – Real 20.80% 21.40% 21% 17% -4% 34% (%) RE – Revised Estimate Table 10: Provincial General Sales Tax on Services PKRs, Billions (B) Province 2013-14 2014-15 2015-16 2016-17 2017-18 2018-19 2019-20 (RE) Sindh 39 49 61 79 99 93 135 Punjab 52 43 59 78 106 90 102 KP 6.0 6.2 7.2 10.3 10.9 10.4 15.4 Balochistan 0.9 2.0 4.5 7.2 8.8 10.7 Table 11: Provincial General Sales Tax on Services, Percentage Change (YOY) (C) Province 2019-20 2013-14 2014-15 2015-16 2016-17 2017-18 2018-19 (RE) Sindh 26 24 30 25 -6 45 Punjab -18 38 33 36 -15 13 KP 3 16 43 6 -5 48 Balochistan 130 125 59 23 22 25 Sales taxes collected as a percentage of total tax collected is detailed in all of the tables. Based on the project target, this is the most relevant indicator for assessing the progress. The sales taxes collected as a percentage of GDP was not considered due to several challenges. Where previous Bank projects had used a tax-to-GDP ratio as an indicator, there were many challenges in assessing the efficacy of tax collection. In this context, GDP is subject to many factors both external (overall growth implications due to international economic climate) and internal (such as the decision to debase the GDP). Hence, this ratio could change irrespective of a tax administration’s successful efforts to increase tax collection. For example, when preparing the ICR of a Public Sector Governance Program-for-Results in Rwanda (P149095), this ratio made it impossible for the ICR to provide a Satisfactory rating, even though the tax authorities had met all of the required (nominal) targets. Page 49 The World Bank PK Sindh Public Sector Management Reform Project (P145617) Figure 3: Sindh GSTS: Increasing Share of Provincial Taxes, but Uneven Growth Sindh GSTS: increasing share in provincial taxes, but uneven growth 70% 60% 50% 40% 30% 20% 10% 0% -10% 2013-14 2014-15 2015-16 2016-17 2017-18 2018-19 2019-20 RE GSTS share in total tax revenue GSTS growth (YOY) Source: World Bank staff calculations Note: GSTS= General Sales Tax on Services; YOY= Year-on-Year. Page 50 The World Bank PK Sindh Public Sector Management Reform Project (P145617) ANNEX 8: PFM PERFORMANCE INDICATORS Table 1: Overview of the Scores of PEFA Indicators PFM Performance Indicator Method 1 2 3 4 Score A. Higher Level Government (HLG) HLG A. Transfers from HLG M1 C D C D+ I. Budget Reliability PI_1 1. Aggregate expenditure outturn C C PI_2 2. Expenditure composition outturn M1 C C A C+ PI_3 3. Revenue outturn M2 C B C+ II. Transparency of Public Finances PI_4 4. Budget classification A A PI_5 5. Budget documentation C C PI_6 6. Government operations outside financial reports M2 D* D* D D PI_7 7. Transfers to subnational governments M2 A C B PI_8 8. Performance information for service delivery M2 D D D D D PI_9 9. Public access to fiscal information B B III. Management of Assets and Liabilities PI_10 10. Fiscal risk reporting M2 D D D D PI_11 11. Public investment management M2 C A C B B PI_12 12. Public asset management M2 C C D D+ PI_13 13. Debt management M2 C C D D+ IV. Policy-based Fiscal Strategy and Budgeting PI_14 14. Macroeconomic and fiscal forecasting M2 NA C NA C PI_15 15. Fiscal strategy M2 D C C D+ PI_16 16. Medium-term perspective in expenditure budgeting M2 D D C NA D+ PI_17 17. Budget preparation process M2 A D D C PI_18 18. Legislative scrutiny of budgets M1 C C A B C+ V. Predictability and Control in Budget Execution PI_19 19. Revenue administration M2 B B C D* C+ PI_20 20. Accounting for revenue M1 A A C C+ PI_21 21. Predictability of in year resource allocation M2 C C B C C+ PI_22 22. Expenditure arrears M1 D* D D PI_23 23. Payroll controls M1 A A A B B+ PI_24 24. Procurement management M2 A B B B B+ PI_25 25. Internal controls on non-salary expenditure M2 A C B B PI_26 26. Internal audit M1 B B D* D* D+ VI. Accounting and Reporting PI_27 27. Financial data integrity M2 D D C B D+ PI_28 28. In-year budget reports M1 C C C C PI_29 29. Annual financial reports M1 C B B C+ Page 51 The World Bank PK Sindh Public Sector Management Reform Project (P145617) VII. External Scrutiny and Audit PI_30 30. External audit M1 C D C C D+ PI_31 31. Legislative scrutiny of audit reports M2 D NA NA NA D Source: PEFA Report, April 13, 2020 Note: NA= not available; PI= Performance Indicator. Page 52