Document of The World Bank FOR OFFICIAL USE ONLY Report No: ICR00004433 IMPLEMENTATION COMPLETION AND RESULTS REPORT (Ln. 7914-ID and Ln. 8438-ID) ON A LOAN IN THE AMOUNT OF USD 220 MILLION AND AN ADDITIONAL LOAN IN THE AMOUNT OF USD 500 MILLION TO THE REPUBLIC OF INDONESIA FOR THE LOCAL GOVERNMENT AND DECENTRALIZATION PROJECT ( P111577 ) June 8, 2018 Social, Urban, Rural And Resilience Global Practice East Asia And Pacific Region CURRENCY EQUIVALENTS (Exchange Rate Effective {Jan 09, 2018}) Currency Unit = IDR IDR 13,500 = US$1 FISCAL YEAR January 1 – December 31 Regional Vice President: Victoria Kwakwa Country Director: Rodrigo A. Chaves Senior Global Practice Director: Ede Jorge Ijjasz -Vasquez Practice Manager: Abhas Kumar Jha Task Team Leader(s): Thalyta Ernandya Yuwono, Marcus John Jin Sarn Lee ICR Main Contributor: Sarah Xinyuan Lin ABBREVIATIONS AND ACRONYMS AF Additional Financing AP Advance Payment APBD Anggaran Pendapatan dan Belanja Daerah (Local Budget) Bappenas Badan Perencanaan dan Pembangunan Nasional (National Planning Agency) BPK Badan Pemeriksa Keuangan (State Audit of Indonesia) BPKP Badan Pengawasan Keuangan dan Pembangunan (Government of Indonesia’s Internal Audit Agency) CPF Country Partnership Framework CPS Country Partnership Strategy DAK Dana Alokasi Khusus (Specific Purpose Grants) DGFB Directorate General of Fiscal Balance Dinas Work Units at the Local Government level EIRR Economic Internal Rate of Return GoI Government of Indonesia GPOBA Global Partnership for Output Based Aid IOI Intermediate Outcome Indicator IPF Investment Project Financing ISP Institutional Support Program ISR Implementation Status Report KPI Key Performance Indicator KPPN Kantor Pelayanan Perbendaharaan Negara (MoF representative office at LG level) LA Loan Agreement LG Local Government LKPP Lembaga Kebijakan Pengadaan Barang/Jasa Pemerintah (National Procurement Policy Agency) M&E Monitoring and Evaluation MIS Management Information System MoF Ministry of Finance MoHA Ministry of Home Affairs MPW Ministry of Public Works MPWH Ministry of Public Works and Housing MTR Mid-Term Review NPV Net Present Value OBD Output-Based Disbursement PAD Project Appraisal Document PDAM Perusahaan Daerah Air Minum (Local drinking water company) PforR Program for Results PIU Project Implementation Unit QP Qualifying Percentage RPJMN Rencana Pembangunan Jangka Menengah Nasional (National Mid-Term Development Plan) RUC Reference Unit Cost SUSENAS Survei Sosial Ekonomi Nasional (National Socioeconomic Survey) TA Technical Assistance TTL Task Team Leader (from the Bank) VPR Value of Potential Reimbursement VQR Value of Qualifying Reimbursement WBRS Web-Based Reporting System WBTMS Web-Based Transfer Monitoring System TABLE OF CONTENTS DATA SHEET ..................................................................................................................... 1 I.PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES ...................................................... 6 A. CONTEXT AT APPRAISAL .........................................................................................................6 B. SIGNIFICANT CHANGES DURING IMPLEMENTATION (IF APPLICABLE) .......................................9 II.OUTCOME ................................................................................................................... 11 A. RELEVANCE OF PDOs ............................................................................................................ 11 B. ACHIEVEMENT OF PDOs (EFFICACY) ...................................................................................... 11 C. EFFICIENCY ........................................................................................................................... 15 D. JUSTIFICATION OF OVERALL OUTCOME RATING .................................................................... 17 E. OTHER OUTCOMES AND IMPACTS (IF ANY) ............................................................................ 17 III.KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME ............................. 19 A. KEY FACTORS DURING PREPARATION ................................................................................... 19 B. KEY FACTORS DURING IMPLEMENTATION ............................................................................. 19 IV.BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME 20 A. QUALITY OF MONITORING AND EVALUATION (M&E) ............................................................ 20 B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE ..................................................... 22 C. BANK PERFORMANCE ........................................................................................................... 24 D. RISK TO DEVELOPMENT OUTCOME ....................................................................................... 26 V.LESSONS AND RECOMMENDATIONS ........................................................................... 27 ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS ..................................................... 29 ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION ................... 37 ANNEX 3. PROJECT COST BY COMPONENT ...................................................................... 40 ANNEX 4. EFFICIENCY ANALYSIS ..................................................................................... 41 ANNEX 5. BORROWER, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS48 ANNEX 6. SUPPORTING DOCUMENTS (IF ANY) ................................................................. 59 The World Bank Local Government and Decentralization Project ( P111577 ) DATA SHEET BASIC INFORMATION Product Information Project ID Project Name P111577 Local Government and Decentralization Project Country Financing Instrument Indonesia Investment Project Financing Original EA Category Revised EA Category Partial Assessment (B) Partial Assessment (B) Related Projects Relationship Project Approval Product Line Additional Financing P123940-AF to the 30-Sep-2014 IBRD/IDA Local Government and Decentralization Project /LGDP Phase II Organizations Borrower Implementing Agency Ministry of Public Works, Secretariat General, BPKP, Ministry of Finance, Ministry of Home Affairs Ministry of Finance, Bappenas Project Development Objective (PDO) Original PDO The objective of the project is to improve the accountability and reporting of the central government’s Specific Purpose Grants(DAK) for the infrastructure sub-sectors within pilot local governments (LGs). This will be done through improved financial andtechnicalreporting, and verification of outputs delivered by LGs with the DAK grants. Page 1 of 62 The World Bank Local Government and Decentralization Project ( P111577 ) Revised PDO The objective of the Project is to improve the accountability and reporting of the central government’s Specific PurposeGrants(DAK) for basic infrastructure, consisting of roads, irrigation, water and sanitation, within Selected Local Governments. PDO as stated in the legal agreement The objective of the project is to improve the accountability and reporting of the central government’s Specific Purpose Grants(DAK) for the infrastructure sub-sectors within Pilot Local Governments. FINANCING Original Amount (US$) Revised Amount (US$) Actual Disbursed (US$) World Bank Financing 220,000,000 219,770,855 219,770,855 IBRD-79140 500,000,000 500,000,000 500,000,000 IBRD-84380 Total 720,000,000 719,770,855 719,770,855 Non-World Bank Financing Borrower 0 8,500,000 8,500,000 Total 0 8,500,000 8,500,000 Total Project Cost 720,000,000 728,270,855 728,270,855 KEY DATES Approval Effectiveness MTR Review Original Closing Actual Closing 08-Jun-2010 20-Jan-2011 03-Jun-2013 30-Jun-2014 31-Dec-2017 Page 2 of 62 The World Bank Local Government and Decentralization Project ( P111577 ) RESTRUCTURING AND/OR ADDITIONAL FINANCING Date(s) Amount Disbursed (US$M) Key Revisions 14-Apr-2014 122.78 Change in Loan Closing Date(s) 30-Sep-2014 171.41 Additional Financing Change in Project Development Objectives Change in Results Framework Change in Components and Cost Change in Loan Closing Date(s) Change in Financing Plan Change in Safeguard Policies Triggered Change in Institutional Arrangements Change in Procurement Change in Implementation Schedule 08-Feb-2016 265.92 Change in Results Framework Change in Procurement Other Change(s) 28-Dec-2017 719.77 Change in Loan Closing Date(s) KEY RATINGS Outcome Bank Performance M&E Quality Satisfactory Moderately Satisfactory Substantial RATINGS OF PROJECT PERFORMANCE IN ISRs Actual No. Date ISR Archived DO Rating IP Rating Disbursements (US$M) 01 29-May-2011 Satisfactory Satisfactory 0 02 11-Mar-2012 Satisfactory Satisfactory 15.11 03 05-Mar-2013 Moderately Satisfactory Moderately Satisfactory 60.66 04 23-Oct-2013 Satisfactory Satisfactory 122.78 05 28-Jun-2014 Satisfactory Satisfactory 122.78 06 07-Jan-2015 Satisfactory Satisfactory 171.41 Page 3 of 62 The World Bank Local Government and Decentralization Project ( P111577 ) 07 22-Sep-2015 Satisfactory Moderately Satisfactory 265.92 08 02-Mar-2016 Satisfactory Moderately Satisfactory 265.92 09 16-Oct-2016 Moderately Satisfactory Moderately Satisfactory 472.31 10 27-Jun-2017 Moderately Satisfactory Moderately Satisfactory 673.40 11 08-Dec-2017 Moderately Satisfactory Moderately Satisfactory 719.77 SECTORS AND THEMES Sectors Major Sector/Sector (%) Public Administration 100 Sub-National Government 100 Themes Major Theme/ Theme (Level 2)/ Theme (Level 3) (%) Public Sector Management 33 Public Administration 33 Municipal Institution Building 33 Urban and Rural Development 67 Urban Development 67 Urban Infrastructure and Service Delivery 67 ADM STAFF Role At Approval At ICR Regional Vice President: James W. Adams Victoria Kwakwa Country Director: Joachim von Amsberg Rodrigo A. Chaves Senior Global Practice Director: John A. Roome Ede Jorge Ijjasz-Vasquez Practice Manager: Sonia Hammam Abhas Kumar Jha Page 4 of 62 The World Bank Local Government and Decentralization Project ( P111577 ) Thalyta Ernandya Yuwono, Task Team Leader(s): Peter D. Ellis Marcus John Jin Sarn Lee ICR Contributing Author: Sarah Xinyuan Lin Page 5 of 62 The World Bank Local Government and Decentralization Project ( P111577 ) I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES A. CONTEXT AT APPRAISAL Context 1. Initiated in 2001, Indonesia’s “big bang” decentralization marked a fundamental shift in the country’s governance structure, turning it from one of the world’s most centralized countries in administrative, fiscal and political terms to one of the most decentralized. Local governments (LGs) assumed several responsibilities that were previously held by the central government, and, as a result, received substantial increases in financial resources, largely in the form of central government transfers. Among LGs’ new responsibilities was the delivery of most services, including education, health and infrastructure. At appraisal, LGs were managing 38 percent of total public expenditures, and were carrying out more than half of public investment. However, the expectation that decentralization would enable subnational governments to respond to local needs more effectively had yet to be met. 2. Part of the intergovernmental transfers came from Specific Purpose Grants, known as Dana Alokasi Khusus or DAK, a grant from the national budget allocated annually to LGs based on criteria set by the Ministry of Finance (MoF). LGs had the flexibility to determine subproject investments in individual sectors, but were limited to the maintenance, rehabilitation and improvement of existing infrastructure, specifically in the road, irrigation, water supply and sanitation sectors. DAK is of central importance for public infrastructure development in Indonesia. In 2012, DAK accounted for 20% of total capital spending by LGs1. 3. Though DAK policies were well-established, studies concluded that improvements were needed in financial, technical, institutional and governance areas, and that monitoring and evaluation (M&E) of DAK-financed activities was inadequate. No reporting or independent verification process was mandated to ascertain that the LGs had completed the agreed works in compliance with technical guidelines issued by the Ministry of Public Works and Housing (MPWH). LGs simply submittted financial reports to MOF stating that DAK funds had been spent in order to remain eligible for receiving the following year’s funds. On average only 50% of LGs submitted DAK infrastructure reports in 2010, and these were typically incomplete and submitted late. Weak institutional coordination, both among central government ministries and the central government and subnational governments, also hindered effectiveness. 4. In this context, MoF requested World Bank support in improving the use of DAK grants at the LG level, as well as in strengthening national systems for reporting and verifying the use of DAK. This was clearly aligned with the Bank’s Country Partnership Strategy (CPS) for Indonesia FY 2009 – 2012, which outlined a strategy to develop both national and LG institutions and systems. In particular, the CPS sought “to support local governments by promoting well-targeted and sustainable public spending through the co-financing of sub-national expenditure programs.” The Bank’s engagements at the time of appraisal also maintained a focus on strengthening existing GoI and LG institutions, rather than developing individual stand-alone operations. This project, the Local Government and Decentralization Project (LGDP), was thus 1 By 2016, this figure had increased to 70%. Page 6 of 62 The World Bank Local Government and Decentralization Project ( P111577 ) developed to enhance the existing intergovernmental transfer mechanism, by strengthening LG and central government systems for reporting and monitoring the use of DAK grants. Theory of Change (Results Chain) 5. Prior to LGDP, no technical reporting or independent verification process was mandated to ascertain how LGs had used DAK funds, or that works had been completed in compliance with technical guidelines. To improve transparency in the use of DAK funds, it was necessary to improve reporting, and thereby accountability. To improve reporting, the project included Technical Assistance (TA) to establish and improve a web-based reporting system, to standardize DAK infrastucture reporting among participating LGs. MoF was to provide training and socialization to LGs on LGDP and its requirements, including proper use of the new reporting system. While the RF did not track the amount or types of training conducted as an output, KPI #1 measured the use of this new standardized system, i.e. “Development and use of an information system to which LGs report information to the MoF and the MPW (later MPWH).” In order to further encourage use of the system to enhance reporting, incentive payments under the project’s Additional Financing (AF) were linked to the use of the Web-Based Reporting System (WBRS). 6. To improve accountability, the improved reports submitted by LGs would be subject to verification by an independent verification agent. This would address the challenge of having un-verified information reported and a lack of transparency over use of DAK funds. For this purpose, the Government of Indonesia’s (GoI) independent internal auditor, BPKP, was tasked to carry out output verification on DAK infrastructure projects in all participating LGs, as designed in Component 3. BPKP was supported with training on verification of technical infrastructure, and on MPWH’s technical guidelines on roads, water, sanitation, and irrigation. Handbooks on output verification were created, providing easily accessible infrastructure specifications and photographs for comparison. Additional staffing and budget for carrying out field verification were also arranged. The Inspectorate General (IG) of MPWH conducted quality assurance following BPKP verification. Hands-on monitoring and evaluation (M&E) training was also provided to MoF, MPWH, Ministry of Home Affairs (MoHA) and Bappenas, the national planning agency. In all, the TA helped to create a system of accountability in the use of DAK funds, and was measured by KPI #2, “% of physical outputs reported, verified and meeting eligibility criteria”. Once the necessary output verification had been completed, MOF would submit a withdrawal request to the World Bank for reimbursement. Project Development Objectives (PDOs) 7. As stated in the PAD and Loan Agreement (LA), the PDO was “to improve the accountability and reporting of the central government’s Specific Purpose Grants (DAK) for the infrastructure sub-sectors within Pilot Local Governments.” Key Expected Outcomes and Outcome Indicators 8. The project aimed to improve both accountability and reporting relating to DAK in pilot LGs. Improved reporting would be tracked with KPI #1, i.e. “a fully operational Web-Based Reporting System (WBRS), used by both MoF and the Page 7 of 62 The World Bank Local Government and Decentralization Project ( P111577 ) Ministry of Public Works (MPW2)”; and improved accountability would be tracked through KPI #2 “physical outputs reported, verified and meeting eligibility criteria”. Components 9. Component 1: DAK Reimbursement (US$220 million, 100% Bank Financed). Using an Output-Based Disbursement (OBD) approach, Component 1 would reimburse MoF for DAK infrastructure transfers in the roads, water supply, and irrigation subsectors, based on physical outputs reported by participating LGs in the five pilot provinces3 and verified by BPKP. The five pilot LGs were identified based on several factors, namely: (i) existing compliance with DAK reporting; (ii) geographical spread; (iii) a mix of high- and low- capacity LGs. Ten percent of the total reimbursement made to MoF would be transferred to LGs as an incentive reimbursement for their counterpart funding of DAK outputs, 10 percent being a GoI requirement for matching funds for DAK provided by LGs. Reimbursements would be made based on GoI’s existing disbursement structure of DAK funds, and would only be provided if output eligibility criteria were met. BPKP’s annual verification report, which consists of information on the number of eligible contracts, formed the basis for the Bank’s reimbursement to GoI. A detailed description of the output verification mechanism is presented in paragraph 27 and an illustrative example is provided in Box 1. 10. Component 2: Institutional Strengthening to Central and LGs, and Project Management Support (US$8.5 million, 100% Borrower financed through MOF) was divided into four sub-components as follows: • Component 2.1: Developing a M&E System. This sub-component would (i) strengthen MoF’s monitoring of financial reports, and build LG’s capacity in developing an integrated system of financial and physical reports that would be linked to MoF’s existing Web-Based Transfer Monitoring System (WBTMS); (ii) develop an overall M&E strategy and support for capacity-building in the Ministry of Public Works (MPW), and strengthen MPW’s management information system (MIS); and (iii) develop instruments to monitor program processes, to ensure quality of service delivery and program implementation, and to detect problems for timely correction. • Component 2.2: Web-Based Monitoring and Reporting System. This sub-component would expand and enhance MoF’s existing WBTMS to become a Web-Based Reporting System (WBRS). The enhanced system would cover financial and technical aspects of DAK outputs, including project location (with GPS coordinates), technical data, physical progress and site photographs. • Component 2.3: Technical Assistance for Central and LGs. Technical Assistance (TA) included (i) strengthening of the operational institutional framework at the central level, including revision of regulations as needed; and (ii) strengthening LGs’ ability to meet requirements on DAK planning, utilization and reporting. • Component 2.4: Project Management Costs. Operating costs of the Project Implementation Unit (PIU) at MoF was financed by this sub-component. 2 MPW was renamed MPWH in 2014. 3 The five pilot participating provinces include: Jambi, East Java, Central Kalimantan, West Sulawesi, and North Maluku. Page 8 of 62 The World Bank Local Government and Decentralization Project ( P111577 ) 11. Component 3: Verification and Eligibility of Outputs for Disbursements under Component 1 (US$4.5 million, 100% Borrower-financed). BPKP would be responsible for verifying outputs and its own capacity-building for output verification, in support of disbursements being approved in line with OBD guidelines. BPKP’s annual Output Verification Report (OVR) was to form the basis for the PIU to request DAK reimbursements from the Bank under Component 1. B. SIGNIFICANT CHANGES DURING IMPLEMENTATION (IF APPLICABLE) Revised PDOs and Outcome Targets 12. A Level 1 restructuring was approved on September 30, 2014, which included an Additional Financing. The PDO was revised slightly as “to improve the accountability and reporting of the central government’s Specific Purpose Grants (DAK) for basic infrastructure, consisting of roads, irrigation, water and sanitation, within Selected Local Governments” to specify the infrastructure sub-sectors involved, and to reflect the scale-up. 13. Furthermore, the roles and responsibilities for each ministry (MoF, MoHA, Bappneas and MPWH) were clarified, including an agreement to allocate funds from their own budgets for TA and dissemination of the project to the LGs. Under the original project, that had been solely MoF’s responsibility. The closing date of both loans was December 31, 2018, a three-year extension for the first loan. Two additional safeguards policies were also triggered: Natural Habitats and Pest Management. (See Safeguards below.) Revised PDO Indicators 14. Based on early-stage Implementation Status Reports (ISRs), the target set for both PDO indicators was 100%. These were lowered as part of the AF to 80% (KPI #1) and 88% (KPI #2), considering the larger number of LGs involved. The wording of KPI #1 was also revised slightly. With the original phrasing, the wording implied that the target would be achieved once the system had been developed and used. However, based on ISR results, the indicator was measured as a percentage of participating LGs who made use of WBRS to submit reports, which was captured more accurately in the revised indicator. As the PAD did not set annual or final targets, the AF Project Paper included the necessary targets as well. Table 1. Comparison of Original and Revised KPIs Original Target Revised Target KPI Development and use of an information system 100% % of DAK reports submitted online 80% #1 to which LGs report information to MoF and through Web-Based Reporting System MPW KPI % of physical outputs reported, verified and 100% No change 88% #2 meeting eligibility criteria Page 9 of 62 The World Bank Local Government and Decentralization Project ( P111577 ) Revised Components 15. Component 1: DAK Reimbursement (US$500 million of Bank financing, bringing total financing to $720 million). The component would continue to reimburse GoI for DAK outputs in the roads, irrigation and water sub-sectors, while also adding sanitation as a fourth sub-sector. A two-stage, criteria-based provincial selection methodology was developed to select provinces for participation including: (i) proportion of DAK to overall local budget (APBD), to measure how much DAK contributed to LG’s development; (ii) Human Development Index (HDI), to be able to prioritize LGs with lower income, access to basic services, and human development levels; and (iii) DAK reporting performance to MPWH, to incentivize improved reporting and accountability performance among LGs. The value of the LG incentive payment was explicitly linked to the use of the WBRS for DAK reporting to provide a clear financial incentive for compliance with reporting requirements. 16. Component 2: Institutional Support Program (US$10 million of Borrower financing, bringing total financing to US$18.5 million) to support the following: • Component 2.1: Policy Advisory. TA and policy advice, as well as capacity-building activities, for MoF’s Directorate General of Fiscal Balance (DGFB) to reform intergovernmental transfers and improve LG service delivery. This would include strengthening the Borrower’s M&E framework for intergovernmental transfers. • Component 2.2: Strengthening LG Capacity to Improve Public Service Delivery. Capacity-building activities to (i) improve DAK utilization by selected LGs; and (ii) improve local public service delivery. • Component 2.3: Strengthening Central Government Capacity for Effective and Efficient Use of DAK: This included (i) implementation support on day-to-day project management, (ii) TA for verification support; (iii) TA for M&E support. 17. Component 3: Verification of Outputs (US$10 million of Borrower financing, bringing the total financing to US$14.5 million) included: (i) TA for BPKP staff at central and regional offices to conduct verification of outputs, including reviewing technical, procurement, safeguards and financial management aspects of DAK infrastructure implementation; and (ii) field verification of outputs. Other Changes 18. The project underwent three Level 2 restructurings, in April 2013, January 2016 and December 2017. The 2013 restructuring was undertaken to enhance the verification and reimbursement impacts, by increasing the threshold of qualifying infrastructure contracts from US$400,000 to US$1 million, and extended the closing date by six months from June 30, 2014 to December 31, 2014. The January 2016 restructuring responded to DAK policy changes, specifically the removal of the 10% LG matching contribution requirement. As this had been a requirement for DAK reimbursement under LGDP, the restructuring removed the IOI that tracked provision of the 10% matching fund among LGs. In addition, the procurement threshold was raised to US$3 million, also in response to DAK policy changes that allowed for larger infrastructure contracts. The December 2017 restructuring advanced the loan closing date from December 31, 2018 to December 31, 2017, as the loan had been fully disbursed. Page 10 of 62 The World Bank Local Government and Decentralization Project ( P111577 ) Rationale for Changes and Their Implication on the Original Theory of Change 19. The project’s components remained broadly unchanged through the various rounds of restructuring, with the design of Component 2 undergoing the most significant changes. The project’s Theory of Change was not significantly affected. II. OUTCOME A. RELEVANCE OF PDOs Assessment of Relevance of PDOs and Rating 20. The PDO remains highly relevant to the context of decentralization in Indonesia, as well as the GoI’s push for increased spending on infrastructure. GOI’s Mid-Term Development Plan (RPJMN) 2015 – 2019 focuses on areas such as human and community development, as well as addressing inequality. Governance and bureaucratic reform are also recognized as one of four key “necessary conditions” for achieving its overarching goals. The 100-0-100 target established in the RPJMN – providing 100% access to water, having zero slums and providing 100% access to sanitation – is clearly supported through LGDP, in the form of financial reimbursement and capacity building to improve reporting and M&E of infrastructure delivery. As DAK also transitions towards performance-based grants, the need for improved reporting and accountability remains high. 21. The current World Bank Country Partnership Framework (CPF) for Indonesia (FY2016 – FY2020) recognizes that governance challenges remain an impediment to Indonesia achieving the Twin Goals. LGDP remains closely aligned with Engagement Area 4: Delivery of Local Services and Infrastructure, which notes that improved delivery of services involves targeted support at both the central and LG levels. Pillar 1 of this Engagement Area specifically aims to strengthen the decentralization framework, including the development of a performance-based transfer system and strengthening the capacity of central government agencies to support and oversee performance of LGs. LGDP is also aligned with Engagement Area 6: Collecting More and Spending Better, particularly in terms of improving the delivery/execution of public spending by strengthening institutional capacity to improve procurement performance. This is done through LGDP’s output verification mechanism. Rating: High B. ACHIEVEMENT OF PDOs (EFFICACY) Assessment of Achievement of Each Objective/Outcome 22. The PDO contains two distinct parts: Improving accountability and improving reporting. Based on the theory of change above, improving reporting is seen as a necessary step in improving accountability. Thus, the assessment of PDO achievement considers improvement of (1) accountability and (2) reporting relating to DAK as a whole. Although the PDO was amended slightly as part of the AF, the PDO assessment is not split because the revised PDO merely reflected the Project scale-up. Page 11 of 62 The World Bank Local Government and Decentralization Project ( P111577 ) 23. Prior to LGDP, monitoring and verification of the use of DAK funds were major challenges for GoI. Existing governance systems were weak, with only manual reporting systems for LGs to report on DAK usage. LGs were compelled to comply with financial reporting requirements as MoF could otherwise withhold transfers, but there were few standards governing financial reporting. Technical reporting to line ministries such as MPWH, if carried out at all, had minimal technical information or details on completed physical outputs. In short, MOF had no way to verify if DAK funds were being used for their intended purposes. 24. WBRS was preceded by MoF’s Web-Based Transfer Monitoring System (WBTMS), which focused only on collecting financial information, and not technical information on physical outputs. Under Component 2.2 of the original Project, WBRS was introduced and implemented to capture both financial and technical information that would enable line ministries to assess the status of physical output delivery. Information that was previously not reported, including GPS data on project locations, technical information and photographs to show physical progress, became available in one comprehensive system. This was complemented by capacity-building and socialization among LGs as part of Components 2.1 and 2.3, to familiarize both local departments and central agencies with WBRS and the reporting standards expected under the Project. 25. Enhancements to WBRS were also made over the course of LGDP. An offline system was added in response to feedback from LGs with poor internet connectivity, enabling LGs to document physical progress by typing notes on problems and achievements, downloading photos, and recording GPS coordinates while still in the field. (See Annex 6 for examples.) To draw a clear link between reporting performance and receiving monetary incentives, the DAK incentive payment was modified under the AF, such that compliance with WBRS reporting was required for LGs to receive the full 10 percent incentive value. Previously, the incentive payment was received regardless of how the LG had performed in terms of reporting compliance. Despite the project closing one year early, per KPI #1, 100% of participating LGs were using WBRS to submit their DAK reports, well exceeding the target of 80%. 26. Improved reporting through WBRS was critical to achieving KPI #2, i.e., for verification of physical outputs to take place. Component 3 provided BPKP with TA to build capacity to carry out technical verification of infrastructure outputs, and allocated additional staffing and budgetary resources for field verification. In total, 350 BPKP staff from 22 regional offices received training on output verification, which consisted of 24 hours of classroom training and 16 hours of hands- on field training. Developed jointly by the Bank, BPKP and MPWH, verification handbooks also provided clear guidance to BPKP staff on criteria for the physical inspection of outputs, with illustrations and photographs of what constituted “good” and “bad” outputs in all four LGDP infrastructure sub-sectors. Similar handbooks, with additional tips on steps to follow during planning and construction stages, were also developed for the LGs. Excerpts from these handbooks are found in Annex 6. 27. The output verification process consisted of two phases. In the first phase, using the data captured in WBRS, BPKP screened contracts to confirm eligibility for verification based on the total contract amount, type of procurement, comparability with Reference Unit Costs (RUCs) from MPWH and full physical completion of output. Using results from the first screening, BPKP would calculate a “Value of Potential Reimbursement” (VPR) by multiplying reported outputs by Page 12 of 62 The World Bank Local Government and Decentralization Project ( P111577 ) their RUC. BPKP would then randomly sample 20 percent of outputs from each participating LG for its second phase of verification, which involved field visits for quantitative and qualitative checks on infrastructure, including compliance with environmental and social safeguards. A Qualifying Percentage (QP) of outputs meeting all the eligibility criteria was multiplied by VPR to obtain the Value of Qualifying Reimbursement (VQR). Final reimbursement from the Bank to MoF was calculated as the sum of VQR across all LGs, less any Advance Payment (AP) initially made. In this way, LGs were reimbursed for infrastructure that met criteria in terms of competitive procurement, compliance with safeguards, as well as completion in line with technical specifications. Box 1 provides an illustration of the output verification method. Box 1: Output Verification in Practice – An Example from East Java The second phase of BPKP’s verification of outputs involved actual physical verification of a sample of outputs in the field. Prior to undertaking field visits, the BPKP verification team would spend time at the offices of the LG, reviewing all relevant documents for completeness and compliance, such as bidding documents, contracts, and technical designs. This office-based review would be aided by the verification checklist, agreed on by BPKP and the Bank. BPKP would then conduct the field visits, accompanied by LG counterparts including from the local department of public works (dinas). An example of how such field verification works in practice can be seen in the case of road upgrading in the district (kabupaten) of Malang in East Java Province. In this case, the physical output verified was the upgrading of 3.97 km of a district road, in the sub-district (kecamatan) of Dau. Prior to the upgrading works funded by DAK transfers, this road was unpaved and often impassable during the rainy season. It serves an important function for local residents, connecting three villages directly to the main provincial road in the area. The upgrading works here involved asphalt paving of the entire stretch of road, to the standard width of a district road as specified in the applicable technical guidelines of MPWH. The value of the contract awarded by the LG for this work was IDR 2,318,800,000 (approx. USD 169,255). The BPKP team first drove along the entire stretch of upgraded road, ascertaining through the vehicle’s odometer that the length of paved road was indeed 3.97 km. The start and end points were checked against the technical drawings and maps contained in the actual contract document issued by the LG. The team then worked their way back along the same stretch of road, stopping at regular intervals to measure the width of the road at each point, to ensure that the width of paving complied with the minimum requirement of the technical guidelines. Staff of the local dinas accompanied the BPKP team, supplying a pavement core drilling machine. This machine was used to drill core samples at three randomly selected points along the road. The samples were inspected to confirm that the depth and quality of asphalt met the technical guidelines. An accompanying crew from the dinas immediately filled and repaved the spots from which the core samples were taken. Using the verification checklist, the BPKP team also confirmed that there were no visible environmental safeguards issues arising from the upgraded road, such as poor drainage or slopes vulnerable to erosion as a result of the upgrading work. The visit concluded with the BPKP team determining that this output was eligible for reimbursement. Page 13 of 62 The World Bank Local Government and Decentralization Project ( P111577 ) 28. From a complete absence of output verification prior to LGDP, the introduction of the two-phase verification process, enabled by improved reporting among LGs and capacity-building among BPKP staff, marked a sea change. As of 2014, the Project reimbursed approximately 60% of the overall DAK infrastructure allocation in the participating LGs for the period 2011 - 2014. This gave GoI an indication of how accountability had improved as compared to the situation prior to LGDP, and led to GoI requesting a scale-up of the Project with AF. In total, during the period 2011 – 2016, some IDR 19 trillion (approximately USD 1.36 billion) of DAK infrastructure outputs – or 54% of the total DAK infrastructure allocation – was verified by BPKP using LGDP’s output verification process. Through the Project, GoI could progress from having no output verification of DAK infrastructure to having more than half of its DAK allocation accounted for. 29. At Project closing, 77% of participating LGs’ physical outputs were reported, verified and had met eligibility criteria (KPI #2), falling slightly short of the final target of 88%. The result achieved for KPI #2 under Phase I was 87%, but the nation-wide scale up in Phase II, going from five participating provinces and 78 LGs from 2011 – 2014, to 22 provinces and 267 LGs from 2015 – 2017, proved a challenge to sustaining good performance. The shortfall in funding for Component 2 reduced the scope of capacity-building and socialization activities, which was particularly detrimental in Phase II when the new LGs needed to become familiar with the project’s requirements. In addition, DAK policy changes also impacted the achievement of this KPI. For example, in 2016, LGs were made responsible for covering 10% of their annual DAK allocation. To do so, either extra funds from the local budget had to be set aside, or the volume of work planned had to be reduced to suit the lower budget. This in turn delayed completion of infrastructure projects in the field, which lowered the number of contracts eligible for reimbursement. These are discussed further in Section III-B. 30. Beyond the project’s RF, improved reporting and accountability allowed for information on the exact outputs funded by DAK to be gathered, which previously was not possible. Among others, LGDP reimbursed LGs for the improvement of 5,355km of roads, enhancement of 3,988km of irrigation channels, and improvement of 669km of water supply pipelines. MPWH, which held annual regional consultations with LGs to discuss and agree on their DAK proposals and provide technical guidance, had a degree of certainty through the output verification process that infrastructure had indeed been built in accordance with technical requirements, to ensure safety and sustainability. As detailed in the Efficiency section and in Annex 4, infrastructure was also constructed in an economically efficient manner. 31. BPKP’s verification results also provided useful information on holding LGs accountable for their performance. BPKP’s annual verification reports contained detailed information on the contracts verified for each LG, and where the outputs were determined to be ineligible for reimbursement, the specific reasons for ineligibility. This enabled MPWH to identify and analyze trends in LG performance across regions of the country and by sector, enabling more targeted capacity building tailored to the specific areas of LG weakness. 32. Impact analyses of LGDP were also carried out at different phases of the project and revealed some results indicating improvements in accountability. For example, an impact analysis commissioned by the Bank in 2016 found that LGDP participation had a positive effect on district budget management and accountability, as measured by the State Audit Board (BPK) opinions. Project districts were found to be 10 % less likely to receive a disclaimer, 1.1 % less likely to obtain an adverse opinion and 9.2% more likely to receive an unqualified audit opinion. While this analysis was carried out based on data for LGs that participated only in Phase I, they point to the positive impacts of LGDP participation on Page 14 of 62 The World Bank Local Government and Decentralization Project ( P111577 ) accountability. Another important finding from this impact analysis shows that LGDP participating districts spend 12-15 percent more on capital expenditures compared to non LGDP participants, resulting in improved access to basic services. Justification of Overall Efficacy Rating 33. The overall efficacy rating is Substantial. All 267 participating LGs used the new WBRS to report their infrastructure outputs, and 77% of all their physical outputs were reported, verified and had met eligibility criteria by loan closing, one year earlier than planned. C. EFFICIENCY Assessment of Efficiency and Rating Economic Efficiency 34. An ex-post economic analysis was carried out at the ICR stage, using the same methodology as at appraisal stages for both Phases. Annex 4 contains full details of this analysis. The economic analysis estimates benefits from the project that are readily quantifiable in monetary terms, such as from transport cost savings from reduced fuel consumption and maintenance costs, and cost savings for purchasing clean water. General benefits expected from the project’s reimbursement of DAK payments are described in Table 2 below. This is not an exhaustive list of potential benefits since LGDP targeted institutional strengthening to improve reporting and accountability relating to DAK, for which positive externalities are likely, but are not quantifiable. The benefits of the project are thus likely to be greater than those quantified here. Table 2: Projected Benefits from LGDP Roads Water Supply & Sanitation Irrigation Transport cost savings from reduced fuel Time savings in obtaining clean water; Increase in crop yields (per unit area); consumption and maintenance costs; Cost savings for purchase of clean water Increase in area under cultivation; Time savings from smoother road travel; from vendors; Lower health expenditures Cultivation of higher value crops and Gains from better market access for and increased productivity from reduced higher yielding crop varieties; Increase in goods; Increased income of transport incidence of water-borne disease. farmer incomes (from increased operators arising from greater demand productivity factors and reduced travel for transport services; Enhanced capital times to markets); Reduced risks of, and values of property made more accessible costs arising from, extreme events such by improved roads. as droughts or floods; Enhanced capital values of agricultural land with improved irrigation. 35. Given the complexity of estimating future DAK investments in all districts and provinces involved in LGDP, representative output types were chosen for each of the sub-sectors, based on outputs most commonly funded by DAK investments in these provinces: • Roads: kilometers of roads upgraded (peningkatan) to a standard width of around 4 meters; Page 15 of 62 The World Bank Local Government and Decentralization Project ( P111577 ) • Water supply and sanitation4: number of houses connected to a clean water supply with a 2-inch diameter pipe connection; • Irrigation: hectares of cropland irrigated as a result of upgrading (peningkatan) of irrigation infrastructure. Unit costs for each output type in each province were obtained, based on RUC data from MPWH. The value of DAK investments were then divided by the unit costs, to obtain the quantity of representative outputs that notionally would be delivered by these DAK investments in each province and sub-sector. 36. The economic analysis was carried out using a cost-benefit method to estimate the net benefit to society arising from the project. The results of the cost-benefit analysis are summarized in Table 3 below. The Net Present Value (NPV) over five years, at a discount rate of 12%, is estimated at US$544.2 million, with an economic internal rate of return (EIRR) of 45 percent. The EIRR figure is greater than that obtained in the ex -ante analysis for Phase I (40%) and Phase II (30%). The ICR evaluation thus demonstrates that actual project benefits exceeded those anticipated at appraisal. Table 3: Cost, Benefits and NPV for Project Investments (USD Million) LGDP Phase 1 LGDP Phase II ICR Cost 267.7 476.2 1,093 Benefit 387.9 756.9 1,915 NPV 56 89.1 544.2 EIRR 40% 30% 45% 37. A sensitivity analysis was also conducted with respect to two variables: (i) a change in the average unit costs; and (ii) a change in the estimated average unit benefits, given their vulnerability to unforeseen variance across LGs. The analysis showed that all else remaining equal, the project could sustain a reduction in the proportion of estimated average unit benefits of 36% in each year of the projection and still be economically feasible. Similarly, it would take an increase of 24.5% in average unit costs to reach a negative expected net present value (ENPV). The results reveal that the overall net benefits have minimal sensitivity to these key variables. 38. In addition, a descriptive analysis comparing actual average unit costs for road rehabilitation (i.e. improvement from soil to asphalt) per kilometer in five LGDP and non-LGDP provinces, or the likely situation without the project, was carried out. The analysis shows that actual road maintenance unit costs under LGDP are relatively lower than those of non-participating provinces in Jambi, Central Kalimantan, East Java and North Maluku. The actual unit costs were higher in West Sulawesi for participating provinces compared to non-participating ones; however, the gap was relatively small5. 4 DAK intervention on sanitation is scattered and mostly on very small investment such as construction of basic bathing, washing, toilet facilities, and communal piped wastewater system. Therefore, for the analysis we combined both water supply and sanitation 5 Actual unit cost for LGDP stands at IDR2.045 million (per km); whereas, for non-LGDP is at IDR1.8 million (per km) in West Sulawesi. Page 16 of 62 The World Bank Local Government and Decentralization Project ( P111577 ) Implementation Efficiency 39. US$210,592 of Bank Budget (BB) was spent on project preparation, and it took 13 months for LGDP to go from concept to Board. Compared to other investment projects in the active Indonesia urban sector portfolio, LGDP had a lower than average period from concept to first disbursement (26.8 months vs. 33.4 months). In terms of implementation, costs amounted to US$2.1 million of BB over a period of 8.5 years, which included two Mid-Term Reviews (MTRs), one Level 1 restructuring for AF and three Level 2 restructurings. Given the size of the loan and the geographical reach of the project, comparisons of implementation costs across the urban portfolio were not necessarily representative. The assessment is that implementation was efficient given the project’s geographical reach and supervision demands, and the achievement of the PDO despite counterpart funding shortages. Early closure of the loans also contributed to greater implementation efficiency in terms of time and costs saved for continued supervision for the Bank and reduced management costs for the borrower. 40. The ICR-stage economic analysis shows that actual project benefits exceeded those anticipated at appraisal. Combined with the implementation efficiency of the project, the overall Efficiency rating is Substantial. D. JUSTIFICATION OF OVERALL OUTCOME RATING 41. Given the high rating for relevance and the substantial ratings for both efficacy and efficiency, the overall outcome rating for LGDP is Satisfactory. E. OTHER OUTCOMES AND IMPACTS (IF ANY) Institutional Strengthening 42. Despite the shortfall in funding for Component 2, LGDP had an overall positive impact on institutional strengthening. The PIU channeled available funds to conduct capacity-building among LGs, focusing on sharing information and criteria relating to LGDP, including improving reporting procedures and encouraging the use of WBRS. LGs were also provided with training in terms of DAK infrastructure technical guidelines, as well as procurement and environmental and social safeguards, to better ensure that output verification criteria could be met. Capacity-building in these aspects was carried out between 2011 to 2016, and covered all participating LGs. MoF also introduced an award for LGs that had shown good performance in reporting in 2014, as a means of encouraging LGs to improve reporting. Performance under KPI #2 benefitted from the institutional strengthening for LGs. 43. With MoF’s funding constraints for Component 2, individual line ministries - as part of the PIU - set aside portions of their own training budgets to provide capacity-building. At closing, US$8.5 million had been provided by GoI, including dedicated funding from the budgets of MPWH, MoHA, and MOF. This was also complemented by two rounds of support totaling US$1 million from the Global Partnership for Output-Based Aid (GPOBA), that were obtained and implemented by the Bank task team. Capacity-building focused on developing capabilities in M&E, in line with the comprehensive M&E framework that the project proposed to develop. While the result was less extensive than originally envisioned – again given the shortfall in funding –positive impacts were observable among individual ministries. The PIU team – including MoF, MoHA, and MPWH – is now able to apply several impact evaluation techniques in their programs, such as correlation analyses, Difference in Difference, and Propensity Score Matching. M&E is discussed further in Section IV-A. Page 17 of 62 The World Bank Local Government and Decentralization Project ( P111577 ) 44. Capacity was also built among BPKP auditors, who previously had no experience with physical verification of outputs. The creation of handbooks to guide the verification process aided the capacity-building significantly. After closing, and following the transfer of DAK disbursement responsibilities from the Directorate General of Fiscal Balance (DGFB) to DG Treasury, MoF sought the Bank’s assistance to share output verification knowledge between BPKP auditors and staff from MOF’s LG offices (Kantor Pelayanan Perbendaharaan Negara - KPPN), with a view to introducing the verification mechanism more widely in DAK administration and strengthen the disbursement mechanism of DAK funds. This is described further in the “Other Unintended Outcomes and Impacts” section below. 45. In May 2017, the Bank and GoI arranged a South-South Knowledge Exchange event in New Delhi and Kolkata, to share LGDP’s experience in the application of performance-based grants. GoI and project representatives from India, Nepal, Sri Lanka, and Bangladesh participated actively in the event, and shared their own experience on the subject. The LGDP experience was particularly attractive as a case study because of its cooperation with BPKP as the Independent Verification Agent as opposed to the use of a private firm, which is more commonly the case. Poverty Reduction and Shared Prosperity 46. LGDP reimbursed GoI for basic infrastructure improvements made in four infrastructure sub-sectors. Aside from the reimbursement being a direct contribution to service delivery improvement, an impact analysis commissioned by the Bank in 2016 demonstrated that LGDP had a positive impact on participating LGs’ capital expenditure, with districts participating in the project spending about 12-15% more on capital than non-participating districts. The increased capital spending of LGDP districts was also found to result in improved service access. District participation in LGDP led to a 4 % increase in household access to water, an increase of about 0.3 kilometers of roads per person, and a 5.5 % increase in rice paddy under irrigation, compared to non-participating districts. Given the links between under-investment in basic infrastructure, poor service delivery and poverty, the improvements in basic service delivery made both directly and indirectly through LGDP could theoretically lead to poverty reduction. Other Unintended Outcomes and Impacts 47. The success of the output verification methodology in LGDP has inspired GoI to adopt the approach more broadly. In 2017, GOI required verification of DAK outputs before the DAK funds could be disbursed by the KPPN. This was a significant change to the DAK funds disbursement methodology, which previously did not involve any verification to confirm realization of outputs. The mandate was institutionalized through PMK No. 50/2017 on Management of Intergovernmental Transfer and Village Funds. BPKP output verification also informed a policy change to DAK operationalization, with MoF revising its regulation on DAK transfers to further enhance reporting compliance. LGs that do not submit a complete set of DAK reports (i.e. both financial and technical reports) would not receive the following year’s DAK funds. 48. MPWH also responded to findings of BPKP’s annual output verification exercise. A significant proportion of water sector sub-projects were found ineligible for DAK reimbursement as the pipes funded by DAK were connected to water resources managed by local drinking water companies. This was not allowed under the prevailing technical guidelines. Based on BPKP verification reports, 20% of ineligible contracts between 2011 – 2015, were affected by this issue. Seeing that the connection to the water resources was necessary and beneficial in ensuring that water was being channeled to Page 18 of 62 The World Bank Local Government and Decentralization Project ( P111577 ) its intended beneficiaries, MPWH modified its technical guidelines to allow such connections. In 2016, the verification report showed that this was no longer a contributing factor to contracts being deemed ineligible. III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME A. KEY FACTORS DURING PREPARATION 49. LGDP was designed in close consultation with GoI, which displayed ownership of the project. Both a stand- alone, input-based operation and a Development Policy Loan were considered as alternatives, but the OBD approach was deemed best suited to strengthen GoI systems at both central and local government levels. The five provinces selected for Phase I provided diversity both in terms of geography, capacity to implement DAK, and existing performance criteria such as compliance with reporting. To further ensure commitment from participants, only LGs that had signed commitment letters with MoF were eligible to participate. 50. The project design incorporated lessons from several relevant projects , including local governance projects in Indonesia, global pilot projects funded by GPOBA, and OBD projects in Mexico and Brazil. Experiences from these projects suggested that, among others, (i) geographic concentration, while also allowing for diversity, would ease implementation complexity; (ii) reliance on government-led reforms that simultaneously strengthened government systems was important for sustainability; (iii) the role of the central government should transition away from implementation to M&E; (iv) disbursement of funds should be linked to outputs to increase economic efficiency; and (v) estimated RUCs were necessary to determine reimbursements. 51. In terms of risk analysis, the project identified all major risks and put in place mitigation measures that were appropriate. In particular, the risk of GoI not providing adequate funding for the TA component was rightly assessed as “High”. Actual implementation proved to be a challenge due to continued budget shortfalls on GoI’s end, but attempts were made to address these with alternative sources of funding and improved design of the project’s institutional support program. B. KEY FACTORS DURING IMPLEMENTATION 52. In 2014, a national policy shift put specific attention on increasing and improving the quality of infrastructure in Indonesia, as reflected in the RPJMN 2014-2019. This in turn resulted in a significant increase in GoI’s infrastructure expenditure, including DAK allocations. Combined with the success of Phase I of LGDP – reimbursement from the Bank to MoF showed more than half of the infrastructure in participating LGs was verifiable and accounted for –, led to GoI requesting AF for a nation-wide scale-up. The AF introduced a two-stage selection criteria to further ensure diversity in terms of geographic reach, income levels and existing performance in terms of reporting compliance. The scale-up, while ambitious, was intended to achieve a broader, more systematic impact on LG performance. 53. The significant increase of DAK allocation, particularly in infrastructure sector, led to the quicker pace of disbursement in the second phase of project funding. While on the one hand this had a positive effect on project implementation efficiency – LGDP closed one year ahead of schedule – it also resulted in a shorter period of time for socialization and capacity-building activities. This was exacerbated by the lack of available funding, as mentioned in paragraph 55 below. Page 19 of 62 The World Bank Local Government and Decentralization Project ( P111577 ) 54. Several DAK policy changes had implications for implementation, especially at the LG level. As described in Section II-B, the requirement for LGs to provide 10% of their annual DAK allocation resulted in delayed completion of infrastructure projects, which reduced the number of reimbursable contracts under LGDP. The restructuring of LG dinas or Work Units also resulted in administrative delays and challenges. For example, previously, one dinas oversaw all LGDP sub-sectors, but the 2016 restructuring resulted in functional separation between roads/ bridges and irrigation/ water supply. Therefore, two different dinas had to be staffed and had to coordinate their work plans and field visits. 55. The shortfall in GoI funding for TA components reduced the number of socialization and capacity-building activities that could be carried out, as well as implementation capacity of the PIU including its M&E capacity. Phase I was planned with US$13 million in GoI funding for institutional strengthening, project management support and output verification, while Phase II was planned with a total of US$20 million. Across both phases, a total of US$8.5 million was eventually provided – US$1.3million in Phase I, and US$7.2 million in Phase II. The significant increase in GoI funding in Phase II was necessary given the significant increase in participating LGs increased significantly over a short period (from 78 participating LGs to 267, or an increase of 242%). It also reflected the commitment of individual PIU agencies to the objectives of the project, as capacity-building funding under Phase II was largely provided for in each agency’s own budget, as opposed to an additional allocation from MoF. 56. Staffing challenges were also faced by the PIUs. The project was implemented by several PIUs including MoF (anchored at the Directorate General of Fiscal Balance (DGFB)), MoHA, BAPPENAS, and MPWH. While the PIUs were to be staffed with a core team of specialists from relevant fields (e.g. FM, procurement, engineers, etc.), it was envisaged that the PIUs would also be supported by consultants. Due to funding shortages, hiring these consultants proved challenging, which in turn negatively affected the PIUs’ ability to carry out socialization, timely reporting and M&E, including the submission of Interim Financial Reports (IFRs). The IFRs were submitted on time in Phase I, but were often delayed later on. This was exacerbated by an internal MoF restructuring in 2016 that caused a management turnover in the PIUs. Over the period of implementation, the MoF PIU saw four different directors. IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME A. QUALITY OF MONITORING AND EVALUATION (M&E) 57. Two aspects of M&E design are assessed – first, M&E of the project itself in terms of the Results Framework (RF) and monitoring of progress and achievements; and second, the M&E framework designed as part of Component 2. M&E Design 58. The RF sought to address the main concerns associated with DAK implementation as identified during project preparation, including inadequate DAK reporting, unsubstantiated and unverified reports from LGs and limited monitoring by provincial governments. The two KPIs were adequate for measuring improved reporting and implementation of the output verification process, as well as, by extension, improved accountability. However, the seven IOIs focused on the achievement of milestones (e.g. submission of annual reports, receiving annual DAK payment by a specified date), as opposed to time-bound, quantifiable targets that could track progress clearly. Not all the IOIs were clearly linked to the KPIs, and the RF could have included more indicators that measured progress of the TAs, given the links between capacity- Page 20 of 62 The World Bank Local Government and Decentralization Project ( P111577 ) building and the achievement of the PDO. Another weakness was that the RF did not establish any baselines or target values. At appraisal, the indicators in the RF also had no accurate baseline data (e.g. technical reporting to MPWH). 59. Results from the MTR in 2013 showed that there was overlap between some of the IOIs, such as timeliness of the LG budgeting process, timeliness of DAK project implementation, and operationalization of WBRS, which appeared as both an IOI and a PDO-level indicator. The Phase II RF was strengthened by removing redundant indicators, but could have been improved by adding indicators relating to the TAs. All IOIs, except #5 in the original Project, #6, #8, and #9 in the AF, have direct linkage and contribute to the improved accountability and reporting of DAK infrastructure, while those excepted indicators were key to support the PDO achievements. M&E Implementation 60. Most RF indicators were milestone-based and collected annually. As the indicators were part of the eligibility criteria for verification by BPKP, data was readily available through BPKP’s annual output verification report. During Phase I, project monitoring activities focused on the achievement of RF indicators, without any systematic or thematic evaluation by GoI. In 2014, the Bank conducted an interim impact analysis to measure the impact of LGDP participation on LG capital spending and service delivery quality, as described in paragraph 46. The Bank team also carried out a series of impact evaluations in two consecutive years (2016 and 2017) to observe the impact of LGDP interventions, particularly the output verification mechanism and incentive provisions to DAK implementation and specifically intended to identify the spillover effect that LGDP might have to a few areas of LG service delivery quality. The result suggests that through participation in LGDP, changes in service outcomes occurred, includes, inter alia: (i) 0.3 km increase in paved roads per capita, (ii) 2.3% rise in household access to protected water; and (iii) 3.5% increase in rice paddy under irrigation. 61. In 2013, the Bank secured grant funding from GPOBA to strengthen M&E capacity of the PIU, with M&E trainings delivered from 2014 to 2016 to MoF, MoHA, Bappenas, and MPWH. The training modules included basic results-based M&E training, impact evaluation training, and logical framework training. M&E was further revised and improved in the AF. Following capacity-building and M&E training, MoF carried out LGDP impact analyses in 2015, followed by MPWH on LG performance analysis, and MoHA on LG performance analysis in 2016 and 2017 respectively. The agencies employed various impact evaluation methodologies, as learned from the M&E training. This reflected improved ownership of M&E among the different PIU members, which had been identified as a weakness in Phase I. M&E Utilization 62. MoF’s Regulation No. 112/2016 on M&E for Specific Intergovernmental Transfer reflected GoI’s commitment to improved M&E. This regulation introduced a new approach to the monitoring of intergovernmental transfers, and provided detailed guidelines on how to evaluate results for various intergovernmental transfer funds. Applied to all DAK sectors, this marked a substantive change in GoI’s M&E methods. It is important to note that the development of the Regulation as well as the various analyses of LGDP performance by PIU members were financed by the national budget, which shows acknowledgement on GoI’s end of the importance of M&E. Notably, MoF indicated that this Regulation was influenced by the results-based M&E training offered by LGDP. Page 21 of 62 The World Bank Local Government and Decentralization Project ( P111577 ) 63. PIUs carried out LGDP performance analyses and used these in different ways. The results of MPWH’s performance analysis informed their capacity-building methodology, enabling it to be more targeted on LGs that had performed poorly. The National Procurement Policy Agency and the MPWH Inspectorate were able to target capacity building to specific LGs on procurement and technical specifications respectively. MoHA carried out a separate performance analysis among all LGs in 2016 and 2017, developing an in-house dashboard system to monitor and evaluate the timeliness and quality of DAK reporting. The analysis found that the DAK infrastructure reporting rate was higher among LGDP participants than non-LGDP participants, and was used to rank LG performance in terms of DAK reporting. MoHA has indicated that the creation and sharing of a ranking system among LGs – making performance transparent and allowing LGs to see how well others have performed – has created an incentive for LGs to improve DAK reporting compliance. Based on MoHA’s database, the quarterly reporting rate in 2017 increased significantly, hitting the highest rate of DAK reporting since 2010. Justification of Overall Rating of Quality of M&E 64. Quality of M&E is rated Substantial. The design and implementation of M&E in Phase I were rather weak, but considerable efforts were made to enhance M&E in Phase II, particularly by the Borrower. While individual ministries have had capacity strengthened and are utilizing M&E skills in their own areas of work, the project could have developed a more holistic framework for overall evaluation and analysis of LGDP results, including potential impacts on policy changes, coordinated by the PIU. B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE Environmental and Social Safeguards 65. Potential adverse environmental and social impacts were expected to be minor to moderate in scale, as infrastructure reimbursed under LGDP focused mainly on maintenance or rehabilitation as opposed to new construction. The Bank’s environmental and social safeguards requirements, procedures and protocols were incorporated as a Supplement of MPWH’s Technical Guidelines, that LGs had to comply with in order to qualify for DAK reimbursement. The guidelines were updated as required. The project was appropriately given a Category B classification, with three policies triggered at Phase I - OP/BP 4.01 (Environmental Assessment), OP/BP 4.10 (Indigenous Peoples) and OP/BP 4.12 (Involuntary Resettlement). With an expanded geographical scope in Phase II, two additional policies were triggered – OP/BP 4.04 (Natural Habitats) and OP 4.09 (Pest Management). The Category B classification was retained, and revised Technical Guidelines were disclosed on the MPWH website on June 24, 2014, and in Infoshop on June 23, 2014. 66. The safeguards rating for LGDP was Moderately Satisfactory. Throughout project implementation, the MPWH Technical Guidelines and Supplemental technical guidelines which include safeguards requirements were applied by the participating local governments. During the output verification process, DAK subprojects that fail to comply with the environmental and social safeguards requirements under the MPWH Technical Guidelines were declared ineligible by BPKP, and expenditures were not reimbursed. The Bank team also provided support to confirm these results during its Page 22 of 62 The World Bank Local Government and Decentralization Project ( P111577 ) implementation support missions. However, the awareness and understanding of the MPWH Technical Guidelines varied among LGs. Intensive socialization and capacity-building were required to ensure familiarization with the various safeguards requirements, and such activities were impacted by shortfalls in funding for the TA Components especially in Phase II. Financial Management (FM) 67. FM arrangements were generally adequate for the project in both Phase I and Phase II, with a rating of Moderately Satisfactory given at loan closure. Thorough assessments of FM capacity were carried out at appraisal. FM risk at Phase I was rated High (Substantial after mitigation) considering the lack of verification of DAK funds by MOF, as well as the variation in payment verification mechanisms across LGs. Similar shortcomings were identified during AF appraisal, but with Substantial (Moderate with mitigation) overall risk. Budgeting and reporting of DAK implementation at LG level generally improved over the course of the project, but funding shortages for capacity-building limited the amount of training that could be provided in other areas of weakness, including the improvement of internal controls for DAK implementation at the LG level. 68. The timeliness of IFRs submission worsened over time, likely because of management turnover at the MoF PIU. Seven IFRs remained overdue even during preparation of the ICR, after the Project had closed. The annual audits of the project were conducted by BPK, the independent State Audit Institution in Indonesia, and were found to be satisfactory to the Bank. Audit reports were received in a timely manner, and unqualified opinions were received with no significant findings in the last 5 years of project implementation. Procurement 69. Procurement was rated Moderately Satisfactory at project close. The Project adopted national procurement regulations and procedures, and targeted institutional strengthening by (i) introducing LA covenants requiring establishing and improving capacity of local procurement services units (ULPs) with proper staffing and budgeting; and (ii) designing the output verification process to ensure that only competitively bid contracts procured through the respective ULPs under a specified threshold were eligible for reimbursement, for BPKP’s review. The RF also contained an IOI on percentage of LGs with a functioning ULP. This indicator had a 99.6% achievement rate at project closure as compared to the 80% target, largely because the establishment of ULPs also became a national requirement under revised national procurement regulations (Perpres 54/2010 and its amendment of Perpres 70/2012). 70. LGDP reimbursed DAK expenditures for the verified contracts procured and awarded under the national procurement procedures and did not conduct procurement review process of the selected contracts. At appraisal, it was assessed that national procurement procedures under Keppres 80/2003 generally contained elements of international best practice, and that they were well-suited to the output-based approach. LGs thus followed applicable national procurement procedures as laid out under GoI’s procurement regulations. Given that the value of contracts for the eligible DAK sub-sectors was small (capped initially at US$400,000, but raised over time to US$1 million and US$3 million), it was anticipated that no international bidders would participate. Use of national competitive national procurement was agreed as eligibility criteria for reimbursement of outputs under three of four of the DAK sub-sectors Page 23 of 62 The World Bank Local Government and Decentralization Project ( P111577 ) under LGDP; the sanitation sector was required to adopt acceptable competitive, community-based procurement procedures. C. BANK PERFORMANCE Quality at Entry 71. The design of the project was innovative, as it moved away from financing stand-alone projects to enhancing the overall government framework within which infrastructure investments take place. The introduction of output verification as a condition for DAK reimbursement also emphasized the need for delivery of results in line with GoI’s own laws and regulations. LGDP was well aligned with the 2009-2012 CPS, and was designed in close consultation with GoI, weighing the pros and cons of alternative design options and taking into account lessons from several similar operations, both in Indonesia and globally. Major risks were correctly identified and mitigated. 72. The project’s logical chain and RF could, however, have been improved. LGDP aimed to improve both accountability and reporting of DAK. While improved accountability arguably begins with enhanced reporting, accountability could have been better defined and measured. The RF retained a focus more on enforcement of standards related to monitoring and reporting, as opposed to time-bound, measurable project impact. The absence of sound baseline measurements, e.g. the existing rate of financial and technical reporting prior to LGDP, also affected the assessment of impact. There was also no clear way to track the implementation of the various sub-components under Component 2, making it difficult to monitor progress and eventually attribute progress to project interventions. 73. The TA component was ambitious, particularly considering that it was to be funded entirely by GoI and that no other external sources of funding had been secured at appraisal. The project correctly identified the risk of having insufficient TA funds, and attempted to address this by introducing a LA covenant for GoI to set aside adequate funding and provide a TA work plan before DAK reimbursements were made. However, the total funding amount and scope of TA activities were perhaps over-stated. As the roll-out of LGDP also required extensive socialization to familiarize LGs with the various requirements and standards – reporting through WBRS, compliance with procurement and safeguards requirement, as well as technical guidelines from MPWH – project design could also have included a clear plan to ensure sufficient capacity-building was carried out prior to the first round of verification and reimbursement. 74. In terms of ensuring sustainability, the project’s TA component included a handful of sub-components that referenced institutional arrangements and regulation changes pertaining to the effective use of DAK. The design of these sub-components could have been more specific in terms of how practices developed under LGDP could be formally institutionalized. It was unclear, for example, how output verification would be continued after LGDP ended. MOF had requested the output verification by BPKP in an ad-hoc basis, and there is currently no formalized process for verification to take place for all DAK sectors. As discussed further in Section IVD, this has implications for the continuity of the process, which was a necessary component of enabling the project to achieve its PDO. Page 24 of 62 The World Bank Local Government and Decentralization Project ( P111577 ) 75. Quality at Entry is rated Moderately Satisfactory. Although the project thoroughly considered lessons learned from other similar operations and was innovatively designed to improve GoI frameworks at both the central and LG levels, there were some aspects that could have been improved, particularly in terms of M&E and ensuring sustainability. Quality of Supervision 76. The project had three different Task Team Leaders (TTLs) over the course of implementation, but retained a core team of specialists who were familiar with the project and key stakeholders from preparation to closure. All TTLs were based in Jakarta which allowed them to maintain a close working relationship with the PIUs. Supervision of projects in the field was a challenge given the geographical spread of the project (particularly in Phase II), but the task team visited selected provinces on almost every supervision mission, and ensured that a mix of provinces in terms of geographical spread and project performance were included. Safeguards and fiduciary staff were actively involved at all project stages, including preparation, implementation, and completion. 77. Throughout implementation, the task team was pro-active in identifying problems and proposing solutions. For example, weaknesses in the output verification process were identified early on and were attributed to a lack of technical skills in BPKP, as well as insufficiently thorough verification fieldwork. The Bank, together with BPKP and MPWH officials, revised the project’s Verification Manual to provide more detailed explanations of verification checklist items, as well as other supporting documents required to carry out verification. Project restructurings undertaken during implementation responded to identified challenges (e.g. raising the procurement thresholds to increase the number of contracts eligible for verification), as well as to DAK policy changes at the national level. 78. The task team also designed the AF in close consultation with GoI, and considered alternative project design options. AF discussions were initiated in January 2012, only a year after Phase I had become effective. This gave the team and the client sufficient time to carry out stakeholder discussions and study different project design options, including a Program for Results (PforR) approach. This was eventually decided against in favor of an AF, given GoI’s interest in scaling up the Project in an effective manner, and the relative familiarity of GoI with the original LGDP approach. 79. The task team helped address the shortfall in GoI funding for TA. Two tranches of GPOBA funding, totaling US$1 million, were obtained to implement part of the M&E capacity development activities, without which no M&E capacity- building would have taken place. The project’s AF was candid in acknowledging the funding shortfall and sub-par performance of Components 2 and 3 in Phase I. However, enforcing the mitigating measures established as LA covenants proved difficult. GoI’s reluctance to borrow for TA – this would have necessitated complex external indebtedness approvals that would have significantly delayed AF implementation – also limited alternative options for securing funding for capacity-building. The task team attempted to address this weaknesses in consultation with GoI by proposing a more comprehensive, clearly delineated Institutional Support Program (ISP) in Phase II. 80. Quality of Supervision is rated Satisfactory. The Bank provided close supervision, was candid in its ratings of project progress and assessment during the MTR, and sought to pro-actively address challenges encountered. Page 25 of 62 The World Bank Local Government and Decentralization Project ( P111577 ) Justification of Overall Rating of Bank Performance 81. Bank performance is rated Moderately Satisfactory. Performance was strong in terms of introducing an innovative project design, maintaining a consultative relationship with GoI and close project supervision, but the project also contained an overly ambitious TA component and could have improved both its M&E framework and measures taken to better ensure sustainability. D. RISK TO DEVELOPMENT OUTCOME 82. Decentralization remains a key priority of GoI, as are community development and the reduction of inequality. Transfers to LGs in support of basic infrastructure development and fiscal decentralization are thus expected to continue being priorities, with DAK playing a central role. The risk of reduced government commitment to funding DAK transfers and improving effectiveness of DAK funds is low. However, other risks remain. 83. Firstly, output verification has not been fully institutionalized as a process to determine effective use of DAK. Interviews with participating ministries during ICR preparation indicate that there is wide recognition of the importance of output verification and appreciation of the system introduced under LGDP, as well as a desire to continue and possibly expand usage to other sectors. At the same time, more widespread and consistent implementation of output verification requires additional government funding and training of staff. As such, short of MoF requesting output verification by BPKP, there is currently no formalized process for verification to take place. Regulation 50/2017 includes an article requiring that DAK transfers undergo verification with administrative and physical checks; however, it lacks detail in terms of exactly how the verification is to take place and by whom. Such institutional arrangements remain under discussion. While BPKP developed the capacity to do so under LGDP, MoF is considering whether the function should be moved within the Ministry. Unless the process is mainstreamed, output verification will continue on an ad-hoc basis. Risk: Moderate. 84. Changes to the structure of incentive payments may hamper continued reporting practices and possibly future expansion of reporting requirements. As part of LGDP, incentive payments were offered to LGs who met criteria for reporting and verification of outputs. Impact analyses carried out by the Bank demonstrated that these payments had a small positive effect on LG capital spending. However, policy changes in 2016 led to a grouping of all government incentive transfers under one fund, such that LGs could no longer distinguish between incentives for LGDP performance and those tied to other government schemes. Without this clear link, LGs may not continue established practices. While the impact analysis also indicated that undergoing the output verification process had a clearer impact on LG performance than the incentive payment, the lack of a monetary incentive may hinder the larger-scale implementation of improved reporting standards with LGs that did not participate in the project. Risk: Moderate. 85. Third, in terms of reporting, WBRS was found to be an effective tool in the context of LGDP. However, WBRS has been replaced and others have been developed in parallel by MoF (OM-SPAN) and MPWH (eMON). These have adopted some key features of WBRS, but also have some weaknesses. WBRS for instance allowed for offline reporting, which was critical in provinces with poor internet connectivity. The system also required GPS data and photographs to show physical progress of implementation, which were useful for monitoring. Critically, WBRS contained information that BPKP auditors needed to carry out output verification, but not all this information exists in either MoF’s or MPWH’s current reporting Page 26 of 62 The World Bank Local Government and Decentralization Project ( P111577 ) systems. The lack of integration across both systems may result in fatigue among LGs in terms of needing to comply with multiple forms of reporting. LGDP participants had to submit reports through WBRS in order to qualify for reimbursement, but a similar requirement may need to be instituted to compel LGs to continue reporting using separate systems. Risk: Moderate. 86. On balance, Risk to Development Outcome is rated Moderate. V. LESSONS AND RECOMMENDATIONS 87. A project design that centers on strengthening country systems helps to ensure continuing relevance in a changing policy environment. The core design of LGDP focused on improving reporting and accountability on the use of DAK infrastructure funds transferred to LGs, by utilizing BPKP – GoI’s own internal auditor – as the independent verifier of outputs, and by introducing WBRS as part of GoI’s standard reporting requirements. This basic design proved robust and remained relevant throughout implementation, despite changes in GoI’s DAK policy, that introduced different categories of DAK allocations and modified the incentive structure for LGs. Regardless of how DAK allocations were made to LGs, and how LGs were supposed to utilize the funds, LGDP helped ensure that GoI received reporting and verification on the use of the DAK infrastructure funds in those LGs that participated in LGDP. 88. Scaling-up from a successful pilot project requires careful analysis of readiness and full government commitment for implementation. Given the initial success achieved by the Project in the five pilot provinces, replication and scale-up to many other provinces aimed to widen the Project’s positive impacts. Given the size and diversity of Indonesia, however, such replication requires considerable financial and human resources. This in turn requires that the government is fully ready to do so, including to commit the necessary resources. Owing to the lack of funding for technical assistance, the significant scale-up in LGDP created many challenges to achieving the same level of success recorded in the initial phase of the Project. 89. Adequate time and financial resources for capacity-building activities should be established upfront, with funding secured from various sources aside from the government. While LGDP still achieved institutional strengthening among participating agencies and LGs, the full scale of the project’s ambitious TA components was hampered by a lack of funding from GoI’s budget -- the only planned source under this project. The identification of confirmed funding sources (e.g. trust funds or other development partners) as early as project appraisal, and the development of an implementation timeline that takes into account the time needed for socialization and capacity-building, can better ensure that project outcomes are achieved. More flexibility in the design of the various sub-components, including the possibility of partial financing from the loan, could also have responded to constraints during project implementation, given the high risk of a funding shortfall. 90. With well-structured arrangements, using an independent government agency for output verification can be a viable option that promotes implementation efficiency and sustainability. A significant innovation in the LGDP design was to use the government’s own internal auditor, BPKP, as the independent verification agent. The established practice with Results-Based Financing approaches, at the time of project preparation, was to use a verification agent that is Page 27 of 62 The World Bank Local Government and Decentralization Project ( P111577 ) external to government. The experience of LGDP has shown how verification arrangements can be put in place that not only ensure the independence of the verification process, but also significantly strengthen government institutions and capacity. The additional benefit of this approach was increased efficiency in project implementation, since the cost of verification by BPKP was lower than what would have been spent to contract an external verification agent. 91. Institutionalization of any reforms needs to be designed upfront to ensure sustainability beyond the project. Despite the acknowledgement by GoI of the usefulness of output verification, the process now only takes place at the request of MoF, and has not been mainstreamed into fiscal transfer processes. Feedback from GoI indicates that institutionalization requires additional funding for capacity to be built among the relevant agencies, which will also take some time to develop. Clearer identification of how such a process can be institutionalized, including an analysis of any supporting policies and regulations that may need to be put in place, should be carried out early in project design. 92. To better facilitate iterative learning in projects, a robust M&E framework with clear responsibilities among participating agencies is necessary. Some capacity for results-based M&E was developed under LGDP, and at closing, individual ministries had adopted some of these lessons in designing their own M&E frameworks. However, the results produced by output verification and performance analysis were not effectively used to inform the design of future DAK transfers. It was suggested during project design that results of LGDP could have implications for how fiscal transfers were designed and implemented, but there did not appear to be a deliberate mechanism by which this could occur. . Page 28 of 62 The World Bank Local Government and Decentralization Project ( P111577 ) ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS A. RESULTS INDICATORS A.1 PDO Indicators Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion % DAK reports submitted Percentage 80.00 0.00 80.00 100.00 online through Web-Based Reporting System 31-Jul-2014 31-Jan-2011 31-Dec-2018 07-Dec-2017 Comments (achievements against targets): 100% of participating LGs submitted reports online using WBRS, exceeding the target of 80%. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion % of physical outputs Percentage 87.00 0.00 88.00 77.00 Page 29 of 62 The World Bank Local Government and Decentralization Project ( P111577 ) reported, verified, and 31-Jul-2014 31-Jan-2011 31-Dec-2017 07-Dec-2017 meeting eligibility criteria Comments (achievements against targets): The project fell slightly short of the 88% target, with 77% of physical outputs reported, verified and meeting eligibility criteria at project close. This is attributed to the short period of time for the expansion of LGDP in Phase II, as well as the shortage of funds for socialization and capacity-building among LGs. A.2 Intermediate Results Indicators Component: DAK Reimbursement and incentive Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Percentage of LGs that Percentage 54.30 60.00 60.00 5.00 receive DAK payment 1 from MoF by March 31. 30-Jun-2010 02-Sep-2014 31-Dec-2018 07-Dec-2017 Comments (achievements against targets): The project fell significantly short of achieving this target. The IOI was included as a means of showing that LGs were on track for improved reporting, as receipt of a first payment from MoF was an indication that LGs had successfully submitted a DAK utilization report for the 4th quarter of the preceding year. However, MoF policy changes in 2016 necessitated that LGs submit a DAK utilization report for the entire preceding year in order to receive their first payment tranche. This severely impacted the timeliness of reporting among LGs, leading to only a small number of LGs receiving their first DAK payment by March 31. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Page 30 of 62 The World Bank Local Government and Decentralization Project ( P111577 ) % of contracts with work Number 0.00 80.00 80.00 99.00 completed with documented physical handover by 30-Jun-2010 31-Jan-2011 31-Dec-2015 07-Dec-2017 December 31 Comments (achievements against targets): The target was exceeded at project close. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Contracts with quality of Percentage 0.00 45.00 45.00 25.00 outputs delivered according to contract by December 31 30-Jun-2010 02-Sep-2014 31-Dec-2018 07-Dec-2017 Comments (achievements against targets): The target was not achieved. Socialization and capacity-building activities could not keep pace with the ramp- up of the project in Phase II; as such, LGs were not all familiar with the technical standards required for output delivery and did not have the capacity to monitor delivery of infrastructure in line with these standards. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion % of LGs with functioning Percentage 73.00 80.00 80.00 99.60 procurement unit (ULP) 31-Jul-2014 02-Sep-2014 31-Dec-2018 07-Dec-2017 Comments (achievements against targets): The target was exceeded at project close, as functioning ULPs were also a national procurement requirement that LGs had to comply with. Page 31 of 62 The World Bank Local Government and Decentralization Project ( P111577 ) Unlinked Indicators Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Supplement of MPW Text The original MPW text Issuance of the MPW Issued Technical Guidelines on Technical Guidelines Supplement Technical Safeguard issued by MPW do not include Guidelines, include Ministerial Circular letter Safeguards issues. Safeguards. 30-Jun-2010 31-Jan-2011 30-Jun-2014 07-Dec-2017 Comments (achievements against targets): The target was achieved. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion MoF produce annual M&E Text N/A text Annual and quarterly PIU has submitted the reports for the project, mid- project report and LGDP impact analyses term evaluation report and IFRs, Annual M&E, report but not yet final evaluation report Mid-Term Evaluation, submitting the IFRs for and Final Evaluation Q2-Q4 FY 2016 and Q1 Reports are submitted FY 2017, as well as the on time. 2016 project annual report. 30-Jun-2010 31-Jan-2011 31-Dec-2018 07-Dec-2017 Page 32 of 62 The World Bank Local Government and Decentralization Project ( P111577 ) Comments (achievements against targets): The target was not fully achieved at project close as key reports were submitted late. This was attributed to impacts of PIU restructuring in 2016, including shortage of trained staff. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Maintain an issued Date 20-Dec-2010 20-Dec-2010 30-Jan-2015 27-Jan-2015 Supplement of the MPW's Technical Guidelines on 20-Dec-2010 20-Dec-2010 31-Dec-2018 27-Jan-2015 Environmental and Safeguards through MPW Circular Letter applicable to all Selected Local Governments in each FY Comments (achievements against targets): This IOI is a repeat of an earlier indicator and was included due to a system error. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion BPKP submits to PIU a Yes/No N Y Y Y quantitative and qualitative Verification Report 02-Sep-2014 02-Sep-2014 31-Dec-2018 31-May-2017 consistent with the TORs for the same as specified in the Verification Arrangement by Page 33 of 62 The World Bank Local Government and Decentralization Project ( P111577 ) May 31 Comments (achievements against targets): The target was achieved. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Issuance by BPKP of Yes/No N Y Y Y Verification Manual and socialization of BPKP regional 02-Sep-2014 02-Sep-2014 31-Dec-2018 30-Mar-2017 representatives by February 28 of each FY Comments (achievements against targets): The target was achieved. Page 34 of 62 The World Bank Local Government and Decentralization Project ( P111577 ) B. KEY OUTPUTS BY COMPONENT Objective/Outcome 1 To improve the accountability and reporting of the central government’s Specific Purpose Grants (DAK) for basic infrastructure 1. Percentage of DAK reports submitted online through Web- Based Reporting System. Outcome Indicators 2. Percentage of physical outputs reported, verified and meeting eligibility criteria. 1. Percentage of LGs that receive DAK payment 1 from MoF by March 31. 2. Supplement of MPW Technical Guideline on Safeguard issued by MPW Ministerial Circular letter. 3. MoF produce annual M&E reports for the project, mid-term evaluation report and final evaluation report. 4. Percentage of contracts with work completed and documented physical handover by December 31. Intermediate Results Indicators 5. Contracts with quality of outputs delivered according to contract by December 31. 6. Percentage of LGs with functioning procurement unit (ULP). 7. BPKP submits to PIU a quantitative and qualitative Verification Report consistent with the TORs for the same as specified in the verification arrangement by May 31. 8. Issuance by BPKP of Verification Manual and socialization of BPKP regional representatives by February 28 of each FY. Key Outputs by Component 1. Reimbursement for DAK infrastructure that has undergone (linked to the achievement of the Objective/Outcome 1) output verification and meets eligibility criteria (Component 1). Page 35 of 62 The World Bank Local Government and Decentralization Project ( P111577 ) 2. Implementation of a Web-Based Reporting System for LGs to submit financial and technical information on DAK infrastructure (Component 2). 3. Capacity built among LGs for use of WBRS (Component 2). 4. Capacity built among LGs in terms of MPWH technical guidelines, environmental and social safeguards, and procurement (Component 2) 5. Implementation of an output verification process for DAK infrastructure sub-sectors (Component 3). 6. Capacity built among BPKP auditors to carry out output verification (Component 3). Page 36 of 62 The World Bank Local Government and Decentralization Project ( P111577 ) ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION A. TASK TEAM MEMBERS Name Role Preparation Peter Ellis Task Team Leader Rizal A. Rivai Procurement Specialist Novira Kusdarti Financial Management Specialist Indira Dharmapatni Social Safeguards Specialist Andrew Sembel Environmental Specialist Gregorius Daan Pattinasarany Senior Economist Andri Wibisono Infrastructure Specialist Thalyta Ernandya Yuwono Economist Marcus John Jin Sarn Lee Economist Adri Asmoro Laksono Poesoro Economist Griya Rufianne Operations Analyst Rentanida Simatupang Municipal Finance Specialist Ira Marina Team Assistant Supervision/ICR Thalyta Ernandya Yuwono, Marcus John Jin Sarn Lee Task Team Leader(s) Achmad Zacky Wasaraka Procurement Specialist(s) Novira Kusdarti Financial Management Specialist Ahsan Ali Team Member Indira Dharmapatni Social Safeguards Specialist Dea Widyastuty Team Member Bastian Zaini Team Member Page 37 of 62 The World Bank Local Government and Decentralization Project ( P111577 ) Ihsan Haerudin Team Member Tatong Permana Anggrimulja Team Member Griya Rufianne Team Member Rebekka Hutabarat Team Member Achmad Affandi Nasution Team Member Adri Asmoro Laksono Poesoro Team Member Eduardi Prahara Team Member Krisnan Pitradjaja Isomartana Environmental Safeguards Specialist Aswin Arif Hidayat Team Member Sarah Xinyuan Lin ICR Author B. STAFF TIME AND COST Staff Time and Cost Stage of Project Cycle No. of staff weeks US$ (including travel and consultant costs) Preparation FY09 8.025 35,578.16 FY10 31.601 172,152.63 FY11 1.600 2,861.44 Total 41.23 210,592.23 Supervision/ICR FY10 0 12,429.53 FY11 26.621 105,662.05 FY12 25.435 77,093.30 FY13 21.025 104,395.32 FY14 32.131 134,590.70 FY15 80.936 739,776.05 Page 38 of 62 The World Bank Local Government and Decentralization Project ( P111577 ) FY16 89.028 286,792.87 FY17 70.885 509,563.85 FY18 19.095 217,763.42 Total 365.16 2,188,067.09 Page 39 of 62 The World Bank Local Government and Decentralization Project ( P111577 ) ANNEX 3. PROJECT COST BY COMPONENT LGDP Phase 1 Amount at Approval Actual at Project Percentage of Components (US$M) Closing (US$M) Approval DAK Reimbursement 220.00 219.77 99.9% Institutional Support 8.50 1.10 13.0% Program Verification of Outputs 4.50 0.2 4% Total 233.00 221.07 94.8% LGDP AF Amount at Approval Actual at Project Percentage of Components (US$M) Closing (US$M) Approval DAK Reimbursement 500.00 500.00 100% Institutional Support 10.0 6.9 70.0% Program Verification of Outputs 10.0 0.3 3% Total 520.00 507.2 97.5% Page 40 of 62 The World Bank Local Government and Decentralization Project ( P111577 ) ANNEX 4. EFFICIENCY ANALYSIS A. Background The objective of the project was to improve the accountability and reporting of the central government’s Specific Purpose Grants (DAK) for the infrastructure sub-sector. The project gradually expanded from five pilot provinces (i.e. Jambi, East Java, Central Kalimantan, West Sulawesi, and North Maluku) during 2011–2014 to 22 pilot provinces6 gradually throughout 2015–2016. In the end, the project covered 267 districts/municipalities across those 22 provinces. Based on independent verification of physical outputs, Component 1 of the Project reimbursed up to the full DAK expenditure allocations in three infrastructure sub-sectors (roads, water and sanitation, and irrigation). Working with existing mechanisms for the transfer of DAK resources, Component 2 of the Project was to strengthen institutional capacities at both the national and sub-national levels, by improving technical guidelines on DAK expenditures from line ministries, and enhance existing systems and mechanism for monitoring and verifying DAK implementation. Component 3 was designed to strengthen BPKP’s capacity to conduct verification of outputs. B. Economic Analysis The ex-post economic analysis follows the same methodology that was used at appraisal stages for both LGDP and the AF. The economic analysis was based on data collected from several sources including the Evaluation of Economic Feasibility and Cost Effectiveness of ILGR Infrastructure Investment study (2009); SUSENAS (National Socio-Economic Survey) (2016), Technical Data for DAK roads, Irrigation, Drinking Water and Sanitation, MPWH (2010 – 2017), RUCs for Roads, Bridges, Irrigation, and Drinking Water (2012), and the World Bank data such as IFR for Component 2 and 3 disbursements. The exchange rate used throughout the analysis is IDR 13,500. The discount rate is set at 12 percent, same as the original analysis. Costs. The cost estimates under Component 1 are based on the actual DAK allocations by province and sub- sector that were reimbursed over six years of implementation (i.e. 2011 to 2016). In order to analyze the full economic costs and benefits, annual operating costs for the infrastructure resulting from DAK investments under this Project were included in the analysis, expressed as a percentage of the DAK investment outlays (Roads: 5%; Water: 2%; Irrigation: 5%). It was assumed that Project investments would have a service life of at least three years. For this analysis, additional costs arising from Components 2 and 3 over the six years of implementation 66Jambi, East Java, Central Kalimantan, West Sulawesi, North Maluku, Bengkulu, Lampung, West Java, Central Java, West Kalimantan, North Sulawesi, South Sulawesi, NTT, NTB, West Sumatera, South Sumatera, Bangka Belitung, Bali, South Kalimantan, Central Sulawesi, Gorontalo, and Maluku. Page 41 of 62 The World Bank Local Government and Decentralization Project ( P111577 ) are also considered. Table 1 breaks down the costs of Component 1 by province and three sub-sectors used for the analytical model. Table 1: Estimated project costs (USD) Project Cost Region Irrigation Roads Water & Sanitation Bali 2,800,339.50 4,911,807.50 1,363,111.75 Bangka Belitung 1,498,391.50 4,892,619.00 1,249,898.88 Bengkulu 2,617,328.00 7,725,099.00 1,744,005.88 Gorontalo 1,718,196.50 5,736,776.50 1,055,484.25 Jambi 2,112,907.00 7,742,129.00 1,736,570.25 West Java 6,283,794.00 12,222,672.00 5,169,175.00 Central Java 12,102,504.00 20,817,016.00 7,898,516.50 East Java 10,084,100.00 19,503,794.00 7,629,156.00 West Kalimantan 4,288,177.00 14,436,560.00 2,906,116.75 South Kalimantan 2,718,787.75 7,858,940.00 1,682,842.13 Central Kalimantan 4,239,006.50 12,387,455.00 1,895,116.25 Lampung 3,765,767.00 12,367,546.00 2,660,982.00 Maluku 2,387,064.50 9,052,712.00 1,926,297.75 North Maluku 2,576,312.25 9,219,653.00 1,517,820.00 NTB 4,235,408.50 7,970,473.50 2,293,040.25 NTT 6,738,324.50 18,020,754.00 3,903,925.00 West Sulawesi 2,194,219.00 5,163,658.00 942,401.19 South Sulawesi 6,881,519.50 18,595,694.00 3,910,161.50 Central Sulawesi 3,752,335.00 10,521,841.00 1,961,556.88 North Sulawesi 3,002,299.75 10,675,110.00 2,697,680.25 West Sumatera 6,009,156.50 15,115,357.00 3,867,466.75 South Sumatera 2,581,109.50 11,919,012.00 2,954,808.00 Total 94,587,047.75 246,856,678.50 62,966,133.19 Overview of Project Benefits. The analysis estimates economic benefits that are readily quantifiable in monetary terms– such as from transport cost savings from reduced fuel consumption and maintenance costs and cost savings for purchasing clean water from the tap rather than vendors. This is not an exhaustive list of potential benefits arising from the project since this project helped to strengthen government institutions and mechanisms for managing DAK funding and implementation, substantial positive externalities are likely such as in the effective use of DAK in other sectors, and potentially in the management of other categories of fiscal transfers to sub-national governments. Thus, the total benefits of the project are likely to be greater than those quantified here. The general types of direct benefits from the Project are described by project component below. Specific benefits may vary from province to province and within a province, depending on the unique circumstances of, and specific investments in each location. Page 42 of 62 The World Bank Local Government and Decentralization Project ( P111577 ) Component 1: DAK Reimbursement No Roads Water Supply & Sanitation Irrigation 1 Transport cost savings from Time savings in obtaining clean Increase in crop yields (per unit reduced fuel consumption water; area); and maintenance costs; 2 Time savings from smoother Cost savings for purchase of clean Increase in area under road travel; water; cultivation; 3 Gains from better market Lower health expenditures from Cultivation of higher value crops access for goods; reduced incidence of water-borne and higher yielding crop disease and lower income varieties; foregone from illnesses. 4 Increased income of Increase in farmer incomes (from transport operators arising increased productivity factors from greater demand for above); transport services; 5 Enhanced capital values of Reduced risks of, and costs property made more arising from, extreme events accessible by improved such as droughts or floods; roads. 6 Enhanced capital values of agricultural land with improved irrigation. Component 2: Institutional Strengthening for Local and Central Governments and Project Management Support • Improved rate of completion and quality of outputs obtained from infrastructure investments funded by DAK; • Reduced inefficiencies and irregularities in use of DAK funds; and • Greater accountability and transparency in the use of DAK funds. Component 3: Verification of Outputs • This component contributes to the achievement of results in Components 1 and 2 above. Valuation of Project Benefits. The first step in estimating Project benefits was to estimate the quantity of outputs from DAK investments under this Project in each sub-sector and province. Project benefits are estimated using detailed data collected from participating LGs in the three pilot provinces (i.e. Sumatera, Java and Sulawesi) for ILGR (Initiatives for Local Governance Reform) project. Benchmark figures from these local governments added to the relevant data from SUSENAS are then used to estimate benefits for the remaining 267 participating local governments. Given the complexity of estimating future DAK investments in all districts across 22 provinces, representative output types were chosen for each of the sub-sectors, based on outputs most commonly funded by DAK investments in these provinces: Page 43 of 62 The World Bank Local Government and Decentralization Project ( P111577 ) • Roads: kilometers of roads upgraded to a standard width of 4 meters; • Water supply and sanitation: number of houses connected to a clean water supply with a 2-inch diameter pipe connection; • Irrigation: hectares of cropland irrigated as a result of upgrading of irrigation infrastructure. Unit costs for each output type in each province were obtained, based on RUC data from MPW. The values of DAK investments were then divided by the unit costs, to obtain the quantity of representative outputs that notionally would be delivered by these DAK investments in each province and sub-sector. The economic analysis was carried out using a cost-benefit method to estimate the net benefit to society arising from the project. This net benefit was calculated as the difference between incremental benefits and incremental cost as shown in figure below. Figure 1: C-B Analysis Framework for LGDP Furthermore, Table 2 below summarizes the unit costs obtained7, calculated output quantities, and estimated unit benefits, for the representative output types in each province. 7 Unit: road (Km), water (household), irrigation (ha) Page 44 of 62 The World Bank Local Government and Decentralization Project ( P111577 ) Table 2: Unit costs and unit benefits for representative outputs (USD) Unit Benefit Unit Cost Region Irrigation Roads Water & Sanitation Irrigation Roads Water & Sanitation Bali 41.95 14,046.69 320.85 181.78 60,871.21 300.10 Bangka Belitung 81.29 36,279.07 357.50 175.27 78,219.70 327.07 Bengkulu 96.48 32,196.97 366.91 316.96 84,052.20 303.22 Gorontalo 132.56 22,843.04 366.10 629.55 72,045.45 492.52 Jambi 96.59 32,196.97 366.91 189.39 69,418.47 335.68 West Java 61.36 28,906.79 366.91 416.67 125,267.35 180.86 Central Java 61.36 40,361.19 366.91 378.79 174,904.92 343.18 East Java 61.36 17,040.17 366.91 170.45 73,843.45 365.61 West Kalimantan 94.70 43,560.61 366.91 189.10 124,241.74 474.75 South Kalimantan 67.64 50,594.03 348.47 189.39 75,012.80 384.95 Central Kalimantan 94.70 43,560.61 366.91 265.15 87,124.59 365.61 Lampung 96.48 32,196.97 366.91 303.03 68,181.82 332.29 Maluku 150.17 58,186.59 390.63 936.36 69,090.91 472.56 North Maluku 185.61 43,560.61 366.91 757.58 92,288.99 503.11 NTB 120.01 43,560.61 38.22 695.40 137,387.27 386.25 NTT 185.61 56,818.18 354.42 681.82 116,382.58 520.83 West Sulawesi 109.85 55,456.35 366.91 568.18 86,614.38 503.11 South Sulawesi 109.85 55,456.35 392.98 591.10 157,033.78 548.48 Central Sulawesi 91.72 46,068.89 289.11 474.39 71,952.59 380.47 North Sulawesi 109.85 55,456.35 402.35 591.10 157,033.78 548.48 West Sumatera 47.98 36,860.54 282.29 103.45 79,473.38 258.26 South Sumatera 120.78 57,238.16 371.77 260.42 123,408.67 340.13 Cost-Benefit Analysis The net present value (NPV) of the Project investments were then calculated for the Project as a whole, over a period of six years (2011-2016) combining the time length of both initial and the AF, again using a discount rate of 12 percent. For illustrative purposes, the internal rate of return (based on net economic benefits) was also calculated, to show the discount rate at which net present value would be zero. It should be noted that the cost and benefit figures for each province do not take into account the national-level investments for Components 2 and 3 of the Project; these are, however, included in the analysis and results shown for the Project overall. The results of the cost-benefit analysis are summarized in Table 3 below. The Project NPV over five years, at a discount rate of 12%, is estimated at US$544.2 million, with an economic internal rate of return (EIRR) of 45 percent. The EIRR figure is greater than ex ante analysis for the first LGDP and AF LGDP which stood at 40 percent and 30 percent, respectively. The economic re-evaluation thus confirms and exceeds the project benefits anticipated at appraisal. Furthermore, in addition to this economic analysis, which shows these DAK investments to be economically sound, a benefit arising specifically from this project is the improved accountability and reporting in the utilization Page 45 of 62 The World Bank Local Government and Decentralization Project ( P111577 ) of DAK funds. While it is relatively difficult to quantify the value of this benefit ex-post, the positive impact of institutional strengthening in the use of DAK resources is expected to be substantial. Table 3: Benefits, costs and net present value for Project investments (USD Million) LGDP Benefit Cost LGDP AF Benefit Cost ICR Benefit Cost Total 387.9 267.7 Total 756.9 476.2 Total 1,915 1,093 NPV 56 NPV 89.1 NPV 544.2 EIRR 40% EIRR 30% EIRR 45% Sensitivity Analysis Sensitivity analysis of the project was done with respect to two variables: (i) a change in the average unit costs; and (ii) a change in the estimated average unit benefits. These variables were selected for their vulnerability to unforeseen variance across local governments. Results of the sensitivity analysis are shown in Table 4 below, summarized in terms of the percent change in each variable required to switch the NPV from positive to negative or vice-versa—the “switching value”. All else equal, the project could sustain a reduction in the proportion of estimated average unit benefits of 36 percent in each year of the projection and still be economically feasible. Similarly, it would take an increase of 24.5 percent in average unit costs to reach a negative ENPV. The results of two scenarios reveal that the overall net benefits are barely sensitive to these key variables. Table 4: Sensitivity Analysis LGDP Additional All four sub-sectors LGDP I (PAD) ICR Financing (PAD) Average unit costs +24.5% +27.7% +57% Estimated average unit benefits −20% -11.6 % -36% In addition, a descriptive analysis comparing actual (average) unit costs for road rehabilitation (betterment from soil to asphalt) per km in five initial provinces for both LGDP and non-LGDP, or the likely situation without the project, was also conducted. The analysis shows that actual unit costs under LGDP for road maintenance are relatively lower than costs of others in Jambi, Central Kalimantan, East Java and North Maluku. However, the actual unit cost for LGDP in West Sulawesi is a bit higher than that of non-LGDP although the gap is relatively small.8 8 Actual unit cost for LGDP stands at IDR2.045 million (per km); whereas, for non-LGDP is at IDR1.8 million (per km) in West Sulawesi. Page 46 of 62 The World Bank Local Government and Decentralization Project ( P111577 ) Figure 2: Average Unit Cost in Five Initial LGDP Provinces: Road Sector 5,000.00 4,000.00 IDR, Thousands 3,000.00 2,000.00 1,000.00 - Jambi Kalimantan Jawa Timur Sulawesi Barat Maluku Utara Tengah Actual Unit Cost (LGDP), Average Actual Unit Cost (Non-LGDP), Average Source: BPKP contract data (2016) and MPWH e-monitoring (adjusted) (2017) Page 47 of 62 The World Bank Local Government and Decentralization Project ( P111577 ) ANNEX 5. BORROWER, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS Summary of Borrower’s Implementation Completion Report Context Country Summary Decentralization. With over 13,000 islands and spread out over 5,000 kilometers, decentralization is more suitable for a geographically dispersed nation as Indonesia. However, it was not until 2001, that Indonesia went from being one of the most centralized countries in the world in administrative, fiscal, and political terms, to one of the most decentralized. The implementation of Law 22/1999 on Regional Autonomy and Law 25/1999 on Fiscal Balance marked the beginning of this reformation. Both laws have since been amended multiple times, most recently known as Law 23/2014 and Law 33/2004, respectively. Specific Purpose Grants (DAK). Law 33/2004 article 10 specified the various types of Equalization Fund, which consists of Revenue Sharing (DBH), General Purpose Grants (DAU), and Specific Purpose Grants (DAK). The DAK is a grant funded by the national budget (APBN) and allocated to particular LG with the purpose of financing specified activities that are aligned to national priorities and conducted under the jurisdiction of LGs. Over the years, the DAK allocation has been growing very rapidly, increasing from IDR 2.2 billion in 2003 to IDR 35.8 billion in 2015. Similarly, the number of sectors funded by DAK grants increases each year; in 2015, 14 sectors are eligible for DAK, an increase from just 5 sectors in 2003. Stakeholder Ministry of Finance (MoF). In the original Project Appraisal Document, MoF was the only Project Implementation Unit (PIU), precisely under the Directorate Generale of Fiscal Balance (DGFB). Later in the Additional Financing, the PIU was expanded to include MoHA, Bappenas, and MPWH; while MoF remains as the Head of PIU. As the head of PIU, MoF is responsible for issuing Minutes of the quarterly PIU Management Committee Meetings, aggregating all Project reporting and monitoring reports to the Bank, and taking final decisions on all Project implementation issues in the event that consensus is not reached across the PIU Management Committee. Ministry of Public Works and Housing (MPWH). MPWH is the line ministry responsible for developing and managing physical infrastructure in Indonesia. As infrastructure was chosen as the pilot sector for LGDP, MPWH was therefore invited as the Steering Committee. Some of MPWH responsibilities are updating and issuing annual Reference Cost Unit (RUC), draft and issue relevant Ministerial Circular Letter for Supplemental Guidelines on Social and Environmental Management, and conduct socialization regarding the Supplemental Guidelines. The responsible unit for LGDP is Bureau of Budget Planning and Foreign Cooperation (BPAKLN), under Secretariat General of MPWH. Page 48 of 62 The World Bank Local Government and Decentralization Project ( P111577 ) Ministry of Home Affairs (MoHA). MoHA is the ministry responsible for formulating and implementing policies on politic and general governance, regional autonomy, and regional administration. There are two units of MoHa that are involved in LGDP; they are Directorate Generale of Regional Development (Bangda) and Directorate Generale of Regional Finance (Keuda). MoHA’s key role is to support outreach and coordination with participating Local Governments on all aspect of Project implementation. National Planning Agency (Bappenas). As policy maker for development planning, Bappenas has the authority to regulate DAK menu, decide on priority regions, and enact other policies that are influential to the project. As such, Bappenas responsibilities under LGDP includes: contribute to and support policy analysis in areas related to DAK and intergovernmental transfers, and support the continued development and operationalization of the Web-Based Reporting System. The responsible unit for LGDP is Directorate of Regional Autonomy, under Deputy of Regional Development. Government of Indonesia Internal Auditor (BPKP). BPKP is the only stakeholder that is not included in Steering Committee and PIU. BPKP’s role in LGDP is essentially to conduct Verifications of Outputs (VO), which aligns with its main function to: audit, review, monitor, evaluate, and other form of supervision towards national development and/or other activities that are funded by state/region budget and/or subsidies. The VO function is financed by BPKP own budget resource, including new staff employment and capacity enhancement. Rationale Challenges in DAK Implementation. A Bappenas study revealed that in five years since 2005, DAK management has encountered financial, technical, institutional and good governance problems. The problems can be summed up as follow: there is a mismatch between the amounts of allocation and the local needs (financial); LGs do not have clear guidance on the appropriate use of DAK or the timeline of achievements of DAK-funded projects (technical); weak or less-than-optimal institutional coordination between the central and the regions, non-existence of a coordination team in the center and province, and less-than-optimal performance of coordination teams in the districts (institutional); and the application of the principles of transparency, accountability, and participation in DAK management (good governance). Inadequate M&E Mechanism. Additionally, according to a study by the SMERU Institute, the existing monitoring and evaluation (M&E) of DAK-financed activities is still inadequate. The findings of this study corroborate Bappenas study, in which LG and NGO official gave relatively low ranking to M&E aspect of DAK. This is said due to the fact that both central and local government do not allocate any fund in their budget for M&E; and the lack of benefit of M&E for LG, since they have never received any feedbacks on the reports they made for respective line ministries. Infrastructure as Pilot Sector. Infrastructure sector was chosen as pilot sector because of several of reasons; firstly, because its relatively stable Technical Guidelines (Juknis), shown by the multiple years Juknis during 2010 to 2015. MPWH as the line ministry responsible for basic infrastructure is also Page 49 of 62 The World Bank Local Government and Decentralization Project ( P111577 ) committed to organize Technical Coordination Meeting annually with all LGs, to provide structured and clear information on how to implement and monitor DAK-funded projects. Infrastructure is currently one of the government’s top priority, with total projected spending of IDR 4.7 billion until 2019. Project Development Objective Project Development Objective (PDO). The objective is changed slightly in 2014 as a part of the additional financing. The revised PDO reflects the geographical scaled-up of the project, from initially 5 pilot provinces to 30 of total 34 provinces, excluding provinces with special autonomy status (Aceh, Papua, and West Papua) and the Special Capital District (DKI) of Jakarta. The new PDO also specify the types of infrastructures included in the project. The change is illustrated below: Original PDO Revised PDO The objective of this Project is to improve The objective of the Project is to improve the accountability and reporting of the the accountability and reporting of the central government’s Specific Purpose Grants central government’s Specific Purpose Grants (DAK) for the infrastructure sub-sectors (DAK) for basic infrastructure, consisting of within pilot local governments (LGs) roads, irrigation, water and sanitation, within Selected Local Governments. Achievement of Project Indicators Achievement of PDO, which is to improve the accountability and reporting of the DAK for basic infrastructure, was rated Satisfactory throughout project implementation, except in 2012, 2016, and 2017. The 2012 lower rating was due to the disbursement lag and the issues found in the quality of implementation. In 2016, the reduced TA and socialization led to a decline in the percent of physical outputs reported, verified, and meeting eligibility criteria. Fortunately, according to data obtained in 2016 by MPWH, 100% of LGs that received DAK infrastructure allocations submitted their DAK technical reports to MPWH through the e-monitoring system, exceeding the end target of 80%. For component 1 of Disbursement indicator, the decline in 2014 was caused by stronger internal control in government system. Starting in 2014, MoF requested all LGs to submit the previous year’s budget accountability report to receive the 1st DAK transfer. Only 76% of total participating districts have received their first DAK payment from MoF by 15 December 2014. Achievement Outside of PDO Indicators Institutionalizing DAK reporting system. The GOI developed and implemented their own web-based reporting system, which spanned to broader DAK projects; including non-LGDP participants and non- Infrastructure Sectors. The MPWH introduced e-Monitoring for DAK infrastructure reporting and the MOF introduced OM-SPAN for reporting on fiscal transfer. Page 50 of 62 The World Bank Local Government and Decentralization Project ( P111577 ) Standardization of Monitoring & Evaluation. M&E is officially standardized in 2016 through issuance of PMK 112/PMK.07/2016 on Implementation of Monitoring and Evaluation of Predetermined Transfer to Local Government. LGDP’s series of M&E workshop for PIU are one of the catalysts for this standardization, as indicated through the PMK annex, which includes LGDP’s M&E Standard Instrument. Relevance of the Project Policy Improvement. The project started as a pilot in 5 provinces. The project introduced a performance- based incentive based on achievement of DAK outputs. However, there is no mechanism for providing sanction for non-performing LGs. Results from BPKP report provides input to the GoI for DAK policy improvement and to make DAK become a more performance-based transfer. Institutional Strengthening for Local Governments. Although institutional strengthening for Local Government is already within Central Government’s main responsibilities, the quality of implementation was a challenge. Through LGDP, the GOI requested for support from the Bank to design a more targeted and focused TA to local governments. Feedback Mechanism. With BPKP’s verification of outputs and introduction of complaints handling mechanism in DG Fiscal Balance’s website, LGDP has introduced methods to measure the degree of DAK achievements and method for providing feedback from LGs to MOF and from BPKP to LGs. Project Impacts DAK Incentive to Maintain LG’s Performance. Local government incentive mechanism in LGDP, has introduced a reward mechanism for good performance local governments. This is an innovative feature of LGDP, considering that since decentralization took place in 2001, Central Government no longer has direct command line towards Local Government. Output-based Mechanism to encourage good performance. Correlated to the use of incentive, output verification has motivated LGs to perform better, by conducting their projects in compliance to the eligibility criteria. This mechanism has also served as a reassurance for LGs that their performance is within their control, because the criteria that determine their results are foreseeable. This output-based mechanism within LGs budget could be considered as idea for future policy. Web-Based Reporting System to Improve DAK Reporting. LGDP introduced a web-based reporting system to improve LG reporting for both financial and technical reporting. The modules consist of pictures, GPS coordinate for the location of the DAK project, amount of DAK received and amount of contract, progress and annual DAK reporting. As part of the Project, the MoF issued Ministerial Decree to enforce participating LGs to submit their DAK reports using the Web-Based Reporting System (WBRS). Since 2014, the MoF added a new requirement to the Project, which stated that if LGs don’t submit their reporting using WBRS, the LG incentive would be reduced by a certain percentage. Targeted Capacity Building. As a result of the characterization, Central Government will then be able to design specific capacity building material for every Local Governments based on their strengths and weaknesses, as opposed to the current generic capacity building. Page 51 of 62 The World Bank Local Government and Decentralization Project ( P111577 ) Performance as Criteria for DAK Allocation. Besides strength and weakness characterization and targeted capacity building, LG’s DAK performance can also be utilized as one of the criteria for next year’s DAK allocation. Current DAK practice has implemented entry point screening by enacting the proposal-based allocation starting in 2015; whereas the exit point screening could be executed by this performance-based mechanism. Key Factor & Event Issues that Affected the Project Removal of 10% Matching Fund. In Law 33 Year 2004 concerning Fiscal Balance Between the Central Government and The Local Governments, Local Governments were required to provide “Matching Fund” amounted to minimum 10% of its DAK allocation. With increasing DAK allocation every year, 10% Matching Fund becoming more of a burden for LG. Eventually in 2015, the Matching Fund requirement was repealed; and this policy, together with the newly enacted proposal-based system, have made the DAK transfer more effective. Significant Increase in the number of LG participants. From the initial 5 provinces in 2010 to almost nationwide by 2017, the number of LGDP participants has increased significantly, which affect the quality and quantity of knowledge dissemination and supervision from Central Government, as evidenced by the poorer performance of the newly participated LGs. In the future, project scale up from pilot to nationwide implementation should be supported by adequate strategy to prevent the performance from declining. Transformation of DAK Menu. DAK policy is gradually improved with time, to cope with arising problems and to anticipate future concerns. Similarly, DAK menu was continuously changed to answer the needs of cities and regions. One of the example is the addition of “new construction” menu on DAK road infrastructure 2017 Technical Guidelines. While this addition created greater option for LG to utilize their DAK allocation; in relation to the LGDP, it also meant new kind of output verification that is needed to be done by BPKP. Revised Procurement Arrangements. The Additional Financing in 2015 included an increase in the threshold amounts for eligible contract, from maximum US$ 1,000,000 to US$ 2,000,000, making room for more eligible projects. The additional financing also expanded the menu of eligible procurement procedures to include community participation in procurement through simplified competitive procurement, in order to adjust with sanitation sectors technical guidelines. Proposal-Based DAK Mechanism. Proposal-based mechanism is first implemented in 2015 as a deciding factor for DAK allocation. Before this requirement, exact DAK allocation for each Local Governments is based on the accumulation of these three criteria: General Criteria, Specific Criteria, and Technical Criteria. General Criteria is LG fiscal capacity; while Specific Criteria is Regional Index determined by MoF with Bappenas input; and Technical Criteria is the indicator of priorities program, based on the technical index from each line ministries. Proposal submission as additional criteria has some positive and negative Page 52 of 62 The World Bank Local Government and Decentralization Project ( P111577 ) outcome. On the bright side, the Local Governments are forced to forge plan on how to disburse DAK allocation. However, on the negative side, proposal submission could corrupt DAK allocation mechanism by providing more room for negotiation between LG and policymaker; which may result in greater or smaller allocation in certain LGs. This imbalance in allocation could then affect the LG’s performance, which then will be shown through verification in LGDP. Monitoring and Evaluation Monitoring and Evaluation for DAK. Monitoring and Evaluation of DAK is a mandatory for Central Government, as outlined in PP 55/2005 about Fiscal Transfer and PP 43/2013 about Procedure of APBN Implementation. Additionally, the enactment of PMK 112/PMK.07/2016 serves as the standardization of Monitoring and Evaluation of DAK and other kind of fiscal transfers. Within this regulation, the M&E mechanism for fiscal transfer is: (i) designed as performance-based; (ii) Integrated between various stakeholders; (iii) measured by established indicators; and (iv)collected through reliable system information with nationwide coverage. However, integration between various stakeholders has not yet been decided, thus hindering this law implementation. Deliberation between various stakeholders is necessary to divide the role for this impact evaluation; based on each ministry/agency function and deliverables. Monitoring and Evaluation for LGDP. According to PIU, Monitoring and Evaluation for LGDP is relatively good, considering annual M&E report as one of the key deliverables of PIU. The main difference of M&E between DAK and LGDP is that there has not been any M&E on accountability and reporting in DAK, whereas in LGDP it is one of the main indicators. Moreover, existing DAK M&E only serves as troubleshooter for LG problems in DAK, without providing information on physical and financial progress of fiscal transfer. Fiduciary and Safeguards Procurement. The procurement for LGDP is fully funded by the government and has followed Perpres 54/2010 on Government Goods and Services Procurement and its following amendments: Perpres 35/2011; Perpres 70/2012; Perpres 172/2014; and Perpres 4/2015. Financial Management. Aside from Interim Financial Report (IFR) submission to the World Bank, there is no specific arrangement of financial management for LGDP. Furthermore, submission of IFR did not affect or burden the PIU as much; because the data required in the IFR is well within PIU’s knowledge. Environmental and Social Safeguards. Environmental and Social Safeguards in Indonesia is regulated through Law 32 Year 2009 about Environmental Protection and Safeguard, and its subsequent regulations. While this law has specified which document to prepare in order to control the environmental impact, it did not explain how to control the impact itself. Alas, to execute Component 2 of LGDP, The GoI is required to supplement Guidelines on Environmental and Social Safeguards, which realized through MPWH Circular Letter. Overall, GoI already has adequate regulation on safeguards, and this circular letter only acted as an emphasis. Furthermore, from PIU’s point of view, the implementation of this supplemental guidelines Page 53 of 62 The World Bank Local Government and Decentralization Project ( P111577 ) is still unsatisfactory; and in the future, they prefer to use existing GoI regulation instead of producing additional guidelines. Implementing Agency Performance Coordination Besides the division of responsibilities among PIU, there are certain deliverables that required all PIU participation, such as: Semi-Annual Management Committee Meeting; Annual Steering Committee Meeting; Semi-Annual Project Report; Mid-Term Project Evaluation Report; and End-of-Project Report. This coordination, while led by MoF, required attention and input from every PIU; which is why “Coordination” one of the deciding factor to measure the Implementing Agency performance. The joint - report submission achievement is outlined in Chapter 4, ‘Assessment of Intermediate Outcome’, under Component 2: Institutional Strengthening and Project Management Support; the PIU performed quite well by submitting the reports fairly on time. Meanwhile, the joint PIU meeting was not held as frequent as it supposed to be, due to the conflicting schedule between different agencies. Substance The Government of Indonesia is relatively well prepared for LGDP, having learned from multiple years of DAK implementation. Some of the innovations that are introduced through LGDP are: output verification, online monitoring and reporting system, and data-driven policymaking. These innovations are gradually being institutionalized within the government institutions, most prominently the implementation of PIU’s own web-based reporting system. Commitment According to the PIU, each of them has executed their responsibilities as stated in the agreed terms of reference. Here are the most notable achievements, according to each PIU: • MoF: improved accountability and reporting; better M&E mechanism through web-based reporting system; standardized M&E through PMK 112/PMK.07/2016. • MPWH: verification data as feedback of policymaking; targeted capacity building program. • MoHA: characterization of Local Governments; providing appropriate recommendation; increasing Local Government participation. • Bappenas: increasing Local Governments’ commitment; more measurable national priorities achievement. • BPKP: improved human resource capacity in physical output verification; disciplined Local Governments; conduct Government Internal Control System (SPIP). World Bank Performance Throughout LGDP implementation, the Bank has continuously conduct capacity building program, with emphasis on Monitoring and Evaluation (M&E) workshop. The lesson was deemed detailed and insightful, but there are rarely follow-ups. The M&E practice has also been discussed in FGD involving all PIU, but it has not been implemented and only realized through PMK 112/2016, with role sharing between PIU not Page 54 of 62 The World Bank Local Government and Decentralization Project ( P111577 ) yet decided. The Bank has provided support to the design of Technical Assistance to help PIU deliver their key responsibilities. Overall, in terms of institutional strengthening, the Bank has performed adequately well. Supervision by The World Bank for LGDP is deemed quite well by the PIU, although it could be improved to be more comprehensive. Lesson Learned Web-Based Reporting System. Before LGDP, DAK reporting is done manually through printed document and/or field visit. With web-based system, the reporting could be done in real time, which simultaneously increase both the accuracy and the efficiency in reporting. Data-driven Policymaking. In accordance to good policymaking practice, DAK policy is regularly iterated, in order to adapt with current circumstances. For that reason, the availability of accurate, up-to-date data is very important to make sure that the policy can respond to existing problems. This is where output verification comes in handy, because it manages to provide data for policy iteration in DAK, whether for nationwide policy, or simply as a basis for capacity building program. Output Verification as DAK Performance Motivator. With increasing DAK allocation each year, it is important to develop a mechanism to ensure that DAK implementation is accountable. Output verification serves as one of the solution. By sampling reported outputs and compiling the outputs that meet the eligibility criteria, the Central Government can figure out where the DAK transfer had gone. Not only that, output verification has also became a motivator to boost LG performance, because the amount of reimbursement depends directly on the percentage of the qualified output. However, because the current reimbursement scheme only imposes upon the Central Government, it is still debatable whether output verification truly motivated LG. Impact Evaluation conducted by the World Bank also shows that incentive from eligible output only matters significantly to low-income regions. For future implementation, reimbursement scheme that charges directly to Local Governments can be considered. Project vs Program Mindset. Beyond LGDP, the PIU hope that for similar project in the future, the objective is not only limited to project execution, but also on project selection. Executing the project well is important but selecting the right group of projects is more critical in the long run, as it determines the course of development and how the goal is going to be achieved. Working Systematically. This lesson is obtained from directly working together with The World Bank. This cooperation became an example for GoI on how to plan and execute project properly, with explicit targets and strategy on how to achieve them. Sustainability Institutionalizing Verification. The GoI is planning on institutionalizing verification into DAK operation. There are some assessments and policy discussions that needs to take place prior to having verification of outputs included as part of DAK transfer process, which includes: (i) redefinition of output verification Page 55 of 62 The World Bank Local Government and Decentralization Project ( P111577 ) and its implication towards existing policies and allocations; (ii) hold meeting with related institutions to decide on verification terms and position; (iii) prepare the national regulation and standard for output verification; (iv) decide on responsible ministry/agency as auditor. Enhance Reporting System. One of the most signficatnt impacts of LGDP is the institutionalization of web- based reporting system, for example the OM SPAN in MoF and e-Monitoring in MPWH. These web-based systems are far better improvement from the manual reporting system before LGDP, however, it does not necessarily mean that there are no more rooms for innovation. Some ideas for improvement include: GPS location of project; project plan; contract document; output data; photo; et cetera. Of course, before these improvements are finalized, they need to be assessed exhaustively; and afterwards, it is mandatory to conduct dissemination for local officers to ensure the system effectiveness. Set Standard Unit Cost for All Sector. By determining the standard Unit Cost of infrastructure during LGDP, the GoI is guaranteed the value for money of DAK infrastructure in participating regions. Ensuring value for money of fiscal transfer is one of the main functions of MoF, therefore ensuring its continuity is one of the top priorities of LGDP sustainability plan. This practice has to be replicated in every sector, nationwide: firstly, to ensure the value for money; secondly to limit the room for mark-ups and manipulations. Strategy for Nationwide Scale Up. Expanding the project from five provinces in LGDP phase I to thirty provinces in LGDP II needed a well-planed transition. One of the major factors for declining performance was the consequence of limited socialization and untargeted capacity building from Central Government. In the future, strategy for scaling up should be predetermined. Additional Resource for Daily Project Management. Even though LGDP posed only as a slight addition to the existing DAK mechanism, it was still tricky for the Borrower to handle its day-to-day management, because LGDP deliverables were beyond the Borrower’s main obligations. Therefore, the responsibilities given by LGDP often conflicted with Borrower’s existing functions; and as expected, they prioritize their designated function first. To resolve this, hereafter the GoI should assign additional manpower to conduct this daily project management, which currently only been done by MoHA. Borrower Comments to the Draft ICR BPKP: • Suggest removing the sentence in paragraph 7: “This would address the challenge of having un- verified information reported and a lack of transparency over use of DAK funds prior to LGDP”. This was based to the fact that the use of words “lack of transparency” was not in-line in the context of improving accountability. • Disagree with the statement that the outcome was not achieved, as stated in paragraph 23, which was inconsistent with statement in paragraph 9. In para.9, it has been described that the 2 KPIs, which was the outcome indicator of LGDP: the use of Web-Based Reporting System by the Ministry Page 56 of 62 The World Bank Local Government and Decentralization Project ( P111577 ) of Finance and the eligibility criteria has been met. The WBRS was not only used by the MoF, but also used by GoI as a whole in providing incentives to LGDP participants. • Suggest removing LKPP as the institution that needs to be involved in output verification process, to address challenges identified in BPKP verification. Verification was based on the compliance to the Presidential Regulation on government procurement. BAPPENAS: • Change of DAK policy in 2015-2017 provides considerable effect to LGDP performance, particularly to the achievement of Component 1: “DAK Reimbursement and Incentives”. This could be drawn as a lesson learned (for future operation), so that the design could easily adapt to any DAK policy changes. • For component 2: “Institutional Strengthening to Central and Local Governments and Project Management Support”, an apparent challenge remains in the task division between PIUs, particularly on the partial M&E mechanism, where each institution still has their own M&E instrument. Furthermore, the provision of supporting consultant to assist the PIU in developing policies and DAK information system has not succeeded. • Result from Component 3: “Verification of Outputs” that has been developed by BPKP has a potential use as an input and basis for Bappenas, MoF, and MPWH to plan the future DAK, particularly to assess the DAK (proposal). • Specific inputs for the continuation of LGDP are as follows: (i) adapt the verification system that carried out by BPKP for DAK planning (and not only limited to DAK infrastructure); (ii) involve BPKP in the DAK planning process as verification team; (iii) Identify the future (DAK) activity menu based on the LGDP evaluation results on the best performing activity in infrastructure sectors. MOF: • There are several steps that needs to be made before the GoI could internalize the output verification mechanism within the overall DAK (and intergovernmental fiscal transfer) system: o Need to develop a detailed concept and possible mechanism for the use of output verification results to inform decision-making purpose o Need to arrange a cost-benefit analyses to compare the additional cost required to undertake output verification with the improved efficiency of DAK, particularly to illustrate the value for money of the output verification activities. • There are many DAK policy changes that was particularly informed and influenced by LGDP implementation, including: o The requirement to verify the Work Plan (Usulan Rencana Kegiatan – URK) in the DAK disbursement process. This policy started in 2017 and the verification was carried out by KPPN. Page 57 of 62 The World Bank Local Government and Decentralization Project ( P111577 ) o The requirement to use contract data as the cross-checking instrument to ensure the project accuracy, as mandated in the PMK 112/2016 on DAK management. This applies not only to DAK infrastructure, but to all DAK sectors and all LGs. Page 58 of 62 The World Bank Local Government and Decentralization Project ( P111577 ) ANNEX 6. SUPPORTING DOCUMENTS 1. Original Project Appraisal Document (PAD) 2. Original Loan Agreement (LA) 3. Original Verifiication Arrangement (VA) 4. Additional Financing Project Paper (AF-PP) 5. Additional Financing Loan Agreement (AF-LA) 6. Additional Financing Verification Arrangement (AF-VA) 7. Amended Loan Agreement(s) 8. Implementation Status Report(s) 9. Aide-Memoire(s) 10. LGDP Impact Evaluation Study by MOF 11. LGDP Performance Evaluation by MPWH 12. DAK Reporting Performance Analysis by MoHA 13. LGDP Impact Evaluation by the Bank Page 59 of 62 The World Bank Local Government and Decentralization Project ( P111577 ) Figure 6: Sample View of Web-Based Reporting System Figure 7: Excerpt of Output Verification Handbook Page 60 of 62 The World Bank Local Government and Decentralization Project ( P111577 ) Page 61 of 62