Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review National Community Empowerment Program (P125405) Report Number: ICRR0021763 1. Project Data Project ID Project Name P125405 National Community Empowerment Program Country Practice Area(Lead) Indonesia Urban, Resilience and Land L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IBRD-82130,TF-12192 31-Dec-2015 281,566,854.91 Bank Approval Date Closing Date (Actual) 20-Nov-2012 31-Dec-2018 IBRD/IDA (USD) Grants (USD) Original Commitment 266,000,000.00 23,500,000.00 Revised Commitment 286,957,306.72 20,957,306.72 Actual 281,573,783.95 20,957,306.72 Prepared by Reviewed by ICR Review Coordinator Group Ranga Rajan J. W. van Holst Christopher David Nelson IEGSD (Unit 4) Krishnamani Pellekaan P131296_TBL Project ID Project Name P131296 National Community Empowerment Program ( P131296 ) L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) 0 Bank Approval Date Closing Date (Actual) Page 1 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review National Community Empowerment Program (P125405) 20-Nov-2012 IBRD/IDA (USD) Grants (USD) Original Commitment 0.00 0.00 Revised Commitment 0.00 0.00 Actual 0.00 0.00 2. Project Objectives and Components DEVOBJ_TBL a. Objectives The Project Development Objective (PDOs) as stated in the Loan Agreement (Schedule 1, page 6) and the Project Appraisal Document (PAD, page 7) were: " To support the Republic of Indonesia to ensure the urban poor in participating Kelurahans (urban wards) benefit from improved local governance and living conditions". The PAD (page 7) states that, to achieve this objective, the project will: (a) strengthen and institutionalize elected representative organizations held accountable to communities: (b) provide grants to communities directly and transparently to finance an open menu of activities aimed at poverty alleviation: (c) enhance the capacity of central and local governments to partner with community organizations in public service provision: and (d) increase awareness of disaster risk mitigation and mainstream measures of resilience. For the purpose of this review’s assessment of the extent to which the PDO was achieved in Section 4 below it will be parsed into two objectives, namely: Objective 1: To ensure that urban poor in participating kelurahans benefit from improved local governance conditions Objective 2: To ensure that urban poor in participating kelurahans benefit from improved living conditions b. Were the project objectives/key associated outcome targets revised during implementation? No c. Will a split evaluation be undertaken? No d. Components There were four components (PAD, pages 8 -9). Page 2 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review National Community Empowerment Program (P125405) 1. Capacity Building for Communities and Local Governments. The estimated cost at appraisal was US$85.1 million. The actual cost was US$79.6 million. This component aimed at providing capacity building support on community development to: • the Board of Trustees of the Badan Keswadayaan Masayarakat (BKMs) for preparing community development plans, using Kelurahan Grants for implementing sub-projects, and supporting consolidation of poverty alleviation and Community Driven Development (CDD) plans. • community groups in participating Kelurahans (community volunteers and local government officials) for planning and monitoring sub-projects. and • the Revolving Loans Fund (RLF) unit staff to support well-performing RLFs. (Under the approach adopted, revolving loans to non-performing RLFs,were to be phased out and well-performing RLFs would continue to receive grants as an incentive to maintain good loan repayment among its members). 2. Kelurahan grants. The estimated cost at appraisal was US$373.9 million. The actual cost was US$544.0 million. This component co-financed Kelurahan grants for two types of activities. • sub-projects identified in the community development plans for: (i) prioritized infrastructure investments: (ii) income generating activities: and (iii) social programs to the poorest and the vulnerable groups: and • piloting three interventions: (i) empowering women's group in Aceh Province (This province was targeted in view of its more conservative, male-dominated culture): (ii) spatial development programs for neighborhood development targeting the poorest neighborhoods in about 200 kelurahans: and (iii) strengthening the livelihoods of individuals in poor communities. 3. Implementation Support and Technical Assistance (TA). The estimated cost at appraisal was US$41.3 million. The actual cost was US$36.7 million. This component financed the following activities: • project management support and technical assistance at the central level, to the teams of Oversight Service Providers (OSP) in provinces and to the OSP offices in participating district governments, city coordination teams and community facilitators. • monitoring support and maintaining a complaint handling system • conducting evaluation surveys for monitoring the outcomes and impacts of the National Community Empowerment Program. 4. Contingency for Disaster Response. Given the high risk of catastrophic events, a provisional zero- dollar component was included to allow for rapid allocation of funds in the event of a national disaster. No costs were incurred for this component. e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project cost. The estimated cost at appraisal was US$500.0 million. The actual cost was US$660.3 million. The actual cost was higher in view of the increase in the scope of project activities during implementation (discussed in section five). Page 3 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review National Community Empowerment Program (P125405) Project financing. The project was financed by an IBRD loan of US$266.0 million and a US$23.5 million grant from the National Community Empowerment Program (PNPM) to support program activities in Aceh province. US$260.6 million of the IBRD loan and US$20.9 million of the grant were disbursed. The difference between the appraisal estimates and actual disbursements were due to the significant exchange rate fluctuations during implementation (During appraisal the exchange rate was US$1 = 9,550 Indonesian Rupiah (INR) and at closing US$1 was approximately equal to INR14,000). There was parallel financing for similar activities in the thirteen provinces not covered in this project from the Islamic Development Bank (PAD, paragraph 36). Borrower contribution. Counterpart funding from national, local and Community Organizations was estimated at US$210.8 at appraisal. Their actual contribution was significantly more than planned at US$370.0 million. Dates. The project approved on November 2012, became effective on December 28, 2012 and was scheduled to close on December 31, 2015. The project closed three years behind schedule on December 31, 2018. Other changes. There were four Level 2 restructurings (including three for the IBRD loan and one for the PNPM Trust Fund). The following changes made through the first restructuring on December 11, 2014: There were disbursement delays in the wake of the reorganization of the executing agencies (the Ministry of Public Works and the Ministry of Housing which were merged into a single agency - the Ministry of Public Works and Housing). The closing date was extended by a year to December 31, 2016, for completing the affected sub-projects and for financing TA for preparing a follow-on project (the National Slum Upgrading Project), that was expected to serve the same kelurahans as this project. The project was restructured on December 12, 2014, to extend the closing date for the PNPM Trust Fund by 23 months and fifteen days to December 15, 2016 for completing the ongoing activities. The closing date was extended by nine months to September 30, 2017. Savings of US$17.0 million, realized due to the exchange rate changes during implementation, were allocated for supporting the start up of the follow-on project mentioned above. Two changes were through the fourth restructuring on September 29, 2017. Following flash flooding in Bima, West Nusa Tenggara province, US$14.5 million was reallocated for reconstruction of the affected areas. This activity was financed from the project savings to provide additional support to 300,000 beneficiaries. The closing date was extended by two years to December 31, 2018 for completing the reconstruction activities. 3. Relevance of Objectives Rationale Country context. The PDOs were relevant in the country context. At appraisal, about 44 percent of Indonesia's population (233 million) lived in urban areas. Over the next four decades, the country was Page 4 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review National Community Empowerment Program (P125405) expected to add 56 million persons in its urban areas and urban poverty was expected to surpass rural poverty by 2020. Delivery of basic services remained inadequate in urban areas, with fewer than three percent of urban households having access to piped sewage connection and only 20 percent of the solid waste was being collected and transported to final disposal. Meeting the challenges associated with rapid urbanization was important to the government strategy. Government strategy. The PDOs were well-aligned with those of the Community Driven Development (CDD) articulated in the National Program for Community Empowerment Program, widely known by its Indonesian acronym PNPM Mandri, launched in August 2006. This program was established as an umbrella program for bringing together community driven poverty related initiatives. The premise of the program was that community level infrastructure will better serve community needs and do so at lower investment cost when it is planned and implemented by communities themselves. By 2008, the program was operating in all rural and urban areas. Alignment with the Bank strategy. The PDOs were well-aligned with the Bank’s assistance strategy for Indonesia. The core engagement area on community development and social protection of the Country Partnership Strategy (CPS) for 2009-2012, articulated the need for designing and expanding programs that promoted inclusive growth and CDD programs. The PDOs continued to be relevant to two of the six key areas of engagement under the Country Partnership Framework (CPF) for 2016-2020 when the project closed. Engagement area one highlighted the need for developing national infrastructure for growth and development, through supporting sectors such as domestic water supply and sanitation. Engagement area four of the CPF underscored the need for strengthening the decentralized framework for improving service delivery in urban areas through infrastructure development. The project's emphasis on gender and governance was aligned with the CPF's cross-cutting engagements. At the time of appraisal, the Bank had supported the PNPM program in urban areas through five urban program projects (UPPs) (UPP1, UPP2, UPP3, PNPM Urban 1 projects and the ongoing PNPM Urban III project). This project aimed at consolidating the PNPM urban experiences and mainstreaming innovations piloted under the ongoing PNPM Urban 1 project, including addressing universal access to clean water and sanitation. The other features incorporated at design, included empowerment of women in Aceh province, increasing the number of kelurahans and grant allocations to the poorest areas through the neighborhood development approach, oversight of revolving loan funds, and strengthening the complaints handling mechanism. Given that this project built on the experiences of the prior projects, the overall level of ambition was highly appropriate and therefore the relevance of the PDOs to the government and Bank strategy is by this review rated as high. Rating Relevance TBL Rating High 4. Achievement of Objectives (Efficacy) EFFICACY_TBL Page 5 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review National Community Empowerment Program (P125405) OBJECTIVE 1 Objective To ensure that urban poor in participating kelurahans benefit from improved local governance conditions. Rationale Theory of Change. The TOC was based on capacity building of the Board of Trustees (BKMs) in community groups for preparing community development plans and implementing sub-projects which were aimed at improving urban governance. These activities along with establishing a complaints handling mechanism were aimed at ensuring the commitment of local governments to CDD to benefit the urban poor and thereby contribute to the long-term development objective of reducing urban poverty. The causal links between the project activities, their outputs and outcomes were logical and the intended outcomes were measurable. Outputs (ICR, pages 13-14 and pages 36-37). Technical assistance was provided for building the capacity of communities and local governments and the following activities were completed as targeted. • The communities formed Boards of Trustees (BKMs) and voted in elections for the Boards. The percentage of adult population voting for the management of BKMs at the neighborhood level increased to 35.4 percent from a baseline of, exceeding the target of 30 percent. • An advanced approach to community development plan was formalized with community engagement on spatial and integrated planning in priority areas, gender empowerment and the livelihoods pilot. • Technical assistance was provided to the Kelurahans to manage grant allocations, channeling other local government poverty programs through the BKMs and to align and incorporate the community development plan into city development plans. • Program operation manual, training materials, accountability matrix for project operators and monitoring and evaluation were developed. • A complaint handling system was developed and penalties for misused funds and rewards for well- performing BKMs and community groups were instituted and publicized. The information collected by the Monitoring Information System (MIS) were publicized. 98 percent of the provincial consultants provided updated data on complaints via the MIS, exceeding the target of 90 percent. • The percentage of local governments providing cost-sharing increased to 90 percent exceeding the target of 80 percent. • The BKMs with completed annual financial audits increased from 90 percent in 2012 to 97 percent in 2018 (The audits were conducted by external auditors). • 4,000 community facilitators and consultants at the provincial level were trained (there was a zero baseline and no targets for this indicator). Outcomes. Page 6 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review National Community Empowerment Program (P125405) • The participation of the poorest and vulnerable community members in planning and decision making increased to 47.1 percent from a baseline, exceeding the target of 40 percent. (Page 8 of the ICR notes that the community leaders were trained to perform a community self-survey to identify its "poorest" and most vulnerable members. The poorest members were defined by the community itself, to ensure that the definition of the poorest took into account national standards, while also being sensitive to the local context). • The participation of women in planning and decision making increased to 41 percent from a baseline, marginally exceeding the target of 40%. • According to Annex 6 in the ICR a beneficiary survey of the urban poor was conducted by a consultant firm between April 2016 to November 2016 in fifty municipalities in ten provinces included in the project. The methodology entailed a two-stage sampling provinces by first randomly selecting provinces and then selecting Kelurahans within selected provinces. The main conclusions of the survey were: (i) 82.2 percent of beneficiaries expressed satisfaction with local governance practiced at the community level. This exceeded the target of 80 percent.(ii) 90.5 percent were satisfied with their participation in community meetings and 99 percent with BKM's accountability reports (there were no targets for these indicators): and (iii) 81 percent of beneficiaries expressed satisfaction with the complaints handling mechanism (There were no targets for this indicator). • 99.6 percent of the complaints were resolved, exceeding the target of 90% (As of December 31, 2018, the project had resolved 67,090 of the 67,349 reported complaints). This review concludes that since 82.2 percent of the urban poor in participating kelurahans were satisfied with local governance practiced at the community level they presumably also benefited from improved local governance and hence the efficacy with which Objective 1 was achieved was “substantial”. Rating Substantial OBJECTIVE 2 Objective To ensure that urban poor in participating kelurahans benefit from improved living conditions. Rationale Theory of Change. The TOC anticipated that community-based infrastructure sub-projects (such as improved tertiary roads, drainage networks, waste disposal facilities and health, social and cultural facilities) and investments to support enhanced livelihoods (such as improved opportunities for petty trading and micro-business activities such as food making, craft making and vehicle repairs) would result in improving the living conditions of the urban poor. The outcomes of these activities were likely to aid the long-term development objective of reducing urban poverty. The causal links between the project activities, outputs and outcomes were logical and the intended outcomes were measurable. Outputs (ICR, pages 14-18 and page 33). Page 7 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review National Community Empowerment Program (P125405) • 99 percent of planned infrastructure, economic and social activities sub-projects were completed each year, exceeding the 80 percent target. (The activities, included 8,405 kilometer (km) of tertiary roads and footpaths, 4,180 km of drainage networks, 43,160 units of waste disposal facilities, 7,694 educational, health, social and cultural facilities, construction of 896 markets and shops, 88,810 housing units and 119,470 sanitation facilities and rehabilitation of 35,550 potable water facilities. The project supported 19,000 groups of individuals who took revolving fund loans to support economic activities pertaining to petty trading, and micro- business activities such as craft making and services such as cloth making and vehicle repairs). • Based on infrastructure certification and spot checking by 99 percent of infrastructure works were evaluated as "good quality", exceeding the target of 70 percent. • Employment was provided for 4,000 trained community facilitators. The ICR (page 18) notes that the average employment of a facilitator was about four years, meaning that the project provided roughly 16,000 man-years in employment for facilitators. The project activities also created 28.5 million workdays for construction workers. There were no baselines or targets for these indicators. • 24.70 percent of kelurahans with revolving loan fund had a Portfolio-at-Risk (PAR) ratio (PAR is the percentage of the total loan portfolios that have either outstanding or overdue repayments) of greater than ninety days. This was short of the target of 50%. Outcomes. • 82.20 percent of the respondents in the beneficiary survey reported in Annex 6 of the ICR, expressed satisfaction with the improved services supporting improved living conditions. This exceeded the target of 80 percent. • 98.60 percent of the urban poor had improved access to infrastructure, social and economic services in the participating kelurahans, exceeding the target of 80%. • 8.7 million people in kelurahans benefited from project activities, exceeding the target of 6.0 million. 51% of the 8.7 million beneficiaries were women, exceeding the target of 40%. This review concludes that 82.2 percent of the urban poor in participating kelurahans benefited from improved access to infrastructure, social and economic services financed by the project which arguably generated improved living conditions among the poor in the project area and hence the efficacy with which Objective 2 was achieved was rated “substantial”. Rating Substantial OVERALL EFF TBL OBJ_TBL OVERALL EFFICACY Rationale Page 8 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review National Community Empowerment Program (P125405) Overall efficacy is rated substantial on the basis that the efficacy with which Objectives 1 and 2 were achieved was substantial. Overall Efficacy Rating Substantial 5. Efficiency Economic analysis. The Net Present Value (NPV) and the Economic Internal Rate of Return (EIRR) were not calculated at appraisal, as the project activities to be financed were not predefined but would be demand-driven (PAD, paragraph 43). A Traditional EIRR was not calculated at closure, as the project exclusively financed non- revenue generating sub-projects (ICR, page 43). Efficiency was therefore assessed by unit rate norms. Cost effectiveness conditions. The PAD (paragraph 43) noted that most recent studies (a study by the Indonesian National Planning Board - Finding of Post Construction Economic Impact Study of CDD projects in 2005, found that average construction costs of community based contracting was 40 percent lower, due to the high level of voluntary labor by residents in CDD projects. A 2010 impact evaluation of CDD projects in Indonesia confirmed that the cost of village infrastructure built by communities was, on average 56 percent lower than the cost of equivalent works built by the Ministry of Public Works or local government contractors, due to the savings from elimination (or reduction) of the cost of cost of contractor overheads. An annual assessment conducted by the National Management Consultant confirmed that in 91.2 percent of the participating kelurahans, infrastructure financed by the project was at least 20 percent less expensive than if the infrastructure were to be constructed using local government contractors, exceeding the target of 80%. The methodology for this assessment compared cost estimates prepared by the community against standard unit prices issued by local government and the actual cost of construction. The average project cost was about 35% less expensive. Further, the lower cost was not at the expensive of quality, as based on certification records and spot checking, 99 percent of the infrastructure works were rated as being of good quality, exceeding the target of 70 percent. Administrative and Operational Issues. The project used the services of Oversight Service Providers. This lowered oversight costs (as the company overheads were no longer included in the cost) and enabled faster procurement process (the procurement period was reduced by more than 50 percent) compared to using consultants). The project utilized the cost savings from exchange rate changes for activities associated with Bima's reconstruction and the transition to a recently launched project (the National Slum Upgrading project (NSUP). Further, the supervision cost was (0.6 percent of the loan) was lower that it would have been, since missions for this project were combined with the supervision of other related ongoing projects (PNPM Urban 111 and the recently launched NSUP. Although the project closing date was extended by three years due to a combination of factors, including restructuring activities, the increased counterpart funding enabled the project to significantly expand its reach to new beneficiaries, and according to the ICR (para 41) the project benefited roughly 2.7 million more individuals than was targeted at appraisal. There was no traditional economic analysis presented in the ICR because the project generated institutional and service benefits that could not be measured. Nevertheless, the ICR provided adequate evidence of the project’s Page 9 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review National Community Empowerment Program (P125405) sound cost-effectiveness and enhanced basic services for the poor. The project’s efficiency is therefore rated substantial. Efficiency Rating Substantial a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 0 Appraisal 0  Not Applicable 0 ICR Estimate 0  Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome The relevance of the PDO to the government and Bank strategy is rated High. Efficacy of the two objectives - to ensure that urban poor in participating kelurahans benefit from improved local governance conditions and to ensure that urban poor in participating kelurahans benefit from improved living conditions - are rated substantial, given that most of the outcomes were realized or exceeded. Efficiency is rated substantial, given that there is adequate evidence of the ways costs were economized. This review therefore rates the project’s overall outcome as satisfactory. a. Outcome Rating Satisfactory 7. Risk to Development Outcome The ICR identified a number of risks to the project's development outcome. Technical risk. The ICR (paragraph 65) noted that most of the project's physical infrastructure investments, were not technically complex. With the technical assistance provided by the project for the community groups, the technical risks were assessed as likely to be low. Financial risk. There are financial risk facing local authorities, given that there were no clear arrangements for covering the operational and maintenance costs of the infrastructure created under the project. Page 10 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review National Community Empowerment Program (P125405) Political risk. The ICR (paragraph 65) noted that political risk is likely to be low, given the commitment of the central and local governments to CDD projects. Natural disaster risk. There is ongoing risk, given that many project activities are in locations that are prone to seismic activity, volcanic eruptions, flash floods and other natural disasters. 8. Assessment of Bank Performance a. Quality-at-Entry This project was prepared based on the experiences from a number of prior Bank-financed urban projects in Indonesia mentioned in Section 3. Lessons incorporated at design, included separating project monitoring from the project evaluation function to maintain objectivity of the consultants and incorporating greater balance between the social, economic and infrastructure themes covered by the project. The project was prepared by a team which had prepared the previous PNPM project, based in Jakarta. Several risks were identified at appraisal, including high governance risk and substantial risk associated with the weak implementation capacity (PAD, page 13). Mitigation measures incorporated at design, included allocating significant resources to maintaining an active project website and establishing a complaints handling mechanism. With mitigation measures, the overall project risk was rated as moderate (PAD, page 13). The implementation arrangements were appropriate, with the Project Management Unit (PMU) housed in the Ministry of Public Works (the main implementing agency), as in the prior Bank-financed projects supporting the National Program for Community Empowerment (PAD, Annex 3, paragraph 2). The arrangements made at appraisal for monitoring and evaluation and safeguards and fiduciary compliance were to a large extent appropriate (discussed in sections 9 and 10 below). Quality-at-Entry Rating Satisfactory b. Quality of supervision According to the ICR supervision missions were conducted twice a year. These missions were supplemented by shorter site missions every year. In addition to traditional supervision methods, the project introduced teleconferences and web-based supervision to monitor project progress. The supervision teams were multi-disciplinary, with expertise in project management, financial management, procurement, monitoring and evaluation and safeguards. The ICR also noted that supervision costs were kept low, as most missions did not only cover this project, but other related projects (the ongoing PNPM Urban 111 project and the recently launched NSUP) (ICR, paragraph 61). The process of engagement with the government was aided, given that all team members were based in the country office. Loan savings resulting from exchange rate changes were used for reconstruction activities and for supporting a follow-on project. The Bank's management of the project supported efficiencies in project administration and supervision when providing support for Bima's reconstruction. The Bank team decided to finance support for Bima through existing funds under Component two, instead of reallocating funds for Component Page 11 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review National Community Empowerment Program (P125405) Four. Allocating the additional funds through component two allowed those kelurahans to receive support without creating the administrative burden of reallocating funds under a new component. There were minor issues associated with financial management during implementation, which were eventually resolved (discussed in section 10). Quality of Supervision Rating Satisfactory Overall Bank Performance Rating Satisfactory 9. M&E Design, Implementation, & Utilization a. M&E Design The results framework was well-defined and the key performance indicators were realistic in monitoring performance (discussed in section four). The M&E design included monitoring through a web-based Monitoring Information System (MIS) that was publicly accessible. The M&E design also envisioned developing a complaints handling mechanism and an impact evaluation of the specialized program for poorer areas. One shortcoming at design, was that there was no indicator for specifically tracking progress on improving operation and maintenance. b. M&E Implementation The ICR (paragraph 49) notes that monitoring was done through a variety of ways, including through the Monitoring Information System (MIS) website, complaints handling mechanism and spot-checking. A teleconferencing system was used by the Bank and government to expand supervision and by the National Management M&E Consultant employed by the project to monitor progress and trouble-shoot in real time. The complaints handling mechanism system allowed complaints to be submitted via the Project's website, phone or face to face with the community facilitators. These data were then published on the project website to ensure transparency. A beneficiary survey was also implemented in 2016 to track project performance. c. M&E Utilization According to the ICR (para 50) the data collected at the central and local levels were utilized for: (i) identifying problems and incorporate better controls: (ii) target resources to the areas with the greatest needs to maximize impacts: and (iii) demonstrate the project's benefits to communities and households. The project shared common data with other CDD projects through the same MIS and this provided a reference for ministries to identify active projects and allocate resources. Information from the website Page 12 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review National Community Empowerment Program (P125405) was also used as a primary reference to all quantitative surveys and qualitative studies undertaken for the National Community Empowerment Program. The project's website at closure not only provided access to data on this project, but also provided information on the recently launched NSUP. M&E Quality Rating Substantial 10. Other Issues a. Safeguards The project was classified as a Category B project under the World Bank safeguard policies. Four safeguard policies were triggered at appraisal: Environmental Assessment (OP/BP 4.01): Physical Cultural Resources (OP/BP 4.11): Indigenous Peoples (OP/BP 4.10): Involuntary Resettlement (OP/BP 4.12) (PAD, page 42). Environmental Assessment and Physical Cultural Resources. The PAD (page 23) noted that as in the past Urban Poverty Projects, the environmental impacts were expected to be localized environmental impacts that were reversible. The PAD (paragraph 55) notes that the project was to adopt the revised and updated Environmental Guidelines used for the PNPM Urban project. The guidelines included measures for management and conservation of Physical and Cultural Resources. The ICR does not provide details but notes that environmental safeguards were satisfactory. The ICR does not, however, indicate whether there were compliance issues associated with physical cultural resources during implementation. Involuntary Resettlement. The PAD (paragraph 51) noted that some physical investments under the neighborhood level program could involve involuntary land acquisition and resettlement. The updated guidelines used for the PNPM Urban 111 project were to be used for addressing issues pertaining to compliance with involuntary resettlement guidelines. The ICR (paragraph 52) notes that about 603 hectares of land were voluntarily donated from the rehabilitation of 48,633 subprojects. Indigenous Peoples (IP). The PAD (paragraph 52) noted that IP communities lived in eleven villages in eight provinces covered by the project outside Java. The IP framework developed for the PNPM Urban 111 project was to be used for addressing IP issues. The ICR does not provide details on whether there were compliance issues associated with the safeguards on indigenous peoples during implementation. b. Fiduciary Compliance Financial management. A financial management assessment was conducted at appraisal (PAD, paragraph 47). The financial risk was rated as substantial, in view of the scale of the project and limited capacity at the community level. Several mitigation measures were therefore incorporated at design, including intensive training arrangements for the district consultants and facilitators and the hiring of additional staff to the Project Management Unit for verifying consultants’ payments. With these mitigation measures, the financial risk was rated as moderate at appraisal. During implementation the project audits were unqualified in all years from 2013 to 2017. The ICR notes that the final audit of 2018 was in progress Page 13 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review National Community Empowerment Program (P125405) when the ICR was prepared and was to be submitted to the Bank by the end of June 2019. The ICR (paragraph 53) notes that the project website published details on consultant contracts audit results and follow up actions. Using the complaints handling system, the Project Management Unit (PMU) resolved 67,090 out of 67,349 complaints and that the PMU recovered US$465,000 of the misused funds This was only 66% of the total misused funds according to the ICR (para 54); there is no mention in the ICR whether there was a plan for recovering the remainder. Procurement. A procurement assessment was conducted at appraisal (PAD, paragraph 50). The assessment concluded that the procurement risk was high, based on the overall fiduciary risk environment in the country. Mitigation measures included extending the high-value contracts under the existing contracts under the ongoing PNPM - Urban 111 project and recruiting two local individual consultants to handle day-to-day management of procurement activities. With mitigation measures, the procurement risk was rated as substantial at appraisal. The ICR notes that during implementation the procurement processes were in compliance with Bank's procurement guidelines. c. Unintended impacts (Positive or Negative) --- d. Other --- 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment Outcome Satisfactory Satisfactory Bank Performance Satisfactory Satisfactory Quality of M&E Substantial Substantial Quality of ICR --- Substantial 12. Lessons The ICR draws the following four main lessons from the experience of implementing this project, with some adaptation of language. 1. Strong government ownership at all levels is crucial to overcome project challenges. The success of this project highlighted the importance of strong ownership for stakeholders at all levels of government. This is particularly important for CDD projects. 2. Concentrating investments in poor urban areas with the highest population densities may result in higher impact then spreading investments widely to reach all poor populations within an urban ward (kelurahan). Under the earlier projects, investments were dispersed widely Page 14 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review National Community Empowerment Program (P125405) in an attempt to reach all poor populations within the urban ward. This project experimented a different method for targeting investments by scaling up an advanced neighborhood development scheme, which provided increasing financing to the poorest areas with higher population densities based on medium term plans and spatial planning that was linked to city development planning. The lesson is that to achieve poverty reduction a targeted approach will often provide better results than a broader approach. 3. Both special grant allocations and gender sensitive meeting arrangements can help in increasing female participation. This project aimed at increasing female participation through training and a pilot initiative in Aceh where special grants were provided to improve women's access and control of the decision making. The lesson is that female involvement in project management can be achieved. 4. Online training (targeted for groups) may be critical for a national program that is implemented across a wide geographical area and diverse target groups. This project required annual training 4,000 facilitators and more than 120,000 people at the urban ward level. The training was achieved through newer forms of training such as through online training. The lesson is that modern training techniques can be effectively used in the context of development projects for rapid and cost-efficient training programs 13. Assessment Recommended? No 14. Comments on Quality of ICR The ICR is clear and provides a good, candid and comprehensive description of the CDD projects. For example, it clearly discusses the advantages of having a transparent complaints handling mechanism in such projects. The ICR is consistent with the OPCS guidelines and draws useful lessons from the experience of implementing this project. However, the ICR provides very few details on compliance with safeguards. The text of the ICR could have benefited from better editing to reduce its length; the recommended length for the main body of an ICR is 15 pages. a. Quality of ICR Rating Substantial Page 15 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review National Community Empowerment Program (P125405) Page 16 of 16