Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review 2nd Urban Poverty Reduction Pj-PREPUD II (P145848) Report Number: ICRR0022193 1. Project Data Project ID Project Name P145848 2nd Urban Poverty Reduction Pj-PREPUD II Country Practice Area(Lead) Djibouti Urban, Resilience and Land L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IDA-54580 30-Jun-2019 5,139,658.44 Bank Approval Date Closing Date (Actual) 14-May-2014 31-Dec-2019 IBRD/IDA (USD) Grants (USD) Original Commitment 5,600,000.00 0.00 Revised Commitment 5,600,000.00 0.00 Actual 5,139,658.44 0.00 Prepared by Reviewed by ICR Review Coordinator Group Chikako Miwa Vibecke Dixon Christopher David Nelson IEGSD (Unit 4) 2. Project Objectives and Components DEVOBJ_TBL a. Objectives The objective of the project was to increase access to basic urban services in Quartier 7 (Q7) in Djibouti City (Financing Agreement dated July 23, 2014, Schedule 1, page 5; PAD, para 22). b. Were the project objectives/key associated outcome targets revised during implementation? No Page 1 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review 2nd Urban Poverty Reduction Pj-PREPUD II (P145848) c. Will a split evaluation be undertaken? No d. Components Component 1: Infrastructure Upgrading in Quartier 7 (Appraisal cost: US$3.57 million. Actual cost: US$3.85 million.) intended to finance key urban infrastructure in Quartier 7 (Q7) in Djibouti City and the persistent challenge of employment in the neighborhood, with a special focus on women and young people (PAD, para 28). The main activities included: a) upgrading urban roads and drains in Q7 to improve access to basic infrastructure, reduce the road’s exposure to flood risks, and improve the Q7 residents’ mobility and safety; and b) revamping, upgrading, and landscaping the Omar Gouled Square and other small public spaces in Q7 to reinforce social cohesion and create an area of social exchange for the neighborhood. Component 2: Building Institutional Capacity and Citizen Engagement (Appraisal cost: US$800,000. Actual cost: US$522,513.) intended to: a) support urban development through preparation of urban restructuring plan of Djaga Bouldouq slum pocket in Q7, as well as development of two local plans for two areas on the south and west of the city; and b) support community development through targeting direct improvement at the community-level activities including establishment of the Quartier 7 Community Development Fund. Component 3: Support to Implementation, Monitoring and Evaluation (Appraisal cost: US$700,000. Actual cost: US$680,225.) intended to finance operating costs, equipment, short-term technical assistance and capacity building for project implementation staff, as well as technical and financial audits. In addition to the above components, physical and financial contingencies of US$530,000 was set aside at appraisal. Actual cost was US$90,000 at project closing. e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project Cost and Financing: At appraisal, the original cost estimate was SDR3.7 million (US$5.6 million equivalent) which was planned to be financed by the World Bank (PAD, para 38). The actual cost was SDR3.7 million (approximately US$5.14 million equivalent), which was fully financed by the World Bank (Operations Portal; ICR, page 2). The difference in the original and actual costs was due to the change in exchange rate between SDR and US$. Dates: The project was approved on May 14, 2014. It had a Level-2 restructuring on June 7, 2019 to extend the project closing date for 6 months from June 30, 2019 to December 31, 2019. There were no changes to the PDO and the PDO outcome indicators. 3. Relevance of Objectives Rationale Page 2 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review 2nd Urban Poverty Reduction Pj-PREPUD II (P145848) Country Context. Djibouti, which had a GDP per capita of US$1,430 in 2011 (PAD, para 3), experienced an average of 7 percent economic growth per year between 2013 and 2016, driven mainly by a large inflow of Foreign Direct Investments (FDI) (ICR, para 1-2). The country also benefitted from its political stability, its port-related activities, and the presence of military bases in its territory to sustain the economic growth. Despite the economic growth, absolute poverty declined only slightly from 42.2 percent to 41.9 percent between 2002 and 2012, according to the Government’s statistics (PAD, para 3). Sector Context. Approximately 71 percent of Djibouti’s estimated 865,000 inhabitants lived in cities (Djibouti Census 2009; PAD, para 1), where population grew at an average annual rate of 5.2 percent between 1960 and 2014, which was higher than the total population growth rate of 4.4 percent (ICR, para 2). On top of the demographic reason, urban population growth was accelerated by repeated climate-related hazards in rural areas and conflicts in neighboring countries. Quartier 7 (Q7) in Djibouti City hosted 25,000 inhabitants living on 0.56 km2 (PAD, para 10), including a large number of urban poor with a high unemployment rate. In Q7, only 24.4 percent of men under 35 (categorized as youth) and 21.2 percent of all women reported being employed, compared to 46.5 percent for all men (PAD, para 12). The neighborhood’s houses were essentially built below the street levels, but seventy percent of Q7 households had no sewage system, facing risks of flooding and health hazard (PAD, para 10). The northeastern section of Q7 was inhabited by about 8,000 urban poor, forming a slum-type area called Djaga Bouldouq, where narrow streets constrained an accessibility for public services like solid waste collection, ambulances, and firefighters (PAD, para 11). Relevance to Bank Assistance Strategies. At appraisal, the PDO was in full alignment with the Country Partnership Strategy (CPS) for FY14-17, as it directly supported the pillar on “Reducing Vulnerability.” At project closing, the project was in line with the Country Partnership Framework (CPF) for FY20-FY25, contributing to Objective 6 “Strengthening the resilience of vulnerable groups” of the pillar on Building Human Capital. The PDO was also aligned with the World Bank Group’s Regional Strategy for the Middle East and North Africa updated in 2019, particularly with its pillar on renewing the social contract. Relevance to Government Strategies. The project aligned with Vision Djibouti 2035, which was a long-term development plan launched by the government in 2014. The project supported the Djibouti City Urban Master Plan 2013-2035, which attempted to regulate city expansion and prevent new slum development. The project also supported improving the government’s Zero Slum Program launched in 2018. Institutional Capacity and Realism. As the implementing agency, the Djiboutian Agency for Social Development (ADDS: Agence Djiboutienne de Développement Social) leveraged the administrations and entities used for the preceding IDA-financed project. The Law and Decree to establish the ADDS defined its responsibility to conduct delegated contract management and implement development projects financed by international donors. Previous Sector Experience. The Bank had a long engagement in the country’s urban development sector since 1984, including constructing urban infrastructure for flood management and population resettlement in response to the flooding in 2004. Most recently, the Bank completed the Djibouti Urban Poverty Reduction Project (PREPUD) (P088876), which aimed to improve access to social and economic infrastructures (i.e. roads with drains, community facilities for health and education) and access to community development opportunities (i.e. literacy programs and vocational training for the neighborhoods especially women and youth, capacity development for community associations) (PAD, para 13). Based on the lessons learned from the preceding project which focused on the richer section of Q7, this project included activities to improve the living conditions of the slum residents, but unfortunately the scope and lack of specificity in the objective does not adequately reflect this background, nor does it illustrate the way in which urban Page 3 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review 2nd Urban Poverty Reduction Pj-PREPUD II (P145848) development can build and impact the lives of those living in the designated region. This outcome entails more than simply improving "access" to services. Clarity and Ambition of the Objective. The project’s objective, “to increase access to basic urban services in Q7 in Djibouti City,” did not fully capture the project’s expected outcome. The word “access” could refer to the targeted beneficiaries’ physical means or opportunities to approach the basic urban services. If this was the case, the stated objective had set its target lower than it could have done otherwise, as such, the target was achievable by completing the project activities. Alternatively, if “access” was meant to also refer to the usage of the basic services by the target beneficiaries, then the objective could have been stated differently for more clarity, for example: “to increase access to and use of basic urban services in Q7 in Djibouti City.” The definition of “basic urban services” was also vague, not specifying which services were targeted. The objective was substantially relevant to the strategies of the Bank and the Government, with sufficient considerations on the institutional capacity and realism and the previous sector experience. On the other hand, the objective really needs to be more precisely worded to articulate its targeted outcome and how the project activities were expected to change people’s lives. Overall, given the considerable shortcoming around how the objective is framed, the relevance of objective is rated modest. Rating Relevance TBL Rating Substantial 4. Achievement of Objectives (Efficacy) EFFICACY_TBL OBJECTIVE 1 Objective To increase access to basic urban services in Quartier 7 (Q7) in Djibouti City Rationale Theory of Change: There were two paths to achieve the objective: (1) infrastructure upgrading, and (2) institutional building and citizen engagement. Theory of Change for Path 1 - Upgrading urban infrastructure such as roads, drainage network, and lighting, as well as rehabilitating Omar Gouled central square with its public facilities would improve physical access to services such as safe transport and living conditions such as sanitation for Q7 residents. The improved access to services were expected to contribute to the long-term outcomes such as greater mobility, greater resilience to floods, a safer and more pleasant environment, and reduced health risks. Key assumptions to enable the change were: (a) the Djiboutian Agency for Social Development (ADDS: Agence Djiboutienne de Développement Social) would demonstrate appropriate capacity to carry out urban infrastructure upgrading; and (b) the word “access” meant physical access to the basic urban services, as implied by the PDO outcome indicators. Upgrading urban infrastructure would also enhance satisfaction of Q7 residents with public services and infrastructure. According to the Theory of Change in the ICR (Figure 1, page 6), improved service access and living conditions, combined with satisfaction of Q7 residents, were expected to increase Page 4 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review 2nd Urban Poverty Reduction Pj-PREPUD II (P145848) land ownership, willingness to upgrade houses, and expand land titling, ultimately leading to better socioeconomic opportunities for development of Q7 neighborhood with community participation in the long- term. A critical assumption for the change was that improved service access and living conditions, as well as enhanced social cohesion, would expand land titling and create better development prospects for the neighborhood (Ibid). However, the assumption would not be sufficient to fill the gap in the logical link between the cause and the expected long-term outcome. The indirect logical link and existence of many other conditions affecting land titling and social cohesion would cause attribution challenges. Outputs: Key outputs of Path 1 were as follows:  1,600 meters of roads and drains including street lighting were constructed, meeting the target (ICR, page 30). As a result, the risk of flood was reduced for 120 houses and shops situated on the renovated streets and more than 130 houses situated on the adjacent streets (ICR, page 46). In addition, a large number of houses in Djaga Bouldouq benefited from 400 KVA transformer that provided stable electric power. (ICR, page 46).  70 percent of Omar Gouled Square development was completed, not meeting the target of 100 percent (ICR, page 30). The project revamped and landscaped Omar Gouled Square, and financed construction of an open space for the Square, a community center/library, a kiosk for youth, and a mosque. The implementation of the activity was delayed due to the resettlement issue discussed below. The construction of an amphitheater was cancelled as the community opted out of this activity due to existing of temporary mosque (ICR, page 52).  Operation and Maintenance (O&M) arrangements of the Omar Gouled central square was in place, meeting the target. An O&M manual was developed and adopted, and a budget line was made available by the Djibouti City Municipality (ICR, page 47). Nevertheless, the implementation of O&M was inadequate, as the muds on the upgraded roads were not removed regularly even though a cleaning machinery was available at the Office of Highways (ICR, para 84).  100 percent of grievances related to delivery of project benefits were registered and addressed, meeting the target (ICR, page 47). Some verbal complaints regarding the construction works of track 39, track 44, and the mosque were raised by the residents or representatives of Q7 (ICR, Annex 6, page 62-63). The ADDS and the construction companies remedied the problems swiftly. Outcomes: The achievements of PDO outcome indicators relevant to Path 1 were as follows: 1. Direct project beneficiaries from at least one subproject, e.g. urban road, drainage, street lighting, community development fund were 28,424 persons, exceeding the target of 25,000 persons (114 percent of the target; ICR, page 46). Of which, the percentage of female beneficiaries was 55 percent, exceeding the target of 50 percent (Ibid). Though the PDO outcome indicator’s target was surpassed, it would have been more informative if how the beneficiaries’ lives changed due to the project was described. According to the ICR (page 47), the target was surpassed due to the new beneficiaries of the NGO activities. On the other hand, the scope of road upgrading activity was reduced during project implementation, due to the reallocation of funds from the road upgrading activity to the old Makhi Mosque reconstruction activity (ICR, para 30). At appraisal, five streets with a total length of 1,665 meters were planned to be upgraded Page 5 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review 2nd Urban Poverty Reduction Pj-PREPUD II (P145848) with street lighting, paving, drainage, and sidewalks. At project closing, two streets with the total length of 900 meters (54 percent of the plan) were fully upgraded, while three streets with the total length of 665 meters were only provided with lightings (ICR, page 51). According to the map of Q7 with specific upgraded roads highlighted (ICR, page 48), the upgraded roads were located in the south of Q7, not running into the northeastern section where the slum area existed. This raised a question on to what extent the poor neighborhood in the slum area benefitted from the road upgrading activity. The remaining three roads were not prioritized, as the ADDS confirmed that upgrading these roads would be able to be conducted by the follow-on project, as intensive engineering work was not required for these roads (ICR, page 47). 2. Residents of Q7 within 50 meters of rehabilitated and new roads, drains, and street lightning were 8,160 residents, exceeding the target of 7,500 residents (108 percent of the target; ICR, page 46). Though the PDO outcome indicator’s target was surpassed, it was questionable to what extent the achievement of the outcome was attributable to the project, given that the smaller scope of road rehabilitation activity as mentioned above. In addition, the population growth in the project area from 16,060 in 2014 to 20,020 in 2019 (ICR, Annex 4, Table 5, page 41) might have contributed to the achievement of target. It would have been more useful if the outcome indicator measured what changes were brought after the rehabilitation activities were completed. For example, a decrease in travel time on the rehabilitated and new roads, a decrease in frequency of flooding where drains were installed, and an increase in the number of pedestrians on the streets with lightnings disaggregated by gender could have been alternative indicators. 3. According to the end-of-project beneficiary survey, 83.2 percent of the surveyed users agreed on the following statement: “Omar Gouled central square is well maintained and operated,” exceeding the target of 75 percent (ICR, page 46). Regarding the methodology of the end-of-project beneficiary survey held in December 2019, a total of 80 households were surveyed randomly, 20 for each of the modules or types of activities (i.e. the road construction works for street number 39 and 44, other road works, the electrical and public lighting works, and the rehabilitation of the Omar Gouled Square such as Mosque, Kiosk for youth, and Community Center/Library). The objective was to survey at least 20 percent of the total amount of beneficiaries in relevant areas. The survey used the concepts and tools of the SERP methodology (Participatory Monitoring, Evaluation and Recovery) through Citizen Assessment Sheet. Interviewers and facilitators received training conducted by a Bank expert focused on mastering the tools and concepts of the methodology. Both quantitative and qualitative data was collected from the residents as well as through the focus groups. The two types of data were analyzed together with the transcripts of the audio recordings of the discussions held during the interviews. Though the target of the outcome indicator was surpassed at project closing, the ICR (para 62) noted that the late completion of the Omar Gouled Square rehabilitation activities right before project closing did not allow sufficient time for the residents to feel the improvements in maintenance and operation of the Omar Gouled Square. This raised a question on the attributability of the survey result to the project. Moreover, the ICR (para 91) rightly pointed out that beneficiary surveys might not be comprehensive enough to measure the implementation progress towards reaching the intended outcome, as the beneficiaries’ perceptions on the results of the project might be susceptible to exogenous factors. Land Ownership and Titling: According to the ICR (para 56), there was an increase in the official recording of land tenure rights by the residents of Q7 after the project intervention. However, to what extent the increase in land titling was attributable to the project is uncertain due to the lack of evidence. The upgraded urban infrastructure increased the targeted beneficiaries’ accessibility to the basic services such as safe transport, drains, and community development. The infrastructure upgrading investments Page 6 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review 2nd Urban Poverty Reduction Pj-PREPUD II (P145848) contributed to making some progress even before project closing towards the enhancement of land titling, which was one of the outcomes expected to be materialized in the long-term. On the other hand, it was uncertain to what extent the achievements were attributable to the project due to the methodologies to measure the outcomes, such as counting the number of direct beneficiaries without explaining the possible effect of population growth and conducting the beneficiary survey before the residents were able to provide comprehensive feedback. Theory of Change for Path 2 - Developing urban planning tools (i.e. the Djaga Boudlouq restructuring plan, city expansion plans), combined with capacity building support and training, would strengthen institutional capacity on urban planning and enhance coordination at national and local levels. The strengthened capacity would enable the ADDS to develop and implement a comprehensive slum program to upgrade the existing slums and preventing new unplanned settlements. Key assumptions for the change were that: (a) the strengthened institutional capacity on urban planning and the enhanced coordination would increase access to the basic urban services; and (b) there was sustained political will to implement and advance upgrading of the slum-type area, Djaga Bouldlouq. Implementing community development subprojects and developing a citizen engagement platform would encourage more active participation of women in neighborhood development, leading to empowerment of the community. A key assumption for the change was that the community development subprojects and the citizen engagement mechanism were effectively designed to ensure participation of women. In the long-term, strengthened institutional capacity would lead to development and implementation of comprehensive program of upgrading slums and preventing new settlements. Outputs: Key outputs of Path 2 were as follows:  25,841 person-days of short-term employment for residents of Q7 were generated under the project investments, exceeding the target of 15,000 person-days (172 percent of the target; ICR, page 47). The road rehabilitation works employed the urban poor, especially the youth, to provide them with means for income generation (ICR, para 32). To what extent the short-term employment contributed to poverty reduction was uncertain, as these were temporary jobs with low skills transfer.  3 local development plans for the new neighborhoods in Balbala South, Khor Ambado, and Nagad were completed and approved by the authorities, exceeding the target of 2 local development plans (ICR, para 36; page 47). The local development plans supplemented the Djibouti City Urban Master Plan 2013-2035 with more detailed cost-recovery plans to facilitate expansion of the city in the low- income neighborhoods. According to the ICR (para 36), the preparation process of the local development plans contributed to enhance institutional capacities on urban planning, by strengthening coordination and hands-on capacities of the Ministry of Urban Development and Housing, the Ministry of Finance, and the Ministry of Interior.  A restructuring plan/study of Djaga Bouldouq, a slum-type area in Q7, was approved by the authorities, meeting the target (ICR, page 47). The restructuring plan included all socioeconomic studies, feasibility studies, preliminary technical studies, cost assessments, economic analysis, environmental and social assessments, and safeguard studies, as well as a specific citizen engagement mechanism aimed at easing the implementation of the restructuring operation (ICR, para 37). The restructuring plan was being implemented under the Integrated Slum Upgrading Project financed by the Bank. Page 7 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review 2nd Urban Poverty Reduction Pj-PREPUD II (P145848)  50 subprojects funded by the Community Development Fund (CDF) were implemented, meeting the target of 50 subprojects (ICR, page 47). Of which, 51 percent of the subprojects was from women-led associations, meeting the target of 50 percent (Ibid). The CDF aimed to support sustainable initiatives within the neighborhood, especially those led by and targeting women and youth. The CDF funded various activities, including cleanliness and sanitation, maintenance and operation of roads and public buildings, education and training, and culture and sports, as shown in Annex 6, page 58-60 in the ICR. The CDF-funded subprojects also provided direct supports to youth employment, such as driver license education, schooling, after-school support, and building local NGO capacity. Outcomes: The achievements of PDO outcome indicator relevant to Path 2 were as follows: 1. Direct project beneficiaries from at least one subproject, e.g. urban road, drainage, street lighting, community development fund were 28,424 persons, exceeding the target of 25,000 persons (114 percent of the target; ICR, page 46). Of which, the percentage of female beneficiaries was 55 percent, exceeding the target of 50 percent (Ibid). According to the ICR (page 47), the target was surpassed due to the new beneficiaries of the NGO activities supported by the CDF-funded subprojects. No evidence on how the beneficiaries’ lives changed due to the project was provided. Institutional Capacity Strengthening: There were some indicators to measure the achievements of outputs by the project on institutional capacity strengthening, but not outcomes. According to the ICR (para 52-54), the project supported the urban planning agencies to deepen their understanding on urban management and planning. The project strengthened institutional and financial foundations of the urban planning agencies at the municipality by improving system coordination among various urban players. The project also supported the Government to improve the Zero Slum Program by adopting a new institutional approach to set the priorities and principles of the investments and streamline regulations on land administration and urban planning. Moreover, the project provided the local associations and the ADDS with training on managing the CDF and developed a manual on the CDF procedure, but how the training and the manual brought changes to their institutional capacities were unspecified. Citizen Engagement: The PDO outcome indicator 1 above counted the direct beneficiaries of the CDF activities in aggregation with other project activities, which did not adequately measure outcome-level achievements brought to the community by the CDF activities, such as operational and financial sustainability of the local associations without the project support or improvements in the living conditions in the neighborhood due to the CDF activities. As noted in the ICR (para 62), the indicator of the number of person/days of short-term employment generated measured a temporary impact of project interventions and did not provide feedback about the sustainable impact the project made in creating job opportunities for residents. According to the end-of-project beneficiary survey (ICR, Annex 6, page 61-62), the CDF activities were perceived positively by the beneficiaries and the community association presidents who participated in the focus groups. Nevertheless, the survey result was only to indicate the community’s level of satisfaction with the CDF activities at the time of project closing. Page 8 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review 2nd Urban Poverty Reduction Pj-PREPUD II (P145848) The institutional and financial capacities of the urban planning agencies were strengthened by enhancing the coordination system, establishing the priorities and principles of the investments, and reorganizing regulations on land administration and urban planning. On the other hand, the outcome-level achievements on institutional strengthening were not fully measured or explained. Moreover, it was uncertain to what extent the CDF activities and the short-term jobs created under the activities contributed to the increase in access to the basic urban services in a sustainable manner. Overall, the achievement of the Objective is rated substantial with shortcomings on justifying the achievements of outcomes. Rating Substantial OVERALL EFF TBL OBJ_TBL OVERALL EFFICACY Rationale No evidence on the outcomes (i.e. greater mobility, greater resilience to floods, a safer and more pleasant environment, and reduced health risks) of the infrastructure rehabilitation was provided. The institutional capacity building and citizen engagement improved access to some of the basic urban services such as cleanliness and sanitation, education and training, and culture and sports. This was enabled by supporting local association and creating short-term employment opportunities under the project activities. However, there were uncertainties regarding the level of achievement and sustainability. Overall, the efficacy is rated substantial with shortcomings on justifying the achievements of outcomes. Overall Efficacy Rating Substantial 5. Efficiency Economic Analysis: At appraisal, an internal economic rate of return (ERR) was 12.4 percent with a discount rate of 8 percent and the net present value (NPV) of US$6.29 million (PAD, Annex 7, para 7, page 58). With a discount rate of 6 percent, the internal ERR at appraisal was 10.2 percent (ICR, para 39). At project closing, with a discount rate of 6 percent over a project life of 30 years, the internal ERR was 10.4 percent (ICR, para 39; Annex 4, Table 4, page 40), as well as the NPV of US$2.6 million (ICR, Table 1, page 14). Comparing the internal ERRs at appraisal and at project closing, the efficiency at project closing was in the range of the level expected at appraisal. Regarding the methodology, the econometric analysis conducted at project closing calculated the economic benefits based on (i) the housing values and the elasticity of consumption per capita to changes in nightlights calculated by using the data of the household survey of 2017 (EDAM4), (ii) the average housing value and per capita expenditures in the targeted Arrondissement, (iii) the number of houses on the Page 9 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review 2nd Urban Poverty Reduction Pj-PREPUD II (P145848) upgraded and adjacent streets, (iv) the difference in the growth of the number of nightlights throughout the project in Q7 in comparison to Q6, which was considered at appraisal as a similar area (ICR, para 41). Aspects of Design and Implementation that Influenced Efficiency: The development plan of the Omar Gouled central square was re-designed during implementation to address an involuntary resettlement issue, which started with a nearby mosque’s claim for ownership of a portion of land in the square and its occupation of the land with a temporary mosque. The mosque subsequently was considered by the Bank Safeguards as a formal squatter. After long deliberations involving various Government authorities and the President of Djibouti, the Resettlement Action Plan (RAP) was revised and identified a rehabilitation of an old mosque beside the square as an in-kind compensation. The project therefore financed the reconstruction of the old Makhi Mosque by reallocating funds from the road rehabilitation activities. However, the temporary mosque was not removed, leading to reductions in the scale of the road upgrading activities and modifications in the plans on upgrading the Omar Gouled central square, as described in Section 4. No explanation was provided by the ICR why the temporary mosque remained after the reconstruction of the old mosque. This resettlement issue indicated the lack of beneficiary involvement at design stage, as described in Section 8. The internal ERR at project closing was in the range expected at appraisal though the urban infrastructure upgrading activities were delayed and modified to address the involuntary resettlement issue. Overall, the efficiency is rated substantial. Efficiency Rating Substantial a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 0 Appraisal  10.20  Not Applicable 0 ICR Estimate  10.40  Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome The relevance of objectives was substantial despite the somewhat limited scope of the objective. The institutional capacity building and citizen engagement improved access to some of the basic urban services by providing direct supports. Due to the uncertainties on the level of achievement and its sustainability, the overall efficacy is rated substantial with notable shortcomings on justifying the achievements of outcomes, which have affected the outcome rating. The efficiency was substantial, as the internal ERR at project closing was in the range expected at appraisal but there were implementation delays due to design and implementation aspects. Overall, the outcome is rated satisfactory. Page 10 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review 2nd Urban Poverty Reduction Pj-PREPUD II (P145848) a. Outcome Rating Satisfactory 7. Risk to Development Outcome 1. Risk for insufficient institutional capacity and ownership of stakeholders: There was a stakeholder risk identified at appraisal given this project’s multisectoral nature and the need for close collaboration among several public entities (PAD, para 49). To mitigate the risk, the project enhanced knowledge and skills of the entities responsible for urban planning and delivery and involved the community in project activities from the beginning to increase their ownership of the future project assets (ICR, para 83). At project closing, the project results verified that CDF effectively engaged communities, as well as that community planning made the community a main player to implement the decided plans who were capable to influence the decisions in a more appropriate manner (ICR, para 88). In addition, the ongoing Bank-financed projects in the country, including Djibouti Integrated Slum Upgrading Project (P162901), were expected to continue strengthening institutional capacities of relevant public entities and securing commitments of authorities. 2. Risk for inadequate maintenance of rehabilitated roads: There was an Operation and Maintenance (O&M) risk identified at the closing of the preceding project, given the limited resources allocated for road maintenance (ICR, para 84). To mitigate the risk, the project integrated O&M arrangements and ensured budget allocation for O&M. At project closing, however, the maintenance of the roads upgraded under the project was insufficient given the accumulation of mud on them. According to the Djiboutian Agency for Social Development (ADDS: Agence Djiboutienne de Développement Social), the mud collected on the rehabilitated tracks 39 and 44 needed to be cleaned by a certain type of machinery with mechanical brushes. Even though the machinery was already granted by Japan International Cooperation Agency (JICA) and was available at Djibouti’s Office of Highways (OVD) facilities, the machinery was not used regularly for maintenance (ICR, para 84), which might be going to lead to an early deterioration of the roads. There were no risk mitigation measures presented in the ICR to ensure regular removal of mud from the rehabilitated roads after the project closing. 8. Assessment of Bank Performance a. Quality-at-Entry The project’s strategic relevance, technical, financial, social development, environmental, and fiduciary aspects were thoroughly considered at entry. The project design built on that of its predecessor and reflected its many strengths and addressed its weaknesses in the new design. It also benefited from the procedures already in place for project implementation as well as reporting arrangements. It used innovative techniques, such as the placemaking methodology for the Square’s landscaping. The composition of the Bank team at entry ensured the compliance with the Bank policies and procedures. It was noted that, according to the ICR (para 89), “the project design was intentionally made simple given the challenging context, and its objective was realistic and achievable,” however, the project development Page 11 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review 2nd Urban Poverty Reduction Pj-PREPUD II (P145848) objective could have been more clearly stated to specify its expected outcomes and approaches to achieve them, as described in Section 3 and 9. The vagueness of the stated objective weakened the results framework, especially the designs of the outcome indicators and the methodologies to measure the achievements. Though the project was the second one in the series, the objective was not pitched at a level to address a development problem, as described in Section 3.Moreover, the resettlement issue for the temporary mosque in the Omar Gouled Square was not detected through beneficiary engagement at planning stage, which resulted in the delay and scaling down of the activity. Overall, the Quality at Entry is rated moderately satisfactory. Quality-at-Entry Rating Moderately Satisfactory b. Quality of supervision The supervision inputs and processes were adequate, as the composition of the Bank team showed continuity which contributed to having an effective implementation support. A total of eleven implementation support missions were conducted over the five years of implementation. The Bank also exhibited flexibility in adopting corrective measures, such as updating the abbreviated RAP with the formal inclusion of such temporary mosque as a squatter, as well as an in-kind compensation until the old mosque beside the project site was rehabilitated, yet the resettlement issue of the temporary mosque remained unsolved. The Bank team adjusted the ISR ratings to candidly reflect the implementation status, in order to motivate the Borrower to focus on critical issues. For example, they downgraded the Implementation Progress rating to empathize the importance of finalizing the design and launching the civil works on the Square and accelerated the activity. Moreover, the project recently received an ITS Knowledge and Information Services (ITSKI) award for outstanding project filing. The team’s efforts were recognized for their contribution to the management of the Bank’s information by being one of the two Top Projects of the 120 projects monitored by ITSRI in FY19 (ICR, para 80). On the other hand, during implementation, mitigation measures against the sustainability risk regarding inadequate road maintenance after project closing were not thoroughly considered and implemented, as mentioned in Section 7. The issues of weaknesses in Results Framework with inadequate indicators were not addressed by the Bank’s supervision team, which made it difficult for the project to measure and report relevant project outputs and outcomes. Overall, the Quality of Supervision is rated moderately satisfactory. The Quality at Entry was moderately satisfactory, as strategic relevance, technical, financial, social development, environmental, and fiduciary aspects were ensured though there were some weaknesses in the objective and the Results Framework as well as the inadequate beneficiary engagement at design stage. The Quality of Supervision was moderately satisfactory, as the Bank team demonstrated flexibility to address the resettlement issue but did not fully address the issues of the Results Framework. Thus, the Overall Bank Performance is rated moderately satisfactory. Quality of Supervision Rating Moderately Satisfactory Page 12 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review 2nd Urban Poverty Reduction Pj-PREPUD II (P145848) Overall Bank Performance Rating Moderately Satisfactory 9. M&E Design, Implementation, & Utilization a. M&E Design The stated objective caused an ambiguity in what the word “access” meant in terms of the expected outcomes, as described in Section 3. If the stated objective aimed to increase the means to approach basic urban services, then it could be argued that the stated objective set a modest target, as such target could be directly achieved by completing the project activities. To observe the development results brought to the residents by the project activities, another objective on the utilization of the urban basic services could have been added. Moreover, it was not clear how the increased institutional capacity on urban planning, which was stated as a short/medium-term outcome, would fit into or contribute to the stated objective. If the increased institutional capacity on urban planning was intended to be a separate outcome than the increased access to the urban basic services, then an addition of another objective could have been considered. The ambiguity in the stated objective affected the design and the measurement methods of the three PDO outcome indicators. Two of these outcome indicators counted the number of beneficiaries, which were adequate to capture only the achievement of outputs. In addition, the outcome indicator 2 did not take the natural change in the number of residents in the target area into a consideration, as described in Section 4. Furthermore, the outcome indicator 3 was designed to measure citizen satisfaction with the maintenance and operation of Omar Gouled Square, which became inadequate during the project implementation, as the delay in the upgrading of the Square did not allow enough time for the residents to feel the improvements and be able to evaluate the impact of the intervention and its maintenance. it would make more sense to measure the result of Outcome Indicator 3 at a later stage which is beyond the Project purview. Lastly, there was no indicator to measure the achievement of outcome derived by the interventions on institutional capacity strengthening and citizen engagement, as discussed in Section 4. On the positive side, annual targets of indicators were set at appraisal and all target values remained unchanged throughout the project. Data sources were defined for all indicators. The M&E design and arrangements were well-embedded institutionally, as the ADDS was given the overall responsibility to collect information and maintain the reporting building upon the experience accumulated by the preceding project. b. M&E Implementation M&E systems were implemented as per the original design. All the indicators in the Results Framework were measured and reported as per the agreed frequency. The ADDS ensured attention to effective M&E implementation, which diligently carried out the monitoring function of the project with a sense of ownership. The ADDS also submitted quarterly progress reports on key activities, progress, and results to the Bank in a timely manner. On the other hand, the weaknesses in M&E design described above were not brought to discussion during implementation. Page 13 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review 2nd Urban Poverty Reduction Pj-PREPUD II (P145848) c. M&E Utilization The M&E data helped the ADDS to make decisions on implementing the project, such as extending the project closing date. The data collected through the end-of-project survey was also utilized to assess the residents’ satisfaction with all activities delivered by the project, capturing the perception of Q7 residents of various dimensions of project intervention. In addition, the M&E system supported the ADDS to collect and address the grievances. The M&E design had some weaknesses in the stated objective, the outcome indicators, and their measuring methods, which affected the implementation and utilization of M&E. Nevertheless, the M&E implementation and utilization were well managed in a responsible manner, with an additional data collection on the beneficiaries’ satisfaction. Overall, the M&E Quality is rated modest. M&E Quality Rating Substantial 10. Other Issues a. Safeguards The project triggered OP 4.01 (Environmental Assessment) and OP 4.12 (Involuntary Resettlement), and was classified as Category B. The project complied with the Bank safeguards. An Environmental and Social Management Framework (ESMF), Environmental and Social Management Plans (ESMPs), and a Resettlement Policy Framework (RPF) were developed and used during procurement, M&E, and E&S mitigation measures. No major accident/incident was reported during project implementation. An abbreviated resettlement action plan (ARAP) was also prepared and updated three times to address the issues related to a group of 15 encroachers, who affected the rehabilitation of the Omar Gouled Square (ICR, para 68). A wide range of stakeholders, including central and local authorities and civil societies, were consulted with for the implementation of the ARAP. b. Fiduciary Compliance Financial Management: The institutional arrangements for financial management at the ADDS were adequate. All the independent audits were completed in a timely manner with unqualified clean auditor’s opinions. Unaudited Interim Financial Reports (IFRs) and Project Financial Statements (PFS) were acceptable and provided on time. The project was challenged by a financial loss due to the fluctuations in the exchange rate between SDR and USD; nevertheless, a risk on shortage of funds were well mitigated by a tight control over the budget (ICR, para 70). Procurement: The procurement was carried out in accordance with the Bank’s Procurement and Consultant Guidelines. The procurement review procedures for the ADDS were modified in 2017 to align Page 14 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review 2nd Urban Poverty Reduction Pj-PREPUD II (P145848) with that for the other public institutions, requiring the ADDS to submit procurement documents to the Commission Nationale des Marchés Publics (CNMP) for prior reviews. Overall, the Djiboutian procurement Manual of Procedures was in line with the Bank’s procedures. The Bank’s standard procurement documents were used throughout the implementation, as a total of 26 contracts for works, goods and consultancy services were procured and successfully implemented in accordance with procurement plans (ICR, para 69). c. Unintended impacts (Positive or Negative) n/a d. Other n/a 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment Outcome Satisfactory Satisfactory The weaknesses in the objective and the Results Framework as well as the inadequate Bank Performance Satisfactory Moderately Satisfactory beneficiary engagement at design stage remained during the implementation, and were not adequately addressed. Quality of M&E Substantial Substantial Quality of ICR --- Substantial 12. Lessons The following two lessons in the ICR stood out as important and relevant to other projects in Djibouti with infrastructure components and are presented here with some editing. The Bank’s two consecutive operations in Q7 (PREPUD and PREPUD II) prove that the Bank needs to ensure the development and enforcement of an operations and maintenance program prior to launch of implementation. Without such effort, there would be challenges to sustainability of the project assets, as described in Section 7. Additional consultations at the design stage with the responsible parties about the future of each such asset would be essential. Utilizing its convening power, the Bank may be able to also support the Government to develop an action Page 15 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review 2nd Urban Poverty Reduction Pj-PREPUD II (P145848) plan for providing systematic maintenance of existing investment. Moreover, a need for an integrated approach to address the maintenance issue needs to be well communicated to the active donors in the country. Key project activities need to be designed to be within the area of influence of the implementing agencies to ensure their capacities to resolve any potential issues with stakeholders, especially when a faith-based organization is involved. The project encountered the implementation delays due to the claimed ownership on the land allocated to the Square and the construction of a temporary mosque. The ADDS had to seek higher-level support to resolve the issue, as such issue was outside of its scope of responsibilities and control. 13. Assessment Recommended? No 14. Comments on Quality of ICR The ICR provides a good overview of the project with a concisely and well written narrative, which is structured around the project development objective (PDO). The ICR includes a clear figure describing the project’s results chain and offers a detailed description of the implementation challenges and of how they were addressed. Moreover, the ICR includes a candid discussion of Bank and Borrower performance in general. The lessons appropriately respond to the specific experiences and findings for the project. On the other hand, the quality of evidence could have been improved by providing data where possible and elaborating the reasons of over- and under-achievement of the targets of the indicators to show to what extent the achievements were attributable to the project. In addition, the ICR could have done a more thorough job of clarifying the causal relationship between the activities on strengthening the institutional capacity on urban planning and the stated PDO. Overall, the quality of the ICR is rated substantial. a. Quality of ICR Rating Substantial Page 16 of 16