ICRR 13030 Report Number : ICRR13030 IEG ICR Review Independent Evaluation Group 1. Project Data: Date Posted : 01/29/2009 PROJ ID : P104357 Appraisal Actual Project Name : Yogyakarta And US$M ): Project Costs (US$M): 6.6 2.3 Central Java Earthquake Roof Structure Project Country : Indonesia Loan /Credit (US$M): Loan/ US$M ): 6.6 2.3 Sector Board : UD US$M): Cofinancing (US$M ): Sector (s): Housing construction (100%) Theme (s): Access to urban services and housing (100% - P) L/C Number : Board Approval Date : 12/22/2006 Partners involved : Closing Date : 08/31/2007 08/31/2007 Evaluator : Panel Reviewer : Group Manager : Group : Anna Amato Ridley Nelson Monika Huppi IEGSG 2. Project Objectives and Components: a. Objectives: The project development objective was to respond to the most basic needs of the households affected by the earthquake through the provision of roof structures that transition to permanent housing . b.Were the project objectives/key associated outcome targets revised during implementation? No c. Components (or Key Conditions in the case of DPLs, as appropriate): The individual component amounts both at appraisal and completion were not available in the project documents . However, the costs were available by implementing agency : Cooperative Housing Foundation International - Appraisal: US$2.38 million; Actual: US$0.91 million International Organization for Migration - Appraisal: US$4.26 million; Actual: US$1.39 million (i) Set-up of the prefabrication facility . (ii) Preparation of Roof Structure Kits (RSKs). (iii) Targeting and distribution of kits to beneficiaries, which included : a) Site identification; b) Finalization of beneficiaries list and raising awareness; c ) Ex-post distribution/monitoring. (iv) Assistance to beneficiaries during construction . (v) Quality control and reporting. d. Comments on Project Cost, Financing, Borrower Contribution, and Dates: Over the course of the project, it became apparent that the GoI reconstruction program for permanent housing was moving ahead at a rate far beyond what this transitional housing project had anticipated, eventually succeeding in constructing the approximately 150,000 houses it had promised within one year of the earthquake . As a result of the reduced scale of project operations, it was clear that total grant allocation would not be fully utilized . At the time of the mid-term review mission, when the decision was made to reduce the scale of the project, US$ 1.27 million of the grant had been disbursed and an additional US$ 0.64 million was scheduled to be disbursed . At project closure, US$2.3 million had been disbursed and US$ 4.3 million was cancelled. 3. Relevance of Objectives & Design: The relevance of the project objective at the time of appraisal was substantial. The Roof Structure Project (RSP) was designed to provide approximately 24,000 temporarily homeless families in targeted villages in the affected areas with safe and durable housing through the provision of roof structures while their permanent house was being rebuilt. The lack of adequate housing impacted the capacity of people to resume their livelihoods . A large percentage of the affected population was and remains engaged in small home -based industries. Hence, in the absence of durable shelter, these communities would not be able to resume their previous economic activities . Also, the lack of adequate housing could cause severe health problems, which would worsen during the rainy season, imminent at the time of project preparation.The project promoted the transfer of earthquake -resistant construction knowledge to beneficiaries as well as strengthening local institutional capacity , strongly supporting the final pillar of the FY2004-2008 Indonesia Country Assistance Strategy -- that is, to enhance disaster risk preparedness and response. The relevance of design is assessed as substantial.The components were well-designed to achieve the objective . The Results Framework was also designed very clearly to monitor the results outcome . All planned monitoring appears to have been done, including a beneficiary survey . This project was prepared while the new emergency response Operational Policy was about to be approved and represents the new approach of responding quickly with targeted interventions. As one of the first interventions attempting this approach, this grant, though funneled through the GoI, was given to two international NGOS, the Cooperative Housing Foundation (CHF) and the International Organization for Migration (IOM). These NGOs were thought to be the most effective way to quickly implement this project, since they were already familiar with conditions on the ground . 4. Achievement of Objectives (Efficacy): The objective of responding to the most basic needs of households affected by the earthquake through the provision of roof structures that transition to permanent housing was substantially achieved . The most significant impact on the achievement of the project's objective was the outstanding success of the GoI permanent housing reconstruction program that substantially decreased the total number of roof structures required, increased the number of recipients transitioning to their permanent houses and, consequently, decreased the occupancy rate of the roof structures by the time of the final survey undertaken by the World Bank in July 2007. At mid-term review (MTR) three months after project commencement, the project scale was significantly reduced from a total initial target of 24,000 units to 6,141 units. The implementation of the permanent house reconstruction in Yogya was unprecedented in terms of speed and coverage . The GoI considered Yogyakarta an important province, and had a strong commitment to rebuilding it quickly. Although the numerical target was not achieved as planned, the Project was able to cover all of the remaining needs for roof -structures. Therefore, only the key outcome indicator concerned with recipient satisfaction was met (and significantly exceeded ). The lack of success in meeting outcome indicators does not point to a failure of the project, but rather to a combination of the sorts of unexpected circumstances that arise in fast-changing disaster relief situations . 5. Efficiency (not applicable to DPLs): Since this was a rapid response grant, calculation of an ERR was not required . However, calculations of the per-unit costs were made. The RSP experienced significant cost overruns in the per -unit costs as compared with the projections during project preparation . The realized per-unit cost for CHF was 37% greater than the anticipated per-unit cost (US$446 vs.US$297) and for IOM was 150% greater than the anticipated per -unit cost (US$658 vs.US$266). The increased costs were largely a result of the loss of economies of scale when the project was downsized. The materials that had already been procured and were no longer needed were passed on to the existing World Bank Urban Poverty Project's (UPP) housing program. The contribution of the project in permitting earthquake affected persons to resume their lives is quite high . The internal rate of return for thousands of children returning to school, beneficiaries returning to work and social well -being is impossible to calculate, but it is surely substantial. ERR )/Financial Rate of Return (FRR) a. If available, enter the Economic Rate of Return (ERR) FRR ) at appraisal and the re- re -estimated value at evaluation : Rate Available? Point Value Coverage/Scope* Appraisal % % ICR estimate % % * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome: Although the project's relevance and efficiency decreased in the course of the project, it was able to complement the outstandingly successful GoI permanent housing reconstruction program by bridging the shelter gap between emergency shelter and permanent housing for those thousands of families that would have otherwise had to remain in tents or other sub-standard dwellings throughout the rainy season . It is also a common phenomenon in large scale relief operations, that the “last 10%� of beneficiaries to be reached in need of support are harder to reach, more scattered and more problematic to identify and this was exactly the population targeted by the project . This type of intervention increases the time and costs involved compared to early stages of an intervention and contributes to higher unit costs. a. Outcome Rating : Satisfactory 7. Rationale for Risk to Development Outcome Rating: Since this project's results were by their nature supposed to be temporary in a post -disaster situation, risk to development outcome is negligible . The skills and knowledge transfer from the academic, government and donor community on earthquake resistant construction techniques should mitigate the scale of future destruction from a similar size earthquake. a. Risk to Development Outcome Rating : Negligible to Low 8. Assessment of Bank Performance: In hindsight, the need for transitional shelter was vastly over -estimated, The World Bank team relied on the disaster appraisal done by a team of international partners which estimated that 300,000 homes were destroyed or heavily damaged. They also looked at recent past experience given the lessons learnt in the response to the Aceh tsunami in 2004 where the permanent reconstruction of houses took over two years and local institutional capacity was severely disrupted . The team closely monitored the situation and were flexible once the fast -changing pace on the ground was noted.The close coordination between the World Bank and the implementing organizations in their respective field offices and head offices in Jakarta permitted early detection and appropriate solutions to issues as they arose. This close relationship resulted in the improvement of targeting, and the expansion of the program into UPP villages. at-Entry :Satisfactory a. Ensuring Quality -at- b. Quality of Supervision :Satisfactory c. Overall Bank Performance :Satisfactory 9. Assessment of Borrower Performance: GoI’s main role in the RSP was as coordinator and supervisor in the overall reconstruction program in Yogyakarta and Central Java, although its involvement was minimal due to its focus on its own permanent housing reconstruction program. The Yogyakarta Provincial Government was responsive in the reconstruction assistance and actively undertook coordination with other actors in the overall reconstruction program . The Central Java Provincial Government was slightly slow in processing the grant agreement and involved only at the district level. Both implementing agencies provided good quality roof structures under difficult circumstances with high beneficiary satisfaction. The prefabrication process, training, collaboration and coordination with other stakeholders were all conducted in a satisfactory manner . This process was accelerated by the local institutional capacity and community participation . a. Government Performance :Satisfactory b. Implementing Agency Performance :Satisfactory c. Overall Borrower Performance :Satisfactory 10. M&E Design, Implementation, & Utilization: Overall the M&E design of the RSP and its implementation and utilization from the Bank appears to be good, although there was not much written about it in the ICR and so this determination is based on this limited information . The indicators in the Results Framework were appropriate and well chosen, and were filled in both for the Mid -Term Review and upon completion for both implementing agencies . Further detail about implementation experience was given during a telephone interview with the Task Manager . The World Bank’s field office monitored key factors during implementation and played a major role in the timely and creative responses to issues facing the project . A Beneficiary Survey was completed . a. M&E Quality Rating : Substantial 11. Other Issues (Safeguards, Fiduciary, Unintended Positive and Negative Impacts): There were no safeguard issues identified . A significant factor affecting the occupancy rate (ranging from 24-81% with a target of 60%) was the extent to which the roof structures could be used for purposes other than dwellings. Even at the time of the mid-term review, the task team noted substantial numbers of Roof Structures that had been converted into small shops or other business places and community meeting places . The usefulness of the Roof Structures as places of business, warehouses, animal shelters and other community functions decreased the percentage of beneficiaries that re -used the materials used in the Roof Structure to construct their permanent house. 12. 12. Ratings : ICR IEG Review Reason for Disagreement /Comments Outcome : Satisfactory Satisfactory Risk to Development Negligible to Low Negligible to Low Outcome : Bank Performance : Satisfactory Satisfactory Borrower Performance : Satisfactory Satisfactory Quality of ICR : Satisfactory NOTES: NOTES - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate . 13. Lessons: 1. Close coordination between the World Bank and the implementing agencies both in their respective field offices and head offices permits early detection and appropriate solutions to issues as they arise . In this case, this close relationship resulted in the improvement of targeting, the expansion of the program into UPP villages and an appropriate downscaling of the project before too much wastage occurred . 2. Transitional Shelter delivery is better when carried out by local people with their existing social networks/capacity. The capacity of the local communities to band together to form community work teams was well utilized by the project permitting minimal paid work in the construction phase of the project and ensuring value-for-money. 3. The Transitional Shelter approach will only be an appropriate approach if it does not have to compete with the government permanent housing program . Both the Bank, as the trustee of the grants, and the implementing agencies have to be equipped with project preparation procedures that are tailored to rapid relief requirements and mechanisms. 14. Assessment Recommended? Yes No 15. Comments on Quality of ICR: The ICR quality is rated, on balance, satisfactory but there are some weaknesses . There was no breakdown of costs by component. This information was requested of the Task Manager, who needed a few weeks to get it from the archived files so it had not been received as of the time of submission of this ICR Review . There was little evaluation of M&E, although more detail was given during a phone interview with the Task Manager . Also, the project was listed as an Emergency Response Loan (ERL), when it was actually a grant which funded emergency response activities . a.Quality of ICR Rating : Satisfactory