ICRR 13019 Report Number : ICRR13019 IEG ICR Review Independent Evaluation Group 1. Project Data: Date Posted : 09/24/2008 PROJ ID : P055823 Appraisal Actual Project Name : Second Rural US$M ): Project Costs (US$M): 40.7 34.3 Municipal Development Project Country : Nicaragua Loan/ US$M): Loan /Credit (US$M): 28.7 34.3 Sector Board : ARD Cofinancing (US$M): US$M ): 5.0 Sector (s): Sub-national government administration (76%) Central government administration (24%) Theme (s): Municipal finance (29% - P) Municipal governance and institution building (29% - P) Environmental policies and institutions (14% - S) Other environment and natural resources management (14% - S) Decentralization (14% - S) L/C Number : C3480 Board Approval Date : 03/27/2001 Partners involved : UN Capital Closing Date : 12/30/2004 12/31/2007 Development Fund Evaluator : Panel Reviewer : Group Manager : Group : Roy Gilbert John R. Heath Monika Huppi IEGSG 2. Project Objectives and Components: a. Objectives: The project development objective is to improve the performance of rural municipalities, by building their capacity in three major areas in which municipal governments can make a critical contribution to sustainable rural development and environmental management: (a) the provision of local public infrastructure and the delivery of municipal services; (b) the protection of the environment and the sustainable management of natural resources, and (c) the promotion of local economic development . b.Were the project objectives/key associated outcome targets revised during implementation? No c. Components (or Key Conditions in the case of DPLs, as appropriate): Part A, Rural Municipal Development (appraisal cost US$27.9m.; actual cost *US$21.8m. - IDA only) incl: financing of municipal sub-projects through the Municipal Development Fund (FONDEM); institutional strengthening of municipalities; institutional strengthening of the (central) Nicaraguan Institute for Municipal Development (INIFOM); and technical assistance for evaluation and policy development . Originally targeting 43 municipalities, later expanded to include all 153 municipalities in Nicaragua except the capital Managua . Part B. Municipal Environmental Management (appraisal cost US$11.2m.; actual cost *US$10.5m. - IDA only) incl. support for decentralization of environmental management; technical assistance and training to indigenous groups; environmental information system; setting up a Municipal Environmental Facility (FAM). Targeting 70 rural municipalities and their communities. Part C.** Decentralization Reform Process (appraisal cost US$0.0; actual cost *US$2.0m. - IDA only) incl. technical assistance to strengthen ministries and public agencies; and hurricane Felix emergency assistance . [* "actual" cost figures include only that part financed by IDA, as reported in the ICR **Added as a new project component through a formal amendment to the Development Credit Agreement (DCA) in December 2004 in support of the government's Municipal Transfers Law of 2003 and other municipal reforms.] d. Comments on Project Cost, Financing, Borrower Contribution, and Dates: As reported in the ICR. actual cost figures summarized here include only IDA expenditures and do not correspond to the full amounts. At appraisal, IDA was to have financed 71 percent of the total project costs, the Borrower 7 percent, Local Governments 10 percent and the UN Capital Development Fund 12 percent. The actual shares at completion are not known, although the Region later informed IEG that it is it working with the Borrower and implementing agency to obtain these numbers . Even without presenting the figures the ICR concludes that a shortage of counterpart funding was an issue early on in implementation, but not so later on . For its part, the Borrower ICR refers to a lack of counterpart funds in 2006, however. At the time of the original project closing date in December 2004, the DCA was amended but project objectives remained unchanged . The amendment added Part C and restructuring Part A to replace FONDEM's financing mechanism with Nicaragua's new revenue sharing arrangements introduced by the 2003 Municipal Transfers Law. The amendment also extended the closing date by 18 months to June 2006. Another 18 month extension to December 2007 followed. According to the ICR, these extensions were to allow complementary actions to support the effective implementation of the new revenue sharing mechanism. 3. Relevance of Objectives & Design: The project objectives are substantially relevant to government and Bank priorities reflected in the 2007 Country Partnership Strategy (CPS), which calls for modernizing state institutions (including municipalities) and promoting citizen participation by strengthening governance and accountability . Project objectives are also consistent with the emphasis upon decentralization of the latest 2005-2009 Poverty Reduction Strategy Paper . The project design was substantially relevant, for including investment and technical assistance components with performance -based financing criteria as an incentive for municipal capacity building . 4. Achievement of Objectives (Efficacy): (a) greater municipal capacity for providing infrastructure : substantially achieved . 547 infrastructure sub-projects in roads, markets, bus terminals and solid waste facilities, were financed through FONDEM in the original 43 participating municipalities. Municipal drainage works contributed to flood prevention in several municipalities . However, the ICR does not provide data on the coverage of these services before and after the project at the municipal level. Municipal planning for infrastructure provision was strengthened through the preparation and implementation of municipal development plans . Project training in financial management for staff from all municipalities also helped better prepare them for assessing the costs and financing of municipal infrastructure investments. Although not a guarantee in itself, the project's success in increasing municipalities' revenues from US$9.5 million to US$17.2 million (above the target of US$12.5 million) meant that municipalities have greater potential to finance infrastructure. Deconcentration of INIFOM staff helped strengthen the human resources of local government for providing better infrastructure and services . (b) greater municipal capacity for environmental management : substantially achieved . 70 municipalities passed 317 municipal laws relating to environmental standards . 43 municipalities prepared environmental plans and 39 set up special environmental units. The appraisal target had been just 24 municipalities (60 percent of the original 40 participating municipalities). The project helped set up Municipal Environmental Units (UAM) in 140 municipalities. For the first time in Nicaragua, at least 10 municipalities directly managed protected areas within their jurisdictions, something that had previously been an exclusive responsibility of the central government . The Ministry of the Environment (MARENA) nevertheless increased its support to municipalities through setting up its own permanent regional delegations throughout the country for the first time . The project successfully increased community involvement and participation in municipal decision making about environmental issues especially . All 153 urban and rural municipalities in Nicaragua (with the exception of the capital Managua ) prepared five year plans with the participation of communities, nearly fives time the original target of 32 rural municipalities. The Environmental Information System (SINIA) created under the project has proved to be of great interest to NGOs, government departments, the media, universities and municipalities themselves, all of whom are look to the project for evidence of a greater municipal capacity in this area . (c) greater municipal capacity for promoting local economic development : modestly achieved . Municipalities' gains in revenue transfers were significant . While this may give them the potential to promote local economic development, it is not a sufficient condition by itself . In this project, as in general, better performance on the revenue side is no guarantee of better performance on the expenditure side . Also, to have more capacity for promoting economic development, municipalities would have to demonstrate that they can deploy knowledge and programs in this area, something that the ICR does not report . The creation and continued operation of the Ministry of Finance's website, TRANSMUNI, can act as an incentive for municipalities to increase this knowledge and improve their performance in supporting local economic development so that their good results are disseminated, attracting productive businesses to their areas. 5. Efficiency (not applicable to DPLs): ERRs were estimated for a stratified sample of subprojects in the main regions covered by the project . The ICR does not say whether the sample was randomly selected or not within the strata . Across sub-projects, ERR estimates varied considerably with the highest values obtained by road sub -projects and also for sub-projects of different sectors that were financed under the post -FONDEM arrangements. Cost figures for the individual sub -projects evaluated for ERRs are more likely to be include all costs (unlike cost figures for the project as a whole ). The ICR reports that sub-project costs were specifically re -estimated for the purposes of the cost -benefit analysis. ERR )/Financial Rate of Return (FRR) a. If available, enter the Economic Rate of Return (ERR) FRR ) at appraisal and the re- re -estimated value at evaluation : Rate Available? Point Value Coverage/Scope* Appraisal Yes 24% 6% ICR estimate Yes 34.86% 0% * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome: The project achieved its relevant objectives, particularly increasing municipal capacity to provide infrastructure, legislate on the environment. But there were moderate shortcomings arising from the thin evidence of the institutional aspect of municipal capacity strengthening specifically in those areas that the project intended, especially in promoting local economic development . Indicators of generic improvements, such as increased revenues and greater coverage of municipal audits do not by themselves constitute convincing evidence of achievements in the areas cited by the project objectives . The ICR reports other positive outcomes of the project unrelated to its formal objectives, in particular improved living conditions resulting from project infrastructure and services . a. Outcome Rating : Moderately Satisfactory 7. Rationale for Risk to Development Outcome Rating: The 2003 Municipal Transfers Law in particular can be expected to assure the transparent provision of resources to municipalities into the future. This sound legislative basis for continuing support to municipalities in Nicaragua reduces the risk to the development outcome . As the principal central government agency for dealing with municipalities, INIFOM looks set to continue receiving political support . However, continuing activities and municipal investment will still depend to some extent upon external funding into the future, the level of which is far from certain . a. Risk to Development Outcome Rating : Moderate 8. Assessment of Bank Performance: Overall, Bank Performance was good . While the design of the project was generally sound, the Bank did tend to overestimate the ability of INIFOM to implement the project . In hindsight, the design would have been stronger had it required INIFOM to contract out more technical services to private suppliers . With some foresight, the project design incorporated elements of an impact evaluation process to help evaluate the operation . While a good idea in itself, the management and data requirements of the proposed impact evaluation proved burdensome and impossible to implement given the local capacity constraints . But during implementation, the Bank responded creatively and quickly to the new opportunities presented by the new 2003 Municipal Transfers Law and other reforms, by launching two project restructuring missions shortly thereafter . Supervision paid close attention to helping project stakeholders adapt to the new circumstances and ensure that project financial allocations were made in a timely fashion . at -Entry :Satisfactory a. Ensuring Quality -at- b. Quality of Supervision :Satisfactory c. Overall Bank Performance :Satisfactory 9. Assessment of Borrower Performance: The Borrower introduced four major pieces of legislation, constituting a municipal reform program and strategy that the project was able to support, namely : the Municipal Transfer Law; the Municipal Financial Administration, Participatory Budgeting, and Solvency Law; the Municipal Civil Servants Career Law; and the Municipal Cadastre Law. INIFOM played an important advocacy role in promoting these laws and was strong on delivering technical assistance to municipalities. But INIFOM had to spread its capacity thinly among its competing advocacy and technical roles, as well as its responsibility as implementing agency for the project itself . Borrower performance weakened around mid-term when project management was rated unsatisfactory by supervision missions, following procurement inconsistencies by INIFOM and MARENA . Following technical assistance and close supervision by the Bank's Managua office, there was some improvement in the performance of these agencies but it was still found that INIFOM had overspent by more US$ 1.0 million through undocumented expenditures made through the project's Special Account . INIFOM's effectiveness was hampered by frequent changes of leadership that were determined by the government . On the other hand, the Borrower ICR gives a more positive opinion about INIFOM, arguing that the agency became a more efficient and effective institution measured by its greater decentralization and share of professional staff thanks to support received from the project . a. Government Performance :Satisfactory b. Implementing Agency Performance :Moderately Satisfactory c. Overall Borrower Performance :Moderately Satisfactory 10. M&E Design, Implementation, & Utilization: Initial M&E design was complex and onerous, especially in needlessly requiring repeat surveys of municipal officials and other stakeholders . The M&E design also relied upon data collection, processing and analysis that was beyond the capacity of INIFOM. Implementation of key aspects of M&E related to municipal finance and investment planning went well, however . Information collected was maintained in the project's web -based system called TRANSMUNI, making it transparent and accessible to users . The data was utilized to highlight where more project capacity building efforts needed to be concentrated among different municipalities. The project had specifically intended to establish and operate policy evaluation mechanisms, but enjoyed little success in so doing . The frequent turnover of INIFOM leadership meant that continuous and close attention needed for this work was not forthcoming from the highest level . The ICR reports a long list of the results of 34 indicators, but not all of them can be used to evaluate the project . For instance, there are cases of target data expressed in shares of something (say the proportion of municipalities involving communities in budgeting ) while actual results data is expressed in absolute values (say the number of municipalities achieving the desired result ). Thus the actual result cannot be compared with the target . Also, several indicators are presented as measures of greater municipal capacity, when they illustrate something else, such as greater financial revenues, for example . a. M&E Quality Rating : Modest 11. Other Issues (Safeguards, Fiduciary, Unintended Positive and Negative Impacts): The project's Operational Manual restricted the financing of new project roads in areas with fragile ecosystems . While safeguards affecting indigenous peoples ensured that the project avoided areas with large populations of this kind, a dispute between MARENA and the Sutiava indigenous group who complained that their planned participation in the selection of sub-projects in their area had not taken place . The components they wanted to include --particularly mangrove reforestation and black shell cultivation were subsequently financed by the Millennium Account, however . 12. 12. Ratings : ICR IEG Review Reason for Disagreement /Comments Outcome : Moderately Moderately Satisfactory Satisfactory Risk to Development Moderate Moderate Outcome : Bank Performance : Satisfactory Satisfactory Borrower Performance : Satisfactory Moderately Despite progress made under the Satisfactory project, implementing agency performance still shows some weaknesses. Quality of ICR : Satisfactory NOTES: NOTES - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate . 13. Lessons: If an organization as project implementation agency does not have basic institutional capacity and commitment at the outset, it can prove very difficult to build its capacity further . This suggests that alternative implementation arrangements need to be considered in such circumstances . Project investment can bolster policy reform when it responds to incentives of the reform itself and applies long standing technical assistance and training programs designed to provide the expertise that municipalities need to handle the new requirements of the reform . An "add-on" component, such as environmental management in this case, can be successfully incorporated into a multi-sectoral project when it is clear that the new component contributes to achieving that project's objectives, in addition to any sector -specific goal it might also serve . Municipal Associations can play an important role in the success of municipal development projects when they provide continuity in supporting key project aims at times of turnover of municipal leaders and staff . 14. Assessment Recommended? Yes No Why? To learn more about the transition of a project that initially only supported rural municipalities to become one that covered all rural and urban municipalities in the country (except Managua). Also to find out what the final project costs were in order to be able to accurately assess project efficiency in achieving its objectives, and to review accountability. 15. Comments on Quality of ICR: Overall, this is a satisfactory ICR, but it has some shortcomings . It makes clear reference to the learning from previous project experiences incorporated into the design and implementation of this operation . It also contains a lot of information about the implementation of the project . While the project very successfully transitioned through the major municipal reforms adopted in Nicaragua in 2003, the ICR could have provided more information about how this was done. It could also have explained how some performance targets were surpassed ten -fold in some cases. A shortcoming of the report that weakens the accountability function of the evaluation, is the lack of data for the final costs of the project. The ICR does not give a clear explanation of why the data on counterpart funding is not provided. The ICR nevertheless provides opinions about the performance of counterpart funding . Without final cost data, evaluation findings concerning the efficiency of achieving the results obtained must remain questionable . The ICR also provides few cost breakdowns, such as how the US$ 10 million on technical assistance for the municipal environmental management component were spent . Nor is there any information about how much the project spent in each participating municipality . Furthermore, the evaluation would have been tighter and more convincing if the ICR had covered fewer performance indicators . Several of the 34 indicators refer to variables unrelated to the project objectives. The more relevant ones only could have been treated in the main text of the report . Some indicators report baseline, target and actual data on a different basis, making the before and after project observations incomparable. While the ICR has put a lot of thought into the lessons, it could have highlighted more clearly the single main point of each one. The present complexity of the lessons, involving several ideas within each one can make it difficult for a reader to unravel the evaluation point being made . Finally, this is a long report that would have been more succinct had it focused upon a fewer relevant performance indicators and eliminated repetition of some material. a.Quality of ICR Rating : Satisfactory