ICRR 13709 Report Number : ICRR13709 IEG ICR Review Independent Evaluation Group 1. Project Data: Date Posted : 06/30/2014 Country : Niger Project ID : P065991 Appraisal Actual Project Name : Community Action US$M ): Project Costs (US$M): 43.83 52.41 Program L/C Number : CH025; CH346 Loan/ US$M ): Loan /Credit (US$M): 35.00 43.50 Sector Board : Social Development US$M): Cofinancing (US$M ): 4.50 4.18 Cofinanciers : GEF Board Approval Date : 03/20/2003 Closing Date : 06/30/2007 12/31/2010 Sector (s): Sub-national government administration (35%); General agriculture fishing and forestry sector (20%); General water sanitation and flood protection sector (20%); General education sector (15%); Health (10%) Theme (s): Decentralization (23% - P); Rural services and infrastructure (22% - P); Participation and civic engagement (22% - P); Other social protection and risk management (22% - P); Poverty strategy; analysis and monitoring (11% - S) Prepared by : Reviewed by : ICR Review Group : Coordinator : Elaine Wee-Ling Ooi George T. K. Pitman Christopher David IEGPS1 Nelson 2. Project Objectives and Components: a. Objectives: The project was the first phase of a four -phase 12-year Adaptable Program Loan: the Community Action Program (CAP). The program aimed at sustained poverty reduction and improved local governance through empowering communities and local governments, improving natural resource and ecosystem management, and raising levels of health, education, and food security, thereby stimulating economic growth (page 3, PAD). The Project Appraisal Document (PAD, p. 4) states that the development objectives of this first phase project were : to establish and operationalize decentralized, participatory, and transparent financing mechanisms that "to empower poor communities to take charge of their own development, with the support of their local governments ." In addition the PAD separately states (PAD, p.5) the Global Environmental Objective as : "to promote community -based integrated ecosystem management as a means of reducing the region's vulnerability to desertification and poverty, while fostering multiple global benefits ." The Development Grant Agreement (DGA p.18) stated the objectives of the project were to : (i) to design and implement decentralized, participatory and transparent financing mechanisms that enable "( Communities and Local Governments to carry out their own development plans; and ii) promote (ii) Community -based Integrated Ecosystem Management (CBIEM) CBIEM ), and foster multiple global environmental benefits ." The DGA's statement of objectives is the basis for this Review . The second-phase project in the CAP APL series became effective in December 2008 and overlapped with the first phase project by two years . b.Were the project objectives/key associated outcome targets revised during implementation? No c. Components: While there were no changes to the project objectives an additional component, protection against Avian Flu, was added in 2008. 1) Community Support (IDA Planned US$3.1 million, Actual US$0.98 million; Planned GEF US$0.50 million, Actual US$0.29 million). This aimed at introducing decentralized and participatory planning procedures by building capacity of community-based organizations (CBOs), and maximizing the capacity of existing local indigenous institutions to design, implement, and manage micro-projects. It was to be implemented in all eight regions and cover about 15-20% of the total population, and included all villages or village groupings within a selected sample of communes. The development plans agreed within the CBOs were to be implemented by the communities themselves under the leadership of committees created for this purpose. The GEF resources were to support (a) CBOs to gain an understanding of the linkages between poverty reduction and integrated management of the natural resources; and (b) to be used in the identification and inclusion of primary stakeholders within communities having interest in the maintenance of the various type of production functions of ecosystems . Stakeholders were: sedentary and non-sedentary livestock holders; croppers; collectors of household products such as food, water and energy; fishermen; hunters and traditional healers and nature conservationists . GEF funds were also to support communities to design and implement relevant micro-projects related to community-based integrated ecosystem management . 2) Local Governance Support (IDA Planned: US$3.15 million, Actual US$3.46 million: GEF Planned US$0.40 million; Actual US$0.22 million). This aimed at strengthening administrative and fiscal dimensions of integrated local development. It was intended that decentralization in this first -phase project would be modest and experimental and that it would be scaled-up in subsequent phases based on successful models developed . This involved local and central levels within three sub -components: Local governance. Working with a selected set of local governments within each of the eight regions, the project was to build their capacity in participatory preparation of integrated local development and ecosystem plans, and in the administrative, fiscal and technical management required to support the effective execution of these plans, by ensuring that they work effectively with their constituents and local indigenous institutions . This set of new governments will be selected from the communes involved in the first phase of the CAP . The goal of this component was to increase capacity such that these governments "graduate" and are able to assume the fiscal and administrative responsibilities performed by the project . Policy and institutional reforms: working with central government, notably the High Commission for Administrative Reform and Decentralization (HCRAD), the project was to assist acceleration of the process of decentralization through provision of studies and support for a communication program that helps enforce and publicize the legal and regulatory framework for decentralization, while strengthening the capacity of central Ministries responsible for decentralization . Working with line ministries, the project was to assist in planning for reallocating centrally- and regionally-based staff and resources to the lowest possible level of local government . Since use and tenure rights over natural resources remain particularly important in the context of decentralization, the GEF funds was be earmarked to support the Secretariat of the Rural Code in the development of national and local environmental governance policies and frameworks for promoting community-based integrated ecosystem management . It was to do this by discouraging cultivation of marginal lands and banks of surface waters, and by resolving conflicting interests and use of surface waters . This component was to be technically supported through the United Nations Capital Development Fund (UNCDF). 3) Local Investment Fund (LIF) LIF ) (IDA Planned: US$24.6 million, Actual US$19.55 million; GEF Planned US$2.10 million, Actual US$1.99 million). This aimed to channel small fungible capital grants to communities and local government for the financing of micro -projects. Communities could utilize the funds for any micro -project it aligns with the local integrated development plan . Micro-projects could include, but were not limited to, natural resource management activities (soil fertility control, erosion control, tree plantings and nurseries, fuel wood ); and integrated micro-projects such as: production of crops, fish, and livestock (irrigation, gardening, seedling production, livestock fattening, cereal banks, fish farming, food processing, stockraising, beekeeping ), water and sanitation (wells, boreholes); education (village schools, literacy programs); health (clinics, health posts, HIV/AIDS and other specific disease prevention ); rural transport (rehabilitating local roads, small works ); etc. The LIF was a matching grant and communities and eligible local authorities are expected to contribute a certain percentage of the value of the investment, either in cash or kind . The amount of the initial grant was to be small, in the majority of cases from $2.0 - $2.8 per inhabitant. Communities and eligible local authorities that effectively accessed and executed the LIF would be eligible in subsequent financing rounds for additional funding . In contrast, those who fail to respect the terms of the grant would be excluded for a certain period . Half of the available GEF funds will be blended with the IDA funds in this component . To promote and support the global environmental interests activities the community contribution would be lower for certain types of micro-project, including (i) community awareness raising activities relating to the interest in integrated ecosystem management; (ii) the maintenance of the productivity, biodiversity and soil cover of the natural rangelands, including the preservation and /or recovery of marginal lands and banks of surface waters, (iii) the introduction of valuable native species of the natural rangelands within the cropping systems; and (iv) allocation of cultivated areas to fodder production by native fodder species . 4) Poverty and Environmental Monitoring (IDA Planned: US$3.44 million, Actual US$2.83 million; GEF Planned:0.50 million, Actual 0.60 million). This had three aims: 1. To measure, in the context of the PRSP, national levels and trends of poverty; to establish baselines and to monitor national trends in the quality of status of Niger's ecosystems and its management of natural resources, and to monitor trends in community access to social and economic services . Rather than generating new data, the project was to integrate several disparate data sets on socio -economic and environmental issues into a spatial geographic information system (GIS). GEF funds were to be used to ensure that this monitoring system explicitly incorporated socio -economic and natural resources data relating to the use and condition of the existing ecosystems in their multiple function seen from a local, national and regional /global perspective. The coordinators of the data system would have two initial objectives : (i) to work with Niger's research institutes, government statistical bodies and NGOs to create the initial nationwide integrated spatial assessment and decentralised mapping of poverty and the condition of the natural resources, and (ii) to develop a strategy for sharing, using and maintaining data within Niger, and making harmonized data sets as necessary . The sub-component would finance a small team (two or three people), some hardware and software, training, and consultants. 2. To monitor the CAP in order to provide timely feedback to both communities and program management in terms of relevance, efficiency, effectiveness and impact of program interventions, in order that rapid corrective action can be taken if necessary; and 3. To strengthen local communities' capacity to analyze and manage their own integrated local development and ecosystem planning process, a community development agent was to support a community -chosen Monitoring and Evaluation Committee . The Committee was to define the indicators that will be used to : (i) monitor micro-projects; (ii) from a locally elaborated baseline evaluate the effect of the Program and, potentially, that other local service providers to the community ; and (iii) systematically feed these observations into regular re-adjustments of the local development plan . The Committee was to be responsible for data collection and for the primary data analysis. 4. 5) Project Managemen t (IDA Planned US$4.21 million, Actual US$9.59 million; GEF Planned US$0.50 million, Actual US$0.90 million). This was to support staffing, project MIS systems, office equipment and transportation 6) Avian Flu (Planned $4.5 million; Actual $4.15 million). This was added in 2008 to strengthen veterinary and public health diagnostic, surveillance, control and outbreak containment capabilities against the Avian flu . It had four subcomponents: support for national veterinary services; human health services; improving public awareness and communications campaigns; and strengthening M&E of highly pathogenic avian influenza related activities, including with respect to environmental and social matters . d. Comments on Project Cost, Financing, Borrower Contribution, and Dates: Project Cost : Final project cost was US$52.41 million or 20% more than initially planned primarily because of additional funding for Avian Flu (US$4.23 million), and an increase in project funds available due to exchange rate variations. There were substantial departures from planned project costs . Most notably, expenditure for project management was 234% of the budgeted amount of the CAP, and 180% more than planned for the GEF activities . In contrast only 26% of the amount budgeted for community support (component A) was spent. Similarly, only 58% of the budget for GEF community support, and 55% of the budget for local governance support was spent . Other components typically had variations around 20% in budget vs actual costs . Even though the US$ equivalent amount of the original IDA Grant increased, the ICR does not give any account of how the additional US$ 4.35 million was distributed over project components . Financing : Total project financing at US$ 43.50 million was 24% more than the appraised amount. An IDA Grant for the Debt Vulnerable was US$ 35.00 million and by project closing this had increased to US$ 39.40 million because of exchange rate variations between SDR and the US$, and by closing US$ 39.35 million had been disbursed and US$ 45,864 was cancelled. Additional Grant financing from the Avian Flu Trust Fund was US$ 4.50 million, and at project closing US$4.23 million had been disbursed and US$ 0.27 million was cancelled. A GEF Grant of US$ 4.00 million was fully disbursed. Borrower : As agreed at appraisal, the government provided US$ 2.83 million and local communities provided $ 2.00 million. Dates : The project was originally expected to close on June 2007. A Midterm Review was carried out on March 2006. Implementation delays and country conditions (drought, change of government ) and the planned inclusion of the Avian Flu component in February 2008 led to a level 2 restructuring and the closing date was extended to June 2010. A military coup on February 18, 2010 led the Bank to invoke suspension under OP 7.30 and disbursement resumed on April 30, 2010. Because of the delays, an extension of the closing date by six months to December 2010 was agreed to enable completion of all activities including the Avian Flu component . 3. Relevance of Objectives & Design: a. Relevance of Objectives: High The decentralization objective was highly relevant at appraisal and remains so . A major obstacle to implementing an effective community development strategy in Niger was the scarcity and management of public resources. Decision-making powers, staff, and financial resources were concentrated at the center and about three quarters of recurrent expenditure in line ministries were devoted to personnel salaries and allowances, with much of the remainder to utility costs, leaving very little for recurrent expenditures . Given that in 2012 over 80% of Niger's population still lived in rural areas and relied on natural resources for their livelihoods, reaching beyond the center and fostering grassroots development was and remains an important policy objective . Project objectives were relevant to Niger's 2002 Poverty Reduction Strategy Paper (PRSP) that considered rural development the cornerstone of a poverty reduction strategy in view of the fact that rainfed agriculture provides the vast majority of employment, food, and income for Nigeriens . The main objective was particularly relevant to the the PRSP's fourth pillar "strengthening institutional and individual capacity with and outside government, at the central and local level." Improving environmental management and stewardship underpins poverty reduction efforts in Niger . At the time of appraisal, 84% of men and 97% of women were involved in growing crops or raising livestock . Even so, the yields of the staple crops - millet and sorghum - were low and declining. Pastoralism is also very important, but the national herd had suffered tremendously from recurring droughts and the decreased availability, declining condition and reduced accessibility of grazing and water areas during the dry season . Apart from low and variable farm incomes, the combination of low yields, scarce water, declining soil fertility, and inadequate marketing infrastructure were the main reasons for widespread food insecurity . In 2012 more than half the population was affected by food and nutrition insecurity . The most recent PRSP - the Plan for Economic and Social Development 2012-2015 - reaffirms relevance through its four axes : (i) creation of conditions conducive to sustainable, equitable and inclusive development; (ii) food security and sustainable agricultural development; (iii) promotion of a competitive and diversified economy; and (iv) promotion of social development. Project objectives were and remain relevant to the Bank's country strategies . The Country Assistance Strategy (CAS) for FY03-05 fully endorsed the PRSP and adopted its four pillars . The FY08-11 CAS objectives followed a similar approach but were organized under two pillars : i) accelerating sustainable growth that is equitably shared by maintaining macroeconomic stability and strengthening competitiveness, sustainable management of natural resources, the investment climate, and economic infrastructure; and ii ) developing human capital through equal access to quality social services and improving preparedness to deal with natural disasters . Population growth and Governance were cross -cutting issues affecting both pillars . Project objectives were also aligned with the objectives of the Bank’s Strategy for Africa that emphasizes enhancing competitiveness and resilience through improved natural resource and environment management . b. Relevance of Design: Substantial At appraisal the project did not have a results framework but had a satisfactory logical framework in accordance with World Bank guidelines at that time . The objectives of the program, the causal relationships between different interventions and underlying assumptions about how program actions would lead to intended outcomes were logical and properly laid out . Project design was based on a series of pilot studies assisted by the African Development Foundation, the UN Capital Development Fund, and the European Union . Design gave inadequate attention to developing guidelines demarcating locations to be assisted and how the project could work with other donor programs to provide complementary services and avoid duplication . However, it could have paid more attention to the logistics of mobilizing government and NGOs to remote areas, and the means to ensure the timely production of the multitude of manuals and guidelines required . The participatory appraisal techniques to be used to establish the problems that poor communities face, to discuss solutions, and to plan project investment were relevant and essential to bolster local self -determination. This process was facilitated by NGOs or other facilitators that assisted communities to (i) conduct needs assessments, (ii) draft local development plans, and (iii) facilitate implementation of micro-projects. While giving communities the ability to exercise choice over source of technical assistance, technology type, and investment design, design did not give enough attention to incentives, particularly for environmental micro -projects. The scope of the proposed monitoring and evaluation encompassing a national poverty monitoring system and community-level one was ambitious given the well -known lack of local capacity and institutional limitations present in Niger; it would have been better to focus on the M&E system for the project's activities . As it was, the indicators, particularly for the environmental component, were poorly designed . The original indicators (level of degradation of land and water resources, changes in land cover, household energy use, harvesting of medicinal products and biodiversity loss ) were too complex, poorly defined and not easily measurable . Consequently they had to be redesigned at mid-term review. The Avian flu component was only modestly relevant as it had nothing to do with decentralization or community development. Even so, it was well designed. Its emphasis on enhancing prevention and preparedness capacity, strengthening disease surveillance and diagnostic capacity, and control and containment plans, along with parallel support for human health services and surveillance system and awareness raising, was highly relevant . 4. Achievement of Objectives (Efficacy): (i) to design and implement decentralized, participatory and transparent The project's statement of objectives "( financing mechanisms that enable Communities and Local Governments to carry out their own development ii) promote Community -based Integrated Ecosystem Management (CBIEM) plans; and (ii) CBIEM ), and foster multiple global environmental benefits " contains five objectives: (a) to design and implement decentralized financing mechanisms that enable Communities and Local Governments to carry out their own development plans; (b) to design and implement participatory financing mechanisms that enable Communities and Local Governments to carry out their own development plans; (c) to design and implement transparent financing mechanisms that enable Communities and Local Governments to carry out their own development plans; (d) promote community-based integrated ecosystem management; and (e) foster multiple global environmental benefits . The achievement of each objective is discussed below . Not all the outcomes can be attributed to the first -phase CAP because there were many other donor -funded operations, including 5 Bank projects that included or had impacts on the rural sector. All the first-phase CAP outcomes and outputs are only reported to 2007 when that phase of the CAP ended. This was because the ICR was only prepared in June 2011 because project closing had been extended to 2010 for the Avian Flu component. (a) to design and implement decentralized financing mechanisms that enable Communities and Local Governments to carry out their own development plans : Substantial Outcomes US$21.54 million, or 90% of the Local Investment Fund was disbursed to rural areas for 1,546 community and village development micro-projects. These micro-projects derived from 675 community and local government micro-plans (target 540) and their ability to mange several rounds of successive projects after meeting performance standards: 70% of 54 communes implemented at least 3 micro-projects (target 60%); and 75% of communities have implemented at least 5 micro-projects (target 60%). The project helped establish and reinforce decentralization mechanisms as planned for 20% of the country’s newly established communes, laying the groundwork and architecture for decentralized local development . At the central level the project helped the government in preparation of policies (3 out of 4 planned decentralization laws and decrees were prepared ). The CAP also supported the establishment of the High Commission on Modernization of the State to channel finances to the decentralized communes . The ICR reported the government was committed to the decentralization process, and many sectoral ministries (at least 12) actively provided technical assistance and oversight in the design and implementation of the microprojects . Stakeholder feedback was generally positive, indicating the financing mechanisms were working as intended, although there were delays in counterpart financing especially in the early years . Outputs Local governance was improved in 54 communes (target 54) and all received technical assistance through the project: A financing preparation guideline document was prepared and disseminated . 1,134 persons (target 787) in commune organs were trained in administration and management . 1,051 persons (target 787) were trained in budget management. 1,179 persons (target 787) were trained in fiscal management. 274 persons (target 270) were trained in procurement. 185 persons (target 270) were trained in M&E. All 54 communes prepared and validated community development plans . 675 micro-plans (target 540) for communes were prepared and these plans enabled 1,546 community projects to be implemented: Production, processing and trading - 621 projects or 43% of all projects Education - 302 projects or 19% Natural resources management - 252 projects or 16% Water and purification - 182 projects or 12% Health - 105 projects or 7%. The remaining projects were in other infrastructure and transport . The ICR provides no information about the developmental value /outcome of the microprojects, i.e. whether they have led to income improvement, increased access to schooling, cleaner potable water etc .,. Local governance was improved at the village level and 178 self-organized village clusters (target 150) were established and technically supported : 534 management organs in the village clusters were established and trained meeting the target . 1,167 persons (target 890) were trained to be members of procurement committees . 170 local development plans (target 150), and 2,934 micro-project plans (target 2,000), were prepared for villages. b) to design and implement participatory financing mechanisms that enable Communities and Local Governments to carry out their own development plans : Substantial Outcomes Participating communes and villages contributed 10% of micro-project funding. This added US$2.00 million to the project as planned. The target of providing 25% of productive micro-projects to participating women’s groups was reached, and the target of having women account for one -third of the members of elected bodies at communes, communities, and micro-project committee was reached in many places, especially where women were active enough to present themselves for elections. Where women were traditionally more withdrawn and less educated, this target was not attained. However, participation may have favored men over women . The ICR (p.63) reports that even women who have official roles could not confirm to the [ICR] study team that they are effectively involved or consulted when important decisions are taken or when priority activities are chosen into the Local Development Plans or Commune Development Plans. Outputs Participation was improved in 54 communes (target 54) and all prepared communal plans 450 persons (target 270) in commune organs were trained in civil society activities . 1,202 persons (target 787) were trained in communal planning. 229 persons (no target) were informed and sensitized to cooperative laws . As planned, 5 savings and credit cooperatives from the earlier natural resources management project were reactivated. All 54 communes prepared and validated community development plans . 675 micro-plans (target 540) for communes were prepared and agreed . Local participation was improved at the village level and 178 self-organized village clusters (target 150) were established and technically supported : 534 persons (target 534) were trained in communal management. 5,609 persons (target 3,750) were trained in the collaborative approach . 1,160 persons (no target) were trained to establish forums for inter -cluster meetings. As planned, 5 savings and credit cooperatives from the earlier natural resources management project were reactivated. 170 local development plans (target 150), and 2,934 micro-project plans (target 2,000), were prepared and agreed for villages. (c) to design and implement transparent financing mechanisms that enable Communities and Local Governments to carry out their own development plans : Modest Outcomes : The ICR provides no information about the transparency of the financing mechanisms nor are there any indicato rs. The Task Team responded " the project had procedures in place to ensure transparency at all stages of the financing process, i.e. (i) in the allocation of resources, (ii) in the execution of activities, and (iii) in results reporting." While procedures were said to be in place there is still no evidence that demonstrates the financing mechanism were transparent. The substantial level of stakeholder participation in development of communal and village plans would indicate that some of the process was transparent . The ICR (p.73) notes, for example, that competition has allowed the communes to choose the best and least expensive NGOs to carry out the field activities . Even so, the ICR (p.14) also notes that the project coordination unit did not collect separate information about the amounts used for commune-level grants and community-level productive micro-projects. The Task Team in response stated that "during project implementation, grants to both communes and communities were tracked by the project’s M&E system. Details can be found in the various semester and annual reports of the project ." Outputs : Two Commissions foncieres departmentales (target 2) and 9 Commissions foncieres communes (target 54) were established. It is not clear what impact the lack of Commissions foncieres communes in most communes had on the governance of decentralized financing . The Task Team subsequently stated that the rule that any micro-project requiring land for its installation would not be financed without an “act� provided by the departmental or communal “commissions foncières� was upheld. Not a single micro-project requiring such “act� was financed without such confirmation . Accordingly, the lack of Commissions foncieres communes did not affect some aspects of the governance of decentralized financing, although it might have caused some delays in the approval of such micro-projects. (d) promote community -based integrated ecosystem management : Substantial Outcomes : 252 or 16% of the microprojects were environmental or related to natural resources management . An economic analysis of a sample of 9 environmental protection sub-projects indicated that 78% were fully profitable and 11% were marginally profitable - these micro-projects had the highest profitability among all the 1,546 mirco-projects financed. Among the most popular activities supported were 82 micro-project sites that were planted with 7,800 ha of acacia aenegalensis trees following the establishment of 110 tree nurseries. In addition, nearly 9,000 hectares of land (including a portion of BioCarbon Fund initiatives in sub -Saharan Africa) benefited from sustainable land management technologies . Water erosion was reduced in 88% of these sites based on monitoring survey results carried out by the project . Main benefits were the value of increased wood and fodder production. Outputs 615 persons (target 787) were trained in environmental 'activities .' 117 persons (target 280) were trained in land management. (e) foster multiple global environmental benefits : Substantial Outcomes : The project helped set up the operations of one of the first BioCarbon Fund (BioCF) initiatives in sub-Saharan Africa and 7,800 ha of land (target 9,000) were recovered and acacia senegalensis planted in 2006 and 2007. Each hectare of land forested stores 31 tons of carbon once the tress are fully grown . Thus if the success rate of planting trees was 60%, the potential to store about 0.25 million tons of Carbon was achieved . The project was also involved in 7 ongoing GEF sites and successfully secured the commitment of local communities and their involvement in the protection and rational exploitation of their natural resources and rehabilitation of biodiversity in 86% of these sites. According to the Region, the project supported the construction of 2,200 improved cooking stoves that reduced the consumption of firewood by 40-50%. The ICR also indicated that "some studies" have shown an increase in land animals and birds especially in project supported reforestation and land . No evidence is presented in the ICR . Outputs : The original indicators were too aggregate, poorly defined and not measurable . They were: level of degradation of land and water resources, changes in land cover, household energy use, harvesting of medicinal products and biodiversity loss. After the MTR in 2008, they were replaced with i) area of marginal lands under crop cultivation; and ii) area of marginal land being protected or actively recovered . Progress towards the next phases of the APL Triggers for the second-phase project were: (i) 80% of communities and local stakeholders are satisfied with CAP and wish it to continue; (ii) at least two-thirds of communities targeted in the first phase received capacity-building support and have drafted local development plans; (iii) one half of the communes selected for local governance capacity -building are considered ready to assume key project management roles, in particular for fiscal management; and (iv) government, through continued implementation of reforms and the provision of financial support, has demonstrated strong support for decentralization and community -based development. The phase one project met the triggers for a second -phase project and the second phase was prepared and appraised under accelerated procedures in early 2008. Additional Financing for Avian Flu . The ICR provided a lengthy analysis on the implementation of the Avian Flu component that was introduced in 2008. While this effort has the potential to control infectious disease outbreaks, and had the means of mitigating the impoverishing effects of the flu among farmers, its efficacy could not be tested . 5. Efficiency: Modest Financial and economic efficiency The PAD estimated a financial internal rate of return (FIRR) of 36% only for a small segment of the potential revenue generating microprojects (cattle and donkey carts, and seed mills ). The ICR was not able to do a calculation based on the same microprojects because the data collection was not good enough to enable reliable cost-benefit of microprojects. Instead the ICR selected a sample of 145 micro-projects that was randomly chosen, however only 118 of these micro-projects were able to be evaluated due to difficulties reaching some of the more remote areas . These micro-projects represented 23% of the 631 revenue-generating micro-projects and 7.6% of all 1,526 micro-projects. The FIRR of the PAD and the ICR are not comparable . They found 44% to be fully profitable, (FIRR more than 10%); 16% to be marginally profitable, and 40% to be unprofitable. Only 33% of the 104 agro pastoral subprojects were profitable, compared to 74% of the 23 artisanal and 78% of the 9 environmental projects. They also found that the projects managed by women were the most profitable while those managed by communities were the least profitable . Factors of profitability include beneficiaries ’ discipline, their qualifications and rigor of village clusters’ management committees, which conduct regular monitoring activities, despite the fact that these activities rely heavily on volunteering. Externally, micro-projects’ profitability is based on the availability of local labor, local markets, growing demands, and incidentally monopoly (lack of competition). In contrast, the lack of profitability is mainly the result of weak management (to sell below purchase price, distribution of cash to the management committee members in the form of compensation, irrational use of feeds in the case of cattle fattening, etc .) by the management committees. The study found that contracts for the supply of products (cereal banks, agricultural inputs banks) were awarded without any concern for profitability and by inexperienced members who ignored procurement procedures, but also because of interference from project staff that identified local suppliers. Delays in delivery are also a factor that handicapped the profitability of these micro -projects. Finally, the ICR (p.14) notes: " the uncertainty about the project's efficiency (that is, whether the results could have been achieved less expensively ) ....could not be fully verified because of a lack of relevant information in the government's study of project results ." Administrative Efficiency Government indecision about which ministry would supervise the project delayed the project's effectiveness by 9 months. The large scope of the project required preparation of multitudes of guidelines, and training modules that consumed project resources and delayed implementation . Throughout the project there was lack of funds for some activities because of delays and failure to provide timely counterpart fund. In the first two years of the project the M&E system did not provide, apart from financial monitoring, satisfactory information to gauge implementation progre ss. According to the ICR expenditure on the government's administration and management was 234% more than the appraisal budget for the CAP and 180% more than appraisal for the GEF. The Task Team subsequently stated that the final figure for project management expenditure was 170% of the appraised amount (rather than the 234% as in indicated in the ICR). Even so this overexpenditure cannot be attributed to the Avian Flu component as all CAP and GEF activities had been completed by the end of 2007, and the Avian Flu component had its own separate management budget that was fully spent . ERR )/Financial Rate of Return (FRR) a. If available, enter the Economic Rate of Return (ERR) FRR ) at appraisal and the re- re -estimated value at evaluation : Rate Available? Point Value Coverage/Scope* Appraisal No ICR estimate No * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome: Relevance of objectives was high but that for design was substantial.The CAP achieved decentralization of and beneficiary participation in financing and that enabled communities and communes to implement local development plans, and the efficacy of these two objectives is rated substantial . However, it remains unclear how transparent that process was, and the efficacy of that objective is rated modest . Community eco-system management and global environmental benefits were substantially achieved . Efficiency is rated modest. a. Outcome Rating : Moderately Satisfactory 7. Rationale for Risk to Development Outcome Rating: Government and the donor community continue to support the decentralization efforts . Some 44% of the microprojects sampled were profitable . In addition, maintenance of constructed facilities is seen a significant problem as it is beyond the ability of local communities to manage and finance . The few environmental projects may be sustainable as they yield useful benefits such as fuel wood . While the Avian Flu achievements were considerable, without continued external donor funding, the established mechanisms (staff salaries, surveillance system and laboratory equipment ) will not be continued. a. Risk to Development Outcome Rating : Significant 8. Assessment of Bank Performance: a. Quality at entry: The Bank provided strong analytical work and financed a Project Preparation Facility in support of the Government's decentralization initiative and the project took three years to prepare . The preparatory work also included coordination and consultation with other donor partners to ensure the same approach and practices were applied in other rural development projects . The project was over-ambitious in scope given the very low level of local capacity and government resources, a finding endorsed by IEG's CASCAR Review of 2013. Many preparatory tasks (guidelines, manuals etc) that should have been undertaken during appraisal were left to implementation and this created delays . A similar problem occurred when additional financing was appraised for the emergency Avian Flu component and the project duration agreed for this new component was over-optimistic. While most risks were thoroughly appraised, preparatory studies showed the communities had a low interest in environmental projects for the common good compared to pr ojects delivering short-term returns. M&E was modest. at -Entry Rating : Quality -at- Moderately Satisfactory b. Quality of supervision: Bank supervision was regular and thorough . Bank supervision was inclusive and strong technical support was provided to the project. To promote their buy-in other donors were solicited to provide input and participate in Bank missions. The Bank had to work also under very challenging conditions - during implementation there was a locust invasion and 2 years of prolonged drought . A military coup took place in 2010 halting Bank operations and a change in government took place . Despite such circumstances project supervision continued. The MTR was proactive in addressing the shortcomings in the M&E systems and revised the indicators for the GEF objectives . The project also did a good job implementing the avian flu component which successfully set up a strong network/program of surveillance, reporting and response against the avian flu . Although this component had little to do with creating strong decentralized networks for local development, a lot of demand was made on the Bank team. The main project had an environmental category F and while an EA and EMP were prepared, no information is given in the ICR on compliance except to say that there were environmental issues early on, but not what they were or whether they were resolved . Regarding Involuntary Resettlement OP 4.12, there is no indication on the scope of the problems or whether the compensation provided was sufficient . The Region's comments also do not provide this information . While supervision, was overall well carried out, the lack of attention to reporting safeguard compliance is a significant shortcoming. Quality of Supervision Rating : Moderately Unsatisfactory Overall Bank Performance Rating : Moderately Satisfactory 9. Assessment of Borrower Performance: a. Government Performance: The government was proactive in promoting policies and legislation that were consistent with the projects objectives. It also endorsed the involvement of other donors in the design, discussion, and supervision of the program, and encouraged the use of the project's approach and mechanisms in other rural development efforts. On staffing the government retained the same experienced project staff from another Bank financed project to manage the CAP. It was also proactive in responding to the threat of the Avian Flu which could have had disastrous effect on the population and farming economy and has supported the implementation of the Avian Flu component. However, the Government indecisiveness about selecting the counterpart ministry (Ministry of Economy and Finance or the Ministry of Agricultural Development ) for the project delayed the project ’s effectiveness. The Government was constantly late also in paying the annual counterpart funds, again delaying project implementation activities. On the positive side, even after project closure, the Government has continued to pay outstanding counterpart funds for this first phase of the APL . This has enabled local project staff to continue working during this bridge period before the second phase of the APL becomes effective . Government Performance Rating Moderately Satisfactory b. Implementing Agency Performance: The implementing agency was the Program Coordinating Unit although the contributions of several ministries and other line agencies were indispensable in this project . Together they laid the foundation for Niger's decentralized governance structure; and supported the establishment and functioning of local communes and communities in 20% of the country. The Unit was quick in setting up its field network in all eight regions to support the new decentralized structure, without which the 1,546 microprojects would not have been implemented. According to the Stakeholder and Beneficiary feedback, the Unit has adhered to the "inclusive participation and consultative " tenet of the decentralization strategy, applying it to all stakeholders and donors. While the weak M&E quality was improved considerably after the midterm review, reliable cost figures are still not available for the microprojects . While this was possibly due to lack of feedback from village clusters and local communes monitoring and reporting quality, more attention should have been given to this vital activity. As a result, there was some over -commitment on micro-projects because budgets had already been fully committed and this indicates a lack of budget controls . Implementing Agency Performance Rating : Moderately Satisfactory Overall Borrower Performance Rating : Moderately Satisfactory 10. M&E Design, Implementation, & Utilization: a. M&E Design: The project had sufficient funding for M&E . US$ 3.9 million was allotted to develop and implement a Poverty and Environmental Monitoring system as well as the project M&E . Training was provided to key national agencies (Secretariat monitoring the PRSP, the Environment Ministry etc ) - and local communities and communes also had a role in data collection. Initially M&E quality was low with poorly elaborated indicators . Input indicators were sometimes used as output indicators and some output indicators were not precise of quantitative . Specifically the GEF Environmental indicators were too broad, aggregated and not well defined, and some outcome indicators for the CAP were missing (participation, transparency ). b. M&E Implementation: The M&E was originally led by an inexperienced coordinator who focused only on monitoring financial data . Two years later, implementation picked up after the Coordinator was replaced . New environmental indicators (area of marginal land under cultivation by annual crops, area of marginal land that has been protected or has been recovered) replaced the old, and baseline values were established . Villages and communes collected and reported data about the micro-projects to the regional and project coordination units, but it is unclear how effective this process was, particularly for providing cost information . In the final year of the project, 90 % of the 54 communes and 75% of the 178 village clusters were reporting to the center . A number of evaluation studies on the performance of intermediary and support organizations were conducted as well as an impact study of the use of technology in natural resources management. The ICR had little information about the functioning of the poverty monitoring system but noted that the project was “unable to fully implement the national level poverty monitoring system .� c. M&E Utilization: Halfway through the project the M&E system became an important tool for project management and the Bank ’s supervisory missions. M&E results led to adjustment and simplification of the guidelines governing the selection of micro-projects, to improve their mix and spread . The monitoring system also helped guide the project to discontinue Component A activities (developing the capacity of local NGOs and village communities in identification and implementation of microprojects) before project end because enough knowledge had been generated in the communities clusters. Even so, the M&E system did not prevent the continued commitment of funds to new microprojects even when the budget was fully utilized . M&E Quality Rating : Modest 11. Other Issues a. Safeguards: The project was classified as F - a financial intermediary assessment - under OP 4.01 (Environmental Assessment). In addition OP 4.04 (Natural Habitats), and OP4.12 (Involuntary Resettlement) were triggered. An environmental assessment and an environmental management plan were prepared before the project began, a nd a study on conservation of natural habitats was expected at appraisal . It is not clear if the latter study was undertaken. In the early stages of project implementation, some of the microprojects did not have any environmental considerations or assessments done . While an EA and EMP were prepared, no information is given in the ICR on compliance except to say that there were environmental issues early on, but not what they were or whether they were resolved . At appraisal it was not expected that there would be involuntary resettlement . Because the project scope was slightly altered towards project end, and environmental activities included larger scale schemes (water schemes, Acacia Senegal, and other indigenous plantations ) that resulted in actual movement of households, the implementing agency and the Permanent Secretariat of “Code Rural� prepared and implemented compensation rules for these cases. Regarding OP 4.12, there is no indication in the ICR on the scope of the problems or whether the compensation provided was sufficient . Regional comments also do not provide this information . The avian flu component was assigned a category B under OP /BP 4.01 because of the potential medical waste generated in health centers and laboratories, thus requiring an environmental assessment . It also triggered OP/BP 4.09 (Pest Management) because of the use of chemicals in the disinfection of carcasses and the decontamination of burning sites, and OP /BP 4.12 (Involuntary Resettlement) because of the potential need for land acquisition for burning and burial sites . During the life of the project the concerns over an Avian Flu occurrence did not materialize. No medical waste was generated, there was no decontamination or burning of animal carcasses, and thus there was no need for land acquisition and resettlement did not occur . b. Fiduciary Compliance: The project accounts, including regional sub -accounts and Special Accounts, were audited regularly . All audits were unqualified. The problems that existed were in budget management and control; that is, coordination between the financial accounts and commitments for project implementation, resulting in an over -commitment, especially in micro-project approvals. Bank missions highlighted these problems . These issues were resolved toward the end of the project with reallocation of project funds, and micro -project applications were no longer accepted. During the project period two Procurement Reviews were carried out . In addition, the Bank’s procurement specialists regularly participated in supervision missions . Apart from some advisory notes, the reviewers have made no major comments on CAP’s procurement activities. The ICR reports (p.10) that the procurement activities were considered as moderately risky in the Procurement Reviews but were satisfactory in most Implementation Support Reports. The avian flu accounts were audited for 2008 and 2009. Both audits were unqualified. c. Unintended Impacts (positive or negative): None d. Other: 12. 12. Ratings : ICR IEG Review Reason for Disagreement /Comments Outcome : Satisfactory Moderately There were moderate shortcomings in Satisfactory the project's achievement of the transparency objective and in efficiency. Risk to Development Moderate Significant Only 44% of micro-projects were Outcome : profitable and there were inadequate maintenance provisions for the majority of the microprojects. Without continued donor funding, the achievements under the Avian Flu component are unlikely to be sustained. Bank Performance : Satisfactory Moderately There were moderate shortcomings in Satisfactory appraisal and quality-at-entry is rated moderately satisfactory. Supervision performance is rated moderately unsatisfactory due to inadequate information about safeguard compliance both in the ICR and the Region's comments. Borrower Performance : Satisfactory Moderately While the government was very Satisfactory supportive of decentralization, the impact of the failure to provide timely counterpart funding was a moderate shortcoming. Implementing agency performance had moderate shortcomings over M&E and committing project funds for micro-projects. Thus both government and the implementing agency performance are rated moderately satisfactory. Quality of ICR : Satisfactory NOTES: NOTES - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate. 13. Lessons: The ICR offers a number of lessons of which the following are the most important (with some reformulation of language). Promoting the value of natural resource management and environmental improvements requires tangible incentives . To incentivise poor people in long -term issues as the environment (eg maintenance of trees), tangible incentives (food or cash for work) are necessary. Additionally the public-sector environmental staff also require incentives (transport, training, allowances ). Political- and higher-level administrators must strongly encourage elected local officials and higher -level administrators to view natural resource management as a high priority . Women can be more successful at specific types of micro -projects In Niger, women proved to be more energetic and responsible in their commitments for financial and in -kind contributions, and better at making their micro-projects profitable than men, mixed groups, or commune administrations . It is not clear, however, whether the better profitability of women’s projects might be attributable to their selection of activities with which they are traditionally more familiar (e.g., batik coloring, petit trade, and raising small animals ). 14. Assessment Recommended? Yes No Why? The CAP-1 and its follow-on phases could provide important lessons on how to design and nurture decentralization and foster local participation and self -reliance. There is also the question of how reliant the decentralization process is on continued financial support from donors . 15. Comments on Quality of ICR: This was a challenging and complex project to assess, and the ICR has for the most part done a satisfactory job of analyzing and writing up the issues . The difficulties of achieving the environmental goals were well presented but the project response and details of the "environmental microprojects" were not well described. Very little was said about the exogenous factors (droughts, locust invasion, military coup and change of government ) which are likely to have effected the project. Insufficient information is given in the ICR on compliance with the environmental safeguard in the main project and information is also insufficient on compliance with OP 4.12. Otherwise the ICR contained a lot of useful information much of which was in the Annexes . More of these contents should have been summ arized into the main text. a.Quality of ICR Rating : Satisfactory