Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review TZ-Agr Sec Dev (FY06) (P085752) Report Number : ICRR0020725 1. Project Data Project ID Project Name P085752 TZ-Agr Sec Dev (FY06) Country Practice Area(Lead) Additional Financing P120930,P125484,P132838,P1 Tanzania Agriculture 15873 L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IDA-41920,IDA-46390,IDA- 31-Dec-2011 216,500,000.00 47400,IDA-51720,TF-11170 Bank Approval Date Closing Date (Actual) 15-Jun-2006 30-Sep-2016 IBRD/IDA (USD) Grants (USD) Original Commitment 90,000,000.00 14,250,000.00 Revised Commitment 176,704,593.62 14,250,000.00 Actual 173,823,509.35 12,167,838.41 Prepared by Reviewed by ICR Review Coordinator Group Hassan Wally Ridley Nelson Christopher David Nelson IEGSD (Unit 4) PHPROJECTDATATBL Project ID Project Name TZ:ASDP: Japan-PHRD Additional P125484 Financing ( P125484 ) L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) Page 1 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review TZ-Agr Sec Dev (FY06) (P085752) 0.00 Bank Approval Date Closing Date (Actual) 19-Dec-2011 IBRD/IDA (USD) Grants (USD) Original Commitment 0.00 0.00 Revised Commitment 0.00 0.00 Actual 0.00 0.00 2. Project Objectives and Components a. Objectives The project was the first phase of a fifteen-year investment program intended to support the implementation of Tanzania’s Agricultural Sector Development Strategy (ASDS). The Project Development Objective (PDO) in the Project Appraisal Document (PAD, p. 3) and the Financing Agreement (FA, p. 5) was identical and aimed to: "(i) enable farmers to have better access to and use of agricultural knowledge, technologies, marketing systems and infrastructure, all of which contribute to higher productivity, profitability, and farm incomes; and (ii) promote agricultural private investment based on an improved regulatory and policy environment." These two objectives were complementary and expected to contribute to agricultural growth and poverty reduction. At the time of the second Additional Financing (approved on May 28, 2010), the wording of the first part of the PDO was modified slightly by removing the link to higher order sector objective outcomes, i.e. the phrase “all of which contribute to higher productivity, profitability, and farm incomes” was removed "to circumscribe the project’s accountability and attribution (ICR, p. 2, para 5)." The objectives read as follows: "(i) to enable farmers to have better access to and use of agricultural knowledge, technologies, marketing systems and infrastructure; and (ii) to promote agricultural private investment based on an improved regulatory and policy environment." This review assesses the outcome of the project against the objectives as stated in the Financing Agreement. Page 2 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review TZ-Agr Sec Dev (FY06) (P085752) b. Were the project objectives/key associated outcome targets revised during implementation? No PHEVALUNDERTAKENLBL c. Will a split evaluation be undertaken? No d. Components The project included two components: 1. Local Support (appraisal cost: US$297.00 million, actual cost: US$288.4 million). It included three sub-components: 1.1. Local Agricultural Investments. This would include the provision of District Agricultural Development Grants (DADGs) for local agricultural investments on a cost-sharing basis, including, among other things, in public infrastructure, such as rural roads; small-scale irrigation schemes; environmental investments; productive community investments, such as in risk-bearing innovative equipment, crops and livestock; food storage facilities; market infrastructure; reforestation of degraded areas; and community nurseries. Also, provision of District Irrigation Development Fund (DIDF) Grants to pilot the competitive selection of local investments in small-scale irrigation schemes above the established budget ceiling for DADGs. 1.2. Local Agricultural Services. This would include the provision of Extension Block Grants (EBGs) for contracting by farmer groups of local agricultural services through private agricultural service providers, such as for advice on agricultural production and marketing, and facilitation of farmer-to-farmer visits and learning. 1.3. Local Agricultural Capacity Building and Reform. This would include the provision of Capacity- Building Grants for local agricultural capacity building, among other things, to improve district agricultural planning and agricultural investment appraisal, and for reform of agricultural services. Also, training would be provided to promote farmer empowerment for activities such as interacting with Local Government, and procurement and management of contracted services; and facilitation of farmer fora for networking, leadership and technology testing. The project would support the promotion of development of private sector agricultural service providers through awareness raising activities, and provision of training on operating modalities, and technical and business practices. 2. National Level Support (appraisal cost: US$243.00 million, actual cost: US$186.00 million). It included five sub-components: 2.1. Agricultural Services. The project would support agricultural services’ reform, primarily in the field of research and extension, through: improvement of management and accountability of Zonal Agricultural Research and Development Institutes (ZARDIs) through implementation of a client-oriented research development and management approach (CORDEMA); and the establishment, financing and expansion of Zonal Agricultural Research and Development Funds (ZARDEFs) across all ago-ecological zones; and through facilitation of policy and institutional reforms, such as preparation of a code of practice for extension and research, and finalization of an agricultural services reform strategy for other related technical services. Page 3 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review TZ-Agr Sec Dev (FY06) (P085752) 2.2. National Irrigation Development. The project would provide support through the National Irrigation Development Fund for the carrying out of due diligence preparatory work for future small-, medium- and large-scale irrigation investments in National and International Basins, including: participatory development and operationalization of appropriate identification, screening and prioritization mechanisms; support for technical designs, studies, and environmental impact assessment, including the strategic environmental assessment; capacity strengthening at the national, zonal and district levels, including in monitoring and evaluation; and carrying out of activities to attract private investment, such as awareness raising and improving the policy environment for public-private partnerships. Also, support would be provided in carrying out of physical infrastructure investments in irrigation at the national level in National Water Basins through public-private partnerships. 2.3. Marketing and Private Sector Development. The project would support scaling up of new approaches to private sector led agricultural market development, including support to smallholder marketing associations, linkages to external markets, and capacity building and investment along the entire marketing chain; empowerment of producer marketing groups at district level; improvement of formulation of agricultural regulations and laws and strengthening capacity for their implementation; support for agricultural policy analysis and formulation; carrying out of annual assessments of public expenditure in agriculture, at both national and district levels; and annual sector reviews. 2.4. Food Security. The project would support inclusion of vulnerable and food insecure groups in planning, preparation and implementation of District Agricultural Development Plans through technical advisory services and training to Local Government Authorities, and carrying out of rural vulnerability assessments. 2.5. Coordination, Monitoring and Evaluation. The project would support strengthening of national, regional and district level mechanisms for planning, implementation and reporting of agricultural investments and services, including quality control; overall project coordination; and monitoring and evaluation. Revised Components At the Mid-Term-Review (MTR) in September-October 2008, it was realized that some of the activities for supporting the second objective under the “market and private sector development” were overly ambitious. These activities were reconfigured as follow: (i) improving local regulatory environment for private investment in small, medium and large scale interventions; (ii) promoting forward and backward linkages along value chains; (iii) targeting investments in processing; (iv) promoting contract farming and out-grower opportunities; and (v) promoting access to financial services. In addition, during the first Additional Financing in June 2009, a new activity was added to support research and dissemination of soil fertility management technologies. Page 4 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review TZ-Agr Sec Dev (FY06) (P085752) e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project Cost. The total project cost was estimated at the appraisal stage to be US$315.56 million (PAD, Annex 5). However, this amount was revised upwards to reach US$539.70 million (ICR Annex 1). In a further communication during the preparation of this Review, the project team explained that the increase was necessary given the length of the implementation period. The actual amount disbursed according to the ICR (Annex 1) was US$474.40 million. The difference between the two amounts reported in the ICR (US$539.70 and US$474.40) was mainly due to a shortfall of US$50.75 on behalf of the development partners (see below) and another US$12.6 million on behalf of the Government (see below). The table above (in section 1) covering the Japanese Grant is left blank because it is covered under the main table above it which aggregates all the donor contributions. Financing. The project was financed through a basket fund that included six development partners: The World Bank, European Union (EU), Japan International Cooperation Agency (JICA), Irish Aid (IA), the International Fund for Agricultural Development (FAD), and the Danish International Development Agency (DANIDA). The World Bank financed the project through an IDA credit that was equivalent to US$90.00 million. The project received three rounds of additional financing (AF), the first (AF1) on June 9, 2009 for US$30.0 million to respond to the food crisis; the second (AF2) for US$35.00 million on May 28, 2010 to respond to the financial crisis; and the third (AF3) for US$30.00 million on October 23, 2012 to create a bridge between the project and the anticipated second phase. The total amount of World Bank funding was US$185.00 million; and the actual amount disbursed was US$189.99 million. On January 17, 2012, the project received a complementary grant worth US$14.2 million from the Japan Policy and Human Resources Development (PHRD) to supplement the credit resources with a focus on scaling up activities related to rice production in 20 irrigation schemes. The African Development Bank (ADB) funded the project through a credit worth US$62.6 million, the International Fund for Agricultural Development (IFAD) provided a credit worth US$98.9 million, the EU funded the project through a Grant worth US$9.4 million, the Irish Aid provided a Grant worth US$48 million, the Japan International Cooperation Agency (JICA) provided a Grant worth US$26.05 million. Actual amounts disbursed as reported by the ICR (Annex 1) were: US$62.5 million for ADB, US$98.5 million for IFAD, US$9.4 million for the EU, US$40.4 million for the Irish Aid; and US$2.97 million for JICA. Total disbursed funds amounted to US$393.45million. The difference between this and the US$474.4 million given above under Project Costs is due to borrower contribution. Borrower Contribution. The Borrower was originally expected to contribute US$8.0 million of counterpart funds (PAD, summary table). The ICR (Annex 1) reported that the Borrower was expected to contribute US$95.5 million a later revised figure. Actual disbursement was US$82.9 million. The project team explained that the difference in borrower contribution was due partly to the increased length of implementation; and that there was an in-kind portion to the government contribution. Dates. The project was expected to close on December 31, 2011 (the ICR recorded the closing date as June 30, 2011 and in another place in the ICR as June 30, 2012), however, the actual closing date was four years and nine months later on September 30, 2016. The project was restructured four times as follows: (the ICR did not report on the Level of each restructuring (1 or 2) and did not provide figures for the amount of funds disbursed at each restructuring) 1. On June 9, 2009 in order to approve the first additional financing (AF1) and extend the closing date from June 30, 2011 to March 31, 2014. Page 5 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review TZ-Agr Sec Dev (FY06) (P085752) 2. On May 28, 2010 in order to approve the second additional financing (AF2) and change the wording of the PDO and introduce some changes to the Results Framework. 3. On January 17, 2012 in order to approve a complimentary PHRD Grant and extend the closing date from March 31, 2014 to September 30, 2016. 4. On October 23, 2012 in order to approve the third additional financing (AF3). The ICR (p. v) reported that the Midterm Review was conducted on September 18, 2008 compared to an original date on November 15. 2008 (PAD, Annex 11), however, the ICR (p. v) reported the original date as April 30, 2009. 3. Relevance of Objectives & Design a. Relevance of Objectives Rated High. In 2004 agriculture dominated the economy in Tanzania and accounted for 46% the country's GDP. Improvements in overall economic growth relied heavily on the performance of the sector. In addition, about 87% of the poor lived in rural areas and 75% of rural income was earned from agricultural activities. Therefore, improvements in agricultural performance were expected, in due course, to have a direct impact on the incomes of the poor. With an abundant resource base, comparative advantage in the production of many crops, and expanding market opportunities both locally and regionally, the potential for improved agricultural performance in Tanzania was high. Sustained achievement of the agricultural targets needed improvements in agricultural productivity, a reduction in transactions costs, and improved management of risks (PAD, p. 2). At project appraisal objectives were highly relevant to the Government's priorities for the agriculture sector as outlined in the National Strategy for Growth and Poverty Reduction (MKUKUTA). MKUKUTA was based on three pillars: (i) growth and the reduction of income poverty; (ii) improved quality of life and social well-being: and (iii) good governance and accountability. The main focus of the strategy was on achieving shared growth in which agricultural development plays a key role. Objectives were also in line with the Agricultural Sector Development Strategy (ASDS) which recognizes that growth was to be private sector led through an improved enabling environment for enhancing the productivity and profitability of agriculture. Also, objectives were in line with the World Bank's Country Assistance Strategy (CAS) for Tanzania (2000-2003) which focused its interventions in four areas of strategic importance including sustainable rural development, to improve the livelihood of the majority of the poor who live in rural areas, among others. At project completion, objectives continue to be highly relevant to the Government priorities for the agriculture sector. Specifically, objectives continued to be in line with the 2010/11 - 2014/15 Second National Strategy for Growth and Poverty Reduction which emphasized agricultural modernization and commercialization. Objectives were also in line with the World Bank's 2012-2015 Country Assistance Strategy (CAS) for Tanzania which emphasized that Tanzania needed to increase agricultural productivity and value addition, among other things, in order to sustain high growth and to make growth more inclusive to reduce poverty. Page 6 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review TZ-Agr Sec Dev (FY06) (P085752) Rating High b. Relevance of Design Rated Modest. • Design included two complementary objectives that were expected to contribute to productivity, profitability and farm income, however, these three aspects were not included in the Results Framework. The second objective, was broad and vague. The Results Framework did not provide clear links between inputs, outputs and the expected outcomes; and the intermediate outcome indicators were not all well aligned with the PDO. • Design featured a basket funding approach to ensure coherence among development partners and reduce transaction costs, however, coordination among multiple donors proved to be challenging. Design also sought to align disbursements with the Government’s funding mechanisms for Local Government Authorities; however, the project suffered from delayed release of funds throughout implementation. Design also promoted local community empowerment to better respond to the needs of communities and increase ownership. • To achieve the stated objectives, design included two components with multiple activities, which made design complex. The first component would contribute to achieving the first objective through improving the capacity of the Local Government Authority capacity to plan, support and co-ordinate agricultural services and investments in a more efficient, participatory and sustainable manner. Support would also be provided to develop and implement District Agricultural Development Plans, including increasing farmer influence in resource allocation decisions for services and investments; progressing agricultural services reform and improving the quality of public expenditure. Activities included supporting local agricultural investments on a cost-sharing basis, providing grants to support private service (extension) to farmers; and providing agricultural capacity building activities and supporting the promotion of development of private sector agricultural service providers through awareness raising activities, and provision of training on operating modalities, and technical and business practices. While these activities were relevant and directly linked to the first objective, they lacked a detailed description in the PAD. • The second component would support achievement of both project objectives where the first objective would be supported through improvements to the relevance and responsiveness of the agricultural research system including greater linkages with extension; and the second objective would be supported through improvements to the national level policy environment partly, and through developing mechanisms for greater public-private partnerships. Support would be provided to reform agricultural services, primarily research and extension; to improve the overall sector policy framework; to carry out preparatory work and investment in national level irrigation through public-private partnerships; to simulate market development; and to improve food security and sector co-ordination. These activities were relevant and directly related to both objectives. However, the sub-component addressing food security seemed as an out of place add-on to the project. • Design suffered from a number of shortcomings including an overambitious territorial coverage Page 7 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review TZ-Agr Sec Dev (FY06) (P085752) which diluted the project's impact, and made close monitoring challenging. Design promoted value chain development, yet it included no deliberate actions aimed at its development; and at the same time it promoted a demand driven approach that resulted in a diverse and "dispersed set of activities with no organizing principles to drive meaningful value chain development" (ICR, p. 28, para 20). Design also promoted a demand driven private extension service, however, this approach did not take into account the existing extension service provided by Local Government Authorities, which eventually contributed to the underachievement of this activity. Rating Modest 4. Achievement of Objectives (Efficacy) PHEFFICACYTBL Objective 1 Objective To enable farmers to have better access to and use of agricultural knowledge, technologies, marketing systems and infrastructure, all of which contribute to higher productivity, profitability, and farm incomes. Rated Modest. Rationale Outputs • Local District Agricultural and Irrigation Development Grants financed the following: 492 kilometers of feeder roads were rehabilitated/constructed, 160,345 ha were developed and/or rehabilitated on 386 irrigation schemes (target:600), 473 floodwater retention dams were constructed, 80 shallow wells were built; and 65 tractors, 1,972 power tillers, 1,321 ploughs were cumulatively provided to farmers through cost sharing arrangements, 105 oxen training centers were established, 104 veterinary clinics were built, 680 (target:640) dip tanks were constructed and rehabilitated, 1,852 general processing machines were installed including 598 agro-processing machines for various crops, including coffee, maize, paddy, sunflower, cassava, ginger, palm fruit, meat, and milk, 921 warehouses were constructed 351 crop markets and 58 livestock markets were built. • A total of 774,156 framers received capacity building training through farmer field schools. Also, 16,556 extension officers received short courses while 1,519 received long courses. The ICR did not elaborate on the content of these courses. 2,328 private agricultural service providers also received training, however, no information was provided in the ICR on the nature of this training activity. • Agricultural research: by project completion 236 research projects were funded through the Zonal Agricultural Research and Development Funds (157 on crops and 79 on livestock); and 86 improved crop varieties were developed and released after validation by the National Seed Release Committee. These varieties had various positive attributes including high yielding, drought tolerance, diseases and pest Page 8 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review TZ-Agr Sec Dev (FY06) (P085752) resistance, good marketability and early maturing. Also, 31 PhD, 76 MSc and 37 Bachelors were supported to improve human resources, 22 vehicles and 7 motorcycles were procured for logistical support to researchers, 2 new staff houses were constructed and 23 were rehabilitated, 87 computers, 52 printers and 22 photocopiers were provided for research stations. • Extension services: by project completion 16,330 farmer field schools were established, 50 private sector service providers were contracted (target: 558), 106 motor vehicles, 2,343 motor cycles and 3,389 bicycles were distributed in 131 Local Government Authorities for extension staff to improve their effectiveness, 319 Ward Agricultural Resource Centers were established, and 475 computers and printers were procured to equip offices of extension staff. • To strengthen research-extension linkages, Zonal Information and Extension Liaison units (ZIELUs) headed by zonal information and extension liaison officers were established to link research activities at agricultural research institutes with extension services in the districts through farmer field schools, various farmer trainings. ZIELUs were responsible for technology verification, transfer and knowledge dissemination. • The Strategic Environmental and Social Assessment (SESA) for the national irrigation master plan and the national irrigation policy was completed in May 2011. The SESA identifies potentially adverse environmental and social impacts emanating from the implementation of the national irrigation policy/national irrigation master plan, such as degradation of river catchments and riparian ecosystems/biodiversity, soil salinization, loss of forests and other vegetation, reduction of environmental flows, degradation of ecologically sensitive areas. It also provided strategic guidance on how to minimize and mitigate those impacts when implementing irrigation development projects/programs in the sector. Outcome The project achievements under the first objective suffered from major shortcomings (discussed below) and failed to attain a number of the outcome targets (despite the cumulative closing date extension of about 57 months or 187% of the original expected duration of the project). The ICR (p. viii) reported that the project reached 228,000 beneficiaries compared to a target of 285,000 (this target was set after 6 years of implementation at AF3 stage). However, the ICR text (para 31) reported that the exact number of beneficiaries in the various activities supported by the project was not known. This casts doubt on the exact outreach of the project and makes it challenging to adequately gauge the project’s sub-sectoral outreach. There were also concerns about long term sustainability of some of the irrigation infrastructure and warehouses -financed by the project- where according to the ICR (p. 10, para 31), the construction quality of both was "generally low". Further, the project suffered from attribution issues that were exacerbated by the use of sector-wide indicators (ICR, p. 10, para 29). Finally, the absence of baseline data and a relevant counterfactual combined with a poor M&E system makes assessing the impact of project activities on productivity, profitability and farm incomes not possible. Therefore, outcome of this objective is rated modest. The project activities impacted the following areas: • Access to knowledge and technology. According to the ICR (p. 10, para 31) adopters of improved technologies enjoyed income gains of about US$396 over non-adopters due to multiple factors, including Page 9 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review TZ-Agr Sec Dev (FY06) (P085752) adopting better crop and livestock husbandry, utilizing improved genetic materials, practicing value addition, and enhancing their labor productivity. However, the project fell short on attaining important outcome targets including: percentage of farm households using improved seeds reached only 19.80% of households compared to a target of 35% of households and a baseline of 18% of households. Also, percentage of farm households using fertilizers reached 16.80% of households compared to a target of 25% of households and baseline of 12% of households; and percentage of farm households using improved livestock breeds reached 4% of households compared to a target of 5% of households and a baseline of 25 of households. It was not possible to assess the number of farm households using improved soil fertility management practices because mechanisms for its measurement were not put in place. The ICR also did not report on adoption rates of new technologies among project beneficiaries. Further, the Zonal Information and Extension Liaison Units were underperforming due to shortage of staff, funds, transport, and necessary communication facilities, which resulted in limited dissemination of research messages to beneficiaries (ICR, p. 26, para 14). • Impact of project activities on Crop/Livestock Production. According to the ICR (p. 11, para 33) maize production in the Southern Highlands increased by 20% (50% percent among farmer field school participants, and by 153% in Arusha region (from 1.5 to 3.8 tons/hectare). This was achieved through better seed and management practices. Also, farmers practicing the System for Rice Intensification (SRI) had 34% higher yields than "their peers", however, the ICR did not indicate whether those peers received any project support. Livestock mortality was also down by 20%, however, the baseline was not clear in the ICR. Also, due to better animal husbandry and genetic improvement through artificial insemination and bull rotation, milk yield improved in many instances by up-to 100% (e.g. in Meru District, from 6 to 12 liters/cow/day). Cross-breeding between local chicken and improved cockerels increased egg production from 40 to 60 eggs per hen per year and improved the average weight of local chicken from 1.5 to 2.5kg. While these results seem encouraging, there is doubt about the accuracy of the data due to M&E weaknesses including the absence of a clear baseline and counterfactual to accurately assess the impact of the project. • Impact of project activities on Value Addition. According to the ICR (para 34) agro-processing machines had a monthly processing capacity of 36,655 tons of sunflower, 30,772 tons of paddy, 11,142 tons of coffee and 5,513 tons of cassava. The ICR reported that properly pulped coffee sold at a 36% premium in 2008/09 over traditional practices. The ICR did not provide quantitative data on the impact of project activities on profitability and farm incomes as a result of agro-processing. • Impact of project activities on Labor productivity. The project promoted ox plowing, however, by project completion the percentage of smallholders using oxen reached 24% compared to a target of 30% and a baseline of 20%. Well-trained oxen reduced the time for plowing one hectare from 30 days using a hand hoe to only 2 days. At the price of TZS 2,500 per labor-day, this translates into gross savings of TZS 70,000 per hectare. According to the ICR (para 35) cropped area per household increased from 2 to 5 acres on average among oxen users. Similarly, utilizing powered tillers resulted in 8 to 10 times reduction in plowing time compared to hand hoe. However, the ICR provided no information on the impact of these activities on productivity, profitability and income at project completion. Page 10 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review TZ-Agr Sec Dev (FY06) (P085752) • Access to Transport and Market Infrastructure. According to the ICR (p. 12, para 36) the rehabilitation of feeder roads resulted in a reduction of transportation costs by 75% in communities benefitting from these feeder roads. Also, farmgate prices increased significantly, for example, in Nkasi District maize prices increased by 56%, from TZS 15,000 to TZS 27,000 per bag. Improved feeder roads also led to increased frequency of extension services by 33%. However, it was not clear in the ICR how these results were estimated. The ICR (p. 12. para 36) also claimed that the use of market infrastructure financed by the project resulted in an increase between 20% and 25% in farmer's sale prices; and the use of ware houses enabled farmers to sell their produce at better market prices rather than be forced to sell during the peak harvest season when prices would typically be lower due to the increase in supply. In Singda region, for instance, sunflower farmers obtained an unprecedented 243% price increase from storage (from TZS 210 to TZS 720 per kg) through better market timing; while rice farmers are obtaining up to 60%intertemporal price gains by avoiding selling during the glut period. That said, the ICR (para 36) highlighted that "a key concern with respect to market infrastructure, especially of warehouses, is the sustainability of the developed infrastructure as various program reviews showed inadequacies in quality assurance during construction." • Access to Irrigation Infrastructure. According to the ICR (p. 12, para 37) the rehabilitation and development of 160,345 hectares under 386 irrigation schemes (target: 600) resulted in a 100% increase in rice paddy production from an average of 1.47 tons per hectare to 3.43 tons per hectare. However, the ICR (p. 36, para 8) stated that actually none of the irrigation schemes reported their production data which totally contradicts the first statement. Finally, there were "lapses in quality assurance during construction of the irrigation infrastructure and this will undermine long-term functioning and therefore continued access to irrigation infrastructure (ICR, p. 12, para 37)." Rating Modest PHREVDELTBL PHEFFICACYTBL Objective 2 Objective To promote agricultural private investment based on an improved regulatory and policy environment. Rated Negligible. Rationale Outputs • By project completion 23 agricultural marketing regulations and legislations were in place compared to a target of 21 and a baseline of 7. Outcome The project funded warehouses could have played a positive role in facilitating the flow of funds to the Page 11 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review TZ-Agr Sec Dev (FY06) (P085752) agriculture sector through the warehouse receipt financing system. Farmers using the warehouse receipt system were storing their produce in project funded warehouses. Commodities stored included: cashew- nuts, rice, cotton and coffee. The ICR (para 38 & para 39) refers to several laws and regulations that could positively impact agricultural investments. However, whether the project played a role in supporting these laws and regulations was unclear from the ICR. Other than the warehouses, there was no tangible evidence provided in the ICR to show the impact of the project activities on the achievement of this objective. Therefore, outcome is rated negligible. Rating Negligible PHREVDELTBL PHREVISEDTBL 5. Efficiency Economic and Financial Efficiency ex ante • The PAD did not include a detailed ex ante financial and economic analysis. It stated that beneficiary coverage and productivity improvements would need to yield at least a 12% rate of return at the minimum to consider the project's investments feasible. ex post • The project's economic value was expected to include: (i) increased production and productivity; (ii) incremental impacts from access to markets; (iii) value addition to raw products; and (iv) labor productivity. However, the absence of reliable data for the principal activities, including the level of gains and the corresponding number of beneficiaries did not allow a rigorous economic and financial analysis of the project activities at completion (ICR, para 41). • An Internal Rate of Return (IRR) of 38% was estimated for irrigation investments. • While the ICR (para 40) reported that project costs for some activities (irrigation development and rehabilitation) seemed to be in line with other projects in Tanzania and elsewhere in Sub-Saharan Africa, the project had attribution issues and monitoring relied on sector wide indicators with little evidence linking benefit streams to project implementation. Annex 3 in the ICR reports some encouraging largely qualitative results, but there was no meaningful analysis of the efficiency of investments made to achieve these results. Administrative and Institutional Efficiency Page 12 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review TZ-Agr Sec Dev (FY06) (P085752) The project closed 57 months later than expected, a 187% increase over what was planned. Construction of warehouses faced significant delays due to multiple factors including: delays in procurement, delays in remitting advances to contractors, low frequency of supervision of sites by the supervising consultant, and in two of the cases, due to land disputes (ICR, p. 31, para 7). Also, training on rice value chain development was hampered due to delays in the release of funds. The ICR (para 16) also stated that fiduciary factors caused significant delays in implementing key project activities which contributed to the failure of the project to fully achieve its development objectives. The project also suffered from the lack of harmonization of donor deposits into the basket fund, this was later resolved by making annual deposits by development partners before 1st July. Efficiency is rated negligible due to the lack of a meaningful ex post economic and financial analysis; and due to major weaknesses in administrative and institutional efficiency. It is also noted that the lack of evidence is a particular concern with such a large project of US$474 million that was under implementation for about ten years. Efficiency Rating Negligible a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 0 Appraisal 0 Not Applicable 0 ICR Estimate 0 Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome Relevance of objectives was rated high while relevance of design was rated modest. Efficacy of the first objective was rated modest because the project failed in attaining a number of important outcome targets. Also, the project suffered from attribution issues that were exacerbated by the use of unfocused sector-wide indicators and poor M&E-- including the absence of a clear baseline and counterfactual needed to accurately assess the impact of the project. Efficacy of the second objective was rated negligible because the impact of the project activities on the achievement of this objective was not clear, and the project-funded warehouses with evidence only at an output level were the only tangible evidence of the project contribution. Efficiency was rated negligible due to the absence of any meaningful ex post economic or financial analysis; and due to major shortcomings with regards to the administrative and institutional efficiency. There were major shortcomings in the operation's achievement of its objectives and efficiency. Therefore, outcome is rated unsatisfactory. Page 13 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review TZ-Agr Sec Dev (FY06) (P085752) a. Outcome Rating Unsatisfactory 7. Rationale for Risk to Development Outcome Rating Risk to the development outcome is rated high for the following reasons: • The maintenance of irrigation infrastructure poses the greatest challenge to sustainability of the development outcome. In Tanzania O&M has been a frequent problem that is further complicated by the low quality of the irrigation infrastructure developed under the project. The ICR (para 49) stated that the irrigation impact assessment study revealed that O&M needs for the project-financed irrigation suffered from low skills. None of the schemes visited had established a proper O&M budget. Contribution by beneficiaries towards O&M ranged between 0.2 and 0.5%of annual gross income per unit area - well below the typical 7-8% global average. This raises serious doubt about the availability of funds to carry out proper O&M for the irrigation schemes. • Downstream irrigation schemes could suffer from reduced water availability due to significant unauthorized irrigated area expansions and water abstractions by some upstream irrigators. If these actions are not addressed, the sustainability of project -supported downstream schemes is highly questionable (ICR, para 49). • Another threat faced by a number of schemes is the excessive sedimentation loads which could threaten the sustainability of the water supply. It is unclear whether there was a plan to address sedimentation (ICR, para 49). a. Risk to Development Outcome Rating High 8. Assessment of Bank Performance a. Quality-at-Entry • At the request of the Government of Tanzania, the World Bank prepared this project. It featured an integrated approach of Development Partner support (European Union, Japanese International Cooperation Agency, Danish International Development Agency, Irish Aid; and the International Fund for Agricultural Development) to the agriculture sector; which was in line with the Government's priorities. • The project preparation benefitted from a sound background analysis that included a number of working papers that were prepared related to agricultural research, agricultural extension, farmer empowerment, Page 14 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review TZ-Agr Sec Dev (FY06) (P085752) and information and communication. However, a thorough analysis of the policy framework was not carried out. This would have helped to gauge the extent to which the policy environment was limiting private investment flows into the agricultural sector. Furthermore, the "lack of such analysis subsequently undermined the case for attribution of broad sector outcomes to policy actions undertaken by Government during the project period (ICR, para 13)." Project preparation also lacked a thorough capacity assessment of the Government institutions/entities involved in project implementation as this would have contributed to better implementation arrangements. Also, the central Government’s budgetary and treasury processes were not fully understood which contributed to difficulties in the flow of funds during implementation (ICR, para 13). Fund flow patterns should have been modeled at appraisal. The PAD also lacked a detailed economic and financial analysis of project activities. • The project's design benefitted from a number of lessons most notable were related to: building public agricultural service provision around demand-based approaches; using incentive based systems to reform Local Government Authorities; and integrating Development Partner financed projects into Government systems to reduce duplication and transaction costs and better align development partners’ support to the Medium Term Expenditure Framework (MTEF) for agriculture. • Ten risks were identified at the appraisal stage, one was rated high, another was rated substantial while eight were rated moderate. Several of the identified risks materialized during implementation. However, the project’s proposed mitigation measures were generally "inadequate to mitigate them (ICR, para 20)." For example: delayed release of funds by the Government persisted throughout project implementation; the lack of practical modalities for involving local communities in complex procurement processes contributed to the persistence of a top-down approach by Local Government Authorities in procurement management, especially in large irrigation schemes; construction of some warehousing and irrigation infrastructure suffered from low quality of construction due to limited engineering skills and erratic funding; and the limited capacity at the lead ministry contributed to slow procurement processing which persisted throughout the project’s life (ICR, para 20). Poor quality of construction is an ongoing challenge in Tanzania and has been highlighted earlier by IEG. • M&E suffered from major weaknesses including an inadequate monitoring and evaluation system (see section 10 for more details). The ICR (para 13) stated that the "size and dynamics of the agricultural sector seemed to not have been fully understood" during the preparation stage which undermined the M&E design. It appears also that the scale of the project itself was insufficiently understood. Quality-at-Entry Rating Unsatisfactory b. Quality of supervision The project implementation benefitted from constant support through a task team leader based in the country office throughout most of the project’s life. Quarterly meetings of the Basket-Funding Steering Committee (including the World Bank) aimed to ensure the projects smooth implementation. Annual multi- stakeholder implementation support missions included a wide range of skills which provided comprehensive action plans for the following year. However, supervision suffered from two notable shortcomings: first, the failure to ensure environmental and Page 15 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review TZ-Agr Sec Dev (FY06) (P085752) safeguard compliance and secure adequate funding to address these activities; and second the failure to ensure that the M&E system was functional at an early stage of the project implementation, including the introduction of necessary and relevant amendments to the Results Framework to align the project’s indicators with the expected outcomes. IEG also questions the candor of the ISRs and how a project that did not have sufficient oversight of safeguards or fiduciary compliance continued to be rated consistently Satisfactory or Moderately Satisfactory for about ten years. Finally, the ICR (p. 3, para 10) stated that some activities were revised at the MTR stage, however, that there was no subsequent formal restructuring by the World Bank. Quality of Supervision Rating Unsatisfactory Overall Bank Performance Rating Unsatisfactory 9. Assessment of Borrower Performance a. Government Performance According to the ICR (para 52) the Government undertook several actions to facilitate the project’s implementation, including: "enacting a number of laws (for example: the Irrigation Development Act of 2013 has set up a framework that facilitates private investment in irrigation) and adopting a series of regulations to create a conducive environment for private sector investment in agricultural value chains; adopting laws and establishing institutions to improve irrigation management and oversight; and providing counterpart funding." However, the project suffered from delayed release of government and donor funds throughout implementation which contributed to poorly monitored carryovers of funds at Local Government Authorities, and incomplete designs and works which in some cases undermined the quality of the program’s investments. Government Performance Rating Moderately Unsatisfactory b. Implementing Agency Performance The institutions responsible for implementation of the national level component of the program were the Ministry of Agriculture, Food Security and Co-operatives (MAFC), the Ministry of Livestock Development (MLD), and the Ministry of Industry, Trade, and Marketing (MITM) while implementation of the local level component was the primary responsibility of the Prime Minister's Office-Regional Administration and Local Government and Local Government Authorities (LGAs). According to the ICR (para 52) the implementing agencies facilitated the project’s implementation through enacting a number of laws and adopting a series of regulations to create a conducive environment for private sector investment in agricultural value chains; and through adopting laws and establishing institutions to improve irrigation management and oversight. However, there were a number of notable shortcomings including: poor enforcement of social and Page 16 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review TZ-Agr Sec Dev (FY06) (P085752) environmental safeguard policy compliance, failure to ensure better quality of both irrigation and warehousing infrastructure; weak financial management; and poor management of M&E activities including limited collection of the project's outcome-related data--critical to assess the project's performance (ICR, para 52) and particularly important for such a large project. Implementing Agency Performance Rating Unsatisfactory Overall Borrower Performance Rating Unsatisfactory 10. M&E Design, Implementation, & Utilization a. M&E Design The overall responsibility of M&E was within the office of the Director of Policy and Planning at the Ministry of Agriculture, Food Security & Cooperatives. Design of the project's M&E relied on monitoring overall rural poverty and sector growth indicators as part of the Government's own program under the National Strategy for Growth and the Reduction of Poverty. The National Sample Survey of Agriculture completed in 2003 was used as a baseline for the project. However, M&E responsibilities were not clear and there was no coherent M&E implementation plan for the project. In addition, M&E design was short on specific impact assessments/studies that would have measured the project's impacts at baseline, midpoint, and completion. The Results Framework (RF) covered two complementary objectives, however, in the design the linkage between the intermediate outcomes and the objectives was unclear, and there were too many indicators most of which were not easy to measure (ICR, p. 42, para12). The RF also included sector- wide indicators to gauge project performance, even though a case for attribution could not be possibly made, for example, growth in processed exported agricultural goods and increased usage of tractors which could hardly be attributed specifically or wholly to the project’s activities (ICR, p. 7, para 22). b. M&E Implementation M&E implementation started three years late in 2009 (the project was effective on October 18, 2006). It largely focused on financial data and project activities, with no reporting on outcomes such as yield. Data from National Panel Surveys (NPS), which would have helped fill some of the data gaps, was intermittent and too general to draw attribution. M&E implementation was also undermined by weak capacity at the Local Government Authorities combined with the overambitious territorial coverage of the project which made close monitoring of activities difficult (ICR, p. 7, para 22). Some new PDO indicators were added to the Project’s Results Framework, while others were reformulated during the first AF (approved on June 9, 2009) in order to align the Project’s results framework with that of the Government Program’s Results Framework. It is questionable whether this adjustment helped M&E activities Page 17 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review TZ-Agr Sec Dev (FY06) (P085752) or better aligned the indicators with the PDO. c. M&E Utilization M&E design was poor and implementation was weak. The weakness of the M&E system prevented a thorough and credible assessment of the project’s impact. It did not adequately document the changes and attirbutable benefits induced by the project. It was not clear in the ICR whether the M&E generated data was used to inform any management decisions related to the project. M&E Quality Rating Negligible 11. Other Issues a. Safeguards The project’s environmental category was B. The following safeguard policies were triggered: Environmental Assessment (OP/BP/GP 4.01), Pest Management (OP 4.09), Involuntary Resettlement (OP/BP 4.12), Safety of Dams (OP/BP 4.37) and Projects on International Waterways (OP/BP/GP 7.50). The environmental and social impacts of the project were expected to come from the implementation of sub-projects contained in the District Agricultural Development Plans, and also from national level investments in irrigation (financed through resources allocated from the project). The ICR (p. 7, footnote 6) noted that among other problems in the irrigation works, inadequate funding resulted in neglecting carrying out of Environmental and Social Impact Assessments. Environmental Assessment (OP/BP/GP 4.01). An Environmental and Social Management Framework (ESMF) was prepared and disclosed prior to the project’s appraisal. In 2008, the Midterm Review highlighted inadequate adherence to the ESMF (ICR, p. 8 para 23). Despite efforts by the Bank, compliance remained a problem for most of the project's implementation period. According to the ICR (para 23) "in most cases the mandatory Environmental and Social Assessments (ESIAs) or the Environmental and Social Management Plans (ESMPs) were not done.....and ....overall environmental safeguard compliance and oversight was very poor." Pest Management (OP 4.09). An Integrated Pest Management Plan (IPMP) was prepared prior to the project's appraisal. However, the ICR did not provide any further information on compliance and mitigation efforts. Involuntary Resettlement (OP/BP 4.12). The Resettlement Policy Framework (RPF) had been prepared and disclosed prior to appraisal. An activity added as part of the PHRD Grant required about 50 acres of land being used by farmers to grow paddy who would need to be compensated. Therefore, an Abbreviated Resettlement Action Plan was prepared and disclosed in 2013. A sample audit revealed that compensation under the resettlement framework had not yet been carried out in some instances. According to the ICR Page 18 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review TZ-Agr Sec Dev (FY06) (P085752) (para 24), the Bank followed up before project closure and "all the families that had lost land were compensated with other land, however, without due consideration to any impacts that the loss of their original land had on their livelihoods." The ICR also stated that social safeguard compliance was rated "unsatisfactory." Safety of Dams (OP/BP 4.37). The ICR did not report on this safeguard policy. Projects on International Waterways (OP/BP/GP 7.50). The ICR did not report on this safeguard policy. To summarize, it is clear that none of the Bank’s safeguards were adequately adhered to. b. Fiduciary Compliance Financial Management. According to the ICR (p. 8, para 25) the project audits of the Agricultural Sector Line Ministries by the Controller Auditor General for all the fiscal years were unqualified, except for FY08/09 and FY13/14 where US$177,569 were reported to be ineligible expenses. These funds were later accounted for using "substitute documentation." While there was an adequate number of financial staff, funds from the project’s designated account tended to be held up in the exchequer system, sometimes for several months. The ICR (p. 8, para 25) correctly highlighted this matter and stated that the delay in releasing project funds raised "the possibility of diversion of project resources to other uses." The delayed release of project funds not only caused implementation delays, but also "led to carry over funds which were not adequately monitored or accounted for." As a result, Interim Financial Reports were submitted late to the World Bank. Overall, financial management was weak and contributed to implementation delays. Procurement. Procurement activities followed the World Bank guidelines, however, procurement operations at the national level were generally slow. Procurement activities benefitted from project supported capacity building to national and local staff; and sub-project committees at community level. However, there were challenges stemming from delays in the preparation of tender documents and inadequate implementation of Procurement Plans due to insufficient procurement capacity (ICR, p. 9, para 27). c. Unintended impacts (Positive or Negative) The ICR (p. 16, para 47) reported that "unintended outcomes and impacts have not been properly documented." d. Other --- Page 19 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review TZ-Agr Sec Dev (FY06) (P085752) 12. Ratings Reason for Ratings ICR IEG Disagreements/Comment Relevance of design was rated modest. For reasons given in section 6, Efficacy of Moderately the first objective was rated Outcome Unsatisfactory Unsatisfactory modest while efficacy of the second objective was rated negligible; and efficiency was rated negligible. Risk to Development High High --- Outcome QAE and supervision both Moderately Bank Performance Unsatisfactory suffered from major Unsatisfactory shortcomings. Government performance suffered from moderate Moderately shortcomings; while the Borrower Performance Unsatisfactory Unsatisfactory implementing agency performance suffered from major shortcomings. Quality of ICR Substantial --- Note When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate. 13. Lessons The ICR included 12 lessons. The following four are emphasized with some adaptation of language: • Investments in irrigation at a national scale should be prefaced with a proper understanding of the water balance at the appropriate scale. In this case, water abstraction for irrigation purposes proceeded without an understanding of the cumulative impact of the irrigation needs on the availability of water for other uses. This has contributed to an observed increased competition for water between the energy, agriculture and tourism sectors in the country. • Appropriate skills mix is key to quality supervision. The project made significant investments in infrastructure which ended up being of a low quality. The presence of engineers with adequate expertise on the supervision team would have mitigated the poor quality issues observed for some of the infrastructure. • Basket-Funding reduces transaction costs, but harmonizing implementation support procedures Page 20 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review TZ-Agr Sec Dev (FY06) (P085752) among participating development partners can be a challenge. The approach and rigor to implementation support varies across development partners. Less rigorous implementation support standards can undermine ensuring effective fiduciary and safeguard compliance. In cases where challenges associated with basket-funding outweigh the risk, one alternative is for each development partner to, while maintaining the basic tenets of donor-harmonization, identify aspects of the Government program to support in a separate operation, albeit well-coordinated with other development partners. • It is important to design the M&E system prior to project appraisal to ensure capturing project achievements. Relegating the system’s design to a project’s implementation phase fails to capture early stage data, the system often becomes under-resourced, and eventually becomes of limited value as a management tool. It is also important to evaluate M&E performance during an MTR and undertake corrective actions. One corrective measure in cases like this would be the commissioning of targeted case studies to evaluate the returns to a sample of project investments. 14. Assessment Recommended? Yes Please explain This was a very large project in terms of total project costs. Further assessment of the project would focus on how such a large and diffuse project came to be prepared and approved given Tanzania’s history of finding large complex projects difficult to implement and on water basin issues and efficiency. Also, it would allow probing the often problematic issues of donor coordination in a large project. 15. Comments on Quality of ICR The ICR was handicapped by the limited availability and poor quality of data due to the weaknesses of the M&E system. This in turn limited the ability of the ICR to document and measure outcomes that were attributable to the project. That said, the ICR provided candid coverage of project shortcomings and attempted to report on project achievements despite the poor M&E. The ICR included a good discussion of outcomes that reflected the different elements of the stated objectives. It included 12 generally thoughtful lessons that reflected the project's experience. The ICR could have been improved in the following areas: • Provide a clear breakdown of project costs including different sources. • Report full detail on restructuring including whether Level 1 or 2 and the amount of Bank funds disbursed at the time of restructuring. • Report on all the triggered safeguard policies and include a clear statement of compliance. Page 21 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review TZ-Agr Sec Dev (FY06) (P085752) • Report on the status of external financial audits. • Restructuring and refinancing due to the food crisis and the financial crisis should have received more coverage with regards to the impact of the project in terms of what was actually achieved. • Reporting was inconsistent in some cases, for example, the ICR reported on rice paddy production in irrigation schemes then in a different section reported that none of the schemes reported their production. • ICR recorded two different original project closing dates; and both dates differed from the closing date reported in the PAD. Also, original MTR date recorded in the ICR was different from the one in the PAD. a. Quality of ICR Rating Substantial Page 22 of 22