Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MZ Agriculture DPO-1(P129489) Report Number : ICRR0021235 1. Project Data Country Practice Area(Lead) Mozambique Agriculture Programmatic DPL Planned Operations: 2 Approved Operations: 3 Operation ID Operation Name P129489 MZ Agriculture DPO-1 L/C/TF Number(s) Closing Date (Original) Total Financing (USD) IDA-52140 31-Dec-2013 50,000,000.00 Bank Approval Date Closing Date (Actual) 25-Apr-2013 31-Dec-2013 IBRD/IDA (USD) Co-financing (USD) Original Commitment 50,000,000.00 0.00 Revised Commitment 50,000,000.00 0.00 Actual 49,725,650.00 0.00 Prepared by Reviewed by ICR Review Coordinator Group Francesco Cuomo J. W. van Holst Christopher David Nelson IEGSD (Unit 4) Pellekaan PHPROJECTDATATBL Operation ID Operation Name P146930 MZ Agriculture DPO-2 ( P146930 ) Page 1 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MZ Agriculture DPO-1(P129489) L/C/TF Number(s) Closing Date (Original) Total Financing (USD) IDA-52140,IDA-55810 28-Mar-2016 50,000,000.00 Bank Approval Date Closing Date (Actual) 30-Mar-2015 28-Mar-2016 IBRD/IDA (USD) Co-financing (USD) Original Commitment 50,000,000.00 0.00 Revised Commitment 50,000,000.00 0.00 Actual 47,826,648.00 0.00 2. Program Objectives and Policy Areas a. Objectives The original Program Development Objective (PDO) in the Program Document (PD) for the first Agriculture Development Policy Operation (AgDPO-1) was to: "to promote private sector-led agricultural growth in order to improve food and nutrition security" (AgDPO-1, PD, page vii) The program objective was revised in the PD for the second policy operation becoming "to improve agricultural technology, enhance access to productive assets and financial services, and improve monitoring of sector performance" (AgDPO-2, PD, page vii) Due to Mozambique's non-compliance with IDA policies on the disclosure of non-concessional loans (acknowledged by the Government in April 2016) the third DPO was dropped from the proposed series by the World Bank. In accordance with harminized OPCS/IEG Guidelines, this Review will assess the results of the program based on the achievements during the incomplete series against the revised objective for the series. . b. Pillars/Policy Areas Policy Area 1: Improving Agricultural Technology. This policy area aimed at supporting improvements in agricultural productivity and production by promoting increased use of seeds, fertilizers and irrigation (AgDPO-2, paras 44,50,55, table 6). This policy also included actions to improve nutrition "that directly impacts on-farm labor productivity" . Policy Area 2: Enhancing Access to Assets and Financial Services. This area aimed at strengthening of land administration and the development of commodity value chains by improving the access to productive and financial assets for producers and entrepreneurs in rural areas to stimulate private sector investments in agriculture (AgDPO-2, PD, paras 67, 75, table 7). Page 2 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MZ Agriculture DPO-1(P129489) Policy Area 3: Monitoring Sector Performance. This policy area aimed at developing and implementing reliable systems for monitoring purposes in agriculture, with the objective to build a sustainable agricultural statistics system "that allows to report objectively on the performance of the agriculture sector" (AgDPO-2, PD, para 83, Table 8). c. Comments on Program Cost, Financing, and Dates Program Cost. At appraisal, the total cost of the program was estimated at US$150 million based on three operations of roughly US$50 million each in the first DPO PD (AgDPO-1 PD, Annex 1, page 60). The actual cost was US$100 million due to the cancellation of the third development policy operation in April 2016. The cost of each of the two operations was US$50 million (ICR, page i). Financing. The AgDPO operation was designed as a programmatic DPO consisting of a series of three operations with achievements under each operation triggering a single-tranche payment. IDA disbursed two tranches of payments of US$50 million in 2013 and 2015. Borrower Contribution. There was no counterpart funding from the borrower, the Republic of Mozambique. Dates. The first Development Policy Operation was approved on 04/25/2013 and became effective on 09/15/2013 and closed on 12/31/2013. The second Development Policy Operation was approved on 03/30/2015, became effective on 06/29/2015 and closed on 03/28/2016. The Bank program team confirmed that the third operation was dropped in April 2016. 3. Relevance of Objectives & Design a. Relevance of Objectives Alignment with Government Strategy. The Government had developed a medium-term investment plan for the agricultural sector (Programa Nacional de Investimento do Sector Agrário, PNISA). The plan identified the need to create legal framework and policies to promote agricultural investment and critical public investment programs for achieving production and productivity objectives in the sector (ICR, para 4; PNISA, para 24, Table 1, para 30). Despite the existence of other programs supporting business environment reform, infrastructure development, financial inclusion, institutional capacity strengthening, service delivery and connectivity, some policy-related challenges remained for the objective of crowding in private sector investments in agriculture due to an outdated legal and regulatory framework that prevented access to larger regional markets (ICR, para 6). Furthermore, the AG DPO series, the first in the country, was instrumental in reducing the estimated financing gap of 78 percent of the total budget of the Government Agriculture National Strategy 2013–2017 (PNISA) (ICR, para 7). Alignment with Bank Strategy. The program was also relevant to the Bank Strategies both at approval and program end. In 2013, the Country Partnership Strategy (CPS) for FY12-16 identified agriculture as "a natural source of employment, livelihood, and inclusive growth and poverty reduction" under pillar I and set to address Page 3 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MZ Agriculture DPO-1(P129489) "constraints to agricultural productivity", also relevant to the program objective of promoting "private sector-led agricultural growth to improve food and nutrition security" (CPS, table 3, para 74, 78). At the end of the program, the main objective of the CPS for FY 17-21 was to "creating more inclusive growth through employment promotion and improving productivity and competitiveness in a sustainable manner" (CPS, para 57). In particular, the program was well aligned to support the objective of increasing agriculture productivity, access to finance of household enterprises, and promote private sector development under focus area 1 (Promoting Diversified Growth and Enhanced Productivity) (CPS, table 2, para 58, 70, 71, 74). Relevance of original and revised objectives. The original program development objective was to "promote private sector-led agricultural growth to improve food and nutrition security" (AgDPO-1, PD, page vii). This program objective was then revised in the PD for AgDPO-2 dated February 27, 2015 becoming "to improve agricultural technology, enhance access to productive assets and financial services, and improve monitoring of sector performance" (page vii). The original program development objective was consistent and substantially relevant to the development challenges identified in the Government and World Bank strategies. However, the Country Management Unit and Government agreed to change the program development objective in the context of the second operation because "it was understood that the original PDO was set at a level significantly higher than what the program could achieve, and the Results Framework in the Program Document of DPO-1 did not contain indicators that could measure private sector-led growth, food, or nutrition security" (ICR, para 11). This Review notes that eliminating the objective to "promote private sector-led agricultural growth to improve food and nutrition security" in the revised program objective significantly reduced the programmatic perspective of the development policy operation to a narrow focus on improving various aspects of the agricultural sector's performance. Some of the prior actions were no longer closely related to a results chain leading to the revised objectives. The program's revised objective was still relevant to the strategic development objectives of the Government and the World Bank in Mozambique, but it was no longer relevant to an operation that would lead to "a set of policy, regulatory, and institutional reforms that are selected for their capacity to galvanize Mozambique’s agricultural transformation".(AgPDO-1 PD, para 5). Conclusion. Despite high consistency with Government and Bank strategies, the Relevance of Objectives is rated Modest due the narrow focus of the revised objective which is no longer congruent with most of the program's prior actions. Rating Modest b. Relevance of Design Theory of Change - The ICR states that the program "sought to stimulate better flow and speedier availability of improved and new varieties of seed, fertilizer technology, and better access to irrigation resulting from reforms to the legal, regulatory, and institutional environments that would, in turn, stimulate private sector investment in development and distribution of agricultural input technologies. The expectation is that availability of technologies would facilitate uptake by farmers and consequently lead to increased agricultural productivity. The rationale for promoting reforms that is conducive to the private sector is that the decades of public sector interventions has delivered too few technologies to farmers, thereby limiting the choices available from the Page 4 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MZ Agriculture DPO-1(P129489) outset" (para 41). The ICR also refers to evidence that research at the World Bank has concluded that "differences in domestic policies account for close to 50 percent of the variation in improved input usage at the household level in Africa, even where biophysical, demographic, and socioeconomic variables within the region are similar" (para 42). While agricultural development needs a conducive policy environment this Review notes that the revised program objective reduced the programmatic perspective of the development policy operation to a narrow focus on improving various aspects of the performance of the agricultural sector. Looking at the revised PDO, private sector investments completely disappeared from the PDO, and the focus of the program shifted to providing inputs to improve agriculture productivity (to improve agricultural technology, enhance access to productive assets and financial services, and improve monitoring of sector performance) (ICR, para 42-44). This narrow focus is inconsistent with a program operation and would be more appropriate for the PDO of an investment project. Moreover, the theory of change did not explain how the private sector would be incentivized to invest after the proposed regulatory reforms. It was simply assumed that the lack of growth in the agricultural sector was caused by "unpredictable business environment" that "discouraged private companies from introducing new varieties, especially for crops and countries with small markets" and that, once these conditions were fixed, private sector investment would follow (ICR, para 41-42). There was no evidence for these assertions and the policies needed to address the alleged problems. Hence, to the extent that this operation has any policy content, it did not address the policies required to establish incentives in the agricultural sector to adopt improved technologies, and enhance access to productive assets and financial services. Prior Actions. The program design was based on Government agreement to achieve three policy areas and after implementing 13 prior actions during three program phases (operations). According to the ICR, all prior actions for all three phases of the program were completed, but due to the cancellation of the third phase by Bank management, there is no legal and independent evidence that the prior actions for the third phase were satisfied. Policy Area 1: Improving Agricultural Technology. The objective of this policy area was to undertake policy, legislative and institutional reform to increase the use of seed and fertilizer, improve access to irrigation and improved nutrition (ICR, para 45). While the increased use of seed and fertilizer and the inclusion of targets for irrigation were relevant to the Policy Area and the program development objective, it is unclear why there was a separate focus on nutrition, as it was not relevant to the revised program development objective. The policy and related reforms were expected to lower entry cost for private sector participation in the agricultural sector and to speedier adoption of agricultural technologies and therefore support the achievement of the program objectives (ICR, para 45). The following prior actions were achieved: • The Government ratified Southern Africa Development Community (SADC) compliant seed regulations, and approved regulations for the protection of plant varieties that recognize SADC seed harmonization protocol • The Government ratified the fertilizer subsector regulations that include fertilizer authority provision of public services and requirements for economic agents that encourage private sector participation in the fertilizer value chain. These include the composition and operational procedures technical evaluation committee for the Registration of Fertilizers (CATERF). • The Government ratified the medium term investment plan for agriculture (PNISA) that includes the irrigation subsector, and approved an updated regulatory framework for irrigation associations defining Page 5 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MZ Agriculture DPO-1(P129489) mandates and accountabilities for the operation, maintenance and management of public irrigation infrastructure • The Government adopted the food quality standards for nutrient-fortified wheat flour and edible oil. The Ministry of Industry and Commerce and the Ministry of Health have jointly created of a commission on food fortification and the commencement of a food fortification program with the private sector. Policy Area 2: Enhancing access to Assets. The objective of this policy area was to support investment in agriculture and adoption of agricultural technology by improving access to land and finance (ICR, para 56). The reasoning behind this policy area was, as stated in the ICR, that when land rights are secured, farmers invest more in their farms and productivity improves, and purchase of inputs and technology adoption is often linked to better access to financial services (ICR, para 56). On land, the focus of the policy action was two-fold— simplification of procedures for issuance of land rights to associations, communities, and (individuals); as well as the simplification of the policy/procedures for leasing land. Regarding access to financial services, the focus was to improve access to finance for farmers and implement a warehouse receipt system. These issues were well aligned to the program objectives. The following prior actions were achieved: • The Government completed the simplification process and reduction of time lines for the transfer of rural land user rights for rural land parcels measuring less than 10 hectares. • The Ministry of Agriculture ensured completion of stakeholder consultation process on new procedures for issuance of third party land use rights by communities, individuals, and associations • The Government submitted to its Council of Ministers for ratification the national strategy for agribusiness (PNDA) including a sound policy agenda and a comprehensive implementation plan to promote the development of rural financial services Policy Area 3: Monitoring sector performance. The objective of this policy area was to improve availability and reliability of agricultural statistics in order to better inform priorities for research and agricultural investment. Availability and reliability of agricultural data was relevant to the program objectives, original and revised, as an essential step to formulate other policy. The following prior actions were achieved: • Implementation of the new integrated agriculture household survey covering the 2012/13 agriculture season that replaced the early warning system (aviso previo) and the Agricultural Household Survey (TIA), as evidenced by the publication of the 2013 agriculture sector review Timing. All prior actions for phase 1 and phase 2 were on time, and, according to the ICR, all prior actions for all three phases of the program were completed by 2015/2016 but, as mentioned, due to the cancellation of the third phase before the ROC, there is no legal and independent evidence of that the prior actions for the third phase were satisfied. Macroeconomic Framework. The AgDPO-1 PD stated that an adequate macroeconomic framework was in place at appraisal, including an appropriate treatment of macroeconomic risks and mitigating factors (AgDPO- Page 6 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MZ Agriculture DPO-1(P129489) 1, paras 33, 47, 175, 179, 181; AgDPO-2, para 118). Fiduciary risks were also considered low (AgDPO-2, para 126). Despite these statements, Mozambique's macroeconomic framework in the years 2013-2015 was assessed in many reports to be weak because of substantial balance of payments deficit and rapidly rising public debt. Conclusion: While prior actions were well designed and identified key areas where policy and institutional framework change were needed, the restructuring of the program development objective significantly reduced the programmatic thrust of the program. Together with a weak macroeconomic framework, the conditions for a successful development policy operation were, in retrospect, not ideal. Therefore, relevance of design is rated Modest. Rating Modest 4. Achievement of Objectives (Efficacy) PHEFFICACYTBL Objective 1 Objective To improve agricultural technology, enhance access to productive assets and financial services, and improve monitoring of sector performance Rationale This objective was made up of three subobjectives and will be evaluated accordingly. Sub-objective 1: To improve agricultural technology The Government undertook actions to increase the uptake of improved seed, and fertilizer and to develop additional irrigation areas. The production data presented were based on estimates of maize, the most common crop in Mozambique. According to the ICR, the program policies led to an increase in supply of maize, driven by a streamlined import system for technology, availability of seed in small packages of 2 kg or less, and a growing network of agro-dealers, driven by the upgrading of five regional laboratories and the training of 20 staff. Outputs: • Between 2010 and 2016, the number of seed companies had increased by 85%, especially in the last 3– 5 years. (ICR, para 48) • An additional 3000 Ha of irrigated land is managed by irrigation associations (10% improvement), consisting in 32 irrigation schemes (ICR, PDO Indicator 3). Of these, about 43 percent of irrigation organizations supported by the PROIRRI have at least 90 percent recovery of operation and maintenance costs. • Increase in the number of improved seed varieties released between 2011 and 2014, 52% of the total Page 7 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MZ Agriculture DPO-1(P129489) was released between 1988 and 2014. Aggregate improved seed sales in 2016 were 4,375 tons of maize, 650 tons of rice, 364 tons of cowpea, and 689 tons of soyabean seeds. (ICR, para 48) • Increase in the number of agrodealers, from 150 to 600 (ICR, para 51) • Increased access to fertilizers, as availability doubled from 50,000 to 100,000 tons in 2015. Availability doubled as 20 imported fertilizer products entered the import market and blending, testing and approval by the National Directorate of Agricultural Services was undertaken. (ICR, para 51) Outcomes • 8.4% of farmers use certified seeds from a baseline of 6% (target 9%). According to the ICR, the seed- to-grain price ratio for hybrid maize and variety maize also suggests that hybrid maize is an attractive option for smallholder farmers (ICR, para 47). • 6% of farmers use fertilizers from a baseline of 2.7% (target 4%). According to the Bank's program team, this improved use of fertilizers was also supported by a public fertilizer and seed catalogue, which was intended to increase awareness about available seed and fertilizers, and was supported by the DPL. • Absolute fertilizer consumption quadrupled, increasing from 51,400 metric tons in 2010 to about 189,160 metric tons in 2014. Relative fertilizers consumption also increased from 6kg/ha to 15kg/ha between 2012 and 2015 (ICR, para 51). • While average yield for cereals has improved from 630 kg/ha to 703 kg/ha (12% increase) and production has increased from 1.5m MT in 2012 to 1.7m MT in 2014, it is unclear if this increase is entirely or in part attributable to the program (ICR, table 6). Sub-Objective 1 conclusion: Despite most outcomes being achieved and most targets surpassed, the achievement of these outcomes represented only a marginal improvement in Mozambique's agricultural sector. Sub-objective 2: To enhance access to productive assets and financial services The Government undertook actions to improve land rights, access to land in order to facilitate investments in productive assets and access to finance (which in turn facilitates investments in productive assets). Outputs: • Over 10,000 land user rights were issued to associations (target 260) and 325 certificates of delimitations were issued to communities (target 325) (ICR, table 4) • As third-party lease approval for land rights were not completed during the implementation of the series, land obtained under the Land Use Right system is not considered to be acceptable as collateral. (ICR, para 57). Under the current system, land is owned by the state, and the Land Use Right holder only has the rights to use the land (ICR, 57) • Banks are still reported to be reluctant to provide loans based on this land rights system even at interest rates in the range of 25%–30% (ICR, para 57) • A nascent warehouse receipt system is being implemented n Chimoio where the first warehouse receipt system issued in June was used to access the loan (ICR, para 58) • Accumulated Savings and Credit Association reached 300,000 members in 2015 from roughly 75,000 in Page 8 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MZ Agriculture DPO-1(P129489) 2012 (ICR, paras 57-58) Outcomes • Share of rural households with access to credit improved from 2.3% in 2012 to 8% in 2016 (target 3.5%) (ICR, PDO indicator 6) • Despite being a key reason behind the operations (according to the Bank's program team), investments from the private sector were not “crowded in”. Only US$612,526 worth of investments were pledged as a consequence of the Land Rights System reform, and only US$152,710 was invested between 2013 and 2015 (ICR, para 58). To provide a proxy of comparison, according to WITS, in 2016 Mozambique imported US$70 million worth of mixed mineral or chemical fertilizers (HS code 3105). Sub-Objective 2 conclusion: This sub-objective achieved and surpassed its targets. Nevertheless, the achievements these outcomes were small and insignificant compared to the size of an economy such as Mozambique. Sub-objective 3: To improve monitoring of (Agricultural) sector performance The Government’s actions consisted of the adoption of a master plan for agricultural statistics, creation of a new department, harmonization for all data collection instruments and institutions into a single agency, development of core set of indicators, and implementation of a new integrated household survey and the decentralization of operational resources for the implementation of the Agricultural Household Survey (TIA) at the provincial level. The adoption of these prior actions has resulted in: Outputs: • Data from the TIA has been used extensively by the Ministry of Agriculture and Food Security to analyze the agricultural economy and set priorities for research and agricultural investment. (ICR, para 60) • The Ministry of Agriculture now publishes annual performance reports and generates agricultural statistics, for example, the Anuario Estatistico 2012–2014 (ICR, para 60) • Coordinating the synchronization of disbursement for operational expenses of the TIA needs further improvement. Outcomes: • According to the ICR, improved monitoring of the performance of the sector by the Ministry of Agriculture and improved monitoring of the progress on some shared development commitments were achieved by the program (ICR, para 61) Sub-objective 3 conclusion: Sub-objective 3 achieved in its major targets Conclusion: In summary, while the program achievements led to only modest improvements for the productivity and growth of Mozambique's agricultural sector, most of the these achievements met their targets. This Review has therefore rated the program's efficacy as substantial. Page 9 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MZ Agriculture DPO-1(P129489) Rating Substantial PHREVDELTBL PHREVISEDTBL 5. Outcome The relevance of objectives and design was rated modest due to the revision of the program's objectives that diminished the programmatic perspective of this development policy operation to a narrow focus of improving some specific aspects of the performance of the agricultural sector. On the other hand there were substantial, though limited, outcomes that met the targets set at the program's appraisal. This Review therefore rates the project's outcome as moderately satisfactory. a. Outcome Rating Moderately Satisfactory 6. Rationale for Risk to Development Outcome Rating This Review agrees with the ICR that there are substantial risks to the Development Outcome. According to the ICR (para 70), substantial risks were related to land policy reform, warehouse receipt and access to finance, donor financing affected by nondisclosure of debt, irrigation schemes not maintained and cost recovery of irrigation not enforced. This Review also agrees with the stated risks, and also consider failure of private sector investments materializing regardless of the regulatory changes as another significant risk to development outcome. The ICR points out that many of these risks may not materialize and lists a number of other projects addressing these issues (ICR, para 70). The few explicit risk mitigation measures discussed in the Project Documents for AgDPO-1 and 2 revolve around the highly participatory nature of the program among government institutions, the participation of development partners in the sector reform agenda, and policies designed to counter real exchange rate appreciation supported by the Government, the Central Bank and the IMF (AgDPO-1, para 175 and following; AgDPO-2, para 117 and following). However, the lack of private sector investment in the agricultural sector by small or large-scale enterprises in Mozambique is not discussed in the ICR. Neither does the ICR discuss the prospects for an increase in private sector investment in the future. a. Risk to Development Outcome Rating Substantial Page 10 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MZ Agriculture DPO-1(P129489) 7. Assessment of Bank Performance a. Quality-at-Entry • As mentioned in Section 3, the original program development objective to "promote private sector-led agricultural growth to improve food and nutrition security". However, the Country Management Unit and Government agreed to amend this objective to "improve agricultural technology, enhance access to productive assets and financial services, and improve monitoring of sector performance" in the context of the appraisal of AgDPO-2 because "it was understood that the original PDO was set at a level significantly higher than what the program could achieve, and the Results Framework in the Program Document for DPO-1 did not contain indicators that could measure private sector-led growth, food, or nutrition security" (ICR, para 11). This revision reduced the original programmatic perspective of the development policy operation to a too narrow focus on improving some aspects of the performance of the agricultural sector. • The development policy operation supported the Government-owned priorities and focused on selected policy areas that, according to the ICR, were grounded in a sound background and analytics (ICR, para 74). Policy areas and prior actions of the project focused on key elements of Mozambique's very low agricultural productivity, in an effort to crowd in (private) investments (ICR, para 1-2, table 1). The design of the program activities was modestly relevant to the revised program development objective. A few changes were approved in the prior actions between the first and second operations, but these were minor (ICR, Table 2). • The team drew lessons from similar operations from similar projects in West Africa (Ghana, Niger, Nigeria and Senegal) and the series of PRSCs in Mozambique (ICR, 71 and AgDPO-1 PD, 72-78). As noted already, according to the ICR, the design of the project benefited from a large body of analytics and knowledge related to identification of binding constraints and critical issues to be supported by the prior actions (ICR, 71). • While many indicators were output oriented, the results framework was appropriate to measure results relevant to the revised program development objective of "improving agricultural technology, enhancing access to productive assets and financial services, and improving monitoring of sector performance". It would have not been for the original objective. • Key risks were identified and the mitigation measures were mostly adequate. However, political and macroeconomic risks that materialized and caused the lapsing of the series were not foreseen. Nevertheless, according to the ICR, the risk that a sectoral programmatic series would lapse is common in Africa, and, therefore, "the team should have considered this risk in sequencing policy reform" (ICR, para 28). Conclusion: While policy areas were relevant and the result framework relevant in measuring results from the revised program objective, the revision of the program development objective significantly worsened the programmatic perspective of the DPOs such that the entire design of the DPO series became questionable. Therefore, Quality at Entry (based on AgDPO-2) is rated Moderately Satisfactory. Page 11 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MZ Agriculture DPO-1(P129489) Quality-at-Entry Rating Moderately Satisfactory b. Quality of supervision • According to the ICR, there was a strong engagement with Government officials and other development partners throughout the series, and remained strong even after a change in Government (ICR, para 72). The Bank's project team accordingly fit the preparation timeline with the client’s political transition cycle. • The Bank's project task team leaders were based in-country and able to mobilize several TA to strengthen institutional capacity for project implementation, including on land administration (ICR, para 72). • The Bank's project team was able to recognize shortcomings in the results framework in the extent to which it did not provide evidence to substantiate claims of achievements of the PDO, which led to the restructuring of the PDO (ICR, para 11). Despite this positive initiative, the revised PDO significantly reduced the programmatic perspective of the program series. • According to the ICR, the Bank's program team effectively leveraged links with other World Bank operations and broad support from donors. For example, it organized targeted TA from donors that covered many areas of reform, including institutional capacity strengthening for implementing the reforms (ICR, para 28). Conclusion: The Bank's project team well supervised the project, and realized the shortcomings in the Results Framework. Instead of changing the Results Framework, the team and the Government decided to restructure the PDO of the series. This restructuring, as previously mentioned, reduced the programmatic perspective of the series, and weakened the relevance of the PDO. Bank supervision is rated by this Review as Moderately Satisfactory. Quality of Supervision Rating Moderately Satisfactory Overall Bank Performance Rating Moderately Satisfactory 8. Assessment of Borrower Performance a. Government Performance • The Government showed interest and support in the Bank’s program, and remained engaged with the World Bank during appraisal and implementation, the Fund, and other donors (ICR, para 75). • There were delays ion meeting prior actions of AgDPO-2, which affected the program disbursement time line (ICR, para 75) • Failure by the Government to disclose commercial debt (the hidden loans crisis) was the reason for the cancellation of the third AgDPO (ICR, para 33) Page 12 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MZ Agriculture DPO-1(P129489) Conclusion: While, overall, the Government supported the program, it was also the cause of the cancellation of AgDPO-3 which was due to its non compliance with a commitment to the Bank to maintain an adequate macroeconomic framework . Coupled with delays in meeting prior actions, Government performance is rated Moderately Unsatisfactory. Government Performance Rating Moderately Unsatisfactory b. Implementing Agency Performance According to the ICR, the Agency for the Development of the Zambezi Valley Corridor was responsible for the coordination of the program (ICR, para 76). While the agency overall performed well, there were delays in meeting effectiveness conditions for the first two operations and in completing some triggers for AgDPO-3 because of coordination problems with the line ministries responsible for implementing reform actions (ICR, para 76). Implementing Agency Performance is, therefore, rated Moderately Satisfactory. Implementing Agency Performance Rating Moderately Satisfactory Overall Borrower Performance Rating Moderately Satisfactory 9. M&E Design, Implementation, & Utilization a. M&E Design • The policy areas for intervention were selected and appropriate for a DPO series targeting the agricultural sector • The results framework at approval was inadequate as it did not contain appropriate indicators to provide evidence for "promoting private sector-led agricultural growth". During the preparation for the second operation, the PDO was changed to better reflect data coming from the Results Framework, and therefore this issue was resolved. Nutrition related aspect were still included in the Results Framework, although it is unclear why as nutrition related references were cut from the revised PDO. • There were indicators for each policy area that were well designed. Two indicators were output oriented (indicators 3-5, 7) and three were outcomes (indicator 1, 2 and 6). However, according to this Review, the targets were low and unambitious, and as a results, they were often surpassed by over 100%. • Between the first and second operations there were minor changes in indicators. b. M&E Implementation • According to the AgDPO-1, the Ministry of Planning and Development (MPD) had the overall Page 13 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MZ Agriculture DPO-1(P129489) responsibility the implementation of the AgDPO series, while, according to the ICR, it was the Ministry of Economy and Finance (MEF) (ICR, para 31; AgDPO-1, PD, para 162). The Bank's program team confirmed that this discrepancy was due to institutional changes following the 2015 elections. The designated agency to lead interministerial task force for the operation was the Agency for the Development of the Zambezi Valley Corridor (ADVZ) (ICR, para 31; AgDPO-1, para 162). Furthermore, implementation of sector-specific reforms was carried out by line ministries and sector agencies (ICR, para 31). Reporting on the implementation of the reform actions was done as part of the internal progress report for each of the line ministries. • According to the AgDPO-1, the World Bank program team was to use the preparation of the AgDPO-2 and AgDPO-3 as an opportunity to monitor overall progress toward the expected outcomes of the DPO series, as well as ratification by the Council of Ministers of the National Strategy for Agribusiness (PNDA). The Bank's program team focus was on the impact outcomes of the program and identify the adjustments needed as the series evolves, particularly with respect to the subsequent selection of Triggers and Prior Actions (AgDPO-1, PD, para 164). • While the results framework was inadequate at first, the PDO was restructured during implementation between the first and second operation. The restructuring of the PDO made the results framework adequate and more relevant. c. M&E Utilization • The Bank's team, together with the Government of Mozambique, used data from the Results Framework to realize the need to restructure the program development objective. • The performance of the series was tracked using the MASA integrated annual household survey and studies by other donors active in the sector. • According to the ICR, Information from the DPO series was used for the tracking of progress of the Cooperation Framework under the New Alliance in Mozambique and the Comprehensive Africa Development Programme Joint Sector Review (ICR, para 33). M&E Quality Rating Substantial 10. Other Issues a. Environmental and Social Effects • According to the AgDPO-1 Program Document, the Prior Actions and Triggers proposed under the AgDPO series are not likely to cause significant negative effects on the country’s environment, forests, or other natural resources. However, special attention is required in implementing the agricultural technology Page 14 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MZ Agriculture DPO-1(P129489) measures under AgDPO Policy Area #1 (AgDPO-1, para 160). For example, the Technical Evaluation Committee for the Registration of Fertilizers (CATERF) was designed to include a representative from Ministry for the Coordination of Environmental Affairs. • The Ministry for the Coordination of Environmental Affairs (MICOA) already conducted a comprehensive Strategic Environment and Social Assessment for the Sustainable Irrigation Development Project (PROIRRI), and produced environmental safeguards tools for mitigating the potential impact of increased used of agricultural inputs and land PROIRRI and for the Agricultural Productivity Program for Southern Africa (APPSA). These two operations require on-farm soil testing to be carried out prior to any investments (e.g on irrigation infrastructure) to help determine the incremental nutrient value required form fertilizer applications. b. Fiduciary Compliance According to the DP for AgDPO-1, there have been significant improvements in the management of public finances and there have been successful PFM reforms in the country, despite this challenges persisted around internal controls and information gathering at decentralized levels, PFM staffing, delayed releases of budget and off-budget spending (AgDPO-1, para 165) Concerning specifically the agriculture sector, the sector was facing a series of public finances management problems constraining the efficient execution of the budget due to "(i) complicated and lengthy procurement processes, (ii) the need to close the accounts of one year before beginning to spend against the following year’s budget, and (iii) delays in the disbursement of external funds" (AgDPO-1, para 168). As a consequence, the Ministry of Agriculture (MINAG) was only able to execute 68 percent of its recurrent budget and 32 percent of its investment budget from government own resources. • There has been significant support for PFM reform from the Bank and other donors, through various projects and the PRSC series. The Bank's program team also confirmed that there were many ongoing PFM initiatives and a Governance Program for Results (P124615), which the DPL operation leveraged. The Bank is also planning to support Government’s new PFM for Results Program which aims to deepen the PFM reforms in sectors and at decentralized levels (AgDPO-1, para 169). • There was no need for ongoing financial management support from the DPL series as the Bank credit would be disbursed in a single tranche payment. As a consequence, it was decided at approval that the proposed credit would be disbursed following standard IDA procedures (AgDPO-1, para 165). • Because of the failure by the Government to disclose non-concessional commercial loans, the financing by the Bank of the third DPO was cancelled. c. Unintended impacts (Positive or Negative) According to the ICR, The operations benefited the Government in the preparation of the Cooperation Framework for the New Alliance, and provided confidence for the development partners as the operation Page 15 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MZ Agriculture DPO-1(P129489) aligned to the priority of the donor community and reinforced mutual accountability for sector development (ICR, para 69). d. Other --- 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment Relevance of objectives and design were both rated modest due to the diminished Outcome Satisfactory Moderately Satisfactory programmatic approach of the revised program objective. Efficacy was rated substantial. Risk to Development Substantial Substantial --- Outcome Moderately Bank Performance Moderately Satisfactory . Satisfactory Moderately Borrower Performance Moderately Satisfactory --- Satisfactory Quality of ICR Substantial --- Note When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate. 12. Lessons Summary of lessons learned: • DPO series objective should be concrete and measurable, linked to to implementation. Achievements stemming from prior actions should be strongly focused on the PDO and well-measurable. • Data availability and instrument should be well planned in advance. It is important to ensure that any risks related to data availability are identified and accounted for upfront before concluding project preparation, and inform the Results Framework (ICR, para 80). • Technical assistance facilitates the positive results of a DPL. The team rightly identified the need to build both administrative and technical capacity of institutions that implemented reforms and was instrumental in shortening the learning curve (ICR, para 81) Page 16 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MZ Agriculture DPO-1(P129489) • The design of a programmatic series should account for the risk of cancellation, particularly if it is a common occurrence in the region. The termination of the third operation in the series could have been mitigated by better sequencing. (ICR, para 83). 13. Assessment Recommended? No 14. Comments on Quality of ICR • The ICR is clear and well written • The ICR could have better discussed the shortcomings of the program in its efficacy section, including the limited achievements. a. Quality of ICR Rating Substantial Page 17 of 17