ICRR 13469 Report Number : ICRR13469 IEG ICR Review Independent Evaluation Group 1. Project Data: Date Posted : 11/24/2010 PROJ ID : P075184 Appraisal Actual Project Name : Diversified Farm US$M ): Project Costs (US$M): 69.11 28.67 Income And Market Development Project Country : Philippines Loan/ US$M): Loan /Credit (US$M): 60.00 26.98 Sector Board : ARD Cofinancing (US$M): US$M ): Sector (s): Agricultural marketing and trade (60%) Agricultural extension and research (15%) Sub-national government administration (15%) Central government administration (5%) Roads and highways (5%) Theme (s): Rural services and infrastructure (25% - P) Export development and competitiveness (25% - P) Infrastructure services for private sector development (24% - P) Technology diffusion (13% - S) Rural markets (13% - S) L/C Number : L7236 Board Approval Date : 06/22/2004 Partners involved : Closing Date : 06/30/2009 06/30/2009 Evaluator : Panel Reviewer : Group Manager : Group : Ridley Nelson Robert Mark Lacey IEGSE ICR Reviews IEGSE 2. Project Objectives and Components: a. Objectives: The objectives of the project, stated the same in both the PAD and the loan agreement, were : to strengthen the capacity of the Department of Agriculture (DA) to provide market oriented services to increase agricultural competitiveness and rural incomes . b.Were the project objectives/key associated outcome targets revised during implementation? No c. Components (or Key Conditions in the case of DPLs, as appropriate): There were five components: 1. Support for Market Development Services (Appraisal US$17.16 million, Actual US$8.75 million). To strengthen the capacity of the Agriculture Marketing Assistance Service of the DA to provide more effective market promotion in conjunction with the private sector and to establish an Agriculture and Fisheries Market Information System. 2. Market Development Investments (Appraisal US$22.25 million, Actual US$2.33 million). For market development investments applying improved results allocation criteria in partnership with local government units and the private sector in four regions . 3. Strengthening Safety and Quality Assurance Systems for Market Development (Appraisal US$17.33 million, Actual US$8.57 million). To improve the implementation capacity of the DA ’s regulatory services and in particular to meet international standards supporting the DA ’s six regulatory agencies . 4. Market -linked Technology Development and Dissemination (Appraisal US$9.66 million, Actual US$5.73 million). To improve the DA’s R&D and to provide training through the strengthening of the Bureau of Agricultural Research and the Bureau of Postharvest Research and Extension using competitive grants and the DA ’s Agricultural Training Institute. Enhancing Budget Resource Allocation and Planning (Appraisal US$2.72 million, Actual US$3.29 million). To support a government-wide initiative to improve public expenditure management giving more emphasis to the funding of the DA's core functions related to market development . d. Comments on Project Cost, Financing, Borrower Contribution, and Dates: Due to weakening support for the project and implementation difficulties, the project disbursed only about 40% of the planned amount. There was no restructuring or extension . The component for Marketing Development achieved only about 10% of the planned expenditure. 3. Relevance of Objectives & Design: Relevance of Objectives . Rated Substantial. The objectives were consistent with the current 2004-2010 Medium Term Philippine Development Plan and the Agriculture and Fisheries Modernization Act . They were also consistent with the Bank’s 2003-2005 CAS which aimed at sharpening lending support to address the declining competitiveness of Philippine agriculture. Relevance of Design . Rated Modest. The reason for this lies partly in the area of relevance of the design to the objectives and partly in the more general area of quality at entry picked up again in Section 8. QAG, while being inclined to rate Quality at Entry less than moderately satisfactory, was in the end persuaded by the region that it fell above the line on that rating. QAG's reservations were due to a lack of clear link between inputs and outputs . This review agrees with that earlier position . There is some question about whether the project as designed could have actually met the objectives. This review questions whether the shift towards performance -based-budgeting was realistic for such a project since a switch of this type is usually controversial within the public service; it might be expected to take a long time and have repercussions across other departments . If the reform was essential for achieving the objectives then there was a design weakness in not having it in place given the time generally needed for such a shift. If it was not essential then it was not needed within this sectoral project . It seems that the DA was being used in this project as an experiment . The ICR notes that the choice of lending instrument in the form of a Sector Investment and Maintenance Loan was appropriate since the alternative, a Sector Investment Loan, being a non-regular budget fund source, may not have been able to sustain the increase in the budget for market oriented services beyond the project period . 4. Achievement of Objectives (Efficacy): The extent to which the project assisted in strengthening the capacity of the DA to provide market oriented services. Rated Negligible. The ICR notes that the performance indicators called for a fundamental shift in the way the DA conducted its business . This was to call for a 15% increase from the established baseline in the proportion of resources supporting market development services and a 35% increase from the established baseline in client satisfaction with the delivery of market information, development services and market related investments . The trend in the budget share of market oriented services and investments was small but positive for the first three years but subsequently declined to below the baseline as an increasingly production -oriented approach took hold . While the actual amount spent on market services did, in fact, increase due to a rising total budget, the share, which was the agreed target, fell. The initial client satisfaction survey showed a very low level of satisfaction with services, a follow-up survey has been delayed to December 2010. The ICR anticipates little if any improvement in client satisfaction. Notwithstanding this poor performance, there were some positive achievements in intermediate outcomes . For example, the use of the operations manual was standardized, a web -based regulatory information system was established, under the quality assurance component there was an approximately 20% increase in the number of accredited private sector operations, and implementation of a competitive research grants program . Overall there was a lack of consistent support for the project and the reforms partly due to strong political pressures for self-sufficiency and productivity . The extent to which the project supported an increase in agricultural competitiveness and rural incomes . Rated Negligible. There is no evidence that the project contributed to increased agricultural competitiveness and rural incomes. In any case, given the poor levels of client satisfaction any positive gains would be difficult to attribute to the project. Indicators for agricultural competitiveness and rural incomes include a decrease in the number of export rejections at destination ports, an increase in the number of private -sector firms engaged in processing, marketing and exporting, reduced post -harvest losses, and increases in farm profitability and diversification . These indicators were inadequately monitored by the DA . However, EU data shows that the number of detentions and rejections of Philippine agri-food exports increased over the period and these rejections and detentions increased at the same rate as exports. Therefore, the ICR notes that it appears that the project has been unable to strengthen the quality assurance system. With respect to post-harvest losses in rice, there was only a marginal decline in losses since 1994 although corn performed better . With respect to profitability, data from the Bureau of Agricultural Statistics suggests a tendency towards improved profitability of import competing products but a worsening profitability of exportable farm commodities over the project period . The ICR notes that this is consistent with a focus on self -sufficiency contrasted with the promotion of competitiveness for the whole Philippine agri -food system. 5. Efficiency (not applicable to DPLs): While there was no economic or financial analysis in the PAD, the ICR carried out some economic and financial analysis for selected infrastructure sub -projects under the Market Development Investments component . These analysis covered particularly farm to market roads, trading centers, hauling trucks and other facilities . These exhibited satisfactory to high ERRs . For example, farm to market roads predominantly exhibited ERRs over 25% and trading posts exhibited ERRs of 19% and above. However, the ICR notes that the total amount spent for the market development investment component represented only 8% of total project cost (compared to the 32% estimated at appraisal) so these percentages do not offer sufficient basis for an overall project rate of return estimate . Only 49% of the loan was disbursed although the DA spent many times that amount on activities that would have been eligible for reimbursement but were not claimed, apparently because of unwillingness to complete the documentation and because the budget had, in any case, already been released to the DA so there was no incentive to do so. Government therefore paid substantial unnecessary commitment charges . Given the negligible rating for the achievement of the objectives and the failure to enhance competitiveness, notwithstanding the lower than planned expenditure, Efficiency cannot be rated more than negligible . ERR )/Financial Rate of Return (FRR) a. If available, enter the Economic Rate of Return (ERR) FRR ) at appraisal and the re- re -estimated value at evaluation : Rate Available? Point Value Coverage/Scope* Appraisal No ICR estimate Yes 20% 8% * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome: While project design was largely consistent with the borrower development framework and the CAS, the project failed to bring about changing the way the DA does business . The project was inefficient in its use of resources . Overall, although it delivered some of its intermediate outcomes, it failed to improve agricultural competitiveness and rural incomes. a. Outcome Rating : Unsatisfactory 7. Rationale for Risk to Development Outcome Rating: The ICR rates risk as High. This review concurs. The failure to implement the AFMA for 12 years, and the failure to utilize this project to implement it, poses a high risk for what limited achievements were made and for Philippines agriculture. a. Risk to Development Outcome Rating : High 8. Assessment of Bank Performance: Quality at Entry . Rated Moderately Unsatisfactory on balance . The ICR rated Moderately Satisfactory and a QAG quality at entry review rated project preparation Moderately Satisfactory but with reservations . As noted under Relevance of Design, QAG found weakness in the linkage between expenditures and outcomes . The Bank team responded that previous project experience cautioned against over -ambitious objectives. However, the ICR notes that the slow implementation of the new AFMA act in the first six years leading up to the project and the dropping of a planned condition of negotiations to restructure the system of budgeting should have raised more questions about the level of commitment to the strategy and the objectives . This review agrees. There was an unwarranted degree of optimism. The ICR notes that both the Bank and the Oversight Agencies believed that, through the planned public expenditure reforms and liquidity support, the DA could be sufficiently induced to modify its procedures for planning and resource allocation . The ICR places more blame on the borrower than the Bank, in particular noting that it was predominantly the waning of political support for the AFMA and public expenditure management reforms during implementation that undermined the project . But the Bank also played a significant role in the design weaknesses . Quality of Supervision . Rated Moderately Unsatisfactory . The ICR rates supervision Moderately Satisfactory indicating that there was responsive supervision by the Bank and a high degree of coordination with the various responsible agencies. On the positive side, there were comprehensive review missions and cluster reviews of project components. Since the project was managed from the Manila office it was possible to have regular monthly and sometimes weekly reviews . The Midterm Review was comprehensive . There was substantial input from Bank procurement financial and safeguards staff . Feedback from the borrower indicates that the support provided by the Bank was greatly welcomed . On the negative side, the ICR notes that the Bank team did not appear to have sufficiently tackled the DA ’s M&E performance. But the larger question is whether the Bank acted sufficiently strongly to the emerging disconnect between the project and the government strategy . Apparently, early in project implementation the Bank team was raising questions and was open to a redesign of the project . Starting as early as 2006 there was consideration of partial or full cancellation . However, the Bank team continued to give the implementing agency the benefit of the doubt in spite of repeated reneging on remedial actions. It should be noted that, the global food crisis did influence the government ’s attitude to productivity, trade and marketing strategies. The ICR itself raises questions about whether the Bank was sufficiently proactive given the emerging problems. The Bank chose to remain engaged . Bank staff in discussion with IEG argued that a project to respond to the food security issue was being prepared and that it would have sent conflicting signals to cancel the project while another in the same sector was being launched . However, on balance this review finds that the evidence suggests that the team and particularly Bank management should have drawn a line at some point in the first half of the project prior to or at the MTR and have suspended the loan . The food security problem was simply one aspect of an already diverging strategy that was for some time drifting away from the stated project objectives. It is mainly for this reason that, notwithstanding a number of aspects of strong supervision, quality of supervision is overall rated Moderately Unsatisfactory . At some point it should have been clear that government commitment to the project strategy was simply insufficient . at -Entry :Moderately Unsatisfactory a. Ensuring Quality -at- b. Quality of Supervision :Moderately Unsatisfactory c. Overall Bank Performance :Moderately Unsatisfactory 9. Assessment of Borrower Performance: Government . Rated Unsatisfactory. There were a number of failures caused by government . There was a failure to introduce the anticipated public expenditure reforms . Both the Department of Budget and Management and the National Economic and Development Authority were unable to meet their commitments . This was constrained by the administration’s increasing focus on production targets of rice and corn . As noted by the ICR, overall adequate oversight and follow through by the borrower was lacking . Implementing Agency .Rated Unsatisfactory. The ICR offers evidence of considerable commitment and ownership at the technical staff levels of the DA . However, DA management, due mainly to the changing priorities of the administration, lost commitment to the AFMA and the PEM reforms . There was repeated reneging on agreed actions to improve project implementation . How far responsibility for this should be placed on the agency itself and how much on the government as a whole is debatable . Both the ICR and this review place a degree of responsibility on both. a. Government Performance :Unsatisfactory b. Implementing Agency Performance :Unsatisfactory c. Overall Borrower Performance :Unsatisfactory 10. M&E Design, Implementation, & Utilization: Design . The ICR found strong links between the project objectives and the indicators and between the intermediate and secondary outcome indicators . Trends in export rejections and in the number of participants in post -production and agricultural activities were useful measures of the state of agricultural competitiveness . However, no baseline information was established. The location of M&E in the Program Monitoring and Evaluation Division (PMED) of the DA’s Planning Service was appropriate and used existing functional linkages with line agencies and service agencies and linked to the DA’s Regional Field Units. Implementation . Reporting was done by the five line and service agency clusters not by the PMED . This resulted in ad hoc reporting primarily undertaken in response to the Bank ’s regular supervision data requirements . The project M&E reflected the weak state of M&E in the DA due to limited staffing and a low value placed on M&E beyond simply tracking quantitative output targets . Utilization . With weak quality of data, utilization appears to have been limited although the ICR does not offer much evidence on utilization. The Bank missions appear to have been the main uses of the output data emerging . a. M&E Quality Rating : Modest 11. Other Issues (Safeguards, Fiduciary, Unintended Positive and Negative Impacts): Bank supervision missions predominantly rated compliance with environmental safeguards satisfactory and social safeguards moderately satisfactory . The main social concern was for the inclusion of marginalized social groups . On social safeguards there were some compliance issues due to the late training of project specialist staff and inadequate project site documentation . The main environmental issues were chemical use and some potential environmental impact of market development investments . The ICR reports that where deficiencies were noted they were acted on. In 2007, a safeguard policy refresher course was held . It is concluded that safeguards were complied with albeit with some delay in some cases . Procurement suffered from the usual problems of lack of familiarity with Bank procedures . Considerable technical support was required to familiarize government staff . Financial management was, for the most part, rated unsatisfactory in supervision reports due to delays in submitting reports (it is not clear in the ICR whether this includes audit reports ) and inadequate or incorrect documentation . The ICR finds the difficulties encountered to reflect a culture of inadequate financial accountability in the DA with inadequate oversight by the Department of Budget and Management . 12. 12. Ratings : ICR IEG Review Reason for Disagreement /Comments Outcome : Unsatisfactory Unsatisfactory Risk to Development High High Outcome : Bank Performance : Moderately Moderately IEG places more weight on the design Satisfactory Unsatisfactory weakness in the linkage between inputs and outcomes and the optimism of the institutional change sought and on the response to the declining commitment of the borrower and the weak remedial action. Borrower Performance : Unsatisfactory Unsatisfactory Quality of ICR : Exemplary NOTES NOTES: - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate . 13. Lessons: The ICR offers a number of well-crafted and institutionally and politically perceptive lessons . The most important are summarized here with some adaptations . 1. In a Sector Investment and Maintenance Loan that supports a sector reform program of difficult institutional reforms, it is important to determine that the client agency is fully committed and operationally prepared to mainstream the reforms within the project period . The responsible department needs to have adequate budgetary control. Risk is lowered if core elements of the reforms are already under way prior to project approval . This is especially important in countries or sectors where there is past evidence of a disconnect between policy and operations. Indications of sufficient commitment may include : an already evident shift in the relationship with the private sector; increased focus on diversification in the sector in line with comparative advantage; progress in complementary policies and programs; and evidence of a strong external constituency demonstrated by the parallel emergence of similar initiatives in other sectors . 2. In the absence of clear operational manifestations of reforms in the concerned agencies the Bank should limit its involvement to helping establish a solid foundation for reforms through focused Sector Investment Loans and Technical Assistance support, pilot programs, building capacity, and AAA activity . 3. In challenging sector reform situations the Bank may need to expand its range of engagement towards a broader constituency beyond simply a lead government sectoral department, engaging with the legislature including staff of various committees, private sector initiatives, agro -industry, agro-processers, supermarkets, and government agencies responsible for trade, industry or technology . (In this case, the oversight committees of the legislature expressed, in a stakeholder workshop, that they were especially keen on establishing a regular venue for knowledge sharing with the Bank .) 4. Cost-sharing issues between national and local governments, and the incremental resource incentives for government departments to participate in donor funded projects, need to be well understood not simply as issues of project design but as an aspect of the analysis of implementing agency commitment . 14. Assessment Recommended? Yes No 15. Comments on Quality of ICR: This Intensive Learning ICR is exemplary . While IEG experience has tended to be that it is easier to write an exemplary ICR about a good project, this was a poorly performing project with limited data . The ICR is very easy to follow, is open about the faults of the project, has an exceptional depth of institutional and political understanding of the complex issues, has balanced judgment that commends positive aspects of performance where due, and has formulated some exceptionally thoughtful lessons . It offers the results of two stakeholder consultations with useful findings. In a project which did not offer any quantitative economic analysis in the PAD, the ICR attempted to do at least some economic analysis of selected project investments . The ICR carefully distinguishes between outputs and outcomes. It offers a succinct and informative summary of the Borrowers comments . It is not entirely clear on page 10 of the ICR whether the delays in issuing reports referred to audits or to other financial reporting, but this is a very small point. a.Quality of ICR Rating : Exemplary