Document of The World Bank FOR OFFICIAL USE ONLY Report No. 20424 PERFORMANCE AUDIT REPORT MEXICO MEXICO ENVIRONMENTAL PROJECT (LOAN 3461-ME AND GET GRANT 28604) April 28, 2000 Operations Evaluation Department Sector and Thematic Evaluations Group This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed without World Bank authorization. Currency Equivalents Currency Unit = New Mexican Peso (N$) US$1.00 = N$ 3.12 (1993 average) US$1.00 = N$ 3.38 (1994 average) US$1.00 = N$ 6.42 (1995 average) US$1.00 = N$ 7.60 (1996 average) US$1.00 = N$ 7.92 (1997 average) US$1.00 = N$ 9.07 (1998 average) Special Drawing Rights: US$1.00 = 0.73 Weights and Measures Mexico uses the Metric System Abbreviations and Acronyms BANOBRAS Banco Nacional de Obras y Servicios P6blicos (National Bank of Public Works and Services) CNA, CONAGUA Comisi6n Nacional del Agua (National Water Commission) Cr6dito Piblico Undersecretary of Public Credit in the Secretary of Finance Egreso Undersecretary of Expenditures in the Secretary of Finance FMCN Fondo Mexicano para la Conservaci6n de la Naturaleza (Mexican Fund for the Conservation of Nature) GEF Global Environmental Facility GET Global Environmental Trust INE Instituto Nacional de Ecologia (National Institute of Ecology) MEP Mexico Environmental Project NAFTA North American Free Trade Agreement PAM Programa Ambiental Mexicano (Mexico Environmental Project) PROFEPA Procuradoria Federal de Protecci6n Ambiental (Office of the Attorney General for Environmental Protection) SARH Secretaria de Agricultura y Recursos Hidricos (Secretary of Agriculture and Water Resources) SEDESOL Secretaria de Desarrollo Social (Secretary of Social Development, the country's environmental agency from 1992 to 1995) SEDUE Secretaria de Desarrollo Urbano y Ecologia (Secretary of Urban Development and Ecology, the country's environmental agency from 1985 to 1992) SEMARNAP Secretaria de Medio Ambiente, Recursos Naturales y Pesca (Secretary of Environment, Natural Resources and Fisheries, the country's environmental agency since 1995) SINAP Sistema Nacional de Areas Protegidas (National Protected Areas System) SHCP Secretaria de Hacienda y Cr6dito Piiblico (Secretary of Finance) TAC Technical Advisory Committee Fiscal Year Government: January 1 - December 31 Director-General, Operations Evaluation : Mr. Robert Picciotto Director, Operations Evaluation Department : Ms. Elizabeth McAllister Manager, Sector and Thematic Evaluations Group : Mr. Gregory Ingram Task Manager : Mr. Andres Liebenthal FOR OFFICIAL USE ONLY The World Bank Washington, D.C. 20433 U.S A. Office of the Director-General Operations Evaluation April 28, 2000 MEMORANDUM TO THE EXECUTIVE DIRECTORS AND THE PRESIDENT SUBJECT: Performance Audit Report on Mexico Mexico Environmental Project (Loan 3461-ME and GET Grant 28604) Attached is the Performance Audit Report prepared by the Operations Evaluation Department on the Mexico Environmental Project (Loan 3461-ME) for which a loan in the amount of US$50 million was approved in April 1992. The project also included a GEF grant (GET Grant 28604) for the equivalent of US$30 million. The project was made effective on May 1993 and both the Loan and the Grant closed on December 1997, when $26.2 million of the loan amount and $21.3 million of the GEF grant amount were cancelled. The Mexico Environmental Project had four major objectives (a) to enhance Mexico's ability to carry out key environmental protection and natural resources management functions in the short term; (b) to promote the institutional strengthening and policy measures required to allow Mexico to carry out such functions more efficiently in the long term; (c) to transform the federal environmental agency into a small but highly qualified second tier organization that supervises and coordinate environmental work carried out by other levels of government; and to conserve biodiversity in selected protected areas. The objective of the GEF Grant was to undertake conservation management actions in 17 protected areas around the country. Mexico's 1994 financial crisis, two major institutional reorganizations of Mexico's environment sector, and a high turnover of Bank staff hampered project implementation. Altogether, these resulted in major delays, changes, restructuring, and cancellations. The GEF component also underwent major changes to cope with slow implementation, resulting in a full project restructuring in May 1997. Despite restructuring and downsizing, the project managed to deliver in many important areas. It enabled recruiting and training of personnel for the National Institute of Ecology (INE) and the Office of the Attorney General for Environmental Protection (PROFEPA) the newly created environmental agencies. It supported 6,000 environmental assessments and the supervision of 750 environmental audits and gave partial support for some 70,000 industrial inspections. The project also supported the establishment and operation of 5 wildlife rehabilitation centers and 12 sea turtle protection centers. In several cases the training activities backed by the project were delivered through local universities and reached a wider public outside the project staff, including businesses and NGOs. Finally the GEF Protected Areas Program supported the preparation of management plans for 9 protected areas, emergency and operation plans and follow-up activities in 10 protected areas, several productive pilot This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed without World Bank authorization. 2 projects in protected area buffer zones, and a strategy study for the management of the National System of Protected Areas. However, these achievements were far below the project's original design and even below its restructured goals. This was due mostly to lack of local funds, a cumbersome project design that tried to cater to too many demands, and major institutional changes in the implementing agencies. The audit upgrades the ICR outcome rating of unsatisfactory to marginally unsatisfactory, noting that some important project goals were achieved. The institutional development impact is rated modest; sustainability is rated likely; and borrower performance is rated satisfactory, all three in agreement with the ICR. The audit downgrades the ICR Bank performance rating of satisfactory to unsatisfactory, to underline the flaws and poor quality of project preparation and appraisal, that were only partially overcome through significant supervision efforts. This project is a good illustration of the Bank's 1999 Mexico Country Assistance Strategy statement that Bank lending to the country should move from activities to goals. Environmental projects usually encompass many small activities that are ill-suited to traditional Bank procedures. Such projects would be good candidates for the Bank's Adaptive Program Lending (APL) whereby full slices of the project are cleared in advance and the emphasis moves to auditing the outcomes and away from tracking progress in a myriad of intermediate activities. Another lesson is the need for more caution when addressing institutional strengthening strategies in the environmental sector, since evolutionary institutional arrangements have been the norm around the world over the last 20 years, as governments try different approaches to environmental management. Attachment 1 Contents Preface ............................................................................................................................................. v 1. Introduction and Background ........................................................................................... 1 Introduction........................................................... 1 Background.........................................................1 2. Objectives, Components, and Design .............................................................................. 2 Objectives........................................................... 2 Components......................................................... 2 Design ....... ...................................................... 3 3. Im plem entation........................................................................................................................4 4. Outcomes and Follow-up Activities ................................................................................... 5 Outcomes... ........................................................5 Follow-up Activities ......................................... ..........7 5. Findings .................................................................................................................................... 7 Project Achievements and Shortcomings .............7........ .............7 Sustainability........................................................ 8 Institutional Development Impact ..............................................9 Expectations for a GEF Project..............9............ ...............9 Bank Performance.................................................... 9 Borrower Performance ................................................. 9 Ratings Summary ............................................. ........... 10 6. Lessons .................................................................................................................................... 10 Basic Data Sheet ........................................................................................................................... 13 Annex: Comments from the Borrower .................................................................................. 17 This report was prepared under the supervision of Andres Liebenthal (Task Manager) by Mr. Pablo Gutman (Consultant) who visited Mexico in June-July 1999. William Hurlbut edited the report. Soon-Won Pak provided administrative support.  111 Principal Ratings ICR Audit Outcome Unsatisfactory Marginally Unsatisfactory Sustainability Likely Likely Institutional Development Partial Modest Borrower Performance Satisfactory Satisfactory Bank Performance Satisfactory Unsatisfactory Key Staff Responsible Task Manager Division Chief Country Director Appraisal Robert Kaplan David de Ferranti Edilberto Segura Completion Adrian Demayo Michael Baxter Olivier Lafourcade  V Preface This Performance Audit Report (PAR), prepared by the Operations Evaluation Department (OED), evaluates the Mexico Environmental Project (Loan 3461-ME), for which a loan of US$50 million was approved in April 1992. The project also included a Global Environmental Trust Grant (GET-28604) for US$30 million equivalent. The project was made effective on May 8, 1993, and both the loan and the grant closed on December 31, 1997, when $26.2 million of the loan amount and $21.3 million of the GEF grant amount were cancelled. This audit covers both the WB and the GEF component of the Mexico Environmental Project, but it should be noticed that as a result of the May 1997 restructuring of the GEF component US$16.3 million of the original grant were transferred to a new project (GET-2867) which is still active and is not evaluated in this audit. Following standard OED procedures, the draft PAR was sent to the borrower for comments which is attached as Annex to this report.  1 1. Introduction and Background Introduction 1.1 OED selected the Mexico Environmental Project (MEP) for audit because it was among the first and largest of the Bank's stand-alone multipurpose environmental loans and because the GET grant support for the project was also large. The implementation of the project was subject to major macroeconomic shocks and institutional changes and was sometimes affected by conflicting Bank and client views regarding management procedures and public participation. Since similar challenges may occur elsewhere, OED considered that MEP could offer valuable lessons for future project design and implementation. Background 1.2 By the late 1980s Mexico's authorities were fully aware of the severity and extent of the country's environmental problems, and of the very limited capacity of the government to address them. Rapid growth of industries like oil, petrochemical, and energy, all of them heavy polluters; improper disposal of solid and toxic wastes; extreme air pollution in major cities, particularly in Mexico City, which is home to more than 18 million people; overuse and pollution of water resources; soil erosion; loss of biodiversity and degradation of coastal and marine ecosystems. These problems were compounded by a limited or non-existent regulatory framework, poor institutional arrangements for dealing with environmental issues, lack of financial and human resources, and a weak environmental management tradition among government, industry, and public alike. 1.3 In 1985, the Government of Mexico (GOM) grouped federal environmental responsibilities into one body, he Secretary of Urban Development and Ecology (SEDUE), and in 1988, the Congress passed the country's environmental framework law, the General Law of Ecological Balance and Environmental Protection (LGEEPA). At the same time, several national and international events reinforced the visibility of environmental issues, and the government's willingness to address them. Among these influential events were: the opening up of the economy, the preparatory activities for the United Nations Conference on Environment and Development (Rio 1992), negotiations for the North American Free Trade Agreement (NAFTA), and becoming a member of the Organization for Economic Cooperation and Development (OECD). The 1989-94 National Development Plan was the first to identify "ecological balance" as a prime national objective, and soon after, the GOM and the Bank opened discussions on an array of large environmental projects of which MEP was part. The GOM interest in expanding its investment in environmental management matched a similar concern in the Bank, which was at the time increasing its environmental lending. 1.4 In preparing the batch of environmental projects that were launched in Mexico at the beginning of the 1990s, both Mexico's sectoral agencies and the Bank had in mind the lessons of a recent failure: the 1982 Mexico Pollution Control loan (2154-ME) the first operation of its type in the Bank. Designed to finance industries willing to invest in air pollution abatement, the project never took off, and at closure, less than 15 percent of the loan had been disbursed. Difficult macroeconomic conditions, high interest rates, and the absence of any enforcement of environmental regulations made industries reluctant to invest in pollution abatement. In the early 1990s the Bank supported the government's decision to take a comprehensive approach. Air pollution, water and sanitation, solid wastes, natural resources development, biodiversity protection, ozone protection, and more were targeted with specific projects. The project was 2 expected to strengthen institutions, regulations, and policies at the federal level and, to a lesser degree, to initiate decentralization of environmental management from the federal to the state level. 1.5 The Protected Area Programs supported by the GET Grant (28604) paralleled the MEP, taking advantage of the fact that the same institutions were involved in both initiatives. Mexico's rich biodiversity makes it the fourth ranking "megadiversity" country in the world. Yet its protected areas at the beginning of the 1990s were in very poor condition. The government had recently created the National System of Protected Areas (SINAP) to address conservation more systematically, but most protected areas remained "paper parks," without management plans, directors, or personnel. The global value of Mexico's biodiversity convinced GEF to support effective conservation management in a sizable number of protected areas. 2. Objectives, Components, and Design Objectives 2.1 The Loan Agreement for the project lists two major objectives for the project: (a) to enhance Mexico's ability to carry out key environmental protection and natural resources management functions in the short term and (b) to promote the institutional strengthening and policy measures required to allow Mexico to carry out such functions more efficiently over the long term. The SAR added several more-specific objectives: (c) to transform SEDUE into a small but highly qualified second-tier organization that supervises and coordinates environmental work carried out by other levels of government; (d) to support actions in key areas where pre- investment work is already well developed and institutional absorptive capacity is adequate- (e) to reform and strengthen the administrative structure of SEDUE's federal and state offices in preparation for increased decentralization of responsibility and authority to state environmental agencies over the medium term; and (f) to support development of a sectoral strategy. 2.2 The objective of the GET Grant was to undertake conservation management actions in 17 protected areas around the country. Those areas were selected based on the global importance of their biodiversity and endemic species. Components 2.3 The MEP was a complex project, with a large number of components.' The summary that follows is distilled from the Loan Agreement. Monitoring and Control of Pollution * Water Pollution: (a) redesigning the national water monitoring system; (b) training personnel; (c) information and studies, hardware, and software * Air Pollution: (a) installing air monitoring systems in 36 cities and enforcing compliance of stationary sources; (b) updating the National Information System for Stationary Sources; (c) increasing training and equipment in seven regional centers * Environmental Laboratories: (a) expanding, remodeling, equipping, and operating a central laboratory in Mexico City and six others in regional centers; (b) training personnel * Inspection and Enforcement: (a) training inspection and enforcement personnel 1. Presentation of the components took 14 pages in the SAR, and more than 3 pages in the Loan Agreement. 3 Environmental Impact Minimization * Impact and Risk Assessments: (a) staffing and training the office in charge of EIA review; (b) increasing public participation in EIA review process * Training on ecological planing and zoning, plus some 700 studies on environmental risks and accident prevention * Planing: (a) preparing ecological plans for 14 sectoral activities; (b) 20 environmental studies and 10 ecological plans in selected areas; (c) purchase of satellite imagery Biodiversity Conservation (excluding the GET Grant) * Wildlife Protection: (a) establishing and operating a network of seven surveillance centers; (b) conducting educational campaigns; (c) constructing and operating seven wildlife rehabilitation centers * Protection of Marine Turtles: (a) constructing and operating 12 camps; (b) conducting educational campaigns; (c) research Institutional Strengthening * Institutional strengthening of SEDUE * Pilot decentralization of environmental management to five states * Twelve studies in environmental and natural resource economics * Legal studies regarding environmental crime and the consistency among existing environment and natural resources legislation * Support for the preparation of some 100 standards and regulations and training on how to use them * Studies in support of the national protected area system * Studies regarding the creation of biological corridors * A study on environmental effects on health * A study on environmental health hazards in Tijuana city * A study on scenarios of Mexico's environmental future 2.4 The GET Grant project had four components: (a) preparing emergency plans for up to 13 protected areas; (b) preparing management plans for up to 17 areas; (c) carrying out activities and operating plans derived from the emergency and management plans; and (d) developing a tourism strategy for the national system of protected areas (SINA). Design 2.5 The complex and ambitious design of the project was the outcome of a dialogue between Bank staff and officers of the small environmental department of SEDUE. Although the process negotiated all required steps of the project cycle both at Mexico's Government and the Bank, there is no evidence in the project's files that a clear endorsement and support from higher government levels-not even from SEDUE's Secretary- was either sought or secured. The only expression of borrower's ownership put forward at preparation was a vaguely worded letter from the director of SEDUE' s environmental department. Implementation of the project was also left fully in the hands of SEDUE's environmental department, plus a project coordination unit (PCU) set up at the National Ecological Commission (an Inter-Ministerial consultative body, chaired by SEDUE). While the appraisal document acknowledges as a major risk SEDUE's limited capability to undertake the project, the only safeguards proposed were to closely monitor the project progress. Three supervision missions a year was proposed for the two initial years as well as annual comprehensive reviews. 4 2.6 An important design component of the GET Grant project was to strive for participation and consultation. This was to be achieved through the creation of local Technical Advisory Committees (TACs) in each protected area, the preparation of Indigenous Peoples Development Plans, and ample consultation on management plans with local communities. 3. Implementation 3.1 The original project total costs of US$126.6 million were to be met by the Bank loan of US$50 million, the GET grant for US$30 million, and a GOM contribution of US$46.6 million. The Bank loan, approved in April 1992, was to be disbursed during 1992-95 with some retroactive financing for eligible expenditures incurred in 1991. The GET grant was on the same disbursement schedule. All this changed during implementation because of several major events. 3.2 In May 1992, just one month after the Bank approved the loan and grant, Mexico's Government drastically reorganized the environmental sector, closing the Secretariate of Urban Development and Ecology (SEDUE). Environmental responsibilities were redefined. Water issues were transferred to the Secretary of Agriculture and the remaining environmental responsibilities were divided among two new institutions: the Office of the Attorney General for Environmental Protection (PROFEPA), in charge of enforcement, and the National Institute of Ecology (INE), in charge of policy development and natural resources management. Both institutions became part of a new Ministry of Social Development (SEDESOL). The audit could not find an explanation for how the Bank and SEDUE's staff responsible for preparing the project could have been so unaware of forthcoming changes, or at least of the weak institutional base of the project design. Confronted with these sweeping changes, the Bank seriously considered canceling the whole project but decided to continue. It took almost a year of negotiations to redefine the project activities. The result: * All the water-related components were cancelled, due to the transfer of water issues to the new Secretary of Agriculture and Water Resources. * Studies to analyze how to restructure SEDUE were cancelled. * Remaining activities were reorganized and redefined to match the responsibilities of the two new agencies, PROFEPA and INE. * Either explicitly or implicitly, MEP was redirected to build up and staff the new agencies, particularly INE, where at times a third of its employees, and a much larger proportion of its technical staff, was on MEP's payroll. 3.3 The initial restructuring shock resulted in a one-year delay in project effectiveness. No disbursements were made during fiscal 1992, and a letter of amendments to the loan was negotiated and signed in April 1993. Project implementation and loan disbursement proceeded slowly thereafter. 3.4 In December 1994, Mexico fell into a severe financial and economic crisis. Exchange rates plummeted and capital fled the country. The GOM had to bail out a large portion of the country's banking system, aided by a US$30 billion bailout package from the United States Treasury, and enforced a severe fiscal and budgetary tightening. Public expenditure was curtailed, and environmental investment, not considered a priority, was greatly downsized. 3.5 In 1995, the GOM requested a two-year extension of the closing date as well as cancellation of half the loan. This resulted in the second amendment to the project. Thereafter Hacienda (the Treasury), by law the sole point of entry for all foreign loan moneys, tightened its release to implementing agencies. Each public agency had access only to its annual budget to pay 5 both for its regular activities and its special programs. This effectively canceled the incremental character of the Bank's financing. In fact, the budgets of PROFEPA and INE shrank after the 1995 crisis even as, by engaging in the project, those agencies had agreed to undertake new activities. Furthermore, Hacienda released the approved annual budgets with considerable delay. Typically, 60 percent of the yearly budget would reach the environmental agencies in the last quarter. The project ICR assessed that this budget cramming prevented full use of the resources and compromised the quality of the outcomes. Adding to the tightening of government budgets, clashes between Mexico's and the Bank's administrative procedures were a common source of delays, with the Mexican implementing agencies complaining of Bank micro-management of each change in a highly complex project. 3.6 In early 1995, a Ministry of the Environment, Natural Resources, and Fisheries (SEMARNAP) was created. The preexisting Office of the Attorney General for Environmental Protection (PROFEPA), and the National Institute of Ecology (INE), became part of this new ministry, together with other disciplines like fisheries and water resource management. Again, these changes seem to have been neither recommended nor foreseen by the project's team studying how to improve the country's environmental management. Although adjustment delays ensued, they were less severe than in the 1992 reorganization, since INE and PROFEPA, the focal points of the project, migrated to SEMARNAP and were mostly unaffected. 3.7 As the GOM comments on the project's ICR make clear, the Bank also contributed to implementation hurdles, with five task managers in seven years, some of whom had difficult personal relations with local counterparts, and repeated changes in the Latin American Department that caused delays, loss of institutional memory, and supervisory discontinuities. 3.8 The GEF component, being a grant, was less impaired by the 1995 economic crisis, but it too had severe implementation problems. Following the advice of an international panel of experts, the number of SINAP protected areas included in the project was reduced from 17 to 12, and the grant was decreased from US$30 million to US$25 million (this change was part of the amendments to the loan signed in April 1993). Start-up delays continued until 1994, and thereafter the project proceeded very slowly, confronted with numerous problems, including slow disbursements, difficulties moving the funds down to the local level, and lack of participation of local and native communities, delays in completing management plans for some reserves, and delays in appointing the Technical Advisory Committees. A second grant amendment, in April 1996, focused on streamlining disbursement to the local level and enhancing the participation of local and native communities. It also opened discussions for a major project restructuring. As a result, in May 1997, the GEF and the GOM restructured the project as follows: termination of the grant by December 1997, cancellation of US$16.3 million, and use of these moneys for an endowment, the Mexican Fund for Nature Conservancy (see paragraph 4.6). 4. Outcomes and Follow-up Activities Outcomes 4.1 Despite major changes to the original implementation framework and cancellation of more than half of its original budget, the project managed to deliver on many of its original and revised components.' 2. The multiplicity of MEP components precludes a full listing, what follows is a qualitative summary. A quantitative list and comparison with original monitoring indicators can be found in the ICR (report No. 18922, pp. 22-27). 6 The following components substantially achieved expected outcomes (based on the 1997 restructuring): * The air pollution program put in place 20 meteorological and 79 monitoring stations, approximately half the original target. Planning and policy was strengthened, but it was far from effective in controlling stationary sources as originally envisioned. * Enforcement activities were strengthened, including supervision of 750 environmental audits and partial support for some 70,000 industrial inspections. * The impact and risk assessment programs also surpassed its original targets for staffing, training, and operating the environmental assessment office, that during MEP implementation handled some 6,000 environmental assessment presentations, compared with 3,380 originally envisaged. * The wildlife protection program established and operated 5 out of the originally proposed 7 rehabilitation centers. * The turtle program surpassed its original target of establishing and operating 12 protection centers. * Training activities that were part of many project components were delivered through local universities and reached the wider public outside the project staff, including businesses and NGOs. * Staff were recruited and trained from INE and PROFEPA to carry many of the above activities. The following components achieved far less than the expected outcomes (based on the 1997 restructuring): * The component to install one national reference laboratory for INE and 6 regional ones for PROFEPA. The INE laboratory was cancelled. PROFEPA built five, one of which was still not operative in June 1999. * The project envisaged 977 studies of various types, with the largest number concentrating on ecological zoning, legislation, and standards components. A total of 514 studies were completed. Some of them seemed to have been influential: e.g., several were used as background documents in the 1996 reform of the National Environmental Law. Then again, many proved of little use. For instance, many industrial and emissions studies in support of specific environmental assessment and discharge regulations were discarded when SMARNAP opted for a more simplified regulatory approach. * The pilot decentralization program for five states was delayed by the central government's reluctance to transfer resources to the states with the arrival of the 1995 crisis. 4.2 Overall, outcomes were far from those envisaged in the original design of the project. Instead of tackling major environmental problems, the project was mostly devoted to staffing and strengthening the newly created federal environmental agencies. 4.3 The GEF Protected Areas Program, for its part, delivered (a) the preparation of management plans for 9 protected areas, (b) emergency and operation plans and follow-up activities in 10 protected areas, (c) support to several pilot productive projects in protected area's buffer zones, and (d) strategy studies for the management of the National System of Protected Areas. 4.4 The 10 protected areas reached by the program should be compared with the 17 originally envisaged, and 12 targeted after the 1993 amendment of the grant. The program also fell short on local community participation and consultation, which proved to be one of the most contentious 7 issues between the Bank and the Mexican implementation agency. In several protected areas setting up the Technical Advisory Committees was delayed or never achieved. Furthermore, local participation and consultation was not up to Bank standards. Finally, by 1996, it was clear that the program would not have resources to manage the protected areas once the GET Grant was exhausted. All these factors lead to the 1997 restructuring of the program. Follow-up Activities 4.5 Focusing on supporting decentralization of environmental management to 11 states, the PROMAD (Programa de Medio Ambiente y Descentralizacion) project, currently under preparation and scheduled for late fiscal 1999 or early fiscal 2000, is seen by GOM and the Bank as a follow up to MEP. It would feature the design and establishment of state-based environmental funds as a source of sustainable state-level environmental budgets. 4.6 The restructuring of the project's GEF component resulted in the GET Grant 28678 that will continue conservation programs in the 10 protected areas where the project is already working; will support programs that would benefit more than one protected area; and has redirected US$16.3 million of the original GET Grant to an endowment fund administered by the Mexican Fund for the Conservation of Nature, a local NGO. The fund income would be devoted to conservation activities in SINAP protected areas. The total cost of this project is US$46.15 million, of which the GET Grant 28678 total contribution amounts to US$26.18 million (including the US$16.3 million mentioned above). 4.7 Conservation management, biodiversity monitoring, and local participation to protected areas not covered by the first GEF project will be provided by GEF Protected Areas II, scheduled for fiscal 2000. 4.8 Several other GEF projects in the pipeline will integrate biodiversity conservation as part of Bank loans targeting rural poverty alleviation or production development projects. This is the case for: (a) GEF Oaxaca Hillside Management (FY00), GEF Community Forestry Reserves (FY00), GEF El Triunfo Reserve (FY99), GEF Meso American Corridor (FY01), GEF Land Conservation Mechanisms (FY00), and GEF Conservation and Sustainable Use of the Gulf of California (FY01). 5. Findings Project Achievements and Shortcomings 5.1 When measured against its objectives, what the project managed to achieve makes a relevant contribution to the country's development. But equally relevant is what the project failed to achieve. 5.2 Many of the project activities helped enhance Mexico's ability to carry out key environmental protection and natural resource management, in the short and long run, as was called for in the project objectives. Advances in air pollution control, environmental impact assessment, biodiversity protection, and environmental legislation, are all good examples. On the other hand, important objectives like water pollution control were not achieved, and many others fell short of original and of restructured targets. 8 5.3 A major concern is the lack of borrower's ownership of the project's original design, as evident in the large gap between what the Bank, the national environmental agencies, and the higher levels of GOM perceived was called for in institutional design and strengthening. The project SAR-prepared by the Bank-included among the project objectives "...transforming SEDUE (the then federal environmental agency) over time into a small but highly qualified second tier organization that supervises and coordinate environmental work carried out by other levels of government..." This objective was clearly not shared by Mexico's agencies at that time and did not make it to the Loan Agreement. Furthermore, both the Bank and its counterparts in the implementing agencies grossly failed to be aware, let alone influence, GOM intentions regarding the sector's institutional design. This failure triggered major project changes and delays. Also unrealistic planing by SEDUE and the Bank resulted in a project that was a large collection--over one thousand--of unrelated products whose only possible rationale was to give something to everybody. This proliferation of components hindered implementation by the Mexican agencies and supervision by the Bank. Partly as a result, as mentioned by the ICR, the GEF component was inadequately supervised. 5.4 The bulk of MEP resources were spent on staffing and strengthening the federal environmental agencies, with many of its consultancy and study items paying for scores of new staff for INE and PROFEPA. Paying for salaries and operation costs is not among the Bank's priorities, although it can be said that in this case it helped build Mexico's environmental agencies at a time when there were tight budgetary limitations for the country. 5.5 Many of the project's deliverables seem to have added little value, as exemplified by the little use made of a good number of the 514 studies commissioned by the project, or the few complaints aroused by the absence of the other 413 studies that the project failed to deliver. 5.6 Since institutional development projects are not amenable to current project economic assessment techniques, rates of return were not estimated either at appraisal, closing, or audit. On the other hand, it is clear to the audit that the design of the project was very inefficient from the perspective of trying to achieve meaningful results in a manageable and monitorable fashion. Sustainability 5.7 Several MEP outcomes, particularly the most successful ones (e.g., air pollution monitoring and policy, environmental assessment and legislation) have been adopted by Mexico's environmental agencies programs and are likely to be sustainable. 5.8 The sustainability of other MEP outputs, particularly those that demand significant operational costs, are more uncertain owing to Mexico's persistent budgetary constraints. The operation of some of the infrastructure paid for by the project (e.g., laboratories) was delayed due to lack of counterpart funding for personnel and operation costs, and were later paid by PROFEPA. At the end of MEP, some of the staff hired by INE during the project were moved to the payroll of another ongoing Bank project (Northern Border Environmental Project, Loan 3750- ME). and as late as June 1999, INE had advanced but not completed the transfer of MEP staff to their regular payroll. 5.9 Sustainability of the GEF Protected Areas component is a special case, in that concerns about lack of funds to sustain the outcomes of the original project design triggered its restructuring (see paragraph 4.6). 9 Institutional Development Impact 5.10 Mexico's environmental management has made significant advances during the 1990s in terms of legal framework, institutions, human resources, and programs, particularly at the federal level and, on a more modest scale, in some states. Also, public information and public environmental-related demands have grown dramatically. MEP has contributed to this development, particularly through its steadfast support of PROFEPA, INE, and SEMARNAP through a time of considerable economic, budgetary, and organizational turmoil. 5.11 Yet in Mexico, as elsewhere, the direction and pace of institutional development is driven by the country's management culture and its current political agenda, as exemplified by the sudden institutional changes from SEDESOL to SEDUE to the Environmental Ministry, or by the reluctance to use the Bank's suggestions regarding decentralization and market incentives. In the end, the institutional development that actually took place bears little resemblance to the original SAR proposals.3 Expectations for a GEF Project 5.12 GEF projects are expected to be innovative, replicable, sustainable, incremental, and, where appropriate, include participation and consultation. In this regard, the original design of the program was far from innovative, but it did address all the other issues. The most interesting issue is that with participation and consultation lagging and growing concerns over the sustainability of the whole program, the Bank and INE were able to restructure the project to address these concerns, including the innovative approach of redirecting part of the grant to a permanent NGO- managed fund for protected areas. Bank Performance 5.13 The project's design left much to be desired. It was overly ambitious. It failed to grasp the lack of broad-based government support for the project's sector reform proposals. It was crammed with too many unrelated components, detailed to a level that invited micro- management. It grossly underestimated the implementation difficulties to be faced by Mexico's agencies and the Bank. That being said, it is commendable that through six years of intense work, the Bank and the implementing agencies were able to partially correct the poor initial design, weather two sectoral reorganizations and the difficulties triggered by the country's 1995 economic crisis, and still deliver important and useful outputs. Borrower Performance 5.14 The project required a lot of adjustments and experiential learning from the borrowers. Tough economic conditions made it difficult to respond both to GOM and Bank procedures in a framework of scarce resources and job insecurity. Even so, the staff of the Mexico participant agencies increased their comnitment to the project over time. The project ICR singles out the performance of the project coordination unit, which has now been transformed in INE's External Credit Unit, in charge of coordinating its foreign-financed projects. 3. The point that institutional change is a country-specific process, with its own cultural dimension and times, is argued in a recent Bank review of environmental sector institutional development projects that questions the ability of the Bank project-by-project approach to make a difference in the country learning curve. See Marguils S. and Vetlesseter T. (1999) Environmental Capacity Building: A Review of the World Bank's Portfolio. Environmental Department Paper No. 68. 10 Ratings Summary 5.15 Based on the above analysis, this audit rates the project as follows: Outcome: Marginally Unsatisfactory. The audit upgrades the ICR rating (Unsatisfactory) to reflect that some important project's goals were achieved. Institutional Development: Modest. The audit agrees with the ICR rating. Sustainability: Likely. The audit agrees with the ICR rating. Bank Performance: Unsatisfactory. The audit downgrades the ICR rating (Satisfactory) to underline the flaws and poor quality of project preparation and appraisal, while acknowledging the efforts made to overcome these flaws, and other difficult externalities during supervision. Borrower Performance: Satisfactory The audit agrees with ICR rating. 6. Lessons * Project designs need to be focused. This project was an extreme case of trying to cater too many demands. A design that has many deliverables and many detailed activities is a good sign of a project lacking a core group of objectives. High level of complexity invites micro- management and hinders the flexibility needed to accommodate changes during the project cycle. * Expect surprises, particularly in the environmental sector. Project risks are routinely treated as incremental within a few percentage points around expected values. Few analyses face up to the risk of major surprises. No one would blame the MEP for not anticipating Mexico's 1995 financial crisis, but it is difficult to understand how it could have overlooked the signs and directions of the 1992 and 1995 institutional reforms in Mexico's environmental sector. The fact is that changes in institutional arrangements have been the norm for environmental institutions around the world during the past 20 years as governments try different approaches to handle this new area of government. The chance of major changes in the institutional framework should be acknowledged in the design of institutional strengthening projects. * Major changes in a sector need to be done in tandem with adjustments to the national policy. The project ICR correctly points to the difficulty of fostering decentralization of one sectoral agency if it does not makes part of a general government decentralization policy. * Move from financing activities to financing goals. The Bank's 1999 Country Assistance Strategy for Mexico makes the point that Bank lending to the country should move from activities to goals. Environmental projects usually encompass many small activities that are ill suited to traditional Bank procedures. Such projects would be good candidates for the Bank's Adaptive Program Lending (APL), whereby full slices of the project are cleared in advance and the emphasis moves to auditing the outcomes and away from tracking progress in myriad intermediate activities. 11 * Bank staff, and staff at the local agencies, should be better aware of their counterpart operational frameworks. Conflicts and misunderstandings between the country and the Bank legal and administrative frameworks were a usual source of friction and delay throughout the project cycle. These difficulties were sometimes exacerbated by high rotation of the Bank staff assigned to the project. Complex projects should acknowledge the cost of these problems and seek ways top address them. In these regard the recently enlarged Bank country office in Mexico is well positioned to help negotiate these situations in future projects.  13 Basic Data Sheet Key Project Data (in US$ million) Appraisal estimate Cancellations Actual at project closing Actual as % of appraisal estimate Total project costs 126.6 57.9 46 Loan amount 50 26.2 19.5 39 GET grant amount 30 21.3* 8.2 27 Government of Mexico 46.6 11.6 30.2 65 (estimated) Source: SAR/ICR * A restructuring of the GET grant was negotiated resulting in $16.3 million being transferred to a privately managed endowment fund for protected areas in July 1997, which became GET Grant 278678. Cumulative Estimated and Actual Disbursements, Loan 3461-ME (in US$ million) FY92 FY93 FY94 FY95 FY96 FY97 FY98 Appraisal estimate 8.0 25.3 37.0 46.4 50.0 Adjustment (in 1996) 3.2 6.8 Actual 5.016 12.161 17.705 18.189 18.189 19.5 Actual as % of appraisal estimate 0 20 33 38 36 - Source: ICR/590 Cumulative Estimated and Actual Disbursements, GET Grant 28604 (in US$ million) FY92 FY93 FY94 FY95 FY96 FY97 FY98 Appraisal estimate 3.1 9.9 19.1 28.1 30.0 Adjustment (April 1993) 5.0 14.2 23.2 25.1 Adjustment (June 1997) 8.8 Actual 3.018 3.600 4.289 6.981 8.191 Actual as % of appraisal estimate 21.1 15.5 17.1 79 - Source: ICR/590 14 Project_Timetable ______________________ Original Actual Identification March 1990 Preparation March 4, 1991 Appraisal June 17, 1991 November 13, 1991 Negotiations November 22, 1991 February 11, 1992 Board presentation April 14, 1992 Signing April 28, 1992 Effectiveness April 15, 1993 (according to ICR) May 8, 1993 (according to SAR) Project completion June 30, 1995 June 30, 1997 Loan/Grant closing December 31, 1995 December 31, 1997 Final disbursement January 31, 1996 December 31, 1997 Source: SAR/ICR Staff Inputs, Loan 3461-ME Stage of project cycle Time-frame Staff weeks Identification-appraisal Mar 1990 - Nov 1991 95.0 Negotiations -effectiveness Dec 1991 - Apr 1993 Not given Supervision May 1993 - Jun 1997 131.1 Completion Jul 1997 - Dec 1997 4.0 Total 230.1 Source: ICR Staff Inputs, Grant GET 28604 Stage of project cycle Time-frame Staff weeks Identification-appraisal Mar 1990 - Nov 1991 25.7 Negotiations -effectiveness Dec 1991 - Apr 1993 Not given Supervision May 1993 - Jun 1997 122.0 Completion Jul 1997 -Dec 1997 0.0 Total 147.7 Source: ICR Mission Data Date Staff Weeks Staff days in the field Identification/Prep 3/90 - 3/91 aration 95.0 30.4 Appraisal 4/91 - 11/91 Supervision 5/93- 6/97 153.1 48.1 Completion 7/97 - 12/97 Source: ICR (note: The mission data on specializations represented, performance ratings, and rating trends are typically provided in the ICR project tables, but were not available for this project.) 15 Follow-up Operations Operation Credit/Grant Amount no. ($US million) Approval FY Protected Areas Restructuring Program GET 28678 16 July 1997 El Triunfo Reserve GET 22748 0.725 1999 PROMAD (Environmental Management and MX-PE-36005 1999 Decentralization) Loan Oaxaca Hillside Management GET 22766 0.712 2000 Community Forestry Reserves n/a n/a 2000 Meso American Corridor n/a n/a 2001 Protected Areas II n/a n/a 2001 Land Conservation Mechanisms n/a n/a 2000 Conservation and Sustainable Use of Gulf of California n/a n/a 2001  17 Annex: Comments from the Borrower AANCO ACIONA. CE OeA3 Y SERvClos PUBUCO, e. m e. Marzo 23, 2000 GOF1/0634/200q SR. GREGORY INGRAM Deprtamento de Operación y Evaluación Banco Mundial 138a H. Street N.W. Washington, D.C. 20433 G7-035 Hacemos referencia a su carta fechada el 11 de febrero del presente, a través <4e cual solicita los comentarios al "Reporte dA Auditoría de Desemoeño de la Donaci4n GEF 28601. y del Progrmnm Ambi^ntYt1 de México" pcr parto de emta frtituoión y lo las Dependencias del Gobierno Federal encargadas de la Ejecución de dich" proyectos. Sobre el particular, hago de su conocimiento que el Banco Nacional de Obras ly Servicios Públicos, S.N.C. (BANOBRAS) no tiene comontario alguno al ROpo4 mencionado en la Referencia. Por lo que toca al Instituto Nacional de Ecología (INE) y a la procuraduría Federal e Protección al Ambiente (PROFEPA) anexo estamos enviando sus comentarios el rnismo. Sin otro particular, aprovecho la ocasión para enviarle un saludo cordial. ATENTAMENTE. .MAEL DA iLERA nte. rcp. Lk. Alejarro Peralta Moreno. Subgerente de Proyectos da O n rn~ Urbano. aANOBRAS.