ICRR 14424 Report Number : ICRR14424 IEG ICR Review Independent Evaluation Group 1. Project Data: Date Posted : 09/08/2014 Country : Costa Rica Project ID : P057857 Appraisal Actual Project Name : Cr Equity And US$M ): Project Costs (US$M): 50.0 44.28 Efficiency Of Education L/C Number : L7284 Loan/ US$M): Loan /Credit (US$M): 30.0 24.28 Sector Board : Education Cofinancing (US$M): US$M ): Cofinanciers : Board Approval Date : 03/29/2005 Closing Date : 06/30/2011 12/31/2013 Sector (s): Primary education (50%); Secondary education (30%); Central government administration (20%) Theme (s): Education for all (67% - P); Education for the knowledge economy (33% - S) Prepared by : Reviewed by : ICR Review Group : Coordinator : Pia Schneider Judyth L. Twigg Lourdes N. Pagaran IEGPS2 2. Project Objectives and Components: a. Objectives: According to the Loan Agreement (2005), the objectives of the project were to : "(a) reduce education quality gaps in the Borrower’s primary and secondary education system in rural areas; and (b) improve the equity and efficiency in the allocation, administration and use of the Borrower ’s education sector resources ." The Project Appraisal Document (p. 7) provided a slightly different set of objectives : "the project will: (i) reduce existing rural education gaps in primary education quality, equity, and intemal efficiency; (ii) develop cost-effective strategies to increase access to, and improve the quality of, secondary education rural modalities; (iii) improve the impact of equity programs for low -income students; and (iv) enhance the efficiency of the education sector’s institutional and economic resources allocated to the rural sector ." This review will use the project development objectives (PDO) as stated in the Loan Agreement . The objectives remained unchanged . However, in June 2011, during a Level-2 Restructuring, four original PDO indicators were replaced by three new indicators (see Section 4). The project's scope was changed from originally "municipalities and regions in the four Macro -Regions (Norte, Atlantico, Puntarenas and Guanacaste ) with low socio-economic and education indicators " to "nine rural districts." Accordingly, this review will use a split rating and assess project performance against the overall objective before and after the June 2011 restructuring, when 36.6% of the actual loan amount had been disbursed . b.Were the project objectives/key associated outcome targets revised during implementation? Yes If yes, did the Board approve the revised objectives /key associated outcome targets? No c. Components: The three components supported under the project are presented below, showing IBRD financing plus borrower contribution estimated at appraisal, revised amounts (June 2011 restructuring), and actual project costs, excluding contingencies and IBRD front -end fee. 1. Quality and Equity of Rural Education (Appraisal: US$ 34.24 million; Revised: US$ 23.60 million; Actual: US$ 20.50 million). This component was to finance Annual Operational (POA) Subprojects to improve rural education attainment and institutional development . To orient the preparation of the POA Subprojects, strategies and expected results to increase the level, quality and equity of basic (up to 9th grade) rural education were included in a Policy Activity Schedule . The Rural Education Quality and Equity Subprojects were to improve the targeting, education quality and organizational efficiency of rural education . The Local Institutional Development Subprojects were to strengthen the institutional capacity of regional departments, schools and Collaborative School Networks to contribute to general institutional efficiency . The POA Subprojects were to be executed by the technical units of the Ministry of Public Education (MEP), supported by the Project Coordination Unit (PCU). The Subprojects were to target municipalities, communities and schools with the lowest education indicators. 2. Equity of Education Services (Appraisal: US$ 3.86 million; Revised: US$ 22.75 million; Actual: US$ 21.80 million). This component was to increase the capacity of the MEP to reduce equity gaps by supporting activities to identify, reach and monitor the delivery of education services to regions with low education indicators and students from low income households . It financed technical support, tools and programs (i) to raise the capacity of MEP units managing the MEP’s demand-based equity programs (scholarships, vouchers, transportation and school meals); and (ii) to develop and implement an integrated information system to track education outcomes across municipalities, communities and schools (Sistema de Información de Desarrollo Educativo, SIDE). 3. Improved Institutional Efficiency (Appraisal: US$ 10.20 million; Revised: US$ 3.50 million; Actual: US$ 1.90 million). This component was to improve the institutional capacity of the MEP through inter -departmental integration strategies and working alliances across central, regional and school organizations . It was expected: (i) to strengthen the capacity of the MEP ’s staff (both pedagogical and administration units ) to conduct participatory sector diagnosis, plan strategically, and implement, monitor and evaluate education programs; (ii) to improve the efficiency of edu cation services provided in the rural sector, through the integration and strengthening of the MEP units leading such services; and (iii) to increase capacity of the Project Coordination Unit (PCU) for coordination, fiduciary, monitoring and evaluation activities of externally financed projects . The PAD lists 13 different units and programs as beneficiaries for this component (including Indigenous Education Unit, Curriculum Development and Multi-grade Education Units, National Teaching Center etc ). The Indigenous Education and the Multi-grade Education Units at the MEP were responsible to monitor, analyze and evaluate subproject proposals (POA) and other activities to develop a strategy for inter -cultural exchange. During the June 2011 restructuring, all three components were revised and renamed . Activities were moved from the first to the other two components, and overlapping activities were dropped . Accordingly project costs were revised by component (as shown above under Revised Amounts ): Component 1 was renamed “Efficient and Equitable Access to Rural Education ." This revised component was to finance infrastructure investments to improve access to quality education including: (i) renovating existing classrooms and constructing new classrooms, and providing furniture, equipment, computers and software; (ii) enhancing school building facilities, including construction and provision of furniture, and equipment for kitchens, teachers' lounges, student dormitories, and restrooms in schools belonging to rural School Networks; (iii) constr ucting common school network facilities, such as technology centers, cultural, arts and physical education facilities, and energy and connectivity facilities; and (iv) providing furniture and equipment for the Regional Indigenous Education Directorate in Sula, created by Decree in 2012 to represent seven indigenous territories in Limon and six administrative circuits . Activities related to MEP's institutional strengthening (including support to transform Telesecundarias to Liceos Rurales ) were moved to the second component . Activities to support quality of rural education were moved to the third component. Other activities were dropped. Component 2 was renamed “Improving MEP ’s Institutional Efficiency ." It was to finance: (i) computer equipment and technical assistance to develop and implement an education sector information systems at the school level; (ii) technical assistance to support for MEP ’s Implementing Technical Units that manage the Equity Programs, and resource transfers to school councils to finance operating costs and fund transfers to beneficiaries through the equity program (transportation, school meals, and education vouchers ) ; (iii) training and technical assistance to support implementation of MEP ’s institutional reform; and (iv) support for project management. Component 3 was renamed “Quality of Education ." It was to finance: (i) development and implementation of a training program in intercultural education for MEP staff at the central and regional levels; (ii) a rural education quality improvement program adapted to the local and c ultural context, including the development of a strategy to transform (as approved by the Superior Council of Education in July 2009) Telesecundarias with two years of secondary rural education to new Liceos Rurales with four years of quality secondary education preparing for a Bachillerato degree with modern teaching methods including information /communication technology (ICT); (iii) provision of computer equipment and professional development programs for teachers assigned to rural areas; and (iv) improvements in MEP staff and teacher capacity to analyze student learning assessments, including a standard methodology and related training modules . d. Comments on Project Cost, Financing, Borrower Contribution, and Dates: Project Cost, Financing and Borrower Contribution The actual project cost was US$ 44.28 million, compared to the original appraised project amount of US$ 50 million. Project funds were revised by component during the June 2011 restructuring as shown above . Actual costs were lower than appraised because of the can cellation of some project activities, including the procurement of computers and software (see Section 11b) and consultancies to compile information on school performance and design a teacher training strategy . The project was financed by a US$ 30 million IBRD loan, of which US$ 24.28 million actually disbursed. At closure, the undisbursed balance of US$ 5.72 million of the loan was cancelled. Activities related to computers and consultancies were cancelled (Section 11.b). The actual borrower contribution was US$ 20 million, as planned at appraisal. Dates There were four restructurings : August 2007: to modify procurement methods and disbursement percentage . September 2008: to change disbursement percentage . June 2011: to eliminate project activities that were overlapping, define target areas as 9 rural administrative territories, replace four original PDO indicators by three indicators to better reflect project objectives and activities, align the project coordination unit with the MEP, update the environmental assessment and Indigenous People Planning Frameworks, align expenditure categories with revised project activities as shown above, and extend the closing date by 18 months to December 31, 2012 to ensure that revised targets were met. Disbursement at this point was US$ 8.9 million (36.6% of the actual loan amount). September 2012: to extend the closing date by 12 additional months to December 31, 2013. 3. Relevance of Objectives & Design: a. Relevance of Objectives: Substantial under original and revised project scope The objectives of reduced quality gaps and improved equity and efficiency in the education system were substantially relevant. They supported the government's 2003 plan to close the rural-urban education gap, improve educational attainment among low -income groups, and improve the education system . They were still relevant to the government's 2010-2014 National Development Plan, which guarantees for all the right to education. The focus on primary and secondary education was appropriate given low coverage and completion rates in secondary education; by 2009, only 40% of youth finished secondary education, which points to structural problems in the education system . The objectives were also substantially relevant to the country's goal to become a knowledge economy . The objectives were also in line with the goals of the Bank's 2004 Country Assistance Strategy, which aimed to close the gap in learning quality between regions and social groups, and with two clusters in the current Bank Country Partnership Program (FY2012-2015), namely developing competitiveness and improving efficiency and quality in the social sectors . However, the objectives of closing the quality gap, increasing equity and increasing efficiency may have been too ambitious given the project's time frame. b. Relevance of Design: Original project scope : Modest Revised project scope : Substantial The original design had weaknesses that were partially addressed during the restructuring . The original design was too complex and lacked clarity, and needed to be restructured . The objectives were not realistically aligned with the original project activities, and the design was too ambitious considering the resources available . The original design could have benefited more from the available analytical work and detailed socio -economic and demographic data by region. It targeted municipalities with the lowest education indicators in rural, indigenous and low-income communities; however, it was not clear how these areas were be selected . To carry out the targeting, the MEP was to develop the relevant indicators that were to be provided by an information system to be developed under the project . Activities in the Policy Activity Schedule (Component 1) were vaguely defined, and some activities were overlapping across components and had to be consolidated or dropped during the restructuring. The results framework was too detailed and had to be reformulated . Critical risks were not adequately identified and addressed in the design . The revised design explicitly targeted 9 administrative rural districts. The revised activities were more clearly defined and linked to the objectives . The key indicators were refined and better aligned with the objectives and activities. The revised results framework became more logical . 4. Achievement of Objectives (Efficacy): Objectives : To reduce education quality gaps in the Borrower ’s primary and secondary education system in rural areas; and improve the equity and efficiency in the allocation, administration and use of the Borrower ’s education sector resources . The project uses 2006 as the baseline year. However, the project became effective in August 2007 and implementation started in 2008 (see Section 8a). Thus, the project did not contribute to changes that happened before 2008. 2013 : Outputs relevant for all objectives by 2013: 236 classrooms were built or renovated in the school network and provided with equipment, surpassing the target of 186 classrooms. Classrooms were fully functional except the computer laboratories, for which procurement was cancelled (see Section 11b). 81 schools were built and furnished, surpassing the target of 71. 14 liceos rurales were built and furnished, almost meeting the target of 15. 164,311 students benefited from scholarships nationwide, surpassing the target of 100,299 students. 102,623 beneficiaries nationwide received transportation benefits, surpassing the target of 93,423. 673,129 students received food benefits nationwide, surpassing the target of 656,000. 451 national/regional advisors were trained in intercultural education, surpassing the target of 179. 6 studies and workshops were financed under component 2, surpassing the target of 4. A participatory assessment was implemented to define training needs for secondary education teachers . 11 education materials were produced with intercultural perspective, surpassing the target of 9. 15.5% of schools and liceos were equipped with an education sector information system, not meeting the target of 100%, because of the cancellation of the computer procurement . 80 MEP technical staff were trained to improve supervision and evaluation of the Equity and Transfer programs, not meeting the target of 600 staff. Transfers to School Councils increased from 31.611 million CR colones in 2008 to 81.076 million CR colones in 2013, not meeting the target of 90.918 million CR colones. Funds for scholarships increased from 6.861 million CR colones in 2008 to 28.320 million CR colones in 2013, not meeting the target of 38.883 million CR colones. 632 staff were trained in MEP institutional reform programs, not meeting the target of 5,170 staff. 13.5% of rural secondary teachers (1,697 of 12,854 teachers) were trained in Liceos rurales and intercultural education, not meeting the target of 100%. Standard methodology to analyze student assessment was not developed and not implemented, and teachers and MEP staff were not trained in this area . Objective 1: reduce education quality gaps in the Borrower ’s primary and secondary education system in rural areas Original scope : Modest Revised scope : Modest Outcomes Graduation rates in primary education in target areas remained on a similar level from 87.9% in 2006 to 89.1% in 2013, compared to a nationwide increase from 93.5% (2006) to 95.8% (2013). Graduation rates increased in rural areas (which benefited from the project's equity and transfers program ) from 92.3% in 2006 to 96.1% in 2013. The graduation gap between targeted areas and the rest of the country remains significant at more than 5%. Enrollment in liceos rurales increased from 3,113 students in 2009 to 9,501 in 2013, following the transfer of 139 telesecundarias into 109 liceos rurales and after the Higher Education Council had approved the new curriculum in 2009. While infrastructure and equipment financed under the project may have improved the quality of the education setting; the contribution of teacher training to improved learning quality is not clear as teacher training activities did not progress as planned (see above). Also, pedagogical material was mainly produced for teachers and less for direct use by students (ICR, p. 36). New standard methodology to analyze student learning assessment was not developed . The impact on quality as a result of the pedagogical changes in the classroom was not assessed . Most quality improvements seem to have happened before the project started . The percentage of students passing secondary education bachillerato exams in the target areas increased from 14.1% in 2006 to 41.6% in 2008 before project start (August 2007) and substantially reduced the gap to the national average of 69.8%. However, from 2008 until 2013, the rate declined to 40.5% in target areas, and the gap to the national average increased during selected years (ICR, p. 48). The increase in 24 indigenous territories was from 11.4% in 2006 to 46.5% in 2013, and in rural areas from 53% to 60.3%, not meeting the target of 70%. The pass rate was 49.9% in the 14 newly established liceos rurales supported by the project (no target). The values for 2008 are not provided for these subgroups . Objective 2: improve the equity in the allocation, administration and use of the Borrower ’s education sector resources . Original scope : Modest Revised scope : Substantial Outcomes Enrollment in formal education among 13-17 year old students increased for lowest income groups from 69.7% (2006) to 77.1% (2012), which reduced the gap with the students in highest income groups who reported enrollment of 95.3% (2006) and 92.7% (2012). A disproportionately high number of poor and marginalized populations live in the target areas supported by the project . This increase went along with government policy to increase the education budget from 5.2% to 7.1% of GDP and the number of teachers from 77,446 to 88,548. The graduation rate in target areas in 11th grade and 7th grade increased from 22% in 2008 to 34.4% in 2013, surpassing the target of 27.5% and substantially reducing the gap to the national average of 43.6%. The graduation gap was narrowed from 13.4 to 9.2 percentage points. The number of students benefiting from the MEP equity and transfer programs increased substantially (see above). 76% of scholarships went to students in the lower two income quintiles, but no information is provided on whether this was a positive or negative change . Objective : improve the efficiency in the allocation, administration and use of the Borrower ’s education sector resources . Original scope : Negligible Revised scope : Modest Outcomes No information was provided on changes in internal education efficiency in terms of average percentage of over-age students and dropout rates . Improved cooperation among rural schools in networks has not been assessed . No information is provided on the efficiency of transfers and equity programs in reaching low -income students and facilitating their access to education . The number of students benefiting from the MEP equity and transfer programs increased substantially (see above). 76% of scholarships went to students in the lower two income quintiles, but no information is provided on whether this was a positive or negative change. Administrative cost per beneficiary of the MEP equity and transfer program was reduced from 10,855 CR colones in 2008 to 7,013 CR colones in 2013, not meeting the target of 6,574 CR colones. 5. Efficiency: Modest At appraisal the Bank team estimated that the project would reduce the number of repetitions by 105,826, reduce dropouts to 148,353 students, and save US$ 6.5 million in education demand subsidies . Primary school graduation was estimated to increase by 4%, with an incremental annual earning of US$ 600 benefiting families from lowest income quintiles. Using a discount rate of 10%, it was estimated that the project would yield an internal rate of return (IRR) of 51% over 10 years. Based on the same assumptions, the ICR conducted a cost-benefit analysis and estimated an IRR of 29% over 8 years. The ICR does not assess the accuracy of the PAD's estimates on repetitions, dropouts, and savings in education demand subsidies . Several factors affected the cost -effectiveness of implementation of the project . Effectiveness was substantially delayed due to the country's approval process . Collaboration between MEP and PROMECE was not smooth (see Section 9b), impacting the efficient use of project resources . Lack of clarity in project design also delayed project start and required a restructuring . ERR )/Financial Rate of Return (FRR) a. If available, enter the Economic Rate of Return (ERR) FRR ) at appraisal and the re- re -estimated value at evaluation : Rate Available? Point Value Coverage/Scope* Appraisal Yes 51% 100% ICR estimate Yes 29% 100% * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome: Project before restructuring : Unsatisfactory Relevance of the project's objectives is rated Substantial and of design Modest . Achievement of the objective to reduce the education quality gap is Modest, to improve equity in education is Modest, and to improve efficiency in the allocation and use of resources is Negligible . Efficiency is rated Modest. Project after restructuring : Moderately Unsatisfactory Relevance of the project's objectives and design are rated Substantial . Achievement of the objective to reduce the education quality gap is rated Modest, to improve equity in education is Substantial, and to improve efficiency in the allocation and use of resources is Modest . Efficiency is rated Modest. According to the harmonized OPCS/IEG guidelines for restructured projects, the final outcome rating is determined according to the percentage of the loan disbursed before and after project restructuring . The amount disbursed at the restructuring in June 2011 was US$ 8.9 million (36.6% of the actual loan amount of US$ 24.28 million). The project is rated Unsatisfactory (value of 2) before restructuring and Moderately Unsatisfactory (value of 3) after restructuring. The weighted rating is (2 * 0.36) + (3 * 0.64). The weighted average is the sum of the two: 0.72 + 1.92 = 2.64, which rounds to 3, Moderately Unsatisfactory. Therefore the final outcome rating is Moderately Unsatisfactory. a. Outcome Rating : Moderately Unsatisfactory 7. Rationale for Risk to Development Outcome Rating: The risk that the quality education gap will increase is moderate given the substantial investment by the government in school improvement in rural areas and increased government funding for education . The risk to improved equity and efficiency in the allocation of resources is moderate because of the improved targeting mechanisms developed with the support of the project . Given the strong government commitment to education reforms, the support provided by MEP is expected to sustain reforms . Computers are expected to be procured by the government. There is still a need to improve monitoring and control of the use of funds . a. Risk to Development Outcome Rating : Moderate 8. Assessment of Bank Performance: a. Quality at entry: The Bank team prepared a project that was not adequately calibrated to the local context and had to be restructured. The team did not identify the targeted schools and areas during preparation despite the availability of analytical work and detailed socio -economic and demographic data by region . The Bank team did not clearly identify the project implementation arrangements when preparing the project . Implementation arrangements were unclear and relied on a bottom -up approach that was too ambitious given the institutional context. Thus, the Operations Manual was not completed until 2008. Also, the Bank team initially overestimated the capabilities of local communities and regional technical units . The M&E framework missed relevant indicators and needed to be revised . The risk assessment was not realistic and did not consider the political risk that emerged from an earlier political crisis . The Bank team did not adequately assess the project approval process by the government and did not ensure that the project was included in the 2007 Budget Law, which further delayed approval until August 2007. at -Entry Rating : Quality -at- Unsatisfactory b. Quality of supervision: The Bank team was flexible and processed a total of four restructurings . After project effectiveness in 2007, the team processed two minor restructurings to update procurement and consultant guidelines, but these revisions did not address the design weaknesses . In 2009, a new task leader took over . The new team recognized the need for a restructuring, started the process, and selected the Mid -Term Review at the point for finalizing it. The team did not revise the PDOs, as this would have required Congressional approval and would have prolonged project implementation . However, the restructuring in 2011 did address most of the targeting and implementation weaknesses and helped speed up project implementation . The M&E design was revised and improved. The Bank team worked closely with the team of the Inter -American Development Bank (IDB), which supported secondary education nationwide . The team also took the necessary steps to identify and address weaknesses in fiduciary management (Section 11). Supervision teams struggled with monitoring and evaluation of the project (see Section 10). Quality of Supervision Rating : Moderately Satisfactory Overall Bank Performance Rating : Moderately Unsatisfactory 9. Assessment of Borrower Performance: a. Government Performance: Project effectiveness was delayed by 21 months due to a government change that delayed Congressional approval of the project. The change in government caused staff changes in the MEP implementation team . The high counterpart funding of 40% indicates strong commitment by the government to the reforms supported under the project . The government decision to transform "telesecundaria" to "liceos rurales" in April 2009 was to promote equity in access to secondary education and supported the project objectives . The government passed decrees in 2007 and 2009 to introduce institutional reforms at the MEP and eliminate duplication of functions and inefficiencies . These reforms were implemented by 2011 and changed responsibility for project management, including for infrastructure . The Ministry of Finance closely monitored the project and worked on resolving controversial issues in project implementation . The same Education Minister was in charge throughout the project's lifetime, which contributed to continuity . Government Performance Rating Satisfactory b. Implementing Agency Performance: PROMECE was the country's project implementation unit that managed all donor funded education projects . PROMECE was restructured in 2010, along with the MEP restructuring. PROMECE was in charge of fiduciary, administrative and monitoring activities within the MEP . The MEP had management and technical responsibility for the project. However, several responsibilities were not clearly identified until the 2011 restructuring. This lack of clarity created tension between PROMECE and MEP about project management . The PROMECE director was dismissed in 2009 and replaced by three successive directors in 2010. The previously dismissed director was rehired after a legal process . The MEP did not carry out the targeting as originally planned, which led to restructuring as the process was inconsistent with project investment sequencing. MEP and PROMECE were never able to reconcile their institutional issues . Some technical units did not show the necessary commitment to the project . The Department for Intercultural Education put in place an intercultural education policy . There were weaknesses in financial management and in procurement that led to contract cancellation (Section 11). Monitoring and evaluation was weak . Implementing Agency Performance Rating : Moderately Unsatisfactory Overall Borrower Performance Rating : Moderately Unsatisfactory 10. M&E Design, Implementation, & Utilization: a. M&E Design: The original design included four key indicators that were not clearly defined . They were replaced during the 2011 restructuring that clarified the meaning of indicators . Some relevant indicators were missing and only provided in the ICR, including an indicator on primary education . Baseline values were missing for some indicators and only collected in the ICR . The design does not include other relevant indicators such as the distribution of teachers by education and experience across schools with students of different socioeconomic background, the quality of teacher pre -service training, or the number of classroom hours per day and year encountered by an average student and low -income students. b. M&E Implementation: The results framework was not regularly updated, and supervision teams had difficulties with reporting on project implementation in targeted areas. There was no systematic assessment of the impact on quality as a result of pedagogical changes in the classroom . c. M&E Utilization: The M&E design was limited to the project . Data to assess indicators were provided with delays . The monitoring and control of the use of funds under the Equity program still needs improvement . A computerized student record system is used in only about 25% of the education system. Main statistical data on education are not publicly available. M&E Quality Rating : Modest 11. Other Issues a. Safeguards: Two safeguard policies were triggered by the project : Environme ntal Assessment : School construction caused the project to be classified as Environmental Screening Category B. An Environmental and Social Management Framework was prepared in 2003 including environmental assessment templates and guidelines for contractors . The template was updated in 2009. PROMECE prepared environmental impact files for each project site . Environmental supervision was carried out. A Resettlement Framework was prepared . Land use was screened to ensure it was free of legal issues . All land usage rights issues were resolved, and school construction took place except in Vesta where the land use was still to be determined (ICR p. 12). Indigenous Peoples : The project targeted indigenous students, and school construction was in indigenous areas. In 2003 a Social Assessment and an Indigenous Peoples Planning Framework (IPPF) were prepared in consultation with indigenous and rural education stakeholders . The IPPF was updated during project restructuring in 2011, and three workshops were held in 2012 with different ethnicities. b. Fiduciary Compliance: Financial Management : had several weaknesses . There were delays in submission of financial reporting and audit reports in 2012. These delays were fixed by April 2013, and reports were submitted with moderate delays . Poor contract monitoring and ineligible expenditures were detected before project closure . This was addressed by additional Bank supervision . With the exception of two audit reports, all other reports were delivered with substantial delays. The ICR does not report whether audits were qualified . The Bank team subsequently confirmed that audit opinions were unqualified . Procurement : encountered problems that affected implementation and disbursement . The project's largest procurement contract of US$ 3.7 million to purchase computer equipment for teachers and students was cancelled because of differences between Bank procurement procedures and the country's procurement rules . The procurement of 12 consultants was cancelled and triggered an investigation by the MEP audit unit and the Controleria of the government. MEP requested two ex-post procurement reviews. The Bank identified mistakes but did not find evidence to declare misprocurement (ICR p. 13). c. Unintended Impacts (positive or negative): None reported. d. Other: 12. 12. Ratings : ICR IEG Review Reason for Disagreement /Comments Outcome : Moderately Moderately The project is rated Unsatisfactory Satisfactory Unsatisfactory before restructuring and Moderately Unsatisfactory after restructuring . Weighting these ratings by the percentage of the loan disbursed before and after restructuring, the final outcome rating is Moderately Unsatisfactory. Risk to Development Moderate Moderate Outcome : Bank Performance : Moderately Moderately Major design weaknesses were not Satisfactory Unsatisfactory formally addressed until the 2011 restructuring. Borrower Performance : Moderately Moderately If the rating for one dimension is in the Satisfactory Unsatisfactory satisfactory range while the rating for the other dimension is in the unsatisfactory range, the rating for overall Borrower performance normally depends on the outcome rating . In this case, outcome rating is MU, hence overall rating for Borrower is MU following IEG/OPCS harmonized guidelines for Bank and Borrower performance. Quality of ICR : Satisfactory NOTES: NOTES - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate. 13. Lessons: Lessons drawn from the ICR (p. 25) are: Risk assessments are most useful when they are comprehensive and derive mitigation activities . In this case, during preparation, the political risk of a government change and mitigating measures were not adequately identified. Insufficient understanding of the government and congressional approval and budgetary process delayed project effectiveness . Participatory approache s are necessary to adapt curricula and institutions to rural and indigenous areas . In this project, a better understanding about the capabilities of local communities and regional technical units in the investment design would have been useful to strengthen the institutional context for a participatory approach. A participatory approach can help by adapting the curriculum to the needs of local populations, including indigenous groups . 14. Assessment Recommended? Yes No Why? Some of the results under the project will materialize only after some time, including the sustained impact of targeting low-income students with Equity benefits (scholarships, food, transport ) on their learning outcomes and labor market entrance. The project had a well-performing Indigenous Peoples element that could be portrayed and lessons learned in a thematic evaluation . 15. Comments on Quality of ICR: The ICR is well written and concise. It is satisfactory in its presentation of evidence . The ICR is consistent with ICR guidelines. The author made efforts to collect additional indicators to better interpret progress under the project and enhance information collected under the project . a.Quality of ICR Rating : Satisfactory