ICRR 14331 Report Number : ICRR14331 IEG ICR Review Independent Evaluation Group 1. Project Data: Date Posted : 06/24/2014 Country : Colombia Project ID : P105164 Appraisal Actual Project Name : Second Student Loan US$M ): Project Costs (US$M): 388.0 395.0 Support Project, Apl Phase I L/C Number : L7515 Loan/ US$M): Loan /Credit (US$M): 300.0 300.0 Sector Board : Education Cofinancing (US$M): US$M ): Cofinanciers : Board Approval Date : 03/04/2008 Closing Date : 12/31/2010 06/30/2013 Sector (s): Tertiary education (100%) Theme (s): Education for the knowledge economy (67% - P); Improving labor markets (33% - S) Prepared by : Reviewed by : ICR Review Group : Coordinator : Judyth L. Twigg Denise A. Vaillancourt Lourdes N. Pagaran IEGPS2 2. Project Objectives and Components: a. Objectives: According to the Loan Agreement (p. 5) and the Project Appraisal Document (PAD, p. 14), the project’s development objectives (PDOs) were “to: (a) improve coverage by increasing the enrollment and graduation rates of students in tertiary education; (b) improve equity by increasing enrollment and graduation rates of tertiary education students from economically disadvantaged backgrounds; and (c) increase and diversify the sources of alternative funding available to the Instituto Colombiano de Cre'dito Educativo y Estudios Te'cnicos en el Exterior (Colombia’s student loan agency, ICETEX ) in order to increase ICETEX's sustainability .� The objectives remained unchanged . However, one PDO-level indicator was modified at an October 2010 restructuring. Due to a lack of data, the original indicator “increase in the net enrollment of students in income quintile 1 and 2� was replaced by “percentage of students enrolled in tertiary education whose income is below twice the minimum salary.� Targets for several other indicators were amended at the same time to reflect the larger resource envelope. At that restructuring, US$ 173.02 million, or 57.7% of total Bank financing, had been disbursed. b.Were the project objectives/key associated outcome targets revised during implementation? Yes If yes, did the Board approve the revised objectives /key associated outcome targets? No c. Components: The project contained two components . Original cost estimates are taken from the PAD and reflect total costs . The ICR reports appraised and actual costs only for the Bank -financed portion of the project ): 1. Expanding equity and access to tertiary education (appraisal, US$ 381.62 million; actual, US$ 388.67 million), was intended: (i) to finance new student loans and renew existing loans to qualified students from economically disadvantaged families; and (ii) to form new alliances with tertiary education institutions to increase support and services to low-income students, foster a culture of repayment among students; support collection efforts after graduation; and co-finance tuition. 2. Institutional strengthening (appraisal, US$ 6.3 million; actual, US$ 6.3 million) was intended to: (i) strengthen ICETEX loan administration, portfolio management, and collection procedures, and upgrade its data technology hardware and software; (ii) expand the ICETEX funding base to ensure long -term sustainability; and (iii) monitor the project and perform impact evaluation studies . d. Comments on Project Cost, Financing, Borrower Contribution, and Dates: Project Cost: Original total costs were US$ 388 million, and final total costs were US$ 395 million, inclusive of all financing. The ICR reports only Bank financing as the total project cost . Financing: The project was financed by a US$ 300 million loan from the International Bank for Reconstruction and Development, whch was 100% disbursed. It was conceived as the first phase of a six -year APL Program, “Improving Access and Quality in Higher Education .� The second phase was approved by the Board on April 1, 2014 but is not yet effective. Borrower Contribution: The Borrower met its full counterpart obligations, providing US$ 88.67 million. Dates: In October 2010, the project’s closing date was extended by 18 months, from December 31, 2010 to June 30, 2012, due to slower disbursement than originally intended . At this point, only US$ 173 million had been disbursed, for two main reasons : (i) ICETEX had chosen to pursue a conservative financial leverage policy; and (ii) a slowdown in demand for student loans and improvement in the management of the portfolio had contributed to a budget surplus in 2010. The restructuring allowed the project to adapt to a changing context, as the global financial crisis had prevented ICETEX from pursuing its original aim of diversifying funding through private sector investment. Extension of the closing date allowed for a more realistic assessment of the operating environment, and for effective diversification of funding sources . Targets and the financing plan were revised to reflect the new implementation schedule and a larger resource envelope than originally anticipated . In an April 25, 2012 Restructuring Paper, the closing date was extended again, from June 30, 2012 to June 30, 2013, to allow for completion of activities A June 14, 2013 restructuring reallocated Bank funding in the amount of US$ 6.3 million from Category 2 ("Goods and Services for Component 2") to Category 1 ("Tertiary Education Loans"), because ICETEX decided to use all project resources to finance student loans in Component 1 and to fund remaining activities under Component 2 with its own resources. 3. Relevance of Objectives & Design: a. Relevance of Objectives: Relevance of Objectives is rated High under both the original and revised indicators /targets . Since the 1990s, government strategy has focused on the development of human capital to improve competitiveness, diversify the economy, and sustain growth . At appraisal, Colombia had achieved nearly universal enrollment in primary education and was making good process at the secondary level . However, it could not afford to increase tertiary education supply on its own . The 2006-2010 National Development Plan set a broad sector goal of 35% enrollment in higher education. The project’s objectives are highly relevant to this government strategy and also to current Bank strategy to improve opportunities in education under the strategic theme “Expanding Opportunities for Social Prosperity � (Country Partnership Strategy, FY 2012-2016). b. Relevance of Design: Relevance of Design is rated Substantial under both the original and revised indicators /targets . The expansion of targeted student loans is a global standard practice to increase coverage, graduation, and equity in tertiary education. The project’s approach was to expand student loan coverage through collaboration with institutions of higher education, while increasing support and services for low -income students to encourage graduation and foster a culture of repayment . Project design incorporated an appropriate targeting methodology (the System of Identification and Classification of Potential Beneficiaries for Social Programs, or SISBEN ) to ensure coverage of students from vulnerable backgrounds . The project’s planned institutional strengthening activities were plausibly connected to the objectives of diversifying ICETEX ’s funding sources and thereby improving sustainability. 4. Achievement of Objectives (Efficacy): Improve coverage by increasing the enrollment and graduation rates of students in tertiary education is rated Substantial for both the original and revised indicators /targets . Outputs : 259 agreements were signed between ICETEX and higher education institutions, exceeding both the original target of 30 and the revised target of 206. These agreements obligated the institutions to increase support and service provision for students with ICETEX loans to facilitate their graduation, help students manage their education loans, foster a culture of loan repayment among students, and support loan collection efforts post-graduation. The agreements also established the procedures through which the higher education institutions would receive and manage student loan funds, and participate in a sustainability fund to cover losses derived from dropouts (wherein ICETEX deducts a variable percentage of loan proceeds and transfers those funds to the institutions to cover defaults ). 187,610 new students received ICETEX ACCES loans, exceeding both the original target of 100,000 and the revised target of 134,875. 810,072 renewals were disbursed for students currently in the loan program, exceeding both the original target of 432,000 and the revised target of 650,487. Outcomes : The transition rate between secondary and tertiary education increased from 62% in 2006 to 75% in 2011, exceeding both the original target of 67.4% and the revised target of 71.5%. The overall enrollment rate in tertiary education increased from 30% in 2006 to 42.4% in 2012, exceeding the original target of 37.1% and essentially meeting the revised target of 42.8%. The number of tertiary education graduates increased from 150,270 in 2006 to 241,049 at the end of 2012, exceeding both the original target of 182,270 and the revised target of 199,828. The overall national gross graduation rate in tertiary education using OECD methodology increased from 19.4% in 2006 to 33.8% in 2012; using cohort methodology, the overall national graduation rate remained stable at 22.2% in 2006 and 22.9% in 2013. The graduation rate in tertiary education for student loan beneficiaries, using cohort methodology, increased substantially, from 29.2% in 2006 to 50.0% in 2013, demonstrating the positive impact of the student loan mechanism on graduation rates . Improve equity by increasing enrollment and graduation rates of tertiary education students from economically disadvantaged backgrounds is rated High , based on significant overachievement of targets for percentage enrollment in tertiary education among disadvantaged students, and on significantly higher graduation rates among disadvantaged students who received project -supported student loans compared to those who did not. Outputs : 156,170 new students from strata 1 and 2 obtained an ACCES loan, exceeding both the original target of 80,000 and the revised target of 111,426. This increased the percentage of loan recipients from disadvantaged families from 39.3% in 2006 to 49.5% in 2012. Interest rates were reduced and flexibly repayment plans were offered to disadvantaged students, and loan forgiveness options were put in place for disadvantaged students who completed their degrees. Outcomes : The number of students whose family income is below twice the minimum salary and who enrolled in the first year of tertiary education increased from 239,216 in 2008 to 301,645 (no target was set). The percentage of students enrolled in the first year of tertiary education whose family income is below twice the minimum salary increased from 45.6% in 2008 to 58.1% in December 2012, exceeding both the original target of 49.7% and the revised target of 49.9%. The overall graduation rate of students whose family income is below twice the minimum salary increased slightly, from 20.9% in 2006 to 21.6% in 2013. By comparison, the graduation rate of these disadvantaged students who received student loans under the project increased from 11.8% in 2006 to 39.4% in 2013, illustrating the positive impact of the student loan program on graduation rates among vulnerable students . Increase and diversify the sources of alternative funding available to ICETEX in order to increase ICETEX's sustainability is rated Modest under the original indicators /targets, and Substantial under the revised indicators /targets . Outputs : Technical assistance was provided to improve ICETEX ’s portfolio management and collection processes . Studies were conducted under the project to explore various options for financial sustainability . 15 new or upgraded management and information technology systems were put in place, exceeding the original, unchanged target of 10. ICETEX established a Sustainability Fund in 2009, through contracts with institutions of higher education . It also leveraged funds through alliances with 89 local governments, districts, departments, and municipalities . It explored the potential to tap into the private capital market by issuing Educational Saving Securities and other options (bonds, securitization) on the private equity market; these efforts were not successful . ICETEX also manager about 300 targeted funds and donations . The percentage of total resources available for student loans generated from new external sources other than ACCES credit or government/public funding increased from zero in 2006 to 14.9% in 2012, but decreased to 7.6% in 2013, essentially achieving the revised target of 7.5% but falling short of the original target of 9%. The indicator measured funding diversification from private sources (firms and non-governmental organizations) and some public sources (the higher education institutions themselves plus local governments and municipalities ). Outcomes : The amount of ICETEX resources available for credit generated from external sources other than the World Bank and government subsidies increased from Colombian Pesos (COP) 10.6 billion in 2006 to COP 56.7 billion in 2013. The ICETEX Sustainability Fund increased from zero in 2006 to COP 22.5 billion in 2013. Annual ICETEX administrative costs as a percentage of total annual income decreased from 6.1% in 2006 to 2.6% in 2013, exceeding both the original target of 3.7% and the revised target of 3.0%. The percentage of the ICETEX portfolio at risk over 30 days as a percentage of the total portfolio decreased from 19% in 2006 to 17.4% in 2013, exceeding the revised target of 18% but falling short of the original target of 15%. 5. Efficiency: Efficiency is rated Substantial . Cost-benefit analyses were performed, using the same reasonable assumptions, at appraisal and for the ICR (ICR, pp. 33-38). They found internal rates of return as follows : for the Colombian economy as a whole, 10.7% at appraisal and 19.67% for the ICR, with the actual result higher due to the larger than estimated number of tertiary graduates; for the Colombian government, 14.1% at appraisal and 12.2% at the ICR, with the return decreased due to a higher default rate than estimated (30% compared with 20% estimated at appraisal); and for the average ICETEX student loan recipient, 28.5% at appraisal and 29.2% actual for the 2008-2010 period and 35.1% for 2011-2013. For the 2008-2010 period, the increased return was due to higher salaries and increases in tuition subsidies; for 2011-2013, due to subsidies for living costs, partial debt cancellation, and reduction of real interest rates . ICETEX reduced administrative costs from 6.1% in 2006 to 2.6% in 2013, indicating increased efficiencies in expenditure of project resources . The reduced costs stemmed from decentralizing management and monitoring of student loan disbursements to the higher education institutions; computerization and simplification of administrative processes; improvements in information systems; development of a comprehensive website to provide information to students; and maintenance of the permanent staff level at 200 individuals, even as the volume of student loan transactions increased six -fold. ERR )/Financial Rate of Return (FRR) a. If available, enter the Economic Rate of Return (ERR) FRR ) at appraisal and the re-estimated value at evaluation : re- Rate Available? Point Value Coverage/Scope* Appraisal Yes 10.7% 100% ICR estimate Yes 19.67% 100% * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome: Relevance of objectives is rated High and relevance of design Substantial under both the original and revised indicators/targets. Under the original indicators/targets, achievement of one objective is rated High, one Substantial, and one Modest. Under the revised indicators/targets, achievement of one objective is rated High and two Substantial. Efficiency is rated Substantial . These ratings are indicative of minor shortcomings in the project’s preparation and implementation under both sets of indicators /targets, and therefore outcome ratings of Satisfactory under both sets of indicators /targets. The project’s overall outcome rating is therefore Satisfactory . a. Outcome Rating : Satisfactory 7. Rationale for Risk to Development Outcome Rating: A Bank-financed Student Loan Support Project, Phase II, was under preparation at the time of the ICR . It is intended to expand access to tertiary education through ICETEX student loans . The project team later clarified that this second phase was approved by the Board on April 1, 2014. According to the ICR (p. 16), ICETEX is strongly backed by the government and higher education institutions . Demand for student loans continues to be strong, and ICETEX has developed strong and efficient administrative processes . However, there are moderate risks to the development outcomes . Student loan policy has faced some resistance from student groups demanding free public education and /or lower interest rates. The ICETEX funding base is still highly dependent on public resources and is insufficient to satisfy total demand . According to the ICR (p. 16), if the gap between ICETEX lending capacity and student demand widens, student protests could expand. a. Risk to Development Outcome Rating : Moderate 8. Assessment of Bank Performance: a. Quality at entry: Project preparation benefited from “exemplary cooperation� between the Bank, ICETEX, and the government (ICR, p. 3). Considerable sector knowledge had been accumulated during a previous higher education project in Colombia. To respond to the needs of ICETEX and the government, the Bank extended the repayment period of the Bank loan, made repayment disbursement -linked, made the loan directly to ICETEX with a government guarantee, and converted the loan to Colombian pesos on disbursement so that ICETEX did not take a foreign exchange risk . This flexibility in financial terms allowed ICETEX to adapt to changing circumstances, and it also created a strong incentive for ICETEX to manage seriously the repayment and default risk of the student loans, and to increase and diversify sources of alternative funding . High-quality economic and financial analysis informed preparation, including a sensitivity analysis to analyze the behavior of the student debt burden under various interest rates and tuition levels . However, there were shortcomings . Risk assessment did not anticipate the conditions that led to slow disbursement of the Bank loan and implementation delays . The Bank team could not have foreseen the severity of the global financial crisis . Also, the formulation of the objectives should not have included graduation rates, as the duration of studies (eight semesters) exceeded the initial three-year time frame of the project. The expectations of expanding the funding base through the private capital market may have been unrealistic (ICR, p. 16). at -Entry Rating : Quality -at- Moderately Satisfactory b. Quality of supervision: Supervision benefited from close cooperation with the Bank's country office in Colombia . The Bank team collaborated closely with ICETEX to reduce administrative costs, limit exposure to exchange risks, and manage delinquency and default associated with dropouts . The Bank’s flexibility enabled adjustment to the global financial crisis and other issues as they arose . Quality of Supervision Rating : Satisfactory Overall Bank Performance Rating : Satisfactory 9. Assessment of Borrower Performance: a. Government Performance: Changes in government higher education policy during implementation introduced more favorable conditions for student financial assistance . Student loans remain at the core of government policy to increase coverage of higher education. The percentage of tertiary education students subject to ICETEX loans who received these loans increased from 13% in 2006 to 20% in 2013. Government Performance Rating Satisfactory b. Implementing Agency Performance: Implementation responsibility was transferred in 2011 from a Project Implementation Unit to ICETEX. ICETEX (which absorbed PIU staff) maintained strong ownership of the project from preparation through implementation, with highly qualified professional staff and “experience, maturity, and effectiveness � in the assignment and disbursement of student loans (ICR, p. 18). It reacted appropriately to the global financial crisis, improving management of the loan portfolio and reducing its exposure to risk . In particular, ICETEX was effective in engaging collaboratively with the higher education institutions to address the issue of dropouts among student loan recipients, which was the main source of repayment delinquency and default . When the management of student loans was decentralized to the higher education institutions and there were issues with reconciliation of accounts, ICETEX implemented the necessary measures to strengthen financial management. Implementing Agency Performance Rating : Satisfactory Overall Borrower Performance Rating : Satisfactory 10. M&E Design, Implementation, & Utilization: a. M&E Design: Some of the project's outcome indicators did not directly and comprehensively measure achievement of the objectives; in particular, the formal outcome indicators did not measure enrollment and graduation rates for all students and for disadvantaged students . Since the duration of studies (eight semesters) was longer than the planned duration of the project, increases in graduation rates should not have been part of the objectives; it is likely, however, that data on graduation rates will be useful for the second phase of the APL . The methodologies to calculate the PDO-level and monitoring indicators were not included in the PAD or in the operations manual . b. M&E Implementation: ICETEX closely monitored and evaluated the progress of activities carried out under the project . It responded effectively to the Bank’s request for specific data through semi -annual reports. Substantial efforts were made to monitor repayment trends and identify and reach students in default, to monitor the main indicators at the level of the higher education institutions, and to conduct all necessary and planned studies . At the October 2010 restructuring, all but four targets were made more ambitious to reflect the extended implementation schedule and larger resource envelope . Indicators were appropriately modified . c. M&E Utilization: ICETEX efficiently used the M&E framework to manage its relationships with higher education institutions and improve its portfolio. By linking higher education institution contributions to the Sustainability Fund to the dropout rate of students, ICETEX created an incentive for improvement in academic achievements . A planned impact evaluation was not completed in a timely manner and therefore could not provide inputs to the ICR . M&E Quality Rating : Substantial 11. Other Issues a. Safeguards: The project was environmental category C and did not trigger environmental safeguards . The project did trigger the Bank’s Indigenous Peoples safeguards (OP 4.10), as it targeted low-income students. Approximately 3.4% of Colombia’s population self-identify as indigenous peoples, and they are disproportionately represented among the poor. ICETEX introduced ethnicity in the application form for student loans in 2008, and in 2010 it implemented a policy to provide a 50% subsidy to students of indigenous descent . As a result, an increasing percentage of student loan recipients were indigenous people and Afro -Colombians. b. Fiduciary Compliance: A thorough financial management assessment was well documented in the PAD . ICETEX complied with all contractual fiduciary obligations . Only the 2009 audit was qualified because an internal control issue was detected. Difficulties with the reconciliation of accounts became acute when ICETEX decentralized the management of student loans to the higher education institutions in 2009. Nine corrective actions were implemented, and the 2011 and 2012 audit reported were unqualified. No procurement was financed by the Bank loan. c. Unintended Impacts (positive or negative): None reported. d. Other: 12. 12. Ratings : ICR IEG Review Reason for Disagreement /Comments Outcome : Satisfactory Satisfactory Risk to Development Moderate Moderate Outcome : Bank Performance : Satisfactory Satisfactory Borrower Performance : Satisfactory Satisfactory Quality of ICR : Satisfactory NOTES: NOTES - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate. 13. Lessons: The following lessons are adapted from the ICR (p. 19): Student loans can reduce dropout rates, especially among disadvantaged populations . Agreements with institutions of higher educations, in this case, permitted the introduction of a more results -oriented culture. It should not be assumed that funds diversification through private capital markets will be successful . In this case, raising funds from capital markets has been less successful than leveraging ICETEX resources with funds from other sources (due, at last in part, to the impact of the global financial crisis ), including municipalities and universities and through Sustainability Funds . It is possible, through incremental changes to formulas used to allocate student loans, to improve targeting of disadvantaged students and make that targeting compatible with risk reduction . 14. Assessment Recommended? Yes No 15. Comments on Quality of ICR: The ICR is clear, concise, and evidence -based. It effectively harnesses data not only on formal project indicators, but also other available data, to demonstrate achievement of the objectives . However, Annex 1 is incomplete, lacking data on project spending by component . Overall, the ICR handles cost and financing information and analysis poorly, with incomplete and confusing information on contributions from ICETEX equity resources. a.Quality of ICR Rating : Satisfactory