Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review College Education Development Project (P154577) Report Number: ICRR0024174 1. Project Data Project ID Project Name P154577 College Education Development Project Country Practice Area(Lead) Bangladesh Education L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IDA-58410 30-Jun-2022 88,821,603.97 Bank Approval Date Closing Date (Actual) 03-Jun-2016 31-Dec-2023 IBRD/IDA (USD) Grants (USD) Original Commitment 100,000,000.00 0.00 Revised Commitment 93,579,641.92 0.00 Actual 89,474,731.34 0.00 Prepared by Reviewed by ICR Review Coordinator Group Sarveshwari Singh Salim J. Habayeb Susan Ann Caceres IEGHC (Unit 2) 2. Project Objectives and Components DEVOBJ_TBL a. Objectives The development objective of the project as stated in the Financing Agreement and Project Appraisal Document (PAD) was “to strengthen the strategic planning and management capacity of the college education subsector and to improve the teaching and learning environment of participating colleges.” (Financing Agreement, September 8, 2016, p. 4). During implementation, one key associated outcome target was revised downward, along with a revised description of other indicators. Both original and revised outcome targets were achieved, rendering a split Page 1 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review College Education Development Project (P154577) rating evaluation moot for deriving the overall outcome rating. Therefore, this ICR Review does not apply a split rating. b. Were the project objectives/key associated outcome targets revised during implementation? Yes Did the Board approve the revised objectives/key associated outcome targets? --- c. Will a split evaluation be undertaken? No d. Components The project design included three components as stated in the PAD. Component 1: Strengthening Strategic Planning and Management Capacity (appraisal: US$48.2 million; actual cost: US$48 million). The objective was to address planning, governance, and management challenges and undertake reforms at the system level. The component consisted of two subcomponents. Subcomponent 1.1: Development of a Strategic Plan for the College Subsector aimed for a consensus- based plan of action to provide direction and goals for the college education subsector. The Ministry of Education would oversee the process to prepare analytical background documents on significant sectoral issues including access and equity, quality and relevance; management of the college system, financing of college education, and science, technology, and information and communication technology (ICT). Subcomponent 1.2: Improvement of the Teacher Management System would support the Ministry of Education’s efforts to fill teacher vacancies in government colleges and reform the teacher recruitment procedures for non-government colleges. Component 2: Improving Teaching and Learning Environment in Participating Colleges (appraisal: US$70.4 million; actual cost: US$42 million). This component aimed to address the challenge of providing quality and relevant college education at the institutional college level. This component consisted of two subcomponents. Subcomponent 2.1: Improving Teaching-Learning Environment in Colleges aimed to support quality and management enhancement activities. Institutional Development Grants (IDGs) would be provided to eligible colleges that meet selection criteria to improve the teaching and learning environment and institutional management practices. The grants would support upgrades to basic teaching-learning facilities and internet connectivity, improving labor market relevance of college education and linkages with the industry/employers, introduction of quality assurance, and strengthened management capacity and upgrade of colleges’ fiduciary system. Subcomponent 2.2: Strengthening Teaching and Management Capacity in Colleges would provide qualitative improvements in teaching and management in the college subsector through supporting the Page 2 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review College Education Development Project (P154577) training of trainers, teacher training and capacity building of college principals, policymakers, leaders, and managers for subsector development. Component 3: Project Management, Communication, and Monitoring and Evaluation (appraisal: US$11.4 million; actual cost: US$10 million). The objective was to support project management, communication, and monitoring and evaluation (M&E) with the establishment of a Project Management Unit (PMU), capacity building, and technical assistance necessary for project implementation and several analytical studies. This component had two subcomponents. Subcomponent 3.1: Project Management and Communication would ensure sufficient capacity for implementation of policy interventions and dissemination of information. Subcomponent 3.2: Monitoring and Evaluation activities would include Disbursement Linked Indicator (DLI) verification, progress reporting, satisfaction surveys, tracer studies of graduates, and effectiveness assessments on key activities. A web-based Project Management Information System (PMIS) would be established and M&E activities would be carried out by the PMU with support from concerned implementing partners. e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project Cost, Financing, Borrower Contribution Estimated total project cost at appraisal was US$130 million. Project financing at appraisal was US$130 million with US$100 million financed by an IDA credit. The Borrower contribution was US$30 million. Actual total project cost reported in the ICRR is US$89.4 million, while a cost of US$93.5 million was reported by the TTL. Actual IDA project financing was US$93.5 million as US$6.4 million cancelled in December 2023 (ICR Annex 3, p. 33). Actual borrower contribution was approximately US$14 million as reported by the TTL. Dates: The project was approved on June 3, 2016 and became effective on October 13, 2016. A Mid-Term Review was conducted on January 14, 2019. The original project closing date was June 30, 2022 and the date was extended to December 31, 2023 during the first project restructuring. Restructuring: The project was restructured four times. Restructuring 1, May 1, 2019: The project’s first restructuring extended the closing date by 18 months due to implementation that had been stagnant for the first two years owing to the absence of project leadership in the implementing agency as well as key technical staff in Project Management Unit (PMU). The number of rounds of Institutional Development Grants (IDGs) to colleges was reduced from two to one, although the total number of grants remained the same as originally planned. In addition, the description of outcome indicator 3 for teacher training activities was revised to include “certificate of completion” to reflect the addition of enhanced continuous professional development. Restructuring 2, June 14, 2021: Page 3 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review College Education Development Project (P154577) For the project’s second restructuring, two targets were revised. For outcome indicator 1 pertaining to the college sector strategic plan, the end target was revised by removing “the first-year activities have been implemented” and adding “Ministry of Education prepared an investment plan for quality enhancement of college education in line with the college subsector strategic plan.” The target was revised because it did not fully account for the time needed to be in compliance with all the government policies and procedures to develop the National Strategic Plan (NSP), obtain additional funding with the approval of a Development Project Proposal (DPP), and initiate activities based on an NSP. Also, after the project was approved, government policy changed and required the development and approval of a Pre-Development Project Proposal. As a condition for submitting a Pre-Development Project Proposal, an investment plan was required to be included. In addition during the restructuring, for outcome indicator 3, the target for the number of teachers and managers trained was reduced from 8,000 to 5,000 due to cancellation of in-person trainings during the COVID-19 pandemic and limited online training capacity. Restructuring 3, June 1, 2023: This restructuring had the following change in the results framework: for outcome indicator 3 on teacher training, the target for the number of teachers and managers trained was increased from 5,000 to 6,500. Restructuring 4, December 5, 2023: Ninety-four percent of the loan amount had been utilized and the remaining 6 percent was canceled. 3. Relevance of Objectives Rationale Project objectives were well-aligned with the Government of Bangladesh’s 7th Five Year Plan, the National Education Policy (NEP) 2010, and the Strategic Plan for Higher Education (2006–2026). Additionally, the Government noted in its Perspective Plan (2010–2021) that strengthening the quality of tertiary education institutions (TEIs) is necessary to enable its vision for Bangladesh to attain middle-income status. (PAD, para 12). At appraisal, objectives were relevant and aligned with the World Bank’s Country Partnership Framework for the period FY2016-FY2020 (Report No.141189-BD). The project supported Country Partnership Framework objective 2.1 on improved equity in access, quality, and relevance in education. The project built on prior World Bank experience in the higher education sector in Bangladesh including the Skills and Training Enhancement Project (P090807) and the Higher Education Quality Enhancement Project (HEQEP) (P106216, 2009-2018). The objectives were appropriate for the stage of the Government’s plan and focused on supporting system and institutional level reforms to establish an enabling environment for teaching and learning. The project aimed to address inadequate teaching and learning infrastructure especially for science and technology, lack of relevance to employers, absence of quality assurance mechanisms at the institutional level, and inadequate planning and management capacity for institutional development. Expanding access and Page 4 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review College Education Development Project (P154577) enrollment before such an environment was in place leading to producing graduates without the requisite skills to move Bangladesh to middle-income country status (PAD, para 13). During implementation, restructurings were undertaken that aimed to expedite implementation while maintaining focus on project objectives. In response to the COVID-19 pandemic, teacher trainings were shifted to online format and an online learning platform was established. At closing, project objectives remained responsive to Government of Bangladesh’s priorities. The National Strategic Plan for higher education colleges 2023-2031 that was supported by the project is closely aligned with Strategic Plan for Higher Education 2018–2030, the Eight Five-Year Plan, and the Government’s Vision for 2021 and 2041. The objectives continued to be fully aligned with the World Bank’s Country Partnership Framework (CPF) for Bangladesh for FY2023–FY2027 (Report No: 181003-BD) as it contributed to the CPF’s High- Level Objective on Improved Socioeconomic Inclusion. The CPF further notes that the World Bank Group will help the country promote investments in skills development by filling current quality gaps in higher education. Rating Relevance TBL Rating High 4. Achievement of Objectives (Efficacy) EFFICACY_TBL OBJECTIVE 1 Objective To strengthen the strategic planning and management capacity of the college education subsector. Rationale The ICR prepared a retrofitted theory of change (TOC) and results chain as this was not required at the time of preparation. The aim of Objective 1 was to strengthen the National University (NU) college system’s capacity to plan, manage, implement and monitor institutional programs. This would be achieved by supporting the development of a sector-wide strategy to guide policymakers, academia and college administrators in achieving the Government’s goals in higher education. In addition, activities included the preparation of an investment plan to enable funding for implementation of the strategy. The college subsector would also be strengthened by filling government college teacher vacancies and preparing a needs assessment to inform the recruitment of government college teachers for the next three years. By implementing these actions at the college education subsector level, strategic planning and management capacity would be strengthened. Outputs and intermediate results: Page 5 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review College Education Development Project (P154577) Tracer studies of graduates were conducted to strengthen sector planning capacity (IRI 6), and the target was achieved. Prior to the project, information was not available on labor market outcomes of students from the affiliated college sector. Baseline, midterm, and follow-up tracer studies of graduates were conducted and highlighted serious employability issues, initiating national policy dialogue on poor graduate outcomes. Findings from tracer studies of graduates informed the National Strategic Plan which included recommendations for improving employability of graduates. Outcomes: A sector strategy and action plan were developed (PDO 1, DLI 1) and the target was met. The project supported the development, approval, and publication of a National Strategic Plan for Higher Education Colleges 2023-2031, the first plan for the tertiary college subsector. The NSP included an action plan to improve employability of graduates and sector outcomes in the following areas: (a) vision, size, and shape; (b) access and equity; (c) quality and relevance; (d) management of the college system; (e) science, technology, and information and communication technology (ICT); and (f) financing. The draft National Strategic Plan was the subject of a national consultative workshop with stakeholders in December 2022. (ISR #15, February 2023). The Ministry of Education approved and published the National Strategic Plan for Higher Education Colleges 2023-2031. With regard to the investment plan and revised target, it was prepared based on the National Strategic Plan. It was discussed in regional and national consultations and then approved by the Secondary and Higher Education Division in August 2023. With this plan, a Pre-Development Project Proposal from the Ministry of Education to the Economic Relations Division was submitted. This would enable the allocation of government and donor funds for continued development of the college subsector. The second outcome indicator (PDO 2, DLI 2) was 3,500 government college teacher vacancies were filled, and the target of 2,700 filled vacancies was exceeded. In addition, the target included completion of a needs assessment for teacher posts to assess gaps and inform recruitment needs and this was met. The project aimed to support the Government to expedite the deployment of teachers to ensure that government colleges were adequately staffed. Competitive exams to fill vacancies had not been held by the Public Service Commission in the prior three years. To improve the teacher management system for non-government colleges (DLI 3) and select and hire quality teachers, the Government aimed to amend the Amendment of the Non-Government Teachers’ Registration and Certification Act (NTRCA) 2005. The Act was amended. Objective 1, to strengthen strategic planning and management capacity of the college education subsector, was achieved with the preparation and approval of the college sector’s first national strategic plan and investment plan which provided the basis for allocating funds for the college subsector. Teacher vacancies were filled and a needs assessment was prepared to inform future recruitment requirements to enable better planning and execution in the recruitment of teachers. Recommendations were made to improve graduates’ employability based on findings from the tracer studies. Rating Substantial OBJECTIVE 2 Page 6 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review College Education Development Project (P154577) Objective To improve the teaching and learning environment of participating colleges. Rationale Objective 2 aimed to support improvements in the teaching and learning environment at the college level. Main activities included: training teachers and managers in subject and pedagogical training; integrating courses on subjects with market relevance such as science and technology; publishing timely annual reports by colleges; and conducting self-assessments. Grants provided to participating colleges aimed to upgrade basic teaching and learning facilities and internet connectivity, enhance the market relevance of college education through the development of students’ soft skills and industry/employer links, and strengthen management capacity in colleges. By undertaking these activities at the individual college level, these steps would contribute to improving the teaching and learning environment. Outputs and intermediate results A total of 2,342,802 college students in the NU college system benefitted from the direct interventions to enhance learning (IRI 1), exceeding the original and revised targets of 2,000,000 and 2,100,000 students, respectively. Of the total number of beneficiaries, 1,124,924 were female students, surpassing the target of 1,000,000. While the indicator measured the number of beneficiaries, it does not measure how students benefitted from an improved teaching and learning environment. Eighty-six percent of colleges that received Institutional Development Grants (IDGs) published timely annual reports (IRI 2) for 2023 that highlighted the progress made on institutional development activities, exceeding the target of 80 percent. The aim of this activity was to strengthen management of the participating colleges. The reports provide information on college activities, development projects, students and teachers, academic programs, and financial data. The reports were accessible to the public for accountability, transparency, and informed student choice. Prior to the project, the colleges did not publish regular annual reports. Thirty-five grantee colleges completed self-assessments for quality assurance (IRI 3), exceeding the target of 30 colleges. Institutional Development Grants were used by colleges to prepare, for the first time, self- assessments for quality assurance that included a review of the quality of teaching, curricula, and feedback from teachers and students about the learning environment. The self-assessments aimed to support colleges to develop their own action plans to improve their teaching and learning environment and quality. The assessments were externally peer reviewed and enabled the colleges to prepare Institutional Development Plans to guide continuous quality assurance activities. The Institutional Development Grants aimed to improve the teaching and learning environment in selected government and non-government colleges (DLI 4). Grants of up to US$1 million provided to the 120 participating colleges were used to refurbish 1,889 classrooms and establish 158 Information and Communication Technology (ICT) labs, 269 science labs, 624 libraries, and establish 62 mothers’ and childcare corners, which were mainly childcare centers and included facilities to support lactating mothers. The Bangladesh Research and Education Network connection was extended to 110 colleges, and campus networking was established in 94 colleges, enhancing access to high-speed internet and global academic resources. An evaluation of these grants found that they significantly improved the teaching and learning environment, particularly in poorly-equipped colleges. In addition, management capacity of more than 9,988 teachers and managers was strengthened in areas such as procurement, financial management, ICT, office management, recordkeeping, and M&E. A second round of smaller performance-based grants of US$10,000 Page 7 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review College Education Development Project (P154577) were provided in the last year of the project to 35 colleges, a subset of the 120 colleges that had performed well, and were used to purchase classroom furniture and equipment. Satisfaction levels of students of grant-supported colleges about the teaching and learning environment increased 37% from baseline, exceeding the target of 30%, while satisfaction levels of teachers increased 10%, falling short of the same target. (IRI 4). Three rounds of surveys were conducted and the end-line survey found that student satisfaction increased from baseline for teaching-learning facilities, soft skill development, academic infrastructure, industrial cooperations, and internet connectivity. Based on the survey findings, recommendations were made to establish mentoring programs, offer internship opportunities to students, and integrate English language programs. The ICR would have been strengthened by indicating whether recommendations were used to improve the teaching and learning environment. Outcomes A total of 8,306 college teachers and managers in Honors and Masters colleges were trained in subject and pedagogical training (PDO 3, DLI 5), exceeding the revised target of 6,500 and the original target of 8,000. An evaluation of the teacher training program was conducted that confirmed positive outcomes on improved teaching skills, subject knowledge, English proficiency, technological competence and alignment with international benchmarks for teaching standards. Grantee colleges achieved 84 percent of the milestones identified in their respective Institutional Development Plans (PDO 4), exceeding the target of 70 percent. The Institutional Development Plans included goals, actions to attain them, and milestones to measure achievements. Rating Substantial OVERALL EFF TBL OBJ_TBL OVERALL EFFICACY Rationale Objective 1, to strengthen the strategic planning and management capacity of the college education subsector, was achieved. The college subsector’s first national strategic plan and investment plan were approved, teacher vacancies were filled, and the findings from tracer studies of graduates contributed to recommendations for improving graduates’ employability. Objective 2, to improve the teaching and learning environment of participating colleges, was achieved. Teachers and managers were trained and grantee colleges met milestones in their Institutional Development Plans. The completion of self-assessments enabled colleges to develop action plans to improve the teaching and learning environment. Evidence for achievement would have been strengthened with information on how colleges used annual reports, self-assessments, and achievement of milestones to improve the teaching and learning environment. Objective 1 is rated Substantial, and the achievement of Objective 2 is rated Substantial resulting in an aggregate achievement of Substantial for Overall Efficacy. Page 8 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review College Education Development Project (P154577) Overall Efficacy Rating Substantial 5. Efficiency At appraisal, a cost-benefit analysis was conducted for the project. The analysis assumed that students would have an increased likelihood of graduating from college and better prospects for employment. Costs included the project costs and the opportunity costs of foregone income. The net present value (NPV) and the internal rate of return (IRR) were estimated for three scenarios that used varying assumptions of graduation rate, employment, and wage premium. In the medium-case scenario, net present value (NPV) was expected to be US$283 million with an internal rate of return (IRR) of 28 percent. The analysis assumed a discount rate of 3 percent. The ICR conducted a cost-benefit analysis using the same approach with several different assumptions. For example, the ex ante analysis used employment of graduates at the national level whereas the ex post analysis used extrapolated data from the tracer studies conducted during the project. Based on the ex-post analysis, the NPV was expected to be US$620 million and the project’s IRR is 14 percent. The analysis assumed a discount rate of 7 percent. For both analyses, the focus was on the individual beneficiary level and excluded positive externalities from improved education quality. The economic returns are likely to be greater than the simple costs and benefits analysis presented. With regard to implementation efficiency, teacher training activities leveraged facilities and logistical support of four government entities, enabling cost efficiencies compared with full outsourcing. The Project Management Information System (PMIS) that was established enabled data collection from 120 colleges that received IDGs, reducing the cost of manual collection and analysis. The first two years of the project experienced delays across all project activities due to the lack of project leadership in the implementing agency and the absence of procurement and financial management specialists in the PMU. This led to the project’s first restructuring which extended the closing date by 18 months. By the Mid- Term Review, performance had improved. In addition, a 21-month Government approval process for the second restructuring delayed the release of a second round of small performance grants to selected colleges. At closure, 6 percent of total committed funds were cancelled and savings were derived from the technical assistance component, small works and services that were not required, and the development of e-content for a Learning Management Information System which was dropped. On balance, the shortcomings were moderate, and coupled with the positive economic analysis, Efficiency is rated Substantial. Efficiency Rating Substantial Page 9 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review College Education Development Project (P154577) a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 0 Appraisal  28.00  Not Applicable 0 ICR Estimate  14.00  Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome Relevance of Objectives is rated High as the project objectives were aligned with the Bank’s CPF FY2023-2027 and the government’s Strategic Plan for Higher Education (2006–2026). Efficacy is rated Substantial due to the substantial achievement of the two objectives. Efficiency is rated Substantial for positive economic benefits and implementation efficiency. The overall Outcome is rated Satisfactory. a. Outcome Rating Satisfactory 7. Risk to Development Outcome The project aimed to build the capacity of the college subsector and the teaching and learning environment. Financial sustainability depends on annual college budgets and the ability to accommodate operations and maintenance of new facilities including high-speed internet connectivity. After project closure, the Government evaluated the results of the Institutional Development Grants and provided US$2.36 million for an additional year to support 120 colleges through December 31, 2024. With the change in government in August 2024, government support to the project’s objectives continues, per the TTL. 8. Assessment of Bank Performance a. Quality-at-Entry The project was relevant and aligned with the Government’s strategy to improve the access, quality, and management capacity of the teaching and learning environment in the college subsector. The project appropriately focused on supporting system and institutional level reforms to establish an enabling environment for teaching and learning. The project was informed by prior World Bank operations in higher education and addressed significant and relevant challenges. Page 10 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review College Education Development Project (P154577) Overall responsibility for implementation was with the Ministry of Education while the National University oversaw day-to-day implementation activities. The University Grants Commission was to play a coordinating role between the MoE and the National University. The Project Management Unit (PMU) was established at the NU. The risk rating for institutional capacity for implementation and sustainability was Substantial, which underestimated the challenges. During project preparation, a financial management assessment was conducted to evaluate the adequacy of the proposed financial management arrangements with the implementing entities. Recommendations included introduction of an automated record keeping system, continuous fiduciary training, development of an FM Action Plan to ensure a sound internal control system. Several measures needed to minimize FM-related risks included establishing a web-based accounting system and continuous training of stakeholders at all levels. An in-depth procurement capacity and risk assessment was conducted. The key drivers of procurement risks were associated with the acute capacity constraint in managing procurement and contract management-related activities, inexperience, delay in processing procurement activities, inadequate record management system, and vulnerability of the agencies with respect to inappropriate procurement practices undertaken by the bidders. While measures to mitigate risks were put in place, they did not suffice to enable implementation readiness and contributed to the initial two-year implementation delay. Also, M&E design had shortcomings discussed in section 9a. Quality-at-Entry Rating Moderately Satisfactory b. Quality of supervision The World Bank team conducted twelve formal implementation support missions that included both technical and fiduciary support. In addition, weekly and monthly meetings were conducted with the NU, PMU, and grantee colleges. During the first two years of the project, supervision was appropriately focused on addressing significant implementation challenges and problem project status due to serious fiduciary issues (see section 10b). ISRs and Aide Memoires were candid, documenting challenges and progress. The Bank team established time-bound action plans. The team carried out restructurings in response to implementation challenges and the COVID-19 pandemic. The Bank team was proactive in providing implementation support which contributed to improvements in implementation performance by the Mid- Term Review and through project closing. Quality of Supervision Rating Satisfactory Overall Bank Performance Rating Moderately Satisfactory 9. M&E Design, Implementation, & Utilization Page 11 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review College Education Development Project (P154577) a. M&E Design Objectives clearly stated the project’s focus on two areas: strengthening the strategic planning and management capacity of the college subsector and improving the teaching and learning environment at the individual college level. Evaluations of the Institutional Development Grants and teacher training were included in the project design. Tracer studies aimed to track graduates’ employability. Beneficiary surveys were planned to track satisfaction of students, teachers, and employers. As for the indicators, measuring the number of beneficiaries (IRI 1) did not incorporate how students benefitted from an improving the teaching and learning environment. Also, while surveys of beneficiaries (IRI 4) can provide a perspective on project activities, they fall short of providing evidence of improvement in the teaching and learning environment. b. M&E Implementation At the project level, monitoring and evaluation was the responsibility of the Project Management Unit (PMU). The PMU submitted 14 semiannual monitoring reports. The World Bank team conducted biannual implementation support missions in a timely manner. During implementation, a Project Management Information System (PMIS) was developed and was functional. The PMIS collected grant-related data from 120 colleges and facilitated the consolidation and reporting of the data. Planned evaluations were conducted for the teacher training program and the Institutional Development Grants. Tracer studies of graduates were undertaken. Baseline, midline, and endline satisfaction surveys of principals, teachers, students and employers were conducted. c. M&E Utilization Findings from the tracers studies were used to develop recommendations to improve graduates’ employability and they were included in the National Strategic Plan. While recommendations were made from the findings of the beneficiary satisfaction surveys, how the recommendations were used to improve the teaching and learning environment were not noted. M&E Quality Rating Substantial 10. Other Issues a. Safeguards Project activities triggered Environmental Assessment OP/BP 4.01 related to renovation and refurbishment of buildings, classrooms, laboratories, and establishing fiber optic cable connectivity. The small-scale civil works were not anticipated to cause any major adverse environmental impact and the project was rated as Category B at appraisal requiring partial assessment. An Environmental Management Framework was Page 12 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review College Education Development Project (P154577) prepared by the Borrower to enable compliance with the World Bank’s safeguard policies. Indigenous Peoples OP/BP 4.10 was triggered as project activities would take place at colleges throughout the country, including areas with small ethnic and vulnerable communities. The project was not expected to cause adverse impacts on the indigenous people including their lands, culture, livelihood, or way of life. During project preparation the Borrower prepared a Social Management Framework. Ratings were Satisfactory during implementation and at project closure (ICR, p. 17). Risk ratings were Moderate at appraisal and throughout implementation, and at project closure. The project included a Grievance Redress Mechanism and minor work injuries were reported and addressed. Ten grievances were reported regarding personal protective equipment and they were satisfactorily addressed. b. Fiduciary Compliance In the first two years after project effectiveness, four Interim Unaudited Financial Reports were overdue and auditors expressed a qualified opinion on financial statements. The fiduciary rating was downgraded to Moderately Unsatisfactory in March 2018. By the Mid-term Review, financial management improved due to progress in deploying a financial management specialist, clearing pending interim unaudited financial reports, and installing off-the-shelf computerized accounting software. Ratings improved to Moderately Satisfactory in February 2019 and then to Satisfactory in November 2019. In August 2022, performance was downgraded for a short period to Moderately Satisfactory until February 2023 when the rating improved to Satisfactory. This brief period of downgrade was due to delays in submission of Interim Unaudited Financial Reports. The project closed with a Satisfactory rating. The ICR indicated that the overall financial management performance of the project was Satisfactory. (ICR, p. 18) With regard to procurement, during the first two years of implementation, a procurement specialist had not yet been deployed in the PMU. Procurement performance was downgraded to Moderately Unsatisfactory from March 2018 to August 2018. By the Mid-Term Review, a senior procurement specialist was hired and an updated procurement plan was maintained. Additional time-bound milestones to be reached were agreed upon including initiating the use of electronic government procurement, hiring at least two additional procurement specialists, and sharing an updated procurement training plan. These actions led to an upgrade in the procurement rating to Moderately Satisfactory in February 2019 and then Satisfactory in May 2021 which remained until project closing. The ICR noted that the overall procurement management performance was Satisfactory. (ICR, p. 18) c. Unintended impacts (Positive or Negative) None reported. d. Other N/A Page 13 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review College Education Development Project (P154577) 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment Outcome Satisfactory Satisfactory Shortcomings in risk assessment and mitigation, including underestimation of Bank Performance Satisfactory Moderately Satisfactory known implementation challenges and shortcomings in M&E. Quality of M&E Substantial Substantial Quality of ICR --- Substantial 12. Lessons The ICR provided insightful lessons including those below: When engaging for the first time in a subsector, simple and realistic project design can enable smooth implementation. As a first operation in the college subsector, project design incorporated targeted interventions for system and institutional capacity building at the national and college subsector levels. Sufficient time needs to be built into a project to allow for engagement and consultations among many stakeholders undertaking new strategic-level and management activities to achieve project objectives. Establishing project leadership and the PMU in a timely manner enables implementation readiness. During preparation, the risk assessment for institutional capacity for implementation and identification of mitigation measures can help ensure readiness. Dialogue with the Government on implementation readiness could include, for example, consideration of assignment of a project leader from the Government before approval. This can enable timely staffing of the PMU with the requisite technical expertise and commencement of project activities. Continuous hands-on support for capacity building can enable successful implementation of new activities. The Institutional Development Grants provided to 120 colleges supported new activities including goal setting, planning, implementation and reporting. When there is a lack of experience for implementation, scalable on-the-ground capacity building, training, and rapid response are essential throughout project duration for institutional development and successful implementation. 13. Assessment Recommended? Page 14 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review College Education Development Project (P154577) No 14. Comments on Quality of ICR The ICR provided a good description of project implementation and restructurings in a concise manner. Annexes included detailed information on the Institutional Development Grants and their outputs. Lessons were clearly drawn from implementation experience. With regard to the quality of evidence and analysis, while the ICR provided information on targets being met and achievement of objectives, the ICR would have been strengthened by reporting how findings from the college annual reports, self-assessments, and achievement of milestones were used, and thereby providing evidence of improvement in the teaching and learning environment. With regard to consistency, the ICR noted that all financial reports were submitted on time with acceptable quality when serious fiduciary lapses in the first two years of implementation including delays in submission of financial reports were noted in an Aide Memoire (July 2018). Also, for a short period in 2022, there were delays in submission of financial reports. For procurement, the ICR reported performance as Satisfactory while elsewhere in the ICR, it is noted that the project lacked procurement specialists during two years of implementation which contributed to significant delays. a. Quality of ICR Rating Substantial Page 15 of 15