Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MR - Skills Dev. Support Project (FY11) (P118974) Report Number : ICRR0021344 1. Project Data Project ID Project Name P118974 MR - Skills Dev. Support Project (FY11) Country Practice Area(Lead) Additional Financing Mauritania Education P144575 L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IDA-49210,IDA-H9650 30-Apr-2016 22,405,137.92 Bank Approval Date Closing Date (Actual) 26-Apr-2011 31-Dec-2017 IBRD/IDA (USD) Grants (USD) Original Commitment 16,000,000.00 0.00 Revised Commitment 27,300,000.00 0.00 Actual 23,150,469.18 0.00 Prepared by Reviewed by ICR Review Coordinator Group Judith Hahn Gaubatz Judyth L. Twigg Joy Behrens IEGHC (Unit 2) 2. Project Objectives and Components a. Objectives According to the Financing Agreement (page 5) and the Project Appraisal Document (PAD, page 4), the project objectives were as follows: • To improve the quality and efficiency of training institutions and create an enabling environment for a more market-driven Technical and Vocational Education Training system. Page 1 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MR - Skills Dev. Support Project (FY11) (P118974) The project objectives remained the same throughout the project period. Additional Financing (AF) was approved for the project in April 2014, in order to scale up activities and project impact. Accordingly, outcome targets were revised upwards; as the project revisions are considered an expansion, rather than a narrowing, of project scope, a split outcome rating is not applicable. b. Were the project objectives/key associated outcome targets revised during implementation? Yes Did the Board approve the revised objectives/key associated outcome targets? Yes Date of Board Approval 30-Apr-2014 PHEVALUNDERTAKENLBL c. Will a split evaluation be undertaken? No d. Components 1. Strengthening and Diversifying Technical and Vocational Education Training (TVET) (Appraisal: US$ 13.0 million; Appraisal + AF: US$ 23.6 million; Actual: US$ 19.8 million): This component aimed to improve the quality, effectiveness and relevance of training provided by TVET institutions. Activities included: development and financing of School Program Implementation Agreements with individual TVET institutions, which included a results-driven framework, upgrading of equipment, curriculum revision, teacher training, and management capacity building; and financing of apprenticeship and short-term training subprojects targeting unskilled youth, according to proposals submitted by training providers (public, private, non-governmental organizations, or employers). Participating TVET institutions were selected according to the following criteria: demonstrated economic and social demand for the skills training offered; prepared mission statement and action plan; and the availability of options for increasing capacity and improving quality without new construction. 2. Improving the Institutional Environment of Technical and Vocational Education Training (Appraisal: US$ 4.6 million; Appraisal + AF: US$ 5.3 million; Actual: US$ 4.6 million): This component aimed to strengthen the capacity of the Ministry of Employment, Vocational Training, and New Technologies (MDEFPNT) to create a more demand-driven TVET system, and the capacity of the National Institute for the Promotion of Vocational and Technical Training (INAP-FTP) to manage the TVET system. Activities included: support for updating the TVET regulatory framework (particularly as pertains to private sector training providers); developing strategic planning and monitoring capacity and a human resources development plan; strengthening capacity to monitor performance agreements with TVET institutions and to guide curriculum development and teacher training; developing a framework for skills certification; and conducting of tracer studies. Activities also included a communications strategy to increase demand for TVET and a survey to identify bottlenecks for women entering the training and labor market. Page 2 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MR - Skills Dev. Support Project (FY11) (P118974) The April 2014 AF financed the following activities: rehabilitation of four TVET centers, increase in the number of performance-based contracts from 8 to 14 TVET institutions, and expansion of apprenticeship programs and TVET programs. A February 2015 project restructuring added a third component: Prevention of the Ebola Epidemic (Appraisal: US$ 0.7 million; Actual: US$ 0.7 million) that financed the establishment of a rapid-response team and an isolation unit for case management. See below for further details. e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project Cost • The original appraised project cost was US$ 17.6 million. With the AF of US$ 11.3 million, the revised appraised project cost was US$ 28.9 million, of which US$ 24.75 million disbursed. The shortfall was in part due to exchange rate fluctuations, such that the actual amount disbursed from the original Credit was lower (91.7% disbursed), as well as lower than expected spending from the AF Grant (82.6% disbursed), with the remaining unused funds still to be returned to the Bank. The project team reported that the process of returning the funds was delayed due to the time needed to close and audit the accounts as well as the holiday period, and that three letters have been sent from the client to the Al Amana Bank specifying the amounts to be transferred from each of the two project accounts. Financing • The project was financed by an International Development Association (IDA) Credit of US$ 16.0 million and an additional IDA Grant of US$ 11.3 million. The sources of funds for the additional grant were recommitted IDA funds from three other IDA operations in Mauritania (Higher Education Project (P087180), Multi-sector HIV/AIDS Control Project (P078368), and Public-Sector Capacity Building Project (P082888)) that were not expected to be utilized in those projects. Borrower Contribution • The Borrower was expected to provide counterpart funds in the amount of US$ 1.6 million, which was provided in full. Page 3 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MR - Skills Dev. Support Project (FY11) (P118974) Dates • April 2014: AF in the form of an IDA grant of US$ 11.3 million was approved, in order to expand the scope of the project commensurate with the increased financing. The project activities of performance- based contracts and short-term training were scaled up to increase the number of project beneficiaries, and rehabilitation of four TVET centers was added. Targets for key indicators were increased accordingly. The Environmental Category was revised from "C" to "B" due to the addition of civil works. • February 2015: The project was restructured to add a third component, in response to the emerging Ebola Virus Disease in neighboring countries. The rationale for including this component in the project was that risks associated with potential school closures could have a significant negative impact on project outcomes. The amount for the component (US$ 0.7 million) was allocated from other components that were not expected to be utilized by project closing. Intermediate indicators were added to track the new activities. • February 2016: The project closing date was extended from April 2016 to December 2017, to allow for completion of activities associated with the AF. 3. Relevance of Objectives Rationale Mauritania has been experiencing steady economic growth in recent years, although it remains highly dependent on the exploitation of mineral resources and has seen only limited growth in the manufacturing sector. In order to ensure stable and sustainable growth, the country needs to support small and medium enterprise development and improve competitiveness. Although the country's recent poverty reduction strategy (2009) prioritizes private sector development, the strategy identifies key constraints of a lack of a supportive legal environment to encourage business development and a shortage of skilled labor. According to the 2006 Investment Climate Survey, one in every five firms in Mauritania reported that low worker skills and education were major productivity constraints (as reported in the ICR, page 1). Small firms are generally unable to finance necessary skills training, yet the TVET subsector has historically been too small to fill this gap; at the time of project appraisal, there were fewer than 5,000 students enrolled in TVET institutions, representing only three percent of total secondary education enrollments. Moreover, the TVET education that has been provided is of low quality, marked by inefficient student:teacher ratios, high training costs, and high unemployment rate among TVET graduates. Mauritania also has a significant out-of-school youth population that is unskilled and hence unemployed. Page 4 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MR - Skills Dev. Support Project (FY11) (P118974) Given these significant constraints, the government created the MDEFPNT in 2009 and adopted a TVET strategy for 2010-2020 that focuses on improving the quality and efficiency of TVET programs, strengthening TVET institutional capacity, and increasing outreach to unemployed youth and employers. The Bank's Country Partnership Framework for FY18-23 identifies increasing employability, particularly of youth and women, as a key focus area, including the number of young entrepreneurs employed as a key indicator. The project objectives are strongly consistent with both country and Bank strategies. Rating High 4. Achievement of Objectives (Efficacy) PHEFFICACYTBL Objective 1 Objective To improve the quality of training institutions Rationale The theory of change was overall sound, with project outputs likely to lead to the intended outcomes of improved quality and efficiency of the TVET sector. Outputs to improve quality included improving teacher training, revising curricula, and upgrading training equipment. Outputs to improve efficiency included increasing enrollment to maximize capacity and the provision of short-term training opportunities. The performance-based contract approach used also aimed to improve efficiency by ensuring optimal resource allocation. Outputs to increase relevance of the training to market demand (derived from outreach campaigns to small industries and TVET service providers) were likely to contribute to both improved quality and improved efficiency. Outputs • Training of 400 teachers and TVET institution staff, to provide job training in the sectors of agriculture, construction, mechanics, electronics, technical education, and information technology. • Upgrading of four TVET facilities in Nouakchott, Nema, Kaedi, and Boghe. • Provision of short-term training to 8,042 out-of-school youth, including in audio-visual media and fisheries (target: 6,500). The ICR (page 18) noted that these were industries not covered by existing TVET institution programs, and thus private sector training providers took advantage of the funding made available to provide short-term training courses in these industries. For example, the 2014 deregulation of news media led to the creation of private and online media outlets; the 2016 government restriction on foreign workers in the fisheries sector led to an increase in demand for domestic workers. Page 5 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MR - Skills Dev. Support Project (FY11) (P118974) • Financing of Performance-Based Contracts (PBCs) with 12 TVET institutions, based on individualized School Performance Implementation Agreements that included revision of curricula to better align with market needs, purchase of training equipment, and development of partnerships with local firms (original target: 8; revised target: 14. Two of the 14 institutions were re-arranged and/or merged such that there were 12 actual institutions eligible for funding). The ICR (page 17) noted that PBCs enabled institutions to hire tutors to provide on-site training within private companies and coaching for students to help with transition to the job market. However, • International standards for certification were not adopted by the TVET institutions as planned, although four training sessions on the topic were organized to raise awareness among TVET managers and prepare for future actions. The ICR (page 44) suggested that this may have been an overly ambitious planned output, as international standards appear "misaligned" or not compatible with the standards sought by the labor market served by the project. Outcomes • The cumulative number of graduates from project-supported TVET institutions increased from 993 in 2011 to 8,367 in 2017. This surpassed the original target of 5,161 but fell short of the revised target of 9,411. The ICR (page 19) notes that this shortfall was due to the reduction in participating TVET institutions (12, compared to target of 14). The average graduation rate from the TVET institutions having PBCs with the project increased from 71% to 86%. The pass rate on national exams increased from 23% in 2008-09 to 80% in 2016-17. • The overall insertion rate of TVET graduates nationwide increased from 61.5% in 2012 to 65.2% in 2017. This may be partially attributable to the project's interventions, as project schools enrolled two- thirds of all TVET students in the country. The highest levels of insertion were in the industries of dyeing, metal construction, masonry, auto mechanics, building electricity, wood carpentry, market gardening, and sewing and confection. • 71% of trainees in the short-term training programs obtained employment within six months, surpassing the target of 64%. • The drop out rate in project TVET institutions decreased from 9% in 2011 to 5.8% in 2017, falling short of the target of 3%. The ICR (page 18) suggests that the target may have been ambitious given historically high drop-out rates in the TVET sector due to higher opportunity costs for TVET training compared to other educational sectors. • The proportion of unskilled female youth enrolled in training programs was 34%, surpassing the target of 27%. The ICR (page 23) reports that the project was also able to provide female-led NGOs and female Page 6 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MR - Skills Dev. Support Project (FY11) (P118974) employers the opportunity to benefit from short-term training, as both beneficiaries and service providers. Approximately 20 qualifying training sessions were organized for activities traditionally assigned to women. • The highest insertion rates among the short-term training participants were the poorer regions of Tagant (96.4%), Guidimagha (95.4%), Assaba (94.7%), and Gorgol (88.6%), followed by Nouakchott (80.9%). Achievement is rated Substantial due improved graduation outcomes and evidence of substantial insertion rates for TVET graduates, although there were shortcomings in achieving the targets for number of graduates and decreased drop-out rates. Rating Substantial PHREVDELTBL PHEFFICACYTBL Objective 2 Objective To improve the efficiency of training institutions Rationale Outputs See outputs reported above. Outcomes • Enrollment rates at project TVET institutions increased by 47.2% over the project period, surpassing the original target of 38% and meeting the revised target of 45%. • The unit cost for short-term training decreased from US$ 671 in 2014 to US$ 653 in 2016/17. The ICR (page 19) referred to the project's final evaluation report, which stated that the decreased unit cost did not adversely affect the quality of training, but was achieved primarily through the use of more efficient practices such as part-time trainers. (Note: A baseline figure of US $640 was provided in the results framework but was deemed unreliable; the baseline figure was updated to US$ 671 at the time of project restructuring.) • The student:teacher ratio in project TVET institutions increased from 9:1 to 12:1, achieving the target of 12:1. Page 7 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MR - Skills Dev. Support Project (FY11) (P118974) Achievement is rated Substantial due to evidence of increased enrollment, decreased unit cost of short-term training, and increased student:teacher ratio, all of which indicate increased efficiency in the use of resources in the TVET sector. Rating Substantial PHREVDELTBL PHEFFICACYTBL Objective 3 Objective To create an enabling environment for a more market-driven Technical and Vocational Education Training system Rationale The theory of change for this objective was overall sound, as it directed capacity building activities towards both the MDEFPNT, which had been recently established and was responsible for the strategic oversight and management of the TVET sub-sector, and the INAP-FTP, which was responsible for day-to-day operations of an autonomous training fund. Outputs • Technical assistance in updating of the TVET regulatory framework, including extensive consultations with stakeholders. The updated framework addressed issues of expansion of private sector provision of TVET, the strengthening of funding mechanisms, the reinforcement and development of the apprenticeship system, support for inclusivity and equity in the TVET sector, and the strengthening of control and supervision mechanisms. • Conducting of three tracer studies (2015, 2016, and 2017) with the participation of 26 TVET institutions, and an out-of-school youth tracer study (2016). In addition to demonstrating results for this project, the tracer studies were intended to inform management decisions for adjusting training curricula. • Preparation of annual statistical yearbook with updated information on key TVET indicators. • Development of human resource development plan to strengthen the capacity of the MDEFPNT. • Establishment of an Ebola Virus Disease rapid-response team, isolation unit for case management, guidelines for detection and treatment, and relevant training. Outcomes Page 8 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MR - Skills Dev. Support Project (FY11) (P118974) • Draft legislation to update the TVET regulatory framework was completed and approved by the Council of Ministers in November 2017. At the time of ICR preparation, the draft legislation was under review by the National Assembly and pending final adoption, as recommended by a technical committee of the National Assembly, and was scheduled for a vote for the session ending in July 2018. The project team subsequently reported that the legislation (Law No 2018-038) was adopted by the parliament on August 22, 2018. • Increased collaboration between TVET providers and employers, as evidenced in part by the updating of 28 specialized programs according to input from employers. • The share of TVET students enrolled with private sector training providers at project closing was 25%, achieving the target of 25%. • The ICR (page 24) reported findings from an independent assessment of the performance-based contracting mechanism, including that there was "improvement in the implementation capacities of the MDEFPNT, at both central and local levels." However, no specific information was provided. • The ICR reported that the key indicator on "the share of government contribution to the training fund's annual budget" achieved the target of 25%. However, the reported result includes project funding and therefore not core public budget. Achievement is rated Substantial due to passing of legislation updating the TVET regulatory framework, evidence of increased capacity to monitor the TVET sector, and increased engagement with the private sector. Rating Substantial PHREVDELTBL PHOVRLEFFRATTBL Rationale The three sub-objectives to improve quality, improve efficiency, and create an enabling environment for a more market-driven TVET system, are all rated Substantial. Therefore, the overall Efficacy rating is also Substantial. Overall Efficacy Rating Substantial PHREVISEDTBL 5. Efficiency Page 9 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MR - Skills Dev. Support Project (FY11) (P118974) At appraisal, the economic analysis (PAD, Annex 7) identified the expected benefits of the project as reduction in youth unemployment, improved TVET institutional performance that increases cost efficiency of training, and strengthened TVET sector management that optimizes resource allocation. Unit cost calculations for the annual cost of training per student was estimated to decrease from US$ 852 in 2010 to US$ 762 in 2016, due to increased enrollment in existing facilities, reduced drop out rate, and increased student:teacher ratio per classroom (using contract teachers). However, the discussion primarily focused on efficiency of the sector as a whole, rather than efficiency in the use of project resources. There was no net present value (NPV) estimate or other cost-benefit calculation. The ICR (Annex 4) provided unit cost calculations for rehabilitation of project schools and annual cost of training per student. The cost for rehabilitating the four TVET institutions was low (US$190 - 210 per square meter) compared to non-project schools in Mauritania (US$ 300 - 342 per square meter). The unit cost for short-term training decreased from US$ 671 in 2014 to US$ 653 in 2016/17. Cumulative enrollment numbers at the TVET project institutions also increased, indicating higher utilization of existing space, but still fell short of targets. There was no NPV or internal rate of return estimation at closing. There were also some initial implementation challenges which likely affected efficiency in the use of project resources. The TVET institutional arrangements significantly changed following government restructuring, such that main counterpart agency - the Ministry of State for National Education - was subsequently divided into two ministries (see ICR, page 25). There were procurement delays and deficiencies in civil works, as well as fiduciary challenges due to lengthy administrative procedures. Efficiency Rating Modest a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 0 Appraisal 0 Not Applicable 0 ICR Estimate 0 Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome Page 10 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MR - Skills Dev. Support Project (FY11) (P118974) Relevance of the project objectives is rated High. Achievement of each of the three objectives -- to improve quality and efficiency of the TVET system, and to create an enabling environment for a more market-driven TVET system -- is rated Substantial. Efficiency is rated Modest. These ratings indicate moderate shortcomings and therefore an Outcome rating of Moderately Satisfactory. a. Outcome Rating Moderately Satisfactory 7. Risk to Development Outcome The new TVET regulatory framework will likely sustain focus on a market-driven TVET system with strong involvement of the private sector. The Bank will continue support to these policy directions through the First Competition and Skills Development Policy Operation, currently under preparation. Institutional arrangements and mechanisms are likely to be sustained due to the capacity building activities of the project. The government intends to expand two models introduced by the project - short-term training and PBCs - although funding will largely come from development partners rather than from the government's own budget. Therefore, risk to development outcome is considered Modest. 8. Assessment of Bank Performance a. Quality-at-Entry This project represented the first Bank-funded operation in the TVET sector in Mauritania; therefore, preparation was marked by extensive stakeholder consultations and capacity building, as well as a well- considered project design. The project design built upon a comprehensive analysis of the sub-sector in the Bank's Education Country Status Report (2010) and various other studies highlighting strategies to develop a more skilled labor force. Political commitment to key institutional and legislative reforms was ensured through consultations. Donor coordination was addressed through a Memorandum of Understanding signed by all active development partners in 2007, ensuring that the project covered areas not being currently supported by other donors. The project design also drew upon lessons learned such as the effectiveness of PBCs in creating incentives to improve performance, the usefulness of a designated training entity (i.e. INAP-FTP) to administer funds for TVET training, and the positive impact of competitive selection processes on quality and accountability. Overall implementation risk was appropriately assessed as High due to the fragile country macro-context and new approaches proposed in the project; these were effectively mitigated through intensive stakeholder consensus building and project supervision. The M&E design was overall sound, with measurable and realistic indicators and targets, with the exception of overambitious targets for two indicators (drop-out rates and international certifications) and the lack of more robust indicators to measure improved efficiency. Page 11 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MR - Skills Dev. Support Project (FY11) (P118974) Quality-at-Entry Rating Satisfactory b. Quality of supervision The initial project period was marked by slow implementation progress and high turnover in task team leadership (the latter of which was noted by the client as a significant shortcoming in supervision). On this basis, Implementation Status Report (ISR) ratings for this period ("Satisfactory" for both Implementation Progress and Development Objective from September 2011 - January 2013) appear overly positive. However, supervision and project implementation improved significantly with the eventual placement of an Education Specialist and procurement specialist in the field office. Significant steps taken include the securing of AF to scale up the project's impact, setting up a Technical Committee to monitor project implementation given early challenges in M&E, and refining the results framework to better capture project achievements. As noted in the ICR, however, the sub-objectives of "quality" and "efficiency" were not monitored or reported separately in ISRs, and over-ambitious targets for two indicators were not revised. Quality of Supervision Rating Moderately Satisfactory Overall Bank Performance Rating Moderately Satisfactory 9. M&E Design, Implementation, & Utilization a. M&E Design The outcome and intermediate outcome indicators covered a range of measures for each of the project objectives. The sole key outcome indicator identified for efficiency (drop out rate) was not a strongly attributable result given that there were numerous factors outside the project that would likely drive drop out rates. Baseline and target figures were clearly identified. The M&E arrangements were well defined, including assigning primary responsibilities to the M&E Unit in the INAF-FTP (which had prior experience in monitoring and demonstrated adequate M&E capacity), collecting data from multiple sources (surveys and administrative data), developing capacity of implementing entities to monitor and report data, and developing an M&E manual. As noted previously, the indicator on the drop out rate may not have been appropriate (not attributable given multiple factors), and the target for international Page 12 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MR - Skills Dev. Support Project (FY11) (P118974) certifications appears to have been ambitious (international standards not applicable to the country- specific context). Also, the indicator on "resources generated from services provided by TVET institutions as part of recurrent budget" may not have been valid, as the increase over the project period was primarily due to the PBCs funded by the project. b. M&E Implementation According to the ICR (page 27), the M&E unit in the INAF-FTP was adequately staffed to collect data and prepare reports, although data was collected manually and therefore it was difficult to compile aggregate date and produce analytical reports on a more timely basis. Project implementation progress was tracked regularly, complemented by evaluative activities including tracer studies and assessments of specific activities (PBCs and short-term training). Indicators were refined at project restructurings, and targets were adjusted according to the expanded scope of the project and updated data from improved data collection methods. c. M&E Utilization The ICR (page 28) reported the following examples of the use of M&E information: during the AF/restructuring, indicators were amended and targets were revised to reflect the reliability and accuracy of data collection and to better reflect the scaling-up of activities financed under the AF; findings of the tracer studies by INAP-FTP and TVET institutions helped adjust/improve TVET institution curricula. M&E Quality Rating Substantial 10. Other Issues a. Safeguards The original project was designated an Environmental Category "C" project as it did not require civil works, nor did it trigger any safeguard policies. However, with the expected rehabilitation of TVET institutions under the AF, the project was revised to an Environmental Category "B" project, and safeguard policies on Environmental Assessment (OP/BP 4.01) and Involuntary Resettlement (OP/BP 4.11) were triggered. An Environmental Management Plan and Resettlement Action Plan were prepared. Safeguards compliance was rated satisfactory throughout the project period, and the ICR reported no major problems. However, the ICR provides no further details to verify what safeguards measures were implemented. Page 13 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MR - Skills Dev. Support Project (FY11) (P118974) b. Fiduciary Compliance Financial management: Financial management performance was rated in the satisfactory range throughout the project period. The ICR (page 29) reported that financial management staff had the proper qualifications, financial reporting was regular, internal controls were reasonable, external audits were timely and unqualified, and minor audit issues were resolved in a timely manner. Procurement: Procurement performance was rated moderately satisfactory due to persistent procurement delays in civil works and acquisition of imported equipment. Procurement staff had the proper qualifications, with regular review of procurement decisions by the Public Procurement Control Commission and the Regulatory Authority. The ICR (page 29) reported one procurement shortcoming, in which computers and information technology equipment (in the amount of US$ 160,000) were purchased in 2015 to be used by the MDEFPNT to support an education management information system that had not been implemented. The ICR (page 30) reports that the equipment was subsequently "deployed to other IT activities for pedagogical use." c. Unintended impacts (Positive or Negative) None reported. d. Other --- 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment Moderately Outcome Moderately Satisfactory --- Satisfactory Moderately Bank Performance Moderately Satisfactory --- Satisfactory Quality of M&E Substantial Substantial --- Quality of ICR Substantial --- Page 14 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MR - Skills Dev. Support Project (FY11) (P118974) 12. Lessons Lessons drawn by the ICR (Lessons, page 33): • A competitive funding mechanism can effectively promote demand-driven skills development and training. In the case of this project, the competitive selection process for providers of short-term training courses helped to fill current gaps in the labor market identified by small industries and TVET providers themselves. • The involvement of the private sector in the upstream identification of training needs can increase the relevance of training provided and thus insertion rates for graduates. In the case of this project, outreach efforts were directed to small industries to ensure their input into training programs and form partnerships to facilitate eventual placement of graduates. 13. Assessment Recommended? No 14. Comments on Quality of ICR The ICR quality was overall satisfactory, with important lessons drawn on what worked well in this specific country context. The quality of evidence was also satisfactory, with strong quantitative data drawn from the project's monitoring system and verified by targeted evaluative studies. The economic analysis did not appear to clearly distinguish between project efficiency and sector efficiency (the former of which is the basis for the Efficiency rating). The project indicator on "resources generated from services provided by TVET institutions as part of recurrent budget" does not provide convincing data as it includes project funds in its calculation of "resources generated." This indicator was reported in both the Efficacy and Efficiency sections as part of the evidence base. There was limited information reported on safeguards performance to verify satisfactory implementation of measures. a. Quality of ICR Rating Substantial Page 15 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MR - Skills Dev. Support Project (FY11) (P118974) Page 16 of 16