Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BIH Employment Support Program (P152347) Report Number: ICRR0023269 1. Project Data Project ID Project Name P152347 BIH Employment Support Program Country Practice Area(Lead) Bosnia and Herzegovina Social Protection & Jobs L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IBRD-86870 01-Oct-2021 55,699,539.61 Bank Approval Date Closing Date (Actual) 05-Jan-2017 01-Apr-2022 IBRD/IDA (USD) Grants (USD) Original Commitment 55,800,000.00 0.00 Revised Commitment 55,800,000.00 0.00 Actual 55,699,539.61 0.00 Prepared by Reviewed by ICR Review Coordinator Group Judyth L. Twigg Eduardo Fernandez Eduardo Fernandez IEGHC (Unit 2) Maldonado Maldonado 2. Project Objectives and Components DEVOBJ_TBL a. Objectives The project's objective, as stated in the Loan Agreement (p. 6), was "to increase formal private sector employment among targeted groups of registered job seekers." Although one outcome target (job placements for job seekers from disadvantaged groups) was revised downward at a September 2020 restructuring, a new comparably-sized group of beneficiaries was added (job Page 1 of 14 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BIH Employment Support Program (P152347) seekers who became economically vulnerable due to the COVID-19 pandemic), such that the scope of the project was not changed. A split rating methodology is therefore not warranted. b. Were the project objectives/key associated outcome targets revised during implementation? Yes Did the Board approve the revised objectives/key associated outcome targets? Yes Date of Board Approval 21-Sep-2020 c. Will a split evaluation be undertaken? No d. Components The project consisted of two components: 1. Support for Employment Promotion (International Bank for Reconstruction and Development, IBRD at appraisal: EUR 46.4 million; actual: EUR 48.12 million) was to help the government scale up and increase the effectiveness and efficiency of its active labor market programs (ALMPs) and job intermediation to help improve labor market outcomes. Funds allocated under this component were to be disbursed upon achievement of agreed results to be measured by disbursement-linked indicators (DLIs). It contained two subcomponents:  Employment Promotion through Improved ALMPs was to finance a package of financial incentives directed at job seekers and employers, accompanied by contractual obligations and co- responsibilities to ensure financing of medium-term productive jobs in the formal private sector. These measures were to include targeted wage subsidies, on-the-job training, and self-employment programs. The programs to be funded each year were to change based on learning and results from previous years. Target groups were to include youth, long-term unemployed, those with low skills, individuals above 40 years of age, and members of disadvantaged groups. All ALMPs were to be co-financed 50/50 from general entity government revenues.  Employment Promotion through Improved Job Intermediation was to finance improved employment services for job seekers and employers. Services for job seekers were to include improved methodologies for the intake/registration of newly unemployed persons, categorization of job seekers by risk of falling into long-term unemployment, and tailored counseling to clearly identify and focus resources on the job seekers most in need of support. Services for employers were to consist of development of new employer outreach methodologies, local skills needs surveys/interviews with employers, promotion of online services to increase the number of registered vacancies with the employment services, and facilitation of vacancy opening and filling by providing tailored human resource services to employers. These activities were to be financed through a combination of government funds and a small portion of the loan (specified in the Project Appraisal Document, PAD, Table 3). Page 2 of 14 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BIH Employment Support Program (P152347) 2. Support for Management Systems, Monitoring, and Communications (IBRD at appraisal: EUR 1.8 million; actual: EUR 0.51 million) was to support effective monitoring of the project, providing financing for strengthening management systems to monitor ALMPs and employment services as well as development and implementation of an effective communications strategy targeting job seekers, employers, policy makers, and the overall population. Through traditional investment lending mechanisms, it was to finance enhancements of information technology systems to allow data exchange and monitoring among relevant institutions; creation and implementation of a communications and outreach strategy; and process and impact evaluations of selected programs. As with the first component, these activities were to be financed through a combination of government funds and the loan. e. Comments on Project Cost, Financing, Borrower Contribution, and Dates The project was approved on January 5, 2017, became effective on January 26, 2018, underwent a mid- term review in December 2019, and closed on April 1, 2022, six months later than its original closing date of October 1, 2021. The total planned project cost was EUR 87.3 million (US$97.2 million). It was to be financed by an IBRD loan of EUR 50 million (US$55.8 million), of which EUR 21 million was allocated to the Republika Srpska (RS) and EUR 29 million to the Federation of Bosnia and Herzegovina (FBH). EUR 1.8 million of the Bank loan was initially unallocated, to be used for any unforeseen expenditures. EUR 1.33 million was undisbursed at project closure because of failure to achieve one DLI target in the RS. Actual Bank financing was US$55.7 million. Government funding, planned at EUR 37.3 million (EUR 16 million from RS, and EUR 21.3 million from FBH), was fully provided. The project was restructured twice. On September 21, 2020, the outcome/DLI target for increased job placement through ALMPs in RS was revised downward. Funds were allocated from the unallocated amount and across components to support benefits for 6,500 registered job seekers who became unemployed during the COVID-19 pandemic lockdown period (March-September 2020), and a new DLI was added to reflect support for those beneficiaries. On June 30, 2021, the closing date was extended to allow time for completion of activities following pandemic-related delays. 3. Relevance of Objectives Rationale Bosnia and Herzegovina (BiH) is a small, upper middle-income country with a complex governance structure comprising two autonomous and diverse entities: the Federation of Bosnia and Herzegovina (FBH) and the Republika Srpska (RS). High unemployment and inactivity rates challenged both entity governments' ability to reduce poverty. In 2015, 27.7 percent of the adult population (ages 15 and older) was unemployed, one of the highest unemployment rates in the region. An even larger share of the adult population -- 56.3 percent -- was inactive, neither working nor looking for work. Almost 44 percent of the unemployed had not been able to find a job for more than five years. Only 31.9 percent of the adult population was employed. According to the PAD (p. 1), only 12 percent of the employed were poor, compared to 22 percent of the unemployed. Moreover, a higher share of the bottom 40 percent income group who were employed had temporary jobs or no contracts, and they were much more likely to be in Page 3 of 14 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BIH Employment Support Program (P152347) occupations and sectors of activity that rely on unskilled labor. Overall, the poor had difficulty getting and keeping formal jobs. In 2015, the BiH Council of Ministers and the governments of the RS and FBH officially adopted a reform agenda that recognized the importance of addressing high unemployment. It set out a wide range of policy reforms related to employment issues: improving the enabling environment for existing and new businesses; delinking health benefits from unemployment registration to reduce the incentives for informal work; making secondary, tertiary, and adult education curricula more responsive to the needs of the private sector; restructuring or selling state-owned enterprises (SOEs) to increase productivity and private sector job creation in those sectors; and reforming social assistance programs to encourage work. As a key element of social assistance program reform, it was recognized that the institutional capacity of public employment services (PES) needed to be strengthened so that they could effectively intermediate labor demand and labor supply. Having strong PES and active labor market programs (ALMPs) was seen as critical for supporting the overall reform agenda, especially as they would serve as tools to mitigate potential adverse effects on employment of other pending reforms. Prior to the project, qualified employment counselors and administrative staff were stretched thin, and services were almost exclusively offered in person with little use of electronic platforms or automation. Twice as many people were registered as unemployed than the actual number of unemployed -- with many people registering primarily to obtain free benefits -- creating a significant administrative burden on the PES that hindered their focus on real job seekers. A 2014 analysis of the functionality of the PES in the FBH showed that staff of those agencies spent about 70 percent of their time on administrative tasks rather than on direct employment support. There was wide divergence in the capacity of local offices. Coverage and targeting of ALMPs was low and ineffective, and monitoring and evaluation (M&E) was almost nonexistent. The project responded to this country context and government policy by seeking to improve employment outcomes through increasing the effectiveness and scale of ALMPs and upgrading the job intermediation services provided by the PES. The project's objectives were also highly aligned with the Bank's Country Partnership Framework at appraisal (CPF, FY2016-FY2020), which contained focus areas on creating conditions for accelerated private sector growth and increasing public sector efficiency and effectiveness. Specifically, the project supported CPF Objective 1c, Improve Public Service Delivery, by supporting enhanced PES capacity to respond in a timely and adequate manner to redundancies resulting from SOE reform, and Objective 2b, Improve Labor Market Flexibility and Social Protection, by making the country's labor market more agile in hiring and supporting people who were hard to employ. The project remains relevant to the current CPF (FY2023-2027), particularly its second objective on increased inclusive market-relevant employability, under its first higher-level outcome, increased inclusive private sector employment. Rating Relevance TBL Rating High 4. Achievement of Objectives (Efficacy) EFFICACY_TBL Page 4 of 14 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BIH Employment Support Program (P152347) OBJECTIVE 1 Objective Increase formal private sector employment among targeted groups of registered job seekers Rationale The theory of change for this objective held that the project's interventions would reduce labor market frictions stemming from high search costs, supporting job seekers to transition into formal employment more quickly and firms to find hires more quickly. For ALMPs, expanded coverage, better targeting of vulnerable groups, and development of an online job portal would plausibly lead to better and faster matches between job seekers and employers, ultimately increasing the number of registered job seekers placed in jobs. Production of an annual report on monitoring of ALMPs, which would provide recommendations on how to improve program design, would lead to redesigned programs based on those recommendations, improving the quality of ALMPs and their capacity for job placement. For intermediation, developing of a profiling methodology and employer outreach methodology, training of counselors, conducting of client satisfaction surveys, and expanded vacancy canvassing would plausibly lead to increases in the number and quality of employer contracts, the number of collected vacancies, the number of job seekers in need of PES support with a risk profile, and the number of collected vacancies filled with registered job seekers, ultimately increasing the number of registered job seekers de-registering because they found a formal job. The results chains for both ALMPs and PES assumed that adequate and effective investments would be made in technical assistance and information technology. Achievement of the project's objectives was measured by three outcome indicators and five DLIs linked to the two results areas of increased formal private sector job placements through improved ALMPs and increased formal private sector job placements through improved intermediation. Outputs The Federation Employment Institute in the FBH issued 42 public calls, and the Employment Bureau of the RS issued four public calls, enabling participation of 12,971 firms in the real sector. A new rulebook of cantonal employment services was adopted in the FBH, along with a new rulebook for employment service staff in the RS, meeting the target (DLI 3.1). 30,635 job seekers in need of support in the FBH, and 27,000 in the RS, were profiled and had an individual action plan (IAP) reflected in the PES database, exceeding the respective targets of 30,000 and 20,000 job seekers (DLI 3.3). A job seeker self- assessment was introduced, meeting the target. The number of vacancies collected increased by 10.63 percent in the FBH and by 10 percent in the RS, meeting or exceeding the target of 10 percent (DLI 3.3). Successful matches increased by 21 percent from 2017 to project closure in the FBH, and by 15 percent in the RS, meeting or exceeding the target of 15 percent (DLI 3.5). IAPs were a key instrument in employment intermediation; at least twice annually, an evaluation of completed IAPs was conducted to facilitate understanding of job seeker needs and optimal means of support. A total of four annual reports were completed to monitor programs according to various subgroups (age, gender, long-term unemployed, branch office), providing recommendations for improving program design and targeting (DLI 4). Four programs were redesigned on the basis of these recommendations, meeting the target (DLI 2). The PES adopted new methodologies to analyze labor market data to better understand employer Page 5 of 14 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BIH Employment Support Program (P152347) needs and labor market demand, and then made corresponding changes to the ALMP menu in order to increase job placements. Entrepreneurship training was introduced, clearer criteria for target groups were defined to reduce overlaps, and experience sharing was facilitated between ALMP designers and job counselors. Wage subsidy amounts were increased for women, long-term unemployed, and low-skilled persons to additionally support employment of targeted beneficiary groups. Information technology systems were developed and enhanced for job counseling, profiling, on-line matching, and communication processes. PES and ALMP information management and job portal systems were upgraded, reducing administrative burdens, making data more available and consistent, and disaggregating beneficiary data by gender, educational background, residency, and other characteristics. PES systems are now connected with the Tax Administration system, enabling employment institutions to check employers' suitability for participation in public calls and apply any needed corrective measures. The satisfaction level of job seekers accessing PES support reached 90 percent in 2020, exceeding the target of 81 percent. No baseline was provided. Intermediate outcomes The cost per job created by the program 12 months after completion for youth and disadvantaged people decreased from EUR 3,000 to EUR 2,016.35, achieving the target of any decrease. Unemployment benefit payments were made during the COVID-19 emergency to 3,600 recently unemployed persons in the FBH and 2,900 recently unemployed persons in the RS, meeting the targets (DLI 5). Outcomes The number of registered job seekers in need of PES support de-registering from the unemployment registry because they found a formal job increased from 102,000 in 2016 to 552,387 in 2022, far exceeding the target of 110,543. These totals reflect an increase from 76,000 to 414,572 in the FBH, exceeding the target of 81,844, and an increase from 26,000 to 137,815 in the RS, exceeding the target of 28,699. The number of registered job seekers placed in private sector jobs through wage subsidy ALMPs increased from 5,000 in 2016 to 44,854 in 2022, exceeding the original target of 41,500 and the revised target of 37,575. These totals reflect achievement of 32,261 placements in the FBH (DLI 1, exceeding the target of 23,500) and 12,593 in the RS (DLI 1, not achieving the target of 14,075). Within these totals, sub-targets were met and exceeded in the FBH for placements of youth and members of disadvantaged groups, and sub- targets for these groups were nearly met in the RS. The ICR (p. 20) explained that 8,469 additional job seekers were placed in the RS in agricultural and public sector jobs and were not counted against the project's achieved result. 60 percent of ALMP beneficiaries remained employed as a result of participation in ALMPs 12 months after completion of the program, exceeding the target of 50 percent. An impact evaluation conducted in the RS found that there was an average 70 percent difference in employment probability in favor of the treatment group compared to the control group at the end of the program, and that the effect remained positive three to Page 6 of 14 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BIH Employment Support Program (P152347) twelve months later; an analogous impact evaluation in the FBH found 51 percent higher employment compared to the control group six months following program completion, and 52 percent higher employment at twelve months. Rating Substantial OVERALL EFF TBL OBJ_TBL OVERALL EFFICACY Rationale The project's interventions supporting ALMPs and intermediation services of the employment service agencies attracted more firms to participate in ALMPs, streamlined participation for disadvantaged job seekers who rely on PES to find a job, facilitated more efficient and effective matching, and ultimately achieved the objective of increasing formal private sector employment among targeted groups of registered job seekers. The only shortcoming in achieving targeted change was a minor shortfall in the number of job placements in the private sector in the RS. Overall Efficacy Rating Substantial 5. Efficiency The PAD's cost-benefit analysis (Annex 5), focusing on the ALMP component of the project, found a net return for each dollar spent of 40 percent to 103 percent, depending on the program, and taking into account potential leakages to new hires who would have found a job without financial incentive. The analysis considered cost and benefit streams over a time span of 20 years for young beneficiaries, and 15 years for beneficiaries older than 45 years. Programs varied in impact on labor outcomes and cost-effectiveness, which in turn depended on the target population, the parameters of the program, and program implementation mechanisms. These estimates were conservative, using only the ALMPs' direct costs (money transfers to employers and job seekers) and benefits measured as the present value of future income revenues over the working life of newly placed job seekers, or the increase in the present value of future income revenues due to wage increases resulting from higher productivity jobs (resulting from improved job matching). The ICR's ex post analysis (Annex 4) was based on a similar valid methodology and assumptions as those used at appraisal, updated to reflect actual results at the time of the project's impact evaluation (conducted prior to the COVID-19 emergency). It found an internal rate of return of 61 percent, within the range of appraisal estimates. Implementation efficiency benefited from project design that acknowledged the learning curve that would be associated with the first-time implementation of a hybrid investment lending project with a large results-based component. The DLI matrix deliberately set realistic targets in line with capacity constraints and the control span Page 7 of 14 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BIH Employment Support Program (P152347) of implementing institutions, and it distributed risk between DLIs that were more difficult to achieve and those that were achievable more quickly and with minimal time, effort, and financing. The mix of indicators/results, targets, and attached disbursements was meant to provide the two entity governments with steady cash flow, in particular for the financing of ALMPs. Moderate shortcomings included slow start-up and eventual inability to agree on an Additional Financing that was proposed in response to the COVID-19 pandemic due to "lengthy and difficult" decision processes arising from the unique, complex governance structure of the country (ICR, p. 28). Based on strong economic efficiency and efficient implementation, with some moderate shortcomings that did not meaningfully affect achievement of outcomes, efficiency is rated Substantial. Efficiency Rating Substantial a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 94.80 Appraisal  40.00  Not Applicable 94.80 ICR Estimate  61.00  Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome The objectives were highly relevant to country context, government strategy, and Bank strategy. The project substantially achieved its objective of increasing formal private sector employment among targeted groups of registered job seekers. Both economic and implementation efficiency were substantial, with moderate shortcomings. These ratings indicate that there were only minor shortcomings in the project's preparation, implementation, and achievement, leading to an Outcome rating of Satisfactory. a. Outcome Rating Satisfactory 7. Risk to Development Outcome The ICR (p. 25) observed that the results-based design of the project helped build a likely sustainable focus on outcomes rather than processes in the sector. Although the adverse impact of the pandemic and other Page 8 of 14 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BIH Employment Support Program (P152347) global crises might cause structural and short-term vulnerabilities for the country's macroeconomy, undermining labor market demand and supply, government policies set out for the 2023-2027 period continue to support increased inclusive private sector employment and provision of financial incentives to firms, mitigating the risk. The ICR also noted that the project's requirement to report indicators to the Bank in order to guarantee disbursement of the loan forced closer interactions between the Ministry of Finance, MoLSP, Federation Employment Institute, and the cantons in the FBH, "possibly laying the ground for future collaboration." In the RS, "it was evident that following the indicators became common practice from the central level to the employment agencies," facilitated by the comprehensive and user-friendly information systems developed under the project. The ICR (p. 26) also noted that sustainable capacity was built among PES staff through the process of using impact evaluation findings to improve ALMPs. The government has requested a follow-on project, approved by the Board on July 19, 2022. 8. Assessment of Bank Performance a. Quality-at-Entry As this was the first investment project financing with DLIs in the BiH social sector, the project received extensive technical support from the Bank task team at the design stage through a technical assistance (TA) project (Strengthening Employment Outcomes in Bosnia and Herzegovina ASA, P162489). This TA contributed to robust technical design of the project and high responsiveness to the government's reform strategy, which in turn built government ownership of the operation. Project design drew from key lessons learned from prior experience implementing similar programs in other countries (detailed in the PAD, Annex 6). It was recognized that wage subsidies, while a useful instrument to help employers and employees weather times of crisis, were also a means to build skills among specific groups of workers, especially youth and low-skilled and older de-skilled workers. Lessons from impact evaluations in developed countries showed that wage subsidies were worthwhile only when they provide job opportunities to workers who would otherwise remain unemployed or take jobs that do not maximize their potential productivity. In addition, international evidence showed that employment programs have higher impact when combined with effective intermediation services. Lessons were also taken from the prior Bank-financed project in the sector, the Social Safety Nets and Employment Support Project (P116774, US$15 million, 2010-2015), including the need to focus on four key program features -- delivery modality, level of financial compensation granted to participants, method and extent of targeting, and duration and sustainability of programs -- and the possibility of achieving results without overly generous financial incentives. Finally, the prior project illustrated that strong administrative systems to support implementation of multiple ALMPs, case management, and employer outreach are critical. Risk assessment was multidimensional. Political and governance risks were related to the possibility of weak commitment from the FBH and RS governments, as well as the ten cantonal governments within the FBH, to be mitigated through strong communications with the general public, private sector, civil society, and relevant international organizations (including European Union [EU]representatives) to maintain a broad-based platform of support for the reforms. Macroeconomic risks stemmed from the possibility of weak economic growth in the EU, with adverse effects on BiH's economic outlook through decreased exports, remittances, and capital flows. The Bank mitigated these risks through continued dialogue with entity governments and the International Monetary Fund. There were also risks related to the complexity of the project's implementation arrangements, especially in the FBH, where the PES is Page 9 of 14 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BIH Employment Support Program (P152347) highly decentralized. The entity governments were reluctant to borrow large sums for TA, creating risk that TA resources would be inadequate to support implementation capacity. This risk was mitigated through setting aside a small portion of the loan as unallocated, so that TA needs that arose during implementation could be covered, and through the specification of a detailed Implementation Support Plan (PAD, Annex 4). Overall, the ICR (p. 32) attributes the project's success to the Bank team's "enormous efforts" in leading policy dialogue with the government and coordinating project preparation among the two entity governments and other stakeholders. Quality-at-Entry Rating Highly Satisfactory b. Quality of supervision The Bank team provided intensive and proactive implementation support. Three supervision missions were conducted during each year of implementation to review progress, identify problems that were causing delays, provide technical advice, and offer solutions contributing to timely resolution of issues. The task team leader was located in Washington, DC, but a locally recruited consultant was based in the World Bank office in Sarajevo to ensure daily just-in-time technical and communication support. During the COVID-19 pandemic, the team switched to virtual missions and conducted regular on-line meetings with the project management unit (PMU) to ensure uninterrupted communication. The ICR credits the Bank team's efforts for the client's request for a follow-on project. Quality of Supervision Rating Highly Satisfactory Overall Bank Performance Rating Highly Satisfactory 9. M&E Design, Implementation, & Utilization a. M&E Design The project's objective and theory of change were straightforward and focused. Outcome indicators and DLIs were set out clearly in the PAD (Annex 1), with appropriate baselines, targets, data collection instruments, frequency of collection, responsibility for collection, and verification protocols for DLIs well specified. Indicators were drawn from government administrative data, monitoring data collected by the implementing units, and data from impact evaluations of selected ALMPs conducted annually. There were 20 intermediate results indicators for the FBH and 16 for the RS, a long list that could have been streamlined, given the complexity of working with two different government entities. Page 10 of 14 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BIH Employment Support Program (P152347) In FBH, the Ministry of Labor and Social Policy (MoLP) was in charge of project monitoring, verifying achievement of DLIs, and reporting to the Bank. A Steering Commitee, to include representatives from relevant ministries and the Prime Minister's office, was to receive biannual reports from the MoLP on progress toward achievement of DLIs and project indicators. In RS, the PES was responsible for implementation of ALMPs being financed by the project. The Ministry of Labor and Veterans Affairs (MoLVA) was to verify achievement of DLIs as reported by the PES, monitor the project, and report to the Bank. A Steering Committee, to include representatives from representative ministries and the Office of the President of the Government, was to receive quarterly or biannual progress reports from the MoLVA on progress toward achievement of DLIs and project indicators. b. M&E Implementation Both the MoLP (FBH) and MoLVA (RS) had a dedicated M&E specialist to oversee project monitoring. M&E data were collected as planned. ALMP implementation data were sent from the entity-level PES to the M&E specialists for analysis and production of annual and biannual reports to the Bank and the Steering Committees, as envisioned. In the RS, a special online platform was developed for employment services, with statistical data and reports filtered by criteria including gender, region, workplace, and occupation. The ICR (p. 29) noted that "the ministries' M&E capacity was significantly strengthened under this project." c. M&E Utilization The impact evaluations were effective monitoring and feedback tools, described by the ICR (pp. 29-30) as "an innovative way of using impact evaluations in a lending operation." Information systems developed under the project were critical in supporting the M&E function, enabling data to be used to inform meaningful course corrections. The performance monitoring reporting format developed during project implementation was incorporated into PES annual reporting and continues to be used. However, the ICR also noted that implementing agency capacity in verifying DLI achievement remained limited and requires further strengthening. M&E Quality Rating Substantial 10. Other Issues a. Safeguards The project was rated Environmental Assessment category C. None of the Bank's safeguard policies were triggered. All selected ALMPs were implemented within the confines of national guidelines on environmental safety and health standards, and necessary checks were part of the selection process of ALMP beneficiary firms (ICR, p. 30). Page 11 of 14 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BIH Employment Support Program (P152347) b. Fiduciary Compliance Overall fiduciary responsibility lay with the FBH MoLSP and RS MoLVA. Both ministries were to be reinforced by PMUs that had experience with Bank financial management and procurement processes through prior projects. Financial management performance was rated satisfactory overall, with adequate systems for accounting, budgeting, organization and staffing, internal controls, counterpart funding, and audit and financial reporting. Project financial statement audit reports, prepared for each entity annually, were received by the Bank on time and contained unqualified opinions. However, the RS PMU did not submit interim financial reports on time, and there was an undisbursed amount of EUR 1.33 million on the RS side at the time of the disbursement deadline (August 1, 2022). Procurement was also overall satisfactory, carried out in compliance with the Bank's procurement guidelines. However, the ICR (p. 31) noted that the RS PMU faced challenges in the use of Bank procurement systems, including filing of proper documentation, due to technical difficulties and weak capacity. The follow-on project contains activities to build capacity of the procurement specialist. c. Unintended impacts (Positive or Negative) None reported. d. Other --- 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment Outcome Satisfactory Satisfactory Bank Performance Highly Satisfactory Highly Satisfactory Quality of M&E Substantial Substantial Quality of ICR --- High 12. Lessons The ICR (pp. 33-35) offers a series of insightful lessons stemming from project experience, including the following: A comprehensive approach to labor market policies, addressing both supply- and demand- side factors, can allow for flexibility in times of crisis. One of the project's approaches to increasing formal private sector employment among targeted groups of registered job seekers was through wage subsidies, an intervention that is by nature flexible and responsive to labor market Page 12 of 14 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BIH Employment Support Program (P152347) needs and to employers' constraints (high labor costs). This proved to be particularly valuable during the COVID-19 emergency, when other types of interventions (such as training) might have increased labor supply productivity but not addressed labor market needs. Targeting specifically vulnerable populations and redesigning policies to meet their needs is important for achieving higher-level policy objectives. In this case, defining eligibility for specific vulnerable groups rather than allowing universal eligibility guaranteed access for populations who were less likely to have obtained employment on their own. Project impact evaluations also showed that these groups remained employed longer, and that satisfaction was increased among both job seekers and employers. Cultivating a culture of continuous monitoring and impact evaluation can encourage effective feedback loops, improving the design of interventions and achievement of outcomes. For example, in the RS, an annual impact evaluation showed that the employment program was most effective for job seekers between the ages of 25 and 30, people with lower qualifications, and the long-term unemployed, prompting the program to focus on and increase targets for those groups in 2021. 13. Assessment Recommended? No 14. Comments on Quality of ICR The ICR clear, concise, and well written. It conformed with ICR guidelines. It was consistently results focused, and its analysis was of high quality and based on a range of available evidence. Its explanations of implementation challenges in a complex governance environment were candid and thorough. Its lessons were insightful and should prove highly useful for design and implementation of future projects in the country and the sector. The ICR could have gone into more detail on the extent to which the project's interventions increased beneficiary populations' resilience to reform of state-owned enterprises and other elements of the government's 2015 reform agenda, but that is a minor shortcoming. Overall, this was a high-quality, best-practice document. a. Quality of ICR Rating High Page 13 of 14 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BIH Employment Support Program (P152347) Page 14 of 14