Report No: AUS0000198 Functional Reviews of the Public Employment Services in the Western Balkans Overview . July 4, 2018 . Social Protection and Jobs Global Practice Europe and Central Asia Region . . © 2018 The World Bank 1818 H Street NW, Washington DC 20433 Telephone: 202-473-1000; Internet: www.worldbank.org Some rights reserved This work is a product of the staff of The World Bank. The findings, interpretations, and conclusions expressed in this work do not necessarily reflect the views of the Executive Directors of The World Bank or the governments they represent. The World Bank does not guarantee the accuracy of the data included in this work. The boundaries, colors, denominations, and other information shown on any map in this work do not imply any judgment on the part of The World Bank concerning the legal status of any territory or the endorsement or acceptance of such boundaries. Rights and Permissions The material in this work is subject to copyright. Because The World Bank encourages dissemination of its knowledge, this work may be reproduced, in whole or in part, for noncommercial purposes as long as full attribution to this work is given. Attribution—Please cite the work as follows: “World Bank. 2018. Functional Reviews of the Public Employment Services in the Western Balkans - Overview. © World Bank.” All queries on rights and licenses, including subsidiary rights, should be addressed to World Bank Publications, The World Bank Group, 1818 H Street NW, Washington, DC 20433, USA; fax: 202-522-2625; e-mail: pubrights@worldbank.org. Co-financed by Austrian Ministry of Finance Available on Content Acknowledgements.................................................................................................................... 2 Introduction.............................................................................................................................. 3 Labor Market Challenges in the Western Balkans ........................................................................ 3 Public Employment Services..................................................................................................... 4 Functional Reviews of the PES in the Western Balkans ................................................................. 4 The DEA Model: The Basics ...................................................................................................... 5 Data requirements and use...................................................................................................... 6 Main Findings...........................................................................................................................11 Key findings ..........................................................................................................................14 Going forward ..........................................................................................................................19 References...............................................................................................................................21 Figures Figure 1: Employment to population ratio by education level and gender .......................................... 3 Figure 2: Unemployment rate by duration and age group ................................................................ 3 Figure 3: DEA of branch offices .................................................................................................... 9 Figure 4: Differences in resources and challenges: Montenegro offices. ............................................14 Figure 5: Changes in productivity in jobs outcomes given inputs (M3): FYR Macedonia .......................15 Figure 6: Pay-off in jobs placements from increasing output by one unit (M2): Serbia .........................16 Figure 7: Albania: Average scale efficiency by model ......................................................................17 Tables Table 1: Variables used in the country DEA models ........................................................................10 Table 2: Key findings: Summary Matrix ........................................................................................12 Boxes Box 1: Data Envelopment Analysis (DEA) ....................................................................................... 7 1 Acknowledgements This overview was prepared by Stefanie Brodmann and Sara Johansson de Silva, in collaboration with Maddalena Honorati, Josefina Posadas, and Gonzalo Reyes. It draws on a set of technical reports prepared by a team from the Institute for Advanced Studies (Vienna, Austria), led by Alexander Schnabl and including Helmut Hofer, Jan Kluge, Sarah Lappöhn and Hannes Zenz. Special thanks go to the staff of the Public Employment Services in Albania, Federation of Bosnia and Herzegovina, Republika Srpska, FYR Macedonia, Montenegro and Serbia for sharing detailed data on employment services and training program participation. The team is grateful for editorial support provided by Bardha Ajeti and Carey Jett. 2 Introduction Labor Market Challenges in the Western Balkans Western Balkan countries face significant labor market challenges . Just half of the population aged 15- 64 years are employed, and women of working age are more likely to be jobless than to work. The gap in employment levels between the Western Balkans and the European Union is substantial (Figure 1). Although country specific challenges vary, the Western Balkan labor markets are generally characterized by high inactivity rates, high unemployment rates, and high levels of informal work. Unemployment is large structural in nature, and 25 percent of youth have been unemployed for more than one year (Figure 2). Young people, females, those with low education levels and ethnic minorities are particularly excluded from access to productive employment in the formal sector. As of 2016, there were nearly 1.4 million unemployed in the region (World Bank and Vienna Institute for International Economic Studies, 2018). The low levels of productive employment carry high social, economic, fiscal and political costs. Figure 1: Employment to population ratio by Figure 2: Unemployment rate by duration and education level and gender age group 45 100 % of working age population (15-64) 88 40 17 90 81 80 76 73 % of the active population 73 35 65 67 70 62 58 60 30 60 53 9 50 44 25 41 42 < 12 months 37 40 20 26 > 12 months 30 15 5 20 25 Total Total Low Low Medium Medium High High 10 21 2 13 5 10 Male Female - WB 6 EU 28 WB 6 EU 28 15-24 25-29 25-54 55-64 Source: SEE Jobs Gateway and Eurostat. WB6 includes Albania, Bosnia and Herzegovina, FYR Macedonia, Kosovo, Montenegro, and Serbia. Fostering job creation (more generally), increasing skills, and expanding access to formal jobs, especially for socially vulnerable groups, is of critical importance for the Western Balkans. These goals are reflected in employment strategies across the region which center on facilitating business growth and job creation at the firm level (including growth-friendly policies and labor market regulatory reform, improving skills levels, broadening active labor market policies, and promoting access to jobs for more vulnerable groups; FREN, 2017). 3 Public Employment Services The objectives of public employment services (PES) are to assist adults in looking for jobs and improving their employability, thereby facilitating their transition to work. These agencies are typically responsible for registering the unemployed and paying out unemployment benefits, providing information about employment opportunities, supporting job search and placement services, and delivering active labor market programs (ALMPs) (ILO, 2015; Kuddo, 2009). The PES in Western Balkan countries operate under difficult conditions. The economic environment is challenging, with low levels of job creation in the formal sector, high levels of long-term unemployment, and a high share of young and unskilled persons among the unemployed. At the same time, firms in Western Balkan countries report difficulties in identifying qualified staff, pointing to significant skills mismatches (Koettl-Brodmann et al., 2017, Honorati and others, 2018). The PES also lack necessary resources to address challenges . Across the Western Balkans, spending on ALMPs amounts to just over 0.1 percent of GDP (about one sixth of the levels registered in Western Europe). The number of unemployed clients per PES staff member is high, ranging from over 200 in Montenegro to nearly 1000 in the Republika Sprska in Bosnia and Herzegovina. 1 Focusing on specialists/counselor staff only, the ratios are even higher – up to 2000 unemployed per counselor in some Serbian offices, and the situation is similar elsewhere. Caseloads in the Western Balkans are significantly higher than those of most EU countries where the staff-unemployed ration generally ranges between 50 and 200 (European Commission, 2016). Lack of resources and staff preclude tailored approaches for assisting clients, which is especially important for hard-to-serve clients (FREN, 2017). When recruiting new workers, Western Balkan firms are much more likely to employ family and friends and poach from other firms than to use public employment services, which may reflect limited confidence in the PES (Koettl- Brodmann and others, 2017, Honorati and others, 2018). In view of these challenges, empirical evidence on how to effectively monitor, evaluate and improve performance is of utmost importance. The PES in the Western Balkans need to acquire tools and methods to monitor labor markets and evaluate programs to improve the effectiveness of ALMPs, job search assistance, and other services. Building the capacity to establish regular performance management and identify strengths and weaknesses across the PES portfolio of actions and interventions are essential steps toward strengthening their effectiveness and efficiency. Functional Reviews of the PES in the Western Balkans As part of the World Bank’s analytical and technical support to help build the capacity of Public Employment Services in the region, a set of functional vertical reviews of the PES have been prepared for five countries : Albania, Bosnia and Herzegovina, FYR Macedonia, Montenegro, and Serbia. Functional vertical reviews entail a review of an agency's performance and its decision units in the underlying hierarchy – in this instance, the regional branch offices of each PES. In the case of Bosnia and Herzegovina, estimation results are presented only for the cantonal offices of the Federation of Bosnia and Herzegovina (FBG) as the information available for the Republika Sprska remains limited. The functional reviews are based on the methodology of Data Envelopment Analysis (DEA), best described as a benchmarking exercise comparing the inputs (resources), outputs (activities) and 1 Details on staff-client ratios and other basic monitoring data in Western Balkans are drawn from the data-base prepared for the technical reports by Schnabl et al. (2017 and 2018, various). 4 outcomes (job placements) across branches . The methodology helps identify areas in need of modification for each branch office as well as offices that can serve as peer model offices in exchanges around policy options and approaches. The DEA based functional reviews are but one piece in a broader performance management tool set . The method has several strengths, including its multi-variate analysis and its potential to analyze inputs and outputs without assigning a monetary value to each variable. Importantly, it also allows for detailed results at a regional branch office level. Although the functional reviews focus on the mandate to help the unemployed find jobs, they do not (i) distinguish between hard-to-serve and other clients (ii) review management or operational processes in detail (iii) consider the point of view of firms or job seekers, including their satisfaction with existing services. Each of these aspects is necessary for developing a broader and more effective performance management approach. This note summarizes key results from the functional reviews of Albania, Bosnia and Herzegovina, Former Yugoslav Republic of Macedonia, Montenegro, and Serbia . For each country, a detailed technical report and a synthesis of the main analytical results and policy implications are available. This note summarizes information on the method, main analytical results from each review, and key takeaways in terms of performance management processes. Methodology and data The DEA Model: The Basics The functional reviews of PES in the Western Balkans are based on Data Envelopment Analysis (DEA), a technique used in various settings to compare performance across different units, such as a firm's production units (e.g., different airplane hubs ((Box 1)). The DEA method identifies a best practice “frontier”, in this case among the different branch offices of PES in each country. Other branch offices can then compare their office with the “frontier” to identify sources of inefficiency and modify accordingly their own organization and processes. For inefficient offices, the model can identify a “best practice” office that uses a similar mix of inputs and outputs but with better results and that can serve as a peer office for comparison and inspiration. The analysis focuses on the relationships among inputs, outputs, and outcomes, measured on an annual basis: • Inputs are the resources used by offices, such as number of staff and expenditures. • Outputs are the products of these resources (i.e., activities that the offices undertake, such as training and profiling). • Outcomes describe the final result, in this case, number of registered unemployed transitioning into formal jobs (irrespective of how long they had been registered with the office). This performance evaluation includes the following models: • Efficiency of Activity (Model M1) measures the relationship between office inputs and outputs. Some offices are particularly good at utilizing financial and staff resources to provide more activities and services (e.g., training, counseling, and referrals) for the unemployed. • Impact (Model M2), assesses how various outputs relate to the outcome. 5 • Effectiveness (Model M3) looks at how outcomes are related to the inputs. This can be seen as the “reduced form” of the two models above, namely how the financial resources and staffing of different offices affect their ability to secure employment for the unemployed. The DEA provides benchmarking for a given level of registered unemployed . The degree to which the unemployed use these services (i.e., the ratio of registered unemployed to total unemployed according to labor force survey data) and the degree to which firms are using PES for recruitment are not considered. Environmental factors are not captured within the model. Labor market conditions and challenges likely vary significantly within countries, both on the supply and demand side, for example the overall skills levels and ages of the unemployed, the number and type of job offers available, and dominant economic sectors. The outcomes of the benchmarking exercise should therefore be interpreted in the context of the environment in which the regional office operates. In three countries (Albania, Montenegro, and Serbia), the DEA models have been complemented with econometric analyses of how different environmental variables impact the effectiveness of the model at the branch level. For FYR Macedonia and Bosnia and Herzegovina, econometric analyses were not undertaken due to data constraints. The DEA provides benchmarking within e ach country’s PES system only. A branch office deemed fully efficient in one country might be the best among that country’s PES branch offices, but not in another country. The results are therefore not comparable across countries. Ongoing work to review performance and best practice across countries in the Western Balkans, under the auspices of the Regional Cooperation Council, is an important complement to the DEA reviews. 2 The DEA model is described in more detail in Box 1. The analysis focuses on changes in performance over time, on the efficiency, impact and effectiveness of branch offices, and on the efficiency of scale of these branch offices (i.e., whether their size, measured in inputs such as staff and expenditures or outputs such as activities, is optimal to achieve the highest level of productivity). Data requirements and use The DEA model requires basic monitoring indicators only. As will be shown, the DEA model can provide important insights using basic data that the PES should already routinely collect : jobs placement, expenditures, staff numbers, and outputs that the office produces. These data are not always available, however. In what follows, the model varies somewhat by country in terms of the indicators used, largely depending on the availability and quality of data. The final outcome is defined as placement of the unemployed in formal jobs. As seen in Table 1, the outcome variable is in all but one case the same across countries: the number of unemployed that were registered with the relevant PES but transitioned into a job in the formal sector in that year. The one exception is the Republika Srpska, for which only information on unemployed transitioning into subsidized jobs is available. This indicator is of lower relevance from a performance evaluation perspective and could even be considered an employment promotion activity rather than an outcome. Input and output indicators vary across countries, however, depending on data availability and the specific approach of the PES, as well as the number of offices available for analysis (fewer variables can be assessed for countries with a smaller number of offices). Input indicators in this analysis include (i) expenditures on active measures, as well as on maintenance costs, operational costs, or goods and 2 https://www.rcc.int/priority_areas/27/esap-employment-and-social-affairs-platform 6 services and (ii) staff numbers, including counselors that specialize in assisting the unemployed and other staff. For Serbia, the number of participants in active labor market programs is budgeted for at a central level beyond the control of local branch offices, and is thus considered an input. Outputs include activities with the unemployed (profiling, meetings without profiling) and activities related to identifying and promoting links between the unemployed and employers, registering vacancies, referring the unemployed to job interviews, and contacting employers directly. For all countries except Serbia, outputs also include the number of persons enrolled in active measures on education and employment. Overall, these are basic indicators of resource use and performance that should be available in monitoring systems. Outputs that are closely linked to local labor demand are more context driven than others and as such only partly under the control of the office . For example, if there are no job vacancies in the area, then there cannot be any registration of vacancies. There must be sufficient employers to contact and job opportunities to which the unemployed can be referred. These activities are hence more dependent on local jobs conditions than those that center on the unemployed themselves, such as enrollment in active measures, profiling, and meetings with counselors and clients. The functional reviews present additional multivariate analysis, outside of the DEA model, to illustrate the links between environmental factors and effectiveness levels . Potential control variables include those that capture challenges related to clientele and case load, socio-economic conditions and local labor demand. The first category includes total number of unemployed (whether registered or not) in the area, and the share of youth, females, or persons with low-level skills among the registered unemployed. Socio- economic conditions are captured by average salary, average household budget, social welfare beneficiaries, and similar indicators. Local labor demand includes measures of geographic size, population density, and the number of formal jobs in the area. Table 1 does not present the control variables for the econometric analysis of the influence of factors outside the purview of the offices. These vary depending on data availability, as they draw on a mix of administrative and other sources of data such as firm registries and household surveys. Box 1: Data Envelopment Analysis (DEA) First developed for nonprofit organizations, DEA is a nonparametric frontier estimation technique that is used to benchmark performance. The “frontier” that is estimated consists of the most efficient organizations within a group, which are used as a standard against which the performance of other organizations is evaluated. The DEA hence assesses whether a specific unit is efficient relative to others and identifies an “ideal” unit that the specific decision- making unit can imitate to increase efficiency. In Box Figure 1a, Offices P1 –P4 are fully efficient – on the so-called “technological frontier” – whereas Offices P5– P7 could achieve more with fewer resources. As such, the DEA focuses mostly on relative, not absolute, efficiency – the “best” unit can still be inefficient in absolute terms, for example compared to offices in other countries. The Malmquist productivity index is used to detect changes in the productivity of these processes over time, which in turn determines whether the DEA exercise should be undertaken under the assumption of productivity development or not. The productivity index can be decomposed into a “catch -up” effect and a “frontier shift” effect. The former relates to convergence of poorer-performing units to better ones. The latter refers to technological improvement that helps a unit produce more with the same input. In Box Figure 1b, such a shift in the frontier is illustrated. “Technical development” describes not only progress in a technical sense but also changes in processes, applied methodologies, and the political environment that potentially affect outcomes. In the Western Balkans Functional Reviews, the Malmquist index indicated technological progress and the results reported are based on a model which assumes that the “technology” is different each year. The “best practice” office is hence defined for that year only, rather than for the entire period. 7 Compared to parametric methods for data analysis, such as Ordinary Least Square regressions, the DEA method requires fewer data, allows for combining multiple inputs and outputs into a single summary measure of efficiency without a priori weights, and allows for comparisons of performance when prices of inputs or outputs are not known. Box Figure 1: DEA analysis and the technological frontier a. Basics of the DEA model b. Frontier shift between t and t+1 Output (M1) Outcomes (M2, M3) 10 Constant Returns to 9 Scale Variable 8 P4 Returns to Scale 7 P3 6 5 P2 P7 4 3 P6 P5 2 P1 1 0 0 1 2 3 4 5 6 7 8 9 10 Input (M1, M3) Output (M2) Scale efficiency measures whether efficiency is affected by the size (or scale of the operation) of the office. Each model is calculated for the case of constant returns to scale (CRS: double inputs lead to double outputs, in the efficient cases) and variable returns to scale (VRS: diminishing or increasing returns to scale may occur), as shown in Box Figure 1a. The use of both models allows for a comparison of scale efficiency. If the ratio of the efficiency value of the CRS model to the VRS model is lower than 100 percent, the unit is not of optimal size. The CRS model imposes stricter conditions and so efficiency values tend to be lower than those in the VRS model. Because scale efficiency on average was relatively high, suggesting that the sources of inefficiencies is not the scale of operation per se but how processes and programs are managed, most of the information presented in this report is based on the CRS model. 8 Figure 3: DEA of branch offices Table 9 Table 1: Variables used in the country DEA models 1. INPUTS 2. OUTPUTS 3. OUTCOMES Expenditures Staff ALMPs ALMPs Other Albania Active measures Total staff Total Contacts with Jobs 2013-2016 Maintenance Participants* employers Placements** Vacancies (all) registered Bosnia and Total staff Total Contacts with Jobs Herzegovina (FBH) Participants* employers Placements** 2016 Referrals (all) Bosnia and Active measures Total staff Jobs Herzegovina (RS) Maintenance placements** 2016 (subsidized) FYR Macedonia Active measures Counselors Total Contacts with Jobs 2012-2016 Operational Other staff Participants* employers Placements** Vacancies (all) registered Montenegro Active measures Total staff Total Contacts with Jobs 2014-2016 Maintenance Participants* employers Placements** Vacancies (all) registered Serbia Goods and Counselors Total Contacts with Jobs 2012-2016 services Other staff Participants* employers Placements** Vacancies (all) registered Referrals Contacts with unemployed Profiles created Source: Technical reports by Schnabl et al. (2017 and 2018), various. *Participants in active measures on education and employment. ALMP Participants is counted as an input in FBH and Serbia, because the number of participants is mandated by the central office and not under the control of local offices. **Registered unemployed transitioning into jobs. 10 Main Findings This section provides an overview of the key messages emanating from the DEA analyses undertaken of the PES in, respectively, Albania, the Federation of Bosnia and Herzegovina, FYR Macedonia, Montenegro and Serbia. As explained, the definition of best practice is specific to each country and hence, the different national PES cannot be ranked against one another. Moreover, the inputs and outputs differ across countries, depending on the specific approach and mandate of the PES as well as data availability, and there are likely differences in how expenditures are recorded, in budget procedures, and in how different activities are defined. The overview therefore concentrates on identifying patterns and trends that are common, or different, across countries . Results for Republika Srpska are not presented because the policy relevant outcome variable, total jobs placements, is not yet available. 3 The findings center on jobs outcomes . Results presented generally relate to the M3 model (inputs lead to outcomes), and is outcome oriented rather than input oriented. Thus, the analysis is geared towards identifying ways of producing more jobs placements for the PES clients with a given level of resources, rather than on how to preserve current jobs outcomes with fewer resources. Table 2 summarizes the main findings for each country in relation to six questions: • Taken together, have the PES offices become more effective in producing outcomes? Why? • How effective are branch offices on average, in other words, distance to national best practice – the productivity frontier? • What gains, if any, could be made from closing the effectiveness gaps, in other words, if all offices became as effective as the best one(s)? • What measures would be most effective in increasing jobs placements? • Is the scale of operation efficient or could geographical reorganization improve outcomes? • To what extent are local labor market and socio-economic conditions or the characteristics of the clientele associated with differences in office performance? 3 Results from the DEA model for RS using the subsidized jobs placements are available in the country reports. 11 Table 2: Key findings: Summary Matrix Question Albania Federation of Bosnia FYR Macedonia Montenegro Serbia 2013-2016 and Herzegovina 2012-2016 2014-2016 2012-2016 2016 Taken together, have the Yes, productivity Not available (data for Yes, productivity Yes, productivity Yes, productivity PES offices become more increased, mostly due one year only). increased, both due to increased, entirely due increased, mostly due effective in producing to catch-up by poor catch-up and frontier to catch-up. to a frontier shift, outcomes? Why? performing offices. shifts. Productivity in the best while poorer performing offices performing offices actually fell. remained behind. How effective are branch Average effectiveness Average effectiveness Average effectiveness Average effectiveness Average effectiveness offices on average, i.e., was low, at 41 percent is moderately high, at is moderate, at 54 is low, at 25 percent. is high, at 78 percent, distance to national best in 2016. Most offices’ 67 percent. However, percent in 2016. This reflects distance suggesting that practice? performance remains four offices out of ten from most offices to performance is similar far below that of the are less than half as the best offices, Tivat across branch offices. best-performing effective as the most and three other offices. effective offices. (coastal) offices. What gains, if any, could High gains: Jobs Moderate gains: Jobs High gains: Jobs Job gains would Moderate gains: Jobs be made from closing placement could placements could placements could increase tremendously, placements could the effectiveness gaps, increase by 120 increase by 25 percent. increase by 73 percent by over 300 percent. increase by 20 percent. i.e., if all offices became percent with full Limited potential gains reflecting a large share This result is driven by The limited gains as effective as the best? effectiveness. Large reflect small, of low performing the stronger reflect the overall high potential gains reflect differences among offices. performance of offices levels of effectiveness large existing offices in effectiveness in coastal regions than (small distance to best differences between (moderate average in others and is not practice) in Serbia. best and worst offices. effectiveness). likely to be achievable in practice. 12 Table 2, cont. What measures would A mix of actions is The analysis could not Reducing case loads Reducing case loads, Reducing case loads be most effective in likely to be most be undertaken due to (increasing the number contacting employers, and contacting increasing jobs effective: increasing data constraints. of counselors) and and increasing employers pays off placements? staff, increasing increasing participation participation in ALMPs most. Contacting contacts with in ALMPs have the have the largest effect employers is employers and strongest impact. on jobs placements. significantly more registering vacancies effective than any have the highest pay- other PES activity. off. Is the scale of operation The scale of operation The scale of operation The scale of operation The scale of operation The average scale efficient or could is comparatively is not very efficient. is quite efficient with is comparatively efficiency is high and geographical efficient, although a the exception of a few efficient although a no office has very low reorganization improve few offices are scale offices. few offices appear too levels of scale outcomes? inefficient. large to be efficient. efficiency. To what extent are local Context variables, Econometric analysis is Analysis not possible Context variables such A larger share of hard- labor demand, socio- especially differences not possible (too few due to data as the characteristics to-serve clients (e.g., economic conditions and in labor demand, are offices), but a constraints. of the unemployed and older, female or long the characteristics of the strongly linked to comparison of the labor market term unemployed), as clientele associated with office performance. characteristics of the conditions are not well as poorer local differences in office unemployed and labor associated with office labor demand performance? market conditions performance. conditions are linked to suggest they are not lower office related to office effectiveness. performance. 13 Key findings In all countries, there are significant variations in the offices' local socio-economic context and labor market challenges : the size of the formal sector in the area they serve and the potential demand for labor in those firms and institutions, the number and characteristics of the registered unemployed, and the resources at their disposal. There are also pervasive differences in starting points, which must be taken into account when comparing outcomes across offices, when identifying peer offices , and when devising changes to office activities or organization. The impact of these different context variables on office effectiveness is mixed, however. When tested formally in multivariate analysis few context variables emerge as linked to the effectiveness levels. In Albania and Serbia, for example, offices for which indicators point to low labor demand and a high share of hard-to-serve clientele are less effective. In Montenegro, however, only population density, a potential indicator of labor demand and economic size, is consistently related to office performance. This is surprising because labor market conditions are considerably more favorable in coastal regions (Podgorica, the capital, is an exception), reflecting the influence of tourism on labor markets. The number of unemployed per counselor (caseload) is ten times higher in some offices than in others. There are also tenfold differences in the number of formal jobs in relation to the number of unemployed (Figure 4). Figure 4: Differences in resources and challenges: Montenegro offices. 12 Bud 10 More -----> less difficult labor market conditions Kot 8 (more jobs/unemployed) Pod Other 6 Tiv Her Coastal 4 Bar Zab Ulc Cet Dan Nik 2 Plj Plu Bij Kol Sav Moj Roz Pla Ber Andr 0 600 500 400 300 200 100 0 More -----> less resource constrained (lower case load) Source: Administrative data, database prepared for Schnabl et al. (2018c). Overall productivity has increased steadily . At the national level, the different PES have become better at placing the unemployed in formal jobs, using a given level of resources and activities. These improvements were driven by different forces, however. In Albania and Montenegro, poor performing 14 offices approached the better performing offices. In Serbia, by contrast, higher productivity resulted almost exclusively from a shift in the frontier – the better performing offices became even more effective, but poor performing offices did not manage to reduce the distance between themselves and best practice offices. In FYR Macedonia, productivity increased because of both: the best offices increased in effectiveness, and the worst performing offices approached the frontier to some extent (Figure 5). Figure 5: Changes in productivity in jobs outcomes given inputs (M3): FYR Macedonia 180% 160% 140% 120% 100% 80% 60% 40% 20% 0% 2012-2013 2013-2014 2014-2015 2015-2016 2012-2016 Total Catch-up Frontier shift Source: Prepared based on Schnabl et al. (2018b). Productivity is given by the Malmquist index (see Box 1). A productivity level about 100% indicates improvement. Overall increases in productivity mask substantial variations in effectiveness across branch offices. In several countries, a large share of offices struggle with low efficiency and effectiveness in terms of fulfilling their mandate, implying that they are far behind the most productive offices. In FYR Macedonia, Albania and Montenegro, many offices are below 40 percent in effectiveness, and average effectiveness levels are therefore low, at 54, 47 and 25 percent respectively. In FBH and Serbia, average effectiveness levels are higher, at 67 and 78 percent. Reducing the effectiveness gaps between offices could result in substantial gains in terms of jobs placements . Large differences in effectiveness also imply large potential but unexplored gains. If less productive offices could emulate the outcomes of the best offices, (i.e., move to the “frontier” ) total jobs placements could increase significantly. The highest potential gains are in countries with a significant spread in effectiveness among offices. Closing these gaps would mean that job placements could increase by 120 percent in Albania and over 300 percent in Montenegro, where the least effective offices fall far below the best performing offices. Gains are consequently much smaller in Serbia and Bosnia-Herzegovina, where office performance is more compressed. 4 4 An important caveat to these results is that job markets need not be local for offices, firms or job seekers. High jobs placements in one office may be the result of tapping into firms and job opportunities in the area where another office operates. If this is the case, total jobs gains would likely be significantly smaller even with fully effective offices. 15 Case load (input) and demand side activities (output) appear to be most effective in increasing jobs placements although there are variations across countries. On the input side, reducing case-load by increasing staff, especially specialists, consistently pays off. On the output side, interventions to approach the demand side (contacts with employers, registering vacancies) appear to be most effective. In Serbia, the pay-off to increasing contacts with employers is very high, much higher than for any other action (Figure 6). Of course, a high number of contacts with employers or vacancies to register may reflect relatively healthy local labor demand conditions. These findings are only a first step towards identifying useful methods. The relative merit of different activities cannot be compared without converting them to the same scale and units so that, for example, the cost of adding a counselor or enrolling a person in ALMPs in local currency is compared to an increase in expenditures in local currency. Figure 6: Pay-off in jobs placements from increasing output by one unit (M2): Serbia 2.50 Additional job created from increase in 2.13 2.00 1.50 one unit 1.00 0.50 0.29 0.27 0.06 0.11 0.00 Contact Register Meet with client Create profiles Make referral employers vacancies Source: Prepared based on Schnabl et al. (2017b). Based on the M2 model. So what can the PES and the separate branch offices do to improve effectiveness? Geographical reorganization of the offices is not a promising solution – improving practices within offices is a better way forward . Scale efficiency measures whether offices are operating at the right scale or instead economies or diseconomies of scale are holding back office effectiveness. Average scale efficiency is high in most Western Balkan countries. In Albania, for example, average scale efficiency is above 80 percent for the effectiveness model. A few offices have low scale efficiency in most countries. In the case of Albania, the worst performing office has a scale efficiency below 30 percent. However, the economic and political costs associated with merging or splitting offices are unlikely to pay off given that most offices are already operating at a relatively efficient scale. Hence, each office, and the PES as a whole, need to identify ways to increase effectiveness within branch offices. 16 Figure 7: Albania: Average scale efficiency by model Scale efficiency 90 80 70 60 50 40 30 20 10 0 M1 M2 M3 Mean score Share over 80% Least productive office (score) Source: Schnabl et al. 2017a. For branch offices, this would entail evaluation of office performance, together with best practice peer-to-peer learning . Using the DEA as a starting point, branch offices could begin by disentangling where in the process of converting resources into job placements that difficulties emerge. Are there inefficiencies in how resources (staff, expenditures) are used to produce various activities aimed at improving job outcomes? Or are these activities simply ineffective in helping the unemployed find jobs so that offices should consider changing the mix of activities? Tailored approaches are likely to work best. Offices choose different mixes of inputs and outputs, depending on the level of resources, local context, organizational approach, mandate, capacity, etc. However, for each office in the analysis, the DEA models identify peer offices – offices that use a similar approach to the office in question in terms of resources and activities but achieve better results in terms of jobs placements . In addition to evaluating the office’s own performance, it would be useful to examine in detail the processes and approaches of these peer offices and exchange experiences. The local context for job creation is beyond the influence of any PES but also matters for the choice of outputs. For example, in areas where the job creation capacity of the local private sector is low, offices may need to tap into regional or national labor markets and provide information about opportunities elsewhere. Approaches to training and the degree of mobility of job seekers also depends on their characteristics in terms of age, family formation, gender and education. At a national level, the PES needs to help foster peer learning, and work towards establishing more and better performance management practices across its units . At the aggregate level, PES have a critical role in establishing a platform that can help branch offices exchange ideas. The core PES functions are also central in identifying tools and approaches to facilitate and improve performance management, in communicating these to branch offices, and in ensuring that data collection on basic monitoring indicators is detailed, accurate and comparable. Data constraints limit the analysis and are a constraint to all forms of monitoring and evaluation efforts at the branch level as well as at the aggregate level of each public employment agency . Very basic indicators such as expenditures and job placements are in some cases not available. Improving 17 data collection and monitoring is crucial to improve performance evaluation and, ultimately, to help the public employment agencies fulfil their mandate of helping the unemployed transition into jobs. Technical solutions exist that facilitate harmonization, recording and sharing of such basic data. 18 Going forward Improving performance management of public employment services is part of a broader reform package necessary to help the PES in Western Balkan to fulfill their objective of transitioning the unemployed into productive job opportunities. This note has summarized key findings from a benchmarking exercise within the decentralized structures of PES in five countries in the region. The analysis has shown that the suboptimal performance of some branch offices is costly and that there could be significant gains, in terms of more job placements for the unemployed, if poor performing offices could bring their performance up to that of the best performing offices in each country. Regular functional reviews using the DEA method can assist the branch offices and headquarters of PES in increasing their efficiency (i.e., approaching the best practice offices): by providing information on (i) weaknesses in processes and programs in the way that inputs are converted into outputs or in how those outputs result in job placements, and (ii) whether performance compared to other offices has changed over time (iii) what accounts for these changes, in terms of changes in offices' use of resources and mix of programs and activities. The DEA model also provides the foundation for additional analysis of the impact of changes in local economic conditions, in new, externally imposed strategies, or budget cuts, for example. This information can be used to identify where action is needed to improve performance. Establishing a functional review as a recurring exercise (annual or bi-annual) could be a step in the right direction. Peer-to-peer learning within each institution could increase effectiveness significantly. For each office, the benchmarking exercise indicates which other offices use a similar approach (in terms of mix of input/outputs) but have better results. Communication with best practice offices would be a good start in terms of performance review and the identification of these clusters of offices is a significant strength of the DEA. In addition, offices could team up with offices that face similar challenges (e.g., a high share of low skilled unemployed or low labor demand). The DEA based functional reviews should be part of a broader set of performance management practices . The DEA model also has limitations for monitoring and evaluation. In the DEA model, the “top performing” offices often vary significantly over time, suggesting that the definition of best practice is not very stable. Familiarity with quantitative research methods is needed both to conduct the analyses and to interpret its results. Like many quantitative methods, the DEA does not provide any information on or evaluation of differences in process or management. Hence, complementary exercises include, inter alia, benchmarking with other countries in the region and in the European Union, undertaking enterprise surveys, process and organizational (“management”) reviews, labor market monitoring, and evaluation of existing active measures. At present, consistent measures of expenditures and programs across these countries do not exist. The European Union’s network of Public Employment Services has established both peer managerial reviews and a basic set of common performance indicators against which the performance of each country’s PES is evaluated. Countries in the Western Balkans could choose to adapt these indicators to their specific context, and work towards a common monitoring framework. Strengthening data collection, monitoring and consistency across countries are of critical importance to improving performance management and, over the long run, labor market outcomes. The DEA analysis uses only basic indicators of expenditures, staff, type of activities, and jobs outcomes that should form part of the core data available for each branch office. However, an important finding from this multi-country exercise is that, in many instances, even this basic information is lacking. Lack of internal processes, lack of capacity and lack of ICT solutions that could support data collection preclude performance monitoring of the PES and of employment strategies more broadly. Again, the EU has developed a set of basic common indica tors that can be used “at -a-glance” to review the performance of PES. These include evaluations of the number of unemployed who find jobs and how 19 quickly they find jobs (subsidized and unsubsidized), the share of low skilled and youth among those that find jobs, the share of filled vacancies, labor force survey information on the use of PES in job search, and job seeker satisfaction with PES services. 20 References European Union, Directorate-General for Employment, Social Affairs and Inclusion, 2016, Assessment Report on PES Capacity 2016, available online at: http://ec.europa.eu/social/BlobServlet?docId=16967&langId=en Foundation for the Advancement of Economics (FREN), 2017, Regional analysis of employment and social measures in the Western Balkans 6. Report commissioned by Regional Cooperation Council. Honorati, M., S. Johansson de Silva, and O. Kupets, 2018, Western Balkans - Demand for skills in Albania: an analysis of the skills towards employment and productivity survey (English). Washington, DC: World Bank Group. International Labour Organization (ILO), 2015, Employment Services in the evolving world of work. Johansson de Silva, S. and O. Kupets, 2018, Skills for more and better jobs in Serbia, unpublished mimeo, Washington, DC: World Bank. Koettl-Brodmann, J. S. Johansson De Silva, O. Kupets, and B. Naceva, 2017, Looking for skills in the former Yugoslav Republic Macedonia. Washington, D.C: World Bank. Kuddo, A., 2009, “Employment Services and Active Labor Market Programs in Eastern European and Central Asian Countries.” Washington DC: World Bank. Schnabl, A., H. Hofer, S. Lappöhn, A. Pohl, and H. Zenz (2017a): “Albania Functional Review: Efficiency Analysis of the National Employment Service – 2013–2016, Technical Report.” Background paper. Washington, DC: World Bank. ________________(2017b): “Serbia Functional Review: Efficiency Analysis of the National Employment Service – Update 2012-2016. Technical Report.” Background paper. Washington, DC: World Bank. ________________(2018a): “Bosnia and Herzegovina Functional Review: Efficiency Analysis of the Employment Agency of Bosnia and Herzegovina – 2016. Technical Report.” Background paper. Washington, DC: World Bank. ________________(2018b): “FYR Macedonia Functional Review: Efficiency Analysis of the Employment Agency of the Republic of Macedonia – 2012-2016. Technical Report.” Background paper. Washington, DC: World Bank. ________________(2018c): “Montenegro Functional Review: Efficiency Analysis of the Montenegro Employment Agency- 2016. Technical Report.” Background paper. Washington, DC: World Bank. World Bank and the Vienna Institute for International Economic Studies (wiiw), 2018, Western Balkans Labor Market Trends 2018. 21 Annex Annex Table 1: Basic data for Public Employment Offices Staff Registered unemployed Formal jobs in area Total Per office Per staff Total Per office (thsds) (thsds) member Per unemployed Albania 285 8 94 3 329 10.5 Bosnia and Herzegovina FBH 601 60 442 44 619 1.2 RS 130 22 232 39 969 1.8 FYR Macedonia 221 7 102 3 463 n.a. Montenegro 222 11 49 2 223 3.6 Serbia 1608 50 701 22 436 2.7 Source: Technical reports by Schnabl et al. (2017 and 2018), various. 22