NATIONAL EMPLOYMENT SERVICE Assessment and Recommendations for an Action Plan 2022 NATIONAL EMPLOYMENT SERVICE NATIONAL EMPLOYMENT SERVICE ACKNOWLEDGEMENT CONTENTS This Document was prepared by a World Bank team comprised by Marijana Jašarević (Social Protection Specialist), Gonzalo Reyes (Sr. Social Protection Specialist), Zlatan Šabić (Sr. Operations Officer), Abbreviations 4 Lara Lebedinski (Consultant) and Irena Kalmić (Consultant). Comments were provided by Jamele Rigolini EXECUTIVE SUMMARY 5 (Program Leader), Cem Mete (Practice Manager) and Mauro Testaverde (Sr. Economist). The team would INTRODUCTION 11 like to thank the National Employment Service and the Ministry of Labour, Employment, Veterans and Social Affairs of the Republic of Serbia for the information and support provided in the preparation of this document. 1. BRIEF REVIEW OF NATIONAL EMPLOYMENT SERVICE ACTIVITIES Comments were also provided by UNICEF.   AND MONITORING 12 1.1. Registration of unemployed people 13 1.2. Passive labour market programmes: unemployment benefits 17 1.3. Delivery of active labour market programmes 19 1.4. Budget and personnel 20 2. REVIEW OF ACTIVE LABOUR MARKET PROGRAMMES 23 2.1. Coverage of active labour market programmes 23 2.2. Characteristics of beneficiaries by type of ALMP 24 2.3. Geographical distribution of ALMPs 27 2.4. Summary of key findings 27 3. THE NES DURING THE COVID-19 PANDEMIC 28 4. IMPROVED USE OF LABOUR MARKET INFORMATION 32 4.1. The National Employment Service Management Information System (NES MIS) 32 4.2. Assessment of NES MIS 33 4.3. Recommendations to improve use of labour market information captured by NES 36 4.3.1. Improvements of the current NES MIS 37 4.3.2. Introducing a Labour Market Observatory 41 5. PROCUREMENT OF ALMPS 45 5.1. Public procurement procedure applicable to the classroom-based vocational trainings based on the old PPL 46 5.2. Obstacles in procurement of classroom-based vocational trainings 49 5.3. Improvements with the new PPL 51 5.4. International examples of public procurement 54 5.5. Summary 59 6. SKILLS DEVELOPMENT TRAINING IN THE CONTEXT OF LIFELONG LEARNING 60 7. RECOMMENDATIONS 62 7.1. Impact matrix 65 Annex 1. Statistical information 67 Annex 2. Main reports published by NES 71 Annex 3. Conceptual definitions of labour market information systems 74 4 NATIONAL EMPLOYMENT SERVICE NATIONAL EMPLOYMENT SERVICE 5 ABBREVIATIONS EXECUTIVE SUMMARY ALMP Active Labour Market Programme CROSO Central Registry of Compulsory Social Insurance OVERVIEW OF THE NES EFQM European Foundation for Quality Management EMIS Education Management Information System The National Employment Service (NES) is a public agency under the supervision of the Ministry of EU European Union Labour, Employment, Veteran and Social Affairs (MoLEVSA) in Serbia. The NES in Serbia has two GDP gross domestic product main responsibilities: (i) it administers unemployment benefits and (ii) it supports the unemployed HZZ Hrvatski zavod za zaposljavanje – Croatian Public Employment Service through job-counselling and provision of Active Labour Market Programmes (ALMPs). ILO International Labour Organization To register as unemployed, a person is required to visit the NES branch office where she/he is resident. LMI labour market information Beneficiaries of unemployment benefits are required to confirm their unemployment status every LMIS Labour Market Information System month by appearing at the NES, while other unemployed people need to confirm their status every MIS market information system three months. After employability assessment and profiling are conducted, the NES case worker and MoESTD Ministry of Education, Science and Technological Development the unemployed person develop an individual employment plan that considers both the professional MoLEVSA Ministry of Labour, Employment, Veteran and Social Affairs background and professional interests of the unemployed person and labour market demand for spe- NEAP National Employment Action Plan cific occupations, based on which the unemployed person can be referred to a job interview or to one NES National Employment Service or more ALMPs. OECD Organisation for Economic Co-operation and Development PES Public Employment Service Unemployment benefits can be claimed by individuals who were insured for the case of unemploy- PPL Public Procurement Law ment based on an employment contract, self-employment or contract for seasonal or temporary work. RSD Serbian dinars Workers are entitled to unemployment benefits for a period ranging from 3 to 24 months. The amount SME small and medium enterprise of unemployment benefits is the product of the daily cash benefit and the number of calendar days in SORS Statistical Office of the Republic of Serbia each month for which the right is exercised. The daily cash benefit allowance depends on the income in the 12 months preceding unemployment. Based on their individual employment plans, unemployed people are referred to one or more ALMPs. The NES offers a wide variety of ALMPs for unemployed people. Using the standard economic nomen- clature, these can be broadly classified as: (i) job search assistance programmes; (ii) skills development trainings (classroom and/or on-the-job); (iii) subsidized private sector employment programmes; and (iv) subsidized public sector employment programmes. The largest share of the NES budget from 2017 to 2019 was spent on unemployment benefits and other benefit claims, although this share has fallen during the observed period by 8 percentage points. During the same period, the share spent on ALMPs has increased from 15 per cent to 21 per cent. The share of GDP spent on ALMPs was 0.08 per cent between 2017 and 2019, which is extremely low compared to other countries. In 2019, only 12 per cent of newly unemployed people claimed unem- ployment benefits, primarily due to very restrictive requirements. The most commonly implemented ALMPs are job search assistance programmes, with approximately 1,100,000 beneficiaries in 2019, a significantly higher number than in 2017. This increase reflects the efforts of NES to strengthen the advisory role of case workers and intensify contact with clients. The number of participants in skills development training programmes, which stood at 10,489 in 2019, has been falling from 2017 to 2019. The number of private sector incentive programmes had moderate deviations of around 10,000 in the observed period. Finally, public sector employment programmes, which stood at 6,365 in 2019, have been decreasing over the observed period. 6 NATIONAL EMPLOYMENT SERVICE NATIONAL EMPLOYMENT SERVICE 7 While there are only few rigorous studies shedding light on the effectiveness of the ALMPs implement- to provide the first labour market experience to approximately 8,000 young people. The programme is ed by NES, international evidence suggests that job search assistance programmes have the lowest cost implemented by NES, and the first trainees entered the programme in December 2020. but also a small impact in the long term. Skills development trainings are most effective in the long term, but, due to the ‘lock-in effect’,1 the positive effect on employment is not reflected in the short term. Private sector incentive programmes and public sector employment programmes are known to IMPROVED USE OF LABOUR MARKET INFORMATION be successful in the short term, but their long-term impacts are, at best, slightly positive. The NES monitors gross employment rates by ALMP for each calendar year but does not systemat- The NES Management Information System (NES MIS) is a modern, operational system that supports ically use empirical models to evaluate ALMPs.2 In the past, MoLEVSA would commission external key NES business processes. It contains several modules, including: Activities with employers; Job in- consultants to evaluate certain subtypes of skills development trainings provided by NES. Thus, there termediation; Work ability assessment; Professional orientation; Active labour market programmes; are few rigorous studies relying on a credible comparison group that shed light on the effectiveness of Unemployment benefits; and Analytical reporting. It also has an online portal, which offers job-match- NES ALMPs. While these studies provide useful information on the impact of specific programmes, ing services to jobseekers and companies, with a customized matching functionality. there is no systematic evaluation of ALMPs or a methodology for measuring the net placement rates NES produces various reports that are used for planning and decision-making processes.4 The Report that result from interventions. Therefore, the provision of ALMPs, and the resources devoted to them, on the Implementation of Annual Work Plan presents information on the implementation of each ac- do not seem to be informed by evidence of their effectiveness. Net placement rates (correcting for job tivity foreseen in the Annual Work Plan that NES performed each year. The Monthly Statistical Bulle- matchings that would have resulted even in the absence of the intervention) would provide an accurate tin of NES provides the numbers and structure of unemployed and employed persons and the numbers metric to assess the effectiveness of interventions and a basis for policy decisions. of registered job vacancies. An employer survey is conducted annually to identify mismatches between In summary: (i) there is limited information on the effectiveness of ALMPs; (ii) there is a relatively demand and supply of labour in terms of occupations and skills. low take-up of ALMPs and, in particular, skills development trainings; (iii) there is some evidence of The NES captures and uses a vast amount of information on registered unemployed people, alongside ineffective distribution of services; and (iv) there is no evidence of budget reallocation towards or away vacancies and programmes and interventions that are offered to its clients. This wealth of information from specific programmes according to effectiveness, based on the data collected. is probably underutilized in the process of improving labour market interventions and in the analysis of labour market conditions. A broader view of the data and information that the NES manages, en- riched with the information from external sources, can improve (i) the NES monitoring and evaluation THE NES DURING THE COVID-19 PANDEMIC capabilities and (ii) labour market efficiency and performance of additional stakeholders beyond NES. In addition to further improvements of the NES MIS operational features (including more integrated From the date when the state of emergency in Serbia was declared in March 2020, the NES moved support for management and procurement of training and more online services), NES could consider some of its services online. As of the date of writing,3 the NES still allows use of email to submit dif- strengthening the MIS support to key monitoring and evaluation functions (dynamic reporting and ferent types of requests as it did during the state of emergency; however, all services are also offered generation of indicators; impact and process evaluation of ALMPs; generation of indicators related to in person again. There were no procurements in 2020 due to the COVID-19 pandemic. However, the GRM processes; easy access to analytics through dashboards; support for online surveys; etc.). NES has proposed some adjustments with respect to the manner of performance of certain trainings (for example, allowing online teaching if possible) and presented them to MoLEVSA. MoLEVSA did Broader use of the labour market information collected by NES would provide different types of deci- not approve of delivering trainings online. sion makers, such as workers, jobseekers, firms and policymakers, access to better information. While the NES uses its information primarily for job-matching purposes, other stakeholders might find this The labour market indicators do not suggest an overall negative impact of the COVID-19 pandemic information useful for other purposes, such as career guidance, educational or human resources de- on the labour market. However, there is no doubt that the COVID-19 pandemic has a negative impact cisions, and labour market analysis. This need can be fulfilled by the introduction of a Labour Market on the labour market outcomes of some subgroups. Ideally, in times of economic downturn, spending Observatory (LMO) that could produce systematic information on labour market conditions to help on ALMPs should be increased. In addition to standard activities, increased spending could finance relevant stakeholders make informed choices leading to better alignment between skills demand and specific programmes that target those who lost their jobs with the objective to protect them from long- supply. The LMO would host the labour market information (LMI) repository (created not only from term labour market exclusion. In addition, specific policies would need to be designed for young peo- the NES information but also by combining the information from external databases), provide labour ple to minimize the negative effects of the current crisis. As a response to the current economic situa- market analyses for a wide audience, support policy-making and planning, and support a wide spec- tion, the Government of Serbia designed a traineeship programme called ‘My First Salary’, which aims trum of cross-sectoral monitoring and evaluation processes. 1 The lock-in effect refers to the situation where individuals decrease or cease completely their job search efforts while attending an 4 The National Employment Strategy for 2011–2020 provides the framework, strategic direction, and priorities for employment policy. intensive training. This causes a drop in their initial employment probability after the training ends. The National Employment Action Plan (NEAP), adopted by MoLEVSA annually, sets out a plan to implement goals from the Strategy 2 As part of the GIZ project Youth Employment Programme, a data cleaning methodology and matching technique has been developed to and defines annual goals of employment policy with corresponding measures and activities for the year. The Annual Work Plan is analyse NES ALMPs. This methodology has been applied to evaluate the impact of ALMPs financed by the project. adopted at the beginning of each year by the NES. It operationalizes the NEAP. The NEAP is adopted in December for the following 3 25 March 2021. year, after which the Annual Work Plan is adopted. 8 NATIONAL EMPLOYMENT SERVICE NATIONAL EMPLOYMENT SERVICE 9 PROCUREMENT OF ALMPS nent of a broader lifelong learning system that, if conceptualized and implemented well, could enable all citizens to seamlessly engage with formal and non-formal education and training throughout their As a public agency, the NES is obliged to implement the Public Procurement Law (PPL) when procur- lifetimes. ing goods and services on the market. The PPL, adopted in 2012 and amended twice in 2015, was, until Creating a lifelong learning environment in Serbia where a person can re-enter education and training recently, the applicable Law. However, a new PPL, adopted in 2019, came into effect from 1 July 2020. whenever the need to update skills arises will be important to minimize skills mismatches. In addition, There is only one ALMP of NES, called the Labour Market Trainings programme, a classroom-based with the support of a well-functioning LMIS and simplified procurement processes, such a frame- vocational training, delivered by external service providers. The NES has identified two major obsta- work would complement the skills development efforts of the existing ALMPs strategy. In practice, this cles in conducting public procurement for the Labour Market Trainings programme: (i) a lengthy and would mean that more relevant curricula (informed by the private sector) would produce graduates complex procedure, which must be repeated every year; and (ii) limitations in selection of the highest whose skills are better aligned with the needs of the labour market. It would further mean that employ- quality service provider. The period between identifying labour market needs and execution of the ers, employees and education and training providers would eventually shift focus to skills as outcomes, trainings after selection of service providers can take up to six months. In addition, the NES repeats the rather than relying on broad qualifications as a proxy for competence. procurement procedure every year because it is prevented from using Framework Agreements by its annual strategic documents and by restrictions stemming from budget regulations. It also experiences RECOMMENDATIONS difficulties in choosing the best-quality service provider, particularly with defining the most appropri- ate elements for assessment of training quality in line with market needs. The level of resources devoted to ALMPs seems low for international standards and for the needs The new PPL brings certain improvements to the public procurement procedure, which may facilitate arising from a crisis and recovery period due to the COVID-19 pandemic. Increasing resources would procurement of Labour Market Trainings for NES. This includes: (i) introduction of higher thresholds allow more participants to benefit from labour market interventions. However, more information is for implementation of public procurement rules for social and other special services (which include needed to assess which interventions are most effective, which requires improved monitoring and eval- educational and professional development services); (ii) a special regime for award of contracts for uation capacity and better use of existing information captured by NES. Flexibility to adapt interven- social and other specific services, which includes a simplified procedure and shorter deadlines; (iii) tions based on evidence and evaluations as well as to procure external providers for training courses introduction of the possibility to use a simplified type of negotiated procedure without publishing the would improve the effectiveness and relevance of labour market programmes offered by NES. invitation for submitting tenders, for repetition of similar services; (iv) a simplified method of proving the fulfilment of the criteria for qualitative selection of service providers; and (v) implementation of The COVID-19 crisis also provides an opportunity to adapt services offered by NES, making further principles for determining the most economically advantageous tender from the European Union (EU) use of online channels and adapting processes like registration, benefit claiming or follow-up of indi- Directive (on the basis of the best price–quality ratio, assessed on the basis of various sub-criteria in- vidual employment plans more user friendly. A list of the main recommendations in the report is as cluding qualitative, environmental and/or social aspects). NES will apply the new PPL for the first time follows: in the second quarter of 2021 to hire external service providers of trainings. 1) At 0.08 per cent, the share of GDP spent on active labour market programmes is low by internation- The practices of national employment services in the EU countries (specifically in Austria and Cro- al standards and should be raised. atia) show that the length and complexity of the public procurement procedure could be overcome 2) The number of clients in skills development trainings should be increased gradually, as this type of by a more flexible procurement planning phase and implementation of simplified procurement pro- ALMP has the best employment effects in the long term. One strategy for achieving this could be by cedures and mechanisms. This might include: (i) direct award of contracts for contract values below investing in public–private partnerships (PPPs). legally determined thresholds; (ii) contracts award in negotiated procedures without competition for 3) In parallel to increasing the supply of skills development trainings, NES should improve its analyti- repeated services; and (iii) using Framework Agreements for procurement of trainings on a multi-year cal capacity. It should conduct regular independent impact evaluations of its programmes. It should level. When it comes to the definition and application of quality award criteria for selection of the best also use evaluations and results from employer surveys to make decisions to adjust, create or dis- service provider, EU practice shows that most focus is dedicated to assessment of quality of training continue ALMPs. programmes and training staff. 4) NES should explore the possibility to apply statistical profiling to identify individuals at risk to be long-term unemployed. IMPLICATIONS FOR LIFELONG LEARNING 5) The employer survey is and should remain a central tool to identify the skills needs and skills gaps. This instrument should be regularly reviewed to align it with international best practices. Its repre- The National Employment Action Plan (NEAP) promotes the advancement of skills development, par- sentativeness at the district level and sector level should not be compromised. ticularly through ALMPs such as job search assistance, subsidized employment, and a range of skills development trainings. However, these efforts are mainly targeted at helping the unemployed find 6) The COVID-19 crisis has shown that the NES can move some of its services online and this direc- work. While a strong focus on getting the unemployed into jobs is necessary, this is only one compo- tion should be further explored as a long-term strategy. 10 NATIONAL EMPLOYMENT SERVICE NATIONAL EMPLOYMENT SERVICE 11 7) Steps should be taken to improve the NES MIS: a) Improve NES MIS support for monitoring and evaluation; INTRODUCTION b) Improve NES MIS to support procurement and provision of training. This report provides an overview of the tasks and responsibilities of the National Employment Service 8) Introduce a Labour Market Observatory (LMO): (NES) in Serbia with a focus on: (i) the monitoring and evaluation system of active labour market pro- a) Create an LMI repository that feeds from NES databases and from external databases through grammes (ALMPs); (ii) the existing Labour Market Information System (LMIS); (iii) the procurement cooperative agreements with other institutions; process for ALMPs; and (iv) NES functioning during the COVID-19 pandemic. b) Provide a wide audience with labour market analytics to support policy-making, planning and In recent years, administrative data of high quality have been generated and there is significant poten- cross-sectoral monitoring and evaluation. tial to use these data to improve the quality of NES services, including via the development of a com- 9) The analytical capacity of the NES Department of Labour Market Analysis and Planning should be prehensive LMIS. This report explores how to upgrade the current LMIS and build a comprehensive enhanced to use new NES MIS and LMO capabilities. The Department should be given the func- monitoring and evaluation system. The procurement process for externally delivered skills develop- tion and responsibility to adapt ALMPs to local conditions, design interventions, carry out pilot ment trainings is lengthy and inefficient in meeting the demands of a dynamic labour market, so this programmes, and discontinue programmes that the evidence shows to be ineffective. report explores how to improve this process. Finally, the COVID-19 pandemic has negatively affected delivery of public services as in-person contact with clients is reduced, so we assess NES’s functioning 10) The NES should be able to start with procurement processes early in the year, to avoid potential and delivery of services during the lockdown. fluctuations of market needs and capacities which may happen due to the lengthy procurement procedure. Also, it would be ideal to enable the NES to commence procurement of specific, urgent The first section of the report provides an overview of the core processes of the NES (registration of classroom-based vocational trainings before the final adoption of the Public Procurement Plan and unemployed and delivery of ALMPs) and its available budget. The second section reviews the ALMPs verification of the Annual Work Plan. provided by NES, while the third explores NES operations during the state of emergency caused by the COVID-19 pandemic. Section 4 discusses characteristics of the current LMIS and how to broaden 11) The NES should analyse all the possibilities for applying a simplified procedure or a mechanism for the scope of NES by relying on more comprehensive data and analysis. Section 5 describes the pro- easier procurement of planned classroom-based vocational trainings that are available under the curement procedure for externally delivered skills development trainings and obstacles that the NES new PPL. is facing in implementation of this procedure. It also provides an overview of crucial improvements 12) The NES should consider various options to strengthen implementation of quality criteria for as- in the public procurement procedure introduced with the new Public Procurement Law and explores sessment of the most economically advantageous tender, based on the provisions of the new PPL international practices in procurement from Austria and Croatia. Section 6 reflects briefly on the im- and EU practice. plications of the above for lifelong learning in Serbia. Section 7 concludes the report with a summary of key recommendations. 12 NATIONAL EMPLOYMENT SERVICE NATIONAL EMPLOYMENT SERVICE 13 1 BRIEF REVIEW OF NATIONAL 1.1. REGISTRATION OF UNEMPLOYED PEOPLE EMPLOYMENT SERVICE ACTIVITIES Registration of the unemployed is conducted in person in the respective NES branch either in the place of permanent residence or place of temporary residence. A person can register as unemployed in the AND MONITORING corresponding branch of temporary residence if she/he remained unemployed in this location. The un- employed person registers with an administrative officer by filling in the unemployment form (Figure 2). At the registration, she/he shows: (i) identity card or other valid public document with a photograph and personal data; (ii) proof of education or professional qualifications (original document for inspection); The NES is a public agency under the supervision of the Ministry of Labour, Employment, Veteran (iii) act on termination of employment, if the person was previously employed; and (iv) other evidence and Social Affairs (MoLEVSA). The NES in Serbia has two main responsibilities: (i) it administers in accordance with the law (certificates, confirmations, statements). When registering as unemployed, unemployment benefits and (ii) it supports the unemployed through job-counselling and provision of a person receives an appointment for a first individual meeting with her/his case worker. At least once ALMPs. The responsibilities of NES are regulated mainly in the Law on Employment and Unemploy- in three months, or if called by the NES, the unemployed person appears at the NES to get information ment Insurance.5 about the possibilities and conditions for employment and employment mediation, based on the indi- Given its responsibilities, the activities and performance of NES are strongly linked to the character- vidual employment plan. If an unemployed person does not visit the NES by the required date and time, istics and dynamics of the labour market. In Serbia, the labour market has evolved favourably since she/he is removed from the register and cannot register as unemployed for the next six months.6 2010. Activity rates increased from 49.4 per cent in 2011 to 54.6 per cent in 2019, while employment rates grew from 37.9 per cent to 49 per cent in the same period and the unemployment rate declined Figure 2. Registration and first meeting with case worker from 24.2 per cent in 2012 to a record low 10.4 per cent in 2019. This has translated into lower levels of activity for NES during this period, with fewer registered unemployed and fewer beneficiaries of labour STEP 2: MEETING WITH CASE WORKER market programmes. However, long term unemployment remains relatively high (almost 60 per cent STEP 1: REGISTRATION of total unemployment) and there are persistent differences in labour market outcomes for women 1. Employability assessment and profiling 1. Register with administrative officer (activity and employment rates are 15 percentage points lower than those for men) and younger pop- 2. Development of individual employment plan 2. Receive appointment for meeting with case worker ulations (unemployment rate of 27.9 per cent in 2019), making them a natural target for labour market 3. Assignment to ALMP (optional) interventions provided by NES. Contrary to expectations, the key labour market indicators for 2020 do not demonstrate a worsening The first individual meeting between the case worker and the unemployed person takes place within of the labour market situation as a result of the COVID-19 pandemic. In comparison to 2019, the em- 90 days of registration. At the first meeting, the unemployed person is asked whether she/he needs ployment rate in 2020 has marginally increased by 0.1 percentage point while the unemployment rate support from NES in finding employment. Should she/he reply affirmatively, the case worker conducts has dropped by 1.4 percentage points to 9.0 per cent in 2020. The activity rate in 2020 dropped by 0.6 an employability assessment. Based on the qualification level, work experience, and expectations of the percentage points to 54.0 per cent. unemployed person, as well as the current labour market situation, the case worker profiles the unem- ployed person to assess her/his likelihood of finding employment. There are three different profiling Figure 1. Key labour market indicators in Serbia, 2010–2019 categories: (i) high/easy employability; (ii) employable with the provision of intensive services support 60 – subsidized employment; and (iii) employable with the provision of intensive services support – com- prehensive support for reintegration into the labour market. Most individuals fall into the second cate- 50 gory.7 The criteria to categorize the unemployed currently used by NES are as follows: 40 1) Easily employable category in the open labour market with the provision of basic services in medi- 30 ation and active job search: 20 > Unemployed in this category: (i) can immediately join the mediation process; (ii) have relevant qual- 10 ifications, realistic expectations of labour market conditions, as well as the required level of activity, 0 and the job offer for occupations in which they will be mediated is very good or good. 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 Activity rate Employment rate Unemployment rate 6 In the case of removal from the register, an unemployed person can write a complaint explaining the reasons why she/he did not appear Source: Statistical office of the Republic of Serbia, Labour Force Survey, Annual Indicators. at the specified date and time. If this happens more than once in a year, she/he requires written proof to confirm the claim in the complaint. 5 NES has 32 branch offices all over Serbia and roughly 1,800 employees. The number of case workers working directly with clients is 711. 7 For 2019, the distribution of categories was the following: 19 per cent high, 79 per cent fair, and 2 per cent poor. 14 NATIONAL EMPLOYMENT SERVICE NATIONAL EMPLOYMENT SERVICE 15 2) Employable with the provision of intensive services support – subsidized employment category: those who belong to more than one hard-to-employ category are prioritized for inclusion in ALMPs. If a beneficiary of unemployment benefits finds employment, she/he stops receiving the benefits. > Unemployed assessed to have less chances for employment due to irrelevance of qualifications, expectations or motivation, as well as limited or weak job offers in their respective field, or because Different countries use different methodologies to profile the unemployed, and these can be catego- the person cannot get employment unless included in programmes such as subsidized employment rized into three broad types: rule-based profiling, case worker-based profiling and statistical profiling.9 (allocation of financial resources for their employment in any form). Rule-based profiling refers to a profiling mechanism that assigns the unemployed into different groups based on certain criteria such as age, education level and unemployment duration. Case worker-based 3) Employable with the provision of intensive services support – comprehensive support for reintegra- profiling means that case workers classify the unemployed into categories based on an interview and tion into the labour market category: additional quantitative and qualitative information. Statistical profiling uses a statistical model to pre- > Unemployed assessed as hard to employ; their employment is not possible even by engaging in sub- dict labour market disadvantage, and these models frequently aim to identify the individuals who are sidized employment. For instance, the person cannot be referred to the employer or included in the most likely to become long-term unemployed. Compared to case-worker-based profiling, the main ALMP due to health issues. Persons with disabilities who do not have an estimated working capacity advantage of statistical profiling is that it is less time consuming and less expensive. Statistical profiling are also hard to employ. makes better use of the information collected by the Employment Agency and has the advantage over > Unemployed who based on employment case worker assessment, or their personal statement/as- rule-based profiling because it considers the full set of characteristics of each individual and does not sessment, has health problems preventing them from performing their work. treat them as members of larger groups. Individuals can be classified into one or more hard-to-employ categories. Hard-to-employ categories The profiling methodology used by NES in Serbia can be classified into case-worker-based profiling. are defined each year in the National Employment Action Plan (NEAP). In 2020 the following cate- Some countries use two or even all three types of profiling, and NES should explore the use of statistical gories were classified as hard-to-employ: (i) persons with no qualifications or low qualifications; (ii) profiling either exclusively or as an addition to the case-worker profiling. Large administrative datasets persons aged 50 years or older; (iii) long-term unemployed (12 months or more and especially those (individual employment and unemployment spell data, firm-level data, etc.) have become available over unemployed longer than 18 months); (iv) women and especially long-term unemployed women; (v) the last years and they offer high quality information which can improve upon the profiling process. persons younger than 30 years old and in particular women, persons without qualifications or with low Applying statistical profiling could help save some time of the case workers and, in addition, it would qualifications; (vi) disabled people; (vii) Roma; (viii) beneficiaries of Financial Social Assistance (FSA); make it possible to adjust the individual employment plan more frequently than every six months. and (ix) redundant employees. The individual meeting lasts between 20 and 60 minutes. An unemployed person can be removed from the register of the unemployed if the person: After the employability assessment and categorization is conducted, the NES case worker and the i) does not show up at NES within the deadlines stipulated by the law; unemployed person agree on an individual employment plan that considers both the profession- ii) does not submit proof of active job search at the request of NES; al background and professional interests of the unemployed person and labour market demand for the specific occupations. Based on the employability assessment and individual employment iii) rejects an offer of mediation for adequate employment; plan,8 the unemployed person can be referred to a job interview or to one or more ALMPs. If some iv) does not contact the employer to whom she/he is referred by NES in case of NES mediation for types of ALMPs are highly demanded (e.g., self-employment subsidies, employment subsidies for appropriate employment; the private sector, etc.) funds from other less demanded programmes can be transferred to highly demanded programmes. If additional funds allocated are not enough to meet the high demand for v) refuses to establish an individual employment plan or does not comply with its provisions; them, a first-come first-served principle is then applied and eligible unemployed can be put on a vi) refuses or through her/his own fault drops out from ALMP; waiting list and included in the programme once it becomes available. The individual employment plan is updated at least every six months, when the ALMPs and activities envisaged by it are ad- vii) does not respond to a call by NES; justed to the labour market needs and the characteristics of the unemployed at least once every six viii) does not notify NES of any change that affects the acquisition or loss of rights or obligations stip- months. In addition to regular meetings, the unemployed person can request a meeting with the ulated by the Law on Employment and Unemployment Insurance, no later than five days after the NES case worker at any time. change; For any unemployed person who states that she/he can find employment without support from the ix) performs work without an employment or other form of legal contract (e.g., freelance contract, NES and thus declines the available support, the case worker and unemployed person determine activ- services contract, etc.). ities that the unemployed person will undertake. Furthermore, to encourage and motivate the benefi- The unemployed person is deleted from the records on the day of non-fulfilment of the obligation de- ciaries of FSA to search for jobs, it is recommended that FSA beneficiaries contact the case worker once termined by the law. If there is a justified reason for not reporting, the unemployed person is obliged to a week. Recipients of unemployment benefits, hard-to-employ categories as defined by the NEAP and 8 The individual employment plan contains the personal data on the unemployed, information on occupations in which she/he seeks 9 Desiere, Sam, Kristine Langenbucher and Ludo Struyven, Statistical profiling in public employment services: An international employment, a list of activities that will be performed and a list of active labour market measures recommended for the unemployed. comparison, OECD Social, Employment and Migration Working Papers 224, OECD Publishing, Paris, 2019. 16 NATIONAL EMPLOYMENT SERVICE NATIONAL EMPLOYMENT SERVICE 17 report in person and inform NES immediately after the termination of that reason. If an unemployed 1.2. PASSIVE LABOUR MARKET PROGRAMMES: UNEMPLOYMENT BENEFITS person gets removed from the register of the unemployed through their own actions, they may not register as unemployed for six months. Unemployment benefits can be claimed by individuals who were insured in case of unemployment A person’s unemployment registration is terminated for the following cases: based on employment, self-employed or contract for temporary and occasional work. More specifi- cally, all the following criteria need to be fulfilled for an individual to be able to claim unemployment i) the unemployed deregisters from the records; benefits: (i) she/he was insured in case of unemployment over the previous 12 months without breaks10 ii) the unemployed finds formal employment (either employment contract or other types of employ- or for 12 months within the previous 18 months with breaks; (ii) the employee’s insurance was termi- ment such as services contracts); nated without her/his will or fault; and (iii) if the employee was declared redundant, and she/he has iii) a person becomes a full-time student – i.e., a student if she/he is younger than 26 years of age; not received severance payment from the Budget of the Republic of Serbia in excess of the amount of severance payment determined by the Labour Law. iv) a person fulfils conditions for exercising the right to an old-age pension or exercising the right to a family or disability pension; Unemployed individuals who have the right to claim unemployment benefits are required to submit their application within 30 days of termination of their compulsory insurance. If they miss this deadline, v) unemployed reaches the age of 65; the period for which they can claim benefits is reduced by the number of days by which they missed the vi) a person becomes completely incapable of working in accordance with the law; deadline. If the unemployed submits the claim after the expiration of the period for which she/he has the right to unemployment benefits, she/he fully loses the right to this financial compensation. vii) a person enters into serving a prison sentence, imposed security measures, educational or protec- tive measures, for a period of at least six months; The unemployed are entitled to unemployment benefits for a period ranging from 3 to 24 months: viii) for foreign residents, if the permanent or temporary residence permit is terminated; i) 3 months if the person has between 1 and 5 years of service; ix) in case of death. ii) 6 months if the person has between 5 and 15 years of service; Table 1 presents the number of newly registered unemployed for the years 2017 through 2020. Overall, iii) 9 months if the person has between 15 and 25 years of service; the distribution by gender is similar and no age group is over-represented compared to others, when iv) 12 months if the person has more than 25 years of service; and considering the ranges of the different age groups. With respect to education, individuals with sec- ondary education are the largest group among the newly unemployed, followed by those with primary v) 24 months if the person is less than 2 years away from reaching the legal retirement age. school or less, with the smallest group being those with college education or higher. For 2020, we ob- A year of service is considered to be the period of 12 months for which the person had compulsory serve a drop in total registrations of unemployed, which can likely be explained by an increase of the insurance. inactive population due to the pandemic. The amount of unemployment benefit depends on the previous salary of the unemployed in the 12 months preceding the employment termination and the latest available statistical information on the Table 1. Number of newly registered unemployed average salary in the Republic of Serbia. By dividing these two amounts a personal coefficient is ob- 2017 2018 2019 2020 tained, and by multiplying the same with the nominally determined amount of the daily cash benefit Abs. Abs. Abs. Abs. base, which currently amounts to RSD  1,039 (€8.8), the amount of the daily cash benefit is deter- % % % % mined. The cash benefit is determined in the gross amount, which includes the amount of pension and number number number number Gender disability contributions and health insurance contributions, and currently the lowest net amount is Males 195 50% 191 49% 180 48% 158 48% RSD 14,940 (€127), and the highest is RSD 34,635 (€295). The basis of the daily cash benefit as well as Females 197 50% 200 51% 193 52% 173 50% the lowest and highest monthly amount of the cash benefit are adjusted at the beginning of each year Age categories with the annual consumer price index in the calendar year preceding the year for which the adjustment Age 15 to 29 148 38% 141 36% 129 35% 120 36% is made. Age 30 to 54 201 51% 204 52% 196 53% 170 51% Beneficiaries of unemployment benefits have the same obligation as other registered unemployment Age 55 and higher 43 11% 46 12% 47 13% 42 13% individuals (as outlined above). They meet their case worker on a regular basis and have an individ- Education categories ual employment plan. In addition, they are required to visit the NES office every month (rather than Primary school or less 104 27% 104 27% 100 27% 85 26% every three months) to confirm their unemployment status. They are given priority in participating in Secondary school 216 55% 215 55% 205 55% 183 55% ALMPs. College or higher 71 18% 72 18% 68 18% 64 19% Total 392 100 391 100 372 100 332 100 10 Periods lasting at least 30 days are considered to be breaks. Source: National employment service data, author’s calculations 18 NATIONAL EMPLOYMENT SERVICE NATIONAL EMPLOYMENT SERVICE 19 A beneficiary of unemployment benefits can lose this right if: (i) the person is removed from the regis- Table 3. Share of unemployment benefit recipients among new unemployed ter of the unemployed through their own actions, in accordance with the law; (ii) a person’s unemploy- 2017 2018 2019 2020 ment registration is terminated, in accordance with the law; (iii) a person fails to notify NES within five Gender days of the change, which is a condition or basis for the acquisition, exercise or termination of the right Males 18% 17% 16% 16% to financial compensation; (iv) if the responsible authority determines that the person is informally Females 21% 21% 21% 21% employed without any contract; and (v) if a person submits a request for termination of rights. Age categories Payment of cash benefits is suspended for the period during which the rights based on unemployment Age 15 to 29 11% 11% 10% 10% are suspended; i.e., for the duration of the contract of temporary and occasional jobs, stay in a foreign Age 30 to 54 23% 21% 21% 22% country for reasons provided by law, military service, or due to imprisonment of less than six months. Age 55 and higher 36% 33% 32% 31% In these cases, once the reason for suspension falls away, the person can resume receiving benefits for Education categories the remaining period initially granted, but she/he needs to register and reapply for the benefit. Primary school or less 15% 12% 12% 13% Secondary school 21% 20% 20% 20% Table 2 presents the distribution of unemployment benefits in terms of socio-economic background. College or higher 22% 25% 24% 23% Women are somewhat more likely to be recipients of unemployment benefits than men. There do not Total 20% 19% 18% 18% seem to be large differences in terms of age, when considering the ranges of the different age groups. Source: National employment service data, author’s calculations With respect to education, most recipients of unemployment benefits have completed secondary school, while the others are equally distributed between those with primary school or less and those with college or higher education. In 2020, surprisingly, the number of new recipients of unemployment benefits dropped to 61,120 from 68,514 in 2019. 1.3. DELIVERY OF ACTIVE LABOUR MARKET PROGRAMMES Table 2. Number of new recipients of unemployment benefits Based on their individual employment plans, unemployed people are referred to one or more ALMPs. 2017 2018 2019 2020 The NES offers a wide variety of ALMPs for unemployed people. In line with the nomenclature in eco- Abs. Abs. Abs. Abs. nomics, these can be broadly classified as: (i) job search assistance programmes; (ii) skills development % % % % number number number number trainings (classroom and/or on-the-job); (iii) subsidized private sector employment programmes; and Gender (iv) subsidized public sector employment programmes. Males 35,115 46% 31,709 43% 28,171 41% 25,002 41% 1) Job search assistance programme Females 41,336 54% 41,757 57% 40,343 59% 36,118 59% Age categories Job search assistance programmes include various soft measures intended to enhance the individual’s Age 15 to 29 15,606 20% 14,893 20% 13,099 19% 11,490 19% effort to search for a job. Job search assistance programmes include job search training, counselling, Age 30 to 54 45,307 59% 43,566 59% 40,539 59% 36,652 60% job search monitoring,11 job clubs and job fairs, among others. Additionally, sanctions (measures ap- Age 55 and higher 15,538 20% 15,007 20% 14,876 22% 12,978 21% plied if the unemployed does not comply with the requirements) are considered part of this type of Education categories programme in accordance with the Law on Employment and Unemployment Insurance. Primary school or less 15,606 20% 12,114 16% 11,871 17% 10,699 18% 2) Skills development trainings Secondary school 45,307 59% 43,622 59% 40,548 59% 35,801 59% College or higher 15,538 20% 17,730 24% 16,095 23% 14,620 24% Skills development trainings aim to increase the human capital of the unemployed. They include class- Total 76,451 100 73,466 100 68,514 100 61,120 100 room vocational/technical training, work practice (on-the-job training), basic skills training (mathe- Source: National employment service data, author’s calculations matics, language), life skills training (socio-affective, non-cognitive skills), and job placement.12 Table 3 presents the share of the newly unemployed who claim unemployment benefits. The share of unemployed who claim unemployment benefits experienced a drop of 2 percentage points from 20 per 11 For example, the unemployed could be asked to show some proof that she/he applied for a certain number of jobs within a specified cent in 2017 to 18 per cent in 2020. There are differences with respect to the age and education cate- period. gories. The older an unemployed person, the more likely it is that she/he will receive unemployment 12 According to NEAP 2020, the following further education and training programmes are offered in 2020: (i) Professional traineeship benefits. With respect to education, unemployed people with primary school or less are less likely to programme; (ii) Internship programme for youth with higher education; (iii) Internship programme for unemployed persons with secondary education; (iv) Programme for the acquisition of practical knowledge for non-qualified persons, redundant workers and long- have the right to unemployment benefits compared to those with more education (secondary or higher term unemployed persons; (v) Training in response to labour market needs; (vi) Trainings at the employer’s request – for unemployed education). persons; (vii) Training for the needs of the employer for an employee; (viii) Functional elementary education for adults. 20 NATIONAL EMPLOYMENT SERVICE NATIONAL EMPLOYMENT SERVICE 21 3) Subsidized private and public sector employment at 0.08 per cent even though there were no labour market trainings due to COVID-19. The funds for labour market trainings were actually spent on other ALMPs. The largest part of the budget of NES (59 Subsidized private sector employment includes incentives both for the unemployed and for the private per cent in 2019) is spent on unemployment benefits. Surprisingly, the share of GDP spend on unem- sector to hire unemployed individuals. There are two types of incentives: wage subsidies or self-em- ployment benefits dropped from 0.25 per cent in 2019 to 0.18 per cent in 2020. This is contrary to the ployment assistance. There are also additional support programmes for persons with disabilities who economic theory, which would predict that in times of lower economic activity the expenditure on un- are employed under special conditions such as job adjustment and work assistance. employment benefits increases as presumably unemployment rises. We know that unemployment did With respect to delivery, only classroom training programmes belonging to the second category skills not increase in 2020 and, as shown earlier in Table 2, the number of new unemployment benefit recip- development trainings are outsourced, and training providers are externally hired in Serbia. Providers ients dropped in 2020 with respect to 2019. The Government provided financial support to companies for classroom training programmes are selected through public procurement procedures. Conversely, which helped them retain the employees. All these developments together explain this counterintuitive job search assistance programmes are provided by in-house employees. finding that the share of GDP spent on unemployment benefits dropped. Table 5. Share of GDP spent on ALMPs and unemployment benefits 1.4. BUDGET AND PERSONNEL 2017 2018 2019 2020 Table 4 shows the distribution of the budget of NES for 2017 to 2020. Overall, the largest share of the Share of GDP spent on ALMPs and unemployment benefits 0.41% 0.38% 0.33% 0.26% budget is spent on unemployment benefits and other benefit claims, although this share has fallen dur- Share of GDP spent on ALMPs 0.08% 0.08% 0.08% 0.08% ing the observed period by 10 percentage points. During the same period, the share spent on ALMPs Share of GDP spent on unemployment benefits and other related rights 0.33% 0.29% 0.25% 0.18% has increased from 15 per cent to 21 per cent. Source: NES Budget execution reports for 2017, 2018, 2019 and 2020. Table 4. Annual national employment service budget Figure 3. Public expenditure in ALMPs in OECD as percentage of GDP 2017 2018 2019 2020 Abs. Abs. Abs. Abs. 2.00 % % % % number number number number 1.80 Administration and support 4,208 18% 3,538 15% 4,621 20% 5,582 21% 1.60 Unemployment benefits and other 1.40 15,851 67% 14,930 65% 13,308 59% 15,289 57% 1.20 benefits Unemployment benefits 11,610 49% 11,502 50% 10,536 47% 12,229 46% 1.00 0.80 Other 4,241 18% 3,428 15% 2,772 12% 3,060 11% 0.60 Active labour market programmes 3,641 15% 4,643 20% 4,671 21% 5,833 22% 0.40 ALMPs general 2,368 10% 2,795 12% 3,151 14% 4,000 15% 0.20 ALMPs for professional 0.00 481 2% 452 2% 444 2% 550 2% rehabilitation and disabled Denmark Sweden Finland Belgium France Luxembourg Hungary Austria Spain Germany Netherlands Switzerland OECD countries Norway Estonia Poland Ireland Republic of Korea Czech Republic Lithuania Slovenia Australia Canada Slovak Republic Latvia Israel Portugal Japan Chile ALMPs co-financed with local 792 3% 836 4% 817 4% 900 3% self-governments ALMPs other 0 0% 560 2% 49 0% 383 1% Total budget 23,700 100% 23,111 100% 22,600 100% 26,705 100% Source: NES Budget execution reports for 2017, 2018, 2019 (numbers in RSD) Source : OECD. Stat https://stats.oecd.org/Index.aspx?DataSetCode=LMPEXP# Table 5 shows the NES budget in relation to gross domestic product (GDP). Overall, the share of GDP The NES organizational units are as follows:14 spent on ALMPs was 0.08 per cent between 2017 and 2019, which is extremely low compared to other countries. For instance, the share of GDP spent on activation policies in Organisation for Economic 1. Directorate with its working unit in Belgrade: 268 employees. Co-operation and Development (OECD) countries ranges from 0.14 per cent in Chile to 1.96 per cent 2. Regional employment offices in Vojvodina and Kosovo: 18 employees. in Denmark, as can be seen in Figure 3.13 Note that in 2020 the expenditure on ALMPs was maintained 3. 32 branch offices: 1,503 employees. 13 https://www.oecd.org/employment/activation.htm 14 National employment service Fact Sheet updated 17 March 2021. 22 NATIONAL EMPLOYMENT SERVICE NATIONAL EMPLOYMENT SERVICE 23 The cabinet of the director is part of the directorate. The directorate has eight sectors15 and employs 268 people. This includes the Sector for Employment Support, which employs 40 people. This Sector 2 REVIEW OF ACTIVE LABOUR MARKET PROGRAMMES is responsible for planning, organization, evaluation and operational issues related to ALMPs. The Sec- tor for Unemployment Insurance and Legal Issues employs 24 people. It includes the Department for Unemployment Insurance, which ensures that regulations related to employment and unemployment benefit claims are universally enforced. It also provides professional support and advice in this area to regional branches. The two regional offices for Vojvodina and Kosovo and Metohija each employ nine 2.1. COVERAGE OF ACTIVE LABOUR MARKET PROGRAMMES people. Most NES employees, 1,503 in total, work in branch offices. This includes 598 case workers16 and 177 administrative officers (evidentičari) who register the unemployed and schedule appointments Table 6 presents the number of unemployed participants in each type of ALMP between 2017 and with case workers. The average number of employees per branch office is 47. Branch Vršac17 is the 2020. Dividing the annual number of participants by the average number of unemployed in the same smallest (excluding Kosovo), with 22 employees, and Belgrade is the biggest, with 277. year indicates the coverage of each type of ALMP. The most implemented ALMPs are job search assis- tance programmes, which increased over the period 2017 to 2019, totalling approximately 1,100,000 in 2019. This increase reflects the efforts of NES to strengthen the advisory role of case workers and to intensify contact with clients. There was a drop in 2020, but this Is not surprising, as in 2020 there were several periods when the contacts were limited and personal meetings were not possible. The number of participants in skills development training programmes, which stood at around 10,000 in 2019, has been falling from 2017 to 2019. The number of private sector incentive programmes had moderate deviations around 10,000 per year in the observed period. Finally, public sector employment programmes, which stood at roughly 6,000 in 2019, have been decreasing over the observed period. In 2020, the number of participants in skills development programmes increased as a result of the intro- duction of a new ALMP, ‘My First Salary’, targeting labour market entrants. The number of participants in private sector incentive programmes fell while the number of participants in public works remained stable in 2020. In terms of coverage of ALMPs, in 2019, the last year prior to the pandemic, the number of job search assistance interventions was more than two per registered unemployed. This is not surprising, as this type of ALMP includes meetings between case workers and unemployed people, which are required at least once every six months. The other three types of ALMPs require more resources than job search assistance programmes and thus cover a smaller number of unemployed people. In 2019, skills devel- opment training and private sector employment programmes covered 2 per cent of the unemployed and public sector employment programmes 1 per cent. Table 6. Number of participants and coverage of ALMPs by type Participant numbers Coverage (in thousands) 2017 2018 2019 2020 2017 2018 2019 2020 Job search assistance 967 1,016 1,095 968 149% 174% 207% 190% Skills development training 13 11 10 15 2% 2% 2% 3% Private sector incentive 9 10 11 8 1% 2% 2% 2% programmes Public sector employment 10 9 6 6 2% 1% 1% 1% 15 The eight sectors are: Sector for Employment Support; Sector for Unemployment Insurance and Legal Issues; Sector for Economic programmes (public works) Affairs; Sector for Human Resources; Sector for Material Resources; Sector for Projects and International Cooperation; Sector for Total ALMPs 999 1,047 1,123 997 Development and Information Communication Technologies; and Sector for Internal Audit. 16 There are five types of case workers: case workers for careers; case workers for claiming unemployment benefits; case workers for Registered unemployed 651 583 530 509 career planning; case workers for employment; and case workers for employment of people with disabilities. (mean all months) 17 There are two branches with even smaller number of employees, but both are in Kosovo: Priština has 8 employees and Gnjilane 17. Source: National employment service data, author’s calculations. 24 NATIONAL EMPLOYMENT SERVICE NATIONAL EMPLOYMENT SERVICE 25 2.2. CHARACTERISTICS OF BENEFICIARIES BY TYPE OF ALMP As mentioned, the NES monitors whether an unemployed person belongs to the target groups, and preference in ALMPs is given to the target groups that are considered ‘hard to employ’ in the specific This section discusses whether some subgroups of the unemployed attend more frequently certain year. Each year, the National Employment Action Plan defines which target groups22 are considered types of ALMPs compared to other groups. Table A1 in Annex 1 shows the distribution of the charac- ‘hard to employ’ in that specific year based on labour market indicators of previous years. For instance, teristics of all unemployed people. To establish whether specific types of ALMPs are biased towards in 2019 the following target groups were in the hard-to-employ category: individuals younger than 30, some groups of the population, the distribution of each type of ALMPs (shown in Table A2 through individuals older than 50, redundant workers, individuals with primary school or less, long-term un- Table A4) needs to be compared to the overall distribution of the unemployed (Table A1). In this sec- employed (more than 12 months of registered unemployment), people with disabilities, Roma, recipi- tion, our reference year is the last year before the pandemic, 2019, and we discuss in detail the ALMPs ents of Financial Social Assistance, and a few other small groups. Aside from job search assistance pro- in 2020 in the section on the COVID-19 pandemic. grammes, which cover the entire unemployed population, between 90 and 100 per cent of participant places of skills development trainings, private sector incentive programmes and public sector incentive First, job search assistance programmes18 cover the entire population of the unemployed, as all unem- programmes were assigned to at least one of the target groups considered hard to be employed in 2019. ployed individuals meet case workers and need to have an individual employment plan. Males attend job search assistance programmes more frequently than females, while the largest group of beneficiaries The NES monitors gross employment rates by ALMP for each calendar year but does not systemati- comprises middle-aged people (30 to 54 years). The largest share of beneficiaries has secondary school cally use empirical models to evaluate ALMPs.23 In the past, MoLEVSA would commission external education, followed by primary school or less, while the smallest category is college educated or higher. consultants to evaluate certain subtypes of skills development trainings provided by NES. Thus, there are few rigorous studies relying on a credible comparison group which shed light on the effectiveness With respect to skills development trainings,19 women are more likely (66 per cent in 2019) to attend of NES ALMPs. During 2015, an impact evaluation of some active employment policy programmes this type of ALMP. The most frequent age category (57 per cent) is young people aged 15 to 29. This was conducted24 and found that the programme Training upon Request of an Employer25 had a positive can be explained by the fact that, of the approximately 10,000 participants, 5,000 attended the Profes- effect on the employability of participants two years after exiting the measure, while the Labour Market sional Traineeship Programme in companies, a programme which targets young people. In terms of Training26 programme had a positive effect only on persons with disabilities.27 A study from 2016 ex- educational background, the largest share of participants has higher education, reflecting again that amined the impact of the Youth Services Package targeting young unemployed people and empirically roughly half of the participants were included in the Professional Traineeship Programme. This ALMP evaluated two specific programmes: Professional Traineeship Programme28 and Acquisition of Practical is frequently used to place young people in the labour market. This programme was used mostly by Knowledge.29 It found that the Professional Traineeship Programme did not have an impact on key la- individuals who were unemployed for less than a year. In this category, there is a specific programme bour market outcomes (employment status and wages), while the Acquisition of Practical Knowledge called Acquiring Practical Knowledge, an on-the-job training programme that exclusively targets hard- programme helped individuals become employed and reduced the inactivity rate. While these studies to-employ individuals with, at most, primary school education, redundant workers and the long-term provide useful information on the impact of specific programmes, there is no systematic evaluation unemployed; any individual falling into one or more of these categories is able to participate in the of ALMPs or a methodology for measuring the net placement rates that result from interventions. programme. In 2019, 910 individuals participated in the Acquiring Practical Knowledge programme. Net placement rates (correcting for job matchings that would have resulted even in the absence of the The NEAP for 2020 specifies that 200 individuals who completed functional adult education will be intervention) would provide an accurate metric to assess the effectiveness of interventions and a basis included in classroom-based vocational trainings. The proportion of long-term unemployed in skills for policy decisions. development trainings has been decreasing since 2017. Other important and informative descriptive studies conducted in the last five years, which are, how- For private sector incentive programmes,20 there are no discernible patterns that some population ever, not identifying the causal impact and are not able to estimate the effectiveness of ALMPs, include subgroups are more frequently represented than the overall unemployed population. Beneficiaries of the following: (i) In 2017, two relevant studies were commissioned: ‘Analysis of the subsidized employ- these programmes are predominantly middle-aged unemployed people with a secondary school edu- ment programme’ (subsidy for self-employment and subsidy for employment of unemployed persons cation. With respect to job search duration, the largest share of participants has been unemployed for less than a year. 22 Younger than 30, older than 50, redundant workers, individuals with primary school or less, long-term unemployed (more than 12 months of registered unemployment), people with disabilities, Roma, recipients of Financial Social Assistance, refugees, internally Looking at public sector incentive programmes,21 men are more likely to participate in this type of displaced persons, returnees, victims of trafficking, victims of violence, single parents, both parents unemployed, and prison returnees. 23 As part of the GIZ project Youth Employment Programme, a data cleaning methodology and matching technique has been developed to ALMP (55 per cent in 2019). No clear pattern can be observed for age distribution, but these benefi- analyse NES ALMPs. This methodology has been applied to evaluate the impact of ALMPs financed by the project. ciaries more frequently have a lower educational background. The largest share has been unemployed 24 International Labour Organization, ‘Active Labour Market Programme Impact Assessment: The impact of three programmes for less than a year. The proportion of long-term unemployed in public sector incentive programmes implemented by the National Employment Service of Serbia’, ILO, 2015. 25 This programme finances the training of unemployed to acquire the knowledge and skills for a specific open position at a given has been decreasing since 2017. employer. 26 Class-based vocational/technical trainings. 27 Marjanović, Dragana, et al., Service Package and the Relevant Programmes and Measures Funded from the Republic of Serbia Budget 18 See Table A2 in Annex 1 for details. and Targeted at Youth, SIPRU, Belgrade, 2016. 19 See Table A3 in Annex 1 for details. 28 This programme aims to provide a first work experience and to prepare the participants for independent work. For professions where a 20 See Table A4 in Annex 1 for details. professional exam is required (e.g., lawyers, teachers, etc.), the programme aims to prepare the participant for this type of exam. 21 See Table A5 in Annex 1 for details. 29 This is a work-based learning programme aiming to teach skills required for a specific job. 26 NATIONAL EMPLOYMENT SERVICE NATIONAL EMPLOYMENT SERVICE 27 from the categories of hard-to-employ persons) and ‘Assessment of public expenditures from the budget activity targeting. As regards the macroeconomic situation, the ALMPs are more successful in periods of the Republic of Serbia spent in the period 2013–2015 years on policies aimed at increasing employa- of positive GDP growth and when the unemployment rate is low.33 bility and youth employment’; (ii) In 2018, within the project ‘Promotion of Youth Employment – YEP’, the following reports were written: ‘Analysis of local employment action plans for 2018’, ‘Analysis of the position of Roma men and women in the labour market’ (with the support of GIZ), and ‘Analysis 2.3. GEOGRAPHICAL DISTRIBUTION OF ALMPS of women’s participation in active employment policies and spending of funds for those purposes in the period 2015–2017’; (iii) In 2019, in accordance with the procedure for adopting planning policy doc- Table A6 in Annex 1 shows the geographical distribution by branch for 2019 for each type of ALMP, uments, and as part of the process of preparation of the NEAP for 2020, the following analyses were as well as separately for classroom trainings (third column). The capital Belgrade and the larger cit- carried out: ex-post analysis of the NEAP for 2017 and NEAP for 2018, as well as an ex-ante analysis of ies (Kragujevac, Niš and Novi Sad) lead in all types of ALMPs. Classroom trainings, the only pro- the NEAP for 2020. Finally, in cooperation with UNDP, an analysis of the difficulties faced by Financial gramme provided by external providers, are spread over all branches except Priština in Kosovo. Only Social Assistance beneficiaries in accessing the labour market was prepared. two branches have more than 100 participants, whereas the rest had fewer than 100 participants of classroom trainings in 2019. Table 7. Summary of key characteristics for different types of ALMPs Job search assistance Skills development Private sector incentives Public sector employment 2.4. SUMMARY OF KEY FINDINGS trainings Government cost Low Medium/High High High Short-run effect Positive Negative Positive Positive This section has highlighted the following issues related to ALMP delivery: (i) there is limited infor- Long-run effect (Best case) Small positive Large positive Small positive Zero to small positive mation on the effectiveness of ALMPs; (ii) there is a relatively low take-up of ALMPs and, in particu- Long-run effect lar, skills development trainings; (iii) there is some evidence of unequal distribution of services (for Small negative Small negative Negative Large negative (Worst case) example, participants in skills development trainings predominantly have higher education); and (iv) Displacement Medium Low High High there is no evidence of budget reallocation towards or away from specific programmes according to Source: Kluve (2016).30 effectiveness, based on the data collected. Section 7 will present recommendations to tackle the issues identified in this section. Due to the limited evaluations of NES ALMPs, Table 7 provides a summary of the main characteristics of ALMPs from a meta-analysis of studies carried out in several countries. This table provides an over- view of the government cost, short- and long-term effects, and the displacement effect for each type of ALMP analysed in the international evidence. In the case of ALMPs, we refer to a period of 12 months after completion of the programme as short-term effects, while long-term effects refer to a period more than two years after completion. Job search assistance programmes have the lowest cost but also have a small impact in the long term. Skills development trainings are most effective in the long term in terms of employment, but, due to the ‘lock-in effect’,31 this positive effect is not reflected in the short term. Private sector incentive programmes and public sector employment programmes are known to be suc- cessful in the short term, but their long-term impacts are, at best, slightly positive. However, it is important to point out that the effectiveness of each individual programme depends on the target group, the design of the programme and the macroeconomic environment. The impacts of a programme can be heterogeneous across groups (e.g., youth, gender, vulnerable groups). For in- stance, skills development trainings, which are successful on average, have shown mixed results in the case of youth.32 Aside from the target group, additional elements of the programme are considered to determine its effectiveness: programme duration, monetary incentives, individualized follow-up and 30 Kluve, Jochen, A review of the effectiveness of Active Labour Market Programmes with a focus on Latin America and the Caribbean, International Labour Office, Geneva, 2016. 31 The lock-in effect refers to the situation where the unemployed participates in the programme during a certain period of time, after which she/he searches for a job. Hence the positive effect of such measures is not seen in the short term. 32 Caliendo, Marco, and Ricarda Schmidl, ‘Youth unemployment and active labor market policies in Europe’, IZA Journal of Labor Policy, 33 Yeyati, Eduardo L., Martin Montané, and Luca Sartorio, What works for Active Labor Market Policies? A meta analysis, (No. wp_ vol. 5, no. 1, 18 January 2016, article 1. gob_2018_9), Universidad Torcuato Di Tella, Buenos Aires, 2019. 28 NATIONAL EMPLOYMENT SERVICE NATIONAL EMPLOYMENT SERVICE 29 3 THE NES DURING THE COVID-19 The number of new recipients of unemployment benefits dropped from 68,514 in 2019 to 61,120 in 2020. The evolution of the number of new recipients of unemployment benefits is shown in Figure 5. PANDEMIC From April onwards, the number of new beneficiaries was lower in 2020 than in 2019. This is a sur- prising finding, but it is somewhat in line with the drop in the number of new registered unemployed. Figure 5. New recipients of unemployment benefits The COVID-19 pandemic in Serbia affected the functioning of various institutions, including NES. The state of emergency in Serbia was declared on 15 March 2020 and lifted on 6 May. When the state 8,000 of emergency was in place, all government institutions tried to reduce in-person contact with clients to prevent the spread of the virus. 6,000 New beneficiaries During this period, the NES moved some of its services online. Unemployed persons could register with NES via email, while unemployment benefit claims could be submitted electronically. Both em- 4,000 ployers and unemployed people could apply electronically via email to be included in ALMPs. Other less frequent requests could also be submitted via email. As of the date of writing,34 the NES still allows 2,000 use of email to submit different types of requests as it did during the state of emergency; however, all services are again also offered in-person since that time. Offering registration and other NES services 0 1 2 3 4 5 6 7 8 9 10 11 12 permanently online would require changes in the Law on Employment and Unemployment Insurance, Month which currently requires in person registration and reporting. 2019 2020 In 2020 there were no externally delivered skills development trainings. NES has proposed some ad- Source: NES data justments with respect to the manner of performance of certain trainings (for example, allowing online teaching if possible) and presented them to MoLEVSA, but MoLEVSA did not approve the online Figure 6 shows the evolution of the number of ALMP participants by type of ALMP. Overall, the num- delivery. ber of participants in job search assistance programmes (including predominantly job-counselling) fell In 2019 the total number of new registered unemployed stood at approximately 372,000, while it in 2020 with respect to 2019. The drop was sharp during the lockdown in the months of March through dropped to roughly 332,000 in 2020. Figure 4 shows the monthly evolution of new registered unem- May, and it rose above the 2019 numbers in the months June through August. In the last quarter of the ployed for 2019 and 2020. We observe a sharp drop in new unemployed in March and April in 2020 year the number of participants was stable, but below the 2019 numbers. during the period of lockdown and a sudden increase in June when the country fully reopened. The Turning now to skills development trainings, the number of participants was higher in 2020 than in numbers of new unemployed dropped again in the months October to December. 2019 due to the introduction of a new programme called ‘My First Salary’, which enrolled 8,172 partic- ipants in December 2020. Aside from this particular programme, the number of participants in skills Figure 4. New registered unemployed development programmes was lower in 2020 than in 2019. 40,000 The ‘My First Salary’ programme was first introduced in 2020 and it was designed as a response to the current economic situation. It aims to incentivize youth employment and provide support to the 30,000 economy by addressing the problem of labour shortages. The programme is aimed for unemployed New unemployed youth up to the age of 30 without professional work experience who have completed either secondary 20,000 or higher education. The programme is a work-based traineeship with a nine-month duration. The programme foresees a financial compensation of RSD 20,000 (€170) for secondary school graduates 10,000 and RSD 24,000 (€204) for higher education graduates. 0 The number of participants in private sector incentive programmes was lower in 2020 with respect to 1 2 3 4 5 6 7 8 9 10 11 12 2019, and this reduction is reflected in all months with the exception of July. These numbers suggest Month that private companies did not make use of the existing incentive schemes to hire new employees, 2019 2020 probably due to the shrinking economic activity and the unpredictability of the economic situation. Source: NES data Lastly, the number of participants in public works remained stable in 2020 with respect to 2019. Figure 6 shows that the implementation of public works was delayed in 2020 but, overall, the numbers are in 34 25 March 2021. line with the numbers from the previous year. 30 NATIONAL EMPLOYMENT SERVICE NATIONAL EMPLOYMENT SERVICE 31 Figure 6. Participants in ALMP types solvent immediately after the state of emergency was lifted at the beginning of May. To be eligible for this financial transfer, SMEs were expected not to reduce the number of permanent employees by more a. Job search assistance b. Skills development training than 10 per cent until July when the last instalment was transferred. In August a second package of 120,000 8,000 measures was implemented, and SMEs could apply for additional wage subsidies for each employee in 6,000 the amount of 60 per cent of minimum wage for a two-month period. Similar to the first package, SMEs Participants Participants 80,000 4,000 benefiting from the second package committed not to reduce the number of employees by more than 40,000 10 per cent until the end of the year. Both support packages offered to large companies the opportunity 2,000 to apply for wage subsidies only for furloughed employees, and they would receive in this case 50 per 0 0 1 2 3 4 5 6 7 8 9 10 11 12 1 2 3 4 5 6 7 8 9 10 11 12 cent of the minimum wage per furloughed employee. Month Month The strict lockdown in Serbia was implemented only in the period March through May, and most businesses are open since then, but some sectors of the economy were and still are affected more c. Private sector incentive programs d. Public works than others, and some population subgroups are more vulnerable to economic shocks. Containment 2,500 2,500 measures and decreasing aggregate demand will have harder effects on non-essential sectors (such as 2,000 2,000 tourism and trade, among others), where vulnerable employees are frequently employed. In particular, Participants Participants 1,500 1,500 informally employed, self-employed, low-wage earners, employees with non-permanent contracts and 1,000 1,000 in small firms, and women and young people are at most risk of suffering from the economic down- 500 500 turn.36 Women are considered particularly vulnerable because they are not only over-represented in 0 non-essential sectors but were also the main caretakers of children during school and kindergarten 0 1 2 3 4 5 6 7 8 9 10 11 12 1 2 3 4 5 6 7 8 9 10 11 12 closures. Likewise, in times of economic crisis, young people are often first to be laid off. Additionally, Month Month research indicates that if young people enter the labour market in times of recession, many suffer long- term negative effects on their employability and wages.37 2019 2020 Ideally, in times of economic downturn, spending on ALMPs should be increased. In addition to stand- Source: NES data ard activities, increased spending can usefully include specific programmes to target those who lost their jobs, to protect them from long-term labour market exclusion. Such times can be used for edu- cation, reskilling and training programmes as the demand for labour falls. According to ILO,38 young There is no doubt that the COVID-19 pandemic has a negative impact on some sectors in the labour people will be among the hardest hit by the current crisis, and the introduction of the programme ‘My market. Several explanations can be offered as to why this is not yet reflected in NES data. First, the First Salary’ by NES is expected to alleviate this problem. pandemic may most significantly have affected employment of informally employed people, who are not captured either by NES data or data from the Central Registry of Compulsory Social Insurance (CROSO). Informal workers are especially hard hit by the crisis caused by the lockdown measures, because they typically cannot rely on income replacement or savings and/or because they work in the hardest hit sectors.35 Second, it is reasonable to assume that individuals who became unemployed during this period did not have a strong incentive to register with NES, because they assumed they could get little support from NES given that the economic activity dropped and many companies froze their hiring processes. How- ever, the number of people who were formally employed and lost their jobs in the period of COVID-19 should be reflected in the CROSO data, to which we did not have access. Third, wage subsidies were provided to small and medium enterprises (SMEs) and to a smaller degree to large companies. As part of the first package of economic measures, SMEs were given wage subsi- dies for all SME employees, amounting to three payments of the minimum wage per employee. These subsidies, together with other ALMPs introduced by government, helped SMEs to remain financially 36 https://www.ilo.org/wcmsp5/groups/public/@dgreports/@dcomm/documents/briefingnote/wcms_745963.pdf 37 Kahn, Lisa, ‘The long-term labor market consequences of graduating from college in a bad economy’, Labour Economics, vol. 17, no. 2, 35 International Labour Organization, ILO brief: COVID-19 crisis and the informal economy: Immediate responses and policy changes, April 2010, pp. 303–316. ILO, Geneva, May 2020. 38 https://www.ilo.org/wcmsp5/groups/public/@dgreports/@dcomm/documents/briefingnote/wcms_745963.pdf 32 NATIONAL EMPLOYMENT SERVICE NATIONAL EMPLOYMENT SERVICE 33 4 IMPROVED USE OF LABOUR The NES also has an online portal, which offers job-matching services to jobseekers and companies, with a customized matching functionality. Both jobseekers and companies can register and examine MARKET INFORMATION the demand for and supply of jobs. If they wish, companies can receive additional support in the job- search process from NES employees. However, the portal is not widely used, and the quality of job matches through the portal is not high, which discourages potential users from accessing this service. The NES MIS receives information daily from CROSO. CROSO is a central registry recording employ- 4.1. THE NATIONAL EMPLOYMENT SERVICE MANAGEMENT ment information on all employed individuals such as employment start and end (if available), contract INFORMATION SYSTEM (NES MIS) type, occupation, and so on. It was established in 2014 to collect taxes efficiently and to ensure better control of payment of contributions. The information exchange between NES and CROSO helps to The National Employment Service Management Information System (NES MIS) is a modern, increase the quality and reliability of the data. NES uses information from CROSO to remove newly operational system that supports key NES business processes. It is demand and supply oriented and employed people from its unemployment register and to ensure that beneficiaries of unemployment provides services to jobseekers and employers to enhance their efforts to improve their work situation benefits do not hold any type of employment. or their labour force, respectively. It helps in reducing the information gap that leads to mismatching between demand and supply, improves job placement and matching, provides information on profes- sions and training, helps in managing the ALMPs, and provides information for labour market policy 4.2. ASSESSMENT OF NES MIS formulation. In order to assess the NES MIS, we will adopt a broader view of not only focusing on the technical ca- The NES MIS contains eight functional modules: pabilities of the current MIS but also on how it can be seen as the stepping stone to building a broader 1) General subsystem: includes information on the system’s functionalities. Labour Market Information System (LMIS)40 based on the data it contains. 2) Activities with employers: records contact with employers, letters of intent,39 and job fairs. This broader view of an LMIS would provide different types of decision makers, such as workers, jobseekers, firms and policymakers, access to better information. While the NES uses the information 3) Job intermediation: covers the main business processes of NES and is central to the functioning of contained in its MIS primarily for job-matching purposes, other stakeholders might find this informa- the NES. It is used to record personal data and unemployment spells of the unemployed, record data tion useful for other purposes, such as career guidance, educational or human resources decisions, and on employers, and keep track of rights and obligations of the labour market participants. labour market analysis. 4) Work ability assessment: covers the business process that evaluates the work ability of an individual. While improving use of information within the NES would allow better job-matching functions, en- 5) Professional orientation: records data on information provision to users regarding professional ori- abling broader use would improve measures of labour market efficiency and performance.41 In this entation and data on services provided to unemployed people as part of professional orientation. section we will categorize the NES information system according to its uses and characteristics, based 6) Active labour market programmes: covers all processes related to ALMPs. It keeps track of all on a framework developed in collaboration with other World Bank projects and initiatives.42 Based on ALMPs and participants in these programmes. this categorization, we will provide initial recommendation to boost the use of the information system for NES purposes, as well as building a broader LMIS around it. 7) Unemployment benefits: covers the full process of claiming unemployment benefits, from the initial request, through decision-making and appeal procedures, and all the way to expiration of the right Table 8 outlines the different stages of an LMIS, from basic to advanced, in relation to three dimen- to financial compensation and possible subsequent changes, calculation of financial compensation, sions: main functions, data used, and actors involved. An advanced LMIS both provides data for var- and records of special financial compensation. ied users (jobseekers, business, education providers, etc.) and direct services tailored for the different users. An advanced system relies on real-time big data and has several subsystems of information and 8) Analytical reporting: NES has an advanced system of analytical statistical reporting integrated in digital tools. An advanced LMIS involves both private and public actors as producers of information its MIS. These reports are used to prepare statistical bulletins that are made publicly available, as and users of data and services. well as for monitoring and analysis of ALMPs, to monitor the efficiency of the NES, and to analyse trends in employment using different dimensions. The main internal beneficiaries of these reports are NES employees from the Department of Labour Market Analysis and Planning and associates for statistics in the branch offices. While external users of these reports are numerous, the most important ones are: MoLEVSA and other ministries, the Statistical Office of the Republic of Serbia (SORS), and the general public, which has access to the periodical statistical bulletins. 40 See Annex 3 for a conceptual discussion on building an LMIS. 41 In times of economic crisis or economic recovery, such as the ones stemming from the COVID-19 pandemic, the availability of timely information becomes even more crucial. 39 The letter of intent is signed by the NES and a private company and defines the nature of the cooperation between the two parties. 42 The framework used here was developed in a collaboration with the Korea Employment Information Service and its Worknet platform. 34 NATIONAL EMPLOYMENT SERVICE NATIONAL EMPLOYMENT SERVICE 35 Table 8. Different stages of LMIS according to international standards More effort could be made to collect data on vacancies from private portals, while SMEs should be ap- LMIS type proached to establish collaborative partnerships. The number of vacancies shared on the NES portal is Basic Intermediate Advanced not large, and many are for low-skilled jobs. Raising usage of the job-matching portal by posting more Provide both data and some basic Provide both data for different users vacancies would attract more jobseekers and make the job-search process more efficient. Additionally, Provide basic labour market the NES would get better insight into the current demand for occupations and supply of jobseekers. Main functions services to different stakeholders, in as well as direct services tailored to information via statistical indicators particular jobseekers and private firms different users A comprehensive LMIS would also use data from the country’s education management information Several subsystems of information National surveys and administrative Different data sources beyond surveys and digital tools are used to provide system (EMIS), the complete CROSO dataset, and other available administrative data sources. Data Data used are used to produce labour market data updated (often real-time) information from the EMIS provide insight into the qualifications and occupations of new labour market entrants. intelligence based on big data The CROSO dataset is a census of all formally employed individuals, which contains information on Involve more public actors (education the characteristics of all formally employed individuals and their job contract conditions, together with Big private and public actors as Few public sector actors as users systems, job-counselling and job- Actors involved and producers matching services, social security, producers of information and users the flows into and out of formal employment. An additional relevant source is the Social Card Informa- of data and services tion System, which was developed by the Government of Serbia. The goal of this system is to provide a statistics offices, business registry comprehensive picture of the social status of individuals and entities (family/household, municipality, Source: World Bank region, country). One of its planned functions will support the collection, systematization and report- ing of data on vulnerable populations and groups of people facing particularly serious threats. The The following description of the NES information system is based on the categorization presented Social Card Information System could be used in real time to identify both vulnerable individuals and in Table 8 and identifies what is required for the system to become a fully functional, comprehensive clusters of vulnerable individuals that could be targeted by ALMPs.43 LMIS. The three categories – main functions, data used, and actors involved – are discussed in terms of their current state and best-case scenario. Actors involved Main functions Currently, the NES MIS involves several public sector actors: the NES, CROSO (social services), and SORS. In this category, the NES can be classified as intermediate. There is room to include both more At present, the Serbian LMIS provides basic labour market information relying predominantly on NES public sector actors and to strengthen cooperation with private sector actors. data and somewhat on CROSO data. Further, it provides basic services to different stakeholders and thus can be classified as intermediate in relation to its functions. On the public sector side, a partnership with the Ministry of Education, Science, and Technological Development (MoESTD) should be established, and existing collaborations with CROSO and SORS The NES should seek to improve the provision of direct services to different users. The online portal should be strengthened. NES should seek to strengthen cooperation with the private sector to get should seek to broaden the base of registered jobseekers and firms to provide better options on both access to the information produced by key private sector players. After establishing the available pool sides of the market. This can be achieved through targeted outreach to SMEs and collaboration with of data from different sources, NES should tailor its services depending on the needs of different part- private portals. The services of career guidance should be improved and should include both short- and ners. Services should aim to make the job-matching process more effective and efficient. Additionally, long-term guidance. The career guidance system should be linked not only to available government they should serve to deepen understanding of current labour market needs and to predict the future programmes but also to the information system on education and external training providers. Data employment demand. from different sources should be used for decision-making, real-time analysis, and prediction of labour market trends. Currently, the LMIS is underused, mostly only for job search assistance and produc- In summary, at present, the NES has a well-developed and rich MIS and a good level of reporting based tion of statistics. It does not sufficiently inform processes such as decision-making, evaluation of pro- on its own data (administrative and survey data). However, it could usefully develop its MIS further and grammes, and adaptation of programmes, among others. The LMIS has high quality data which can be should aim to build an even more comprehensive LMIS in the medium term. To achieve this, the NES used to support rigorous evaluations and design data-driven decision processes. should, in the short term, aim to expand the base of jobseekers and firms who use the NES online portal and develop the capacity to obtain metrics such as net placement rates that would help assessing the effectiveness of NES interventions and policies. In the short to medium term, it should seek to cooper- Data used ate with other institutions so that data from different sources can be used to support decision-making At present, the NES uses data collected through its system, elements of CROSO data, data from the processes in the NES. In the medium to long term, it should aim to develop different segments of the employer survey, and data from the labour force survey. In this category, the NES system can be classi- LMIS based on big data (for example, career guidance services). fied as basic, and there is room for gathering more data for decision-making purposes. The NES should seek to develop partnerships for new sources of data from the private and public sector. 43 The NES does have information in its current system on dimensions of vulnerability (e.g., recipients of financial social benefits, single parents), but real-time synchronization with the Social Card Information System would ensure that NES data on vulnerability are not outdated. 36 NATIONAL EMPLOYMENT SERVICE NATIONAL EMPLOYMENT SERVICE 37 4.3. RECOMMENDATIONS TO IMPROVE USE OF LABOUR MARKET b) Introduction of the Labour Market Observatory (LMO) as the core system that provides de- INFORMATION CAPTURED BY NES scriptive data on the labour market that are mostly intervention and observation oriented, for the use by policymakers, stakeholders from other sectors, researchers and the wider public. While we observe that the current NES MIS is particularly strong in support to labour market services provision and general statistical reporting, the information it contains is probably underutilized for the 4.3.1. Improvements of the current NES MIS monitoring and evaluation of programmes managed by NES. The NES MIS can also evolve to provide The previous assessment has shown that there is room to further develop the NES MIS to support the more online support to its users and support more business functions such as procurement and pro- monitoring and evaluation capabilities in NES and of ALMPs and programmes implemented by NES vision of trainings. in general, to move some of its services online (based on the COVID-19 crisis experience) and to incor- The NES MIS captures and uses a vast amount of information on registered unemployed people, along- porate further capabilities. Building on the existing NES MIS functions, the following improvements side vacancies and programmes and interventions that are offered to its clients. While this informa- could be implemented in the short- and medium-term horizon: tion is used in the daily operations of the NES to facilitate the employment of registered unemployed people, it also constitutes a relevant source of information that can be used by other stakeholders, like 1. Improve NES MIS support for monitoring and evaluation policymakers, academia, education providers and the private sector. > Enable the setup of performance management at the NES by allowing monitoring of set of key per- While improving use of information within the NES would allow better job-matching functions, en- formance indicators, with built-in flexibility to report dynamically and at different levels of aggrega- abling broader use would improve measures of labour market efficiency and performance.44 Serbia is tion (corporate business intelligence). missing an intervention- and observation-oriented labour market information system that would: > Facilitate the impact evaluations of specific ALMPs to inform MoLEVSA and NES about the success > Provide timely and accurate information about strengths and weaknesses in the labour market; rate determining whether NES ALMPs have a causal impact on employment and the size of the effect. > Identify skills gaps; > Facilitate the process evaluation of specific ALMPs (cost-effectiveness analyses), which consists of looking at the relationship between resources, activities and outcomes; i.e., what the cost for the > Help firms to understand strengths and weaknesses of the available labour force; NES (per its branches as well) is in order to successfully place a jobseeker back on the labour mar- > Help the less-served population to access useful information to inform their education, training and ket. More in-depth analyses should also be facilitated by the MIS, which may additionally look into job search decisions; and efficiency of scale; that is, whether the available staff, expenditures or outputs such as activities are > Promote research in labour markets using up-to-date information to improve planning for future optimal to achieve the highest rate of return on the labour market. human capital needs and knowledge-based policy decision-making. > Accommodate key performance indicators in relation to the GRM process, such as the percent- This broader view of LMI use would provide different types of decision makers, such as workers, age of grievances resolved internally before they reach the media or courts; targets for closure of jobseekers, firms and policymakers, access to better information. While the NES uses its information grievances within a specified time frame; target of closure within the first two internal rounds of primarily for job-matching purposes, other stakeholders might find this information useful for other discussions, avoiding the need for external mediatory bodies; number of complaints expressed as a purposes, such as career guidance, educational or human resources decisions, and labour market monthly value for critical, major and minor grievances; and percentage change year by year of the analysis. number and degree of grievances. Consequently, while maintaining the current structure of the NES MIS, our recommendation is to see > Provide dashboards in relation to the monitoring indicators for the high-level monitoring and su- the NES MIS as one element in a set of cooperative systems that serve the purpose in a coherent and pervision of the ALMPs and the performance monitoring of the NES. coordinated LMIS architecture. Under such architecture, the NES MIS can be strengthened by making > Automatically generate reports with the monitoring indicators on a periodic basis and notify all the further use of existing information, but also new systems could be introduced to provide a wider audi- involved users (e.g., through email). The dashboards which are based on these reports should be ence with the Serbian LMI. automatically updated. Following this simple framework, Serbia should look for the improvements of its LMIS through invest- > Enable the involved users to compose dashboards based on the available reports and share them. ments into two key information systems: > Enable the involved users to access the available reports and dashboards and drill down for more a) Further functional and technical improvements of the NES MIS, to support monitoring and detailed information if required. evaluation of programmes managed by NES, more business processes and more online functions. > Enable the involved users to analyse the available data (including comparisons among datasets, trend analysis, etc.), generate descriptive analytics and design and produce reports supporting dif- 44 In times of economic crisis or economic recovery, such as the ones stemming from the COVID-19 pandemic, the availability of timely ferent levels of disaggregation (i.e., overall, sectoral, regional, and TS ) on the available data. information becomes even more crucial. 38 NATIONAL EMPLOYMENT SERVICE NATIONAL EMPLOYMENT SERVICE 39 > Enable the involved users to modify the monitoring indicators and the respective calculations, as of participants who are then evaluated in a second round. At the end of the selection procedure, the well as to add monitoring indicators and calculations as necessary. candidates are ranked and NES signs a contract with the selected candidates. The training providers receive a list with the names of the training participants. > Support the ability to set alert mechanisms (e.g., visual alerts, email alerts) when certain conditions are met in relation to the available data. > Support monitoring of the implementation of the training. The organizer of the trainings visits the training provider to monitor the implementation of the training. If the implementation is not > Enable the involved users to embed reports and dashboards with the monitoring indicators into proceeding as agreed in the contract, NES has the right to terminate the contract. NES pays for the the NES MIS portal as well as external portals, accompanied by the respective data, for use by the training of participants. Financial support is paid to the participants of the trainings. After the end external users. of the training, participants receive certificates or a confirmation of attendance. > Enable the NES monitoring and evaluation service to design online questionnaires and e-surveys > Reporting on the implementation of the training. After the end of the training, the training provider in relation to the LMPs (passive or active) provided by the NES and get feedback, either by a closed needs to provide additional information on the implementation of the training using a predefined group of beneficiaries, employers and service providers, or by the general public through the NES form. The training provider submits a list of certificates/confirmation of attendance that were is- MIS portal. The NES MIS should also support the ability to correlate a questionnaire/survey with a sued. The organizer of the training completes a report on the implementation of the training. specific LMP, as well as support export functionality for the data collected. > Archiving. Support digital archiving of all documents related to the specific training. 2. Support procurement and provision of training 3. Seek more online functions through an improved integrated online portal to provide infor- > Facilitate determining the labour market needs and determining the possibility to conduct specific mation for stakeholders and assure their effective interaction with the NES: trainings. The NES MIS should support each NES branch to collect available data on local labour markets and do analyses of the educational and professional characteristics of the unemployed and > The NES portal should be the single point of interaction between the jobseekers and the agency, and the available training offer of educational institutions at the local level. it should provide detailed information on the available LMPs (active or passive), available job va- cancies, allowances, news, announcements, publicity material, labour market/careers information > Support development of a draft list of trainings and a Training Catalogue. Based on the NAPS, fi- on jobs and skills in demand in the local and national labour market, available internships (without nancial plan and quotas for branches, the sector for employment support asks branches to submit specifying the employer’s name), detailed information about the training providers offering training their draft lists of trainings. Each branch needs to submit the following information for each train- services under the NES, detailed information about other service providers offering services under ing: name of training, location of training provider, number of participants, minimum educational the NES, FAQs, links to career development services, links to careers information attached to each requirements, number of hours of training, possible training providers and a narrative on the need of the ALMPs, obligations stemming from registration with NES and participation in LMPs (passive to conduct each individual training. The draft list of trainings is devised by using the following in- or active) and continuous provision of notifications, grievance redress procedures and timetables, puts: the information on the local labour market needs, structure of the registered unemployed, the and guidance concerning the introduction of an unemployed person into an ALMP and the relevant offer of training providers, available financial resources and quotas. Designated employees in the benefits. headquarters of NES check the adequacy of the proposed trainings and whether they are in line with other documents and the budget. If there are some required corrections, the branch offices correct > Enable the potential beneficiaries to create an account and log in to the NES MIS through the NES their lists, and the final outcome of this consultative process is the Training Catalogue. MIS portal. > Facilitate selection of training providers. Based on the Training Catalogue, designated employees > Enable the potential beneficiaries to apply for registration (re-registration) with the NES through an at the headquarters initiate the public procurement process. Branches are informed when the pro- online application form in the NES MIS portal. The application should be signed with the use of the curement process was initiated. NES employees pay visits to potential training providers. Finally, electronic signature service. after the public procurement process ends, NES signs a contract with the training providers and > Automatically connect with the available official databases/records to cross-reference the potential sends the contracts to NES branches. The NES MIS should facilitate the procurement process as beneficiaries’ information. described in Section 5. > Enable the potential beneficiaries to upload scanned copies of the confirmatory documents. > Support the management of the classroom-based vocational trainings at branches. After establish- > Enable the potential beneficiaries to arrange an appointment with a case worker from their local ing an initial contact with the training provider, the designated employees of the branch office TS in order to proceed with the profiling process. The system should provide a list of the available determine the target groups for the trainings, the selection procedure and the point system to rank appointment slots, and the potential beneficiaries should be offered three alternatives at least. the applicants. Afterwards, the plan of the realization of the training is prepared which includes all the characteristics of the training, it specifies the target group and the selection criteria. Case > Enable the NES managers to assign the appointments to the case workers. workers conduct a first preselection, and the selected candidates are instructed to attend an infor- > Notify the potential beneficiaries by electronic means (email and/or SMS) about the appointment mation session. After the information session, the so-called organizer of the training prepares a list details and ask for a final confirmation through an interaction form within the NES MIS portal. 40 NATIONAL EMPLOYMENT SERVICE NATIONAL EMPLOYMENT SERVICE 41 > Alerting, notification and information delivery mechanisms (email, web access, SMS, instant mes- 4.3.2. Introducing a Labour Market Observatory saging) to both the beneficiaries and the employers related to their appointment for a job interview. As noted, NES produces various reports that are used for planning and decision-making processes (re- > Support advanced search and matching capabilities through the NES MIS portal for both the em- port on implementation of annual work plan, monthly statistical bulletin, employer surveys and special ployers and the jobseekers. reports – described in more detail in Annex 2). However, the Serbian NES MIS provides basic labour > Facilitate the online communication (including text messages, video calls, chat, e-messages, social market information relying predominantly on NES data and somewhat on CROSO data. The CROSO networks, etc.) between employers and jobseekers, in case they both have an account in the NES dataset is a census of all formally employed individuals, which contains information on the characteris- MIS portal. tics of all formally employed individuals and their job contract conditions, together with the flows into and out of formal employment. > Enable the potential beneficiaries to create/update their online CV in the NES MIS portal (min- imum information: level of education, skills, competences, previous experience, certificates and More effort could be made to collect data on vacancies from private portals, while SMEs should be achievements). The information in the CV, as well as any update on it, should be confirmed by the approached to establish collaborative partnerships. More comprehensive analytics would be possible responsible case worker during a scheduled appointment. through the use of data from the country’s education management information system (EMIS). Data from the EMIS provides insight into the qualifications and occupations of new labour markets en- > Provide online access to the potential beneficiaries through the NES MIS portal, in order to be able trants. An additional relevant source is the Social Card Information System, which was developed by to check the status of their application, their appointment details, as well as have full access to their the Government of Serbia. The goal of this system is to provide a comprehensive picture of the social CV and personal file in the MIS. status of individuals and entities (family/household, municipality, region, country). One of its planned > Information concerning the eligibility criteria set by the NES for the inclusion of a service provider in functions will support the collection, systematization and reporting of data on vulnerable populations the NES registry of service providers, the application procedure, as well as guidance and FAQs. Share and groups of people facing particularly serious threats. The Social Card Information System could information about the activation measures the service providers can provide – i.e., vocational train- be used in real time to identify both vulnerable individuals and clusters of vulnerable individuals that ing, on-the-job training, lifelong/adult learning, and professional rehabilitation – while the NES MIS could be targeted by ALMPs.45 portal should provide the list of trainings to be provided by occupation/profession and display clear Although the necessary data are often available, there is no standard practice to continually track com- and detailed information about the certification/preselection process, and the application process. prehensive, cross-sectoral labour market status and trends. Surveys produce short-term indicators of > For employers: information, online guidance and FAQs concerning the offer of job vacancies and labour market trends, and there does not appear to be any agreed indicators and related methodologies the relevant benefits, labour market information (LMI) and information about the pool of regis- that ALMP implementers are expected to produce regularly, in order to be reviewed by policymakers. tered jobseekers. The NES produces regular statistics but there are no regular rigorous evaluations carried out to assess the implementation of LMPs (active or passive) and individual labour market programmes or to care- > Managing online counselling sessions on a regular basis and changes of the jobseeker’s status if fully identify their impacts. needed (e.g., achievement or certification that could change the individual action plan). Several public authorities (for example, MoLEVSA and MoESTD) and other organizations (e.g., indus- > The status of jobseekers’ CVs and latest upgrades based on their achievements. Active involvement try associations) also possess information systems containing data on the working situation, education- of the jobseekers in the continuous update of their CVs and other relevant information. They should al status and ALMP participation of the labour force. Although there are cases where data are available confirm every update of the CV and be able to update it with progress not related to the NES. The from these information systems, these data are not linked at the reporting layer of the NES MIS, in counsellor should always have the ability to supervise the process. order to provide a wider range of indicators for the monitoring and evaluation of LMPs, or to a further > Acquisition of certificates online, not only after completing a training programme but also on com- extent provide the means for monitoring the beneficiaries’ full pathway to employment between the pletion of training modules. various calls and their status after the completion of a programme. > Be capable of interfacing with social media (minimum: Facebook, Twitter, LinkedIn and Instagram) There is also currently an existing weakness in the interconnection and interoperability of the various and supporting the necessary functionality to promote information, news, publicity material and information systems involved in the LMPs, as well as a lack of a single digital space to gather all the announcements on them. available data concerning the LMPs. Involved labour market actors are forced to resort to ad hoc solu- > Provide chatbot functionality to facilitate the online interaction with the potential beneficiaries, the tions with the various public authorities.46 employers and the service providers. The NES does have information in its current system on dimensions of vulnerability (e.g., recipients of financial social benefits, single > Empowering of clients to demand high quality services by introducing online communication be- 45 parents), but real-time synchronization with the Social Card Information System would ensure that NES data on vulnerability are not tween jobseekers and case workers. outdated. 46 For example, the Government of Serbia is working on building a system through an initiative in the Prime Minister’s Office, based on > Client satisfaction report cards where clients can provide feedback on the quality of service they data collected by MoESTD and CROSO. The goal is to inform MoESTD about the job market placement and career paths of graduates have received. of specific programmes at secondary and higher education level. 42 NATIONAL EMPLOYMENT SERVICE NATIONAL EMPLOYMENT SERVICE 43 A more systemic approach to LMI analytics is obviously needed. It seems that the systematic solu- The LMO IS should build on existing NES MIS reporting and analytical modules but should develop tion cannot be deployed under the organizational borders and mandate of the NES. While the NES an institutionally and technically separated data warehouse and business intelligence system that shall seek for the improvements of the analytics and monitoring and evaluation based on the NES MIS collects and processes data not only from NES but also from other sources. The information to be database (as described in the NES MIS improvements section), Serbia will need a separate national provided to stakeholders will not be only in the form of reports and indicators – the data sets will be Labour Market Observatory (LMO) to resolve the LMI analytics in the wider sense. As defined in a provided in raw form (based on the rules for data use and privacy) for further processing. That will add World Bank policy brief:47 a completely new quality dimension to the LMI analytics in Serbia. “LMOs are institutions aimed at producing systematic information on labor market conditions in order Overall, the introduction of the LMO IS should serve the following business objectives: to help labor market actors make informed choices leading to better alignment between skills demand LMI repository and supply. Their main stakeholders are current and future labor market participants (i.e., jobseekers and students) who need timely and relevant information to make educational and occupational deci- > Create a data repository that feeds from NES databases and external databases. sions; intermediaries who help these end users make informed choices (i.e., public employment service > Create a platform for systematic and efficient dialogue of actors in the field of labour market and case workers and career guidance counselors), policymakers, especially in the areas of labor and educa- education in order to support the adjustment and correlation of education/training processes to tion, since the information provided by the LMO will be used to inform their respective policies; training labour market requirements. institutions, whose programme design will be based on the information available on labor markets; and investors and employers, whose investment decisions will also be informed by such data. Despite serv- > Facilitate the availability of information related to ALMPs’ operation, participation and results on ing multiple audiences, the key objective of LMOs is to guide individuals in making informed choices a timely basis to support decision-making in relation to the ALMPs and their role in supporting on skills development. LMOs around the world first aggregate relevant existing data on skills and labor unemployed workers and improving the operation of the labour market. demand (from employers) and supply (from the working-age population). Their core function is to then Labour market analysis analyze these data and to ensure that timely, relevant, and accessible labor market information reaches > Provide continuous and updated information on the labour market, quantitatively and qualitatively, their primary audience—jobseekers and students—as well as intermediaries, such as public employment with regard to supply and demand of professional competences on the labour market. service (PES) case workers and career guidance counselors. LMOs can also generate essential new data, for instance through tracer studies, conduct research that aims to promote a better alignment between > Publish analytical research relating to specific areas that are not covered by regular statistical data. the skill supply and demand, and convene stakeholders to improve collaboration between them.” > Analyse skills needs on the labour market and prospects of various professions on the labour market The LMO will need a strong LMO Information System (LMO IS). It should become the primary and their attractiveness. mechanism for fetching and dissemination of labour market information (LMI) and intelligence, build- > Elaborate on medium-term forecasts, foresights of labour market development, estimate the ing on existing fragmented data and analyses. The LMO IS will gradually address certain data exchange perspectives and the needs for various occupations/competences. gaps between labour market information systems and enhance the anticipatory focus of the outputs. It will make a difference by providing well-grounded responses on the underlying causes of skills mis- Policy-making and planning matches and propose solutions engaging all interested stakeholders, including both expert and non-ex- > Elaborate recommendations for concrete action and support policy decisions, and enhance the ef- pert users of labour market information. fectiveness of policies in different occupational categories of target groups (labour market inclusion The LMO IS will seek to integrate data and analyses contributing to an improved understanding of la- of young people, people from various disadvantaged categories), and support the participatory (ac- bour market and skills dynamics (short and medium term); foster cooperation and dialogue of partners tive) ageing of population. and stakeholders in making sense of the produced information; enhance coordinated data collection > Support the medium- and long-term strategic planning for utilizing the resources available to fund and production of needed analysis; disseminate readable information on labour market trends for so- ALMPs, both from the state budget (national resources) and other sources (co-financed resources). cial and economic decisions for the wider public. Monitoring and evaluation Finally, the LMO IS will facilitate the timely and systematic availability of data and data analyses on > Monitor and evaluate the public and private education/training system in order to determine the LMPs (active or passive) and the labour market, so that MoLEVSA can monitor LMPs, including se- compliance between the training delivery services system of vocational training and the demand for lected data on individual programme implementation and results, and use evidence to plan how to labour. utilize available resources to fund LMPs. The system will be aligned with the objectives of LMPs, as the system will be able to track LMP participation and its results. The system will be a tool for overall > Enable the high-level monitoring of active labour market policy, including, for example, the decision-making for labour and employment policy, including redesign of LMPs as needed. implementation and outcomes of the various programmes by making use of the monitoring data of individual programmes. 47 Rutkowski, Jan, Carmen de Paz, and Victoria Levin, Labor Market Observatories: Critical Success Factors, Job Notes, No. 4, World Bank, Washington, DC, 2018. https://openknowledge.worldbank.org/handle/10986/29594 44 NATIONAL EMPLOYMENT SERVICE NATIONAL EMPLOYMENT SERVICE 45 > Promote the integration of the systematic monitoring processes in relation to the design, im- plementation and reform of ALMPs. 5 PROCUREMENT OF ALMPS > Provide the monitoring data necessary to support evaluations (information on inputs, processes, outputs and outcomes to contribute to the evaluation of implementation [process evaluation] and As a public agency acting as a purchaser, the NES is obliged to implement the Public Procurement Law outcomes [impact evaluation] of these programmes), and, more specifically, provide evidence that (PPL) when procuring goods and services on the market. The PPL, adopted in 2012 and amended twice the necessary conditions for a programme to have an impact have been met and data on the param- in 2015, was, until recently, the applicable law. However, a new PPL, adopted in 2019, came into effect eters that define the intervention. from 1 July 2020. The transitional provisions of the new PPL define that all public procurement pro- cedures initiated by 30 June 2020 shall be finalized in accordance with the old PPL. Finally, the NES is obliged to harmonize its public procurement plan with the provisions of the new PPL since 1 July 2020 and, at the latest, before initiating the public procurement procedure in accordance with the provisions of the new law. Additional legal instruments governing public procurement conducted by the NES include the Law on Employment and Unemployment Insurance, the NES Statute, and the NES Rulebook on Closer Regu- lation of the Public Procurement Procedure. NES implements public procurement rules and procedures in procurement only in the case of class- room-based vocational trainings, as it hires external service providers for this type of ALMP.48 Service providers for classroom-based vocational trainings are legal entities engaged in adult education, of which a small number have been awarded the status of ‘Publicly Recognized Activity Organizer’ by MoESTD in accordance with the Law on Adult Education. There are not many accredited service providers because the accreditation process is highly bureaucratic and makes compliance difficult. Al- though new procedures have been introduced as part of the implementation of the National Qualifica- tions Framework, the new procedures are reportedly even more complex and costly than the old ones, and most training providers simply do not invest the time and effort required to secure accreditation. Consequently, these processes act as a disincentive for providers to develop new courses and pro- grammes that meet changing skills development needs.49 Since 2018, the NES has conducted public procurement of classroom-based vocational trainings at a national level.50 In past three years, the total average value of procured classroom-based vocational trainings on an annual level is approximately RSD 65 million (€550,000) for general trainings, which include educational trainings (languages, computer skills, etc.) and vocational trainings. For training of persons with disabilities, it is approximately RSD 20 million (€170,000).51 48 Note that NES calls this ALMP Labour Market Trainings, but in this report we refer to this type of skills development trainings as classroom-based vocational training, which is the standard term in the economic nomenclature. 49 Serbia Restructuring Systems of Lifelong Learning, World Bank, April 29, 2020. 50 Before 2018, public procurement of labour market trainings was defined and conducted for each NES branch office and particular training types. 51 The data are based on the information received from NES staff. 46 NATIONAL EMPLOYMENT SERVICE NATIONAL EMPLOYMENT SERVICE 47 5.1. PUBLIC PROCUREMENT PROCEDURE APPLICABLE TO THE CLASS- preparation of tender documentation. During this step, depending on the needs, MoLEVSA can be ROOM-BASED VOCATIONAL TRAININGS BASED ON THE OLD PPL52 actively involved in the provision of recommendations that may be related to the tendering process itself (for example, the definition of ‘Publicly Recognized Activity Organizer’ as one criterion for award of contracts) or identifying new development and unplanned occurrences in the labour market (exam- Figure 7. Outlines the steps in the public procurement procedure ples have included harmonization of the tendering procedure with a new law or regulation that has an impact on the planned trainings, or adjusting to the challenges of the COVID-19 pandemic). Amend- STEP 2: PREPARATION OF STEP 3: ASSESSMENT OF STEP 1: PROCUREMENT ments to the Training Catalogue can be made during this process (within the budget determined by TENDER DOCUMENTS AND TENDERS AND AWARDING PLANNING COLLECTING TENDERS CONTRACTS the Public Procurement Plan).56 In 2018, the invitation for submitting bids was published in May, while 1. Defining Training catalogue 1. Rejection of non-compliant in 2019 it was published in April.57 1. Defining mandatory/ additional 2. Adopting Procurement Plan selection requirements and contract tenders 3. Verification of Annual Work Plan awarding criteria 2. Applying contract awarding Preparation of tender documents and collecting tenders 4. Consultations with the Ministry 2. Submission of tenders and criteria and ranking of tenders material proofs 3. Contract awarding The NES is expected to prepare tender documents in accordance with the public procurement prin- ciples defined by the PPL.58 Data contained in the tender documentation must be identical to data contained in the invitation to submit tenders. Tender documentation, according to the type of proce- dure and the nature of the subject matter of public procurement, must contain mandatory elements Procurement planning prescribed by the PPL. The PPL prescribes mandatory requirements that all bidders must fulfil,59 while In accordance with the old PPL, the NES is obliged to adopt an annual Public Procurement Plan con- providing for the contracting authority to set additional service-related requirements in tender docu- taining the following information: (i) subject matter of public procurement; (ii) estimated value of pub- ments for participation in the public procurement procedure. Bidders prove that they fulfil the defined lic procurement; (iii) type of public procurement procedure; (iv) tentative date for initiating procedure; requirements by submitting material evidence for each requirement60 with their offer. This represents (v) tentative date for concluding contract; and (vi) tentative duration of contract. a significant administrative burden for the NES during assessment of tenders. In accordance with its Rulebook on Closer Regulation of the Public Procurement Procedure,53 every The NES conducts public procurement of classroom-based vocational trainings using an open procure- relevant NES department expresses its needs for procurement of goods, services or works within its ment procedure and publishes the invitation for submitting tenders to all interested parties. The invita- competence for the next calendar year and submits these to the NES Public Procurement Department tion must contain a list of information predetermined by the PPL. Simultaneously, the NES should also by 15 December of each year. The Public Procurement Department prepares a proposed procurement publish tender documents on the Public Procurement Portal and on its web page. In accordance with plan no later than 31 January and the plan is adopted by the NES Steering Committee. the PPL, the deadline for submission of bids in open procedure may not be shorter than 30 days from the day of publication of the invitation to submit tenders, in cases where the estimated value of public Classroom-based vocational trainings are represented only as a general item in the Public Procure- procurement of services does not exceed the amount of RSD 250 million (approximately. €2 million). If ment Plan, with concrete details defined through the Training Catalogue. The Training Catalogue is the NES amends or supplements tender documents eight days or fewer than the expiry of the deadline prepared internally in December to reflect labour market needs that the NES has identified for the for submission of tenders, it has to extend the deadline for submission of tenders and publish a notice following year. The Training Catalogue includes a list of trainings based on employers’ needs that could on this extension. not be realized during the year, anticipated market needs expected due to changes in the market (for example, introduction of a new law or amendments to existing regulations), professional preferences of Assessment of tenders and awarding contract unemployed people, and trainings that can be realized given the investors’ availability.54 In addition to the Public Procurement Plan and Training Catalogue, the NES adopts an Annual Work Opening of tenders takes place immediately after the deadline for their submission. Prior to assess- Plan, to which the Government gives formal consent. The Government usually gives its consent to the ment of tenders, the NES rejects proposals that do not comply with defined mandatory and additional NES programme of work in February of each year.55 requirements. In assessing proposals, the NES may apply only those criteria for awarding contract that After adoption of the Public Procurement Plan and verification of the Annual Work Plan, the NES 56 Information is based on interview with NES staff. can initiate procurement of trainings in accordance with the Training Catalogue, commencing with 57 Based on information from tender documents available on the NES website. 58 In preparation of tender documents and during the entire procurement procedure, the NES is obliged to comply with the following 52 The whole section is based on the provisions of the Public Procurement Law (‘Official Gazette of RS’ nos. 124/2012, 14/2015 and public procurement principles defined by the PPL: efficiency and cost-effectiveness, ensuring competition, transparency, equality of 68/2015) which is applicable until 30 June 2020. bidders, environmental protection, and ensuring energy efficiency. 53 Based on the provision of the NES Rulebook on Closer Regulation of the Public Procurement Procedure (consolidated text) dated 30 59 Mandatory requirements include registration with the competent authority, non-conviction of specific criminal offences, regular March 2017. payment of taxes, etc. 54 Information is based on interview with NES staff. 60 These proofs could be: excerpt from the register; certificates of the competent authorities; licenses; list of professional references; and 55 Information is based on interview with NES staff. other evidence of fulfilment of financial, business, personnel and technical capacities; etc. 48 NATIONAL EMPLOYMENT SERVICE NATIONAL EMPLOYMENT SERVICE 49 were defined in the invitation to submit tenders and in tender documents. The assessment criteria for Within five days of receipt of a duly completed request for protection of rights, the NES is expected to: awarding a contract and its elements need to be described and evaluated and should enable objective (i) accept the request for protection of rights; or (ii) deliver the request with NES’s elaborated response verification of tender evaluation. and all documentation to the Republic Commission to decide on the request for the protection of rights. The Republic Commission is obliged to decide on the request for protection of rights within 20 The PPL currently defines the following criteria for awarding a contract: days from receipt of complete documentation.62 1) Economically most advantageous tender, or 2) Lowest price offered. 5.2. OBSTACLES IN PROCUREMENT OF CLASSROOM-BASED VOCATIONAL The criterion of ‘economically most advantageous tender’ is based on implementation of various ele- TRAININGS63 ments such as: (i) offered price; (ii) discount to prices from the pricelist of the contracting authority; (iii) deadline for delivery or performance of service or works within the minimum acceptable deadline that does not compromise the quality; (iv) costs and cost effectiveness; (v) quality; (vi) environmental The NES has identified two major obstacles in public procurement for classroom-based vocational advantages and environmental protection; (vii) warranty period and type of warranties; and (viii) num- trainings: (i) a lengthy and complex procedure that must be repeated every year; and (ii) limitations in ber and quality of engaged staff. The NES assigns a relative significance (weight) to each element of selecting the highest quality service provider. the criterion in tender documents. The total sum of weighted points for these elements should be 100 Given all the stages in the procurement planning process for classroom-based vocational trainings, as points. The NES chooses from submitted tenders by ranking them based on weighted points. well as the mandatory minimum timeframes for collecting and evaluating bids, the process from the Based on its evaluation of tenders, the NES has to make a decision on contract awarding within 25 days point of identifying the labour market needs and preparation of the Training Catalogue to the point of from the opening of tenders, except in specially justified cases (such as the volume or complexity of execution of the trainings after selection of service providers can last six months. Moreover, if a request tenders, or complexity of the methodology of points allocation) where the term may be 40 days. The for protection of rights is submitted, commencement of trainings may be further delayed by up to two NES usually makes its decisions within this prolonged timeframe, due to the comprehensive tender more months. During this lengthy process, the situation in the labour market may significantly change, documents submitted and on-spot inspections that NES carries out during this phase. in such a way that the planned investors and adequate staff allocated to conduct the trainings are no longer available. In addition, this complex procedure is applied every year, even for trainings that are The NES publishes its decision on awarding the contract on the Public Procurement Portal and on its organized every year under the same or similar terms and capacities. web page within three days of making it. The NES should deliver the public procurement contract to the bidder to which the contract was awarded within eight days of the day of expiry of the time limit for The current PPL provides for NES the possibility to conclude a Framework Agreement after the con- filing the request for protection of rights.61 ducted procurement procedure, with one or more tenderers, for a period of up to three years (or two years where the Framework Agreement is concluded with one tenderer).64 The purpose of the Frame- work Agreement is to determine the general terms of the contract to be awarded during the defined Protection of rights period, relating to the price and quantities (where appropriate). This mechanism contributes to the A tenderer, applicant or candidate (i.e., interested person) who has an interest in an awarded con- flexibility and efficiency of the procurement procedure and provides the NES business continuity and tract in a specific public procurement procedure and who has suffered or could suffer damage due to confidence that, when the need for procurement arises, it will be able to implement this without a non-compliance with PPL rules and procedures may file a request for the protection of rights through- complex and lengthy procedure. However, the NES does not use Framework Agreements as a mecha- out the entire public procurement procedure, within the terms specified by the PPL. nism for procurement of classroom-based vocational trainings65 because: (i) the strategic documents governing implementation of ALMPs by the NES (i.e., NEAP, Training Catalogue) are adopted annually By law, in cases where the request for protection of rights has been filed, the public procurement pro- and therefore focus on only annual planning of ALMPs without projections for longer periods; and (ii) cedure is suspended until a decision on the submitted request is made. However, the NES may continue budget regulations66 restrict procurement of ALMPs through any mechanisms that assume an NES with the procurement procedure before adopting a decision on the submitted request in the following cases: (i) if retention of procurement activities would cause great difficulties in conducting NES work 62 Exceptionally, in particularly justified cases, this term may be extended by 15 days. The Republic Commission is obliged to deliver its decision to the interested parties within five days of the day of adoption. or business that are disproportionate to the value of the public procurement; or (ii) if the Republic 63 The whole section is based on the information received during an interview of NES staff and information available on the NES website, Commission finds that retaining the NES’s activities in the public procurement procedure would sig- as well as on NES’s experience with implementation of the current Public Procurement Law ‘Official Gazette of RS’ nos. 124/2012, 14/2015 and 68/2015. nificantly endanger the interest of the Republic of Serbia. In addition, if the request for protection of 64 The Framework Agreement mechanism is also recognized in the new Public Procurement Law (‘Official Gazette of RS’ no. 91/2019), rights is submitted after conclusion of the contract, the NES may not execute it until the final decision which became applicable from 1 July 2020. Moreover, the new PPL expands the total period of a Framework Agreement’s validity to four on the submitted request. years. 65 NES uses the Framework Agreement in limited cases, mostly in procurement of maintenance services. 66 Law on Budget System (‘Official Gazette of RS’ nos. 54/09, 73/10, 101/10, 101/11, 93/12, 62/13, 63/13, 108/13, 142/14, 68/15, 103/15, 99/16, 113/17 and 95/18); Decree on criteria for determining the nature of expenditure and conditions and manner of obtaining consent 61 If the chosen bidder refuses to sign the public procurement contract, the NES may conclude the contract with the next most for conclusion of certain contracts which, due to the nature of expenditure, require payment in several years (‘Official Gazette of RS’ nos. advantageous services tenderer. 21/2014 and 18/2019); Rulebook on standard classification framework and accounting plan for the budget system (‘Official Gazette of RS’ nos. 6/2016, 49/2016, 107/2016, 46/2017, 114/2017, 20/2018, 36/2018, 93/2018, 104/2018, 14/2019, 33/2019, 68/2019 and 84/2019). 50 NATIONAL EMPLOYMENT SERVICE NATIONAL EMPLOYMENT SERVICE 51 commitment that is longer than 12 months (such multi-year commitment is only allowed for procure- 5.3. IMPROVEMENTS WITH THE NEW PPL70 ment of services that are classified under specific account groups, which excludes ALMPs).67 Regarding the assessment criterion of economically most advantageous tender, the NES considers that The aim of the new PPL is harmonization with solutions from European Union (EU) Directives, es- the old PPL did not provide clear instructions or sufficient flexibility for selection of the highest quality pecially Directive 2014/24/EU of the European Parliament and of the Council of 26 February 2014 on service provider. Also, the NES considers that, without adequate standardization of training services, public procurement and repealing Directive 2004/18/EC (‘EU Directive’), and achieving better efficien- service price is not a good selection criterion because it does not guarantee quality of service. The pro- cy of public procurement procedures, their simplification, and reduction of administrative loads for cess of service quality standardization by awarding the status of a ‘Publicly Recognized Activity Organ- both the contracting authority and tenderers. It is also intended that the improved procurement system izer’ to training providers is still ongoing, but the number of accredited training providers is increasing. should lead to wider competition by ensuring greater participation of SMEs. From the perspective of The NES expects that standardization of training services will contribute to facilitating procurement of procurement of classroom-based vocational trainings, the new PPL introduces various procedural im- classroom-based vocational training services by simplifying assessment of a service provider’s quality.68 provements, outlined below. In addition to the ongoing accreditation process, the NES defines certain standards that each training programme must fulfil (for example, ratio of theoretical and practical classes/trainings, mandatory Higher thresholds for public and other specific services number of classes, teaching dynamics, number of trainees per group, providing written manuals and literature, etc.) to participate in the procurement process. However, even if service providers and their In accordance with the old PPL, the general procurement threshold under which the PPL does not ap- training programmes satisfy these predetermined standards, the NES has difficulties in choosing the ply is RSD 500,000 (approximately €4,000), while the new PPL introduces a higher threshold for public best quality service provider, specifically in relation to defining the most appropriate elements for as- and other special services, including educational and professional training services, of RSD 15,000,000 sessment of training quality pursuant to the economically most advantageous criteria. In 2019, the (approximately €128,000). Even though the threshold has been significantly increased, it may still not NES has introduced several additional elements (beside price) for quality assessment, including: (i) affect procurement of classroom-based vocational trainings given the average annual values of train- possession of an online platform for learning and interaction between trainees and teachers; (ii) use ings procured in recent years. Thus, it is expected that the NES will continue to apply rules and proce- of premises and teaching aids and availability of lecturers outside the training period; (iii) number and dures defined in the PPL for procurement of classroom-based vocational trainings. quality of hired staff; and (iv) status of a publicly recognized activity organizer.69 However, in assess- ment of these quality elements, NES is limited to objectively measurable facts that do not necessarily Special regime for award of contracts for social and other specific services guarantee the quality of the training programme or staff performing the training. For example, the first criterion is limited to determining whether the service provider possesses an online platform, rather The new PPL envisages a special regime to award contracts for the procurement of social and oth- than assessing its functionality. Likewise, quality of training providers is only based on years of expe- er special services listed in Annex 7 of the Law (which includes educational and professional devel- rience and formal qualifications and not on their actual skills in performing training. The effects of opment services). This special regime gives the NES an opportunity to implement procurement of these additional selection elements thus proved limited. It is still highly likely that the service provider classroom-based vocational trainings in a simplified procedure, by applying shorter deadlines than are formally meets all the conditions but still performs unsatisfactorily during the training. defined for regular open procedures. However, it will still be obliged to determine appropriate dead- lines for the submission of tenders, especially taking into account the complexity of the subject of pro- curement and the time required to prepare tenders, as well as the principles of transparency, equality of tenderers, and efficiency. In addition, the NES would be obliged to regulate, through a special act, the manner of planning and implementation of procurement of social and other special services. In accordance with the new PPL, the minimum deadline for submission of tenders in the open proce- dure is: (i) 35 days from the date of publishing the invitation to submit tenders, for public procurement with an estimated value equal to or higher than the amount of EU thresholds;71 (ii) 25 days from the date of publishing the invitation to submit tenders, for public procurement with an estimated value less than the amount of EU thresholds; (iii) 10 days for procurement of services with an estimated value 67 The specific services for which the regulations recognize NES’s option to make financial commitments for a period that exceeds 12 months are divided into two groups: (i) group 42 – Use of goods and services, which include: employees’ trainings (internally), 70 This whole section is based on the provisions of the new Public Procurement Law (‘Official Gazette of RS’ no. 91/2019) which becomes professional services (lawyers, auditors), maintenance services, administrative services, etc.; and (ii) group 62 – Procurement of financial applicable from 1 July 2020. assets. ALMPs are classified under group 47 – Social insurance and protection (specifically Training services through the National 71 European thresholds are value limits, defined in three EU directives related to public procurement: Directive 2014/24/EU, Directive Employment Service), which is excluded from the multi-annual commitment option. 2014/25/EU, and Directive 2009/81/EC. Above these thresholds, all member states and candidates for EU membership are obliged 68 According to information received from the market, training providers are reluctant to participate in the accreditation process, due to apply the provisions of these directives; i.e., to implement the provisions in their national legislation. The height of thresholds is to the complex and burdensome procedure. Therefore, growth in the number of accredited training providers might take place after determined by the European Commission, and they are changed every two years. The last change was 1 January 2020, when those simplification of the procedure itself. thresholds were somewhat lowered. The thresholds for goods and services are: for public procurers that are state bodies €139,000; for 69 This element does not have an important value (it is weighted with less than 10 out of 100 points) and it does not significantly affect the all other public procurers €214,000; for sectoral procurers and for procurement in the field of defence and security €428,000. European position of service providers without this status in the tender assessment process. thresholds for works are the same for all contracting authorities and public procurements and amount €5,350,000. 52 NATIONAL EMPLOYMENT SERVICE NATIONAL EMPLOYMENT SERVICE 53 below RSD 10 million (approximately €85,000). In urgent cases, the NES may set a shorter deadline The idea of introducing this statement is to reduce the administrative burden, both on tenderers and for submission of tenders than prescribed under (i) or (ii), but not shorter than 15 days from the date contracting authorities, associated with obtaining a significant number of certificates and other doc- of publishing the invitation to submit tenders. In addition, these deadlines could be shortened by five uments related to the reasons for exclusion and criteria for selection of service providers. This should days if tenders can be submitted electronically. Conversely, deadlines must be extended by five days, streamline the phase of bid assessment, so that the NES can focus on the contract award criteria. except in urgent cases, if tender documents are not available electronically. With respect to the phase of assessment of bids and awarding contracts, the new PPL stipulates that decisions on awarding contract Adjustments in applicable criteria for award of contract should be made within 30 days of expiration of the deadline for tender submission, unless the contract- ing authority has determined a longer deadline in the tender documentation. The new PPL implements a principle from the EU Directive under which the exclusive contract award Implementation of the new PPL will determine whether the rules on implementation of the special criterion in public procurement procedures is the most economically advantageous tender. The most regime for public and other special services would be more favourable for the NES in procurement of economically advantageous tender can be determined using three different approaches, which are un- classroom-based vocational trainings than the standard procedure. However, this special regime does der the disposition of the contracting authority: (i) based on price; (ii) based on cost, using a cost-ef- not affect the length of the procurement planning process. fectiveness approach such as life-cycle costing; or (iii) based on the best price–quality ratio, which is assessed using criteria (including qualitative, environmental and/or social aspects) linked to the subject matter of the public tendering contract in question. Such criteria may, for instance, comprise: Awarding the contract without competition > Quality, including technical characteristics, aesthetic and functional characteristics, accessibility, The new PPL extends the possibility for using a simplified process, in the form of a negotiated proce- solution for all users, social, environmental and innovative characteristics, and trading and its con- dure which does not require publication of an invitation to submit tenders, in cases of repetition of ditions. similar services. The NES would be able to use this type of procedure for new services that repeat sim- > Organization, qualification and experience of staff assigned to implement the contract, where qual- ilar services awarded to the service provider with which NES has already concluded the basic contract. ity of staff can have a significant impact on the level of performance. This can only be done if all the following conditions are met: (i) such services are in accordance with the basic project for which the basic contract has been concluded; (ii) the basic contract was concluded > After-sales service and technical assistance, delivery conditions such as delivery date, delivery pro- after the public procurement procedure in which the invitation for submitting tenders was published; cess and delivery period, or deadline for execution. (iii) the scope of possible services to be repeated and the conditions under which they are awarded The listed factors are provided on a non-exhaustive basis, and the contracting authority may apply oth- were specified in the procurement documents for the basic contract; (iv) the public invitation for the er criteria (factors) to determine the most economically advantageous tender, considering the specific basic contract envisaged the possibility of applying this procedure; (v) in determining the estimated circumstances of the public procurement procedure. In addition, the contracting authority is free to tender value for the basic contract, the total estimated value of the new repeated services has also been predetermine the relative value of each defined criterion (factor) used to award the contract. The con- taken into account; and (vi) this procedure is carried out within three years of conclusion of the basic tracting authority may also pre-prescribe the value of price or cost, to ensure that the most economi- contract. cally advantageous tender is determined only on the basis of quality criteria. This would provide the NES an option to continue cooperation in a simplified manner with services Notwithstanding the above, contract award criteria must be described and evaluated, non-discrimina- providers that have proved their quality during contract implementation for those classroom-based tory, linked to the public procurement contracts in question, and must allow for effective competition. vocational trainings that are usually conducted each year in the same or a similar manner. In addition, they should be defined in a way that enables objective verification and evaluation of ten- ders, as well as verification of data submitted by the tenderers, to assess the extent to which tenders Simplified method of proving the fulfilment of the criteria for the qualitative selection meet the contract award criteria. of service providers The possibility of introducing qualitative criteria for awarding contracts was included in the old PPL as well, although the new PPL does not give detailed instructions on selection and implementation of The new PPL introduces a formal statement on fulfilment of criteria as a mandatory mechanism for qualitative elements for assessing the most economically advantageous tender. However, it seems that proving the criteria for qualitative selection of a service provider (i.e., both mandatory and optional implementation of the principles for determining the most economically advantageous tender from grounds for exclusion, as well as selection criteria). This is similar to the European Single Procurement the EU Directive into the new PPL is intended to make it possible to apply good practices of EU coun- Document prescribed in the EU Directive. The statement must be submitted with the tender, in the tries in drafting and implementing qualitative criteria for awarding contracts. Such an approach could format prescribed by the Public Procurement Office, and serves as preliminary evidence in lieu of ex- potentially help the NES to widen the range of qualitative criteria and to improve the methodology of cerpts and certificates issued by the relevant authorities. At the end of the procedure, only the service assessment of these criteria to choose the best quality service provider. provider with the economically most favourable tender is required to submit certificates and other documentation as proof of fulfilment of the prescribed criteria. 54 NATIONAL EMPLOYMENT SERVICE NATIONAL EMPLOYMENT SERVICE 55 5.4. INTERNATIONAL EXAMPLES OF PUBLIC PROCUREMENT plex programmes. However, it is possible to award the contract in a procedure without competition for repetition of similar services, provided that a tender procedure with competition was initially im- plemented. This type of procedure requires approximately two months, and it is possible to award the Public procurement practice of the Austrian Public Employment Service (PES)72 contract to the best tenderer (of the competitive procedure) in this simplified procedure (in the event of successful and satisfactory implementation of the training programme) within three years of award Legal framework. The legal framework governing public procurement of ALMPs consists of: (i) Di- of the first contract. It is also possible to conclude a Framework Agreement with the best tenderer, for a rectives adopted on EU level (Concessions Directive 2014/23/EU, the Public Procurement Directive period longer than one year.73 This model can be chosen when it is unclear to what extent services will 2014/24/EU, etc.); (ii) Federal Procurement Act 2018 (BVergG 2018), which applies to all contracting be required and gives the PES more flexibility when the trainings are needed but the actual capacities authorities in Austria and adapts the national legal situation to EU law; (iii) Guideline of the Public requested are unclear.74 Employment Service Austria (‘BM1’), which regulates awarding of training programmes in the area of the PES Austria; and (iv) Federal guideline for the compensation of personnel and material expenses at Evaluation of tenders. The contract is awarded to the economically and technically (in terms of con- educational institutions (‘BM2’), which regulates accounting. tent) most favourable tender, in accordance with the specified award criteria and the weighting given, provided that the offer reaches the quality limit of 700 points. The maximum number of points that Procurement planning. The annual planning process for ALMPs for any given year starts in June can be achieved is analogous to the percentage weighting of 1,000 points, of which 400 points are usu- of the previous year. The relevant PES management is consulted on type of programmes and labour ally allocated to the costs of the tender. By rule, the following evaluation criteria and sub-criteria are market policy objectives (issued by the Ministry of Labour) that must be included. In addition, region- regularly applied in the context of the utility value analysis, with different weightings adapted to the al offices provide their feedback on each individual programme and whether a further or repeated respective educational offer: assignment is desired. Afterwards, three-day planning meetings are organized in September where participants (PES management, the specialist departments of the PES national office involved, and rep- 1) Quality of the personnel employed (weighted 30 per cent), with following sub-criteria: (i) formal resentatives of the regional offices) discuss each individual programme. At this point, the PES budget qualification (weighted 45 per cent); (ii) experience in the intended field of work (weighted 50 per (including ALMPs) available for the following year has not been determined, but after years of practice, cent); and (iii) proportion of gender among trainers according to measure hours (weighted 5 per the PES can assume that the budget will be similar to the budget for the current year or will take into cent). account known budget commitments and reductions. After internal approval of planned programmes 2) Conceptual quality, including gender mainstreaming, diversity and radar logic as cross-sectional by all required PES bodies, calls for tenders with different types of procedures are launched. There is issues (quality of the submitted concept of measures) (weighted 29 per cent). This which includes no fixed date when tendering should happen, as training courses can start at different times of the year. the following topics that are evaluated based on the European Foundation for Quality Management Therefore, tendering is done at different times, starting around six months before training starts. In (EFQM) model: (i) achieving permanently balanced and sustainable results; (ii) creating benefits March of every year, after the budget for that year has been approved, an additional planning day is for customers; (iii) managing processes and changes effectively and efficiently; (iv) implementing held to examine how the objectives and budget have developed, during which further ALMPs may be competence-oriented quality standards; (v) seeing employees as the key to success; (vi) leading with planned if funds are still available. During the entire planning process, the PES constantly exchanges vision, inspiration and integrity; and (vii) innovation, continuous improvement, good practice and information with the Ministry of Labour, which is involved in each PES decision. creativity. Procurement procedure. Procedural rules depend on the estimated contract value. For contracts up 3) Equipment and transport connections of the planned training location (weighted 1 per cent), with to €100,000, the service can be purchased directly from suitable and efficient providers. This is the the following sub-criteria: (i) areal equipment (weighted 50 per cent); (ii)  technical equipment simplest type of award procedure, lasting between four to six weeks. Above this value, a procedure (weighted 40 per cent); and (iii) transport connection (weighted 10 per cent). involving publication of a contract notice and competition of several tenderers must always be chosen. Competitive procedures usually take the form of a single-stage award procedure with publication of 4) Cost of the educational offer (weighted 40 per cent). a notice. There are specific rules of procedure for procurement of social and special services, which also include adult and other education services, and PES has a wide margin of manoeuvre in designing Austrian PES comparisons with the NES the specific tender documents for these services (for example, there is no obligation to use the types of award procedure specified by the law). However, these contracts must be awarded in compliance with Procurement planning. Like the NES, procurement of training programmes by the Austrian PES the principles of Union law, in particular equal treatment of all candidates and tenderers. The duration is a complex and lengthy procedure, involving many stakeholders (PES management, specialist de- of procurement procedures depends on the type of educational and training services required, as well partments, regional offices, and representatives of the Ministry of Labour). Planning for the next year as on the procedural technicalities of the PES regional office procuring the service. Tendering can take begins mid-year and runs to the end of the year. Afterwards, the regular procurement procedure for up to six months (which includes the publication notice, approximately four weeks for submission of tenders, opening of tenders, assessment, and awarding the contracts) especially for the new and com- 73 The maximum period provided by the law is four years. In exceptional cases it can be prolonged, but only with justified reasons. 74 The use of Framework Agreement model differs by regions. For example, the PES in Vienna has never used it as it traditionally has very large capacities. However, the PES in Styria has used it for German courses during the refugee crisis, since it was unclear how many 72 This section is based on the information received during an interview with PES staff. places would be needed. 56 NATIONAL EMPLOYMENT SERVICE NATIONAL EMPLOYMENT SERVICE 57 educational trainings takes 6 months, which can potentially create a gap of 6 to 12 months between Procurement planning. To start the procurement procedure, each HZZ regional office needs to pre- identification of a market need and implementation of training. However, unlike the NES, the PES has pare an annual Education and Training Plan (which includes a list of planned programmes, their dura- greater flexibility in terms of budget planning. It can commence procurement of trainings even before tion, number of participants, and corresponding costs), based on a local labour needs analysis. Due to the final budget for the next year has been approved (based on PES budget projections and experience a great diversity of adult training programmes on the market and to standardize the quality of trainings, from previous years). Afterwards, modifications of the planned trainings and their adaptations are HZZ prepares technical specifications for each educational and training programme in the Education possible when the PES receives the actual budget for that year (i.e., in March). Training procurements and Training Plan. Those specifications define minimum requirements that the programme must meet are performed several times a year, which enables the PES to respond better to actual market needs. In (type of programme, number of participants, enrolment requirements, duration of the programme, addition, the Ministry of Labour is actively included in all stages of planning, as well as in PES’s deci- hourly rate, teacher qualifications, and so on). Prior to preparation of tender documents, a preliminary sion-making, enabling efficient communication and prompt responses to adjustments of the planned analysis of the market and potential bidders is performed. In doing so, HZZ will consider the legislation ALMPs and their execution. governing the field of adult education and, if necessary, seek the advice of independent experts, com- petent authorities or market participants. Their expertise may be used in planning and implementing Procurement procedure. As in the Serbian case, the regular public procurement procedure is lengthy the procurement procedure and preparation of tender documentation, provided that such advice does and complex, so the PES commences procurement six months before the estimated start of training. not distort competition and does not violate the principles of non-discrimination and transparency. In However, for the purpose of procuring skills development trainings which have a status of social and addition, the Annual Procurement Plan is prepared, which includes procurement in education. Funds special services, the PES applies specific rules, which gives it more flexibility and a wide margin of from the Procurement Plan are an integral part of the HZZ’s annual budget, which is prepared for the manoeuvre in designing specific tender documents for these services. Moreover, it applies shorter and following year with additional projections for two years. Training programmes that are not foreseen in more simplified procedures and mechanisms for procurement of skills development trainings when- the Procurement Plan cannot be implemented. However, there may be some changes in the number ever it is legally possible (which enables it to avoid regular time-consuming procedures. These are: (i) of participants within the Framework Agreement (or sector), while the number of people involved in directly awarding service agreements when the training value is lower than €100,000; (ii) awarding education depends on the interest of the unemployed persons and the situation in the labour market. contracts in a procedure without competition for repetition of similar services; and (iii) concluding a Framework Agreement for a longer period where trainings are needed but the actual capacities re- Procurement procedure. Since 2017, HZZ has been procuring educational and training programmes quested are unclear. through an open public procurement procedure according to the criterion of the most economically advantageous offer. To start the procedure, HZZ sends a tender notice for each economic sector where Evaluation of tenders. Like the NES, the PES accords a high value to price in the context of the ten- trainings are planned. The notice contains information on the intention of concluding a framework der utility value analysis (400 out of 1,000 points). It also applies objective sub-criteria in evaluation agreement lasting up to two years, the number of planned groups, and other relevant information. Ten- of staff quality (i.e., formal education and professional experience). However, unlike the NES, the PES der documents are published in the Electronic Public Procurement Bulletin. Procurement of training evaluates the content and quality of training concept based on the EFQM model, which enables it to programmes is divided into economic sectors (for example, accommodation and serving of food and measure the strengths and weaknesses of training programmes offered in a more flexible manner and beverages, construction, metal industry, shipbuilding, and so on), and a separate procurement proce- provides an in-depth analysis of the training programme. Also, the PES allocates the highest number of dure is undertaken for each sector. Eligible providers (institutions, schools) may bid for one, several or points to quality of staff and the training concept (590 out of 1,000 points), and the fewest for technical all groups of training programmes for which a public procurement has been announced, provided that characteristics and equipment (only 10 points), which provides it a better opportunity to choose the the tender for a particular group includes all the training programmes listed within each group. Ten- best quality training provider and training concept. ders need only be submitted electronically. After publication of the tender documents, the usual period for submission of tenders is 15 days. After submission, tenders are analysed and evaluated according to Public procurement practice of the Croatian Public Employment Service75 the given criteria. The decision on selection of the best bidder is made within a maximum of 150 days from the expiration of the deadline for submission of tenders. The selected service provider is invited Legal framework. The procurement of training programmes by the Croatian Public Employment Ser- to submit updated supporting documents within five days, after which minutes on the review and eval- vice (Hrvatski zavod za zaposljavanje – HZZ) is carried out under the Public Procurement Act (Offi- uation of tenders are drawn up and the decision on selection is made. The standstill period (15 days) cial Gazette 120/16). The procurement process also relies on legislation in the field of education: the and appeal period (10 days) follow, after which a Framework Agreement is concluded with the selected Adult Education Act (OG 17/07, 107/07, 24/10); Rulebook on standards and norms, the manner and service provider. HZZ concludes a separate Framework Agreement with each provider for each group procedure for determining the fulfilment of conditions in adult education institutions (OG 129/08); of training programmes, for up to two years. During the period of validity of the Framework Agree- Rulebook on records in adult education (OG 129/08); and the Vocational Education Act (OG 30/09), ment and after the selection of candidates, the HZZ regional office signs individual public procurement among others. contracts with the selected training providers for each individual training programme. Evaluation of tenders. When setting criteria for evaluating the most economically advantageous offer, the selected criteria may differ somewhat from activity to activity, due to the nature of jobs for which 75 This section is based on the information received during an interview with HZZ’s staff and information from tender documents the training programmes are organized, the ratio of theoretical and practical classes, and other reasons. prepared by HZZ, which are available at https://eojn.nn.hr/Oglasnik/. 58 NATIONAL EMPLOYMENT SERVICE NATIONAL EMPLOYMENT SERVICE 59 The following evaluation criteria and sub-criteria are regularly applied in the context of the most eco- through a report on a survey and satisfaction analysis for the previous two years. The combination nomically advantageous offer: of quality recognition by state institutions and quality assessment based on satisfaction and practical experience of students can provide a solid basis for choosing the best quality training provider and 1) Price (weighted 20 per cent). training concept. 2) Professional qualifications of teachers or mentors (weighted 30 per cent), with the following sub-cri- teria: (i) work experience in the profession; (ii) teaching/mentoring experience; and (iii) pedagogi- cal–psychological education, master’s examination and/or additional qualifications. 5.5. SUMMARY 3) Methodology and organization of implementation of the educational/training programme (weight- ed 30 per cent), with the following sub-criteria: (i) proportion of hours of practical classes; (ii) pres- The NES has identified two major obstacles in conducting public procurement for classroom-based entation of the organization of teaching; (iii) strategies for assessing the progress of students; (iv) vocational trainings: (i) a lengthy and complex procedure which must be repeated every year; and (ii) envisaged possibility of individual consultations of students with the teacher; and (v) possibilities for limitations in selection of the highest quality service provider. The period between identifying labour independent learning, including online access to learning materials. market needs and execution of the trainings after selection of service providers can take up to six months (which can be further prolonged in cases where procedure for protection of rights is imple- 4) Quality assurance of implementation of the educational/training programme (weighted 20 per mented). This negatively affects NES’s responsiveness to labour market needs. In addition, the NES cent), with the following sub-criteria: (i) programme developed according to the old/new meth- repeats the procurement procedure every year because it is prevented from using Framework Agree- odology of educational programme; (ii) implementation of analysis of satisfaction of programme ments by its annual strategic documents (NEAP and Training Catalogue) and restrictions stemming participants; and (iii) ensured architectural accessibility for people with disabilities, etc. from budget regulations. It also experiences difficulties in choosing the best-quality service provider, particularly with defining the most appropriate elements for assessment of training quality pursuant HZZ comparisons with the NES to the economically most advantageous criteria. The quality criteria that NES has applied in the past two years have not proved sufficient, so there is still a high likelihood that the service provider formally Procurement planning. The HZZ planning process is very similar to the NES approach, as all major meets all conditions but performs poorly during training. aspects of ALMP procurement are defined in the Education and Training Plan (equivalent to the NES Training Catalogue) and Procurement Plan. Likewise, the standards for training programmes are set in The new PPL brings certain improvements to the public procurement procedure, which may facilitate advance, and trainings are fixed for a specific period. However, unlike the NES, HZZ plans its budget procurement of classroom-based vocational trainings for NES. This includes: (i) introduction of higher with a projection of two subsequent years, which allows extended training planning and commitments thresholds for implementation of public procurement rules for public and other special services (which for periods longer than 12 months. include educational and professional training services); (ii) a special regime for award of contracts for social and other specific services, which includes a simplified procedure and shorter deadlines; (iii) Procurement procedure. Like the NES, HZZ procures skills development trainings in an open pro- introduction of the possibility to use a simplified type of negotiated procedure without publishing the curement procedure, which is lengthy and complex. However, due to electronic publication and sub- invitation for submitting tenders, for repetition of similar services; (iv) a simplified method of proving mission of tenders, HZZ can shorten the period for submission of tenders to 15 days. As a rule, HZZ the fulfilment of the criteria for qualitative selection of service providers; and (v) implementation of enters into Framework Agreements with the best tenderer for up to two years, which reduces the principles for determining the most economically advantageous tender from the EU Directive (on the administrative burden and makes execution of trainings more efficient and adaptive to market needs basis of the best price–quality ratio, assessed on the basis of various sub-criteria including qualitative, since the public procurement process does not need to be repeated each year. environmental and/or social aspects). Evaluation of tenders. HZZ weights the training price in evaluation of the most economically advan- The practices of national employment services in the EU countries (specifically in Austria and Cro- tageous offer in a similar way to the NES. Also, HZZ applies objective sub-criteria in evaluation of staff atia) show that the length and complexity of the public procurement procedure could be overcome quality (i.e., formal education and professional experience) and organization of trainings (for example, by a more flexible procurement planning phase and implementation of simplified procurement pro- the proportion of hours of practical activities and classes, the possibility for individual consultations cedures and mechanisms. This might include: (i) direct award of contracts for contract values below of students with the teacher, and availability of an online teaching platform). However, unlike the NES, legally determined thresholds; (ii) contracts award in negotiated procedures without competition for HZZ identifies quality assurance of implementation of the educational/training programme as a spe- repeated services; and (iii) using Framework Agreements for procurement of trainings on a multi-year cial quality criterion and gives it a significant weight (20 per cent). This criterion is evaluated through level. When it comes to the definition and application of quality award criteria for selection of the two major sub-criteria. First is the methodology of the educational programme, which is assessed and best service provider, EU practice shows that most focus is dedicated to assessment of quality of train- verified by the state Agency for Vocational Education and Training and Adult Education using a legally ing programmes and training staff. In Austria, quality of training programmes is evaluated using the prescribed procedure that assumes a similar importance in recognizing the quality of the teaching pro- EFQM model, while Croatia uses a combination of quality recognition by state institutions and quality gramme as the verification of the status of a Publicly Recognized Activity Organizer in accordance with assessment based on satisfaction and practical experience of the students. Serbian regulations. Second is analysis of the satisfaction of programme participants, which is verified 60 NATIONAL EMPLOYMENT SERVICE NATIONAL EMPLOYMENT SERVICE 61 6 SKILLS DEVELOPMENT TRAINING learning platforms, including online, blended or traditional face-to-face. Lastly, PPPs in the Precision Training Framework context are contractual agreements between educational institutions and the pri- IN THE CONTEXT OF LIFELONG vate sector, where employees could benefit from formal courses, students could gain experience in workplaces, and curriculums could be influenced by the realities of workplace skills demands. LEARNING Figure 8. Precision Training Framework (World Bank, 2019) The NEAP is dedicated to advancing skills development, particularly through ALMPs, such as the job search assistance programme, subsidized employment initiatives, and a range of skills development trainings. However, as this report has demonstrated, these efforts are mainly targeted at helping the unemployed find work by job-matching, upskilling or reskilling efforts. While a stern focus on get- ting the unemployed into jobs is necessary, it forms part of a broader lifelong learning system that, if conceptualized and implemented well, could enable all citizens to seamlessly engage with formal and non-formal education and training throughout their lifetimes. Earlier in 2020, and in response to skills gaps in Serbia, the World Bank developed two technical notes to guide higher education reform and advance a lifelong learning system in Serbia.76 Lifelong learning embraces all forms of learning throughout life and is based on the premise that traditional educational pathways for young people through schools, colleges or universities do not provide them with all the knowledge and skills they will need throughout their lives. Therefore, people will continually need to enhance knowledge and skills to keep up with changing workplace demands. The ever-increasing developments in scientific and technological innovation, as well as renewed focus on the importance of soft skills – such as communication, creativity, leadership, time management and problem-solving – require employers and employees to adapt to the demand. Ultimately, a lifelong learning system reduces fragmentation and encourages a culture of learning that spans beyond formal schooling and post-schooling qualifications. For the Serbian context, creating a lifelong learning environment, where a person can re-enter education and training whenever the need arises to update skills, will be an important contributor to The two technical notes outline the necessity for such a framework to guide skills development in Ser- minimizing skills mismatches. In addition, with the support of a well-functioning LMIS and simpli- bia by exploring the extent of horizontal and vertical skills mismatches. Some examples of these skills fied procurement processes, such a framework would complement skills development efforts of the mismatches include that almost a third of recent tertiary graduates between the ages of 20 and 34 are ALMPs. In practice, this would mean that more relevant curriculums (informed by the private sector) unemployed, and a quarter of tertiary graduates are employed in semi-skilled professions. Moreover, would produce graduates whose skills are better aligned with the needs of the labour market. It would there is limited engagement in further education and training by employers and employees. These mis- further mean that employers, employees and education and training providers would eventually shift matches require systemic intervention. focus to skills as outcomes, rather than relying on broad qualifications as a proxy for competence. A useful conceptualization of workforce skills development is the Precision Training Framework (see Figure 8). This Framework was developed by the World Bank to support frontier economies high in productivity, innovation and income, to further advance competitive skills development. It brings to- gether three key avenues of skills development among employees – individualized learning, workplace training, and public–private partnerships (PPPs) – within a context of good governance, connectivity and an enabling business environment. Individualized learning is technology-driven and allows employees to upskill at their own pace and through a variety of teaching and learning platforms. Workplace training is employer-driven, involves some form of skills assessment and development plan, and takes place through a range of teaching and 76 World Bank, Serbia: Human Capital and Jobs, ‘Technical Note 1: Supporting effective higher education reform’ and ‘Technical Note 2: Restructuring systems of lifelong learning’, World Bank, Washington, DC, 2020. 62 NATIONAL EMPLOYMENT SERVICE NATIONAL EMPLOYMENT SERVICE 63 7 RECOMMENDATIONS 5) The sample size of the employer survey has been decreasing over the recent years, as it was difficult to allocate sufficient resources in the budget for this activity. The survey is and should remain a central tool to identify the skills needs and skills gaps. This instrument should be strengthened by aligning it regularly with other similar international surveys. Its representativeness at the district This report has given a wide overview of some of the main functions carried out by NES. It has focused level and sector level should not be compromised. on the use of labour market information data and the provision of labour market interventions, such as classroom-based trainings, which are subject to the national public procurement regulations. 6) The COVID-19 crisis has shown that the NES can move some of its services online, and this di- rection should be further explored as a long-term strategy. For example, offering registration of In order to implement many of these recommendations, additional resources will be necessary. Ex- unemployed people and processing claims for unemployment benefits are two processes that could panding the use of ALMPs should be coupled with improved information on which interventions are be offered online permanently through the secure NES portal. the most effective, have disproportionate effect on the most disadvantaged populations and are most efficient. At the end of the following list of recommendations we have included a matrix that provides 7) Steps should be taken to improve the NES MIS: a timeframe, priorities and responsible agencies to implement these recommendations. a) Improve NES MIS support for monitoring and evaluation by enhancing its capabilities to track 1) At 0.08 per cent, the share of GDP spent on active labour market programmes is low by internation- key indicators, use dashboards, and analyse data at various levels of disaggregation in real time, al standards and should be raised. The National Employment Strategy for the period 2011–2020 among other measures. envisaged increased spending on ALMPs to 0.5 per cent of GDP, and, as this number has not yet b) Improve NES MIS to support procurement and provision of training and serve as a tool to sup- been reached, it still provides a good initial target. port the process from service provider selection to follow-up of beneficiaries after completion of 2) At present, the NES is primarily providing job search assistance to its clients. The number of cli- training. ents in skills development trainings should be increased gradually, as this type of ALMP has the c) Improve NES MIS to support more online services through an integrated NES portal. best employment effects in the long term according to the international evidence. One strategy for 8) Introduce a Labour Market Observatory (LMO): achieving this could be by investing in public–private partnerships (PPPs). These partnerships can take on various forms and include apprenticeships, internships, learnerships and other forms of a) Create an LMI repository that feeds from NES databases and from external databases through work-based learning. In the bigger picture, such initiatives form part of a flexible lifelong learning cooperative agreements with other institutions. system proposed by the World Bank in a technical note earlier in 2020.77 If done well, PPPs can be b) Provide a wide audience with labour market analytics to support policy-making, planning and used to expand participation in skills development trainings of the unemployed, as well as engaging cross-sectoral monitoring and evaluation. the private sector more in guiding curricula of education and training to align skills, competences 9) The analytical capacity of the NES Department of Labour Market Analysis and Planning should be and knowledge with labour market demand. enhanced to use new NES MIS and LMO capabilities. The Department should be given the func- 3) In parallel to increasing the supply of skills development trainings, NES should improve the quality tion and responsibility to adapt ALMPs to local conditions, design interventions, carry out pilot of its analytical tools. It should conduct regular independent impact evaluations of its programmes. programmes, and discontinue programmes that the evidence shows to be ineffective. It should also incorporate new metrics to follow up the effectiveness of interventions in its moni- 10) To be more responsive to market needs, the NES should have more flexibility in planning and pro- toring cycle, such as measuring net placement rates of labour market programmes. The NES De- curement of skills development trainings. As a first step, it should be able to start with procurement partment of Labour Market Analysis and Planning should also use evaluations and results from processes early in the year, to avoid potential fluctuations of market needs and capacities, which employer surveys to make decisions to adjust, create or discontinue ALMPs. may happen due to the lengthy procurement procedure. This could be achieved if MoLEVSA be- 4) NES should envisage the use of statistical profiling as a tool to predict labour market disadvantage comes more actively involved in NES proposals for preparation of tender documentation immedi- and target individuals who are at risk to become long-term unemployed. ately after adoption of the Training Catalogue, to make it possible for the NES to start procurement immediately after final approval of the Annual Work Plan and Public Procurement Plan. Also, it would be ideal to enable the NES to commence procurement of specific, urgent skills development trainings even before the final adoption of the Public Procurement Plan and verification of the An- 77 Two technical notes were prepared in 2020: one on higher education reform and the other on the importance of a lifelong learning system that allows for seamless movement in and out of education and training opportunities throughout Serbians’ careers. The latter nual Work Plan, based on NES’s budget projections and experience from previous years. note proposes implementation of the Precision Training Framework, which was developed by the World Bank to support frontier economies high in productivity, innovation and income, to further advance competitive skills development. It brings together three 11) Given the complexity and length of the public procurement procedure, it is recommended that key avenues of skills development among employees – individualized learning, workplace training, and PPPs – within a context of the NES should analyse all the possibilities for applying a simplified procedure or a mechanism good governance, connectivity and an enabling business environment. Sources: World Bank, Europe, Skills for Competitiveness, Leveraging Skills for Competitiveness in Europe, World Bank, Washington, DC, 2019. http://documents.worldbank.org/curated/ for easier procurement of planned classroom-based vocational trainings that are available under en/657511543953734339/pdf/Nov-1-2018-Final-report-leveraging-skills-for-competitiveness-in-EU-P165578.pdf the new PPL. Specifically, the NES should, in cooperation with the Public Procurement Office, World Bank. Serbia: Human Capital and Jobs, ‘Technical Note 1: Supporting effective higher education reform’, and ‘Technical Note 2: explore if there is a possibility to single out specific or urgent classroom-based vocational trainings Restructuring systems of lifelong learning’, World Bank, Washington, DC, 2020. 64 NATIONAL EMPLOYMENT SERVICE NATIONAL EMPLOYMENT SERVICE 65 to exclude them from the mandatory procurement process where their estimated value is below 7.1. IMPACT MATRIX the prescribed threshold for public and other specific services. It is also recommended that, in following a wider EU trend, the NES be enabled to procure classroom-based vocational trainings Time horizon (short using Framework Agreements in some cases. This could be performed by adjusting the concept term <1 year; medium Expected Role players Cost of the strategic documents governing ALMPs (i.e., NEAP and Training Catalogue), as well as with term 1–3 years; longer impact term 3–5 years) modifications of the Public Procurement Plan concept and the budget regulations (either through Governance amendments to regulations or by giving specific interpretations to current provisions) to include High (if appointing the option of multi-year projections and commitments by NES regarding classroom-based voca- Addressing the analytical capacity of the new talent). Less if tional trainings. The NES should also use the opportunity provided by the new PPL to carry out Department of Labour Market Analysis and restructuring/realigning Planning to enable the Department to adapt Ongoing (continuous procurement of classroom-based vocational trainings in a simplified procedure available for public ALMPs to local conditions, design interventions, MoLEVSA (NES) evaluation of capacity existing capacity that High and other specific services (by applying shorter deadlines), as well as to implement the simplified might be displaced by carry out pilot programmes and discontinue needs) automation of certain negotiated procedure without publishing the invitation for submitting tenders, in cases of repeti- programmes that the evidence show to be functions/moving tion of classroom-based vocational trainings that are usually conducted each year in the same or a ineffective. certain services online. similar manner. The impact of COVID-19 Ministry of Finance; on the economy might 12) With respect to implementation of quality criteria for assessment of the most economically ad- Raise share of GDP spent on ALMPs to 0.5%. Treasury; MoLEVSA demand greater High High (NES); and other vantageous tender, the NES should assess the following options, which are based on the provisions investment in the short relevant role players of the new PPL and EU practice: to medium term Improved processes a) Predetermine the price/cost of classroom-based vocational trainings in tender documents, to be Expanding on current able to choose the most economically advantageous tender only based on quality criteria. MoLEVSA (NES); efforts (short term); b) In cooperation with MoLEVSA and the Public Procurement Office, explore the possibility of MoESTD; National introducing new Increasing the number of clients in skills Qualifications Agency; programmes aligned introducing quality assessment of classroom-based vocational training programmes based on development trainings. and stakeholders from with the Precision Medium to high High the EFQM evaluation model. public and private Training Framework c) Introduce analysis of the satisfaction of programme participants as a criterion/sub-criterion for sectors (medium to longer term) quality assessment of the classroom-based vocational training programmes. Using the NES online portal to improve job- NES; alternative private Short term Low Medium d) Dedicate more points to quality of staff and quality of training concepts (compared to technical matching efficiency. sector job portals and organizational elements) when weighting the criteria for assessment of the most economi- Short to medium term cally advantageous tender. Permanently moving some services online, such (some functions are Low (if infrastructure is as registration of unemployed people and claims NES already online as a Medium already in place) for unemployment benefits. result of the COVID-19 pandemic) Data and information Improve NES MIS support for monitoring and evaluation. Improve NES MIS to support procurement and NES Short to medium term Medium High provision of training. Improve NES MIS to support more online services through integrated NES Portal. Introduce Labour Market Observatory (LMO) to create LMI repository that feeds from NES NES; MoESTD; SORS; databases and from external databases through CROSO; Social Card cooperative agreements with other institutions. Information System; Medium to longer term Medium to high High Provide a wide audience with labour market and other relevant analytics to support policy-making, planning and stakeholders cross-sectoral monitoring and evaluation. 66 NATIONAL EMPLOYMENT SERVICE NATIONAL EMPLOYMENT SERVICE 67 Role players Time horizon (short term <1 year; medium term 1–3 years; longer Cost Expected impact ANNEX 1 STATISTICAL INFORMATION term 3–5 years) Conduct regular and continuous independent NES (Department of evaluations of the impact of programmes to Table A1. Characteristics of registered unemployed (in thousands, monthly averages) Labour Market Analysis Continuous Medium High inform decisions to adjust, create or discontinue and Planning) 2017 2018 2019 2020 ALMPs. Strengthening of the employer survey by aligning Short term (then Abs. NES Low High Abs. number % Abs. number % Abs. number % % it regularly to other similar international surveys. continuous) number Procurement Gender To enable a more flexible procurement system, Males 312 48% 275 47% 243 46% 228 45% the processes need to start early in the year; Females 339 52% 308 53% 287 54% 281 55% MoLEVSA needs to engage with tender proposals Age categories immediately after adoption of the Training Age 15 to 29 150 23% 126 22% 110 21% 106 21% Catalogue; and NES has to be able to commence procurement of certain trainings even before the MoLEVSA (NES); Public Age 30 to 54 383 59% 341 59% 307 58% 289 57% Short term Low Medium final adoption of the Public Procurement Plan and Procurement Office Age 55 and higher 118 18% 116 20% 113 21% 114 22% verification of the Annual Work Plan. Further, NES Education categories can simplify procurement processes by applying shorter deadlines, as well as implementing Primary school or less 213 33% 194 33% 178 34% 172 34% a simplified negotiated procedure without Secondary school 344 53% 304 52% 274 52% 262 51% publishing the invitation for submitting tenders. College or higher 94 14% 85 15% 77 15% 75 15% Implement an initial screening of applications Job search duration to determine whether certain trainings could At most 12 months 204 31% 184 32% 180 34% 177 35 be excluded from the procurement process – for MoLEVSA (NES); Public Short term (continuous Low Medium Between 1 and 5 years 244 37% 205 35% 176 33% 169 33 example, when their estimated value is below the Procurement Office evaluation) prescribed threshold for social and other specific More than 5 years 203 31% 194 33% 173 33% 163 32 services. Total 651 100% 583 100% 530 100 509 100 Enable NES to procure classroom-based vocational Source: National employment service data, author’s calculations. trainings by using the Framework Agreement MoLEVSA (NES); Public mechanism – where the option of multi-year Procurement Office; Short to medium term Low High projections and commitments by NES with MoESTD respect to classroom-based vocational trainings is included in policy. Align processes of selecting the most MoLEVSA (NES); Public economically advantageous tender with the Short term Low Medium Procurement Office provisions of the new PPL and EU practice. 68 NATIONAL EMPLOYMENT SERVICE NATIONAL EMPLOYMENT SERVICE 69 Table A2. Job search assistance: number of participants by characteristics (in thousands) Table A4. Private sector incentive programmes: number of participants by characteristics 2017 2018 2019 2020 2017 2018 2019 2020 Abs. Abs. Abs. Abs. Abs. Abs. number % Abs. number % Abs. number % % % % % % number number number number number Gender Gender Males 456 47% 471 46% 494 45% 431 45% Males 4,686 51% 5,661 50% 5,096 48% 3,963 50% Females 511 53% 545 54% 601 55% 536 55% Females 4,455 49% 5,738 50% 5,418 52% 3,969 50% Age categories Age categories Age 0 to 14 9 1% 8 1% 9 1% 4 0% Age 15 to 29 2,863 31% 3,201 28% 2,616 25% 1,909 24% Age 15 to 29 294 30% 282 28% 280 26% 249 26% Age 30 to 54 4,515 49% 6,042 53% 5,596 53% 4,171 53% Age 30 to 54 539 56% 571 56% 615 56% 531 55% Age 55 and higher 1,763 19% 2,156 19% 2,302 22% 1,852 23% Age 55 and higher 126 13% 154 15% 192 18% 183 19% Education categories Education categories Primary school or less 1,706 19% 2,289 20% 2,146 20% 1,674 21% Primary school or less 271 28% 295 29% 329 30% 284 29% Secondary school 5,519 60% 6,806 60% 6,292 60% 4,586 58% Secondary school 519 54% 537 53% 573 52% 499 52% College or higher 1,916 21% 2,304 20% 2,076 20% 1,672 21% College or higher 178 18% 183 18% 194 18% 185 19% Job search duration Job search duration At most 12 months 4,112 45% 4,867 43% 4,820 46% 3,870 49% At most 12 months 528 55% 537 53% 570 52% 512 53% Between 1 and 5 years 3,885 43% 4,652 41% 3,937 37% 2,968 37% Between 1 and 5 years 268 28% 279 27% 292 27% 255 26% More than 5 years 1,144 13% 1,880 16% 1,757 17% 1,094 14% More than 5 years 172 18% 200 20% 232 21% 200 21% Total 9,141 100% 11,399 100% 10,514 100% 7,932 100% Total 967 100% 1,016 100% 1,095 100% 968 100% Source: National employment service data, author’s calculations. Source: National employment service data, author’s calculations. Table A5. Public sector incentive programmes: number of participants by characteristics Table A3. Skills development trainings: number of participants by characteristics 2017 2018 2019 2020 2017 2018 2019 2020 Abs. Abs. Abs. Abs. % % % % Abs. number number number number Abs. number % Abs. number % Abs. number % % number Gender Gender Males 5,696 55% 4,627 54% 3,474 55% 2,995 54% Males 4,177 33% 4,250 37% 3,602 34% 5,893 39% Females 4,581 45% 3,873 46% 2,891 45% 2,591 46% Females 8,323 67% 7,154 63% 6,887 66% 9,356 61% Age categories Age categories Age 15 to 29 6,832 55% 6,640 58% 6,011 57% 12,712 83% Age 15 to 29 2,746 27% 2,005 24% 1,454 23% 1,232 22% Age 30 to 54 5,185 41% 4,357 38% 4,005 38% 2,271 15% Age 30 to 54 5,863 57% 4,916 58% 3,618 57% 3,208 57% Age 55 and higher 483 4% 407 4% 473 5% 266 2% Age 55 and higher 1,668 16% 1,579 19% 1,293 20% 1,146 21% Education categories Education categories Primary school or less 2,738 22% 2,041 18% 2,268 22% 1,484 10% Primary school or less 4,679 46% 3,885 46% 2,640 41% 2,362 42% Secondary school 4,918 39% 3,957 35% 3,980 38% 7,368 48% Secondary school 4,216 41% 3,646 43% 3,081 48% 2,712 49% College or higher 4,843 39% 5,406 47% 4,241 40% 6,397 42% College or higher 1,382 13% 968 11% 644 10% 512 9% Job search duration Job search duration At most 12 months 5,031 40% 6,751 59% 6,312 60% 10,479 69% At most 12 months 3,440 33% 2,781 33% 4,127 65% 3,478 62% Between 1 and 5 years 5,267 42% 3,368 30% 2,918 28% 3,880 25% Between 1 and 5 years 4,719 46% 3,615 43% 1,264 20% 1,348 24% More than 5 years 2,202 18% 1,285 11% 1,259 12% 890 6% Total 12,500 100% 11,404 100% 10,489 100% 15,249 100% More than 5 years 2,118 21% 2,104 25% 974 15% 760 14% Total 10,277 100% 8,500 100% 6,365 100% 5,586 100% Source: National employment service data, author’s calculations. Source: National employment service data, author’s calculations. 70 NATIONAL EMPLOYMENT SERVICE NATIONAL EMPLOYMENT SERVICE 71 Table A6. Geographical distribution of participants by branch and ALMP type in 2019 Skills development Private sector Public sector ANNEX 2 MAIN REPORTS PUBLISHED BY NES Branch Job search assistance Classroom trainingsa training incentives employment Belgrade 168,593 890 252 969 182 Kragujevac 64,447 434 126 512 222 Niš 68,518 934 88 963 500 NES produces various reports that are used for planning and decision-making processes. The National Novi Sad 97,924 901 98 941 314 Employment Strategy for 2011–202078 provides the framework, strategic direction and priorities for Priština 1,441 0 0 0 0 employment policy. The Strategy envisages the establishment of an efficient, stable and sustainable Bor 16,944 190 47 197 230 trend of employment growth in the Republic of Serbia by the end of 2020, together with the harmo- Valjevo 21,848 113 10 275 207 nization of employment policy and labour market institutions with the Acquis Communautaire. The Vranje 52,536 350 35 343 139 National Employment Action Plan (NEAP), prepared by MoLEVSA and adopted by the Government Zrenjanin 23,838 192 25 140 147 of the Republic of Serbia annually, sets out a plan to implement goals from the Strategy and defines Kikinda 22,923 196 22 163 132 Kosovska Mitrovica 12,396 475 25 142 185 annual goals of employment policy with corresponding ALMPs and activities for the year. Kraljevo 31,164 647 55 437 200 Key strategic and operational documents of employment policy in the Republic of Serbia are drafted by Kruševac 37,022 420 44 459 246 the Working Group for drafting the NEAP. This working group consists of representatives of the fol- Leskovac 39,986 668 38 494 394 lowing ministries: Ministry of Economy, Ministry of Finance, Ministry of Youth and Sports, Ministry of Pančevo 25,629 202 50 159 142 Education, Science and Technological Development; other institutions: Republic Bureau of Statistics, Požarevac 20,904 144 32 150 220 Republic Secretariat for Public Policies, Team for Social Inclusion and Poverty Reduction, Standing Sombor 28,805 191 48 330 162 Conference of Towns and Municipalities and National Employment Service; social partners: Union Sremska Mitrovica 35,292 445 77 353 153 of Employers and representatives of two representative unions and the Serbian Chamber of Com- Subotica 18,393 73 29 212 93 merce. The working group is an exemplary case of social dialogue within the employment policy and Užice 24,766 294 36 308 136 a sustainable and efficient mechanism of cooperation between the different stakeholders. During the Čačak 30,361 282 47 368 184 meetings and/or via email, the working group members exchange information and collect suggestions Šabac 23,231 154 16 252 138 for the new NEAP. Additionally, local government units are an important partner of the national level Pirot 17,496 178 33 215 218 of government in this whole process. The consultative process takes place annually in two cycles: first, Prokuplje 22,691 255 25 315 419 after the adoption of the NEAP to inform about the newly adopted ALMPs and the local NEAPs; and Gnjilane 5,920 84 10 42 55 second, during the process of drafting a new NEAP. Zaječar 14,966 222 72 188 125 Smederevo 21,874 180 35 391 141 The first part of the NEAP provides a thorough analysis of labour market trends of previous years, with Jagodina 35,174 369 42 271 248 special attention given to analysis of the labour market situation of the larger target groups mentioned Novi Pazar 42,854 359 35 229 161 previously. The second part of the plan defines all the planned ALMPs for the given year. In preparing Loznica 27,663 243 22 318 320 this, all existing evidence, such as available reports (for example, Implementation of the Annual Work Prijepolje 24,502 250 15 254 275 Plan) and experience of employers with current ALMPs, is considered. Employers have an opportunity Vršac 15,151 154 29 124 77 to say what has and has not worked well, which is considered in further development of ALMPs. As a Total 1,095,252 10,489 1518 10,514 6,365 result, the ALMPs can be changed based on evidence and input from members of the working group. The following are the four main public reports published by NES: Source: National employment service data, author’s calculations. Note: Classroom-based vocational trainings are a subtype of skills development trainings, and they are the only ALMP which is provided externally. Classroom-based vocational training providers are selected through procurement procedures. Annual Work Plan The Annual Work Plan is adopted at the beginning of each year by the NES. It operationalizes the NEAP. The NEAP is adopted in December for the following year, after which the Annual Work Plan is adopted. The Annual Work Plan is a public document, divided into three parts, and it explains each 78 In the process of writing this report, the Employment Strategy for the period 2021–2026 with Action Plan for 2021–2023 has been published. 72 NATIONAL EMPLOYMENT SERVICE NATIONAL EMPLOYMENT SERVICE 73 activity to be performed by NES in detail. For each activity, the Annual Work Plan specifies goals, tar- Employer surveys get groups, implementation and dynamics, expected results, and the responsible department. The first part is dedicated to ALMPs and it describes in detail each ALMP offered by NES and the correspond- An employer survey is conducted annually to identify mismatches between demand and supply of ing activities. It further reports the number of individuals expected to be included in each programme labour in terms of occupations and skills. This survey collects information from employers on the for the given year. The second part describes processes and procedures for unemployment benefit demand for certain occupations, qualifications and skills of those hired in the current year and those claims and other financial claims. The third part describes all supporting activities79 of NES in detail. expected to be hired in the following year. Moreover, employers are asked which occupations are cur- rently in declining demand or are expected to be so in the coming period. Results from this survey are used to plan externally commissioned skills development trainings. Based on its findings, existing Report on the Implementation of Annual Work Plan labour market trainings are modified and/or completely new trainings can be created. The Report on the Implementation of Annual Work Plan reports the implementation of each activity The sample size of the employer surveys has decreased from 5,55482 in 2015 to 1,48483 in 2019. Howev- foreseen in the Annual Work Plan that NES performed in a given year. It is adopted by NES at the be- er, it is still representative at the national, regional and district level (districts coincide with the branch ginning of each year for the previous year. Like the Annual Work Plan, it is in three parts: active labour offices). In 2018, the survey was not conducted at all. From the reduction of the sample size over the market policies; unemployment benefits and other financial benefits; and supporting activities. In the recent years, it appears that NES is struggling to allocate necessary financial resources to conduct a first part, the document reports, for each ALMP, the purpose and goals, the planned target and actual high-quality survey with employers. However, it is the primary tool available to identify skills needs number of participants, and a review of implementation of each activity. The second part on unem- and gaps, giving crucial input for the design of externally commissioned classroom-based vocational ployment and other financial benefits reports on the number of claimants and the size of benefits. The trainings and assessment of local labour market needs. Thus, there is a need to strengthen the quality of third part reports on implementation of each supporting activity, and these findings are considered the surveys and ensure they are conducted regularly. Alternative scenarios such as outsourcing design when developing the new NEAP. or data collection and cooperation with the SORS could be explored to reduce costs. Monthly Statistical Bulletin Special reports The Monthly Statistical Bulletin of NES provides the numbers and structure of unemployed and em- MoLEVSA commissions external consultants on an ad hoc basis to undertake evaluations of NES ployed persons and the numbers of registered job vacancies. It aims to provide an overview of the ALMPs targeting a specific population category. Past evaluations have examined the impact of ALMPs labour market situation for the given month. The Monthly Statistical Bulletin is not only an instru- on Roma, youth, women and other target groups. Each report is independent, and they often use very ment designed for NES employees80 to monitor the labour market situation, but also informs other different methodologies; for example, some look at gross impacts and others at net impacts, while stakeholders, including other government institutions, researchers and the public. The report relies some are qualitative and others quantitative. Usually, these reports evaluate specific programmes tar- on data from NES, specifically on registered unemployed and on vacancies gathered from employ- geting the population of interest and estimate the effectiveness of these programmes. MoLEVSA uses ers reporting to NES, as well as CROSO data on employment and recruitment. The following tables these reports to readjust the ALMPs. are contained in the report: i) general data on unemployment, job vacancies and employment dis- aggregated by gender and educational level;81 ii) occupation of the unemployed and most frequently demanded professions by educational level; iii) unemployment, job vacancies and employment disag- gregated at the district level and further disaggregated by gender, educational level, age and duration of unemployment; and iv) number of recipients of unemployment benefits disaggregated at the level of gender and educational level. 79 These include public information and publishing, projects and international cooperation, IT system, labour market analysis, quality management system, human resources, material resources, financial affairs, legal affairs, and internal audit. 80 NES employees from the Department of Labour Market Analysis and Planning and associates for statistics in the branch offices. 82 In 2015 7,017 companies were contacted and 5,554 were surveyed. The response rate was 79.2 per cent. 81 Gender and educational level. 83 In 2019 2,500 companies were contacted and 1,484 were surveyed. The response rate was 58.4 per cent. 74 NATIONAL EMPLOYMENT SERVICE NATIONAL EMPLOYMENT SERVICE 75 ANNEX 3 CONCEPTUAL DEFINITIONS The LMIS uses information systems as technical tools to fulfil its purpose. These subsidiary systems are cooperating and exchanging information. Interfaces to subsidiary systems are tailored to different OF LABOUR MARKET INFORMATION types of users (youth and workers, employers, intermediaries, students, policymakers, researchers) and provide support specific business needs, such as: job-matching services; career and skills guidance; SYSTEMS information on vacancies, salaries and job prospects; information on education and training opportu- nities; pathways to apprenticeships; information on local, state, national and regional labour markets; relevant research and analyses on labour market and industry trends, skills shortages and needs, etc. Over the past decades, a major shift has been observed in the institutional and organizational culture In general, subsidiary systems of an LMIS can be conceived to fulfil one of the two following core sets of national employment agencies worldwide as most public employment services (PES) have moved of functions:86 away from being a self-sufficient organization, to one that sets out to provide more dynamic and flex- a) Service-oriented information systems that provide labour market services that are demand and ible client-centred services and develop and manage working relationships with other agencies, both supply oriented. They provide information to workers and employers (and labour market interme- public and private, in the delivery of employment services. To support such transformation and related diaries) to enhance their efforts to improve their work situation or their labour force, respectively. changes in business processes, the agencies stand-alone management information systems (MIS) have b) Data-driven information systems that provide descriptive data on the labour market that are mostly evolved or become part of a much broader labour market information system (LMIS). intervention and observation oriented. They produce information describing the situations that To understand the purpose of the LMIS, we will first refer to the labour market information (LMI). prevail in the labour market. Such systems are especially useful for policymakers and for designing Depending on its applicability, labour market information has four main purposes:84 interventions aimed at improving the situation in, or the functioning of, the labour market. They typically build on a set of statistical indicators such as macro-level labour market performance indi- 1. Supply-oriented information is used by workers to improve their standing in the labour market cators including unemployment rates, new job formation by sector, information on labour market (e.g., find work, improve their skills). demographics, etc. 2. Demand-oriented information improves employers’ ability to hire efficiently, to become more successful in optimizing new hires, or to improve the capacity of their human resources. 3. Intervention-oriented information caters to decision makers and other labour market stakehold- ers who work to improve the functioning of labour markets (i.e., removing obstacles to employ- ment, distortions, inequities and facilitating good quality jobs, labour market access and training). 4. Observation-oriented information serves a general purpose for overall research on the labour market to contribute to the study of the economy and society. The LMIS can be defined as an ecosystem of organizations, individuals, procedures and technical tools that produce, store and utilize LMI in order to support provision of labour market services, de- cision-making, and formulation and implementation of labour market policies.85 This means that an LMIS is not one technical system, not even set of technical systems: it is more a network of information needs and flows between stakeholders. 84 Sorensen, Kjartan, and Jean-Michel Mas, ‘A Roadmap for the Development of Labor Market Information Systems’, Workforce Connections and USAID, August 2016. 85 We will use this definition as it includes support to provision of labour market services. Definitions that can be found in the literature are typically oriented to data collection and analytical dimension of an LMIS. For example: “A Labour Market Information System … can be defined as a set of institutional arrangements, procedures and mechanisms that are designed to coordinate the collection, processing, storage, retrieval and dissemination of Labour Market Information.” From: Mangozho, Nicholas, ‘Current practices in labour market information systems development for human resources development planning in developed, developing and transition economies’, EMP/SKILLS Working Paper No. 13, International Labour Office, Geneva, January 2003, retrieved 28 Sept 2020, from: http://www.ilo. org/skills/pubs/WCMS_103881/lang--en/index.htm Another example is an ILO definition that can be retrieved from: https://ilostat.ilo.org/resources/labour-market-information- 86 Sorensen, Kjartan, and Jean-Michel Mas, ‘A Roadmap for the Development of Labor Market Information Systems’, Workforce systems/ Connections and USAID, August 2016. NATIONAL EMPLOYMENT SERVICE