Technical support to the Ministry of Labor and Social Affairs of Greece and the Greek Manpower Employment Organization (OAED) GREECE: IMPROVING THE DESIGN AND DELIVERY OF ALMPS – Phase II MONITORING REPORT #2 – ELEFSINA PILOT PROGRAM April 27, 2021 Administrative Agreement of February 11, 2019 EC Contract No SRSS/S2019/012 World Bank Report No. AUS0002197 Funded by the European Union via the Structural Reform Support Programme and in cooperation with the European Commission's Directorate-General for Structural Reform Support This report is a product of the Social Protection and Jobs Unit at the World Bank. It was prepared in the context of the Technical Assistance for Improving the Design and Delivery of ALMPs – Phase II, funded by the European Union via the Structural Reform Support Program and in cooperation with the European Commission's DG REFORM. The report corresponds to Output B.4: Second Monitoring Report of the Elefsina pilot program, under Administrative Agreement of February 11, 2019; EC Contract No SRSS/S2019/012; World Bank Project Number P168146-TF073206. The report has been prepared by a World Bank team led by Stefanie Brodmann and including Polly Jones, Ioanna Pantelaiou, Gordon Betcherman, Cornelius Claus Von Lenthe, and Giannis Tzimas. The team is grateful to the Greek Manpower Employment Organization (OAED) and particularly to Mr. Dimitris Zissimos from OAED’s IT department, for the close cooperation throughout the technical assistance project. The team also appreciates the support of Dorina Giouroukou, coordinator of the technical assistance for DG REFORM, and managerial support from Cem Mete, Goran Tinjic and Nikolaos Schmidt at the World Bank. This report is a product of the International Bank for Reconstruction and Development / The World Bank. The findings, interpretation and conclusions expressed in this paper do not necessarily reflect the views of the Executive Directors of the World Bank, the European Commission, or the Government. 2 Highlights ✓ Out of the 3,617 individuals eligible to participate in the Elefsina pilot as of September 2019, through March 2020, 948 individuals underwent the profiling exercise and met with a counselor. This translates to a take-up rate of around 26 percent. ✓ Through March 2020, 38 pilot participants entered wage subsidy positions and 893 individuals were referred to training. ✓ Through October 2020, 80 participants entered wage subsidy positions, close to half of which entered in the second half of 2020. Considering the timing, these entries are likely related to the Greek Government’s response to COVID-19. ✓ Through October 2020, 900 pilot participants were referred to training, of which 840 received a voucher for technical training and 781 activated them and became trainees. ✓ The most commonly offered training specialties are office clerk, retail salesperson, building guard, warehouse worker and bartender. ✓ Nearly 80 percent of trainees in specialties for which OAED made certification arrangements have completed the certification assessment and passed. The only trainees not successful at the assessment were 4 in the forklift driver specialty (out of 7 assessed). ✓ Over a quarter of basic skills voucher recipients (41 out of 151), failed to enroll in courses, suggesting the need in the future for follow-up of these individuals to encourage them to take classes so that they can benefit from skills training. ✓ According to ERGANI Data on employment status, program participants were more likely than non-participants to have found a job and less likely to be in the unemployment registry in October 2020. These findings are only suggestive of the positive results of the pilot, since they are not the product of an impact evaluation. Note: The data used in this report are from OAED’s main management information system (called OPS) and the demand-responsive training program (DRTP) Dashboard. With the data received from OAED, the World Bank team was able to update part of the indicators from the first monitoring report (covering November 2018 – March 2020) for this longer time period (through October 2020). Following the first monitoring report, OAED did not update the data for the ALMPS Dashboard, the source for some monitoring indicators (for example, related to appointments with counselors and wage subsidies). Both the DRTP and ALMPS dashboards are a temporary solution until the ALMP M&E System is functional. The findings on outcomes presented in the last section of the report are based on administrative data from OPS and ERGANI (the Information System of the Ministry of Labor and Social Affairs) and analysis by World Bank staff. 3 1. Introduction The first phase of the new ALMP model was initiated with a pilot program in three municipalities (Elefsina, Asporpyrgos, and Mandra) that are covered by the local employment office (KPA2) of Elefsina. The pilot is an opportunity to test an enhanced case management approach, where the job seeker receives personalized services, including profiling, individual action plans (IAP), and counseling, and to introduce innovations in ALMP design. The previous monitoring report covered the period from November 2018 – March 2020. This report focuses on the period from April 2020 to October 20201. It was possible to update most information on training as more recent data from the DRTP Dashboards was available. This was not the case for other sections (wage subsidies, for example) because of the unavailability of updated data for the ALMPS Dashboard 2 . For this reason, some information is repeated from the previous report in cases where substantial changes are unlikely (target group and characteristics of pilot participants). The report concludes with a section on outcomes, making use of data from ERGANI. 2. Target Group3 The target group for the pilot are people 45 years old and over registered as unemployed at the KPA2 of Elefsina for at least 6 months. Table 1 shows the number and characteristics of the target group as of August 2018, a couple of months before the pilot started, while Table 2 shows the same in September 2019. The size of the target group increased by about 10 percent between August 2018 and September 2019, while its composition changed little.4 The share of more recently unemployed, women, and GMI recipients rose slightly. Several reasons may explain the changes in the size of the target group over time. For example, new individuals may have registered as unemployed for the first time, the period of unemployment for already registered unemployed may have exceeded 6 months, or some who had been unemployed for 6 months may have found jobs, among others. 1 It was intended for 2 monitoring reports to be prepared in 2019 in order to track progress and identify issues that needed to be addressed during pilot implementation but was not possible owing to difficulties in accessing OPS data. 2 OAED was not able to provide the data needed to update the ALMPS Dashboard due to other critical tasks. 3 Data on the target group is the same as presented in the previous monitoring report. 4 As for data on the target group after September 2019, data from October 2020 is available. However, these data cannot be used as a base for analysis given that they might be affected by COVID-19 and that they might include some unemployed who had exceeded 6 months unemployment duration only by late 2020, by which time pilot activities were winding down. OAED was not able to provide more frequently updated data on the target group. 4 Table 1: Profile and Size of Target Group (August 2018) Total Percent Men Women Education Level Unclassified/blank 10 .3% 3 7 Less than primary 366 11% 147 219 Primary/compulsory 1812 56% 713 1099 Secondary 866 27% 253 613 Post-secondary 45 1% 21 24 Tertiary 150 5% 40 110 Post-tertiary 3 .1% 1 2 Age 45-54 1537 47% 479 1058 55+ 1715 53% 699 1016 Duration of Unemployment >=6 months and <12 months 539 16% >=12 months 2713 84% GMI GMI Recipient 942 29% 429 513 Non-GMI Recipient 2310 71% 749 1561 Total 3252 1178 2074 Table 2: Profile and Size of Target Group (September 2019) Total Percent Men Women Education Level Unclassified/blank 631 17% 225 406 Less than primary 460 13% 167 293 Primary/compulsory 1536 42% 547 989 Secondary 793 22% 235 558 Post-secondary 51 1% 13 38 Tertiary 144 4% 49 95 Post-tertiary 2 0.1% 2 0 Duration of Unemployment >=6 months and <12 months 623 17% 290 333 >=12 months 2994 83% 948 2046 GMI GMI Recipient 1110 31% 476 634 Non-GMI Recipient 2507 69% 762 1745 Total 3617 1238 2379 5 3. Profiling and Appointments with Job Counselors Between November 2018 and March 2020, 948 individuals in the target group completed the on-line profiling questionnaire and 943 met with a job counselor to finalize the identification of their profile category and update their individual action plan. This corresponds to a take-up rate for the pilot of around 26 percent. Since the data required to update the ALMPS Dashboard was not accessible to the World Bank team at the time of writing this report, this indicator was not updated as of a later date. However, considering that training was by far the most common referral for pilot participants (95 percent based on data through March 2020) and the number of those referrals changed very little between March and October 2020 (893 vs. 900), there were probably relatively few additional individuals who entered the pilot after March 2020. Results of Profiling and Appointment with Counselor The results of the profiling and appointments with the jobs counselors are shown in Table 3. This is the same information that was presented in the previous monitoring report, since more recent data to update the ALMPS Dashboard was not available, as mentioned above. Slightly over 80 percent of those interviewed by a counselor were classified in Group 4, while 15 percent fell in Group 3. Compared to the characteristics of the overall target group (Table 2), the individuals who decided to participate in the pilot tended to be somewhat younger and longer-term unemployed and were more likely to be male. Only 10 percent of GMI recipients in the target group met with a counselor whereas slightly over a third of non-GMI recipients did so. For individuals who met with a counselor, information is available on previous participation in labor market programs for which OAED has some responsibility, mainly different types of unemployment assistance and Kinofelis (public works). About 60 percent of those who met with a counselor had received some type of unemployment assistance, most commonly the unemployment benefit. About a fifth had participated in Kinofelis. 6 Table 3: Characteristics of Unemployed with Counselor Appointment Profile Category Group 1 Group 2 Group 3 Group 4 Total Percent Total 2 10 143 788 943 Share of total 1% 15% 83% Men 1 1 40 225 269 28% Women 1 9 103 563 679 72% Age <45 1 3 2 1 7 1% 45-54 1 6 91 381 479 51% 55-64 1 47 340 388 41% 65-74 3 66 69 7% Education Primary/compulsory 1 50 453 504 53% Secondary 1 7 62 203 273 29% Post-secondary 1 8 31 40 4% Tertiary 1 1 16 32 50 5% Master 2 2 blank 7 67 74 8% GMI beneficiaries YES 9 93 102 11% NO 2 10 134 695 841 89% Unemployment Duration GMI<6 months 0 Non GMI<6 months 2 6 9 17 5% GMI>=6 months and <12 months 1 2 3 Non GMI >=6months and <12 months 1 2 17 25 45 5% GMI>=12 months 8 91 99 10% Non GMI>=12 months 1 6 111 661 779 83% Total< 6months 2 6 9 17 2% Total >=6 months and <12 months 1 2 18 27 48 5% Total>=12 months 1 6 119 752 878 93% 4. Wage Subsidies The previous monitoring report provided information on trends related to the 38 wage positions recorded through March 2020. Although updated information from the ALMPs Dashboard is not available, data is available from ERGANI through December 2020. This data indicates that post March 2020, there were relatively few additional wage subsidy placements until September 2021: 1 in the 2nd quarter of 2020, and 5 in July-August 2020. In September 2020, there was a large increase in entries to wage subsidy positions covering 30 individuals, followed by roughly another 20 individuals during the last quarter of 2020. Considering the timing, these are likely related to the Greek Government’s response to COVID-19 and may also have served some unemployed who had exceeded 6 months unemployment duration only by late 2020, by which time pilot activities were winding down. 7 5. Training Overview The bulk of training activities took place between the beginning of the pilot and April 2020 (with the exception of internship completions) and the associated trends were covered in the previous monitoring report. The focus of this monitoring report is on the updated figures and certification results. Table 4 presents an overview of the status on training activities as of October 2020. In the intervening six months since the period covered in the first monitoring report, there were very few additional entrants to training (referrals, vouchers issued). Rather during these six months there were advances in the later stages of activities (activation of voucher, start of theoretical and practical training, and certification). Table 4 provides an overview on training for the Elefsina pilot as of October 2020. In all, job counselors referred 900 pilot participants to training. Out of those referred, 93 percent received a voucher for classroom (theoretical) training. The gap between referrals and training vouchers issued is probably explained by two factors. One, not all those who received a basic skills voucher enrolled in classes; about 40 individuals did not enroll. According to the requirements for trainees, if an individual received a basic skills voucher, a skills training voucher would only be issued after completion of the basic skills training. Individuals who for some other reason did not return to the KPA2 to receive a voucher presumably explain the rest of the gap. The skills self-assessment is intended for those in profiling Category 3 and about 60 percent (113) of them took the test. Of those who received a voucher for technical training, 93 percent (781 individuals) activated it, and all were scheduled to have completed classes by end October 2020. 5 Slightly more than 660 trainees had reached the completion date of the internship for the same period. Table 4: Training Overview (as of October 2020) Total Completed/Activated In Did Not Enroll Process Referred to Training 900 Basic Skills Vouchers 151 110 41 Skills Self-Assessment 113 113 Training Vouchers 840 781 59 Practical Training (internship) 663 118 5Since the DRTP MIS is not set up to capture dropouts from training, the number of those who completed the theoretical training is unknown. 8 Basic Skills Job counselors issued 151 basic skills vouchers between April 2019 (when the test became available) and October 2020; nearly all were issued during 2019. Table 5 shows the distribution of vouchers by both the specialty of the trainee and the subject (language, math, or IT). The highest numbers of basic skills vouchers were issued for the specialties of warehouse worker (47), retail salesperson (37) and bartender (32). A voucher could cover any combination of the 3 modules depending on the needs of the individual, language being the most common module recommended. Table 5: Basic Skills Vouchers by Training Specialty and Module Training Specialty Language Math IT Vouchers Bartender 32 19 8 32 Building Security Guard 13 10 8 16 Office Clerk 9 4 5 11 Retail Salesperson 23 13 25 37 Waiter 3 3 3 3 Warehouse Guard 4 2 2 5 Warehouse Worker 47 28 18 47 Total 131 79 69 151 Characteristics of Recipients of Basic Skills and Technical Training Vouchers Table 6 shows the characteristics of basic skills voucher recipients as well as the recipients of technical training vouchers. The former are more likely to be female, older (65-74), less educated (the unspecified category probably includes those with less than primary education), and profile category 4. GMI recipients accounted for about one-fifth of basic skills vouchers, higher than their 14 percent share of technical training vouchers. The higher vulnerability of these individuals is not unexpected given the focus of the training. As mentioned above, over a quarter of basic skills voucher recipients failed to enroll in courses, a substantial share, suggesting the need for further follow-up of these individuals in the future to encourage them to take the classes so that they can benefit from skills training. 9 Table 6: Recipients of Basic Skills and Technical Training Vouchers Basic Skills Specialty Training Number Percent Number Percent Total 151 100% 840 100% Male 36 24% 232 28% Female 115 76% 608 72% 45-54 67 44% 397 47% 55-64 65 43% 376 45% 65-74 19 13% 67 7% Primary education 95 63% 453 54% Unspecified education 34 23% 53 6% Secondary education 15 10% 252 30% Other (education) 7 5% 82 10% Profile Category 3 23 15% 181 22% Profile Category 4 119 79% 605 72% Profile Category N/A 9 6% 54 6% GMI Recipient 31 21% 120 14% GMI Non-recipient 120 79% 720 86% Unemployed >=6 and<12 1 3% 6 1% Unemployed >=12 mo. 140 89% 778 93% Unemployed <6 or Blank 10 7% 56 7% 10 Technical Training Through October 2020, 840 technical training vouchers were issued. Table 7 shows the breakdown by specialty. The most common specialties were office clerk and retail salespersons, followed by building guard, warehouse worker and bartender. Three- quarters of vouchers were issued in June (45 percent), July (20 percent) and September (10 percent), all 2019. The skills self-assessment was taken by 13 percent of voucher recipients, accounting for 10-15 percent of most specialties, with the exception of office clerks (25 percent) and heavy truck drivers (60 percent). Table 7: Vouchers Issued and Skills Self-Assessment by Specialty Vouchers Percent Skills Self Percent of Assessment Vouchers Bartender 109 13% 9 8% Building Guard 143 17% 13 9% Forklift Driver 13 2% 2 15% Heavy Truck Driver 10 1% 6 60% Office Clerk 181 22% 45 25% Retail Salesperson 175 21% 15 9% Waiter 21 3% 2 10% Warehouse Guard 41 5% 7 17% Warehouse Worker 139 17% 13 9% Wholesale Salesperson 8 1% 1 13% Total 840 100% 113 13% Technical Training Delivery During the period since March 2020, only 48 additional unemployed began technical training. Since internships start after the completion of classroom training, about 100 additional trainees were scheduled to have entered their internships during this period. As shown in Table 4, 118 individuals had yet to reach the end date for their internship as of October 2020. The last individuals were scheduled to complete their internships by April 2021. Figure 1 below shows the beginning and end dates for training (including the internship). 11 Figure 1: Beginning and end dates for technical training Certification OAED made arrangements for trainees in 7 out of 10 specialties to be certified after completion of practical training, the exceptions being building guards, heavy truck drivers and warehouse workers. This meant that about 60 percent of those who activated their training voucher had the possibility to be certified. These individuals received a voucher, which they could use at an ISO Center. As of the end of October 2020, 371 trainees had activated a certification voucher, representing 80 percent of those in specialties for which OAED made arrangements for certification and who had completed practical training (the internship). Except for 4 trainees in the forklift driver specialty (out of 7 assessed), all received a “passing” certification result. 12 6. Results A key question for the pilot is whether it achieved its objective of helping participants integrate into the labor market. To assess this, in this section the post-program outcomes for pilot participants are compared with those for non-participants from the target group. According to the data, program participants were more likely than non-participants to have found a job and less likely to be in the unemployment registry at the end of the pilot. These findings are not the result of a rigorous impact evaluation and are only suggestive of the impact of the pilot. The basis for this analysis of the effects of the pilot is described below. The sample for the comparison included individuals eligible for the pilot, based on a minimum age of 45 and at least 6 months of registered unemployment, who were in the OAED unemployment registry between November 2018 and February 2019. This sample included 3,423 individuals. The rationale for this sample is that this allows for including the effect of pilot interventions (training and wage subsidies), which largely occurred in the second half of 2019 and 2020. Among the eligible individuals, participation in the pilot was defined by whether the individual received an Individual Action Plan (IAP). In the sample, 1,050 6 (30.7 percent) had an IAP and were thus identified as participants and 2,373 (69.3 percent) had not, so were identified as non-participants. Three outcomes were compared. The first was whether the individual had a job at any point between the end of their unemployment registration and October 2020. The second was whether the individual was employed in October 2020. The source of the information for both employment outcomes was ERGANI, the national employment registry managed by the Ministry of Labour and Social Affairs. It should be noted that ERGANI does not include unregistered or informal employment, or self-employment. The third outcome was whether the individual was in the OAED unemployment registry in October 2020. Only a small share of eligible individuals found employment and the majority were registered as unemployed at the end of the pilot period. Table 8 (columns 5 and 6) shows the employment and unemployment outcomes for all eligible individuals on the three outcomes identified above. Just under 20 percent had had some employment after leaving the unemployment registry; only 12.1 percent were employed in October 2020. On the other hand, nearly 70 percent appeared on the unemployment registry in that month. It is not surprising that these outcomes are not very favorable. The target group for the pilot was a difficult group to integrate into the labor market. Moreover, starting in March 2020, the Greek employment situation was badly hit by the COVID-19 pandemic. 61050 is about 100 more individuals than the 943 shown in Table 3. Members of the target group who participated in Kinofelis (public works) account for this difference. While Kinofelis was dropped from the pilot, individuals in the target group were eligible to participate in the nation-wide program. 13 Table 8: Comparison of post-program employment and unemployment outcomes for pilot participants and non-participants Participants (n=1050) Non-participants All eligible individuals (n=2373) (n=3423) Number Percent Number Percent Number Percent (1) (2) (3) (4) (5) (6) Employed at some 319 30.4 354 14.9 673 19.7 point after unemployment Employed in 177 16.9 236 9.9 413 12.1 October 2020 Unemployed in 680 64.8 1698 71.6 2378 69.5 October 2020 Nonetheless, employment outcomes were more positive for individuals participating in the pilot compared to non-participants. Table 8 shows that pilot participants were twice as likely to have been employed at some point after being registered as unemployed and 70 percent more likely to have been employed in October 2020. The differences were less striking in terms of unemployment but a smaller share of participants than non- participants were in the OAED unemployment registry in October 2020. However, these differences do not reflect the impact of the pilot because it is likely that participants were more employable to begin with. It would be expected that the pilot would be more attractive to individuals who were employable and were motivated to find employment. With the data available from OPS, we can compare the two groups in terms of observable individual characteristics. While the differences are fairly modest, participants were on average about 2 years younger, slightly better educated, and less likely to self-identify their specialty in areas, such as elementary occupations and plant and machinery work, where employment opportunities were likely to be scarce for this demographic group. Also, participants were more likely to be male than non-participants. Given higher employment rates for males in Greece7, this may be a relevant difference in the two sub-samples. These findings are similar to the differences between the characteristics of the individuals who met with a counsellor and the target group as a whole, discussed above. When individual characteristics are controlled for using propensity score matching, pilot participants still had more favorable employment outcomes. In order to take into account these differences between participants and non-participants, propensity score matching was used to construct “treatment” (participant) and “control” (non-participant) groups that were matched as closely as possible on many relevant individual characteristics (age, sex, duration of eligible time on the unemployment registry, receipt of unemployment and social assistance benefits, occupation, education, and municipality). Then probit functions were estimated to see if there were any differences in employment and unemployment outcomes for participants compared to matched non-participants (i.e., very similar 7According to the International Labour Organization, ILOSTAT, in 2019, female labor force participation in Greece was 44 percent compared to 60 percent for males. 14 characteristics). The finding, according to the results shown below in Tables 9-10, is that participating in the pilot (i.e., having an individual action plan) increased the likelihood of being employed in October 2020 by roughly 3 percentage points. A similar estimation on unemployment status in October 2020 found that the likelihood of being on the OAED unemployment registry in October 2020 was 5-6 percentage points lower for the pilot participants (Table 10). All of the estimates are significant at the 10 percent level. Table 9: Probit estimates of the effect of participation on employment in October 2020 (1) (2) (3) (4) (5) 1:1 nn- 1:2 nn- 1:3 nn- 1:4 nn- 1:5 nn- Observations matching matching matching matching matching 0.0267* 0.0273** 0.0332** 0.0319** 0.0326** 3,423 (0.0147) (0.0136) (0.0132) (0.0131) (0.0128) Robust standard errors are reported in parentheses, stars indicate the following level of significance: *** p<0.01, ** p<0.05, * p<0.1 Table 10: Probit estimates of the effect of participation on being in the unemployment registry in October 2020 (1) (2) (3) (4) (5) 1:1 nn- 1:2 nn- 1:3 nn- 1:4 nn- 1:5 nn- Observations matching matching matching matching matching -0.0616*** -0.0590*** -0.0572*** -0.0542*** -0.0486*** 3,423 (0.0219) (0.0202) (0.0194) (0.0191) (0.0188) Robust standard errors are reported in parentheses, stars indicate the following level of significance: *** p<0.01, ** p<0.05, * p<0.1 These estimates are consistent with a conclusion that the pilot had positive impacts but they do not demonstrate a conclusive causal effect. This ex-post analysis is able to compare selected employment outcomes between pilot participants and non-participants, taking into account various observable individual characteristics. However, while the technique used closely matched the two groups on these characteristics, it does not take into account unobserved characteristics, such as motivation to work and to acquire employable skills, personal networks, and “soft” skills. These attributes would almost certainly be associated with the likelihood of finding a job and may well be more prevalent in the group that chose to have an individual action plan. A more rigorous methodology would have taken this into account, but this was not possible for this analysis. 15