Policy Research Working Paper 9345 Job Search and Hiring with Two-Sided Limited Information about Workseekers’ Skills Eliana Carranza Robert Garlick Kate Orkin Neil Rankin Social Protection and Jobs Global Practice & Africa Gender Innovation Lab July 2020 Policy Research Working Paper 9345 Abstract This paper presents field experimental evidence that limited assessment results only to workseekers has similar effects on information about workseekers’ skills distorts both firm and beliefs and search, but smaller effects on employment and workseeker behavior. Assessing workseekers’ skills, giving earnings. Giving assessment results only to firms increases workseekers their assessment results, and helping them to callbacks. These patterns are consistent with two-sided credibly share the results with firms increases workseek- information frictions, a new finding that can inform the ers’ employment and earnings. It also aligns their beliefs design of information-provision mechanisms. and search strategies more closely with their skills. Giving This paper is a product of the Social Protection and Jobs Global Practice and Africa Gender Innovation Lab. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http://www.worldbank.org/ prwp. The authors may be contacted at ecarranza@worldbank.org, robert.garlick@duke.edu, and kate.orkin@bsg.ox.ac.uk. The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent. Produced by the Research Support Team Job Search and Hiring with Two-Sided Limited Information about Workseekers’ Skills∗ Eliana Carranza† Robert Garlick‡ Kate Orkin§ Neil Rankin¶ Abstract This paper presents field experimental evidence that limited information about workseekers’ skills distorts both firm and workseeker behavior. Assessing workseekers’ skills, giving workseekers their assessment results, and helping them to credibly share the results with firms increases workseekers’ employment and earnings. It also aligns their beliefs and search strategies more closely with their skills. Giving assessment results only to workseekers has similar effects on beliefs and search, but smaller effects on employment and earnings. Giving assessment results only to firms increases callbacks. These patterns are consistent with two-sided information frictions, a new finding that can inform the design of information-provision mechanisms. JEL codes: J23, J24, J31, J41, O15, O17 ∗ We are grateful for helpful comments from Martin Abel, Peter Arcidiacono, Vittorio Bassi, Paolo Belli, Emily Breza, Rulof Burger, Stefano Caria, Jishnu Das, Giacomo di Giorgi, Taryn Dinkelman, Rebecca Dizon-Ross, Stefan Dercon, Erica Field, Simon Franklin, Doug Gollin, Marek Hanusch, Rachel Heath, Lukas Hensel, Clement Imbert, Anett John, Lisa Kahn, Julien Labonne, Melanie Morten, Simon Quinn, Imran Rasul, Duncan Thomas, Gabriel Ulyssea, Chris Woodruff, and seminar participants at the NBER Summer Institute, Barcelona Summer Forum, Duke, Georgia, Notre Dame, NYU, Oxford, UIUC, and the World Bank. We appreciate exceptional research assistance from Wim Louw, Wendy Trott, Shelby Carvalho, Allegra Cockburn, Goodwill Mpofu, Emmanuel Bakirdjian and Nilmini Herath in South Africa and Lukas Hensel, Svetlana Pimkina, Laurel Wheeler, Gabriel Cunha and Rachel Sayers at our institutions. Thanks to staff at the Harambee Youth Employment Accelerator, the Abdul Latif Jameel Latif Poverty Action Lab Africa Office, and our institutions. The project is conducted in collaboration with the World Bank Jobs Group and Africa Gender Innovation Lab, and received funding from the National Science Foundation (# 1824413), Private Enterprise Development in Low Income Countries (# 3024 and # 4728), W. E. Upjohn Institute for Employment Research, and the UKRI GCRF Accelerating Achievement for Africa[Pleaseinsert\PrerenderUnicode{âĂŹ}intopreamble]s Adolescents Hub. This study has been approved by the University of Cape Town Commerce Faculty Ethics in Research Committee, the Duke Institutional Review Board for the Protection of Human Subjects in Non-Medical Research (# D0368) and the Oxford Department of Economics Research Ethics Committee (# ECONCIA15-055). This study is pre-registered at https://doi.org/10.1257/rct.1631-8.0. The authors’ views do not necessarily represent the views of the World Bank or the governments of the countries they represent. † World Bank, ecarranza@worldbank.org ‡ Duke University, robert.garlick@duke.edu § University of Oxford, kate.orkin@bsg.ox.ac.uk ¶ University of Stellenbosch, neilrankin@sun.ac.za 1 Introduction Workseekers make job search decisions and firms make hiring decisions using potentially limited information about workseekers’ skills. Limited information for firms can lead to hiring poorly-matched workers and to wedges between wage offers and productivity (Altonji and Pierret, 2001; Arcidiacono et al., 2010; Farber and Gibbons, 1996; Kahn and Lange, 2014). These hiring distortions can reduce both employment and average wages conditional on employment (Aigner and Cain, 1977; Pallais, 2014). Limited information for workseekers can lead them to search for jobs that poorly match their skills or withdraw from search entirely (Belot et al., 2018; Conlon et al., 2018). These search distortions can also lead to lower employment and lower wages conditional on employment. When both sides of the market receive credible information on workseekers’ skills or past performance, these workseekers’ outcomes in the labor market can improve (Abebe et al., 2020a; Abel, 2019; Bassi and Nansamba, 2020; Pallais, 2014). These information problems may be particularly important in settings where hiring is less formal and education provides less information about skills (Pritchett, 2013). Limited information may exacerbate other frictions in developing country labor markets, such as high search and migration costs (Abebe et al., 2020b; Bryan et al., 2014; Franklin, 2017). We study how providing new information about workseekers’ skills affects job search and hiring decisions and workseekers’ labor market outcomes. We run a series of field experiments that separately manipulate firms’ and workseekers’ information about workseekers’ skills. We provide the first evidence that both firms and workseekers adjust their behavior when they acquire new information about workseekers’ skills, improving workseekers’ outcomes in the labor market. The existence of two-sided information frictions is important both conceptually and for the design of information-provision products and policies.1 Many existing policies target only one-sided frictions and may have limited returns if two-sided frictions are present. For example, firms testing job applicants’ skills has limited returns if well-matched workseekers do not apply to these firms, while workseeker skill assessments offered in job search assistance programs have limited returns if workseekers cannot credibly communicate their skills to firms. Existing papers cannot identify two-sided information frictions because they study only one side of the market or study simultaneous two-sided information revelation. We study firms’ and workseekers’ responses to learning workseekers’ results on standardized skill assessments. The assessments measure non-specialist skills such as communication, numeracy, and grit and draw on existing tools used by job placement agencies and large firms. The 6,891 assessed workseekers are drawn from a population where information frictions may be important. They are unemployed or underemployed black youths in urban South Africa with limited post-secondary education, work experience, and access to referral networks. This population faces statistical 1 We use the term ‘information friction’ to refer to distortions in behavior due to limited information about workseekers’ skills. Our experiments do not speak to limited information about other factors such as vacancy characteristics or workers’ effort. 2 discrimination in this labor market (Malindi, 2017). We demonstrate the existence of two-sided information frictions in three steps. First, we show that giving workseekers their results from these assessments and enabling them to easily and credibly share the results with firms improves the workseekers’ labor market outcomes. To show this, we randomly select some workseekers for a ‘public’ certification intervention. We give them electronic and physical certificates describing the assessments and showing their results. The certificates show their names and national identity numbers and are branded by the widely known agency that conducts the assessments and the World Bank. We compare these workseekers to a control group of workseekers who receive no certificates and do not learn their results. In the three to four months following certification, publicly certified workseekers use certificates in job applications, shift their beliefs about their skills closer to their assessment results, and target their search toward jobs that they think value their skills. Their employment rate increases by 17% (5 percentage points), weekly earnings increase by 34%, and hourly wages increase by 20% relative to the control group. The rise in earnings reflects both higher employment and higher earnings conditional on employment. This first step shows that information frictions exist, but does not show who has limited information about workseekers’ skills. The treatment effects are consistent with firms changing hiring and wage offer decisions as they learn about workseekers’ skills from seeing certificates, with workseekers changing search behavior as they learn their own assessment results, or with both types of frictions. In the second step of our argument, we show that workseekers face information frictions. We randomly select some assessed workseekers for a ‘private’ certification intervention. This intervention gives them one physical certificate that shows their assessment results and describes the assessments, but excludes identifying information and branding that might make the certificate a credible source of information to firms. Private certification has the same effects as public certification on workseekers’ beliefs and search targeting, a positive effect on earnings that is smaller than the public certification effect, and no effect on employment. These results show that giving workseekers more information about their skills changes their behavior and outcomes in the labor market. However, effects are smaller than when they can also share the information with firms. This pattern is consistent with both sides of the market having limited information about workseekers’ skills, but inconsistent with only one side having limited information. In the third step of our argument, we run an audit/correspondence experiment as a further check for firm-side information frictions. This experiment manipulates firms’ information without scope for changes in workseeker behavior. We submit applications to real job vacancies using real resumes from workseekers in our sample. We submit multiple applications per vacancy, randomizing whether applications include public certificates. Applications including certificates get 11% more interview invitations, consistent with firms having limited information about workseekers’ skills and acquiring more information from skill certification. 3 These three experiments demonstrate our main finding: both firms and workseekers have limited information about workseekers’ skills. In addition, we present four secondary findings that further characterize how information frictions affect this labor market using heterogeneity analysis and smaller experiments. First, learning specific assessment results is important, not just learning that workseekers have been assessed. The certification effects are not driven, for example, by firms using workseekers’ decisions to get assessed as a signal for tenacity or proactivity, or by firms basing hiring decisions purely on the certificates’ branding. Second, in this context, preferences for different skills vary across firms and relative performance in different assessments varies across workseekers. This pattern is more consistent with horizontal than vertical differentiation: certification helps firms identify which workseekers are suited for specific jobs, rather than identify a subset of workseekers suited for all jobs.2 Third, certification has larger effects on the labor market outcomes of workseekers who lack other ways to communicate their skills to employers, like work experience and university education. Fourth, although we do not directly observe if certified workseekers become employed at the expense of workseekers outside our sample, most of our results are consistent with economic mechanisms in which certification can increase total employment. Our primary contribution is to show that both firms and workseekers face information frictions and that both sides of the labor market respond to information provision. This extends existing work documenting labor market patterns consistent with either firm-side or workseeker-side information frictions in both developed and developing economies.3 We build on this work by showing that both firms and workseekers face limited information about workseekers’ skills. Our work is most similar to papers that study information frictions by simultaneously revealing information to both firms and workseekers about skill assessment results (Abebe et al., 2020a; Bassi and Nansamba, 2020) or evaluations from workseekers’ past employers (Abel et al., 2020; Pallais, 2014). These papers show that information revelation changes workseekers’ outcomes and interpret this as evidence of firm-side information frictions. But they do not separately manipulate firms’ and workseekers’ information, so cannot distinguish one-sided from two-sided frictions.4 The distinction between one- and two-sided frictions is important for designing mechanisms that private actors or government can use to address information frictions. Interventions targeted at one side of the market are common, but may not be optimal if both sides of the market face 2 This finding is consistent with recent work on information frictions in matching models with multidimensional skills (Fredriksson et al., 2018; Guvenen et al., 2020; Lise and Postel-Vinay, 2020). 3 Altonji and Pierret (2001), Arcidiacono et al. (2010), Farber and Gibbons (1996), and Kahn and Lange (2014) show that wages align more closely with skills as tenure increases, consistent with limited information about skills at the time of hiring. Wage and retention patterns for workers hired through referrals are also consistent with information frictions (Ioannides and Loury, 2004; Heath, 2018) and some researchers find that workers have better labor market outcomes when they have formal educational qualifications, conditional on measured skills (Alfonsi et al., 2017; MacLeod et al., 2017). Workseekers’ job search decisions can change when workseekers learn more about their labor market prospects (Ahn et al., 2019; Altmann et al., 2018; Belot et al., 2018), although these papers do not specifically examine limited information about workseekers’ skills. 4 Abel et al. (2020) reveal information to both sides of the market and only to firms, but do not test for workseeker-side frictions. 4 frictions. On the workseeker side, some job search assistance programs offer skill assessments to workseekers (McCall et al., 2016). This can inform workseekers and improve their search targeting. But if firms face information frictions and do not learn these assessment results, then firms’ hiring choices and wage offers will remain distorted and workseekers’ improved search will have limited returns. On the firm side, skill assessments are sometimes used to inform firm hiring decisions (Autor and Scarborough, 2008; Hoffman et al., 2018). But if workseekers face information frictions, they might not apply for jobs that match their skills, leaving firms to assess and select from a sub-optimal pool of applicants. Two-sided frictions may reduce the return to one-sided information acquisition enough to deter investment in one-sided screening by firms or signaling by workseekers. Alonso (2018) shows that introducing one-sided information acquisition in labor market matching can reduce welfare when information frictions are two-sided. This paper complements work on the aggregate implications of information frictions. Canonical models show that search and matching frictions facing individual workseekers and firms can generate aggregate unemployment (Mortensen and Pissarides, 1999). Our finding of two-sided information frictions offers an experimental foundation for general equilibrium models in which both firms and workseekers have limited information about match productivity, leading to aggregate employment distortions (Jovanovic, 1979; Menzio and Shi, 2011). In particular, our findings complement work by Donovan et al. (2018), who show that models with limited information about workseekers’ skills can explain aggregate labor market dynamics in developing countries. Our findings on information frictions are also relevant to the design of active labor market programs (ALMPs). We show that a skill assessment and certification intervention, delivered during recruitment for an ALMP, can substantially and cheaply improve participants’ employment and earnings. The employment effect is almost three times larger than the mean effect size of the active labor market programs reviewed by Card et al. (2018). The average earnings gain in the first three months after treatment is 5.6 times the average variable cost of this certification intervention alone and 2.3 times the average variable cost of assessment and certification.5 Skill assessment and certification may enhance the value of ALMPs to participating workseekers even when other mechanisms for learning about workseekers’ skills exist. Importantly, certification is available to first-time workseekers, unlike reference letters or performance evaluations from past employers. Assessment results can be certified to multiple employers, while workplace performance at one employer may be imperfectly observed by other employers (Kahn, 2013). Certification can help workseekers excluded from referral networks or firms who receive referrals based on factors poorly aligned with workseekers’ skills (Beaman and Magruder, 2012; Beaman et al., 2018; Chandrasekhar et al., 2020). We describe the economic environment in Section 2: a simple conceptual framework, the context, 5 In a similar spirit, several papers show that making low-cost changes to ALMPs so they provide more information to firms and/or workseekers improves their effectiveness (Abel et al., 2020; Belot et al., 2018; Wheeler et al., 2019). 5 the sample, and the skill assessments. In Section 3, we describe the public skill certification experiment and the treatment effects on workseekers’ labor market outcomes. In Section 4, we analyze the roles of firm- and workseeker-side frictions. In Section 5, we discuss secondary results about what workseekers and firms learn from certification, what this implies for the effects of certification on different types of workseekers, and what this might imply for certification at a larger scale. We conclude in Section 6 and discuss questions around markets for assessment-based certification. 2 Economic Environment 2.1 Conceptual Framework In this section, we sketch a simple conceptual framework with two goals. First, the framework illustrates how either workseeker- or firm-side information frictions can lower two labor market outcomes: the employment rate and the mean wage conditional on employment. Hence, observing that employment and/or wages rise when both firms and workseekers acquire more information does not show which side(s) of the market face information frictions. This highlights the importance of the separate variation we generate in firms’ and workseekers’ information sets. Second, the framework illustrates the mechanisms that link limited information to distortions in firm and workseeker behavior and hence to lower wages and employment. This guides our empirical tests of these mechanisms. Consider a stylized economy consisting of infinitely many type W1 and W2 workseekers and type J1 and J2 jobs. Workseekers choose not to search, to search for type 1 jobs, or to search for type 2 jobs. Searching for either type of job incurs fixed cost C > 0. A type i workseeker searching for type j jobs meets a firm offering such a job with probability Pi,j . Conditional on meeting, the match produces output with pecuniary value Vi,j and pays wage Wi,j ≤ Vi,j . The workseeker receives utility U (Pi,j · Wi,j ) − C if she searches and zero otherwise, implying that she has a reservation wage W i (C, P ).6 Non-employment is possible if search costs are high relative to the expected utility of working (which leads to some non-participation) or if the meeting probability Pi,j is less than one for some (i, j ). We make some additional simplifying assumptions for this discussion, but none of the results in the framework depend on these additional assumptions. First, we assume fraction p of all workseekers and all jobs are type 1. Second, we assume that type i workseekers are better at searching for type i jobs, produce the most output in type i jobs, and earn the highest wages in type i jobs, and similarly for type j workseekers and type j jobs. Under these assumptions, if type i workseekers choose to search, they will choose to search for type i jobs rather than type j jobs, and vice versa. 6 For simplicity, we assume that firms post and commit to wages before workseekers make search decisions. This implies that all workseekers who choose to search for type j jobs will accept them if offered. 6 Limited information about workseekers’ skills can enter this environment in two ways. First, we consider the case where only workseekers observe their types with error. This type of friction can occur if, for example, workseekers receive limited information about their own type from education or work experience or if they have little education or work experience. With this type of friction, each workseeker chooses whether and where to search based on her perceived type. If a type i workseeker ‘incorrectly’ searches for a type j = i job, she is less likely to meet a firm and, conditional on meeting a firm, will produce less and earn a lower wage. This type of friction reduces mean wages conditional on employment by generating some mismatches between workseeker and job types. This can also reduce the employment rate through two mechanisms: workseekers who search for the wrong type of jobs are less likely to meet firms, and mismatches between workseeker and job types may not generate enough output to offer wages above the reservation wage. The former mechanism can occur, for example, if firms offering different types of jobs hire using different channels, like posting formal adverts versus hiring walk-ins. The latter mechanism can occur if, for example, search costs and hence reservation wages are high or if there is a legal minimum wage. Belot et al. (2018) and Falk et al. (2006) prove results of this flavor formally. Second, only firms may observe workseekers’ types with error. This type of friction can occur if attributes observable to firms, like educational qualifications or past work experience, provide limited information about skills. With this type of friction, workseekers search for the ‘right’ types of jobs but firms do not know the type of the workseekers they meet. If type j firms believe that fraction q of the workseekers they meet are type j , then the expected output from each hire is q · Vj,j + (1 − q ) · Vi,j . If firms’ utility is a concave function of their output, then they will offer a wage lower than q · Wj,j + (1 − q ) · Wi,j . Concavity can arise from firms’ production technology or from uninsured risks from bad hires. Possible uninsured risks include lost customers or damaged equipment from hiring the ‘wrong’ workseekers or severance pay and dispute resolution costs when firing workseekers. This reduces mean wages conditional on employment and, if the offered wage is below the reservation wage or minimum wage, reduces the employment rate. Aigner and Cain (1977) and Pallais (2014) prove results of this flavor formally. This simple framework shows that observing a rise in employment and/or wages when both firms and workseekers acquire more information does not show which side(s) of the market face information frictions. This highlights the importance of the separate variation we generate in firms’ and workseekers’ information sets. Depending on the structure of the model, frictions on both sides of the market might interact to generate larger distortions or partly offset each other. We do not explore this in detail because our experimental design does not identify the size of any interaction effect without additional assumptions. We focus on the static case for simplicity, but recognize that the effect of information frictions may differ in a dynamic framework with learning by firms or workseekers (Conlon et al., 2018; Lange, 2007). The framework allows either horizontal or vertical differentiation of workseekers. We define 7 horizontal differentiation as type i workseekers being more productive than type j workseekers in type i jobs and vice versa. We define vertical differentiation as type i workseekers being more productive than type j workseekers in all jobs. In both cases, either firm- or workseeker-side information frictions can lower the employment rate and the mean wage conditional on employment. With horizontal differentiation, frictions on either side of the market can lower wages conditional on employment for all types of workseekers. With vertical differentiation, firm-side frictions can lower wages for type i workseekers mistaken for type j workseekers and raise wages for type j workseekers if they are mistaken for type i workseekers. 2.2 Context We work in the metropolitan area of Johannesburg, South Africa’s commercial and industrial hub. Johannesburg’s labor market has four salient features for our study. First, information frictions are likely, as alternative sources of information on workseekers’ skills are limited. Many young workseekers have no work experience several years after completing education, limiting the scope to learn about or signal their skills through experience (Ingle and Mlatsheni, 2017). Grades and grade progression in most primary and secondary schools are weakly correlated with independently measured skills (Lam et al., 2011; Taylor et al., 2011). Workseekers who have completed secondary school typically report their grades in the nationally standardized graduation exam in job applications. But examination grades weakly predict performance in post-secondary education and firms report in interviews that the grades convey limited information about skills (Schöer et al., 2010).7 This limits the scope for firms and workseekers to learn about workseekers’ skills from their educational attainment. Certification is thus likely to provide both firms and workseekers with additional information on workseekers’ skills. Second, ‘wrong’ hires are costly to firms. Firing a worker requires a complex and lengthy process and can be challenged by even temporary employees in courts and specialized dispute resolution bodies.8 Probationary work is permitted but regulated and probation periods cannot exceed three months (Bhorat and Cheadle, 2009). Firms report challenges understanding labor regulation, contributing to the perceived cost of separations.9 Consistent with these factors, giving firms free consulting on labor regulation increases hiring (Bertrand and Crépon, 2019). Third, reservation and minimum wages exist. Minimum wage compliance in the formal sector is high (Bhorat et al., 2016; International Labour Organization, 2016). Commuting costs are high and likely to raise reservation wages (Kerr, 2017). The nearly universal state pension system 7 The limited information content of education qualifications is consistent with the large role of referrals in hiring: more than half of all firms report that referrals are their preferred recruitment mechanism (Schöer et al., 2014). 8 Small firms report an average of two dispute resolution cases in the previous year, requiring an average of 11 days of staff time per case (Rankin et al., 2012). 9 Only 18% of SME owners know the conditions that made a contract valid or rules governing severance pay (Bertrand and Crépon, 2019). 8 gives workseekers in multi-generation households access to non-labor market income, increasing reservation wages (Abel, 2019). Fourth, employment rates are low. In our study period, unemployment in Johannesburg was 28% for the working-age population, 51% for ages 15-24, and 32% for ages 25-34 (Statistics South Africa, 2016).10 Low employment in the presence of information frictions, costs from ‘wrong’ hires, and reservation or minimum wages are consistent with our conceptual framework. Particularly low employment for youths is also consistent with information frictions, as youths have less search and work experience to reveal their types. Many other factors can contribute to low employment rates; we merely argue that a role for information frictions is plausible. 2.3 Sample Recruitment and Data Collection We recruit a sample of 6,891 young, active workseekers from low-income backgrounds with limited work experience. Workseekers in our sample have limited access to traditional ways to learn about their skills and communicate their skills to prospective employers: university education, work experience, or access to referral networks. We recruit only active workseekers, so we do not examine the relationship between information frictions and labor market participation decisions. This is a sample from a policy- and theory-relevant population likely to face information frictions, rather than a population-representative sample. To recruit the sample, we work with the Harambee Youth Employment Accelerator, a social enterprise that assesses the skills of inexperienced workseekers and matches them to employers looking for entry-level workseekers, among other activities. Harambee recruits candidates through radio and social media advertising and door-to-door recruitment in low-income neighborhoods. Interested candidates register online and complete a phone-based screening questionnaire.11 Eligible candidates are invited to two days of standardized skill assessments. Some candidates are invited to further job readiness training based on their assessment results and residential location, but only 0.2% of candidates in our sample get jobs through this training. Our sample consists of all candidates who arrive at Harambee for the second of these two testing days, on 84 operational days. We conduct three surveys to measure workseekers’ labor market outcomes, job search, and beliefs about their skills and the labor market. The baseline is a self-administered questionnaire that candidates complete on desktop computers at Harambee under supervision. This is administered after candidates have done skills assessments but before they receive information about their results. 10 Throughout the paper, we use Statistics South Africa’s definition of an employed person as someone who did any income-generating activity, for at least one hour, during the reference week. Unemployment rates exclude those in full-time education or not in the labor force. 11 Candidates are eligible to work with Harambee if they are aged 18-29, have legal permission to work in South Africa, have completed secondary school, have at most 12 months of formal work experience, have no criminal record, and are from disadvantaged backgrounds. This information is self-reported but checked against administrative data for some candidates. 9 We collect endline data in a 25-minute phone survey 3-4 months after treatment.12 The phone survey response rate is 96%, leaving an endline sample of 6,607 respondents. The response rate is balanced across treatment groups (Table D.5) and unrelated to most baseline covariates (Table D.6). We also conduct a short text message survey 2-3 days after treatment. Respondents receive mobile phone airtime payments for answering the text message and phone surveys. 2.4 Job Search and Employment in the Sample This section describes relevant patterns around labor market outcomes and job search in our sample. We report summary statistics for key baseline and endline variables for the 6,891 workseekers in Tables D.1 and D.2. Respondents are 99% Black African, 62% female, and on average 24 years old. And 17% have a university degree or diploma, 21% have some other post-secondary certificate, and 99% have completed secondary school. Malindi (2017) shows that young, black workseekers with relatively low levels of education face discrimination in this labor market, with wage dynamics consistent with information frictions and statistical discrimination. Of the sample, 38% worked in the week before the baseline and 70% had ever worked, but only 9% had ever held a long-term job. Conditional on working, mean weekly earnings in the week before the baseline was 565 South African rand (94 USD in purchasing power parity terms), slightly below the minimum wage for a full-time worker in most sectors. Wage work was eight times more common than self-employment. Most work was relatively short-term, with median and mean tenures of 2 and 7 months respectively. Of the sample, 97% searched for work in the week before the baseline. In that week, they spent on average 17 hours and 242 South African rand (40 USD PPP) searching. The relatively high search costs suggest that welfare gains for workseekers are possible from improved search targeting. Workseekers submitted on average 10 applications in the preceding month and received 1.2 offers, though the medians for both measures are zero. The job search and application process is somewhat formal: 38% of the candidates employed at endline reported that they submitted written applications for their current job and 47% reported that they had a formal interview. 2.5 Assessments We conduct six assessments with workseekers: communication, concept formation (similar to a Raven’s test), focus, grit, numeracy, and planning. Firms have demonstrated interest in the results of these assessments, though they obviously also use other information in hiring decisions. Client firms have paid Harambee to screen roughly 160,000 prospective workers using these assessments. Appendix A describes each assessment in detail, their psychometric properties, and how some Harambee client firms use them in hiring. Each assessment session is led by two or three industrial psychologists, who manage a team of 12 See Garlick et al. (2019) for an experimental validation of labor market data from phone surveys in this setting. 10 facilitators. Assessments are conducted in English and are self-administered on desktop computers. Appendix Table D.1 shows standardized scores on all six assessments. There is a fairly even spread of candidates over the distribution and little evidence of ceiling effects. Appendix Table A.2 shows the correlation matrix between different skills. We interpret candidates with different assessment results as different worker types, in the language of the conceptual framework. Scores are weakly correlated across assessments, with pairwise correlations between 0.05 and 0.51. Hence, the assessments horizontally differentiate candidates based on their relative skills rather than only ranking them in a single dimension of skills. Candidates have inaccurate beliefs about their own types, suggesting a role for workseeker-side information frictions. We ask candidates in which tercile they believe they ranked for each of the communication, concept formation, and numeracy assessments, after taking the assessments but before any candidates learn their results. Only 8% of candidates answer correctly for all three assessments and 28% of candidates answer incorrectly for all three assessments. Overconfidence is more common than underconfidence: 22% of candidates overestimate their tercile on all three assessments and 1% underestimate their tercile all three assessments (Appendix Table D.1). 3 Labor Market Effects of Certification 3.1 Intervention Our first certification intervention gives candidates information about their assessment results and allows them to share the results with prospective employers. The effects of this intervention may reflect reductions in firm- or workseeker-side information frictions. In either case, the framework predicts that certified workseekers will have higher employment and higher earnings conditional on employment. Candidates receive a certificate describing the assessments and their performance (Figure 1). They receive 20 color copies printed on high-quality paper and an email version. Each certificate briefly describes Harambee and its placement and assessment work. To provide credibility to the assessments and results, the certificate is branded with the World Bank logo and Harambee logo. Harambee is a widely recognized brand in South African marketing surveys (Mackay, 2014). The certificate describes the skills measured by each assessment. The certificate directs the reader to www.assessmentreport.info for more information on Harambee and the assessments. The website shows sample questions for each assessment and describes how psychologists have designed and evaluated the assessments. For each skill, the certificate shows the tercile in which the candidate ranked on each assessment, compared to other candidates assessed by Harambee.13 The candidates assessed by Harambee are described as South African high school graduates aged 18-34 from disadvantaged backgrounds. To link candidates with certificates, each certificate shows the 13 In piloting, both workseekers and firms found certificates with only rankings easier to interpret than certificates with only cardinal scores or both rankings and cardinal scores. 11 candidate’s name and national identity number. National identity numbers are typically shown on resumes and school transcripts in South Africa. Each candidate receives their certificates during a group briefing with a psychologist. The psychologist explains what each assessment measures and how to interpret the results on the certificate. They explain that workseekers can, but do not have to, attach the certificate to future job applications and that they can request more certificates from Harambee. To ensure briefings were standardized, the research team and Harambee psychologists jointly developed a briefing script and PowerPoint presentation. Research assistants monitored each briefing to ensure psychologists used the script. 3.2 Experimental Design We randomly divide our workseeker sample into a certification group, a control group, and other groups discussed in the next section. We randomize treatment by assessment date to reduce risks of spillovers between treated and control workseekers, assigning 2,247 workseekers assessed on 27 days to certification and 2,274 workseekers assessed on 27 days to the control group. Table D.1 shows that the randomization generates balanced treatment assignments. Treated and control workseekers differ in only one way: treated workseekers receive the certification intervention described above, while control workseekers receive no information about their assessment results and no certificate to enable them to share results with firms. All treated and control workseekers receive roughly one hour of job search counselling before the assessments on how to create an email address and how to prepare and dress for an interview. They also receive an email with a CV template, interview tips, and job search tips.14 This differs from the design in Abebe et al. (2020a), where treated workseekers receive both skill certification and job search counselling while control workseekers receive neither. We estimate treatment effects using models of the form Yid = Td · ∆ + Xid · Γ + Sd + id , (1) where Yid is the outcome for workseeker i assessed on date d, Td is a vector of treatment assignments, and Xid is a vector of prespecified baseline covariates. Sd is a block fixed effect, to account for the fact that we randomly assign days to treatment groups within blocks of 6-10 sequential days. We use heteroskedasticity-robust standard errors clustered by assessment date, the unit of treatment assignment. All labor market and job search measures use 7-day recall periods, except where we specify otherwise. We apply an inverse hyperbolic sine transformation to right-skewed variables such as earnings; the distributions of these variables in our sample allow us to interpret these 14 Harambee invites some workseekers for further training and job search assistance. These invitations depend partly on their assessment results and may only be issued months after assessment. By the endline survey, only 1.4% of our sample are invited for further interaction with Harambee and only 0.17% receive a job offer through their further interaction with Harambee. These outcomes are uncorrelated with treatment status and all our results are robust to dropping these workseekers. 12 Figure 1: Sample Public Certificate REPORT ON CANDIDATE COMPETENCIES name.. surname.. ID No. id.. This report provides information on assessments conducted by Harambee Youth Employment Accelerator (harambee.co.za), a South African organisation that connects employers looking for entry-level talent to young, high-potential work-seekers with a matric or equivalent. Harambee has conducted more than 1 million assessments and placed candidates with over 250 top companies in retail, hospitality, financial services and other sectors. Assessments are designed by psychologists and predict candidates’ productivity and success in the workplace. This report was designed and funded in collaboration with the World Bank. You can find more information about this report, the assessments and contact details at www.assessmentreport.info. «name» was assessed at Harambee on 13 September, 2016. «name» completed assessments on English Communication (listening, reading, comprehension), Numeracy, and Concept Formation: 1. The Numeracy tests measure candidates’ ability to apply numerical concepts at a National Qualifications Framework (NQF) level, such as working with fractions, ratios, money, percentages and units, and performing calculations with time and area. This score is an average of two numeracy tests the candidate completed. 2. The Communication test measures a candidate's grasp of the English language through listening, reading and comprehension. It assesses at an NQF level, for example measuring the ability to recognise and recall literal and non-literal text. 3. The Concept Formation Test is a non-verbal measure that evaluates candidates’ ability to understand and solve problems. Those with high scores are generally able to solve complex problems, while lower scores indicate an ability to solve less complex problems. «name» also completed tasks and questionnaires to assess their soft skills: 4. The Planning Ability Test measures how candidates plan their actions in multi-step problems. Candidates with high scores gener- ally plan one or more steps ahead in solving complex problems. 5. The Focus Test assesses a candidate’s ability to distinguish relevant from irrelevant information in potentially confusing environments. Candidates with high scores are generally able to focus on tasks in distracting surroundings, while candidates with lower scores are more easily distracted by irrelevant information. 6. The Grit Scale measures whether candidates show determination when working on challenging problems. Those with high scores generally spend more time working on challenging problems, while those with low scores choose to pursue different problems. «name»’s results have been compared to a large benchmark group of young (age 18-34) South Africans assessed by Harambee. All candidates have a matric certificate and are from socially disadvantaged backgrounds. The benchmark group is 5,000 for cognitive skills and 400 for soft skills. «name» scored in the «tercile_num» THIRD of candidates assessed by Harambee for Numeracy, «tercile_lit» THIRD for Communication, «tercile_cft» THIRD for Concept Formation, «tercile_tol» THIRD for Planning Ability, «tercile_troop» THIRD for Focus and «tercile_grit» THIRD for the Grit Scale. DISCLAIMER: This is a confidential assessment report for use by the person specified above. The information in the report should only be disclosed on a “need to know basis” with the prior understanding of the candidate. Assessment results are not infallible and may not be entirely accurate. Best practice indicates that any organisation’s career management decisions should depend on factors in addition to these assessment results. Harambee cannot accept responsibility for decisions made based on the information contained in this report and cannot be held liable for the consequences of those decisions. This figure shows an example of the certificates given to candidates in the certification treatment. Each certificate shows some information about the assessments, the candidate’s assessment results, the candidate’s name and national identity number, and the logo of the World Bank and the implementing agency. Each workseeker received 20 printed certificates, an email copy of the certificate, and guidelines on how to request more certificates. 13 Table 1: Treatment Effects on Labor Market Outcomes (1) (2) (3) (4) (5) Employed Hoursc Earningsc Hourly wagec Written contract Treatment 0.052 0.201 0.338 0.197 0.020 (0.012) (0.052) (0.074) (0.040) (0.010) Mean outcome 0.309 8.85 159.3 9.84 0.120 Mean outcome for employed 28.85 518.3 32.28 0.392 # observations 6607 6598 6589 6574 6575 # clusters 84 84 84 84 84 Coefficients are from regressing each outcome on a vector of treatment assignments, randomization block fixed effects, and prespecified baseline covariates (measured skills, self-reported skills, education, age, gender, employment, discount rate, risk aversion). Heteroskedasticity-robust standard errors shown in parentheses, clustering by treatment date. Mean outcome is for the control group. All outcomes use a 7-day recall period. Outcomes marked with c use the inverse hyperbolic sine transformation for the treatment effects but the control group means are reported in levels. All monetary figures are reported in South Africa Rands. 1 Rand ≈ USD 0.167 in purchasing power parity terms. The sample sizes differ across columns due to item non-response, mostly from respondents reporting that they don’t know the answer. treatment effects as percentage changes. We assign zeros to job characteristics for non-working respondents (e.g. earnings, hours) and to search measures for non-searching respondents (e.g. number of applications submitted) to avoid sample selection. We thus analyze treatment effects on realized outcomes, rather than latent outcomes that may be non-zero for the non-employed or non-searching. We also estimate quantile treatment effects on selected outcomes, which allows us to focus on the distribution of outcomes for only employed candidates. The estimating equations and variable definitions are prespecified. We report treatment effects on some outcomes that are not prespecified but note when we do so. In Appendix D, we show that our results are robust to prespecified adjustments for multiple testing and omitting the prespecified covariates Xid . 3.3 Certification Improves Labor Market Outcomes The first main effect of certification is to increase employment. Current employment rises by 5.2 percentage points from a control group mean of 30.1 percentage points (Table 1, column 1). We also ask about employment in each calendar month between treatment and endline and show in Table D.10 that certification increases employment in every month between treatment and follow-up. Certification increases average weekly hours worked by 20% (column 2). We code hours worked as zero for non-employed candidates. So the treatment effect on hours may reflect two effects: an extensive margin effect if treatment increases the employment rate and an intensive margin effect if treatment increases the hours that employed candidates work. We adapt a decomposition proposed by Attanasio et al. (2011) to identify these two effects (details in Appendix C). We define the extensive margin effect as the treatment effect on employment multiplied by mean hours worked for employed control group candidates. Intuitively, this is the rise in hours we would see if treatment 14 Table 2: Treatment Effects on Labor Market Outcomes at Extensive and Intensive Margins (1) (2) (3) (4) Hoursc Earningsc Hourly wagec Written contract Total effect 0.201 0.338 0.197 0.020 (0.052) (0.073) (0.039) (0.010) Extensive margin 0.189 0.269 0.141 0.020 (0.042) (0.059) (0.031) (0.005) Intensive margin 0.013 0.069 0.056 -0.000 (0.020) (0.040) (0.028) (0.008) Treatment effect conditional 0.036 0.195 0.159 -0.001 on employment (0.058) (0.113) (0.078) (0.024) This table reports decompositions of treatment effects on job characteristics into extensive and intensive margin effects. The extensive margin effects are the treatment effects on job characteristics due to the treatment effect on employment, evaluated at the mean job characteristics for the control group. The intensive margin effects are the differences between the treatment effects and extensive margin effects, which must be due to changes in job characteristics for the employed candidates in the treatment group. The conditional effect is the implied mean change in job characteristics per employed treatment group candidate. Treatment group employment is 36%, so the conditional effects on all outcomes are roughly three times larger than the corresponding intensive margin effect. Heteroskedasticity-robust standard errors are shown in parentheses, clustering by treatment date. All outcomes use a 7-day recall period. Outcomes marked with c use the inverse hyperbolic sine transformation. increased employment but the marginally and inframarginally employed treated candidates worked the same average hours as the inframarginally employed untreated candidates. We define the intensive margin effect as the difference between the total treatment effect on hours and the extensive margin effect on hours. We find that the entire effect on hours is explained by the extensive margin effect (Table 2, column 1). This shows that treated candidates do not work longer hours conditional on employment, but are simply more likely to be employed. The second main effect of certification is to increase earnings. Weekly earnings increase by 34% (Table 1, column 3). 27 percentage points of the 34% increase in earnings is explained by the rise in employment (the extensive margin effect, shown in Table 2, column 2). This implies an intensive margin increase in earnings of 7 percentage points per candidate. Hourly wages, calculated by dividing earnings by hours, also increase by 20% (Table 1, column 4). The extensive and intensive margins account for respectively 14 and 6 percentage points of the 20% increase in wages (Table 2, column 3). These results show that treatment increases earnings mainly by increasing employment, but also increases earnings conditional on employment. These results are consistent with the conceptual framework: more information about workseeker skills (i.e. types) increases the latent value of some workseeker-job matches, leading to higher employment and mean earnings conditional on employment. This explanation also matches the quantile treatment effects on earnings. These are positive throughout the earnings distribution, consistent with broad increases in match value rather than an increase only in low-value, low-wage matches (Figure 2). The earnings results are not consistent with a special case of the framework where more information increases only job-finding rates but not the output of firm-worker matches, 15 which would not change earnings conditional on employment. Finally, certification increases by 2 percentage points the probability of having a written contract, Statistics South Africa’s definition of a formal job (Table 1, column 5). This effect is entirely explained by the higher employment rate (Table 2, column 4). Consistent with this result, 4 percentage points of the 5.2 percentage point increase in employment are in wage employment, and only 1.2 percentage points into self employment. Wage and self employment status are not prespecified outcomes. The effects on employment and earnings are substantial and easily exceed the cost of the program. The employment effect is almost three times larger than the mean standardized short-run effect size of active labor market programs reviewed by Card et al. (2018), larger than the effect of an intervention that helped similar South African workseekers get reference letters from past employers (Abel et al., 2020), and similar to the effect of a program that subsidized firms to hire South African workseekers from similar backgrounds (Levinsohn et al., 2013). The average earnings gain in the first three months after treatment is 778 South African rand (USD 130 PPP) – 5.6 times the average variable cost of certification alone and 2.3 times the average variable cost of assessment and certification (details in Appendix B). The average weekly earnings gain is equal to 17% of the weekly adult poverty line in South Africa (details in Appendix D.2). 4 Separating Workseeker- and Firm-Side Information Frictions Certification may increase employment and earnings by providing information to firms, to workseekers, or both sides of the market. This distinction matters for modeling information frictions and designing government or market-based remedies to information frictions. In this section, we show that both sides of the market change behavior in response to new information. Our argument proceeds in three steps. First, we show that public certification changes workseekers’ beliefs and search behavior in ways that are consistent with the existence of either firm- or workseeker-side information frictions, or both. Second, we discuss another arm of our workseeker experiment that reveals information only to workseekers. The results of this intervention, compared to the effects of public certification, are consistent with both firm- and workseeker-side information frictions existing. The results are not consistent with only one-sided frictions, either for workseekers or for firms. Third, we discuss an audit-style experiment that reveals information only to firms. The results of this experiment are consistent with firm-side information frictions. 4.1 Certification Changes Job Search and Beliefs We document three patterns in the effects of certification on workseekers’ beliefs and job search behavior. First, certification shifts workseekers’ beliefs about their skills closer to their measured skills. We ask candidates if they think they scored in the top, middle, or bottom third on each of the six assessments compared to other candidates assessed by Harambee. Certification increases 16 Figure 2: Quantile Treatment Effects on Earnings Panel A: Empirical Distributions of Earnings in Control and Public Certification Groups 1 .9 Empirical CDF Control .8 Treated .7 .6 0 2 4 6 8 10 Earnings (inverse hyperbolic sine) Panel B: Quantile Treatment Effects of Public Certification on Earnings 6 Treatment effect on earnings (IHS) 0 2 −2 4 .6 .7 .8 .9 1 Percentile Panel A shows the empirical distributions of earnings in the control and public certification groups. Earnings are the inverse hyperbolic sine transformation of earnings in South African rand, with 1 rand ≈ 0.167 USD in purchasing power parity terms. Earnings are coded as zero for candidates who are not working. The vertical axis in Panel A is truncated below at the 60th percentile because earnings below that value are zero. Panel B shows the quantile treatment effects (QTEs) of public certification. These are unconditional QTEs, estimated without controlling for any covariates or stratum fixed effects. The 95% pointwise confidence intervals allow heteroskedasticity and clustering by treatment date. The confidence intervals exclude zero at all percentiles except 73-74, 86, and 93-99. 17 the fraction of assessments where candidates’ self-assessments match their measured results from 0.39 to 0.55 (Table 3, column 1).15 In contrast, certification has no effect on candidates’ self-esteem (column 2). This shows that their updated beliefs about the skills do not lead to more general updating about their self-worth. Second, certification changes the types of jobs that candidates target. We ask candidates if the types of jobs they are applying for most value communication, concept formation, or numeracy. Certification increases the fraction of candidates searching for jobs that most value the assessment in which they scored strictly highest from 0.16 to 0.21 (column 3).16 Third, candidates use certificates in job applications (columns 4-7). 70% of candidates use the certificates with at least one job application between treatment and endline, with an unconditional average of 6.7 applications sent per candidate.17 Applications with certificates generate an average of 0.43 interviews and 0.11 job offers over the 3-4 months from treatment to endline. The first two patterns suggest a role for workseeker-side frictions: candidates align their beliefs and job search more closely to their assessment results, potentially leading to better outcomes in the labor market. The third pattern suggests a role for firm-side information frictions: candidates use reports with job applications, potentially making the applications more informative to employers, leading to more job interviews and offers. Some combination of these patterns leads candidates to expect 11% more offers in the next month, from a control group mean of 4.2 offers (column 8). Taken together, these three patterns are consistent with certification reducing both firm- and workseeker-side information frictions. However, these three patterns are not sufficient to show whether reducing only one of these frictions could generate the employment and earnings effects of certification. For example, the revised search targeting might by itself increase employment and earnings. Alternatively, the certificates attached to job applications might by themselves increase employment and earnings. We therefore run two more experiments that separately manipulate the information available to firms and workseekers. Before proceeding to the next experiments, we note that certification does not change multiple prespecified measures of job search effort in the month before the endline: the probability of doing any search, number of applications submitted, hours spent searching, and money spent on search (Table D.10). There are two possible explanations for this pattern. First, certification may change 15 This question measures candidates’ beliefs about their assessment results. These may differ from their beliefs about their domain-specific skills if, for example, they believe the assessments are noisy measures of their skills. To address this possibility, we ask candidates if their communication and numeracy skills are in the top, middle, or bottom third of people aged 18-34, from disadvantaged backgrounds, with high school education (the population typically assessed by Harambee). This is not a question about their result on a specific assessment. Treatment increases the share of the two skills where candidates’ beliefs match their assessment results by 12.4 percentage points (standard error 2.2 p.p.). This shows that candidates’ updated beliefs are not assessment-specific. We collect this measure only for a random 50% sample of the first 3,000 candidates to complete the survey. 16 This search targeting measure is not prespecified. The result is similar for the fraction of candidates searching for jobs that most value the assessment in which they think they scored highest. 17 The 6.7 additional applications with reports follows from the 1.682 unit effect on the inverse hyperbolic since of the number of applications in column 5, and the fact that control workseekers send zero applications with reports. 18 Table 3: Public and Private Certification Effects on Beliefs, Search, and Labor Market Outcomes (1) (2) (3) Skill belief > median Targeted accurate self-esteem search Public certification 0.158 0.001 0.052 (0.008) (0.013) (0.010) Private certification 0.123 -0.002 0.047 (0.008) (0.014) (0.010) p: public = private 0.000 0.806 0.698 Mean outcome 0.389 0.553 0.155 # observations 6607 6609 6609 # clusters 84 84 84 (4) (5) (6) (7) (8) Used Applications Interviews Offers Expected reportb with reportb,c with reportb with reportb offersa,c Public certification 0.699 1.682 0.432 0.112 0.106 (0.013) (0.040) (0.023) (0.011) (0.019) Private certification 0.289 0.572 0.144 0.036 0.053 (0.012) (0.033) (0.017) (0.008) (0.023) p: public = private 0.000 0.000 0.000 0.000 0.025 Mean outcome 0.000 0.000 0.000 0.000 4.198 # observations 6609 6598 6597 6597 6531 # clusters 84 84 84 84 84 (9) (10) (11) (12) (13) Hourly Written Worked Hoursc Earningsc wagec contract Public certification 0.052 0.201 0.338 0.197 0.020 (0.012) (0.052) (0.074) (0.040) (0.010) Private certification 0.011 0.066 0.162 0.095 0.017 (0.012) (0.048) (0.078) (0.046) (0.009) p: public = private 0.002 0.011 0.028 0.030 0.769 Mean outcome 0.309 8.848 159.291 9.840 0.120 # observations 6607 6598 6589 6574 6575 # clusters 84 84 84 84 84 Coefficients are from regressing each outcome on a vector of treatment assignments, randomization block fixed effects, and prespecified baseline covariates (measured skills, self-reported skills, education, age, gender, employment, discount rate, risk aversion). Heteroskedasticity-robust standard errors shown in parentheses, clustering by treatment date. Mean outcome is for the control group. Skill belief accurate is the share of the six assessments where the candidate’s perceived tercile matches their actual tercile. Targeted search is an indicator equal to one if the candidate reports mainly applying for jobs that most value the skill in which the candidate scored highest. Above-median self-esteem is an indicator equal to one if the candidate’s response on a shortened version of the Rosenberg (1965) self-esteem scale is above the sample median. All outcomes use a 7-day recall/forecast period unless marked with a (30-day recall/forecast period) or b (since treatment). Outcomes marked with c use the inverse hyperbolic sine transformation for the treatment effects but the control group means are reported in levels. All monetary figures are reported in South Africa Rands. 1 Rand ≈ USD 0.167 in purchasing power parity terms. The sample sizes differ across columns due to item non-response, mostly from respondents reporting that they don’t know the answer. 19 how workseekers search – targeting different jobs and using certificates in applications – without changing their search effort. This is consistent with a special case of the conceptual framework where information frictions change how firms and workseekers match but do not change the share of workseekers who choose to search. Second, certification may temporarily change extensive or intensive margin search effort but the endline may occur too late to detect this change. Employment already rises in the first month after treatment (Table D.10). This suggests that any changes in workseeker behavior that increase employment occur soon after treatment. The search effort questions use 7- or 30-day recall periods, which miss the period soon after treatment when candidates may have increased effort and found jobs. The questions on certificate use ask about the entire period between treatment and the endline survey, which will capture any short-term changes in search behavior.18 4.2 Workseekers Face Information Frictions In this section, we explore whether revealing information only to workseekers can replicate the employment and earnings effects of our certification intervention. If not, then it is likely that certification does not work by addressing only workseeker-side information frictions. We implement a ‘private’ certification intervention, distinct from the ‘public’ certification intervention described above. Candidates assigned to the private certification intervention receive an unbranded, anonymous certificate with the assessment results rather than the branded, identifiable ‘public’ certificate (Figure 3). We interpret the private treatment as primarily providing information to the workseekers about their own types. Candidates in this group receive only one black-and-white, unbranded copy of the report, printed on low-quality paper, and do not receive an electronic version. Candidates receive a briefing from a psychologist about the assessment results. But this briefing does not encourage them to share the certificate with firms or suggest that this is possible, unlike the briefing received by candidates in the public certification group. Candidates in the public certification, private certification, and control groups all receive the same one hour of job search counselling and email with job search advice. We assign 2,114 candidates assessed over 27 assessment days to private certification. We simultaneously randomize days to public certification, private certification, and control. The three groups are balanced on baseline characteristics (Table D.1). The private and public certification interventions have similar effects on workseekers’ beliefs and search targeting. Private certification makes workseekers’ beliefs about their own skills more accurate and has no effect on self-esteem (Table 3, columns 1-2).19 The private and public effects 18 Consistent with this timing explanation, effects on all search effort measures are marginally larger for respondents with a shorter time between treatment and endline. This result is robust to instrumenting the treatment-to-endline time with the random order in which candidates were assigned to be surveyed. 19 The private certification effect on beliefs about skills is slightly smaller than the public certification effect. The former effect may be smaller because the public treatment conveys information differently (e.g. the branding makes 20 Figure 3: Sample Private Certificate REPORT ON CANDIDATE COMPETENCIES -Personal Copy- This report contains results from the assessments you took at Harambee in Phase 1 and Phase 2. These results can help you learn about some of your strengths and weaknesses and inform your job search. You completed assessments on English Communication (listening, reading and comprehension) and Numeracy today in Phase 2. In Phase 1, you completed a Concept Formation assessment which asked you to identify patterns. 1. The Numeracy tests measure various maths abilities. Your score is the average of the two maths tests you did today at Harambee. 2. The Communication test measures English language ability through listening, reading and comprehension. 3. The Concept Formation test measures the ability to understand and solve problems. Candidates with high scores can generally solve complex problems, while lower scores show an ability to solve less complex problems. You also did some games and questionnaires to measure your soft skills: 4. The Planning Ability Test measures how you plan your actions in multi-step problems. Candidates with high scores generally plan one or more steps ahead in solving complex problems. 5. The Focus Test looks at your ability to pick out which information is important in confusing environments. Candidates with high scores are able to focus on tasks in distracting situations. 6. The Grit Scale measures candidates’ determination when working on difficult problems. Candidates with high scores spend more time working on the problems rather than choosing to pursue different problems. Your results have been compared to a large group of young South African job seekers who have a matric certificate, are from socially disadvantaged backgrounds and have been assessed by Harambee. You scored in the MIDDLE THIRD of candidates assessed by Harambee for Numeracy, MIDDLE THIRD for Communication, LOWER THIRD for Concept Formation, LOWER THIRD for Planning Ability, MIDDLE THIRD for Focus and TOP THIRD for the Grit Scale. Planning Ability DISCLAIMER Please note that this is a confidential assessment report and is intended for use by the person specified above. Assessment results are not infallible and may not be entirely accurate. Note: This figure shows an example of the certificates given to candidates in the private treatment arm. The certificates contain the candidate’s assessment results but no identifying information and no branding. Each candidate received one copy of this certificate. 21 on search targeting are almost identical (column 3). Candidates in the private arm expect to receive 5% more offers than control candidates, significantly less than the 11% increase in expected offers in the public arm (column 8). This suggests that workseekers view the new information as useful, but less useful than also being able to communicate their assessment results to firms. The private certification intervention has substantially smaller effects than public certification on candidates’ outcomes in the labor market. Private certification effects on the probability of employment and hours worked are positive but small, not significantly different from zero, and significantly smaller than the public certification effects (columns 9-10). Private certification increases earnings and hourly wages but both effects are less than half the size of the public certification effects and significantly smaller (columns 11-12). Results from both the public and private certification interventions are consistent with quantitatively important information frictions facing both workseekers and firms. Giving information to workseekers in both the public and private certification interventions increases the accuracy of their beliefs about their skills, allowing better search targeting, and leading to higher earnings. When workseekers can also credibly convey that information to firms, as in the public intervention, they are more likely to be employed and have even higher earnings and wages. There are two ways that these results might reflect only firm-side or only workseeker-side information frictions, but we view both as unlikely. First, the private certification may deliver some information to firms and this information, rather than changes in workseeker beliefs or search targeting, may generate the positive private effect on earnings. This explanation is consistent with the fact that some candidates report using private certificates in job applications, although private effects on certificate use are on average one third as large as the public effects (Table 3, columns 4-7). We cannot conclusively rule out this interpretation but we view it as unlikely. The private reports do not have the candidate’s name and identity number, so they cannot be linked to a particular candidate. They have no branding, from Harambee or the World Bank. They do not explain that Harambee has used these assessments widely to place candidates with companies or that assessments predict workplace productivity and do not link to a website. None of the 15 hiring managers interviewed during piloting reported that they would view the private reports as credible. Second, the public certification effects on employment and earnings may be larger than the private effects because workseekers incorrectly believe that firms face information frictions. Under this explanation, workseekers in the public certification group believe that firms are more likely to respond to job application submitted with certificates, hence they search more or search differently it more credible to workseekers) or because the information is more likely to be retained (e.g. workseekers are more likely to keep copies of the public report or discuss it in recent job interviews). To separate these hypotheses, we measure workseekers’ beliefs about their skills using a text message survey 2-3 days after treatment. The public and private effects in this survey are not different to each other, suggesting that workseekers’ beliefs update in the same way straight after receiving the reports and that the difference in the endline survey 3-4 months later is due to differential retention. See Appendix D and Table D.8 for details. 22 using a method or in a time period we do not measure. These changes in search behavior may generate higher employment and earnings, even if firms face no information frictions and do not respond to certificates. In the next section, we address this possibility by discussing an experiment that directly manipulates firms’ information, without any scope for changes in workseeker behavior. 4.3 Firms Face Information Frictions In this section, we show that revealing information only to firms changes their responses to job applications, without allowing any potentially mediating behavior by workseekers. This is consistent with firms facing information frictions and the employment and earnings effects of certification being partly explained by firm-side information frictions. We describe results from an audit-style study, with more details on the experiment in Appendix E. We invite a random sample of assessed candidates to send us a resume that we will forward to prospective employers on their behalf. We create a list of job vacancies by scraping online job advertisements. We eliminate scam vacancies and vacancies that require work experience or university education, where many candidates in our sample would be ineligible. We send resumes from 4 randomly chosen candidates to each vacancy, each from a different email address. We generate two outcome variables based on the email responses from firms. ‘Interview invitations’ are invitations to interview with the firm. ‘Any responses’ are similar to ‘callbacks’ in other audit studies and include interview invitations and requests to provide more information by email or by visiting the firm in person. We randomize each vacancy to receive either 1 or 3 resumes with public certificates attached. We also randomize which of the resumes are chosen to receive public certificates. This design motivates the estimating equation Yrv = Certificaterv · β1 + Certificaterv · HighIntensityv · β2 + Vv + Xr · Γ + Erv + rv , (2) where Yrv is the response to resume r sent to vacancy v , Certificaterv is an indicator equal to one if the application includes a public certificate, HighIntensityv is an indicator equal to one if the vacancy receives 3 applications with certificates rather than 1, Vv is a vector of vacancy fixed effects that subsumes the main effect of HighIntensityv , Xv is a vector of prespecified resume covariates, and Erv is a vector of fixed effects for the email addresses used to submit the applications. We cluster standard errors by resume and vacancy.20 β1 , the effect of using a public certificate when other applications do not, is positive. Applications with a public certificate are 1.6 percentage points more likely to get any response and 1 percentage point more likely to get an interview invitation (Table 4, columns 2 and 4). These are substantial 20 Like most audit studies, we submit the same resume to multiple vacancies. Each resume includes a certificate for half of these vacancies. Audit studies generally cluster standard errors by resume (Neumark, 2018). Abadie et al. (2017) recommend clustering by the unit at which treatment is assigned. We therefore cluster by both vacancy and resume. Results are very similar when clustering only by vacancy or only by resume. 23 Table 4: Treatment Effects of Additional Information in Audit Study (1) (2) (3) (4) Any response Interview request Certificate (β1 ) 0.015 0.016 0.009 0.010 (0.009) (0.009) (0.004) (0.006) Certificate × HighIntensity (β2 ) -0.027 -0.028 -0.014 -0.017 (0.013) (0.014) (0.009) (0.010) Mean outcome 0.130 0.130 0.088 0.088 # applications 3992 3992 3992 3992 # vacancies 998 998 998 998 # resumes 717 717 717 717 Vacancy fixed effects × × Email address fixed effects × × Resume covariates × × Note: Coefficients are from regressing each outcome on a vector of treatment assignments and, in columns 2 and 4, vacancy fixed effects, email address fixed effects, a vector of prespecified workseeker covariates (measured skills, education, age, gender, past employment, and the scan quality of documents they include in their application). The vacancy-level treatment HighIntensityv is included in columns 1 and 3 but omitted in columns 2 and 4 because it is collinear with the vacancy fixed effects. Heteroskedasticity-robust standard errors shown in parentheses clustered by resume and vacancy. The mean outcomes are for applications sent without public certificates to vacancies that receive only one application with a public certificate. effects, both equal to 11% of the control group means, although they are only statistically significant at the 10% level. These results show that more informative applications lead to higher callback and interview invitation rates in a low-information environment. This suggests firm-side information frictions play a role in the earnings and employment effects of public certification. Combining this result with the observed effects of the public and private certification on workseekers’ beliefs, search behavior, and outcomes in the labor market suggests that both firms and workseekers’ face information frictions. β2 , the difference between the effect of being the only application with a public certificate sent to a vacancy and the effect of being one of multiple applications with public certificates sent to a vacancy, is negative. Applications that include a public certificate are 2.8 percentage points less likely to get a response and 1.7 percentage points less likely to get an interview invitation when they compete against other applications with reports (Table 4, columns 2 and 4). However, more informative applications may still be valuable in a higher-information environment for job offers, which we do not observe in the audit study, than callbacks and interviews. If firms use callbacks and interviews to get more information, then certificates may allow them to interview fewer candidates and still make better-matched hires. Jarosch and Pilossoph (2019) make a similar argument, both theoretically and empirically, about audit studies that manipulate firms’ information about applicants’ employment history. There are, however, some caveats to the interpretation of the audit study results. This examines only one hiring method (online applications) and one stage of that process (interview invitations). 24 These are standard limitations of correspondence-based audit studies. But it does mean that the design cannot easily quantify the importance of firm-side information frictions relative to workseeker-side information frictions on the same outcome. Furthermore, we randomly match workseekers to vacancies in the audit study. This omits any role for search targeting, which the public and private certification results suggest may be important. We therefore view the audit study as less important evidence than the arms of the workseeker-facing experiments. It simply provides additional evidence that the difference between the public and private certification results can be explained by firm-side information frictions. 5 What Do Workseekers and Firms Learn from Skill Certification? The preceding two sections show that skill certification provides information that improves workseekers’ outcomes in the labor market. In this section, we explore what workseekers and firms learn from skill certification, what this implies for the effects of certification on different types of workseekers, and what this might imply for the effects of certification at scale. This section relies on smaller experiments and heterogeneity analysis of the main experiments, so we view these as secondary results. 5.1 Assessment Results Matter, Not Just Being Assessed The public certification and audit results above are consistent with three interpretations. First, our preferred interpretation is that firms and workseekers acquire information about workseekers’ skills from the assessment results. Second, firms may acquire information about workseekers’ tenacity or proactivity from their choice to get assessed, not their actual assessment results. Third, the assessment results may provide no useful information to firms but may be visually appealing or attention-grabbing because they are colorful, branded, and printed on high-quality paper. In this section, we discuss two smaller experiments whose results are consistent with the first, but not the second or third interpretations. The first interpretation is also consistent with the private certification results, while the second and third are not. In the first experiment, we manipulate information about workseekers’ assessment results, holding constant the information that workseekers have been assessed and any visual appeal of the certificates. We randomly assign 254 candidates from our workseeker sample, assessed over 3 days, to a ‘placebo’ certification group. These candidates receive placebo certificates that are identical to the public certificates except that the actual assessment results are omitted (Figure F.1) and the psychologist’s briefing does not discuss the assessment results. The placebo certification treatment has minimal effects on labor market outcomes (Table F.1). It increases an index of labor market outcomes by 0.03 standard deviation. This is not significantly different to zero and is significantly smaller than the public certification effect of 0.12 standard deviation. This index is an inverse covariance-weighted average of the five labor market outcomes 25 discussed in Section 3.3: employment, hours, earnings, wages, and contract status. The placebo certification effects on the five individual outcomes are all smaller than the public certification effects and are on average only 26% as large. But we cannot reject equality of the public and placebo effects for some of the individual outcomes because the small size of the placebo sample leads to large standard errors. The second experiment measures firms’ willingness-to pay (WTP) for information on workseekers’ assessment results, conditional on knowing candidates have been assessed. We recruit 69 establishments located in commercial areas near the low-income residential areas in Johannesburg where most workseekers in our sample live and are likely to work.21 We conduct a survey and WTP exercise with the person responsible for hiring decisions at each of these establishments. We show this person a secure online database containing assessment results, contact information, and selected resume-style information for our 6,891 candidates. This database allows users to filter and search for candidates with specific assessment results and obtain their contact information. See Figures G.1 and G.2 for selected screenshots of the database. We use a Becker-DeGroot-Marshak mechanism to measure WTP for access to this database relative to a placebo database with candidates’ contact information and selected resume-style information, but no assessment results.22 Firms’ WTP for access to the database with assessment results is substantial: 68% of firms report positive WTP and the unconditional mean WTP is 1,161 South African rand or USD 195 PPP (Figure G.3). Mean WTP is 224% of the mean weekly earnings for employed candidates in our workseeker sample. This shows firms value information on specific assessment results, conditional on knowing candidates have been assessed. Both the placebo experiment and WTP measurement are consistent with the first but not second or third interpretations above: information about assessment results is valuable, not just information about whether candidates have been assessed or any visual appeal of the certificates. This provides additional support for our preferred interpretation: public certification provides information about workseekers’ types and facilitates more productive firm-worker matches. 21 We recruit establishments by asking if they are willing to participate in a study on hiring and tell them we can provide some useful information on hiring. We restrict the sample to establishments that have hiring responsibilities, either single-establishment firms or branches of larger firms that hire independently. Most firms are in retail, have multiple entry-level workers, expect to hire entry-level workers in the next year, and take on average four weeks to fill a vacancy (Table G.1). 22 We tell firms the database normally costs 10,000 South African rand (USD 1,670 PPP) for three months access, ask how much they are willing to pay for access, and then randomly offer them a discount between 0 and 100%. If their stated WTP is higher than the normal price minus the discount, we give them access to the database. If their stated WTP is below the normal price minus the discount, we give them access to a placebo database with candidates’ contact information and selected resume-style information but no skill assessment results. We first explain the entire mechanism and run a practice round with a bar of chocolate. 26 5.2 Certification Facilitates Horizontal More Than Vertical Differentiation Our conceptual framework distinguishes two types of workseeker differentiation. Under horizontal differentiation, type i workseekers are more productive than type j workseekers in type i jobs, and vice versa. Under vertical differentiation, type i workseekers are more productive than type j workseekers in both type i and j jobs. The distinction is important for understanding how information frictions affect different types of workseekers. Under horizontal differentiation, alleviating information frictions can help both types of workseekers by matching them with jobs where they are more productive. Under vertical differentiation, alleviating information frictions can hurt type j workseekers by reducing their probability of being mistaken for more productive type i workseekers. In this section, we discuss three relevant patterns in our data. Two are consistent with horizontal differentiation and one is inconsistent with vertical differentiation, providing suggestive evidence of horizontal differentiation in this setting. First, there is substantial heterogeneity in firms’ relative demand for different skills. We show this using an incentivized choice experiment with the sample of 69 establishments described in the previous subsection. We ask the person at each establishment responsible for hiring to rank profiles of seven hypothetical candidates and tell them we will use their ranking to match them with workseekers from the online database, following Kessler et al. (2019). Six of the profiles have middle terciles for five assessments, and a top tercile for one assessment. There is substantial variation in firms’ relative ranking of profiles (Table G.2). All six profiles’ median rank is between second and fourth. The share of firms ranking each profile highest ranges from 6 to 33%. The seventh profile has middle terciles for all six assessments and has a one-year post-secondary education certificate, while the other six profiles have only completed secondary school. Only 9% of firms rank this profile first and 76% of firms rank this last, showing that firms value the assessed skills relative to an alternative signal of productivity in which workseekers might invest.23 Second, assessment results are weakly correlated across skills within candidate. Numeracy and concept formation are most highly correlated, with ρ ≈ 0.5. But most other pairwise correlations are substantially lower, with ρ < 0.1 for some pairs of skills (Table A.2). As a result, most candidates’ certificates show substantial variation across skills. Table A.3 shows that 88% of the candidates have at least one top tercile but only 24% have four or more top terciles and only 2.3% have all top terciles. 76% of the candidates have at least one bottom tercile but only 12% have four or more bottom terciles and only 0.7% have all bottom terciles. 64% of candidates have both top and bottom terciles. Other studies that measure multidimensional skills also find weak correlations across skills within candidates (Almlund et al., 2011; Poropat, 2009). 23 We conduct a second experiment where we ask firms to rank profiles with assessment results shown for some skills and concealed for others. This assess whether firms value information about specific skills as well as the level of the skills. The two experiments may yield different results if, for example, firms find skill S1 most valuable but believe the assessments of skill S2 yield more new information. This second experiment also shows substantial heterogeneity in firms’ ranking of different profiles. 27 Third, we do not find strong evidence of vertical differentiation in the public certification experiment. To test for vertical differentiation, we construct three indices that combine the six assessment results in different ways: the number of top terciles minus bottom terciles, the first principal component of the cardinal scores, and a weighted average of the cardinal scores with weights based on their association with earnings.24 The first index weights all skills equally, the second gives more weight to skills that are highly correlated with each other, and the third gives more weight to skills with higher associations with earnings. For each index, we construct an indicator for above-median values of the index. We then include this indicator and its interactions with treatment assignments in equation (1). The interaction effects with public certification on employment are smaller than 2 percentage points and not significantly different to zero for all indices (Table D.9) panel A.25 Taken together, these results are more consistent with horizontal than vertical differentiation. Firms and workseekers both seem to face information frictions, and certification provides information that facilitates more productive firm-worker matches that generate higher employment and earnings. Employment and earnings rise for many different types of workers, not just those with high values of a skill index of the six assessment scores. This is consistent with models of multidimensional skill where information frictions can lead to poor matches between workseeker skills and firm requirements (Fredriksson et al., 2018; Guvenen et al., 2020; Lise and Postel-Vinay, 2020). However, our experiment is not designed to provide a general test of horizontal against vertical differentiation. Certification may facilitate vertical rather than horizontal differentiation when it is based on assessment of a single skill or assessments of multiple highly correlated skills; or when certificates report a single summary measure of multiple skills. Certification may also facilitate vertical rather than horizontal differentiation when it covers a larger share of the workforce. Our sample excludes workseekers with advanced degrees or less than completed secondary education, whose skills may be more vertically differentiated from the workseekers in our sample. 5.3 Certification Is More Effective When Other Information on Workseekers’ Skills Is Limited If certification changes labor market outcomes by providing information about workseekers’ skills, then it should be most effective when there are limited alternative sources of information on workseekers’ skills. These sources might include past work experience and post-secondary education, which allow workseekers and firms to learn about workseekers’ productivity in specific tasks. We 24 The weights equal the coefficients from regressing earnings on the cardinal scores using control group data. Results are similar for weighted averages based on the coefficients of regressions of control group earnings on polynomial or spline functions of the skills. 25 Results are similar using the continuous indices instead of binary indicators and when using alternative model specifications: allowing nonlinear interactions between skill indices, using different single indices, or using machine learning methods to estimate heterogeneous treatment effects simultaneously across all individual scores. 28 test this idea by augmenting equation (1) to include interactions between treatment and proxies for alternative sources of information. Public certification effects on employment are 2.7 percentage points smaller for candidates with post-secondary education (standard error 2.8 p.p.) and 4.3 percentage points smaller for candidates with prior work experience (standard error 3.2 p.p.) (Table D.9, panel B). We also estimate the latent probability of being employed at endline as a single summary measure.26 Candidates with above-median latent probabilities of employment have 6.9 percentage point smaller public certification effects than candidates with below-median latent probabilities (standard error 2.8 p.p.). These results show that certification can substitute for traditional sources of information about workseekers’ skills.27 This is consistent with evidence that educational qualifications are more useful for members of groups facing statistical discrimination (Arcidiacono et al., 2010). 5.4 Skill Certification at Different Scales We show that skill certification at a relatively small scale increases employment and earnings for certified workseekers. In this section we discuss conditions under which effects may vary with the scale of skill certification.28 First, employment and earnings effects may depend on scale if certified workseekers displace non-certified workseekers. It is unlikely that our experimental results are due to displacement of non-certified workseekers in the control group. We certify only 2,247 workseekers in a metropolitan area with roughly 8 million people and 2 million employed workers (Statistics South Africa, 2016). The probability of certified and control group workseekers applying for the same jobs by chance is very small, and Harambee does not encourage recently-assessed workseekers to apply to specific jobs or search for work in specific areas. It is possible that certified workseekers displace non-certified workseekers who are not part of the experimental sample. We cannot directly test for this, but we can evaluate the mechanisms that might generate it. Displacement is less likely if certification improves match quality and hence increases the share of latent vacancies that are worth filling, as in our conceptual framework and general equilibrium models of information frictions (Donovan et al., 2018; Jovanovic, 1979; 26 We estimate the latent probabilities following Abadie et al. (2018). We regress endline employment on baseline demographics, education, assessment results, beliefs about assessment results, employment, earnings, and search behavior in the control group. We use the predicted values from these regressions in all treatment groups as latent probabilities for employment, adjusting the predicted values in the control group using leave-one-out estimation to avoid overfitting. 27 This result is not explained by a correlation between workseekers’ skills and their education and past employment. We regress employment on treatment assignments, a single index measure of skill from Section 5.2, a measure of information about workseekers’ skills from this section, and a full set of interactions. The interactions between public certification and the single index skill measure remain close to zero, while the interactions between public certification and the measure of information about workseekers’ skills remain negative. 28 There are few existing papers that study how the effects of specific active labor market policies change with scale. For job search assistance policies specifically, Crépon et al. (2013) and Lise et al. (2004) find larger-scale policies generate negative spillovers on non-participants, while Blundell et al. (2004) find no spillovers on non-participants. 29 Menzio and Shi, 2011). Displacement is more likely if firms value certification for some reason other than information (e.g. visual appeal) or if certification helps firms to identify a small set of universally-demanded workseekers and compete for them. Our results are more consistent with the match quality mechanism. We find that firms’ demand for different skills is heterogeneous, firms value learning about workseekers’ specific skill types, and the gains from certification are not limited to workseekers with specific skill profiles. All these patterns suggest that firms and workseekers use certification to learn about workseekers’ skills and facilitate better matches between workseekers’ skills and firms’ demand. We also find that certification increases earnings and hourly wages conditional on employment, suggesting that certified workseekers are in more productive matches. We do find that the callback premium to certification drops when certified applicants compete against each other. This is consistent with some certified workseekers displacing other certified workseekers from interviews. However, as discussed in Section 4.3, this does not necessarily imply that certified workseekers displace other certified workseekers from hiring. Certification may allow firms to call back and interview fewer candidates and still make better-matched hires, as shown by Jarosch and Pilossoph (2019). Second, employment and earnings effects may depend on scale if the extent of limited information varies across the population of either firms or workseekers. Consider the case where the population is divided into fraction p of uninformed workseekers, who do not know their skills and cannot convey their skills to firms, and fraction 1 − p of informed workseekers, who know their skills and can convey this information to firms. Assessing and certifying more than p share of workseekers will have limited returns. Our finding that certification has larger employment and earnings effects when there are limited alternative sources of information on workseekers’ skills is consistent with this possibility. Our experiment does not identify the population shares of workseekers or firms facing information frictions. The share of relatively uninformed types may be higher in our sample than the population, as we study workseekers with poor baseline labor market outcomes. But Harambee’s workseeker recruitment does not explicitly advertise assessments or information frictions, so workseekers are unlikely to select into the sample specifically for assessment and certification. Third, employment and earnings effects may depend on scale if certificate (non-)use conveys information in general equilibrium. If, for example, all workseekers get assessed and certified but only use certificates when applying to vacancies where their match quality is high, then firms may infer that workseekers without certificates are poor matches for these vacancies. In another example, some firms may choose not to use assessments in hiring if assessments are costly and they believe they can infer workseekers’ types by observing their interactions with other firms (Lockwood, 1991). Our experiments cannot speak to these general equilibrium mechanisms. But adding either of these two mechanisms to our conceptual framework still predicts that any non-zero use of assessment and certification will raise employment and earnings relative to no assessment and certification. Even if reducing information frictions has decreasing effects on employment and earnings at 30 larger scales, it may still raise workseeker or firm welfare by reducing job search costs, vacancy posting costs, and the frequency of bad hires that lead to separations. This interpretation is consistent with models showing that firm- and workseeker-level search and matching frictions, including information frictions, can lower aggregate utility through multiple mechanisms, not just through unemployment (Donovan et al., 2018; Mortensen and Pissarides, 1999; Poschke, 2019). 6 Conclusion Workseekers make job search decisions and firms make hiring decisions using potentially limited information about workseekers’ skills. Providing more information about workseekers’ skills may reduce these information frictions and hence improve workseekers’ outcomes in the labor market. We show that assessing workseekers’ skills and communicating the assessment results to both workseekers and firms increases assessed workseekers’ employment by 17% (5 percentage points), earnings by 34%, and hourly wages by 20%. This shows that certification gets more workseekers into work and into higher-paying jobs. The main contribution of our paper is to show that this reflects both workseeker- and firm-side information frictions. The distinction between workseeker- and firm-side frictions is important, as it informs how economists, government, or private firms might design information-provision products. We study a context and sample where information frictions are likely: work experience is limited, education-skill relationships are relatively weak, hiring mistakes are costly, and reservation and minimum wages are relevant. However, none of these features is unique to young workseekers in South Africa. Formal education qualifications are weakly related to measured skills in many countries (Pritchett, 2013). Many labor markets face more regulations governing hiring, firing, and probation than in South Africa (Botero et al., 2004). Hiring mistakes may be costly even when separations are unregulated, due to reposting and retraining costs. High rates of youth unemployment in many countries are consistent with information frictions, as youths have less job search and work experience that can reveal their skills to themselves or to firms (International Labour Organization, 2017). Our results suggest that, in similar contexts, providing information about workseekers’ skills may be a valuable focus of government policy. Some existing job search assistance programs offer skill assessments to workseekers (McCall et al., 2016). Adding certification to these assessments might enhance their effectiveness at low cost. We find that adding certification to an existing assessment program generates earnings gains for workseekers that easily exceed the cost of both assessment and certification. Government involvement, through public-sector assessment programs or subsidies to private-sector assessments, is likely to be particularly important for credit-constrained workseekers (Abebe et al., 2020b). Better information about workseekers’ skills could also come from more accurate assessments during formal education (MacLeod et al., 2017). Our results suggest there may also be scope for market-based provision of information about 31 workseekers’ skills. We show that firms are willing to pay for access a database with information on workseekers’ skill assessment results and contact information. We also ask workseekers in our sample how much of a hypothetical job search subsidy they would be willing to spend on certification. They report 17%, compared to 24% on training and 27% on transport, suggesting the possibility of charging workseekers for assessment services. Some large firms already use in-house psychometric assessments in hiring (Autor and Scarborough, 2008; Hoffman et al., 2018). Anecdotally, psychometric assessments seem rarer in small firms, perhaps because in-house assessment systems are unlikely to be cost-effective when hires are infrequent. There are some third-party providers of assessment services around the world, including Harambee, LinkedIn, and the Manpower Group. Our results show that providing more information through certification can be valuable even in a labor market where some firms already use assessments, suggesting scope to grow this market. There are important market design questions around third-party provision that might be addressed in future work, such as which side(s) of the market will pay for assessment services, how third-party providers can establish reputations, how precisely or coarsely information should be reported, and under what conditions participants will opt into or out of assessment. This work might incorporate existing models of screening and signaling when both agents and principals have limited information (Alonso, 2018; Rosar and Schulte, 2012). Our results also motivate future work on the interaction between different information provision mechanisms. For example, we find that public certification is most effective for workseekers with less work experience and without university education. This suggests that skill assessment and certification can substitute for alternative sources of information about workseekers’ skills. Future work could examine conditions under which skill assessment and certification are complements or substitutes for network referrals, reference letters, or outsourcing agencies.29 References Abadie, A., S. Athey, G. Imbens, and J. Wooldridge (2017): “When Should You Adjust Standard Errors for Clustering?” Working Paper 24003, National Bureau of Economic Research. Abadie, A., M. Chingos, and M. West (2018): “Endogenous Stratification in Randomized Experiments,” Review of Economics and Statistics, 100, 567–580. Abebe, G., S. Caria, M. Fafchamps, P. Falco, S. Franklin, and S. Quinn (2020a): 29 We find one result consistent with certification enhancing the effectiveness of referrals, potentially by helping network links to target referrals or making their referrals more credible to employers. Public certification slightly increases the probability of securing a job through a formal application or interview after a referral. There is no large or significant treatment effect on the probability of securing a job in other ways we measure: by approaching an employer in person, dropping off an application, emailing an application, getting hired by a social contact directly, or working at an employment broker. However, this result is only marginally statistically significant once we account for multiple testing across the different ways of finding a job. Hence, we view this as a suggestion for future work, rather than a core result. 32 “Anonymity or Distance? Job Search and Labour Market Exclusion in a Growing African City,” Review of Economic Studies, forthcoming. Abebe, G., S. Caria, and E. Ortiz-Ospina (2020b): “The Selection of Talent: Experimental and Structural Evidence from Ethiopia,” Manuscript, University of Bristol. Abel, M. (2019): “Unintended Labor Supply Effects of Cash Transfer Programs: New Evidence from South Africa’s Pension,” Journal of African Economies, 28, 558–581. Abel, M., R. Burger, and P. Piraino (2020): “The Value of Reference Letters: Experimental Evidence from South Africa,” American Economic Journal: Applied Economics, forthcoming. Ahn, S., R. Dizon-Ross, and B. Feigenberg (2019): “Improving Job Matching Among Youth,” Working Paper, Columbia University. Aigner, D. and G. Cain (1977): “Statistical Theories of Discrimination in Labor Markets,” Industrial and Labor Relations Review, 30, 175–187. Alfonsi, L., O. Bandiera, V. Bassi, R. Burgess, I. Rasul, M. Sulaiman, and A. Vitali (2017): “Tackling Youth Unemployment: Evidence from a Labor Market Experiment in Uganda,” Manuscript, University College London. Almlund, M., A. Duckworth, J. Heckman, and T. Kautz (2011): “Personality Psychology and Economics,” in Handbook of the Economics of Education, ed. by E. Hanushek, S. Machin, and L. Woessmann, Elsevier, 1–181. Alonso, R. (2018): “Recruiting and Selecting for Fit,” Manuscript, LSE. Altmann, S., A. Falk, S. Jäger, and F. Zimmermann (2018): “Learning about Job Search: A Field Experiment with Job Seekers in Germany,” Journal of Public Economics, 164, 33–49. Altonji, J. and C. Pierret (2001): “Employer Learning and Statistical Discrimination,” Quarterly Journal of Economics, 116, 313–335. Anderson, M. (2008): “Multiple Inference and Gender Differences in the Effects of Early Intervention: A Reevaluation of the Abecedarian, Perry Preschool, and Early Training Projects,” Journal of the American Statistical Association, 103, 1481–1495. Arcidiacono, P., P. Bayer, and A. Hizmo (2010): “Beyond Signaling and Human Capital: Education and the Revelation of Ability,” American Economic Journal: Applied Economics, 2, 76–104. Attanasio, O., A. Kugler, and C. Meghir (2011): “Subsidizing Vocational Training for Disadvantaged Youth in Colombia: Evidence from a Randomized Trial,” American Economic Journal: Applied Economics, 3, 188–220. Autor, D. and D. Scarborough (2008): “Does Job Testing Harm Minority Workers? Evidence from Retail Establishments,” Quarterly Journal of Economics, 123, 219–277. Bassi, V. and A. Nansamba (2020): “Screening and Signaling Non-Cognitive Skills: Experimental Evidence from Uganda,” Manuscript, University of Southern California. Beaman, L., N. Keleher, and J. Magruder (2018): “Do Job Networks Disadvantage Women? Evidence from a Recruitment Experiment in Malawi,” Journal of Labor Economics, 36, 121–153. 33 Beaman, L. and J. Magruder (2012): “Who Gets the Job Referral? Evidence from a Social Networks Experiment.” American Economic Review, 102, 3574–3593. Belot, M., P. Kircher, and P. Muller (2018): “Providing Advice to Jobseekers at Low Cost: An Experimental Study on Online Advice,” Review of Economic Studies, 86, 1411–1447. Benjamini, Y., A. Krieger, and D. Yekutieli (2006): “Adaptive Linear Step-Up Procedures That Control the False Discovery Rate,” Biometrika, 93, 491–507. Bertrand, M. and B. Crépon (2019): “Teaching Labor Laws: Evidence from a Randomized Trial in South Africa,” Manuscript, University of Chicago and CREST. Bhorat, H., T. Caetano, B. Jourdan, R. Kanbur, C. Rooney, B. Stanwix, I. Woolard, L. Patel, Z. Khan, L. Graham, K. Baldry, and T. Mqehe (2016): “Investigating the Feasibility of a National Minimum Wage for South Africa,” Manuscript, University of Johannesburg: Centre for Social Development in Africa and University of Cape Town: Development Policy Research Unit. Bhorat, H. and H. Cheadle (2009): “Labour Reform in South Africa: Measuring Regulation and a Synthesis of Policy Suggestions,” Development Policy Research Unit Working Paper 09/139, University of Cape Town. Blundell, R., M. C. Dias, C. Meghir, and J. Van Reenen (2004): “Evaluating the Employment Impact of a Mandatory Job Search Program,” Journal of the European Economic Association, 2, 569–606. Botero, J., S. Djankov, R. L. Porta, F. Lopez-de Silanes, and A. Shleifer (2004): “The Regulation of Labor,” Quarterly Journal of Economics, 119, 1339–1382. Bryan, G., S. Choudhury, and A. M. Mobarak (2014): “Under-Investment in a Profitable Technology: The Case of Seasonal Migration in Bangladesh,” Econometrica, 82, 1671–1748. Card, D., J. Kluve, and A. Weber (2018): “What Works? A Meta-Analysis of Recent Active Labor Market Program Evaluations,” Journal of the European Economic Association, 16, 894–931. Chandrasekhar, A. G., M. Morten, and A. Peter (2020): “Network-Based Hiring: Local Benefits; Global Costs,” Working Paper 26806, National Bureau of Economic Research. Conlon, J., L. Pilossoph, M. Wiswall, and B. Zafar (2018): “Labor Market Search with Imperfect Information and Learning,” Working paper 24988, National Bureau of Economic Research. Crépon, B., E. Duflo, M. Gurgand, R. Rathelot, and P. Zamora (2013): “Do Labor Market Policies Have Displacement Effects? Evidence from a Clustered Randomized Experiment,” Quarterly Journal of Economics, 128, 531–580. Donovan, K., J. Lu, T. Schoellman, et al. (2018): “Labor Market Flows and Development,” in 2018 Meeting Papers, Society for Economic Dynamics, vol. 976. Falk, A., D. Huffman, and U. Sunde (2006): “Self-Confidence and Search,” IZA Discussion Paper 2525. Farber, H. and R. Gibbons (1996): “Learning and Wage Dynamics,” Quarterly Journal of Economics, 111, 1007–1047. 34 Franklin, S. (2017): “Location, Search Costs and Youth Unemployment: Experimental Evidence from Transport Subsidies,” The Economic Journal, 128, 2353–2379. Fredriksson, P., L. Hensvik, and O. N. Skans (2018): “Mismatch of Talent: Evidence on Match Quality, Entry Wages, and Job Mobility,” American Economic Review, 108, 3303–3338. Garlick, R., K. Orkin, and S. Quinn (2019): “Call Me Maybe: Experimental Evidence on Frequency and Medium Effects in Microenterprise Surveys,” World Bank Economic Review, forthcoming. Guvenen, F., B. Kuruscu, S. Tanaka, and D. Wiczer (2020): “Multidimensional Skill Mismatch,” American Economic Journal: Macroeconomics, 12, 210–244. Heath, R. (2018): “Why Do Firms Hire Using Referrals? Evidence from Bangladeshi Garment Factories,” Journal of Political Economy, 126, 1691–1746. Hoffman, M., L. Kahn, and D. Li (2018): “Discretion in Hiring,” Quarterly Journal of Economics, 133, 1–36. Ingle, K. and C. Mlatsheni (2017): “The Extent of Churn in the South African Youth Labour Market: Evidence from NIDS 2008-2015,” Manuscript, University of Cape Town: Southern Africa Labour and Development Research Unit. International Labour Organization (2016): Enabling Environment for Sustainable Enterprises in South Africa, Geneva: International Labour Office. ——— (2017): Global Employment Trends for Youth 2017: Paths to a Better Working Future, Geneva: International Labour Office. Ioannides, Y. M. and L. D. Loury (2004): “Job Information Networks, Neighborhood Effects, and Inequality,” Journal of Economic Literature, 42, 1056–1093. Jarosch, G. and L. Pilossoph (2019): “Statistical Discrimination and Duration Dependence in the Job Finding Rate,” Review of Economic Studies, 86, 1631–1665. Jovanovic, B. (1979): “Job Matching and the Theory of Turnover,” Journal of Political Economy, 87, 972–990. Kahn, L. (2013): “Asymmetric Information between Employers,” American Economic Journal: Applied Economics, 5, 165–205. Kahn, L. and F. Lange (2014): “Employer Learning, Productivity, and the Earnings Distribution: Evidence from Performance Measures,” Review of Economic Studies, 84, 1575–1613. Kerr, A. (2017): “Tax(i)ing The Poor? Commuting Costs in South African Cities,” South African Journal of Economics, 85, 321–340. Kessler, J., C. Low, and C. Sullivan (2019): “Incentivized Resume Rating: Eliciting Employer Preferences without Deception,” American Economic Review, forthcoming. Lam, D., C. Ardington, and M. Leibbrandt (2011): “Schooling as a Lottery: Racial Differences in School Advancement in Urban South Africa,” Journal of Development Economics, 95, 121–136. 35 Lange, F. (2007): “The Speed of Employer Learning,” Journal of Labor Economics, 25, 1–35. Levinsohn, J., N. Rankin, G. Roberts, and V. Schoer (2013): “Wage Subsidies and Youth Employment in South Africa: Evidence from a Randomized Control Trial,” Manuscript, University of Stellenbosch. Lise, J. and F. Postel-Vinay (2020): “Multidimensional Skills, Sorting, and Human Capital Accumulation,” American Economic Review, forthcoming. Lise, J., S. Seitz, and J. Smith (2004): “Equilibrium Policy Experiments and the Evaluation of Social Programs,” Working paper 10283, National Bureau of Economic Research. Lockwood, B. (1991): “Information Externalities in the Labour Market and the Duration of Unemployment,” Review of Economic Studies, 58, 733–753. Mackay, A. (2014): Building Brands in a Rapidly Changing Market: Lessons for South Africa, Yellowwood. MacLeod, W. B., E. Riehl, J. Saavedra, and M. Urquiola (2017): “The Big Sort: College Reputation and Labor Market Outcomes,” American Economic Journal: Applied Economics, 9, 223–261. Malindi, K. (2017): “Imperfect Information and The Racial Wage Gap for South African Men,” Manuscript, University of Stellenbosch. McCall, B., J. Smith, and C. Wunsch (2016): “Government-sponsored Vocational Education for Adults,” in Handbook of the Economics of Education, ed. by E. Hanushek, S. Machin, and L. Woessmann, Elsevier, vol. 5, 479–652. Menzio, G. and S. Shi (2011): “Efficient Search on the Job and the Business Cycle,” Journal of Political Economy, 119, 468–510. Mortensen, D. and C. Pissarides (1999): “New Developments in Models of Search in the Labor Market,” in Handbook of Labor Economics, ed. by O. Ashenfelter and D. Card, Elsevier, 2567–2627. Neumark, D. (2018): “Experimental Research on Labor Market Discrimination,” Journal of Economic Literature, 56, 799–866. Pallais, A. (2014): “Inefficient Hiring in Entry-Level Labor Markets,” American Economic Review, 104, 3565–3599. Poropat, A. E. (2009): “A Meta-Analysis of the Five-Factor Model of Personality and Academic Performance,” Psychological Bulletin, 135, 322–338. Poschke, M. (2019): “Wage Employment, Unemployment and Self-employment across Countries,” Manuscript, McGill University. Pritchett, L. (2013): The Rebirth of Education: Schooling Ain’t Learning, Washington, DC: Center for Global Development. Rankin, N., C. Darroll, and T. Corrigan (2012): “SMEs and Employment in South Africa,” Small Business Project, Johannesburg. 36 Rosar, F. and E. Schulte (2012): “Optimal Test Design under Imperfect Private Information and Voluntary Participation,” Manuscript, University of Bonn. Rosenberg, M. (1965): Society and the Adolescent Self-Image, Princeton University Press. Schöer, V., M. Ntuli, N. Rankin, C. Sebastiao, and K. Hunt (2010): “A Blurred Signal? The Usefulness of National Senior Certificate (NSC) Mathematics Marks as Predictors of Academic Performance at University Level,” Perspectives in Education, 28, 9–18. Schöer, V., N. Rankin, and G. Roberts (2014): “Accessing the First Job in a Slack Labour Market: Job Matching in South Africa,” Journal of International Development, 26, 1–22. Statistics South Africa (2016): Quarterly Labor Force Survey Quarter 4 2016, Pretoria: Statistics South Africa. Taylor, S., S. Van Der Berg, V. Reddy, and D. Janse van Rensburg (2011): “How Well Do South African Schools Convert Grade 8 Achievement Into Matric Outcomes?” Working Paper, Stellenbosch University. Wheeler, L., E. Johnson, R. Garlick, P. Shaw, and M. Gargano (2019): “LinkedIn(to) Job Opportunities? Experimental Evidence from Job Readiness Training,” Working paper, Duke University. 37 Online Appendices A Assessments We assess each workseeker’s skills in six domains. Most of the assessments are already used by Harambee and by some large firms in South African during hiring. We do not claim that these are the best possible assessments for predicting workplace performance. But these are assessments that some market agents have chosen to use, have reasonable psychometric properties, and are correlated with workplan performance in some settings. A.1 Firms’ Use of Assessments Harambee has used the numeracy, communication, and concept formation assessments since 2011 to select candidates for further job readiness training and recommend candidates to vacancies at partner firms. Harambee has placed over 160,000 candidates in entry-level jobs using these assessments. Table A.1 shows how 33 large client firms in retail, hospitality, logistics and corporate services require Harambee to use assessments when recommending candidates for interviews. All firms used at least one assessment to screen candidates and 73% of firms used all three assessments. In contrast, only 57% required certified results on the national high school graduation exam and only 3% required references. This shows firms find this skill information useful relative to other sources of information about prospective workers’ skills. Harambee also administers a set of career aptitude measures provided by a psychometric testing firm. 67% of firms in this sample used this assessment score to screen applicants, suggesting they value horizontal differentiation. We could not include this assessment in the certification because it is a proprietary instrument. We therefore selected three alternative measures of skills which would be unlikely to be correlated with numeracy, communication, and concept formation. To select these, we conducted interviews with 20 hiring managers to understand which other skills they valued in successful hires. Elsewhere, we conducted a detailed literature review of measures and selected those most overlapping with what firms valued (Esopo et al., 2018), which were also correlated with either earnings or measures of workplace performance in some settings. A.2 Description of Assessments Concept formation is very similar to the Raven’s Progressive Coloured Matrices assessment (Raven and Raven, 2003). It is a non-verbal measure of fluid intelligence, which captures the rate at which people learn and their conceptual reasoning. It specifically assesses the ability to ignore superficial differences and see underlying commonalities across situations and to use logic in new situations. Meta-analyses identify measures of fluid intelligence as strong predictors of worker productivity (Schmidt and Hunter, 1998; Schmidt et al., 2016). The Raven’s test is widely used in hiring and selection (Chamorro-Premuzic and Furnham, 2010), including in recent research in 38 Table A.1: Firms’ Use of Psychometric Assessments in Hiring % of firms using each piece of information to screen candidates Assessment result for Career Criminal High school # Communi- Concept aptitude record graduation Reference Sector Numeracy firms cation formation profile check certificate Hospitality 11 0.82 1.00 0.91 0.64 0.91 0.64 0.00 Retail 16 0.69 0.56 0.88 0.81 0.94 0.75 0.06 Corporate 6 1.00 1.00 0.83 0.33 1.00 0.00 0.00 Total 33 0.79 0.91 0.79 0.67 0.94 0.58 0.03 Table shows use of assessment results and other information by 33 firms that have long-term recruiting relationships with Harambee. Firms are coded as using an assessment if they require candidates to reach a certain threshold score on the assessment to be eligible for interviews or training programs. Firms are coded as using other documents if they require these to be submitted with the candidates’ application packages. The criminal record check is a set of checks against government records that the candidate had no criminal record or bad credit history. We observe only what information these 33 firms request from Harambee for candidates whom Harambee shortlists for interview, not how firms use the information. economics (Abebe et al., 2020b; Beaman et al., 2018). Scores on this assessment are correlated with interview ratings, technical scores and supervisor ratings in several South African firms (De Kock and Schlechter, 2009; Lopes et al., 2001; Taylor, 2013). Numeracy focuses on practical arithmetic and pattern recognition. We calculate a single numeracy score using the inverse variance-weighted average of two numeracy assessment scores. The more advanced assessment is developed by a large retail chain and used in their applicant screening process, as they believe it identifies some of the skills needed by cashiers. The simpler assessment was developed by a South African adult education provider (www.mediaworks.co.za) and assesses proficiency in arithmetic used in high school: comparing different types of numbers; working with fractions, ratios, money, percentages and units; and performing calculations with time and area. Communication captures English language listening, reading and comprehension skills. The assessment was developed by a South African adult education provider (www.mediaworks.co.za) and is designed to assess English proficiency for high school students. It evaluates both listening and written comprehension. It focuses on ability to identify and recall the main message of a text or passage, infer meaning of vocabulary through context clues, and infer meaning when information is not directly stated. Both numeracy and communication skills are correlated with educational attainment and wages in OECD countries (Heckman et al., 2006; Heckman and Kautz, 2012; Hanushek et al., 2015). There are also correlations between wages and numeracy (du Rand et al., 2011) and wages and English communication skills (Casale and Posel, 2011) in South Africa, conditional on education. Grit is a self-reported measure of a candidate’s inclination to work on difficult tasks until they are finished and whether they show perseverance to achieve long-term goals. This assessment is a validated self-reported 8-item psychological scale (Duckworth et al., 2007). Grit correlates with academic performance and workplace retention in the US (Eskreis-Winkler et al., 2014). The assessment labeled Focus on certificates captures inhibitory control, the ability to distinguish 39 relevant from irrelevant information, control one’s attention to focus on what is needed for a task (Diamond, 2013) and guide thought and action in accordance with a goal (Posner and DiGirolamo, 1998). The assessment is a computerized version of the widely-used Stroop Test, using colors (Stroop, 1935). Similar measures are correlated with employment status (Kalechstein et al., 2003) and moderate the negative effects of workplace related stress, such as burnout and absenteeism, in service sector jobs (Schmidt et al., 2007). Planning measures how candidates behave when faced with complex, multi-step problems. The assessment is adapted from the Hit 15 lab task (Gneezy et al., 2010). The computer and the subject take turns adding either one, two or three points to the points basket. The goal is to be the first player to reach 15 points. It captures ability to search for relevant information and anticipate the consequences of actions. High planning scores predict retention rates among truckers in the US, conditional on cognitive skills (Burks et al., 2009). Similar measures of complex planning skills are correlated with wages in South Africa, controlling for fluid intelligence and education (Ederer et al., 2015). For the first 17 of the 84 assessment days, covering 26% of candidates, computer problems meant that we used two self-reported psychological scales, labeled Control and Flexibility on the reports instead of focus and planning. We used two subscales of the Personal Problem-Solving Inventory (Hepner and Petersen, 1982). The Personal Control scale (control) captures whether candidates take a systematic or impulsive and erratic approach when faced with new, challenging problems. The Approach Avoidance (flexibility) scale captures whether candidates actively consider several approaches to solving a problem or whether they pursue their first idea without thinking about alternatives. These are not exact analogues of the tasks: they capture self-perceptions as well as behaviors (Heppner, 1988). But scores are correlated in other samples: for example, the PSI is correlated with the Stroop task (Rath et al., 2004). None of the main results in the paper are substantially different between the sample using the focus and planning assessments and the sample using the control and flexibility assessments. We use the assessment scores in the paper in three ways. First, we use assessment scores as a prespecified conditioning variable when estimating treatment effects. We use the concept formation, communication, grit, and numeracy scores individually for this purpose. We combine the remaining scores into a single measure by taking the first principal component of control and flexibility and standardizing it, taking the first principal component of focus and planning and standardizing it, and then appending the two principal components together. Second, we use assessment scores in the heterogeneity analysis described in Section 5.2. We use only the scores observed for all candidates (concept formation, communication, grit, and numeracy) for this analysis. Results are similar when we restrict to the 74% of candidates who took the focus and planning assessments and use all six assessments. Third, we use assessments in the firm-facing experiments described in Sections 5.1 and 5.2. The online platform reports all eight assessment results and explains that each candidate took 40 Table A.2: Correlations of Assessment Results Panel A: Correlations In First 17 Days of Assessment (1,615 workseekers) Concept formation Grit Numeracy Control Flexibility (Personal (Approach Control) Avoidance) Communication 0.337 0.127 0.386 0.237 0.126 Concept formation 0.108 0.489 0.174 0.098 Grit 0.163 0.507 0.334 Numeracy 0.212 0.107 Control 0.173 Panel B: Correlations In Remaining 67 Days of Assessment (5,276 workseekers) Concept formation Grit Numeracy Focus Planning (Stroop) (Hit 15) Communication 0.346 0.088 0.393 0.171 0.258 Concept formation 0.094 0.519 0.225 0.292 Grit 0.129 0.049 0.106 Numeracy 0.162 0.325 Focus 0.181 Table shows pairwise correlation coefficients between assessment results. The sample is split because two of the assessments changed after the first 17 days of assessment, from the control and flexibility scales to the focus and planning tasks. None of the pairwise correlations between the four assessments used for the entire period (communication, concept formation, grit, and numeracy) are substantively or statistically significantly different between the two periods. Table A.3: Distribution of Top, Middle, and Bottom Terciles Shown on Candidates’ Reports Fraction with _ top terciles Total 0 1 2 3 4 5 6 0 0.001 0.007 0.025 0.032 0.029 0.018 0.007 0.119 Fraction with _ bottom terciles 1 0.009 0.036 0.059 0.064 0.037 0.011 - 0.215 2 0.027 0.077 0.079 0.040 0.011 - - 0.235 3 0.054 0.076 0.048 0.009 - - - 0.187 4 0.070 0.059 0.009 - - - - 0.138 5 0.060 0.024 - - - - - 0.084 6 0.023 - - - - - - 0.023 Total 0.243 0.279 0.220 0.146 0.076 0.029 0.007 Table shows the share of the sample with i top terciles and j top terciles on their reports for each i, j ∈ {0, 6}. The number of middle terciles equals 6 − i − j . 41 only six of the eight assessments. The profile-ranking exercise does not use the control or flexibility scales. A.3 Administration of Assessments All assessments are conducted in English, the same language used for all Harambee interaction with candidates. All assessments are conducted on desktop computers, so the assessment results may be sensitive to candidates’ computer skills. To minimize this sensitivity, all candidates do some practice computer exercises before the assessments and all assessments are designed to be completable within the available time limit. Before starting assessments, candidates consent to their assessment results being shared with Harambee, the research team, and external firms. Registered industrial psychologists employed or contracted by Harambee oversaw administration of all assessments. They also delivered briefings to candidates to interpret results. Finally, the lead psychologist at Harambee approved the language on certificates. This ensures compliance with South African law on psychometric testing in workplace settings. A.4 Validation of Self-Reported Psychological Scales and Tasks We use four self-reported psychological scales in the paper: grit, control and flexibility are used as skills measures, while self-esteem is used as an outcome measure. We followed standard procedures in psychology to ensure the self-reported scales were well-understood and valid as measures. See Esopo et al. (2018) for a full discussion of the process followed. We use the same seven-point Likert scale for all scales. The Problem-Solving Inventory had already been validated in South Africa with young black African students of a very similar demographic profile to our sample and we used this item wording (Pretorius, 1993; Heppner et al., 2002). For grit and self-esteem, we ensured language used was well-understood by conducting cognitive debriefings with 20 Harambee candidates. Cognitive debriefing captures the underlying cognitive processes that respondents use to answer questions to detect and solve problems in questionnaires (Tourangeau, 2003; Willis, 2008, 1999). For example, the interviewer asks for specific information relevant to the question or the answer given. Examples of probes used are “What does the term mean to you?”, “Can you repeat this question to me in your own words?” and “What made you answer the way that you did?” We simplified the wording of some items and altered some culturally specific idioms in response to the cognitive debriefings. Second, we estimated the extent to which different items in each scale move together, using Cronbach’s alpha (Cronbach, 1951). All assessments have α > 0.65. Third, we administered the scales twice for 150 candidates, ten days apart. We estimated Lin’s Concordance Correlation Coefficient (Lawrence and Lin, 1989) between the two administrations. All assessments have ρc > 0.62. Fourth, we check if any items on the scales have very low variation across candidates using maximum endorsement frequencies. No items meet the threshold for being dropped due to 42 insufficient variation from Bowling (2014). The terciles shown on the assessment results are based on assessment results from candidates assessed before the study started: 5,000 workseekers for communication, numeracy and concept formation test, and 500 workseekers for the other skills. Tercile assignments are largely unchanged if we retrospectively construct them using our full sample of assessed workseekekers. Table A.2 shows the correlation of assessment results for the different skills. Numeracy, concept formation and communication have pairwise correlations of 0.34 to 0.52. Numeracy and communication assessments capture acquired knowledge, often from schooling, which is often positively correlated with fluid intelligence. This is potentially because learning at a higher rate improves acquisition of knowledge (Heckman and Kautz, 2012; Nisbett, 2009; Roberts et al., 2000). However, as we intended, these are less strong correlations between the other tasks (focus and planning) and the scales (grit, flexibility, and planning). These suggest the certificates will horizontally differentiate workseekers from one another. B Implementation Costs This appendix reports the costs of the public certification intervention and compares these to gains experienced by treated workseekers, showing that the latter easily exceed the former. We measure costs from the Harambee and J-PAL Africa financial statements. All costs are reported in 2016/7 PPP USD terms and are averaged over the 2,247 candidates who received the public certification intervention. The cost figures in nominal USD are 42% of the cost figures in PPP USD, though this does not affect the cost-benefit comparisons. We report average variable costs and, where these are possible, total and average fixed costs. The average variable costs may change with scale but we do not attempt to project scale effects on costs. The average variable cost of adding certification to Harambee’s existing assessment operation was USD 23.10. This included report printing, software license fees, website hosting fees, the time of J-PAL and Harambee staff used to prepare the reports, and the time of Harambee psychologists used to conduct briefings. This also included a USD 10.32 transport subsidy to each participant to cover the cost of travel to the Harambee office, which is arguably not a necessary cost of the intervention. These cost calculations exclude the private and placebo certifications, audit study, and firm-facing experiments. The average variable cost of certification and assessment was USD 57.27 per participation. This included all certification-only costs, facility rental, computer rental, data and internet costs, and the time of Harambee staff who administered the assessments. Facility and computer rental costs were the largest line items for the assessment cost, jointly accounting for USD 23.43. The average variable costs exclude fixed costs such as licenses for the assessment tools, market research into firm preferences over assessments, and senior management fees. For these costs we either cannot calculate a meaningful average fixed cost or cannot reliably separate Harambee’s 43 total fixed costs for developing the assessment program from its costs of other activities. J-PAL Africa’s fixed cost for developing the certification program on top of the assessment program was approximately USD 17,685 or USD 7.87 per candidate who received the public certification intervention. This covered J-PAL Africa staff costs during development and all costs of piloting the certificates with firms and workseekers. This includes the cost of developing and piloting the private and placebo certifications, which we cannot easily separate from the public certification, but excludes the costs of developing and piloting the audit study and firm-facing experiments. We compare these average costs to the average benefit per participant who received the public certification intervention over the first three months after the intervention. Public certification increases average earnings by USD 9.05 in the week before the endline survey and the endline survey occurred on average 14.4 weeks after treatment. Multiplying these together gives an average effect on earnings since treatment of USD 130.2: 5.6 times higher than the average variable cost of certification, 2.3 times higher than the average variable cost of assessment and certification, and 2.0 times higher than the average variable cost of assessment and average variable and fixed costs of certification. The gains to treated workseekers over just three and a half months easily exceed the cost of public certification and assessment. The preceding calculation assumes that the treatment effect on weekly earnings does not vary through time from treatment to the endline. The public certification effect on earnings does not substantially vary with the time period from treatment to endline. But the treatment effects on recalled employment in the first and second months after treatment are not identical, suggesting a possible time trend (Table D.10). To account for this, we convert the weekly earnings effect into monthly terms and multiply this by the sum of the employment effect in the first month after treatment, the second month after treatment, and the week before the endline. This gives an average on earnings since treatment of USD 110.1, which also easily exceeds the cost of public certification and assessment. C Labor Market Effects at the Extensive and Intensive Margins Treatment effects on labor market outcomes such as earnings and hours can occur at the extensive margin – due to treatment effects on employment – and at the intensive margin – due to treatment effects on job characteristics conditional on employment. This distinction is important, as intensive margin effects indicate that treatment is changing the type of jobs candidates secure. The intensive margin effects are not identified from regressions of labor market outcomes on treatment indicators for employed candidates, as the set of employed candidates may be selected based on treatment assignment. We adapt a method from Attanasio et al. (2011) to decompose of labor market effects into extensive and intensive margins. We describe the decomposition here for earnings, but the same idea applies to any labor market outcome that is observed only for the employed. We use the term 44 “treatment” to refer to the public certification. Using the law of iterated expectations and the fact that observed earnings are zero for non-employed candidates, we can write the average treatment effect on earnings as: E[Earn|T reat = 1] − E[Earn|T reat = 0] (3) ATE for earnings = (E[Earn|T reat = 1, W ork = 1] − E[Earn|T reat = 0, W ork = 1]) · P r[W ork = 1|T reat = 1] ATE for earnings | employment Treated employment rate + E[Earn|T reat = 0, W ork = 1] · (P r[W ork = 1|T reat = 1] − P r[W ork = 1|T reat = 0]) . Control earnings | employment ATE for employment We define the second line on the right-hand of the regression as the extensive margin effect. Intuitively, this is the average treatment effect on employment ‘priced’ at the mean earnings value in the control group. If treatment has no effect on the employment rate, then this expression is zero. We define the first line on the right-hand side of the regression as the intensive margin effect. If treatment only changes the employment rate but has no effect on earnings for employed candidates, then this term is zero.30 All terms in equation (3) except the average treatment effect on earnings conditional on employment are identified by the experiment and can be consistently estimated using sample analogues. Hence, we can consistently estimate the remaining term using the formula in (3). We obtain standard errors by estimating all quantities as a system and using the Delta method. This decomposition applies to realized earnings, which are zero by definition for non-employed candidates. This decomposition does not apply to latent earnings, which may be non-zero for non-employed candidates. Alternative methods are available for studying latent earnings. One set of approaches point identifies the average treatment effect on latent earnings by modeling the selection process into employment and adjusting observed earnings for selection (e.g. Gronau, 1974 and Heckman, 1974). Another set of approaches bounds the average treatment effect on latent earnings by assuming that the earnings for the non-employed fall in some region of the observed earnings distribution (e.g. Lee, 2009 and Manski, 1989). Neither approach is ideal in our setting: the former methods require an instrument for selection into employment that we do not have and the latter methods will yield wide bounds given the large effect of public certification on employment. Another set of approaches point identifies quantile treatment effects on latent earnings by assuming that the earnings for the non-employed fall in some region of the observed earnings distribution 30 Attanasio et al. (2011) show that the intensive margin effect can be further decomposed into two terms: the treatment effect on earnings conditional on candidates’ baseline characteristics, and the difference in baseline characteristics between employed candidates in the treatment and control groups. However, neither of these terms is point identified. Separating these effects is not important in our application. Our conceptual framework is consistent with certification either increasing the same workseekers’ latent treated wages conditional on employment, or increasing mean wages conditional on employment by helping workseekers with higher latent treated wages get employed. 45 (e.g. Powell, 1984). Our analysis of quantile treatment effects has a similar flavor to this approach, though we do not directly interpret these as effects on latent earnings. As discussed in Section 3.3, this decomposition shows that the earnings and wage effects of public certification occur at both the extensive and intensive margins. The hours and contract type effects occur only at the extensive margin. The intensive-margin effect on earnings is also visible in the distributions and densities of earnings for the public certification and control groups. Figure 2 (in the main text) shows the distributions of earnings for each group and the quantile treatment effects of public certification. Figure C.1 shows the densities of earnings for employed candidates in the control and treatment groups. We rescale the latter density by the ratio of treatment group to control group employment. Hence, the vertical difference between the densities at each earnings level E represents the treatment effect on the share of all candidates earning E , not on the share of employed candidates earning E . The treatment effect on the earnings density is almost entirely above median earnings for employed control group candidates. This shows that either the marginal candidates employed only when treated earn more than most inframarginal control candidates, or treatment increases earnings for inframarginal candidates, or both. 46 Figure C.1: Density of Earnings in Control and Public Certification Groups .3 Density (scaled by relative employment rate) Treated .2 Control .1 0 0 5 10 Earnings (inverse hyperbolic sine) This figure shows the densities of earnings in the control and public certification groups. To account for the positive treatment effect on employment, the treatment density is scaled by the ratio of employment in the treatment group to employment in the control group. Hence the vertical difference between the densities at each earnings level E represents the treatment effect on the share of all candidates earning E , not on the share of employed candidates earning E . The density is estimated only for the employed, so candidates with zero earnings are excluded. 47 D Additional Results about Workseeker Experiments D.1 Summary Statistics and Balance Tests This section reports summary statistics for the baseline workseeker sample (Table D.1) and endline workseeker sample (Table D.2). Balance tests for equal means of baseline measures are also reported in the final column of Table D.1. Table D.3 compares our workseeker sample to the broader population of the country and of Gauteng province, where the study took place. 48 Table D.1: Summary Statistics for Baseline Variables Variable # obs Mean Std 10th 90th p:balance dev. pctile pctile Panel A: Demographic Measures Age 6891 23.6 3.3 19.8 28.3 0.583 Male 6891 0.382 0.486 0.267 University degree / diploma 6891 0.167 0.373 0.889 Any other post-secondary qualification 6891 0.212 0.409 0.642 Completed secondary education only 6891 0.610 0.488 0.794 Panel B: Assessment Results Numeracy score 6891 0.052 0.988 -1.187 1.411 0.523 Communication score 6891 0.050 0.992 -1.093 1.694 0.206 Concept formation score 6891 0.047 0.991 -1.516 1.260 0.764 Grit score 6891 0.031 0.992 -1.313 1.279 0.089 Other scores 6891 -0.002 1.070 -1.305 1.318 0.859 Panel C: Labor Market Measures Employed 6891 0.378 0.485 0.468 Earnings 2116 565 740 100 1400 0.083 Ever worked 6877 0.704 0.457 0.418 Ever held a long-term job 6877 0.090 0.286 0.696 Panel D: Job Search Measures Searched 6891 0.968 0.175 0.058 Applications submitted a 6815 9.9 18.6 2.0 20.0 0.809 Search cost 6147 242 1520 30 400 0.276 Search hours 6699 17.0 20.8 2.0 48.0 0.231 Offers receiveda 6810 1.20 7.20 0.00 2.00 0.280 Panel E: Belief Measures Planned applicationsa 6840 48.9 1629.9 4.0 36.0 0.252 Correct about all assessment results 6891 0.082 0.274 0.960 Incorrect about all assessment results 6891 0.290 0.454 0.961 Overconfident about all assessment results 6891 0.219 0.413 0.732 Underconfident about all assessment results 6891 0.010 0.100 0.783 Table shows summary statistics for selected baseline variables. Percentiles are omitted for binary variables. All monetary figures are reported in South Africa Rands. 1 Rand ≈ USD 0.167 in purchasing power parity terms. In this table, intensive-margin labor market measures (e.g. earnings) are set to missing for non-workers, and intensive-margin search measures (e.g. search cost) are set to missing for non-searchers. All assessment results are standardized to have mean zero and standard deviation one in the control group. Missing values reflect item non-response, mostly due to respondents reporting that they don’t know the answer. All period-specific outcomes use a 7-day recall/forecast period unless marked with a (30-day recall/forecast period). The final column reports the p-value for testing equality of means of the baseline variables across all treatment groups, using heteroskedasticity-robust standard errors clustered by treatment date. 49 Table D.2: Summary Statistics for Endline Variables Variable # obs Mean Std dev. 10th pctile 90th pctile Panel A: Labor Market Measures Employed 6607 0.323 0.468 Earnings 2112 623 1183 2 1500 Hours worked 2121 28.5 21.6 4.0 56.0 Hourly wage 2097 33.1 72.3 0.1 77.8 Wage employment 2102 0.885 0.319 Self employment 2102 0.114 0.318 Panel B: Job Search Measures Any search 6608 0.692 0.462 Applications submitteda 6577 12.8 21.5 1.0 27.0 Hours searched 6601 9.9 14.2 0.0 25.0 Search cost 6599 116 167 0 300 Responsesa 6593 0.861 2.147 0.000 2.000 Offersa 6592 0.207 0.680 0.000 1.000 Panel C: Belief Measures Fraction of assessments overconfident 6607 0.345 0.237 Fraction of assessments underconfident 6607 0.176 0.166 Targeted search 6891 0.175 0.380 Planned applicationsa 6591 16.1 29.7 3.0 30.0 Expected offersa 6531 4.49 5.70 1.00 10.00 Table shows summary statistics for selected endline variables. Percentiles are omitted for binary variables. All monetary figures are reported in South Africa Rands. 1 Rand ≈ USD 0.167 in purchasing power parity terms. Intensive-margin labor market measures (e.g. earnings) are set to missing for non-workers. Intensive-margin search measures (e.g. search cost) are set to zero for non-searchers. Missing values reflect item non-response, mostly due to respondents reporting that they don’t know the answer. All period-specific outcomes use a 7-day recall/forecast period unless marked with a (30-day recall/forecast period). Table D.3: Comparison between the Workseeker Sample and External Populations (1) (2) (3) (4) Gauteng Gauteng Study sample South Africa province age 18-29 of workseekers Age 28.914 31.461 23.776 23.646 (19.606) (19.099) (3.326) (3.299) Male 0.489 0.504 0.512 0.382 Currently Employed 0.290 0.375 0.346 0.378 Currently Searching 0.102 0.151 0.302 0.984 < complete secondary school 0.737 0.612 0.429 0.011 Complete secondary school 0.184 0.255 0.449 0.610 > complete secondary school 0.072 0.120 0.116 0.379 Table compares the sample of workseekers in this study (column 4) to several external benchmarks: the country (column 1), the province of Gauteng where the study takes (column 2), and people in Gauteng in the eligible age range for the study (column 3). National and provincial statistics are calculated from the Quarterly Labour Force Surveys (QLFS), averaging over all 2016 waves and using post-stratification weights supplied by Statistics South Africa. QLFS data are not available by city but the greater Johannesburg metropolitan area where the study is conducted accounts for over half the population of the Gauteng province. Standard deviations are shown in parentheses for all continuous variables. 50 D.2 Benchmarking the Magnitude of the Earnings Effects In this section we show that the earnings effects are substantial relative to two local benchmarks. Minimum wage: During our study period, minimum wages in South Africa varied by sector and location. Sector- and location-specific minimum wages were either set by the Ministry of Labour or in bargaining councils, where large firms and unions agreed minimum wages that applied to all firms (Budlender et al., 2015; Isaacs, 2016). Table D.4 shows minimum wages for urban areas at the time of the study for several sectors relevant to workseekers in our sample. Poverty Lines: South African poverty research often uses poverty lines based on the cost of purchasing 2100 calories plus the average amount spent on non-food items by households whose food expenditure equals the food poverty line (Budlender et al., 2015; Leibbrandt et al., 2012). Using this definition, the adult monthly poverty line just before the study period was 1,386 South African rand or USD 232 in purchasing power parity terms (Isaacs, 2016, p.22). The average treatment effect on earnings is equal to 17% of the adult monthly poverty line or 7-9% of the monthly minimum wage at the time of the study. Table D.4: Benchmarking Earnings Figures to Minimum Wage and Poverty Lines Panel A: South African poverty lines and minimum wages at baseline Monthly Weekly aaa Date ZAR USD ZAR USD Poverty line Adult Early 2016 1386 232 320 54 Household (4 people) Early 2016 5544 927 1279 214 Minimum wage Domestic work 2015-2016 2550 427 588 98 Hospitality 2015-2016 2750 460 634 106 Wholesale and retail 2015-2016 3250 544 750 125 Private security/contract cleaning 2015-2016 3500 585 808 135 Panel B: Benchmarking sample earnings and certification treatment effects on earnings Weekly As % of poverty line As % of min. wage Date ZAR USD Adult Household Hospitality Retail Baseline mean earnings if employed Late 2016 562 94.1 1.76 0.44 0.89 0.75 Endline mean earnings Early 2017 159 26.6 0.50 0.12 0.25 0.21 Endline mean earnings if employed Early 2017 518 86.7 1.62 0.41 0.82 0.69 Treatment effect Early 2017 54.1 9.05 0.17 0.04 0.09 0.07 Calculations assume 1 rand ≈ 0.167 USD in purchasing power parity terms; 4.33 weeks per month. Household poverty lines assume households of four people with only one earner. Control group respondents work 29 hours per week conditional on being employed; earnings for those in full time work will be higher than mean earnings here. Poverty lines are from Isaacs (2016, p.22) and minimum wages are from the Department of Labor for 2015. Minimum wages are for large urban areas (Area A). They are for hospitality businesses with less than 10 employees and shop assistants in the wholesale and retail sector. 51 D.3 Non-response The phone survey after 3-4 months is our main source of endline data. We use a text message survey after 2-3 days only to measure beliefs about numeracy and self-esteem. The response rates for the text message and phone surveys are respectively 83 and 96%. Non-response does not differ by treatment arm (Table D.5). Non-response does not differ over most baseline characteristics (Table D.6). Men are less likely to respond in both surveys. Higher numeracy and concept formation scores predict higher response rates in the text message survey. Higher grit predicts lower response rates in the endline survey. Table D.5: Non-response by Treatment Group in Each Post-Treatment Survey Round (1) (2) Text Message Survey Endline Phone Survey Control 0.170 0.040 (0.013) (0.006) Public 0.177 0.039 (0.011) (0.004) Private 0.182 0.044 (0.010) (0.004) Placebo 0.142 0.047 (0.032) (0.026) p: Control = Pvt. 0.481 0.632 p: Control = Pub. 0.670 0.855 p: Pvt. = Pub. 0.785 0.388 p: Control = Pvt. = Pub. 0.778 0.681 p: Control = Plc. 0.414 0.787 p: Pvt. = Plc. 0.238 0.888 p: Pub. = Plc. 0.297 0.746 p: Control = Pvt. = Pub. = Plc. 0.641 0.841 # observations 6891 6891 # clusters 84 84 Coefficients show the fraction of each treatment group that does not complete each follow-up survey round. Heteroskedasticity-robust standard errors clustered by treatment date are shown in parentheses. 52 Table D.6: Non-response by Baseline Covariates Group in Each Post-Treatment Survey Round (1) (2) Text Message Survey Endline Phone Survey Completed at most high school -0.008 -0.003 (0.012) (0.005) Numeracy score -0.029 0.003 (0.006) (0.003) Communication score 0.008 0.003 (0.006) (0.003) Concept formation score -0.019 0.002 (0.006) (0.003) Grit score -0.001 -0.007 (0.005) (0.003) Other scores 0.001 -0.002 (0.004) (0.003) Perceived numeracy score -0.000 -0.000 (0.000) (0.000) Perceived literacy score 0.014 -0.003 (0.010) (0.005) Perceived concept formation score 0.010 -0.003 (0.009) (0.004) Self-esteem index 0.006 0.002 (0.004) (0.002) Age -0.002 0.001 (0.001) (0.001) Male 0.049 0.014 (0.010) (0.005) Employed -0.005 -0.001 (0.008) (0.005) Above median discount factor 0.012 0.007 (0.009) (0.005) Respondent is present-biased 0.016 0.007 (0.011) (0.006) Above median risk aversion -0.007 0.001 (0.008) (0.005) p: All coefficients jointly zero 0.000 0.041 Mean outcome 0.170 0.040 # observations 6891 6891 # clusters 84 84 Coefficients are from regressions of round-specific attrition on the list of baseline covariates displayed here. All assessment scores are standardized to have mean zero and standard deviation one in the control group. Heteroskedasticity-robust standard errors clustered by treatment date are shown in parentheses. 53 D.4 Additional Treatment Effects Table D.7 shows the public certification effects of our main labor market outcomes without conditioning on the prespecified covariates. The results are very similar with or without the covariates. Table D.8 shows public and private certification effects at two points in time: in the text message survey conducted 2-3 days after treatment and the endline phone survey conducted 3-4 months after treatment. This table shows four patterns, which expand on the discussion in footnote 19 of the paper. First, both treatments make candidates more likely to report that their assessment result matches their actual assessment result immediately after treatment. Second, both treatment effects decline over the following 3-4 months, although the different survey methods mean the time comparison should be interpreted cautiously. Third, the public treatment effect on self-beliefs is significantly larger than the private effect after 3-4 months but not after 2-3 days. This suggests that the larger public treatment effect at 3-4 months does not occur because the information it conveys is immediately more credible or easier to understand than the private treatment. Instead, it may be larger because the information is more memorable or the public treatment generates other effects, such as more job interviews or employment that provide more opportunities to learn about skills. Fourth, neither treatment affects self-esteem at either point in time. Table D.9 shows how treatment effects on employment vary by single index summary measures of candidates’ skills (Panel A) and baseline candidate characteristics that might provide alternative measures of candidates’ skills (Panel B). We discuss these treatment effects in Sections 5.2 and 5.3 of the paper. Table D.10 reports public and private certification effects on all prespecified workseeker-level job search and labor market outcomes. These are organized into families of conceptually similar outcomes, which we use for multiple testing adjustments. First, we report q -values that control the false discovery rate across outcomes within each family (Benjamini et al., 2006). None of the q -values in this table is substantively different to the corresponding p-values reported in the main paper. Second, we estimate treatment effects on inverse covariance-weighted averages of the outcomes within each family (Anderson, 2008). This provides a single summary test of the information contained across all outcomes in the same family. None of the treatment effects on these averages provides substantively different information to the treatment effects on individual outcomes. We omit some prespecified outcomes related to beliefs from this paper and analyze them in separate work. The search targeting measure discussed in Section 4 is not prespecified. We did not prespecify an analysis plan for the smaller extension experiments discussed in Section 5. 54 Table D.7: Treatment Effects on Labor Market Outcomes Without Covariates (1) (2) (3) (4) (5) Employed Hours c c c Earnings Hourly wage Written contract Treatment 0.046 0.175 0.336 0.206 0.018 (0.013) (0.058) (0.076) (0.041) (0.010) Mean outcome 0.309 8.848 159.291 9.840 0.120 Mean outcome for employed 28.847 518.291 32.283 0.392 # observations 6607 6598 6589 6574 6575 # clusters 84 84 84 84 84 Coefficients are from regressing each outcome on a vector of treatment assignments and randomization block fixed effects without any other covariates. Heteroskedasticity-robust standard errors shown in parentheses, clustering by treatment date. Mean outcome is for the control group. All outcomes use a 7-day recall period. Outcomes marked with c use the inverse hyperbolic sine transformation. The sample sizes differ across columns due to item non-response, mostly from respondents reporting that they don’t know the answer. Table D.8: Treatment Effects on Self-Beliefs through Time (1) (2) (3) (4) Perceived numeracy tercile correct Above-median self-esteem Public 0.233 0.315 0.001 -0.001 (0.013) (0.015) (0.013) (0.015) Private 0.200 0.333 -0.002 0.016 (0.015) (0.016) (0.014) (0.015) p: public = private 0.010 0.240 0.806 0.239 Mean outcome 0.396 0.399 0.553 0.479 # observations 6601 5297 6609 5027 # clusters 84 84 84 84 Coefficients are from regressing each outcome on a vector of treatment assignments, randomization block fixed effects, and prespecified baseline covariates (measured skills, self-reported skills, education, age, gender, employment, discount rate, risk aversion). Heteroskedasticity-robust standard errors shown in parentheses, clustering by treatment date. Mean outcome is for the control group. Above-median self-esteem is an indicator equal to one if the candidate’s response on a shortened version of the Rosenberg (1965) self-esteem scale is above the sample median. Columns (1) and (3) report results from the main phone follow-up survey. Columns (2) and (4) report results from the text message survey conducted 2-3 days after treatment. The sample sizes differ across columns due to item non-response, mostly from respondents reporting that they don’t know the answer. 55 Table D.9: Heterogeneous Treatment Effects on Employment (1) (2) (3) Panel A: Heterogeneous Effects by Single Index Skill Measures Public treatment 0.052 0.052 0.054 (0.011) (0.011) (0.012) × Share top - share bottom terciles 0.019 (0.028) × PC1 (Scores) 0.004 (0.025) × Earnings-weighted average of scores -0.007 (0.029) # observations 6607 6607 6603 # clusters 84 84 84 Panel B: Heterogeneous Effects by Alternative Information Sources Public treatment 0.052 0.052 0.051 (0.011) (0.012) (0.012) × post-secondary education -0.027 (0.028) × employed at baseline -0.043 (0.032) ˆ × Pr(Employed at endline | X) -0.069 (0.028) # observations 6607 6607 6607 # clusters 84 84 84 Coefficients are from regressing each outcome on a vector of treatment assignments, displayed interaction terms, randomization block fixed effects, and prespecified baseline covariates (measured skills, self-reported skills, education, age, gender, employment, discount rate, risk aversion). Heteroskedasticity-robust standard errors shown in parentheses, clustering by treatment date. The measures used for interactions in Panel A and column 3 of Panel B are indicators for above-median values of the underlying indices. All measures in panels A and B are demeaned before being interacted with treatment, so the coefficient on the treatment indicator equals the average ˆ treatment effect. Pr(employed at endline | X) is estimated by regressing endline control group employment status on the baseline covariates listed above and predicting employment for all candidates. Prediction for control group candidates uses leave-one-out-estimation to avoid overfitting. PC1 (Scores) is the first principal component of the skills. The earnings-weighted average of scores is the weighted average of the assessment results, with weights derived from a regression of control group earnings on assessment results. 56 Table D.10: Treatment Effects on Prespecified Outcomes with Multiple Testing Adjustments (1) (2) (3) (4) (5) Index Any search Applicationsa,c Search hoursc Search costc Public -0.012 -0.020 0.018 -0.035 -0.092 (0.032) (0.014) (0.042) (0.048) (0.081) Private 0.006 -0.006 0.036 -0.035 -0.031 (0.032) (0.014) (0.038) (0.049) (0.088) q: Public effect = 0 1.000 1.000 1.000 1.000 q: Private effect = 0 1.000 1.000 1.000 1.000 q: Public = private effect 1.000 1.000 1.000 1.000 Mean outcome 0.001 0.695 12.356 9.791 112.684 # observations 6608 6608 6577 6601 6599 (1) (2) (3) (4) (5) Responses per Offers per Index Responsesa,c Offersa,c applicationa applicationa Public 0.016 0.023 0.006 0.000 -0.000 (0.029) (0.024) (0.013) (0.004) (0.003) Private 0.019 0.016 0.013 -0.005 0.001 (0.026) (0.022) (0.013) (0.004) (0.004) q: Public effect = 0 1.000 1.000 1.000 1.000 q: Private effect = 0 1.000 1.000 1.000 1.000 q: Public = private effect 1.000 1.000 1.000 1.000 Mean outcome -0.023 0.871 0.195 0.099 0.030 # observations 6593 6593 6592 5944 5943 (1) (2) (3) (4) (5) b Applications Interviews Offers Index Used report with reportb,c with reportb,c with reportb,c Public NA 0.699 1.682 0.432 0.112 (0.013) (0.040) (0.023) (0.011) Private NA 0.289 0.572 0.144 0.036 (0.012) (0.033) (0.017) (0.008) q: Public effect = 0 0.001 0.001 0.001 0.001 q: Private effect = 0 0.001 0.001 0.001 0.001 q: Public = private effect 0.001 0.001 0.001 0.001 Mean outcome 0.000 0.000 0.000 0.000 # observations 6609 6598 6597 6597 (1) (2) (3) (4) (5) Employed Employed Employed Index Hoursc in last week in month 1 in month 2 Public 0.137 0.052 0.036 0.058 0.201 (0.025) (0.012) (0.011) (0.014) (0.052) Private 0.049 0.011 0.028 0.009 0.066 (0.028) (0.012) (0.013) (0.015) (0.048) q: Public effect = 0 0.001 0.001 0.001 0.001 q: Private effect = 0 0.504 0.142 0.504 0.336 q: Public = private effect 0.003 0.132 0.002 0.008 Mean outcome 0.001 0.309 0.465 0.437 8.848 # observations 6609 6607 6604 6607 6598 (1) (2) (3) (4) Hourly Written Index Earningsc wagec contract Public 0.106 0.338 0.197 0.020 (0.028) (0.074) (0.040) (0.010) Private 0.069 0.162 0.095 0.017 (0.030) (0.078) (0.046) (0.009) q: Public effect = 0 0.001 0.001 0.019 q: Private effect = 0 0.066 0.066 0.066 q: Public = private effect 0.047 0.047 0.345 Mean outcome 0.006 159.291 9.840 0.120 # observations 6609 6589 6574 6575 Coefficients are from regressing each outcome on a vector of treatment assignments and randomization block fixed effects. Heteroskedasticity-robust standard errors shown in parentheses, clustering by the 84 treatment dates. Sharpened q -values control the false discovery rate across outcomes in each panel, following Benjamini et al. (2006). The first column of each panel shows inverse covariance-weighted averages of outcomes in each panel, following Anderson (2008). The index is omitted for the report use variables because these are zero for all control group candidates, so the covariance cannot be estimated. Mean outcome is for the control group. All outcomes use a 7-day recall period unless marked with a (30-day recall period) or b (since treatment). Outcomes marked with c use the inverse hyperbolic sine transformation. The sample sizes differ across columns due to item non-response, mostly from respondents reporting that they don’t know the answer. 57 E Audit Study We conduct an audit study to identify the effect of information provision on firm decisions, without any scope for mediating behavior by workseekers. We submit real workseekers’ applications to entry-level job vacancies and randomly vary the information firms see about workseekers’ skills. This appendix reports more information about the process and sample to help interpret the results reported in Section 4.3. We implement the audit study in nine sequential rounds. In each round, we invite candidates by text message to submit application materials to us, within 7 days, for an undisclosed job opportunity.31 We do not explicitly indicate our affiliations or link the message to Harambee. We send one reminder text message to all candidates 1-3 days after the initial invitation. We invited 2,220 candidates to send CVs over the nine rounds. We randomly sample candidates from those who had already completed the workseeker survey. 717 candidates (28%) submit CVs within the one week period. Most CVs include some information about proxies for candidates’ skills: 91% include a reference letter or contact information for referees and 55% include their secondary school graduation results (Table E.1, panel A). The 717 responders are similar to the full workseeker sample on all baseline covariates except gender, where deliberately oversampled men for an even gender split. Candidates in the private treatment group are slightly more likely to respond to the invitation (Panel B). All treatment effects are robust to reweighting the responders to have the same distribution of treatment assignments and baseline covariates as the full workseeker sample. For each application received, we record information on when the application was received, where it was sent from, what documents are included, and an indicator for scan quality of included documents (e.g. photographs versus high-quality scans). We also send the candidate an acknowledgement of receipt. Simultaneously, we compile job vacancies from several online job posting sites. We selected only vacancies suitable for entry-level workers, so that all candidates in our sample are eligible to apply. We exclude jobs that look suspicious or are discriminatory, for example: jobs that ask for payments of any kind to apply, promise unrealistic salaries or benefits, or discriminate based on appearance, race, or gender. This generates a sample of 1,068 vacancies over the nine rounds, though we exclude 70 vacancies for reasons discussed below. Among the vacancies, 48% are for sales jobs, with the remaining vacancies spread over clerical, call center, factory, restaurant and retail jobs. We submit 4 applications to each vacancy, each “from” a different candidate using a different email address. We do not represent ourselves as the candidate. Instead, we use a generic email address designed to look like the application was scanned at a copy/printing shop, a generic subject 31 We send each individual a text message: “Dear , we have identified a job opportunity for you. We are a group of researchers trying to help young people find jobs. If you are interested, email your CV to or fax your CV to . Find more info at . Please send your CV within 7 days.” A CV in South Africa is generally understood to include all materials relevant to job applications. 58 Table E.1: Comparison Between Audit and Workseeker Study Samples Workseekers in audit sample All workseekers Mean Std Dev. Obs Mean Std Dev. Obs Panel A: Characteristics of applications received from workseekers Includes references or a reference letter 0.91 0.29 713 - - - Includes a copy of ID document 0.47 0.50 714 - - - Includes information about secondary school completion 0.55 0.50 714 - - - Panel B: Characteristics of workseekers Public treatment 0.30 0.46 717 0.33 0.47 6891 Private treatment 0.37 0.48 717 0.31 0.46 6891 Age 23.2 3.12 717 23.6 3.30 6891 Male 0.48 0.50 717 0.38 0.49 6891 University degree / diploma 0.18 0.38 717 0.17 0.37 6891 Any other post-secondary qualification 0.24 0.42 717 0.21 0.41 6891 Completed secondary education only 0.58 0.49 717 0.61 0.49 6891 Numeracy assessment score (z-score) 0.06 0.96 717 0.05 0.99 6891 Literacy/communications assessment score (z-score) 0.02 0.94 717 0.05 0.99 6891 Concept formation assessment score (z-score) 0.11 0.93 717 0.05 0.99 6891 Grit assessment score (z-score) 0.10 0.99 717 0.03 0.99 6891 Worked in the last 7 days (endline) 0.40 0.49 717 0.38 0.48 6891 line, and generic email message.32 We send most applications within 2 weeks of compiling the vacancy list. We use a three-stage randomization process. First, we generate multiple applications per candidate and randomly assign half of these to treatment status and half to control status. Treatment applications are sent with a public certificate and control applications without any certificate. In all other respects, treatment and control applications are identical. This randomization is independent of workseekers’ treatment status in the workseekers’ study. This generates within- and between-candidate variation in the information content of their applications. Second, we randomize vacancies to receive either one or three applications with reports. This generates within-vacancy variation in the information content of the applications received and between-vacancy variation in the overall information environment. Third, we randomly match applications to vacancies, subject to the target number of treated and control applications and the constraint that no candidate’s application is sent to the same vacancy more than once. The realized distribution of treatment assignments shown in Table E.2, Panel A matches the intended design: half of the applications are sent with certificates and, mechanically, applications sent with reports are three times more likely to be sent to vacancies that receive three applications with reports. We monitor and record responses for two weeks after sending the applications. We classify each response into one of these categories: (1) interview invitation, (2) request to send more information or visit the establishment in person, (3) email bounce, (4) scam, and (5) other - mostly personalized 32 We cross-randomize the subject lines “Application for ” and “Application for ” with the email messages “Please find attached the application for as recently advertised online” and “Please find the application for for , as recently advertised online.” 59 Table E.2: Descriptive Statistics for Application-Level Attributes Mean Std Dev. # Obs Panel A: Characteristics of applications submitted Had one report in a vacancy with one report 0.12 0.33 3992 Had one report in a vacancy with three reports 0.38 0.48 3992 Had no report in a vacancy with one report 0.37 0.48 3992 Had no report in a vacancy with three reports 0.13 0.33 3992 Panel B: Responses to applications submitted Any response received 0.15 0.35 3992 Interview request received 0.09 0.29 3992 acknowledgements of receipt. If any application sent to a vacancy receives a type (3) or (4) response, we drop the vacancy from the sample. We define two outcome variables for analysis. First, any application that receives a type (1) response is coded as an ‘interview invitation.’ Second, any response that receives a type (1), (2), or (5) response is classified as ‘any response’. We forward all responses to the relevant candidate so they can contact the firm. The final sample consists of 3,992 applications sent to 998 vacancies, after dropping 70 vacancies with bounce or scam responses. Of these applications, 15% receive any response, including 9% that receive interview invitations (Table E.2, panel B). 60 F Placebo Certification Experiment: Sample Report and Treatment Effects Figure F.1: Sample Placebo Certificate REPORT ON ASSESSMENT PROCESS name.. surname.. ID No. id.. This report provides information on assessments conducted by Harambee Youth Employment Accelerator (harambee.co.za), a South African organisation that connects employers looking for entry-level talent to young, high- potential work-seekers with a matric or equivalent. Harambee has conducted more than 1 million assessments and placed candidates with over 250 top companies in retail, hospitality, financial services and other sectors. Assessments are designed by psychologists and predict candidates’ productivity and success in the workplace. This report was designed and funded in collaboration with the World Bank. You can find more information about this report, the assessments and contact details at www.assessmentreport.info. «name» was assessed at Harambee on «date». «name» completed assessments on English Communication (listening, reading, comprehension), Numeracy, and Concept Formation: 1. The Numeracy tests measure candidates’ ability to apply numerical concepts at a National Qualifications Framework  (NQF) level, such as working with fractions, ratios, money, percentages and units, and performing calculations with  time and area. This score is an average of two numeracy tests the candidate completed. 2. The   Communication   test   measures   a   candidate's   grasp   of   the   English   language   through   listening,   reading   and comprehension. It assesses at an NQF level, for example measuring the ability to recognise and recall literal and non­ literal text. 3. The  Concept   Formation  Test   is   a   non­verbal   measure   that   evaluates   candidates’   ability  to  understand  and  solve problems. Those with high scores are generally able to solve complex problems, while lower scores indicate an ability to solve less complex problems. «name» also completed tasks and questionnaires to assess their soft skills: 4. The Planning Ability Test measures how candidates plan their actions in multi-step problems. Candidates with high scores generally plan one or more steps ahead in solving complex problems. 5. The Focus Test assesses a candidate’s ability to distinguish relevant from irrelevant information in potentially confusing environments. Candidates with high scores are generally able to focus on tasks in distracting surroundings, while candidates with lower scores are more easily distracted by irrelevant information. 6. The Grit Scale measures whether candidates show determination when working on challenging problems. Those with high scores generally spend more time working on challenging problems, while those with low scores choose to pursue different problems. DISCLAIMER: This is a confidential assessment report for use by the person specified above. The information in the report should only be disclosed on a “need to know basis” with the prior understanding of the candidate. Harambee cannot accept responsibility for decisions made based on the information contained in this report and cannot be held liable for the consequences of those decisions. This figure shows an example of the certificates given to candidates in the placebo treatment group. The certificates contain the candidate’s name and national identity number, and the logo of the World Bank and the implementing agency. Each work seeker received 20 of these certificates, an email certificate, and guidelines on how to request more certificates. 61 Table F.1: Public and Placebo Certification Effects (1) (2) (3) (4) (5) (6) Hourly Written Index Employed Hoursc Earningsc wagec contract Public 0.120 0.052 0.201 0.338 0.197 0.020 (0.027) (0.012) (0.052) (0.074) (0.040) (0.010) Placebo 0.027 0.020 0.039 0.069 0.054 0.005 (0.043) (0.027) (0.075) (0.184) (0.129) (0.021) p: public = placebo 0.041 0.245 0.045 0.147 0.266 0.471 Placebo / public ratio 0.222 0.379 0.196 0.204 0.275 0.238 # observations 6609 6607 6598 6589 6574 6575 # clusters 84 84 84 84 84 84 Coefficients are from regressing each outcome on a vector of treatment assignments, randomization block fixed effects, and prespecified baseline covariates (measured skills, self-reported skills, education, age, gender, employment, discount rate, risk aversion). Heteroskedasticity-robust standard errors shown in parentheses, clustering by treatment date. Mean outcome is for the control group. All outcomes use a 7-day recall period. Outcomes marked with c use the inverse hyperbolic sine transformation. The index in the first column shows the inverse covariance-weighted averages of the 5 labor market outcomes, following Anderson (2008). The mean ratio of placebo to public effects is 0.258 for all 5 non-index outcomes. The sample sizes differ across columns due to item non-response, mostly from respondents reporting that they don’t know the answer. 62 G Experiments with Firms: Willingness to Pay and Skill Ranking This appendix provides more information about the firm-facing experiments described in Sections 5.1 and 5.2. We recruit a sample of 69 firms located in commercial areas near the low-income residential areas in Johannesburg where most workseekers in our sample live. We survey them about their hiring practices, measure their willingness-to-pay for a database containing information about assessment results for workseekers in our sample, and measure their preferences for different types of skills using an incentivized resume-ranking exercise. Table G.1 reports summary statistics for this sample. Table G.2 shows summary statistics on firms’ preferences for different types of skills. Figures G.1 and G.2 shows screenshots of the platform marketed to firms. Figure G.3 shows the distribution of willingness-to-pay. 63 Table G.1: Summary Statistics for Firm Sample Variable # obs Mean Std dev. 10th pctile 90th pctile Wholesale & retail trade 69 0.623 0.488 Transport, storage & communication 69 0.014 0.120 Restaurant & hospitality 69 0.188 0.394 Agriculture 69 0.014 0.120 Financial & insurance 69 0.087 0.284 Community & social services 69 0.014 0.120 Hiring decisions made exclusively at 69 0.754 0.434 location interviewed Uses external recruiting services 69 0.25 0.43 # employees 69 15.0 29.6 3.0 32.0 # entry-level employees 67 7.24 14.94 0.00 14.00 # vacancies for entry-level employees 59 1.42 3.70 0.00 4.00 # entry-level hires expected in 58 3.95 5.43 0.00 10.00 next 12 months # applications received for last 56 16.2 21.2 2.0 30.0 entry-level vacancy posted # weeks required to fill last 58 4.17 6.47 1.00 8.00 entry-level vacancy posted Mean monthly compensation for 58 8,447 16,273 2,500 9,000 employees in last financial year Total payroll costs in last 31 1.277 2.766 0.078 3.200 financial year (millions) Table shows summary statistics for selected firm attributes variables. Percentiles are omitted for binary variables. First six rows are indicators for sectors. All monetary figures are reported in South Africa Rands. 1 Rand ≈ USD 0.167 in purchasing power parity terms. # observations varies due to item non-response. Missing values for the final variables are more common because the survey was completed by the person responsible for hiring decisions, who did not always have access to financial records. Table G.2: Firm Ranking of Profiles with Different Assessment Results and Education (1) (2) (3) Profile content Share of firms ranking profile: Median Top tercile Highest education aaaa First Last ranking Communication Complete secondary school aaaa 0.119 0.015 3 Concept formation Complete secondary school aaaa 0.075 0.030 4 Focus Complete secondary school aaaa 0.328 0.060 3 Grit Complete secondary school aaaa 0.134 0.045 4 Numeracy Complete secondary school aaaa 0.060 0.090 2 Planning Complete secondary school aaaa 0.194 0.000 4 None One-year post-secondary diploma aaaa 0.090 0.761 7 Table shows summary statistics from firms’ ranking of profiles with different skill assessment results and different level of education. All profiles have middle terciles for skills except that listed in the first column. 64 Figure G.1: Screenshots of Login Page and Filtering Page 65 Figure G.2: Screenshot of Individual Candidate Profile on Platform 66 Figure G.3: Willingness-to-pay for Database of Workseekers’ Assessment Results 30 20 Frequency 10 0 0 2000 4000 6000 8000 10000 Willingness-to-pay (South Africa Rands) Notes: This figure shows the distribution of willingness-to-pay for access to the database of assessment results described in Section 5.1 and shown in Figures G.1 and G.2. Values are in South African rand, with 1 rand ≈ USD 0.167 in purchasing power parity terms. The maximum possible bid is 10,000 South African rand. 67 Appendix References Abebe, G., S. Caria, and E. Ortiz-Ospina (2020): “The Selection of Talent: Experimental and Structural Evidence from Ethiopia,” Manuscript, University of Bristol. Anderson, M. (2008): “Multiple Inference and Gender Differences in the Effects of Early Intervention: A Reevaluation of the Abecedarian, Perry Preschool, and Early Training Projects,” Journal of the American Statistical Association, 103, 1481–1495. Attanasio, O., A. Kugler, and C. Meghir (2011): “Subsidizing Vocational Training for Disadvantaged Youth in Colombia: Evidence from a Randomized Trial,” American Economic Journal: Applied Economics, 3, 188–220. Beaman, L., N. Keleher, and J. Magruder (2018): “Do Job Networks Disadvantage Women? Evidence from a Recruitment Experiment in Malawi,” Journal of Labor Economics, 36, 121–153. Benjamini, Y., A. Krieger, and D. Yekutieli (2006): “Adaptive Linear Step-Up Procedures That Control the False Discovery Rate,” Biometrika, 93, 491–507. Bowling, A. (2014): Research Methods in Health: Investigating Health and Health Services, Maidenhead, GB: McGraw Hill; Open University Press, 4 ed. Budlender, J., M. Leibbrandt, and I. Woolard (2015): “South African Poverty Lines: A Review and Two New Money-Metric Thresholds,” Manuscript, Southern Africa Labour and Development Research Unit, University of Cape Town. Burks, S., J. Carpenter, L. Goette, and A. Rustichini (2009): “Cognitive Skills Affect Economic Preferences, Strategic Behavior, and Job Attachment,” Proceedings of the National Academy of Sciences, 106, 7745–7750. Casale, D. and D. Posel (2011): “English Language Proficiency and Earnings in a Developing Country: The Case of South Africa,” Journal of Socio-Economics, 40, 385–393. Chamorro-Premuzic, T. and A. Furnham (2010): The Psychology of Personnel Selection, Cambridge University Press. Cronbach, L. J. (1951): “Coefficient Alpha and the Internal Structure of Tests,” Psychometrika, 16, 297–334. De Kock, F. and A. Schlechter (2009): “Fluid Intellingence and Spatial Reasoning as Predictors of Pilot Training Performance in the South African Air Force (SAAF),” SA Journal of Industrial Psychology, 35, 31–38. Diamond, A. (2013): “Executive Functions,” Annual Review of Psychology, 64, 135–168. du Rand, G., H. van Broekhuizen, and D. von Fintel (2011): “Numeric Competence, Confidence and School Quality in the South African Wage Function,” Manuscript, Stellenbosch University. Duckworth, A., C. Peterson, M. Matthews, and D. Kelly (2007): “Grit: Perseverance and Passion for Long-term Goals,” Journal of Personality and Social Psychology, 92, 1087–1101. 68 Ederer, P., L. Nedelkoska, A. Patt, and S. Castellazzi (2015): “What Do Employers Pay for Employees’ Complex Problem Solving Skills?” International Journal of Lifelong Education, 34, 430–447. Eskreis-Winkler, L., A. Duckworth, E. Shulman, and S. Beal (2014): “The Grit Effect: Predicting Retention in the Military, the Workplace, School and Marriage,” Frontiers in Psychology, 5, 1–12. Esopo, K., D. Mellow, C. Thomas, H. Uckat, J. Abraham, P. Jain, C. Jang, N. Otis, M. Riis-Vestergaard, A. Starcev, K. Orkin, and J. Haushofer (2018): “Measuring Self-Efficacy, Executive Function, and Temporal Discounting in Kenya,” Behaviour Research and Therapy, 101, 30–45. Gneezy, U., A. Rustichini, and A. Vostroknutov (2010): “Experience and Insight in The Race Game,” Journal of Economic Behavior and Organization, 75, 144–155. Gronau, R. (1974): “Wage Comparisons – A Selectivity Bias,” Journal of Political Economy, 82, 1119–1143. Hanushek, E., G. Schwerdt, S. Wiederhold, and L. Woessmann (2015): “Returns to Skills Around the World: Evidence from PIAAC,” European Economic Review, 73, 103–130. Heckman, J. (1974): “Shadow Prices, Market Wages, and Labor Supply,” Econometrica, 42, 679–694. Heckman, J. and T. Kautz (2012): “Hard Evidence on Soft Skills,” Labour Economics, 19, 451–464. Heckman, J., J. Stixrud, and S. Urzua (2006): “The Effects of Cognitive and Non-Cognitive Abilities on Labour Market Outcomes and Social Behaviour,” Journal of Labour Economics, 24, 411–482. Hepner, P. and C. Petersen (1982): “The Development and Implications of a Personal Problem-Solving Inventory,” Journal of Counseling Psychology, 29, 66–75. Heppner, P. (1988): The Problem Solving Inventory (PSI): Manual, Palo Alto: Consulting Psychologists. Heppner, P., T. Pretorius, M. Wei, D. Lee, and Y. Wang (2002): “Examining the Generalizability of Problem-Solving Appraisal in Black South Africans,” Journal of Counseling Psychology, 49, 484–498. Isaacs, G. (2016): “A National Minimum Wage for South Africa,” Manuscript, University of the Witwatersrand: National Minimum Wage Research Initiative. Kalechstein, A., T. Newton, and W. van Gorp (2003): “Neurocognitive Functioning is Associated with Employment Status: A Quantitative Review,” Journal of Clinical and Experimental Neuropsychology, 25, 1186–1191. Lawrence, I. and K. Lin (1989): “A Concordance Correlation Coefficient to Evaluate Reproducibility,” Biometrics, 45, 255–268. Lee, D. (2009): “Trimming, Wages, and Sample Selection: Estimating Sharp Bounds on Treatment Effects,” Review of Economic Studies, 76, 1071–1102. 69 Leibbrandt, M., A. Finn, and I. Woolard (2012): “Describing and Decomposing Post-Apartheid Income Inequality in South Africa,” Development Southern Africa, 29, 19–34. Lopes, A., G. Roodt, and R. Mauer (2001): “The Predictive Validity of the APIL-B in a Financial Institution,” SA Journal of Industrial Psychology, 27, 61–69. Manski, C. (1989): “Anatomy of the Selection Problem,” Journal of Human Resources, 24, 343–360. Nisbett, R. (2009): Intelligence and How to Get It: Why Schools and Cultures Count, New York: W. W. Norton and Company. Posner, M. and G. DiGirolamo (1998): “Executive Attention: Conflict, Target Detection, and Cognitive Control,” in The Attentive Brain, ed. by R. Parasuraman, MIT Press, 401–423. Powell, J. (1984): “Least Absolute Deviations Estimation for the Censored Regression Model,” Journal of Econometrics, 25, 303–325. Pretorius, T. (1993): “Assessing the Problem-Solving Appraisal of Black South African Students,” International Journal of Psychology, 28, 861–870. Rath, J., D. Langenbahn, D. Simon, R. L. Sherr, J. Fletcher, and L. Diller (2004): “The Construct of Problem Solving in Higher Level Neuropsychological Assessment and Rehabilitation,” Archives of Clinical Neuropsychology, 19, 613–635. Raven, J. and J. Raven (2003): “Raven Progressive Matrices,” in Handbook of Nonverbal Assessment, ed. by R. McCallum, Boston: Springer, 223–237. Roberts, R., G. Goff, F. Anjoul, P. Kyllonen, G. Pallier, and L. Stankov (2000): “Armed Services Vocational Aptitude Battery (ASVAB): Little More than Acculturated Learning?” Learning and Individual Differences, 12, 81–103. Schmidt, F. L. and J. E. Hunter (1998): “The Validity and Utility of Selection Methods in Personnel Psychology: Practical and Theoretical Implications of 85 Years of Research Findings,” Psychological Bulletin, 124, 135–168. Schmidt, F. L., I.-S. Oh, and J. Shaffer (2016): “The Validity and Utility of Selection Methods in Personnel Psychology: Practical and Theoretical Implications of 100 Years of Research Findings,” Manuscript, University of Iowa. Schmidt, K., B. Neubach, and H. Heuer (2007): “Self-Control Demands, Cognitive Control Deficits, and Burnout,” Work and Stress, 21, 142–154. Stroop, J. R. (1935): “Studies of Interference in Serial Verbal Reactions,” Journal of Experimental Psychology, 18, 643–662. Taylor, T. (2013): “APIL and TRAM Learning Potential Assessment Instruments,” in Psychological Assessment in South Africa, ed. by S. Laher and K. Cockroft, Wits University Press, 158–168. Tourangeau, R. (2003): “Cognitive Aspects of Survey Measurement and Mismeasurement,” International Journal of Public Opinion Research, 15, 3–7. 70 Willis, G. B. (1999): “Reducing Survey Error through Research on the Cognitive and Decision Processes in Surveys,” Short Course Presented at the 1999 Meeting of the American Statistical Association. ——— (2008): “Cognitive Aspects of Survey Methodology (CASM),” in Sage Encyclopedia of Survey Research Methods, ed. by P. J. Lavrakas, Thousand Oaks, California: Sage Publications, 104–106. 71