44187 World Bank's ENTERPRISE SURVEY HOW TO IMPLEMENT THE SURVEY February 2007 Available at www.enterprisesurveys.org INTRODUCTION The World Bank's Enterprise Surveys (ES) collect data from key manufacturing and service sectors in every region of the world. The Surveys use standardized survey instruments and a uniform sampling methodology to minimize measurement error and to yield data that are comparable across countries. Most importantly, the Enterprise Surveys are designed to provide panel data sets. Because panel data is one of the best ways to pinpoint how and which of the changes in the business environment affect firm-level productivity over time and across countries, the Enterprise Survey team has made panel data a top priority. The use of properly designed survey instruments and a uniform sampling methodology enhances the credibility of World Bank analysis and the recommendations that stem from this analysis. The Enterprise Survey Initiative aims to achieve the following objectives: · To provide statistically significant investment climate indicators that are comparable across all of the world's economies; · To assess the constraints to private sector growth and enterprise performance; · To build a panel of establishment-level data that will make it possible to track changes in the business environment over time, thus allowing, for example, impact assessments of reforms and policy changes; and · To stimulate policy dialogue on the business environment and to help shape the agenda for reform. The purpose of this note is to provide information to the implementing contractors on how to implement the surveys. Two complementary notes, the Questionnaire Note, and the Sampling Note complete the documentation for these surveys. The Questionnaire Note is geared to researchers, field managers, field supervisors and enumerators. The Sampling Note may be of interest to researchers. 1. What is in an Enterprise Survey questionnaire To generate internationally comparable data, the questions in the Core questionnaire are asked in all countries and for all industries where the survey is implemented. In addition to this Core instrument, the Manufacturing Module and Services Module questions are asked to establishments in the manufacturing and services sectors, respectively. Attachments A, B and C contain the Core, Core plus Manufacturing Module, and Core plus Services Module, respectively. Attachment D is the Screener Questionnaire. The Screener Questionnaire is used to screen those establishments that do not fit the sampling criteria and should not be surveyed. The Core instrument is comprised of eleven sections. The first group deals with the characteristics of the business and the investment climate in which it operates including: · Section A ­ Control Information · Section B ­ General information: ownership, start-up. · Section C ­ Infrastructure and Services: power, water, transport, and communication technologies. · Section D ­ Sales and Supplies: imports, exports, supply and demand conditions. · Section E ­ Degree of Competition: price and supply changes, competitors. · Section G ­ Land: land ownership, land access issues. · Section I ­ Crime: extent and losses due to crime. 2 · Section J ­ Business-Government Relations: quality of public services, consistency of policy, regulatory compliance costs (management time, bribes); and · Section M ­ Investment Climate Constraints: evaluation of general obstacles. These eight sections contain qualitative questions, asking for the manager's opinion on the business environment and for his motivation for business decisions. Section F, Capacity: use of production capacity, hours of operation, is a section only included in the Manufacturing Module. The second part of the instrument deals with facts and figures specific to the transactions establishments make in order to operate. More specifically, these sections contain questions on production costs, investment flows, balance sheet information and workforce statistics. These sections include: · Section K ­ Finance: sources of finance, terms of finance, financial services. · Section L ­ Labor: worker skills training, skill availability, employment, education levels of workers; and · Section N ­ Productivity: Numbers and figures needed to estimate productivity. 2. Who is surveyed The Enterprise Surveys are targeted to a particular set of firms; manufacturing and retail/wholesale establishments with five (5) or more full-time employees, located in major urban centers. This section contains explanations of what is meant by: the manufacturing and retail/wholesale industries, an establishment, full-time employees, and major urban centers. Manufacturing refers to the mechanical or chemical transformation of materials or substances into new products. Manufacturing operations are generally conducted in facilities described as plants, factories, or mills, and characteristically use power-driven machines and materials-handling equipment. In addition, the assembly of components of manufactured products is considered manufacturing, as in the blending of materials. For the purposes of the Enterprise Surveys, only activities that are undertaken for the production of goods the establishment itself produces are of concern. If an establishment purchases and resells these items, the revenue it earns from the goods it resells is considered revenue from commerce and not manufacturing. For example, an establishment may manufacture and sell men's shirts, but must buy some shirts from other manufactures in order to complete orders. Revenue from manufacturing refers only to the sales that come from the shirts actually produced in the respondent's factories. The retail and wholesale industries are the two service industries targeted by the surveys. Retail services refer to the selling of goods and/or services directly to the consumer while wholesale refers to the sale of goods or services to someone other than final consumers. Sales to a retail business, merchants, manufacturers, industrial firms, commercial, and other businesses which are sometimes called a middleman or distributor. An establishment is a single physical location where business is conducted or where services or industrial operations are performed. A firm may have one or many establishments. For example, a firm that produces beverages can have many bottling and distribution establishments as well as the establishment where the beverages are made. For the purposes of this survey an establishment must make its own financial decisions and have books separate from that of the firm to be included in the survey. An establishment should have its own managerial oversight and control its payroll. 3 The World Bank's Enterprise Survey is designed to survey firms with five or more full-time employees. By full-time, we mean employees that work up to eight or more hours per day. Urban centers are the major metropolitan areas of the countries included in the survey. This does not exclude establishment outside the boundaries of a municipality but located within a metropolitan area. The sampling frame, described below, provides more information as to what industries are included in the services and manufacturing sectors of this survey. 3. Field Work Implementing the survey involves a number of separate tasks, most of them contractual responsibilities of the implementing contracted firm. These activities include: · Identifying and building an appropriate sample frame · Hiring and training enumerators and their supervisors · Piloting the survey · Securing the participation of enterprises · Making and tracking appointments and tracking refusals to participate · Implementing the questionnaire · Tracking survey completion and quality control, and · Entering data and quality control. a. How to identify and build the appropriate sample frame The validity of any statistical inferences drawn from the survey results hinges on having an appropriate sample frame. For the Enterprise Surveys, a sample frame refers to a list that identifies every establishment, firm or any other economic unit within the target population of interest. Such a list is needed so that every individual economic unit of the population can be identified unambiguously. Identifying an appropriate sample frame requires in-depth consultation with local contacts and usually personal visits to possible sources. Sample design and sample frames are discussed in more detail in the Sampling Note, but an important part of the initial field work is developing a sample frame and developing a relationship with the country statistics agency. In some countries, the statistical agencies can help with the sample frame and also provide additional information on firm entry and exit that may affect the accuracy of the sample frame. However, in many other countries, the government may not have an up-to-date sample frame. In these cases, alternative frames must be identified or assembled through business associations, market research organizations, or other institutions. There are instances where the sample frames are incomplete and in almost all cases the sample frame is out of date. In these cases, the implementing contractor must begin with the data in hand and complete the information in order to be able to locate and identify the characteristics of the establishments and/or verify the level of accuracy of the sample frame. For more on this, the reader can consult the Sampling Note. 4 b. How to implement the questionnaire The Enterprise Survey questionnaire is designed to be administered in face-to-face interviews with managing directors, accountants, human resource managers and other relevant company staff. Ideally it is carried out in partnership with the organized private sector, such as a local chamber of commerce or business association. Section A should be filled out by the enumerator before the interview begins. The information required to fill out Section A comes from the Screener Questionnaire and the sample frame. The purpose of Section A is to verify the accuracy of the sample frame, update the sample frame information with the information provided from the Screener, and to guarantee that the establishment is classified correctly with respect to the sample. Given the importance of the information contained in Section A, the enumerator should seek to confirm the accuracy of this information at some point in the interview. The eight sections, sections B through J and M, contain qualitative questions, asking for the manager's opinion on the business environment and for his motivation for business decisions. Consequently, these sections of the questionnaire should be administered to the managing director or direct representative, whose evaluation of such conditions would influence the establishment's investment and management strategy. The second part of the survey, sections K, L and N, where there are questions on production costs, investment flows, balance sheet information and workforce statistics, this section can be administered to the managing director. However, in larger establishments this part of the questionnaire may be best administered to the accounting department/book keeper and/or human resource manager. In enterprises that do not keep formal books, the figures must be estimated or calculated from records that may not be readily accessible. As a result, this part of the survey is the most subject to non-response or erroneous response. It may be necessary to leave this portion of the survey with the establishment to provide time to search for the information and fill out the questionnaire correctly. The enumerator should make special efforts to assist respondents to provide figures for these sections. These sections will be more difficult to fill out because even those enterprises that do keep formal books may be reluctant to reveal their true data since they may be underreporting either their income or wages to public agencies. In this context, assuring respondents of the confidentiality of their information and the potential value to the enterprise of reform recommendations may induce reluctant participants to provide this sensitive information. Survey length affects the quality of the data gathered; the longer the survey instrument the poorer the quality of the data. Experience shows that when interviews go beyond 1 hour and 10 minutes, data quality is compromised. For this reason, the Core plus either the Manufacturing or Service Module has been restricted to fewer than two hundred (200) variables. Field experience suggests that the Core ES will take forty-five (45) minutes to an hour. The instrument is broken into parts and it is usually possible to conduct the separate parts simultaneously, with multiple enumerators, thus collecting a large amount of information without taking too much time. The additional questions that comprise the Manufacturing or Service Module will take an additional twenty to thirty minutes to complete. More on how to implement the survey can be found in the Questionnaire Note. 5 c. Why and how to pilot Every questionnaire must be piloted prior to launch of the surveys. The purpose of the pilot is to determine if questions are properly translated, worded and understood in the context of the particular country's business environment. The pilot might also reveal important business environment issues that are not covered by the initial survey formulation. Piloting consists of 20-25 interviews, depending on the number of establishments in the sector. If regional differences are considerable, an attempt should be made to pilot the survey in all major regions that the survey covers. Also to the extent possible, firms from all size categories and industries should be included in the pilot sample. Data obtained from the piloting should be recorded in a database. Piloting can also be an important test on the consistency checks programmed into the database to detect irregularities, failure to follow skip instructions and other discrepancies with responses. From this data, preliminary checks on the quality of responses can be determined and most importantly, problems that appear to stem from enumerator mistakes can be detected and corrected early in the process. While the content of questionnaires is well set, the purpose of piloting is to determine whether questions do not work or are so poorly worded that they cannot be understood. If piloting uncovers a consistent pattern of problems with the questionnaire, the questionnaire content can be changed. d. How to publicize the launch of the Enterprise Surveys It is vital that the survey be launched with wide publicity and the full support of the local business community. Newspaper advertisements have proven useful in many countries. Another successful technique is to have publicized launch parties attended by business leaders and covered by the local press to lay out the purpose and scope of the survey. Not only do such events build cooperation, but they also begin laying the groundwork for a dissemination campaign and can enhance the business communities' ownership of findings and recommendations. It is useful if such events can be held in all regions that the survey is covering. The World Bank Country Office can be enlisted to help with publicizing the launch of the Enterprise Surveys. Country Directors and the World Bank Public Affairs officers should be informed. It is not unusual that an establishment that has been asked to participate in the Survey will call or contact the World Bank country office in order to verify the validity of the Survey. The country office must be able to vouch that the survey to which they have been asked to participate is in fact a World Bank initiative. Letters, e-mails and sometimes phone calls can be written or made by the contractor in order to ensure participation of the larger establishments. In almost all cases, a formal letter from the World Bank can substantially increase the participation of some establishments. If the country in which the Enterprise Survey is being launched had a previous survey implemented some years back, it is likely that the World Bank can prepare a Country Profile that shows how the data were used in the previous round. The Country Profile has been used mainly to induce participation by medium and large establishments. 6 The implementing contractor should have in mind that it is they, not the World Bank, who should assume the cost and labor for undertaking any letter writing campaign and/or sending out the Country Profile. e. How to make appointments and keep track of progress Once appropriate publicity has been carried out, teams of appointment makers can go to work. One successful technique is to make initial contact by an introduction letter, followed up by telephone. Having a respected businessman or a senior staff member of the co-sponsoring agency involved in making appointments is useful for gaining cooperation, especially from large firms. In some countries the actual enumerators have been used to make appointments as well as conduct the interviews, in other cases the functions have been split, allowing enumerators to spend all their time conducting interviews. How this is actually organized is at the discretion of the contractor, but in all cases, the information contained in the screener questionnaire must be collected and registered prior to each interview. It should be noted that the training of the individuals who make first contact with potential interviewees is too often ignored. Appointment makers should ideally also attend the enumerator and supervisor training. Most importantly, they should know the purpose of the study, how it is carried out, and what the respondent should expect to be asked. These are all questions that establishments asked when they are first contacted by phone and are asked to participate. Appointment makers should be able to be persuasive and know how to make clear the importance of the study to improving the investment climate of the economy in which the Enterprise Survey is launched. A well-practiced, concise and coherent paragraph that convinces the establishment to participate should be a requirement for anyone making first contact with these establishments. At times, companies selected in the sampling process refuse to participate or will not provide essential information asked for in the survey. In this case, appointment makers or enumerators should try to persuade the company to cooperate. If additional support is required to persuade the company to cooperate, it is helpful to enlist the support of the co-sponsoring agency or to seek the intervention of respected business leaders. If the company still refuses, then it should be dropped from the sample and replaced with an establishment drawn from the replacement list. The implementing firm should keep track of refusals to participate by stratum as this information is very useful to quantify the potential non-response bias on any study conducted with the information collected. It is important that the information about establishments that refuse to participate be recorded and kept. Ideally, if the establishment is willing, they will respond to the questions on the Screener Questionnaire, even if they are not disposed to participate in the survey, and this is the information that should be kept. However, in case the establishment refuses to answer the questionnaire, the information contained in the sample frame can be useful as well. The point is not to discard information about those establishments that do not want to participate in the full survey and to encourage them to at least update the information that is in the sample frame by answering the questions that are on the Screener. 7 f. How to train enumerators and field supervisors The proper training of enumerators to elicit accurate and consistent responses is imperative. The quality of the data depends directly on the level of enumerator training. Therefore, enumerators and their supervisors must be thoroughly trained and the best results are achieved if training takes place under the supervision of the World Bank expert. The aim of the training is to ensure that all enumerators thoroughly understand the instrument and are asking questions in a manner consistent with other enumerators in their country and around the world. Training should adhere to the instructions that accompany the Core. Experience shows that training usually takes 8-10 hours of class room work and exercises to familiarize enumerators with the questionnaire. The Questionnaire Note offers some basic enumeration principles that should be covered in the training. We also provide a multiple choice exam that should be administered at the end of classroom training. All enumerators that pass this exam should be considered for the piloting stage of the training and survey implementation. After the classroom work, enumerators must conduct practice interviews under the supervision of experienced interviewers. One technique is to conduct interviews in the morning and return to the class room to provide feedback, discuss results and answer questions in the afternoon. When surveys are conducted simultaneously in a number of locations, the training of all enumerators should take place in the same location. This allows all teams to adopt consistent interpretations of questions and interview techniques. With some surveys, the complexity of the instrument allows for a team of 2-3 people to conduct the interview. One person can interview the manager, while another enumerator works with the accounting department and human resources manager to fill out the second part of the instrument. Enumerators selected to conduct the accounting and human resource section should be knowledgeable of basic accounting principles. They will have to help the companies' staff pull out the required information from their books and in some cases they will have to help construct information for small companies without formal books. In the past, in some countries where costs were not prohibitive, hiring practicing accountants proved extremely useful for quickly extracting accurate accounting data. Field supervisors should go through the whole training that enumerators undertake. Field supervisors provide coordination among enumerators, provide quality checks, accompany enumerators, especially at the beginning of the field deployment, to their first interviews, provide feedback to enumerators, and will also be in charge of those making the first contact, by phone, with potential interviewees and make the appointments. g. How to supervise to maintain quality control The implementing contractor is responsible for tracking survey completion and for quality control. The completed questionnaire, ideally, would go through four levels of quality checks: · Level 1 ­ the enumerator checks the questionnaire immediately after the interview is done; · Level 2 ­ the enumerator and the field supervisor check the questionnaire, ideally, as soon as the interview is done and the enumerator is satisfied that his or her own review of the questionnaire has not turned up any irregularities or inconsistencies and that the markings and handwriting are legible; 8 · Level 3 ­ the supervisor, on their own, should check to see if the answers are legible, clear, and consistent. This should be done without the aid of the enumerator to help the supervisor decipher their handwriting or notes on the questionnaire; and · Level4 ­ the person in charge of inputting the data, if they cannot decipher the hand writing, understand which options was chosen, or their data program finds an error, this person should stop recording the data on the problematic questionnaire into the database and return the document to the field supervisor. The person recording the data in the database should never guess at what is meant by an unclear response. Once an interview is completed, a supervisor within the implementing organization should immediately check to ensure that the questionnaire is filled out in accordance with the instructions in the Questionnaire Note. At this stage the supervisor should check to make sure that all blanks are filled and all critical data has been collected. It is also necessary to check that all answers are legible. In many cases, interview teams gather valuable information that is not a part of the formal survey. To the extent possible, these notes and marginal comments must be communicated to the World Bank survey manager to determine if they should be appended to the data set. Any missing or inconsistent data must be corrected by the enumerator. The more time it takes between interview completion and checking the survey form the more difficult it is to make corrections. It is often necessary to make repeat visits to participating establishments and these visits should be scheduled as soon as possible after the initial visit. Partially completed surveys should remain with the team until they are finished. Once they are complete and have been checked and verified by the field supervisor they should be immediately sent for data entry. Survey supervisors must spot check enumerators' work by following up with companies that they have interviewed. Supervisors should randomly contact participating enterprises and ask if the interview went well. Did the enumerator ask all sections? Was s/he professional? Was the company satisfied with the interview? It is suggested that the implementing contractor follow up with 12 percent of the sampled establishments. For a smaller percentage of establishments, certain key responses should be checked for accuracy. Failure to follow these basic guidelines can lead to spurious information. If an enumerator is found to have made errors or not conducted the interview professionally, the supervisor should provide the enumerator immediate feedback. In some cases, enumerators will need to be replaced. In addition to ensuring survey forms are filled out correctly, the implementing contractor must ensure that the sample meets the criteria set out in the sample design. The completion rate of interviews must be constantly monitored and checked against the stratification requirements and other elements of survey design as explained in the Sampling Note. 4. Data entry and quality Enumeration is not complete until the information is entered into a data base. Data entry should take place in tandem as field interviews are being conducted. Data entry is an integral part of the quality control process. Data entry programs should clearly alert for missing or out of range entries. The World Bank survey manager, along with the contractor, will determine what constitutes a completed survey that can be sent for data entry. However, there are guidelines, provided here, to make that determination. 9 Sections A, B, C, D, E, F, G, I, J, K and M should have no missing data other than that generated by the skip patterns. For these sections, the World Bank's survey manager requires the implementing contractor to submit questionnaires where all questions have responses. Where a respondent refuses to respond, the code "REF" (-8) may be acceptable but only in those questions where the questionnaire provides this option. Other codes, such as -6 "Still in process", as a response to questions where an application or a request for services was submitted by the establishment. The code -5, for "Request denied", is provided as an option. Code -5 indicates that "No specific period of time" was specified to pay off the line of credit or loan in Question K.12: K.12 Referring only to this most recent line of credit or loan, how much time, in months, was this establishment granted to pay off the loan or line of credit when it was received? Months Months to pay off most recent loan loan/line of credit k12 For sections L and N, since many of the questions require that the respondent provide numbers and figures that they may be reluctant or unable to provide, the tendency is for the respondent to provide "DK" answers (-9) or "REF" (-8) when the respondent refuses to respond. However, where there are a predominant number of questionnaires with sections L and N filled with either (- 9) or (-8), this will reduce the ability for statistically significant estimation. For this reason, the World Bank TTL sets performance standards as to what is an acceptable level of "DK" and "REF" responses for these two sections. First, all survey instruments must include a non-zero response, and a response other than "DK" or "REF", for the following question: N.5 For fiscal year [insert last complete fiscal year], please provide the following information about this establishment (in thousands of local currency units): Last Complete Fiscal Year Total annual revenue 000s LCUs Second, for sections L and N, item responses that are not "REF" or "DK" will be no lower than seventy-five percent (75%) per sampling stratum. The seventy-five percent (75%) is calculated at a variable, not question, level. In cases where the enumerator left behind these sections and the respondent returns these sections with many responses left blank, the enumerator will be required to determine whether blanks are a result of the respondent not knowing the answer to the question and which blanks are due to the fact that the respondent refuses to answer the question. During the data entry process, reports should be run every day looking for outliers or anomalies that might be the result of incorrectly reported or incorrectly entered data. The data entry supervisor should look at important ratios such as value added/worker sales/worker, changes in capital stock compared to investment and other ratios that will pick up incorrect data. If this is done while the enumerators are in the field, then enterprises can be re-contacted to sort out discrepancies. The data 10 entry team should return any incorrectly filled-out questionnaires or questionnaires with discrepancies to the enumerator who conducted the interview. Validated questionnaires should be entered using a computer program form (such as EPI-INFO, MS-Access, etc.) that will allow the exclusion of inadmissible entries and automatically follow the required skip patterns. The program must be able to output the data set into a format commonly used in the World Bank. STATA is the preferred format. It is important to agree how they will manage data entry quality control. Some contractors use a process of double blind entry, where all forms are entered twice and then matched to catch mistakes. However, such a process is expensive for a survey as large as the ES. Another approach is to only re-enter a small portion of randomly selected surveys. Implementing contractors that are confident in their ability to check completed questionnaires and in the quality of their data entry team forgo double blind entry. In this situation they run other types of checks, for example printing out common ratios and looking for outliers. The World Bank survey manager will discuss how the contracting firm assures quality control and be satisfied with their system. The quality of the data is the ultimate responsibility of the implementing contractor and the survey manager. In order to avoid errors in data entry and cleaning, the instrument contains coding variables that might lead to non-sampling errors. For instance, variables referring to "Still in process" (-6) "Not Applicable" (NA= -7), and "Refuses to Answer" (REF = -8), "Don't Know" (DK = -9) should be clearly identified with a one-digit negative number as shown in parentheses. 5. Sampling Guidelines The essential feature of the Enterprise Survey is its ability to make international comparisons and, where possible, sub-national inter-regional comparisons. Consequently, success depends on the sample used for the underlying survey. The consistency and efficiency of the national or regional estimates are only as good as the sample design. Budget constraints limit the overall sample size of a typical survey to a very small fraction of the population. To obtain an accurate picture of the investment climate, this small sample must then be spread over a number of industries, and establishment size categories. The sampling methodology is designed to generate the necessary sample sizes per industry and size to conduct statistically robust analyses with levels of precision at a minimum 7.5% precision for ninety percent (90%) confidence intervals about: · Estimates of population proportions (percentages) of all variables asked as proportions (percentages), at the industry level and; · Estimates of the mean of log of sales at the industry level. The log of sales is taken as the most important quantitative variable from the survey because it has been the variable most commonly used to assess establishment performance. The survey's focus imposes further requirements on the sample design. While representation of the broad economy is highly desirable, given the productivity orientation of the survey, a small number of large sectoral sub-samples must be included to provide measures of productivity that can be compared to parallel sectors in other countries. At a minimum, sectors from both manufacturing and services should be covered. With respect to larger countries with substantial inter-regional variation, location may be added as an additional stratification criterion. In large economies like 11 Brazil, China, India, Indonesia, South Africa, and Mexico, capturing regional or state differences is a key objective of sample design. Within the parameters to design a sample that are provided in the Sampling Note, the details and the weighting scheme must be documented in a technical appendix to the Enterprise Survey as well as provided by the implementing contractor along with the data base. A more complete and more technical discussion of the sampling methodology is provided in the Sampling Note. Attachment Description A Core Questionnaire B Core Plus Manufacturing Questionnaire C Core Plus Services Questionnaire D Screener Questionnaire E Sampling Manual I Multiple Choice Exam J Multiple Choice Exam ­ Answer Key 12