WPS4588 Policy ReseaRch WoRking PaPeR 4588 An Evaluation of the Initial Impact of the Medical Assistance Program for the Poor in Georgia Xiaohui Hou Shiyan Chao The World Bank Europe and Central Asia Region Human Development Department and South Caucasus Country Department April 2008 Policy ReseaRch WoRking PaPeR 4588 Abstract As part of the recent health reform effort, the government poor. Second, the benefits have successfully reached the of Georgia launched a Medical Assistance Program poorest among the poor. These two findings indicate in June 2006 to provide health insurance to its poor that government efforts to improve the poor's access to population. So far the program covers slightly over 50 and utilization of health services are yielding results. percent of the poor and provides benefit coverage for The paper emphasizes that the initial dramatic increase outpatient and inpatient care. This paper estimates in surgeries must be interpreted with caution, given initial impact of the Medical Assistance Program and the possible misclassification or misreporting of acute assesses whether the benefits have reached the poorest surgeries in the data. The paper also stresses the need among those eligible, using utilization data from June to continue monitoring implementation of the Medical 2006 to December 2006. Based on the analysis using Assistance Program and further improve program design, a regression discontinuity design and a three-part particularly the targeting mechanism, to achieve better model, the paper presents two main findings. First, the efficiency, effectiveness and overall equity in access to Medical Assistance Program has significantly increased health care services. utilization of acute surgeries/inpatient services by the This paper--a product of Human Development Department and South Caucasus Country Department, Europe and Central Asia Region-- is part of a larger effort in the Bank to support the Government of Georgia in its poverty reduction and improving access to social services by the poor. Policy Research Working Papers are also posted on the Web at http:// econ.worldbank.org. The authors may be contacted at xhou@worldbank.org and schao@worldbank.org. The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent. Produced by the Research Support Team An Evaluation of the Initial Impact of the Medical Assistance Program for the Poor in Georgia Xiaohui Hou and Shiyan Chao1 Human Development Department South Caucasus Country Department Europe and Central Asia Region The World Bank Key words: Impact evaluation, health insurance, Georgia JEL code: I10, I18, I38 1The authors would like thank the Social Assistance and State Employment Agency, the former State United Social Insurance Fund and the Ministry of Labor, Health and Social Affairs in Georgia for sharing data and information with us. We specially thank Donna M. Dowsett-Coirolo, Tamar Manuelyan Atinc, Armin Fidler, Arup Banerji, Owen Smith, Rosalinda Quintanilla, Aleksandra Posarac, Tamar Gotsadze, Mamuka Nadareishvili, Nino Moroshkina, Anne Margreth Bakilana, Rianna Lisa Mohammed and Achil Mamaladze and all participants at the World Bank ECAHD Lunch Seminar for their comments and helpful discussions. The findings, interpretations, and conclusions of this paper are those of the authors and should not be attributed to the World Bank and its affiliated organizations, or to the Executive Directors of the World Bank or the governments they represent. 1. INTRODUCTION Following the dissolution of the central planning economic system in the Eastern Europe and Central Asia (ECA) region, most ECA countries began implementing a series of health care reforms (Dulitzky and Hou 2007); Georgia was no exception. The government of Georgia initiated health sector reforms in the mid 1990s with an aim to salvage a health system that was in crisis. Despite a decade of reform efforts, which saw the development of many sector policy papers and strategies, the actual improvement in health sector performance has been limited. The health system is still oversized, of poor quality, under-funded, inefficient, inequitable, and with low utilization rates. The health status of the Georgian population remains poor, as indicated by a high infant mortality rate compared with other countries in the ECA region. Until 2006, the government had refrained from taking any drastic measures to change the organization of the health care delivery system and thereby improve the distressed health sector conditions in the country. Following deep-seated reforms in other areas, Georgian authorities turned their attention to the health sector two years ago and unleashed a fundamental reform process with the objectives of expanding access to and utilization of health services by increasing allocations from the public budget and by partnering with the private sector. One of the initiatives was the Medical Assistance Program (MAP), which has as its main objective improving the poor's financial access to health care services. Under this program, the government of Georgia provides a Supplementary Benefit Package (SBP)2 to the eligible poor, which covers the cost of acute and planned surgeries and inpatient services without any co- payment or deductibles. The program was launched in June 2006 nationwide. This paper provides the first impact evaluation of the MAP based on data from June 2006 to December 2006. The paper has two main findings based on the analysis from a Regression Discontinuity (RD) design and a three-part model. First, the MAP has significantly increased utilization of acute surgeries/inpatient services among beneficiaries. Second, the MAP has been successful in reaching the poorest among the poor. The paper is organized as follows. Section 2 provides background on the Georgian health care system, and Section 3 describes the details of the MAP. Section 4 describes the data used in the paper. Sections 5 and 6 present the methodologies and results for the first and second finding, respectively. Section 7 discusses the findings and some of the policy implications. 2. BACKGROUND Georgia's health care system was severely damaged in the conflict and economic collapse following the breakup of the Soviet Union and independence. Real per capita public expenditures on health declined from about US $13 in 1990 to less than $1 in 1994 (European Observatory on Health Care Systems 2002). The physical condition of medical facilities severely deteriorated, as did medical technology and equipment. Not surprisingly, health indicators declined sharply in 2At the beginning of the implementation in 2006, the Medical Assistance Program was called the Supplementary Benefit Package Program. Later in 2007, it was called the Medical Assistance Program for the poor in order to avoid confusion; the MAP offers a full coverage of benefits, not supplementary benefits in addition to basic benefits. 2 the early 1990s. For example, life expectancy declined from 72.97 years in 1990 to 69.94 years in 1994, and the infant mortality rate increased from 20.7 per 1000 live births in 1990 to 28.6 in 1994.3 In the mid-1990s, the government implemented an initial reform, intended to transform the health system from the National Health Service model to a Social Health Insurance (SHI) system and to shift the government's primary responsibility from service provision to service regulation and financing (Both 2002). In 1995, entitlement to free health care was removed from the Constitution, and the SHI was created with contribution rates of 3% from the employer and 1% from the employee to finance a Universal Benefit Package (UBP) for the general population. However, due to limited funding, the actual provision of benefits is minimal and some require co-payment (European Observatory on Health Care Systems 2002). For services not covered by the UBP, patients have to pay directly out-of-pocket. Since the early 1990s, out-of-pocket payment (OOP) has become the main source of health financing. The results from a Health Service Utilization and Expenditure Survey indicate that the most common reason (73%) for not seeking medical care is the high out-of-pocket expenditures (Gotsadze 2001). Figure1 illustrates the high OOP spending in Georgia compared to EU and CIS averages. The OOP expenditure in Georgia is about 70%­80%, compared to about 40% among CIS countries and 20% among EU countries. Almost half of the total OOP are informally paid (Belli, Gotsadze et al. 2004). Figure1. Georgia has High Private Out-of-Pocket Spending on Health, 1998-2004 e as 80 ng 70 60 endipS urtidnepxE 50 Georgia et th 40 EU ckoP ealH 30 CIS 20 oftuO Total 10 % 0 1998 1999 2000 2001 2002 2003 2004 Year Data source: WHO European Health for All Database (HFA-DB) Due to high out-of-pocket expenditures, lack of access to health services, and the poor quality of services, Georgia has almost the lowest health services utilization rate in the ECA Region, with less than 2 outpatient visits per person, less than 6 inpatient visits per 100 people, and less than 2 surgical procedures per 100 people per year. Figure 2 illustrates the surgical procedures per year per 100 people from 1998 to 2004. The average number of surgical procedures per year per 100 in 2004 is less than 2 in Georgia, compared to about 5 in the CIS countries and about 7 in the EU countries. 3Data source: WHO European Health for All Database (HFA-DB) 3 Figure 2. Georgia has Low Utilization of Surgical Procedures Per Year Per 100, 1998-2004 8 100 7 per 6 5 Georgia cases 4 EU 3 calig CIS 2 sur 1 #0 1998 1999 2000 2001 2002 2003 2004 year Data source: WHO European Health for All Database (HFA-DB) 3. THE MEDICAL ASSISTANCE PROGRAM FOR THE POOR Despite a series of attempts to address systemic issues in the health sector since the mid-1990s, the results have been limited. Responding to the situation, the government has recently launched an ambitious four-year plan aimed at improving equity and access to essential health services, especially for the poor and the vulnerable. These reform efforts focus on four main areas: (i) improving access to health services through involvement of private investment; (ii) prioritizing limited public funds to finance health care for the poor and other vulnerable groups; (iii) channeling public health financing through private health insurance companies, and (iv) strengthening the regulatory role of the Ministry of Labor, Health and Social Affairs (MoLHSA). In this context, in mid-2006 the government launched the MAP, which provides a benefit package targeted to the extremely poor. Beneficiary Selection The beneficiaries of the MAP were selected based on welfare scores derived from a proxy means test. This proxy means test was developed by the Social Assistance and State Employment Agency (SASEA) in 2005 as the primary tool for the targeted social safety net program. The test uses about 80 variables to estimate a household's welfare status. Each applicant (household) receives a welfare score based on the information collected from the survey. Initially, households with scores below 100,000 were entitled to health insurance coverage under the MAP. However, in early 2007, due to budget constraints, the government changed the eligibility threshold, requiring a score of 70,000 or below to be eligible. In 2007, about 650,000 individuals had received their Medical Assistance Card (Chanturidze, Chitashvili et al. 2007), roughly 15 percent of Georgia's total population and 50 percent of that country's poor4 4Based on HIES data and using a consumption-based measure of poverty, approximately 30% of the Georgian population (4.3 million) was considered poor in 2006, and 12% were classified as extremely poor. 4 Administration of the MAP During the evaluation period, the MAP was administered by the former State United Social Insurance Fund (SUSIF). Each individual in an eligible family received a medical assistance card with a common family ID and a specific individual ID. Beneficiaries who presented their cards at government health facilities were exempted from official user fees. The SUSIF also managed the payments of the Universal Benefit Package (UBP) for the general population. In August 2007, SUSIF was restructured and its purchasing function of health services was transferred to the Health and Social Programs Agency (HeSPA), a new legal entity subordinated to the MoLHSA. The HeSPA is currently managing the MAP nationwide except in Tbilisi and Imereti regions.5 Benefit Package Since the introduction of the MAP in June 2006, the benefit package has been rapidly evolving, making evaluation of the package difficult. Three different packages have been offered: an initial one from June to December 2006, a less generous one from January 2007 to September 2007, and a new, most generous package from September 2007 to the present. For the review period of this paper (June-December 2006), the benefit package covers the following:6 a) Acute out-patient and in-patient treatment; b) Planned in-patient service, except expenses for aesthetic surgery, cosmetic treatment, sanitary-resort treatment, sexual disorder, infertility, treatment abroad, sexually transmitted infections, HIV, and hepatitis C; c) Out-patient examinations prescribed by the doctor that cost no more then 200 GEL per beneficiary a year; d) Reimbursement of baby delivery costs; and e) Services envisaged by the Child Medical Assistance, oncology diseases diagnostics and treatment, and referral components of the State Program for Inpatient Assistance for the Population. The process is different for planned in-patient services and acute inpatient service. In the planned cases, providers have to submit an authorization form to SUSIF, and it takes about two to three weeks before scheduling; in the acute cases, authorization is not needed and patients can simply present their medical assistance card to providers. Despite the lists of procedures SUSIF includes under these two broad categories, there is leeway for providers and patients to pursue their collective interests and classify procedures as acute cases when needed. 5 For details, see next section "The trend of Multi-payers under the MAP." 6Description of the MAP is dated May 28, 2007 (Source: MoLHSA), 5 In January 2007, the benefits for planned surgeries/inpatient services were suspended due to budget constraints and later reinstated in the beginning of September 2007. . The trend of Multi-payers under the MAP Starting in September 2007, the government launched a pilot scheme shifting responsibility for provider payment to private insurers in Tbilisi and Imeriti regions. Vouchers were distributed to the approximately 200,000 people in these two regions eligible for the MAP, and the beneficiaries can select among four participating private insurance companies. Participating insurance companies have to provide the benefit package identified by the state and cannot deny membership to any people with vouchers. The HeSPA reimburses the insurance companies based on an average premium of 7.2 GEL per beneficiary per month. The plan is to change eventually from a single state purchaser to multiple payers in the health insurance market. 4. DATA Data Sources This paper limits its analysis to data from June to December of 2006 to avoid two confounding changes. First, the eligibility score changed from 100,000 to 70,000 starting January 2007 and second, the benefit package was reduced by excluding planned surgeries around the same time. Six months is a short timeframe, but it provides clearer evidence on the initial impact of the MAP. The analysis in the paper uses two data sources. The first is the claim data from SUSIF, which contains information on hospital check-in and check-out dates, the amounts SUSIF paid to the providers, and disease codes for patients covered under the MAP and the UBP. The second source is the targeting database managed by SASEA, which contains household demographic information, individuals' education levels, employment status, and welfare scores based on the proxy means test.7 The two data sets share the common family ID and personal ID. A new data set, based on the common IDs, was created by merging these two data sets. After total number of utilizations and total medical expenditures for each individual were generated from the merged data set, duplicate records with the same family ID and personal ID were removed. The unit of analysis is at the individual level. The final data set contains 983,708 individuals below or at the 100,000 cut-off line.8 The data set is rich and unique, as it has each utilization record, which would be unobtainable from a sample survey. However, the data set has its limitations. First, while it contains inpatient utilization by beneficiaries of the MAP, it does not have any outpatient utilization or use of pharmaceutical drugs. Second, it does not have any information on out-of-pocket spending. Third, the data does not contain pre-program utilization information. 7Due to reasons of confidentiality, we only obtained a subset of the targeting database that included all the households but with only limited information. The information includes the welfare score, demographic structure of the households, each household member's marital status, education and employment status, etc. 8There are 646,954 individuals below or at 70,000. 6 Description of Variables Used Acute and planned surgeries/inpatient services The former SUSIF classifies inpatient cases into three categories: acute surgeries, planned surgeries and maternal deliveries. As explained before, there are differences between acute and planned surgeries in terms of the administrative process. A planned surgery has to have prior authorization by SUSIF, which takes about 2-3 weeks before scheduling. Authorization is not needed for acute cases, and patients can present a medical assistance card to providers for acute care. Acute cases are covered under the UBP for all of the population, but non-MAP beneficiaries have to make a 25%­50% co-payment, depending on type of services. Because there is no authorization process required for seeking acute care, people tend to use acute care for cases that are not necessarily emergencies or even acute. Therefore, the surgeries included under the acute surgery category represent cases that did not receive prior authorization from SUSIF. During the study period, the SUSIF reimbursed providers based on reported cases for three categories of inpatient care for the MAP and only acute surgeries for UBP patients. Therefore, the SUSIF data include reported acute surgeries for both MAP and UBP patients. About 20% of the cases right above the 100,000 cutoff line and 12% of the cases immediately below were the surgeries with zero length of stay. These were ambulatory surgery cases, which SUSIF reimbursed providers for using a case-mix payment system. Total number of utilizations and total medical expenditures Total number of utilizations is the sum of utilizations for each individual from June 2006 to December 2006, including acute, planned and delivery services. The same applies to total medical expenditures, which are the payments made by SUSIF to providers for each eligible poor from June 2006 to December 2006. Since there is no formal cost sharing on the part of patients who receive MAP coverage9, SUSIF's total expenditures serve as total medical expenditures. Welfare scores and quintiles The welfare score is derived from the proxy means test. Quintiles are derived from the welfare scores for beneficiaries in order to compare utilization patterns among the poor covered under the MAP as well as to assess whether the benefits reach the poorest. Demographic and household information Demographic information includes age, sex and education, employment status, and relationship to head of household. The level of education is classified according to five categories: no formal education, primary education, secondary education, high school education or equivalent, and 9It is not clear whether MAP patients pay any informal payment, such information will require a household survey. 7 above high school education. Other information includes urban and rural status and the region of residence.10 5. THE IMPACT OF THE MEDICAL ASSISTANCE PROGRAM ON ACUTE SURGERIES/INPATIENT UTILIZATION Method The first objective of this analysis is to evaluate whether the MAP has had any impact on the utilization of acute surgeries and inpatient care among the poor. Regression Discontinuity (RD) method is used for the analysis. RD design has been widely used in program evaluation, in which treatment (or participation) status depends on whether an observed covariate exceeds a fixed threshold (Campbell 1969). The unique characteristic that sets RD designs apart from other evaluation methods is that treatment/participation depends on a cut-off score. It compares outcomes for observations just above and just below the cut-off score (threshold). The assumption is that observations just above and just below the threshold are very similar in terms of both the observed and the unobserved characteristics. Thus, the assignment of treatment/participation is a proxy randomized process. The similarities and differences of the observables between the observations just above the threshold and those just below the threshold can be tested statistically. This method is suitable for evaluating Georgia's MAP as the program identifies beneficiaries based on a continuous welfare score. From June 2006 to December 2006, households with scores equal or below 100,000 were identified as MAP beneficiaries. Beneficiaries receive surgeries and inpatient care without any formal co-payment or deductible, while non- beneficiaries have to pay about 25%­50% of the total costs. Statistical comparisons of the observables between beneficiaries and non-beneficiaries in the neighborhood of the threshold show small and insignificant difference for most variables. These analyses of quasi-random assignment are listed in Appendix 1. The RD design essentially estimates the Local Average Treatment Effect (LATE) in the neighborhood of the threshold. The LATE is different from the Average Treatment Effect (ATE), which estimates the average impact of the program on all beneficiaries. In some cases, the LATE is quite similar to ATE, while in other cases it is not. In this case, the estimated program impact through the RD design is only applicable to the neighborhood of the threshold during the initial six month period. In order to test the effects of the MAP on all beneficiaries, the study further analyzes inpatient utilization by comparing the lowest quintile of the beneficiaries to the highest. The next section provides the details of this analysis and results. There are two broad methods to estimate the LATE using the RD design: non-parametric and parametric (Hahn, Todd et al. 2001). The non-parametric approach avoids any misspecifications but the standard errors are usually estimated through bootstrap. The parametric approach relies on some sorts of specification. In the paper, we choose a "parametric" approach by using a regression that includes a fourth order of polynomial of the welfare score. This parametric 10The paper examined region of residence and region where medical services were delivered and found that in most cases they were the same. 8 approach assumes that we can adequately control for other determinants of acute surgery utilization using a sufficiently flexible polynomial function of the welfare score. The estimation model is as follows: C=1 if S 100, 000, and Y=m(S) + C + , where C is the dummy based on cut-off score, equal to 1 if the welfare score S is less than 100,000; m(S) is a function including a fourth order of polynomial of the welfare score; is the impact of the Medical Assistance Program. Results This section presents the main results from graphical analysis and empirical estimation. Both illustrate the effect of the entitlement to MAP on the utilization of acute surgeries and inpatient care. The estimated impact of the MAP is quite significant: beneficiaries are 9 times more likely to seek acute surgeries/inpatient services as compared to non-beneficiaries in the neighborhood of the threshold. Figure 3 plots welfare scores against the probability of having acute surgeries/inpatient services. To give an overall picture of the data, each point in Figure 3 is an average of predicted probability of utilizing acute surgeries/inpatient care within 500 wide intervals of welfare scores. The vertical line marks the MAP cut-off point at 100,000. The continuous horizontal line is the predicted probability of the utilization of acute surgeries/inpatient care from a regression that includes a fourth order of polynomial in welfare score fitted separately for points above and below the threshold. The dashed lines represent the 95 percent confidence intervals. 9 Figure 3: The Effect of MAP on Utilization of Acute Surgeries/In-Patient Services Cases Acute ofy Probabilit Note: This figure plots probability of utilization of acute surgeries/inpatient services against welfare scores. Each dot is the average probability within 500 intervals of welfare scores. Solid lines are fitted values from 4th order polynomial regressions on either side of the discontinuity. Dotted lines are 95% confidence intervals. The beneficiaries under the MAP are more likely to use acute surgeries/inpatient services than people who have similar poverty status but are not covered by the MAP as indicated in Figure 3. There is a large discontinuity jump in utilization of acute surgeries/inpatient care at the 100,000 score threshold. The figure clearly shows that the probability of utilization of acute surgeries/inpatient care seems to be a continuous and smooth function of welfare score everywhere, except at the threshold that determines the MAP eligibility. If the RD design is valid and the data are clean and complete, the beneficiaries and non-beneficiaries should appear ex- ante similar in every respect on average. The difference can be attributed to the fact that individuals in one group have the MAP coverage and can seek acute surgeries/inpatient services for free, but the other group of individuals has to make a 25%­50% co-payment for acute cases. The MAP beneficiaries clearly utilized more acute surgeries/inpatient services during the study period. The size of the jump is fairly large, from about 0.001 to about 0.01, which is equivalent to 1 utilization per 1,000 per six months to 10 utilizations per 1,000 per six months. Table 1 (tables follow references) present similar results with the empirical estimation from different parametric specifications. The sample contains individuals with scores from 98,000 to 102,000. There are 29,460 observations at the individual level, with 14,133 in the range from 98,000 to 100,000 and 15,327 in the range from 100,001 to 102,000. Column 1 reports the mean of controls (non-beneficiaries with scores from 100,001 to 102,000). The probability for non- beneficiaries to seek acute surgeries/inpatient care is 0.1% or 0.2 per 100 people per year. This is very low compared to the national level which, for acute hospital admissions and inpatient surgical procedures, is 5.7 and 2.2. per 100, respectively, according to 2005 data from WHO 10 HFA-DA. The probability for beneficiaries to seek acute surgeries/inpatient care is 1 per 100 over six months, as presented in Column 2, which is close to the national average level. Columns 3­6 present parametric estimations of the results, including OLS with a dummy variable defined as equal to 1 if beneficiary (Column 3), OLS with the dummy variable and a fourth order polynomial in welfare scores (Column 4), OLS with the dummy and other covariates to control for additional differences between the beneficiaries right below the threshold and non- beneficiaries right above the threshold (Column 5), and finally a Probit model with marginal probability presented (Column 6). The last column presents the impact size based on the first OLS estimation. Different specifications yield very similar results, and they all indicate that the beneficiaries of the MAP utilized more acute surgeries than the non-beneficiaries in the neighborhood of the threshold. The impact of the MAP, i.e. beneficiaries are nine times more likely to have acute surgeries than non-beneficiaries with a similar poverty status, seems unusually high. One needs to interpret this result with great caution. Possible reasons for such high impact are proposed below. First, the health services utilization rate is very low in Georgia, and the main reason for not seeking care is financial, especially among the poor. Lifting the financial constraint removes the most significant barrier to poor people seeking care and would naturally precipitate a huge increase in health services utilization by this sub-group of the poor population. This finding is consistent with the literature, which shows that when costs of medical care are reduced by health insurance, people tend to use more health services (Holly, Gardiol et al. 1998; Harmon and Nolan 2001; Hofer 2006). A significant increase in utilization of health services has also been observed when user fees are removed (Deininger and Mpuga 2003; Yates 2007). However, the literature on price elasticity of demand for medical care yields changes in demand of a smaller magnitude when cost of care changes.11 12 A strict comparison with the literature is difficult due to the different methodologies and measurements used in the various studies. The most compatible case is the Rand Health Insurance Experiment study (HIE), which assessed responses in demand to various cost-sharing schemes as compared with free care and showed a 20%-39% deduction in hospital admissions due to cost-sharing.13 The extremely high increase in utilization of surgeries seen in this study of the MAP may be partly attributable to accumulated demand, given that the study only covers the first six months of the program. Pent up demand for health services may have caused the poor who qualified for benefits to rush to take advantage of them before yet another change in health policy could result in their revocation, thus causing a dramatic spike in utilization in the first six months. One may observe a gradual decline in the surge once the cumulated demand is exhausted. Second, as mentioned before, the SUSIF data may suffer from a misclassification problem. The increased use of acute surgeries may have resulted from classifying non-acute surgeries as acute 11 P. Zweifel and W. Manning, 2000. "Moral Hazard and Consumer Incentives in Health Care" in Handbook of Health Economics, Vol. 1 Elservier Science B.V. 12 W. Manning et al. 1987 "Health Insurance and the Demand for Medical Care: Evidence from a Randomized Experiment", The American Economic Review, Vol. 77 No. 3 13Rand Health, 2006 "The Health Insurance Experiment: A classic RAND study speaks to the current health care reform debate", Joseph Newhouse and the Insurance Experiment Group. Free for All? Lessons from the RAND Health Experiment. Cambridge. Mass.: Harvard University Press, 1993. 11 cases in order to avoid waiting for SUSIF authorization. Therefore, some of these surgeries may represent accumulated non-acute cases. The SUSIF data also do not capture surgeries/inpatient cases covered through private health insurance. However, private health insurance covers less than 1.5 % of the population and is mainly employer-financed. Therefore, it is unlikely to cover the poorest population, the focus of this study.14 It is also possible that there is misreporting of acute surgeries for the MAP patients vs. UBP patients due to different reimbursement rates between the two programs, i.e., the providers would be fully reimbursed for the MAP patients but only partially reimbursed for the UBP patients. Finally, in spite of the MAP's great impact, it still only managed to raise utilization rates of the insured poor closer to the level of the national average at the approximate measures,15 which is still below the average of many other comparable nations. The recent Georgia Health Utilization and Expenditure Survey (HUES) shows that the poorest quintile's share of inpatient care utilization was still 30% lower than the national average inpatient utilization.16 Limitations This paper is only able to examine the impact of the MAP on acute surgeries/inpatient services, due to data limitations. Acute cases are covered under both MAP and UBP, and the difference in coverage is that MAP beneficiaries do not need to make any co-payment while non-MAP users have to make a co-payment of 25%­50%. Since in both cases SUSIF pays providers at least a portion of the costs, SUSIF keeps the utilization record of the acute cases. This is not the case for planned surgeries, for which non-MAP beneficiaries have to pay total costs out-of-pocket unless these are covered by other private health insurance plans. Since SUSIF does not pay for those services, it does not keep the records for those surgeries; therefore, the comparison between MAP users vs. non-MAP users for planned surgeries was not possible. Due to the very small number of cases of delivery during the study period--e.g., two cases of delivery among women with scores between 100,001 to 110,000--this study could not examine the impact of the MAP on utilization of delivery services. 6. HAVE THE POOREST AMONG THE BENEFICIARIES TRULY BENEFITED? Section 5 estimates the impact of the MAP in the neighborhood of the threshold, which has little implication for the utilization of services by the poorest among the beneficiaries. This section focuses on whether the MAP benefits have reached the poorest of the poor. It is not the case that all welfare programs reach the individuals who need them most. In this case, there are factors that affect the poorest differently in their seeking of inpatient care. Figure 4 illustrates different pathways that affect inpatient service utilization. The poorest tend to be less educated, and therefore are likely to be less informed about the MAP. In addition, high travel costs are usually associated with seeking inpatient care, creating a potential financial barrier; and third, the poorest include those living in remote or mountainous areas, where the supply of physicians and medical 14Ministry of Labor, Health and Social Assistance, 2007. Georgia Health Utilization and Expenditure Survey conducted during June 2006 to July 2007. 15According to WHO Health data base for all, Georgia has less than 2 outpatient visits per person, less than 6 inpatient visits per 100 people, and less than 2 surgical procedures per 100 people per year in 2004. 16Ministry of Labor, Health and Social Assistance et al, 2007. Georgia Health Utilization and Expenditure Survey. 12 facilities tends to be limited, further contributing to their low utilization of health services. Compounding the situation, the poorest often have worse health status due to poor living conditions and nutrition intake as well as lack of access to health services. As a consequence, the need for care among the poorest is high. When inpatient services are offered free of charge, the poorest are more likely to utilize services. Figure 4. The Possible Pathways of the Impact of MAP on Surgeries/Inpatient Services Utilization 1. Less or not informed about the program 2. High travel costs or other costs The poorest use less associated with seeking care benefits/services 3. Limited supply of medical professionals or medical facilities MAP PROGRAM Initial poor health due to long term lack of The poorest use more access to health services, poor diet, etc. benefits/services The literature has documented different international experiences in this area. For example, Abel-Smith and Rawal (Abel-Smith and Rawal 1992) found in Tanzania that many patients had to incur substantial costs, in addition to travel costs, to use the "free" services supported by the government because of inadequate supplies of drugs and food at hospitals. Other literature shows that the poorest beneficiaries are more likely to use health services because of their poor health, which is partly due to their having had little access to health services prior to the program's inception (Trujillo, Portillo et al. 2005; Wagstaff 2007). Method In order to test whether the MAP has reached the poorest in Georgia, all MAP beneficiaries are grouped into five quintiles based on their welfare scores, and the use of services is compared between the lowest quintile and the highest. The RD analysis estimates the impact of the MAP on the beneficiaries who are right below the threshold. The following comparison between the lowest and the highest quintile among the beneficiaries will inform us whether MAP has benefited the poorest. 13 The analysis covers both utilization of services and the distribution of total medical expenditures in the state budget. A central problem in estimating health expenditures is its contingency on the utilization of the health services, which, as a result, affects the distribution of the health expenditures. To address this problem, this paper adopts the multi-part model developed by Manning et al. (Manning, Newhouse et al. 1987) and Duan and Chau (Duan and Chau 1987) and utilized as part of the Rand Health Insurance Experiment (Newhouse and the Insurance Experiment Group 1996) and other papers (Ruiz, Amaya et al. 2007). The multi-part model separates the process into two stages, with the first being the decision to use medical services and the second being the level of expenditures for people who have used the services. This paper follows the same strategy and uses the three part model to estimate the probability of utilizing services, the number of services used and the incurred medical expenditures. The three part model consists of a LOGIT model, an OLS and a LOG-Linear model. The formal utilization model specification is as follows: The LOGIT model estimates the probability of an individual seeking surgeries/in-patient services, especially to analyze whether the poorest and the most vulnerable individuals have utilized the benefits. Prob (USE=1) = 1+ 1X+ 1 , where USE is a binary variable with having received at least one surgery/in-patient care equal to 1; 1 is a vector of coefficients and X represents a set of independent variables. Part two is a regression model analyzing the difference in number of utilizations by welfare quintiles among the eligible individuals who have used the services at least once. N(service | USE=1) = 2+ 2X+ 2 , Part three is the log-linear model that estimates the total expenditures for the surgeries and inpatient services. LOG(EXP| USE=1) = 3+ 3X+ 3 The vectors of the coefficient estimates for the respective models are represented by 1, 2, 3. The respective error terms are symbolized by 1, 2, and 3. It is assumed that E () = 0. The data has been restricted to individuals between 18 and 80 years old. We restrict the sample to 18­80 years of age in order to control for education and employment status, information which is often missing for individuals below 18 years of old. Analysis for individuals between 3 and 17 years old reveals similar results, which are available upon request. In addition to age restriction, records with length of stay greater than 60 days or total expenditures greater than 10,000 GEL have been removed. In the end, the data contains 738,342 individuals, of whom 16,434 (2.23%) have used the services. 14 Results Table 2 shows characteristics of the beneficiaries of the MAP by welfare quintiles. As expected, the bottom quintile of those receiving MAP coverage is comprised of the poorest and most disadvantaged; these individuals are proportionally older, less educated and unemployed. Forty- one percent of the population in the lowest quintile is between the ages of 60 and 80, compared with 36.5% in the second quintile, 31.7 % in the third, 27.3% in the fourth, and 24.2% in the top quintile (P<0.001 for F-test). The lowest quintile has a higher percentage of individuals without any formal education, about 3.2% compared to 1.7% in the second, 1.2% in the third, 1.0% in the fourth, and 0.8% in the top quintile. The same patterns follow for the education of the head of the household. In terms of employment status, the lowest quintile has more unemployed people (70%). There is a significant difference in poverty distribution across the regions, with Imereti and Samegrelo regions having the highest proportion of the poorest population. Poverty distribution also varies significantly between urban and rural areas, with more poverty in rural areas. . Table 3 presents average utilization rates of acute and planned surgeries/inpatient services by welfare quintiles. People in the poorest quintile had more acute surgeries/inpatient services as well as more planned surgeries (p<0.001). The number of deliveries does not vary significantly across quintiles. Among the individuals who have used the services at least once, there is no significant difference in terms of number of utilizations and total medical expenditures. Table 4 presents results for the surgery/inpatient service utilization model comparing users vs. non-users (part I model). Part I indicates a significantly higher probability of utilization by the lowest quintiles compared with the highest quintile among the beneficiaries. The poorest are 47% (OR=1.47, CI=(1.38, 1.57)) more likely to have acute surgery/inpatient services compared with the fifth quintile, and 13% (OR=1.13, CI=(1.02, 1.26)) more likely to have planned surgeries after adjusting for a vector of control variables, including age, sex, education, employment status, region of residence and other confounding factors. The finding favors the hypothesis that the poorest tend to use more services due to inferior health status resulting from financial barriers to access and limited utilization before the MAP. There is no significant difference in the number of deliveries among the five quintiles, which is consistent with the hypothesis because demand for maternal delivery is much more inelastic than other health services. Education plays an important role in determining the utilization of services. People with higher education tend to use more services compared to people with no formal education. The head of household's educational attainment has little impact on the individual's utilization. Table 3 also indicates a significantly lower probability of utilization by family members other than the head of household. Employed people are significantly less likely to use the services. There are at least two possible explanations. First, there is a higher opportunity cost involved for the employed to seek services as they have to take time off from work. Second, employed people are generally healthier as their health is a condition for employment. 15 People living in urban areas are more likely to have acute surgeries/inpatient services or planned surgeries. This pattern may be due to their having fewer geographical barriers in terms of access to health facilities (OR=1.34 for acute and OR=1.37 for planned). There are significant regional differences in utilization for both acute and planned surgeries/inpatient services. However, due to the ambiguity of the definitions for acute and planned cases, there may be a high degree of substitution between acute and planned cases. Table 5 shows the results for Part II and Part III. Part II uses a regression model for number of utilizations among the users while Part III shows results for total expenditures by using a log- linear model. Among the users, there is no significant difference between different quintiles in terms of number of utilizations. For total expenditures, the poorest quintile is slightly more likely to have lower expenditures compared to the top quintile. Other findings from this analysis include: females are more likely to have surgical or inpatient services but incur lower medical expenditures compared to their male counterparts; the employed tend to have lower utilization numbers and incur lower total medical expenditures; urban users incur lower medical expenditures; and there is a significant regional difference in terms of total medical expenditures, with users from Imereti region having the highest total medical expenditures. Since the observations of care seeking behavior are only for six months, a longer follow up is needed to see whether such utilization patterns would persist. Limitations The biggest disadvantage of the data for this analysis is that they do not contain information on the severity of the illness nor the health status of the users. Due to this limitation, the regression can not control for health status, which is a strong determinant for inpatient care or surgery services utilization. As a result, the paper can only assume that the poorest quintile utilizes more services because of their poor initial health status, without being able to formally test this hypothesis. Another limitation is the lack of information on out-of-pocket expenditures. Therefore, it is not possible to assess the impact of health insurance on out-of-pocket spending among the poor who are insured. 7. DISCUSSION This paper estimates the initial impact of the MAP and examines whether the benefits have reached the poorest among the eligible. The findings show that: (1) the impact on utilization of acute services is significant in the neighborhood of the threshold of the program, and (2) the beneficiaries in the poorest quintile are more likely to use services compared to beneficiaries in other quintiles. These two findings suggest that government efforts to improve the poor's access to and utilization of health services are yielding results. Expanding coverage for low income populations through subsidized insurance, sometimes combined with other contributive insurance plans, has been well documented in the literature as an effective way to improve the poor's financial access to health services (Ruiz, Amaya et al. 2007). Georgia's initial experience with the MAP, whereby public funding was used to provide 16 health insurance coverage to the poor, further corroborates this. The MAP design directly addresses the major financial barrier faced by the poor who seek care by covering the costs to this population of the most expensive services (surgeries and inpatient services). Such a scheme mitigates unequal access to and utilization of health services by the poor. The evidence from the Life in Transition Survey, conducted by the World Bank in the Europe and Central Asia Region in late 2006, also shows that Georgia has fewer people complaining about unofficial payments in the public health system than in most of ECA countries.17 The MAP may have reduced informal out-of-pocket payments. The MAP has demonstrated a positive impact by increasing utilization of inpatient services among the poorest. However, this study does not look at the size of the overall impact, the long term impact, or the extent of the financial implications, all of which will require further investigation given the MAP is still at the early stage of implementation and has been in a state of constant flux. Since January 2007, the government has made two major changes in response to the budget constraint caused by the dramatic increase in utilization. First, the government changed the cut- off from 100,000 to 70,000 in order to reduce the number of people eligible18 Second, the government suspended the coverage for planned surgeries until September 2007 several months prior to the election. After September 2007, the government expanded MAP benefits to include a more generous package of outpatient care and has outsourced the MAP in Tbilisi and Imereti regions to private insurances companies as pilots. The government is in the process of providing similar insurance programs to teachers and other groups such as the military, law enforcement personnel and employees of state agencies. However, these changes do not fully address some of the MAP's design limitations. In the following discussion, we focus on four areas for further improvement of the program: eligibility criteria, benefit package design, expansion, and cost control mechanisms. First, MAP eligibility is determined by only one criteria--a cut-off based on the proxy means test score. People with proxy means test scores above 100,000 (now above 70,000) are not eligible for the MAP and therefore are exempt from receiving free care even though they may, in fact, be very poor. This approach may help some poor but it is still far from equitable. Policy makers need to come up with solutions that soften these sharp distinctions and increase the overall equity in terms of access to healthcare services. One option is to introduce a variety of benefit packages, each with a different eligibility cut-off point. The drawback to this, of course, is that it is more administratively burdensome. A detailed discussion on overall targeting performance based on proxy means test is beyond the scope of this study. But generally, targeting based on proxy means test alone invariably suffers from under-coverage and leakage errors. The recent Health Utilization and Expenditure Survey (HUES) shows that the MAP covers about 20% of the population in the poorest quintile while about 30% of MAP beneficiaries are from non-poor quintiles, suggesting that MAP suffers a 17World Bank, "Salient Findings of the Life in Transition Survey, Europe and Central Asia Region," February 2007. 18Except for in the Adjara region, where the regional government budgeted additional monies in order to maintain the cut-off of 100,000. 17 high under-coverage rate as well as a considerable leakage rate compared to other countries using proxy means test for targeting.19,20,21 Despite some issues regarding estimating consumption quintiles under the HUES study,22 the finding suggests the proxy means test did not target the poor well and needs further improvement. One solution is to combine proxy means test targeting with other targeting schemes, such as community targeting. Improving the accuracy of the proxy means test and minimizing leakage errors are essential to MAP targeting and should be a policy priority. Second, the inclusion of outpatient care in the MAP benefits package would reduce incentives for using inpatient care and promote rational use of primary health care, which is important in a culture where people rely heavily on specialist and hospital care. However, the current MAP does not cover outpatient pharmaceuticals, which is one of the major health care costs. Coverage of essential medicines under the MAP would certainly benefit the poor, given that 49% of household expenditures on health are for drugs and medical supplies at retail pharmacies (MoLHSA 2007) Third, the government has been considering MAP's expansion. The cost of the MAP was relatively low as shown by Chanturidze, et al (2007): program expenditures over a six-month period in 2006 amounted to 7.4% of total government expenditures on health.23 The budget for the MAP has increased from 36 million GEL in 2007 to 75 million GEL in 2008, in anticipation of higher utilization rates and medical costs.24 There are two alternatives for expanding the MAP. One is to expand its population coverage, either by lifting the proxy means test score for eligibility so as to include more poor or by including other vulnerable/special population groups such as the elderly, teachers, the police force, the army or veterans. Another possibility is to broaden the benefits covered. Currently, the government has decided to expand similar insurance programs to teachers and other groups such as military and law enforcement personnel. This study indicates that more poor can benefit by raising the cutoff point and expanding MAP eligibility. However, the accuracy of the proxy means test needs to be assessed and improved before such an expansion. For any expansion, simulations and cost assessments are necessary to determine budget implications and financial sustainability as this study shows that the "marginal" impact can be large. Fourth, under the current design, the MAP does not require either a deductible or co-payment, which are cost control instruments to avoid notorious moral-hazard problems (Arrow 1963; Feldstein 1973). Whether or not the program should include cost-sharing schemes for inpatient care is still under debate. During the initial stage when demand tends to be inelastic and the poor are more likely to seek the most necessary care first, cost-sharing is probably not needed. The 19Grosh, M. and J. Baker. 1995, Proxy means test for targeting Social Programs: simulation and speculation." LSMS working paper 118, World Bank, Washington DC. 20Narayan, A, Vishwanath, T, and Yoshida, N, Sri Lanka Welfare Reform. 21Hou X. "Challenges in targeting the bottom ten percent­ Proxy means test for welfare reform in Pakistan". 22The HUES uses the consumption quintiles from the Integrated Household Survey, which have various issues regarding measurement of household consumption.. See the HUES report, December 2007. Ministry of Labor, Health and Social Assistance. December 2007. 23Chanturidze, N. T. Chitashvili, et al (2007) "Evaluation of Medical Assistance Program for Population below the Poverty Line" Ministry of Labor, Health and Social Affairs of Georgia. 24Based on the information authors received from MoLHSA in October 2007. 18 most frequently performed planned surgery under the MAP is for cataracts. This is understandable; the poor are more likely to take advantage of the MAP benefit to have this surgery, which had been postponed prior to the program for financial reasons because cataracts are not life-threatening. However, the continuous provision of benefits and inclusion of outpatient care would reduce a surge in demand for such surgeries as was observed at the beginning of the program. Cost-sharing may, however, need to be introduced once the program expands to cover more people, including those who are less poor. Further refinement of the program's design will require good information and sound evidence. Thus, careful monitoring and constant evaluation of the program is needed to make wise policy decisions. In conclusion, the results from this analysis using claim data show that the MAP has reached the poorest and achieved its initial objective of increasing utilization of surgical and inpatient services by the poor. However, due to data limitations, the full impact of the MAP cannot be assessed under this study. It is clear that the program design needs further improvement to better reach the targeted population and enhance program sustainability. Given the dynamic state of healthcare reform in Georgia, regular monitoring and evaluation are needed to ensure a good understanding of program performance and improve its success. Towards this end, new data have been requested and large surveys are planned to continue monitoring the outcomes of the program. 19 REFERENCE Abel-Smith, B. and P. Rawal (1992). "Can the poor afford 'free' health services? A case study of Tanzania." Health Policy and Planning 7(4): 329-341. Arrow, K. (1963). "Uncertainty and the Welfare Economics of Medical Care." The American Economic Review 53(5): 941-973. Belli, P., G. Gotsadze, et al. (2004). "Out-of-pocket and informal payments in health sector: evidence from Georgia." Health Policy 70: 109-123. Both, J. (2002). Health care financing strategy in Georgia 2002-2007. Tbilisi, Georgia, Technical Report, 2002.057 Leiden, Netherlands: TNO Prevention and Health. Campbell, D. T. (1969). "Reforms as Experiments." American Psychologist 24: 409-429. Chanturidze, N., T. Chitashvili, et al. (2007). Evaluation of Medical Assistance Program for Population below the Poverty Line, MoLHSA, Tbilisil, Georgia. Deininger, K. and P. Mpuga (2003). "Economic and welfare impact of the abolition of health user fees: evidence from Uganda." Policy Research Working Papers, the World Bank. Duan, N. and L. K. Chau (1987). "Distribution-free and link-free estimation for the sample selection model." Journal of Econometrics 35: 25-35. Dulitzky, D. and X. Hou (2007). Health. Fiscal Policy and Economic Growth, Lessons from Eastern Europe and Central Asia. C. Gray, T. Lane and A. Varoudakis. Washington DC, the World Bank Group. European Observatory on Health Care Systems (2002). Health care systems in transition: Georgia. Copenhagen, Denmark, WHO Regional Office for Europe. Feldstein, M. S. (1973). "The welfare loss of Excess Health Insurance." Journal of Political Economy 81: 250-280. Gotsadze, G. e. a. (2001). Health Service Utilization and Expenditure in Tbilisi - 2000, Report of Household Survey. Tbilisi, Georgia, Curatio International Foundation. Hahn, J., P. Todd, et al. (2001). "Identification and Estimation of Treatment Effects with a Regression-Discontinuity Design." Econometrica 69: 201-209. Harmon, C. and B. Nolan (2001). "Health insurance and health services utilization in Ireland." Health Econominics 10: 135-145. 20 Hofer, R. (2006). "Private health insurance and utilization of health services in Chile." Applied economics 38: 423-39. Holly, A., L. Gardiol, et al. (1998). "An econometric model of health care utilization and health insurance in Switzerland." European Economic Review 42: 513-22. Manning, W., J. Newhouse, et al. (1987). "Health insurance and the demand for medical care: evidence from a randomized experiment." The American Economic Review 77(3): 251-277. MoLHSA (2007). Household Survey on Health Service Utilization and Expenditure Draft Report. Georgia Primary Health Care Development Project, MoLHSA, Department of Statistics, MoED, Oxford Policy Management, UK and Curatio International, Tblisi, Georgia. Newhouse, J. and The Insurance Experiment Group (1996). Free for all? Lessons from the RAND health insurance experiment. Cambridge, MA. Ruiz, F., L. Amaya, et al. (2007). "Progressive Segmented Health Insurance: Colombian Health Reform and Access to Health Services." Health Economics 16: 3-18. Trujillo, A. J., J. E. Portillo, et al. (2005). "The Impact of Subsidized Health Insurance for the Poor: Evaluating the Colombian Experience Using Propensity Score Matching." International Journal of Health Care Finance and Economics 5(3): 211-239. Wagstaff, A. (2007). "Health Insurance for the Poor: Initial Impacts of Vietnam's Health Care Fund for the Poor." The World Bank, Policy Research Work Paper Series: 4134. Yates, R. (2007). "The Impact of Abolishing User Fees in Africa--Recent Developments in Six African Countries." Social Science Research Network, Working Paper Series, The World Bank. 21 Table1. The Impact of MAP on Acute Surgeries/Inpatient Care Utilization Control, Mean Treatment, (SD) Mean(SD) Parametric Estimation (6) (1) (2) (3) (4) (5) (7) OLS Marginal Impact (N=14133) (N=15327) OLS (Quartic) OLS (X) probit Size Acute Surgeries/Inpatient 0.001 0.01 0.009 0.011 0.009 0.009 9 Care (0.031) (0.098) (0.001)** (0.002)** (0.001)** (0.001)** Note: Column 3 to Column 6 present parametric estimations of the results. Column 3 is an OLS model with h a dummy variable defined equal to 1 if beneficiary; Column 4 is an OLS with the dummy variable and a fourth order polynomial in welfare scores; Column 5 is an OLS with the dummy and other covariates to control for additional differences between the beneficiaries right below the threshold and non-beneficiaries right above the threshold; and Column 6 is a Probit model with marginal probability presented. The last column presented the impact size based on the first OLS estimation. 22 Table2:Characteristics of beneficiaries by welfare quintiles Variable Name Quintile 1 Quintile 2 Quintile 3 Quintile 4 Quintile 5 Age (%) 18-30 15.6 17.3 19.1 20.2 20.9 30_40 14.0 14.9 15.6 16.2 16.2 40_50 14.4 15.1 16.7 18.0 19.2 50_60 12.0 13.0 14.0 15.5 16.8 60_70 18.5 17.7 16.4 14.9 13.8 70-80 23.8 20.2 16.3 13.1 11.0 Female (%) 57.1 57.0 56.4 56.4 56.3 Education (%) no formal education 3.2 1.7 1.3 1.0 0.8 Primary 25.6 22.9 19.5 16.4 13.7 Secondary 61.1 63.5 65.0 65.4 63.6 high school or equivalent 2.1 2.5 3.2 3.9 5.0 above high school 8.0 9.4 11.0 13.3 16.9 Education of Head no formal education 2.3 1.5 1.4 1.2 1.3 Primary 29.3 27.8 24.7 22.0 19.2 Secondary 60.4 61.8 64.0 65.0 64.6 high school or equivalent 0.6 0.6 0.7 0.8 0.9 above high school 7.3 8.3 9.3 11.0 14.0 Employment Status 29.6 46.7 53.1 56.6 58.7 Household size (SD) 3.6(2.1) 3.8(2.0) 4.0(2.0) 4.1(1.9) 4.2(1.8) Relationship to head head self 45.3 42.5 39.4 37.4 35.9 Spouse 20.3 20.9 20.7 20.1 19.7 Children 27.8 30.1 33.1 35.5 37.0 Other 6.6 6.5 6.8 7.1 7.4 Urban 41.8 29.0 27.5 29.5 34.1 Region Tbilisi 8.2 9.2 9.3 9.6 12.5 Guria 4.8 5.4 4.7 4.4 3.5 Racha-Lechkhumi, qvemo svaneTi 4.1 3.4 2.4 1.9 1.5 Kakheti 11.3 12.4 13.3 12.6 12.2 Imereti 20.6 19.6 18.4 18.8 17.9 Mckheta- Mtianeti 4.6 4.0 4.1 4.0 3.7 Samegrelo 21.1 14.6 11.7 10.2 8.2 samtckhe-Javakheti 2.4 3.6 4.8 6.0 8.1 Qvemo Qartli 5.5 7.7 10.9 12.6 13.8 Sida Qartli 8.9 9.1 8.8 9.0 8.8 Achara 8.1 10.8 11.6 10.9 9.8 Note1: limited to ages between 18 to 80; 2. total number of beneficiaries is 738,342; 3. F-test shows significant difference between groups, with p<0.001 for all variables listed. 23 Table3: utilization of services by welfare quintiles Variable Name Quintile 1 Quintile 2 Quintile 3 Quintile 4 Quintile 5 Prob>F # acute surgery/ inpatient 3139 2559 2096 1787 1596 <0.001 # planned surgery/inpatient 884 741 769 656 680 <0.001 # delivery 285 304 331 307 308 0.4736 Mean of number of utilization (SD) 1.37(1.04) 1.41(1.06) 1.40(1.12) 1.40(1.15) 1.41(1.18) 0.4247 Mean of total expenditures, GEL (SD) 588(985) 558(871) 606(961) 613(1025) 598(883) 0.1535 24 Table4. Part I. The Logit Model of Services Utilization Acute Surgeries/inpatient planned surgeries Delivery Welfare (reference= quintile 5) Quintile 1 1.47 (1.38 , 1.57)** 1.13 (1.02 , 1.26)* 1.04 (0.87 , 1.24) Quintile 2 1.36 (1.28 , 1.45)** 1.04 (0.93 , 1.15) 1.03 (0.87 , 1.22) Quintile 3 1.21 (1.13 , 1.29)** 1.14 (1.03 , 1.27)* 1.08 (0.92 , 1.27) Quintile 4 1.08 (1.01 , 1.16)* 1 (0.89 , 1.11) 0.96 (0.82 , 1.13) Age (reference=30-40 years old) 18-30 0.82 (0.75 , 0.89)** 0.83 (0.72 , 0.95)** 3.4 (2.99 , 3.88)** 40-50 1.27 (1.18 , 1.38)** 1.24 (1.09 , 1.41)** 0.14 (0.11 , 0.20)** 50-60 1.83 (1.70 , 1.98)** 1.34 (1.17 , 1.53)** 60-70 1.83 (1.69 , 1.98)** 1.55 (1.36 , 1.77)** 70-80 1.76 (1.62 , 1.91)** 1.67 (1.46 , 1.91)** Female (=1 if female) 0.88 (0.85 , 0.92)** 0.85 (0.79 , 0.91)** Education (reference: no formal education) Primary 1.28 (1.04 , 1.57)* 1.17 (0.78 , 1.75) 2.62 (1.23 , 5.58)* Secondary 1.28 (1.04 , 1.57)* 1.36 (0.92 , 2.02) 2.56 (1.21 , 5.42)* high school or equivalent 1.03 (0.80 , 1.33) 1.46 (0.94 , 2.26) 1.89 (0.87 , 4.11) Above high school 1.23 (0.99 , 1.53) 1.35 (0.90 , 2.03) 2.84 (1.32 , 6.09)** Education of head (reference: no formal education) Primary 1.18 (0.95 , 1.47) 1.49 (0.94 , 2.35) 1.06 (0.66 , 1.69) Secondary 1.16 (0.94 , 1.45) 1.58 (1.01 , 2.49)* 0.83 (0.52 , 1.33) high school or equivalent 1.55 (1.14 , 2.11)** 1.66 (0.94 , 2.94) 0.76 (0.36 , 1.60) Above high school 1.1 (0.88 , 1.39) 1.8 (1.13 , 2.87)* 0.65 (0.39 , 1.07) Employment (=1 if employed) 0.74 (0.71 , 0.77)** 0.72 (0.66 , 0.77)** 0.69 (0.61 , 0.77)** Household size 0.97 (0.96 , 0.98)** 0.98 (0.96 , 1.00)* 1.03 (1.00 , 1.06) Relationship to head (reference=head) Wife 0.96 (0.91 , 1.02) 0.99 (0.90 , 1.09) 1.95 (1.41 , 2.69)** Children 0.74 (0.69 , 0.80)** 0.89 (0.80 , 1.00)* 1.36 (0.99 , 1.88) Other 0.78 (0.71 , 0.86)** 0.8 (0.69 , 0.93)** 0.67 (0.46 , 0.98)* Urban (=1 if urban) 1.34 (1.28 , 1.40)** 1.37 (1.26 , 1.50)** 0.85 (0.74 , 0.98)* Region (reference = Imereti ) Tbilisi 1.2 (1.11 , 1.29)** 2.3 (2.07 , 2.55)** 0.98 (0.77 , 1.25) Guria 0.92 (0.82 , 1.02) 1.25 (1.07 , 1.46)** 1.1 (0.83 , 1.45) Racha-Lechkhumi, qvemosvaneti 1.62 (1.45 , 1.80)** 0.41 (0.30 , 0.56)** 0.78 (0.48 , 1.24) Kakheti 1.21 (1.13 , 1.30)** 0.81 (0.71 , 0.92)** 0.82 (0.66 , 1.02) Mckheta-Mtianeti 1.43 (1.30 , 1.57)** 1.4 (1.19 , 1.63)** 1.1 (0.83 , 1.47) Samegrelo 1.25 (1.17 , 1.33)** 0.7 (0.61 , 0.79)** 0.97 (0.79 , 1.18) Qvemo Qartli 0.8 (0.73 , 0.87)** 0.48 (0.41 , 0.57)** 1.43 (1.18 , 1.72)** Sida Qartli 1.41 (1.31 , 1.51)** 0.59 (0.51 , 0.70)** 1.08 (0.87 , 1.34) Achara 1.66 (1.54 , 1.78)** 0.8 (0.69 , 0.92)** 1.43 (1.19 , 1.71)** Observations 701,307 701,307 193,517 95% confidence intervals in parentheses * significant at 5%; ** significant at 1% 25 Table 5. PARTII. Regression of Number of Utilizations and Log of Total Expenditures PARTII, Number of PARTIII. LOG (Total Utilization Expenditures) Welfare (reference= quintile 5) Quintile 1 0 (-0.06, 0.06) -0.07 (-0.13, -0.00)* Quintile 2 0.03 (-0.03, 0.09) -0.08 (-0.15, -0.02)* Quintile 3 0.01 (-0.04, 0.07) -0.03 (-0.10, 0.03) Quintile 4 0.01 (-0.05, 0.07) -0.04 (-0.11, 0.02) Age (reference=30-40 years old) 18-30 -0.05 (-0.12, 0.02) -0.05 (-0.13, 0.03) 40-50 0.21 (0.15 , 0.28)** 0.03 (-0.05, 0.11) 50-60 0.32 (0.25 , 0.39)** -0.01 (-0.09, 0.07) 60-70 0.18 (0.11 , 0.25)** 0.04 (-0.04, 0.12) 70-80 0.06 (-0.01, 0.13) -0.04 (-0.12, 0.04) Female (=1 if female) 0.05 (0.02 , 0.09)** -0.24 (-0.28, -0.19)** Education (reference: no formal education) Primary 0.08 (-0.12, 0.28) -0.18 (-0.41, 0.05) Secondary 0.16 (-0.03, 0.36) -0.1 (-0.32, 0.12) high school or equivalent 0.08 (-0.15, 0.31) -0.16 (-0.41, 0.10) above high school 0.26 (0.05 , 0.46)* -0.08 (-0.32, 0.15) Education of head (reference: no formal education) Primary 0.08 (-0.13, 0.29) 0.05 (-0.18, 0.29) Secondary 0.06 (-0.15, 0.26) 0.06 (-0.18, 0.29) high school or equivalent 0.15 (-0.13, 0.43) 0.11 (-0.21, 0.42) above high school 0.05 (-0.17, 0.26) 0.01 (-0.24, 0.25) Employment (=1 if employed) -0.07 (-0.11, -0.03)** -0.09 (-0.13, -0.04)** Household size -0.01 (-0.02, 0.00) 0 (-0.01, 0.01) Relationship to head (reference=head) Wife 0.02 (-0.03, 0.07) -0.02 (-0.07, 0.04) Children 0.01 (-0.05, 0.07) 0.04 (-0.02, 0.11) Other -0.01 (-0.09, 0.07) 0 (-0.09, 0.10) Urban (=1 if urban) -0.03 (-0.07, 0.01) -0.05 (-0.10, -0.01)* Region (reference = Imereti ) Tbilisi 0.27 (0.21 , 0.34)** -0.07 (-0.14, 0.00) Guria 0.03 (-0.06, 0.12) -0.27 (-0.38, -0.17)** Racha-Lechkhumi, qvemosvaneti -0.01 (-0.11, 0.10) -0.22 (-0.35, -0.10)** Kakheti 0.16 (0.09 , 0.22)** -0.26 (-0.34, -0.19)** Mckheta-Mtianeti 0.16 (0.08 , 0.25)** -0.1 (-0.20, 0.00) Samegrelo 0.05 (-0.01, 0.11) -0.21 (-0.28, -0.14)** Qvemo Qartli 0.19 (0.11 , 0.26)** -0.14 (-0.23, -0.05)** Sida Qartli 0.22 (0.15 , 0.29)** -0.41 (-0.49, -0.33)** Achara 0.07 (0.00 , 0.14)* -0.18 (-0.26, -0.11)** Observations 16,434 16,423 95% confidence intervals in parentheses * significant at 5%; ** significant at 1% 26 APPENDIX 1. Test for Quasi-Random Assignment It is important to note that our empirical strategy crucially relies on the assumptions that applicants close to the cut-off score from both directions are randomly assigned to beneficiaries and non-beneficiaries. Thus, all the pre-determined characteristics should not be statistically different between beneficiaries and non-beneficiaries with the scores right above and right below the cut-off score. If this assumption does not hold, our estimates are likely to be biased. Intuitively, this assumption seems to make sense. Statistically, we tested pre-determined characteristics, which should be the same on both sides of the cut off score of 100,000, on average, and found that, as we compare closer and closer to the cut-off score, beneficiaries and non-beneficiaries have similar observable characteristics. This is graphically shown in Appendix Figure 1 and Figure 2, which plot the selected average individual characteristics against beneficiaries and non-beneficiaries. Figure 1 plots personal characteristics, including age, sex, education, education of head and household size. Figure 2 plots the geographical characteristics, including all the regions and the urban/rural status. Generally, the figures indicate that the difference at the cut-off points is small and statistically insignificant. Appendix Table 1 illustrates the same points by quantifying the difference between beneficiaries and non-beneficiaries for a larger set of characteristics. In particular, we examined all the characteristics shown in Appendix Figures 1 and 2, as well as the age categories, education categories and employment status. Column1 includes a sample with scores from 90,000 to 110,000; column 2 includes a sample with scores from 92,000 to 108,000; column3 includes a sample with scores from 96,000 to 104,000, and the last column includes a sample with scores from 98,000 to 102,000. As the sample gets closer and closer to the cut-off point, the differences between beneficiaries and non-beneficiaries get smaller and less significant. Consider, for example, education in years. The difference in education level is large between beneficiaries and non-beneficiaries in the sample with scores ranging from 90,000 to 100,000, with beneficiaries having lower education achievement. However, as the sample is restricted closer and closer to 100,000, the education difference decreases. In the sample with scores ranging from 98,000 to 102,000, 28 variables are not significantly different between the beneficiaries and non- beneficiaries among 31 observed variables. Overall, the evidence strongly supports a valid regression discontinuity design. And as a consequence, it appears that among applicants who obtained a score close to the cut-off point, the process of who obtained a score right above the cut-off point or right below the cut-off point, seems to be a virtually randomized process. This is our identification assumption. 27 Appendix Figure 1. Similarity of Characteristics of Beneficiaries and Non-Beneficiaries in the Neighborhood of Threshold (1) Note: Rows from top left to right are age, sex, education, education of head and household size. Circles represent the average characteristics within intervals of 500 in welfare score. The continuous lines are fitted values from 4th order polynomial regressions on either side of the discontinuity. Dotted lines are 95% confidence intervals. 28 Appendix Figure 2. Similarity of Characteristics of Beneficiaries and Non-beneficiaries in the Neighborhood of Threshold (2) Note: Rows from top left to right are regions of residence from region 1 to region 11 and the last chart is for urban/rural status. Circles represent the average characteristics within intervals of 500 in welfare score. The continuous lines are fitted values from 4th order polynomial regressions on either side of the discontinuity. Dotted lines are 95% confidence intervals. 29 Appendix Table1. Difference in Individual Characteristics between Beneficiaries and Non-Beneficiaries (1) (2) (3) (4) 90,000<=score 92,000<=score 96,000<=score 98,000<=score <=110,000 <=108,000 <=104,000 <=102,000 Age in 2007 0.345 0.341 -0.001 -0.001 (0.117)** (0.131)** (0.002) (0.003) Age 0-5 -0.001 0.000 0.000 -0.007 (0.001) (0.001) (0.003) (0.004) Age 5-10 -0.001 0.001 -0.002 -0.002 (0.001) (0.002) (0.003) (0.004) Age 10-20 0.001 -0.005 -0.003 0.004 (0.002) (0.002)** (0.003) (0.004) Age 20-30 -0.006 0.001 0.003 0.001 (0.002)** (0.002) (0.003) (0.004) Age 30-40 0.000 0.000 0.000 0.002 (0.002) (0.002) (0.003) (0.004) Age 40-50 -0.002 -0.002 0.002 0.002 (0.002) (0.002) (0.003) (0.003) Age 50-60 -0.001 0.001 0.002 0.003 (0.002) (0.002) (0.003) (0.006) Age 60-70 0.003 0.006 0.000 -0.009 (0.002) (0.002)** (0.004) (0.022) Age 70 + 0.006 0.002 -0.032 0.000 (0.002)** (0.003) (0.016)* (0.001) Female -0.001 -0.088 0.001 0.007 (0.003) (0.011)** (0.001) (0.004) Education (years) -0.118 0.001 0.008 -0.007 (0.010)** (0.000)* (0.003)** (0.006) no formal education 0.001 0.010 -0.003 0.001 (0.000)* (0.002)** (0.004) (0.002) Primary edu. 0.011 0.003 0.002 0.000 (0.002)** (0.003) (0.002) (0.004) Secondary edu. 0.009 -0.002 -0.004 -0.012 (0.003)** (0.001) (0.003) (0.019) High school or equivalent -0.003 -0.011 -0.046 -0.001 (0.001)** (0.002)** (0.013)** (0.001) above high school -0.016 -0.076 -0.001 0.005 (0.002)** (0.009)** (0.001) (0.004) Education of head (in -0.090 0.001 0.011 -0.002 years) (0.008)** (0.001) (0.003)** (0.006) Employment -0.008 -0.041 -0.011 0.019 (0.003)** (0.011)** (0.004)** (0.004)** Household size -0.037 -0.019 0.003 -0.001 (0.010)** (0.003)** (0.003) (0.002) Urban -0.027 -0.010 0.000 -0.003 (0.003)** (0.002)** (0.001) (0.001) Region1 -0.015 0.001 -0.002 -0.003 30 (0.002)** (0.001) (0.001) (0.004) Region2 0.003 0.000 0.003 -0.012 (0.001)** (0.001) (0.003) (0.004)** Region3 0.000 0.005 -0.009 -0.007 (0.001) (0.002)* (0.003)** (0.002)** Region4 0.005 0.001 -0.002 0.005 (0.002)** (0.002) (0.002) (0.003) Region5 0.005 0.000 0.004 -0.006 (0.002)* (0.001) (0.002)* (0.004) Region6 0.001 0.009 -0.010 0.001 (0.001) (0.001)** (0.002)** (0.004) Region7 0.011 -0.013 -0.001 0.001 (0.001)** (0.002)** (0.003) (0.003) Region8 -0.015 -0.004 0.005 0.005 (0.002)** (0.002)* (0.002)* (0.003) Region9 -0.004 0.004 0.008 0.000 (0.002)* (0.002)** (0.002)** 0.000 Region10 0.003 0.008 0.000 0.000 (0.001)* (0.002)** 0.000 0.000 Region11 0.007 0.000 0.000 0.000 (0.002)** 0.000 0.000 0.000 Number 145,795 116,605 58,118 29,457 Note: 1. Regions 1 through 11 are as follows: Tbilisi, Guria, Racha-Lechkhumi, Kakheti, Imereti, Mckheta-Mtianeti, Samegrelo, Samtckhe-Javakheti, Qvemo Qartli, Sida Qartli, , Achara. 2. Standard errors are in parenthesis. 31