Exercise 6: Costing an additional year of preschool Background Currently, the Government provides only one year of preprimary education for children who are 5 years old. The Government wants to test the impact of an intervention that extends preschool education to children 4 years of age. It is also interested in whether this is a cost-effective method for increasing children’s school readiness and whether the ultimate benefits of the program would outweigh the costs. The pilot program will be developed and implemented by a non-profit in one province, along with representatives from the Ministry of Women and Children Affairs (MoWCA), and Directorate of Primary Education (DPE) under the Ministry of Primary and Mass Education (MoPME), although the longer-term objective is for the Government to adopt the program if it proves to be successful. The program uses the existing preprimary infrastructure, resources and teachers, but different curriculum and materials. The 4 year old children will simply attend a second shift at current preschools, thus leveraging the infrastructure and resources that the government is already investing in preprimary education. Thus, teachers simply devote extra time in the afternoon or morning to conduct a new class for 4 year olds. The intervention structure is similar to that of the existing preprimary program: Children meet every day for 2 hours. The child-to-teacher ratio is about 15 to 20 children for each teacher. Existing preprimary teachers who are selected from their respective communities will serve as the teachers for the new classes of 4 year old children. To measure impact, an impact evaluation has been designed. Preprimary centers will be randomly assigned to a treatment group (two shifts, each catering to 4 year old children or 5 year old children) and a control group (one shift only for 5 year olds). The evaluation will run for two years. The main outcome of interest is children’s school readiness immediately before they enter primary school. To recap the key elements of the program: • Objective: Provide 1-extra year of preschool (2 years instead of existing 1 year) to 4 year old children. • Timeline: Impact and costs will be measured over 2 years, and 2 separate cohorts of children. • Existing infrastructure: The impact evaluation will use existing schools and teachers, utilizing “shifts.� 1. Setting up a timeline for costing activities. The following graphic presents the timeline for the pilot intervention and the impact evaluation. For each costing activity (i) – (vi) listed below, discuss where on the timeline they would ideally happen (e.g. “before B� or “between B and F�) and insert your responses in the last column. (C) (G) (A) Start of (E) End of (I) Baseline school Midline School Findings survey year 1 survey year 2 published (B) (D) (F) (H) Pilot End of Start of Endline intervention school school survey begins year 1 year 2 Activity Description Timing (i) Take stock of budgets Ascertain all relevant budget Before B. and financial material and discuss data sources with program team (ii) Ex-ante analysis Assess the state of all existing Before B. financial data about the project, and whether the structure of this data can answer all important questions about the cost structure of the program (i) Data collection plan: Determine what data needs to Before B. be collected in real-time to ensure accuracy of cost projection (iv) Finalize cost estimate: Fill in cost model with final Right after G. actual expenditure data, as well as time, effort, quantity and price data; Generate final unit cost estimates (v) Generate cost-effectiveness ratio Calculate final cost metrics for Between G and I. policy makers (vi) Generate cost-benefit ratio Calculate final cost metrics for Between G and I. policy makers 2. The ingredients method. The ingredients method is an investigation that uncovers, quantifies, and values all resources and efforts required to make an intervention happen. Using budgets, financial documents, procurement logs, interviews with project management, and direct observations, the ingredients method involves the following steps. Step 1 List ingredients. List all ingredients required to make the intervention a reality. Step 2 Categorize. Consider what category these ingredients fall under and tag them appropriately (personnel, trianing, equipment and capital assets, supplies, travel, management/overhead, transfer payments, etc..) Step 3 Value ingredients: quantities and prices. List the quantity of the ingredient required for the intervention, as well as the price of each unit of that ingredient. The nature of the unit (example: teacher salary months) should be specified. Step 4 Value ingredients: timing. Appropriate quantities and prices should be applied for each year that the program is implemented. Step 5 Value ingredients: allocation of shared costs to intervention. For any costs that are shared between multiple interventions or activities, determine the appropriate allocation percentage to the intervention. This may require significant investigative effort and interviews with staff. Step 6 Finalize total value of ingredient per year. To get a total cost of each ingredient in each year, muiltiply values together (ex: Quantity year 1 X Price year 1 X Allocation percentage for year 1 = Total cost of ingredient in year 1). An example of a completely disaggregated ingredient is presented in Table 1. Table 1 The challenge you will face with completing the ingredients method is that financial information usually comes in aggregate form. (a) Suppose you receive this aggregate budget for preschools. Is this data sufficient? What information is missing from this? For each aggregate category, discuss some additional information you would like to know to properly cost this program. Budget category Cost School personnel 1,526,545 Management labor costs 1,432,412 School maintenance 356,541 Pre-service training 532,557 In-service training 239,761 Materials 424,534 Equipment 283,449 Travel 43,211 This data is not sufficient to do a cost-analysis. Some questions about each category could include: • School Personnel: Does this include all school personnel or just personnel working on the intervention? Is this the amount of their full salary or just the appropriate portion of their salary allocated based on time and effort on the classes for 4 year olds? Is this gross or net salaries? How many teachers work on the extra-year of preschool program? • Management Labor Costs: How is this aggregate sum calculated? Is it the total management costs of a field office? Does it take into account the specific management personnel required for this intervention, or does it simply take aggregate salaries of program staff and neglect to allocate their time and effort? • School Maintenance: Is this the total cost of maintaining the schools? Does it allocate appropriately based on the extra-year of primary education’s use of school buildings? Does the intervention actually increase maintenance costs? What does school maintenance entail (utilities, cleaning?)? • Pre-service Training: What inputs were required to make the pre-service training happen. Does this aggregate budget line include other trainings being performed by the NGO under the same funding source (perhaps other 1-year pre-primary training)? How many teachers took part in the training? What would it take to replicate this activity? Did this training happen only in one year or in multiple years? • In-Service Training: Same questions for pre-service training. • Materials: What materials were required to make this intervention happen? Are these all the materials for 1 and 2-year preprimary students or just 2-year preprimary students? Were material costs the same in year 1 as in year 2? • Equipment: What specific equipment was required for this intervention? Was it split between other activities? Was it purchased for this intervention, or already owned and then used for the intervention? At what intensity was it used for this intervention, and how quickly does it depreciate / wear out? Were there equipment maintenance costs associated with this category? • Travel: Where do these travel expenses come from? Were they incurred at a specific point in time? Are they fixed or variable costs? Shared with other activities? What vehicles/contractors/tickets pruchased? Do costs vary with distances? (b) A template is provided in Table 2 that lists disaggregated ingredients. There is only one ingredient listed for each of the budget categories (When the cost model is complete, there will be many ingredients for each category, but for now for simplicity, we are just providing one example for each cost category). One ingredient (lunches for participant, in-service training) is filled out already. For each ingredient, please fill in the allocation percentage in the column labeled “Allocation Percentage.� The allocation percentages should not be imaginary; they should correspond to what you know about the program. In addition to the information already discussed in the project description, use the following assumptions about shared costs: 1. Management personnel split their time approximately evenly between 5 projects 2. The preservice and in-service trainings are exclusive to the 2-year preprimary program. 3. Other costs incurred at the school, such as maintenance, are for the entire school, not one activity at the school. The school provides equal services to 1 year preprimary, 2 year preprimary, and primary school classrooms. 4. Materials in this budget are used for both the 1 year and 2-year preprimary program 5. Equipment is used for both the 1 year and 2-year preprimary program. Table 2 Allocation percentage Totals Teachers salaries 50% 1,008,000 Senior Field Officer 20% 32,400 (gross salary) Utilities 33% 68,310 Per diem for participants (5 day 100% 50,000 training) Lunches for participants 100% 18,000 Sheet paper 50% 27,000 Laptops 50% 2,500 Fuel Costs 50% 2,400 (c) Now consider that this project’s costs are incurred over two years, not one. The template in Table 3 has columns for both years for two ingredients, teacher salaries and senior field officer salaries. Assuming that values you listed in question three are correct for the whole two year project, now allocate those values over both years by filling in the columns labeled “Number of units� for 2016 and 2017. Keep in mind, the “units� for teachers’ salaries is total “teacher months� (in the first table, this is a total months worked for the WHOLE project. For Senior field Officers, the nature of the unit cost is simply “personnel,� so the units tell you how many senior field officers are employed at any given time. This “nature of unit cost� will affect what units you fill in for both years. Table 3 • The number of units for Teacher months in table 2 should be split between the two years because the nature of the unit costs suggests that it was recorded as an aggregate for both years in table 2. • The senior field officer (gross salary) units should remain the same as in table 2 in both year columns. Since the number of units column records the number of personnel, not personnel effort like with the teachers, the number of personnel would stay the same presumably in year 2 as in year 1. (d) For each of the eight ingredients in Table 4, discuss where you think you might find the information to value that ingredient (quantities, prices, allocation percentages, all broken down by year). Think of your own experience with your own department’s or organization’s information systems and imagine how you might get the necessary data. Table 4 Ingredients Possible data sources for quantities, prices, and allocation percentages Teachers’ Salaries Senior field officer (gross salary) Utilities Per Diem for participants (5 day training) Lunches for participants Sheet paper Laptops Fuel costs Fairly open ended question. Answers for some ingredients could include: • Budgets • Actual expenditure reports • HR reports (for personnel salaries) • Implementation documents • Interviews with managers, procurement staff, finance personnel • Interviews with frontline personnel • Direct observation of the intervention 3. Calculating total program costs. Consider the aggregate budget provided earlier. In Table 5, calculate the implied unit cost for delivering an additional year of preprimary education to one child if we were to use just this data. Assume there are 1,750 beneficiary children PER YEAR, and therefore 3500 children total that receive the intervention. Given that yearly data is missing, you’ll just have to assume costs are equal across years (which, though inaccurate, is your only option with aggregate data). Table 5 Budget category Cost School personnel 1,526,545 Management labor costs 1,432,412 School maintenance 356,541 Pre-service training 532,557 In-service training 239,761 Materials 424,534 Equipment 283,449 Travel 43,211 Total 4,839,010 Beneficiaries (Total over 2 years) 3500 Unit cost to provide extra year of preprimary 1382.6 education to 1 child Now in Table 6, look at the data in disaggregated form generated by the ingredients method. In Table 7, fill in the implied the unit costs for year 1 and year 2 and again assume there are 1750 beneficiary children PER YEAR receiving an additional year of preschool. Remember, every year a new cohort of children receive an extra year of preschool. How big of a difference is this from the unit cost generated with aggregate data? What are a few reasons for this difference? Table 6 Table 7 Total Cost Beneficiaries Unit Cost Year 1 1,320,674 1750 755 per student per year Year 2 1,228,719 1750 702 / student / year The difference between the unit cost is 627.6 in year 1 and 680.6 in year 2. This could be because the aggregate budget was recording aggregate costs for the whole school, and not allocating between activities. Management costs may have also remained totals for managers salaries prior to allocating their time and effort between 5 different projects they were working on. Numerous other reasons for innaccuracies in aggregate numbers also exist (see answers to question 2). 4. Cost-effectiveness analysis. Now you are ready to combine cost and impact and interpret your findings. You learn that the Government funded another impact evaluation among similar communities of a program that targeted parents of children aged 3-6 years and taught them how to do early stimulation activities in the home and appropriate behavior management to increase school readiness. You have the following data on impacts of the two programs. ⎔ The impact evaluation for the extra year of preprimary suggests that the program increases school readiness scores by an average of 0.6 standard deviations (SD). ⎔ The impact evaluation of the pilot program that targeted parents suggests that the program increases school readiness scores by an average of 0.3 SD. The cost data gives you the following unit costs: ⎔ Extra year of preschool: $750 per student ⎔ Parenting Program: $300 per student What are the cost-effectiveness ratios for the two programs (cost / impact)? Which one is more cost effective? Cost effectiveness ratio for extra year or preschool: 1250 for 1 SD change in test scores. Cost Effectiveness ratio for parenting program: 1000 for 1 SD change in test scores. The parenting program would appear more cost-effective in this scenario. 5. Cost-benefit analysis. Recall that the Government also wants to know whether the total benefits of the extra year of preschool outweigh the total costs. You’ve been provided with information from a longitudinal study that found for every 0.1 SD increase in school readiness, there is an associated half a year of additional schooling. You also find that each additional year of schooling is associated with an average increase in earnings of 10 percent. Recall that we need to calculate a net present value (see formula below) of benefits because some of those happen in the future (which we discount) whereas the costs are all happening today (t = 1 for costs). Using this information, you calculate that the net present value of a 0.1 SD change in school readiness scores using a 3% interest rate is equal to $4,432. Fill in the Net Present Value of the program in the second to last column in Table 8. Which program now looks most attractive? Is it different than what you found in your cost-effectiveness analysis? Now fill in the Benefit/Cost ratio in the last column. Does your answer change? Table 8 Net Present Benefit Net Present Cost of Net Present Benefit cost of Program per Program per child (-) Value of Program Ratio (benefits / child (+) (not discounted, costs costs) incurred in base year) Extra year of $26,922 $750 $26,172 $35.89 preprimary Parenting $13,296 $300 $12,996 $44.32 program When you look at the Net Present Value of the program using a cost-benefit analysis, the extra year of preprimary program appears more effective at improving outcomes over the course of a beneficiaries life. NPV gives you the total value of the program over a beneficiaries lifespan. If you look at the benefit/cost ratio, the parenting program looks more effective. This records how many dollars in benefits per beneficiary you obtain for every dollar spent. Though the parenting program has a higher benefit to cost ratio, the extra year of primary school has a higher total threshold for the value it can provide a participant over the course of their life. This may lead policy makers to choose the preprimary program over the parenting program. It provides more total benefit to a beneficiary, though it is less cost-effective.