Education and Technology Series Volume 1 0 Number 3 l 1996 Cost Analysis of Information Technology Projects in Education: Experiences from Developing Countries by Michael Potashnik Douglas Adkins A publication of the World Bank Human Development Department Education Group - Education and Technology Team COST ANALYSIS OF INFORMATION TECHNOLOGY PROJECTS IN EDUCATION: EXPERIENCES FROM DEVELOPING COUNTRIES by Michael Potashnik, Douglas Adkins I. Introduction ized countries launched major programs for introducing computers into schools. In the OECD countries, there was both experimenta- 1. The New Information Age. Revolutionary advances in in- tion and controversy over the use of computers (Vickers and formation technology and communications are transforming the Smalley, 1995). In one group of countries, those with a strong world economy and presenting new challenges to all countries. The "dual system" of apprenticeships-Austria, Germany, and Swit- challenges are to compete effectively in an emerging information- zerland-computer science and computer literacy became compul- based economy in which computing and communications play a sory subjects, but there was little use and integration of computers central role. They are also to take full advantage of the enormous in other subjects. A second group of countries-the United States, potential of information and communications in advancing all as- the United Kingdom, Austria, and part of Canada - endorsed both pects of economic, social, and cultural life. Electronic superhigh- computer literacy courses and integration into the curriculum from ways of broadband fiber optic lines, satellite channel capacity, data the beginning. However, these integrated approaches were slow to networks, and other essential information infrastructure are emerg- develop and hardware often found its way into schools before suf- ing in many countries. Likewise the increasing efficiency of data ficient attention had been paid to curriculum development and storage, improvements in the quality of software and steadily de- teacher education. France was the first country to commit itself to clining costs of production, are enabling more and more people to integrating the use of microcomputers across the curriculum. In have access to computer and telecommunications technology at Norway, Scotland, and the Netherlands, governments mandated work, home, and in school. Indeed, today's $2,000 laptop com- programs of integrated computer support and funded the creation puter is more powerful than a $10 million IBM mainframe com- of software and in-service teacher training. puter of twenty years ago. Moreover, today's personal computer hard drives can hold 1.2 gigabytes whereas the IBM PC/XT first 4. Developing countries also introduced computers into their edu- introduced in 1983 held 10 megabytes of information (Gates, 1995). cation systems during the past decade, although generally on a smaller scale and almost exclusively for the teaching of computer 2. Today's computers are also relatively user friendly. For ex- science and computer literacy. Many of these initiatives remained ample, graphic user interfaces, exemplified by such systems as pilot projects, although there have been exceptions. In recent years, Windows and Mac OS have all but replaced character-based sys- mainly due to the growing awareness of the need to prepare for the tems like MS-DOS. And, thanks to research by the Xerox Corpo- emerging information-based economy, some developing country ration, the "mouse" has become the vehicle for facilitating human- governments have already launched or are in the process of initiat- computer interactions. Over the past twenty years we have also ing sizable programs to introduce computers in the education sys- seen an exponential growth in computer software of all kinds, in- tem. They are: Belize, Brazil, Colombia, Costa Rica, Chile, cluding educational software. Students today can have access to Grenada, Singapore, Argentina, Uruguay, the Philippines, Turkey, very exciting educational software combining sound pedagogical Egypt, South Africa, and Jamaica. qualities with multimedia applications. These new software pack- ages give computers broad-based appeal serving students at all lev- 5. While in previous years governments had to be convinced of els and with diverse learning styles and capabilities. Finally, ad- the importance of computers in education that is generally not the vances in communications technology give students access to the situation today. Policy makers and educators generally employ world outside and to a rich array of information and other resources one or several of the following rationales in their decisions for adopt- for learning heretofore unavailable. Telecommunications technol- ing information technology: the social rationale which argues that ogy is becoming a common feature of classrooms and schools which schools need to provide students a "comfort level" with technol- have introduced computers into education. Using modems and tele- ogy so they can live and work in a rapidly emerging society where phone lines in their schools, students can now communicate via information and telecommunications play an increasingly impor- local and wide area networks as well as via the Internet. tant role; the vocational rationale which follows from the social * The authors are gratefdul for the comments provided by Jee-Peng Tan and Harry 3. Computers in Education. During the 1980s, most industrial- Patrinos (HDD) on a drat of this paper. 2 Education and Technology Series, Vol. 1, No. 3 rationale, emphasizes the need to prepare students for future em- the high c6sts'ofeducati6n ifithe United States, Perelman has con- ployment in information and telecommunications based economies; cluded there is a serious productivity crisis in education and insuf- the pedagogical rationale argues that computers can improve the ficient research and development to come up with solutions to deal instructional processes and learning outcomes in schools; and, fi- with this crisis. The failure to effectively exploit the instructional nally, the catalytic rationale is that computers can change schools power of the computer is but one example of the resistance to change for the better by accelerating educational innovations in the teach- in education. There is little mystery as to the reasons for this fail- ing learning which goes on in the classroom (Hawkridge, Jaworski ure, according to Perelman. They are the result of a lack of incen- and McMahon, 1990). tives, and disincentives common to government-owned, "bureau- cratically administered monopolistic enterprises." To address these 6. Despite significant cost reductions in recent years, informa- problems, Perleman calls for a major restructuring of schools with tion technology , as presently designed, is not going to be afford- incentives for improving learning outcomes, and more and better able for wide-spread use in most countries and school systems, any funded educational R&D on learning technologies. time soon. However, in a growing number of countries, policy makers and educators are still launching new pilot education tech- 9. How much does educational technology cost? Is it affordable nology projects, usually with the help of external and/ or extra- to most developing countries? Are there ways of reducing the costs budgetary resources and strong community support. In fact, the and increasing the effectiveness of projects? What can we learn experiences of the countries included in this study would suggest from the experience of other countries in designing cost-effective that affordability is a relative concept, not an absolute one. Even project inputs? These are the issues which motivate this study. those countries which have yet to achieve universal primary edu- There are certainly not new concerns. Several other writers have cation and have relatively low secondary school enrollments, gov- attempted to determine the costs of information technology as well ernment leaders at all levels have initiated pilot information tech- as its effectiveness as input to improving educational quality such nology projects. Even where schools lack other instructional in- as Levin (1985), Jamison, Klees and Wells (1978) and Glick (Un- puts such as textbooks and/or have poor sanitation facilities, devel- published). Nonetheless, Anzalone (1991) noted there was no lit- oping country educators and others are attempting to gain experi- erature yet on the costs of in-school application of computers in ence with computers. And in most ofthe countries, governments- developing countries and expressed the hope that pilot efforts would often with significant contributions by the community- are mak- soon produce some usable data on costs, so that an attempt could ing information technology accessible to children of all different be made to develop a projection for the costs of a "typical" applica- socio-economic backgrounds, and in some, mainly to lower income tion in a "typical" country. students in rural areas. 10. Purpose of Study. This study aims to assist education 7. A main tenet of this study is that the introduction of informa- policy makers and planners in analyzing the costs of programs to tion and communications technology in education in developing introduce and maintain information technologies in schools. The countries should not wait until a country has reached some prede- study has three main sections. In the first section, it provides back- termined state of economic or educational development. Even in ground information on the instructional uses and inputs of infor- countries which do not believe in the cost effectiveness of informa- mation technology programs in the primary and secondary schools tion technology as a tool for mass education, it is important that of several Latin American and Caribbean countries. The selected they begin acquiring experience using this technology for educa- countries are: Belize, Chile, Costa Rica, Jamaica and Mexico. In tional purposes. Otherwise, educators in developing countries will the second, it highlights some of the key issues in cost analysis be marginalized in the international dialogue on education. Short- methodology used to calculate the costs, and assess the financial term concerns for equity at the national level must be balanced by feasibility ofproposed investments in information technology. This longer-term concerns for equity at the international level. The World section also provides data, albeit, incomplete, on the costs of se- Bank's recent publication, Priorities and Strategies for Education lected information technology programs. The final section con- (1995) also notes that the failure to fully use cost-effective tech- tains the conclusion and recommendations of the study, which fo- nologies to improve educational quality, "carries the risk of further cus on how to reduce the costs and increase the benefits of pro- increasing the gap between [developing] countries and industrial posed information technology programs. ones." 11. This study is only a first step towards gaining greater under- 8. But what about the computer's promise of greater efficiency standing as to the costs and benefits of information technology in and productivity in education? Can governments expect to achieve education. Further research is needed to learn more about indi- these kinds of benefits from investments in technology? One of vidual country experiences and to test assumptions about the costs the more interesting and provocative observations about technol- and benefits of different models of computer use. We would like ogy in education appears in Lewis J. Perelman's article "Closing to encourage other countries and researchers to join us in develop- Education's Technology Gap "Perelman, (1995). Reflecting on ing good empirical data on the costs of information technology Cost Analysis of Information Technology Projects in Education 3 programs not only in Latin America and the Caribbean region, but Beginning in 1994, a second project at the secondary level, funded in other regions of the world which might offer interesting con- by a World Bank loan, has installed 20 computer laboratories at a trasts or collaboration of the interim findings in this study. While roughly estimated cost of $1.5 million. These schools are offering the data we have managed to collect is incomplete and not totally courses utilizing productivity tools as well as LOGO. reliable, it is the best we could obtain for now. And, despite these limitations, our data serves to define some of the key issues facing 15. Jamaica. The Jamaica Computer Society Education Founda- educators in analyzing the costs and benefits of educational tech- tion (JCESF), with the support of the Business Partners for Educa- nology investments. tion, the Ministry of Education, and the Heart Trust have been imple- menting two experimental pilot programs to improve the quality of 12. The Projects. For this study, we attempted to obtain cost data education in Jamaica's schools through the introduction of com- from several information technology projects in execution in the puters. Under its first program known as Jamaica 2000, the JCESF Latin America and Caribbean region. These particular projects are, and its partners mobilized US$4 million and installed computer by no means, the only information technology activities underway laboratories in 51 secondary and community colleges. An addi- in the schools of the particular countries. However, they are all tional 30 computer laboratories are scheduled to be installed by significant pilots, and in two cases, "Enlaces" in Chile and the Com- January, 1996. While originally intending to provide training in puters in Education program in Costa Rica, are truly national in computer studies in secondary schools, Jamaica 2000 now seeks to scope. We attempted to obtain cost information in three ways: di- increase literacy and numeracy through the use of computer-as- rectly from visits to the project sites, from IBM in Chile, Costa sisted instruction in reading and mathematics. A second pilot pro- Rica and Mexico and from interviews with project consultants. As gram recently initiated and known as EDTECH 20/20, is pursuing noted, we were only partially successful in our attempts to obtain similar objectives within primary and all age schools and has suc- detailed data on all the projects. However, we have utilized some cessfully mobilized some 1.2 million, mainly from the Inter-Amen- qualitative and quantitative data obtained on the following projects: can Development Bank, with the aim of installing 21 computer laboratories in 15 primary schools in four poor rural communities. 13. Belize. Two information technology projects are on-going in Jamaica is also receiving US$.5 million from INFODEV in sup- Belize. One project at the primary level was started by a personal port of these initiatives. donation from Francis Ford Coppola in 1994; the other at the sec- ondary was launched in the late 1980s and has been continued by 16. Mexico. Two of Mexico's on-going information technology the Michael Ashcroft Foundation. Working with the Apple Com- projects were examined. One, known as Genesis, is located in the puter Corporation and in consultation with the World Bank, Mr. state of Aguascalientes and is funded by the state from its own Coppola contributed about US$100,000 to equip a group of six resources. Launched in the 1993-1994 school year, it includes all primary schools with five computers, CD-ROMs, printers and other 61 secondary schools, 26 of 300 primary schools, and 3 normal equipment all of which were placed in classrooms. The software schools. The project constructed laboratories with some 20 com- consisted of ClarisWorks, Compton's encyclopedia, and a variety puters and trained some 400 teachers, and established a mainte- of other educational titles in science, graphics, environmental stud- nance and repair service. LOGO is the predominant instructional ies and reading. The second program provides computers and other tool, although the machines also have Microsoft Works installed hardware and software for laboratories in secondary schools. The and students use its word processing capabilities. The other project major thrust of the program is on preparing students to use com- is located in the municipality of Puebla where secondary education puter productivity software for employment in business and gov- is the responsibility of the Rector of the Benemerita Unversidad ernment. The schools are networked to a server at the Belize Teach- Auton6ma de Puebla (BUAP). The BUAP-funded project has pro- ers College which operates E-Mail. vided computer labs for eight secondary schools, in-service train- ing for some 40 teachers, and technical and curriculum support 14. Costa Rica. The Computers in Education Program is a na- from two US organizations, the Academy for Educational Devel- tional program that has been in operation since 1988 at the primary opment and the Center for Research in Education (INCRE). school level. The main funding has been provided by the Omar Dengo Foundation, created by the country's ex-President, Oscar 17. Chile. There are two important information technology Arias. Supplementary support to the program has been provided projects in Chile. One pilot project known as Genesis, is being by USAID, the Inter-American Development Bank and other carried out in the municipality of &ufloa, which is located in the sources. The project has installed computer labs in some 127 pri- greater Santiago metropolitan area. The project was launched in mary schools reaching about 1 out of every 3 Costa Rican primary 1994 at the initiative of Mayor, Pablo Vergara, a strong advocate school students. The project was among the very' first in the re- for both education and computers. The project has been imple- gion to use LOGO as its main instructional tool. Seymour Papert, mented with the assistance of the Latin American Center for Re- LOGO's creator, and his colleagues from MIT, jointly with IBM, search on Education (CUE), an IBM technical support team, based have delivered most of the project's hardware and software inputs. in Mexico. The first phase of the project includes 13 of 2 schools. 4 Education and Technology Series, Vol. 1, No. 3 Each school has been provided with computer labs containing some dents who take these classes obtain preparation for clerical jobs or 20 PCs and given staff development training by CLIE. The main in some cases for post-secondary technical and university educa- thrust of the project is teaching students thinking and problem solv- tion. These types of classes are commonplace in secondary schools ing skills using LOGO programming language. The second project, in all countries and they fill a strongly felt need by educators and known as Enlaces, (Linkages) forms part of Chile's comprehen- parents to prepare students for the labor market. In Puebla, Mexico sive educational reform program for both primary and secondary where computers have been installed in eight municipal high schools. Enlaces was launched in March 1993 with the goal of schools, administered by the Benemerita Universidad Auton6ma creating a national telecommunications and computer network de Puebla students are being taught to use productivity tools for among 100 Chilean primary schools and associated institutions by problem solving in their regular courses. Belize is using them in 1997. By mid-1995, Enlaces had substantially surpassed that target courses which prepare students for employment in private business and by end 1995 had expanded the network to some 200 schools in and government. the increasing the number of primary schools and incorporating an (iii) Programming LOGO. One of the most original and chal- initial group of secondary schools. Enlaces is projecting to incor- lenging applications of computers in education is the LOGO pro- porate all public primary and secondary schools into the network. gramming language. Logo has been used in schools all over the Enlaces is supported by the Government, the community and loans world since the early 1980s as a tool to promote thinking, problem from the World Bank. solving, and creativity. It was widely popularized by Seymour Papert in his 1980 book, Mindstorms, a publication which, accord- 18. Use of Computers. What are the main applications of com- ing to John Sculley, former Chairman and CEO of Apple Com- puters in schools and for what educational objectives? The infor- puter, Inc., started the computer revolution in schools. The two mation technology projects in the countries we have examined use main premises on which Logo is based are that children can learn computers for the following: teaching basic literacy and numeracy, to use computers in a masterful way and that learing to use cor- use ofproductivity tools, LOGO programming, constructivist peda- puters can change the way they learn everything else. Contrasting gogy, for communications networks, and for instructional enrich- his approach to computer-aided instruction, Papert noted in ment. Mindstorms that according to in his vision: rather than using the (i) Basic Literacy and Numeracy. Since the early 1980s, computer to " program the child, "it is the child who programs the computers have been used to tutor students in basic literacy and computer and in doing so "both acquires a sense of mastery over a numeracy through a variety of programs collectively known as piece of the most modem and powerful technology and establishes Computer-Assisted Instruction (CAI) or Integrated Learning Sys- an intimate contact with some of the deepest ideas from science, tems (ILS). In CAI the computer provides text and multiple-choice from mathematics, and from the art of intellectual model building questions or problems to students, gives and immediate response (Papert, 1980). Logo has been widely used in Latin America, Eu- to the answers given, summarizes students' performance and gen- rope and to a lesser extent in United States schools. erates exercises for worksheets and tests. CAI typically presents (iv) ConstructivistPedagogy. In a growing number of schools tasks for which there is only one correct answer; it can evaluate , particularly in the United States, computers are being used to sup- simple numeric or very simple alphabetic responses, but it cannot port constructivist teaching (Means and Olson, 1995). The model evaluate complex student responses. In ILS, computers are net- of constructivist teaching draws on advances in cognitive psychol- worked and equipped with software that provides a set of sequen- ogy which have fostered our understanding of the nature of skilled tial lessons. Students work through these lessons as prescribed by intellectual performance and the environments conducive to learn- the built-in management system, which tracks individual student ing. In schools where constructivist teaching is employed, teach- progress. These systems are "integrated" in the sense that each ers are working at creating learing environments in which stu- lesson is connected with the next, all lessons are correlated with a dents actively pursue knowledge rather than passively receive it, set of objectives and all tests are matched to the lessons and objec- and where they can develop the advanced skills of comprehension, tives. As noted above, Jamaica is using ILS for teaching literacy composition, reasoning, and experimentation. in constructivist and numeracy at both the primary and secondary levels. ILS is classrooms, teachers serve as facilitators of learing, attempt to being used widely and effectively in the United States, and in tiny create authentic contexts for modeling expert thought processes, Grenada. and seek ways to help students to achieve intellectual accomplish- (ii) Productivity Tools. Many schools today are teaching stu- ments they could not do on their own. In many of these areas, dents to use a variety of computer productivity tools widely used in computers are providing effective interactive learing contexts con- business and commerce: word processing, spread sheets, data bases, sistent with constructivist instructional goals. In addition to the and e-mail. Inmost secondary schools, the purpose of this instruc- United States, Jamaica and Chile are expected to begin experiment- tion is to equip students with marketable skills for employment. ing with these approaches in their computer programs in the com- These classes are usually taught in computer labs which were for- ing years. merly filled with typewriters but now include computers, an array (v) Communication Networks. In both the industrial and of commercial software, printers and possibly modems. The stu- developing countries, a rapidly growing number of schools are us- Cost Analysis of Information Technology Projects in Education 5 ing computers to connect their students with the world outside. By compare the purchase prices of program inputs that are to be bought bringing telecommunications applications into their classrooms, at different times without adjusting for the time value of money. teachers are able to create environments where students can com- municate with other students via electronic mail, participate in col- 20. Cost-Benefit and Cost-Effectiveness Analysis. Cost com- laborative projects with students from other schools, and come in parisons can be used to inform different kinds of decisions. If costs contact with a rich array of information sources that broaden their are measurable in terms commensurate with benefits, as they usu- horizons. Chile currently runs one of the few successful educa- ally are in income generating enterprises and in some other con- tional wide area networks (WAN) of any country in the world. texts, they may fruitfully be compared with benefits in an integrated Belize's secondary schools are also networked, but the network is 'cost-benefit' analysis, using such summary measures as 'rate of not actively used. Other schools are able to link up to the Internet return on investment'. When they are not measurable in terms com- via telephone lines and local servers often located in universities. mensurate with benefits, the cost comparison can be one of 'cost Where countries have fiber optic cables, as in Chile, plans are be- effectiveness', where the costs of alternative methods of achieving ing made with local telephone companies for Internet connections the same qualitative or quantitative benefit are compared. For in- and access to the World Wide Web. A number of school systems stance, the cost of a 10% increase in reading scores may be calcu- in developing countries are linked via the Internet to math and sci- lated for several different interventions, such as a decrease in the ence networks such as the Global Laboratory project and National student-teacher ratio, an increase in the quantity of teaching aids, Geographic Kids Network science series developed at TERC with or the placement of computers in classrooms. Even narrower cost funding from the U.S. National Science Foundation. comparisons are frequently useful. If the decision involves how to (vi) Knowledge. Educational courseware and reference mate- configure computers in a school, the cost comparison may involve rials such as stand along units of instruction in math, science, lan- the cost per student computer contact-hour of a computer labora- guage and other subjects, as well as encyclopedias make up the tory configuration versus a classroom configuration. When more largest number of titles on the market. They are available on dis- than one benefit results from a program, the lack of kette, but also increasingly on CD-ROM and in multi-media for- commensurateness among benefit indices can be overcome by mats. They are used in the classroom to supplement and enrich weighting to form a single weighted benefit index (para. 31). teaching for practice and drill, independent study and research. The data which is now available on CD-ROM is larger and more di- 21. Cost Data. Cost data, of course, are used for accounting func- verse than is currently present inmost school libraries in develop- tions that are important in themselves, for instance, regulating cash ing countries, if they have them at all. Further, some of the new flow, calculating taxes, deterring fraud or allocating budgets. Also, multi-media learning environments and formats offer students the in budget documents, expenditure data often refer to planned ex- opportunity to use those sensory skills which are most suitable for penditures rather than actual. These uses and aspects of cost data their individual learning styles. typically determine how it is aggregated and presented. For cost analysis to be useful to decisions, however, adjustments to account- IL Cost Analysis of Educational Computer Applications ing data are almost always necessary. For instance, the school tech- Cost Analysis Methodology' nology budget may be aggregated across different technologies, may allocate the cost of long-lived equipment expenditures to a 19. The purpose of cost analysis is to assist in making decisions single year or may neglect to include the non-capital costs of tech- about the use of scarce resources. The most general formulation of nology programs m the technology account. Such data must be dis- the concept of cost is that the cost of a set of resources (and the aggregated over program and time, and data from personnel and interventions they permit) is the maximum value of what they can other accounts obtained, before even crude estimates of the costs alternatively be used to produce for the decision maker or the en- of a technology program can be obtained for decision-making. tity he or she acts for. If this entity is society as a whole, as is often the case in educational cost analysis, the usual assumption is that 22. Marginal Cost. Another way in which available cost data - market prices and other equivalent money values can be used to usually average or total cost data - often need to be adjusted for evaluate the cost of resources, as long as they can be seen as repre- senting the revealed preferences of society at large. Expenditure The treatment here is il brief. More extensive treatment of the values are also used without adjustment where program managers methodology of cost analysis relevant to computer applications in educa- aim to evaluate alternative expenditures out of given budgets, even tion can be found in several excellent longer-length discussions: Henry though the budgeted sums do not reflect real economic value. De- Levin, Cost Effectiveness: A Primer, (Beverly Hills, Sage Publications, viation of budgeted values from real economic ones arise for ex- 1985) specifically treats the cost analysis of computer-assisted instruction; ample, when programs can purchase equipment at a preferential Dean T. Jamison, Steven J. Klees and Stuart J. Wells, The Costs ofEduca- exchange rate. In this case, cost comparisons using money values tional Media: Guides/or Planning and Evaluation (Beverly Hills, Sage would not be appropriate for the decisions involving social effi- Publications, 1978) treats the cost analysis of educational broadcasting; ciency. In one particular important case, one should generally not and PeterJ. Glick, CostA nalysis Categories andProcedures (Unpublished) 6 Education and Technology Series, Vol. 1, No. 3 deciding on the allocation of resources is when marginal cost data national government or a particular local government, the interest are needed. Knowing the marginal cost (the extra cost) per addi- rate at which that government borrows can be considered to be its tional student, for example, may be important for modeling the financial cost of capital. costs of different technologies or configurations, because certain of them may be more efficient at relatively lower or higher levels 26. Selecting the appropriate discount rate for a social investment of utilization. Where appropriate, the marginal cost concept may project is, thus, somewhat tricky, and different analysts come up be captured in a simple fixed-variable cost model. Many technol- with different answers. To maintain consistency across different ogy problems can be expressed in this model, because in many analyses, therefore, discount rates have become conventionalized analyses the equipment, installation, software licenses, wages of in different contexts. Organizations often prescribe the particular laboratory instructor, etc. may be considered to be fixed costs re- discount rate that is to be used for projects under their jurisdiction. gardless of the number of students served, and materials, electric- Alternatively, the analyst can use a range of discount rates to deter- ity to run the computers and peripherals, teacher time, etc. are var- mine the sensitivity of total project cost to variation in the discount able (marginal) costs that vary with the number of students served. rate. In any case, analysts and users of their analyses need to un- derstand that the "cost of money" is just as real a cost as the cost of 23. Capital Costs. Because most cost analyses are done for peri- teachers' salaries or supplies and cannot be neglected without bias- ods of a year or less, a special problem arises with the costs of long- ing the analysis in favor of capital-intensive interventions. lived assets, such as facilities, equipment, and training, where the expenditures are made in one period but the benefits reaped over 27. Sources of Funds and Costs. Another guide to determining the course of several other periods. When measured benefits are which costs to include in the analysis is to look at the sources of commensurate with costs and arrive at times different from costs, funds. If the goal is to determine the comparative educational effi- an analysis of the full streams of costs and benefits will be neces- ciency of particular interventions, it is irrelevant who pays for them sary, with all values converted to present values and summarized, - governments at different levels, school fund-raising, teacher vol- if so desired, in an internal rate of return on investment.2 In cost- unteers, parents and students, international donors, etc. All costs, effectiveness analysis, however, where benefits are not commen- however financed, need to be included in theanalysis. If the issue surate with costs, a simplified technique to "annualize" the costs of is one of affordability from an educational budget, however, only long-lived assets can be used (para. 32). It has the additional virtue the marginal budgetary costs of particular types of interventions of allowing the period of analysis to be coterminous with the bud- need be considered. As for the other sources of funds, only their get period. the sustainability would then be relevant. The danger of taking a narrow budgetary approach, however, is that efficiency consider- 24. The costs of long-lived assets are generally of two types - de- ations may be neglected with a resulting waste of scarce resources preciation and the financial cost of capital. Depreciation is that available to the system as a whole. portion of the value of the capital good that is consumed over time, either through wear and tear or through obsolescence. Depending 28. Cost Comparisons. Most cost analyses for program decisions on the purpose, annual depreciation can be estimated in different involve comparisons, which are sometimes only implicit, of 'with- ways. For example, it can be estimated as the average annual de- the-program' and 'without-it'. Carefully defiing them is at the crease in value over the life of the project or the asset or as a stream heart of an effective cost analysis. For instance, if the cost of a of values that differ in amount according to the age of the asset. computer-assisted-instruction (CAI) program in a classroom con- figuration is desired, one clear cost element would seem to be the 25. The second capital cost that must be considered is the finan- teacher time spent in planning, organizing and supervising class- cial cost of capital, which is simply the cost of tying up resources in room CAT activities. If the decision is to install the classroom CAI a particular capital good, when they could be used for other expen- program only if it improves reading scores more cost-effectively ditures. This cost is measured by the appropriate discount or inter- than alternative methods, all of which use different amounts of est rate for the entity bearing the cost. If the entity bearing the cost teacher time, teacher time should be accounted for in the cost com- is considered to be "society", the appropriate discount rate is the parison. If the analysis is designed to examine whether a technol- "social discount rate", i.e., one that reflects the value society would ogy-intensive classroom could be designed to economize on teacher have gained by using the financial resources for current consump- time and thus reduce total classroom costs, clearly, costing teacher tion rather than tying them up in a piece of equipment or another long-lived asset for the purpose of producing consumption in fu- I Discussions of present value and internal rate of return can be found in any ture years. Since "society" cannot be interviewed directly, what financial management textbook. Basically they involve 'discounting' values to society pays for capital, as measured by interest rates in private the present by dc-compounding them using a discount (interest) rate. In present value analysis, the discount rate is given; in internal rate of return analysis, the rate financial markets (when they are deemed to be functioning ad- is calculated to bring future costs and benefits back to the present so that dis- equately), is the measure typically used to approximate the social counted costs and benefits are equal. See para. 32 for a discussion of the use of a discount rate. If the entity bearing the cost is considered to be the discount rate to 'annualize' equipment costs. Cost Analysis of Information Technology Projects in Education 7 time would be central to the analysis. However, if the decision is to ample is shown to be sensitive to the benefit weights; when either ascertain the affordability of the classroom CAI program vis-i-vis reading or math is emphasized, the special reading and math pro- the budget, it would not be appropriate to include teacher time in grams are respectively more cost-effective and more affordable than the calculation of cost, because the decision is not about the de- the CAI program. Finally, unless the decision maker is prepared to ployment of teacher resources. The steps in cost analysis should, forgo the chance of significant improvement in one or the other of therefore, be done in the following order: (i) clearly specify the reading and math, the choice needs to be restricted to those alterna- decision, (ii) define the explicit or implicit cost comparisons and tives that can deliver significant improvements in both math and (iii) specify and develop data on the cost elements with and with- reading - the CAI program and the combination of the special read- out the intervention. ing and math programs. When only these two are considered, the CAI program is most cost effective in all relevant cases. Conversely, Doing Cost Analysis of School Computer Applications the combination program is usually tops when affordability is the criterion. 29. Schematic Example. The costs of different approaches to in- creasing reading and math skills are compared in a schematic cost 32. Annualizing Capital Costs. The special problems that arise analysis and serve as an illustrative example of how to go about with the cost analysis of long-lived assets, such as facilities, equip- doing a cost analysis. A hypothetical CAI program is compared ment and training, were discussed above. Because cost-effective- with Alternative A, a program emphasizing math, Alternative B, a ness analyses are conveniently done on the basis of annual average program emphasizing reading, and a fourth alternative, the simul- cost, often to conform to the budget period, capital costs must usu- taneous use of both A and B (Tables I and 2). The schematic ex- ally also be expressed in annual terms. This is typically done by ample illustrates the importance of clear thinking about the deci- using a simple financial model available on standard spreadsheet sion involved, about the appropriate cost comparisons and about programs, which can be used to provide an annualization factor to the relative importance of various benefits. convert multi-period capital cost into an average annual cost.' This average annual cost combines depreciation and the cost of money 30. The first important decision that the cost analyst needs to make in a single figure. Table 3 gives examples of this calculation for is to decide whether the comparison is on the basis of cost-effec- equipment of varying useful lives. As one example given in the tiveness or affordability. In the sense used, cost-effectiveness means table, a set of equipment with a purchase price of $150,000, no the impact of marginal resources on reading and math outcomes scrap value, and a useful life of 5 years would have an annualized and measures the efficiency of resource use. Affordability means capital cost of $41,611. the impact of marginal budget expenditure on the same outcomes and is a measure of feasibility not efficiency. When the decision is 33. Price Distortions and Taxes. The treatment of price distor- to choose the most cost-effective program, the costs must include tions and taxes in a cost analysis depends on whether the analysis is the opportunity cost of teacher time, since the alternatives use dif- being carried out from the point of view of society as a whole or ferent amounts of teacher time, which could be used for other valu- from that of a particular agency. When the analysis is being car- able teaching activities (Table 1). For this purpose, teacher time red out from the point of view of a particular agency, and the focus could be valued at teacher cost per hour or, in an attempt at greater is on financing out of a budget, the actual prices paid, distorted or realism, at some percentage of this, assuming that teachers would not and the actual taxes paid, are relevant costs. On the other hand, implement the new-programs using time that was less productive when the analysis is being done from the point of view of society than their average. When the decision is to choose the most afford- as a whole, the object is to find the efficient solution, which will able program, teacher time should not be included as an expendi- allow society to gain maximum benefits from its available resources. ture, because there will be no change in the overall expenditures on This may require adjustments in the costs used in the analysis to teachers (Table 2). take account of price distortions and taxes, since they do not repre- 31.3 The model is mathematically the same as that used to calculate equal annual 1. Wen her isa siglebenfit theanaysi isstraghtorwrd. payments on an equipment loan with the cost of money equal to the interest part of The cost-effectiveness index is simply the amount of the benefit the annual payment and the depreciation equal to the principal pat of the pay- per unit cost. When there is more than one non-commensurate ment. Annualized capital cost (or annual loan payments, when that is what is benefit, like reading and math test scores, they must either be evalu- desired) can be calculated using the PMT function of any standard computer spread- ated qualitatively or a weighted index constructed. Care in select- sheet program or the formula c[r(l+r)n]/[r(I+r)n-I], where c is the cost of the asset, r is the discount rate and n is the useful life of the asset. Since in this model the ing weights should be taken, since program choice may be sensi- annual cost of money tied up at any particular time in the remaining value of the tive to the weights selected. When improvements in math and read- asset and the annual depreciation combine to form a constant annual amount, de- ing scores are given equal weighting in the schematic example, the preciation is by implication modelled as a moderately increasing function of time CAI program is shown to be the most cost-effective of the four for relatively short asset lives and low interest rates. While this model is highly convenient and realistic enough in most cases, when asset lives are long-lived and interventions(Table 1) but, with the same benefit weighting, also interest rates very high, a model incorporating a more realistic depreciation profile the least affordable (Table 2). The decision in this schematic ex- may be indicated. 8 Education and Technology Series, Vol. 1, No. 3 TABLE 1. Costs of Computer-Assisted Instruction (CAl) and Alternative Programs for Increasing Reading and Math Scores, Schmatic Comparison Alternative Decision I : Invest in the Most COST-EFFECTIVE Among CAI, A or B Most Cost-L-T. CAI Alternative Alternative Combined AII CAl v. A B A & B Four A & B SCOPE No. of Students Served 300 200 400 400 COSTS INVESTMENT COST, ANNUALIZED Training 10,000 20,000 20,000 Facilities 5,000 0 0 Equipment & Prepaid Services 25,000 10,000 15,000 Total Annualized Investment Cost 40,000 30,000 35,000 RECURRENT COST Personnel Services CAl Coordinator (half time) 25,000 Opportunity Cost of Teacher Time 10,000 25,000 40,000 Other 5,000 5,000 5,000 Subtotal Personnel Services 40,000 30,000 45,000 Software and Supplies 15,000 5,000 10,000 Maintenance and Technical Assistance 5,000 3,000 3,000 insurance and Other Recurrent 5,000 5,000 5,000 Total Recurrent Cost 70,000 48,000 68,000 Total Costs 110,000 78,000 103,000 259,000 Total Cost Per Student 367 390 258 648 BENEFITS Increase in Reading Score 10% 20% 1% 19% Increase in Math Score 15% 2% 15% 14% COST EFFECTIVENESS Cost Per Student per 10% Increase in Weighted Benefit Score Benefit Weights R.eading Math Equal Weights 50% 50% 293 355 322 392 CAI CAl Other Weightings: Reading Emphasized 67% 33% 315 277 458 373 A CAl Math Emphasized 33% 67% 275 491 248 414 B CAI Only Reading Valued 100% 0% 367 195 2,575 341 A A & B Only Math Valued 0% 100% 244 1,950 172 463 B CAI * Long-lived training, facilities and equipment costs are annualized, i.e., presented as average annual costs. See Text. Cost Analysis of Information Technology Projects in Education 9 TABLE 2. Costs of Computer-Assisted Instruction (CAI) and Alternative Programs for Increasing Reading and Math Scores, Schmatic Comparison Alternative Decision 2 Invest in the Most AFFORDABLE Among CAI, A or B Most Affordable CAl Alternative Alternative Combined All CAl v. A B A & B Four A& B SCOPE No. of Students Served 300 200 400 400 EXPENDITURES INVESTM ENT, ANNUALIZED Training 10,000 20,000 20,000 Facilities 5,000 0 0 Equipment & Prepaid Services 25,000 10,000 15,000 Total Annualized Investment 40,000 30,000 35,000 RECURRENT Personnel Services CAl Coordinator (half time) 25,000 Additional Paid Teacher Time 0 0 0 Other 5,000 5,000 5,000 Subtotal Personnel Services 30,000 5,000 5,000 Software and Supplies 15,000 5,000 10,000 Maintenance and Technical Assistance 5,000 3,000 3,000 Insurance and Other Recurrent 5,000 5,000 5,000 Total Recurrent 60,000 23,000 28,000 Total Expenditures 100,000 53,000 63,000 169,000 Total Expenditures Per Student 333 265 158 423 BENEFITS Increase in Reading Score 10% 20% 1% 19% Increase in Math Score 15% 2% 15% 14% AFFORDABILITY Expenditure Per Student per 10% Increase in Weighted Benefit Score Benefit Weights Reading Math Equal Weights 50% 50% 267 241 197 256 B A & B Other Weightings: Reading Emphasized 67% 33% 286 188 280 244 A A & B Math Emphasized 33% 67% 250 334 152 270 B CAl Only Reading Valued 100% 0% 333 133 1,575 222 A A & B Only Math Valued 0% 100% 222 1,325 105 302 B CAl * Long-lived training, facilities and equipment costs are annualized, i.e., presented as average annual costs. See Text. 10 Education and Technology Series, Vol. 1, No. 3 sent resource costs to society as a whole, but merely the transfer of experience with currently available educational computer applica- resources from one group to another. As an example, teacher wages tions in developing countries. Most of the experience with com- can be seriously our of line with private labor markets for similar puters in schools in developing countries to date has been the un- skills if they reflect the political power of public service unions to planned acquisition of computers by secondary schools, often sup- any great extent. As for taxes, import duties and sales taxes, which ported by parents interested in their children acquiring computer may be included in costs paid by even governmental organizations, tool skills useful in future employment. The experience of Belize, are in reality a transfer from one group to another through the gov- which is analyzed in a somewhat outdated USAID consultant re- ernment budget and are not payments for resources used specifi- port, is illustrative of this and is discussed below. Nevertheless, as cally in the production of the item purchased. Thus, they should be shown by the country survey presented in the first section of this deleted from costs when societal efficiency is the objective of the paper, a considerable start has been made in a number of develop- analysis. ing countries. 34. Inflation. Special care needs to be taken of the impact of in- 36. Comparative Cost Analysis. Unfortunately, the experience flation on costs. When the focus is efficiency in resource use, the discussed above has not been accompanied as yet with effective best procedure is to adjust all costs to the money value of the date efforts to do real world cost analysis, neither on the prevailing labo- at which the project starts. That way, the same amount of resources ratory configuration nor on other configurations. The data which is seen to cost the same amount of money at all times. If the focus was obtained for this study, unfortunately, is also not complete for is on budgetary affordability, however, the actual (inflated) prices any of the projects. Both time constraints in country visits and the expected to be paid may be the more appropriate ones to use in the unavailability of reliable data, made it difficult to obtain the kind of analysis. information we would have liked to include in our cost analyses. Since no systematic benefit analyses are also available, it is also The Costs of Educational Computer Applications in Devel- not surprising that little cost effectiveness analysis has been done. oping Countries This needs to be rectified, because greater use of computers in edu- cation in developing countries will at some point need justifica- 35. The large reductions in the cost of ever-more-powerful com- tions based on costs and benefits. Despite the general lack of cost puters and peripherals and the attendant qualitative improvements data that can be used to estimate the cost of information technology in educational software are too recent for there to have been much projects in developing countries, the Jamaica and Chile's Genesis Table 3. Annualized Capital Cost Years of End of Year Annualization Useful Annualized Factor /3 Life /C Cost /2 Cost of Equipment (1) (2) (3) $ 150,000 1 $ 168,000 1,120 2 88,755 0.592 Discount Rate 3 62,452 0.416 12% 4 49,385 0.329 5 41,611 0.277 6 36,484 0.243 7 32,868 0.219 8 30,195 0.201 9 28,152 0.188 10 26,548 0.177 I/ Number of years=NPER in the Excel spreadsheet function, see note 2. 2/ The Excel spreadsheet function is PMT(10%,NPER.150000). The Lotus 123 function is similar. 3/ Calculated by dividing column (2) by the cost of the equipment. Cost Analysis of Information Technology Projects in Education 11 project examples described above have provided enough real world linguistically appropriate for use in most countries. Despite the data about actual laboratory installations to tabulate along with the fact that there are over 13,000 educational software titles on the Belize data in comparative format (Tables 4-6). To round out the market in the United States, most developing country educators are discussion, we also discuss the cost of computer-rich classroom reluctant to purchase educational software from other countries. configurations programs in the USA (Table 7). The English-speaking Caribbean , Chile and Costa Rica, appear to be exceptions. Jamaica is using Autoskill's ILS for teaching lit- 37. Equipment Costs. Leaving aside the Belize data,4 which are eracy and numeracy, Costa Rica has trained teachers to use the both outdated and deficient in other respects for comparative pur- Geometer's Sketchpad, U.S-produced software for teaching ge- poses, the first clear conclusion is that in the other three cases an- ometry, and Chile's Enlaces project has invited software publish- nualized investment costs, predominantly equipment costs, are about ers to offer their educational software titles in Spanish for review half or more of all costs when property protection costs are added and possible purchase. Second, in the Genesis projects in Mexico in. Clearly, equipment costs do matter, and the cost declines of the and Chile, the computer applications in schools have concentrated last decade therefore also make a large difference. Cost in the three on the teaching of the LOGO programming language, so that the countries for computers, at least, appear to fall in a range from only software purchases required have been different versions of $1,600 to $2,000 for machines. While the capabilities of the vari- LOGO. Third, computer manufactures are bundling educational ous machines, particularly their communications and CD-ROM and productivity software into their computers so that country edu- capabilities may vary somewhat, the data suggests that current com- cators may not feel the need immediately to go to the market in puter costs are now similar in widely different countries, search of other software. 38. Market Differentiation. This may not persist for long, how- 40. Professional Development. The project data indicates that ever, if software and hardware interactions in the market allow a countries are allocating widely varying amounts on professional low-end market to develop to the advantage of developing coun- development, ranging from about 18 percent in the Genesis project tries. Because of the high investment costs, many programs in these in Chile 16 percent in Jamaica's new projects, and only 4 percent countries would opt for low-end set-ups with less costly and less in the United States. Training costs are among the most difficult capable (but still educationally effective) inputs. For the last sev- costs to analyze because they can vary widely depending upon how eral years, the personal computer market has had a very simple much training is needed, who does the training, and where it takes structure, with basically two levels of capability - low-end proces- place. Some of the most effective professional development takes sors, now in the Intel 486 capability range, and high-end proces- place outside the classroom when teachers learn from each other. sors, currently the Intel Pentium. Prices of computers at both 1ev- Judging from the personal assessments made by teachers partici- els have been declining, but not as fast as the price of computing pating in projects and from classroom observations in Belize, Costa power. In the process, the capability of even low-end hardware Rica, Chile and Mexico, there are clear indications that teachers and software has risen very rapidly. A reasonable hypothesis is have benefited from in-service training programs. Training has that sooner or later, the hardware market will experience much made teachers more knowledgeable of computers and generally greater product differentiation with more levels than just low-end confident in using them in certain tearing situations. Many are and high-end. Instead of the low-end simply being the high-end of also excited about the potential of the technology and would like to two years ago, some of the increased productivity in the computer develop skills to enable them to use the technology more effec- industry may then be channeled into large price declines rather than tively in the classroom, Initial investments in training have also the increased capability that will undoubtedly continue in high-end built up training capacity by "multiplicadores" in several countries computers. The educational software industry will also adapt to so that there should be continuous training and support to teachers hardware differentiation with a parallel differentiation. If these in the classroom. Likewise, in the Genesis projects in Chile and market events occur, it will be good news for the mass extension of Mexico, CLIE technical support visits are programmed four times computer projects in developing country schools. In most cases, during the school year. These initial training efforts notwithstand- education officials in these countries will undoubtedly find that ing, there is reason to believe on the basis of recurrent cost alloca- having a Ford-Escort type of program widely available is prefer- tion data, that most projects are not providing sufficient follow-up able to having a Mercedes-type program in a few schools, training for teachers and administrators which would permit them to use information and communications technology more effec- 39. Soft are. Initial and recurrent expenditures on software are tively. After their first training event which lasts about two to three less than 3 percent of total project costs in most developing coun- weeks most teachers do not obtain much additional training, ex- tries, randomly reach about 5 percent of total project costs in the United States. Software expenditures are always going to remain The computer cost was presented in annualized form in the source of the Belize data Working backward, the implied cost per computer was a very low $1,115 relatively modest compared to other project inputs for a variety of (see Table 6). Certainly, this was not a commercial rate in 1987, so the relevance reasons, cost being only one factor. First, there continues to be a of the figure is limited at present. The argument in this paragraph, however, would shortage of high quality educational software that is culturally and suggest that such a figure may not be unrealistic in the not-too-distant future. 12 Education and Technology Series, Vol. 1, No. 3 TABLE 4. Costs of Computer-Assisted Instruction, Laboratory Configuration, in Jamaica, 1995* (US$) Useful Average Invest- Annualized Cost Category Item Life Unit No. of Unit ment or Annual % (Years) Description Units Cost Cost Cost" COSTS INVESTMENT" Facilities Comput. Rm., Renovation 15 Contract 1 $ 3,000 $ 3,000 $ 394 1 Airconditioning 7 Unit 1 1,700 1,700 349 1 Contingency & Other 10 2% of Facil. 1 94 15 0 Equipment Computers 5 Unit 15 2,000 30,000 7,914 30 Peripherals 5 Set 1 1,200 1,200 317 1 Power Protection 10 Unit 1 1,300 1,300 212 1 Contingency & Other 8 5% of Equip. 1 1,625 305 1 Subtotal Facilities & Equipment 38,919 9,506 35 Software Site Licenses 7 Set 1 3,200 3,200 657 2 Training (Upfront) Lab Coordinator 7 Person Week 8 250 2,000 411 2 Teacher Training 7 Training Day 5 285 1,425 293 1 Total Investment $ 44,119 $ 10,574 39 RECURRENT"* Personnel Cluster Support Pers. Wk. 3 80 $ 208 1 Lab Coordinator Pers. Wk. 52 8o 4,160 16 Fringe Benefits 15% Salaries 1 655 2 Maintenance Equipment Per Lab Chg 1 1,625 1,625 6 Software Per Lic. Chg 1 85 85 0 Routine Year 1 500 500 2 Insurance Contract 5% of Equip. 1 1,625 1,625 6 Training Training Days 12 285 3,420 13 Electricity Month 12 200 2,400 9 Telecommunications Telephone Year 1 411 411 2 Internet Provider Year 1 343 343 1 Computer Supplies Year 1 1,000 1,000 4 Total Recurrent $ 16,224 61 Total $ 26,798 100 Recurrent Total Cost Per Student (300 students) $ 54 $ 89 % of National Primary Per-Student Recurrent Expenditures (US$139) 39% 64% % of National Secondary Per-Student Recurrent Expenditures (US$315) 17% 28% Source of Basic Data: World Bank Estimated costs of 15-station computer laboratory for proposed World Bank-financed project, based on costs of similar facilities in the recently initiated EDTECH 20/20 project. * Long-lived training, facilities and equipment costs are annualized, i.e., presented as average annual costs, using a 10% discount rate and with varying useful lives. Depreciation is included. ** Does not include marginal costs for classroom teacher time and computer room space, which are assumed to be zero, since the focus of the analsysis is on affordability rather than cost-effectieness. ost Analysis of Information Technology Projects in Education 13 TABLE 5 Costs of Computer-Assisted Instruction, Laboratory Configuration, in Chile, 1995* (US$) Useful Average Invest- Annualized Cost Category Item Life Unit No. of Unit ment or Annual % (Years) Description Units Cost Cost Cost" COSTS INVES7MENP* Facilities Comput. Rm., Renovation 15 Contract 1 $ 5,000 $ 5,000 $ 657 2 Furniture 10 Set 1 2,500 2,500 407 1 Local Area Network 10 Unit 1 5,000 5,000 814 2 Contingency & Other 10 2% of Facil. 1 150 24 0 Equipment Server 5 Unit 1 2,100 2,100 554 1 Computers 5 Unit 20 1,700 34,000 8,969 22 Peripherals 5 Set 1 1,200 1,200 317 1 Backup Generator 7 Unit 1 500 500 103 0 Equipment Installation 5 Contract 1 1,000 1,000 264 1 Contingency & Other 8 5% of Equip. 1 1,785 335 1 Subtotal Facilities & Equipment 53,235 12,443 30 Software Site Licenses 7 Set 1 3,300 3,300 678 2 Training (Upfront) Lab Coord. & Instructors 7 2yr.Vendor Supp. 1 19,250 19,250 3,954 10 Total Investment $ 75,785 $ 17,075 41 RECURRENr** Personnel Cluster Support Share 1 $ 1,000 $ 1,000 2 Lab Coordinator Annual Salary 1 4,200 4,200 10 Other Personnel SeNces Annual Salary 1 5,700 5,700 14 Maintenance Equipment Annual Avg. 1 1,000 1,000 2 Software Per Lic. Chg 1 85 85 0 Routine Year 1 100 100 0 Insurance & Theft Contract+$1,000 1 5,000 5,000 12 Training Training Days 12 285 3,420 8 Utilities Year 12 400 4,800 12 Telecommunications Telephone Year 1 100 100 0 Internet Provider Year 0 - - Computer Supplies Year 1 130 130 0 Total Recurrent $ 24,535 59 Total $ 41,610 100 Recurrent Total Cost Per Student (400 students) $ 61 $ 104 % of National Primary Per-Student Recurrent Expenditures (US$) % of National Secondary Per-Student Recurrent Expenditures (US$) Source of Basic Data: IBM de Chile * Estimated costs of 20-station computer laboratory in the municipality of Nunoa provided by IBM de Chile. Long-lived training, facilities and equipment costs are annualized, i.e., presented as average annual costs, using a 10% discount rate and with varying useful lives. Depreciation is included. Does not include marginal costs for classroom teacher time and computer room space, which are assumed to be zero, since the focus of the analsysis is on affordability rather than cost-effectiveness. 14 Education and Technology Series, Vol. 1, No. 3 TABLE 6. Costs of Computer-Assisted Instruction, Laboratory Configuration, in Belize, 1987* (US$) Useful Awrage Invest- Annualized Cost Category Item Life Unit No. of Unit ment or Annual % (Years) Description Units Cost Cost Cost** INVESTMENTk" Facilities Comput. Rm., Renovation 25 Contract 1 $ 474 3 Furniture - Local Area Network Contingency & Other 2% of Facil. 1 9 0 Equipment Server - Computers 6 Unit 10 [1,115]& [11,150]& 2,486 16 Peripherals 6 Set 1 658 4 Power Regulator 6 Unit 1 268 2 Equipment Install., Freight 6 Contract 1 361 2 Contingency & Other 6 5% of Equip. 1 189 1 Subtotal Facilities & Equipment 4,445 29 Software Acquisition Cost Set 1 117 1 Training (Upfront) Lab Teacher - Total Investment $ 4,562 29 RECURRENT - Personnel Support $ - - Lab Teacher Annual Salary 1 9,180 9,180 59 Fringe Benefits Amount 1 172 172 1 Maintenance Equipment Annual Avg. 1 100 100 1 Routine - Insurance & Theft Training - Utilities & Supplies Year 1 1,515 1,515 10 Telecommunications Telephone - Internet Provider - Total Recurrent $ 10,967 71 Total $ 15,529 100 Recurrent Total Cost Per Student (200 students) $ 55 $ 78 % of National Primary Per-Student Recurrent Expenditures (US$) % of National Secondary Per-Student Recurrent Expenditures (US$) Source of Basic Data: M.T. Rock, P.J. Glick and R.V.A. Sprout (1990) * Estimated costs of 10-station computer laboratory at St. Johns College, Belize City, consultant's report for USAID. Cost data for four other schools were similar. Costs presented here are modified for purposes of comparison. ** Long-lived training, facilities and equipment costs are annualized, i.e., presented as aerage annual costs, in source. using a 9% discount rate and with varying useful lives. Depreciation is included. & Annualized cost presented in source using assumptions there would imply cost per computer of only $1,115 per computer. Cost Analysis of Information Technology Projects in Education 15 TABLE 7. Costs of Computer-Assisted Instruction, Classroom Configuration, in the United States, 1993* (US$) Useful Average Invest- Annualized Cost Category Item Life Unit No. of Unit ment or Annual % (Years) Description Units Cost Cost Cost** C OS T. INVESTMENT** Facilities Wiring, Furniture, Etc. 10 School 1 $ 136,000 $ 136,000 $ 18,478 5 Contingency & Other 12 2% of Facil. 1 2,720 324 0 Equipment Computers 5 Unit 160 1,600 256,000 60,773 17 Peripherals 5 Classroom 30 3,000 90,000 21,366 6 Contingency & Other 5 5% of Equip. 1 17,300 4,107 1 Subtotal Facilities & Equipment 502,020 105,049 29 Software Site Licenses 7 Set 1 75,000 75,000 13,435 4 Training (Upfront) Lab Coord. & Instructors 7 Training Day 5 300 1,500 269 0 Teacher Training 7 Program 1 2,000 2,000 358 0 Total Investment $ 580,520 $ 119,111 33 RECURRENT*** Personnel, Computer Cluster Support Share 1 $ 25,000 $ 25,000 7 Technology Coordinator Annual Salary 1 50,000 50,000 14 Personnel, Additional To Allow Teach. Netwrk Time Total Salaries 1 75,000 75,000 21 To Allow Teach. to Use Tech. Total Salaries 1 50,000 50,000 14 Training Formal Staff Devel. Year 1 13,000 13,000 4 Maintenance Equipment Annual Avg. 1 13,000 13,000 4 Routine Year 1 100 100 0 Telecommunications Telephone + Internet Year 1 12,000 12,000 3 Computer Supplies Classroom 30 1,000 30,000 8 Total Recurrent $ 243,100 67 Total $ 362,211 100 Recurrent Cost Per Student (800 students) $ 304 Total Cost Per Student (800 students) $ 453 Source of Basic Data: Means and Olson, 1995 * Estimated costs of hypothetical US School, adapted from Means and Olson,1995, Table 7, p.99. Long-lived training, facilities and equipment costs are annualized, i.e., presented as average annual costs, using a 6% discount rate and with varying useful lives. Depreciation is included. Does not include marginal costs for classroom teacher time and computer room space, which are assumed to be zero, since the focus of the analsysis is on affordability rather than cost-effectiveness. cept in periodic summer workshops. lower-cost alternative to be evaluated against the Internet. Like- wise, LAN capability alone might be evaluated against the combi- 41. Personnel Salaries. Since the emphasis in this cost compari- nation of LAN and WAN capability. son is on the affordability of the programs rather than on their cost- effectiveness, the cost of teacher time (or of laboratory space) has 43. Infrastructural Support and Maintenance. The not been included. The main personnel costs in Jamaica and Chile Infrastructural support for computer facilities is well-developed in are those of the full-time laboratory coordinator and the technical all the countries as are regular maintenance and repair services. support personnel shared with other clusters. Power outages are generally infrequent and thus not an insurmount- able obstacle to the efficient functioning of computer labs. Like- 42. Network and Communications. Both the Jamaica and Chile wise, all countries have regular maintenance and repair services cost data appear to have underestimated the cost of network com- which are mainly covered under standard service agreements for munications, with $750 and $100 per year, respectively. This can equipment. In Aguascalientes, Mexico a core maintenance and be compared with $12,000 per year for US programs. Clearly, repair team functions along side IBM's, complementing and rein- Internet and WAN costs need to be evaluated for cost-effective- forcing services to schools. Belize's Ministry of Education ap- ness. CD-ROM capability, valuable on its own, might be a partial pointed two full-time technicians to provide for routine mainte- 16 Education and Technology Series, Vol. 1, No. 3 nance and minor repairs of the computers in the Belize Teachers 47. Hardware and Software. Given the large costs of facilities, College and the secondary and primary schools. Local dealers also equipment and software, it is clear that effective deployment, pro- provide hardware, parts and servicing in these countries, but expe- curement practices, and payment arrangements are very important rience has been mixed. for cost reduction. Trade restrictions and other practices that pre- vent buyers from obtaining the best international prices are obvi- 44. Property Protection. In many developing countries schools, ously counterproductive. Buyers should also attempt to exploit security is a continuing difficult problem. Many have experienced any tax exemptions that are available. Even pilot projects should incidents where most valuables have been stripped in a single night. attempt to negotiate 'best' prices with vendors, on the argument This is one reason why many such schools have adopted extraordi- that more sales will be likely in the future. nary security precautions, such as having instructors take valuable equipment and tools home every afternoon. The security problem 48. Computer Deployment in Schools. Computer equipment is also one reason why the laboratory configuration has been uni- costs average 25 to 35 percent in most projects. For most of the versally adopted in developing countries, since a single room can countries initiating information technology in education, the pre- be better defended. Clearly, the security problem of computers left ferred option for deploying computers and related hardware has overnight in classrooms is serious, perhaps insurmountable. One been to place them in computer labs or teaching resource centers. alternative that has been given some attention is to have the invest- (With the exception of Chile, which has many small rural schools ment for a classroom configuration made in notebook computers with only a few computers each, all schools in other countries have which would, like the tools mentioned above, be taken home every some 15 to 20 computers in each lab). The lab option has two afternoon by the teachers or locked in some secure storage area at strong arguments in favor of it. The first is that computers in labs, the end of school each day. In any case, property protection and its usually with 15 to 20 machines provide greater access to students costs are not insignificant and need to be evaluated carefully. In at more affordable prices than do computers in classrooms. Sec- the Jamaica and Chile cases, insurance and related costs alone are ond, schools can more easily protect computers from theft and van- estimated at 5% and 9% respectively. In addition, a portion of dalism when they are in one well-protected room rather than spread facilities and equipment cost (e.g., for locks, security cables, and throughout a school building with poor security. Despite the al- extra-strong walls and doors) should logically also be allocated to most exclusive preference for deploying computers in labs, there protection. may be reason to question whether or not this should be the pre- ferred option in all cases. As the cost simulation in Table 8 sug- III. Conclusions and Recommendations: Strategies for Reduc- gests, the deployment of inexpensive, capable and communications- ing Costs and Increasing Benefits ready portable computers in classrooms could prove to be a more cost effective and educationally advantageous option than the de- 45. The conclusions and recommendations of this study point to a ployment of computers in labs. The easy portability of laptop/note- number of ways that countries and schools systems could improve book computers from classroom to classroom would allow their the benefits and reduce the costs of projects to introduce informa- collection at night in secure rooms (or in the possession of teachers tion and communication technologies into education. While not at home) and thus make this deployment a feasible option in schools all countries or schools systems will be able to benefit, there is little with security problems. For these reasons, classroom deployment doubt that they should explore all available options to make tech- of portable computers should be seriously investigated. nology in education more feasible and cost effective. Among these options, one that should be explored in some countries is the con- 49. Gain Market Knowledge. Superior market knowledge tracting out of instruction using computers to private tutors or pri- will have an important role in cost reduction, especially in the next vate training institutions rather than having schools purchase equip- few years as the computer market becomes more differentiated (para. ment and assume responsibility for maintenance and professional 38). The ability to use low-end hardware will also depend on the development. The experiences of the countries and schools we development of the software market. The recent explosion of educa- have examined in this study suggest that some of the options for tional software for today's low-end (486) machines will probably reducing costs and increasing benefits are not always immediately support the continued use of low-price software on cheaper low- obvious. It is only after some passing of time and with good moni- end machines expected in the future. The key tactic for developing toring and assessment, do new and improved ways of maximizing country buyers will be to successfully gain market knowledge of resources become apparent. the opportunities in the more differentiated market. Retaining a high-priced international consultant may result in greater savings Strategies for Reducing Costs in this market than relying on vendor information even from sev- eral vendors. His or her knowledge of the international market 46. The main strategy for reducing costs involves substrategies would, first, be likely to result in significant hardware savings. for reducing the main cost elements: hardware, computer staff sala- Moreover, his or her specialized knowledge might include knowl- ries and teacher training and support. edge of computers that did not require air conditioning and special Cost Analysis of Information Technology Projects in Education 17 protection against dust, resulting in facilities saving, and ones that but generally reflect market conditions. Competition in the United could drive cost-effective software, resulting in software savings. States among computer manufactures has brought leasing prices down to levels where leasing can be an attractive alternative to 50. Effective Procurement Procedures. Effective procurement purchase. practices are very important for cost reduction. The world market for computer technology and software is highly competitive and 52, Efficient Usage of Equipment. Simply saving resources on offers countries and school districts opportunities for obtaining the cost of hardware and software is only half the battle. Reducing benefits ftom competition. Computer manufacturers wishing to the cost per student contact hour or, better, per unit of benefit by penetrate markets for longer-term sales prospects are often willing maximizing usage is the other half. This will mean especially care- to offer lower prices. For this reason, international competitive ful attention to efficient scheduling by school principals and minis- bidding procedures, will yield better prices than more limited local try officials, so that computers are continuously in use by children competition or direct purchasing. Likewise, procurement of equip- during and in many cases outside of school hours. ment and services in bulk (e.g., for more than one school or one district), will produce economies of scale and the consequent price 53. Computer Staff Salaries. Another potential major area for benefits. A two-step process recommended by the World Bank cost containment is in computer staffsalaries, in particular, the full- will bring better benefits than one step, if the system is large and time laboratory coordinator, who is an essential element in the labo- complex. Following a two step process, a government would first ratory configuration and other staff In the laboratory configura- invite companies to submit technical proposals and evaluate them tion, the coordinator manages the operation and instructs both the based on criteria of soundness rather than cost. Bidders are permit- teachers and students with or without paid assistants. When the lab ted to adjust their technical proposals to produce the same perfor- is open to the community, he or she and/or assistants may manage mance. The financial viability of bidders and their capacity to imple- the activity and handle the cost recovery. The main potential say- ment the proposed solutions are also evaluated in this first stage. ing in these salaries would come from using an alternative configu- The second stage takes place after the government has chosen all ration, probably a classroom configuration. In this configuration, qualified bidders and invites them to submit prices for the techni- the absence of the laboratory can allow a different and less expen- cal proposals approved in the first phase. The final selection is sive arrangement for technical and training support. Custodial, based on price. Chile's Enlaces project has followed this approach management and student teaching activities are the responsibility in its procurement of computers and related equipment with the of teachers; technical support likewise becomes the responsibility result that both Apple Computer Corporation and IBM have won of teachers generally, of teachers with specialized training, of tech- on different occasions with clear price advantages for the govern- nicians shared with other schools and of vendor-provided support ment. teams. There is little experience with the classroom configuration in developing countries, however. Minimizing the security prob- 51. Lease vs. Purchase. National governments and local school lem of the classroom configuration will probably require portable districts can obtain help from some of the large computer manufac- computers in many or most instances. Portability may also aid in tures in putting together suitable packages for leasing or purchas- maximizing usage. Developing country experimentation is urgently ing computer hardware and software. Not only may this help in needed to determine whether the cost savings of feasible classroom financing projects that otherwise might be difficult to finance, but configurations are both positive and significant. Investigating the in some (probably a minority of cases) may even result in cost re- relative cost-effectiveness of various laboratory and classroom ap- ductions, when all maintenance, insurance, technical assistance and proaches would also be essential. The extensive experience of de- training costs are considered. The Genesis projects in Chile and veloped countries in the classroom configuration should be ana- Mexico involve leases of equipment and training, backed up with lyzed to see what if any lessons learned can be transferred to the support services from CLIE. Negotiating a lease is sometimes com- developing country context. The high resource commitment, how- plicated by the fact that governments have to commit in advance ever, reduces the usefulness of developed country cost data for de- annual contributions from their recurrent budgets to pay the leases, veloping country analysis (see Table 7). Developed country expe- and that isn't easy for some governments to arrange. On the other rience in how to train and motivate teachers to integrate technol- hand, leases have their distinct advantages in that the companies ogy in classroom activities, however, should be highly relevant to retain ownership of the computers, which obligates them to obtain the third big cost category in the cost structure of educational com- insurance on their computers and to make repairs of the technology puter applications, training and teacher support. if anything goes wrong. Secondly, upon termination of the lease, usually in about five years, the government is free to buy the equip- 54. Training and Teacher Support. This category includes train- ment out right or return it to the seller and obtain upgrades. In an ing of all kinds and released teacher time for teacher support activi- industry where there is constant improvements in technology, be- dE. Teccrrsmcfcrixp is that schools in which exem- ing able to return equipment to the manufacturer has its distinct benefits. The interest rates on leasing vary from country to country Becker, Hank (1993) as cited in Means and Olson (1995). 18 Education and Technology Series, Vol. 1, No. 3 Table 8: Costs of Various Configurations of Educational Computers - Simulation 10-Computers 20-Computers Laboratory Classroom Laboratory Classroom Portable Portable PROGRAM SCOPE No. of Students Served 200 200 200 200 No. of Computers 10 10 20 20 No. of Classes 10 10 10 10 No. of Students Per Class 20 20 20 20 No. Students Per Computer 20 20 10 10 INTENSITY OF COMPUTER USE Average Hours Per Com puter/Week 30 30 30 30 Avg. No. Students Working Together 2 2 2 2 Avg. Student Computer Contact Hrs./Wk. 3.00 3.00 6.00 6.00 Value of Use - Qualitative Adjustment 1.00 1.25 1.00 1.25 Avg. Quality-Adjusted Hours/Wk. 3.00 3.75 6.00 7.50 School Weeks Per Year 30 30 30 30 EXPENDITURES INVESTMENT, ANNUALIZED Facilities -Alteration, Air Con &Wiring 1,000 200 1,300 300 Use of Space - Opportunity Cost 1,900 - 3,800 - Facilities -Communications Wiring 100 500 100 700 Computers & Prepaid Services 5,300 5,800 10,600 11,600 Peripheral Equipment 500 600 1,100 1,200 Software Site Licenses 1,500 1,500 3,000 3,000 Training (Upfront) & Tech. Assist. Contracts 8,000 5,000 10,700 6,700 Total Annualized Investment 18,300 13,600 30,600 23,500 RECURRENT Personnel Services Lab Coordinator 10,000 - 10,000 - Technical Support 3,000 3,000 4,000 4,000 Program Support 3,000 3,000 4,000 4,000 Additional Paid Teacher Time - - - - Other - - - - Subtotal Personnel Services 16,000 6,000 18,000 8,000 Software and Supplies 1,000 1,000 1,300 1,300 Maintenance and Technical Assistance 1,200 1,000 1,600 1,300 Training and Teacher Release Insurance and Other Recurrent 1,700 1,900 3,500 3,800 Total Recurrent 19,900 9,900 24,400 14,400 TOTAL 38,200 23,500 55,000 37,900 Per Student 191 118 275 190 Per Student Contact Hour 2.12 1.31 1.53 1.05 Per Quality-Adjust. Student Contact Hour 2.12 1.04 1.53 0.84 NOTE: Discount rate = 10% Cost Analysis of Information Technology Projects in Education 19 plary computer-using teachers work provide a strong social net- without specification of educational objectives, and thus its impact work of computer-using teachers and receive in-service training in on teaching and learing in schools has failed to meet expectations. computer software. The strong social network not only requires Likewise, educators need to have realistic expectations regarding equivalently strong school leadership but also an allocation of time the contributions to education which can be made by information when teachers can interact with each other outside of their class- technology. Viewing information technology as a valuable knowl- rooms and also use computers. Doing this at the level costed in edge extension and instructional tool for use throughout the cur- Table 7 would obviously use significant resources both for formal riculum will enable teachers to employ many important applica- training activities and also for substitute teachers to cover for teach- tions to enrich teaching and could provide a powerful impetus to ers having released time to confer with colleagues and experiment new forms of self-leamiig, cooperative learning, problem solving with computer applications and methods. Rather than seeing train- and higher-order thinking skills among their students. ing and support activities as a source of savings, therefore, the im- perative may be to use savings on hardware, software and person- 58. Intensity of Computer Usage. The is little firm knowledge nel to efficiently augment teacher training. about what minimum weekly (or monthly) computer usage is nec- essary to obtain commensurate educational benefits. Judgments as 55. Interaction of Training with Cost Reduction Opportuni- to the minimum access required per student vary from one to three ties. It will also be necessary to have effectively trained teachers in hours per week, depending upon how computers are used for in- order to obtain the savings in these cost categories. Having trained structional purposes. In Mexico, Chile and Costa Rica, students teachers will help to maximizing the utilization of the hardware share computers from two to three hours per week. In Jamaica, the even in the laboratory configuration so as to reduce the cost of goal has been to provide students up to three hours per week of student contact hours. Certainly this will be the case in the class- CAI. Even the goal of two hours per week of access is beyond the room configuration. Effective, committed teachers will be even means of a good many education systems, at least initially. Thus more essential in maximizing the educational effectiveness and the they start with less, as has been the case of projects such as the one cost-effectiveness of technology applications. These interactions in Puebla, Mexico. However, after two years of operation, Puebla together with the large cost of training and teacher support will has plans to double the number of computers to enable students to make these program aspects hardest of all areas to get right. Exten- have at least two hours of computer time weekly. Whatever the sive experimentation with different models of training and teacher minimum usage decided on, it may be necessary in order to achieve support is therefore indicated. this level for schools to limit access to certain grade levels, is done in Rufloa and Aguascalientes. 56. Piloting Innovations. One useful strategy for identifying cost reduction possibilities as well as efficiencies, is to undertake pilot 59. Invest in Professional Development Teachers are the key programs before launching large-scale initiatives. Although this is change agents in education and play the crucial role in the deploy- rather obvious, educational policy makers and administrators are ment of information technology in the classroom. Computers are a reluctant to undertake pilot projects because they often prove diffi- powerful instructional tool and will become more so. But their cult to replicate on a larger scale. Pilot programs are also unattrac- potential will not be realized without teachers training in their use. tive to officials who are looking for urgent responses to large-scale There is an increasing need to give considerably more attention to problems. Nonetheless, Chile has demonstrated the value of pilot the training of teachers in using computers effectively' in the class- projects in planning its national computer network project, Enlaces. room. Where feasible, teachers need to learn to use computers in During the past three years, Enlaces gained valuable experience their teaching when they are first trained. However, in-service train- and cost data to confirm the viability of it project concept, growing ing, combined with support in the classroom, enables practicing from an initial 14 schools in southern Chile to over 200 primary teachers to acquire the necessary knowledge and skills in computer and secondary schools nationwide. The pilot phase has helped de- use for teaching. Effective professional development will enhance fine hardware and software requirements, cost effective training investments, while the failure to invest will almost certainly limit solutions, and overall operating expenses of the network. While the effectiveness of computers in achieving instructional objectives. Enlaces is now a program of national scope, it continues to pilot The cost analysis of professional development in projects also in- different approaches such as the use of the Internet and the World dicates the need for governments to incorporate funding in their Wide Web in selected schools. recurrent budgets to provide teachers regular training opportuni- ties to upgrade their computer skills and familiarize themselves with Strategies for Increasing Benefits. the latest software on the market for teaching in their disciplines. 57. Defining Instructional Objectives. The introduction of 60. Software. The selection of good educational software is criti- information and communications technology into education should cal for maximizing the instructional benefits of information tech- be guided by clear and relevant educational objectives. Experi- nology in schools. Nevertheless, teachers and administrators are ence indicates that quite often, technology has been introduced generally handicapped in making informed judgments as to the best 20 Education and Technology Series, Vol. 1, No. 3 software to support the learning outcomes in their curriculum. This Enlaces. 1994. Informatica Educativa Principios Y Us0s. handicap can be overcome in at least two ways. One way is to invite publishers to submit software for review by curriculum spe- Gates, Bill. 1995. The Road Ahead, Viking Press. cialists and teachers before making selections and purchases. This may not be widespread industry practice, but should be pursued Glick, Peter J. Cost Analysis: Categories and Procedures Unpub- with those publishers willing to do so. A second way is to consult lished. publications which review educational software. Two such guides for the U.S. market are High/Scope's Buyer's Guide to Children's Hawkridge, David., Jaworski, John., McMahon, Harry. 1990: Coi- Softwar and the ASCD's Only the Best: The Annual Guide To puters in Third-World Schools: Examples. Experience and Is- the Highest Rated Educational Software and Multimedia. Since s Mac Millan, London. producing quality software is costly and technically difficult, few companies in developing countries would be inclined to invest in Jamison, Dean T. Klees, Steven J. and Wells, Stuart J. 1978. The new software publishing for their national markets. However, some Costs of Educational Media: Guidenlines for Planning and firms are contracting with software publishers to translate and adapt Evaluation. Sage Publications. exiting software code for use in different countries. Within IBM Corporation, there is an agreement which permits any geographic Levin, H. 1975. Cost Effectiveness: A Primer. Sage Publications region of the company to have access to IBM software code for translations. These type of arrangements offer countries the op- Lockheed, Marlaine E., Middleton, John, Nettleton, Greta S. (Eds.), portunity to take advantage of other major up front software in- 1991. Educational Technology: Sustainable and Effective Use vestments. Another option is for teachers to develop some of their The World Bank. Washington, D.C. own instructional materials, using one of the many authoring soft- ware packages on the market. Hypercard has been the most popu- Means, Barbara and Kerry Olsen. September, 1995. Technology's lar software among U.S. educators, but there are now other pack- Role in Education Reform. Prepared for: Office of Educa- ages including ones which accommodate multimedia formats. tional Research and Improvement, U.S. Department of Educa- tion. SRI International. 61. Providing Benefits to the Community At-Large. Programs to introduce computers into schools generally receive strong sup- Papert, Seymour, 1980. Mindstorms. Basic Books. port from the school community. While parents might be reluctant to support other school programs, they are usually willing to con- Perelman, Lewis J. "Closing Education's Gap" in Kearsley, Fred tribute materials and labor for computers. While community mem- and Lynch, William (Eds.), 1994. Educational Technology bers view school computers mainly as instructional tools, many Leadership Perspectives. Education Technology Publications. quickly appreciate the benefits which computers offer to business and local government. Thus, in many communities in Mexico, Chile "Primer informe anual de las clases de computacion 1994-95 en las and Costa Rica where computers are installed in schools, commu- preparatorias de la BUAP," 30 de Agosto de 1995, Puebla, nity members have obtained access to them in the evenings, week- Mexico. Unpublished. ends, and other times when they are not being used for educational purposes. Using computers in this way, further maximizes their Rock, Michael T., Glick, Peter J., Sprout, V.A. "A Cost Analysis benefits although not exclusively for educational purposes. On the of Computer Instruction in Belize," 1990. Upublished. other hand, the benefits are reflected in tangible ways such as the user fees charged for the community use of computers, an increas- Seminario-Taller Nacional de Reflexion sobre Politica en ingly common practice. Informatica Educativa Memoria, 1994- Editorial Universidad Estatal a Distancia San Jose, Costa Rica References Vickers, Margaret and Smalley, June. "Integrating Computers into Anzalone, S., 1991. "Educatic.- Technology and the Improve- Classroom Teaching: Cross-National Perspectives. In Perkins, ment of General Educat i Developing Countries". In David N., Judah L. Schwartz, West, Mary Maxwell, Stone Lockheed, M.E., Middleto,. 4ettleton, Greta.S. (Eds.), Edu- Wiske, Martha.(Eds.) 1995 Software Goes to School. Oxford cational Technology: Sustai1able and Effective Use, The World University Press. Bank, Washington, D.C. World Bank. 1995. Priorities and Strategies for Education A World Badilla Saxe, Eleonora. 1995. "Evaluacion Formativa del Programa Bank Review. Washington, D.C. de Informatica Educativa Ministerio de Educacion Publica Fundacion Omar Dengo Convenio Bid/Fod." Unpublished.