PHREE Background Paper Series Document No. PHREE/91/32 Educational Technology: Sustainable and Effective Use " Marlaine E. Lockheed John Middleton Greta S. Nettleton Editors Education and Employment Division Population and Human Resources Department The World Bank January 1991 t This publication senes serves as an ouwet for background products from the ongoing work progam of policy research and analysis of the Education and Employment Division in the Populadon and Human Resources Department of the World Bant 7he views expressed arc thosc of the author(s), and should not be atozbuted to the World Bank A -. 0 3 4 * The International Bank for Reconstsction and Development/ The World Bank, 1991 Educational Technology: Sustainable and Effective Use Marlaine E. Lockheed, John Middleton, Greta S. Nettleton Editors Table of Contents Page Overview Chapter 1: Educational Technology: Towards Appropriate and Sustainable Use ... ....................................... 1 Marlaine E. Lockheed and John Middleton Research Reviews Chapter 2: Educational Technology and the Improvement of General Education in Developing Countries .31 Stephen Anzalone Chapter 3: Uses and Costs of Educational Technology for Distance Education in Developing Countries: A Review of the Recent Literature .98 Greta S. Nettleton Chapter 4: Uses and Costs of Educational Technology for Vocational Training: Current Research in Canada and the United States, with Implications for Developing Countries ........ ........... 148 Anna Stahmer, Audrey Mehler, Ingrid Bryan and Murray Richmond Case Studies Chapter 5: Pilot Projects in Educational Technology: The Philippine Case ........ 217 John Middleton Chapter 6: A Comparison of Open Universities in Thailand and Indonesia... 245 David N. Wilson Chapter 7: Computer Applications in Vocational Training in Norway . 272 Arvid Staupe Chapter 8: Sustainability in Four Interactive Radio Projects: Bolivia, Honduras, Lesotho, Papua New Guinea . 293 Thomas D. Tilson Educational Technology: Chapter 1 Chapter 1 Educational Technology: Towards Appropriate and Sustainable Use Marlaine E. Lockheed john Middleton - 1 - I Educational Technology: Chapter 1 Chapter 1: Educational Technology: Towards Appropriate and Sustainable Use Marlaine E. Lockheed and John Middleton I. Introduction Rethinking Technology The potential of educational technologies fc,r improving education and training effectiveness, increasing educational access, and reducing costs, has been the subject of considerable debate, research and experimentation in developing countries for the lact two decades. Two broad observations may be drawn from this work. First, a variety of technologies are educationally effective; both children and adults learn through technologies, often at modest cost. Second, despite evidence of effectiveness and the existence of successful, sustained applications in many developing countries, educational technologies have not been as widely adopted as early experimenters had hoped twenty years ago. One reason for this lack of implementation is that, while technology has provided effective educational options when implemented under the right conditions, little attention has been paid to identifying the "right" conditions. Thus, a new emphasis on conditions for sustained, effective use of technologies is needed. Many experimental implementations in developing countries have emphasized technological elements, while neglecting fundamental educational 'ssues. As a result, many technology projects have failed. These failures have brought disillusionment about what technology can accomplish in developing countries. Major assistance agencies, including the World Bank and USAID, have reduced support for technology applications in education, giving much greater priority to improving effectiveness by strengthening the basic elements of schooling, especially at the primary level. In fact, in the last decade, the major instrument for school improvement has been textbook distribution, accompanied by teacher training, rather than any directly "technological" means. However, over the last ten to fifteen years, the world has not been static. Development has continued apace, and a number of factors now support a reassessment of using technology in education. In particular, four factors should be noted. First, general education at all levels remains ineffective and inefficient, and access to secondary and tertiary education is emerging as a problem in many countries. Budget constraints have limited efforts both to improve effectiveness and to expand the system. The effectiveness and efficiency of training are similarly constrained. These realities have fueled a continuing search for alternative ways to improve effectiveness and access within tight cost parameters. In countries as diverse as Pakistan, Thailand, Malawi and the Dominican Republic, these alternatives have included the use of technology. -2- Educational Technology: Chapter 1 Second, both the managerial and technical capacity for implementing educational innovations has increased in many developing countries. In addition, the infrastructure necessary for using many technologies has been strengthened. Electricity supply, broadcasting, printing and publishing have been improved; cheap and reliable radios have become widely available, and audio cassette technologies can be found in many remote areas. Even video cassettes and video cassette recorders (VCRs) are widely available in some developing countries. Third, technologies themselves are changing: simple technologies have become ubiquitous, while complex technologies have become simpler. The spread of cassette teclnology, for example, allows audio and video cassettes to be used in place of broadcasting, which increases flexibility in audio and video materials use. Print-based technologies, such as textbooks and programmed teaching and learning systems, have benefitted from advances in word processing and desk-top publishing. And a new family of learning technologies based on microcomputers has emerged; these technologies (computer-managed and computer-assisted instruction, interactive videodisc) have been shown to provide effective instruction. Finally, two decades of pilot projects cnd research he."e yielded substantial information about both the theoretical aspects of usir. technology for education and about the technical particulars of implementation. Particularly for distance learning, technology is beginning to be accepted and relied on by policymakers themselves, and has established itself as a functional policy alternative The knowledge gained from these pilot projects ard research efforts has yielded a more focused and practical conceptualization of the appropriate role of technology in the educational process. These programs provide an important experience base for olanning future investments. Purpose and Scope of the Review After more than twenty years of experimentation and sustained use of educational technology, it is time to synthesize this experience into a coherent analytical framework for understanding the conditions of cost- effectiveness and sustainability. What are those key ingredients that allow a new technology to "catch on"? What leads teachers, students, and administrators to adopt the innovation as useful -- and even indispensable -- in their everyday school activities? It is clear that far more than simple educational effectiveness is involved. This book reviews what has been learned from accumulated experience with educational technology. The first chapter summarizes and integrates the lessons that emerge from the subsequent three chapters, each of which surveys a major education area where technology has been used -- general education, distance education, and technical/vocational training. Based on the evidence presented in the survey chapters, a conceptual framework is developed for understanding how theoretical approaches to educational technology have moved into the real world. The book pinpoints key factors enabling educational technology to be applied successfully and to be sustained, as well as key obstacles which have proved fatal to many efforts. -3- Fducational Technology: Chapter 1 The survey chapters are: Stephen Anzalone's chapter on applications of educational technology in general education, mainly at the primary level, (Chapter 2: Educational Technology and the Improvement of General Education In Developing Countries); Greta Nettleton's review of experience with e'.stance education (Chapter 3: Uses and Costs of Educational Technology for uistance Education in Developing Countries); and a chapter on technical and vocational training contributed by Anna Stahmer and colleagues (Chapter 4: Uses and Costs of Educational Technology for Vocational Trairing). Chapters 5, 6, 7 and 8 present case studies of a pilot program of radio education in the Philippines, distance universities in Asia, vocational technical training in Norway, and interactive radio in four countries. The chapters on general and distance education are based on comprehensive literature reviews of research and implementation experience for technological applications in developing countries. Since the research literature on technology for technical and vocational training in developing countries is scarce, Chapter 4 addresses this topic through a series of case studies of enterprises and post-secondary technical education institutions in Cianada and the United States. Overall, the book covers technology applications in more than 50 developed and developing countries. Technological Alternatives to Schooling The variety of technologies reviewed is presented in Figure 1. Print remains the dominant technology, followed by audio cassettes. In developing countries, radio has become the broadcast medium of choice, both for general education and distance learning, because of its low cost and flexibility. Broadcast television, after a number of failed experiments, is used only rarely in developing countries, with a few important exceptions. Telecommunications networks are used to distribute technical and engineering courses in Canada and the US, but so far have had only limited applications in the developing world (West Indies, Indonesia). Use of the more advanced technologies (videodisc, interactive video, microcomputers) is growing rapidly in Canada and the US, Europe and Japan, but is only beginning in a few developing countries on an experimental basis. Use of various other educational "technologies" -- such as pictures, charts, blackboards, or slide projectors -- has received very little policy or research attention. Yet increasing the use of any or all of these educational tools is a policy option that should not be ignored. These relatively "low-tech" tools may have a significant impact, particularly in low-income countries or more impoverished schools where, for example, children might be trying to learn to write without pencils, paper, slates or chalk. The existence of more advanced technologies should not cause policymakers to overlook the commonsense alternative of providing basic learning materials, less glamorous but fundamental to the learning process. -4- Educational Te'hnology: Chapter 1 Figure 1: Technologies Review. Educational Application Vocational General Distance & Technical Technology Education Education Training Print Textbooks X X Other X X Audio-V>-ual Aides X X Prog- X Learn:v X RadiG Interactive X Other X X Audio Cassette X X X Broadcast Television X X X Video Cassette X X X Video Disk X X Micro-Computers X X X Interactive Video Disk X X Electronic Aides X Simulations X ulecommunications Distrihution X X X Tele-conferencing X Interest in technology is greatest when it is embedded in large- scale programs to provide alternatives to traditional schooling. Research and experimentation focused on such programs have been directed at four important policy goals: to improve the effectiveness of education, at affordable marginal costs; to improve efficiency through lower-cost alternatives to conventional instruction; to extend access to those shut out of the educational process by geography, economics, or social status; and to transform education technologically in order to make it more compatible with an information-based society. The first three objectives have been addressed -5- Educational Technology: Chapter 1 through policy and research. Since the fourth is still largely theoretical, even utopian, it will not be addressed in this book. Cost considerations lie at the heart of these goals. Most policymakers, given no resource constraints, would choose good traditional schooling for all students as opposed to technology-based alternatives. It is the economic difficulty of providing good schooling for all, at all levels, that keeps alive the idea of technologically based alternatives. However, the rationale for using educational technologies is different for developed and developing countries. In developed countries, which have more effective schooling and enjoy high enrollment levels, technologies are used to achieve marginal improvements in effectiveness at lower marginal costs than would be possible through traditional methods. In developing countries, on the other hand, where good schools are affordable only for a relative few, the search is for alternatives to make significant improvements in educational effectiveness, while at the same time increasing access to education, particularly at the secondary and tertiary levels, at lower cost. Elements of Sustainability Reflecting the policy goals described above, the studies reviewed in this survey of educational technology applications generally have emphasized both cost reduction and educational effectiveness. The implicit assumption has been that if a technology taught well and/or saved money, its implementation would follow more or less automatically. What actually has occurred is that while overly expensive or ineffective projects indeed have foundered as one might expect, programs with good results have also met with difficulties in becoming established, particularly in the area of general, in- school education. Research that goes no further than investigating whether a particular innovation is educationally effective does not provide enough information for developing policy. After all, past research has shown that nearly any technology can be effective, under the right circumstances and with sufficient resources available. What should be investigated now is how technology can be practical and usable in real life situations. Pure research on the educational effectiveness of a technology is a necessary starting place, but we need to know more in order to ensure the sustainability of the technology. A second analytical pitfall which must be avoided is generalizing about a particular technology without taking into account the fact that the technology teaches only as part of a complete system of courseware and instructional process. Thus, theoretical approaches must de-emphasize the importance of technological tools per se and focus instead on how specific institutions have used technologies to solve specific problems in ways that last. A generalization about a particular technology is not very useful to policymakers when it fails to consider the specific factors of instructional design and institutional context that determine success and sustainability -- or failure. In any realistic conceptual model for educational technology applications, therefore, factors related to sustainability are as important as relative costs and educational effectiveness. The conceptual framework -6- Educational Technology: Chapter 1 proposed in this chapter is organized around a series of issues whieh, taken together, comprise a basis for an integrated analysis of the sustainability of educational technology applications, as well as of their cost-effectiveness. The three major issues are: (i) the "fit" between an educational problem and a potential technology application, in terms of task definition, pedagogical congruence, and economic feasibility (cost- effectiveness); (ii) barriers to change, particularly in terms of the technological environme[t, administrative capacity, and politica' receptivity; and (iii) real and lasting changes in the target education system as a result of the implementation. In considering these issues, we draw on the evidence of what has been found to work and not work, as reported in the three survey chapters. To be successi.l and sustainable, technology does not have to be implemented in a large-scale project; proliferation of small-sccle projects in a country can result in an overall high level of use. For example, different radio frequencies might serve the naeds of several different projects throughout a particular country, from agricultural extension, to lectures for a distance university, to various supplementary broadcasts for schools at the primary and secondary levels. In fact, a technology may be more sustainable when it complements other educational goals than when the purpose is simply to add the technology across the board t1sroughout the school system. In the case where an educational innovation is transferred from one country to another, success can only be evaluated by specifying exactly what is being transferred. Is it a piece of equipment, or a curriculum based on a particular typ' of equipment? Or is it an entire institutional model to provide education, such as a distance learning university? A first transfer is likely to be the equipment itself -- printing presses, radio or television broadcast facilities, cassette players, and so forth. Equipment is relatively easy '- transfer. Recent experience with interactive mathematics and second lri,,uage courses suggests that curricula also can be transferred if sufficient attention is given to adaptation to the receiving educational context. In transferring an edr --tional innovation, what is the most difficult -- and the most challenging Lo educational planners -- is developing the institutional frameworks needed to sustain the innovation. When elements of a particular curriculum are adopted without continuing institutional patterns and linkages to the mother program, this transfer may be seen by some observers as the ultimate success in adaptation. But others may consider it a failure when the original material loses its -7 - Educational Technology: Chaptet 1 inat'.tutional identity and is transformed into a larger, ..,going program, often run very differently.1' 1., Congruence of Tasks and iools Defining the Task The first basic question to ask in planning for educational technology implementations is: What are the specific pedagogical problems that the educationa'l technology is expected to solve? Thi tes6..orn should be asked from a broad pclicy viewpoint that takes into accoua_ .he ev'erall situation of the educational sector. Where are the bottlenecks? '.,ere can a given amount of resources be applied to best advantage? How can technology help in this process? The initial definition of the problem makes it possible to determine the scale and degree of innovation of the planned implementation (Rondinelli, Middleton and Verspoor, 1990). Innovation is the most important factor. What is the scope of the program? Is it to provide the curriculum for an entire level of education, as in the case of the major distance universitLes in Asia, such as China's Radio and Television University system? Or is it intended for supplementary enrichment in a few subject areas, like Thailand's primary school radio broadcasts in music, culture and other subjects? How much does the program depart from current practice? Is it a new delivery system only, or does it require new curricula, teachin'g/learning activities, and roles for teachers and headmasters? Are new managerial and professional skills needed? The extent of need for feasibility testing affects directly the scale of the project. It is often the plan to vary the scale greatly over time, as in the very common case of starting with a small pilot program to test feasibility and effectiveness and then expanding to serve a much larger portion of the potential clientele. Scale variations do not alter the basic relationship between the technology and the pedagogical problem -- that is, for example, interactive radio "works" as well for one class or a thousand -- though scale does affect other important considerations such as unit costs. In the early days of educational technology planning, each new technology was seen as a gener-' solution to a broad array of educational problems. This optimistic exf -;tation has given way to a more realistic recognition that a given technology or combination of technologies is just one policy alternative. Rather than displacing conventional instruction, an educational technology generally is designed to fulfill a specific, rather narrowly defined purpose. As alternatives, technologies are appropriat- for I/ This was the case in the adaptation of interactive radio IRI mathematics courses for use in Thailand's primary school radio broadcasts (see Chapter Two for complete description of IRI projects). The characteristic instructional structures of the original program have been greatly changed, although some of the original curriculum materials continue to be used. Some analysts even question whether the adaptation can still be labelled as IRI, according to the strict definition of the term. -8- Educational Technology: Chapter 1 certain tasks under certain conditions, but not for every educational purpose and context. The bpecified ;bjectives of technology projects have also evolved with experience. Early efforts to use technology to lower unit costs, such as the well-known primary school television schemes of the 1970s in the Ivory Coast, El Salvador, and Niger, were not successiul. Planners now use technologies to try to improve the effectiveness of education or to in.crease access, with the expectation that technology can help to contain costs rather than to reduce them substantially below the existing level. In many instances, technology options will actually add costs to an education system, but so would more traditional options for improving effectiveness. In other cases, technology may allow the creation -r new programs from the ground up for less cost than would be required to establish traditional institutions. Technology also may bring pedagogic and administrative benefits that outweigh the disadvantage of greater capital expenditures. Apart from these broAd generalizations, different specific kinds of problems are addressed In each of the three areas (general education, distance learning, and vocational training), and as might be expected, different technology uses have developed for each area. General educs&.ion. With respect to general schooling, technology has been found to prov e additional wayc to present and deliver teaching and learning activities. In some instances, these alternative methods offer advantages over existing methods, though they are unlikely to provide a basis for comprehensive change in the school system as a whole. A new technology may be adopted simply because it will improve educational effectiveness for a smaller marginal cost increase than other policy options. A technological innovation may also provide equivalent or better instruction at the same cost in some alternative setting. Reviews of cost-effectiveness of alternative educational inputs have found that technological interventions, such as interactive radio, are often more cost-effective than otler educational materials (Lockheed and Hanushek, 1988), as shown in Table 1. Other objectives for educational technology include reaching droports, enriching curriculum, and teacher training. Most successful implementations in the gen.eral education area have been at the primary level, reflecting the priority which basic schooling typically has received in deveioping countries struggling towards providing universal primary education. Under these circumstances, the emphasis has been on development of technology applications that would be suitable for large- scale use, with low levels of infrastructure development, and for use by teachers with inadequate training. The most common application of technology has been to substitute for or to supplement existing teacher expertise. As developing countries over the last two decades have made strenuous efforts to expand the capacity of primary education, the quality of educational inputs has often suffered (Lockheed and Verspoor, 1990). rarticularly affected have been instructional materials, the -9 Educational Technology: Chapter 1 Table 1: Efficiency of Six Educational Policies Effect Cost Size Per-Student Efficiency Textbooks Brazil .34 $1.65 .21/$i Nicaragua .36 $1.75 .21/$1 Philippines .40 $ .27 1.48/$1 Thailand .06 $ .25 .24/$1 Radio education Kenya .53 $ .40 1.33/$1 Nicaragua .55 $1.80 .31/$1 Thailand (Northeast) .58 $ .44 1.31/$1 Teacher education Brazil (4 yrs primary) .21 $2.21 .09/$1 Brazil (Logos II) .09 $1.84 .05/$1 Brazil (3 yrs secondary) .16 $5.55 .03/$1 Thailand (additional semester postsecondary) <.01 $ .09 .06/$1 Technical-Vocational Secondary Colombia (INEM) .39 $ 98.00 .40/$100 Colombia (tech-voc.) .33 $376.00 .09/$100 Tanzania (commercial) .50 $272.00 .18/$100 Tanzania (technical) -.37 $561.00 -.07/$100 Tanzania (agricultural) -.20 $375.00 -.05/$100 Cross-Age Peer Tutoring United States .73 $212.00 .34/$100 Cooperative Learning Israel 1.40 $ 85.00 1.65/$100 Source: Lockheed and Hanushek, 1988. level of teacher expertise and knowledge, and the amount and use of instructional time. Conventional (i.e., non-technological) approaches to improving teaching effectiveness have emphasized teacher education and in-service training.V Where well implemented, these interventions have been effective. Textbook interventions, focusing on increased provision of better quality I/ See next section for discussion of uses of technology for teacher training, through distance education - 10 - Educational Technology: Chapter 1 texts, have also been very successful; this kind of innovation shares many organizational and economic characteristics with new technology projects. Less frequent, but becoming more important, have been investments in improved school management that seek better utilization of all school resources -- teachers, materials, student time. Despite these measures, the effectiveness of existing teachers often cannot be improved rapidly enough, and the alternative of technological solutions may be useful. In a number of instances, technological interventions to substitute for or supplement teacher skills have been successful in enhancing student learning. Unlike teacher training, the principal focus in these interventions is not to increase teacher knowledge and expertise, but to supplement or substitute for teacher knowledge and teaching skill through technology-based delivery of the educational curriculum. Incidental learning by teachers is a common side benefit, but it is not the principal objective. In the case of interactive radio instruction (IRI) and programmed learningW, the instructional design has emphasized supplementing teacher knowledge and skills in the presentation of core curriculum in certain key subjects. Specific goals have been set for these programs. In the case of IRI in Nicaragua and Kenya, for example, the effort was limited to those subjects that teachers found most difficult to teach, mathematics and English (a second language in Kenya) and was limited to two or three grades. The goals of programmed learning projects have been broader and have spanned more subjects. Major applications of programmed teaching/learning in developing countries, including the Philippines, Indonesia, Thailand, Bangladesh, and Liberia, have spanned a wide range of subjects (Anzalone). In some cases, the core curriculum has been taught through programmed materials. These programmed learning projects have also been able to decrease the actual number of teachers needed, which is another way of reducing the shortage of teachers with the necessary expertise. Projects like those listed above have been able to operate with teacher/student ratios ranging from 50 to 1 to over 100 to 1 (Anzalone). In both IRI and programmed learning, teacher support has been provided by giving the instructor a structured curriculum, pre-designed pupil / In interactive radio instruction, there is a high degree of correlation between instructional materials and delivery strategy. Specific instructional principles applied to radio lessons and supporting printed materials include: active and frequent pupil responses, immediate reinforcement, and distributed practice with "segmented" structure (about a dozen topics per lesson). Programmed teaching/learning systems may be presented by teachers, by printed materials, or by computers (not common in developing countries). Programmed teaching modules are step-by-step scripts for a teacher to present a lesson, including the eliciting of student responses and providing of corrective feedback. Programmed learning materials are instructional materials presented in small segments for use in self-instruction, peer learning groups, and cross-age tutoring. - 11 - Educational Technology: Chapter 1 drills and learning exercises, and subject content already prepared in advance, for use in the traditional face to face classroom situation. Of all the technologies used at the primary level so far, only programmed learning has presented effectively an entire grade-level curriculum. In some cases, programmed learning requires a change in classroom management, to allow for small group study and other innovative organizational schemes. Other technology applications used in general education have less impact on traditional teacher roles and instructional practice. Increased textbook provision, the most common application to improve school effectiveness at this level, is also least likely to affect traditional teacher roles, although it may be related to comprehensive curriculum reform. Where supplementary use of radio broadcasts and audio cassettes occurs in general education (frequently for areas such as music where sound plays a vital role in the learning process), this enrichment has little or no impact on the curriculum or on teaching practices. Distance education. Most distance education implementations serve older students, since it is difficult for a distance learning program to provide the discipline, supervision and motivation that young children require in the learning process. Distance learning programs tend to be based in newly created entities, institutionally separate from the regular school system, although leading to equivalent certification. This makes implementation of new technologies easier, since they are used in a brand new context, rather than trying being added to a pre-existing school situation. Distance learning technologies are chosen for a number of reasons. The use of distance learning may be directed at improving educational equity - - to serve students whose access to traditional learning is constrained because they live in remote areas or are barred from attending regular institutions by cultural, gender, or economic barriers. An example of this is the RADECO project in the Dominican Republic, where radio schools using paraprofessional monitors have been established in remote communities previously without primary schools. Students receive interactive radio lessons in mathematics and language for one hour per day (Anzalone). In Pakistan, sociocultural restrictions requiring women's seclusion are very strong, particularly in rural areas, and distance education was Identified as the only way to reach these potential students. This is an important objective of Allama Iqbal Open University (AIOU). The University uses cassettes and diagrams for illiterate learners, while formal coursework is accomplished through correspondence packages, radio, television, and face- to-face tutorials at study centers (Nettleton). Another rationale for distance learning technologies is saving money. Efforts to lower unit costs by using distance methods to create new, complementary education institutions have been very successful. In the Dominican Republic, Nettleton reports, the RADECO project has achieved per- student cost reductions of approximately 50% over formal public school costs, and the CENAPEC project, using printed materials and optional tutorials to provide secondary school equivalency, costs about half as much per student as traditional schools. In Brazil, a print and tutorial-based in-service teacher training system called Logos II was six times more cost-effective than traditional, residential teacher training methods (Nettleton). The costs of - 12 - Educational Technology: Chapter 1 using programmed teaching/learning systems in the Philippines was half that of conventional instruction (Anzalone). Many large and well-established distance education institutions are able to provide acceptable educational effectiveness at a unit cost that is much lower than the cost of equivalent traditional institutions. Among these are Thailand's STOU, Pakistan's AIOU, China's CRTVU, and Korea's KACU. The main goal for most distance education programs continues to be increasing access to secondary and higher education where demand exceeds supply of spaces. Certainly this is a key factor in the explosive growth of large distance universities in Asia and Latin America, several of which now have enrollments numbering in the hundreds of thousands. These institutions have been developed primarily to help satisfy the enormous demand for access to traditional higher education, which cannot be sufficiently expanded for economic and practical reasons. The instructional design required by distance learning is completely different from that used in traditional learning. Whether the technology relies primarily on printed learning modules, makes use of audio and video tapes, or incorporates supplementary broadcasts of course material by radio or television, the basic instructional problem is the same. The subject material must be explained clearly and completely by the materials provided, since students cannot easily ask for clarifications. Apart from special tutoring centers or study groups, which cannot always be arranged, students must be able to follow the material entirely on their own. In addition, motivation must be maintained, feedback on progress and testing must be provided by distant tutors by mail or other means, and testing and certification procedures need to be administered at regular intervals. To deliver effective instruction at a distance requires the cooperation of a number of individuals with specialized roles. The original content is provided by the subject matter specialist, who is usually a top- ranked teacher or professor in his/her field. Instructional designers specializing in the distance learning field help adapt this material to the media that have been chosen for teaching purposes, usually based on printed learning modules plus other supplements, such as videotaped lectures. In addition, tutors supply the necessary manpower to give feedback to the many distance learners enrolled in the courses, through marking papers, grading tests, and in some instances, answering questions about the content. This system makes it possible for the knowledge and expertise of a few outstanding teachers and other specialists to benefit a far greater number of students than would be possible through face-to-face teaching alone. Vocational training. Shortages of qualified instructors have limited the effectiveness of public technical and vocational training in developing countries and in industrialized countries. In the industrialized countries, post-secondary institutions have been able to offer higher wages to attract the required manpower, and firms and factories have been able to provide extensive in-house technical training programs. These responses have significantly reduced the shortage of qualified teachers in industrialized countries and consequently have reduced the incentive to find ways of using technology to compensate for inadequacies in the teaching force. Thus, while developing countries may have reason to use technology to substitute for - 13 - Educational Technology: Chapter 1 instructor expertise, there is relatively little experience .In developed countries that could serve as a model for doing so. Interactive technologies, such as computer-assisted learning and interactive videodisc, have shown the most promise in enabling learners to master the cognitive and psychomotor skills required for complex, multi-step technical tasks. The hardware and software costs of these technologies are high. For developing countries, these technologies are most economically efficient as substitutes for direct instruction when qualified instructors are very costly, and when high utilization rates reduce unit costs. Recently, industrialized countries have increased the use of technology to substitute for instructor time, in order to make the most of instructor resources and to extend the benefit of instructor expertise to larger numbers of students. However, experience with technology applications remains relatively limited in all countries. Where implementations have occurred, the main purpose often has been to increase the flexibility of learning and scheduling. In today's rapidly changing technology environment, flexibility in curriculum design is highly desirable. In addition, when students can complete their work on an individualized schedule, both the institution and the learner benefit. In post-secondary vocational training, the priority on flexibility can be seen in the use of a variety of self-paced, open entry/exit systems. Not only do such programs contribute to efficient use of scarce resources (instructors). They also allow institutions to reduce unit costs per graduate because of shorter completion time, and because completers can be replaced immediately, rather than having to wait for the beginning of the next semester. Generally, applications of technology in the vocational/technical area are focused on the development of these self-paced courses, rather than being added to existing, classroom-based curricula. In one of the few documented instances where technology-based and self-paced learning has been introduced in a vocational school in a developing country, the SENAI school in Rio de Janeiro, Brazil, students learned more rapidly, and the technology resulted in more complete use of facilities. Students took less time to complete the same amount of coursework, with time reductions ranging from 45.7% in mechanics courses to 76.1% in electronics courses. In addition, completers were replaced immediately during the ongoing courses, keeping the school operating at full capacity the entire year (Oliveira and Castro, 1988). This model has since been transferred to other locations in Brazil, and to programs in Chile, Columbia, and Costa Rica, with equal effectiveness (Wilson, 1990). Interestingly, cost-effectiveness was not the principal objective in the beginning. The school's administrators introduced the individualized learning system in order to increase program flexibility, to better serve the needs of students and the fluctuations of the labor market. The increased efficiency occurred as a welcome side benefit to the original plan. Training also takes place in enterprises through in-service training programs. Such training poses particular challenges in the use of instructor and trainee time because time spent on teaching and learning has a direct cost in lost productivity to the firm. The conventional model involves assembling - 14 - Educational Technology: Chapter 1 trainees, materials and an instructor in a classroom or workshop. This model ce be administratively complex and involve high opportunity and travel costs b._n for instructors and employees. When a large number of employees must receive a relatively few hours of training, these costs can be prohibitive. Enterprises thus have a strong incentive to develop individualized training programs, and they have the financial resources to do so. Clearly, it is enterprises rather than vocational schools that have a substantial stake in reducing trainee opportunity costs through technology. Enterprises are also more likely to use technology for simulation training, particularly when it is not safe or feasible to have trainees actually perform certain tasks that they need to learn, such as technical control in nuclear power plants or petrochemical industries. As with distance learning materials, each student works on his or her own with the self-paced learning materials for vocational training; consequently, a similar kind of instructional design is required. An important difference is that training materials typically are used in a context where experienced personnel are available to answer questions. In the case of trade schools, the instructors who formerly led classes now take on the role of tutors, and also help the students when they reach the practical phase of the learning process where they must work with the actual tools or materials of their trade. Broadcast media are rarely used in vocatiinal training, except where college or graduate level lectures are offered in engineering or other technical disciplines in some programs in the United States and Europe. Print modules, audio and video cassettes, and computer simulators are more often used. However, traditional instructor-based training remains the dominant mode both in pre-service training in vocational technical schools and in enterprise-based, on-the-job training programs world-wide. Pedagogic Congruence - Once the educational task has been defined, the second issue is to determine the congruence between the pedagogical problem and a particular technology, or, more commonly, combination of technologies. In the past, it has been taken for granted that, whatever the problem, a technology intervention of some kind would be appropriate. With this assumption, the principal objective of experimentation was finding ways to use technology to lower costs. This assumption should not be made at the outset. Rather, planning should begin with the question of whether there is a technical alternative that is pedagogically appropriate for the problem at hand. To be viable and appropriate, technology(ies) must be appropriate for the learners' needs, the course content, the learning environment, and the teachers' abilities. If no technology meets these conditions, then further discussion of costs clearly is moot. Learners' needs. Careful consideration of the target audience will reveal a great deal about whether technology can be used, and if so, what kind. Learning age and abilities are of crucial importance. For adult learners at advanced levels, as in distance universities or advanced level - 15 - Educational Technology: Chapter 1 training programs, the students' capacity to absorb material is high, and their motivation and self-discipline can be assumed to be fairly strong. Thus, a "talking head" style lecture (broadcast by television or distributed by audio cassettes), for example, could be a viable and low-cost option. Primary students, on the other hand, would have difficulty paying attention to such a program. For audiovisual teaching materials to be effective in reaching young children, they must be tailored for the younger audience and designed with great care to establish and maintain attention and motivation to learn. They will require more time and effort, as well as special expertise -- which means they will cost more to produce. Cogrse content. The appropriate kind of technology to use is also determined by the course content. No hard and fast rules can be made about when it is right to use a particular technology, although many situations are obviously wrong. For example, print is unsuitable for music appreciation, for oral skills in foreign languages, for conveying complex physical tasks in training programs, and for reaching children who do not yet know how to read. Radio cannot teach reading skills unless tied in with printed materials, nor has it been effective in teaching complex concepts at the college level, except as a supplement to other study materials. In the vast majority of cases, it has been found that technologies are best used in combinations. Experienced learning design specialists are familiar with the entire spectrum of strengths and weaknesses of each tool. Discussion of choice between different technologies should not overshadow a more important issue in the learning design process. The principal lesson to be drawn from experience has been that the quality of curriculum and materials design is crucial to the effectiveness of any technological application. In achieving an educational purpose, a particular medium is only as effective as the quality of the message (content) and the skill with which the medium is used to communicate the content. No equipment, no matter how sophisticated, can teach well with badly prepared courseware. Experience has shown that the best choice of technology is not scientifically predictable. Nor does it appear that there is only one right choice of technologies for a given situation. A particular combination of technologies may be very effective in meeting a given educational need; however, the problem might be solved equally well by a different set of technologies. Learning environment. It is important to consider the congruence between the technology and the learning envihonment. Where students already use audio cassettes to listen to popular music, as is common in many developing countries, they will not find cassettes difficult to accept in the learning process, or in the classroom. Television broadcasting is a practical option in a case where television sets and classroom space can be provided through an institutional linkage, such as at the students' workplace, as is the case in the Chinese Radio and Television University. On the other hand, efforts to provide television sets and batteries in non-electrified rural villages in the Ivory Coast and Niger were extremely costly and ineffective; it is likely that a simpler technology would have been more practical in that environment. - 16 - Educational Technology: Chapter 1 Teacher abilities. Finally, consideration must be given to congruence between teachers' needs and abilities and the demands of using the new technologies. When teachers feel overwhelmed by the challenge of presenting material that they do not themselves understand thoroughly, they may quickly embrace a program which gives them support in the classroom, such as programmed learning, better textbooks, or supplementary radio lessons. In contrast, it is unrealistic to expect teachers to introduce computer learning systems when the teachers are inadequately trained and teaching under adverse conditions. It has been found that even in wealthy industrialized countries, well-trained teachers need at least two to three years to learn how to make use of computers in their teaching. Inadequately trained teachers are likely to find the challenge beyond them. In poor areas, moreover, teachers, parents and administrators, are likely to resent expenditures on expensive equipment when money cannot be found in budgets for higher salaries or other pressing needs. Economic Feasibility: Cost-Effectiveness An important aspect of congruence between an educational tool and the task to which it is applied is cost-effectiveness. Despite the fact that technology is rarely inexpensive, and "up-front" costs are generally significant, a technologically based learning system can be more cost- effective than a non-technological one when the ratio of learning gains to costs is considered. Any decision to implement a technologically based learning system must address the question of how much such a system will cost in relationship to what it can deliver. Greater cost-effectiveness in a system can be achieved in several ways: (a) by reallocating resources from inputs that have smaller effects on learning to those that have larger effects, that is, by increasing outputs (learning gains) associated with a given level of resources; (b) by reducing the overall amount of inputs (financial resources) while maintaining existing levels of learning; or (c) by providing new educational services (increased access, previously unavailable programs, etc.) at a lower cost than could be done by using traditional educational methods (Windham, 1988; Lockheed and Hanushek, 1988). The cost-effectiveness sought through technologically based learning systems is generally one of the first two types, although in the case of distance education, the third is generally the focus. Unfortunately, most educational decision makers facing choices about the economic feasibility of introducing technology have little information about the comparative effectiveness or costs of the technological approach versus a more coiuventional approach; moreover, it is quite difficult to obtain such information. However, properly designed research can provide information about the effectiveness of technologies in enhancing learning, and information on costs can be obtained through procedures that measure th3 cost of each "ingredient" of the technological intervention (Levin, 1987). One review of the cost-effectiveness of various educational interventions in developing countries -- textbooks, interactive radio instruction (IRI), and teacher in-service training -- found that when cost is taken into account a particular intervention may be seen to be more promising than when effectiveness alone is considered (Lockheed and Hanushek, 1988). - 17 - Educational Technology: Chapter 1 For example, the study found that textbooks were comparable to IRI in effectiveness, but when costs were also considered in the comparison, in some countries IRI proved to be nearly three times as cost-effective as textbooks. In the United States, peer tutoring was found to be twice as effective as computer-assisted instrt-ction (CAI) in boosting reading skills, but also twice as expensive; thus, the cost-effectiveness of the two methods was roughly equal (Levin, Glass, and Meister, 1984). These examples illustrate the fact that the economic feasibility of a particular technological intervention should not be assessed by cost considerations alone. The effectiveness of the intervention, in comparison with other interventions (including the conventional approach), must also be taken into account. Considering costs only has led to some of the problems facing school systems in many developing countries today; because lowering costs has been the only objective, necessary resources have not been supplied to schools (Lockheed and Verspoor, 1990). In such cases, costs per student have declined, but learning has apparently decreased as well, resulting in no net improvement of cost-effectiveness. Cost-effectiveness is directly affected when a curriculum is ineffective or not suitable for the target audience. Enrollment may be low, students tend to drop out, or they fail their certification examinations. In the long run, though such a curriculum may have low costs, its cost- effectiveness is also low, as in the case of the IRDEB project in Bahia, Brazil, and the Kenya in-service teacher training course in the 1970s (Nettleton). Techrology also affects economic feasibility by changing the way that inputs must be allocated to the educational process. In traditional schooling, inputs, such as students, teachers, school rooms, and traditional learning materials, are assembled at the beginning of the school year and left essentially undisturbed until the academic schedule is completed, resulting in the output of students qualified to enter the next class level or to graduate (minus, of course, those who drop out or fail). By contrast, nearly any kind of technology implementation demands that inputs be continuous throughout the school year. Radio programs have to be broadcast at regular intervals for radio learning projects, worksheets and printed learning modules for programmed learning must be delivered to students throughout the year, distance education programs require constant exchange of materials between students and their tutors, and self-paced learning systems for vocational training work best when they are completely freed from the restrictions of an academic schedule. Under these circumstances, management of continuous inputs puts greater pressure on the resources and management cap^bilities of the agency that is carrying out the project. The factors that have been found most likely to affect the cost- effectiveness of a technology implementation are: the degree of innovation, the scale of the program, the kind of technology used, the amount of fixed capital expenditure required, and the caliber of project administration and management. Financing options include cost recovery through fees, regular funding allocations from the government education ministry, bilateral or multilateral loans, and foreign aid. In certain cases, equipment manufacturers may provide equipment at lower cost or offer other advantageous incentives (such as training), although this kind of arrangement can be fraught with complications. - 18 - Educational Technology: Chapter 1 The evidence on the cost-effectiveness of technology in comparison to conventional alternatives has been much clearer for distance education than for in-school general education programs. This difference could result from the fact that these projects are easier to examine; they are usually undertaken in the context of a brand-new, discrete institutional entity devoted entirely to distance education, such as a distance university. This separate identity makes comparability easier, since effectiveness data such as the ratio of graduates to initial enrollees and dropout rates can be compared with figures from equivalent, traditional institutions. For example, clearcut cost-effectiveness advantages have been shown directly in a teacher training program in Tanzania, and in Asian distance universities such as Thailand's STOU. In contrast, cost benefit advantages of technology applications in primary schools as compared to other options such as textbook provision or teacher training have been less conclusively proven. III. Barriers to Implementation Given a logical, and to some extent empirically demonstrated fit between a technology alternative and an educational problem, what are the next factors that must be addressed if the alternative is to be successfully developed and installed in a real life situation? Clearly E educational alternative can be well conceived, yet still fall prey to &- of a number of implementation problems. Considering the importance of non-pedagogical feasibility factors in determining the sustainability of a program, it is essential to examine these factors carefully. Feasibility is the most complex facet of the sustainability model, and embraces three dimensions which must be analyzed in turn. These are: a. technological environment b. administrative capacity c. political receptivity The record suggests that many of these kinds of implementation obstacles can be overcome, given sufficient resources, good planning and effect4.ve management. Thus, feasibility can best be judged on the basis of experience, rather than theoretical speculation. In the long run, the failure of a particular program based on a particular technology does not automatically mean the technology is ineffective; the failure may have been the result of implementation problems rather than a mismatch between the technology and the pedagogical problem. Technological Environment The first requirement for sustainable use of technology is an appropriate technological environment. The physical environment where the technology will be used is not likely to be changed by the project itself. Existing infrastructure such as electrical power or regular postal deliveries needs to be adequate to serve the technology as required. In nearly every case where a program has tried to supply these basics itself, the effort has proven to be fatal to the entire project, because of either cost or logistical - 19 - Educational Technology: Chapter 1 difficulties. Under such circumstances, a technological intervention is not feasible, no matter how effective it might be pedagogically. For example, if no television broadcasting is available in rural areas of a particular country, television should not be considered as an instructional option for that particular context. Some of the most well- known project failures, such as the instructional television project in the Ivory Coast, occurred in precisely this situation. In other cases, efforts to overcome a lack of electricity through using batteries and generators have run into serious cost problems. Educators should not be deluded into thinking that their reforms can substitute for a complete rural electrification program or the development of national broadcasting systems. Lack of infrastructure is not the only environmental challenge. Climactic conditions may be unusually harsh, involvir.g sand, humidity, high temperatures, etc. Poor roads, lack of expertise, and lack of spare parts can make maintenance of mechanical and electronic equipment extremely difficult, as well as impeding physical distribution of learning materials such as textbooks and audio cassettes. Conditions are likely to vary throughout a country, however, and a difficult rural environment may coexist with fairly well developed infrastructure in larger towns, regional capitals, and major urban centers, where, for example, the majority of secondary schools might be located. The basic lesson is that the more difficult the environmental conditions, the simpler the technology should be. One guide to appropriateness is to determine whether the users already make use of any of the technologies in non-educational situations in the target area. Administrative Capacity Administrative capacity to implement, supervise and maintain the technological intervention is the next requirement. The government agency responsible for the project must be large enough to devote the necessary manpower to it, and the executive level staff must be experienced enough to run a project of the planned scale and scope. If the technology application is innovative and untried, a greater degree of specialized expertise is required, and there is a greater likelihood that outside consultants will have to be sed. Staff development in early stages of the project is essential. When ;.ie intervention follows a model that is familiar to the administrators responsible for its operation, the im-i'.ementation is less demanding of time and resources, particularly since it requires less rigorous testing and evaluation to establish educational effectiveness. The way the project is integrated into existing bureaucratic structures is a critical variable in long-term sustainability. On the one hand, the agency in charge of the project must have sufficient autonomy to make its own decisions on expenditures, strategy, and learning design in order that the particular needs of the project can be met. Without autonomy, it is difficult to collect accurate information about costs because project costs may not be separated out or budgets protected from other institutional demands. Autonomy must be accompanied by the stable financing needed to develop both the new system and institutional capacity. On the other hand, the links between the technology project and existing educational systems must be strong, or the implementation will not be - 20 - Educational Technology: Chapter 1 able to survive as a viable educational option. These links must include clearly specified decision-making mechanisms around and within the project, to help with problems of lack of continuity in top personnel as well as to bridge political gaps between agencies. A persistent example is the need to establish cooperation between educators, broadcasters and creative talent in broadcasting programs. In terms of institutional survival, if the agency that runs the project is too isolated from the ongoing educational decision-making processes, the project is at risk of being dropped by the wayside or seeing its funding melt away. In terms of educational credibility, parents and students will be unlikely to favor a general education technology application that is not supported by the formal school system and tied to the formal curriculum and testing sequences leading to traditional educational credentials. Administrative links should also include financial commitment from appropriate institutional entities, beginning early in the life of the project. A project that depends too heavily on foreign aid will often collapse when that funding runs out, because alternate domestic sources are not available to take up the burden. When a government makes a financial commitment to a project, it is inclined to make a stronger institutional commitment as well. To strengthen the program and reduce costs, increased cooperation with other institutions is an important strategy. Cooperation could involve shared use of buildings and cooperative curriculum development with existing schools and colleges, as well as agreements with other government detpartments, such as the postal service or the government broadcasting authority, for tariff concessions, shared production facilities, or extra broadcast time. Political Receptivity Certain basic political conditions are required for an experimental pilot to make the transition to permanency. Stability and continuity in the education ministry is of first importance. Many promising pilot projects have been abandoned because of changes in top leadership, despite good evaluations in educational effectiveness. In the case of Radio Mathematics in Nicaragua, the 1979 revolution caused a rapid end to the project. In the case of the Philippine Radio Education Pilot Project (1978-1981), leadership changes at the project agency level and the ministerial level, combined with diminishing interest from the multilateral lending agency involved (Middleton, 1985), were factors in project discontinuation, despite successful completion of the pilot. Assessment of the relationship which might develop between the project and bilateral or multilateral donors abroad is an important issue in determining political feasibility. Such funding may be crucial to allow a project to handle the start-up costs at the small-scale, pilot stage of implementation. But, as already mentioned, excessive financial dependence on an external donor hinders long-term sustainability. More'ver, if the impetus for the project is seen as being from outsiders, local political support may be stunted; if the project becomes too heavily identified with foreign interests, local resistance may even develop. On the other hand, skeptical - 21 - Educational Technology: Chapter 1 policymakers may be:ome convinced of the value of an unfamiliar policy option by seeing it suppor;:ed by planners with direct experience from other countries. Today the struggle within the educational establishment for scarce funds is often fierce, and the marketing of educational technologies to developing countries is a sophisticated and highly organized activity (Terry, 1989,. It must be recognized that a new project is in competition with other technologies and must be "marketed" to those with the power to determine its fate. The reality is that many other educational options are available; for some cptions, such as textbooks, manufacturers employ aggressive advocates looking to expand sales. Policymakers need to be actively convinced that a successful pilot really is a better option than another, more traditional option. Evaluation data must not only be accurate, but must also speak directly to the concerns of top-level decision makers, who are much more likely to support a project that promises to contribute to their policy agenda. The danger of taking a marketing approach is that planning objectivity is lost, and the integrity of the research may be skewed in an effort to sell an idea. Short-term political manoeuvering can shape a project in a way that is not beneficial to the students involved. Moreover, since educational experiments require a substantial amount of time to be developed and implemented, they cannot always keep pace with rapid policy changes. In the worst case, political forces might create a temptation to distort or suppress negative evaluation data, leading to inaccurate and deceptive research results. Political feasibility is also affected by sociocultural factors. Language is a thorny problem in many countries. Where more than one language is spoken, choosing the one or ones to be used by the project may turn into a volatile, controversial issue. Concern about imposing a dominant language on minority groups has to be weighed against the practical difficulties of conducting the project in a number of languages. Cultural, religious and ethnic traditions can also present stumbling blocks within a country that has substantial diversity or even hostile conflict among groups. Where transfer of a teaching model from a developed country to a developing country is being considered, different cultural patterns may be a problem, beyond the complications of language and translation. This issue was evident in the adaptation of the preschool program Sesame Street for Latin American audiences. Political sensitivity is most acute when material to be transferred from one country to another is directed at young children, who are presumed to be more susceptible to cultural influence. With older audiences, there is less political sensitivity. This difference is exemplified by the contrast between the problems encountered in transferring Sesame Street (for young children) and the enthusiastic adoption of Britain's distance learning model (Open University) in a large number of developing countries, where little concern over cultural appropriateness has been expressed. - 22 - Educational Technology: Chapter 1 IV. Imnlementing Educational Change It should be emphasized that this chapter is not intended to provide a rigid map of how to develop education policy for technology. Bringing about positive and permanent changes in the education sector is a complex and subtle process, which should not be reduced to simplistic generalizations. Key issues can be discussed, as they are in this chapter, and this discussion can help to guide policymakers, but no hard and fast rules can be established. For those who would reform and improve education, there are two questions to ask about technology-based alternatives before embarking on a program of change. Does a given alternative have a reasonable chance of succeeding in a particular context? If so, what are the conditions which increase the probability of successful implementation? In other words, a two- stage, iterative decision-making process is required: the first stage assesses the "fit" between the alternative and the content of implementation; the second, assuming a reasonable rit, assesses strategies of implementation (Rondinelli, Middleton and Verspoor, 1990). Once a positive decision is made about using a technology-based alternative and implementation of the technology is planned, the ground has been laid for the th'.rd stage -- effecting educational change. In seeking to implement new technologies, planners need to take into account four key factors: (a) the strength and resilience of social and political support for traditional schooling; (b) the need to mobilize political support for any innovation; (c) the design and decision-making factors that underpin effective pilot projects; and (d) the nature of financing for the innovation. The Resilience of Traditional Schooling Since innovations are seen, in whole or in part, as alternatives to "conventional" education, assessments must begin with a close look at traditional schooling as the predominant form of education. The resilience of traditional, classroom schooling in the face of sustained criticism is nothing short of remarkable. The design and implementation of effective technology alternatives must take into account the elements of schooling that contribute to its staying power. Categorical charges that schools are resistant to change will not suffice: they ignore the instrumental strengths of the school as an agent of education, as well as the social ard synWolic place of the school in society. Much of the strength of the school lies in the simplicity with which it brings together the fundamental elements of learning -- students, teachers, materials and time -- in a place where learning is the principal purpose of activity. This simplicity makes school administraticn relatively straightforward. And the school, by design, is isolated in time and place from the rhythms and distractions of daily life. Students receive the adult guidance that historically has b'en central to school-based education. Modern research in developmental psychology reinforces the wisdom of this ancient choice: the complex cognitive skills and strategies necessary for fully independent learning develop gradually up to early adulthood, and even then are not always fully developed. The Importance of learning with peers also has both historic and modern research rationales. - 23 - Educational Technology: Chapter 1 As a group, the educational technologies are far less "self- contained" than the conventional school. As noted previously, distance education programs typically require a flow of new lessons, and these must be scheduled for broadcast or delivery; continuous exchange of materials between students and their tutors also must be arranged. Since the logistics of these processes can be difficult, particularly in developing countries, technological interventions may substantially increase the difficulty of administering educational systems. Consequently, technological innovations are more vulnerable to administrative breakdowns than the conventional educational system. In addition, dependence or a continuous flow of inputs makes technological programs more vulnerable to disruption or influence from outside, while conventional schools, being more self-sufficient, are more able to maintain their independence. Indeed, the separate organization of schooling functions to protect this fundamental social process from casual changes that do not have widespread social support. When the quality of inputs is good and the circumstances stable, schools can do a very good job of education. This is recognized by parents and students in all societies. Knowing from observation that good schools exist and that they make a powerful difference in the lives of students, the first interest and demand of citizens is access to good schooling in the traditional sense, not to an alternative means of education. Schools are powerful symbols of achievement to communities in both developing and developed countries. Particularly in developing couiitries, leaders of newly independent nations generally have understood that schools were important not only for their instrumental value, but also as symbols of progress. In developed countries, economically declining communities hold fiercely to their schools, often at great cost. The possibility of perfecting conventional schooling thus draws more interest and support than any alternative to schooling. This fact has two important implications. First, any technology for improving schooling must acknowledge the comparative advantages of schools for novice learners: they provide adult guidance and a focal time and place for learning. Second, technologies must treat with caution and respect the social and symbolic value of conventional schools. Employing technology-based alternatiyes to schooling for children of the rural poor in developing countries sends a clear, though unintended, message: that their children do not merit a good school -- a perfectible school -- and they must be satisfied with less.!Y The perfectible school, of course, remains elusive even under the most favorable resource conditions. And in many developing countries, the task of providing the inputs that good basic schooling demands -- healthy students, good teachers, effective materials and adequate facilities -- has been and continues to be daunting (Lockheed and Verspoor, 1990). Alternatives must be sought. Whether various technology-based alternatives can be sustained, and how this can be achieved, is not entirely clear. However, the record indicates that, for certain purposes at certain levels of schooling, there is reason for optimism. I We are indebted to Joao Oliveira for this insight. - 24 - Educational Technology: Chapter 1 Mobilizing Political Support Educational change requires two kinds of political support. It must have internal support, that is, support within the government's decision- making process. Political support must also be mobilized externally -- outside of government circles among the public. In terms of internal government processes, the feasibility of a project essentially depends on the extent of the political will behind the new project. Often top-level decision makers are biased against educational technology because they are unfamiliar with it, they perceive it as a threat to traditional educational values and practice. or they are nervous about costs. Those who want to implement any kind of educational change, including a technological alternative, must determine how to make it appealing to key persons within government. Part of the planning process is designing and describing the project so that it clearly addresses needs that key officials perceive as important. A frequen. pattern is that a charismatic individual with substantial political influencc takes up the cause of introducing an educational technology project, and works to convince colleagues and other government departments to back the idea. While this is advantageous in cases where a lot of convincing needs to be done, the project is at risk if the individual falls from power or moves to a different part of the government. Broader support for a technology alternatives project gives it a much better chance of lasting on its own merits, regardless of changes in personnel and other uncontrollable circumstances. Among the external political factors to consider is the political impact of the project at the local level. The local environment includes teachers, unions, local administrators, parents and the students themselves. As Nettleton points out in Chapter 3, strong links with community institutions, such as religious organizations and universities, have helped projects to reduce costs and contributed to long-term institutional survival. For instance, in the Dominican Republic's Project RADECO, local community organizations supply the administrative structure for recruitment and supervision of study group leaders. A failure to consider how the goals of an educational technology innovation may affect all interested parties can cause serious problems. For example, in the case of educational television in El Salvador, teachers quite understandably resisted the program, because they clearly saw that it would increase their workload without giving them additional compensation. What made economic sense -- increasing educational efficiency at a lower marginal cost than salary increases would require -- did not make political sense with teachers, who already felt themselves to be overworked and underpaid. Pilot Projects and Going to Scale Where the degree of innovation in a technology intervention is high relative to conventional practice, a pilot project is often employed. Pilots provide time to develop professional and management systems and staff without the pressure of full implementation. The appropriateness of the intervention for the educational task, the pedagogical congruence, and the economic - 25 - Educational Technology: Chapter 1 feasibility can be tested at low cost. Administrators can learn from experience and adjust the design of the intervention towards improved cost- effectiveness, avoiding costly mistakes when going to scale. Concrete data and experience are available for decision making. Effective pilots are not easy to implement. Three elements are essential for success: an explicit design that models the intervention, comparative data on both costs and effectiveness, and formal decision-making structures that involve key stakeholders in the pilot and in decision making (Middleton, 1985). Explicit design. An innovation, by definition, makes assumptions about the task to be accomplished and about pedagogical congruence and economic feasibility. A radio in-service teacher education project, for example, assumes that: o teachers will receive the broadcast; o they will listen to the broadcast; o they will learn from the broadcast; o they will change what they do in the classroom; o as a result, students will learn more; and o the unit costs of achieving these changes will be less than for conventional alternatives. These assumptions involve not only the instructional design of the intervention, but also the technological environment, administrative capacity, and political feasibility. Evaluations of pilot projects are most effective when they provide information that enables decision makers to test the validity of their assumptions. Without an explicit design, there are likely to be unanswered, and unanswerable, questions at the decision-making stage. For instance, a well conceived pedagogical strategy may fail because teachers do not use it properly, but this reason for failure cannot be pinpointed unless the evaluation design assesses classroom use. Comparative data. The decision to go to scale should be based on comparative data on cost-effectiveness as well as educational effectiveness. Moreover, cost-effectiveness must be demonstrated in relation to conventional alternatives. Decision makers should not be ready to abandon tried and true methods for a technological alternative without reasonable assurance on this point. Evaluations must develop information on costs and effectiveness both for the innovation and for conventional practice. A well executed pilot test of radio education in the Philippines failed to win acceptance in part because the design tested a range of innovations against each other without assessing the comparing their cost-effectiveness with that of conventional instruction (Middleton, 1985). Decision-making structures. To expand a successful pilot project and integrate it into the mainstream educational system, it is essential to have appropriate decision-making structures. Effective implementation of the innovation requires a degree of institutional autonomy. At the same time, the innovation must have widespread support within the education, planning and finance ministries, as well as teacher and parent organizations. - 26 - Educational Technology: Chapter 1 The process can be facilitated by establishing formal structures and procedures for consultation and decision making from the earliest stages of implementation. Advisory groups that meet regularly and receive good information during the pilot phase can be helpful, but project managers must take them seriously and provide resources for their operation. Explicit incorporation of the innovation in formal education plans provides another opportunity for decision making. In the effort to generate a favorable climate for decisions, the importance of providing public information should not be overlooked. Financing How a project is paid for can significantly affect the eventual success of institutionalization. Outside funding from bilateral and multilateral donors is usually an important component of the start-up phase for technology interventions in general education, and it is often important in distance education programs as well. As described by Cummings (1986) and others, monies from these sources come with strings attached, and the institutional character of the donor organizations varies greatly. Bilateral donors may be strongly influenced by domestic political policies, and their implementation processes are subject to heavy-handed control, rigid schedules, and fixed objectives. At the other extreme, some multilateral lenders Ane more interested in broad macroeconomic objectives, and pay scant attention to small-scale pilot efforts. Whatever the funding source, long-term implementation of technology alternatives requires a firm and substantial funding commitment, because educational technology costs tend to be high immediately after transition to a full-scale project. Unit costs may eventually be quite low when the project is carried out on a sufficiently large scale, but at the outset there are often heavy costs, including the costs of developing the curriculum, establishing the project, and buying the necessary equipment. The degree to which unit costs drop with project expansion varies; different technologies have different cost profiles. Broadcasting involves high fixed costs for the production of courseware and signal broadcast, but when a large audience is reached, the unit costs per student can be extremely low. On the other hand, printed booklets and handouts may have lower fixed costs, but as the project expands the variable cost goes up in direct relationship to the number of students receiving materials. Where paper must be imported, or postal fees aze high for mailing out heavy texts, the cost can become extremely burdensome for a large-scale effort. Successful implementation depends on careful attention to each of the four factors discussed -- the resilience of traditional schooling, the mobilizing of political support, the process of going to scale from pilot projects, and the financing of the innovation. - 27 - Educational Technology: Chapter 1 V. Conclusion From decades of experimentation with educational technology in developing countries, a broad picture is beginning to emerge. For general education, applications have focused on serving existing schools, mainly at the primary level, and these applications are less demand-driven by users than they are policy-directed by government planners. Distance education has grown rapidly, but tends to serve only those students who malke it into secondary school levels or higher. It is strongly demand-driven, and has often arisen as a direct response to overwhelming pressure to create more places at higher levels of education. Applications of technology to vocational technical training ha-.e only entered the initial phase of adoption in developing countries. Sustainability has differed greatly for technology programs in each of the three areas. In general education, particularly in primary schools, sustaining technology appears to be problematic. Policymakers still question whether technology can really make a large-scale, useful contribution to Improving educational effectiveness in most developing cotntries. The cautious pace of such adoption is especially salient when technology applications for general education are compared with the use of technology for distance education. The greatest success in general achooling has been in the application of better textbook distribution in many countries worldwide, and in the implementation of programmed learning in a number of Southeast Asian countries. In distance education, the use of technology for educational purposes has not been questioned, given the separation between student and instructor. The number and size of programs is growing steadily. The kinds of technology employed successfully have been diverse, embracing everything from books to television broadcasting, but printed materials remain the foundation of nearly every program. Use of technology in vocational technical training has been only recently attempted in developing countries, hence fewer "failures" as well as "successes' have occurred and been documented. While it can be expected that enterprises in larger, more industrialized countries will be the first to make use of technological applications, it is still too early to be certain what will happen in this area. The "right" conditions for cost-effective and sustainable use of educational technologies are straightforward, taken individually. There must be a fit between the objective sought and the pedagogical capacity and cost- effectiveness of the technology chosen. Barriers to implementation must be identified, and an effective strategy for implementing change must be developed. The challenge to planners -- and to donor agencies -- is to get all of the conditions right at the same time. To do so will require a higher front-end investment in analysis than typically is required when using standard strategies to confront educational problems; such an investment may not always be justifiable. However, the record suggests that for improving th. quality of primary education and for distance learning, the effort and ir-restment can be well justified by results. - 28 - Educational Technology: Chapter 1 References Anzalone, S. "Educational Technology and the Improvement of General Education in Developing Countries." In M. Lockheed, J. Middleton, and G. Nettleton, eds., Technology and Teaching: Elements of Sustainability in Developing Countries. Cummings, W.K. 1986. Low-Cost Primary Education: Imtlementing an Innovation in Six Nations. Ottawa: International Development Research Centre. Levin, H. 1988. "Cost-effectiveness and Educational Policy." Educational Evaluation and Policy Analysis 10, no.1: 51-69. Levin, H., G. Glass, and G. Meister. 1984. "Cost-Effectiveness of Four Educational Interventions." Institute for Research on Educational Finance and Gov:.rnance, Stanford University. Stanford, Calif. Processed. Lockheed, M. E., and E. Hanushek. 1988. "Improving Educational Efficiency in Developing Countries: What Do We Know?" C2Mpare 18, no. 1: 21-36. Lockheed, M.E., A. Verspoor, at al. In press. Improving Primary Education in Develo2ing Countries: A Review of Policy Options. World Bank. Washington, D.C. Middleton, J. 1985. "The Philippine Radio Education Pilot Project: A Case Study of Decision, Design and Interaction." In B. Searle, ed., Evaluation in World Bank Education Projects. Discussion Paper No. EDT5. World Bank. Washington, D.C. Nettleton, G.S. "Uses and Costs of Educational Technology for Distance Education in Developing Countries; A Review of the Recent Literature." In M. Lockheed, J. Middleton, and G. Nettleton, eds., Technology and Teaching: Elements of Sustainability in Developing Countries. Oliveira, J., and C. Castro. 1988. "Individualized Learning Systems for Vocational Technical Training: The SENAI System." Economic Development Institute, World Bank. Washington, D.C. Pro_.essed. Rondinelli, D., J. Middleton, and A. Verspoor. 1990. Planning Education Reforms in Developing Countries: A Contingency ApRroach. Durham, N.C.: Duke University Press. Stahmer, A., A. Mehler, I. Bryan, and M. Richmond. "Uses and Costs of Educational Technology for Vocational Training in Canada and the United States." In M. Lockheed, J. Middleton, and G. Nettleton, eds., Technology and Teaching: Elements of Sustainability in Develonin? Countries. Terry, J. 1989. Personal communication. Wilson, D.N. 1990. "A Comparison of Open Universities in Thailand and Indonesia." Ontario Institute for Studies in Education. Toronto. Processed. - 29 - Educational Technology: Chapter 1 Windham, D.M. 1988. Improving the Efficiency of Educational Systems: Indicators of Educational Effectiveness and Efficiency. Washington, D.C.: United States Agency for International Development, Bureau for Science and Technology/Office of Education. - 30 - Educational Technology: Chapter 2 Chapter 2 Educational Technology and the Improvement of General Education in Developing Countries Stephen Anzalone - 31 - Educational Technology: Chapter 2 Chapter 2: Educational Technology and the Improvement of General Education in Developing Countries Stephen Anzalone I. Introduction The word "crisis" is again being heard with increasing regularity in discussions of the current status and future prospects for education in developing countries. Many countries are confronting an algebraic mismatch whereby increasing population growth and rising demand on one side of the equation are answered by stagnation or even decline in educational resources on the other. The gravity of the current educational crisis is well-described in the current literature (Heyneman and Farrell, 1988; Oliveira, 1988; Orivel, 1987; Eicher and Orivel, 1979; Windham, 1985). Unless there is a reorganization of the way that education is organized and delivered in many countries, there is a significant danger that urogress made during the past two and a half decades toward assuring education for all will reverse itself. The crisis alarms now being sounded will not be heeded in all developing countries. For one thing, similar dire warnings have been heard before in the late Sixties, with the result that most developing countries responded by finding the means to undertake a dramatic expansior. of their education systems, especially at the primary school level. Today, as they did once before, some developing countries will reckon that their prospects for increased economic growth, diminished population growth, or both, are sufficient to permit them to avoid making any fundamental readjustment in the way that education is organized and delivered. However, one of the outcomes of the current situation is that many countries are once again taking a look at the educational and economic possibilities offered by educational technology. There are several reasons for this new attention to educational technology. First, there is the fundamental economic precedent already set in other sectors that changes in the technology of the production process have resulted in the ability to produce more output with less money. Second, after a period of rapid expansion of educational systems in most developing countries, recent strategies for educational development havc emphasized the need to improve educational quality, spurring a search fo; new ways that this could be achieved. Third, as a result of developments in the more industrialized nations, educators and others in de-veloping countries are increasingly becoming interested in the use of newer instructional technologies, especially computers, to improve educational quality. Fourth, interest in the introduction of newer educational technology in schools is in some ways not about educational quality at all; it is about relevance, about a concern for preparing citizens to operate in an "information society" and to be able to produce for the markets of the future. The possibilities that educational technology offers to developing countries are often tied to the development of technology for mass communications and are supported by the complementary use of communications in - 32 - Educational Technology: Chapter 2 other development sectors such as industry, transportation, and national telephone systems. In this way, the potential benefits of using educational technology appeal to the interests of several different professional perspectives. First, there is the educational perspective. Educators, instructional designers, and educational planners look to educational technology as a potentially powerful tool for increasing educational quality and for equalizing educational opportunities. The use of educational technology provides a more focussed and manageable environment in which to develop and deliver better quality instructional interventions. For the newer technologies, the potential for interactivity is also valued. Second, there is the communications perspectivt. Some of those with an interest in extending systems of mass communication hope that adding educational uses will stimulate investments in communications systems. Third, there is the applied gsychology and social marketing 2ersDective. Planners with a professional background in these areas look toward the possibilities of educational technology for shaping attitudes and behaviors in educationally and socially productive ways. Fourth, there is the economics perspective. Some economists and financial planners see educational technology as a way to improve or extend access to education at costs that would be lower than for conventional instruction methods. This does not necessarily mean that educational technology is cheaper in and of itself. One approach is to use educational technology as a part of a planning strategy that, while increasing the level of educational investments, also yields greater educational benefits than other types of educational improvements that could be tried with the increased spending. The recent surge in attention to the possibilities of educational technology is perhaps better described as a renewed interest. As described in Chapter 3, the late Sixties witnessed considerable experimentation in the use of what was then more likely to be referred to as educational media. At that time, many developing countries were seeking solutions to a basic problem -- enormous numbers of children worldwide were not enrolled in primary school. This situation was aggravated by the shortage of trained teachers, outdated curricula, and insufficient instructional materials. The numerous attempts to make use of educational media for extending and improving education at all levels met with mixed results; both successes and failures were extensively reported on. The net result of these implementation efforts to date has been that many planners and practitioners believe that the use of technology to improve or expand formal school systems has not been proved to be, and maybe never will be, a real solution to educational problems in developing countries. Besides the failed projects, they cite the fact that many successful pilot efforts have not been able to be sustained, they point to the lack of widespread adoption of new technologies in the countries involved in the pilots, and to the fact that the use of the few successful models has been slow to spread to other countries. And where classroom applications of educational technology are involved, there is the suggestion that the economic - 33 - Educational Technology: Chapter 2 advantages promised by these applications have not been realized. Orivel (l987, p. 45) notes that: "Within-school educational technology has not dramatically improved the cost-effectiveness ratio of educational services compared with traditional teaching. The labor qualification/media substitution, which assumes that underqualified teachers supported by media will be paid less to compensate media costs, is in most contexts unrealistic... " While doubts about the potential usefulness of educational technology in the general schooling context -ontinue to abound, there are new factors in the decision making context that should be considered. Three in particular are important: the current crisis over resources, the changes in technological development that have taken place in developing countries over the past two decades, and the accumulated knowledge about how to bring about change in classrooms in developing countries. Taken together, these factors justify taking another look at the possibilities offered by educational technology. The parameters for reassessing the role that educational technology might play in strategies to improve educational quality in developing countries are discussed in Oliveira (1988), and the paper is representative of the recent trend in the field toward an attitude of quiet realism and cautious optimism, in contrast to the more contentious quality of the discussion in the past. This chapter aims to play a part in the reassessment of the role that educational technology might play in meeting the education crisis in developing countries. To this end, it will review current and past uses of educational t*chnology for quality improvement of general primary, secondary, and tertiary in-school education in the developing world. Given the anomaly between the good educational effectiveness reported for many instances where educational technology has been attempted, but a subsequent lack of widespread adoption, primary attention will be paid to implementation factors (including costs) that constrain or support regular or institutionalized use. Emphasis is put on the goal of improving quality rather than that of increasing access to education, although these goals are not always distinct. In order to adjust to situations where resources available for education are declining and educational demand is growing, many developing countries are likely to see access and quality issues to be closely linked and view the use of educational technology as a possible means to maintaining or halting the erosion of quality in the face of increasing enrollments. We are concerned with education that is provided in regular school settings and not with distance teaching for school equivalency, which is covered in Chapter 3. This distinction between the two approaches is sometimes not clear cut; both regular school education and distance teaching often rely on the same kinds of technology such as broadcasting and print, and distance teaching often takes place in organized learning environments that resemble traditional classrooms. In fact, the fields have only recently been clearly separated into separate research areas, and some studies done in the 1970s have a bearing on both areas. Thus the classification of certain programs as in-school learning rather than as distance education, such as the - 34 - Educational Technology: Chapter 2 USAID-sponsored interactive radio projects, is unfortunately somewhat arbitrary. The chapter is organized as follows: We begin by examining educational technology as a concept and strategy and the process by which the concept and strategy is brought into practice. The next section reviews the major varieties of experience where educational technology has been used for quality improvement in education in developing countries. In the following section, we examine the question of comparative advantage among the various technologies in terms of implementation for educational quality improvement in developing countries. We then offer a summary of the major obstacles that affect the implementation of educational technology in developing countries. We finish with a set of conclusions and issues to be addressed by planners in developing investment policies and strategies for the use of technology in education. A tentative and probably incomplete listing of various projects and studies of the in-school uses of educational technology is presented in Annex 1. II. Theoretical ADIRroaches to Educational Technology The field of educational technology appears to have more difficulty than most other areas of professional activity in characterizing itself and its products. Definitions of educational technology abound; most of them are either to narrow or too broad to be useful for this paper. Some see educational technology strictly in terms of equipment, electrical or electronic devices, or other products. At the other extreme, some define educational technology in the broadly inclusive, but not very practical, process terms that embrace all activities relating to solving educational problems and defining or guiding the teaching/learning process. (The question of defining educational technology is treated in Thomas, 1987; Ely and Plomp, 1986; and Ely, 1983). In this paper, we put forth a conceptual/strategic definition of educational technology as it is used to promote educational quality in developing countries. We then offer a model to explain how the concept and strategy are put into practice in the process of implementation. Educational Technology as a Concept Educational technology is about combinations of things, and our definition highlights several important combinations. First, at the most general level, we regard educational technology as a combination of concept and strategy, insofar as we are concerned with how educational technology relates to improving general primary, secondary, and tertiary education in developing countries. At the conceptual level, we define educational technology as being made UD of combinations of materials and processes found in the classroom that, taken together. attempt to enhance,or extend the instruction grovided by the teacher. Educational technology may enhance the quality of instruction provided by the teacher by helping to overcome limitations in a teacher's lack of knowledge of a subject or ability to - 35 - Educational Technology: Chapter 2 present it in instructionally effective ways. Educational technology may extend the instruction provided by the teacher by making it possible to deal with more students or to provide more opportunities for students to be effectively engaged in instruction because of the limitat_ons of time that teachers may spend with individual students. More specifically, we see this conceptual definition of educational technology as including a set of instructional interventions that combine the use of instructional materials and means of delivery -- b-th of which are guided by varying applications of different principles of Instruction. We take instructional materials to mean such things as text materials, radio lessons, television lessons, and educational software -- together with the methods contained both in the materials and in the strategies to organize their use. We take the term means of delivery to refer to the vehicles used for presenting instruction in the classroom (e.g., teachers, radio, or computers). The role which we have assigned to instructional materials for the purposes of this conceptual formulation of educational technology may be challenged. It is more common to analyze methods separately, and to view the effects of an educational technology intervention as being the result of the new teaching methods adopted, independently of the type of materials or media used to deliver them. We believe, however, that to treat methods apart from materials is likely to be of little practical value for improving educational quality in developing countries. Instead, we prefer to treat instructional methods or principles of instruction as being embedded in instructional materials and the way they are delivered in the classroom. One justification for this is the element of feasibility; while it is true that in an abstract sense the new teaching methods could be conveyed by conventional teacher education, in the real world, this i8 generally a financial and logistical impossibility, and the element of feasibility is inherent in the new media or materials being proposed. The existing literature on educational technology in developing countries does not yet permit us to be able to specify what instructional methods or principles of instruction are most common or effective. However, some principles, such as active student participation, frequent practice, and the use of reinforcement, have been found in a variety of successful interventions and appear to recommend themselves for inclusion in any application involving the use of educational technology. Again, we must underscore our insistence that the use of educational technology for improving the quality of general primary, secondary, or tertiary education in developing countries involves the combined use of instructional materials and means of delivery. Our concerns are practical. Often, where the literature makes a distinction between methods and means of delivery, there is considerable debate about whether the latter make any difference in educational effectiveness. Clark (1983, p. 445) argues that "...media do not influence learning under any conditions" and any superior gains in learning that have resulted from the introduction of an alternative medium are the result of improved instructional methods and novelty rather than due to the inherent teaching qualities of the item. Although many would argue that Clark's position is extreme, our point is that this kind of issue is often irrelevant for developing countries -- that while it is theoretically - 36 - Educational Technology: Chapter 2 possible to distinguish between instructional methods and means of delivery in the classroom, in practice, they usually are developed together. Educational Technology as Strategy As mentioned, our notion of educational technology is a combination of concept and strategy. As a strategy it points to the efficacy of certain types of educational intervention and investmevt over others. The strategy is to treat student, teacher, and school charactea.stics (inputs in the education production process) as given, as something that will change over the long run as a result of successful economic development but that cannot be greatly changed within the time and resources available to most development projects. The strategy aims at improving educational quality not through the indirect effec.ts of limited improvements in the above-mentioned inputs but directly, through improving the instructional materials and means of delivery of education content in the classroom. As a strategy, this is a prescription for promoting educational quality wit'. scarce resources and not a statement that student, teacher, or school characteristics are unimportant in the instructional process. Educational interventions based on major investments in school textbooks that have occurred during the past decade are an example of this kiD" of strategy. Clearly, they have involved difficult policy choices: mon 's that are invested in educational materials are no longer available to ii«. re teachers' education or to increase their salaries, nor to help studwts with preschool education or nutrition supplements. The political implications of a strategy relying on the use of educational technology are obvious and they have proved to be one of the biggest obstacles to continued or wider implementation of educational technology in developing countries. Educational Quality Outcomes Our definition of educational technology is active, not static, and the combined use of instructional materials and means of delivery must contribute to the attainment of instructional objectives. The nature and size of these outcomes and the value we place on them are represented in the notion of educational quality. For the purpose of this paper, educational quality will be defined as the attainment of desirable levels of (a) effectiveness, (b) efficiency, or (c) appeal in the process of instruction. These outcomes are borrowed from Reigeluth (1983). By effectiveness we mean the overall level and distribution of attainment related to development of (a) knowledge and cognitive skills, (b) attitudes and values, (c) physical skills, and (d) other social behaviors. The most common measure of effectiveness is student achievement. The effectiveness of the use of educational technology in developing countries is a matter of great concern. We do not focus on it in this paper since it has been thoroughly examined elsewhere (see Lockheed and Hanushek, 1987 and Anzalone, 1987). To summarize the current thinking, research addressing the use of educational technology does not permit us to "prove" the effectiveness of any specific te .hnology or to demonstrate its instructional superiority to other technologie.. However, the use of educational - 37 - Educational Technology: Chapter 2 technology will likely be equally effective in schools in developing countries when compared with usual classroom lecture and recitation provided that the application is sufficiently comprehensive, that the materials used are properly designed for the intended audience, and implementation is accomplished in a satisfactory manner. This is an expectation, not an established fact. The expectation follows from the overall demonstration of the relative importance of material resources in schools in developing countries when compared with other factors (Heyneman and Loxley, 1983). For example, the availability of textbooks has been shown to be a consistent predictor of student achievement. In other instances, the positive effects of using technologies such as interactive radio and programmed teaching/learning suggest the importance of instructional materials generally and not just textbooks for improving effectiveness. Again, an expectation of effectiveness is not a suggestion that any application of educational technology will be effective. What it means is that the effectiveness of applications of educational technology are likely to lepend more on how well instructional materials are designed and used than on some general inherent "power" of a specific technology. Efficiency is an outcome that relates outputs to inputs, most commonly effectiveness to costs. The concept of efficiency is treated in detail in Lockheed and Hanushek (1987) and Windham (1988). Improving efficiency can either refer to ways of maximizing output at given levels of input or to ways of minimizing the costs of inputs for a given level of output. Educational quality may improve through increased efficiency if effectiveness is improved without added costs (rare) or where planners are able to obtain improvements with a smaller increase in cost than through some alternative investment strategy. Greater efficiency may also contribute to solving educational budget problems (but not necessarily to improving quality) when effectiveness is held constant as cheaper inputs are used, or when the school system is able to serve more students with the same inputs. Our use of the term educational quality also embraces the notion of appeal. Appeal is an outcome measured by the desire on the part of students to continue to learn (Reigeluth, 1983). This might be reflected in better attendance at school, more active engagement in the instructional process, or in students' staying in school longer. Appeal may contribute indirectly to more effective instruction. But even where it does not, most societies place a high value on having children and young people spend more time learning. Arguably, more educational decisions are motivated out of a concern for appeal than effectiveness. The perceived importaL,ce of appeal explains such apparently illogical situations such as making major investments in improving school facilities when there is strong evidence that better buildings often have little impact on the effectiveness of instruction, although they may have some effect on appeal. - 38 - Educational Technology: Chapter 2 III, The TechnoloRies: Past. Present and Future There are countless educational methods that may be combined in countless ways in applications of educational technology. Those that have been or are likely to be of interest for improving the quality of general education in developing countries include the following: Textbooks; Audiovisual supplements; Instructional radio; - Interactive radio; - Other radio instruction; Programmed/teaching learning; Instructional television; Computer instruction; - Conventional computer-assisted instruction; - Other computer-based learning; Videodisc instruction; - Linear videodisc instruction; - Interactive videodisc instruction; Instruction provided by electronic learning aids. All of these offer, within classroom lessons, an attempt to enhance or extend the instruction provided by the teacher through the combined use of instructional materials and means of delivery -- both of which are guided by varying application of different methods of instruction. In some cases, the instructional principles or methods are specific and visible. In other cases, they are not. In some cases, we have information on actual instructional materials (i.e. interactive radio lessons), whereas in others (i.e. videodisc instruction) particular instructional materials for improving general education are just now being tested. Our intention is not to catalog all the various experiences described in the literature, but rather to highlight the principle categories of intervention that are likely to prove useful for the future. Textbooks In developing countries, efforts to provide textbooks to classrooms where they have been lacking or in short supply is the most common way to enhance and extend instruction. Such interventions may include not just textbooks but also student workbooks, teacher guides, and audiovisual aids. The instructional methods implicit in these materials tend to vary, but sometimes are not well thought out, since the materials are often developed by those with more interest in content than in teaching techniques. Methods may require such things as using locally relevant examples and pictures, study of practice examples, and answering questions at the end of a unit. The instructional delivery that is required for textbook interventions also varies. Typically, teacher lectures, classroom discussion, and individual student work are organized around the textbooks. Some organization of instruction may also be provided through teacher guides and in-service training. - 39 - Educational Technology: Chapter 2 Textbooks act both as organizers and as presenters of instructional content. As organizers, they often are synonymous with the curriculum, since other formal references to the curriculum or syllabi are seldom apparent in the classroom. Even in the most developed educational systems, decisions about curriculum are essentially the same as decisions about what textbooks to use. As presenters of instruction, textbooks are often the main source of subject-matter in the classroom. This is especially important where teachers' knowledge of the subject is deficient. In developing countries, the availability of textbooks has been shown to be a consistent predictor of student achievement (Heyneman and Loxley, 1983). It is not clear why textbooks have such an effect; this may be the result of the organization that textbooks give to the instructional process or to the advantages textbooks offer in the presentation of content. In some ways, there is probably no mystery. Much of formal education involves reading or using skills that normally develop with learning to read. Especially in environments where reading materials are scarce, it is hardly surprising that the availability of textbooks in the classroom has a consistent impact. Obstacles to imolementation. Textbooks projects have often proceeded out of a limited, "supply side" concern for providing a missing and vital input into schools. We believe that in the future there will be a greater concern for quality issues and a greater recognition of the importance of sound instructional design in the development of textbooks and other instructional materials. The development of this capacity -- to be able to do such things as diagnose learning difficulties, track curriculum objectives, and test and revise materials -- will undoubtedly be aided by the use of microcomputers as part of the administrative structure of the educational system. Textbooks, although not a "new" technology, nevertheless require a relatively sophisticated level of authorship and manufacturing in order to be produced in a cost-effective manner. On the one hand, economies of scale are required, so that a small country, or one with several different language groups may be at a disadvantage in producing books locally. This may force importation of books, which may not be suitable for the local cultural and educational requirements. On the other hand, domestic production of large numbers of books requires equally large amounts of inputs such as paper, which, if imported, may require substantial amounts of scarce foreign exchange. Printing equipment may also have to be imported, and maintenance and other considerations may pose problems. Finally, distribution costs vary directly with the number of books that must be distributed to the students, and a large textbook program may tax the ability of an education ministry to get the books out to the schools in a timely and affordable manner. We might forecast, however, that the production and revision of textbooks and other printed materials in developing countries will increasingly be affected by developments in computers and associated technology. New techniques of desk top publishing are spreading rapidly, and they make it more cost-feasible to produce high quality materials for a limited market. This is likely to benefit some countries that have up to now found textbook production to be too expensive or complex for local conditions. - 40 - Educational Technology: Chapter 2 Hicropublishing makes use of a number of different technologies, including: microcomputers, desktop publishing software, image scanners, and laser printers. In addition, CD-ROM discs can now be used to store vast quantities of images (pictures, maps, and diagrams) and this technology is finding increasing use among publishers in developed countries. For developing countries, access to large numbiers of this kind of image in digital form on CD-ROM disc opens new possibilities for the production of textbooks and other printed materials, because these images can be manipulated or customized for local purposes. Audiovisual Supplements What we refer to today as educational technology once consisted mainly of audiovisual aids or supplements to instruction. Audiovisual supplements to instruction comprise materials and means of delivery that are intended to bring into the classroom concrete or representational examples of real world phenomena in order to aid comprehension and retention, facilitate the modelling of behavior, stimulate imagination, or otherwise enrich the curriculum. Audiovisual supplements involve such materials as films, filmstrips, slides, pictures, overhead projector transparencies, diagrams, posters, maps, flash cards, science kits and laboratory apparatus. We would also include the blackboard, an educational aid that may be used both as an extension of classroom lecture and recitation and as an alternative mode of presenting information or directing student activity. We might also include language laboratories in the audiovisual category. It would be impossible to summarize the enormous number of possible combinations of materials and means of delivery that would accompany the use of audiovisual supplements to instruction in developing countries. The introduction of audiovisual supplements into classrooms in developing countries seems to occur most commonly as part of wider curriculum improvement projects. Judging from the experience of schools in more developed countries, we find that the use of audiovisual supplements may or may not relate directly to the regular school curriculum. In the worst cases, teachers may use materials such as films simply to occupy classroom time in order to take a breather from the usual daily routine (Cuban, 1985). From what is known about the overall effect of material inputs on student achievement in developed countries and from practical experience, the contribution of audiovisual supplements to improving educational quality is rarely expected to be large, and the often low expectations resulting from this experience has affected attitudes toward the newer educational technologies. The literature on the effectiveness of audiovisual supplements in school instruction in developing countries is virtually nonexistent. The use of audiovisual supplements (except for science laboratories) does not figure in the important lists of school quality elements, either under the categories of instructional media or materials (see, for example, Fuller, 1985). The lack of investigative attention to audiovisual supplements is the result of both its apparently low priority as a research topic and of the difficulties that would be encountered in attempting to disentangle the independent effects of these supplements from the likely more potent effects of the wider - 41 - Educational Technology: Chapter 2 curriculum or instructional intervention in which the supplements may be embedded. Obstacles to implementation. The major obstacles to continued or wider implementation of audiovisual technology in developing countries are the costs, and the lack of clear demonstration of effectiveness. It is by no means clear what advantages the use of audiovisual supplements would have in improving educational effectiveness over, say, textbooks or interactive radio lessons. Nor is it clear whether these gains would be sufficient to justify the additional investment required to develop or procure materials and delivery devices and distribute them to schools. Moreover, the costs are likely to be the greatest obstacle in the most deprived settings, which are, ironically, the settings where audiovisual supplements might be expected to have their greatest impact. Where audiovisual supplements are delivered by electrical equipment such as film projectors, other obstacles are likely to constrain continued or wider implementation. This includes inadequate infrastructure and environment conditions for operating and maintaining equipment and classroom organization, finding appro:-riate materials for purchase or rental, and factors related to linking available materials to the curriculum and storing, scheduling, and retrieving equipment. Costs. The cost of audiovisual supplements varies widely. Costs may range from a few cents for a mass-produced chart to several hundred dollars plus related materials costs for projectors to thousands of dollars for science or language laboratories. These will be add-on costs with varying degrees of affordability by country and school. Instructional Radio It is useful to distinguish between two varieties of instructional radio for developing countries. The first of these is "interactive" radio (often referred to as IRI, or interactive radio instruction). It is called interactive because the primary school children in classes that use this medium call out responses to questions prompted by the pre-recorded radio instructors, who then give the answers according to carefully timed scripts. This process simulates live interaction, even though the broadcast radio signal is, of course, a one way medium. This educational technology application features a high degree of correlation between instructional materials and delivery strategy, and provides instruction that has proved to be lively and effective. Specific instructional principles -- including active and frequent pupil responses, immediate reinforcement, and distributed practice with "segmented" structure (about a dozen topics per lesson) -- are applied to radio lessons and supporting printed materials. Most of the instructional method is contained in the radio lessons, and the radio does most of the work for organizing and presenting instruction. Applications of interactive radio have taken place in Nicaragua (mathematics), Kenya (language arts), Thailand (mathematics), and the Dominican Republic (reading and mathematics), and all have yielded good effectiveness results, as well as reasonable per student costs. Currently, an application of the interactive model to teach science is - 42 - Educational Technology: Chapter 2 being developed in Papua New Guinea. New applications, some involving adaptations or radio lessons already developed, are underway or beginning in Honduras, Nepal, Lesotho, Bolivia, and Costa Rica. The use of interactive radio is well-documented. General descriptions of interactive radio are contained in Tilson (1988), Block (1985), and Searle (1985). The effectiveness of interactive radio is reviewed in Tilson (1988), Lockheed and Hanushek (1987), Anzalone (1987), and Searle (1985). The Nicaraguan Radio Mathematics project is described in Suppes, Searle, and Friend (1978); Friend, Searle, and Suppes (1980); and Friend (1985). The Kenya Radio Language Arts Project is addressed in Oxford et al. (1986); Christensen (1985); Friend and Kemmerer (1985). The Thai Basic Skills Project is considered in Galda (1985); Friend, Galda, and Searle (1979); and Sang-jan (1982). The Radio-Assisted Community Basic Education Project (RADECO) in the Dominican Republic is examined in Eshgh et al. (1988); Walker (1986); Sanguinetty (1985); Helwig and Friend (1985); Friend (1984); and Hanssen, Kozlow, and Olsen (1983). The more recent applications of interactive radio are discussed in Tilson (1988). Table 1 provides a summary of information on effectiveness. The other major option for radio instruction is to provide traditional broadcasts of enrichment programming, in topics such as music, foreign languages, or national culture. This, and other kinds of models of radio instruction are frequently used around the world, but the methods and results of instruction are not so well-documented as the interactive case. It is impossible to specify how many counitries are using instructional radio and in what manner. Since, as noted by Hawkridge and Robinson (1982), surveys have shown that almost every country reports using radio for education, the number could be large. It is also difficult to estimate how many of the radio interventions in these countries actually constitute significant interventions, and how many have become dormant over the years. We guess that in most of these instances, radio is used to provide optional supplements to conventional lessons. Likewise, we are unable to comment about what instructional methods are followed in these other uses of instructional radio. We suspect, however, that in most cases they recreate a "talking teacher" format or deliver an oral presentation of information written for textbooks. - 43 - Educational Technology: Chapter 2 Table I Effectiveness of Interactive Radio Projects Country Post-program Testing Evaluation Result Effect Size (mean % of questions answered correctly) (standard deviation Traditional IRI units) instruction instruction Bolivia 47.0% 66.3% .91 Costa Rica 48.2 66.2 Dominican Republic (Math) 22.0 57.0 -- (Reading) 24.0 49.0 -- Ecuador (pilot) 47.0 (pre) 62.0 (post) -- Honduras (lst Grade) 34.3 51.9 .80 Kenya (Listening) 51.8 68.0 .75 (Reading) 36.3 41.5 .24 Nicaragua (Ist Grade) 38.8 65.4 1.17 (2nd Grade) 58.4 66.1 .50 (3rd Grade) 43.2 51.7 .39 (4th Grade 33.8 34.7 -- Source: Friend Dialogues, Inc. To represent how and why radio is used to improve the quality of general education at (most frequently) the primary school level, we propose a classification of use for both interactive radio and other applications of instructional radio. This classification, depicted in Table 2, distinguishes between effectiveness/appeal and efficiency uses. - 44 - Educational Technology: Chapter 2 Table 2 Quality Improvement Strategies Using Radio Strategy Application Interactive Radio Other Radio Improving effectiveness/appeal Supplementary x Stand alone direct teaching x x Complementary x Improving Efficiency Enlarging the school x Lowering cost of schools x x Where there is a broad objective to use instructional radio to improve the effectiveness or appeal of education, we see three subcategories of specific use: a) supplementary, b) stand alone direct teaching, and c) complementary. The fizst of these includes those applications where radio is used to improve quality by adding to regular school instruction in existing subjects or by enriching the curriculum by providing instruction in subjects otherwise difficult to provide (e.g. music). Thailand's national educational radio programming is an example of this use. Radio instruction is provided at the primary and secondary level in music, social studies, and for general educational enrichment and entertainment. The second subcategory uses radio for improving effectiveness and appeal as a stand alone provider of direct instruction. The rationale for this approach is that instruction will be more effective than that provided by an underqualified teacher. Examples of this approach include the interactive radio applications mentioned above in Nicaragua and Kenya, and Ecuador's use of radio to provide more effective and culturally relevant primary school education for the Shuar Indians. The third subcategory is to use radio to complement conventional instruction. In these cases, radio is used in conjunction with existing instructional methods and materials for a particular school subject to provide alternative and direct support to specific: instructional objectives. This would include the "mental arithmetic" applications of interactive radio now being developed in Honduras and Costa Rica. For all three of these categories, the overall goal is to increase the effectiveness and/or appeal of instruction. A second broad (and possibly tacit) objective for using instructional radio is to improve educational efficiency. Here, greater efficiency is achieved not so much through increasing effectiveness as it is through attempting to maintain "constant quality" as expenditures are lowered, or kept - 45 - Educational Technology: Chapter 2 at existing levels even as expansion occurs in the size of the educational system. For example, this could be achieved by increasing the number of students served by the school without increasing the number of teachers or by establishing a school using lower-paid paraprofessional instructional personnel in lieu of a conventional school with higher-paid teachers. The first of these efficiency possibilities is to enlarge the school by using radio to make it possible to instruct more students without increasing the number of teachers. This might occur by adding students to existing grades or by adding grades to previously "incomplete" schools. An example of this use of radio was the Mexican Radioprimaria project of the early seventies. The strategy was to provide six grades of instruction in schools where there were only four teachers. Radioprimaria students in grades four to six received five or six radio lessons each day. Unfortunately, the strategy was not successful, because the use of radio was found to occur most often in already "complete" schools (Spain, 1977). The second efficiency possibility is to use radio to establish new schools at a lo-war cost than could be achieved if conventional schools were to be opened in the same communities. A good example of this is the RADECO project in the Dominican Republic. Here radio schools using paraprofessional monitors have been established in remote communities previously without primary schools. Students receive interactive radio lessons in mathematics and language for one hour per day. Although RADECO schools do not contain a complete, full-day curriculum, evidence suggests that students learn nearly as well in mathematics and reading as students in conventional schools in the region at about half the cost. A less successful example was the Mexican Tarahumara Radio Schools that operated until the seventies. Here, radio schools were conceived as an alternative to "official" schools in a remote region. Again, the objective was to provide instruction as good as that offered by conventional schools, but at lower costs. There was some evidence that students did at least as well in Spanish and arithmetic as did students in comparable conventional schools. The system never realized economies of scale, and it is difficult to estimate what annual per pupil costs would have beyond if the model had been expanded beyond a thousand students (Jamison and McAnany, 1978). Neither of the efficiency objectives of instructional radio appears to have been widely pursued. But as a possible means of responding to the current crisis of rising demand and shrinking resources, this might well prove be an important option for the future. In-school radio education might be made to provide the same economic advantages that some believe can only be achieved through distance education; since radio education is structurally well adapted to being implemented on a mass scale, it could serve as a vehicle for a national educational reform effort, for example. The structured educational methods offered by interactive radio would be especially important at the primary school level, where traditional distance education programs that lack a structured classroom context are generally impractical. Finally, the use of radio could be justified from an efficiency perspective if it were found to contribute to improving the internal efficiency of school education by leading to improved indicators, such as better flow-through rates or lower costs per graduate. However, there is not - 46 - Educational Technology: Chapter 2 yet evidence that this has, if fact, been achieved. In any specific case, the potential of using instructional radio (or another educational technology) to promote educational quality through improved efficiency would be realized only when the marginal cost effectiveness of the radio project is better than what could be attained using conventional means. This is most likely to occur when the qualifications of teachers are low and where teachers and schools are widely dispersed and difficult to reach. Obstacles to implementation. Both technical and administrative factors have in the past affected utilization of radio in the classroom with varying degrees of consequence. Being unable to receive clear broadcast signals and to keep radios functioning have been the most frequently encountered obstacles at the technical level. The supervision and support required to prepare teachers, distribute supplementary materials, batteries, and assistance have also been frequent obstacles at the administrative level. Most developing countries have developed the capacity for better support and supervision of projects, making this less likely to be an obstacle in the future. However, most countries have also undertaken curriculum and materials development efforts that have institutional and economic implications that are not favorable to using alternative educational technology such as radio. This obstacle is likely to loom larger in the future. One if the obstacles to implementation that appears to be less of a problem for radio than for other educational technologies is that of teacher acceptance. The literature tends to show positive, if sometimes cautious, acceptance on the part of teachers. Although the believability of teacher acceptance based upon responses gathered from evaluation surveys may be questionable, we do not find mentioned in the literature on radio any record of the vehement teacher opposition that has contributed to the demise of many educational television projects. At an institutional level, the sustainability and wider implementation of promising experimental projects has had mixed results. Thailand has a long and successful history with the use of radio that started with the supplemental/enrichment applications during the Fifties and Sixties and continues through the present-day expansion of the project to serve the entire nation. The Dominican Republic appears to be moving forward with its use of RADECO. On the other hand, some apparently promising efforts have been abandoned. The successful Radio Mathematics Project in Nicaragua did not survive the political revolution in that country in 1979. The Radio Language Arts project in Kenya did not mpve forward, perhaps because of its lack of acceptance at the local level by those responsible for the primary school curriculum and existing educational broadcasting programs. Many of the early instructional radio efforts of the sixties and seventies in Mexico, Paraguay, Nigeria, Ghana, Uganda, and Sierra Leone were discontinued because of a mixture of financial and political difficulties. There has also been a question as to why more countries have not made use of the interactive radio model, especially in view of the positive results obtained. We can only speculate as to the reasons. But this low rate of - 47 - Educational Technology: Chapter 2 adoption may be about to change, as what looks liKe a significant "new wave" of interactive radio experience is now underway in several countries. Table 3 summarizes the status of these activities. Table 3 Status of Interactive Radio Projects Country Status Institutionalized? Students Bolivia Active No 20,000 243 schools Costa Rica Active Yes 36,000 1,200 schools Dominican Active Yes 1,300 Republic Ecuador Completed No 300 (pilot) 21 schools Honduras Aetive Yes 290,000 Kenya Terminat .d No 4,000 31 schools Lesotho Active ? Nationwide Nicaragua Terminated No 10,000 Papua New Active ? 750 Guinea Source: Friend Diaiogues, Inc. We asked Jamesine Friend, who has been involved with all of the uses of interactive radio, for her assessment of the obstacles to implementation that were faced in Nicaragua in the mid-seventies and telose that being faced today in the countries using interactive radio. Fr-.nd reports that the Nicaraguan project enjoyed the advantages of good technical skills and leadership on the part of local personnel and strong support from the Ministry of Education. The project faced few institutional constraints in carrying out its work; problems with respect to broadcasting proved to be minor. Although the project was mounted for purposes of research, there was early interest in moving forward to wider-scale implementation of the radio mathematics instruction. Plans for wider-scale implementation were halted with the onset of the revolution in Nicaragua. Had there been an attempt to implement radio mathematics more widely, Friend believes that the major obstacle would have - 48 - Educational Technology: Chapter 2 been the money and organizational capacity required to conduct the four hours of training given to teachers. In assessing the present situation where successful pilot projects are underway in several countries, Friend sees a different set of obstacles that must be addressed in order for interactive radio to enjoy wide-scale implementation in these countries. First, Friend believes that there is now more resistance to trying innovations on the part of ministries and donors than was experienced during the mid-seventies. Second, few countries want to use instructional materials that were developed for other countries. Radio lessons must be adapted for countries that are new users. Ensuring the needed quality control for successful adaptation of lessons requires time, money, and a commitment to quality. Third, the cost of batteries, which after teacher training is the largest component of the marginal costs of using interactive radio, remains an obstacle. Friend notes, however, that a solar-powered recharging option being tested in Honduras is likely to prove feasible. Although the solar power option would double the initial investment (now about $50 for the radio), this would pay off in three years. For the future, we anticipate that the technical problems related to sending and receiving a clear broadcast signal will pose less of a problem for developing countries that decide to make use of instructional radio. It will continue to be a problem in some countries, however, as will be the related problem of finding suitable broadcast slots during the day in the face of competing uses for radio. Configurations. Based on the interactive experience, the most typical configuration of instructional radio technology is likely to involve a daily radio lesson of 20-30 minutes, followed by a 15-30 minute teacher-led activity after the broadcast. Supplementary activities for the student to complete at school or home are likely to be provided through printed student worksheets. (In Nicaragua, costs were kept even lower by not using worksheets after the first grade, and by use of simple items such as bottlecaps for learning aids.) Typically, a school making use of instructional radio is provided with one battery-operated radio, which is moved from classroom to classroom. Teacher preparation associated with the use of instructional radio in developing countries involves training sessiona that may last for a few hours to one or two days. An optional delivery method for instructional radio (or radio-like) lessons would involve replacing radios and broadcasts with audio cassette players and audiocassettes. This would offer greater flexibility and teacher control in classroom organization, management, and scheduling. It would permit those schools or students who miss lessons to make them up at a convenient time; classes or groups of students having difficulties could repeat a lesson. The technology for rapid copying of cassettes, both in large and small numbers, is already being used for commercial purposes in many developing countries. A cassette can be copied for about a dollar. In some countries, the benefits of audiocassettes may justify the cost of reproducing cassettes and the added costs and complexity of distribution. However, tape cassette players are more likely to malfunction than radios. Cost. The costs associated with the use of instructional radio have been discussed at length in the literature. (For summaries, see Tilson, 1988 - 49 - Educational Technology: Chapter 2 and Anzalone, 1987; for cost analyses, see Jamison and McAnany, 1978; Leslie and Jamison, 1980; Jamison and Orivel, 1982; Friend and Kemmerer, 1885.) The use of interactive radio instruction generally requires substantial inputs of external technical assistance, which entails high capital costs for instructional development. These costs are considerably less when lessons that have already been developed can be adapted for use in a second country. The cost of developing one grade level of interactive radio lessons for a school subject from scratch is estimated to run between $500,000 and $1,000,000. In contrast, the cost of adapting one grade-level of existing radio lessons for use in a different country is estimated to run between $100,000 and $300,000 per subject. Annual per student costs also depend on the number of students being served; economies of scale are crucial to achieving low unit costs. Consequently, annual unit costs will be higher for applications in smaller countries or in pilot projects. Depending on specific circumstances and assumptions, experience with interactive radio in the late seventies and early eighties suggests that the costs of delivering instruction to clsses in existing schools in a single primary school subject would range from about $.40 to $1.17 per student per year. This cost would be an add-on cost to existing expenditures, and would vary according to such factors as the cost of preparing the accompanying classroom materials, and availability of free transmission time. Both fixed developmental costs and recurrent unit costs for interactive radio tend to be similar to those costs for developing and using school textbooks. Programmed Teaching/Learning Systems Another educational technology that has been used successfully to improve the quality of general primary education in developing countries is programmed teaching/learning. Like interactive radio, programmed teaching/learning systems demonstrate a high degree of correlation between materials and means of delivery and a high degree of attention to the systematic application of various principles of instructional development and design. The technologies differ from one another in the greater reliance on printed materials and the more active role for the teacher called for in programmed teaching/learning approaches. Moreover, programmed teaching/learning systems have been used to deliver a complete primary school curriculum, while interactive radio has so far been used mn applications isvolving only one or two subjects. Programmed teaching/learning systems can rely on a varied combination of methods of instruction and classroom organization. When compared to conventional instruction, they typically tend to rely more on deliberately structured presentation of content by teachers and on printed materials, and to employ greater use of practice drills. The characteristics of programmed teaching/learning are described in Nichols (1982), Cummings (1986), and Thiagarajan and Pasigna (1987). They usually include the following: (a) programmed teaching modules, which are step by step scripts for a teacher's presentation of a lesson (for example, during a series of twenty minute lessons, teachers are assisted in eliciting active student responses (often 7 per minute) and in providing corrective feedback); and (b) programmed - 50 - Educational Technology: Chapter 2 learning modules, whic'- are instructional materials sequenced in small segments. Programmed learning materials are often used in self-instruction, peer learning groups, or cross-age or cross-grade tutoring strategies. Programmed teaching/learning systems tend to lean more heavily on programmed teaching for the early primary grades and on programmed learning for the later ones (Thiagarajan and Pasigna, 1987). Programmed teaching/learning systems have been implemented in several important national efforts to improve the effectiveness or efficiency of primary education. The breadth of this experience is treated in Thiagarajan and Pasigna (1987) and Cummings (1986). Seven major applic.tions of programmed teaching/learning stem from the same origin, a research effort begun by educators in Southeast Asia at an organization called INNOTECH, and later given substantia' support by Canada's IDRC. The projects included: (1) Project IMPACT in the Philippines (discussed in Wooten, Jansen, & Warren, 1982; Flores, 1981; Mante, 1981; Pasigna, 1983), (2) Project PAMONG in Indonesia (discussed in Nichols and Dilts, 1984; Mudjiman, 1981; Klees and Supraman, 1984), (3) Project RIT in Thailand (discussed in Potar, 1984), (4) Project UPE/IMPACT in Bangladesh (discussed in Claveria, 1982); (5) Project INSPIRE in Malaysia (discussed in McMaster, 1978); and (6) the IEL Project in Liberia (discussed in Thiagarajan and Pasigna, 1985; Kelly, 1982; Nichols, 1983; Windham, 1983; Chapman and Boothroyd, 1986). Project number (7), in Jamaica, was unsuccessful, and this program is discussed in McKinley (1981j. In addition, an application of programmed teaching/learning is underway in Somalia (IEES, 1985) and Belize is experimenting with a programmed teaching/learning approach that makes use of posters to deliver programmed teaching in reading and social studies. The contributions that the IMPACT project has made to improved educational effectiveness in a wide variety of conditions in developing countries are thoroughly examined in Cummings (1986). He found that IMPACT can improve the quality of education with no increase in cost in rapidly growing, densely populated areas where trained teachers are scarce, as well as in other areas that are poorly served by the conventional education system, and sparsely populated. Where such difficult conditions are not as acute, IMPACT was also found to be effective in supplementing and enriching mainstream education, as for example, in the relatively prosperous country of Malaysia. A valuable aspect of the programmed teaching/learning approach appears to be its ability to greatly increase efficiency (Thiagarajan and Pasigna, 1987). The programmed teaching/learning systems described above achieved their equivalent or improved learning effectiveness with teacher/student ratios ranging from 50 to 1 to over 100 to 1. Consequently, Project IMPACT in the Philippines was able ta provide instruction as good as or better than conventional instruction at about half the cost. Similarly, in Indonesia, equivalent educational effectiveness was cheaper in the large school and school equivalency versions of PAMONG (but more expensive in smaller schools). Although cost data were not available for Thailand, the reduction in instructional time realized through the use of programmed teaching/learning ranged from 15 to 80 percent, which would make it possible - 51 - Educational Technology: Chapter 2 to reduce by 2 the number of teachers required for a school with 6 classes. In Liberia, improved effectiveness occurred with a 71% increase in enrollments without increasing the number of teachers, and the use of programmed teaching/learning proved to be more cost-effective than a quality improvement strategy using textbooks, especially with larger classes. The introduction of programmed teaching/learning systems appears, in many cases, to be more durable than other applications of educational technology in developing countries. For example, the IMPACT/PAMONG classroom materials are reportedly used on a regular basis in about 5,000 schools in Indonesia (Thiagarajan and Pasigna, 1987). The number of schools in Thailand using IMPACT/RIT appears to be increasing and reportedly could reach 10,000 in about eight years (Thiagarajan and Pasigna, 1987). In Liberia, the IMPACT/IEL materials will be disseminated nationwide and will provide the core of the primary school curriculum. However, political reasons apparently led to the termination of Project PRIMER in Jamaica, and slowed the expansion of Project IMPACT in the Philippines. Obstacles to imnlementation. The major obstacle to implementing programmed teaching/learning systems appears to be philosophical objections on the part of some educators. The early use of programmed instruction in countries like the United States was not widely regarded as effective. Despite the changes in perspective and technique that have taken place in instructional programming during the past severa. years, many educators, both in developed and developing countries, find the idea of "programming" objectionable, due to its early associations with behavioral psychology; they fear that it is likely to encourage rigid teaching practices and rote learning. Thiagarajan and Pasigna (1.987) point out that the development of programmed teaching/learning grows out of an instructional systems design perspective that is not widely found in developing countries. It is more likely tl.:.t instructional materials in the Third World will be produced by those with a "curriculum development" or subject matter orientation. Moreover, the use of programmed modules may clash with the use of textbooks. Liberia is attempting to i.econcile such differences (IEES, 1986). The philosophical objections from teacher trainers are likely to be especially significant. Many will disagree with orienting teacher education towards the specific methodology of programmed instruction at the expense of the fundamental educational courses they are more accustomed to teaching. There is evidence that teachers support the use of programmed teaching/learning systems, perhaps due to the fact that more is required from teachers to make the technology work than may be the case with using textbooks, radio, or television. Programmed teaching/learning materials are not "teacher proof." Depe4ding on one's point of view, this may be regarded as either an advantage or an obstacle to implementation. Another obstacle to the use of programmed teaching/learning materials is that distribution of materials, which are usually produced in series of modules, can be more complex than, for example, distributing textbooks. A high degree of organizational effort is required to deal with production and distribution arrangements and with introducing and supporting the considerable degree of change brought to the way schools are organized and to the teaching - 52 - Educational Technology: Chapter 2 practices that take place there. Finally, it has been observed that the success of programmed learning modules for self-instruction or for groups is affected by students' reading abilities, which may be too low to permit following the modules without compensatory instruction. Aida Pasigna, working with the USAID Basic Research and Implementation in Developing Education Systems Project (BRIDGES), is conducting an analysis of the factors that have facilitated and impeded the sustainability of several of the programmed teaching/learning systems that were introduced a decade ago or longer. Pasigna reports that one of the obstacles to sustainability of the efforts in the Philippines, Thailand, and Indonesia was that the issue of institutionalization was not addressed during the activities of the pilot projects. There was no provision made for fitting the program into ministries, supplying materials, and supervising teachers once the projects ended. The continuation of the use of the programmed teaching/learning systems in the Philippines, Thailand, and Indonesia must operate within a wider context of decentralization in these countries. It is necessary for schools and teachers wanting to use the systems to get a commitment of discretionary funds from provincial, district, or local officials. This has been successful in many places, but poorer communities are often unable to support the continuation of programs in their schools. The obstacle of financing programmed teaching/learning systems was also complicated by the fact that whatever the merits of the system, governments opted for the use of textbooks when donors made funds available for this purpose. According to Pasigna, the implementation of programmed teaching/learning in Thailand faces an additional financial obstacle. The use of programmed teaching/learning in Thailand was attractive in part because it generated savings on teachers' salaries because it allows fewer teachers to serve greater numbers of students. However, it proved to be very difficult to reapply these savings within the budget to pay for the IMPACT instructional materials. Notwithstanding these problems, programmed learning has been one of the most successful of the learning technology innovations to be tried at the primary school level. As proof of this, in Bangladesh, IMPACT has met with quick acceptance and is destined to play an important role in the national planning process for a national program for expansion of primary education (Cummings, 1986). Use of IMPACT materials in Indonesia, Malaysia, and Liberia is also spreading due to their popularity among teachers in overcrowded rural schools. Configurations. Programmed teaching/learning revolves around sets of printed materials that may include all or some of the following elements: programmed teaching modules; programmed learning modules for students; booklets for reading, review, and practice; tests; teacher guides; and management guides. Instruction in programmed teaching/learning classrooms may be organized and delivered in a variety of ways. In Liberia, for example, early grade classrooms are divided into two sections. During a 45 minute lesson sequence, the teacher takes one section at a time and teaches a 20 minute lesson following the procedures spelled out in a lesson module. During this time, pupils in the other section work in groups of three or more on review booklets. During the second half of the - 53 - Educational Technology: Chapter 2 third grade, pupils make the transition to programmed learning. Here they work in groups of three to seven on modules, which are completed in one to two days. Pupils are tested on a module after the second day. Teachers provide little direct teaching. In the Philippines and Indonesia, programmed teaching/learning follows a similar pattern. The transition to peer learning takes place at the end of the third grade. The last three years of primary school are spent in peer learning with teachers providing support to groups and help to individual pupils. In the Philippines, secondary school students have been used as unpaid tutors. Community aides assist with the testing, and parent or other members of the community teach crafts and other skills. Costs. The costs of using programmed teaching/learning systems are summarized in Thiagarajan and Pasigna (1987). The costs of using programmed teaching/learning systems in the Philippines was half that of conventional instruction (McMaster, 1978). In Indonesia, large school applications of the PAMONG programmed teaching/learning system were less costly than conventional schools but slightly more expensive in small schools. In Liberia, the costs of programmed teaching/learning materials were compared to those of textbooks and proved to be less expensive (Windham, 1983). The major experiences using programmed teaching/learning have benh large-scale curriculum development efforts that included large provis ns for foreign technical assistance. In Liberia, the costs of developing e primary school curriculum using programmed teaching/learning was about .8 million. In Belize, however, programmed instruction was developed and ested in two subjects with technical assistance costing under $100,000. Recurrent unit costs are dependent on the numbers of students in the school. In Liberia (1982), per pupil unic costs for grade one (based upon amortizing materials costs over their expected life) were estimated to be $5.91 for classes with 20 pupils and $2.36 for classes with 100 pupils. In Indonesia (1984), these costs were estimated to be $5.92 in schools with about 50 pupils and $3.59 in schools with about 200 pupils. These costs are for materials covering instruction in several subjects. Instructional Television Instructional television has been used in a substantial number of attempts by developing countries to improve the quality of general education in their educational systems. It has been used at the primary, secondary, and tertiary levels to teach virtually all subjects. In spite of this, its effectiveness as an educational too'.. is the subject of considerable controversy; if it is now possible to identify a model of radio instruction that produces consistently good results in different subjects and in different countries, the same cannot be said .'or television. This is surprising, in view of the fact that an educational television series developed for use outside of school, Sesame Street, has broad, intercultural appeal and stands out as one of the most widely-used educational products of our times. Television has been found at the forefront of numerous, multifaceted attempts to reform education in developing countries. Applications of - 54 - Educational Technology: Chapter 2 instructional television have been used as stand-alone systems for direct instruction or for enrichment in a host of countries, including: Algeria, American Samoa, Brazil, Colombia, Egypt, El Salvador, Ethiopia, Ghana, India, Ivory Coast, Korea, Malaysia, Mauritius, Mexico, Niger, Nigeria, Peru, Senegal, Singapore, Uganda, Zaire, Zambia, and Zimbabwe. Television, as Saloman and Gardner (1986 p. 13) put it, "is mostly a one-way medium of communication, basically designed for entertainment and converted (or subverted) to education..." In developing countries, the use of instructional television has been driven by the attraction it commands as the medium of modernity. Sometimes its instructional use may have been an alibi for its commercial development. At the same time, however, television's potential for bringing real-life situations and events into the classroom and for motivating and holding the attention of students -- often by blurrir the distinction between education and entertainment -- is inescapable. The most important national experiences involving classroom use of instructional television are discussed throughout the literature. These include: American Samoa (Schramm et al., 1981); Brazil (Oliveira and Orivel, 1980; Arena et al., 1977); El Salvador (Mayo et al., 1976; Ingle, 1976); Ivory Coast (Kaye, 1976; Hawkridge and Robinson, 1982; Eicher and Orivel 1980; Grant, 1976; Evans and Klees, 1976; Carnoy, 1976); Niger (Clearinghouse on Development Communication, 1982; Silverman, 1976; Carnoy 1976); Colombia (Comstock and Macoby, 1966; Schramm, 1967; Lyle, 1982; Carnoy, 1976); India (Sampath, 1978; Shukula, 1979); Senegal (Orivel, 1981); and Nigeria (Schramm, 1967; Carnoy, 1976). Up until this point, most of the major national experiences with television in the classroom have involved open broadcasting of television lessons, but television is now being used in conjunction with other technologies, particularly the videotape recorder. The use of VCRs is becoming very common in many developing countries, and allows education systems to avoid many of the constraints imposed by broadcasting. If the use of videocassettes "frees" instructional use of television from inflexible broadcast schedules, it has not overcome one of the most obvious pedagogic limitations of broadcast television -- lack of interactivity. Until now, interactivity using television was generally possible only with the use of videodiscs, although some work has been done to develop devices that could allow interactive use of videotapes. Obstacles to implementation. Much of the history of instructional television in developing countries is a chronicle of the obstacles encountered and the disappointments that followed. At the top of the list of obstacles have been costs, closely followed by the problem of ambiguous outcomes in the area of improved educational quality (costs will be discussed in a separate section, below). The use of television in classrooms in developing countries has generally proved to be an expensive addition to educational budgets, but little evidence has been gathered to show that the expenditure brought results that were worthwhile. it look at the research evidence shows that the effectiveness of instructional television was never clearly demonstrated in any of the major television projects for non-university level general education that have taken place in developing countries. For example, instructional television had - 55 - Educational Technology: Chapter 2 spotty results in improving achievement in different school subjects in El Salvador. Mayo et al. (1976) compared the performance of the groups of students receiving instructional television with those benefiting from other parts of the reform but not receiving instructional television. Although the television students outperformed the non-television students in the seventh grade, at the eighth and ninth grade, according to Mayo et al., the "advantages of ITV were not apparent." In the Ivory Coast, where instructional television reached more than half a million students, there was similar inconsistency in results. Hawkridge and Robinson's (1982) discussion of these results indicate that television appeared to be most effective in teaching spoken French but less so in teaching students to write in French. Moreover, television students in the second grade encountered difficulties because the curriculum assumed mastery of what had been taught the previous year. The use of television during the sixties in Colombia, Niger, India, and American Samoa also showed results that, on the whole, should be described as mixed. There may have been evidence that television was effective in some subjects at some grades, but in other instances, non-television control groups did as well or better. In general, the effectiveness of instructional television fell far short of expectations. Most importantly, television did not yield the spectacular results that would have been required to justify the high levels of investment involved. Again, however, it should be kept in mind that these results only tell us about the effectiveness of the design and delivery of those particular television programs in the face of the conditions that were encountered at the time. We should not therefore conclude that the medium of television cannot be used to teach effectively. A second obstacle that was encountered was the fact that countries were not usually equipped for the large-scale organizational requirements to produce, distribute, and support what often amounted to an entirely new national curriculum delivered through telev..sion. Often these functions were dependent on expatriate advisors, which created additional problems. Third, in several countries, being able to receive clear broadcast signals, having access to electricity, and keeping sets operational posed enormous logistica'l difficulties, especially in rural areas. Fourth, fitting television into the classroom often proved difficult because of the rigidity imposed by broadcast schedules. Another important obstacle to the use of television has been strong opposition from parents, teachers, and government officials. Such opposition proved to be fatal in American Samoa, El Salvador, and Niger. The experience of El Salvador is especially instructive. We asked John Mayo, who evaluated this experience during the seventies, to look back on the obstacles that were encountered. Mayo reports that El Salvador did many things right in its efforts to use television. Unlike some of the other prominent television projects of the time where activity was dominated by expatriate advisers, Salvadoreans were in charge of the development of instruction. Moreover, television was used thoughtfully, as part of a strategy for comprehensive reform of the education system. However, according to Mayo, the major obstacle to successful implementation was the lack of consultation with teachers at all phases of program development and implementation. The concerns of teachers and their union were simply not addressed in the implementation strategy. Mayo believes that the resistance of teachers was - 56 - Educational Technology: Chapter 2 not directed so much against the use of television as against the way the reform was conducted. One must not therefore automatically conclude that the medium of television is somehow unsuited for educational purposes. It is still certainly possible that, with the right investment of time, talent, money, and commitment to the application of sound methods of instructional design and development, instructional television could produce good results, as interactive radio has, for improving education in developing countries. Configurations. Television is most often used in the classroom as a means for providing direct instruction in one or more subjects. This instruction is provided in regular classrooms equipped with a television or in a special classroom for television viewing. Television lessons received in the classroom are either watched as they come in, or they are recorded on a VCR for later viewing. Malaysia has used the latter option successfully (Gallagher, 1982). Another possibility would be to use television lessons that are recorded on videotape recorders but are distributed without broadcasting. It is not yet clear which of the various configurations for delivering and using instructional television are likely to find wide use in developing countries. But it is clear that the instructional possibilities for using television have changed considerably since the time of the most well-known experiments with instructional television in developing countries. The most important development is the spread of videocassettes and VCRs to many developing countries; despite the costs, they are in relatively wide use in many of the urban areas in countries such as Brazil, Malaysia and South Korea. VCRs allow the use of video for teaching on a much smaller scale than a full-blown broadcast project, and the use of tapes frees teachers and students from the restrictions and limitations of a fixed broadcast schedule. In the United States, some ongoing development efforts are being made to design a microcomputer control system that would allow a videotape system to be made interactive. Some experience already exists in the use of VCRs for microteaching. This is a technique for teacher training that was tried in Lesotho, Zimbabwe, and the Philippines, mainly in the mid 1970s. Microteaching involved practice in teaching a "mini lesson" in which a student teacher practices a particular teaching skill. The lesson is videotaped and then reviewed by the student and a supervisor. The student then usually gets the opportunity to reteach the lesson, incorporating feedback received from viewing of the videotape and the supervisor's comments. Recently, television programs have begun to reach the classroom in an increasing variety of ways. Nugent (1987) reviews some of the new options. The most common way has been via broadcasts from educational, commercial, or government-sponsored television stations using ground transmitters and normal television frequencies. Alternatively, educational applications of television have sometimes relied on closed-circuit broadcasting. Satellite technology could make it easier to reach some remote locations and continues to be mentioned as a possible conduit for educational applications in developing countries. In the U.S., public television stations - 57 - Educational Technology: Chapter 2 now receive many of their educational programs for local transmission via satellite. Satellites have made it technically possible to broadcast directly to schools that have special equipment, if ways to overcome the high. cost of transmission and of reception equipment can be found. Cable television is also finding educational applications is some places in the developed world. Costs. A major obstacle to the use of instructional television has been the high cost of production, broadcasting, and reception. Three points emerge from the literature on the costs of instructional television in developing countries. First, the total cost for educational television projects is high when compared with other media or methods of instruction for equivalent programs. Schramm (1977) estimated that, all othar variables being equal, television will cost about three times more than radio. Second, the use of television seems to require large initial investments and considerable patience in realizing returns to this investment. Jamison, Klees, and Wells (1978) observed that "the heavily front-loaded costs and rear-load utilization of technology projects results in a requirement that projects last 10 to 20 years to allow all unit costs to fall to a reasonable level." Third, the per unit costs of instructional television are highly dependent on the total number of students served. There has been wide variation in the number of students served in the major instructional television projects. Consequently, average annual costs per student varied considerably. Jamison, Klees, and Wells (1978) summarize these costs for several countries. The amounts are in 1972 dollars. These include American Samoa (1972; 8,100 students) $160, El Salvador (1972; 48,000 students) $24, and Colombia (1965; 275,000 students) $3. For the Ivory Coast (1976), based upon enrollments of 336,000 students, average annual costs per student were estimated at $13 (Carnoy, 1976). In Niger, however, with only 800 students taking part in the pilot phase in 1969, these costs were some $1156 (Carnoy, 1976). This variation suggests that instructional television will appear expensive or cheap depending on the number of students it reaches. Perhaps the most important question to be answered regarding classroom television in developing countries is whether recent developments, such as changing technology, greater local availability of facilities and expertise, and new teaching methodologies, might now make it possible for television to be used economically with applications of fewer than a million students. A study for the World Bank (Feliciano et al., 1981) anticipated many of the emerging technical options for production, distribution, and utilization of television programs that might make for cheaper instructional television. We have yet to see, however, any systematic application of these new possibilities in classroom use of television in developing countries. The major question at this time is what hold the past will exercise over future possibilities for using television in education in developing countries. Computers In educational settings, the computer has replaced the television as the medium of modernity. This is as true for developing countries, where computers are increasingly coming into use in production, commerce, and - 58 - Educational Technology: Chapter 2 education, as it is for the industrialized North. The push to use computers in schools stems from a number of specific goals: to develop a computer- literate population for industrial development, to prepare students for computer-related job opportunities at home, to equip students who will work or study overseas, and to improve educational quality. We do not have up-to-date information on how extensively developing countries are now using computers in their educational systems and for what purposes. We believe, however, that there are a number of exploratory efforts taking place in individual schools in many developing countries. We know of efforts that have been undertaken in Mexico, Chile, Peru, Argentina, Colombia, Brazil, the Philippines, Thailand, Malaysia, Trinidad, Belize, Grenada, India, Sri Lanka, Korea, Kenya, Senegal, and Swaziland. Costa Rica is reportedly mounting a major initiative to introduce computers across the board for primary and secondary education. The literature that evaluates or describes actual computer use in developing countries is scant. Outlines of the issues as they are currently understood can be found in Carnoy and Loop (1986), Hebenstreit, (1986), and Papagiannis et al. (1987). Also of interest is Harper's (1985) account of early attempts to introduce computers in Papua New Guinea and Malaysia. There are discussions about experimental work with LOGO (see, for example, Sylla, 1985). There are analyses of experience with computers in secondary schools in Kenya (Makau, 1988; Wray, 1988) and in Trinidad (Williams, 1987). Grenada's use of computer-assisted instruction at the primary school level is described in Freeman (1987) and in one of the early attempts to explore the effects of computers on the achievement of students in a developing country (Anzalone et al., 1988). Computers have seen a variety of applications in general education that are still at the pilot stage, and information is often incomplete. The principal types of use related to general education include the following: o Computer literacy: These are the various attempts to provide students with a basic orientation toward computers. They may address both practical uses and knowledge about the social dimensions of computers and information technology. Applications of this approach are found in Tunisia, Mexico, and India. o Conventional computer-assisted instruction: These applications provide direct instruction in school subjects. Applications of this approach include Malaysia and Grenada. O Multiple-use through the extended computer room: Many schools, often private schools, raise money to buy one or more computers. These are often put aside to teach commercial courses, but their use can be extended to general education purposes and computer literacy for students at school and sometimes for others in the community during the evening. - 59 - Educational Technology: Chapter 2 o "Opening uD" the instructional process: Many educators look toward the use of computers to go beyond the usual objectives of general education and attempt to foster critical thinking, creativity, or problem-solving. Kenya is an example of such an approach. Similarly, attempts to use LOGO have also found application in developing countries such as Senegal, Brazil, and Argentina. o Programming: A common use of computers in education in developing countries is to teach computer programming. This is a frequent -- and many believe counterproductive -- approach to computer literacy and to preparing students for using computers for the sciences and mathemati(s. O Using computers as productivity tools: In developed countries, the educational use of computers is increasingly directed toward teaching students how to use productivity applications like word processing, spreadsheets, and databases. This instruction is linked to varying degrees to various school subjects. An example of such use in a developing country was in a pilot study at the Belize Teachers' College, where Mora et al. (1988) found that instruction in using productivity tools was associated with improvements in mathematics performance. o Imoroving general education through vocational training: The use of computers in vocational training often opens possibilities for general education for school-age and adult participants that might otherwise not be available. A study in Jamaica found that computer-assisted instruction in basic mathematics skills proved to be significantly superior to conventional instruction (Anzalone, 1987). o Complementary uses: Computers may be used in general education for instructional purposes complementary to, but not replacing, the usual school curriculum. This would include such applications as the IBM "Writing to Read Program." This program has been successfully used in the Philippines. The use of computers in classrooms in the Third World is only now beginning to be evaluated, and it is difficult to predict their effectiveness based upon actual experience in developing countries. Some mixed but generally positive indications of potential effectiveness were discovered in studies of computer usage in Belize (Mora et al., 1988), Grenada (Anzalone et al., 1988), and the Philippines (INNOTECH, 1987). Obstacles to Implementation. The biggest obstacle to the use of computers for improving general education in developing countries is, not surprisingly, cost. As mentioned below, even though it is hard to estimate - 60 - Educational Technology: Chapter 2 the exact cost of educational computer applicatio,as in developing countries, few doubt that it will be expensive. Infrastructure conditions vary considerably from country to country, and the political importance assigned to national development of computer expertise will affect the amount of resources each country's educational sector will have at its disposal. There are likely to be affordable options, but this will require careful study, experimentation, and plAnning. The second major obstacle is the problem of software. For the moment, few countries are in a position to undertake or pay for expensive software development projects, and their use of computers will likely require them to make do with educational software developed for use in other countries. How well this software "travels" remains to be tested. Differences in curricula, languages, and operating systems, as well as copyright and software rental arrangements, could present obstacles in many countries. Hardware incompatibility is another facet of this obstacle; careful planning is required to avoid the purchase of incompatible systems throughout the educational sector. The third major obstacle is that of providing the training for teachers to become confident and motivated managers of computer usage in schools. Makau (1988) discusses how tnis obstacle presented itself in Kenya. The technical and logistical problems posed by electrical supply, heat, dust, and access to spare parts must also be addressed in implementing computer technology in many schools in developing countries, much as for other technologies such as television. Configurations. There are many possible configurations of hardware and software for computer use in schools in developing countries. These would include one or more microcomputers (used separately or in a network), a dedicated minicomputer with terminals, or a mainframe computer with terminals. The first option is likely to be most common because of the advantage of being able to begin implementing the use of computers without having to make a large initial investment. The powerful minicomputer systems used for education offer comprehensive curriculum packages and have different degrees of "openness." Some will run only the curriculum packages developed for the system. Others offer the advantage of terminals that may function as stand- alone microcomputer 'clones." The possible configurations of computers with various input, data storage, printing, and data transmission devices are myriad and are not considered here. It must be mentioned, though, that the diversity of hardware standards and configurations could result in the installation of incompatible systems within schools, or throughout a school systim, and close attention must be paid to this issue. Costs, We could find no literature yet on the costs of in-school applications of computers in developing countries. It is difficult to provide even rough estimates because of the great variety in technology configurations and applications, the varying ways that instructional personnel are utilized in applications and their salary levels, and the rapidly changing costs of hardware and software. It is hoped that current pilot efforts will soon produce some useable data on costs, so that an attempt can be made to develop a projection for the costs of a "typical" application in a "typical" country. - 61 - Educational Technology: Chapter 2 This would at least set some benchmarks for planners, as they undertake the task of predicting the costs of using educational computers in their own countries. Videodiscs As recently as two years ago, the use of videodiscs for general education in developing countries seemed, at best, to be a remote possibility. However, within the last year, an African country is said to have developed, with assistance from a U.S. firm, a series of 23 videodiscs for teaching mathematics in teacher education.1' China is also said to be embarking on a major videodisc manufacturing program, but no details on this project are currently available. So it is evident that the use of computers in education will increasingly be enhanced by the rapidly emerging capabilities of videodisc technology. Videodiscs permit large amounts of video and audio material to be stored on a laser disc, and to be subsequently retrieved in a pre-designed instructional sequence. When coupled with a computer, this material may be used interactively. Interactive videodisc technology weds the visual power of television to the interactive power of the computer. For instruction, a variety of print, graphics, still pictures, motion, and audio can be combined into instructional units. There are two types of videodisc applications that might be used in schools in developing countries. The first of these is a linear system. Here, fixed instructional sequences to be followed by individual students or groups of students are stored on the videodisc. Although one may begin at or return to different points on the disc, the instruction does not branch to different points based upon particular responses. Such systems may be "driven" by the teacher using a remote control. The development of teacher- driven linear videodisc instruction is often based on software developers' expectations that schools will not be able to afford multiple workstations in classrooms. The second type of videodisc application is interactive video. This is achieved by using the videodisc player as a peripheral to a computer. The computer permits rapid access to various parts of the disc and allows the lesson to adjust itself according to the user's responses. Interactive video can be used in individual or group applications. The industry is relatively new, and standardization is still a problem. Currently, interactive video is delivered on 12 inch optical discs that can hold about 100,000 still frames of visual material. In the future, compact discs are likely to be widely used. These discs resemble compact musical discs and have the capacity for holding huge amounts of information. There are several "standards" of compact discs that are being developed. These include CD/ROM (compact disc read only memory), CD/I (compact disc I/ The country has chosen to withhold data on its experience to date, and the company involved is not willing to release the name of its client at this time. - 62 - Educational Technology: Chapter 2 interactive), and CD/V (compact disc video). The resources required to produce videodisc software are considerable. It is estimated that it takes about twice the resources to develop an interactive videodisc program as a computer software program (Strauss, 1988). Whether the added capabilities offered by videodiscs prove to be cost-effective for general education in schools in developing countries remains to be explored. Obstacles to implementation. The major obstacles to using videodiscs for improving educational quality in developing countries are the hardware costs and the availability of software. Costs (discussed below) are high, but lower cost options that may be appropriate for use in schools in developing countries are becoming available. The high production costs of software mean that most developing countries will not be able to produce their own software for some time and would have to rely on imported software. (Again, however, we know of one large African country that has developed, with technical assistance, its own videodisc instruction program for teaching mathematics to teachers.) Courseware importation may be more feasible at the university level, and in the less culture-bound subjects such as mathematics, engineering and the sciences. Currently, the selection of software that is available for general education is extremely limited. Another obstacle that is currently constraining the spread of videodiscs in U.S. schools is the variety of incompatible "standards" for hardware and software. This reduces the range of software a user is able to run on whichever system that a school selects. An additional obstacle is that revisions to the software require that the program be entirely remastered, and it is therefore difficult to modify or add to instruction once a videodisc is produced. Obtaining replacement parts and having trained personnel for setting up, troubleshooting, and keeping hardware/software systems in good operational order is also likely to be a significant obstacle. Configurations. There are several configurations of videodisc technology that are available or emerging as possibilities for school use in developing countries. One such configuration is the linear videodisc system. This involves interfacing a videodisc player to a television set and adding a device for zontrol by the teacher or student. Although this configuration lacks the interactivity made possible by using a computer, it does permit instructional sequences to be highly sophisticated and allows a wide range of audio and visual material to be brought into play for learning purposes. A second option is an interactive, "level III" system. A basic system would include a videodisc player, a monitor, and a computer or other microprocessor device. The videodisc player uses the 12 inch analog optical disc. These workstations may be used by an individual student or by an instructor plus a group of ten or more students. A third option, that is just now becoming a possibility, is the compact disc system. This would be comprised of a compact dis; player, a microcomputer, and a monitor. The disc player would use the smaller, high storage digital discs. It is expected that more advanced compact disc players will be available soon that can be connected directly to a television set and will not require a computer. Interactivity will be possible through the use of a remote sensing device. Such a configuration is likely to be commercially developed for entertainment and educational uses in homes. - 63 - Educational Technology: Chapter 2 Costs. The cost of a workstation for using interactive videodiscs would probably begin at about $2000. More sophisticated "Level III" workstations may cost as much as $20,000. It is expected that as compact discs become available, a player that connects to a home television will be available for about $500. For a linear system that operates without a computer and connects directly to a television, the cost of the disc player is about the same as a videocassette recorder. The cost of producing a unit of instruction using videodisc is high. A unit at the high end of quality may run about $150,000; a series of instructional videodiscs used in U.S. law schools cost about $30,000 each to produce. The high fixed production costs require a large number of users in order to make units affordable. A $30,000 disc used on 100 systems would result in unit costs of $300. For 1,000 systems, it would result in per student costs of $30. Electronic Learning Aids A technology area that is rich in promise, but that has not yet been exploited for the purpose of improving the quality of education in developing countries is comprised of an assortment of devices that may be referred to as electronic learning aids. These devices can be divided into two main categories: the directly instructional and the indirectly instructional. Both categories are hand-held, battery-operated aids that are used to provide instructional supplements at the primary school level. The directly instructional aids are microprocessor-driven devices that provide instructional routines in language and arithmetic. The most well-known of these are the Speak & Read, Speak & Spell, and Speak & Math units, manufactured by Texas Instruments, Inc. Others include Video Technology's Learning Window and Coleco's Talking Teacher. The aids present problems to be solved. The Texas Instruments' series, for example, uses synthetic speech to help present problems, give directions, reinforce sound- letter and number correspondences, and provide feedback on the correctness of the response. The user enters a response on a keyboard, which then appears on a small fluorescent display. Electronic learning aids provide drill and practice in important skills related to literacy and numeracy. While few educators deny the importance of drill and practice for developing basic skills, traditional classroom drill and practice is often boring and avoided by teachers. The game-like routines on the electronic learning aids arp appealing and motivating -- indeed, they tend to be sold as toys. Like microcomputers, they bring a degree of interactivity to individual responses that is difficult or impossible to achieve in conventional classroom instruction. There has been one documented pilot study of the use of electronic learning aids in a developing country. Anzalone and McLaughlin (1984) report on the use of the Speak & Read and the Speak & Math in primary schools in the Kingdom of Lesotho. Interventions lasting four to twelve weeks proved to be feasible and effective in primary schools in that country. A study of the effectiveness of the Speak & Math in low-achieving primary schools in Belize - 64 - Educational Technology: Chapter 2 is underway, and early indications point to a strong impact on the development of computational skill. The second group of instructional aids, those that act indirectly, include all of the electronic devices that need to be used with accompanying teacher-directed exercises or printed materials in order to achieve instructional objectives. These include basic electronic calculators and such things as portable spelling checkers. Electronic calculators have penetrated most corners of the developing world. They often furnish solutions to the mathematics problems of daily life for people who are unable to solve them using paper and pencil. Parents and teachers, however, tend to be skeptical about usirg them in the classroom and believe that they may be counterproductive to learning basic arithmetic. Elsewhere (Anzalone, 1987), we have reviewed the evidence from developed countries that suggests calculators can be an aid to learning and not a crutch. Consideration should be given to the notion of "calculator literacy" as part of the primary school curriculum; this might be especially useful for improving the external efficiency of education for children who drop out of primary school without acquiring basic mathematical skills to the degree that ensures retention. Recently, portable spelling checkers have appeared on the market that can be used to verify the correct spelling of as many as 10,000 words. This version of microchip technology sells for about the same price as the interactive aids described above, but its instructional application in developing countries has not been tested, nor are cost comparisons with other alternatives such as dictionaries, available Configurations. Electronic learning aids are not capable of stand alone teaching, and cannot deliver all or most of a year's instruction in a school subject. Their use in developing countries would almost certainly occur as an add-on element to be used in combination with conventional instruction or other technologies. In Lesotho, it was found that each aid could be used for a group of four students. Consequently, electronic aids might be employed together with strategies for cooperative learning (Slavin, 1987). The aids used in Lesotho were battery-operated and contained an exercise book prepared by the manufacturer. They were used as supplements to instruction in mathematics and reading for about twenty minutes per day, three to five times per week. The Speak & Read provides practice with about 200 of the most basic English words. Insertable modules would extend this range to about 1,500. The Speak & Math is capable of generating about 100,000 different mathematics problems. These are generated randomly within different problem groups and within three levels of difficulty. The Lesotho study found that aids could be introduced into the classroom and used effectively with very little preparation of teachers and students. For the other category of indirect electronic learning aids, those that teach indirectly, the devices do not provide direct instruction. Consequently, the teacher would have to present the problem for students to solve or words to be spelled. Alternatively, this could be done using printed materials. - 65 - Educational Technology: Chapter 2 Obstacles to implementation and costs. The major obstacle to the use of electronic learning aids is the cost. Devices like the Speak & Read retail in the United States for about $40. Unlike for microcomputers, prices for these aids have not fallen during the past several years, although this could change as a result of competition from some recent new entries into the market. Unit costs will depend on how the aids are used; costs may be contained by using the aids in groups and rotating them among grades and possibly among schools. Following such a strategy, Anzalone and McLaughlin (1984) estimated that a 60 hour supplement of interactive drill and practice could be delivered, exclusive of battery costs, for about $.94 per student. Spelling checkers sell for about $50 in the United States. Electronic calculators, often solar-powered, sell for as little as $5. A potential obstacle may be found in the area of language. First, there is the synthetic speech contained in the aids. The aids that were tested "spoke" English with a kind of machine-line accent. The volume of the speech is potentially an obstacle in noisy classrooms. Students in Lesotho adjusted quickly to the speech, but this could present a problem in settings where students are less familiar with English. Second, existing learning aids all assume a thorough knowledge of English, and translation and adaptation of aids to other languages, particularly ones with non-Roman alphabets, may be costly and problematic. Other languages and higher quality synthetic speech could be developed in future generations of these aids, but this is only likely if there were a proven educational market for them. Electronic learning aids should not be overlooked as a potential supplement to primary education in some deryeloping countries. The main advantage that they offer, compared to non-electronic learning aids such as blackboards and maps, is that they provide focused learning tasks. Where teaching is of poor quality and where inadequate opportunities exist for students to do drill and practice routines, such devices could take an active role in improving instructional methods. The possibilities of electronic learning aids are not widely recognized, and their potential has not been adequately explored through investigation. There is some evidence that electronic aids are now receiving more attention in schools in the United States. The aids uere at a disadvantage in the past because of the greater power of microcomputers, and because they suffered from the stigma of being an educational/entertainment product developed for use at home. Manufacturers also appear to be making a new effort to improve the instructional capabilities of the aids, which seem to have been ignored during the decade since Speak and Spell first appeared (Sussman, 1988). If interest in using electronic aids were to grow in developing countries, this might well become a new product line for manufacturing in the Third World. - 66 - Educational Technology: Chapter 2 IV. Comgarative Implementation Advantages The preceding discussion has highlighted the qualities and problems of the many educational technologies that could be used, now or in the future, to improve general education in developing countries. We now turn our attention to choosing among the many available educational technology options. A fundamental question arises: Can a developing country make a decision for a particular technology over the others based on demonstrated comparative advantages? With regard to comparative effectiveness, the answer, in a strict sense, is no. Existing research does not provide us with comparisons that permit us to conclude that one technology provides instruction more effectively than another. Defensible generalizations regarding comparative effectiveness simply cannot be made, because it is impossible to control for methods or important contextual conditions that vary from application to application and from country to country. According to Saloman and Gardner (1986), media research of the "does x teach better than y" variety has proved to be futile. In response to this dilemma, media researchers are increasingly posing more restricted questions, and are tending to focus on the effects of particular features of media on specific cognitive functions, but of course the results of this kind of study will be valid only where that medium is used in a specific way with a specific type of learner. Barring effectiveness comparisons as the basis for decisionmaking, it is our feeling that comparative judgments regarding implementation, based on historical experience, provide the only sound approach to making choices in the design of an educational technology intervention. Before pursuing the idea of implementation, mention should be made of two other approaches commonly prescribed in the literature for guiding decisions about the use of educational technology. Both of them, despite some obvious merits, appear impractical for decisionmaking in developing countries. The first of these is the recommendation that decisions on the use of educational technology should place greater emphasis on a problem solving approach. A common criticism that is advanced to explain why flawed technology applications have failed goes that planners choose to use a particular technology by taking the technology solution as their starting point, rather than the problem it was meant to solve. The charge is frequently made that educational technology options are stalking the world in search of problems. No doubt there has been so . truth to this charge, although getting hung up on favorite solutions for the problems of economic and social development is a failing by no means unique to the educational technology camp. It is impossible to deny that paying more careful attention to problem analysis and needs assessment would not be beneficial in addressing any educational problem. But the harder questions remain: Are decisions about educational technology in developing countries actually being made in t'.e absence of an understanding of educational problems? Is more detailed analysis likely to result in better decisions about whether to adopt technology or not, and which kind ? We doubt whetner spending more time and - 67 - Educational Technology: Chapter 2 money on problem solving prior to making decisions on the use of educational technology is likely to make much difference in the ultimate outcome. We believe that educators and educational planners in developing countries are usually well informed as to the causes, characteristics, and consequences of the educational problems and needs in their countries. The increased investments in textbooks that have taken place in developing countries during the past decade are instructive in this connection. We believe that the impetus for this increase has been the changes in technology that now make textbooks less expensive to produce, combined with the mounting evidence of their efficacy as a "solution" to the problem of low educational quality. It is difficult to see how these decisions to increase investments in textbooks would have been any different had there been made available more finely-tuned analysis of the problem of &s.ortages of instructional materials in the classroom. Decisions would have been different only if other solutions had been found to be more attractive, not if the planning process had permitted the problem to be examined in more detail. A second recommendation that is commonly made to developing countries on how to choose and use educational technology is also unrealistic; this advice calls for fitting the medium to the objective. The recommended process is that one must first formulate clear educational objectives, then decide on the best method to achieve them, and then, and only then, consider what medium or technology is best suited to delivering the method. The logic of this process is compelling and, indeed, may actually occur in some instances where resources permit systematic development of instructional units. But even where resources might permit such an approach, political, institutional, or practical realities frequently tend to force the process onwards to the consideration of the choice of medium or technology before specific instructional objectives can be formulated. The reality is that in developing countries, decisions about the possible use of educational technology will often be taken in a context where educational objectives are vague or unspecified, the efficacy of specific instructional methods is unknown, and the range of media options is severely limited. So, decisions about which technology to use for improving educational quality can and should be made on the basis of existing, documented experience with implementation factors. In this context, comparative advantages of one kind of application over another can indeed be found in the literature. This allows planning analysis to match local conditions, preferences, and previous experience to the advantages and disadvantages of various models suggested by applications that have been made in other countries. In addition, the ultimate decisions will be subject to a "push" factor, in the sense that policy makers will have varying degrees of confidence in the potential value of educational technology and varying degrees of commitment to finding alternatives to conventional instruction. Decisions will also be subject to "pull" as a result of planners' concerns about effectiveness, efficiency, and appeal that will be perceived differently in each country. By reason of their nature, the circumstances in which they are made, and the limitations of our knowledge, these decisions cannot be optimized through any known methodology. But we can illuminate possibilities by sharing - 68 - Educational Technology: Chapter 2 experience, demonstrating models, identifying potential obstacles, and providing rough benchmarks of likely outcomes, even if we cannot prove that "x teaches better than y" or show conclusively which features of x make a difference in learning. In order to put the information available on comparative implementation factors into some kind of structure, the discussion will focus on the three different levels of general education in turn, so that we may compare the relative advantages of the various educational technologies that have been reviewed. The comparisons take into account costs, possible obstacles, and potential educational impact and are based upon previous experience and best guesses. These comparisons are meant to open rather than to close discussion. This assertion is founded not so much on humility as on an appreciation of the great variation in educational contexts from country to country in the developing world and the current limitations of our knowledge. Primary Education The strictures imposed by the goal of "universal" access to primary education and the necessity of achieving low unit costs at this level put high demands on the selection of appropriate educational technologies. The possibility that technology could be a substitute for teacher qualification or teaching time so far has not so far been borne out by experience. New developments in the research could change this, a trend which would be especially relevant for countries that now face severe resource constraints. The best options provided by educational technology at the primary level in developing countries are textbooks, interactive radio, and programmed teaching/learning systems. Applications of these technologies have attained a degree of effectiveness and affordability that make them more suited to primary education than other technologies, such as television. Any of these three technologies can be used to provide direct teaching in one or more subjects or for the entire curriculum. The other possibilities that warrant consideration are technologies which are used to provide specific educational supplements, addressing key parts of the primary school curriculum. These might involve drill and practice using electronic learning aids, audiocassettes, or audiovisual aids. In the most favorable circumstances where resources are available in greater amounts, supplementary instruction might be provided by microcomputers. There appear to be differential advantages associated with using each of these technologies. Textbooks and programmed teaching/learning appear to have a "multisubject" advantage and have been used to cover the entire primary school curriculum. We do not yet know whether interactive radio could be used throughout the entire school day. On the other hand, interactive radio may well be more effective than the others in situations where teachers are not well trained and may cost less to deliver where the school audience is large. Both interactive radio and programmed teaching/learning appear to be better suited than textbooks for strategies that attempt to extend enrollments in existing schools or to use paraprofessionals as instructional personnel. The use of textbooks clearly has greater "face validity" with educators than the other technologies. It does not confront the obstacle of - 69 - Educational Technology: Chapter 2 unfamiliarity posed by the introduction of broadcasting to schools as would interactive radio. Nor does it face the difficulties created by the increased procedural demands placed upon the teacher in programmed teaching/learning systems. These advantages could well come at the expense of effectiveness, however. Combinations of these technologies are also possible, but so far remain untested. If textbooks, interactive radio, and programmed teaching/learning appear to offer the most advantages for most developing countries for delivering the core of primary school instruction, other technologies may also provide specific advantages. The use of low cost audiovisual supplements could well make a specific and useful contribution to instruction in conjunction with textbooks, programmed teaching/learning, and interactive radio. Because of costs, versatility, and ease of use, blackboards will probably be better investments than such things as film, filmstrip, or overhead projectors. Similarly, well-designed instructional supplements deliveree on audiocassettes, particularly if they address areas that teachers normally do not teach well (and these might include both low level skills and higher order thinking) might also be cost effective and relatively easy to implement. Finally, the engaging, interactive drill and practice offered by elect-rnic learning aids might also provide cost effective supplements within primary education and merit further exploration. Many primary schools continue to function without even the basic instructional materials -- chalk for the blackboard, pencils, paper, and notebooks. To a large extent, these are prerequisites not just for applications involving educational technole; +- for school instruction in general. Where these prerequisites are lackinr, developing distribution and financial systems to assist schools in obtaining basic instructional materials should be the starting point of efforts ~.o improve educational quality. Secondary Education For the purposes of this paper, secondary education includes both general secondary education md teacher training for primary teachers. In fact, increased demand for teachers is one of the reasons why the expansion of primary education in the developing world has resulted in considerable increases in pressure to increase enrollments at the secondary level. In addition, a wider spectrum of students now have access to secondary education in many countries. As a result, secondary schools must try to address the needs of students who are poorly prepared for study at the secondary level, even though there has been no lessening of pressure to prepare all students to succeed in school-leaving examinations. Enrollments at the secondary level are usually a small fraction of those at the primary level, unit costs are considerably higher, and facilities are better equipped and usually located in cities and towns where access to electricity and distribution points is less problematic than for most primary schools. These factors make some technologies more feasible but tend to rule out others. In many countries, the low enrollments in secondary schools would tend to make the use of television very expensive on a per-student basis, for in-school use, especially when there is no substitution factor for teacher - 70 - Educational Technology: Chapter 2 qualifications or time. Television appears to be more clearly advantageous for distance education. Interactive radio was developed for the early primary grades, and we have no evidence that an equally effective moujel of radio education could be developed for secondary school students. Sim,larly, programmed teaching was designed for the early primary years, although programmed learning modules for group or self-instruction would appear to be a good possibility at the secondary level and, in a well-designed program, could permit teachers to deal with larger numbers of students. However, the systematic development of programmed learning materials is often expensive and clearly an obstacle when there may not be economies of scale as are found at the primary level. Textbooks are likely to be the most common alternative for providing instructional materials at the secondary level. In many countries, the smaller size of enrollments in secondary education means that textbooks will have to be imported or that locally-produced materials will not be well "capitalized," expensive, or both. And apart from Lockheed et al.'s (1986) study of textbooks in Thailand, there are few studies that have examined the effectiveness of textbooks in relation to other factors at the secondary level. Furthermore, these studies tend to focus on the early secondary years (equivalent to the U.S. eighth and ninth grades). The preceding observations suggest that we can speak with much less confidence about the comparative advantages of various educational technologies to improve the quality of general secondary education than was possible at the primary level. At the secondary level, however, there appear to be better prospects for the newer technologies, provided that their use appears to be consistent with what is expected from school instruction in terms of preparing for school leaving examinations. In particular, the use of computers at the secondary level is spreading quite rapidly in developing countries. Even in schools that are acquiring computers for use in commercial courses or for computer literacy, computer-assisted instruction in general education is a possibility. Computers could well assist in improving the quality of instruction in areas like science and mathematics where instructional quality is frequently low, where both remedial work and enrichment are often needed, and where software may be less culture bound than for other subjects. One strategy to contain the high cost of introducing computers could be to share facilities with other schools or with commercial or private users. It would appear to be particularly advantageous to use educational technology in both pre-service ant in-service teacher education programs. Developing countries already have some experience in this connection. Radio and television have been used to reach teachers in remote areas and to compensate for shortages of teacher trainers. Videocassette recorders were used for microteaching applications e. en before the current spreading popularity of videocassette recorders throughout many developing countries. The use of videocassettes makes it possible to help teachers analyze and model effective teaching strategies and techniques. Furthermore, teachers who themselves learn through the use of educational technology are likely to be more effective when using technology in their classrooms in future educational technology applications. - 71 - Educational Technology: Chapter 2 Tertiary Education General education at the tertiary level is provided by colleges and universities that offer both liberal arts programs and professional education. Higher education in developing countries is almost always characterized by high unit costs and uneven quality. As higher education is the gateway to professional and managerial employment at home or in other countries, there is substantial pressure to increase enrollments in institutions of higher education. The only affordable technology-based options in many countries will be (a) distance education or (b) in-school applications that may be difficult to distinguish from distance education. In the former instance, not only could pressure on the conventional university enrollments be eased if a distance university option were created, but also, good quality teaching materials produced for distance universities are likely to find their way into regular university classrooms. In the case of the latter option, a strategy of cost containment could be designed to increase student-teacher ratios while at the same time maintaining or improving learning quality, by means of the use of educational technology and other instructional materials to permit greater amounts of independent learning. Aside from the distance education option, the use of educational technology for in-school applications at the tertiary level is constrained by several factors. For one, the need for an alternative to classroom lecture is not deeply felt in very many places. Even if alternative means of instruction can be shown to be more productive, the lecture method (accompanied by varying amounts of time for student discussion) is the prevailing means of instruction in higher education all over the world. Its prevelence even in the most prestigious universities and its "demonstration effect" for developing countries are unlikely to change anytime soon. Efforts to increase or improve textbook distribution will also be problematic. In developing countries, college textbooks are usually imported, and their cost, availability, and the number needed for several courses often place an onerous burden on students. Imported textbooks may be in a language that the student understands with difficulty or not at all. The problem of instru_tional materials is often addressed by sharing and photocopying of all, parts, or translated parts of books; by relying on lecture notes; or by preparing teacher handouts. On the other hand, textbooks and other printed materials have the advantage of being convenient for students to use, since much of their learning is expected to take place outside of the classroom. Textbooks are also convenient for the writers and can be produced without the teams of technical specialists that would be required for producing high quality materials to be delivered through alternative technologies. What alternatives to sole reliance on printed or duplicated materials might be advantageous at the tertiary level? The use of broadcast television outside of distance education does not appear to be a good possibility in most cases. Because of the many subjects that are offered, the audiences are likely to be too small for each one to make effective use of broadcast, or even closed-circuit, television, or for this to be cost feasible. - 72 - Educational Technology: Chapter 2 However, the use of videotape technology is an option. Such an approach might involve taped lectures given by regular faculty members o' guest lecturers. This would permit delivering all or part of the instruction for a course to the students via videocassette rather than live lecture. If, for example, half of a course were delivered through videocassettes, this would permit the other half of the teacher's time to be used in a manner that might allow efficiencies to be realized. For example, this could make it possible to increase the student-teacher ratio or it could conceivably free faculty members from more basic courses for more specialized teaching or research. We know of at least one college of engineering in Asia that uses an approach such as this. Although some may object that videotaping of lectures would be another example of the "talking face" method of instructional television, videotaping could permit quality improvements by making the best lecturers more widely "available" and by providing incentives for better prepared and organized lectures. We estimate that lectures could be produced in a small television studio (established for about $50,000) at a cost of about $200 per hour (exclusive of any fee paid to the lecturer). Another possibility -- one that, in fact, is already being implemented in Indonesia -- is the use of printed self-instructional materials. The PAMONG project, which produced programmed teaching/learning modules for Indonesian primary schools, also produced a series of "self- motivated learning" packages for use at the university ]evel (Nichols and Dilts, 1984). Greater use of such materials was subsequently recommended in the Indonesia Education and Human Resources Sector Review (IEES, 1986). The use of computers and videodiscs for general education is also something that will occur in the near future in some colleges and universities in developing countries. Computers are essential in courses that teach computer science and programming, and engineering and other technical areas would derive great benefits from access to up to date technology, both for the learning process and as the subject of study. Nevertheless, computer facilities would also be highly relevant for general education uses, particularly in mathematics and the sciences. V. Obstacles to Implementation: A Summary In trying to understand how and why educational technology might contribute to improving the quality of general education in developing countries, we have been concerned with discovering why it has not been more widely adopted. It is evident that adoption of educational technology for classroom use in existing schools has lagged behind technology adoption for distance learning. This problem can be looked at from three standpoints. The first is the problem of underutilization. In these instances, educational technology may have been introduced within schools but teachers do not make .se of the resources that have been provided in a meaningful way. The second problem is when a successful pilot or demonstration project is never expanded on. Here, - 73 - Educational Technology: Chapter 2 the use of educational technol'gy is either abandoned or Is not extended to a greater number of schools, even in the face of good evidence that It was effective. The third problem is a lack of consideration of the possibilities. Here, countries simply do not give serious consideration to the possibility of introducing instructional alternatives within their schools. They may be unaware of successful uses in other countries or they may judge that evidence from these efforts is not compelling or that they are inappropriate for local conditions. Tha implementation process for applications of educational technology proceeds from a demonstration of feasibility and effectiv6ness to a stage of continue4 or wider implementation. Obstacles are most frequently encountered between these two stages. From our discussion of the various technologies some of these obstacles have been identified. In the interests of a concise understanding of these obstacles, we offer a summary of what we believe are the ten most important obstacles to continued or wider applications of new educational alternatives. These are derived from a reading of the litereture, from conversations with practitioners dealing with individual applications, and from monographs that look specifically at the problems of implez-rntation (Friend, 1987; Thomas, 1987; Michel, 1987; Plomp and Ely, 1986; Cuba., 1985; Chadwick, 1979). Although obstacles may be called by different names and may be described in different terms, there is a great deal of similarity in the obstacles identified by different commentators. We might also mention that one has the impression that no failure of educational technology appears to have gone undetected. In part, this is the result of the "high profile" nature of educational technology -- one that lends itself to scrutiny by educators, social critics, and the media. The failure of educational technology is more licely to be grist for a provocative journal article or for the popular press than, for example, a failed attempt to improve educational planning. The literature on implementing educational reform or innovation in developing countries suggests that the obstacles encountered in the implementation of educational technology are often identical to those encountered in trying to promote educational change generally (UNDP/Unesco, 1983; Havelock and Huberman, 1977). Moreover, many of these obstacles are equally likely ,.o occur in the more developed countries as they also pursue policy-directed changes in teaching practices in schools (Berman and McLaughlin, 1977; Cohen, 1987). Our list of the ten most important obstacles that often act to block continued or wider implementation of educational technology in developing countries is as follows: Obstacle 1: Evidence of Effectiveness Not Convincing Even though an initial pilot program may have been satisfactorily implemented, the educational impact--as suggested by tests or the judgments of teachers or administrators--is not significant enough to justify continued or wider implementation. This is often the result of ineffective materials or software. The availability of effective materials and software is especially problematic for developing countries. - 74 Educational Technology: Chapter 2 Obstacle 2: Philosophical Obiections Implementation of educational technology arouses opposition on ph4.losophical grounds. The use of teaching devices is seen as having "dehumanizing" effects, leading to unwanted dependencies, or stunting future intellectual development in students. In the perception of many educators, technology (especially since the category includes television, calculators, and video games) is part of the problem, not the solution, and consequently it is seen as an unwelcome intruder in the classroom. Obstacle 3: Costs and Finance Wider implementation is halted because it would prove to be too costly. This could happen for several reasons. The costs of longer-term or wider-scale implementation are not accurately assessed or their implications not realistically addressed at an earlier planning stage. A country's financial situation may change during the course of an initial effort. Or, a country may count on attracting continued or new financial assistance that is not forthcoming. Even where the costs of using educational technology are not high, the means to finance these costs may not be present. Small recurrent costs for such things as batteries and materials may be difficult for the national government to finance when an application is implemented on a national scale, or by individual communities or schools when implementation operates in a context of decentralized authority. Obstacle 4: Insufficient Local Involvement Implementation of an initial activity or pilot program may fail to develop local identification, a sense of local "ownership," or an institutional "home" in the local political system. This situation could result from reliance on an excessively "top down" approach to the introduction of educational technology by the government and lack of meaningful involvement of teachers and community leaders in planning and implementation. It may also result from project design flaws that do not permit local authorities to direct the decisionmaking process or to organize cooperation needed from various local ministries and agencies. Obstacle 5: Chaiges in Leadershin An ini-ial educational technology project may be closely identified with specific political or administrative leaders who are subsequently replaced as their government goes out of power. Obstacle 6: Inadequate Infrastructure and Environmental Conditions There may be a lack of personnel or facilities to support implementation at either the national or regional level. With respect to educational technology, the inadequate infrastructure obstacle might involve one or more of the following: o materials development and production: A country or region lacks the personnel to develop oc produce instructional materials. This results in loss of effectiveness or efficiency in implementation. - 75 - Educational Technology: Chapter 2 o distribution: A country or region does not have adequate facilities or organizational arrangements to disseminate various materials or other inputs required for using an educational technology. This could relate to transportation, storage, marketing, or broadcasting. o equipment maintenance: A country or region does not have the necessary personnel or organizational arrangements to ensure that equipment is kept operational. This includes being able to repair equipment and obtain replacement or supporting parts and materials. O supervision and suRRort: A country or region may not have trained personnel or other organizational resources to devote to the supervision and support required for continued or wider implementation of an educational technology. Inadequate infrastructure for implementation may be aggravated by local environmental conditions. Implementation of educational technology may face obstacles such as the availability or quality of electricity, damage to hardware and software caused by heat and dust, distance or geographical factors that affect reception of clear broadcast signals, and the size, condition, and noise level found in a school classrooms. Obstacle 7: Comnlexity with ResDect to Classroom Organization Continued or wider implementation of educational technology does not take place because of the complexity of using the technu'ogy within the organization of the classroom. This could include one or more of the following: o operating the technology is too difficult for teachers and this is not resolved through adequate training; o using the technology does not fit well with a school's schedule or daily routine. Here, curriculum sequences or fixed broadcast schedules related to the use of a technology are not easily adjusted to take into account teacher or student absences, school holidays, special events, day-to-day occu:rences requiring adjustments in class schedules, or differences in the time it takes individual students .o master content. O storing, bringing equipment into the classroom, or moving a class to a special room is inconvenient or time consuming and causes teachers to choose to forego its use. Obstacle 8: Teacher. Parent, or Student Resistance Resistance may be strong from teachers, parents, or students. Apart from the effectiveness obstacle discussed above, teachers may resist the use - 76 - Educational Technology: Chapter 2 of a technology because they see it as a threat to their employment, to their expectations for receiving salary increases, or to their authority and auto3nomy in the classroom. Teachers may also resist because che technology requires greater effort on their part. Resistance from parents and students may result from their perceptions that the instruction provided through t.e technology does not adequately relate to what is contained in the examinations determining entry to schooling at the next level. Obstacle 9: Insufficient Attention to Instructional Development The use of educational technology may be driven by a belief in the attractiveness or power of the medium for delivering the instruction -- textbooks, radio, television, or computers. This may result in insufficient time and money being spent on the systematic development of instructional materials. Furthermore, development efforts may not be directed to applying sound instructional principles and methods or to testing whether they work with specific audiences. Obstacle 10: Clashes with Existing Curriculum and Stakeholders Educational technology may run into competition with existing or other new curriculum approaches. Compsting instructional approaches that both rely on educational technology may be tolerated at a pilot stage but not when there is a likelihood that wider dissemination will take place. Even where a new instructional approach may faithfully follow the national curriculum, and even if it is shown to be more effective than existing approaches. it may not be welcomed by those who have a stake in existing approaches or instructional materials currently in use. Contracts, jobs, salaries, or royalties may be at stake. Pedagogical differences may cause a clash. Moreover, these differences may be tied to political and economic understandings that a country has with another country or funding agency. VI, onglusions and Issues Our purpose in this chapter was to review the current possibilities and the past uses of educational technology for quality improvement of general education in developing countries. This review leads us to several conclusions that should be addressed by planners as they develop investment policies and strategies for the use of technology in education. We will suggest how these issues could be addressed through research and other activities. The conclusions we draw and the issues we identify are the following: Conclusion 1: A reassessment of the role that educational technology miuht play in strategies to improve general primary. secondary. and tertiarv education is both justified and timely. The quality of education at all levels in most developing countries remains a cause for concern. Improving, or even maintaining, quality will be a serious problem in those countries caught in the current crisis of ballooning demand and stagnating or shrinking educational resources. In these - 77 - Educational Technology: Chapter 2 cases, it is dlfficult to see how countries will be able to respond to the crisis using conventional means. We believe that reassessment is justified and timely because of what has been learned during the past two decades about promoting educational quality. Most notably, this includes the importance of classroom materials in relationship to other factors that influence learning. The knowledge created and the experience gained by recent successful investments in textbook provision, taken together with the accumulated evidence of the efficacy of other kinds of materials, including such things as interactive radio and programmed teaching/learning materials, suggest that the use of educational technology may be an important option in promoting educational quality, particularly at the primary level. Moreover, in light of increasing experience with other forms of alternative education options such as distance learning and open universities that have now become established as viable educational options, a reassessment or the role of educational technology can now be seen as an evolutionary step, rather than a radical shift in thinking for many countries. Issues. The major issues that should be addressed in policy planning and investment strategies are (a) to evaluate whether classroom materials should continue to be of such overriding importance, as educational development proceeds in many countries and as information is received from more countries and (b) to determine in what circumstances investments in educational materials delivered through means other than textbooks may be attractive to educational planners. Conclusion 2: In-school uses of technology for improving education auality should not be overlooked in educational policies, and connections between existing aDRlications for distance learning and new in-school applications should be considered. Especially at a time when the availability of educational resources is of such concern, the use of technology must be attractive from both the educational and the economic perspectives. This requirement underlies the growing interest in the use of distance education. While much needs to be learned about the effectiveness of distance education, it is clear that distance education programs for a variety of purposes and a variety of settings have proved their viability. Moreover, the best uses of educational technology are often found in distance education programs. It would be shortsighted, however, to forget that in all countries, schools are the main engines of education. Their social role is not likely to diminish in the future, especially as urbanization continues and more women seek wage employment outside the home. This is especially true at the primary school level. And while it appears inevitable that many countries will have to consider alternative forms for organizing and delivering education in schools if current en:oliment rates are to be maintained, schools will continue to be the focus of educational development in developing countries an, neglecting the quality of instruction in schools would be unwise. Issues. The major issues to be considered are: (a) What is the value of investing in in-school educational technology versus distance education? Is it more appropriate to use technology to help schools improve or maintain quality, often in the face of increasing enrollments, or should investments in - 78 - Educational Technology: Chapter 2 technology be used to increase access to education by excluded groups? and (b) Is it worth making the distinction between in-school applications and distance education applications of educational technology at all? As mentioned before, the theoretical distinction between school education and distance education is sometimes unclear. A functional blurring of the distinction at the policy level is likely to have both economic and equity benefits. The use of educational technology in both schools and distance institutions might help to blur the distinction. Furthermore, investments in one may benefit the other. Well-designed materials to teach students where there is no teacher might also work well in school settings where there is a teacher but where that teacher is not able to function at a desirable level of productivity. Conclusion 3:The economic implications of the different Rossibilities for using educational technology for improving the guality of general nrimary. secondary. and tertiary education varX considerably. The in-school use of educational technology that results in add-on costs is freguent but not inevitable. The use of educational technology in school settings has not generally resulted in economic benefits derived through a substitution of less costly technology for more expensive teacher qualifications or time. In most instances, this appears to have been judged to be neither possible r.or desirable. Indeed, the substitution of educational technology for teacher qualifications or time has not been the fundamental goal of most of the initiatives that were reviewed. The emphasis appears to be on developing or demonstrating the effectiveness of a particular application, with a view that, in time, its generalization would reach a point where economies of scale would result in affordable add-on unit costs. Where other obstacles to implementation have been overcome, this has been and will continue to be a reasonable approach to improving educational quality for many developing countries. A growing number of countries will find it hard to commit the resources needed to keep up with pressures to maintain or increase teacher salaries and to increase enrollments. These countries will be unable to support even small add-ons to current educational budgets. Here, the substitution of technology for teacher qualifications or time is more urgent and more likely to succeed. We found a few instances (namely the use of RADECO in the Dominican Republic and some of the programmed teaching/learning applications) that suggested the possibility that the use of educational technology could be used in ways that would result in lower unit costs. Issues. Developing countries should consider carefully those circumstances where investments in quality improvements of general, secondary, or tertiary education through the use of educational technology with associated add-on costs are justifiable and those that are not. The possibilities for providing lower cost in-school alternatives to the conventional organization of classroom instruction are not likely to emerge naturally -- they will have to be deliberately planned and will probably succeed only in those instances where the use of educational technology might be the only alternative to closing the door of the school to many children and young people in the community. - 79 - Educational Technology: Chapter 2 Again, we find few "naturally occurring" models for such use of educational technology. This is something that could and should be addressed in pilot projects. Conclusion 4: The obstacles to successful imp.ementation of educational technology in schools in developing countries are formidable. These vary from technoloey to technology and country to country. as do the Rossibilities that they may be overcome through careful Rlanning and implementation. This paper has summarized the various obstacles that appear to have confronted the implementation of educational technology in schools in developing countries. Obstacles are not unique to any particular technology, and any one may be found to be affecting almost any situation. Issues. Less attention now needs to be paid to "proving" the potential effectiveness of each educational technology and more attention needs to be paid to "proving" that good pilot efforts can lead to continued or wider implementation. Policy makers should consider ways to build this priority into policies and strategies. Other issues tLat should be investigated are: (a) whether changing circumstances and educational needs in a country suggest a greater willingness to address these obstacles; (b) whether the improved capabilities both of the technologies and of countries to implement change in their educational systems are likely to diminish the likely impact of some of the obstacles that have been important in the past; (c) the degree that these obstacles may be overcome through careful planning and design of projects. There is reason for optimism. But there is a need for concrete evidence. Conclusion 5: New models for the use of educational technology in developing countries are needed. These should include applications of the newer technologies. adaptations of the older technologies. and combinations of the old and new. We have looked at the possibilities of using educational technology to respond to current educational needs with evidence from experience that often took place more than a decade ago. The shelf life of some of this experience has probably expired. Costs and other implementation factors are likely to have changed considerably even in the countries where the pilot experiment took place. But the frequency with which we encou'.cer these cases in various anthologies, secondary analyses, and syntheses is a good indication of the usefulniess of the information derived from such .sxperiences. Good case studies aad well-documented project experiences are che building blocks of understanding and provide tha concrete basis for illuminating possibilities, dialogue, decisionmaking, and planning. Frequently, information must be drawn from cases that were not conducted for the purpose of generating information. Even in the best cases, information on effectiveness often covers classroom experience of only a year. Information on implementation that may affect longer term sustainability of the use of the technology is often unavailable. For the newer technologies, empirical investigation is only beginning. - 80 - Educational Technology: Chapter 2 Isues. The issues that policymakers should consider are the availability, timeliness, and quality of the information that goes into the various papers that will guide the formulation of a government's policies and strategies for using educational technology to improve general education in developing countries. Planners should consider whether more attention needs to be paid to the primary sources of information. The repeated use of such information in papers, meetings, and other activities suggests that this is something well worth the investment. Pilot projects are the main vehicles used to develop and test new models; they should be designed so as to yield sufficient useful information for decisionmaking. New models should test applications of educational technology at the best quality level, under the most accurate conditions possible. And this is important. Our understanding of the potential of such technologies as classroom radio and prog.-ammed teaching/learning would be far different, and far inferior, if we had only surveyed existing uses and not attempted to develop and test "high quality" models specifically designed for developing countries. Similarly, we will fai'l to understand the potential of computers to improve general primary, secondary, and tertiary education if we only look at the effects of currently-available software developed for instructional conditions far different from those of developing countries. Pilot projects must look beyond their stated purpose of gathering information on educational effectiveness. They must test the longer-term implementation issues that ultimately will determine whether educational technology is a feasible alternative. Only then will policymakers be empowered t.o respond to the current challenges in developing countries in new ways, and to improve the quality of the educational experience that young people bring with them from school to society - 81 - Educational Technology: Chapter 2 A N N E x 1: IN-SCHOOL APPLICATIONS OF EDUCATIONAL TECHNOLOGY IN DEVELOPING COUNTRIES - 82 . Educational Technology: Chapter 2 IN-SCHOOL APPLICATIONS OF EDUCATIONAL TECHNOLOGY IN DEVELOPING COUNTRIES Country Project/Study Level Interactive Radio Nicaragua Radio Math Primary Honduras Radio Learning Project (math) Primary Bolivia Radio Learning Project (math) Primary Thailand Basic Skills Pilot Project (math) Primary Papua New Guinea Radio Science Primary Kenya Radio Language Arts Primary Lesotho Radio Language Arts Primary Other Radio Dominican Republic RADECO (Radio Assisted Primary Community Based Education Project) Mexico Radio Primeria Primary Mexico Tarahumara Radio Schools Primary Paraguay Rural Radio Education Project Primary Ecuador Shuar Radio Schools Primary Cuba Radio Victoria de Giron Philippines RATES (Radio Assisted Teaching Primary in the Elementary Schools) Uganda Radio Science Primary Kenya English by radio Secondary Ghana English Frimary/Secondary/ reacher training Malawi Supplementary education Secondary Sierra Leone Supplementary (Discontinued) ITV El Salvador National educational reform Secondary Colombia (Enrichment) Primary Peru Telescuela Popular Americana Primary/Adult Brazil SACI Primary American Samoa National educational reform Primary/ Second&ay Indonesia TVRI Secondary Singapore (Enrichment) Primary/Secondary Malaysia National educational TV service Primary - 83 - Educational Technology: Chapter 2 Ivory Coast Primary/Secondary/ Teacher training Niger Tele-Niger Primary Nigeria Primary/Secondary/ Teacher training Algeria Teacher training Zambia (Enricbment) Primary/Teacher Egypt (Enrichment) Ghana Supplementary Primary/Secondary/ Teacher training (Discontinued) Sierra Leone Science (Discontinued) Tunisia Supplementary Zaire TELESTAR Electronic Aids Lesotho Speak and Spell, Speak and Math Primary Textbooks Jamaica Primary Education Assistance Primary Ecuador USAID Primary El Salvador Educational Reform Primary Honduras Primary Education Project Primary Paraguay World Bank Primary Nexico Mexican government policy Primary Brazil World Bank Primary Colombia World Bank Guatemala World Bank Haiti World Bank Philippines USAID, World Bank Primary Indonesia World Bank Secondary Thailand World Bank (math) Primary Papua New Guinea World Bank Solomon Islands World Bank Pakistan World Bank Bangladesh World Bank Niger Maternal Languages Project Primary Africa Primary Science Program Primary Uganda World Bank Primary/Secondary Ethiopia World Bank Malawi World Bank Swaziland World Bank Lesotho World Bank Tanzania World Bank Comoros World Bank Burundi World Bank - 84 - Educational Technology: Chapter 2 Central African Republic World Bank Guinea World Bank Liberia World Bank Botswana World Bank Yemen Arab Republic World Bank Computers Mexico Logo and math, UNESCO Mexico Graphics for training Higher education Mexico Introduce CAI (planned project) Primary/Secondary Colombia Logo in rural areas, UNESCO Colombia CAI subject teaching, UNESCO Chile CAI for pharmacy, engineering Higher UNESCO Education Brazil Logo in rural areas, UNESCO Primary Brazil CAI Higher education Argentina Logo for math, UNESCO Primary Argentina CAI for sciences, chemical Higher Engineering education Paraguay UNESCO Higher education Trinidad Introduction to computers, Secondary Ministry of Education Grenada Crochu CAI Primary Belize EQUATIONS and computers Primary Philippines Innotech: Writing to Read Primary Malaysia Computer Assisted Learning Teacher training Korea Computer science Primary India Computer Literacy and Awareness Secondary in Secondary Schools (CLASS) Sri Lanka Computer familiarity Secondary Kenya Aga Khan Foundation Primary/ Secondary Swaziland Teacher training Senegal Logo Primary Israel "Transparent" computer Primary Israel TOAM CAI (disadvantaged SES) Primary Israel Teaching algorithms Primary - 85 - Educational Technology: Chapter 2 Programmed Learning/Teaching Belize Programmed posters Primary Jamaica Project PRIMER Primary Indonesia Project PAMONG Primary Bangladesh UPE/IMPACT Primary Malaysia Project INSPIRE Primary Thailand Project RIT Primary Liberia Project IEL Primary (Source: Hayman, J. (1988). US AID Learning Technologies Computerized Information Digest (Washington, D.C.: Institute for International Research). - 86 - Educational Technology: Chapter 2 References Anzalone, S. 1987. "Using Instructional Hardware for Primary Education in Developing Countries." Harvard University, Project BRIDGES. Cambridge, Mass. Processed. Anzalone, S., and S. McLaughlin. 1984. "Electronic Aids in a Developing Country: Improving Basic Skills in Lesotho." University of Massachusetts. Amherst, Mass. Processed. Arnove, R.F., ed., 1976. Educational Television: A Policy Critigue and Guide for Developing Countries. New York: Praeger. Bates, T. 1982. "Options for Delivery Media." In H. Perraton, ed., Alternative Routes to Formal Education: Distance Teaching for School Eguivalency. Baltimore, Md.: Johns Hopkins University Press. Bates, T., and J. Robinson. 1977. Evaluating Educational Television and Radio. London: The Open University Press. Becker, H.J. 1987a. The Impact of Computer Use on Children's Learning: What Research Has Shown and What It Has Not. Baltimore: Johns Hopkins University Press. Becker, H.J. 1987b. "What We Know About Computer Use and Its Effectiveness in Schools: Existing Knowledge and New Research Directions." Cupertino, Calif.: Apple Computer. Block, C. 1985. "Interactive Radio and Educational Development." Develonment Communication Report 49. Bryant, J., and D. Anderson. 1983. Children's Understanding of Television. New York: Academic Press. Bunnel, D. 1986. "The Challenge of CD ROM." PC World, June. Burns, P.K., and W.C. Bozeman. 1981. "Computer-Assisted Instruction and Mathematics Achievement: Is There a Relationship?" Educational Technology 21: 32-39. Carnoy, M. 1976. "The Economic Costs and Returns to Educational Television." In R.F. Arnove, ed., Education^l Television: A Policy Critigue and Guide for Developing Countries. New York: Praeger. Carnoy, M., H. Daly, R. DeVillar, E. Earthman, F. Herrmann, L. Loop, M. Misra, and J. Starbuck. 1986. Education and Computers: Vision and Reality in the Mid-1980s. Stanford, Calif: Stanford University. Carnoy, M., and H.M. Levin. 1975. "Evaluation of Educational Media: Some Issues." Instructional Science. Amsterdam: Elsevier Scientific Publishing. Cat.ioy, M., and L. Loop. 1986. Computers and Education: Whi'-h Role for International Research? Paris: UNESCO. - 87 - Educational Technology: Chapter 2 Chadwick, C. 1982. "An Overview of Educational Technology in Latin America." Prospects. 12, no.3. Chapman, D.A., and R. Boothroyd. 1986. "Programmed Instruction as a Means of Improving Student Achievement: A Look at the Liberian IEL Project.* State University of New York at Albany. Processed. Christensen, P. 1985. "The Radio Language Arts Project: Adapting the Radio Mathematics Model." Develotment Communications Report 49. Chu, G., and C. Alfian. 1967. "Programming for development in Indonesia." Journal of Communication 30, no.4: 50-7. Chu, G., and W. Schramm. 1967. "Learning from Television: What the Research Says." National Association of Educational Broadcasters. Processed. Clark, R. ;?R3. "Reconsidering Research on Learning from Media." Review of Educational Research 53, no.4: 445-459. Claveria, O.B. 1982. "Report on the Implementation of the IMPACT Experiment- Universal Primary Education (IDS). Institute for International Research. Dhaka, Bangladesh. Processed. Clearinghouse on Development Communication. 1978. A Sourcebook on Radio's Role in Development. Washington, D.C. Clearinghouse on Development Communication. 1982. Project Profiles. Washington, D.C. Cuban, L. 1985. Teachers and Machines: The Classroom Use of Technology Since 1920. New York: Teachers College Press, Columbia University. Cummings, W.K. 1986. Low-Cost Primary Education: Implementing an Innovation in Six Nations. Ottawa, Canada: International Development Research Centro. Danna, I. 1985. El aRrendizaje del numero y la comoutadora. Buenos Aires: Laboratorio de Psicologia Evolutiva. Donahue, T., and M. Donahue. 1983. "Understanding Interactive Video." Training and DeveloRment Journal. Eicher, J. C., and F. Orivel. 1980. Cost Analysis of Primary Education by Television in the Ivorv Coast: The Economics of New Educational Media. Paris: UNESCO. Eicher, J. C., D. Hawkridge, E. McAnany, F. Mariet, and F. Orivel. 1982. Tha Economics of New Educational Media: Overview and Synthesis. Paris: UNESCO. Ely, D.P. and T. Plomp. 1986. "The Promises of Educational Technology." International Review of Educat,on 32, no. 4. - 88 - Educational Technology: Chapter 2 Ely, D.P. 1983. "The Definition of Educational Technology: An Emerging Stability.' n ducAtional Considerations (Spring), no.2: 2-4. Emery, M. 1985. "Another Exciting Revolution? A Rejoinder to Kupisiewicz and White." ProsRects. 15, no.4: 493-509. Eshgh, R., J. Hoxeng, J. Provenzano, and B. Casals, ads. 1988. Radio-Assisted Community Basic Education (RADECO). Pittsburgh, Pa.: Duquesne University Press. Etlinger, L. 1974. "The Electronic Calculator: A New Trend in School Mathematics." Educational Technology 14, no.12: 43-45. Farrell, J.P., and S.P. Heyneman. 1988. "Textbooks in Developing Nations: Economic and Pedagogical Nations." Ontario Institute for Studies in Education and Economic Development Institute. Toronto. Processed. Flores, P. 1981. "Educational Innovation in the Philippines: A Case Study of Project IMPACT." International Development Research Centre. Ottawa, Canada. Processed. Forman, D. 1982. "Search of the Literasure." The Coputins Teacher, January. Forsythe, R. 1970. "Instructional Radio: A Position Paper." Stanford University. Stanford, Calif. Processed. Freeman, C. 1987. "Computer Classroom Ethnography: Crochu Primary School, Grenada." Institute for International Research (Learning Technologies Project). McLean, Va. Processed. Friend, J. 1987. "Planning Ahead for Large-Scale Success." Friend Dialogues, Inc. Shelby, N.C. Processed. Friend, J. 1984. "Evaluation of First Grade Instructional Materials." The Radio-Assisted Community Basic Education Project, Report 2. InterAmerica Research Associates, Washington, D.C. Processed. Friend, J. 1985. "Classroom Uses of the Computer: A Retrospective View With Implications for Developing Countries." World BAnk. Washington, D.C.: Processed. Friend, J. 1985. "A Model for Interactive Radio Lessons: The Radio Mathematics Project." Development Communication Rego 49. Friend, J., K. Galda, and B. Searle. 1979. "Transfer of Instructional Technol_ 3y: Radio Mathematics in Nicaragua and Thailand." Processed. Friend, J.. and F. Kemmerer. 1985. "Strategies for and Costs of Disseminating the Radio Language Arts Project Throughout Kenya." Interactive Radio in Education. Agency for International Development and Institute for International Research (Communications Support Project). Washington, D.C. Processed. - 89 - Educational Technology: Chapter 2 Friend, J., and S. Kozlow. 1985. "Evaluation of Second Grade Instructional Materials Produced by the Radio-Assisted Community Basic Education Project." InterAmerica Research Associates. Washington, D.C. Processed. Friend, J., B. Searle, and P. Suppes, eds. 1980. Radio Mathematics in Nicaragua. Stanford, Calif.: Stanford University. Fuller, B. 1987. "What School Factors Raise Achievement in the Third World?" Review of Educational Research 57, no.3: 255-291. Gallagher. M. 1982. "The Best Use of Educational TV." Media in Education and Development 15 (September), no.2: 132-137. Galda, K. 1985. "Learning Math by Radio." Development Communication Report, 49. Gibbons, J.F., W.R. Kincheloe, and K.S. Down. 1977. "Tutored Videotape Instruction: A New Use of Electronics Media in Education." Science 195 (March): 1139-1146. Granr, S. 1977. "Administrative History of Out-of-School Television in the Ivory Coast." Academy for Educational Development. Washington, D.C. Processed. Green, C. 1985. "Instruction by Audio Conference: An Alaskan Example." DeveloRmant Communication Report 49. Halesworth, B. 1971. "Radio Broadcasts for Schools -- Some Lessons Learnt." Educational Broadcasting International 5, no.1: 56-59. Hannafin, M. 1985. "Empirical Issues in the Study of Computer-Assisted Interactive Video." Educational Communication and Technology 33, no.4: 235-247. Hancock, A. 1976. Planning for Educational Mass Media. London/New York: Longman. Hanssen, A., S. Koslow, and A. Olsen. 1983. "RADECO: Radio-Based Primary Education in the Dominican Republic." Development Communication Report 42 Hartley, S.S. 1977. Meta-analysis of the effects of individually paced instruction in mathematics. (Doctoral dissertation, University of Colorado, 1977.) Dissertation Abstracts International, 1978, 38(7-A), 4003. University Microfilms No. 77-29, 926. Hawkridge, D., and J. Robinson. 1982. Organizing Educational Broadcasting. London: Croon Helm. Hebenstreit, J. 1986. "The Use of Informatics in Education: Present Situation, Trends and Perspectives." UNESCO (Division of Structures, Content, Methods and Techniques of Education). Paris. Processed. Helwig, J., and J. Friend. 1985. "Teaching Where There are No Schools." Development Communication ReRort 49. - 90 - Educational Technology: Chapter 2 Holloway, R. 1982. "The Videodisc in Education: A Case for Technological Gradualism." Videodisc/Videotext 2, no.2 (Spring): 132-137. Hunter B. 1987. Computer-Based Tools in Learning and Teaching: A Research Agenda. Cupertino, Calif.: Apple Computer. IEES. 1986. "The Feasibility of Integrating Programmed Learning with Conventional Instruction in Liberian Primary Education." Tallahassee, Florida. Processed. IEES. 1985. "Enhancement of School Quality in Somalia." Tallahassee, Florida. Processed. Imhoof, M. 1985. "Interactive Radio in the Classroom: Ten Years of Proven success." Development Communication ReRort 49. Imhoof, M., and P. Christensen, eds. 1986. Teaching English by Radio: Interactive Radio in Kenya. Washington, D.C.: Academy for Educational. Development. Ingle, H. 1986. "Reconsidering the Use of Television for Educational Reform: The Case of El Salvador." In R. F. Arnove, ed., Educational Television. New York: Praeger. Jamison, D. 1982. "An Introduction to the Methods of Cost Analysis." In H. Perraton, ed,. Alternative Routes to Formal Education: Distance Teaching for School Eguivalency. Baltimore, Md.: John Hopkins University Press. Jamison, D., S. Klees, and S. Wells. 1978. The Costs of Educational Media: Guidelines for Planning and Evaluation. Beverly Hills, Calif.: Sage. Jamison, D., and E. McAnany. 1978. Radio for Education and Development. Bevarly Hills, Calif.: Sage. Jamison, D., and F. Orivel. 1982. "The Cost-Effectiveness of Distance Teaching for School Equivalency." In H. Perraton, ed., Alternative Routes to Formal Education: Distance Teaching for School Equivalency. Baltimore, Md.: John Hopkins University Press. Jamison, D., B. Searle, K. Galda, and S. Heyneman. 1981. "Improving Elementary Mathematics Education in Nicaragua: An Experimental Study of the Impact of Textbooks and Radio Achievement." Journal of Educational Psychology 73: 556-566. Jamison, D., P. Suppes, and S. Wells. 1974. "The Effectiveness of Alternative Instructional Media: A Survey." Review of Educational Research 44, no.1: 1-67. Kaye, A.E 1976. "The Ivory Coast Educational Television Project." In R.F. Arnove, ed., Educational Television: A Critigue and Guide for Developing Countries. New York: Praeger. Kelly, E. 1984. "Evaluation of the Improved Efficiency of Learning Project." State University of New York at Albany. Processed. - 91 - Educational Technology: Chapter 2 Klees, S. 1977. "Cost Analysis of Nonformal ETV Systems: A Case Study of the Extra-Scolaire System in the Ivory Coast." Academy for Educational Research. Washington, D.C. Processed. Konan, D., and A. N'Guessan. 1979. "Educational Reform and Technological Innovation: The Ivory Coast Experiment." In Educational Reforms: ExReriences and Prospects. UNESCO. Paris. Processed. K'ilik, J., C. Kulik, and R. Bangert-Drowns. 1985. "Effectiveness of Computer- Basec Edu.:ation in Elementary Schools." journal of Computers in Human Behavior .: 59-74. Kumar, N. 1967. Educational Radio in India. New Delhi: Arya Book Department. Leslie, J., and D. Jamison. 1980. "Policy Implications of Instructional Technology: Cost and Implications." In J. Simmons, ed., The Education Dilemma: Policy Issues for DeveloRing Countries in the 1980s. New York: Pergamon. Levin, H., G. Glass, and G. Meister. 1984. "Cost-Effectiveness of Four Educational Interventions." Stanford University. Stanford, Calif. Processed. Lewis, J. 1975. "A Review of Scientific Calculators for School." School Science Review 57, no.198: 177-182. Linn, M.C. 1985. "The Cognitive Consequences of Programming Instruction in Classrooms." Educational Researcher 14, no.5: 14-16, 25-29. Lockheed, M., and E. Hanushek. 1988. "Improving the Efficiency of Education in Developing Countries: What Do We Know?" Comare 18, no.l: 21-38. Lockheed, M., and E. Mandinach. 1986. "Trends in Educational Computing: Decreasing Interest and the Changing Focus of Instruction.' Edu3cational Researcher 15, no.5: 21-26. Luskin, B.J. 1987. "The Current and Future Implications of CD-1." Technological Horizons in Education 14, no.10: 47. Lyle, J. 1982. "Since 1967: The Original Case Studies Reviewed." In D. Hawkridge and J. Robinson, eds., Organizing Educational Broadcasting. London: Croon Helm. Makau, B. M. 1988. "The Aga Khan Foundation Computers in Education Project in Kenya." In New Information Technology and Ouality Education: The Potential and Problems of Computers in Schools. Washington, D. C.: World Bank, Economic Development Institute. Mante, R.F. 1981. "Multiple Outcomes and Perspectives in the Evaluation of Project IMPACT. Teaching Yourself in Primary School: Report of a Seminar on Self-Instructional Programs." International Development Research Centre. Ottawa, Canada. Processed. - 92 - Educational Technology: Chapter 2 Nasland, L., and G. Masland. 1976. "Some Cross-Cultural Implications of Educational Television: The Samoa ETV Project." In R.F. Arnove, ed. Educational Television: A Critique and Guide for DeveloRing Countries. New York: Praeger. Mayo, J., and R.C. Hornik. 1981. "Evaluation and Research in the Planning, Development and Support of Media-Based Education." International Institute for Educational Planning. Paris. Processed. Mayo, J., R.C. Hornik, E.G. McAnany. 1976. Educational Reform With Television: The El Salvador Experience. Stanford, Calif.: Stanford University. Mayo, J., E.G. McAnany, and S. Klees. 1973. "The Mexican Telesecundaria: A Cost-Effectiveness Approach." Academy for Educational Development. Washington, D.C. Processed. McAnany, E. 1973. Radio's Role in Development: Five Strategies for Use. Vol. 4. Washington, D.C.: Information Center on Instructional Technology. McAnany, E., J. Oliveira, F. Orivel, and J. Stone. 1983. "Distance Education: Evaluating New Approaches in Education for Developing Countries." Evaluation in Education 6: 289-376. McKinley, L. :981. "Adaptation of IMPACT's Instructional and Training Methods in PRIMER. Teaching Yourself in Primary School: Report of a Seminar on Self-Instructional Programs." International Development Research Centre. Ottawa, Canada. Processed. McMaster, J.C. 1978. "Cost-Effectiveness Analysis of Project IMPACT for the Philippines." Regional INNOTECH Centre. Manila. Processed. Mudjiman, H. 1981. "Effectiveness of Learning Modules and Peer Tutors in Student Learning. Teaching Yourself in Primary School: Report of a Seminar on Self-Instructional Programs." International Development Research Centre. Ottawa, Canada. Processed. Michel, C. 1987. "Educational Radio and Television -- Their Transfer to Developing Societies." In R. M. Thomas and V. N. Kobayashf, eds. Educatioral Technology -- Tts Creation. Development and Crossg-Cultural Transfer. New York: Pergamon. Nichols, D.G. 1982. "Low-Cost Learning Systems: The General Concept and Some Specific Examples." Performance & Instruction 21, no.7: 4-8. Nichols, D.G. 1983. "Handbook for IEL Instructional Management." Improved Efficiency of Learning Project. Gbarnga, Liberia. Processed. Nichols, D.G., and D.R. Dilts. 1984. "Final Contractor's Report: Project PAMONG." Institute for International Research. McLean, Va. Processed. Niemiec, R. and H. Walberg. 1985. "Computers and Achievement in the Elementary Schools." Journal of Educational Comguting Research 14: 435- 439. - 93 - Educational Technology: Chapter 2 Oliveira, J. 1! '. Can Technology Advance Educatl^_? Washington, D.C.: World Bank, Eco.. Aic Development Institute. Orivel, F. 1987. Costs and Effectiveness of Distance Teaching Systems: A Methodological Rpproach. Dijon: IREDU - Universite de Bourgogne. Oxford, R., J. Clark, J. Hermansen, P. Christensen, and M. Imhoof. 1986. "Final Report: Evaluation of the Kenya Radio Language Arts Project." Academy for Educational Development. Washington, D.C. Processed. Papagiannis, G.J., C. Dougal, N. Williamson, and R. Le Mon. 1987. Information Technology and Education: Implications for Theory. Research and Practice. Ottawa, Canada: International Development Research Centre. Pasigna, A.L. 1983. "Developing an Integrated Curriculum for IMPACT." In J. Socrates, ed., The IMPACT System of Mass Primary Education. Quezon City, Philippines: INNOTECH. Perraton, H., ed. 1982. Alternative Routes to Formal Education: Distance Teaching for School Equivalency. Baltimore, Md.: John Hopkins University Press. Perraton, H., ed. 1986. Alternative Forms of Post-Primary Education and Training in Africa. London: Marlborough House. Perraton, H. 1982. "The Scope of Distance Teaching." In H. Perraton, ed., Alternative Routes to Formal Education: Distance Teaching for School EguivaJ.ency. Baltimore, Md.: John Hopkins University Press. Phillips, E.A. 1987. "Report on Activities at the Crochu Learning Center - Term 1, 1986." Processed. Potar, N. 1984. "Project RIT (Reduced Instructional Time)." Project RIT, Muang Mai School. Lopburi, ThailarLd. Processed. Reigeluth, C.M. 1983. "Instructional nesign: What Is It and Why Is It? In Charles M. Reigeluth, ed., Instructional-Design Theories and Models: An Overview of Their Current Status. Hillsdale, N.J.: Lawrence Erlbaum. Roblyer, M.D., W.H. Castine, and F.J. Kinig. 1987. "The Effectiveness of Comput.;rs in Instruction: A Review and Synthesis of Recent Research Findings." Processed. Saloman, C., and H. Gardner. 1986. "The Computer As Educator; Lessons from Television Research." Educational Researcher. January: 13-19. Sang-jan, P. 1982. "Media and the Teaching of Mathematics in Thailand." In Media Asia. Bangkok: Centre for Educational Innovation and Technology. Sanguinetty, J.A. 1985. "The Replication of the RADECO Project in the Dominican Republic: Recurrent Cost Implications." InterAmerica Research Associates. Washington, D.C. Processed. - 94 - Educational Technology: Chapter 2 Schmelkes, S. 1972. "Estudio de evaluacion aproximativa de las escuelas radiofonicas de la Tarahumara." Revista del Centro de Estudios Educativos 2, no.2: 11-36. Schmelkes, S. 1977. "The Radio Schools of Tarahumara, Mexico: An Evaluation." In P. Spain, D. Jamison, and E. McAnany, eds., Radio for Education and Develogment: Case Studies. Vol.l. Washington, D.C.: World Bank. Schramm, W. 1962. "What We Know About Learning from Instructional Television." Educational television: The Next Ten Years. Stanford, Calif.: The Institute of Communication Research, Stanford University. Schramm, W. (1973). Big Media. Little Media. Washington D.C.: Academy for Educational Development. Schramm, W., P. Coombs, F. Kahnert, and J. Lyle, 1967. The New Media: Memo to Educational Planners. Paris: UNESCO. Schramm, W., ed. 1967. "New Educational Media in Action: Case Studies for Planners." Paris: UNESCO/International Institute for Educational Planning. Schramm, W., L.M. Nelson, M.T. Betham. 1981. "Bold Experiment: The Story of Educational Television in American Samoa." Stanford University. Stanford, Calif. Processed. Searle, B. 1985. "Evaluations of Three Interactive Radio Projects." af 3lopment Communication Report 49. Siann, G., and H. Macleod. 1986. "Computers and Children of Primary School Age: Issues and Questions." British Journal of Educational Technology 2, no.7: 133-143. Sigg, P. 1982. "The Hand Held Calculator: Effects on Mathematical Abilities and Implications for Curriculum Change." University of Indiana. Bloomington, Indiana. Processed. Slavin, Robert E. 1987. "Cooperative Learning and Student Achievement." The Johns Hopkins University, Center for Research on Elementary and Middle Schools. Baltimore, Md. Processed. Spain, P.L. 1977. "The Mexican Radioprimaria Project." Ir P. Spain, D. Jamiscn, and E. McAnany, eds., Radio for Educati'-a and Development: Case Studies. Vol.1. Washington, D.C.: World bdank. Spain, P.L., Jamison, D.T., & McAnany, E., eds. 1977. Radio for Education and Development: Case Studies. Vol.l. Washington D.C.: World Bank. Suppes, P., B. Searle, and J. Friend, eds. 1978. The Radio Mathematics Project: Nicaragua 1976-1977. Stanford, Calif.: Stanford University Press. - 95 - Educational Technology: Chapter 2 Suydam, M. 1982. "The Use of Calculators in Pre-College Education: Fifth Anrnual State-of-the-Art Review." Ohio State University. Columbus, Ohio. Processed. Suydam, M. 1983. "Research Report: Achieving With Calculators." Arithmetic Teacher 31, no.3 (November): 20. Sylla, F. S. 1985. "Computers and Literacy in Senegal." Doctoral dissertation, Massachusetts Institute of Technology. Cambridge, Mass. Thiagarajan, S., and A.L. Pasigna. 1985. "Final Contractor's Report: Improved Efficiency of Learning (IEL) Project." Institute for International Research. McLean, Va. Processed. Thiagarajan, S., and A.L. Pasigna. 1987. "Using Soft Technologies of Learning for Primary Education in Developing Countries: A Review of the Literature." Harvard University, Project BRIDGES. Cambridge, Mass. Processed. Thomas, R.M. 1987. "The Nature of Educational Technology." In R. M. Thomas and V. N. Kobayashi, eds., Educational Technology -- Its Creation. Development and Cross-Cultural Transfer. New York: Pergamon. Tilson, T.D. 1988. "Interactive Radio Instruction Costs and Other Factors in Implementation." Paper presented at World Bank Seminar on Educational Technology. Washington, D.C. Tilson T., and B. Searle. 1978. "An Assessment of Teacher Attitudes." In P. Suppes, B. Searle, and J. Friend, eds., The Radio Mathematics Project: Nicaragua 1976-1977. Stanford, Calif.: Stanford University. UNESCO. 1967. New Educational Media in Action: Case Studies for Planners. I-III. Paris: UNESCO. UNESCO. 1978. The Economics of New Educational Media: Cost and Status of Research Trends. Vol.l. Paris: UNESCO. UNESCO. 1980. The Economics of New Educational Media: Cost and Effectiveness. Vol.2. Paris: UNESCO. UNESCO. 1984. Media Education. Paris: UNESCO. UNESCO. 1986. Informatics and Education: A First Survey of the State of the Art in 43 Countries. Paris: UNESCO. Vinsonhaler, J.R., and R.K. Bass. 1972. "A Summary of Ten Major Studies on CAI Drill and Practice." Educational Technology, July: 29-32. Walker, H. 1986. Evaluacion cualitativa del programa de radioeducacion comwtnitaria (RADECO) de la Republica Dominicana." Washington, D.C.: U.S. Agency for International Development. White, R. 1976. An Alternative Pattern of Basic Education: Radio Santa Maria. Paris: UNESCO. - 96 - Educational Technology: Chapter 2 1Vhite, R. 1977. "The Use of Radio in Primary and Secondary Formal Education: The Radio Santa Maria Model in the Dominican Republic." In P. Spain, D. Jamison, and E. McAnany, eds., Radio for Education and Deyelooment: Case Studies. Vol.l. Washington, D.C.: World Bank. Windham, D;M. 1983. "Cost Analysis of the Improved Efficiency of Learning Project." Ministry of Education. Monrovia, Liberia. Processed. Windham, D.M. 1985. "Internal Efficiency and the African School." Universite de Dijon. Dijon, France. Processed. Windham, D.M. 1988. Improving the Efficiency of Educational Systems: Indicators of Educational Effectiveness and Efficiency. Washington, D.C.: United States Agency for International Development, Bureau for Science and Technology/Office of Educaticn. Williams, K. 1987. "Computers in Trinidad and Tobago: A Survey of Access and Attitudes Towards the Technology in Secondary Schools". Doctoral dissertation, Harvard University. Cambridge, Mass. Wooten, J., W. Jansen, amd M.K. Warren. 1982. "Project IMPACT: A Low-Cost Alternative for Universal Primary Education in the Philippines." Agency for International Development. Washington, D.C. Processed. Wray, B. F. 1988. "Computers as a Catalyst for Educational Change." Aga Khan Education Service. Kenya. Processed. - 97 - Educational Technology: Chapter 3 Chapter 3 Uses and Costs of Educational Technology for Distance Education in Developing Countries: A Review of the Recent Literature Greta S. Nettleton - 98 - Educational Technology: Chapter 3 Uses and Costs of Educscional Technology for Distance Education in Developing Countries: A Review of the Recent Literature Greta S. Nettleton I. Introduction A Changing Context for Technology Applications Distance education has emerged in the last twenty years as a vigorous educational alternative in nearly every region of the world. Defined by Perraton (1986) as an educational process in which a significant proportion of the teaching is conducted by someone removed in space and or time from the learner, distance education also depends on two-way communication, needs a fixed institutional base to provide administrative and student support, and provides a systematized learning curriculum to the student. In general, distance learniing is an alternative to conventional learning within school walls, and it usually leads to a specific certificate, qualification or degree. This deftn.ition excludes simple teach-yourself texts and informal educational broadcasts, given the fact that these kinds of programs are usually not tied in to a nation's formal school system. (Perry and Rumble, 1987).1 The use of broadcast media in existing classrooms to improve teaching quality is covered in chapter three. The move towards maturity in distance education is reflected not only in the proliferation of research in the field, but much more significantly, in the changes in quantity and of quality of distance education programs themselves. Most spectacular has been the quantitative growth of numbers of distance learners in formal educational systems. In the early 1980s, according to one estimate, about two million non-resident students were enrolled in distance education programs world wide (Perry, 1984). By 1987, this figure was revised by Daniel (1987) who estimated that nearly four million students were enrolled in higher distance education programs alone worldwide, not counting elementary and secondary level programs. Distance educstion is now well-established in many developing and developed countries, especially in Asia, England, the developed British Commonwealth countries, the Soviet Union and Europe. Huge open universities in Thailand, Indonesia and China each serve hundreds of thousands of students. Strong programs also exist in the lUnited States, Latin America, and Anglophone Africa.2 (see Table 1) 1 For a complete discu;sion of the definition of distance learning, see Keegan's article in Sewart, Keegan, and Holmberg (1983). 2 In contrast, very little information is available about distance education in the Middla East, Eastern Europe or Francophone Africa, reflecting less interest and fewer institutions in those areas. - 99 - Table 1 GENERAL DATA ON SELECTED DISTANCE LEARNING INSTITUTIONS c Country Institution Current enrollment, date media employed (in order & graduates to date estab. of importance) o China Chinese Radio and TV > ~~~~(CRTVU) Gt.d.: 300,000tofcturil a Thailand Sukhothai Thamathirat 167,142 (1986) 1978 print, AV cassettes, Open University radio, TV, study centers R(STOU) Gt.d.: 38,316 t Korea Air and Correspondence 48,067 (1987) 1974 print, radio, face-to High School (ACUS) face meetings (51 schools) Gt.d.: 77,944 Korea Air ahd Cor- 150,000 (1985) 1972 print, radio/TV, AV, Orespondenie Univ. t (KACU) Gt.d.: 57,691 KInonesia Universitas Terbuka 150,000 (1988)est. 1983 print, audio cassettes, (Indonesian Opes Gt.d.: NA face to face rUniversity) India University of 45,887 (1983) 1981 print Madras Gt.d.: NA Nigeria University of Lagos 4,000 (1987) 1974 print, face to face Correspondence and Gt.d. 773 (radio discontinued) Open Studies Unit (COSIT) Tanzania in-service teacher 45,534 (total) 1976 - print, face to face, radio training campaign: and supervised teaching primary'level Gt.d.: 35,028 1981 = ~~~~~~~~~~Table 1, ecrt, M alawi M alawi Correspondence 2 000 (1978) 16 rn,rdo td etr 0 College gtd: NA u (secondary level) Zambia National Correspondence 4,000 (1981) 1964 print, radio, study groups X01 Course of Zambia 627 (1987) 0 °4 (secondary equivalency) Gtd,d: NA 0. MPakistan Allama Cqbal Open 64,651 (1986) 1974 print, study centers, radio University (AIOU) Gt,d.: NA TV, AV cassettes Brazil Logos II 46,000 (1981 est) 1976 print, face to face (teacher training) pass rate: avg. 66Z to 80Z Costa Rica Universidad Estatal 11,000 (1987) 1977 print, broadcast, AV a Distancia cassettes, & face to face (UNED) Gt,d,: 487 Mexico Telesecundaria 408,000 (1987) 1968 television, prcent, super- (7th to 9th grades) Gt.d.: NA vised classrooms Brazil Posgrad 862 (1983) 1979 print, face to face (university level instructor trainigng) Gt.d.: 365 Portugal Telescola 50,000 (1977) 1965 television, class mon- (Jr. Secondary) Gt.d.: 904 itors, worksheets paSs rate Brazil ETV-Maranhao 24,009 (1985) 1969 television, class mon- (Jr. Secondary) itors, worktaeets Brazil ETV-Ceara 44,310 (1985) 1974 television, class mon- itors, worksheets Educational Technology: Chapter 3 Qualitative changes have been more subtle. Distance education institutions in developing countries have adopted some modern technologies to convey their subject matter and to perform administrative tasks, but the printed word remains the dominant media for at least three quarters of all institut ons surveyed (Perry, 1984). More important is the established role that distance education now plays in overall educational policy planning worldwide. Twenty years ago, 'distance education' was not even a common term in education jargon (Halliwell 1987). Government policy makers, while occasionally venturing into the untested waters of educational technology innovation via radio and television schools, were in general still unclear in their understanding of how distance education could work with conventional education. This was complicated by the fact that many people consistently failed to make a distinction between "distance education" as a general policy goal, and the experimental media such as television on which early disted projects were sometimes based. During this same period, the research on distance education media was passing through a turbulent phase. Problems with the implementation of innovative proposals led to intense criticism from within and from outside of the field. At the end of the 1970s, there was serious doubt among some r%searchers whether broadcasting media could really be practical as a tool to improve education in developing countries in the wake of a series of well- known pilot projects such as the Ivory Coast, American Samoa, Niger, and Senegal that have acquired (whether completely justified or not) reputations for failure. (World Bank No. 491, 1981) An interesting effect of this disillusion was that criticism of broadcasting applications affected the reputation of the whole field of distance learning in the eyes of multilateral planning organizations such as the World Bank; Bank lending for this area fell markedly (Hawkridge 1987) even as multimedia distance education began its tremendous surge worldwide in the late 1970s and early 1980s. Having survived the disillusionment felt by many researchers and planners in the 1970s, distance education in the 1980s has come to be recognized throughout most of the developing world as a legitimate policy option for formal school systems as well as for informal applications. The magnitude of this change is to a certain degree invisible, since, as Daniel (1987) wryly observed, the amount of formal studies done on any distance education project seems to be inversely proportional to the number of students involved. Countries such as Colombia, India, the USSR, Kenya and China have devoted nearly three decades to research, pilot studies and implementation of their disted programs. Others have begun their involvement more recently, but have chosen to embark on l.rge scale projects straight from the start, such as Thailand, which has nearly two hundred thousand students enrolled in its Sukhothai Thammathirat Open University (STOU) after only five years (1978 - 1983). Political commitment on this scale demonstrates a growing confidence in the value of distance learning by policy-makers themselves. - 102 - Educational Technology: Chapter 3 Current Approaches to Distance Education There is a move in the literature, and in practice, away from the initial emphasis found during the 190Os and 1970s on comprehensive applications of distance teaching 4tiods to improve the quallty of all education in a country. Instead, objectives have been readjusted to embrace less massive, more specialized applications. The hope La that distance education will complement the existing educational system, rather than replacing it, reforming it, or duplicating it at an inferior level (Perraton, 1986). The single strongest interest is In providing diplomas and certifications to those otherwise unable to attend universities for reasons of limited space, lack of money, and barriers of class and gender. Other preferred applications are for teacher training, secondary school at a distance, particularly in Africa, and adult certification at the primary and secondary 1 bels. aneral, *rimary education is least often attempted, due to the need with s, 1. childr - for greater supervision than a distance education program usually can provide. Several other reasons for the lack of primary level programs have been cited. It is difficult and expensive to design appropriate learning mat rials for children, and, most importantly, primary education is more politicized than other levels of schooling. Socialization and the cultural learning process are as important as the educational curriculum in the early grades, and conservatism about how and what young children should be taught . _vails the world around; reforms in this area can be a very delicate issue. 'rhere is also a wider recognition in the literature that distance education is not necessarily a 'bargain basement' option, that can provide learning on the cheap, although it does, in many cases, allow governments to spend less money per student than they otherwise would have if they had to create conventional institutions from the ground up. While marginal costs per student may indeed be extremely small for a very large distance learning institution, the absolute cost required to create such a program is also likely to be high, if the institution is to achieve a minimum level of educational quality and effectiveness. Regional isolation continues to plague planners of distance learning inst.cutions, as well as researchers and Interested government leaders. However, theou has been increased international cooperation among institutions and governments with an interest in distance learning. This has been occurring both between North and South, and South to South, and ranges from exchanges of simple descriptive information to sharing of planning expertise to actual transfer of courseware and academic credits between dnstitutions. Much of the cooperatior follows certain politico-cultural patterns; this is particularly noticeable in the links among British Commonwealth nations, where the example set by the England's Open University and the Canadian experience with providing education to remote territories have been extremely influential in promoting distance education in other member nations. The Commonwealth Secretariat has also been very active in promoting distance education. Spain's distance university, UNED, has been similarly influential - 103 - Edtcational Technology: Chapter 3 in Latin America, through its Associacion Iberoamericana de Educacion Superior a Distancia. The idea of regional cooperation is gaining acceptance in Asia, as evidenced by the major conference on distance education sponsored by the Asian Development Bank in 1986 (Asian Development Bank, 1987, vol 1 & 2), although much more could be done in the future. World-wide, the International Council for Distance Education (ICDE) in Edmonton, Alberta (which works closely with Unesco) has been promoting research and conferences for over 40 years, and has over 500 individual and institutional members from 50 countries (Daniel, 1987), and Unesco is supporting distance education directly, through its regional office for Asia (ROEAP). II, What Does the LiteLature Sav About Technology? The last twenty years has seen a change in the political role of new technology in distance education systems. As the field has matured, project design goals have moved on from the experimental phase of testing the basic question of whether a particular technology can teach effectively or not and at a feasible cost, towards using distance education, whatever the technology, as a tool to attain national educational goals. Thus, the stature of 'technology' per se has been somewhat reduced in favor of the broader concept of distance learning. The literature is nearly unanimous in stating that no hard aTid fast rules can be made to use to determine which is the best medium to use for a distance learning project. Not only are media usually used in combination with each other, but circumstances vary so much from situation to situation regarding level of infrastructure, institutional relationships, available budget and so forth, that few generalizations ihold from one to the next. Bates (1980) affirms that, There is no logical, step-by-step procedure for deciding on the 'best' media configuration or collaborative arrangements. A large number of factors, none of which can be measured or weighted scientifically, have to be taken into consideration. In the end, all these factors have to be thrown in and an intuitive judgement made. Effectiveness of media in teaching has been shown to depend much more on the content and quality of instructional design than cn the type of media employed (Rumble in Jenkins ed. 1988). Bates (in Jenkins ed. 1988) agrees but qualifies his statement by saying that media differ in the kinds of learning that they do best. According to Jamison and McAnany (1978), citing Chu and Schramm (1967), students can learn through any properly used medium (Eicher in Unesco 1980). This point has been the subject of some disagreement. According to Bates (in Perraton ed. 1982), making a wider selection of media available in a distance course appears to improve learning effectiveness. Smith (in ADB 1987) agrees with Bates. Whatever the effectiveness, widening the range of media in a program is likely to increase institutional complexity, since it will involve collaboration between other - 104 - Educational Technology: Chapter 3 agencies such as the national telecommunications ei.tity (PTT) and broadcasters (Bate.i in Jenkins ed. 1988). Sakamoto (in ADB, 1987) classifies the media available for use in distance education accordir.g to the way that the information leai, the gap between teacher and student: (1) physical transport of materials through space (2) broadcasting and (3) electronic signals sent over wires and/or cable. (see table 2) Rumble (in Jenkins ed. 1988) notes that little research has been carried out comparing distance learning media, although generalized comparisons of the various media have been made which list factors peculiar to each that can affect costs (see ?erraton 1982, Eicher et al 1982, Bates in Perraton ed. 1982 and Stahmer and Bryan 1988). The usefulness of any such comparative study, even if it did exist, would be grossly limited by the fact that application conditions vary so much from country to country. Attempts to provide some general basis for comparison have been made by Sparkes (in Sewart et al 1983) and Bates (1980). (see Tables 3,4 and 5). Print While it is apparent that a large number of kinds of media are theoretically available for use in distance learning situations, in practice, particularly in developing countries, many of these are seldom used. As mentioned above, print continues to be the basis of a large majority of programs (McAnany et al. 1983). Courseware leaflets, textbooks, written assignments and tests provide the foundation for learning in almost all cases, while the newer technologies such as radio, audio cassettes, and in scattered instances, television, are usually applied as an addition, in varying mixtures along with face to face tutoring and group learning sessions. The reasons for this are complex, and not well researched. Paradoxically, far more research exists on the less common broadcasting applications than ol. the planning for, and costs of, printed materials in distance education systems. Thus the consensus found in the literature that print is irreplaceable seems to be taken as an heuristic assumption. Noting its pedagogical effectiveness for conveying abstract ideas and for serving as a reference source, Bates states that, ',as -ng as literacy remains a major aim of development, print will be an essent al teaching medium...educational media should be sonsidered as an addition ..o the provision of print material, not as a replacement" (Bates 1982) Among those institutiorq surveyed by Perry in 1984, 100% in Africa, 93% in Asia, and 72% in Latin Amer-a employed correspondence and print materials in their programs. As pointed out by Stahmer and Bryan, print has been affected by advancing technology; desk top publishing has been found to greatly facilitate initial editing and subsequent updating of materials (in the case of the Open Learning Institute in B.C. Canada, reducing per/page manuscript preparation time from 120 minutes to 50 minutes) while computerization of inventory and management can improve efficiency of distribution (Stahmer and Bryan 1988). Little research has been done that compares the use of printed material in distance learning outside of the classroom with use of other technologies. - 105 - 941 0 0 14 l~~~~~~~~~~~~~~~~~~~~~~~~~~~ AI prslMeopue 0 Delivery~~~~~~~~~~~~~~~~~~uitr M ai rtierR4i 0 I~~~~~~~~~~t~~~~~ of ~ ~ ~ ~ ~ ~ lfomi MTermoital evcSsTem Sofawar VHardwareas Tramportazion ~ ~ ~ ~ ~ rapic Service S~~~~~~~~ofi itware Ed~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~9E vrodaotConunion CTam7nal V Seodcaste sU ~~~~~~ ~ ~ ~~~~~~~~Stipitre UHF Teldew PastdSuvea * Pt~~~~Moing itureVhcls*Pojco onsod 'Intorrtgtion FM TV 0ani:Sr T 8rorUstirs ~ ~ ~ ~ ~ ~ ~ ~ ~~G:~a MIErrenf4ur VH jildc Still pialue UHF Teleuxt~IA Educational Technology: Chapter 3 Table 3 MEDIA COMPARED: Number of Man Hours of Teacher Time to Produce One Student Hour of Study (source; Sewart et al) clazses in school 1.5 tutor-text (1; in O.U.) 90.0 lectures 2.0 full O.t. multi- media course 70.0 radio or audio tapes r 6.C CAL 200.0 T.V. 50.0 These figures indicate tho degree of motivation required of an experienced teacher (once the novelty has worn off) if he is to embark on the us* of each method. The actual cost per student of each method depends upon othor factors toot (a) the number of students that are taught as a result of the investment of teacher-time (b) the proportion of a teacher's time spent on teaching (e.q. in most universities only about 10% of a lectur- er's full time appointment (including vacations) is spent on teachingq whereas in schools or in the Open University this figure is nearer 60% though for differ- ent reasons) (c) the imber of man-hours of support ataff needed per man-hour of teacher-tine (d) the cost of the technology (including the buildings) of each mod Table 4 MEDIA COMPARED: Estimated Costs, Excluding Hardware*, for British. Open University (pourc&; Sewart et al) pertin nater teewro- of of multplier hlu tdcher sUxdmt to give ext per per tunm per el staff stxfnt ubtnst spwt en teacher atve per A D hwr ta&s&1g tiffs teacer B C invested (seaff per (A) ib, (C) (0) studnt hoaw) Clasv (achsl) 1.5 0.6 30 1 1 0.09 Lecwres 2.0 0.1 50 1.5 0.6 Fu.l OU. coxue 70.0 0.6 5000 o.0o 0.23 CAL 200.0 0.5 1000 2.0 0.8 'Open University estaate for a 2id level cise WuCh ns for a yas. (ZN,lLXks eatint for awnxtrative ard otwr saff in tbe University needd to opeate the syste. * too variable to estimate - 107 - .. ~~~~~~~~~~Table 5 0 0 /a * not aPPicable (1.e. medium not used) cc 0~~~~~~~~~~~~~~~~~~~ a A 2 i - 5 x ' , Sa,I- 0~~~~~~~~~~~~~~~~~~~~~ transzission 58 000 40 Oo e X NZa Na IL Iut tran5Cn 77 cost0 convlrted t vS Nil Nllnde tol nea 1.0) o TV produ'ction 8N/a00 15 00 N/a 600 5 0N/a (bSoo(i) Te. 2edium00 7not ?s TOTAnsiso AIV00 COT 80°6 l000 ?1/ 11c Nil 490perC* | 140000 T i Radiotl o stsrodULtt t 1300 2000 50 1230 (See TV io 1,186,000S 40~ ~ so re minute98 0~~~~~~~~~~~~~~~~ c~~~~~ 4.5 6~~~roucin / a000 0 . / .0 Ra25- L 6, 0 U~~~~~~~~~~~~~~~~ U ~ ~~~~IOA I OT 3,0 OD - .0 26o 5.0 640OC 4,0 0 a~~~~ ftoa ots lt 2 2.a1 *8 1%35 6 c.e 113,0 vredonna%iutbedrcl t rdcin rtaaMs cc :~~~~~~ .2 6 C sorc:4ats.98 Educational Technology: Chapter 3 Bates (in Jenkins ed. 1988) warns against assuming that printed materials are necessarily a cheap solution in a distance learning situation. Under certain conditions, if the fixed costs of the academic time used in design are included in the total cost, student numbers are high and distribution networks are inefficient, it may end by being relatively expensive. In the case of Indonesia's Open Junior High School, unit costs for students are Rp 174,826 compared to Rp 107,300 for regular schools; although this is partly attributed to the small size of the program, Setijadi (in ADB 1987) found that the major factor in the higher unit cost was the cost of printing, since the distance program requires ten times as much modular text material than the in-school program. Rumble (in Jenkins ed. 1988), divides the costs of print media for distance education into four components: development, text setting, printing, and distribution. A discussion of development costs, which are mainly affected by the decision of whether to employ full time academic writers or to hire temporary consultants, can be found in Perry (1976), Rumble (1986) and Perraton (1982). Text setting costs are affected by considerations of printing method, the need for calligraphy (eg. for arabic script) and the need for translations. Altbach (1983) emphasizes that production of adequate quantities of acceptable quality of printed materials is highly dependent on the level of development of the local printing industry. Certain problems pose limitations to the use of print materials. As with any other media, knowledgeable writers for specialized subjects, especially in higher education courses, may be in short supply. This may require importing expertise or outright purchase of foreign texts and materials. Importing raises cultural concerns about inappropriate content and can be complicated by foreign currency restrictions. Copyrights can hinder transfer of material across borders, and resolution of this issue can be difficult and expensive. (Bates in Jenkins ed. 1988) (Danarajan & Mugridge 1987). Print is not effective in teaching psychomotor skills, laboratory skills in science courses, or oral/comprehension skills in language courses, nor has it been shown effective for pedagogical techniques for teacher training courses. Printing costs exhibit less economies of scale than broadcast media, and the burden of buying paper and printing increases directly with the size of enrollment, although preparation costs become relatively cheaper when spread out over a larger print run. Distribution also becomes miore complex and more expensive as the size of a distance education progrp.i increases. Most institutions will need to rely on the postal service ior distribution of course materials and for the feedback traffic of comp]:tced assignments to and from the students. A very large institution might attempt to distribute independently, if local postal service were very poor, and where sufficient resources would allow, but considerations of warehousing, labor, fuel, vehicle purchase, and maintenance make this a very costly endeavor. The Universitas Terbuka in Indonesia has found it necessary to print and distribute examination papers separately from all other materials and under careful supervision, to maintain security, but uses the postal service for all other materials. While complaints about slow postal service are common, there may not be any realistic alternative available. There may even be some advantage to - 109 - Educational Technology: Chapter 3 using the government mail service: in certain cases, postal fees have been waived for distance educati4n institutions where political support has been strong for the program. Radio By virtue of its generally low cost and wide availability, radio is often assigned the second place in importance behind print as a delivery medium for distance education, although it is most often used as a supplement, in conjunction with other instructional delivery modes. Perry's 1984 survey shows the heaviest reliance on radio to be in Africa, where 75% of the institutions surveyed used radio. In Asia, 36 %, and in Latin America, a suprisingly low 24% responded affirmatively to the use of radio. Use of audio cassettes has nearly caught up to radio in Asia and Latin America (see section below). Eicher et al. (1982) point out that less cost studies have been done for radio than for instructional television, although those that have been done confirm that radio is relatively cheap. They consider the minimum threshold for a radio coursb to be worth attempting to be approximately 2,000 students, and much higher when a complex teaching system is planned. Jamison and McAnany (1978) examined the costs of three distance learning projects; in this instance, radio-correspondence systems cost from 35% to 50% of what traditional instruction would have cost. In addition, the opportunity cost of lost income and productivity was much less since the students, high school aged and adult, did not need to leave their jobs to undertake study. Bates (in Perraton ed. 1982) cites a cost study done by the UK Open Univerisity that compared radio with alternate distribution of audio material, via cassettes, flexi-discs, and via telephone conference tutorials. In this instance, distribution costs were shown to be dominant, and for the size of the course enrollments involved, the use of broadcast radio was found to be more expensive than that of distribution of cassettes except in cases where the audience was more that 1,000 per year over a period of six years. A study done in 1981 in Indonesia indicated that using radio combined with text for in-service teacher training at the primary level was a cheaper and more effective option to raise school achievement in rural areas than if radio were used to teach the children directly (Setijadi in ADB 1987). The cost-effectiveness of a radio distance learning system is dependent not only on the size and nature of the target audience, but on its being a part of a curriculum of a sufficiently high quality that students do not drop out, or fail their certification examinations. Eicher et al. (1982) cite two case studies, the IRDEB project in Bahia, Brazil and the Kenya in- service teacher training course in the 1970s that had doubtful cost effectiveness outcomes, due to high dropout rates and small enrollments, respectively. For a stand-alone distance learning system based on radio, Jamison and McAnany (1978) provide a thorough cost analysis approach: (1) costs of administration and planning, (2) costs of program conceptualization and production (3) costs of transmission, (4) costs of reception and (5) costs of complementary inputs including printed materials and appropriate teacher - 110 - Educational Technology: Chapter 3 training. They note the difficulties of making an' eneral rules about costing for radio, since each one of these variables can fluctuate from negligible to dominant depending on the local situation. For example, -lthough radios are common in developing countries, not everyone has one, nor can the cost of batteries be ignored for areas where no power reaches. Bates (1980) examines the educational functions of radio by starting from the simple question: why use the media at all? What advantages does it give that simple printed material cannot? He frames his response in terms of 'strategic' reasons. First, radio r:ovides access to learning to students who are removed by location or scheduling from a conventional institution. Bates considers this to be more important than reaching large numbers of students; at the time of his research, none of the existing distance learning institutions that he surveyed was larger than 100,000 students, and individual courses would necessarily be offered to smaller fractions of that figure. So reaching a mass audience with radio would be less of a priority under these circumstances than overcoming barriers of access. Other tactical advantages that radio offers to distance education include opportunities for advertising the institution, providing variety, enrichment and motivation to the students, and pacing the student's progress through the coursework. Studies done on the British Open University have shown that these latter advantages are not always true or relevant, however. (Bates in Perraton, ed. 1982)3 Radio has several major handicaps as an instructional medium; it is a one-way information transfer, it is transient, so that the student cannot go back over something that he/she doesn't understand, and it must be broadcast on a fixed schedule, regardless of a student's own scheduling requirements. Educators are limited by the number of hours of broadcast time -hat they can pay for or have access to. There are pedagogic limitations as well. Bates (in Perraton ed. 1982) cites research that indicates even relatively well educated and well-motivated students find it difficult to learn from radio, stating that "...radio is severely limited as a hard teaching medium, where skills, or difficult ideas or concepts, or a systematic and comprehensive development of knowledge is required." On the other hand, Sharma (1987) cites the advantage that radio has of allowing the student to follow printed material at the same time as he/she hears the broadcast. Bates (in Perraton ed. 1982) also notes the advantages it has of providing motivation, exposure to music and radio-dramas, and its 3 Bates' interpretations zre strongly colored by his experience with organizing open univeristy teaching. Another major source of experience with radio as an instructional medium is the series of instructional radio projects carried out in the 1970s (among the most-studied are Radio Mathematics in Nicaragua, Radio Language Arts in Kenya, and radio teaching in Thailand) with the goal of demonstrating radio's potential for improving the quality of in- school education. Under these circumstances, the ability of radio to reach a mass audience at low cost becomes much more of an advantage. For more information on these programs, see chapter three. - 111 - Educational Technology: Chapter 3 immediacy (it can include up to the minute information), and its relative simplicity from the standpoint of administration and management. Broadcast Television Notwithstanding the continued use of print in distance learning, and strong evidence of cost-saving characteristics of radio, television continues to be used in some distance education programs; while on the one hand, televised education at the primary and secondary levels is relatively rare, on the other hand, in the case of some of the large Asian Open Universities, use of television for instruction is actually increasing. For example, the President of Pakistan has called for the installation of separate radio and television channels especially for the Allama Iqbal Open University in Pakistan (Siddiqui in ADB 1987), while the Chinese Radio and Television University is also expanding its use of television. As noted by Jamison, Klees and Wells, (1978) instructional broadcast television usually costs from $.05 to $.15 per student/hour, about 3 to 5 times as much as radio; they maintain that an audience of close to one million is probably neccessary to achieve the lower cost. So any discussion of broadcast television involves a quantum jump in scale and cost past other available educational media. However, video images do not necessarily need to be broadcast; they can be distributed in two other ways, via closed circuit wires or physically, via videocassette. These alternatives blur the scale distinction, making simple conclusions about television and video in education difficult. (see also next section) Disillusionment with the initial promise of educational media has been most acute in the case of television. Compared to other media, expectations were higher, the flagship projects received more attention, up front investments in capital were much higher, goals were more comprehensive and collapses were that much more painfully felt. As a result, current USAID involvement with instructional television is restricted to informal in-home literacy on the Sesame Street model, (the AL MANAAHIL project), World Bank lending for television projects has fallen off dramatically (Hawkridge, 1987), and Bates (in Daniel et al 1982) finds that worldwide, use of television in distance learning institutions is decreasing. Perry's data shows that useage of broadcast television and of video is similar for most regions: in Africa 17% of institutions replied that they used television, and 17% used video; in Asia, 19% used television and 17% used video; in Latin America, 10% used television and 14% used video. Only in Australasia was there a difference--there, none of the institutions surveyed used television, but 42% used video, perhaps due to the vast distances and lack of television penetration in that part of the world. (Perry 1984) Jamison, Klees and Wells (1978) note that more useful research has been done on educational broadcast television than on radio. Some comparison studies of the two media have been made. Bates (in Perraton, ed. 1982) cites a Unesco study done for Afganistan in 1978 that compared the costs for expanding an existing educational radio service for in-school education to the introduction of educational television for the same purpose. While radio would have reached more than 950,000 children, television would have served - 112 - Educational Technology: Chapter 3 only 2' 000 children in the Kabul area. Start up costs for the educational television system in the first three years were estimated to be three times higher than for radio, and subsequent annual recurrent costs would have continued on at twice the rate of those for radio. Costing for television follows the same basic pattern as for radio. Eicher et al. (1982) delineate four possible levels of cost according to the quality of program production: 1. closed circuit video 2. small scale production (semi-professional quality) 3. middle-sized (commercial quality, non-professional personnel) and 4. large scale (network quality programming). They contrast estimated per hour production costs for low quality video to be about $160 and for broadcast quality television to be $50,000 or more. (see table 6) In the past, start up costs have been a significant component of the expense for television projects. Often large infrastructural components must be obtained, such as production facilities or a transmitter, and extensive research and evaluation for design of the television curriculum may be necessary before the long and expensive production process can begin. The educational agency planning the project will most likely need to depend on a separate broadcasting organization for production and transmission of programming, which is often the source of friction and misunderstanding, as each agency has differing objectiven for cost control, educational content, and broadcast 'quality'. This has been pointed out as an area of particular difficulty by many researchers (Stahmer and Bryan 1988), Bates (1980), (McAnany et al. 1983). Many educational television systems were planned in cooperation with foreign governments or agencies, and salaries for foreign consultants can be four or five times as high as local salaries, further inflating set-up costs. (Jamison, Klees and Wells 1978) McAnany et al. (1983) provide an overview of per hour production costs that suggests an inverse relationship between the number of hours of programming produced and the unit cost. (see Table 6) Although cost was generally directly related to technical quality, technical quality (ie professional-level production value) was not found to have much influence on pedagogical impact. Under these circumstances, "it is economically rational to trade off some technical quality against minimally lesser educational impacts" (McAnany et al.1983). It should be mentioned that production standards on this list vary widely. For example, Stanford University merely televised engineering professors as they gave their lectures, using a single camera, while the British Open University produces edited live action material using highly paid, unionized BBC production staff. Transmission costs are perhaps the most variable element for broadcast television; they either must depend on the relationship between the sponsoring agency of the educational television project and the owner of the transmission facilities, or the project must obtain and run its own equipment. Rumble (in Jenkins ed. 1988) notes that prices charged for transmission can be 1. concessional or free, 2. based on marginal cost or 3. based on commercial rates. Negotiation for time of day for transmission can affect cost, lowering it if it is during off-peak hours. However, time of day can also prove to be a problem in situations where the educational television system must share a - 113 - Educational Technology: Chapter 3 TABLE 6 Cost Per Hour of Educational Television Program Production, Compared to Number of Hours Produced per Year No. Hours Cost per Project produced per hour of year programs Ivory Coast (non-formal) 17 $51,200 Senegal (TSS) 49 16,600 Telecurso (Brazil) 75 53,800 Ivory Coast (formal) 201 25,900 Open University (Britain) 288 18,150 Ceara (Brazil) 300 2,750 El Salvador 333 5,665 Maranhao (Brazil) 525 1,815 Telesecundaria (Mexico) 1,080 925 Hagerstown (USA) 1,440 1,450 Stanford (USA) 6,290 175 sources: Evans & Klees (1976), formal Ivory Coast Klees (1977), non-formal Ivory Coast Klees (1980), Telecurso Oliveira and Orivel (1980), Maranhao and Ceara Jamison, Klees and Wells (1978), all others - 114 - Educational Technology: Chapter 3 schedule with other users (Eicher et al. 1982). Often the best time to reach students is in the evening, after their work day ends, which is also the prime time for commercial and government broadcasters, and usually too valuable to donate for educational use. Ownership of facilities may be more convenient, but less economic. Reception costs are most strongly affected by the level of infrastructure and its distribution in the country. In the U.K., most students own their own television sets, lowering reception costs to near zero. At the opposite extreme, in the Ivory Coast, televisions had to be provided in every school, along with alkaline batteries, since most of the more than 2,000 classrooms involved were located outside of the nation's electrical grid. According to Klees and Jamison (1973 and 1976) the cost of these batteries was one of the major factors in high per student costs that ended the experiment.4 Hawkridge (1983) describes an intermediate situation in China where, although home ownership of televisions is not widespread, electricity is available in most regions, and students' viewing takes place at the factory or workplace, where a set is provided. (see also study centers, below) According to Jenkins (in World Bank No. 491 1981), the educational differences between television and radio are negligible. Others, such as Rumble (in Jenkins ed. 1988) see the qualitative differences between television and radio as being of some importance. It can of course transmit a simple 'talking head', lecture style, but this isn't taking full advantage of the medium; television images can clarify hard to observe processes, show practical skills, reinforce learning through visual cues, and affect opinions and beliefs with powerful images. Images are particularly useful for technical and vocational topics. All of these applications require much more effort and money to design and produce than simpler broadcasts, such as transmission of a lecture to multiple viewing sites. Based on his experience with the British Open Univeristy, Rumble asserts that a television component in a distance education program will help reduce drop outs, increase comprehension, and encourage students to think in abstract ways. The main factor usually cited as arguing against the use of television has usually been cost; based on data from the 1970s, Eicher et al. (1982) declared that educational television should not be considered at all for developing countries unless home ownership of TV sets is widespread, and infrastructural conditions reach minimum standards for power and maintenance. In contrast, Bates has come to the conclusion that cost is not the major reason for lack of broadcasting in distance education. Instead, he emphasizes the institutional problems created by forced cooperation between educators and broadcasters. The lack of control over educational content when broadcasters are in charge of production makes educators leery. Moreover, educators are "unable or unwilling to see the educational, as distinct from the distributional advantages of television or radio" (Bates, in Daniel et 4 Eicher and Orivel, (in Unesco vol 2, 1980) cite other factors as being more important, such as excess sophistication of production level and teacher salaries. - 115 - Educational Technology: Chapter 3 al, 1982). Other barriers he mentioned were lack of appropriate transmission times, and lack of access by the target audience to reception facilities. Non-Broadcast AV Aids Bates (in Daniel et al, 1982) has found that although useage of broadcasting media for distance education appears to be decreasing, use of non-broadcast audio-visual media seems to be on the increase. In particular, audio cassettes have proven to be extremely popular, as well as cost effective for the British Open University, and by 1982, the UKOU was producing more audio material on cassette than for radio broadcast. Perry's survey fotmd that distribution of audio cassettes is employed by 42% of African institutions, 38% of Asian institutions, 70% of Australasian institutions, but only 24% of Latin American ones. About 15% of institutions in all regions use video cassettes (Perry 1984). In a more recent study, Bates (in Jenkins ed. 1988) points out that audio cassettes and video cassettes have radically affected the ratio between fixed and variable costs for audio and video learning, which in the past could only be provided by broadcast. Smaller institutions, or institutions offering courses with relatively small enrollments (under 1,000) are now more likely to be able to take advantage of audio visual teaching methods, since cassettes represent lower fixed relative to variable costs than broadcasting methods. It should be pointed out that for developing countries, this is probably true only for audio cassettes. While a cheap audio cassette player now costs about the same as a transistor radio and cassette players are becoming widespread around the world, video cassette players remain out of reach for all but th3 wealthy in developing countries, given their high cost and need for expensive maintenance. The possibility of installing cassette players in study centers or some other centralized location for student use would be a variable cost, and could quickly become very expensive for a mass audience. Both audio and video cassettes offer educational advantages when compared to broadcast media. The student or group can stop the tape whenever neccessary to allow discussion, responses or repetition of the material. This counteracts the passive nature of most television and radio broadcast learning. A recent study done in China found that video cassettes of lectures were a viable substitute for traditional face to face components in distance education programs for teacher training, but that these results were dependent on high quality print design to improve student study skills, and helped by informal group discussion at study centers (Haughey et al. 1988). Another study done at the UK Open University found that students ranked audio cassettes as the most useful component of their coutsework next to the texts (Bates in Daniel et al 1982). A second advantage of cassettes is that students can use them at the time of day that is most convenient to their study. This avoids the "lock step" problem often cited as a disadvantage for broadcasting, and contributes to increase in access, since students with different schedules can still follow the material. Academics like cassettes because they have greater control over design and production of cassette material. - 116 - Educational Technology: Chapter 3 Face to Face Tutorials & Student Support While not a 'technology', this element of distance learning is a large and important component of the delivery of instruction, and must be considered as one of the ingredients in planning the technology mixture, since use of other media is often designed to coordinate with some form of face to face contact. Evidence on the effectiveness of study centers is contradictory. Some studies indicate that study groups are an important element in maintaining student motivation and preventing dropouts, particularly in Korea (Kwon and Chandong in ADB 1987), Latin America and Africa (Curran and Murphy, 1988) (Smith in ADB 1987). Other studies indicato that study centers set up to provide access to media or tutorial help can be problematic, and can have low attendence due to the distance between students' homes and the centers, tutors' ineffectiveness, or student apathy. Bates notes that, in addition, access can be difficult due to scheduling, and/or overcrowding (Bates in Jenkins ed. 1988). In a study done in 1984 at the Chinese Radio and Television Univerisities, students reported that the tutorials were the least useful of the media employed, in that they tended merely to repeat material already presented in television lectures. (Hawkridge 1985) Nielson (I ) found that cultural learning patterns in Indonesia were incompatible with X question/answer learning format expected in the study centers set up for the Indonesian Open Univerisity, and attendence at the centers fell from as high as 85% to less than 10% over the first three years of the the program; students preferred to organize their own informal groups. Adesola reports that the number of study centers in use by the Correspondence and Open Studies Institute of the University of Lagos fell from 14 in 1981 to 9 in 1986, due to lack of student utilization, although the number of students in the program has remained relatively constant during that period and graduation rates have actually increased (Adesola 1988). The fact remains that many institutions base their administrative structures on a farflung base of local study centers, particularly at the university level. Likewise, for lower level courses, group study with untrained monitors or underqualified teachers is often a basic component of distance learning programs. Some, like the Korean Air and Correspondence High School, have obligatory attendance for group sessions. The study centers may also be required for television viewing, special residential sessions, tutorials, examinations, and science laboratory sessions, which cannot be conducted any other way. And certain programs, such as the Chinese CRTVU and the Portuguese Telescola, use television to teach in a classroom setting, making the distinction between distance learning and in-school use of media a vague line indeed. Advanced Technology Computers. In developing countries, computers have not seen much documented application for distance learning purposes. One exception to this is a pilot project begun in 1985 by the University of Guelph, Canada with the Sukhothai Thammathirat Open University in Thailand, which is testing the - 117 - Educational Technology: Chapter 3 appropriateness of a microcomputer based self-testing system for remote learners in the regional study centers in rural Thailand. (Moore, 1987) Most researchers consider computers as being more realistically applied to problems of internal administration for distance learning in developing countries, rather than to the learning process (Stahmer and Bryan 1988). P. Murphy (1988) recommends that for Africa, computers and telecommunications might best be considered from the standpoint of management of distance teaching programs, for monitoring and evaluation, and for text production. Siddiqui (in ADB 1987) mentions the acute need perceived by Pakistan's Allama Iqbal Open University for microcomputers to aid in the administration of its rapidly growing infrastructure; 16 regional centers serve over 100,000 students, and regional directors need a means to manage and administer their needs as well as an adequate two way communication link with headquarters to resolve problems and to transmit administrative information. Telecommunications. Applications of telecommunications to distance learning may be divided into two categories; transmission media and user equipment. In both cases, use in developing countries has been very limited. However, detailed cost studies of the few pilot projects which have been undertaken in the last twenty years, and extensive speculation about the benefits that could be gained from more experience has generated a substantial body of literature in the area. Transmission eq ipment such as satellites, fibre optics, and terrestrial microwave repeaters all exist primarily as part of national and international telecommunications infrastructure. Their use would affect distance learning by increasing the reach of distribution of any electronic signal (ie telephone calls, computerized information, radio, or television signals) that the educational system wishes to transmit to its users, or by possibly decreasing distribution costs in instances where an increase in the volume of telecommunications traffic would allow reductions in tariff rates. Television and radio can also be transmitted by satellite, and are essentially unchanged as teaching media by satellite distribution; however, reception equipment and project economies of scale are affected. Int llation of new transmission media such as satellites and fiber optics have ;.over been made purely for educational purposes in any country, rich or poor, due to the extremely high costs -.nd the technical complexity involved. Hence their availability must be considered a priori to any realistic distance learning application. Availability, however, is only the first step. Experience has shown that plans to use satellites and other telecommunications media for educational applications are usually not seriously backed up by institutional support or funding, and are often blocked by political, economic, and institutional barriers. Expectations in the 1960s and 1970s that use would spread rapidly have not been fulfilled (McAnany and Nettleton, 1987). Telephones are a 'difficult' technology, and in many developing countries, not only are they non-existent outside of urban areas, the overall quality of the terrestrial, non-satellite telephone system may be so low that reliable transmission of an adequate signal to the correct destination cannot be counted on. - 118 - Educational Technology: Chapter 3 Current use of satellites for oducational purposes in developlng countries is low; the number of active programs can be counted on one hand, and international institutional support for continued pilot testing appears to be drying up. USAID, which sponsored some of the most well known experiments through its Rural Satellite Program, and Intelsat, which has been subsidizing alternative applications for surplus satellite capacity in its worldwide system through Project Share, are both withdrawing their financial support for further study and operations. The most active use of satellite technology for educational applications in the late 1980s outside of Canada and the United States is India's Insat, which broadcasts non-formal educational programming to primary- school aged children. It is reported that Chlna is developing a system via Intelsat to distribute television courses for its CRTVU (see Section III A 2). Some limited use by the University of the South Pacific, the University of the West Indies and by two universities in Indonesia appears to be continuing, despite difficulties. Use of telephones for distance learning is also infrequent in developing countries. Perry's study showed that in Africa, no institutions used telephones for instruction, in Asia only 7% did, and in Latin America, 17% made use of this medium (Perry 1984). Analyzing the costs of telecommunications in education is extremely complex. Projects can be small scale, as would be the case when students or professors use lines in the public telephone system (phone calls, leased lines between study centers for teleconferencing or slowscan television transmission), or they could conceivably be very large scale, if an entire satellite transponder were dedicated to distribution of educational broadcasting to a nation. Moreover, the technology is changing rapidly, so cost generalizations are rapidly outdated by advances in microslectronics, as well as by changing economies of scale as different kinds of equipment spread through the marketplace. Finally, political and institutional factors have a tremendous impact on costs because the transmission equipment and tariffs are almost invariably controlled by national telecommunications companies (PTTs), whose primary responsibilities do not include social services. It is common practice to break down satellite costs into the categories of (1) the space segment, which includes purchase and launch of tho satellite, and ongoing transmission costs and (2) the earth segment, the (often unrecognized) larger portion of the total cost which includes up- links, reception equipment, costs of return communications, provision of services, support costs such as teachers and printed material, and administrative costs. (Bates 1986) etc. This costing method implies that the launch of the satellite is an integral part of the educational applicatlons planning, and may be somewhat misleading; given that many satellites with idle capacity adequate for social applications are already financed and in orbit, costing for satellite utilizations might more realistically concentrate on the educational, administrative, and ground equipment categories. Goldschmidt (in Blume ed. 1984) discusses costs for telecommunications projects mainly in terms of problems they can present to planners. Most important is the fact that pre-occupation with the mechanics of setting up the technical linkages nearly always means that software and - 119 - Educational Technology: Chapter 3 programming are seriously shortchanged. His second major point is that in the casf of satellite education projects, it is very difficult to establish an affordable space segment transmission tariff for non-profit applications for television and dedicated networks, given the reluctance of national telecommunications companies (PTTs) to accept the concept of discounted rates based on the marginal costs of using idle satellite capacity and their preference for charging commercial rates with only a small discount. It appea_s that transmission and other recurring costs seem to be the downfall u%f most educational satellite applications. External funding agencies have been reluctant to pay for anything except initial set-up and fixed capital purchases, and the high tariffs and operating costs are often beyond budgetary feasibility for non-commercial users. Alternative sources of revenue, such as special taxes on commercial system users, should be considered in the future. III. ImDlementing Technologv for Distance Education It is important to bear in mind that distance education is not by definition an educational technology, but rather it is a way of providing education that relies on educational technology (here including printed materials) in practice. Hence, a discussion of technologies without consideration of the context in which they must be applied has little relevance to real world policy applications. Distance teaching can take place in school or out of school, it can serve a clientele ranging from school-aged students to adults, and it can teach subject levels from primary school to post-graduate engineering. Institutions can be government-run, commercial, church sponsored, autonomous, linked to existing institutions, large, small, and of varying levels of quality, prestige, and effectiveness. Context includes such exogenous factors as: (i) social attitudes toward distance education as an alternative to traditional modes; (ii) labor market conditions and their effects on the pool of candidates for distance education; (iii) relative supply of places in traditional schools at each level; (iv) adequacy of "infrastructure" such as broadcast communications networks and the distribution of ownership of radios and television sets; (v) administrative arrangements and relationships between, say, the distance education program and the agency controlling broadcast facilities, and (vi) other subtle points such as cultural attitudes toward diligent, self-directed work (McMeekin, 1989). Failure to take such factors into account during the planning process will greatly reduce the chance of implementing a successful program. Suprisingly few attempts to gather comprehensive data on the nature of distance institutions in existence in the world today have been made. Daniel emphasizes the lack of information covering the greater part of distance education enrollment at the tertiary level; 3 out of the total 4 million students in distance universities are found in China, the USSR, South Korea, and Thailand, yet few studies have been done on these countries (Daniel, 1987). - 120 - Educational Technology: Chapter 3 The importance of a sense of time is underrecognized by analysts in the field. Many case studies carried out ten or fifteen years ago have come to stand as symbols, on their own, of achievement for a particular educational technology, although information about what has happened with the project in the years since is unavailable. As pointed out by Stone (1987), a number of these now-famous projects have ceased to ex4ist or limp along in a perpetual embryonic condition. On the other hand, Jamison, Klees and Wells (1978) point out that "high present costs combined with future utilization of technology projects may result in a requirement that projects last ten to twenty years to allow unit costs to fall to a reasonable level". In other words, not only does the passage of time force a 'successful' project to prose its institutional durability, it also allows a 'failure' to reach maturity and redeem itself by showing improved cost/effectiveness results. It is generally accepted that it is nearly impossible to make a single long term cost study over a period of years for a particular institution (Jamison et al. 1978) (Chale 1983). Perraton (1982) compares an economic analysis to a still photograph: "it looks at an institution at the time the study was made. It may catch a good likeness, or it may be distorted." Bearing these cav-eats in mind, Tables 7 and 8 show some rough cost comparisons between distance and conventional learning institutions. The comparisons are necessarily somewhat poorly defined; the fractional comparisons were taken as given in available studies, and could not be standardized to take into account the differences in cost per student year, per completer, or for full-time equivalence. Also, the conventional alternative is seldom well-defined by the researchers. Thus there is no way to know whether they are holding the disted institution up to the finest and probably the most costly equivalent educational institution (making it appear proportionately cheaper), or to an average quality institution. Some of the programs where distance education costs have been equal to or greater than conventional alternatives are not necessarily considered to be unsuccessful. For example, Telesecundaria costs 10% more than conventional schools in easy to reach araas, but Telesecundaria schools exist only in remote rural regions where conventional schools would cost twice as much to set up and run. Financial support for distance education institutions can come from a variety of sources. Large public universities are likeely to be funded by the national government. Smaller programs might depend on state or even municipal level support, or could be backed by the Catholic Church. Some programs are linked to existing educational institutions, such as correspondence units that depend on a university for courseware, admissions, and funding. Foreign aid has played a big role in some of the past experiments that have involved large scale implementation of new technologies, but has generally proved unsuccessful in sparking local support to continue the projects once the aid is discontinued. - 121 - Table 7 COST COMPARISON: CASES WHERE DISTANCE EDUCATION COSTS LESS THAN CONVENTIONAL 2 institution year cost per student dropout source o compared to con- rate a ventional alter- = nati-, e Universities: UKOU (Britain) 1982 2/3 43Z Perraton (1986) STOU (Thailand) 1984 1/3 502 Srisa-An (1986) CRTVU (China) 1984 1/3*to 2/3 302 Otsuka (1984) *includes part-time students UNED (Costa 1986 . 114 to 1/2 402 Sandoval (1988) Rica) -- UNED (Spain) 1984 113 to 1/2 NA Garcia Aretio (1987) Teacher Training: Logos II 1980 1/6 20X Oliveira & Orivel (1988) (Brazil) Tanzania-- 1981 1/4 77Z Chale (1983) Teacher Train- ing at a Dist- ance Primary and Secondary: ralpwi Cor- 1986 1/5 very high Murphy (1986) College Telescola 1976 1/4 to 1/3 NA Taylor (1979) (Portugal) Radec* 1985 1/2 10 - 152 Sanguinetty (1985) (Dominican Rep.) °J* Table 8 co COST COMPARISON: CASES WHERE DISTANCE EDUCATION COSTS MORE THAN CONVENTIONAL o institution year cost per student dropout source o compared to con- rate s, ventional alter- > native o University: UNED 1979 NA 88X Escotet (1980) (Venezuela) Teacher Training: Correspondence 1977 more expensive less than Perraton (1986) Course Unit 1OZ (Kenya) Primary and Secondary: Telesecundaria 1988 101 higher learning Arena (1988) (Mexico) = or better Ivory Coast ETV terminated, perceived as few learn- Hawkridge (1987) (primary) 1982 "grossly un- ing gains cost-effective" Malaysia ETV 1983 very high low useage lawkridge (1987) (primary, & add-on costs secondary) Educational Technology: Chapter 3 Unlike conventional institutions, distance programs are much more likely to charge fees, and depend, to a greater or lesser extent on some cost recovery. At one extreme, in India, many correspondence units are degraded into mere revenue producing arms of existing institutions. (Mallick, ADB 1987) In contrast, in Nigeria, the COSIT unit attached to the University of Lagos found it very difficult to charge fees at all without provoking a student strike. In the case of the Chinese RTVU, employers pay the entire fee for the students, and roceive the benefit of better trained employees in return. The ultimate cost effectiveness of a project is usually dependent on the following factors: the number of students, the sophistication of the media, the amount of face to face education in the program, the educational effectiveness, and the quality of administration and management of the program. Measurements of effectiveness are harder to collect than simple costs data, and for certain kinds of purposes, such as measuring the success rate of students in getting jobs after graduation, are usually unavailable. Most cost data is fairly self-evident in its relevance. The number of students will affect the degree to which economies of scale will help reduce unit costs, although as the total size increases, overall capital expenditures will go up too. Adequate administrative structure is another expensive, but essential component of distance education institutions. Good managers who can keep the complicated, unfamiliar operations of a disted program running smoothly must be paid salaries that are competitive with those in other jobs they might take in the private sector. On the other hand, predicting the costs incurred by the degree of sophistication of media is not always as simple as it might appear; while certain tools such as televisions and microcomputers for computerized learning are obviously going to fall into a high-cost category, comparisons between print and audio cassettes, for example, may not be so clear cut, given the variability of paper costs, and expense of importation where local printing facilities are inadequate. A microcomputer that streamlined the distribution of courseware might increase efficiency and reduce dropouts to the point that the ultimate savings greatly outweighed the original expenditure. What is most desirable is an appropriate match between technological sophistication, the nature of the educational need, and local context factors. Factors directly related to effectiveness may be somewhat more complicated to evaluate. Face to face meetings, lab work at study centers, residential sessions, and student support by prompt feedback on coursework are expensive components for distance institutions to provide; however, cutting corners on this aspect of the program will generally result in higher dropout rates and lowered educational effectiveness. This will, in turn, raise the cost per graduate, if less students are able to complete the degree. A related point is the importance of gaining and holding an adequate academic standard for the institution. Teachers and tutors are sometimes overworked and underpaid, and may not be well-qualified; they need training in teaching techniques specific to distance education, and in the case of higher education, they need time to carry out scholarly research. If the quality of learning is low, and control of student testing is slipshod, graduates will have little academic credibility compared to their counterparts who took their degrees at conventional institutions. The resulting perception by students, - 124 - Educational Technology: Chapter 3 employers, and che population at large that the institution is second rate is in itself a serious degradation in the institution's effectiveness. And finally, certain important goals for a distance education institution may be in conflict with an overall objective to reduce costs. For example, many programs are designed to increase equity, and special efforts aimed at providing effective access to education for new or disadvantaged groups may increase costs, according to Perraton. IV. Regional Survey of Applications In order to convey an overview of the state of distance education worldwide, it has been necessary to organize the huge topic according to some arbitrary criteria. A regional division seemed the best way to break the information down, given the tendency for certain themes and issues to crop up again and again in groups of countries that share cultural and geographical similarities, and given the increasing interest that is evident in promoting cooperation among institutions at a regional level. As mentioned in the introduction, cooperation tends to grow first between countries that have historical and cultural ties, such as among the Latin American countries and UNED in Spain. Africa Perraton (1986), Curran and Murphy (1988), and Dodds and Inquai (in Jenkins ed. 1988) provide an excellent summary of experience with distance education and related technology up to the mid-1980s in Africa. They have found that both concrete experiences and related evidence on cost effectiveness are relatively limited for the region. However, between 1966 and 1977, the number of public correspondence programs in Commonwealth Africa rose from 3 to 12. Primary and secondary school levels. No data is readily available on primary level distance education programs in Africa, but at the secondary level, some experience and research is available, primarily related to secondary equivalency courses for adults. Perraton notes that such projects have generally had low enrollments, leading to relatively hig,h per student costs, and their teaching effectiveness has been seriously compromised by shortages of materia] and financial. resources. Curran and Murphy (1988) advise that costs are likely to be higher than exp.eted, and effectiveness very low for programs where individuals must study alone. They note that group study and use of study centers can improve effectivness by giving the students increased support and motivation. Some successful programs are cited; the Correspondence Education Unit of the University of Ghana reports providing secondary level education at a cost of 1/40 per enrolled student as compared to a conventional school although data on completion is not provided (Ansere in Daniel et al 1982). The Malawi Correspondence College provides 'second chance' secondary education at a cost above conventional day schools but less than for boarding schools, - 125 - Educational Technology: Chapter 3 and the Mauritius College of the Air has been able to improve secondary curriculum and provide schooling for dropouts and adults at costs that compare favorably with other available options (Perraton ed. 1982). Teacher education. Teacher education has been the area of distance education considered most successful in Africa; it is usually offered as a secondary level training program, and has a long history going back into the 1960s, based largely on correspondence courses. The most common media mix for these institutions is print, supported by supplementary radio programs and varying amounts of face to face contact. Chale (1983) found that teacher training at a distance in Tanzania was not only more effective, it was also four times cheape? per graduate than an equivalent conventional residential program. On the other hand, in his survey of teacher training at a distance in Southern Africa, Taylor (1983) found that many of the projects were small, and therefore not necessarily cheaper than conventional methods. He feels that economies of scale for teacher training at a distance can begin for enrollments in the 1,000 to 2,000 range, and he concludes optimistically that given a distance teacher training project of sufficient scale, "...it is unlikely that the same degree of effectiveness could be achieved at lower cost by any other means". Higher education. Distance teaching at the university level in Africa has been generally limited to several instances where existing universities have developed correspondence units to serve students off- campus, pursuing the same coursework for an identical degree as the on-campus students. A very thorough cost analysis has been done of the COSIT unit of the University of Lagos by Cummings and Olalaku (1988), who found that the unit cost for graduates at a distance was nearly the same as for those enrolled in regular programs, given that graduates generally completed the courses in the allotted time, and graduation rates did not fall below 80% for the education degree and 50% for business administration degree. These relatively high completion rates were attributed to high motivation and self discipline in the adult learners, and the incenitives of job advancement upon completion of the course of study. The one open university on the continent is the University of South Africa, which enrolls 60,000 students (13,000 of which are Black). Efforts to create an Open University in Nigeria collapsed in the early eighties. (Daniel 1987). Asia The number of active distance education programs in developing countries in Asia far outstrips the substantive research that has been done on them, and the explosive growth that is taking place in student enrollments today threatens to widen that gap into an abyss. Asia is a region of extreme cultural and economic diversity, and generalizations are hard to make, except on a few points. Most countries in the region have very large populations, and demand for education is very high. Another prevalent characteristic is that di. tance education is highly concentrated at the university level. Almost io primary school applications have been developed and very few exist - 126 - Educational Technology: Chapter 3 at the secondary level. Use of media for informal education is more widespread, concentrating on literacy, agricultural extension, and other rural programming, but does not fall under the scope of this chapter. Primary and secondary education. Thailand is one of the few developing countries in Asia with experience in distance education at the primary level. The government offers in-school radio programming that supports the curriculum directly in three subjects: social studies, music, and English. (Chaya Ngam in ADB 1987) Secondary education at a distance is well developed in Korea, where the Air and Correspondence High School (ACHS) provides self-instruction texts and broadcasts instructional radio programming to working youth and adults who wish to obtain a high school level certification. The use of media in this case is closely coordinated with student supervision and evaluation; notes taken during the radio broadcasts must be turned in along with other written tests for marking, and affect the student's grade. (Kwon .nd Chandong in ADB 1987). In India, correspondence courses at the secondary level enroll only 0.31% of the total school aged population at that level, and cater mainly to school dropouts. These programs are financed by fees paid by the students, and are considered to be of poor quality, and weakly developed (Mullick in ADB 1987). Higher education. The lion's share of distance education programs in Asia exist at the tertiary level. Nearly all of these institutions were started within the last ten to fifteen years, nearly all make use of broadcast media to teach, as well as print and face to face meetings. Many are massive in size, and most are public, being run by their governments in cooperation with varying combinations of other public agencies and organizations. There are several reasons why so many institutions with so much in common have all sprung up at the same time. Population growth has combined with successful expansion of primary and secondary education to swell demand for university spaces to an unprecedented extent. In the case of some countries, such as Malaysia, this problem has been compounded by increasing difficulties in sending students abroad to study. Economic progress has raised social expectations among large portions of the population, who want to advance themse'ves through study. Changing technology has affected the kinds of skills needed by most countries to fill employment needs, and the demand for highly skilled workers in the scientific and technical areas has risen dramatically. Many countries find that the shortage of skilled manpower extends into academia, and not enough professors can be found to staff expanded universities. Costs and effectiveness of distance universities. Costs per student at the large distance universities have been shown to be, in general, substantially less than for equivalent, conventional universities. (See Table 9) Effectiveness has been much more difficult to evaluate, and what data there is tends to support the idea that some of the distance teaching universities may be suffering from serious flaws, while others have been able to maintain relatively high levels of quality and student support to minimize failure and drop out rates. - 127 - Table 9 v ASIAN DISTANCE UNIVERSITIES: COST AND EFFECTIVENESS DATA country/ costs costD effectiveness 0 institution costC % 4 -type cost/stud. cost/stud. 'measure used rate DISTANCE CONVENT. c) U) Thailand 0 44 STOU average B7,023 B 49,957 14.06% Z dropouts 50% operational per grad 1982 Pakistan AIOU average RS 4,585 RS 6,469 41.53% mean rate of 42.5% per stud. dropouts for 1988 est. all courses China total cost Y 1,000 Y 2,000 50.00% % graduated in 69% per stud. 1982 from 1979 CRTVU per yr. enrollees 1981 Korea KACU total cost US$ 125 US$ 1,250 10.00X X dropouts 50Z per stud. after 1st yr. per yr. of study 1985 total cost NA NA 90.00Z Z graduated 30% per grad. Educational Technology: Chapter 3 STOU in Thailand has avoided serious problems with effectiveness through a number of policies: academic standards have been kept up by involving outstanding academics from other universities, implementation was carried out with deliberation, and involved a three year analysis and planning phase that included contact with other open universities for exchange of experience, and a decision was made to rely on print rather than broadcast media for teaching purposes, as it was considered more appropriate for the level of development of the country. Overall, its reputation in the international distance learning research community is high, although the drop out rate is around 50% (Chaya-Ngam, 1987). Completion rates for the Chinese Radio and Television Universities (CRTVU) have been remarkable; of the 110,000 that registered full-time in 1979, 69% graduated in 1982 (Hawkridge, 1983). In the case of the Korean KACU, even with a graduation rate of only 30%, the costs are so low that the program remains cost effective. The high failure rate is attributed to "maladjustment to self-learning methods" and to insufficient free time available to the working students to complete the full load of five or six courses required by the program. The low costs of the KACU depend on the extensive contributions made by cooperating agencies. A five-day residential requirement at the end of each semester for each course, and attendence at regional study centers are made possible through the loan, free of charge, of appropriate facilities by at least 50 different institutions, including the national universities in each province. These face to face meetings are felt to be the most important component in reducing dropout rates, and if KACU had to provide these facilities itself, costs would approach those of conventional education institutions. The KACU is thus highly dependent on the pre-existence of a well developed educational sector on which to base its operations. The Universitas Terbuka in Indonesia has suffered much greater problems in its first few years of operations. Some allowance must be made for the fact that the program is still only five years old, but the project was implemented with very little opportunity for planning or analysis. The decision to create the UT grew out of several decades of involvement with in- service teacher training by correspondence to create and maintain teaching staff at the secondary and tertiary level. When it was decided to create the UT in 1983, the Preparation Committee was given only 9 months to create the entire institution. A decision to increase the size of the first entering class from 25,000 to 65,000 was made quite late in the planning process. As a result, drop out and failure rates have been very high, student services of very poor quality, and administrative confusion created by the adoption of a continuous enrollment system led to an error rate in registration of over 40% (Nielson, forthcoming). Media use in Asian higher distance education. Media employed by the UT in Indonesia has been based on print, but includes tutorials, privately organized study groups, and some use of the PALAPA satellite to train tutors in a few remote locations. Problems with the tutorial system were mentioned above, in section II - E, and many students have preferred to hire private tutors and organize their own study groups. The satellite is being borrowed from the regular university system twice a week to train tutors in the hard- to-reach Eastern regions of Indonesia an effort to improve this component of - 129 - Educational Technology: Chapter 3 the program. In contrast, the printed materials have proved much more successful, and are even boing adopted for use in some of the more modest conventional universities, since there ls a shortage of text material written in Bahasa Indonesian. The huge size of the enrollment has forced the UT to adopt a computerized marking system for its testing, which many feel to be a severe academic disadvantage at the tertiary level. Much greater use of media in the teaching process occurs in the KACU and in the Radio and Television Universities in China. An important component of the KACU is the radio lectures, which are broadcast a total of seven hours per day by the national educational radio network, KNBS, and students apparently make use of these, as well as of audio cassettes and videocassettes, which they can view in the regional study centers. Only one hour per week is available for television broadcast. In both cases, production is subject to pressures of time and expense, and the format is generally lecture only. yse of audio visual material varies greatly, depeoding on the course. The Chinese system of Radio and Television Universities is perhaps unique in the world for relying very heavily on televised lectures produced by the Central Radio and Television University (CRTVU), and distributed along with related course materials to over 1,000,000 (as of 1985) students enrolled at provincial television universities (PTVUs) (Hawkridge, 1987). In 1983, CRTVU offered a total of 91 courses in the areas of mechanics, electronics, mathematics, physics, Chinese, economics, and management. Only one national television channel exists, and it carries five hours a day of the televised lectures, a situation which might not be economically practical in most other countries. Opposite to the situation prevailing in most other countries, there is no available time on the national radio network, and the CRTVU has had to use provincial radio stations where it could get them. As is the case in most other distance universities, cooperation with local level institutions is of great importance. The system has a four tier structure that ranges from the CRTVU down to the Provincial Television University down to branch schools down to the individual factories and institutions that provide the locales for the television classes. Lower levels of the structure provide tutors, local administration of student affairs, examinations etc. and coordinate the classes (Hawkridge, 1983). The other distance universities in China make use of a combination of printed materials and face to face meetings; the latter is considered to be very important, and staffing ratios range between 25 and 40 students per full- time academic (Daniel 1987). While no published study has yet been done on how China is using satellites for education, it has been reported that three Intelsat transponders have been leased and/or purchased for use in the television university system, along with 5,000 television receive-only (TVRO) antennas and 53 uplinks (Pelton, Intelsat, 1988). Middle last and North Africa The Middle East and North Africa have had the least concrete Involvement in distance education of any region in the world. No more than 5% - 130 - Educational Technology: Chapter 3 of the in-school population in Arab countries benefits from any educational broadcasts by television or radio (Int. Rev. of Ed. 32 (3)). Basic and non-formal education. Kinyanjui (1974) mentions a case study from Algeria of two Unesco sponsored correspondence-based basic education programs that were designed to compensate for the loss of many French teachers after independence, and as part of the arabization of the Algerian curriculum. One innovative approach made use of the newpapers to distribute basic literacy and numeracy coursework to factory workers, whose worksheets were then checked and graded at their places of employment. The other was a crash program to qualify in as short a time as possible the 13,000 classroom monitors who had been recruited to replace the teachers. In this program, pure correspondence proved inadequate, and a multimedia approach using radio, television, face to face meetings in the evenings, at work camps etc. was developed. An effort in the direction of informal education by television was recently made by Jordan Television. In conjunction with USAID and The Children's Television Workshop in New York, Jordan Television produced 65 one half hour shows for a series titled AlNanash_il. The series was, in effect, a basic literacy project in Modern Standard Arabic for lower primary aged school children. Evaluation to date has concentrated on the effectiveness of the program to improve children's reading skills, and the acceptability of the program to teachers and parents. R.T. Murphy (1988) found that the shows were very popular with children, and considered to be of high quality by those teachers surveyed. He also reports that positive findings from his detailed study were replicated by other studies conducted in schools in Jordan, Morocco, and Tunisia. No cost analysis has been made, but the project has been generally characterized as relatively expensive (the cost was between $8 and $10 million) on an absolute scale, but very popular and currently being shown in eight countries, and therefore relatively cheap on a per capita scale. Higher education. Pakistan is the only country in the World Bank's EMENA (Europe, Middle East and North Africa) region with much documented involvement in distance education. Suffering from extremely low rates of literacy (only 26.2% in 1981) and an extremely low rate of government spending on education (only 1.86% of GNP during the 1983-86 period) Pakistan has sought an innovative solution in the creation of the Allama Iqbal Open University, which serves many purposes, ranging from literacy and adult training up to masters degree level coursework. An important objective of the Univeristy is to reduce inequality of access to education between males and females; in Pakistan, socio-cultural restrictions requiring women's seclusion are very strong, particularly in rural areas, and distance education was identified as the only way to reach these potential students. Media use has been influenced by this heterogeneous audience; cassettes, pictures and diagrams are required for illiterate learners, while correspondence packages, radio, television, and face to face tutorials at study centers form the backbone of formal coursework. Total enrollment is estimated at 60,000. An ODA evaluation carried out in 1979 found that Allama Iqbal University "would progressively show considerable cost advantages over other conventional institutions", and a subsequent evaluation in 1983 found - 131 - Educational Technology: Chapter 3 that given sufficiently high numbers of students, AIU's graduate -level courses could be offered at a lower cost than conventional universities. Internal costing efforts have shown that AIOU can award an intermediate degree for 44% less than an equivalent conventional institution, and a bachelors degree for 38% less (Siddiqui in ADB 1987). Mention has been made of the Al-Quds Open University in Jordan, which has been set up to serve the needs of Palestinians scattered throughout Jordan, Lebanon and Syria but no published research is yet available on it. (Daniel, 1987). Latin America Distance education in Latin America makes use of a wide range of technologies for diverse objectives, as compared to distance education in other regions. Thus, some of the programs cited here violate the strict definition of distance education put forth in the introduction, but deserve mention for what they indicate about educational media in the region. Although the absolute enrolment numbers in formal school equivalency programs at a distance for the region are not as high as in Asia, political and social support for the concept is strong in several countries, particularly Mexico, Costa Rica, Colombia, Venezuela, Chile, and Brazil. Research is supported by two major journals and at least five professional organizations in the region. Distance education is most commonly used for rural radio education, open university education either via stand alone institutions or as an add-on component, and for post-secondary teacher education. (Chadwick 1982) Other important areas are primary educatior., indigenous programs for Indians, and teacher training for multigrade classrooms. Latin countries are distinguished by generally open communications policies; not only is there an abundance of radio and television stations in most countries (commercial, Church-owned, and government-run), there is also a widespread tendency for governments to take an active role in promoting educational broadcasting. (Friend, 1989) In the case of Brazil, the government has set up a mandatory system requiring of all broadcasters that a certain number of hours per week be allocated free of charge for didactic programming. The use is generally for self-taught courseware and informal learning that falls outside of our definition of distance learning. Limits on the effectiveness of such policies appear to stem from institutional causes rather than government regulations. A study done on empty radio frequencies allocated for educational purposes in the state of Rio de Janeiro found that they were not being used because of a lack of interest, lack of information on their existence, and a lack of money to implement programming. (Blois, 1984) Primary and secondary education. Primary schooling at a distance has been attempted more in Latin America than anywhere else, and while some of the most well-known efforts, such as the in-school Instructional Television system in El Salvador (1965-mid-1980s) have ceased functioning, other innovative technology based projects have been able to continue, such as the Telesecundaria in Mexico and television education in Maranhao and Ceara in Brazil. In view of the fact that children-lack the social development to - 132 - Educational Technology: Chapter 3 study on their own, distance education at the primary level is usually done in an in-school context, although rural basic education schemes such as ACPO in Colombia depend on groupings of families and neighbors to provide the learning structure for the children, who study together with the adults. A project which attempted to be the basis for a comprehensive national educational reform was the Instructional Television system in El Salvador, which at its height served over 223,000 students enrolled in the formal school system. However, despite favorable effectiveness indicators, political problems overwhelmed the project and it was eventually discontinued. Teacher discontent with changes in work loads and salary structures weakened the system, and according to the OAS, when the El Salvadoran Ministry of Education and Culture was split into two separate ministries in the mid- 1980s, the channel was re-assigned to the new Ministry of Culture, and the didactic ITV component of its programming more or less eliminated as a result (Garzon, OAS). Educational television is being used in Brazil for the second half of primary school in the states of Maranhao and Cear4. In the case of the former, it serves poor children where there was a shortage of qualified teachers, and in the case of the latter, it is intended to improve the quality of education and expand schooling. Both projects were found to be quite cost effective in a study done in 1978. (Oliveira and Orivel, in Unesco vol. 2, 1980). The Maranhao project was re-evaluated by Stone in 1987, who observed that the original goals for the project have been met, but that its efficiency has been reduced by overstaffing with non-teaching personnel. Even after two decades of implementation, the overall scale of televisior use for formal distance educ.ation in Brazil is small. One of the few successful and expanding utilizations of television for non-university level education is found in Mexico. The Telesecundaria program provides formal education equivalency for 7th through 9th grade levels in rural areas where population density is very low. Initial studies showed costs to be 25% lower for Telesecundaria than for traditional schooling (Mayo, 1975) and then, six years later. ti.at they were about the same (Molina, 1981). Arena's study in 1988 showed that costs per study group (or per class) were nearly 50% less than for the traditional system, but that costs per student were higher, due to the fact that Telesecundaria classes were much smaller (a function of the low population density). Most interesting is that television costs do not make up more than 9% of the total cost, which Arena attributes to the fact that the project has grown sufficiently to take advantage of aconomies of scale provided by television. Adult education. Adult education to extend primary or secor,dary level coursework to those who never completed school is the second largest group of distance education projects for the region. Many of these programs are informal, and many are supported by the Catholic Church, which is very active in providing community development and literacy, especially in rural and marginalized areas. A strong tradition of idealism and dedication has contributed to the success and durability of many of these projects. For example, Radio Sutatenza was started in Colombia in 1947 by a single priest and has now grown into ACPO (Acci6n Cultural Popular) an organization employing more than 1,000 persons, that has served as a model for programs in at least twenty other Latin countries (Chadwick, 1982). - 133 - Educational Technology: Chapter 3 While radio has been the media most frequently employed to reach rural audiences, due to its wide coverage, to the availability of cheap receivers in most locales, and to its usefulness for reaching illiterates, some adult distance education programs are based entirely on print. and rely heavily on tutors and study centers. In the case of CENAPEC in the Dominican Republic, which provides secondary school equivalency and is based entirely on printed material combined with optional tutorial sessions, the average cost per student was approximately half of that per student in traditional schools, and about 60% of the cost per graduating student in traditional schools (Muniz Aquino, 1988). Oliveira and Orivel found that a print and tutoria.' based in- service teacher training system in Brazil called Logoa II was six times more cost-effective than traditional, residential teacher training methods. (Olivi.era and Orivel, 1981) Higher education. According to a survey made by the OAS, almost one half of all distance education institutions in Latin America are at the post- secondary level, and are run by either universities or government agencies. Teacher training is the goal of two thirds of this group. Much attention has focused on the two open universities, UNED (Universidad Estatal a Distancia) in Costa Rica, and UNA (Universidad Nacional Abierta) in Venezuela, but many other distance education programs are operated as add-on units to existing universities. For example, in Colombia, instead of creating a massive centralized open university, the government has set up a coordinated effort between existing conventional institutions, which each contribute academic programs to a national system of regional study centers. Nearly 35,000 students are currently enrolled in the system, which is supervised by the university UNISUR in Bogota (UNED-Madrid, 1983). The most common media employed in the region are printed material along with face to face instruction, although the UNED Costa Rica uses broadcasting and audio visual cassettes, ard the Universidad Cat6lica de Santiago, the Universidade de Chile, and the Universidad Cat6lica de Valparaiso reportedly use educational television systems for teleducation. Escotet (1980) has attributed the problems which have beset the initial years of UNA in Venezuela to a lack of sufficient funding and time for preparation. As a result, drop out rates for the first several years were in the neighborhood of 88%, and cost effectiveness was obviously very unfavorable. UNED Costa Rica has been more fortunate; a recent study done by Guadamuz Sandoval (1988) showed that after eight years of operation, cost per student of UNED is 30% less than for the University of Costa Rica, and 50% cheaper than the other Costa Rican University, Universidad Nacional. High dropout rates in the first year of near 60% have fallen to a range around 40% in 1986. A study in 1981 analyzed the cost implications of different media mixes for the UNED, and predicted that while the production costs for the printed course units (which are the basic teaching media) would fall from 15% of total expenditures to 2% once the up-front phase of production had been completed, it was also expected that the proportion of the budget devoted to production and broadcast and audiovisual materials would increase from 4% to 22%, if UNED were to purchase its own production facilities. (Rumble 1981) - 134 - Educational Technology: Chapter 3 V. Oercoming Barrirs tg _tation General Lessons Learned -Local Initiatives: In general, it has been observed that distance education projects have a better survival rate when the majority of their sources of funding are local, and when the planning initiative is taken by local governments. -A strong social demand for education: Success for distance education programs is particularly likely where there is a large surplus of qualified applicants who cannot be accomodated in the conventional school system. Costs: In many cases, distance learning has been found to provide education of a similar quality to conventional education at less cost. Generalizations abeout which media are more cost effective are difficult to make in the abstract, because media are almost always used in combination, and because local contexts and variable external factors influence both costs and effectiveness so strongly. -Infrastructure: The delivery of learning material, the feedback process between teachers and students, and the administration of the program are all dependent on the level of development of local infrastructure. The choice of instructional mix and relative costs of delivery among print, broadcast, audio visual aids and face to face study centers are highly dependent on this factor, as is the turnaround tim3 for student feedback on course work. -Institutiogal Autonomy: Budgetary problems can be aggravated for distance learning institutions by a lack of autonomy. Curran and Murphy (1988) note that publicly funded institutions are usually subordinate to a parent university or government unit, which often controls staffing, and determines budget cuts and expenditures. -_Dstitutional Linkages: Relationships with other organizations are often crucial to efficient operations. Facilities can be shared with existing, conventional institutions to save costs, and course materials and expertise can be exchanged internationally between distance institutions. Greater integration between distance and residential programs is growing, especially in teacher training and higher education, and this has helped improve institutional legitimacy. Academic Status: distance teaching institutions have had, in many cases, to overcome a stigma of being second class relative to other academic institutions. Future employers need to be persuaded that the certification from the distance institution is as valid a qualification as that provided by conventional education. Poorly run institutions that act as 'diploma mills' damage the credibility of all distance learning institutions. - 135 - Educational Technology: Chapter 3 - Motivation: minimizing dropouts and maximizing effective learning depends on high student motivation. Good program and materials design, as well as external rewards through examination results and job promotions are very important to this (Perraton, 1988 unpublished). Some Regional Barriers to Implementation Africa. Infrastructure in sub-Saharan Africa is much more limited than in other parts of the world, especially for electronic media such as television, audiocassette players and telephones. However, it appears to be less of an impediment for simple technologies such as print and radio. In their survey of institutions in six African countries including Ethiopia and Malawi, Murphy and Curran (1988) found that postal delivery was of adequate quality and extent to serve most countries, although the cost of postage for heavy, printed materials was found to be high. Radio signal coverage is also fairly widespread, although the tendency in Africa is for governments to exercise rigid control over all the broadcast media, and it can be difficult to obtain broadcast time for a newly developed distance learning institution. (Friend 1989) What researchers have found to be a far greater restriction on the use of media by many African distance education institutions is that they lack the funds to pay to use existing infrastructure. Many secondary level and teacher training distance learning institutions cannot obtain or maintain printing equipment, buy paper (imported), or provide radios for the students who do not have access to them. Likewise, many correspondence courses cannot distribute course materials as required. P. Murphy (1988) found that costs for printed material used as the basis of teaching by the Zambian National Correspondence College had risen by a factor of 14 in seven years, compared to a doubling of teachers salaries during the same period. While professional and planming problems also must be dealt with, such as the absence of science and technical subjects, low status relative to conventional programs, poor results from study groups, the lack of mass adult, non-formal education, and administrative and methodological problems, the major hurdle which functional programs in Africa must face is the lack of foreign exchange and ongoing support for recurrent operating expenses (Dodds and Inquai, in Jenkins ed. 1988). Asia. While the expansion of the large public distance universities in Asia has been impressive, the speed of the process has been damaging to the effectiveness of some of the resulting distance education programs. The urgency appears to stem from the intense political pressure created by a large unanswered demand for higher education. Reddy (ADB 1987) cites the lack of time allocated for the planning process in setting up large Asian distance universities as a major problem encountered in the region. A second major barrier is the lack of training provided for those who will administer and teach in the new institutions. In several cases, such as in Pakistan, insufficient funds were provided by the government to cover the substantial start up costs required to design the programs, prepare the courseware, and pay for distribution of materials. In some cases, no attention has been given to the external context of what will happen to the - 136 - Educational Technology: Chapter 3 students when they graduate, student services have been inadequate, and much needed technology courses have been shortchanged in favor of the easier to offer humanities courses. The Middle East. One explanation for the lack of distance learning in the Middle East and North Africa may be related to the fact that oil wealth has made it possible for many countries in the region to build sufficient conventional educational facilities to answer their needs. Without substantial unserved demand for education, distance education is often not felt to be necessary. In Pakistan, conditions are quite different: AIOU was created to answer unmet educational demand, and has had to overcome a number of difficulties since its introduction, particularly due to shortage of funds to carry out design and program implementation. Siddiqui ( ADB 1987) observes that the extreme low levels of literacy in Pakistan (around 28%) limit the reach of printed correspondence teaching, and the widely varied levels of AIOU's clientele make it difficult to establish uniform academic standards and to design radio and television broadcasts that can be relevant for sufficient numbers of students. Latin America. Institutional survival for educational technology projects is difficult in Latin America, especially for formal education. Tiffin (1980) and Anderson (in Daniel et al ed. 1982) report that many distance education schemes have come and gone as initial enthusiasm and support have faded away time and time again; proyectismo is a regional problem. Strong links with related institutions such as the Catholic Church and other universities have helped reduce costs, and contributed to long term institutional survival in a number of instances. Sanguinetti emphasizes that RADECO could not operate efficiently, or even at all, without the collaboration of local community organizations which supply the administrative structure for proctors (study group leaders) to be recruited, organized, and supervised. In another example, a distance teacher training campaign in Chile in 1979-80 made use of state-run television, the Chilean Teachers' Association, the University of Chile's nationwide testing system, and other existing facilities, and kept its costs low by avoiding expenditure on infrastructure (Chadwick, 1982). Programs are more likely to succeed when they answer a real need as perceived by the students and their parents. Arena (1988) lists a number of factors that have promoted the survival and growth of the Telesecundaria in Mexico. They deserve attention for what they reveal about the possibilities to overcome barriers to implementation of all distance education programs: ... it [Telesecundaria] answers social demand that would be difficult to satisfy otherwise, its effectiveness is comparable to equivalent traditional institutions, the cost per class or group is less than in traditional institutions, provision of the same services in the remote areas involved by traditional means would effectively double costs, and the economies of scale attained have (1) reduced labor component of cost, and (2) reduced television component of cost. - 137 - Educational Technology: Chapter 3 VI. Conclusion and Recommendations Conclusion In concluding this chapter, it should first be mentioned that generalization is difficult for a field as broad as distance education, and is made even more problematic when it must be done for a group of countries as heterogeneous as those that have been described. However, at the risk of introducing some inaccuracy relative to specific cases, a number of general points can be made about applications of technology to learning at a distance in developing countries: 1. Distance education is expanding rapidly in most regions as an educational option. Developing countries use disted to complement existing educational structures, not to replace them. 2. Technology has a fundamental role to play in distance education, but in many cases, it is not being used up to its potential 3. Technology planning and budgeting can sometimes overshadow educational and curricular concerns, especially where more advanced technologies are concerned. Technology applications are more successful when they function as a support to pre-existing, clearly defined educational objectives 4. Accurate technology costing is an extremely important input into the longer decision-making process of choosing a media mix for distance education. However, the decision is ultimately most affected by political, academic, structural, administrative and cultural factors 5. An mix of technology that is appropriate for each course is important, but the quality of the courseware is most strongly related to the factors of time, academic talent, resources and evaluation that are put into content and curriculum design 6. The effectiveness of the media also depends on adequate student support via prompt feedback on tests and other evaluation, and related student services 7. Media technology will be ineffective if a minimum level of financial and planning resources are not made available for the planning phase of a distance education institution; adequate provision must also be made for recurring costs during the operation of the system. - 138 - Educational Technology: Chapter 3 Recommendations 1. The question of quality should be a fundamental concern in distance education. In those cases where this has been given priority, particularly for the distance universities such as the British Open University, the Chinese Radio and Television University, the Costa Rican UNED, and Thailand's STOU, the resulting institutions are now recognized as benchmark achievements, and have had great influence in improving distAnce education in other countries. This is due not only to the experience they can offer for planners, but because the high standards that they have set have helped free distance education from the marginalized status it has suffered in the past. It appears that a strategy of concentrating resources on a few high quality institutions will be more productive than one of spreading them out over a larger number of lower quality ones, especially in the case of higher education. 2. There is a need for more cost effectiveness studies according to the model proposed by Unesco, (see Orivel, 1987) focusing on those regions most ill-served by existing research, that is, Asia and the Middle East. In the case of Asia, the need for research to understand and direct the explosion of implementation of distance universities is general: Educational quality, finance, external efficiency, equity/access, curriculum production, administrative systems, student profiles, student support, motivation, drop out rates, achievement, and occupational mobility (via post-graduate tracer studies), are a few of many possible topics that need attention. In the case of the Middle East, interest in the area remains undefined, and could benefit from some input from the Bank in terms of proposals and speculative analysis. 3. In Africa, the greatest need is for minimum levels of resources to enable existing distance education programs to operate up to their potential, and to allow desperately needed expansion in the future. 4. In Latin America, research on the question of institutional longevity could be productive, particularly focusing on the question of how projects are financed. 5. Overall, planners and specialists in the area of education at NGOs should consider support for: - greater international co-operation in the field, by promoting information exchanges, international meetings, courseware exchange and supporting entities that prouLote these same goals. - funding of pilot projects and implementation of inexpensive computer equipment to aid in administration of records and logistics in large distance education institutions. - 139 - Educational Technology: Chapter 3 - investigation of the external eff3cts posed by successful distance education programs. Student output must match employment needs and opportunities, at the risk of widespread frustration and wastage of humar. capital. 6. For the long run, major development lending institutions should consider support for the investigation of the role that satellites, computers, and telecommunications can realistically play in distance education in the future, to compensate for the decline of support for this important field from other organizations. - 140 - Educational Technology: Chapter 3 References Adesola, A.O. 1988. "The University of Lagos, COSIT." Paper presented at World Bank Seminar on Educational Technology, Washington, D.C. Altbach, P.G. 1983. "Key Issues of Textbook Provision in the Third World." ProsRects 13, no.3: 316. Arena, E. 1988. "Mexican Telesecundaria System: A Cost Analysis." Paper presented at World Bank Seminar on Educational Technology. Washington, D.C. Arger, G. 1987. "Promise and Reality: A Critical Analysis of Literature on Distance Education in the Third World." Journal of Distance Education 2, no.1 (Spring): 41-58. ADB (Asia Development Bank). 1987. Distance Education in Asia and the Pacific. Vols. 1 and 2. Manila: ADB. Bates, A.W. 1988. "Media in Distance Education." In J. Jenkins, ed., Commonwealth CooReration in ORen Learning: Background Papers. London: The Commonwealth Secretariat. Bates, A.W. 1982. "Options for Delivery Media." In H. Perraton, ed., Alternative Routes to Formal Education: Distance Teaching for School Eguivalenc_. Baltimore, Md.: Johns Hopkins University Press. Bates, A.W. 1980. The Planning and Management of Audio-Visual Media in Distance Learning Institutions. Paris: UNESCO. Bates, A.W. 1984. Broadcasting in Ed'zcation. London: Constable. Bates, A.W. 1986. "Satellite Technology: Some Indications of Costs." The Commonwealth Secretariat. London. Processed. Blume, W., and P. Schneller, eds. 1984. Toward International Tele-education. Boulder, Colo.: Westview Press. Chadwick, C. 1982. "An Overview of Educational Technology in *,atin America." ProsRects 12, no.3: 347-355. Chale, E.M. 1983. "Teaching and Training in Tanzanis. Doctoral dissertation, University of London. Chaya-Ngam, I. 1987. "Distance Education in Thailand." In Asia Development Bank, Distance Education in Asia and the Pacific. Vol.2. Manila: ADB. Cumming, C.E., and F.A. Olalaku. 1988. "Costs and Effectiveness of Open University Learning: A Case Study of the Correspondence and Open Studies Institute at the University of Lagos, Nigeria." Processed. Curran, C. and P. Murphy. 1988. "Accessible Technologies for Distance Education in Six African Countries." Processed. - 141 - Educational Technology: Chapter 3 Daniel, J. 1987. "World Trends in Higher Distance Education and Opportunities for International Cooperation." Laurentian University. Sudbury, Ontario. Processed. Daniel, J., M. Stroud, and J. Thompson, eds. 1982. Learning at a Distance: A World Persgective. Edmonton: Athabasca University/International Consortium on Computers in Education. Dhanarajan G., and I. Mugridge. 1988. "A Case Study: The OLI-DISTED Project." Open Learning Institute. Richmond, British Columbia. Processed. Dodds, T. 1982. "The Mauritius College of the Air" In H. Perraton, ed., Alternative Routes to Formal Educatirna: Distance Teaching for School Ecuivalency. Baltimore, Md.: Johns h.opkins University Press. Dodds, T., and S. Inquai. 1988. "Distance Education in Africa." In J. Jenkins, ed., Commonwealth Cooperation in Open Learning: Background Papers. London: The Commonwealth Secretariat. Eicher, J.C., et al. 1982. The Economics of New Educational Media. Vol.3. Paris: UNESCO. Escotet, M. 1980. "La educacion superior a distancia en Latinoamerica: Mito y realidad de una innovacion." Revista de Teenologia Educativa 6, nos.3 and 4: 239-250. Friend, J., B. Searle, and P. Suppes, eds. 1980. Radio Mathematics in Nicaragua. Stanford, Calif.: Stanford University. Garcia Aretio, L. 1987. "Los costes en las universidades a distancia" Paper presented at VI Congreso Mundial de Educacion Comparada, Rio de Janeiro. Gaudamuz Sandoval, L. 1988. "Estudio de Coso-eficacidad en un sistema de educacion a distancia--Caso: UNED - Costa Rica." Paper presented at World Bank Seminar on Educational Technology. Washington, D.C. Goldschmidt, D. 1984. "Economic Issues in Satellite-Based Educational Projects." In W. Blume and P. Schneller, eds., Toward International Tele-education. Boulder, Colo.: Westview Press. Hakemulder, J., ed. 1979. Distance Education for Development. Proceedings of DSE/UN-ECA Seminar. Addis Ababa. Halliwell, J. 1987. "Is Distance Education by Radio Outdated?" British Journal of Educational Technology 1, no.18 (January): 5-15. Haughey M., et al. 1988. "Non-Tradicional Delivery of Distance Education in China: A Field Experiment." University of British Columbia. Vancouver. Processed. Hawkridge, D. 1987. General Operational Review of Distance Education. EDT Discussion Paper No. 68. World Bank. Washington, D.C. - 142 - Educational Technology: Chapter 3 Hawkridge, D., and B. McCormick. 1983. "China's Television Universities." British Journal of Educational Technology 3, no.14: 160-173. Hawkridge, D. and J. Robinson. 1982. Organizing Educational Broadcasting. London: Croom Helm. Hawkridge, D., at al. 1982. "In-service Teacher Education in Kenya." In H. Perrraton, ed., Alternative Routes to Formal Education: Distance Teaching for School Equivalency. Baltimore, Md.: Johns Hopkins University Press. Jamison, D., and E. McAnany. 1978. Radio for Education and Development. Beverly Hills, Calif.: Sage. Jamison, D., and F. Orivel. 1982. "The Cost-Effectiveness of Distance Teaching for School Equivalency." In H. Perraton, ed., Alternative Routes to Formal Education: Distance Teaching for School Equivalency. Baltimore, Md.: Johns Hopkins University Press. Jamison. D., S. Klees, and S. Wells. 1978. The Costs of Educational Media. Beverly Hills, Calif. Sage. Jenkins, J., ed. 1988. Commonwealth Cooperation in Open Learning: Background PaDers. London: The Commonwealth Secretariat. Kinyanjui, P. 1974. Trai-ing Teachers by CorresRondence. International Extension College. Kwon, S., and K. Chandong. 1987. "Distance Education in the Republic of Korea." In Asia Development Bank, Distance Education in Asia and the Pacific. Vol.2. Manila: ADB. Laflin, M., ed. 1985. "Communications Support Project: The University of the South P-^cific: A Course of Action." The University of tee South Pacific. Processed. Lalor, G. 1986. "Report on The University of the West Indies Distance Teaching Experiment." University of the West Indies. Kingston, Jamaica. Processed. McAnany, E., et al. 1983. "Distance Education: Evaluating New Approaches in Education for Developing Countries." Evaluation in Zducation 6: 289- 376. McAnany, E., and G. Nettleton. 1987. "Le satellite national bresilien." Revue Tiers-Monde 28, no.111: 651-658. McCormick, R. 1985. "Students' Views on Study at the Radio and Television Universities in China: An Investigation in One Local Center." BrLtish Journal of Educational Technology 2, no.16: 84-101. Moore, G. 1987. "The Introduction of Asynchronous Electronic Communication for Distance Education in Developing Countries: A Case Study in Thailand." - 143 - Educational Technology: Chapter 3 Mullik, S.P. 1987. "Distance Education in India." In Asia Development Bank, Distance Education in Asia and the Pacific. Vol.2. Manila: ADB. Muniz Aquino, G. 1988. "Analisis del costo-efectividad de un proyecto de educacion a distancia en la Republica Dominicana: El caso de CENAPEC." Processed. Murphy, P. 1988. "National Correspondence Course of Zambia." Processed. Murphy, R. 1988. "Evaluation of Al Manaahil." Educational Testing Service. Princeton, N.J. Processed. Nielson, D. 1988. "Distance Learning in Indonesia." Harvard University, Project BRIDGES. Cambridge, Mass. Processed. Oliveira, J. 1985. Universidade aberta: Uma alternative de ensino superior Brasilia: Associacao dos Bibliotecarios do Distrito Federal. Oliveira, J., and F. Orivel. Forthcoming. "Training Teachers at a Distance: The Case of Logos II in Brazil." In N. Stromquist, ed., Adoption and Institutionalization of Educational Innovation. Oliveira, J., and F. Orivel. 1981. Cost-Effectiveness Analysis of an In- Service Teacher Training System: Logos II in Brazil. World Bank (Population and Human Resources Division) Discussion Paper No. 81-40. World Bank. Washington, D.C. Orivel, F. 1979. "Economic Analysis of Distance Teaching in Higher Education.' Processed. Orivel, F. 1982. "Preparation of a Proposed World Bank Education Project in China." World Bank. Washington, D.C. Processed. Orivel, F. 1987. "Costs and Effectiveness of Distance Teaching Systems: A Methodological Approach." IREDU, Universite de Bourgogne. Dijon, France. Processed. Perraton, H., ed. 1982. Alternative Routes to Formal Education: Distance Teaching for School Equivalency. Baltimore, Md.: Johns Hopkins University Press. Perraton, H. 1982. The Cost of Distance Education. Cambridge: International Extension College. Perraton, H., ed. 1986. Distance Education: An Economic and Educational Assessment of Its Potential for Africa. EDT Discussion Paper No.43. World Bank. Washington, D.C. Perraton, H. 1987. "Theories, Generalization and Practice in Distance Education." Open LearninZ 2, no.3 (November): 3-12. Perry, Lord Walter. 1984. The State of Distance Learning Worldwide. Milton Keynes: ICDL of the United Nations University. - 144 - Educational Technology: Chapter 3 Perry, Lord Walter, and G. Rumble. 1987. A Short Guide to Distance Education Cambridge: The International Extension College. Rajan, M. S. 1987. "Satellite Applications in Distance Education Through TV and Radio." In Asia Development Bank, Distance Education in Asia and the Pacific. Vol.2. Manila: ADB. Rumble, G. 1988. "The Costs and Costing of Distance/Open Education." In J. Jenkins, ed., Commonwealth Cooperation in ORen Education: Background PaRers. London: The Commonwealth Secretariat. Rumble, G. 1986. The Planning and Management of Distance Education. London: Croon Helm. Rumble, G. 1981. "The Cost Analysis of Distance Teaching: Costa Rica's Universidad Estatal a Distancia." Higher Education 10: 375-401. Rumble, G., and G. Borden. 1983. "Meeting Educational Needs in Costa Rica: The Role of the Distance Teaching Universidad Estatal a Distancia." International Review of Education 29: 427-448. Rumble, G., and K Harry, eds. 1982. The Distance Teaching Universities. New York: at. Martin's Press. Sakamoto, T. 1987. "Hardware and Software in Distance Education." In Asia Development Bank, Distance Education in Asia and the Pacific. Vol.2. Manila: ADB. Sanguinetty, J. 1985. "The Replication of the RADECO Project in the Dominican Republic: Recurrent Costs Implications." InterAmerica Research Associates. Washington, D.C. Processed. Satyanaryana, P. 1988. "Distance Education in India." In J. Jenkins, ed., Commonwealth CooReration in ORen Education: Background PaRers. London: The Commonwealth Secretariat. Schramm, W., L.M. Nelson, and M.T. Betham. 1981. Bold Experiment: The Story of Educational Television in American Samoa. Stanford, Calif.: Stanford University Press. Selim, M. 1987. "Distance Education in Asia and the Pacific." In Asia Development Bank, Distance Education in Asia and the Pacific. Manila: ADB. Setijadi, Prof. 1987. "Distance Education in Indonesia." In Asia Development Bank, Distance Education in Asia and the Pacific. Manila: ADB. Sewart, D., D. Keegan, and B. Holmberg. 1983. Distance Education: International PersRectives. New York: St. Martin's Press. - 145 - Educational Technology: Chapter 3 Sharma, N. 1988. "Technologies to Improve In-Class Ir truction." Paper presented at World Bank Seminar on Educational Technology. Washington, D.C. Siddiqui, S.A. 1987. "Distance Education in Pakistan. In Asia Development Bank, Distance Education in Asia and the Pacific. Vol.2. Manila: ADB. Smith, R. 1987. "Growth and Scope of Distance Learning." In Asia Development Bank, Distance Education in Asia and the Pacific. Vol.2. Manila: ADB. Spain, P., D. Jamison, and E. McAnany, eds. 1977. Radio for Education and Developmapit: Case Studies. Vol.l. Staff Working Paper No.266. World Bank. Washington, D.C. Sparkes, J.J. 1983. "On Choosing Teaching Methods to Match Educational Aims." In D. Sewart, D. Keegan, and B. Holmberg, eds., Distancg Education: International PerpRectives. New York: St. Martin's Press. Srisa-an, W. 1987. "Financing and Cost-Effectiveness of Distance Education." In Asia Development Bank, Distance Education in Asia and the Pacific. Vol.2. Manila: ADB. Stahmer, A., and I. Bryan. 1988. "From Books to Satellites: A Review of Technologies for Distance Education." Paper presented at World Bank Seminar on Educational Technology. Washington, D.C. Stone, J. 1987. "School Televi"ion: An Analysis of the Maranha-o and Ceara Projects." Processed. Stone, J. 1986. "A produtividade da televisa-o educativa: o caso da TVE do Maranha-o." ABT. Processed. Suppes, P., B. Searle, and J. Friend, eds. 1978. The Radio Mathematics Project: Nicaragua 1976-1977. Stanford, Calif.: Stanford University. Taylor, D.C. 198_. "The Cost Effectiveness of Teacher Upgrading by Distance Teaching in Southern Africa." International Journal of Educational Develotpmen 3, no.l: 19-31. Taylor, L.C. 1979. "Schooling with Television in Rural Areas: The Case of Portugal's Telescola." In J. Hakemulder, ed., Distance Education for DeeloRMent. Proceedings of the DSE/UN-ECA Seminar, Addis Ababa. Thomas, R.M., and V.N. Kobayashi. 1987. Educational Technology: Its Creation. Development and Cross-Cultural Transfer. New York: Pergamon. Tlene, D., and S. Futagami. 1987. Educational Media in Retrospect. EDT Discussion Paper No.58. World Bank. Washington, D.C. UNED (Madrid). 1983. Evaluation of Higher Distance Education Results. Proceedings of the International Congress of Open and Distance Teaching Universities, Madrid. - 146 - Educational Technology: Chapter 3 UNESCO. 1977. The Economics of New Educational Media. Vol. 1. Paris: UNESCO. UNESCO. 1980. The Economics of New Educational Media. Vol. 2. Paris: UNESCO. Wolff, L., and S. Futagami. 1982. "The Malawi Correspondence College. In H. Perraton, ed., Alternative Routes to Formal Education: Distance Teaching for School EquivalencI. Baltimore, Md.: Johns Hopkins University Press. World Bank. 1981. "The Educational Use of Mass Media." Staff Working Paper No. 491. World Bank. Washington, D.C. Young, M., at al. 1980. Distance Teaching for the Third World: The Lion and the Clockwork Mouse. London: Routledge and Kegan Paul. - 147 - Educational Technology: Chapter 4 Chapter 4 Uses and Costs of Educational Technology for Vocational Training Current Research in Canada and the United States, with Implications for Developing Countries Anna Stahmer Audrey Mehler Ingrid Bryan Murray Richmond - 148 - Educational Technology: Chapter 4 Chapter 4: Uses and Costs of Educational Technology for Vocational Trainlng Anna Stahmer, Audrey Mehler, Ingrid Bryan and Murray Richmond 1. Introduction Much of the discussion about the potential of educational technology to improve education and training over the last thirty years has focused on the use of technologies for general education. Considerably less s^rutiny has been given to the uses of technologies for occupationally-specitic vocational training, even in industrialized countries. For example, disconcertingly, a discussion of future trends in the area of vocational education put out by the National Association of Trade and Technical Schools in ... mentions neither the use of technologies for training, nor the innovation of self-paced learning. In developing countries, very few attempts to introduce technology into vocational technical training have been made at all, and research results are correspondingly sparse. This chapter addresses that lack by exploring the current uses of educational technology for vocational training in Canada and the United States, with the expectation that this experience may prove valuable in planning similar programs in developing countries. For most countries, having a workforce which is properly trained to operate and maintain industrial equipment and manufacturing processes is a major concern and essential for economic productivity. Yet, investment in human resources is often given low priority by governments and by international lending agencies. As noted by the West German agency CEDEFOP in 1978, until investment in human resources is seen as being of equal, if not greater, importance than investment in machinery, there will be constant problems with mis-matches between job qualifications and job opportunities. Traditionally, the vocational sector has lagged behind general secondary and tertiary education in the use of educational technology. One reason was that vocational training requires a fair amount of practical and hands-on skill exercises which were diZficult to present using the then-existing technology options. This obstacle is no longer necessarily valid; in the industrialized countries, vocational training institutions, as well as industry, businesses and service organizations are increasingly making use of innovative technological systems to present learning materials. Although the task of developing human resources is complicated by rapid changes in industrial and commercial processes that result from the constant introduction of new technologies, many of these same new technolog'ies can, under certain circumstances, be applied to the training process itself. This can be done both to enhance learning effectiveness and to reduce the costs of vocational training. State-of-the-art developments in computing and interactive videodiscs now allow simulation of hands-on training, a capability which will be of particular interest to vocational trainers in the future. And, as noted by Levin and Rumberger, it is also likely that many jobs in the - 149 - Educational Technology: Chapter 4 future will require modest skills which might be more efficiently acquired by technology-supported training. Scope This chapter is designed for government planners and others involved in the design of investment policies and strategies for vocational training in developing countries, as they respond to the challenge of meeting the demands of rapid technological change. The chapter reviews current uses of new technologies for training in North America, where a large number of implementations have already been attempted, for lack of examples in this very new field from developing countries themselves. Although it is obvious that the implementation context differs greatly between North and South, a careful analysis of the resulting data can yield a number of insights that should be of value to decisionmakers in any country. Because planners in developing countries often express strong interest in the potential of new educational technologies for vocational training, attention was given to factors which might be most relevant if adaptation of these educational tools is to be made to the Third World context. In particular, these factors include: (a) barriers to the use of educational techonology, such as institutional frameworks, software development, and high capital costs, and (b) incentives for its use, such as improved quality of learning, and potential savings from reduced learning time. In the North American cases that were studied, we looked at two different types of institutional settings for vocational training. One was the formal educational system, in which technician training and education takes place at the post-secondary level in junior or community colleges. The other setting chosen was the private sector, where industrial firms operate their own in-house, on-the-job training and upgrading programs for their employees. Alt-. igh vocational training covers a wide range of subject areas, from industry to commerce to health and social services, this study emphasized vocational training for the industrial sector and for supporting infrastructure sectors such as transportation and communications. It also concentrated on the use of modern educational technologies such as computers, videodiscs and videotapes. These technologies were looked at both individually, and in combination with other media. I, Broadcasting is touched upon only briefly because its use for vocational training is limited. (The UK Open College and the Singapore Productivity Board ara among those few institutions which do use broadcasting for vocational training.) For similar reasons, video- and audio-conferencing systems receive little attention in our study. Further, we de not include innovations which relate to the improved production of print materials for use in traditional classroom learning formats. We have no doubt though, that print materials will remain an important component within vocational training and that new processes in the development and production of print materials will make them increasingly easy to revise and flexible to use. I/ See Appendix A for Glossary that gives definitions of unfamiliar terms used in the chapter. - 150 - Educational Technology: Chapter 4 The data gathering and analysis were guided by the following questions about applications of educational technology for vocational technical training: (1) What technologies are being used, why and for what kinds of learning tasks? (2) What framework of institutional support do the technologies require? (This includes such questions as: What is the role of instructors, what competencies are required to create instructional software and what barriers exist to full integration of new technologies into training delivery.) (3) What kind of success has been achieved in reduced training time, improved learning quality and according to other subjective assessments? (4) What costs have been incurred, including both capital and recurrent costs? In addressing these questions, the report primarily draws on ten case studies done by the authors on the use of training technology in four North American corporations (Ontario Hydro, Bell Canada, American Airlines, and General Motors Canada), and in six junior colleges (Northeast Metro Technical Institute, Seneca College of Applied Arts and Technology, Southern Alberta Institute of Technology, Northern Alberta Institute of Technology, the College of San Mateo, and the Oregon Partnership Education Program). The case studies were made based on field visits and interviews structured around a standard checklist. 2/ In addition, the chapter was augmented by a review of the literature and by telephone interviews with administrators, trainers and managers in other industrial firms and colleges. Researchers, hardware manufacturers, and courseware providers were also contacted. The ten institutions that were included in the study are in no way meant to serve as a scientifically representative sample of all enterprises and colleges. Rather, we chose institutions that use technologies of the widest possible variety in order to have the opportunity to examine all of the possible analytical questions which might support or inhibit the sustained institutional use of educational technology in vocational technical training. In addition, an economic model was developed based on data from the ten case studies that allows some rough comparisons to be made about cost efficiency and the economic viability of several technology options in different implementation contexts. Details on this model and its results can be found in Appendix C. V Summaries of these ten cases can be found in Appendix B, along with detailed information about the specific findings on technology applications. - 151 - Educational Technology: Chapter 4 II. Which Technologv for Which Learning Task? As mentioned above, the introduction in recent years of new information technologies in industry and business has not only created a need for training in the use of these new tools, but has also made possible the introduction of new pedagogic training techniques and methods. In particular, computers, videodiscs and videotapes now permit independent learning in many situations where, before, only classroom-based teaching was thought to be effective. While the use of such technology-based, self-paced training programs is increasing, the majority of vocational training programs still employ a combination of print and electronic technologies to support classroom-based teaching strategies. Analysis of both the case studies and existing literature reveals that a large variety of types of technologies are used to support training for very similar instructional tasks. This suggests that decisions to pick a particular technology for a teaching task are primarily based on cost and managerial considerations. It is also apparent that large enterprises are quicker to adopt technology-based training than small enterprises or public education institutions, because bigger firms are better able to shoulder the high start-up costs of the new techniques, and are in a better position to benefit economically from them. Uses of Technologies -- What the Research Shows Corporations in North America (and in other industrialized countries) have recognized that their workforce needs to be retrained to function more efficiently in a world dominated by electronics. While some corporations have been quicker than others to adapt new technologies to meet their training objectives, a 1986 survey of 2,500 U.S. businesses cited in Training magazine indicated that 80.5% of the firms used videotapes for instruction; 81.5% used classroom lectures; 27.6% used non-computerized independent study; 23.9% used video teleconferencing; 10.8% used audio teleconferencing, and 3.4% used computer conferencing. Computer-Based Training is becoming mc. and more common for on-site industrial training. Back in 1982, another business survey by Training magazine showed that 18.3% of respondees already used computers ror instructional purposes. Electronics, equipment repair and maintenance, and systems operations are subject areas in which CBT is often used to provide simulated operations and fault insertion. Classroom training is still used to provide introductory and/or theoretical information as a basis for more "hands-on" training. Videotapes and prinF materials are also used to augment material presented in class and via the computer. Videoconferencing is gaining wide acceptance among US businesses, many of which are beginning to produce their own videos in-house. Videodiscs have been found to be a stimulating and effective training technology for equipment operation, installation, and repair, sales training and customer service training. A recent survey released by the California market research firm S K & A Research, and reported in Viewdata magazine estimates that 75,000 interactive videodisc systeiDs will be sold in the U.S. in 1988; 80% of these - 152 - Educational Technology: Chapter 4 systems will be used for training in the manufacturing, medical, financial, retail, and government sectors. As videodiscs become more and more popular with large companies, the product catalogues for "generic" disks are expanding in areas such as electrical maintenance, algebra and ac/dc theory. A fair percentage of these programs appear to be targeted primarily for "blue collar" audiences working in industrial facilities, such as power plants, pulp and paper mills, utility companies, pharmaceutical manufacturers, and aerospace firms. Train-the-trainer discs are available for training managers, maintenance managers and operations managers. Users include Bethlehem Steel, Hughes Aircraft, General Dynamics, and Borg-Warner Automotive. All of these companies have large number of trainees. In colleges, a different range of technologies and combinations of technologies are being used to support classroom learning, as well as an increasing number of self-paced training programs. Audiocassettes are used -o guide students through learning materials and exercises. Computer-managed learning systems are used to administer student tests, grade them, provide immediate feedback on assignments, and to keep student records. Interactive video and videodiscs are being used in training for the health care fields, electronics, and basic mathematics. In Stahmer's 1986 survey of distance education in Canada, many of the educational institutions that were contacted reported using audio teleconferencing to deliver general courseware to centers and to on-site, in-service training programs. However print continues to be the major component of most vocational training programs. Print-based student workbooks, study guides, operations manuals, and textbooks are usually combined with classroom instruction, and sometimes with computer-based training, interactive videotapes or videodiscs. Although computer-based training is less common at the collega level than in private industry, CBT is used in several instances in colleges to teach machining, drafting, word processing and accounting. Some vocotional colleges offer self-paced training programs without any classroom instruction at all. These institutions use predominantly print-based learning packages, with videotapes as supplements. Students are required to do hands-on work with captive equipment in a lab and under the supervision of an instructor. Most of the private training institutions contacted offer conventional classroom-based training in trades and technology. Print materials such as textbooks and study guides, filmstrips, audio tapes and/or videotapes are used to enhance the classroom presentation. Computers are mostly used for teaching computer skills. For the ten case studies, Charts 1 and 2 in Appendix B depict the wide range of teaching technologies these institutions have chosen, as well as the t=aining topics for which they are being used. Print materials are used in all ten of the institutions surveyed for nearly all of their course modules. All courses make use of tutors or face-to-face contact with instructors to support the learning process and to assist the trainees. Computer programs are also used by all of the institutions, although for far fewer of the courses. Film and videotapes are used to supplement classroom instruction, and for some self-paced study programs. Captive equipment is used in most courses, although the far more expensive micro- and full-scope simulations are used only in industry, not in public education institutions. Audiocassettes are used by some organizations, primarily in public education. In one case, an organization is pioneering the use of interactive video tape - 153 - Educational Technology: Chapter 4 (i.e. video tapes played on a VCR, but controlled by a microcomputer). Video disc technology can be found in most of the institutions, but in all cases, it is used for a very limited number of course areas. The course areas in which each kind of technology is used are diverse. For industry, the selection depends on the types of skills required for the jobs in the firm in question, and generally, these learning tasks are very focused and specific. Computer programs in this sector support a wide range of specific skills training tasks such as flight panel operations or electricity meter reading. Even with this emphasis on specific job skills, interactive videodisc is being used for broader subject areas, such as management training and for general information on the company's background, in two of the three industrial applications of this particular technology that we looked at. The broader range of subject matters taught in public education institutions means that these institutions use technologies for more purposes, and in much more generalized course areas. Basic courses such as mathematics, algebra, or computer literacy, and general skills courses such as drafting and blue print reading, power engineering or emergency health care are all taught in vocational colleges with various combinations of educational technologies. Interactive videodiscs tend to support fairly specialized courses, such as fire fighting, nursing or surveying as well as basic mathematics and basic electronics. Examination of the case studies shows the following basic trends in applications of educational technology for vocational technical training: - educational technologies are generally used in combinations with each other; - more advanced educational technologie:j are usually employed along side of more basic ones, such as audiocassettes; - print is ever-present, and face-to-face contacts with tutors or instructors is a typical feature; - the range of subjects that can be taught with each kind of media is large and varied, ranging from specific job skills to basic skills; - a corollary to this is that the range of media that can be used to teach a particular subject is also large and varied. The evidence shows that most things can be taught equally well using many different combinations of media; - use of computer-managed learning to combine a number of educational technologies for different learning tasks appears to be increasing; - 154 - Educational Technology: Chapter 4 - courseware for different learning tasks and different technologies is now available from a growing number of sources. A survey of the relevant literature on applications of educational technology for vocational technical education shows that current trends in the applications of technology for vocational training are consistant with the conclusions drawn from the case studies. Charts 3 and 4 in Appendix B present a "snapshot" of major uses of technologies in training. The information for these tables was gleaned from a survey of relevant literature in the field of the last two to three years. Analysis of these data shows that, consistent with the case study investigation, a wide variety of educational technologies are often used for very similar purposes. All appear to be reasonably successful in practice. Choosing the Right Technology for the Task The case studies and other existing research show that combinations of media teach better than any single medium. This is true for all kinds of learning, but in particular, for teaching psychomotor skills in vocational training. For example, we have not found evidence that educational technology can fully replace hands-on practice, but we have found many assertions that interactive technologies can better prepare trainees for the use of actual equipment and reduce the hands-on training time. The case studies and literature also show that different combinations of technologies can effectively teach very similar subjects. Even the small sampling of the ten case studies shows that electronics is being taught with different combinations of the following technologies: computer-based programs, video, hands-on equipment, microcomputer-based simulations and print -- and soon with videodisc programs. Thus, the choice of appropriate educational media is not predetermined by a direct or exclusive link between an educational medium and the module/subject matter to be learned. At the same time, however, there is a certain relationship between types of training modules and learning tasks, i.e., print materials do not lend themselves to teaching psychomotor skills. Another important factor in the choice of a given technology is the relative difficulty of producing the learning materials. Videotapes and audiocassettes are probably the easiest to produce and use of all the new training technologies. The availability and affordability of audio and video production skills, expertise and equipment has led many corporate and educational institutions to produce their own training meterials in-house. In addition to serving industrial or institutional markets, audio- and videotapes are also used extensively by general audiences whe wish to upgrade their skills in vocational areas. Videodiscs and computers, on the other hand, require expensive, specialized expertise to create and manufacture courseware/software, and utilization is thus slowed where resources for this purpose are not readily available. Why Use Technologies for Vocational Training? A variety of reasons can be found in the literature that argue in favor of the use of educational technologies. In general, many of these - 155 - Educational Technology: Chapter 4 advantages only materialize when the technologies are used in conjunction with flexible learning schedules, such as self-paced learning, so that the use of technologies should be looked at as being part of a larger process of rethinking the entire learning process. The incentives to use technology that motivate both private sector firms and public VTE institutions can be grouped into several categories. Different benefits have been found to accrue to the organizational needs of the training system, to the staff of instructors, to the learning process itself, and to the students. (1) Organizational benefits include improved efficiency, greater scheduling flexibility, improved adaptability to a changing technological environment, and cost reduction under certain specific circumstances. Efficiency: o Intensifies utilization of training facilities by moving away from classroom or academic year schedules; o Serves more students than conventional systems; o Reduces training time for individual courses; typically, training time can be reduced by 30% or more when computer based training is involved; o When repeated over time, a program can serve a cumulatively large audience, even though it only serves individuals or small groups at any specific instant. Schedule flexibility: o Allows training to occur whenever convenient for the trainee; o Fills training spots as they open, i.e., due to drop-outs or completion; o Allows job-site training, even in remote locations. Cost reduction: o Can reduce expenditures on training supplies and equipment, where substitution and simulation allow it; computer based simulation systems are significantly less costly than captive equipment or full-scope simulation systems; they take less time to develop and implement, and represent real savings in certain situations; o Can provide the opportunity to develop courseware which can be sold to outside customers and bring in revenue. - 156 - Educational Technology: Chapter 4 Imnroved adaptability: o Allows quicker entry into a new growth/ product area; o Facilitates massive, short-term training efforts for the installation, operation, and maintenance of large scale, new investments; o Allows timely training in subject areas that suffer from high staff turn-over; o Helps a program to keep up with technology developments; the working environment is changing to technology/computer-based procedures in many cases, and the workforce needs to be ready. (2) Benefits that aid the instructors tend to cluster around the fact that it is difficult to recruit sufficient numbers of qualified teaching staff for many kinds of vocational training. In this context, educational technologies and self-paced learning schemes can supplement lack of experience, as well as lack of staff. o Requires fewer instructors for a given number of students; o Improves teaching quality where instructors lack experience; o Fills in the gaps caused by instructor job hopping; o Helps upgrade out-of date instructors' knowledge; o Stimulates the personal interest of instructors in new technology. (3) Utilization of educational technology can also directly improve the learning process because it offers certain pedagogical advantages, and because it may be the vehicle for the introduction of high quality courseware. In general, the evidence shows that educational technology can offer superior instructional quality when compared to conventional instructional methods, given that it is used in a well-planned implementation context. o Allows the demonstration of details of procedures to large groups, through video close-ups; o Where low literacy is a problem, lessens relative importance of words in the learning process through video images; o Allows for safe training in dangerous, impractical or excessively costly procedures through simulation; o Offers a uniform and infinitely repeatable learning experience, giving better control over the learning process; - 157 - Educational Technology: Chapter 4 o Creative courseware can stimulate interest and motivation of low achieving students; o Provides reliable and standardized testing and student performance evaluation; o The learning style of self-paced, technology-supported training more closely approximates real working conditions; it requires active participation and initiative on the part of the trainee, and technology-based simulations allow for practice of operating skills. (4) Students can benefit from the use of educational technologies in the training process because of unique pedagogic design features, as well as from the organizational flexibility that results from self-paced learning programs. O Interactive technologies are self-adjusting, according to the level and ability of each learner; this allows for remedial learning as easily as for accelerated completion; o Students have flexible access to training in terms of both time and place; o Lost wages are reduced through part-time training and shortened training periods; o Students get faster, more specific feedback, increasing their motivation and learning retention; o Increases access, opportunities and equity of training; Given this general survey of advantages offered by technology to the learning process, it must be remembered that the rationales for the use of educational technology differ in emphasis between the private sector and public v3cational technical training institutions. This is a reflection of the very different economic and financial contexts in which these two types of training take place. For large corporations and industrv, cost reductions and time savings are generally the two most important factors arguing for use of technology for training purposes. The other is the special situation in which the high cost of mistakes caused by poor training must be avoided, no matter what the cost. Where employees operate huge, complex equipment such as a nation's power grid, or a nuclear power plant, the cost of a mistake in economic and or human terms is so high that very expensive training methods can be justified if they are also effective. The case studies offer the following generalizations about incentives for use of educational technology in the private sector. Reduced training time allows trainees to function in a particular Job as seon as the skills are mastered. Thus less overtime or replacement costs (while the employee is off the job for training purposes) accrue. This reduces extra-pay and overtime payments for in-service training. In areas where a large number of people - 158 - Educational Technology: Chapter 4 have 1:o be trained, these savings can be significant. Trainee travel and per diem costs can be reduced because most of the training can take place during slack hours at the place of work. At the same time, instructor travel and per diem costs can be reduced because classroom based courses given by specialists to small groups of people are avoided. Corporations also find that individualized training is less disruptive to work schedules, because it does not take whole groups of trainees out of work at once. Finally, large enterprises find that pre- prepared learning equipment and/or software may be offered as a part of the purchase package from the manufacturer where imported, complex capital goods require extensive training to operate. The situation for Rublic education institutions is somewhat different, due to cost and institutional limitations. It is not unusual for vocational colleges to adopt new educational technologies simply because external funds become available for that purpose or because equipment and software are offered for free by manufacturers. In the case of Southern Alberta Institute of Technology and Norther Alberta Institute of Technology, this kind of situation turned out to be beneficial to the institutions' training programs, as the technology that was donated for specific training programs was successful. In some instances, however, it has been observed that public education institutions have introduced educational technologies in order not to "be left behind", rather than because they offer a specific educational advantage. Those institutions in the case studies which have implemented self- paced learning systems based on technology report favorable results similar to the general benefits already listed above. They can handle more students per instructor; learning time is reduced, often significantly; hardware utilization is consistent enough to justify investment; and computer managed learning facilitates better control of the trainees's learning process and is particularly suitable for educationally disadvantaged learners. In certain cases, educational technologies may allow for reduced investment in facilities because training can take place at work or in small learning centers, rather than large centralized buildings. III. The Institutional Context We have found that individualized self-paced learning, combined with an open entry and exit administrative structure is crit:.cally important to the effective and efficient use of educational technologies in vocational technical training. It is not just the physical tools themselves, but the improved efficiency of the entire training process which they permit that generates the often-cited significant savings in training time and costs. Such restructuring of the training process requires an entirely new approach to the organization of the learning process in time and in space; for example continuous year-round utilization of the educational technology system means that traditional academic schedules must be abandoned, while self-paced programs may not even require a classroom. - 159 - Educational Technology: Chapter 4 Industrial Applications vs. Colleges Industry generally has more flexibility to implement such systems than do public education institutions, and it is apparent that large companies are more likely to use technology-based tralning systems than small enterprises or colleges. The four corporations in the case studies are very large corporations which find it worthwhile to set up their own training programs, an option rarely available to small firms. Small firms usually must rely on colleges or equipment suppliers for their training needs. Large firms can benefit more from the cost-saving characteristics of such systems and achieve directly observable productivity gains. They can afford the high front-end development and hardware costs that these systems entail and they have the necessarily large throughput of trainees required to make the investments in equipment and software cost effective. Industrial training, retraining and upgrading are all targeted to satisfy a specific set of training needs resulting from internal requirements, such as the use of new technologies on the production line, or providing customer maintenance support for a given product or service. Industrial trainers have a great deal of autonomy to decide on which method of training they need, based on these objectives and on the relative cost-effectiveness of the different methods. Further support for relatively high expenditures on training technology in industry stems from the fact that inadequate training can cause a direct loss in productivity or revenue, as is evident from the case studies. In industry, a very important objective in using technology-based training is to "get it right", almost at any cost. For example, if a power generation plant malfunctions even for a few days, the result will be the loss of hundreds of thousands of dollars. So for an electric company, it might be cost-effective in the long-run to use an expensive but highly effective tecmrology, such as interactive videodiscs, to train a small number of people in the techniques of heating up industrial boilers, to avoid the much higher costs of a breakdown. In constrast, no educational institution could afford such high costs per trainee; in public education, reduced cost is the key factor in any decision to implement an educational technology. This does not mean that the advantages of increased learning speed and improved learning quality would not accrue from introducing educational technologies in public education institutions. But these benefits are often over-shadowed by the constraints posed by inetitutional structures. The objectives of public vocational training are less targeted than those of industry, and include remedial basic education. Colleges need different kinds of technologies and curricula than industry. And, the generally high costs of introduction of technologies in public education institutions require innovative adaptations in order to take advantage of the benefits of self-paced learning. Self-paced learning without an open entry/exit system is not likely to be effective; while trainees typically master skills in less time in self- paced learning programs, the traditional educational system does not easily allow them to move on to other training modules, or other levels of training. - 160 - Educational Technology: Chapter 4 Lacking an open entry/exit system, they have to wait for the slower trainees to finish or for the end of the training cycle. The combined use of both conventional and technology-based approaches within one institution is likely to be cost-effective only when a significant number of trainees and courses is involved or, at the opposite extreme, for a very specialized, small enrollment course. It appears that self-paced learning and open entry/exit scheduling are not easily grafted onto the operation of an existing conventional vocational training institution. They require administrative procedures, staffing arrangements and facilities management proceedures which differ significantly from those in conventional systems. Such programs are best developed autonomously, from the ground up. Changes in physical plant to accommodate the new systems and to allow their use to full potential are difficult to implement in the case of public education institutions. Self- paced learning does not require traditional classroom space, but does require access to different types of facilities and services, and the current trend to decentralize training to the home or place of work complicates the transition even more. Expenditures for educational technology can be substantial and a reasonable and consistent flow of trainees is necessary to make them cost effective. One of the colleges reports that with appropriate scheduling about 10 students can share the use of a computer workstation for their course module. However, if the college is unable to recruit enough students to keep the equipment operating at capacity, the cost benefits of the new technology may be lost. Moreover, self-paced open entry/exit systems can yield continuous and extended daily and year round utilization of the equipment and courseware. This means that special administrative arrangements are necessary, ranging from renegotiation of faculty contracts and reduced vacation periods, to providing extended service for longer opening hours. The capacity of computer managed learning to combine testing and record-keeping functions with student counseling and referrals makes it central to the effective implementation of self-paced and open entry/exit training. For example, the Power Engineering course offered by SAIT and NAIT uses CML for a self-paced, open entry/open exit learning program that is offered to students on-campus and at remote work-sites. In the future, a CML system could be used as a "nerve center" for self-paced, open entry/exit systems, using a range of different technologies within one vocational training program. Cooperative Agreements Between Schools and Industry Despite the difficulties described above, many vocational colleges have recognized the long-term importance of educational technologier', and have sought out ways to incorporate them into their teaching programs. One adaptive response has been for many colleges to turn to joint ventures with business and industry. Such co-operation ranges from information sharing to joint courseware production. The motivating force behind these co-operative arrangements appears to be that colleges need to gain access to funds to purchase technologies and - 161 - Educational Technology: Chapter 4 to put together course development teams. In general, availability of funds for new ventures is limited for most educational institutions. A high percentage of their budgets is allocated to pay instructors' salaries. The high front-end costs required to develop technology-based courses (both for equipment and for courseware design) simply cannot be met with existing resources; while staff training can usually be carried out using internal resources, major development of technology-based training programs must rely on external funds. These typically are available in three forms: (1) Government support, often under some form of research and development program; (2) Offers by manufacturers of special deals for equipment purchases or outright donations. (While some positive results can be achieved under this kind of arrangement, they often remain one of a kind. Institutions find it hard to pay for eventual replacement, because these costs are usually not included in the institution's operating budget.) (3) Courseware sales arrangements, whereby educational institutions develop and deliver courses for industrial and other clients. The courses are then also used to serve the student body of the institution. The last mentioned option has turned out to be the most successful, and both sides have found benefits in these arrangements. Industrial firms get access to the courses they need without having to carry the costs of major in-house training divisions. The education institutions in turn, can obtain funds for hardware and courseware development, which they can later use to serve the main student body. While it is not necessary that the joint college-industry courses use educational technologies, (instructors could just as easily be used to deliver the newly developed courses at the industrial sites), the flexibility that technologies offer often makes their use desirable. The portable nature of technologies such as interactive videodiscs, print materials and audio conferencing equipment is a great aid to the cooperative schemes, since the delivery of training appears to be moving to the workplace, and the use of educational technologies by colleges for such off-campus training appears to be growing rapidly. These co-operative ventures between industry and education institutions seem to be mutually beneficial. Moreover, they are possibly the only means by which public education will be able to afford access to advanced educational technologies, or to invest in the staff resources needed for solid courseware development. Only brand new institutions that have not already made fixed budget commitments to faculty salaries and facilities are in a position to establish new patterns of resource allocation that would be suitable for the development and use of educational technologies. Such new patterns of allocations would entail high up-front investments, decreasing utilization costs over time, and the expectation of future revenues from sales of courseware to private sector clients. - 162 - Educational Technology: Chapter 4 There are a number of examples in which colleges have joined with each other and with industry to purchase and/or develop courseware that each participant can utilize in their own setting. The Oregon Partnership Education Program exemplifies co-operation in courseware production. It brings educators and industry together to develop training materials that can benefit both sectors. Another example is the Greater Cincinnati Industrial Training Corporation, which is a partnership between business, government and educational groups to develop the instructional tools necessary to ensure the adequate training of present and future industrial workers. This local partnership program is being marketed nationally by the American Vocational Association. Some cooperative arrangements have a broader goal of promoting the use of educational technology in vocational training. For example, The Ontario Colleges of Applied Arts and Technology (CAATs) have undertaken a Computer-Based Learning Project, whose mission it is to co-ordinate the planning, development, dissemination and use of communication and information technologies in order to enhance educational opportunities in the CAATs and industry. Networking is another new trend in institutional cooperation. The Open Learning Agency in British Columbia has introduced a computer network linking colleges across the province, and a variety of computer conferences are planned to allow instructors to share information on computer-based training and computer-managed learning aids, and to encourage instructors to work together on curriculum development and professional development activities. On a larger scale, the National University Teleconference Network is a consortium of higher education institutions linked across North America to share non-credit educational and training programs delivered via satellite. Collaborations between business and education are encouraged, and participants include Hewlett-Packard, J.C.Penny and Oklahoma State University. Further examples of information-sharing can be found in the proliferation of databases available in North America. Professional and technical associations also provide networking among members and market courseware. To encourage the use of their products, and to allow market testing of new educational materials, many educational hardware manufacturers have participated in joint ventures with public institutions. "or example, AT&T has joined up with the government of Alberta, Alberta Government Telephone, ACCESS Alberta and Athabasca University to form the Canadian Distance Learning Development Center. The Center's objective is to develop new methods and tools for instructional design and delivery to better meet the training requirements of industry. Agra Education has undertaken a similar project in the area of agriculture. As a developer and supplier of computer-based training materials for agriculture education and industry, Agra is joining forces with Apple Computers, John Deere and Company and the Pioneer Seed Company to develop a computer-based training program designed for schools, farmers, and equipment manufacturers and retailers in North America and abroad. The software will encompass several educational levels and will include training material for trainers. Along similar lines, Control Data Corporation has worked with - 163 - Educational Technology: Chapter 4 colleges and large manufacturers, such as General Motors, to meet specific training objectives. The Implementation Process: Preparing for New Training Systems The introduction of advanced educational technologies such as computer technologies or interactive videodiscs has sparked a great deal of debate within public education institutions. Institutions have responded in various ways to the challenge. Many have instituted internal studies, special staff training programs and have made seed funds available for research projects. The most successful of these initiatives are those which institute firm long-range timetables governing the rate of introduction of technologies or those which allow educational technology systems to be set up as a relatively independent unit within the existing organization. For example, the Southern Alberta Institute of Technology plans to modularize all its courses by 1990. Adaptation of training and other activities to the new conditions are thus being carried out according to a clear schedule. In the case of Seneca, the Centre for Independent Learning operates quite separately from the main organization and has proven successful. In the case of San Mateo, the main impetus for change came from the opportunity to provide training to industries in subject areas that were also taught on-campus. The issues confronting industrial trainers as they introduce educational technology into their training program's are somewhat different in emphasis, and rest less on the institutional adaptations required, than on the adaptations to be made by the training staff. In particular, the new systems require the instructors to adopt completely new teaching skills and to take on a different role as a learning facilitator, not as a classroom leader. Instructors need to be prepared to guide students through self-paced and individualized materials. This issue is of concern to instructors and administrators alike. In some cases it presents a key obatacle to the introduction of educational technologies. Trainer trainir.g, team building, spec'al project funds and other strategies have been tried to ease the transition from conventional teaching to technology-based tra.ning. Some institutions are beginning to implement special training programs for trainers. Other adaptive arrangements can be made as well. For example, at Bell Canada, some trainers are seconded from line functions to do training for a defined period of time. These industrial trainers are thus not as strongly committed to conventional training practices as are their educational counterparts, and so may find it easier to adapt to new training methods. Courseware Development: Bringing Together Different Skills As a general rule, the more sophisticated the educational technology, the more sophisticated the courseware development process that it requires. At the simplest level, an individual instructor can easily put together a training course which uses audiocassettes, an existing text, and shop equipment on his or her own. However, computer-based programs require - 164 - Educational Technology: Chapter 4 different skills and talents, and at the very minimum, a subject matter expert and computer programmer will both be required. At the top of the scale, when video is added, or computer graphics are part of the instructional design, a number of additional skills and facilities are required and costs will mount steeply. Two main problems emerge in the area of courseware design: (1) Courseware authoring systems are still too complicated to allow content experts to easily develop courses on their own; (2) The need for a number of different skills in the courseware development process makes the maintenance of in-house teams an expensive proposition, unless a continuous load of courses is under development -- which is rarely the case. Some observers speculate that the need for specialization of different team members may decroase in the future. They predict that software systems will mature to allow content experts with minimal training to handle much of the development process. In the meantime, the case studies show how different institutions have designed solutions to these problems. For example, the Oregon Partnership Education Program brings together _he needed expertise on a temporary basis for each course as is required, from any one of the member institutions. Other institutions, such as NAIT, look to wrap-arounds for exLsting courseware as a means to achieve significant savings. As a part of their courseware development strategy, private s ctor entities often use course modules that are specially developed by production tool and equipment manufacturers and made available as a part of the purchase package when these capital goods are installed in the factory. The firm then adapts the training materials to its own specific requirements. This kind of arrangement offers the benefit of economies of scale, since the courseware can be used for training by all purchasers of a particular product. For example, it may be too costly for a company that uses a particular piece of machinery to develop an interactive videodisc module for equipment-specific maintenance trining, because of the limited number of trainees, the length of time it takes to develop courseware and the relatlvely fast changing technologies. However, the picture is very different for the manufacturer of that machinery; for them, the potential number of trainees is large, the courseware can be developed during the product development process, and lt can then be made available at the time of equipment purchase and delivery to many different customers. Barriers to the Implementation of Technologies Educational institutions and indu trias run into similar kinds of obstacles when implementing technology-based training, although it appears that in terms of strategic decision-making and budgetary allocations, educati-,nal institutions tend to encounter more of them. The obstacles are mainly related to organizational structures, to differences in mandate and, most importantly, to access to funds for technologies and courseware - 165 - Educational Technology: Chapter 4 development. This last factor is surely the most important; a large front- end investment is ,aearly always required to purchase equipment, train staff and develop or purchase courseware. Industry-based training programs are more successful in obtaining access to the requisite funds than the public education sector, because they are able to demonstrate observable gains in productivity to their firms. The benefits of educational techv.'logy to public education are much harder to quantify. The front-end costs of courseware can be reduced through the purchase of generic courseware packages. However, some training institutions and enterprises feel they need customized courseware, yet lack the time, expertise and the funds for courseware development. In addition, technological changes and training needs can outpace the in-house ability to develop courseware, and invalidt- generic packages shortly after purchase. In all sectors, the introduction and effective implementation of technologies in vocational training requires the commitment of staff and management alike. If this commitment is not forthcoming, or if doubts exist regarding the need for or effectiveness of the technology, its implementation can be hampered. If staff are resistant to change, relocation, or retraining, this too can pose problems to the process of implementing technology. Friction can arise when funds are redirected away from traditional training programmes to technology-based training. Instructors may feel that traditional teaching methods are best, and may not wish to devote extra time to professional development and retraining, reviewing new materials and developing new courseware. Many traditional teachers still believe that they will be replaced by the new technologies, and feel tivreatened by their new role as tutor/counsellor, as opposed to being a classroom instructor. In general, the same kinds of obstaclcs face both manufacturing firms and VTE institutions as they begin to implement educational technologies. As was true for the benefits, the obstacles can be grouped under four headings: organization, instructor/staff, courseware, and students. (1) Organizational barriers to implementation of technology: o Requires significant commitment and additional work for everyone; o Requires job re-alignments and other institutional changes; o Requires different and new types of staff skills; o Needs additional or different space and space configurations; o Where computers are to be used to train many people, many user terminals are needed; o Support from senior management must be strong to push through with potential changes in staffing and organization or budget realignments; - 166 - Educational Technology: Chapter 4 o Requires strong management awareness of the potential of technologies and of what it takes to make them work effectively; o Requires sufficient seed funds for staff training and experimental course development. (2) Staff/instructor barriers to implementation: o Staff may resist efforts to change traditional teaching methods; o Instructors need comprehensive and effective preparation for changing their role from lecturer to counsellor, through trainer training; o Changes may bring about a fear of loss of job security; (3) Courseware barriers to implementation: o Technology is characterized by high front-end costs; it can be difficult to obtain start-up funds, even if the ultimate result is a cost savings; o There is a long lead-time to develop good courseware; o Software and eourseware may be difficult for teachers or students to use; v Changing technologies in the workplace require frequent and possibly costly updates of course materials; (4) Barriers to students' use of technology: o Unfamiliarity with equipment; o Difficult operating instructions and procedures; o difficult and inconvenient access to technology o low learning motivation However, some obstacles are specific to the context of industry and business. The instructional staff may consist of trainers recruited from a line operational background; these people may be accustomed to teaching actual field practices, not to tutoring, managing and counselling trainees. They may be reluctant to take on courses/approaches developed externally, and may be inexperienced with working with a multi-skilled course development team. Courseware can also cause problems for industry. There is the question of suitability and adaptability of general courseware to specific corporate requirements. The courseware development process may be too slow to respond to real training needs, and due to rapidly changing technologies and procedures in the work place, it may simply not be cost effective to develop a complete, but expensive technology and training program. - 167 - Educational Technology: Chapter 4 Education Institutions face their own set of obstacles specific to their context, particularly in the organizational area. Where technology-based training is introduced within a basically conventional institution, friction is likely to occur between the new and the old systems. Decreasing public education budgets will lead to uncertainty, which fosters conservatism in planning for new educational alternatives. ourseware may be the source of complications for training institutions as weli. Courses that are well-adapted to use by individuals may have limitations when they must to be integrated with group activities. Also, depletion of departmental resources to create courses using the new technologies may threaten the quality of existing courses. IV. Effectiveness and Efficiency of Educational TechnQlogv ,tn Vocational Trainint Educational technologies have chalked up impressive successes in vocational training in terms of educational effectiveness. A review of the literature sliows that researchers have found that the quality of technology- based training appeaLs to be at least equivalent if not better than traditional training, even as training times are reduced. The considerable expertise applied to courseware development means that the very highest quality of training can be offered to a wider audience than could be reached if those same well-qualified teachers could only train in person. Under these circumstances, trainers hired by schools and indi- . -4d not have such high qualifications in specialized subject-matter knowledge. Efficiency improvements are also evidint. Training times have been reduced, the ratio of teachers per trainee has been reduced, and computer simulations have the potential to reduce the need for the use of costly captive systems. In addition, the flexibility in time and location of training that new technologies permit allows the reduction of external training costs and offers more "user friendly" training opportunities. Educational Effectiveness Discussions with training practitioners and a thorough review of the literature give much support to the theory that well-conceived uses of educational technologies can offer training of a quality equivalent to traditional training. For example, in those case studies where trade certification followed the training period, and where comparisons can be made between conventional teaching methods and technology-based individualized methods, equivalent pass rates appear to have been achieved. Course completion at the Centre for Independent Learning at Seneca is around 50%, which is comparable to rate of completion in conventional courses. One case study reports that interactive video combined with hands-on experience is thought to result in a 50% increase in retention of instructional material when compared to traditional lectures. Another indication that decisionmakers have confidence in the educational quality of the new tools is the fact that corporations are adopting educational technologies for training in areas which - 168 - Educational Technology: Chapter 4 are directly related to their own productivity, such as production and maintenance. Interactive educational technologies permit the trainee to explore the consequences of choosing different answers in response to a series of situation-based problems. This process seems to help students to build a model of the operation in their minds, and allows them to practice different procedures before having to apply what they have learned in real job situations. For example, a videodisc training session on management techniques might simulate a business meeting where the student could choose the tone and substance of his "input" into the discussion, and the videodisc would show what kind of responses each kind of input could be expected to bring from the other "people" in the simulation. Microcomputer-controlled training equipment is also used to create such "practice" opportunities through fault insertion and other simulation features. The capacity of educational technologies to cGnvey visual images of the actual work environment helps students to further prepare for actually carrying out a given set of tasks. For example, an ambulance technician may not encounter a "live" case of heart seizure during training. A video of such a case, with close-ups of significant signals and procedures, will give the trainee an advantage over learning from text, lectures and mock demonstrations. Educational technologies can also demonstrate procedures in close-up, review, and slow motion which might be hard to see in a conventional classroom. For example, dental procedures can best be observed by trainees with the help of visuals which replicate the actual steps, since viewing a live demonstration would not allow a large group to see all of the same details. Educational Efficiency The figures on reductions in training time which we obtained from analysis of the ten cases described in Appendix B are impressive. Overall, they relate as much to the introduction of self-paced learning, as to the use of technology. The following examples illustrate achieved savings in time: across the board, CBT reduces training time by about 30%; Bell Canada found that self-paced print-based training can reduce the time to complete a course by as much as 25%; and the Southern Alberta Institute of Technology has used CML to reduce the time it takes students to complete a course by 47% , as compared to the correspondence system previously used. Another important area where reductions in training time can be valuable are situations where substitut'ins can be made for expensive captive equipment, that is often required for fu..l scale, hands-on practice. Bell Canada considers the potential elimination of the use of captive training systems as one of their primary goals in consideration of the use of computer simulation modules. The main advantages offered by these substitutions are that they are usually far less costly (for example, a $2,000 microcomputer could simulate training exercises on a $150,000 lathe) and they are less subject to wear-and-tear and breakdowns caused by inexperienced users. In cases where new, large-scale installations of equipment and technology are being made, simulations can be used for training even during the early stages of the planning and construction phases of the project, and they allow staff - 169 - Educational Technology: Chapter 4 training to begin much earlier than would be the case with full scope simulation alone. However, we are not aware of any instances where computers will entirely replace full scope simulation. Improved utilization of training staff is another area where educational technology shows positive results. Across the board, the case studies report or imply that educational technologies, combined with self- paced learning, require fewer instructors to handle more trainees than do traditional methods. Although we found little hard data in this area, this perception is widespread in public educational systems as well as among industrial trainers. SAIT reports that their computer-managed course required 70% less staff time than a parallel correspondence course. The fact that trainers can handle more students than in conventional systems will be attractive in situations where the numbers of qualified trainers is limited or where they are costly. In addition, we have found repeated reference to the changing role played by trainers in technology-based training. Less content knowledge appears to be required and trainers act more as advisors, counsellors, motivators and facilitators. Where subject matter expertise is scarce, the combination of educational technologies with such tutors could be a valuable method for vocational training. However, these apparent savings in staff time and knowhow need to be seen in the light of total staff investment in the courseware development process. We do know that significant staff resources are typically required for the development of courseware for educational technologies. Thus, the reduced need for direct student contact (and hence reduced trainer wage costs) may not be significant in terms of total course costs, unless large numbers of students are involved. Flexibility in Terms of Time and Location Almost all of the applications of educational technologies reported in the case studies offer flexibility in terms of the time and location where the training is to take place. Flexibility of training in terms of time was seen to be important from three perspectives. It offers trainees and supervisors more options to schedule training sessions, which is particularly important for adult education and in-plant training. It allows trainees to work through exercises at their own pace, and this reportedly encourages educationally disadvantaged trainees to master the materials. And finally, it allows for better and more continuous use of the equipment and facilities, on a daily as well as a year-round basis. Many organizations are moving towards offering courses at the work station or at company learning centers, thus reducing external costs associated with travel, per diem etc, all of which add significantly to the training costs. In addition, training at the workplace can be integrated with work schedules and slack hours, giving line managers flexibility in scheduling individual training. By comparison, in order to make classroom teaching cost effective, a number of students (typically 20) have to be assembled in one place at one time, disrupting the work schedule. At the same time, such flexibility requires relatively sophisticated administrative mechanisms. This is particularly important in the area of - 170 - Educational Technology: Chapter 4 maintaining sufficiently intensive use of the expensive hardware to keep a program economical. Student access to equipment and facilities requires constant balancing between the need to maintain a reasonably high ratio of students to each piece of hardware, and the need to offer reasonable access during peak hours. This problem is more difficult for colleges than for enterprises. In a firm, this kind of scheduling is integrated with worktime schedules, and is thus more predictable. The need for access to expensive, specialized hardware in order to use courseware which has video or computer components appears to be an important factor for continued use of campus facilities, as opposed to being able to offer training courses to students at home, at a distance. Students are unlikely to be able or willing to purchase such equipment themselves, and a fairly high 'mtnimum number' of trainees therefore must be able to use the equipment purchased by a school to make the investment worthwhile. This can only occur at a centralized school facility. V. Costs of Using Educational Technologv in Vocational Training Given that technologies can be educationally effective, a principal implementation issue is the comparative costs of technology-based training versus conventional instruction. The case studies suggest that unit costs of technology-based training are sensitive to scale economies in utilization, with instructor and hardware costs being relatively fixed variables in cost equations. Given a large enough enrollment, the differences among production costs for courseware using different technologies can become relatively small, whent viewed on a per trainee basis, because of economies of scale. So, in large scale situations, the cost ratio of hardware per trainee is more likely to be the main factor that will determine the cost-effectiveness of a given educational technology because only a finite number of trainees can use a given piece of technology in self-paced open entry/exit systems. This cost will remain relatively constant as long as there is a constant student throughput, and only a drastic drop in hardware costs can bring about an improvement. In this analysis, the savings in student costs resulting from reduced training times and comparative costs of instructors become very important. Wher instructor costs for highly specialized and difficult modules are high, more costly technologies may be cost-effective when they &Icrease the time per use per individual student and when instruction is orgraized to permit high rates of hardware utilization. On the other hand, -nere instructor costs are comparatively low, as in basic skills, print and conventional instruction is likely to be more cost-effective. This strategy is well in keeping with the earlier finding that different technologies can be used to teach similar subject matters. - 171 - Educational Technology: Chapter 4 Cost Efficiency of Different Educational Technologies While it is easy to recognize from the preceding discussions that new educational technologies have obvious administrative advantages in certain vocational training situations, it is more difficult to come to conclusions about cost efficiency. The literature regarding the efficiency of vocational training is not very conclusive. In this study we were able to explore internal efficiency to a certain degree, but any information on external efficiency, such as comparisons of the productivity impact of conventional and technology-baase instruction, remained elusive. Costs taken on their own are somewhat easier to measure. As broken down by Metca.rf in a World Bank study in 1985, the cost of vocational training consists of the cost of learning materials, facilities and equipment, the cost of instructors, the fees paid to training institutions and any production loss incurred due to training. The use of learning technologies usually increases the cost of learning materials and equipment, but decreases the need for and the cost of full-time instructors. In addition, since training time is often shortened, production losses would also be reduced in commercial, industrial applications. The use of educational technologies means that the propox 'on of fixed costs will rise relative to variable costs, to the extent that capital is substituted for labor. This in turn, means that per unit costs of equipment will largely depend on the number of trainees enrolled in the program, making economies of scale an important consideration in their effective utilization. Expensive technology can become cost-efficient when used for a few, very heavily subscribed courses, or when used for a large number of courses, which indivicsually may serve only a few students. but which in total ensure heavy use of the equipment (the principal cost component). In addition to the number of students enrolled, savings in student replacement costs resulting from shorter training time have proved to be a critical economic factor to rendering expensive technology cost-effective. Costs as They Appear in the Case Studies & It was not possible to compare cost information across the case studies, since cost data were reported differently in different cases. For example, some used cost per hour of training, while others used cost per module of training as the basic costing unit. It was possible, however, to develop some useful economic models, based on what data we were able to obtain. In collecting cost information, we concentrated on the following headings: (1) cost of courseware or courseware development, (2) hardware, (3) trainee time and (4) instructor contact. This information was not available for all cases. I/ The costs are in $US dollars. Canadian dollars were converted using $0.80 for one dollar. - 172 - Educational Technology: Chapter 4 (1) Cost of Courseware The case studies report the following ratio between time spent on courseware development and resulting units for training: Table 1 Time Spent on Courseware Preparation Technology Ratio of 1 Hour of courseware: X Hours of DeveloRment Instructor-led 1 25 Print-led 1 : 30-50 Computer-led (no graphics) 1 : 50-100 Computer-led (graphics) 1 100-200 Computer-led (specialized) 1 300 From what we can tell, these ratios are within general practices. The development costs go up significantly with more complex technologies and specialized applications. However, several factors act to reduce costs. First, as their experience with technology-based courseware development increases, staff may be able to reduce the time that they require to prepare courseware by about 20%. Second, given large enough numbers of trainees, courseware unit costs for technologies will fall, and can even become negligible; this is less true with print when individual copies of materials are provided to all trainees, and not true at all for instructor-led cov.rses, where the number of instructors required varies directly with the number of students enrolled. Finally, according to the ten case studies, self-paced learning reduces training time -- in the case of print-led by 25% and with the other technologies from 30% up to 50%. Thus, increased "productivity" reduces the cost margin further. In other ce.ses, courseware development costs were reported in terms of total investment in a course or in terms of sales price. Prices for courseware on interactive videodisc ranged from $1,000 for generic courseware, such as principles of AC/DC, up to $200,000 for a company customized training package. The Oregon Partnership sells a videodisc-based course on Port Authority Management for around $18,000. Their interactive video module in electronics costs around $7,000. A four module electronics course developed by San Mateo College cost between $1.5 to $2 million to develop and sells for up to $30,000. A computer managed learning course in power engineering, combining CAI, video, slides, print and text items costs around $350,000 to develop. - 173 - Educational Technology: Chapter 4 Wrap-around courseware is used or discussed in several of the cases. Two of the enterprises make extensive use of courseware developed commercially or by equipment manufacturers, and have adapted these to their own training situations with apparent success. One of the colleges compares costs for a wrap-around course to those of a stand-alone course, both of which used CML and identical test items. The estimated costs are in the order of $13,000 and $150,000 respectively, with test items accounting for about $7,000 in each case. From a cost point of view, it is obvious why purchase of wrap-around courseware is an attractive option. Revisions of existing courseware are easily made in those systems which combine videotape with microcomputer capabilities. The Oregon Partnership estimates a cost of about $250 per module revision. In contrast, the cost of revising interactive videodiscs is significant because the discs have to be fully reprocessed. The process requires access to the same type nf facilities, equipment and expertise as was required for the creation of the original disc. Revision of print materials used to be labor intensive, but is becoming much easier with wordprocessing and desk-top publishing. The life-span of courseware depends on the pace of change in a given field. Basic courses such as algebra can be expected to remain useful for a long period, and durable courses such as this are available on all types of technologies at relatively low unit costs. Even generic videodiscs are available at prices in the range of $1,000. It is important to note that even when very low cost courseware exists, this does not necessarily make the use of videodisc attractive in these basic areas, because of the high and relatively constant costs of the hardware per trainee. In the final analysis, the unit cost of courseware is not the deciding economic factor for basic courses. In the economic model presented later in this section, we will see that the unit cost for hardware utilization and comparative i,structor costs are more important. Costs of Hardware Hardware systems can be broken down into two main components. One is the hardware which the trainee uses and the other is the hardware required to produce the courses. Costs of educational computer hardware for workstations range from just over $1,000 to $2,000. Basic video playback units used are around $750. Interactive educational videodisc units are around $10,000. Costs for centralized c.omputer capacities (VAX) which work with small distributed computers are quoted at about $200,000 with an additional $400 for each communications modem. These costs take on meaning when we consider the number of trainees who can usefully access a technology unit. In the case of NAIT, in a CML- based course, a continuous load of 10 students per technology unit was considered optimal with the workstations accessible at extended hours. In the case of General Motors, a continuous load of seven trainees is planned, since the course relies heavily on computer courseware. General Motors is planning to switch from computer to interactive videodisc training, increasing equipment costs about seven-fold, but we are not able to predict if the ratio of technology units per trainee will change. - 174 - Educational Technology: Chapter 4 Further, the relatively high cost of even small central computers, suitable to support the kinds of CML in use in our cases, requires intensive use to arrive at a favourable cost-per-trainee unit. At SAIT at the present, there is a total enrollment level of 2,000 students in CML courses, and the $100 cost of each student's sharc in the total central computer costs is not insignificant. These costs are expected to tumble once the institution has fully converted to modular training with CML back-up. Hardware costs incurred during course production were not available in a useful format. They are, however, reflected in the overall development costs and in the selling price of courses. It is generally accepted -hat interactive videodiscs require the most expensive production technology, because otherwise the unique educational quality and capacity of the disk would be wasted. Video production technology can also be relatively expensive, but more flexibility in expenditures is possible if a simple production style is adopted. The cost of computer hardware, software and authoring systems can vary enormously, but is generally relatively high. Similarly, the cost of print production technologies such as desk-top publishing vary enormously, but $10,C00 can buy a unit of reasonable capacity and quality. In addition, someone specially trained to operate it must be hired. Operations and Maintenance Costs Operations and maintenance costs were not a major concern in the cases. The technologies appear to be performing satisfactorily. We did find, however, that videotapes suffer wear-and-tear and need to be replaced occasionally. Staffing, power etc. appear to have been adequately budgeted for. An Illustrative Economic Model In order to pull the above information together, we found it useful to build an economic model that would compare the economic viability of three different combinations of technologies, used in conjunction with instructor- led teaching. The comparison is enriched by an examination of each combination under a number of different conditions. The model is designed to explore the economies of scale that would affect implementation decisions by looking to see how these are affected by varying costs of the different inputs, and is based on trends and data taken from the ten case studies. The model was set up in the following manner. A base case was proposed in which a 40-hour, one-semester vocational course was offered using only conventional, instructor-led teaching methods. The instructor was assumed to be paid at the typical Canadian rate ($3,000), and to be teaching a maximum class size of 30 students, and overhead for the course was put at $500. The different technology combinations used in the model were examined in terms of how they might affect costs as found in the base case. The three combinations tested were: (1) half the hours would be provided by interactive videotape, half by conventional methods, (2) one quarter of the hours by videotape, one quarter by self-paced print modules, and the remaining half by conventional methods, and (3) one half by the much more sophisticated - 175 - Educational Technology: Chapter 4 interactive videodisc technology, and the other half by conventional methods. A final variation (4) used data from variation (3), but included the factor of savings in student costs (meaning the costs in lost wages and productivity during the training period) that would result from the technology use in the final determination of viability. 1 The model shows how per student costs for each input item (Instructor Costs, Overhead, Courseware, Hardware, Savings in Student Costs) would vary across enrollment levels ranging from 30 students per class per semester to 5,000 per semester. The sum of these per student input costs is shown for the whole range of enrollment sizes. The model then compares these five figures with the per student cost of the conventional, base course. The required scale of utilization that would make a technology combination viable can then be determined. For purposes of the calculation, we used a factor of 125% of Instructor costs to identify a range of viability. (Please note that the parameters of the model cannot be calculated wlth accuracy and that it is inappropriate to identify exact viability or break-even points between instructor-only and technology-supported costs from this kind of model.) Thls same analysis was done for each of the three technology combinations, as well as the variations proposed, so that a total of nine comparisons were made with the per student cost of the base case of $117. The base case ltself varied ln only one instance, (lb), the only one in which reduced instructor salary was introduced as a variable. A summary of the results is shown in Table 2, and detailed information is found in Appendix C. / Further detailed description -if rhe assumptions used in the model can be found in footnotes 1 and 2 in Appendix C. - 176 - Table 2 Sunmary of Economic Model: The Effects of Enrollment Scale on Economic Viability for Various Combinations of Technologies Course- Students Bighpy Efficient Inatructor warc sardware Sharing Saving in Cnsider Above (5 50S) Above ? Case Cost Cost Cost Bardvare student Costa ? Students Students ls 1/2 video, balance Canadian level Medium Modest Low not included 120 W/i tutorial (custom (8) video) lb Reduce salary cost; 50S of Canadian Mediur Modest Low Not included 300 nfs otherwise some as l level (cuntom (8) video) 2 1/4 video, 1/4 Canadian level Medium Modest Low Not included 60 n/D self-paced, balance (custom (16) tutorial video) 3s 1/2 videodisc, balance Canadian level Low (off- Bigh LoW Not included n/a n/a tutorial; low cost shelf) (16) videodisc courzmewre 3b Sigb cost videodisc Canadian level Kigh Bigh LoW Not included n/a W/ coursaewrc; otherwise (custom (16) j ame as 3a videodisc) 3c Increase hardware Canadian lvol Bigh Nigh Bip lot included 360 /a V utilization; other- (custom (64) wise sam as 3b videodisc) C as Recognize savings in CanadiaD level Low (off- Bigh LOW Recognized 30 D/F student costs; shelf) (16) 0 otherwise sm- as 3s 4b High cost videodisc Canadian level High High Low Recognized 300 n/a hardwaro; otherwise (custom (16) am* as A videodisc) Ac Increase hardwre Canadian level Bigh Disb Bigh Recognized 210 330 utilization; other- (custom (64) F wise am as 4b videodisc) ° et Educational Technology: Chapter 4 Implications of the Model Our simple examples show that five factors are the most important in determining the viability of the new technologies: (1) instructors' salaries, (2) the number of students, (3) the intensity of hardware utilization, (4) the cost of hardware and (5) on savings in student replacement costs. The higher the salaries of the instructors are, the more viable the technology alternative becomes. In our model, cutting instructors' salaries by half raised the number of students required to make use of educational technologies economically efficient from 120 to 300. Therefore, if qualified instructors are available at low cost, sophisticated learning technologies probably do not make sense. However, in cases where instructors are avaiiable at low cost, but are also under-qualified, educational technologies may be justified in order to improve the quality of learning, and may actually cost less than other options for instructor training and upgrading, particularly in demanding technical and scientific fields, where expertise is likely to be the most expensive. The economic model al*o shows that the cost of hardware is crucial. In Case Example 2 of the model, the cost per student of using a combination of interactive video technology, print-based modules and an instructor approaches the per student cost of the base case for a semester course size of just 60 students, and begins to offer savings in cost per student for a course size of 240 or greater. By contrast, Case Example 3 demonstrates that no matter what the level of cost of courseware or enrollment size, the use of interactive videodisc workstations will not become viable when a low number of students (16) is using each piece of equipment. However, by assuming higher utiliza- tion of the hardware by more students, the picture changes significantly. A fourfold increase to 64 trainees per videodisc unit means that the technology becomes viable for a course enrollment of around 360 students. Intense levels if usage such as this reinforce the importance of self-paced open entry/exit systems in the economic viability of educational technology for vocational training. In order to achieve this level of intensity of equipment use in the public education concext, flexible admininstrative structures are acsolutely necessary. They allow quick replacement of drop-outs (which can be as high as 50% of the students) and extended opening hours, which make the figure of 64 trainees sharing a single piece of equipment for 20 hours of instruction per student throughout the semester a practicable option. When the factor of "student replacement costs" (the cost to the firm of paying for substitute labor while the trainee is absent) is included in the calculations, the results are quite different. The savings associated with reduced learning times is substantial, even at the relatively low Lwurly replacement cost used in Case Example 4 in the model. Under these circumstances, use of interactive videodisc systems changes from being inviable for any class size to being viable for a class of less than 30 students (assuming that low-cost, off the shelf courseware is used) or 300 students, when high-cost, custom designed courseware is used. In the latter case, increasing the intensity of utilization of the hardware can bring this number cown to 210 students. - 178 - Educational Technology: Chapter 4 The importance in the economic model of savings in student time needs further exploration in the context of developing and newly industrialized countries, as welb as in the context of public education systems and training in enterprises. For public education systems, savings in student time may not always be a relevant economic factor. In some cases, savings in student time may not even be relevant for enterprises. However, it is a fair assumption that most enterprises will experience reduced productivity or increased costs when they have released staff for training, and any reduced training time will result in savings. In addition, intensive training for project start-up or major re-training activities should benefit significantly from savings in student learning time. The models assume that qualified instructors are available. The argument changes if no such resources are available or if cost and time of training for instructors need to be considered. Other variables not included in the model are important as well. Reduced costs of training materials, the need to have trai..ed staff in place quickly, requirements for highly precise learning and exact replicability of skills and high overheads for travel or per diem in some training situations are other factors that will ultimately determine the feasibility of each kind of technology. VI. What Are the ImRlications of This Study for DeveloRing Countries? The reasons for the use of educational technologies are likely to be basically the same for both developed and developing countries; planners always want to reduce costs, increase learning quality, and increase educational efficiency. And as the pace of technological change continues to accelerate, developing countries need ways to get e:posure to the new technologies in the same way that the industrialized countries do, if they are to avoid being permanently margin-.-zed in the world economy. In both North and South, particularly in the non-commercial sphere, access to funds for high front-end development costs will be the biggest problem slowing implementation of new teaching technologies in vocational technical training. For many countries, substantial components of the training equipment may have to be imported, and foreign exchange shortages will aggravate the funding problem. Joint arrangements between governments and businesses, cooperative efforts among countries, and wrap-around courseware adapted to local needs, languages, and cultural contexts are possible routes to get around the barrier of high front-end costs. The fact that many different kinds of educational technologies can successfully teach similar subject areas is encouraging for developing countries, because this means that planners will have a great deal of flexibility to choose the most appropriate one for their needs. Decisions can be based on cost and managerial considerations, and on the level of existing technical infrastructures and maintenance support, as well as on the pedagogical character of the system. - 179 - Educational Technology: Chapter 4 Institutional Issues We have seen that many of the benefits of educational technologies depend greatly on the establishment of a flexible training environment. The establishment of self-paced open-entry/exit systems is new to many countries, and will require long-term planning and training; no program c"'. be successful without the informed input and commitment of local planners and administrators. The adoption of such systems is likely to be more easily achieved in newly industrialized countries than in less developed countries, given their greater access to human and financial resources. However, it is conceivable that in situations where no vocational training systems have been set up at all, the introduction of individualized learning methods could be aided by the lack of pre-existing institutional structures. We assume that enterprises and manufacturers in developing countries have as much flexibility to plan their training strategies as their counterparts do in North America, and that they will obtain comparable economic benefits. On the other hand, the public education system in most countries has to contend with structural rigidities that are at least as great, if not greater than, in Canada and the U.S. and it is likely that grafting new teaching systems onto existing institutions will be very difficult. It might be desirable for developing countri - to begin the process by building up the necessary administrative str cures first, before moving on to the introduction of advanced technologies for training. In the initial stages, technologies that are already in use worldwide, such as print modules and audiocassettes, would probably be more easy to introduce. Co-operative arrangements between industry and public education insLitutions for courseware development and for the delivery of training will work best in those countries that have well-developed, large scale industries where the need for in-house industrial training is recognized. A key reason for vocational colleges to seek out co-operative ventures is to gain access to funds to cover high front-end costs, and unless industry has these funds and recognizes the importance of training, co-operation will be unproductive. However, in cases where a sector, such as public utilities, receives funds for training through external arrangements or loans, public education institutions could propose joint training ventures with greater possibilities of success. This kind of co-operation makes more sense for basic skills than for specialized company-specific training, because the resultant courseware would have to be relevant as regular coursework in the training institution as well. This would not be unbeneficial to the companies however, since in many countries, industry finds that it has to train its workforce in basic skills before more advanced upgrading can begin. Co-operation might ultimately help upgrade teaching basic skills in the public education sector, thereby lessening this task for industry in the future. The relative portability of technology-based training also makes it an attractive add-on component for major procurements of imported machinery, allowing a route to low cost procurement of training materials for skill- specific learning that would be difficult to arrange under the kind of cooperative venture described above. As part of the purchase agreement, the supplier would be asked to provide technology-based training packages on the operating and maintenance procedures for each piece of equipment, and the - 180 - Educational Technology: Chapter 4 materials would then be adapted to local conditions. Such a process could be expected to contribute to improving the level and quallty of training at a minimal cost to the purchasing country. The literature reports that there is a tendency in some newly Industrialized countries for companies to hire graduates from traditional universities rather than from the lower prestige vocational training system. It may be possible that technology-based training can quickly provide these new employees with the requisite skills for their new technical jobs, although this solution would only encourage an already Inefficient situation. The Learning Experience The production of original courseware is another area where expenses can be very high, compllcating the adoptlon of educational technology in the developing world. In North America, courseware adaptation and wrap-arounds are achieving success both In terms of lowering costs and in terms of maintaining teaching quality. Not much experience has been reported with these techniques in develnping countries, but it would be surprising if such approaches did not work the same way. In particular, they would be useful where there is a common language of instruction, such as English In Commonwealth countries, Spanlsh in Latin AmerLca, or Arabic throughout the Middle East, and for technologies where export of important infrastructural equipment oc^.urs on a wide scale, such as for power or telephone companies. Where the local language of instru-tion serves only a small number of trainees, the situation may require additional support from different donors to serve these special needs. Because of the technical nature of vocational training, culturally inappropriate courseware and mis-matched subject matter would not be as big a problem as it has been In efforts to adapt other educational materials from one region to another. In many countries, the shortage of qualified instructors, or the problem of instructors lackiikg in up-to-date training in the subjects they teach is a considerable barrier to effective technical training. Educational technology can often be used to compensate for this problem, if it can be established in an appropriate administrative framework. Where qualified instructors exist, but in insufficient quantity, technologies can provide a multiplier effect. The qualified instructors can reach many more students if they design effective courseware for wider distribution vla print modules, audiocassetter, videotaped lectures, or other means, than they could in person. Where instructors need to be upgraded in difficult or highly technical subjects, the use of more advanced technologies might make sense, given the difficulties and high associated costs of providing such training by alternate means, usually via overseas studies. How do Costs Compare? Based on data from North America, the cost model described In the previous section showed that the cost of the Instructor, the cost of hardware, the number of trainees using each hardware unit, and the savings in student opportunity costs are critically important in determining efficiency of - 181 - Educational Technology: Chapter 4 educational technologies. Results will be quite different in developing countries, as the cost assumptions are adjusted to fit the non-industrialized context. Salaries are typically much lower than in North America and the cost of hardware is much higher. Educational technologies require supporting infrastructure that is often not readily available in less developed areas, and this also increases costs. A stable power supply, air conditioning, spare parts, and maintenance and repair services are among the factors which significantly influence costs and usefulness of different technologies. Very often, these costs are underestimated; a safe estimate is that they will run about three times higher in non-industrialized countries. Although adding in the factor of "savings in student replacement ists" in the economic model dramatically increased the theoretical economic viability of the use of educational technologies in the U.S. and Canada, when applying the same model to developin3 countries, many questions remain regarding how these savings should actually be weighed in the decision-making process, s'.nce worker salary levels are much lower in developing countries. For large enterprises and infrastructure projects, savings in student costs will still be a factor, particularly when upgrading of technical personnel is required; suitable replacements may simply not be available, at any cost, given the chronic shortage of this kind of skilled worker. The situat.on for smaller enterprises and public education is more complex, and when budgets are being drawn up, it may be hard to relate abstract opportunity costs of student training to real expenditures on equipment. Thus, while technology choices should take into account any savings in student replacement costs for business enterprises, for public education, the most important factors are likely to be the cost of hardware and intensity of its utilization. Choosing Educational Technologies The simpler technologies such as print msdules and audiocassettes have fewer infrastructure requirements and are already in general use in most countries. Given the equal success they appear to have in pedagogic terms when compared with many of the other, more advanced technologies, they may prove to be an appropriate choice in many situations. To obtain the greatest benefit from their introduction, it is important that they be linked with open entry/exit systems. This makes possible the quick replacement of drop-outs, a particular problem in public education in developing countries, as well as ensuring high utilization rates to compensate for the comparatively high cost of designing and installing the new learning systems. Educational technologies might be most effective for the project- .specific training that is usually associated with large scale investment for national development. Here, needs can be clearly established, and significant demand for high quality technical training exists. In economic terms, these projects have well defined economic and productivity output objectives, and the cost of training can more easily be justified, given that there exists the potential for large losses in revenue if poorly trained employees were to be the cause of equipment failure. This same reasoning has been sufficient justification for many large Narth American companies to turn to technologies for their training needs, both for first time learners and for refreshers. The same benefits might be expected to accrue to developing countries. - 182 - Educational Technology: Chapter 4 Another area where planners could turn their attention is to the application of computer technology to the management of the training process (CML) and to the simulation of occupational skills for hands-on learning. Improving the management of the training process spreads the benefits of the use of one machine to all of the students participating in the program, unlike direct teaching applications, such as videodisc players, that can only serve one or two students at a time. And, CML is a key element in setting up a successful self-paced, open entry/exit program, especially where administrative expertise may be in short supply. In the other instance, using computers for simulation of hands-on practice of technical skills would allow savings on the use of expensive captive equipment. For instance, $2,000 microcomputers could be used to simulate preparatory exerciLes on a $150,000 lathe, greatly reducing the time each trainee needed on the actual machine. And in cases where no captive equipment is available at all, simulations could be used as a substitute. In general, consideration of more expensive technologies for non- school training should be limited to situations where all of the following conditions exist: (1) highly technical subjects require complicated, skill-specific learning; (2) qualified instructors are scarce or not available, or available only at very high cost; (3) significant savings in student replacement costs are foreseen through shortened training and/or flexible on-site training schedules; (4) a minimum threshold of infrastructure exists, such as electricity, reliable communications and access to maintenance. For vocational technical training institutions, additional factors relevant to the choice of new technologies are: (1) a need for improved administration and learning mangagment to increase cost efficiency for an entire program; (2) excessive cost or difficult access to captive training equipment calls for use of computers for simulation or substitution. Conclusion In sum, any final recommendation will need to be situation-specific and will vary depending on the training objectives to be met. The following diagram outlines factors wnich need to be taken into account to develop a least cost strategy for the effective use of educational technologies. - 183 - Educational Technology: Chapter 4 Find least cost strategy to meet objectives, subject to constraints. Objectives Constraints Variables o Achieve training o Characteristics of o Cost of teacher time objectives existing system o Cost of student time o Allow self-paced learning o Availability of o Capital cost of (if/where possible) capital equipment/systems o Allow learning to take o Know-how to develop o Nature of the place at a variety of courseware materials to be locations; at different o Know-how to give/ learned (on a module times (if/where possible) supervise courses by module basis) o Achieve cost/time o Know-how to maintain o Number of students efficiency infrastructure o Number of locations By considering the issues examined, and by looking at the cases, it is possible to begin to develop a feeling as to what technologies may or may not be appropriate in various situations. However, it is important to be careful in attempting to draw sweeping conclusions about the usefulness of new educational technologies in vocational training. Yes, generalizations can be useful in many cases. However, the practical challenge facing the educational planner is to concentrate on specific situations, and to find the technology or technologies which deliver the best training for each individual case, under the constraints and variables which we have described in this chapter. VII. Recommendations Developing Least-cost Strategies Educational technologies can achieve educational effectiveness and efficiency as good as or better than conventional alternatives. Generalized answers as to the least cost options are not advisable, because training objectives, constraints and parameters differ widely across institutions, industries and countries. WE RECOMMEND that the economic model presented in section V of this chapter be developed further; in particular, it would be valuable to undertake a comparative analysis of the specific vocational training requirements for a business enterprise, a major investment project and a large public vocational school. Existing costs could be compared to scenarios based on the use of different combinations of educational technologies for training modules. Modularized Training Sophisticated interactive training technologies such as interactive videodisc or advanced computer-based training are used in specialized - 184 - Educational Technology: Chapter 4 situations in combination with other educational resources. The targeted use of sophisticated and costly educational technologies within the larger context of a full course of training appears to be an efficient option, because it allows the utilization of the technology across a large number of students for selected modules within the course which have a high economic value, (i.e., maintenance of an instrument panel). This particularly applies equally to industry and educational institutions. WE RECOMMEND that the training componenc of an upcoming major infrastructure project in a developing country (possibly in the telecommunications or power sector) be studied with the goal of developing a case study for modularized training using this kind of educational technology mix. Courseware Adaptation and Sharing Full courses or course modules have been developed by industry and educational institutions for a variety of subject areas and for a variety of educational technologies. Tried and tested modules and courses in electronics, for example, are available using different combinations of technologies. Technical training institutions would benefit economically from the opportunity to share relevant courses. Adaptation of the module to individual training situations might be achieved through wrap-around materials. WE RECOMMEND that studies be done in collaboration with vocational training clients to test several courses that teach the same material but which use different educational technologies. Evaluative research could provide valuable comparative data across technologies as to costs, effectiveness, institutional arrangements and long-term t.ining implications for further analysis. Courseware Development A strong trend exists among North American industry, business and educational. institutions to develop courseware jointly and to contract out part of the courseware production process, in order to reduce courseware production costs. WE RECOMMEND that ruch partnership arrangements be encouraged among developing countries for and that the involvement of the private business sector in this process be promoted wherever feasible. Training Technical Trainers Where the number and qualifications of vocational trainers are severely limited, long-term investment strategies can follow two paths. One way is to build on the apparent successes of individualized, self-paced training, arid commit the student learning process to educational technology on a large scale. To do so, investment must be targeted towards the establishment of appropriate institutional structures as well as the training - 185 - Educational Technology: Chapter 4 of instructional designers, programmers and technologists who can look after courseware development and hardware maintenance. Trainers need to be trained as counsellors/tutors. The second option is to use educational technology to improve training of instructors; they would then be available in greater numbers, and with better qualifications to teach in the existing conventional schools. WE RECOMMEND increased efforts to develop self-paced training systems. There is a need to examine the role of educational technologies within them and to explore their implications for instructor staff training. A series of well-publicized case studies and seminars would help to bring about greater awareness of these potentially attractive systems. Information Sharing Good, up-to-date information about available courses, modules, and technologies is critically important to planning efforts. A thorough knowledge of the effectiveness of each of the available options will reduce costs and avoid mistakes. In North America, specialized information bases are increasing. The British Commonwealth is undertaking a related activity. The International .elecommunication Union operates a network for standardization and exchange of courseware. The U.S. Department of Labor also sponsors courseware exchange. WE RECOMMEND that NGOs work with this growing network of international training institutions and other organizations to help set up a system of information exchange which server a wide range of users in the field. Training for Small/Medium Businesses and Institutions Smaller institutions and businesses may not have the necessary resources to justify financial investment in advanced technologies and courseware development, but they can, nonetheless, benefit significantly from technology-based training. Experiences with individualized and self-paced learning materials using print, audio-cassettes and interactive videotape are encouraging for this sector, sinc- the hardware costs are low enough that utilization is feasible for smaller numbers of trainees. WE RECOMMEND that research be done to explore the use of educational technologies from the lower end of the hardware cost spectrum for medium sized businesses and industry, particularly in the agricultural sector and for urban and regional development projects. This would require a combination of project analyses and field studies. Educational Technologies in Industry Informal, firm-based training has been shown to be more effective than formal classroom-based training, both in industry and in the education sector. In North America, industrial firm-based training appears to be leading the way for innovations in educational technologies. ThB benefits - 186 - Educational Technology: Chapter 4 offered by educational technologies, such as self-paced learning, reduced learning time, reduced demand for instructor contact, standardized quality, and the flexibility of on-site training could increase the performance of this sector even further. WE RECOMNEND that a series of studies and field activities be carried out to test these assertions, in particular in industries where the quality of training can be directly linked to economic performance, e.g. in power or telephone companies. Administrative Structures for Public Educat'on Educational technologies in public education, when used in conjunction with self-paced open entry/exit systems, can achieve significant economic and educational benefits. However, the development of administrative procedures for such systems is a complex task. WE RECOMMEND that work be done to encourage the development of administrative models for self-paced open entry/exit systems in public education. We further recommend that initizl efforts should concentrate on the use of simple technolog4es, so that full attention can be paid to administrative systems and procedures, rather than technical details. Computer Managed Learning The use of individualized, self-paced learning and modularized instruction based on different combinations of technologies is growing increasingly popular at the workplace and in vocational schools. The administrative support for testing, grading, and certification provided by Computer Managed Learning (CML) systems appears to be very important to the success of such initiatives. Software for CML is being developed through a variety of initiatives. Some software packages are broad-based and can handle full training courses, whereas others are designed to be specifically applied to learning modules or particular brands of equipment. WE RECOMMEND that assessment be done on the different approaches to CML development and their respective utility for a number of different training situations and countries. Field trials of a selected t:aining systems would usefully complement such a review. Broadcast Television As reported earlier, some countries such as Singapore and the United Kingdom are beginning to use broadcast television as part of their national skills training programs. The technology is already in place in most countries, it is familiar and can motivate learners. Television can reach large numbers of students at home, in the workplace, or in community learning centers. In combinetion with hands-on training, tutorials and print materials, television may be an effective learning tool for basic skills. - 187 - Educational Technology: Chapter 4 WE BECOMIEND that developments of the National Skills Productivity Board's project in Singapore be studied to assess the potential of television for vocational training. Pilot projects could be undertaken using television for vocational training in other countries where appropriate infrastructures are in place. - 188 - Educational Technology: Chapter 4 APPENDIX A Defining the Terms: A Brief Glossary Computer-Based Trainine (CBT), or Comouter-Assisted Instruction, (CAI), refers to the use of computers to provide information, drill and practice simulation, and testing. When used with a videotape or videodisc player, computers control the sequencing of video material according the needs of the learner, as determinied by his or her responses to computer-generated questions. Comouter-Managed Learning (CML), uses computers to provide testing, grading, student record-keeping, remedial assistance and counselling. These applications can be run on any size computer and in a stand-alone or network configuration. Computer-Controlled Trainina refers to training systems which use actual equipment interfaced with a microcomputer to allow the insertion of faults which the trainee will have to identify and rectify. Audiocassettes are used in self-instructional situations to guide a trainee through study guides and other materials. Random Access Audio describes an educational technology which combines audiocassettes (or compact discs or sound/slide technologies) with computer software such as CAI. Interactive Videodisc Training combines video images stored on laser discs with microcomputer capabilities to allow learners to work through a task at their own pace, responding to computer generated questions, with support from visual images including graphs and moving pictures. If a task is mastered, the student p)roceeds to the next segment; if not, the program branches into remedial material. The most salient characteristic of the learning process is that it allows actual interaction between the student and machine, unlike video tape systems, which require passive absorption of information. Film and Viaeo are well-known. Video has the advantage over film in that training materials can be produced less expensively. Video also allows for individual trainees to stop, rewind and replay a portion of the tape to reinforce understanding. Audio-. Video-. and Computer Conferencing allow for interaction and discussion among a number of locations. The first two are often used for continuing education where they are combined with follow-up activities on-site. With computer conferencing, it is not necessary for participants to log onto the system at the same time. Ca2tive Trair.ng Equipment refers to an actual piece of equipment which has been fully assigned to training. Full Scope Simulation is an exact scale model replica of an operations system, e.g., an airplane cockpit. - 189 - Educational Technology: Chapter 4 Wrap-around Courses are adaptations of purchased, pre-existing courseware, usually by creation of additional materials and test items. They are designed to save on course preparation costs, and are used in the great majority of commercial training plans. - 190 - Educational Technology: Chapter 4 APPENDIX B Case Study Summaries Enterprises: Case One -- Ontario Hydro The Simulator Services Group at Ontario Hydro develops and provides on-site, computer-based training for control technicians, operators, and engineering and technical staff of the nuclear generation division. Training is done in general theory areas such as reactor or boiler control, as well as on specific equipment. Captive equipment, computer simulations, audiotapes, films, videotapes and print manuals are used for training. Case Two -- Bell Canada Bell Canada's Network Services Training Department offers training for digital switch operations and maintenance, microwave transmission, telephone operators, electronic office services, engineering economics, and electrical theory. The majority of courses are instructor-led and delivered in a classroom setting. A number of courses are computer based, with graphics capacity used for simulation. Case Three -- American Airlines American Airlines offers technology-based training programs at the Flight Academy in Dallas, and at any one of seven regional training centres. The flight crew training program incorporates computer- based training for simulation and random-access slide-tape presentations on aircraft system operations and safety procedures. Print home study guides are included in the learning package. Training videotapes are available for individual viewing at each air base. For maintenance personnel, computer-based training is used to simulate flight panel operations. Videotapes, interactive videodisc, and classroom presentations with audiovisual aids are used for general training and fur new course material. Case Four -- General Motors Canada General Motors Canada offers training at the work-station to its employees using computer-based training programs. Remedial to advanced levels of instruction are available in the following subject areas: basic and industrial electronics, microprocessors, industrial hydraulics, pneumatics, robotics, programmable controllers, health and safety, and dealez training. The computer modules provide general instruction, simulation, overview, testing, student record- keeping, and referrals to other learning resources. For training on specific machines, classroom instruction is combined with equipment manuals and hands-on lab work using captive equipment. Videotapes are often used to augment classroom presentations and for independent study. Videodiscs are beginning to replace computer-based training modules. The training supervisor at each GM plant is responsible for co-ordinating and soheduling the plant's training program which - 191 - Educational Technology: Chapter 4 tyjp '.y consists of computer hard&are for five workstations, plus software, equipment manuals, workbooks, textbooks and vldeotapes. Case Five -- Nortaeast Notro Technical Institute The Northeast Metro Technical Institute in Little Bear Lake. Minnesota, runs entirely on a self-paced, competency-based training model. The Institute offers two-ye&r (or less) diploma programs in transportation, office occupations, cosmetology, dental, and technical areas. No formal classroom sessions are held. Students are provided with a print-based learning packet for each course module they undertake. Instructors are available for assistance and guidance, and are responsible for course development and management. The Institute operates under an open entry/open exit system. Videotapes and lab equipment are used as supplements to the learning packets. Computers are used in those programs where the student will be required to use a computer on the job. Case Slz -- Seneca College of Applied Arts and Technology The Seneca College of Applied Arts and Technology in Ontario has pioneered the use of independent, student-centered study l&bs using mixed media at its Centre for Independent Learning. The Centre for Independent Learning is a completely self-paced program, where students work with competency-based learning materials, independent of classroom inbtruction. Learning packages consist of a study guide and textbook, with supplementary videotape. Tutors are available daily and in the evenings for consultation. Areas of study include business, travel and tourism, computer studies, floral design, and liberal arts. Students can study at home or at the on-campus resource centre. An open-entry/open-exit policy allows students to begin and complete their programs at any time of the year. Case Saven -- Southern Alberta Institute of Technology The Southern Alberta Institute of Technology (SAIT) has developed and implemented a computer-managed learning system for student record- keeping, test-banking, electronic mail between tutor and student, and remedial assistance. The system currently serves about 2,000 students, and is used in conjunction with self-study programs combining print, video, slides and computer modules in the following areas: electronics, power engineering, mathematics, physics, and emergency medical training. The CML system is also used to administer and grade tests at remote work sites via telephone lines. SAIT employs interactive videodisc in its surveying and tourism courses; audiotapes are used in the health sciences, and film and videotapes are used extensively in medical sciences and in hospitality training. Case Right -- Northern Alberta Institute of Technology The Northern Alberta Institute of Technology (NAIT) integrates print, and video and slides (which are both produced in-house), with computer-assisted instruction and computer-managed learning. In the Electrician Apprenticeship Program, the traditional classroom is completely el-imnated. Students are required to complete 70 print- - 192 - Educational Technology: Chapter 4 based modules at their own pace, in any sequence. The CML system is used for testing students after each module. Tutors are available for assistance. NAIT has purchased videodiscs for use in studying haematology, cytology, paracytology, bacteriology, histology, geology, and meteorology. The discs serve as a visual, database to supplement classroom lectures and administer student assignments and exams. Case Nine -- College of San Mateo The College of San Mateo offers technical and vocational training on- causpus, as well as over-the-air via radio and television stations. The College develops and implements video-based courseware for industrial markets, distributed through private vendors. Self- paced, competency-based interactive videotape programs designea for use at the work-site are available in the following subject areas: electronics, microprocessors, blueprint reading, measuring instruments and algebra. The videotapes in these courses will soon be converted to videodisc, due to industrial market demand. Accompanying print materials include study guides and diagrams. The College provides instructors who make periodic field visits to business clients to monitor and advise on program implementation. Instructors are also available via a telephone "hotline" when urgent help is required. Case Ten - Oregon Partnership Education Program The Oregon Partnership Education Program (OPEP) is a co-operative program between government, educational institutions and private businesses, working on a cost-recovery basis. Participants in OPEP pool their resources and expertise to develop interactive video programs for any area of training. So far, module series have been produced in electronics and Port Authority Management. The electronics training series consists of eight modules, including videotapes, computer courseware, student workbooks, and an instructor's guide. The modules are designed to supplement classroom instruction, not to replace it. - 193 - Educational Technology: Chapter 4 Appendix C Economic Model Viability of New Technologies for V/T Training Based on Ten Case Studies - 194 - Educational Technology: Chapter 4 . 5.4.1 On Ca _ _ muzUU f -- (Sbalt) M SIUM a - 30 30 60 240 IOW SOOO am IMUIfL 040-IE utzWa 2a Ehm Cas-0/20 Izutzuotor $100 $ 50 $ 50 $ 50 $ 50 $ 50 O dethwA 17 17 3.7 17 17 17 Crummre - 200 100 25 6 1 Hardwe - 23 23 23 23 23 SAg in S nt - - - - - - 1U1M $117 S290 TWO 15 $ 91 A~oac z rayo ew (i.e. 1250) at Q 0 8l~~~~~~~.b~ t ayaJzt1. 2b 1rA smazy Rat*i *1/2" 4 IzutKuoto $ s0 $ 25 $ 25 $ 25 $ 25 $ 25 Overhad 17 17 17 17 17 17 eir..mre - 200 100 25 6 1 Hardware - 23 23 23 23 23 Svih Tn 'Sbx.ft - - - - TOM63L S 7 S265 $16 $ 70 1 66 .Wraches "Instrutw Oily" cm (i.e. 12%) at Q uus* apcdam.y. 2 "3/4" V1,; 01/4" SelnaLf-d PEint Insructor $100 $ 50 850 $50 $o $50 Ovezhead 17 17 17 17 17 17 zru..re - 145 73 18 4 1 Harare _ 11 11 11 11 11. Saving In Stdm - Se TUble 4 TO.RL Wll$7 $223 11 W96 $ 82 5$79 AW:rraah "Instructor Only"' Cam. (i.e. 125%) at fig students aPpz!02d,ately. - 195 - Educational Technolopy: Chapter 4 ~~~frSmse (Sbet) PER STrUI ;oCr1 - 30 30 60 240 1000 5000 CPE NUUNAL N40-H I r ExamP aU Ol_ 3a V22 Videodis, UKw ont Instructor $100 $ 50 $50 $ 50 $ 50 $ 50 Ouerhed 17 17 17 17 17 17 ~Jxs~iaze - 3 2 1 - - HardIare - 94 94 94 94 94 Savi In Student - See Table 4 costs ~TOM $117 $164 $163 $162 $161 $161 Does not approadh "Instructor Only" case (i.e. 125%) 3b /2" Videodisc; High cost InsFtructor $100 $ 50 $ 50 $ 50 $ 50 $ 50 overhead 17 17 17 17 17 17 ca2rsare - 666 333 83 20 4 Hardware 94 94 94 94 94 Saving In Student - See Table 4 cost TOMTL $117 $827 $494 $244 $181 $165 Does not approadh "Instructor Only" case (i.e. 125%) 3c ")/20 Videodisc, High cOt Ouinmre, Inease Videodisc Utilizaticn Instructor $100 $ 50 $ 50 $ 50 $ 50 $ 50 Overhead 17 17 17 17 17 17 Crtirseware - 666 333 83 20 4 Hardware - 23 23 23 23 23 Saving In Student - See Table 4 o,st 'JDTAL $117 $756 $423 $173 $110 $ 94 Approaches "Instructor Only" case (i.e. 125%) at 360 stude1ts. - 196 - Educational Technology: Chapter 4 BuUmrt for Senter (Stxuit) !ER S'UEENT COS - 30 30 60 240 1000 5000 can NMiUNAL "40-HIXR" It ±r F,EXPLEq a1M On,. 4a 01/2" Vi 1eodisc, 1w Cbs memmr; saving in fiy*n QCk3ts- Instructor $100 $ 50 $ 50 $ 50 $ 50 $ 50 Overhead 17 17 17 17 17 17 3zseware - 3 2 1 - - Ha~¶dware - .94 94 94 94 94 Saving In Sbtuent - (80) (80) (80) (80) (80) csts TOML $117 $ 84 $ 83 $82 $ 81 81 Approaches "Instructor Only" case (i.e. 125%) at less than 30 students. 4b ./2w Videodis, High Ct Ea; Savings S Sbln CX) Instructor $100 $ 50 $ 50 $ 50 $ 50 $ 50 Overhead 17 17 17 17 17 17 Qursearze - 666 333 83 20 4 Hardware 94 94 94 94 94 Saving in Student - (80) (80) (80) (80) (80) costs YAL $117 $747 $414 $164 $101 $ 85 Approaces "Instructor Only" case (i.e. 125%) at 300 students. 4c 03/2" Videodisc, High Cbt I- ~ ~ ~ ~ ~ ~~I so X $ L. c -206- CHART 3: Range of Technotogies: Educational Institutions I I I I I I I ~ ~~~~~~I lBroad-I I I I I I lEtectro- I IALdia IRandomllnter-I I I INicro Icast I ITutor lindust.lAudio lVideo lCotiputerlnic I Icasset-lAccesslactivelVideol IReat jSiouta- IRadio/i CICoucl-lor IConfer-IConfer- IConfer- IBtack- Sutbject Area ICoepterltes jAudio IVIdeo 10isk IvideojEquipmeltion ITV IPrintlior IOn-Sitelencing lencing lencing lboard TECHNICAL I I I I I I I II IIII PowierSystemi I I. M4aintenanceI I I I I I I II I I II (boiler system)I I I I I I I I I I I I I I I I Power System l I I I 1 1 1I I Steam PlantI I I I I 1 I I I I I I II Operators I I I I I I II I 1 Gas ProcessI I I I I I I I I I I I I II Controllers 1I II II I I II Oil Refinery-I I I I I II I II I I I Naintename. I ' l II Project I 1 I I I I I I I I Supervision I I I I II I I II I I IIr 0 Heavy Gas/airi Comcentration i' I I I I I II I II It Electricalt wiring 1* * I I 1 ElectrcianI * I I IIs . Instatlationl I I I 1. Telephone I I I I I I I I I II IIII System I I I I I II I I ' Technician j I I I I I I I I I II IIII Telepone I I I I I I I I System I I I II 1 1II I I II TechnicianI I I I I I I I I I I I I I CUART 3: Rage of Techntogies: Edxeaticntl Institution I I I I I I | | ISroadoI I I I I I I |Electro- | |Audio IRgadmllnter.I I I I|ICrO cast I ITutOr linb"st.IAudio iVidee ICaeouterlnic | fCasSet-IACcefslactlvelyVdeol Meal lSIlUtlas-RadiO/I l lCouncetlor IConfer-ICanfer-IConfer- lBeak- Subject Area lCaIzutertes |Audio iVideo IDisk lVideolEquipmaltion ITV |Printlior lOn-Sitelencing |eneing lencing Iboud Etectronics _ Is 1 1 _ I I I I I I I _I I BasiI I c 1 I , I I I I I I 1 I I Electricity | I I I I * I I I I I AutoI I I .. .1 1 I I _I _ I I I I _I I_1- 1 Auto-| | | | | | mechanics ' 1 1 I I I I I I I I | FiLSrvc I I I I I I I I I I I I I I I I fleld service| | | |* | | | | | MaintenonmeI FieldSe e1 I I I I_IiI I I_I I I I _I .aintenam I_ I I I I I I I I I I I I I _I I Computer I i i i | | Servicing II I I I I_ I I I I I 1 Industrial I I I ILevell I I I I I I | | I I Service I | I 1111 I I I I I I I I I Technicians I I I I I I I I l I I I I I I ln~st Ia I I I ILe I . I 1 I 1 I 1 1 1. 1 I Irndutriat Ltl Nesithand I I I 11111 I I I I I I I I I I Safety I I I Machine I I I I I I I I I I 1 I 1 I Operatiomns I I I I I I I I I I . I I I I Maintenane s I I 10^ * I I | I I I I I I 0 mechanics, I I I I I I I I I I I I I I I Nanagers | | | Managers i I I I I l- I I I I- I I Ir I I ,. Educational Technology: Chapter 4 MM AI , - - - - - - - - - - - F I jill1 1 1 1111 1 1 1-h11 -n I I I I I I I I I I .111s,§1 lilI'lllll1l1I1l1l111 , ,,i I , ,, I* I I1 II II 111 1 11 11'1 1111 }2 j j g :2 J g j j g~~~ I III - 209- .~~~~ . Educational Technology: Chapter 4 1_ 1 1 1 1 1 1 l - ---- -------1- -- ------- -1-------- ---* - -- -- F- o t 3 4.2 ii a. --m l 1 a j 11 ___ in rIi I I i P I o 21~~~~~~~~~~1 o. ,l I', J~ I _ __I 4 * 1 1-11171 * I - *~~~~~~~~ 21 -I CUART 4: Range of TedwnooLogies: Induistry I I I I ~~~ ~~I I I I IeroadI I Iclass- I lEtectrol I I I IRandomllnter-I I I imicro Icast I ITutor Iroom lAudio Inic IELectro-I I jAudlo IAccesslactlvelVideol ICapt. ISImula-IRadio/I ICouncet-1Indust.IConfer-1Black- Inic I Subject Areaa IComrputerlCassettlAudio IVideo 10isk IVideolEquip. Ition ITV IPrIntIlor lOn-Sltelencing lboard INallt HEALTH &I I I I I I I I I III II HUNANSERVICESI Envirohmentatl I I I I ' * . HealthI I I I I I I I I IIIIII Laboratory I I I I I I I I I I I I II TechniciansI I I I 1 1 I I' I I I .*m Dental : I I I TechnoLogy I : 1 I I I I1 DentaL I I I I I I I III II Technolog 1I 1I Pharmacy 1 1I I I I I III I I ' I TEHNatICaLreI I I I I I I V IEACHER I I I I I I I I TRAINING I : I I I1 Sign Language I I I I* I II I I*I GOVERNMENT I WII I II I i Corrections I 1 II 0 I I Fire Fightingi I 1 I I I I I V~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~r * I~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~0 Educational Technology: Chapter 4 ACCESS Network. "Videodisc Production at ACCESS Network's Educational Technology Centre." Alberta Educational Communications Corporation, march, 1988. Apple Education Advisory Council "Multimedia in Education" - Learning Tomorrow, Apple Computer Inc, Spring 1987. AT&T., jgjJg-- a. Volume 1, Issue 1. AT&T Canada Inc., 1988. Bayardo, Barbara, Alpha Camara and Suzanne Grant Lewis. QLaSL Str_teiesl__;mgagtsl_the__ln_ystry.__^n_ Q SQliggiQs D*cember, 1983. Blang, Mark d QtgMIrite Geneva: International Labour Organization, 1973. Bore, Robert "Reach Out and Train Somkeone" Training and Development Journal, July 1984 p. 26. Browning, Terry "Laser Equipment Enhanced My Program" YokLgr__L Ukjfa1_Lon_JqUnL..jVolume 61, Number 5, August 1986, pp. 41-45. Campbell, Sandra, Audrey Mehler, Donna Sharon and Judith Tobin. Working Paper of the Office of Planning and Development Research, TVOntario. October, 1986 Campbell, Sandra. *muni.L2flafl1ai tlt12gitL. Ed4cat2o earfL.3. Office of Planning and Development Research, TVOntario, August, 1994. Cohen, Stephen, L., James J. L'Allier and Douglas Stewart. "Interactive Videodisc -- Then, Now, and Minutes from Now." Tt^inieg n_DeXtl2Lmsat-Joure«Lw October, 1987, pp. 31-36. SommuticitiQX_T££hnology_in__iducitkg_gad_Ir^aTing Proceedings of the Fourth National Conference on Communications Technology in Education and Training - March 15-17, 1982 Silva Spring, Marylands Information Dynamics, Inc., 1982. Donahue, Thomas J. and Mary Ann Donahue. "Understanding Interactive Video." r Deeember, 1983, pp. 27-33. Eurich, Nell. Q2E2gLat! --- QLAEI22C2t - 212 - Educational Technology: Chapter 4 Princeton New Jerseys The Carnegie Foundation for the Advancement of Teaching 1985. Emerson, Lawrence "Q & As Houston says Optical Media won't gain rapid acceptanco in the nation's schools". I_tty£ti20_QgiX&ry ftL& Volume 1,. Number 3 May/June 1987, p.p. 20-27. European Centre for the Development of Vocational Training. Berlin CEDEFOP, l997. ____Pr omotjon 2LQg£etrao_mos_Rs^h_nd2e£Lllt- rgmi &zatk2as_in--h---Lel__f-2o-^£-oL--rfiin Berlin: CEDEFOP, 1987. Feliciano, Glori i, et al Ibt - World Bank Staff Working Paper No. 491 Washington D.Ct The World bank, October 1987. Felstein, Roma "Hats Off to a New AGe of Learning" r_AGsAt September 11, 1997, pp 6-7. Fretwell, David "Technology Applications in Vocational/T*echnical Training" Paper presented to the Seminar on Using Technologies for Education and Training, Economic Developrment Institute, The World Bank, Washington, DC April 29, 19889 Galagan, Patricia "Computers and Trainings Allies or Enemies?" Iraoioi9-..angi2gYake oL.m1en yrn} April 1, 1987, pp 73-76. Ganley, Rosemary and Ron Wood I*&bno}129y_Adn___ang*L_Ib_arjais io-caniE_an.yj.qaio s Toronto: McClelland and Stewart Limited, 1975. Gizard, X and A. Josephe ,agi2naLRattlgen a__ a jLgtiaL aLaiLa_g Berlins European Centre for the Development of Vocational Training, 1997 pp.31-36 Govaerts, Kenneth C and Thomas A Grillot "Computer-Based Industrial Training. A Primer" T November 1954. Hawkridge, David: U_w_---nfg Q gjL2_ n Baltimore, Maryland: The Johns Hopkins University Press, 1983. Herschbach, Dennis s Ibrgb...igu9aki9o.Ial-bog1gloyLg1ipx......e1karosgiya, Columbus, Ohio: The National Centre for Research in Vocational Education, 1984. Herzing, Henry "Trends: Will Your School Be Ready?" Qrj tr IrZi&ing&_ Volume 4, Number 3, March 1988 pp 12-15. - 213 - Educational Technology: Chapter 4 Heyneman, Stephen P and Daphne Sier White eds. The_±ua1it_y__of Egdu…ati_n__and__Eg-onomic__D_-velogmanI A World Bank Symposium !Nashington D.C.: The World Bank 1986 Howes, Mike "Distance Learning - Getting Started" Journal of European and Industrial Training, Volume 11, No. 3, 1997. Hunter, Beverly and Aiken, Robert "The Evolution of Computer Courses in Vocational Education: Perspectives and Directors" IgTechnogical_Horizons_in__Edgcati2a Volume 14, Number 7, March 1987 pp 54-57. Huly, Paul, ed. Re12tces_Ain_ Content Proceedings of the Association of Media and TEchnology in Education in Canada '82 Conference. Winnipeg, Manitoba, June 6-9, 1982 Hudson, Larry and Steven Sorg. "Linking Classes by Phone." Vocational_Education__Journal, Volume 61, Number 6, September, 1986, pp. 45-46. Jha, C.S. et al The__ARRlication of Cgmguters in Technical and Vocationa-l duatilon International Congress on the Development and Improvement of Technical and Vocational Education. Berlin, June 22 - J41y 1, 1987. The I..lications of _Raid_Scientific -an -TEh-n-lo-gal -_Q--e-- on Technical and__Vocational Education International Congress on the Development and Improvement of Technical and Vocational Education Berlin: June 22 - July 1,. 1987. Johnson, Jerome g*lcttonic_Learning,From Audi,otage toVideodi£ Hillsdale, New Jersey: Lawrence Erlbaum Associates, 1987. Jones, Glen and E. Dorenda Taff "Using Today's Teaching Tools" YVoational_Education_Journal Volume 61, Number 6, September 1996, pp 43-44 Kaplin, Marjina M. "Cincinnati's Industry/Education Partnership ____ Nationwide" yocational Ed_uation Journal Volume 61, Number 7, pp 53-54. King, Edward Y: and Russell N. Rado "Teletraining Applied" Training-and Develogment-Journal, July 1984, pp 27-28. Mehler, Audrey Educational_Videodiscs:__Current _Research and DevYl2ogment. Working Paper of the Office of Planning and Development Research, TV Ontario. May, 1985. Merrill, Paul F and Junius L. Bernian "Videodisc Technology in Education: The Current Scene" National Society for Performance and Instruction NSPI_Jo_unal Volume 18, Number 9, November, 1979, - 214 - Educational Technology: Chapter 4 pp 18-19. Roach, John, V. "High Tech: A Place in Our Lives and In Our Schools." Ibe_Tchrology Teacher3 Volume 46, Number 2, November, 1986, pp 5-e. Rockmaker, Gordon and Gail G. Medlin "Updating Your Training Programs" Caroer Traini ng2 Volume 4, Number 3, March 1988, pp 24-26. Sakamoto, Takashi "Hardware and Software in Distance Education" Regional Seminar on Distance Education November 26 - December 3, 1986 Bangkok Shane, Harold G ITa&tag and Learning in Microelectronic Bloomington, Indiana: Phi Delta Kappa Educational Foundation, 1987. Sharon, Donna. Technology_in_Education__j28Z&_ Working Paper of the Office of Planning and Development Research, F Ontario. October 1987. …________ Tech ologyin Education 198 Working Paper of the Office of Planning and Development Research, TV Ontario April, 1988 Shi, H, A.L. Pater and F.A. Rossini "Microcomputers in Developing Countries: Industrialization in the Information Age The -intgEna i--l--Journal-of-ARl2ied_Engineering_Egucation Volume 1, Number 5, 1987, pp 321-327. Spitzer, Dean R. "'Megatrends' in Educational Technology." _g _Atioaa__Techn2jogy, Volume 27, Number 9, pp. 44-47, September, 1967. Stahmer, Anna and Ingrid Bryan. From Books to Satellites A Review of Technologies for Distance Education. Paper prepared for the Seminar on Using Technologies for Education and Training: An Economic Perspective; April 16-29, 1988, Economic Dev-lopment Institute, The World Bank, Washington, D.C. Stewart, Margaret Ihf_..aiing__ireDi2ector_ ang-th__CBT De-cision Paper presented to the National Society f'r Performance and Instruction Conference, San Antonio, Texas, 'iarch 15-21, 1987. Stover, Teresa S "Technical Training by Videotape" Iraining July, 1986 pp 27-30. Van Stry, Robert F. "Using Technologies for Education and Training: Vocational Technical Education Applications" Paper presented to the Seminar on Using Technologies for Education and - 215 - Educational Technology: Chapter 4 Training' An Economic Perspective, April 18-29, 1988, Economic D*eveopment Institute, The World Bank, Washington, D.C. Van Zwiet*n, John "Managers and Microcomputers: Getting the Right Mi x" I^Btu_LA t Qm2&t _JQUL. July 1984. - 216 - .. . .4... Educational Technology: Chapter 5 Chapter 5 Pilot Projects in Educational Technology: The Philippines Case John Middleton - 217 - Educational Technology: Chapter 5 PILOT PROJECTS IN EDUCATIONAL TECHNOLOGY: THE PHILIPPINES CASE John Middleton World Bankl1 Pilot projects accompanied by extensive research and evaluation have been a m&4or means of introducing educational technologies in developing countries. The Radio Mathematics experiments in Nicaragua and the Radio Language Arts project in Kenya are among the better documented and often cited examples of this approach, no doubt in part because technology - in this case, interactive radio -- was found to have positive effects (Friend, Searle and Suppes, 1980; Imhoff and Christensen, 1986). Pilot projects can offer a number of advantages generally, and these apply to educational technology innovations as well. First, they can provide decision makers with information about costs and effectiveness of new practices, assisting in decisions for adoption. Second, they can help implementing agencies learn more about how the technology works, leading both to ongoing improvements in the intervention and a significant degree of learning from experience. The latter can have useful institution building effects. These benefits are most readily achieved when pilot projects are successful: that is, when the innovation works and the potential for learning and institutional development are realized. But pilot projects are not always successful, and little is known about such cases. Nevertheless, the imrortance of pilot projects as a means of introducing technological innovations indicates the need to assess such efforts for lessons that might improve future efforts. One such case is a pilot project of radio education in the Philippines launched in 1978. This pilot project drew directly on the Radio Mathematics experience in Nicaragua in developing materials for direct instruction in the national language (Pilipino) in elementary classrooms. In- service teacher training using radio was also introduced, along with smaller experiments with radio support for agricultural extension and satellite telecommunication links between Manila and remote regional education offices. A complex summativa evaluation was carried out to measure the effects of the radio interventions, and the evaluation was timed to coincide with the planning of a large-scale World Bank assisted program to reform and improve primary education. The project was well supported by the government of the Philippines and a loan from the World Bank, and Bank staff played significant roles in the design and supervision of the project. The pilot project and the evaluation were completed on schedule. A highly competent implementation team was developed over the course of the project. Th.e project generated a large body I/ The findings, interpretations, and conclusions of this paper are the author's own. They should not be attributed to the World Bank, its Board of Directors, its management, or any of its member countries. - 218 - Educational Technology: Chapt4t 5 of materials and reached all utilization targets. The evaluation was completed as designed, and report. were produced on time Evaluat.on dat Indicated that the pilot project led to modest learning gains, and provided lntight into the relative cost-effectiveness of various combinations of the program elements included in the study. The effort was completed well wlthin budget, wlth approxliately 40 percent of available funds unspent. Despite successful Implementation, the project was not contlnued, much less expanded. Moreover, the extenslve evaluation data had little to do with final decLiLons. Although the project was Liplemented successfully, lt dld not succeed ln Lts priLcLpal purpose of supportlng ratlonal decisio;" making. Three years after the pilot project was completed, the World Bank commLssioned a case study to draw lessons from the experlence that could be helpful in desLgnLng future pllot projects. Specifically, the study was to examlne the way ln whlch evaluatLon functLoned ln the Educational Radlo Project -- how lt was conceptuallzed and designed, how lt was implemented, and how the results were used ln docisLon maklng, A modlfled form of naturallstic inquiry was employed to develop insight lnto the complex processes through which project outcomes imerged. The boundaries of the study were defLned to encompass the actions and lnteractions of the World Bank and the Phillpplne agencles engaged ln the project, with a central focus on the issue of use of evaluation information in declsion maklng. Data wero gathered through examinatlon of project documents in the World Bank and the Philipplnes, two rounds of unstructured intervLews with Bank staff, and one round of interviews with key actors ln the PhliLppLnes. Thli chapter reports the prlnclpal flndings of the study. The flrst section examlnes the orLglns of the pilot project and its evaluation. Subsequent sections examine implementatLon, ldentlfy patterns that help explain the project outcomes and processes, and consider general lessons for the deslgn and LmplementatLon of pilot projects. The focus is on the evaluatlon of the use of radlo for d1.rect Lnstruction in prlmary classrooms and for in-eervlce teacher trainlng. A more complete report coverLng other components of the pilot -- the use of radlo for extenslon and testing of satellite telecommunication linkages -- has been ptiblished elsewhere (Mlddleton, 1985). As the 1970s began, the Phllippines government Initiated a decade of concentrated effort to improve education. A number of Important milestones in educational development had already been achieved. Enrollment rates were high, with virtually all chlldren entering prLrary school and 60 percent completlng grade slx. In hlgher educatLon, 21 percent of the relevant age group was enrolled. Males and females beneflted equally from educational opportunity, and adult llteracy stood at 90 percent. - 219 - Educational Technology: Chapter 5 Serious challenges remaiuied. Repetition rates were high, reducing the efficiency of schooling. The overriding problem was the quality of education. There was, on the average, one tertbook for every ten students. '*hile student/teacher ratios were favorable, teachers had limited opportunity for in-service education. The curricula of primary and secondary schools needed reform, and higher education needed improvement. The problem of educational quality was most serious in rural areas. Rural primary and secondary schools were weak. And, despite a widespread non-formal education effort, a large rural population of out-of-school youth and adults lacked opportunities for basic, agricultural and vocational education. The government had addressed a number of these problems, with the support of multilateral and bilateral donor agencies. Higher education and teacher-training institutions were strengthened. A major effort was launched to revise curricula and increase the availability of textbooks at the primary level. This large-scale program of educational development led to the creation of a specialized agency of the Ministry of Education and Culture to develop and manage foreign-assisted projects, including those funded by World Bank loans. Created in 1970, the Educational Development Projects Implementation Task Force (EDPITAF) was chartered for a ten-year period. EDPITAF rapidly assumed a vital leadership role in the development of Philippine education. Planning for Educational Technology In 1975, EDPITAF turned its attention to the potential of communication technology to improve the quality of education. The Philippines had used both radio and television for education, but lack of broadcasting infrastructure and other problems had led to the abandonment of the effort. Nevertheless, evaluation results from other countries showed that broadcast education could improve student achievement. In addition, a number of countries were successfully using communication technology, notably radio, for educational purposes. In particular, a primary-level radio mathematics project in Nicaragua was evolving effective curricula and materials design procedures and demonstrating significant impact on student performance at very low cost. World-wide interest in the use of satellite communication for development was also increasing, and some government leaders were interested in this aspect of the technology. EDPITAF received World Bank support in early 1975 for a Pre- investment Study of Communication Technology for Education. Part of a package of initiatives for continued collaboration with the Bank in educational development, the study was broadly conceived to include both formal and non- formal education. It would encompass the use of satellites and would focus on priority goals for improving the quality of rural education. The study clearly was seen as leading to proposals to the Bank for incorporating communication technology into the educational development effort. The proposed study was supported at the highest levels of government. An Interagency Planning Committee was formed by executive order from the Office of the President of the Philippines. It was chaired by the Assistant Director of the President's Executive Office and included all - 220 - Educational Technology: Chapter 5 relevant agencies: the Department of Local Government and Community Development, EDPITAF, the Department of Public Information and its National Media Production Center, the Department of Public Works and Telecommunications, the National Economic Development Authority, the Philippine Communication Satellite Corporation, the Broadcast Media Council, the Rural Broadcasters Association, and the Univervity of the Philippines (representing schools and universities). The Interagency Planning Committee convened a workshop in June 1975 to review and establish the terms of reference for the study (Office of the President of the Republic of the Philippines, 1975). The meeting brought together senior education and communication leaderq from the Philippines. A team of five consultants sponsored by UNESCO contributX_ international experience. Participating in the effort were the eventual directors of the pilot project and its evaluation component, as well as Bank staff and external consultants who would play continuing roles in project design and implementation. The workshop led to the design of a pre-investment study for the massive use of educatinnal technology for both formal and non-formal education. The use of satellite communication was a major focus in this one- year feasibility study coordinated by EDPITAF. Faasibility Study The first phase of the feasibility study was devoted to assessing needs for educational development and setting objectives for the uses of technology. The four primary objectives identified were to promote more effective learning, increase the relevance of what was learned, reduce education costs, and serve a large population in and out of school. Importantly, radio was identified at this early point as the medium of choice for the Philippines, given the existing availability of broadcast resources and the nature of identified needs. With advice from the Office of the President, it was determined that basic (primary) education and rural development were the top priorities. The study culminated with the production of a five-volume project proposal to establish the National Educational Communication Program in order to meet the directive to support primary education and rural development. The proposed program rested on six strategy elements: 1. Development of a teaching apnroas.zh that integrated interpersonal and multi-media communication. Drimarily radio. This approach would be followed in both formal and non-formal settings; 2. Decentralization of educational communication activity. Regional production and coordination centers, as well as a monitoring and feedback system would be developed; 3. Establishment of a ventral office to manage the Rroject. This would be primarily a coordinating unit; - 221 - Educational Technology: Chapter 5 4. Development of collaborative relationships with education and development institutions. This mechanism would mobilize existing resources and expertise while maximizing institutional development; 5. Utilization of existing Rroduction. dissemination, and research resources. The study had demonstrated that adequate resources already existed; and 6. AdoRtion of a technical support design. The design relied heavily on the use of existing facilities, with new acquisition as needed to supplement current system resources. These elements added up to a national educational communications system addressed to needs in primary and rural education, with significant decentralization. Decision for a Pilot Project The proposal was submitted to the National Economic Development Authority (NEDA), the National Board of Education, and the World Bank. NEDA's reaction was that the pre-investment study had not demonstrated the market, technical, and economic feasibility of the proposed program. It recommended a pilot project to test the primary and rural education components, as well as the use of satellite communication in linking Manila with distant education offices. After more than two years of intensive study for a national system, the proponents of educational communication had gained the opportunity for a pilot project. What had begun as a push for a satellite-based national system had become, through the process of policy research and decision, an exploratory effort on a far smaller scale than originally envisaged. The focus still was on primary and rural education. Moreover, there was now a group of committed persons and a base of data and analysis, all of which would prove useful as the pilot project moved ahead. Pilot Project Design. With the government's feasibility study completed and fundamental policy decisions reached, the Bank sent a team of two to the Philippines to appraise plans for the pilot project. One staff member had considerable experience in the evaluation of educational technology in developing countries; the other was a broadcasting specialist. Plans called for the use of radio to upgrade in-service primary teachers, to provide direct primary classroom instruction, and to improve rural education. Radio would be used in conjunction with interpersonal communication and printed materials. The Bank team worked with Philippine counterparts for two weeks to develop a proposal for the pilot project. The design developed during this period grew out of the earlier proposals prepared by EDPITAF and benefited from the extensive analysis and planning that had been done under the pre- investment stuidy. It became the basis for Bank funding and, with only minor variations, was followed throughout the pilot project. - 222 - Educational Technology: Chapter 5 The pilot project that emerged focused on the use of radio to improve the quality of education tlhrough: (1) in-service teacher training; and (2) direct instruction in language arts in rural primary schools. There were two additional components: an exploratory study of using radio to support rural., non-formal education; and the study of a number of technical issues assoc.ated with the use of technology in education. Thus, the project had four action components. The design also specified a comprehensive set of evaluation activities. There would be extensive formative evaluation of materials as the project developed. Large-scale summative evaluation would be conducted to assess the costs and effects of the two instructional components -- teacher in-service education and direct primary instruction. At the time that the design was being developed, the Government and the Bank had initiated a national project to strengthen the quality and availability of textbooks in the schools. There was considerable interest in the potential interactive effects of the new textbooks and the radio project in improving educational quality. At the suggestion of the Bank evaluation expert, the summative evaluation design incorporated measurement of the effects of various combinations of the two radio components with different ratios of new textbook availability to students. The result was a complex experimental design with eight treatment groups and a control group. For the World Bank, the main objective was to develop research- based information on the costs and effectiveness of radio in improving the quality of primary education. Bank objectives also included exploration of the use of radio for rural education and "continued examination of the technical options for production, interconnection, transmission and reception that will be open to the Philippines should the government subsequently decide on a major expansion of its use of communication technology for education." These objectives centered on learning from the project to support future decisions. The Government of the Philippines had two additional objectives. One was service: the project was expected to have beneficial effects for teachers, students, and rural people. A second was institution building: the project would create a "nucleus of management, coordination, [and] operations.. .resources" useful in establishing an overall national system. Project managers at EDPITAF also placed considerable emphasis on developing "instructional models" in each of the applications areas. The component of the pilot project designed to test strategies for improving learning achievement in primary schools had two components: 1. Continuing Education for Teachers (CETI. The in-service teacher education component was known as Continuing Education for Teachers (CET). Primary school teachers were to receive semester-length courses by radio in Pilipino (the national language), English, social studies, science, mathematics, and teaching skills. The courses combined broadcast radio, print materials, and periodic forum meetings. Teachers could receive in-service or academic credit for successful course completion. Regional colleges and universities were to cooperate in the forums and in granting credit. Radio lessons and print materials were to be developed centrally and broadcast through regional stations. The plan was to develop and broadcast courses on a - 223 - Educational Technology: Chapter 5 staggered schedule, wlth Piliplno and teachlig skills In the first yoer and subsequent courses added as the project progressed. It ws anticipated that more than 12,000 teachers would participate In the program. 2. Maio-Assisted leaching in lelqntary Schools (RATESR. The direct classroom instruction component, known as Radio-Assisted Teaching In Elementary Schools (RATES). was designed to Improve student abilitles in Piliplno. The RATES prograwR was built around daily 30-minute radio lessons. supported by teachers' guides and student workshoets. Materials were to be developed centrally by EDPITAF and broadcast/disseminated at the regional level. A total of 140 lessons per year were developed, following the official Department of Education currlculum and ensuring consistency with the scope and sequence of teaching and with the new textbooks. According to plan, the first year of RATES was directed at grade 4 students. Grade S materials were developed for the second year, and grade 6 atorials were added in the third year. It was expected that approximately 2,500 classes would be reached with RATES. 8vmmative Evaluation Design. The summative evaluation was designed in some detail. The principal question was: "How is student performance affected by continuing education for teachers (CET), radio-assisted teaching in elementary schools (RATES), and additional textbook availability -- introduced separately or In combination?" To answer thla question, a multi- treatment field experiment was developed. Classrooms were randomly assigned to the nine groups -- eight treatment combinations of RATES, CET. and different textbook ratios, and a ninth group that served as a control. The primary dependent variable was student learning gains as measured with pretests and post-tests. The desLgn Included provislons for a cost analysis of the various treatment combinations. Since the purpose of the pllot was to test impact on rural education, rural pilot sites were selected: the provinces of Pangasinan and Leyte. In both provinces, significant percentages of the population did not speak Pilipino (45 percent and 72 percent, respectively), providing a good test of the impact of the Pillipino courses. Pangasinan is located about 100 miles north if Manila, and Leyte is an island province located about 500 miles to the southeast. Project Organization and Dlvision of Labor. With a large and complex design and firm time constraints for the pilot project, a strong and effective project management structure was clearly needed. At the same time, It was important to maintain the overall government strategy of Involving key organizatlons. It was decided that the work of carrying out the project would be shared among several agencies. Within EDPITAF, the Educational Communications Office (ECO) was established with overall responsibility for project management and specific responsibility for implementing RATES and CET. The Pilot Project Director led this ECO office, hereafter referred to as the Project Office. The summative evaluation was conducted by the University of the Philippines. It was led by the Dean of the Institute of Mass Commuication (IMC). also a member of the Interagency Planning Committee. The Department of Econtiics would conduct the cost analyses. The Project Office provided overill administration; the work was performed under a contract between the - 224 - Educational Technology: Chapter 5 Project Office and IMC. IMC (hereafter identified as "the evaluating agency") was the logical choice to carry out the evaluation since it was one of the leading communication research organization in Asia, with a long record of success in field research and evaluation. Linkages. The Project Office established linkages with other agencies. Of particular importance was the Bureau of Elementary Education within the Ministry of Education and Culture. The technical expertise of the Bureau was essential in the development of CET and RATES materials. Linkages were formalized through the establishment of a Project Advisory Group (PAG) consisting of four consultants to the project. One of these was the Pilot Project Director. The others were the IMC Evaluation Director; the Head of the Bureau of Elementary Education; and the Vice- President of the Philippine domestic satellite corporation. The Project Advisory Group was to serve several purposes, including providing overall policy guidance for the project. During the first two years of the pilot, it would also advise on operational issues. Perhaps most importantly, the Advisory Group could link the pilot project with higher level decision makers. The Head of the Bureau of Elementary Education, for example, was a potentially important link to decision makers in the Ministry of Education and Culture. The organizational structure of the pilot project is shown in Figure 1. FIGURE 1: PROJECT ORGANIZATIONAL STRUCTURE 4 I * * * I 1 1 , * * EDPITAF DIRECTOR ****** EDUCATION MINISTRY* * II I * * I* * F~~~~~~~~~~~~~ BANK & OTHER , PROJECT * INTERNATIONAL ********** ECO DIRECTOR ****** ADVISORY * AGENCIES l GROUP * * * * * * * IMC * SUMMATIVE *** EVALUATION | PELOT OPERATIONS| - 225 - Educational Technology: Chapter 5 Negotiation and Funding. By July 1978, the project had been approved by the responsible entities within the World Bank and the Government of the Philippines, and the loan negotiated and approved. The project officially began in August 1978. The total value of the project was estimated at US$ 3.86 million, of whicht $2.0 million would be foreign exchange costs funded under the Bank loan. Bank funding would provide a number of critical inputs, including vehicles, a recording studio, 62 months of technical assistance, and 24 months of fellowships to enable project staff to study abroad. Government counterpart funds would fund local costs of implementation and evaluation. II. Implementation The pilot project would be implemented in three one-year stages designed to produce evaluation information for development of the upcoming World Bank assisted project for the improvement of primary education. The schedule of implementation and decision making was as follows: the first year for developmental activities and formative evaluation of materials; the second year for running the full pilot test, with the summative evaluation report due at the end of this period; and a third year in which the program was continued in order assess the effects of two years of RATES on student learning. A World Bank appraisal mission four months after completion of the summative evaluation would provide a focal point for decisions on the incorporation of radio education in the final project for the improvement of primary education. Development Year Broadly stated, the objective for the formative year was to develop and test the full range of mechanisms necessary for the pilot test of the project in the next year. For the Project Office, this meant designing procedures for materials development; developing prototype materials and conducting formative evaluation; establishing field offices and operations in the pilot sites; and implementing both CET and RATES in the process. The Project Office confronted a number of difficult administrative challenges. With contracts in place, effective working relationships with the evaluating agency had to be developed. Linkages needed to be established with the Ministry of Education and Culture and with other cooperating agencies. The resources needed for the project -- tape players, vehicles, a recording studio -- had to be acquired. The Bank was providing funding for technical consultants and a fellowship program for external training of key staff; consultants needed to be identified and fielded, and staff members selected and sent for training. It was an equally busy year for the staff of the evaluating agency. The complex summative evaluation plan had to be actualized. Instruments and procedures had to be developed and tested as RATES and CET were fielded in prototype form, leading both to a preliminary evaluation report to guide the - 226 - Educational Technology: Chapter 5 Project Office in the pilot year and to the establishment of an effective, full-scale pilot year evaluation effort. The main tasks - - getting CET and RATES in operation, testing, and developing summative evaluation procedures - - were accomplished proficiently and on time. However, two problems emerged that significantly affected implementation and the eventual use of the evaluation in decision making. These included disagreement on the design of the summative evaluation and difficulties in mobilizing resources. Evaluation Design Issues. The project design did not provide much detailed guidance on the relationships between implementation and summative evaluation. While it was clear that the evaluating agency would conduct an independent evaluation, it was also clear that the Project Office had overall responsibility for the full pilot project. This relationship was reflected in the contractual arrangements between the Project Office and the evaluating agency, which made provision for a consultative and reporting relationship. At the same time, it was recognized that the expertise for evaluation was in the evaluating agency. The way in which the two organizations worked together would be important. The Project Office appointed a summative evaluation consultant from the Ministry of Education Planning Office to advise the Project Director on summative evaluation. This appointment not only provided an important level of technical competence within the Project Office but also established a potentially valuable link with the Ministry. The Evaluation Consultant's role included review, on behalf of the Project Office, of various elements of the summative evaluation design and instruments being developed by the evaluating agency. Inl October 1978, the Project Office and the evaluating agency met to go over the evaluation design and instruments. Project Office staff brought to the meeting a number of concerns, one of which related to the cost analysis. While the design made provision for comparative analysis of the costs of the various combinations of RATES, CET, and textbooks under study, there was apparently no plan to gather cost data on current teaching of Pilipino as reflected in the~ "no treatment" control groups. Thus, while it would be possible to make cost-effectiveness comparisons among treatments, it would not be possible to compare the costs of the treatments with the costs of current teaching. Staff pointed out that the lack of this information would make decisions on adoption of any of the pilot models difficult. There were also concerns regarding the learning gain tests to be used to assess the impact of the various treatments. The tests had 80 items, ten measuring listening skills and the other 70 assessing reading competencies. The Project Office felt that these tests would not be sensitive to the primary impact of radio instruction and urged a better balance between listening and reading skill items. The evaluating agency chose not to modify the plan for collecting cost data or the achievement tests. This meeting marked an important turning point in the pattern of working relationships between the Project Office and the evaluating agency. After the meeting, relationships became less collaborative. The evaluating agency continued to submit progress reports, but evaluation instruments and documentation were shared with the Project - 227 - Educational Technology: Chapter 5 Office only after they had been adopted for use, effectively cutting the Project Office out of the proce3s of evolving the evaluation system. Resource Constraints. A number of resource constraints made the work of the Project Office and evaluating agency more difficult. A restriction on vehicle procurement, which lasted the entire life of the project, made it difficult for staff to carry out their responsibilities in field sites. Though ingenious ways were found to deliver materials, conduct formative evaluation, and coordinate project activities in the field, a great deal of staff time was diverted from substantive to logistical activity. Contact with field-level activity also suffered. Government of the Philippines and World Bank interr,ational competitive procurement procedures delayed the acquisition of cassette players and recording/duplicating equipment. For instance, tape players arrived four months later than planned. Requirements for acceptance of lowest cost bid lent to purchase of equipment of lower quality than desired, which in turn created maintenance problems. Delays in acquiring tape duplication equipment adversely affected the quality of radio materials and, until the equipment arrived in 1980, made duplication a slow and cumbersome process. During the construction of the Bank-funded recording studio in EDPITAF facilities in Manila, audio materials were produced in rented facilities. When the studio was completed, it was found to be inadequately insulated for sound and poorly air conditioned and was used only for editing and duplication. Despite such difficulties, implementation proceeded. Evaluation instruments were developed, tested, and revised. The World Bank staff visiting the project in January 1979 were encouraged by what they saw and reaffirmed the purpose of the pilot project in a supervision report: "If the study gives any encouraging indications as to the effectiveness of primary level instructional radio... it would be worthy of Bank financing to follow up these project for developing primary education region by region with new textbooks and educational radio along with improved curriculum." Preliminary Evaluation Findings. The evaluating agency's preliminary report, based on data collected during field trials of procedures and instruments, provided useful insights for the pilot year. In combination with formative evaluation data collected by the Project Office, this report pointed toward improvements in the learning materials and utilization systems. Problems encountered in test administration were identified. It was noted that teachers in control areas were exerting special effort to "win" in competition with RATES classes. This teacher behavior appeared to explain, at least in part, the finding of "no significant difference" between RATES and control classes during the formative year. Pilot Year In the pilot year, RATES materials were used in 120 grade 4 classes (4,800 pupils) and 30 grade 5 classes (1,100 pupils). CET reached 2,300 teachers for the teaching skills course in Pangasinan and Leyte; and 2,500 for the Pilipino course in the same sites. The material used in the pilot year reflected the lessons learned from formative evaluation during the development - 228 - Educational Technology: Chapter 5 year; project operations were also improved by the experiences of the first year. The evaluating agency successfully met demanding schedules for data collection and analysis and produced both preliminary and final reports on schedule. Implementation Issues. Project Office and evaluating agency staff were still struggling with a number of implementation problems as the pilot test began with the school year in June 1979. The Project Office studio could not be used for recording, and continued reliance on contract facilities was necessary. There were still no project vehicles. Staff training fellowships could not be used because of government restrictions. Procurement of cassette players for the schools was delayed. Serious implementation problems were encountered in the broadcasting component of the project. Radio signals were weak, resulting in erratic reception and poor quality. This problem was compounded by frequent power failures that cut off broadcasting completely. Administrative problems began to appear in the financial relationships between the Project Office and the evaluating agency. The evaluating agency had difficulty in complying with government financial regulations; as a result, EDPITAF had to cut off cash flow to the evaluating agency for more than a year. The continuing discussions between the two agencies on this problem occupied considerable attention, and together with the lack of funds for evaluation, kept the two at some distance from each other. Technical and administrative coordination continued, but probably at a lower level of effectiveness than might have been achieved without the funding problem. Completion of the Summative Evaluation. The evaluating agency completed data collection by the end of the school year in March and began data analysis. Bank staff continued to provide technical advice on the evaluation. The evaluating agency submitted preliminary summative evaluation findings to the Project Office, followed by a joint data review with the Project Office. A month later the evaluating agency sent the Project Office a letter summarizing the gain score data for the various treatments. The data are somewhat equivocal. They show substantial learning gains for various treatments, particularly the combination of RATES with either CET or a 2:1 student/textbook ratio. Surprisingly, when all three of these treatments were used in combination, the learning gain was lower than for the control group. This finding and some of the other results are perplexing. But despite these anomalies, the data indicate that those treatments with a radio component in any configuration scored higher than either configuration of textbooks alone. The learning gains of the control groups were unexpectedly high, which made interpretation of the data more difficult. A partial explanation for this phenomenon may be the "contamination" caused by teachers in the control areas working extra hard to "beat" the experiment. This tendency was observed during the formative year, and modifications were made to the pilot year design, including the addition of a post test-only control group outside the experimental area. The changes that were made apparently were not sufficient to eliminate this factor. - 229 - Educational Technology: Chapter 5 The preliminary cost effectiveness data were interesting. Using as a measure the cost per point of gain score achieved, the treatments using only textbooks were most cost-effective at P 0.20 for the 2:1 ratio and P 0.31 for the 1:1 ratio. CET and a 1:1 textbook ratio cost P 0.74 per pupil point gained; RATES and a 2:1 textbook ratio cost P 13.33. Thus, while the data indicated that radio made a difference in learning gains, textbooks were much more cost-effective. Both the evaluating agency staff and the World Bank staff believed that more thorough analysis of the data to control for a number of key student characteristics would help explain some of the anomalies in the findings. Continued evaluation in the third year of the project was seen as important to full assessment of the effects of radio. It was also felt that impact would increase after two years of programming; one year might have been too short a time for the various treatments to h-:e major effects. Clearly, problems with broadcasting weakened the pilot; equally clearly, broadcasting reliability was a crucial element that should be tested. The design of the summative evaluation created a heavy logistical burden in test administration and data collection in two geographically distant sites. The evaluating agency was hard pressed to carry out the evaluation as designed, especially with the lack of vehicles and heavy field activity demands. One result of this problem is a lack of organized qualitative information about RATES and CET in the field. Although the Project Completion Report in 1982 provided useful summary information about project operations, these data were not available at the time of the preliminary summati*.e evaluation in an integrated form. Making a full interpretation c; the learning gain data was not possible without this qualitative background. It is difficult to determine the impact of the findings reported at that point in th project. The Bank staff apparently did not reach strongly supportive conclusions on the basis of the data. The Project Office and evaluating agency staff were more hopeful. The first two years of the project had led to significant accomplishments and a strong project team. Third year implementation could yet show strong impact. The evaluating agency agreed to conduct a follow-up summative evaluation of impact during the third project year (1980-81). This would include assessment of effects on students with two and three years of cumulative exposure to radio education. In addition, the third year would provide the opportunity to strengthen the evaluation design and data analysis methods. Decisions on the Adoption of Radio. Meanwhile, ds the Project Office and the evaluating agency moved into third year implementation, planning for PRODED continued. The full Bank appro'isal team arrived in September 1988 and stayed into October. This Bank mission was the event toward which the radio pilot project had been directed from the outset. During the visit, the Government and the Bank would decide on the shape of investment in education for the next five years. Working from the EDPITAF proposal of the prior year, and the work of the Bank pre-appraisal team, the mission collaborated with the Government of the Philippines in the development of a five-year program to strengthen elementary education. This program would be the first half of the Government's projected ten-year effort. - 230 - Educational Technology: Chapter 5 The mission was led by the same individual who led the previous pre- appraisal mission, and the team included a consultant on textbook publishing. There was no Bank official with prior direct experience with the radio pilot project. The draft appraisal report and the final version outlined a large and comprehensive program of investments to strengthen the elementary education sector. The strategy rested on policy changes, management improvement, and a series of actions to improve sector performance. There was an emphasis on improving equity through investments in disadvantaged regions. Bank funds would be used to finance curriculum development; development and production of instructional materials; facilities and equipment; the monitoring of teacher supply; staff development; and sector evaluation. The radio project was noted as an ongoing Bank-supported effort. However, the appraisal report made no substantive reference to the pilot project, and radio was not incorporated into the PRODED project. The Pilot Project Director recalls being given only a week's notice by EDPITAF leadership to develop a proposal for use of radio prior to the Bank team's arrival. This proposal outlined region-based use of radio and tape- assisted instruction to strengthen language arts instruction in Pilipino and English, a strategy consistent with the thrust of PRODED toward assistance to educationally disadvantaged regions. The team, with strong leadership from the textbook consultant, was heavily focused on textbook production. The radio component appeared to complicate the strategy and thus did not become an integral part of the appraisal report. A major component of the proposed strategy was the creation of an Instructional Materials Development Corporation (IMDC). This entity would be responsible for developing, producing, and disseminating learning materials. Although the report emphasized print materials, Project Office and EDPITAF staff saw the corporation as a possible structure through which radio education might eventually be expanded. While broadcast education was not explicitly included in the emerging PRODED effort, neither was it excluded. The Goveinmert of the Philippines position. Planning for a Bank- financed project is a collaborative process. EDPITAF and the Ministry of Education and Culture had a great deal to say about what went into PRODED. Government support for the radio pilot project was weak at this point; leadership had changed during the preceding two years. The individual who was EDPITAF Director when the pilot project was initiated -- a strong supporter of the project -- had left the agency. The new EDPITAF Director supported the project but did not have the same history of close involvement in its development. Moreover, EDPITAF was approaching the last year of its mandated existence. The future of the agency was uncertain. Much time and attention was necessarily being devoted to the larger issue of EDPITAF's future. The Minister of Education was to retire at the end of 1980. With his successor's attitude toward radio education unknown, the leadership situation was further clouded. Thus, the pilot project had no strong senior advocate within Philippine leadership circles. With somewhat equivocal preliminary summative - 231 Educational Technology: Chapter 5 evaluation findings, a strong push for radio education did not appear possible. The World Bank position. Similarly, on the Bank team there was no strong advocate for radio education. The team members had nui prior association with the pilot project; none of them were experts in broadcast education. The influence of the consultant who was a strong advocate for textbooks is said to have been important in the joint Government/Bank decision to emphasize textbooks. The two World Bank staff members who helped design :he pilot maintained a continuing interest and involvement in the project during this period. But since they were not a part of the project appraisal process and were not working in the part of the Bank responsible for Philippine lending operations, their direct influence in the decision-making process was minimal. Given the situation and the key actors in the appraisal process, it is not surprising that radio education did not become a formal part of PRODED at that time. Final Year While all these decision-making processes were going on, the Project Office continued with implementation of the third year of RATES and CET. Dissemination switched from broadcasting to classroom use of cassette tape, partly in recognition of difficulties encountered in broadcasting and partly as a trial of an alternative delivery system for the future. The third year evaluation reflected changes growing out of the pilot year experience. The number of treatments was reduced to simplify the design. Data analysis was more sophisticated, including a number of multi-variate analysis methods. Qualitative data were obtained through unscheduled visits to project classrooms. A post test-only control group in a different area was added to control for contamination from extra effort by control group teachers in the experimental area, a feature that had also been added to the pilot year evaluation design. The final report for the third year evaluation indicates that two continuing problems in the design were not fully addressed. The cost analyses still did not include data on regular classroom instruction. There is no indication that the learning gain tests were revised to include more items directly measuring the impact of radio. The third year evaluation results were similar to the findings of the pilot year. RATES students scored slightly better than students in other treatment or control groups (Institute for Mass Communications, 1982). The project officially ended in July 1981. The evaluating agency submitted the first draft of its evaluation report in January 1982, and the Project Office began drafting the Project Completion Report at about the same time. Planning for P2ODED went ahead, with the final Bank appraisal report completed in April 1981. Negotiations were completed for the project to begin in 1982. Radio was not formally included in the final design of the project. - 232 - Educationrl Technology: Chapter 5 Continuing Efforts for Utilization. The effort to see the pilot project forward to a useful conclusion did not stop with the PRODED appraisal missaon. Though disappointed by the outcomes at that stage, the Project Office and EDPITAF continued to seek support for the project and to evolve a pattern of follow-up action to utilize project outputs. The objective of establishing radio as an Important part of elementary education had not yet been achieved, but the staff's belief in the project had not lessened. The materials (in broadcast and cassette format) were having an impact, however small. Moreover, much had been accomplished toward the Philippine objectives of service and institution-building. It was important, at least, to capitalize on these accomplishments. A new Minister of Education was appointed in January 1981. He came to the post from the presidency of the University of the Philippines. In that capacity, he had been aware, administratively, of the radio project through the role of the University's Institute of Mass Communication (IMC) as the evaluating agency. The Project Office sent copies of the final evaluation report to the new Minister and began an informal campaign to develop support for the project. There was still tiwe to have radio included in PRODED. The proposed Instructional Materials Development Corporation, for example, could develop and provide radio materials as well as textbooks. In line with the regional development strategy of the Ministry, EDPITAF requested proposals for continuation of radio education from the regions that had been involved in the pilot program. Support for RATES and CET wae strong, and all three regions responded with proposals in March. Wal"q t: y differed in detail, they all proposed continuation and expansion of use of TES and CET materials on a tape-assisted basis. The projects would be carrie(' out within the context of PRODED. These proposals provided the basis for continued use of the materials developed and would serve as the framework for transition from the pilot project to the future. Efforts to develop top-level support for radio education faltered. The Project Advisory Group, which might have served as a bridge between the Project Office and the Ministry of Education, began to wind down as the project neared completion. EDPITAF itself was in a critical transition period during which the Government wrs studying alternatives for the future of the agency. Direct management of EDPITAF activities from the Ministry gradually increased during the year, culminating with the official appointment of a Vice-Minister of Education to lead EDPITAF in December 1981. The EDPITAF Director recalls the uncertainty of communication and decision lines during this period. He noted that, as the radio project neared completion, he attempted to conserve the large balance of unspent Bank funds in the project to support "technician education," hoping to establish a basis for continuation in a different framework. This strategy did not succeed; the Bank eventually cancelled the outstanding amount of the loan. The new Minister was not supportive of educational radio. Attempts to secure support for the project met with little success. Planning proceeded without an educational radio component. The Suimmative Evaluation R-viewed. After the end of the pilot project in July 1981, there were six more months in which Bank funds could be - 233 - Educational Technology: Chapter 5 used to close out project activities. Project Office staff used this period to transfer project materials and resources to the three cooperating regions to support their plans for continuation. The evaluating agency completed data analysis for the third year evaluation, preparing its report for submission. Two consultants from the Northwest Regional Educational Laboratory in the U.S., retained to advise on guidelines for continuing media-assisted education, commented favorably on the project's instructional designs, quality of materials, and formative evaluation efforts. Their findings in reviewing the pilot year summative evaluation report shed light on issues regarding the design of the evaluation and its utility for policy decisions (Nafzinger and Rath, 1981). Their conclusions are reflected in the following excerpts from their report: We conclude that little information is provided in the summative evaluation that is useful in making decisions about the further use, modification, or development of ECO [the Project Office] materials. Because of the large effort that the evaluation represents, we would have welcomed any opportunity to have reached a different conclusion.... At one level it appears that the evaluation has a clear purpose that is served by a large-scale experimental study. Upon further examination, the clarity of planning becomes blurred. First, there is no indication of the audience for the evaluation or the information needs of the audience. This results in a lack of a rationale for conducting the evaluation. Second, little thought has been given to the methods of Pilipino instruction which could be used in comparison with the RATES program. Only control groups and combinations of RATES with other Project Office programs were used. Thus, a tape-assisted approach (currently being used in many locations) and a textbook approach were overlooked as possible "competitors" to the RATES program. In short, realistic alternatives, one of which is being extensively adopted, were ignored, and no information about the costs or uses of alternatives were provided.... To surrarize, the evaluation is narrowly based, and there is little indication about the questions that were intended to be answered by the evaluation... this lack of clarity is reflected in an analysis plan that seems to lack any direction.... Because of the large investment in the experimental part of the study, scant attention was given to other important elements, such as qualitative information. Cost data were included in the evaluation, but the cost analysis did not provide information helpful to those who would be interested in adopting RATES.... (T]he evaluation gives us no information about who can and who cannot profitably adopt RATES and under what conditions. Neither do we receive information about how RATES differs from what typically occurs in the classroom.... According to the evaluation report and staff reports, clear contamination of control groups occurred. The control teachers - 234 - Educational Technology: Chapter 5 apparently were aware that they were part of a study, and they responded by trying to "beat" the treatment. To some extent, the control group became an alternative treatment comprising more motivated teachers. This occurrence demonstrates the lack of robustness of complex experimental designs.... It also demonstrates the need for evaluations to avoid relying solely on experimental studies and to adopt a flexible approach.... In comparison to the effort apparently required to develop the sampling plan, the thought and effort devoted to the development of a criterion measure seems minimal... .Our judgement is that the test is weighted so greatly in the direction of reading items that it does not reflect the heavy listening component of RATES, making it insufficiently content valid to be used as a sole criterion measure. These are harsh criticisms. As with all limited P RAot facto reviews, they do not -- and could not -- take into account the many contextual variables that contributed to the design and execution of the suimmative evaluation. Some of the observations seem in error; for example, textbooks as a competitor to radio were assessed. However, they do provide insight into the relative effectiveness of the summative evaluation reports in generating support for the pilot project. :;dgame Projects do not end abruptly. There is always a period during which the momentum engendered by project activity is gradually dispersed. It is during this period that the opportunity for organized learning from project experience is at its strongest. For "successful" pilot projects, this is the time when lessons learned are applied to implementation, usually on a larger scale, of some version of the program that was tested. For less successful efforts, it is a period of uncertainty, a time during which the ambiguity of outcome and process that characterized the pilot effort roll forward, complicating efforts to learn systematically from what was attempted -- and what was accomplished. The Bank loan for the radio pilot project formally closed in May 1982. In the five months after the project completion date, the evaluating agency completed its analysis and report on the second year evaluation. The Project Office transferred its materials and much of its equipment to the three regional education offices that would continue to use RATES and CET on a tape-assisted basis. Project Office staff dispersed, many going to the Development Academy of the Philippines, where they formed the core staff for the Educational Reorientation Project (ERP). A significant in-service teacher training component of PRODED, this project became a principal repository of the expertise in media-assisted education developed by the Project Office. The three regional projects, along with ERP, constitute an important level of institutionalization of the resources developed through the pilot project. Direct efforts to have RATES and CET included in PRODED continued, but at a very low level. The new Minister of Education had not changed his mind about radio education. There was some hope that when the Instructional Materials Development Corporation was established it would provide a mechanism through which radio materials might be developed for wider use. - 235 - Educational Technology: Chapter 5 summary A review of the period beginning with the pilot year and ending with the closing of the project leads to the conclusion that, intentionally or not, the Bank gradually abandoned the pilot project after an initial period of great enthusiasm and significant support. The continuity represented by key Bank staff working closely with project managers and evaluators began to erode as other duties increasingly prevented the participation of both. The one staff member able to maintain contact with the pilot remained a supportive presence, but his departmental placement limited his ability to influence the principal Bank decision makers. From midway in the project, key Bank missions to the Philippines did not involve either of the two Bank staff most closely involved with the pilot project. The final two missions were staffed by individuals with very little familiarity with the project and its history. The reports of these missions, given the limitations noted, do not show much learning from the pilot project experience. In the endgame period, decisions had already been made regarding the role of radio in PRODED. Radio was not included. No mechanisms existed in the Bank, or in the Philippines, to keep the project visible other than the continuing interest of key actors, notably the Pilot Project Director and, to some extent, the Evaluation Director. The project was designed to culminate with decisions following the pilot summative evaluation. The subsequent year was in the plan primarily to enable the project to move forward as part of the five-year educational development effort. The design, based on optimism and the expectation that the pilot would be incorporated into the educational system, made no provision for reviewing and learning activities in the event that the pilot year should fail to have the desired impact. In consequence, little was done. The decline in Bank interest and involvement has been discussed. On the Philippine side, there was little formal interaction in the endgame period beyond that necessary to meet administrative competition requirements. The Project Completion report was finished and submitted. Copies were sent on a limited basis to key individuals. There were no seminars, no panels, no inquiries. In the Philippines, as in the Bank, the project appeared to fade away quietly, with little remaining as a result of the great energy and commitment that had been expended. III. Patterns and Lessons Several clear patterns emerge from the story of the Philippine radio education pilot project. These help us understand what happened and why. In turn, within the limits of case study analysis, they enable us to draw lessons from the experience that may help with similar future projects. - 236 - Educational Technology: Chapter 5 Patterns A large-scale pilot project was established to assist with decisions on alternative ways to strengthen basic education in the Philippines. Yet the results of the pilot seemed to have been little considered when those decisions were made. Why this occurred is the fundamental question addressed in this study. The patterns drawn from the project history should help us in reaching answers to that question. As is often the case, there is no single -- or simple -- answer to the question. Rather the outcome followed from the interplay of numerous circumstances, actions, and decisions. These influences fall into three major categories: the design of the pilot project, both with respect to implementation and evaluation; decision-making structures and processes; and interactions among key individuals and agencies engaged in the effort. No single category suffices as an explanation. Taken together, they provide at least a partial answer. Design. From the point of view of the World Bank, the pilot was primarily a research effort to answer this question: "How do CET, RATES, and additional textbook availability -- introduced separately or in combination - - affect student performance?" Philippine leaders saw the project as providing services to education and contributing toward the development of institutional capability in broadcast education. These objectives were compatible. Yet neither approach focused on the principal question of interest for long-term educational development in the Philippines, which might be phrased as follows: How cost-effective is radio as an alternative delivery system for rural primary education in the Philippines? If the pilot program had led to a clear answer to this very different question, decisions on continuing radio education might have been more easily made. A clear and positive answer might well have led to incorporation of radio strategies; a clear and negative answer would have dispelled some of the lingering doubts as to whether or not the pilot had served a useful purpose. But the pilot was not focused on this question. The intervention itself was designed to answer numerous other questions. Thus, instead of a simple intervention design focused on radio as the variable, there was a complex design examining the impact of radio in direct instruction for Pilipino in the classroom, in teacher training in several subject areas, and in interaction with other factors. The analysis design prevented comparison of any outcomes with standard classroom practice, further weakening the design in answering the principal question. rne large-scale, multi-treatment field experiment appears to have been the model of choice for two reasons. One was an overt desire for a high level of "scientific, certainty in conclusions. A more implicit reason was the desire to contribute substantively to the growing world body of knowledge on radio education. Had it worked perfectly, this pilot could have been a landmark effort, advancing knowledge of educational radio on several significant dimensions. It would have provided valuable information of the combination of radio teacher training with direct classroom instruction and on the use of radio in combination with new textbooks. High levels of certainty - 237 - Educational Technology: Chapter 5 in conclusion, of course, would have strengthened these contributions significantly. The sheer size and complexity of the project needed to carry out the design consumed the managerial and time resources of the implementers. Neither the Project Office nor the evaluating agency could do much beyond fulfilling the original plan, particularly given the difficulties encountered in procuring vehicles and equipment. This latter problem greatly affected the efficiency of the implementing agencies. The agency responsible for the evaluation, the Institute of Mass Communication, is a mass communication research agency, but the project involved an educational intervention. Many of the components of evaluation of this type differ from mass communication research, for example, achievement test construction and classroom observation. While there is little doubt that the evaluating agency could have developed competencies in these areas, the struggle to carry out the basic plan left little room for growth of this type. An example of educational data that would have been useful is the amount of time students spent studying Pilipino in the different treatment and control groups. Time on task is a key variable in learning, regardless of medium. Data on this variable might help explain such apparent anomalies as CET and a 2:1 textbook ratio leading to higher learning gains than CET and a 1:1 textbook ratio. Random classroom observations and interviews with students, teachers, and parents could have provided insight in this area. Classroom observation was a feature of the third year evaluation, but had not been done by the time of the decision year. The many treatments led to test instruments that sought to measure both reading and listening skills; this was necessary because of the inclusion of textbook treatments. But reading items and listening items may have been imbalanced, and instruments may not have been sufficiently sensitive to radio effects. The development of very good tests is a difficult and time- consuming task. The size and complexity of the design may have resulted in less attention to this component than was needed. The Project Office conducted a number of small independent studies during the pilot year. One, for example, assessed the impact of tape-assisted CET lessons for teacher training and found good results. A simpler summative evaluation design could have freed human and fiscal resources for a larger number of smaller, decision-focused studies of this nature, yielding a richer base of information for decision making. The size and complexity of the pilot, in combination with a lack of focus on the principal question, led to a situation in which evaluation findings were equivocal. Equivocal findings, in turn, weakened the position of project advocates with respect to the decision-making process, thus compounding the absence of formal decision-making structures. Without the serious disruptions in broadcasting during the pilot year, outcomes of the evaluation might well have been quite different. With more consistent and higher quality radio reception, impact might have been stronger. The summative evaluation design, whatever its weaknesses, would have captured this. Clearer and larger differences between radio and non- radio treatments would have lent strength to arguments for continuation and - 238 - Educational Technology: Chapter 5 expansion of the project. Other weaknesses, such as control group contamination, instrument validity problems, and lack of cost-effectiveness data for standard classroom practice, would have been raised merely as issues that could have kept the evaluation from showing an even stronger impact. Viewed from this angle, a fundamental weakness of the project lay in the optimism regarding implementation. Pilot projects everywhere test both the concept of the intervention and the mechanisms for its implementation. Pessimism on both counts may be the more appropriate initial stance. When the experimental design is relatively simple, more resources can be focused on key conceptual and operational linkages in the intervention. Most crucially, more time and energy can be devoted to ensuring that the intervention takes place in the way it was intended. Unless this is ensured, the evaluation is ambiguous at best. Decision-Making Structures and Processes. No formal mechanisms were built into the project to ensure that project outcomes actually would be used in reaching decisions -- in this case, decisions about the shape of continuing educational development in the Philippines. When the project was planned, it appears to have been assumed that the key actors and decision structures present at inception would continue through the pilot year and still would be in place when results became available. This assumption, of course, turned out be to ill-founded. There were significant personnel changes in EDPITAF and the Ministry. The key Bank official in the design of the project evaluation became increasingly removed from its progress and outcomes. The broad Philippine agency involvement that characterized the initiation of the project faded with implementation, in part because the focus of the pilot was narrower than the initial plan and of less interest to the Government. The single project mechanism that might have served to focus decision making on outcomes -- the Project Advisory Group -- was too narrow in membership to create widespread and continuing interest in the project during implementation, and it disbanded just at the time when it would have been most useful. EDPITAF, in which the Project Office was based, went through complete reorganization precisely at the time when stability was needed to ensure attention to the radio project. Project supervision in the Bank was located outside of the department charged with the larger questions of educational development in the Philippines. This distance, along with the small size of the pilot, led to a situation in which the effort received relatively little attention during implementation. In the Bank, as in the Philippines, there was no recognized and persistent mechanism for integration of the pilot into ongoing decision making. At the stage when the shape of PRODED was being determined, the lack of formal mechanisms combined with personnel changes led to a situation in which the pilot had no senior advocates able to obtain thorough consideration of project outcomes. The project had become, in effect, an orphan. In other projects in other circumstances, of course, the particula-s differ. But there is always the likelihood of changes taking place in key institutions and individuals, and these changes are often unfavorable for the - 239 - Educational Technology: Chapter 5 pilot project and its outcomes. For this reason, it is important to rely on explicit decision-making structures rather than counting on a great deal of continuity in people and organizations. Interactions. In the Philippine project, the overall pattern of interaction among key individuals and agencies was one of increasing distance over time. The pilot began with very close collaboration between the Bank, EDPITAP, the evaluating agency, and the key Philipplne agencies. Much of this collaboration wae supported by close interpersonal contact among staff in the Bank and in the Philippines. This pattern began to break down almost as soon as implementation began. The World Bank evaluation expert's participation dwindled over time, leaving the broadcasting specLalist as the principal linkage. He had little influence over the fundamental dimensions of the pilot design. With the creation of ECO as the Project Office, direct involvement of senior EDPITAF leadership lessened gradually, then more sharply iwhen the original Director left the agency. Differences between the Project Office and the evaluating agency over the summative evaluation design emerged within the first six months of the project and were exacerbated by administrative difficulties. With the evaluating agency leadership worklng directly with the Bank's Evaluation Expert, Project Office staff lost any substantive control over the evaluation. What began as a collaborative relationship became an arms-length, formalized pattern of interaction. In addition, the changes through which EDPITAF was passing made the pattern for utilization of the evaluation unclear. Not only did the Project Office advisory group phase out when it might have helped in developing the formal review of the evaluation, but the relationship between EDPITAF and the Ministry of Education was in flux. As the Ministry assumed more direct control, EDPITAF was less able to bring issues forward strongly. Close collaboration between Bank and Philippine staff was important to the initiation of the pilot project. However, it may have established a pattern in which significant modifications in the pilot design could come only with direct participation of Bank staff, particularly the evaluation expert. This pattern further reinforced the rigidity of the orlginal design and the Project Office's lack of influence over the directions of the summative evaluation. It helps explain, for example, why the issues of collecting cost data on standard classroom practice and relevance of learning gain tests to radio teaching were never resolved. Summary. The significant patterns in the project can be summarized as follows. Hampered by a lack of clear focus on the most relevant questlon, an absence of formal decision-making structures, an extremely complex intervention and evaluation design, and a lack of continuity of key personnel, the pilot project nonetheless succ#eded in developing a large body of educational material and demonstrating that radio and cassette education was feasible and had some impact in the Philippines. For the pilot project to have been judged a success, all or most of these patterns would have had to be different. Formal decislon-making mechanisms alone, for example, mlght have helped with the problems of - 240 - Educational Technology: Chapter 5 continuity of personnel and distance between agencies, but probably could not have entirely overcome problems resulting from the comploxity of the design and lack of focus. A clearer focus might well have led to a simpler and more effective design; a simpler design would have facilitated more flexibility and perhaps greater impact and less equivocal findings. But without continuity of personnel and formal decision-making mechanisms, this still might not have resulted in the adoption of radio as a viable alternative for educational development. Lessons Case studies are, by definition, limited to a specific phenomenon ir a given place at a given time. The validity of the insights they generate is limited to similar phenomena under similar circumstances. The transfer of insight from a case to another situation requires an act of interpretation in which the context of the case is compared with the context of the new situation. Given these caveats, there are lessons in this case for the design of pilot projects. They are most immediately relevant to donor agencies and their partners in developing countries: a principal contextual variable in this case is the partnership between the Bank and developing nation agencies. Some of the lessons apply to situations involving multiple agencies; others may be more generally utseful in pilot project des'gn even when a single organization is involved. 1. Pilot Rrolects shou0 d be clearly focused on policy decisions. The lack of clear focus in the Philippine radio pilot had significant impact on the nature of the pilot design. Although the evaluation question was relevant, it was not asked in a way that centered attention on radio in comparison with alternative delivery systems. This led to a diffuse and far too complex design for the intervention and the evaluation, which in turn diffused energies and attention. 2. kilot projects should be no more comRlex than necessary. A clear corollary of sharp focus is simplicity and economy in design. Such simplicity allows the flexibility needed to adjust to changing circumstances and to maximize the potential of learning from the pilot experience. The complexity of the design in the Philippine pilot clearly hampered fley!.bility of this kind. It is generally u-L.iderstood that the technical sophistication of any project should not exceed the competence of implementing agencies. The Philippine case suggests that, in fact, technical demands of pilot projects should fall well within the demonstrated capabilities of implementers. The special challenges to flexibility and management imposed by pilot projects consume significant amounts of time and energy. To the extent that technical demands strain agency capabilities as well, somethin& has to give -- either flexibility or technical excellence. This seems particularly true where the time available for the pilot is constrained by a rigid decision schedule. The Philippine agencies were extremely competent and skilled and would have been able, given a flexible time schedule, to identify and deal with - 241 - Educational Technology: Chapter 5 problems both of design and implementation. But because of the complexity of the design, such flexibility was not possible. 3. ConceRtual modelling of both the intervention and the evaluation ls essential. In the Philippine pilot, much more attention was given to evaluation design than to modelling for the intervention. A number of key assumptions in the expected chain of causality linking radio intervention and improved student learning were not examined. For example, it was assumed that a semester of radio lessons would change teacher classroom performance to such an extent that student learning would improve significantly, yet there was no mechanism to assess change in teacher behavior in the classroom. 4. Mechanisms for decision makine should be an integral Rart of project designs. Individuals and organizations change. Pilot projects should have strong internal and external mechanisms which enable them to adjust to these changes as they occur. Such mechanisms, moreover, facilitate the ongoing education about the project -- its aims and its methods -- which supports informed decisions at the end. The absence of such mechanisms clearly contributed to the fact that little use was made of the Philippine project evaluation in decision making. Where donor and implementing agencies are involved, mechanisms need to be in place in both. They were lacking on both sides in the radio pilot effort. 5. Agency roles should be clearly defined. In any situation with different agencies sharing responsibility, roles and relationships should be clearly defined. It would be simplistic to say that a single implementing agency should always control evaluation or that evaluation should always be separately managed. It does seem clear that these role relationships should be well specified and understood. This is particularly true where a donor agency with technical competence chooses to be involved in technical and substantive decisions. The influence of the funding agency should not be underestimated. If the agency chooses to have a substantive role, it should accept the concomitant responsibility for continuity, leadership, and, eventually, project outcomes. It should be especially careful in situations where there are multiple implementing agencies, reinforcing established role relationships. 6. Implementation needs as much attention as evaluation -- perhaps more. It may be that suzmmative evaluation should not be attempted until all elements of the pilot have been demonstrated to function effectively. In the Philippine case, for example, summative evaluation and the initiation of broadcasting commenced at the same time. Weak broadcasting led to a weak treatment, contributing to ambiguous outcomes. Problems in procurement of vehicles and other equipment in the Philippine project impaired project implementation. Ensuring smooth implementation before the summative evaluation would often lengthen pilot projects somewhat, but it would be worthwhile in reducing the likelihood of ambiguous results. In some cases, a - 242 - Educational Technology: Chapter 5 clear focus and simplification would provide enough time savings to offset the extra time involved. This study was prompted by concern within the World Bank that it might not be the ideal agency to carry out small-scale, innovative projects. Other donor agencies have the same concern. The study suggests that, to the extent that Bank organization and procedures make it difficult to observe the lessons derived from the case, the World Bank is less than effective as a partner in such an enterprise. There are aspects of Bank operations which, on the surface at least, would seem to make pilot projects problematic. Small projects tend to receive secondary attention. The necessary continuity and depth of involvement of agency staff may be difficult to achieve in the World Bank. The high technical sophistication of Bank staff may tend to lead to very complex designs aimed at generating findings of broad research and policy interest; such designs may be less conducive to finding out what the country needs to know for its specific situation. On the other hand, there is no reason why the Bank or other donor agencies could not counteract these tendencies and be effective in supporting pilot projects in introducing educational technology and in others areas. The question is not the capability of supporting pilot projects, but rather the degree to which an agency wants to take an active role in generating innovative solu" ton.s to development problems. If a donor agency like the World Bank chooses to be active in this way, then pilot tests of innovations are an important tool to that end, and agencies should organize accordingly. - 243 - Educational Technology: Chapter 5 Ref ereces Friend, J., B. Searle, and P. Suppes, ads. 1980. Radio Kath mices in Nicaragua. Stanford, Calif.: Instituto for Mathematical Studies in the Social Sciences, Stanford University. Imhoff, M., and P.R. Christensen, ods. 1986. T2aching English AX Radio. Washington, D.C.: Academy for Educational Development. Institute for Mass Communications. 1982. SMnative EvaluatIon of the Pilot Prolect on Communication Technolog faor Education in the Philiggines:__Final Report. Manila: University of the Philipplnes. Middleton, J. 1985. "The Philippine Radio Education Project: A Case Study of Decision, Design and Interactions." In B. Searle, ed., Evaluation in World Bank EduCation Projects: Lessons From Three Cgse Studies. EDT Discussion Paper No.5. World Bank. Washington, D.C. Nafziger, D.H., and R. Rath. 1981. "Site Visit Report to the Educational Communications Office of EDPITAF, Government of the Philippines." Northwest Regional Education Laboratory. Portland, Ore. Processed. Office of the President of the Republic of the Philippines (Interagency Task Force to Study the Use of Communication Technology for Education) 1975. Renort: _Workshoo on Communication Technology for Education in the Philigpnne. Manila. - 244 - Educational Technology: Chapter 6 Chapter 6 A Comparative Study of Open Universities in Thailand and Indonesia David N. Wilson - 245 - Educational Technology: Chapter 6 A COMPARATIVE STUDY OF OPEN UNIVERSITIES IN THAILAND AND INDONESIA David N. Wilson While the British Open University, established in 1969, was not the first institution of its type, its unique combination of structural and functional attributes was a new model, which has subsequently been successfully transferred to other nations. Earlier innovative post-secondary distance education institutions existed in both developed and developing nations, but most of these earlier institutions had either evolved into more traditional universities or disappeared entirely. From a comparative perspective, this chapter examines the planning, development, and operations of the Sukothai Thammathirat Open Univs rsity (STOU) in Thailand and Universitas Torbuka (UT) in Indonesia, which admitted students in 1980 and 1984 respectively. This comparative examination includes the following aspects of the two open universities: the structures, administration, staffing, enrollment, course operations, curricula, lesson preparation, lesson distribution, study centers, course script evaluation, use of media, relationship of programs to national development objectives, satisfaction of social demand for post-secondary education, and utility of this innovative model for other developing nations. In addition, the chapter traces the divergence of these institutions from the United Kingdom Open University (UKOU) model upon which they are based, in order to examine their structural and functional differentiation and adaptation to the unique requirements of developing nations. The transfer of the innovative open university model from its precursor in the United Kingdom provides the framework to compare the similarities and differences between such institutions in developing nations. Identifying the structural and functional differentiations by which the model has been adapted to the conditions of Thaila-nd and Indonesia adds to our understanding of the institutional transfer process. The lessons learned are of considerable benefit to both donor and developing nations for guidance in future aid and development policy formulation. The study of such innovative approaches to the satisfaction of soc_.l demand for post-secondary educational opportunities in developing nations is warranted by the phenomenon of unsatisfied demand, resulting from the expansion of access to primary and secondary education and the slower expansion of places in post-secondary education. The high cost of providing each place in post-secondary education -- about 3,000 times the cost of providing each place in primary education -- also makes the examination of less costly alternatives important. - 246 - Educational Technology: Chapter 6 Historical and Institutional Background In a comparisc' -f open university systems, Hill (1984) noted that a number of institutions in erance, the U.S., and the U.S.S.R. had previously used media in distance education at the post-secondary level. However, it was noted that until the British Open University integrated media usage with other innovative attributes, the logical next step failed to take place: ... although there was a recognition of the part to be played by correspondence work and face-to-face meetings with teachers ... no extension had occurred into an open university system characterized by the availability of a number of full degree options, the operation of closely coordinated media and correspondence packages and the use of modern technological aids such as computer marked assignments (Hill, 1984, p.143). From our previous knowledge about how models perceived to be successful were transferred from one national milieu to another it is possible to postulate that the establishment of a successful open university model in the U.K. paved the way for the transfer/adoption (and adaptation) of that model in other nations. Such prestige considerations are extremely important and may also account for the failure of earlier innovative open universities. Hill (1984) observes that after the British Open University pioneered its successful combination of attributes, the "successful and well publicized development of a prototype new institution" stimulated "the imagination of those responsible for forward planning" (p.143). This stimulated imagination led to the appearance of open universities of varying types in numerous countries: Spain and Canada in 1972; Germany, Israel, and Pakistan in 1974; Costa Rica and Venezuela in 1977; Thailand and Iran in 1978; China in 1979; Sri Lanka and Korea in 1981; Nigeria, Indonesia, and the Netherlands in 1984; India, Japan, and Jordan in 1985; Taiwan in 1986; and Portugal in 1988. The Nigerian and Iranian open universities subsequently closed, but open universities are being planned in Tanzania and Bangladesh. Therefore, there are 11 open universities in developed nations and nine currently operating in developing nations (Dodd and Rumble, 1984; Wilson, 1990). Further, it is also suggested that the technologies and open university features that were available in the U.K. were not readily available in other nations, or at leact not in the gombination that proved successful in the U.K. In this respect, it is interesting to compare the presence and absence of technologies and attributes in the various open universities that have developed since 1970. Table 1 juxtaposes these data for comparative purposes. The column headings refer to features that open universities may or may not have. QRDn entry indicates that applicants need not have conventional matriculation requirements. Media usage refers to the teaching of courses through correspondence, radio and television, or through alternative media such as audio and video cassettes (these are disaggregated in Table 2). Course teams consist of a cross-section of specialists (subject matter specialists, media producers, educational technologists, tutors, editorial staff) who prepare easily understandable learning modules for students. Study centers are - 247 - Table 1: Comparison Of Open University Attributes Open Media Course Study Part-time Residential Year Ertry Usage Teams Centers Faculty Courses Opened Canada (Athabasca University) x x x 60 - x 1972 Canada (Tele-Universite) b X x 5 x _ 1972 China (CRTU) - x a 500+ x - 1979 Costa Rica (UNED) - x x 29 - x 1977 Germany (FernUniversitat) - x - 45 x 1974 India (Indira Ghandi National OU) b X x 95 - - 1985 Indonesia (Universitas Terbuka) - x x 88 x 1984 Iran (Free U.) - closed - x x 20 x 1978 Israel (OU) x x c 35 - x 1974 Japan (U. of the Air) - x x 6 x x 1985 Jordan (Al-Quds OU) - x x N.A. x - 1985 Korea (Air and Correspondence U.) -x x x 1981 Netherlands (OU) x x x 18 x - 1984 Nigeria (OU) - closed - x a 24 x - 1984 Pakistan (Allama Iqbal OU) b X - N.A. x - 1974 Portugal (Universidad Abierta) - x x N.A. x - 1988 Spain (UNED) - x 50% 72 x - 1972 Sri Lanka (OU) x x x 10 x - 1981 Taiwan (National OU) - - x N.A. x - 1986 c Thailand (Sukothai Thammathirat) - x x 77 x 1978 a United Kingdom (OU) x x x 260 x - 1969 r Venezuela (U. Nacional Abierta) - x x N.A. - x 1977 0 9,_ '-3 OU - Open University 0 N.A. - Information not available. a The CRTU in the People's Republic of China uses the same course texts as conventional universities but 0 study guides are compiled by teams. 0 b Conventional entry qualifications for some programs and open entry for other programs. c The Israel Open University has a Writing Team and an Operational Team. d The Korean ACU has no study centers but arranges lectures at 50 cooperating universities. Table 2: bnstrctional Media Usage At Open Univerites Cofespdece Telephone Audio/Visual Radio Tdeion Texts Teacig Contact Cassettes Lssons Lessons Cnada (Athab U x x x - Canada(Tele.Unveste) x x x - x Ciina (CRTU) x - - - x Costa Rica (UNED) x x x x x x G any (FnUnestat) x x x - X India ( aidiraGandiNational OU) x x - x x x Iadonesia (Universitas Terbka) x x - x x x Iran(FreeU.). dosed x x - x x x Ise (OU) x x x x Japan (U.of Air) x x - x x Jordan (Al-uds OU) x x - x x - AKorea (ACU) x - - x x - Netherlands(OU) x x - x - x * Nigeria (OU) - closed x - - x x x Pakistan(AIamalqbalOU) x x - x x x Portugal (Universidad Abierta) x x x x x x Spain(UNED) x x x x x x Srelana (OU) x x - x x x Taiwan (National OU) x x x - - Thailand (Sukothai lanaat) x x - x x x Umlted Kingdom (OU) x x - x x x Venezuela (U. Nacional Abierta) x x x . x OU = Open University Sources: Dodd and Rumble, 1984; Aitken, 1986; The World of Learning 1986; ST1OU, 1987; Tank, 1986; Wison, 1990. Educational Technology: Chapter 6 regional or local satellites of the main universities where study groups, lectures and tutorial sessions can be held. While instructional media usage appears to be a universal attribute at all open universities, this general criterion conceals a wide variety of arrangements. Therefore, Table 2 disaggregates the media category to further refine our comparative examination. From Tables 1 and 2 it is clear that all open universities utilize "paper-and-pencil" textual materials. In support of this universal feature, most open universities have their own university press to publish texts, workbooks, correspondence materials, and examinations. The open universities in Thailand, Indonesia, China, Sri Lanka, Pakistan, and Venezuela have not established telephone contact between students and instructional staff, because the availability of home telephones is limited and distances constitute an additional barrier. Residential courses are offered by open universities in Iran, Israel, UNA Venezuela, and UNED Costa Rica. With the exception of the Chinese Central Radio and Television University and Universidad Nacional Abierta in Venezuela, most open universities use audio (or video) cassettes, subject to the availability of video playback units (Hill, 1984). With the exception of Athabasca University in Canada, Israel Open University, Sri Lankan Open University, Open Universiteit in the Netherlands, and the Central Radio and Television University in China, the attribute of open entry has not been widely replicated. This reluctance to adopt the notion of open entry in nations with more traditional university heritages and with attention paid to international standards is understandable. Open entry is probably the feature of the open university model that departs most markedly from the traditional university model. As such, it is a feature that could be viewed by some in developing natiorAs as indicating second-class status for universities. Hill (1984) aptly relates the origin of the open-entry feature to the favorable conditions in Britain during the late 1960s, as follows: What first attracted the world's attention to the British Open University was tae concept of oper entry. The strong personal conviction of Jennie (now Baroness, Lee contributed greatly to this approach. The notion that with careful guidance prople without conventional matriculation qualifications could successfully study at university was a striking one which seemed to challenge many cherished views on the background necessary before university studies could be attempted (p. 144). Although the open-entry innovation was not adopted by either Thailand or Indonesia, both adopted the course team feature of the U.K. Open University. In fact, all other open nniversities except FernUniversitat in Germany, Korea Air and Correspondence University, and China's CRTU have adopted the course team approach to staffing and instruction (Hill, 1984). One problem that has plagued the U.K. Open University since its inception appears to be shared by open universities in other nations; this is the problem of accegs to radio and television time. Hill (1984) notes that - 250 - Educational Technology: Chapter 6 tension exists in the relationships between the UKOU and the BBC. Similarly, Tank (1986) wrote that tLh most pressing problem facing Universitas Terbuka was communication with its students. Indonesia has just one television channel and the authorities have offered the Universitas Terbuka just 25 minutes of prime time a fortnight. So Professor Setijadi is encouraging the use of audio cassettes on the principle that the most remote village will have access to a tape recorder (p.7). The CRTU in China uses television, but not radio (in spite of the CRTU title), while the Sri Lankan Open University uses radio, but not television. UNESCO (1989) notes that STOU solved the problem of access to television by establishing its own TV station, Channel 11. However, Hill (1984) observed that "open universities.. .followed the practice that to be pedagogically effective the electronic media.. .must be used in a support role to correspondence tuition and must not be the sole channel of learning" (p.146). Thus, reliance on access to transmitting media must yield to alternative media in countries with limited, or tightly controlled, broadcast facilities. Audio and video cassettes appear to be the most viable alternatives in both developed and developing nations, subject to the availability of playback equipment. In his comparative article, Hill (1984) concluded that the three innovations that achieved a successful combination at the British Open University were: (1) open entry, (2) the use of media, and (3) the course team concept. As noted above, the concept of open entry was adopted by neither Indonesia nor Thailand, while the other two innovations were incorporated. This historical overview of the development and trarsfer of open university models now sets the stage for our detailed examination of Sukothai Thammathirat Open University and Universitas Terbuka. Sukothai Thammathirat Open University STOU had been three years in the planning when it admitted its first 82,139 students in 1980. Unlike traditional universities, which are organized along faculty and departmental lines, STOU is comprised of major schools, each of which has its own board of studies with a chairman and three to seven members. These schools reflect the subjects that are considered related or essential to national development: educational studies, management sciences, liberal arts, health sciences, law, economics, home economics, political science, agricultural extension and cooperatives, and communications arts. Lacking any centralized infrastructure, STOU relies upon regional and local study centers for counseling and tutorial facilities and upon distance education techniques -- correspondence, textbooks, radio and television broadcasts, handbooks, and audio and video tapes -- for delivery of information. - 251 - Educational Technology: Chapter 6 Enrollment and Staffing In 1985 STOU reported an enrollment of 150,822 students -- P8,10C males and 62,719 females -- who were serviced by 162 full-time and l,700 part- time instructional personnel. Admission requirements are a secondary school certificate (Mathayon Suksa 5) or equivalent. Tuition is 2,400 baht per annum, which is equivalent to US$ 90 (Aitken, 1986). Tables 3 and 4 indicate that the largest component of early STOU enrollment and instructional staff was in the education field. STOU's experience suggests the possibility that other open universities in developing nations may find an important role to be the in-service upgrading of teachers dispersed throughout the nation, who would otherwise have little access to formal post-secondary educational opportunities. Another of the early programs was construction management, which is also a skill in high demand in developing nations. It appears, therefore, that the stated commitment to offer courses relevant to national development is more than a rhetorical one. Unfortunately, the high enrollment in law appears to replicate one of the more dysfunctional tendencies common to developing nations. As indicated by the data in Tables 3 and 4, the School of Liberal Arts does not enroll students directly, but rather performs a service function to the other nine faculties. Part-time staff increased from 110 tutors and co-writers in December 1980 to 835 tutors and 236 co-writers by 1983 (STOU, 1983a). - 252 - Educational Technology: Chapter 6 Table 3 STOU Enrollment By Programme: 1980-1983 1980 1981 1982 1983 Certificate of Education (1 year) 1,729 1,148 1,167 n.a. Primary Education (4 years) 3,634 1,850 1,733 n.a. Primary Education (2 years) 29,960 16,502 17,427 n.a. Secondary Education (4 years) 1,545 816 702 n.a. Secondary Education (2 years) 23,115 14,222 12,511 n.a. Educational Administration (2 yrs) 19.115 12.474 12,439 n.a. TOTAL Educational Studies 75,234 47,012 45,979 39,950 Construction Management (2 years) 6,805 5,239 6,041 n.a. Business Administration (4 years) - - 6,946 n.a. Business Administration (2 years) - - 7.638 n.a&_ TOTAL Management Science 6,805 5,239 20,625 21,731 Law (4 years) - - 21,097 n.a. Law (3 years) - - 8,864 n.a. TOTAL Law - - 29,961 30,392 Public Health Management (4 years) - - 3.990 4.078 TOTAL Health Science - - 3,990 4,078 Economics (4 years) - - 1.553 2.558 TOTAL Economics - - 1,553 2,558 Community Nutrition (2 years) - - 2.335 3.841 TOTAL Home Economics - - 2,335 3,841 Agricultural Extension (4 years) - - 1,253 n.a. Agricultural Extension (2 years) - - 3,519 n.a. Cooperatives (4 years) - - 448 n.a. Cooperatives (2 years) - - 159 n.a. TOTAL Agricultural Ext. and Cooperatives- - 5,379 4,941 TOTAL Political Science - - - 4,092 Communications (Began 1984) - - -- …-----------------------------------------_ ....----.---.--------......-. ZQTIL 82,139 52,251 109,842 111,583 Sources: Popa-Lisseanu, 1986; STOU, 1983a. - 253 - Educational Technology: Chapter 6 Table 4 STOU Full-Time Staff: 1981-1983 1979 1980 1981 1982 1983 Office of the Rector 15 46 84 87 114 Office of Educational Services 2 14 27 39 53 Office of Educational Technology 2 30 80 106 155 Office of Registration Records and Evaluation 2 15 38 55 76 Office of Academic Affairs 2 7 25 50 66 School of Liberal Arts 11 11 14 20 27 School of Law 1 1 13 14 35 School of Economics - - 8 9 14 School of Health Science - - 6 10 15 School of Home Economics - - 5 8 11 School of Agricultural Extension and Cooperatives 1 1 3 3 19 School of Political Science - - 4 6 12 TOTAL 59 148 34 451 666 Source: STOU, 1983a. Structure and Administration The STOU academic year is organized in two semesters from July to October and December to April with a summer session during May to August (World of Learning, 1985). Founded in 1978 by Royal Decree, STOU is a state Institution under the supervision of the Ministry of University Affairs, as are all post-secondary institutions in Thailand. The STOU governing body is the University Council. The King appoints the chairman and four to nine members. One member is chosen by the Acadewic Senate. The Vice-Rectors for academic, administratlve, and planning and development serve as secretary and assistant secretaries. Ex officio members include representatives of the Mlnistry of University Affairs, the Post and Telegraph Department, the Mass Communications OrganLzation of Thailand, the Communications Authority of Thailand, and the Rector (STOU, 1983b). The Academic Senate is composed of the Rector (as chairman), the chairpersons of the individual schools, directors of institutes and offLces, and permanent faculty. One representative is also elected by the faculty of each school. Between 1978 and 1984, STOU operated from five temporary locatlons in Bangkok. In 1984 STOU moved into its permanent headquarters ln the Pakkred District of Nonthaburi Province, 25 kilometres from Bangkok (STOU, 1983a). Academle Operations - 254 - Educational Technology: Chapter 6 The STOU University Press publishes correspondence texts for its courses each semester and monthly newsletters (Focus on STOU) to keep students informed of relevant university matters (World of Learning, 1985). The STOU library system includes the central library, operated by the Office of Documentation and Information, plus a "STOU Corner" established in 72 provincial public libraries (STOU, 1988). In view of the dispersion of STOU students throughout Thailand, the University established a nationwide network of 77 local study centers that are located in existing educational institutions. These local study centers distribute textbooks, instructional materials, and audiovisual materials to students in their areas. At least one part-time instructor is assigned to each local study center to provide academic counselling and instructional assistance, as well as to monitor examinations. Academic tutors are recruited from faculty members of existing institutions of higher education in each region and number over 2,000 (STOU, 1983b). Instructional media consist principally of correspondence texts and other printed materials for home-based education. These paper-and-pencil media are supplemented with radio and television broadcasts, and video and audiotapes, using equipment available at local study centers. The STOU Educational Broadcasting Production Center produces 10,000 radio and 2,000 television programs dnnually. The latter are broadcast on Channel 11 in the Bangkok area and on 11 local TV stations of the Public Relations Department and the Mass Communications Organization of Thailand throughout the country (STOU, 1988; OLS News, 1987). Courses and Degree/Diploma Programs In 1988 the School of Liberal Arts listed seven foundation courses in English and Thai languages, mathematics, history, and sociology, plus elective courses to supplement major courses in other schools. Until recently, no degrees were conferred by the School of Liberal Arts as it constituted a service program for other schools. However, both certificate and bachelor's courses are now also offered. The BA programs offer majors in Thai studies, information science, and history. Bachelor's degree programs are offered in education, management science, law, health sciences, economics, home economics, political science, agricultural extension, and communication arts. All of these offer a four- year degree, and most offer other options as well. STOU allows students from four to 12 years to earn their degrees. The course descriptions suggest that STOU has tailored courses to the requirements of developing nations, while preserving the apparent integrity of degree programs. While the applied and national development focus suggested by course descriptions is viewed positively, the nature of course content deserves an in-depth examination; however, this is beyond the scope of the present study. - 255 - Educational Technology: Chapter 6 Budget For those accustomed to the astronomically high costs per student at most traditional Third World universities, the per-student cost of STOU operations appears extremely reasonable. Tuition and fees constitute the main source of STOU revenues. In 1985 these totalled 2,850 baht per annum per student: a university fee of 300 baht, tuition of 1,200 baht, an admission fees of 150 baht, and educational materials of 1,200 baht (for six courses). This income from students totalled 303,808,460 baht, or 81.6 percent of total STOU revenue, while the remaining 68,136,000 baht, or 18.32 percent, was derived from the Thai government budget (Srisa-an and Wangsotorn, 1985). Table 5 compares per-student operating costs at conventional Thai universities with those at STOU. Averaging these data yields a per-student cost of 15,828 baht for conventional universities, as compared with 4,665 baht (or US $173) at STOU -- a cost advantage of 3.4:1. Table 5 Comparison of Per-Student Operating Costs Open University as Discipline Type of University Percentage of Conventional Open Conventional University Business Admin. $14,942.07 $1,695.95 11.35 and Commerce Natural Science 19,778.15 972.72 4.91 Education 20,507.39 638.08 3.11 Social-Behavioral 13,435.97 591.84 4.40 Science Law 11,970.81 461.34 3.85 Humanities 14,332.56 305.36 2.13 Source: Srisa-an and Wangsotorn, 1985. The salary component was a lower percentage of recurrent (or operational) costs than in traditional universities -- only 57.14 percent in 1983. Capital expenditure consisted of 13,814,100 baht on equipment and 60,304,600 baht on capital construction between 1980 and 1983, probably to construct the new headquarters at Pakkred, Nonthaburi (STOU, 1983a). - 256 - Educational Technology: Chapter 6 From these data it appears that STOU is significantly more self-stupporting and cost-effective than most traditional post-secondary institutions in the Third World. The Thai government subvention averages 25 percent and the tuition and other fees appear to cover most recurrent, or operational, costs. Thus, this model addresses one of the major difficulties encountered by developing nations, the problem of support for a burgeoning post-secondary education sector. Universitas Terbuka The Indonesian Universitas Terbuka is younger than STOU, having opened in 1984 with 65,000 students selected from 268,000 applicants and housed in temporary accommodation at the IKIP Jakarta (Institute of Teacher Education). Planning for Universitas Terbuka can be traced to the wholesale movement of the Rector, Professor Setijadi, and his entire staff from the BP3K.11 This migration in 1983 brought a highly professional cadre of 35 officers and support staff to Training and Education Science) to commence formal planning for the open university. The UT planning phase received technical assistance from the United States Agency for International Development. Originally, two staff members from STOU were to assist in the planning of UT. This innovative measure, however, was frustrated by USAID regulations prohibiting the use of "third country nationals" on a technical assistance project. Had this not been the case, our study of the transfer of the STOU model to the UT might have been enhanced by directly attributable -- and, hopefully, documentable -- transfers. The UT planning phase was antedated by the establishment in 1981 on a pilot basis of diploma programs for the professional upgrading of secondary school teachers at IKIP Jakarta, using distance education techniques -- largely paper- and-pencil correspondence. Similarly, teacher certificate (AKTA V) courses for university lecturers, which also commenced in 1981 on a pilot basis, were facilitated by correspondence from IKIP Jakarta. These two pilot programs were absorbed by UT upon its formal creation 1984 (Daroesman, Teschner and Wilson, 1984). Thus, the first similarity between STOU and UT can be identified as their primary concentration upon the in-service upgrading of practicing teachers. The additional concentration of UT upon the upgrading of university lecturers at outlying Indonesian universities can be identified as a significant difference between the two open universities. Programs and CurrLcula During the 1988/89 academic year, UT offered degree programs in public administration, business administration, economics, development studies, applied statistics, science, mathematics, development administration, taxation, and education. Table 6 juxtaposes program offerings at STOU and UT for preliminary comparisons. I/ Badan Penelitian dan Pengembangan Pendidikan dan Kebudayaan: BP3K is the Educational Planning Division of the Indonesian Ministry of Education. - 257 - Educational Technology: Chapter 6 Table 6 Comparison of Course Programmes at STOU and UT STOU UT Common Programmes Education X X Management Science X X X Law X Health Science X Economics X X X Home Economics X Agriculture X Political Science X Communications X Public Administration X Development Studies X Development Administration - X Business Administration X X X Applied Statistics X Taxation x Mathematics X Science X It is feasible that the differences in programmatic offerings might be attributable to the difference in age of the two institutions, rather than any other factor. According to Tank (1986), "the curriculum offered by the Universitas Terbuka has been designed to complement existing universities in Indonesia. It was decided to offer a course in taxation, "but not ... one in law which is already well catered for" (p.7). This aspect of the UT planning and development process differs significantly from that at STOU, seemingly more competitive with other Thai universities, rather than complementary as UT appears to be. Moreover, this writer can only applaud the decision not to offer law courses at UT, particularly in view of the over-provision of this course at existing Indonesian (and most Third World) universities. This suggests that the UT planning process has paid more than mere lip service to the criterion of "national development requirements." Structure and Administration Universitas Terbuka was first conceived in 1981, but did not receive approval from the Directorate-General of Higher Education of the Ministry of Education and Culture until October, 1983. After only one year of planning, UT was launched in the fall of 1984 by Indonesian President Suharto. UT operates under - 258 - Educational Technology: Chapter 6 the direct control of the Directorate-General of Higher Education (DGHE) in a similar manner to Indonesia's 44 other public universities. In comparison, 60 percent of Indonesian university enrollment of over one million attend the 572 private universities that have proliferated in response to nearly open-ended demand. Indonesian universities only date from the establishment of Universitas Gajah Mada in 1949. UT administration comprises the Rector and three Vice-Rectors for administration, academic affairs, and plant and operations. The university is organized into four faculties: Economics and Development Studies, Social and Political Education, Mathematics and Natural Science Education, and Teacher Training and Educational Science. Each faculty has a dean (Universitas Terbuka, 1989). Although detailed information about UT governance is not available, Doeroesman, Teschner, and Wilson (1984) provide a description of public universities that is applicable to UT. Public institutions of higher education in Indonesia are not autonomous or semi-autonomous corporate bodies, but integrated parts of the ministerial hierarchy itself. Each institution is headed by a Rector directly appointed by the President of the Republic and is responsible to the Minister of Education and Culture. He is assisted by three Vice-Rectors (one each for academic affairs, student affairs and administration) and two administration bureaus. The faculties are headed by a Dean and three vice-deans, each of them nominated by the Minister of Education and Culture out of three-name-proposals submitted through the Rector by the senior faculty on the basis of an election. The Rector may consult the Senate, a non-structural academic body comprising the senior academic staff (p.15). These arrangements differ significantly from the greater autonomy prevalent in Thailand and elsewhere. They are, undoubtedly, attributable to the pre-independence Dutch traditions transferred to Indonesia, while other nations inherited the more liberal British traditions of university governance. In Mdy 1986 UT moved to its new permanent headquarters at Pondok Cabe, south of Jakarta. Initial problems of staffing, in a nation that has been critically short of qualified university faculty, were overcome by the part-time recruitment of existing university faculty from both public and private established universities. These faculty were initially charged with the preparation of self- instructional learning modules, as well as with tutorial sessions and the grading of assignments. All academic staff at UT continue to be employed at other universities and are engaged by UT on a freelance basis (Tank, 1986). Course Development The preparation of learning modules in an easily understandable format commenced in 1983. They consist of textbook abstracts and lecture notes, organized in a self-contained workbook that includes all necessary reading assignments. These materials are prepared by a course team comprising a subject matter specialist, a producer from RRI or TVRI (Radio and Television of Indonesia), a - 259 - Educational Technology: Chapter 6 tutor, an educational technologist, a subject matter coordinator, and production and editorial staff (Amidjaja, 1983). According to Tank (1986), the Institute of Technology at Bandung in Central Java developed the UT science program, which includes physics, chemistry, biology, and technology, with an emphasis on electronics. Some of the study centers now include laboratories to facilitate science experiments. Financial constraints have hindered the implementation of these programs, however, and electronics modules have yet to be developed. Operations Perhaps the most innovative aspect of Universitas Terbuka has been its contractual arrangement with the Indonesian Post Office to stock, distribute, and process admissions materials, lessons, and instructional materials, as well as return completed course scripts to the tutors (Daroesman, Teshner, and Wilson, 1984). Until about 1985, students had to travel to one of the 32 main post offices in Indonesia's 27 provinces. Beginning in 1986, new operational arrangements were made for course materials to be collected at local post offices. This enhanced strategy appears to have improved the innovative use of the post office system. However, the plan to undertake home deliveries was abandoned, due to the volume of materials and the inability of postal personnel to carry bulky items. In this regard, the author has stated for many years that in most developing nations the single national institution that pervades the viable geographic expanse of a nation has been the post office. The use of the post office for the two-way processing of distance education materials impressed this author when the plans were described to him in 1984. Another mode of distance education transmission used by UT has been national television. However, as noted earlier, the authorities have allocated only 25 minutes of prime time each two weeks on the slngle national television channel to the UT. The Rector announced that the UT would encourage the use of audio cassettes, on the assumption that even the most remote village has access to a tape recorder. A difficulty that plagued early UT students was that they were given little choice of course options. Now, according to Tank (1986), "students can pick courses, enroll when they want and take examinations when t1.ey think they can pass them," (p.7). The UT has established 32 regional offices and 88 study centers throughout Indonesia to stage two examinations each year. Study groups, lectures, and tutorial sessions are held at these centers and at regional universities during vacation periods. A full-time UT student can complete a degree in four years and obtain an honors degree by writing a thesis (Scrigsi) that demonstrates an acceptable level of scholarship. There is no limit on the length of time for completion of a UT degree. Although Indonesia inherited the Dutch university system, four decades of technical and capital assistance, largely from USAID and U.S.-based foundations, - 260 - Educational Technology: Chapter 6 has resulted in many American attributes and practices being superimposed upon the original heritage. Among these is the system of accumulating course rredits towards a degree, which was introduced in 1979 (Amidjaja, 1983). Also in 1979, the government of Indonesia introduced a new system of educational stratification. The four new strata are: Diploma (SO), Sariana (Sl), equivalent to a bachelor's degree, Pasca Sarjana (S2), or master's degree, and doctoral degree (S3). The Sarjana requires 140 credits and the honors degree 160. Enrollment Universitas Terbuka literally means "open university" and, as such, it is stated to be open to enrollment for any graduates of senior secondary schools above the age of 20. Although UT was expected to enroll 500,000 students per year bv 1990, problems with materials development, distribution by mail, and establishment of the 32 regional offices reduced these expectations. While total enrollment was listed at 137,000 in 1988-89, active enrollment in Sl (degree) programs totalled 80,171 in 1988 (Universitas Terbuka, 1989). The distinction between total enrollment and active enrollment becomes evident upon examination of the 1988 data provided by the Indonesian Ministry of Education and Culture (1989), which shows 9,117 remaining from the 1984 intake, 7,177 remaining from the 1985 intake, 4,209 remaining from the 1986 intake, 2,705 remaining from the 1987 intake, and 7,040 remaining from the 1988 intake. The first graduates in 1988 comprised 258 in the diploma and 185 in the Sl (bachelor's) programs in teacher training and educational science (FKIP). In 1989, 433 D-2 and 755 S-1 FKIP students graduated, together with 30 from the Faculty of Economics and Development Studies (EKON), 119 from the Faculty of Social and Political Education (FISIP), and two from the Faculty of Mathematics and Natural Science Education (FMIPA). The open-entry commitment constitutes an important one from a political perspective. In 1985, about 1 million Indonesian secondary school graduates sat the university entrance examination. However, the continuation rate has declined from a high of 56.4 percent in 1984 to 52.1 percent in 1988-89, and the number sitting the entrance exam declined to 480,000 in 1988. As Wilson (1988) suggests, the satisfaction of social demand for university places, through the expansion of the Universitas Terbuka, has become an important policy measure. Budget UT income from students includes Rp. 5,000 for the registration manual, 2,500 for the course catalogue, either 40,000 for students taking 12 semester credit hours or 60,000 for those taking 13 to 18 semester credit hours, and the purchase of course materials at regional office bookstores for either 3,000 for each semester credit hour or 20-25 per page (Universitas Terbuka, 1988). Therefore, for a UT student taking 12 semester credit hours the total cost would be Rp. 83,500 per year (equivalent in 1988 to US $50), or for those taking 18 semester credit hours Rp. 121,500 per year (equivalent in 1988 to US $72). Tuition fees at public universities range from Rp. 120,000 to 240,000 per annum, while those at private universities in Java can be as much as Rp. 1.5 million per year (equivalent to US $900 in 1988) (Pernia and Wilson, 1989). In comparison, the 1987 GNP per capita was US$ 450. - 261 - Educational Technology: Chapter 6 Upon the basis of available data it is appears that less than 30 percent of the cost of university operations accrues from tuition fees. In comparison to the STOU self-financing noted earlier, it can be concluded that UT operational costs are heavily subsidized by the government of Indonesia (Wilson, 1988). Institutional Comparisons Although the available data make it difficult to effect balanced comparisons between the Sukothai Thammathirar Open University and Universitas Terbuka, there are discernible similarities and differences that make comparisons feasible. The studies of the British Open University cited above provided range of structural and functional attributes for comparisons between the original model and two of its replications in developing nations. These attributes will be examined in appropriate groupings to effect comparisons between STOU and UT and where appropriate between them and the U.K. Open University. Structure, Administration, Staffing, and Study Centers All three institutions operate under the authority of their respective governments. While the U.K. Open University operates on the customary arms-length basis of the University Grants Commission, associated with the Higher Education Branch of the U.K. Ministry of Education and Science, STOU and UT are more directly governed by the Thai Ministry of University Affairs and the Indonesian Directorate- General of Higher Education of the Ministry of Education and Culture, respectively. However, only STOU and the UKOU have Governirg Councils, while UT is directly responsible to the Ministry of Education and Culture. Similarly, all three institutions have an Academic Senate, although the UKOU appears to devolve greater autonomy upon its senate than either STOU or UT. The autonomy of faculty and university bodies appears to have been influenced by the evolution and transfer of university tradition. The U.K. experience evolved along lines of academic freedom. Though this tradition has not always transferred to British-heritage Third World universities, Thailand has adopted much of the British influence and heritage. In marked contrast, UT inherited the Dutch-European heritage of direct control of universitier. Two of the three institutions are organized into faculties with deans as their titular heads. The 1988 Directory of the European Association of Distance Teaching Universities lists the UKOU as having five faculties: Arts, Mathematics, Science, Social Sciences, and Technology, plus Schools of Education and Management and a Center for Continuing Education. The STOU organization into schools parallels this faculty structure. However, the UT organization into course programs, under four faculties, appears to diverge from the more traditional faculty structure of UKOU and STOU. UT relies mainly upon experienced faculty from existing Indonesian universities, under the supervision of deans of education, economics, mathematics, and socia. science. With few permanent faculty, and the use of lecturers from other universities on a contractual basis, UT appears markedly different from other open universities. As noted above, Indonesia &.as been critically short of qualified university faculty, and the UT staffing and organizational divergence is, undoubtedly, an artifact of this shortage. - 262 - Educational Technology: Chapter 6 In comparison with UT staffing practices, STOU has 10 chairmen in its 10 schools, plus 133 full-time staff, 835 part-time tutors, and 236 part-time co-writers. The 1987 UKOU Statistics bulletin listed 46 professors, 126 readers and senior lecturers, and 240 lecturers with a total of 2.733 full-time staff. Academic year organization at UKOU is from January to November, with instruction during the first 10 months and examinations during October and November (World of Learning, 1985). In contrast, STOU is organized into two semesters from July to October and December to April, plus a summer session during May to August. The UT academic year commences with enrollment deadlines in mid-September, February, July, December, and June and with flexible course commencement times. The open-enrollment and examinations-when-desired policies suggest that UT is considerably more flexible than the other two institutions (STOU 1983b). Similarly, reliance upon 88 study centers throughout Indonesia and the use of regional university facilities during vacation periods also suggests that UT academic year organization is considerably more flexible than most other open and traditional universities. All three institutions are now located at permanent headquarters, following periods during which they utilized temporary accommodation. The UKOU headquarters is supplemented by 13 regional offices administering 265 study centers throughout the U.K. (World of Learning 1986). The STOU headquarters coordinates 10 regional offices and 77 local study centers. Similarly, the UT headquarters coordinates the 88 study centers and 32 regional offices throughout Indonesia. Therefore, it is clear that both STOU and UT hpve adopted the central headquarters, regional office, and study center organizational model pioneered by the UKOU. While STOU provides at least one staff member per regional and local study center, in conformity with the UKOU model, the central difference appears to be that UT does not provide full-time staff at its 88 study centers, relying instead upon freelance, part-time faculty from regional universities. Er.rollment, Admissions Policies, and Degrees After four years of operations, STOU had a 1985 enrollment of 176,987 students, while UT reported 138,148 students enrolled in 1988, in its fourth year of operation. In contrast, UKOU opened in 1969 and enrolled 96,224 part-time students in 1988 (Wilson, 1990). Both STOU and UT offer only certificates and bachelor degrees, but UT plans to offer a master's (S-2) in education. STOU does not appear to be considering higher degree programs, which is understandable in view of the demand for places, faculty qualifications, and the availability of higher degrees at traditional Thai universities. In comparison, UKOU operates undergraduate, graduate, and continuing education programs. One central difference between the STOU and UT and UKOU is that UKOU courses can be taken as single entities or accumulated towards a BA degree (World of Learning, 1985). It is not clear whether students can only take "single entity" courses at either STOU or UT, but The inference is that this is not encouraged. Since STOU operates on a credit system, this process may be possible, but the demand equation in a developing nation probably fa 'ors those students intend.ng to complete degree programs. - 263 - Educational Technology: Chapter 6 All three open universities target their programs toward students who are employed full-time and interested in part-time study. UKOU admits students over the age of 18 in an open-admission policy, regardless of whether they possess matriculation qualifications. STOU, on the other hand, confines admission to Thais in possession of the Mathayom Suksa 5 (secondary school certificate), and UT accepts only those Indonesians who pass the PP3 university entrance examination. This suggests that neither STOU nor UT have adopted the UKOu's open-admissions policy. However, the unlimited enrollment announcement by Rector Setijadi of UT, quoted above, leaves this matter unclear. All three institutions also allow their students generous time periods in which to complete their degree requirements. While UT states that there is no limit on the length of time for degree completion, STOU allows its students between four and 12 years (Tank, 1986). This time limitation is, most likely, related to the minimum of one block of six credits that a student must take each semester, as well as to the maximum of three blocks per semester that a student may take. Moreover, both UKOU and STOU appear to have the necessary faculty resources to enable degree completion, while this is less certain at UT. The openness of UKOU includes no limitations upon time in which to complete a degree. It is likely that neither STOU nor UT can afford such openness, in view of the demand, faculty resource constraints, and fiscal/administrative constraints noted earlier. While UT was noted to have graduated a cumulative total of 1,861 BAs after five years of operation, the policy of permitting nonactive status was noted above to have contributed to large numbers of the 1985-1988 intakes remaining on the books, but not graduating. Comparable data are not available for STOU. However, at UKOU over 60 percent of the initial undergraduate cohort completed a BA degree with the 1972-1987 cumulative total at 89,1.01, including 15,052 honors graduates. The 1987 UKOU Statistics bulletin lists the cumulative total for graduate degrees in that year as 351 PhDs and 186 other higher degrees. UKOU graduates in 1987 totalled 8,074 (including 1,610 honors), plus 67 masters degrees and 55 PhDs. Curriculum Development and Course Operations The course team approach, pioneered by UKOU, was earlier noted to have been adopted by both STOU and UT. The STOU boards of studies, consisting of a chairman and three to seven members, organize course teams, drawing from the five to eleven academic assessors on staff and, when required, seek assistance "from experts from outside bodies and staff members of other universities" (STOU, 1983b). The STOU course teams also include staff from the Offices of Educational Services, Educational Technology, and Academic Affairs. In marked contrast, UT appears to rely almost exclusively upon contracted faculty from Indonesia's established universities to function as course team members. As noted earlier, the UT course teams consist of subject matter specialists, production staff from RRI or TVRI, a tutor, an educational technologist, a subject matter coordinator, and production and UT editorial staff. According to Hill (1984), the first vice-chancellor of UKOU, Lord Perry, believed that the course team concept "with a number of specialists contributing their own expertise and all subject to criticism by their peers, was the greatest achievement of the British Open University" (p.145). - 264 - Educational Technology: Chapter 6 This writer is lnclined to agree and, moreover, to commend the usefulness of the course team concept to Third World universities -- both open and traditional -- as a mechanism to facilitate quality instruction. This is particularly important in new universities, because it is impossible to plan for seasoned faculty resources at newly developed universities. The synergistic nature of course teams improves the availability of quality instruction from the outset. Similarly, the use of media was noted by Hill as a UKOU innovation that made the model successful. As noted earlier, both UT and STOU have adopted a multi-media approach to distance teaching. Tables 1 and 2 indicated that all three institutions utilize correspondence texts and teaching methods, audio and video cassettes, and radio and television lessons. It was also noted earlier that all three open universities have encountered difficulties with access to television time and that only Thailand had established its own television station. Neither STOU nor UT use telephone lines for distance teaching. However, while STOU (1983b) has decided against using media and distance techniques for "areas that do not require a great deal of laboratory work" (p.23), UT has contracted with the Institute of Technology at Bandung to develop science and nursing course materials (Tank, 1986). This development follows the pattern established by UKOU, which has developed courses requiring laboratory work in science and computer subjects. The UKOU Institute of Educational Technology is paralleled at STOU by the Office of Educational Technology and at UT by the Office of Educational Media Production. Similarly, all three institutions publish their own workbooks and text materials. Therefore, with regard to media usage, it is reasonable to conclude that the media concepts and practices pioneered at UKOU have transferred, virtually intact, to both STOU and UT. Lesson distribution was noted earlier to have been contracted to the Indonesian Post Office by UT. STOU, on the other hand, relies primarily upon its regional and local study centers for the distribution of course materials. Similarly, UKOU distributes course materials through its 265 study cen_ers. Course script evaluation is undertaken by STOU tutors at the regional and local study centers, which also undertake face-to-face tutoring and grading of examinations. UT has course scripts evaluated by contracted tutors from regional universities, while examinations are computer-graded at UT. Study groups are encouraged in local areas and tutorial sessions are held twice per semester at study centers. UKOU utilizes both tutor and computer-grading of assignments and examinations, varying according to program and/or subject matter. UKOU schedules examinations in October and November, while STOU schedules exams twice per semester at centers in all provinces. Although the announcement that UT will schedule examinations on demand constitutes a radical departure, this has yet to be implemented. UT continues to offer its exams twice yearly at the end of each semester (STOU, 1988; Universitias Terbuka, 1988). Relationship of Programs to National Development Objectives While this criterion appears to be more relevant to developing nations, Hill (1984) notes that UKOU teaches "specific techniques needed for the economic expansion of the country" and that one of its principal functions has been "the upgrading of the qualifications of practicing schoolteachers" (p.147). The press has criticized UKOU as "the housewives' university," an inaccurate description, according to the evidence. - 265 - Educational Technology: Chapter 6 These same comments are germane for both STOU and UT, particularly in the context of developing nations, the teaching of specific development-related techniques and upgrading of practicing teachers assume even greater relevance. Both STOU and UT also manifest commitments in their course offerings to skill areas vital for national development. Specifically, the common program areas of education, economics, and business administration address such vital areas. The decision of UT not to offer law courses differentiates it from STOU, because it suggests that UT has not catered to student demand, but rather made a conscious decision not to offer courses in a field that is over-provided at most Indonesian (and Third World) universities. In like manner: 'JT courses in public administration, applied statistics, and development studies, u.s noted in Table 6, provide tuition in areas related to national development efforts. STOU course offerings in management science, health science, home economics, agriculture, and cooperatives also manifest a commitment to fields related to natioi.al development. The UT plans, noted above, to add courses in ac-_ounting, nursing, physics, chemistry, biology, and technology indicate that this commitment is to be expanded. One unique ripple effect, which attests to the guality of the UT course modules, is their use by faculty at many public and private universities in Indonesia in lieu of textbooks and/or lecture notes. Satisfaction of Social Demand for Access to Post-Secondary Education Again, it is apparent that all three op--- -._.7ersities do satisfy the open-ended demand for access to post-secondary education in their respective nations. Although the U.K. is generously sur lied with post-secondary options, ranging from the traditional universities to che newer 'red-brick" universities, to polytechnics, to further education centers, it is quite apparent that Harold Wilson was quite astute in sensing that a new form of distance post-secondary education was needed. In both the Thai and Indonesian contexts, with their limited post-secondary opportunities, STOU and UT serve a different clientele, albeit one with needs and aspirations comparable to their British student colleagues. The concentration of both STOU and UT upon in-service upgrading of practicing teachers provides learning opportunities that are likely to have a multiplier effect upon the quality of education in both nations -- a quality that is in dire need of improvement. Similarly, the impact upon agricultural extension and cooperative development in Thailand is likely to have a similar effect. If one examines the dynamics of student flows in developing nations, it is apparent that post-secondary demand far outstrips available places. As noted earlier, Indonesia has places at its 45 state-operated universities for only 100,000 of the 600,000 who annually sit university entrance examinations. While comparable data are not available for Thailand, it is likely that the dynamics are of a similar magnitude. Thus, it is easy to conclude that open universities do address the problem of social demand for access to post-secondary opportunities. The question of quality remains a subject for further inquiries of an evaluative nature. From a financial perspective, it is clear that open universities are far less expensive in - 266 - Educational Technology: Chapter 6 both capital and operational outlays and, therefore, can serve more clientele with fewer resources. Utility of the Model for Developing t7ations The foregoing comparisons show that the .open university model is an extremely useful one for developing nations. Apart from the satisfaction of social demand and financial considerations, the opportunities afforded to upgrade skills and knowledge of the existing workforce will make considerable contributions to national development, as does every effort at adult education. These considerations are aqually applicable to the U.K. (and any other developed nation). Moreover, from a geographic perspective and the perspective of redressing unbalanced development of post-secondary opportunities, the provision of open university access is almost tailor made for most developing nations. Given Indonesia's vast dispersion among more than 13,000 islands that stretch for about 5,000 km. from east to west and 2,000 km. from north to south, and the concentration of existing universities in major centers, UT is bound to redress existing inequities. Similarly, in Thailand where "growth poles" exist in Bangkok and lesser urban centers, STOU is certain to redress the imbalance, as well as possibly moderating ur&an drift toward Bangkok. The utility of these remedial aspects for most developing nations, which face similar growth, fiscal, and access problems, renders the concept of an open university quite attractive. However, as one who continually resists notions of panaceas, or cure-all solutions to the problems of developing nations, this writer does not purport to offer the open university as a solution to such problems. Rather, it should be perceived as an attractive element of solution. Institutional Transfer This paper closes with an examination of the notion of institutional transfer, particularly as it applies to the open university model. This discussion is facilitated by Dodd and Rumble's (1984) notation that: In the planning and devalopment of new DTUs (Distance Teaching Universities], the involvement of the U.K. Open University has been a particularly prominent feature. Based upon the success of the university's Consultancy Service set up in 1974, the Open University established in 1977 a Center for International Co-operation and Services (OUCICS) whose primary purpose was to provide policy and technical assistance in the design and implementation of new distance learning institutions (p.248). Dodd and Rumble also note that the OUCICS involved UKOU staff in the planning and early development of STOU, in addition to open universities in Costa Rica, Venezuela, Iran, Israel, Pakistan, S_i Lanka, and Nigeria. However, the closure of OUCICS in 1980, due to a lack of funding, relegated such endeavors to the British Council. Thus, there has been no UKOU assistance provided to UT in Indonesia. - 267 - Educational Technology: Chapter 6 In addition to direct assistance, UKOU has apparently become somewhat of a tourist attraction. Dodd and Rumble (1984) refer to UKOU as a primary port of call for visiting Third World educators. They quote Daniel and Stroud (1981), who wryly noted that: Buses were rented to cope with the international jetset of academic pilgrims descending on Milton Keynes to watch the OU campus rising from the mud, and governments of many political hues, noting an open university to be the educational equivalent to an airline in terms of international status, sponsored similar projects for t-he people (Dodd and Rumble, p.246). Given the long-standing impact of study visits upon subsequent educational reforms, the importance of such visits for che transfer process is a factor worthy of serious consideration, albeit one that is difficult to document. Although somewhat unfair, the "status" comment made by Daniel and Stroud does have some foundation in fact. However, this writer is of the opinion that the transfer of the open university model to most developing nations has been stimulated more by need than by considerations of the status of having such an institution. It was noted earlier that one component of USAID technical assistance to UT was to have been the secondment of two staff members from STOU, but this did not take place. However, this indicates that contacts did take place between STOU and those planning UT. The subsequent transfer of many STOU attributes has been noted in the previous section and, doubtless, arose from such contacts. What this does suggest is that both first-order and second-order transfers appear to have influenced Universitas Terbuka. First-order transfers have been those resulting from visits to and assistance from UKOU to UT, while second-order transfers have been those resulting from visits to and assistance from STOU to UT. UKOU structural attributes that have been replicated at STOU and UT are: UKOU STOU UT Ministerial Authority Indirect Direct Direct Academic Senate Autonomous Dependent Dependent Faculty Organization Faculties Schools None Academic Year Feb - Dec Jul - Apr Sep - Aug Regional Study Centers 265 77 88 Degree Completion Time No limits 4-12 yrs. No limits Degrees Conferred Undergrad Diploma Undergrad Post-grad Ur.dergrad - 268 - Educational Techuology: Chapter 6 UKOU functional attributes that have been replicated at STOU and UT are: mm K AmU Y Enrollment Policy Open Traditional Traditional Course Team Yes Yes Yes Media Usage Multi-media Multi-media Multi-media Media Production Headquarters Headquarters Headquarters Faculty Location Headquarters Reg. Centers Reg. Univ's. Lesson Distribution Reg. Centers Reg. Centers Post Office Course Script Eval. Tutors Tutors Contracted From these lists of attributes it is easy to discern the effects of transfer and replication of UKOU structural and functional aspects at both STOU and UT. Furthermore, it is also easy to discern which aspects have been adapted to prevalent local conditions. Such adaptation is central to the theory and operation of the transfer process, since adaptive changes to local realities mirror our quest for how things came to be the way they are. The juxtaposition of transferred attributes and the comparison of similarities and differences between the UKOU model and its Thai and Indonesian replications provide another chapter in our continuing quest for knowledge about the worldwide spread -- or tranisfer -- of attributes and inTiovations i.n education. - 269 - Educational Technology: Chapter 6 References Aitken, D.J., ed. 1986. International Ha,ndbook of Universities. 10th od. Now York: Stockton Press. Amidjaja, D.A.T. 1983. Higher Education Develo=ment in Indonesia. Jakarta: Directorate General of Higher Education, Ministry of Education and Culture. Daniel, J.S., and M.A. Stroud. 1981. "Distance Education: A Reassessment for the 1980's." Distance Education 2, No.2: 146-163. Darcesman, R E., W.P. Teschner, and D.N. Wilson, 1984. Indonesia: Sector Studv on Eduation. Manila: Asian Development Bank. Dodd, J., and G. Rumble. 1984. "Planning New Distance Teaching Universities." Higher Education 13: 232-248. European Association of Distance Teaching Universities. 1988. "The United Kingdom Open University." Eurogean Association of Distance Teachinb Universities DirectorX 1988. Milton Keynes: The Open University, p.77-85. Hill, J. 1984. "Comparing Open University Systems.' Adult Education 57, No.2: 143- 148. Ministry of Education and Culture. 1989. Laporan Statistik Mahasisva Universitas Terbuka Tahun 1984-1989. Jakarta: Ministry of Education and Culture. "Thailand's Sukhothai Thammathirat Open University." 1987, December. Open Lean.ing Systems News. Southampton: National Council for Educational Technology, p.9- 10. Pernia, E.M., and D.N. Wilson. 1989. Education and Labor Markets in Indonesia: A Sector Survey. Economic Staff Paper No.45. Manila: Asian Development Bank. Popa-Lisseanu, D. 1986. Un Reto Mundial: La Educacion a Distancia. Madrid: UNED. qrisa-an. W., and T Wangsotorn. 1985. "The Management and Economics of Distance Education: The Case of Sukhothai Thammathirat Open University, Thailand." Paper presented at the International Council for Distance Education, 13th World Conference. Melbourne, Australia. STOU. 1983a. Fact Sheet-Sukothai Thammathirat Open Universitv. 1983. Bangkok: Planning Division, STOU. STOU. 1983b. STOU Administrative and Academic Structure. Bangkok: Planning Division, STOU. STOU. 1988. Sam Sorn Academic Bulletin. Bangkok: STOU. Tank, A. 1986. "Rise and Rise of the Open University." Times Higher Education SuRglement 721:7. UNESCO. 1989. Never Too Far. Bangkok: UNESCO. - 270 - Educational Technology: Chapter 6 UKOU. 1987. Ooen University Statistics - 1987. Milton Keynes: UKOU. Universitas Terbuka. 1988. General Informatlon on Universitas Terbuka. Jakarta: Universitas Terbuka. Universitas Terbuka. 1989. Lima Tahun Universitas Terbuka 1984-1989. Jakarta: Universitas Terbuka. Wilson, D.N. 1988. "Indonesia: Sector Study of Education." Manila: Asian Development Bank. Processed. Wilson, D.N. 1990. "A Comparative Study of Open Universities in Developed and Developing Nations." Paper presented at the Annual Conference of the Comparative and International Education Society. Anaheim, Calif. World of Learning. 36th Edition. 1986. London: Europa Publications. - 271 - EduQational Technology: Chapter 7 Chapter 7 Computer Applications in Vocational Training in Norway Arvid Staupe - 272 - Educational Technology: Chapter 7 COMPUTER APPLICATIONS IN VOCATIONAL TRAINING IN NORVAYI/ Arvid Staupe I. Introduction Between 1984 and 1988, the Norwegian Government undertook a four- year Program of Action for experimental activity with computers in the nation's primary and secondary schools and for teacher education. A new secretariat, the Norwegian Datasecretariat, was created in the Ministry of Church and Education to administer and coordinate the program. The main objective of the program was to begin experimental activity that would provide a sound basis for the future introduction of computers into the school system. This chapter will focus on applications for computers that have been developed under this program for vocational training, particularly in the area of simulation. Norway's strategies for minimizing costs and for ensuring timely adaptation of pedagogy to rapidly changing technical environments are likely to be valuable for other countries seeking to introduce computers into their vocational training systems. This is true for developing countries as well as developed countries, although different conditions of infrastructure, financial resources, and access to technology limit computer applications in some cases. II. The Program for Action It is now an established policy goal in Norway that the country must modernize its manufacturing sector through the use of computer technology, and that it should build up new industries, particularly in the area of high technologies. Efforts must also be made to use computer technology in the trade and service sectors. Becauie of this policy of modernization, great demands will be made on the vocational education system in the coming decade. Implementation Phases Part of the policy response to this challenge can be found in the Program for Action, which was aimed at promoting the introduction of computers into schools. The four-year pilot program was based on the realization that a period of experimental activity and competence raising at all levels was necessary before general introduction of computers could be attempted in all schools. The program was designed to benefit both the general education and the vocational training areas. Implementation of the program involved the following specific phases: I/Under the Norwegian Program for Action for Experimentation with Computers in Education - 273 - Educational Technology: Chapter 7 * Total school experiments occurred in 26 selected schools in various parts of the country, involving all the teachers in each school. Participating schools included: 10 primary and lower secondary schools, 13 upper secondary schools, and three agricultural/horticultural schools. * Small project schools participated by having activity limited to a smaller group of tea-hers, to one class, or to the testing of specific software or teaching plans. Approximately 60 schools, including 40 schools giving vocational training, were involved in this phase. * A national network of resource centers was developed to provide the schools with the necessary support to implement computer use, and to encourage the exchange of experience among the participants. In addition, conferences and courses were arranged for groups of teachers. * Development and testing of software for educational purposes was undertaken in two areas. At least 100 teaching and applications programs were created, and at least 10 software tools were designed to help teachers create applications programs and make better use of existing programs. * Cooperation was fostered among schools participating in the program, among computer specialists both within and outside the school system, and with colleges and universities that offer teaching training. - International contacts were made with other Nordic countries through the Committee on Educational Software and Technology, established under the Nordic Council of Ministers. Financing and Costs Financing for the Program of Action and its follow-up activities comes from the Ministry of Education. Table 1 shows the annual budgets during the four years of the program. - 274 Educational Technology: Chapter 7 Table 1 The Program for Action: Annual Budgets, 1984-88 Year Millions (US$) in Budget 1984 $1.7 1985 3.1 1986 3.6 1987 3.8 1988 1.9 Total 14.1 Source: Datasecretariat The budget for the program represents an extra allocation over and above the regular Ministry of Education budget; it is separate from the regular budgets of the regional and local authorities. Over the four years of the initial pilot program, from August 1984 to August 1988, the total budget was US$ 14.1 million. As a follow-up to the program, the Ministry of Education has continued to support the introduction of computers into schools and to foster the development of educational software. For this purpose, US$ 6.5 million was allocated in 1988, 12.6 million in 1989, and 11.7 million in 1990. A separate allocation of US$ 5.3 million was made to support the introduction of computer technology in vocational training schools (US$ 2.6 million in 1987 and 2.7 million in 1988). Implementation Strategy Overall, the Program for Action has emphasized five points in its implementation strategy. Experimental pedagogical activity was sponsored in selected schools to develop appropriate teaching methods for the new technolog:;es. In-service teacher training methods were developed to introduce teachers to the new equipment and develop their skills witb its use. Software development was emphasized in order to correct the defiriency of available learning courseware. Participants were encouraged tc exchange information among themselves to spark creative energy and encourage interest in the program. Finally, cooperation was arranged with other organizations and businesses, both in Norway and internationally, to make additional expertise available to the program as a whole. Pedagogical and technological experimentation takes time. It was apparent at the beginning of the program that nearly all of the participating teachers lacked any understanding of computers. Consequently, a gradual implementation strategy was adopted: teachers first gained experience through a trial-and-error phase. Only after this phase were teachers expected to have developed sufficient knowledge and confidence to handle larger and more - 275 - Educational Technology: Chapter 7 complex tasks, such as incorporating self-designed software into their own teaching methods. Technical specifications for computer equipment to be used in primary and secondary schools had been drawn up by the Ministry before the Datasecretariat was established. These specifications, which were based on sound pedagogical input, were the basis for an invitation for tenders to supply the schools with computer equipment for the experimental activity. Because of the pace of technological development, it was decided to choose technology that would be reasonably up-to-date throughout the whole experimental period. Since this strategy meant avoiding technology that was well tested but already out-of-date, the challenge of designing effective educational applications was greater. In order to rur. a program such as this, the Datasecretariat needed creative employees with strong technical and professional skills. It was especially important to have staff well versed in computer technology in order to guide, coordinate, and evaluate the program's development, since sound economic decisions must be based upon well-founded technological judgments. In this connection, experts at universities and colleges provided a great deal of invaluable assistance. The program has been managed by a relatively small number of people. In the first year of Program for Action, there were never more than four full-time staff members. This number incrvased to 11 during the rest of the experimental phase, and currently there are 21 full-time posts in the Datasecretariat. Software Development Throughout the national Program of Action, one of the main obstacles to gaining experience with computers as tools in education was the lack ot appropriate software. The Norwegian Datasecretariat set out to develop a national supply of programs and accompanying teaching materials for the introduction of computers as a new teaching aid in many different subjects ranging from Aqua-chemistry for Fish-farming to Weaving. In addition, a group of about ten programs were developed as tools to help with such problems as courseware authoring (The Director) and improved portability of graphics software between different kinds of hardware (GrafTrix). Experience gained during the implementation of the program has shown the importance of linking software development to in-service training and teacher preservice education. Both technological and pedagogical innovations were necessary in the program, and there was always interaction between the two, since technological development is based on pedagogical specifications and demands. In Norway several approaches were taken to the problem if increasing availabl.e educational courseware for use with computers in schools: * Individuals (especially teachers) and companies were urged to come up with ideas and products; * Educational institutions were encouraged to produce software; and - 276 - Educational Technology: Chapter 7 * Software packages were translated from other languages (e.g., English, French, Danish). To date, the philosophical approach has been to give the most support to the ideas for application software that originate with the teachers themselves, rather software development projects planned centrally. This method has yielded many advantages: teachers have shown a great deal of initiative and creativity; local interest in the program has been bolstered; and teachers have learned more about computers in the process of using them to create the software. As of the end of 1989, about 50 educational programs (applications) in different areas have been completed, and approximately 50 new programs are under development. Development of the tool programs has simplified the production of educational software, which means that more software can be produced for the same amount of money. Because of GrafTrix, most programs can be used on most equipment with only minor adjustments. In addition, some tool software programs can be used as an open-ended software package to teach subjects such as Dynamic Systems or Local History. The tool programs also make it easier to rewrite, translate, and upgrade other courseware programs. At the center, the role of the Datasecretariat has been to coordinrte all these activities and to set certain overall policies. Accordingly, most of the software that has been developed under the program makes use of standardized user interface, menus, graphics, and other features. Teacher Training According to a report on the program by the Norwegian organization, International Learning Cooperative (IMTEC), at least two years of in-service training are required for 50 percent of teachers to feel that they have mastered the use of computers in their teaching and for them to use computers In their classes. After this twu year training effort, only 25 percent of the trained teachers regularly try out new software or feel that they have mastered the computer adequately to use it for word processing and design of pedagogical software. Only 15 percent feel that they are able to make new teaching plans with the computer. These figures (IMTEC, 1987) point to the tremendous importance of thorough and long-term teacher training in order to implement computers in education. Information technology should be an integral part of the teaching method for a particular subject, or it is likely that it will not be used effectively. Under these circumstances, the most important change in teacher training should be to integrate computer technology into subjects and teaching methods as they are presented during teacher training courses. This will be far more effective than offering separate, specialized computer courses for teachers. With such separate courses, individual teachers have to struggle to incorporate the computer technology into their ongoing teacher training curriculum. In preservice, in-seriice, and post-graduate education, computer teclnology should be integrated into the courses on various subjects and the methods courses rather than being provided as a separate course or workshop. - 277 - Educational Technology: Chapter 7 A major problem in the implementation of a policy of this kind, however, is the lack of experience and competence among teacher trainers themselves. Academically trained in their respective subjects, the staff of teacher training institutions are rather reluctant to make changes in their daily work. Many faculty members may like using word processing to compose lecture notes or organize equipment, but fewer will use a comouter for such purposes as improving a presentation on how to teach art, ec;,logy, or writing. The Norwegian example suggests that, in order to overcome this inertia in the training system, teacher training institutions must become involved in the software development process. This process -ill function as in-service training for the teacher trainers, giving them inspiration to revise the way that they train student teachers. Most subjects and areas of teacher training would benefit from an arrangement like this. In the area of vocational training, it is particularly important to involve instructors in software development in order to expose them to modern technology during their training. Evaluation The Program of Action has undergone several evaluations, including one by the Organization for Economic Co-operation and X 'elopment in 1986. IMTEC has also done beveral studies on the Program anc As preparing a process evaluation, as well as a final evaluation (IMTEC, 1987). The OECD experts concluded: "Norway's effort is impressive compared to many countries. At the beginning of this program, Norway probably was lagging in this area, as compared to some other OECD countries. This in no longer the case" (pp. 19-20). The report went on to cite several specific strengths in the program: * computers have been used as a learning aid in all areas; * teachers and administrators are making increased use of computers as a result of the program; * teacher training has been singled out as the most important part of the program's success; and * extensive software materials have been developed and have received proper emphasis relative to hardware. Vocational Education In Norway, upper secondary education serves students aged 16 to 19 and spans the tenth to the twelfth grades. In addition to general education, upper secondary education includes vocational and apprenticeship training. There are about 200,000 pupils studying at the upper secondary level (including those in apprenticeship training) enrolled at 800 upper secondary schools. All training in general and vocational subjects has been coordinated into one comprehensive system. General and vocational studies now exist side by side in the same schools and can even be combined within a course. There are nine areas of study at present: General ')ubjeccs; Technical and - 278 - Educational Technology: Chapter 7 Industrial Subjects; Aesthetic Subjects; Maritime Subjects; Physical Education; Commercial and Clerical Subjects; Home Economics; and Social Services and Health Subjects. All areas of study follow the same basic timetable; one or two years of basic courses are followed by one or two years of advanced, specialized courses. The basic courses share a common core of general subjects regardless of study area, and these are supplemented by electives in other subjects, such as basic vocational training, art, and physical education. Advanced courses imply specializations. For pupils having chosen vocational training, two years of basic courses is equivalent to one year of apprenticeship. Upon completion of their basic learning, they can either start a job as an apprentice or apply for a place in the advanced vocational training course. It is important for the national authorities to avoid regional disparities and to ensure that training is available in all desired subjects. For this purpose, extra state subsidies and provisions are made in some cases. For example, courses in some specialized trades have such a limited recruitment that they ca-not be organized in each region. Instead, they are given on a national basis. Apprenticeship Training Apprenticeship training is based on close cooperation between schools and places of employment. The development. of Norway's apprenticeship training system has been a key element in the government's policy to ensure coordinated theoretical and practical education and training. The aim of the apprenticeship system is to provide the country with skilled workers in the various crafts and industries, as well as to reduce unemployment among young people. Various incentives have been put in place to stimulate companies and other public and private organizations to enroll apprentices, Aa order to increase the number of workers passing the craft examination during the 1990s. The central body in charge of the apprenticeship system is the Council for Systematic Vocational Training, which is an advisory body in the Ministry of Education. Within each craft or industry there is also a training committee that assists the Council; these committees play an important role in setting up the training programs and curricula. New Pedagogical Approaches As the twentieth century draws to a close, the pace of technological change is increasing. The rapid rate of change creates two kinds of problems for vocational training. First, it is more and more costly to obtain and maintain the needed shop equipment. Second, the increasing role of electronics and computers in manufacturing means that tha pedagogic approach must be adapted to the demands of this new area. Technological development has directly affected the subject matter in most vocational training courses; one important effect has been to make more and more work processes "invisible" to the laborer. Formerly, mechanical - 279 - Educational Technology: Chapter 7 processes were visible and easy to follow; now a lot of manufacturing work is controlled electronically with computers. While students formerly undertook realistic work training to galn mechanical skills and dexterity, now the problem is to galn an understanding of system control and problem solving. Technological development has also led to the use of more expensive machinery ln trade and lndustry. Even if an unlimited budget were available, the large and expensive machines would not be suitable for use in schools. When something goes wrong with the equipment, or if inexperienced students cause breakdowns, special expertise is necessary to fix the problem. Such repairs are expensive and time-consuming. In addition, there are often safety risks lnvolved both for the users and machinery. As a result, theoretical training, simulation, and the use of models are often the only possible methods for giving the students an understanding of the real work processes. Therefore, the Program of Action has given priority to the development of software, equipment, and courseware suitable for these new educational tasks. The goal is to better prepare students to handle the operation of the new kinds of equipment that they will eventually encounter in the workplace when they complete their training. Simulation The introduction of computer technology to the learning process makes possLble new ways of handling complex problems. One such new method is simulation. A simulation model imitates a selection of characteristics in a concrete or abstract systems. The user may alter the parameters and study the changes that occur under varled conditions. Simulation can be used in situations where the real-life process would be too large, too hard to see, too complex, too dangerous, or too expensive for students to attempt hands-on practice. Using slmulation, we can replace complex and expensive systems and give the students a chance to practice and evaluate before using the real thLng. For example, at the simplest level, one starts with models, such as robots, lathes, or conveyer belts, which are connected to the microcomputer. Processes such as cutting, turning or coordinate drilling can be steered and controlled by the computer on these small versions of the machines actually used in industry. Finally, the microcomputer is connected to the regular machLnery used for production, and the close connection between the model and reality becomes obvlous. The transltlon from model to actual equipment is important, especially for students who are about to complete their training and enter the workplace. For example, in the case of machine tools, the full-scale machines dlffer from the models in speed as well in the kinds of materials that they can process (hard stainless steel versus softer metals). To serve these trainees, some Computer Numeric Control (CNC) centers have been established and equipped with both the training models and the regular machinery, to give the students an opportunity to experience the transition firsthand. The same computer equlpment and software that were used in the schools are used in these centers, and the tralnees perform the same tasks. In some locatlons, local Lndustry cooperates wlth the schools by allowing the students to use thelr machinery from tlme to time. - 280 - Educational Technology: Chapter 7 Simulation cannot completely replace real equipment in the learning process. For example, it does not give students the same practical experience as they would get in a chemical laboratory, but it reinforces and supplements the students' lab work. In many cases, the instructors seem to be satisfied with the kind of training they are able to give by using models instead of full-scale equipment. In the case of machine tools, the full-scale equipment is not missed, and there is little demand for additional CNC centers. The schools still have a substantial amount of expensive equipment that was bought before computers were introduced. In order to incorporate this equipment into the new computerized teaching methodologies, it is necessary to reconstruct it. As part of the program, the Datasecretariat has developed an adaptable unit for computer control that can be used with many kinds of equipment. This unit allows the schools to choose whether they want the students to use modern computer control or to practice traditional control methods. The Datasecretariat has developed software for various simulation purposes for vocational training. Among them are MI-2000, which simulates computer-guided numerical control (CNC) for machine tools, a flight-booking program, and a fish farming program and also creates a simulation of various biological, physical, and chemical processes. Figures 1 and 2 show example screens from two of these programs. II. ComRuters and Costs The Datasecretariat has found a number of different ways by which carefully planned use of computers in the teaching process has allowed vocational training programs to control rapidly rising costs or even to reduce costs. The major ways of keeping down costs through computer use are in minimizing infrastructure costs, use of models and reconstructed equipment, improved efficiency in student utilization, and controlling software costs. Infrastructure Costs One objective of the Program for Action is to make it possible for all vocational schools eventually to be able to offer their students training in the use of the latest computer-based equipment found in the workplace. In the case of machine tools, for example, all students should be able to learn CNC. However, it is economically unfeasible to provide enough full-scale, computer-controlled production machinery in the classrooms. Each class with 12 to 15 students would need about five sets of equipment. A single typical machine tool set-up of this type of cost as much as US$ 1,5 million. This kind of tooling equipment, such as the milling, throwing and platecutting machinery is particularly expensive, but even in other areas, obtaining the necessary equipment is far beyond the means of the educational system. It appears to be virtually impossible to combine the need for suitable equipment with the need to maintain costs at an acceptable level. - 281 - Educational Technology: Chapter 7 Simulation and the use of models for training are two ways that computers can be used to overcome the problems posed by expensive learning infrastructure. In the case of a simulation program, such as the fish farming program described above, no equipment is required other than the microcomputer needed to run the software. Once the costs of computer purchase and software development are accounted for, additional costs for using this kind of simulation are largely limited to the outlay for software, which amount to about US$ 200 per copy. The more complicated simulators, such as those used for command and control on board ships, will naturally be more expensive. These kinds of simulators require additional specialized equipment to be linked to the microcomputer in order to execute the simulation. However, the cost of using this equipment can be compared with the cost of giving the student experience with the full-scale situation. In the case of the operation of an ocean- going oil tanker, the cost per hour of training on the simulator is obviously far less than the cost of gaining experience on an actual tanker. Models and Reconstructed Equipment Savings gained through the use of models and adaptation of existing equipment for use with computer simulation. Where new equipment must be purchased, the simulation apparatus can be substituted for expensive computer control mechanisms, reducing the overall purchase price substai±tially. Where equipment already exists in the school, reconstruction costs must be considered to be add-on costs, but the adapted piece may allow for savings through increased learning efficiency and avoiding the need to purchase costly new equipment with the computer features. Greatest savings typically come from the use of computer-controlled models. The mechanical engineering course offers a good example of the comparative costs involved. A new computer-controlled cutting torch costs between US$ 70,000 and 140,000 in Norway. However, if a new cutting torch is purchased without the steering unit and then reconstructed at the training school so that it can be computer-controlled with school computers, the total cost for the cutting torch, the adaptive unit, and the computer equipment is to US$ 3 ,000. Reconstruction of existing equipment, including the cost of the CAM workstation and installation, amounts to approximately US$ 12,000. The software costs an additional US$ 2,000. Purchase of a model, together with the adaptive unit and computer equipment, costs US$ 14,000. Reconstruction yields savings of at least US$ 35,000 per unit when buying new equipment, and the use of a model or a reconstructed piece of existing school equipment saves at least US$ 56,000 per unit. An important additional cost advantage of the model is that it incurs lower maintenance costs than regular machinery. - 282 - Educational Technology: Chapter 7 Figure 1: Fish Farming Simulation. The diagram shows a screen dump from the program Sea-farming, a part of the course on aquaculture. On the screen, several different variables have been entered into the simulation model, which in turn shows how these variables change over time. The resulting curves are shown in different colors. One may choose from up to 70 different variables, and a maxiu.umi of five different variables may be shown simultaneously. In the screen shown, one curve represents the number of fish in Generation 1. The program simulates two generations simultaneously. The fish in Generation 1 are entered into farming at day 160. The diagram shows th&t after this point, the number of fish slowly decreases; some die, and some are harvested. At the same time, the average weight is shown to be increasing until the final harvesting. The simulation in this example is stopped after day 960. It would be possible to run the simulation until day 1200. At the top of the screen, a menu bar is shown. This presents tCie user with a choice of functions, making it possible to control economic factors such as wages, loans, and daily expenditures. Choices can also be made to control envirounmental factors such as temperature, quality of surroundings, and harveating plans. - 283 - Educatioi.al Technology: Chapter 7 - .\t -:rh} ;--j9 t < i< 3 1tt 1 _' '' npp I snl nuer oun kaluiniq IP Ful .jtc PA fj '; ..- -, , !_ittr o: .in skpe d1.iq Va . 0" ld Q l forho ..'16 u1i! aui-''1'a 199' Fiur 2: Reidee Farmin- Simulatio Sam [Lpp Edcton.I thesimlaton It is ,g on the.plain, but stillwinterin... years_ ol an -i -z-*s *shon by sex with th bar in th prmdsape bo. The I _a -rr , . ct - _ a.!_ _ ' ...... . - . . | J 1L3 -4.j - - Full utntir pn lly lcnt erchntorrg og utnd s, prr drit qct US$12o00r fmirhold undtr )the]U 1mc c u ci3 Figure 2: Reindeer Farming Simulation (Sami [Lapp] Education). In the simulation, it is springtime on the plain, but still winter in the mountains The screen shows the number of reindeer at the present time: 323 males and 455 females. The reindeer age distribution ranges from 0 to 15 years old and is shown, by sex, with the bars in the pyramid-shaped box. The black stripe in the sun shows the time of day in the simulation. In another example, a certain type of cutting machine used in training has automatically interchangeable tools, and it costs US$ 120,000 or more, even without the commercial computer control equipment and accompanying software. The model that can substitute for this machine costs approximat.ely US$ 9,000, including the software. With tne cost of the microcomputer added in, the entire model set-up costs about US$ 16,000. The figure represents a savings of 80 to 90 percent, compared to the cost of the full-scale production machiner.y. - 284 - Educational Technology: Chapter 7 Improved Efficiency in Student Utilixation Another way that computers make it possible to contain or reduce training costs is through the use of non-traditional teaching methods, such as individualized self-study plans. When compared with traditional class-based methods, indivldualized plans make use of teaching modules that allow each student to work on a different topic if so desired. With this system fever pieces of equlpment are needed for a class of a given size. For example, with the task of learning the operation of machine tools, it was noted that at least five machlnes were needed per class of 12- 15 students. However, by using computer technology, this number can be reduced to three. It is easy to keep the qtudents active with other tasks, depending on the availability of the practice machinery. Some students can start by drawing or constructing and then simulate the tooling. At the same time, others can work on the more traditional equipment. Our experience is that good planning can reduce the equipment units needed in the classroom by almost 40 percent over the usual levels. Other kinds of efficiencies are possible with computers; for instance, the execution of practical learning exercises can be considerably speeded up. This reduces the time that each student needs to use the equipment. Rubbestadneset Technical School experienced a considerable increase in practice efficiency on its machine tools after the introduction of the MI-2000 computer learning system, which is described in a later section. It used to take 12 to 14 hours for students to make the first product required in a certain laboratory exercise. By using ?I-2000, this production time was reduced two to three houirs. The time savings were further increased by the fact that the simulation program allowed students to change and correct their work before processing began. Software Costs The Datasecretariat has entered into several general agreements with commercial software suppliers in order to make it possible for the schools to obtain commercial sottware. The Norwegian Government was thus able to secure a general price reduction of 85 to 90 percent below market value for a number of programs. The government also sponsored the required adaptation and translation of the software into Norwegian. One goal of this policy is to make software programs that are widely used in trade and industry available to students. The combined efforts of the many teachers involved in software development, as a part of the Program for Action, are also making a substantial contribution to keeping software costs low for the program as a whole. These programs emerge ready to go and in the proper language; they are well adapted to the curriculum in which they will be incorporated, since they have been developed by the same teachers who will use them in teaching. Additional cost advantages are gained because the time that the teachers spend on software development is a part of the time they would need to spend anyway on trainlng to gain competence in computer teaching methods. Finally, no royalties need be paid to commercial owners for the use of this software, since the government-funded software is distributed to schools at cost. - 285 - Educational Technology: Chapter 7 III. The MI-2000 Project The MI-2000 is a software program that has been developed by SresProdukt A/S, Kristiansund, in cooperation with the Ministry of Education and Research under the auspicos of the Datasecretariat. The program teaches Computer Numerical Control (CNC) to students who are learning to operate machine tools in the mechanical engineering courses of the vocational training schools. The principal idea throughout the project has been to create a learning tool that enables students without computer background to gain experience at a professional level. To this end, the program reproduces a system that is in the forefront of industrial development. The MI-2000 program can be used together with a Computer-Aided Design (CAD) system (e.g. as VersaCAD, AutoCAD) and either a small-scale machine model or a full-scale machine tool to make a complete learning system. The MI-2000 course consists of three main modules: milling, throwing, and platecutting. The program simulates the three processes, and the student may edit his work to correct mistakes as he learns, with very little trouble. The result is safe and efficient learning. The program can be used at all levels of vocational training, ranging from CAD for technical designers, to CAD/CAM for production engineers, down to CNC operators. The MI-2000 software was developed in response to the problem of giving students equal access to CNC equipment for learning modern machine tool techniques. In the past, a few schools were able to give instruction on full- scale CNC equipment, but most schools found this too expensive; students in most schools had to use traditional equipment without connection to any computer. An effort to make full-scale CNC equipment available to all the schools was impractical. In addi6ton to the huge expenditures involved, the whole pedagogic method used with the full-scale CNC equipment (manual programming) was out of date. There was also a risk that the students could hurt themnselves if they operated the equipment on their own. This safety issue that close observation by the teacher was required, and this resulted in limited use of the equipment. Coursevare Design Problems The aim was to design an effective teaching system at a reasonable price. The system needed to be useable with both models and full-scale manufacturing equipment. The system also had to be user-friendly, flexible, and powerful enough to handle the demands of a professional level of CAD/CAM utilization. Computer Equipment. One goal of the course designers was to develop a teaching arrangement where computer-assisted design as well as computer- assisted manufacturing could be taught using reasonably priced microcomputer equipment rather than minicomputers or mainframe computers. The software used with minicomputers and mainframe computers typically is not user-friendly, and it is difficult to learn. By contrast, the learning process for the use of microcomputers is very quick. Personal computer software offers variety, user friendliness, full-screen graphics to communicate with the learner, and ease of operation. - 286 - Educational Technology: Chapter 7 Transferability. It was important to design a program that could be used either with a model or with full-scale manufacturing equipment with the student performing the final manufacturing process. To do this, the course was designed to include these steps: (1) the student would use the computer to design the piece to be manufactured; (2) the student would use the computer to do the calculations to set up the machine or simulator; (3) he would simulate the manufacturing process; and (4) finally, when the simulation showed that the exercise had been done correctly, he would actually be manufacturing the piece on the real equipment. It was also necessary for the same basic system to be used for all three parts of the course -- milling, throwing and cutting. Realism. Simulation techniques were incorporated into the design and calculation process and into the penultimate simulated processing phase, when the user has to be able follow the use of the tools through the processes of throwing, cutting, and milling In a realistic way. Simulation makes it possible to check if calculations are inaccurate and would result in tool parts accidentally hitting each other. The trainee can go back and rework his design and calculations if they are not satisfactory. Once the simulated exercise is complete, the student is able to operate the real machine without moving to other computer equipment. Design Generation. MI-2000 offers three ways for the student to generate the original design. It is possible to design the object directly in Mi-2000, or the student can select a finished drawing from a CAD/CAM system such as AutoCAD, VersaCAD etc. It is also possible to choose a shape from a library of finished prototype drawings, and to specify the size needed. For example, ring shapes in many sizes are kept in the standard library. When the drawings are completed, the details for each shape are stored in a temporary file, so the student can choose the best layout for cutting them from the plate, without wasting materials. Edit Function. A construction drawing contains certain information and details, such as dimensions, which do not need to be sent to the manufacturing equipment. However, it is necessary to add data that determine processing speed, processing direction, type of tools, and other aspects of the manufacturing process. Thus, the computer system must include edit functions to allow deletion of redundant information from the construction drawing and the addition of pertinent processing data. It is also important to be able to edit the operating data before transmitting them to the processing machine to correct mistakes. Process Control. The program is designed to allow the use.r to follow the final manufacturing phase on the screen. During this processing phase, the flow of finished processing data can be interrupted if tools are broken or the cutting fire goes out. If processing is interrupted, the trainee can choose to operate the machine manually or to start an automatic reverse searching procedure to find out where to pick up the procedure and continue. Telecommunications Links. Since CNC machines are only available in a few locations, the software designers needed to make it possible for students to transmit the data from the exercise via an internal computer network to the processing machine (at another location within the school) or via the telephone network to more remote locations. This is also important because in industry, the design office is usually physically separated from the - 287 - Educational Technology: Chapter 7 processing areas, and it is important to prepare the pupils for these working conditions. Host of these linkages are still under development. Currently, MI-2000 is used in many Norwegian schools, and the number of users is increasing rapidly. Although it was created to meet the requirements of the educational system, MI-2000 has turned out to be very useful In industry as well. As a powerful, flexible, and userfriendly system, it provides industry with a useful training tool. A large number of companies, ranging in size from small one-person firms to the largest company for oil platform production in Norway, currently aro using MI-2000. In schools, MI-2C00 i,. used with cutting benches of ;no to three meters in size; industry uses it w,A-h benches of up to 60 meters. The remarkable thing is that the software and the computer equipment are the same for either case. In the future, the MI-2000 will have the capability to be linked to other projects that are being developed under the Program for Action, such as the distance learning network called Pegasus. Using the telecommuaiications links provided by Pegasus, it will be possible for students and instructors to work together on MI-2000 across long distances. The system will be able to transmit simple processing data as well as complicated graphs and pictures. III. Case Study: MI-2000 in the Kvadraturen Upper Secondarv School The Kvadraturen Upper Secondary School, which is in Kristiansand, in southern Norway, is one of the largest schools in the country and has a variety of branches, including a technical school. The school has played a large role in the development and testing of the MI-2000 program and provides a good opportunity for a c'.ose examination of how computer technology has been successfully introduced into a particular educational context. Adopting MI-2000 Teachers say that MI-2000 is the kind of project that inspires them to put energy into their work. The user threshold is low and the software is unable right from the basic level. However, adopting the new computer methods has not been uniformly easy for the staff. The school has found that the students learn to use the new information systems far more quickly and easily than many of the teachers. Many of the instructors are accustomed to working mostly with physical objects and find the switch to conceptual processes difficult. In contrast, students who are used to handling abstract problems can produce simple objects after just four hours with the MI-2000. Eventually, the teachers that do master the MI-2000 take on tasks that they did not previously think they could attempt. They also allow the pupils to experiment and to work on their owr., even with tasks that the teachers themselves have not yet mastered. - 288 - Educational Technology: Chapter 7 VCAD Detail Pretreating Record files Cut layout CAM-construction SIMULAT ION Code generating 2 mm ~~~Part 1 oZlete C@Layut *lles Parameterized slation drawings Plate cutting [ ~~~~~~~Preparation rep, 3 LX ~~L-3I- LCI Figure 3: With MI-2000, the trainee can create the design shape in tiiree ways: 1. Take a completed shape drawing from a CAD system like VersaCAD. 2. Use the MI-2000 cc.-puter design capabilities to construct a shape from scratch. 3. Take a prototype drawing from a shape library and use MI-2000 to specify the parameters. - 289 - Educational Technology: Chapter 7 OFFIC OFFICE _ WRSO | C^t l,~~ * PRNOiIPO _ na..m ,n | , Figure 4: An overview of the NI-2000 system. School Contributions to 1MC-2000 As a practical contribution to the program, Kvadraturen Upper Secondary School used the expertise units electronics and mechanical engineering departments to develop a way to rebuild existing optically managed cutting machines into computer-controlled cutting machines. The school has also used the program to build up a high level of competence in computer applications among its staff and instructors in the area of mechanical engineering for vocational training. As a result, the school has become a learning center for teachers interested in computer applications, both locally and throughout the country. In less than a year, the school has organized 14 courses for teachers in the region and three countrywide courses in mechanical subjects. Cooperation with the Private Sector Currently, contact between the school and the private sector is quite good. However, people at the school have described the difficulties they encountered in establishing a mutual cooperative arrangement between the school and industry. Before the computer systems were introduced, the school was perceived to be lagging behind in industrial technology. As a result, companies felt that they had nothing to gain from cooperative arrangements. For example, ten years after industry started to use~ computer numerical controlled processing, teachers were only just beginning to learn CNC techniques. - 290 - Educational Technology: Chapter 7 Now, however, according to the teachers the tables have been turned, and industry is lagging behind in the adoption of CNC processing. A considerable portion of the industry still uses traditional methods for programming equipment. Those involved with the MI-2000 project at Kvadraturen Upper Secondary School feel that this kind of program is the key to developing cooperation between their school and industry because it offers industry access to the latest technology. In order to adopt this technology, companies are eager to work with the school to reap the benefits of the training set-up that is already developed. Small and medium-sized firms are particularly eager to do so. It is still too early to say what practical outcomes the contact with the industry will have for the school. The school has made a profit on some of the services and products sold to customers outside the school system that are by-products of che MI-2000 project. The basic situation is good; by participating in the development and use of MI-2000, the school has gained a good reputation in the eyes of regional and national authorities. This has resulted in valuable contacts being made throughout the country. Among others, the Confederation of Norwegian Business and Industry has recognized that the school has succeeded in making a timely contribution to manufacturing development. The Royal Norwegian Council for Scientific and Industrial Research (NTNF) has also been supportive and has contributed resources to the school to support training courses that it offers to the private sector. - 291 - Educational Technology: Chapter 7 References International Learning Cooperative (INTEC). 1987. The Introducing of Computers in Schools: The Norwegiar Experience. Paris. Processed. - 292 - Educational Technology: Chapter 8 Chapter 8 Sustainability in Four Interactive Radio Projects: Bolivia, Honduras, Lesotho, and Papua New Guinea Thomas D. Twlson - 293 - .5ducational Technology: Chapter 8 SUSTAINABILITY IN FOUR INTERACTIVE RADIO PROJECTS BOLIVIA, HONDURAS, LESOTHO, AND PAPUA NEW GUINEA Thomas D. Tilson If you don't have the whole sequence of instruction designed correctly, if you haven't thought of every detail, if you're n3t out there collecting information about whether it works or not, you're going to end up with a product that sounds like interactive radio and doesn't teach any better than an average teacher could teach. And if you're not teaching any better than an average teacher, you have no business being on the air.1 Since its development in Nicaragua in 1974, interactive radio instruction (IRI) has earned its place on the air again and again. In Kenya, Thailand, the Dominican Republic, Papua New Guinea, Honduras, Bolivia, Lesotho, Costa Rica, Ecuador, Belize, and Swaziland, the methodology has proved supple and effective, adaptable to the major subjects in the elementary curriculum and to a variety of geographical, technological, sociological, and economic constraints. By providing superior instruction for students in the short range and enhancement of teachers' skills in the long range, IRI answers two crucial needs in development education, and it answers them rapidly. Interest in IRI has increased significantly in the last few years. Approximately 600,000 children are now learning from interactive radio in 10 nations across Latin America, Africa, and Asia. The effectiveness of the model depends on several factors. First, the lessons are designed to teach the core instructional materials; they are not just to supplement or to enrich. Thus, the lessons are intensive -- in most subjects, a half-hour lesson is broadcast each day. Second, each curriculum is designed according to proven instructional design principles. Third, the lessons incorporate sound pedagogical principles. The most obvious one is the active participation of the children: every few seconds, students are asked to speak, write, read, sing, manipulate materials, or do physical activities. For a conventional one-way delivery system, the lessons are remarkable in appearing to be truly interactive. The lessons also incorporate principles of distributed learning of new subject matter, immediate reinforcement oi correction of student responses, and a segmented format that offers continuous variety while providing repeated exposure to new topics and repeated opportunities to practice skills learned earlier. Finally, a system of rigorous formative evaluation that regularly monitors the quality of instruction as the programs are being developed also helps to ensure that the lessons are highly effective. lJamesine Friend, Project Director of Radio Mathematics Nicaragua and instructional design consultant for interactive radio instruction (IRI). - 294 - Zducarional Technology: Chapter 8 But imprersive learning gains are not enough to ensure sustainability. It is lncreasLngly obvlous ttihat other factors -- the technologLcal setting, the administrative capacity of local public and prlvate-sector educational instltutions, political eoLiditions in and among those instltutlons, and the cultural appeal of craditional methods of scbooling -- must be the focus of efforts to achieve long-term implementation for lnteractlve radlo Lnstruction. Barrlers to sustainability vary from country to country and from project to projett, but they are always powerful. Like all other educational lnterventlons in developLng countries, Lnteractive radio instruction depends on forces outside Itself for continulng success. The first and most dramatic example of this dependence came wlth the Nicaraguan revolution in 1979: it brought the pioreering IRI project, the Radio Mathematics Project, to a sudden and declslve halt. Other barriers to sustainability are less intractable than revolutions, however, and each IRI project since Nicaragua has lessons to teach us about how to confront and overcome them. The four case studies that follow examine interactive radio projects in Bollvla, Honduras, Lesotho, and Papua New Guinea. Together they constitute a representative record of the challenges to long-term implementatlon in developing countries. In the first section, Michelle Fryer, Chief of Party of the Radio Learning Project (RLP) activities in Bolivia, describes the future of IRI efforts in mathematics and health in Bolivia; this is followed by a cost study prepared by Dean Jamison of UCLA. In the second section, David Edgerton, Chief of Party for the Radio Lsarning Project activity in Honduras, sketches a model for private-sector lnstitutional sustainability. This is followed by a cost study based on the work of Patricia Godoy-Kain of Development Technologies, Inc. In the third section, Maurice Imhoof and Philip Christensen, both with the USAID/BANFES Project in Lesotho, outline the adaptation for use in Lesotho of Englibu in ActLon, a three-year cycle of lessons first developed in Kenya in the early 1980s, and Thomas T'lson, director of the Radio Learning Project at the Education Development Center, presents recent cost data for Lesotho. Finally, in the fourth section, Tom Roy, dlrector of the Radio Science Project in Papua New Guinea, describes the process of institutionalization of the Radio Science Project, which presents a complex new curriculum to students in the upper primary grades in Papua New Guinea. D.ch cost study was undertaken with distinct objectives, and the analysis for each study was somewhat different from the analyses for the other studies. Nevertheless, we have sought to adjust the findings of each study where possible to arrive at a common set of assumptions in such matters as the social discoant rate (7.5S), the life of ttie project (15 years), the life of the radio (5 years), etc. Thus, it is possible tc make direct comparisons among the results of the projects. These results, in turn, can be compared with findings from other studies, such as that of LocAheed and Hanushek (1988). - 295 - Educational Technology: Chapter 8 1. Nath and Health Education Throufth Interactive Radio: The Bolivian Radio Education Project Beginning in 1987, educators in Bolivia, with financial support from the U.S. Agency for International Development (USAID), began to develop, test, and implement a program of interactive radio to carry the main burden of instruction in mathematics in grades 2 through 5. Lessons for grades 2 through 4 are now available, and in 1991 a total of 18 transmitters will broadcast lessons in grades 2 through 5 to 70,000 students in five of Bolivia's nine departments. By that time, the development phase of the project will have been completed and implementation will be well under way. The Bolivian Ministry of Education and Culture (MEC) is now actively considering expansion of the project to all 600,000 Bolivian children in grades 2 through 5. In addition, the MEC, the Ministry of Public Health, and the Radio Education Project (REP) are contemplating the design of a radio- based health sciences curriculum fur grades 3 through 5. This new program would be based upon the health-education pilot project undertaken by REP in 1989. This review is, therefore, oriented toward assessing the cost and effectiveness of the Bolivian projects for decision makers who are considering national implementation of interactive radio instruction. The State of Education in Bolivia, 1980-86 Educational reform in Bolivia presents challenges. In 1986 the national economy was on the verge of collapse. The gross national product of this poor Andean counti.y had been steadily declining since 1980, and hyperinflation had firmly established itself at 24,000% (W.orld Bank 1989). Although the national constitution declares education the highest function of the state, Bolivia's unstable economy severely limited the government's ability to meet the public demand for high-quality schooling. As a result, the public education system was plagued by shortages of qualified teachers, an absence of inservice training opportunities, and numerous interruptions due to school strikes. According to MEC data for 1980, only 21% of all rural students who enter primary school complete fifth grade. Less than 1% of this cohort continues through high school. In 1987 a USAID study of primary school desertion in Bolivia noted an alarming decrease in the number of children who matriculated between 1976 and 1984 (CEBIAE, 1989). During this time, the primary-school gross enrollment rate dropped from 154% to 143% in urban areas, and from 91% to 79% in -ural areas. The study concluded that economic crisis was not only a motivating factor in parents' decisions about whether to send their children to school, but also a deciding factor in teacher abandonmernt. The inefficiency of the public school system has fueled movement in the wealthier urban centers toward private, fee-paying schools. At the same time, there is a parallel movement in the peripheral areas towards decentralization of education, or the private administration of public schools by nongovernmental organizations (NGOs). It was the collaboration of the - 296 - Educational Technology: Chapter 8 Education Development Center (EDC) of Newton, Massachusetts, with one such institution, Fe y Alegria (FYA), that gave birth to the Radio Education Project. Interactive Radio Education in Bolivia: Program Evolution and Prospects Radio Mathematics. The Bolivian Radio Education Project was initially designed to provide high-quality instruction at the primary school level through national dissemination of locally adapted interactive radio curricula. Before the initiation of large-scale activities, the staffs of FYA and EDC conducted a small pilot study with 450 students in 11 second-grade classrooms to assess the feasibility of adapting the Nicaraguan radio math curricula to Bolivia. Based on strong learning gains in the radio cohort, REP concluded that: (1) the radio math curriculum developed in Central America and adapted in the pilot not only meets but expands upon the official Bolivian curriculum; (2) Bolivian school children entering second grade have the necessary prerequisite skills in numeration, basic addition, and simple subtraction to "take-off' with the radio lessons; and (3) Bolivian teachers can effectively use the radio programs in their classrooms with a minimum of training. Encouraged by these positive findings, the REP began to adapt the radio mathematics curriculum for Bolivian primary school students in grades 2 through 5. As a first step, staff members compared the Nicaraguan radio mathematics curricula and the official BoAvian curricula. REP was able to match the two curricula at the second-grade level with straightforward cultural modifications and minor alterations in the way subtraction and multiplication were taught. The overall structure and mathematical content of the original programs remained largely untouched. Review of the third- and fourth-grade curricula indicated that a more extensive revision of the instructional package, equivalent to about 20% for third grade and 50% for fourth grade, would be necessary. REP began with the second-grade programs so that the curriculum team could gain experiences with IRI before undertaking the more complicated task of adapting the third- and fourth-grade lessons. A fifth grade radio math curriculum was not originally planned, but teachers soon began to suggest that REP maximize the longitudinal impact of its program by providing interactive radio math instruction through the final year of primarv school. In 1990 REP began development of the first interactive radio mathematics curriculum for fifth grade. The 25-minute interactive radio lessons (supported by complementary print materials for teachers) are broadcast every day of the school year. So far, REP has developed and broadcast 405 radio mathematics lessons for approximately 50,000 second-, third-, and fourth-grade students in Bolivia. Five of the country's nine departments have participated in the program to date; and in 1991 more than 70,000 students are expected to receive the radio lessons. Table 1 summarizes the growth in geographical, curriculum, and student coverage of REP. - 297 - Educational Technology: Chapter 8 Table 1 Growth of Radio Mathematics Coverage and Enrollment in Bolivia No. of No. of No. of No. of Grades Year Degartments a Transmitters Schools Students b Covere 1988 3 8 145 10,284 2 1989 5 10 182 12,600 2,3 1990 5 18 367 25,000 2,3,4 1991 5 18 900 70,000 2,3,4,5 a Out of nine departments nationwide. b Out of approximately 600,000 primary students in grades 2-5 nationwide. Radio Health. In 1989, encouraged by the success of the radio- based mathematics program, REP d3cided to test the feasibility of using interactive radio to teach health sciences concepts and behaviors to children in fourth and fifth grades. In a departure from traditional projects that target the adult population, particularly women of childbearing age, Radio Health was designed for primary school children in the 8-13 age group who often act as caretakers for younger siblings and engage in household activities relating to food preparation and sanitation. Because these older children have a major influence over the health and nutritional status of their younger brothers and sisters, it is critical that they be able to make informed decisions and to provide responsible care. "Diarrhea Prevention and Oral Rehydration" was selected as the subject of the test module because of its fit with ongoing child survival activities. Although infant mortality has decreased from 169/1000 in 1976 to 96/1000 in 1989, it remains the highest in South America. Despite massive efforts during the 1980s to promote child survival themes among the adult population, diarrheal disease (with resulting dehydration) is the leading cause of death among children under the age of five; and the National Institute of Alimentation and Nutrition estimates that one in two Bolivian children suffers from some degree of malnutrition. Although children living in marginal urban settlements are frequently victims of Bolivia's health crises, health education is not included in the official urban curriculum. Therefore, we developed the Radio Health intervention using a modular format consisting of ten 25-minute lessons related to a common theme that could be taught once a week during science class, physical education, or home economics. The IRI methodology was also adapted to slightly older students by emphasizing individual rather than group responses, incorporating role-play activities into the lessons, and using stories as dramatic illustrations of important messages. - 298 - Fducational Technology: Chapter 8 Cochabamba, a city in the high central valley, was chosen as the pilot site because of the recent proliferation of squatter sett: ants In its periurban area, the city's notoriety for problems relating to water contamination and availability, and its large bicultural population (Quechua). Since health behaviors and eating habits are influenced by sociocultural beliefs and attitudes, REP's ability to intr6rate these beliefs into acceptable messages was crucial. Staff memters undertook extensive field research into the perception of specific bealth problems and interventions, conducting individual and focus-group interviews with 50 parents and 50 children from the target area. Firsthand knowledge of the contextual reality of children in the _a:get group was gained during visits to participating communities to observe common sanitation practices; to evaluate how families acquire, use, and care for their water supplies; and to interview community health workers on outreacht acti ities. The health team also observed neighborhood schools. Once tk., joeconomic profile of the target audience had been established, REP ible tc d'e_velop an effective Instructional methodology that combines at... - model. .- and behavior change. Implementation REP is designed to develop and validate the interactive radio curriculum in Bolivian school3 and to transfer that developed technology to the MEC for large-scale implementation. The project is unique, however, in that the financial and administrative responsibi'lity for all research and development (R&D) activities is managed oy a local NGO, Fe y Alegria, which manages more than 2G0 schools under a government contract. Once the four- year R&D activity is complete in 1992 and the entire radio math curriculum validated, the finished programs will be integrated into the MEC operational portfolio with a minimum of restructuring and at a low cost. One great advantage of FYA management is that REP's R&D activities have been cushioned from external political forces. A second advantage of the arrangement is that the REP can control the validation of each new curriculum in FYA schools before large-scale dissemination. Al*hough REP does not have any formal linkage with the MEC at the central leN it does collaborate at the departmental level with MEC supervisors, who can g_ant permission for local dissemination of the radio lessons; and staff members are now participating in discussions with officials at the highest levels of the MEC about the transfer of the radio curriculum from FYA to MEC management in 1992. Summative Evaluation Results To assess the impact of the instructional radio programs, the REP employs both formative and summative evaluation strategies. Daily classroom observations and interviews provide qualitative data -- whether the children enjoy the lessons, whether teachers work well with the radio -- and tests provide quantitative data on achievement. This section reviews the second- - 299 - Educational T ehnology: Chapter S grade summative evaluation conducted in 1988, and states preliminary results from the third-grade longitudinal evaluatLon conducted in 1989. It also reports results from the summative evaluatlon of the radio health pilot. Second-Grade Summative Evaluation, Nathematies, 1988. The purpose of the summative evaluation is to measure the effectiveness of the interactive radio mathematLes curriculum in terms of learning outcomes. In the final two weeks of the 1987 school year, a mathematics achievement teat was administered to chlldren who were completing second-grade mathematies using conventional (teacher-led) instruction and who had not been exposed to the radio treatment. The following year, the same classrooms became part of the experimental group and a new cohort of second-grade students was given the radio mathematics lessons in place of thelr regular math program. At the end of the 1988 school year, the same posttest was administered to the students who used the radio lessons. A total of 1,554 students in 50 randomly selected urban, periurban, and rural classrooms ln three departments were Lneluded in the sample. REP and student learning. To measure the lmpact of the interactive radio math program ln comparison with traditional instruction, we used the effect size statistic, which calculates the mean difference between the control and experimental groups in terms of the standard of deviation of the control group. Effect size is a powerful educational statistic because it can measure the impact of an intervention using criterion independent of the specific test. The results in Figure 1 indicate that the Bolivlan Radio Education Project has yielded extremely high gains. - 300 - Sducational Technology: Chapter 8 Figure 1 Comparison of Posttest Means for Experimental and Control Classrooms Grade 2 Mathematics in Bolivia, Nicaragua and Thailand MEAN % CORRECT * E Radic Control 70 66% 66% 68% 60 l58 50 40 ||- 30 20 10 0 BOLIVIA NICARAGUA THAILAND THAILAND CENTRAL NORTH EAST PLAIN Effect Sizea: .94 .58 .24 .58 mEffoet sizes showqn here were computed from the same data. Lockheed and Hanushek (1988) have prov a--d a valuable sujmmary of the results of evaluations of the Nicaraguan, Thai, and other interactive radio projects as one element of a wide-ranging review of available evf'l-ice on the effectiveness and cost-effeztiveness of educational investments in developing countries. Rather than further describe specific results here, we simply note that interactive radio consistently had the highest effectiveness and among the highest cost-effectiveness of the projects studied. The effect of urbanization on student achievement. The test data from the Bolivian schools were next analyzed by location (urban, suburban, rural) to see what effect urbanization might have on student learning. In general, the distribution of learning gains in Bolivia follows the same pattern '.hat has been established in other countries using interactive radio. Figure 2 shows that the highest mean gains are recorded in suburban and rural - 301 EducatiouaA Technology: Chapter 8 schoo'ls where truditional teacher-led instruction is the weakest. The lowest gains are in the urban areas where cna generally finds better students, better teachers, and better schools. It is interesting, however, to aote that the experimental group in rurai schools not only demonstrated strong learning gains, but their mean test score (57%) exceeded the mean score for urban students in the control group (55%), thus lending support to thk. hypothesis that interactive radio instruction has a positive impact on narrowing the urban-rural learning gap. Figure 2 Comparison of Posttest Scores for Urban, Suburban, and Rural S-udents in Bolivia POSTTEST MEAN 80% |l Radio l 73 8 Control 70% I 67 60% 57 50% 55 MEAN GAINS: 40% Urban - 18% Suburb - 26% 30% 35 Rural - 23% 20% MEAN RANGE: R _ l _ _ a ~~~----------- 10% Control- 20% II Radio - 16% 0% 0 URBAN SUBURBAN RURAL Third-Grade Longitudinal Evaluation, Mathematics, 1989. In an attempt to measure the cumulative impact of the treatment over time, REP modified the third-grade evaluation design to include a tracer study of those children who were originally tested as part of the second-grade expr'rimental cohort, and who had gone on to complete the third-grade radio curriculum. The results indicated in Figure 3 show that after one year of radio mathematics instruction (third grade only), the experimental cohort's mastery nf basic mathematics was clearly superior to the control group's. The gain of the longitudinal group who had taken radio math in both second and third grades - 302 - Educational Technology Chapter 8 strongly suggests a cumulative impact of the instructional radio curriculum over time. Figure 3 Comparison of rosttest Means for Control, Experimental, and L-'ngitudinal Classrooms, Third Grade Mathematics in Bolivia MEAN % CORRECT Longitudinal * * Radio I Control 70 62 60 - 52 50 - 40 35 30 20 - 10 0 Effect Sizea: 1.29 2.03 aEffect sizes shown here were computed from the same data. Summative Evaluation, Health, 1989. The field evaluation of Radio Health was designed to assess the effectivehass of IRI in teaching health concepts and behaviors to upper primary grade students. In addition to regular formative evaluation, summative evaluation was used to validate the inpact of the instructional intervention, and focus-group interviews were employed to gain insight into parental acceptance of the Radio Health programs. The results of summative evaluation show improvement in students' understanding between the pretest and the posttest (see Table 2). - 303 - Educational Technology: Chapter 8 Table 2 Summary of Average Test Scores by Grade Grade 4 Grade 4 _ N 8 Correct N % Correct Pretest 224 51% 214 57% Posttest 211 70% 199 76% In particular, there were five areas En which we were able to note highly significant gains (p<.001) in student knowledge over the course of the pilot. These include: 1. Knowing the amount of liquid that an infant or child with diarrhea should drink (pretest-22%, posttest-89%) 2. Understanding the correlaeion between bottle feeding and frequent diarrhea (pretest-43%, posttest-72%) 3. Recognizing one or more symptoms of dehydration in a drawing of a babv (pretest-12%, posttest-80%) 4. Identifying one or more ways in which older children can help younger children with diarrhea (pretest-40%, posttest-80%) 5. Describing how to mix homemade oral rehydration therapy (ORT) correctly (pretest-l%, posttest-46%) Assessing Effectiveness: The Impact of Time and Radio on Learning For the cost-effectiveness analysis, two basic measures of outcome were sought: the effect of exposure to the REP on test scores and the effect of exposure to one year of traditional schooling on test scores. While many analyses of the impact of educational interventions like interactive radio provide an assessment of impact in terms of increase in parcent correct on a test or, better, in terms of effect size, this assessment remains, inevitably, only partial because it lacks a metric by which to judge the aducational± (as opposed to statistical) significance of whatever effect size is observed. Hence the importance of providing a measure of the effect size associated with a year of schooling; this provides a natural metric for judging the educational significance of alternative interventions like REP. - 304 - Educational Technology: Chapter 8 The Effect of a Year of Traditional Schooling. One important facet of the cost-effectiveness evaluation was the collection of a new set of achievement data that would allow explicit comparison of the impact of the REP with the impact of a year of traditional mathematics instruction. To undertake this assessment, the same test was given simultaneously to second and third-grade students in 12 schools in and near the city of Cochabamba. Students from both radio and control groups at both grade levals were tested, and the selected schools included rural, suburban, and urban. The result of this pattern of testing provides a better sense of "normal" (non-radio) progress, and allows both for interpretation of the meaningfulness of time of exposure to traditional and radio mathematics instruction. Table 3 summarizes our analyses of the effect of one year of schooling (from mid-second to mid-third grade) on student performance in mathematics in Bolivia. The results are reported in the aggregate as well as separately for the rural, suburban, and urban samples. The first row shows, for each sample, the average percent correct for second graders on the test; below that mean is the standard deviation for the group. Note how much better the urban children did than the rural ones, but note too, that the increment in performance of urban children over the others is being eroded as children remain longer in school (row 3). The effect sizes reported in row 4 are substantial--clearly showing considerable student cognitive improvement associated with additional schooling. Table 3 The Effect of One Year of Traditional Schooling on Mathematics in Bolivia Rural Suburban Urban Total Sample Grade 2 Mean (%) 16.5 17.5 31.0 22.5 Grade 2 S.D. 8.5 14.5 15.5 13.5 Increment of 3rd over 2nd Grade Score 23.5 31.0 16.5 23.5 Effect Siza 2.8 2.1 1.1 1.7 The Effect of the REP. Table 4 shows the effect of the REP for both second and third graders in all three educational environments. The base score (in row 1) is the control school mean; row 3 reports the increment in percent correct of the REP schools over the radio schools; and, again, row 4 reports effect sizes. Findings from the third grade in suburban schools are anomalous in being only very slightly positive; effect sizes averaging .9 are very substantial indeed. Another way of conveying this finding is that increasing educational quality (by introducing the REP) provides a 50% - 305 - Educational Technology: Chapter 8 increment in the achievement provided by a year of schooling, i.e., it is increasing achievement by a remarkpble 50%. Table 4 The Effect of the Radio Education Project (REP) on Mathematics in Bolivia Rural Suburban Urban Total Sample 2nd 3rd 2nd 3rd 2nd 3rd 2nd 3rd Control School Mean (%) 16.5 40.0 17.5 48.5 31.0 47.5 22.5 46.0 Control School S.D. 8.5 15.0 14.5 17.0 15.5 20.0 13.5 17.5 Increment of REP over Control Score 17.5 19.0 13.5 0.5 8.5 26.5 12.5 15.5 Effect Size 2.1 1.3 0.9 1.03 0.5 1.3 0.9 0.9 Assessing Costs The REP's cost structure reflects, principally, incremental costs to those of the ongoing system of primary education. However, since the REP operates within the regular classrooms and requires the availability of the regular teachers, the total costs of REP must also include those costs. This section begins with a very brief assessment of the incremental costs of REP and concludes with estimates for the costs of nationwide expansion. Costs of Traditional Instruction. Jamison (1990) provides an assessment of the cost of primary education in Bolivia. In summary, per- student annual salary costs (teacher and administration) come to about $35; non-salary recurrent costs come to about $9; and (annualized) capital costs come to about $20. The total per student annual costs are about $64. (Capital costs are annualized throughout this report at a 7.5* rate of interest with an appropriate estimate of the lifetime of .ne capital.) Bolivian schools typically operate on a schedule of 25 periods per week for a cost, given the above, of about $2.60 per year for one period per week. Traditionally, mathematics instruction is allocated six periods per w*dk for a prorated cost of $15.60 per year. The REP provides mathematics instruction in only five periods per week for a cost of $13 per year plus the incremental cost of the radio instruction. The important point to note is that the transition from traditional to radio-based mathematics instruction frees up one period per week. - 306 - Educational Technology: Chapter 8 Costs of the REP. To a reasonable first appros' Ation, the incremental costs of educational technology systems can b4 divided into those that are fixed independently of the number of students (F) and those that vary directly with the number of students (V). (Jamison, Klees, and Wells, 1978, provide a description and application of methodologies for costing educational technology systems, including a fuller description of fixed and variable costs.) The total annual cost TC(N) of serving N students per year is, then, in this formulation: TC(N) - F + VN The average cost per student, AC(N), is the total cost divided by the number of students: AC(N) - TC(N)/N - F/N + V To put this slightly differently, the average cost is simply the cost directly associated with serving each student (V) plus each student's share of the fixed costs (F/N). Jamison (1990) provides detailed information on the various components of REP's fixed and variable costs, the results of which we suimarize here. In the REP the main fixed cost is the capital cost of lesson preparation; somewhat less important are project start-up costs. Capital costs must be "annualized" to put them into their annual equivalent terms -- think of annual mortgage payments as the annualization of the value of a mortgage loan. As with the annualizatior. of school construction costs reported earlier in this section, we use a 7.5% interest rate and an appropriate estimate for lifetime of the capital costs. Something over $1.1 million is the estimated cost for preparing 540 lessons, resuliing in a cost of about $2100 per 30-minute lesson. Assuming a 15-year lifetime for the lessons and start-up activity, the annualized costs for these items is $140,000 per year (F). (Note that the way the REP cost analysis is structured, there are no fixed recurrent costs.) Radio receivers constitute the main variable capital cost, and, assuming a five-year lifetime for the receivers, the annualized receiver cost comes to about $0.11 per student per year. Variable recurrent costs are estimated to come to $0.70 per student per year. These figures allow us to be speciiic about the total cost function for the REP: TC(N) - $140,000 + $0.81 (N) Table 5 shows the values for total and average cost of the REP for a range of values of N. For the estimated 1991 enrollment of 70,000, total cost is $197,000 (-140,000 + .81 x 70,000), and the average cost is $2.80 per student. Because the fixed costs are spread over far more students at the nationwide implementation level of 600,000 students, per-student costs fall to $1.04. - 307 - Educational Technology: Chapter 8 Table 5 Variation in Total and Average Annualized Costs (Including Start-up and Lesson Preparation Costs) With Student Usage Annualized Cost Number of Students (N) Total Average (per Student) 25,000 (1990 actual) $160,000 $6.40 70,000 (1991 est.) 197,000 2.80 150,000 262,000 1.74 300,000 383,000 1.28 600,000 (all Bolivia) 626,000 1.04 It should be remembered that this $1.04 is only the incremental cost of adding the REP; in addition, the estimated $13 cost per year for classroom and teacher must be added to get the total cost of $14.04. This total cost is still less than that for traditional mathematics instruction ($15.60) because traditional instruction requires six periods per week rather than five. Costs of Nationwide Expansion. The ongoing developmental phase of the REP comes to a close late in 1991. The product of that effort will be lessons and teachers' guides for radio mathematics in grades 2 through 5, in addition to well-developed experience with implementation structures (including teacher training and supervision) for reaching large numbers of geographically dispersed students. By 1992 the REP will be ready for nationwide implementation, and the MEC has expressed its interest in moving to national implementation at that time. The costs of lesson preparation and start-up are no longer relevant from the perspective of national expansion, since those costs will, by then, have been sunk. The main relevant cost ingredients are the recurrent cost estimates and the costs of the receive's. Given an assumed Rac2 of national implementation, one can use the available information on costs to generate an estimate for the cost of national implementation; Table 6 provides such an estimate. The estimates in Table 6 assume only a slight expansion in coverage between 1991 and 1992 (from 70,000 to 100,000) but rapid expansion in the following two years, and then a tapering off. On this schedule, virtually all of the (by then) 700,000 students in grades 2 through 5 would be served. The estimated cost for the five-year program is about $2.5 million, including a "contingency" estimate of 20%. About 29% of the costs are for the students' notebooks and pencils, leaving a cost of $1.8 million to be centrally - 308 - Educational Technology: Chapter 8 tinanced. (Receivers and batteries, about 20% of the cost total, could be financed at tb school level, although national implementation will be reaching more .emote and poorer areas, there is a case that these expenses, too, should be centrally co-ered.) Conclusion: The Cost-Effectiveness of REP To establish the context for the discussion of cost-effectiveness, two general observations are relevant. First, the finding of an effect size of .9 for one year of implementation of radio math places the effectiveness of interactive radio's implementation in Bolivia at the top of the range of what IRI has achieved so far. The finding reported here is consistent with results from previous summative evaiuations of the Bolivian REP, which also show, significantly, that gains continu- over at least two years of implementation (with effect sizes around 2 for that period). Second, although REP costs are within the range of cost experience with other implementations of interactive radio, they fall within the high side of that range. (See Tilson (1990), for a summary of other findings on costs.) Most previous assessments of the cost-effectiveness of interventions to improve the quality of education in developing countries formed the ratio of incremental effectiveness to incremental cost; Lockheed and Hanushek (1988) refer to this ratio as the "efficiency" ratio when incremental effectiveness is measured in units of effect size and incremental cost is measured in terms of dollars per student per year. Providing textbooks, according to the findings of Lockheed and Hanushek, results typically in a very attractive efficiency ratio of about .2 effect size units per dollar per year; in one exceptional example, in the Philippines, the gain was almost 1.5 per dollar. Most other interventions reviewed were less cost-effective except for interactive radio, which had cost-effectiveness in the .3 to 1.3 range. Assuming, for the Bolivian REP, an effect size of .9 (Table 4) and an incremental cost per student per year of $0.81 (see section above, "Costs of the REP"), we obtain an efficiency estimate of 1.1 units of (incremental) effect per dollar per student per year. By assessing the effect size associated with a year of traditional instruction in Bolivia this evaluation was also able to weigh the cost- effectiveness of the REP against that of traditional instruction. Here total costs must be weighed against total learning gains. It was estimated above that the annual cost of providing traditional mathematics instruction for the scheduled six periods per week was about $15.60. The total annual cost of the REP is equal to its incremental cost of $0.81 plus the cost of teacher and classroom for its scheduled five periods per week; this total is about $13.80 per year. From Table 3, an effect size of about 1.7 per year was estimated for traditional instruction across all locales tested; this results in a ratio of 0.11 for total efficiency (i.e. total gain in achievement divided by total cost). For the REP the total achievement gain is about 2.6 (- 1.7 + .9) and the total cost is $13.80 per year for a total efficiency ratio of .19. The cost-effectiveness of mathematics instruction in Bolivia using the REP is, therefore, about 70% higher than the cost-effectiveness of traditional instruction. The efficiency argument for switching to REP is, therefore, very strong. - 309 - Educational Technology: Chapter 8 Table 6 Costs of National Implementation, 1991-1996 1992 1993 1994 1995 1996 Totals Amount %b Students Served (thousands) 100 350 600 700 700 2450 Costs (thousands of dollars) Radio Transmission 35 35 45 45 45 205 10% Radio Receiversc 24 120 120 48 20 332 16% Batteries (for 50% of 45 13 24 28 28 97 4% receivers)' Mainline Power (for 50% of 0.5 1.5 2 3 3 10 0% receivers)O Teachers' Guides 0 20 14 7 20 61 3% Teacher Training 14 50 85 100 100 349 17% Supervision, Administration 18 65 110 125 125 443 21% Student Notebooksd 25 90 150 175 175 615 29% Subtotal 116.5 394.5 550 531 516 2112 100% Contingency (at 20%) 23.5 78.5 110 106 103 422 Total 140 473 660 637 619 2534 a This table gives cost estimates based on the unit cost assumptions and equipment lifetimes described in Jamison (1990), and the pace of implementation indicated in row 1 of this table. Slight adjustments have been made where appropriate and there has been some rounding. b The figures in this column indicate the percent of the subtotal, excluding contingency. c During the project development phase (1987-91) these costs have been covered by the local school or community. d During the project development phase (1987-91) this cost has been covered by the students' families. alternatives, is extremely attractive -- despite the relatively high- cost environment that the Bolivian terrain imposes. - 310 - Educational Technology: Chapter 8 II. Sustainabilitv and the Private Sector: Interactive Radio gnd the Honduras AVANCE Proleat The AVANCE Project in Honduras was the first attempt at private- sector institutional development in the history of interactive radio. All interactive radio projects previous to AVANCE had been designed mainly as methodology and materials development, not as institution-building efforts. Though conceived with hopes of institutionalization in some form, all were public-sector pilot educational interventions, run from within ministries of education or ministry-run school media offices. The AVANCE project's principal distinction was that its interactive radio component evolved into an attempt to establish a permanent, private, nonprofit educational media agency specializing in the interactive radio system. This section describes how and why the AVANCE interactive radio effort was undertaken; reviews the technical success of the undertaking; discusses factors that contributed to and hindered its development as a private-sector institution; and suggests ways in which the AVANCE experience might apply to institution-building efforts in school radio and other appropriate-technology interventions in development education. Historieal Overview Interactive radio is one of seven interventions that together comprise the Primary Education Efficiency Project, a broad-based educational reform effort initiated by the USAID Mission to Honduras in 1986 and now in its final stages. The interactive radio activity was separated from the other six and instituted under the aegis of AVANCE, a small Honduran PVO that had already been in existence for nearly two years. AVANCE's sole activity at the time was the publication of El Agricultor. a weekly rural development newspaper. In April of 1986, USAID/Honduras drew up a cooperative agreement with AVANCE extending funding and technical assistance for El AgrLcultor and instituting an interactive radio activity. The total amount earmarked for the extension was just over $5 million for a period of six years. Technical assistance for all AVANCE operations, including heavy assistance in administration, management, and marketing, was set at $2.16 million over four years (1986-1990). It was provided by the Radio Learning Project, a consortium led by the Education Development Center (and funded by USAID's Bureau of Science and Technology/Office of Education), and included the Academy for Educational Development (AED) and Friend Dialogues, Inc. (FDI). The arrangement wlth AVANCE came about in part because USAID viewed the organization as a convenient institutional home for interactive radio. The Honduran Mlnlstry of Education agreed willingly to cooperate with an interactive radio undertaking, but did not want the activity directly within - 311 - Educational Technology: Chapter 8 its purview, perceiving that, at the time, it could not successfully absorb or manage a school broadcast operation. USAID viewed the marriage of interactive radio to AVANCE as an opportune experiment in privatization. AVANCE was instituted under an oversight assembly of Honduran business leaders. USAID believed that this assembly: and the nine-member AVANCE Board of Directors elected from it, represented an opportunity to test the application of private-sector energies to the problems of development education. AVANCE's mandate was greatly expanded under the terms of the 1986 cooperative agreement. Sani Radio, a USAID-funded 10-kilowatt shortwave community radio station in the Moskitia, on Honduras's remote Atlantic seaboard, was placed under AVANCE's administrative provenance. A desktop publishing system was procured with technical-assistance funds, and a commercial "quick-print" operation was opened in AVANCE's administrative offices in an attempt to generate income. Unrealized plans were drawn up for procuring a rotary press and putting AVANCE into the textbook-publishing business. An AVANCE Marketing Division emerged in early 1987 in an attempt to sharpen and professionalize AVANCE as a commercially competitive operation. AVANCE's Private-Sector Mandate in Its Beginnings Given this growth in AVANCE's resources and activity, the nature of the private-sector mandate itself was vaguely defined in the text of the Cooperative Agreement. The terms of the agreement specified financial break- even by the project end-date in 1992, but did not elaborate. Some 50 person- months of technical assistance in small-business management, financial planning, and so on, were mandated, plus a long-term advisor in social marketing. AVANCE's administration responded with a set of ambitious income- generating schemes. It was decided that El Agricultor should grow more commercial, and would then become profitable, in spite of a history of nearly four years of losing money. It was envisioned that by the end of 1988 AVANCE would be operating a rotary press at a profit; that revenues would be generated from the sales of commercial marketing services; that the quick- print operation would flourish; that Sani Radio would bring in income from local advertising and private foundation grants. Interactive radio, it was decided, would be AVANCE's public- spirited enterprise, the educational half of the Cooperative Agreement's dual mandate to make education and also to make money. Interactive radio would be sustained by the other, more profitable, AVANCE divisions. Early Attempts at Income Generation Via Interactive Radio AVANCE determined that the interactive radio operation could raise some income through sales of radio receivers and related print materials, and through commercial sponsorship, though it would always depend on AVANCE's net solvency to sustain it. - 312 - Educational Technology: Chapter 8 The radio receivers were marketed directly to Honduras's rural school teachers. Commercial soft-drink sponsorship was obtained for the student workbooks and classroom posters used with AVANCE's interactive radio programming, in return for printing soft-drink ads on the back covers of the workbooks and on the posters. Airtime was donated by the country's two largest private broadcast networks. AVANCE's Interactive Radio Output The interactive radio operation developed well. Interactive radio had been identified originally as an appropriate intervention for Honduras, not only because it is proven and powerful, but also because its pattern of program development -- the complex but predictable cycle of design, production, and evaluation activities -- is by now a matter of established, replicable practices. AVANCE's first interactive radio undertaking was La Familia de los N4meros. a course in mental calculation for children in grades 1 through 3. The project design called for AVANCE to replicate Radio Mathematics, the original interactive radio course developed in Nicaragua during the 1970s. Radio Mathematics/Nicaragua is a complete, largely self-contained courE However, USAID was simultaneously planning to fund the printing and distribution of new elementary mathematics textbooks, and therefore, the Nicaragua radio math series would have resulted in two competing curricula. The solution to this potential conflict was to design a new radio math project, a course aimed at mental calculation, a commonly neglected and vitally useful skill. AVANCE decided that this new series should be designed generically to complement conventional elementary math instruction using any standard textbook. An additional result of the generic design was that the course might actually generate income through sales to other countries. AVANCE market research showed solid rates of acceptance among teachers. As of this writing, La Familia de los Ndmeros is in use by nearly 200,000 students and over 6,000 teachers in Honduran classrooms. During the 1988-1990 period, the participation of teachers and students in the program increased continuously. Table 7 below shows the distribution of teacher and student participants at the national level. The number of teacher participants in 1990 comprises 42% of total teachers in grades 1-3 in the nation in 1990. The number of student participants in 1990 comprises 34% of total students registered in grades 1-3 in 1990. - 313 - Educational Technology: Chapter 8 Table 7 Teachers and Students in 1st, 2nd and 3rd Grades Participating in La Familia de Los Numeros in Honduras Grade 1 Grade 2 Grade 3 Combined 1988 Teachers 1,815 1,815 Students 55,550 55,550 1989 Teachers 2,237 902 3,139 Students 67,110 25,936 93,046 1990 Teachers 2,498 2,446 1,092 6,036 Students 92,221 75,666 27,795 195,682 Total Teachers 6,550 3,348 1,092 10,990 Students 214,881 101,600 27,795 344,278 Source: AVANCE/SEI, Tegucigalpa, Honduras, September, 1990 La Familia de los Ndmeros was introduced to first-grade classrooms simultaneously with the new textbooks. Thus, the experimental design had three groups: * traditional classes with a few old textbooks and no radio lessons * textbook classes in which all the children received new textbooks and teachers were provided limited training * radio and textbook classes in which both the new textbooks and the radio series were introduced La Familia de los Ndmeros proved to be a markedly effective interactive radio course. The evaluation team of SEI has collected annual information on learning gains for first, second, and third graders. This information was used by Friend Dialogues to prepare in-depth statistical analysis of learning gains. Statistical analysis of first graders was made at the end of the 1988 school year and of second graders at the end of 1989. - 314 - Educational Technology: Chapter 8 Third-grade analysis is not available because 1990 was the first year of transmission. The results of evaluation reported in Galda et al. (1990) found significant improvements in children exposed to the interactive radio program stressing mental calculation. She performed a comparative analysis between three different groups: Group A: children with traditional instruction consisting of a few old textbooks and no radio lessons Group B: children with new textbooks, whose first-grade teachers received limited training Group C: same as group B, but the with the addition of the interactive radio math lessons The traditional group, Group A, was tested at the end of 1987 before the new textbooks were available. Groups B and C were tested at the end of 1988. Groups A and C are composed of children from the same region (Francisco Morazan) and from the same schools, tested a year apart. These groups are considered highly comparable since they come from similar socioeconomic conditions and had similar teachers. Group B is in a different area of the country (El Paraiso) and could have better or less qualified teachers and/or better or worse economic conditions. Because there was a time constraint, a pretest for this group was not given. In order to compensate for the lack of pretest, comparisons between school systems and the children who attend them were made. It was found that the average score in Francisco Morazan was 51.4% and in El Paraiso 52.5%. With a t-statistic of 0.85 and p-0.395, differences in average scoring are insignificant and thus the groups are compirable. Effect size is the ratio of average score difference between the traditional and control group to the standard deviation of the control group. In general, effect sizes of .8 are rarely achieved, and thus there is confidence in asserting that the interactive radio mental calculation program with textbooks has been of great benefit (see Table 8). The size of the differences is significant for all three groups. In grade 1 the effect size of improvements over traditional instruction as a result of new textbooks is .43. The effect size of radio combined with new textbooks is nearly twice as large: .80. It is evident that new textbooks and radio lessons have been effective, and it is clear that a combination of the two yields significantly high learning gains. It is not possible to say with certainty that radio alone could have produced the same learning gains because there was not a group that used only the radio program as a learning tool. Also, because the radio program was designed to work on mental calculation, a skill in which the children were weak, and does not cover the entire official curriculum, it could not be credited with gains in other areas. - 315 - Educational Technology: Chapter S Table 8 Statistical Analysis for First Grade Mathematics in Honduras, 1988 Traditional Textbooks Textbooks and Radio Sample Size 553 613 473 Mean Score 34.3% 43.7% 51.9% Standard Deviation 21.9 22.1 25.2 Effect size of difference: Radio & textbooks vs. Textbooks .37 Radio & textbooks vs. Traditional .80 Textbooks vs. Traditional .43 Source: Jamesine Friend, 1989 Preliminary statistical analysis of second graders (see Table 9) shows significant differences between Group C (children with new textbooks and radio) and Group A (children with traditional instruction). However, the schools using only the textbooks in the second grade did no better than schools in the traditional or control group. In fact, they scored somewhat lower (48.6% with textbooks vs. 52.5% in traditional classes, resulting in a negative effect size of -.19). This may be accounted for by the fact that the textbooks arrived late, about halfway through the year. In addition, there were accompanying teachers' guides, but no teacher training as was provided for first-grade teachers. Whatever the reason, assuming that the circumstances with the textbooks were the same in both the "textbook classes" and the "radio-plus- textbook" classes, it appears that the positive impact in the latter group of classes can be accounted for by the radio lessons. The Familia de los Nimeros development/production cycle is a model of contemporary interactive radio practice. The key factors accounting for the success of the series were curriculum design, formative evaluation, and scriptwriting/production. The curriculum design effort was led by the math educator who developed the original Nicaraguan math series. Much of the work, in fact, was done in the United States with very close collaboration with the curriculum specialists at AVANCE in Honduras. Contemporary communications technology enabled this type of international collaboration. The curriculum was based on proven principles of instructional design, results of up-to-date research on how children learn mathematics, and sound pedagogical principles, one of which is critical to IRI: children learn best - 316 - Educational Technology: Chapter 8 Table 9 Statistical Anal-sis for Second Grade Mathematics in Honduras, 1989 Traditional Textbooks Textbooks and Radio and Radio Sample Size 393 500 411 Mean Score 52% 49 66% Standard Deviation 19 19 17% Effect size of difference: Radio & textbooks vs. Textbooks .77 Radio & textbooks vs. Traditional .61 Textbooks vs. Traditional -.19 Source: Jamesine Friend, May 1990. when they are actively involved. Interactive radio lessons are noted for the way in which children actively participate. Every few seconds they speak, read, write, manipulate materials, sing, or do physical activities. It is this activity that leads to the descriptive term, interactive radio instruction. The Familia de los Ndmeros formative evaluation component consisted of daily visits to several schools to observe how well the children understood each lesson and to identify problems. In addition, every couple of weeks the children were tested to see how well they were learning the math content. The findings from the classroom observation and achievement tests were used to modify future lessons as well as to revise the lessons already produced. By the end of 1988, by all measures, the 17-person AVANCE interactive radio staff was both the largest and the most highly qualified technical group of its kind in the world. Production never fell behind over three years of intensive work. Production values were high. Over the next two years the technical capability and flexibility of the interactive radio operation continued to grow at a steady rate. In January 1988, AVANCE installed a small, but excellent audio recording facility in the Interactive Radio Division offices. The personal computers in those offices constituted the first full computerization of interactive radio instructional design and script development. Also in 1988, AVANCE initiated an adaptation of the Kenya Radio Language Arts Project, an interactive radio course in English as a second language for children in grades 1 through 3, developed in the Republic of - 317 - Educational Technology: Chapter 8 Kenya during the early 1980s. The result, Aprendamos Ingles (Let's Learn English), a 100-lesson introductory English course for Spanish-speaking preschoo;rs, was the interactive radio operation's first attempt at a commercial venture. With funding from the Radio Learning Project, AVANCE piloted the series among Hispanic minority populations in nearby Belize. The interactive radio operation's marketing unit prepared a commercial package of the series on cassettes for sale to schools. Meanwhile, with funding and support from the Radio Learning Project, AVANCE sold to the Ministry of Education of Costa Rica the La Familia de los Numeros series (tapes, scripts, reports, etc.); AVANCE also provided technical assistance. By the first months of 1989, in addition to the mathematics and English series, the interactive radio operation was at work on several small public-interest radio series under commercial contracts. It had also begun initial design work on a major new interactive radio series in reading designed to complement the reading texts developed by the Primary Education Efficiency Project's textbook component. With a recording studio, computerized design capability, local commercial contracts, growing international connections, and a dedicated, highly qualified staff, AVANCE's interactive radio operation was developing the contours of a permanent, self-sufficient institution. Nevertheless, in spite of the success of the interactive radio series, by 1990 AVANCE was not achieving economic sustainability. The newspaper El Agricultor was excellent and served an important role in Honduras, but it continued to operate at a substantial financial loss. The social marketing expertise, while providing useful support to the AVANCE activities, never generuted outside income as originally planned. The new quick-print business was the only component showing a potential for profit. Thus, the overall financial weakness of AVANCE, combined with an unsettled management condition, led to a decision by USAID/Honduras to withdraw financial support from AVANCE by the end of 1990. USAID will continue support for sustaining the radio math series, although the exact mechanism for doing this has not yet been worked out. AVANCE will seek to continue with some of its operations without USAID support. The following description of the lessons learned about sustainability focus on the interactive radio activities of AVANCE -- the most successful component of AVANCE and the one relevant component for this book. Lessons in Sustainability from AVANCE Interactive Radio Experience There are established international precedents for privately based, public-interest media institutions. Elements of the model can be found in public-interest media agencies in Latin American nations, particularly Mexico and Colombia; in school television agencies at the state level in the United States; and in the fundraising and income-generating practices of the most successful agency in the history of educational broadcasting, the Children's Television Workshop. - 318 - Educational Technology: Chapter 8 The elements of the AVANCE interactive radio sustainability model are: * A three-pronged income-generating capability, consisting of: - Fundraising to cover program development costs. Public-interest broadcast production of any kind must always be subsidized. In the British and French models, the tradition of government subsidy of broadcasting in the public interest assumes powerful cultural proportions. In the U.S., the practice of corporate grants to public-interest broadcasting is suv-orted by tax advantages and growing tradition. In the developing- world version of the U.S. model, international donor agencies substitute for corporate grant sources. While AVANCE's fundraising skill remained largely undeveloped, AVANCE and USAID both recognized the importance of fundraising to the sustainability of the interactive radio operation, and some efforts were made to develop such capacity. - Commercial sale of secondary products and spinoffs. - Income from international contract packages by selling instructional materials and technical assistance for regional adaptations of programs. Beginning nearly two decades ago, Children's Television Workshop uncovered a vastly profitable market niche for itself in the sale of sophisticated technical-assistance and rights packages for international adaptations of "Sesame Street". On a far more modest scale, but appropriately and impressively, the AVANCE interactive radio operation developed regional international contacts with very hopeful potential for enhancing sustainability. * A directing or governing body with a genuine stake in agency operations. The need for such a body is a key lesson to be learned from the AVANCE experience. The interactive radio operation might have been instituted more successfully as a very small, independent, private, nonprofit corporation, governed by a body consisting entirely and exclusively of educators, professional broadcasters, and others with a genuine professional stake in project operations. Alternatively, in developing nations with stable higher academic institutions, a sciiool media agency might be safely instituted as an operation within a college or university. Several long-established state sehool-television agencies in the United States are operated from the campuses of state universities, enjoying the benefits of proximity to the university's creative and academic resources. - 319 - Educational Technology: Chapter 8 * A close, stable relationship with public education. This may seem too obvious to bear discussion; but use of any instructional technology in a nation's public schools must begin with its collaborative acceptance by the office of government that empowers public education; and its sustained use depends on its thorough and willing institutional incorporation into public education. The AVANCE Experience With the AVANCE-inspired sustainability model, instructional radio is provided to public education from the private sector with funding derived in part from income-generating capabilities. Nothing in this model is radical or unusual. Nothing in it strains against realism or common practice in development. On the contrary; the AVANCE experience shows that the private-sector self-sufficiency model is a natural, even predictable, evolutionary course for an instructional- technology development project to follow. The AVANCE interactive radio operation evolved to the threshold of permanent, sustainable self-sufficiency. Though its future remains uncertain, it may yet achieve permanence. Whether or not it does so, the model for sustainability that emerged from the AVANCE interactive radio experience validates a variety of sustainability measures in use worldwide in similar circumstances, and it offers a composite practical model for reference in planning for private-sector sustainability of instructional technologies in development. Cost Analyses Development Phase. The IRI program La Familia de los Numeros has been developed and implemented over a four-year period from 1987 to 1990. The development stage of the project consisted of informational research, production and design of instructional material (i.e., lesson design, scriptwriting, teacner guides, student workbooks, radio production, transmission of lessons, evaluation of learning gains, and teacher training), and distribution of packages throughout the country to teacher and student participants. Development and implementation of a given grade level instruction program occurred simultaneously in the sense that the final preparation of individual lessons preceded actual airing on the radio by as little as one week. Also part of the development process were the establishment of a network for the sale of the packages to the teachers through supervisors and the establishment of relationships between project staff and radio station owners. - 320 - Educational Technology: Chapter 8 Total Investment Expenditures. Total investment in this project, from inception through 1990, amounts to 6,930,952 lempiras or US $3,465,476, using an exchange rate of US $1 - 2 lempiras. This number was arrived at by adding the expenditures for technical assistance provided by the Radio Learning Project through the Education Development Center, the Academy for Educational Development, and Friend Dialogues, to the expenses of AVANCE. All costs for developing the lessons and some of the implementation costs were provided by the USAID mission in Honduras. The remaining implementation costs were provided by private business (through donated radio transmission time and payment for producing some of the instructional materials), and by teachers who purchased radios and instructional materials. The total investment, including costs for developing the programs and implementing them on a national basis, is shown in Table 10. Table 10 Distribution of Total Investment Costs ('000 Lempiras) 1987 1988 1989 1990 Total Development 739 1,101 1,481 643 3,964 Implementation 6 798 1,126 1,036 2,966 TOTAL 745 1,899 2,607 1,679 6,930 Sources: AVANCE, AED, EDC, and FDI Table 11 shows the distribution of total project investments by source of funds. Table 11 Distribution of Total Project Investments by Source of Funds (Percent of Total) 1987 1988 1989 1990 Total USAID/HOND 100 81 92 84 89 Local Business 0 19 8 16 11 Source: AVANCE/SEI., Honduras, Sept. 1990 a Local business: airtime cost (1987-90), cost of posters and workbooks (1988) - 321 - Educational Technology: Chapter 8 The preceding figures reflect the actual investment costs. A full costing of the development costs, however, requires that an opportunity cost be included. The use of available funds for creating La Familia de los Ndmeros was, theoretically, only one possible option. The money could have been used for other projects, or put into the bank to earn interest for future use. Therefore, an opportunity cost, sometimes referred to as a social discount rate, needs to be applied to the investment amount. The table below shows the total investment cost assuming three different discount rates and a 15-year life of the instructional materials2: Table 12 Total Investment Cost Discount Rate 0.0% 7.5% 15.0% Total Cost in Lempiras 6,930,952 7,716,141 8,116,263 Total Cost in Dollars 3,465,476 3,858,071 4,058,132 Recurrent Costs. The main question for this section is "What will it cost in 1990 prices to continue the implementation of La Familia de los Numeros to the current number of about 200,000 children?" We will refer to these costs as recurrent costs, although technically, the radios, teachers' guides, and posters are capital costs in that they have a life span of more than one year. The amounts do not include amortization of previous investments nor do they include depreciation of existing fixed assets. A staff of one administrator and four technicians is required to sustain the current level of implementation and allow for a modest expansion of about 10 percent per year; this staffing level will allow for minor program changes, but not for major revisions nor for the development of new programs. The recurrent costs based on 1990 prices with 200,000 students are presented in Table 13. The costs in dollars are based on an exchange rate of US $1 - 3.4 lempiras. Note that this is a different rate t-'an that used for the development costs presented above. The reason for t1ie change is that during all but the last few months of the developmer't phase of this project, the exchange rate was approximately 2 lempiras per dollar. Beginning in January 1990, the lempira began to be devalued, reaching 5.5 lempiras per US dollar by mid-September when these data were collected. Thus, the impact of the devaluation on local costs of imported goods had not been fully realized 2 The study from which these data were taken assumed a 17% discount rate and a project life of 10 years. Given these assumptions, the total investment cost was 8,4,8,727 lempiras, or US$ 4,209,364. - 322 - Educational Technology: Chapter 8 when these data were collected. Therefore, the development costs of the project are statad in terms of the predominant existing exchange rate at that time of US $1 - 2 lempiras. The recurrent costs, however, are stated using 3.4 Lempiras per dollar. The 3.4 rate is calculated using a combination of two rates; the rate of 5.5 lempiras per dollar is applied to costs not tied to imported goods, and a rate of 2 lempiras is applied to items that must be imported and, thus, be paid in dollars. Table 13 Recurrent Costs Based on 1990 Prices US$ Lempiras AVANCE Administrative Charge 16,750 56,963 Salaries and Benefits 37,503 127,510 Travel - Domestic 6,443 21,905 Radios & Print Materials 50,524 171,781 Radios (10,256) Radio Repair (1,469) Batteries (18,052) Teacher's Guides (4,259) Workbooks (14,388) Posters (515) Carrying Bag (115) Supervisor Commission (1,469) Transmission 78,824 268,000 Distribution 676 2,299 Teacher Training 7,101 24,144 Other Direct Costs 5,083 17,282 Total $202,907 6P9,883 Cost Per Student $1.01 3.45 (200,000 students) The recurrent cost is $1.01. The largest single item is for radio transmission; other large items include the radios (amortized over five years) and the batteries. In addition, the cost for student notebooks is high. A breakdown of the expenses by major cost categories is presented in Table 14. An important consideration is the allocation of the costs among different constituencies or institutions. The costs can be allocated among three major groups: AVANCE, the private sector, and the local community as represented by parents and teachers. Table 14 presents a breakdown of these allocations. Of the $1.01 total recurrent cost, AVANCE will need to pick up - 323 - Educational Technology: Chapter 8 $0.40. Should the Ministry of Education take over implementation of the project at some later date, it would need to budget accordingly. Private businesses will assume $0.43 per student. Of this cost, 94% is donated radio transmission time; the remaining amount is for printing half of the student notebooks, t'.e plastic posters, and the plastic carrying bags for the instructional materials. About 18% of the costs are to be met by parents or teachers. The breakdown of costs between parents and teachers depends of the circumstances in each school. These local community costs are for the radios (and assume a 50% subsidy by AVANCE unless AVANCE raises the fee for the packages) and the repair of the radio, batteries, teachers' guides, and about half the cost for the atudent notebooks. Table 14 Breakdown of Costs by Source Parents Private and AVANCE Sector Teachers AVANCE Administrative Charge 16,754 Salaries and Benefits 37,503 Travel - Domestic 6,443 Radios & Print Materials Radios 5,128 5,128 Radio Repair 1,469 Batteries 18,052 Teachers' Guides 4,259 Notebooks 7,194 7,194 Posters 515 Carrying Bags 115 Sup. Commission 1,469 Transmission 78,824 Distribution 676 Teacher Training 7,101 Other Direct Costs 5,083 Total $80,157 $86,647 $36,103 Cost Per Student $0.40 0.43 $0.18 (200,000 students) The projected recurrent cost per student can be compared to the per- student recurrent government expenditure. The latest figure in 1988 is 285 lempiras per primary school student; assuming a growth rate of 15% to 1991, - 324 - Educational Technology: Chapter 8 the total government expenditure per primary school student would be 328 lempiras. The add-on cost to the Ministry of Education (e.g., $0.40) to maintain La Familia de los Numeros would be only 0.4% of the annual pe'- student recurrent government expenditure for primary education. If the full recurrent cost is considered (e.g., $1.01) the add-on cost is 1% of the government expenditure per primary school student. Cost Effectiveness. Cost-effectiveness is the ratio of the incremental gain in student achievement, stated in terms of effect size, to the incremental cost (e.g., recurrent cost) of the educational inputs (Lockheed and Hanushek, 1988). Effect size is the ratio of average score difference between the traditional and control group to the standard deviation of the control group. The effectiveness measure for La Familia de los Numeros is difficult to calculate precisely because the radio lessons were introduced along with new textbooks which, in turn, were introduced along with teacher training. As we noted earlier in this section, the impact of the total package (radio, texts, and teacher training) in grade 1 was substantial, with an effect size of .80. The textbooks and teacher training resulted in a significant improvement in achievement, and the addition of the radio programs produced another substantial gain. An estimate of the additional impact of the radio produces an effect size of .37. In the second grade, the impact of the total package was .61. But in contrast to the first grade, the addition of textbooks alone had no impact at all; in fact, the test scores for classes using the textbooks was a bit lower. For reasons explained earlier, it is reasonable to conclude that the total positive impact on the learning in the experimental classes is most likely to be attributable to the radio lessons alone. Therefore, the effectiveness measure to be used for the cost- effectiveness analysis will be the average of the effect sizes for the two grades (.37 and .61) or .49. This represents the incremental learning as a result of the radio programs. The per student recurrent cost for the radio programs is $1.01. Therefore, the cost-effectiveness ratio is .49. Average Annual Total Cost The total annual cost of the La Familia de los Numeros series is the sum of the development costs (including both the development of the materials and national implementation) amortized over 15 years and adjusted for an opportunity cost or social discount rate of 7.5%. - 325 - Educational Technology: Chapter 8 Table 15 Total Annual Cost US$ Development and Implementation Cost $385,807 Recurrent Cost $202,000 Total Annual Cost $587,807 Cost Per Student $2.94a I If the opportunity cost is not included, the total cost per student would be $2.74. It should be noted that the development cost (implementation costs are included) on an annualized basis is far larger than the recurrent costs. These costs are an important consideration when assessing alternative interventions. Once the decision has been made to proceed and the development work has been completed, these costs can be considered as "sunk." That is, in terms of future decisions about expanding the implementation of the lessons and sustaining their use over a long period of time, the initial investment is irrelevant. When all costs are considered, the per student cost is $2.94; but the incremental cost to continue the programs in the schools is only $1.01. Given that the incremental benefits are substantial, the La Familia de los Ndmeros series appears to be a good investment compared to many other options. But even that cost must be broken down into groups depending on the source of funds. In the case of Honduras, the central administrative body, whether that be AVANCE or the Ministry of Education, must cover only $.40 per student. Almost three-quarters of the remaining costs will be picked up by private business; the remaining costs are to be shared by teachers and parents. III. Lesotho Radio Language Arts Program Educational Challenge The Kingdom of Lesotho is one of the smallest countries in Africa. Its 1.6 million people (the population's annual growth rate is 2.3%) inhabit a mountainous territory entirely above 5,000 feet in elevation, where only 13% of the land is suitable for cultivation. The country possesses few natural resources. About 60% of adult males between the ages of 20 and 44 work in the - 326 - Educational Technology: Chapter 8 mines of South Africa. They generate over 50% of Lesotho's total GNP. During the last decade, agricultural yields have declined by an average of 3% per year because of soil erosion, the absent male labor force, and poor farming practices. Although Lesotho's terrain is well-suited to animal husbandry, the country suffers from declining animal quality and inadequate disease control. The economic situation demands an effective educational system capable of preparing citizens for greater self-reliance; yet here, too, the nation confronts major challenges. Formal education is undertaken by a partnership of the government, the churches, and the community. It consists of a seven-year cycle of primary education followed by three years of junior- secondary schooling and two years of high school. There is one university in the kingdom. Enrollment in primary schools in 1980 was 546,828. The number of primary teachers was 6,276, 19% of whom were un-qualified. Although these statistics produce a pl'pil-teacher ratio of 56:1, the tendency to concentrate teachers in upper primary classes in fact leads to more severe overcrowding at the lower primary level. Lesotho has unacceptably high dropout and repeater rates; they result in an efficiency ratio of 2.11 at the primary level. That is, for every successful graduate of the seven-year primary program, the government has paid for 14.78 years of schooling, a terrible waste of scarce resources. To further complicate the situation, the highly academic curriculum has focused on preparing children for further schooling rather than on preparing them to contribute to community and national development. The Basic and Non-Formal Education Systems (BANFES) project was designed to meet such challenges. With funding from the Agency for International Development, a six-member consortium led by the Academy for Educational Development began in 1985 helping the Ministry of Education improve its capacity to provide efficient and effective education relevant to Lesotho's development needs. One focus of these efforts was strengthening the curriculum through work at the National Curriculum Development Centre (NCDC) and the Instructional Materials Resource Centre (IMRC). Command of English-language skills is crucial to Lesotho's national curriculum and to individual success at school and ini work. After the first four years of primary education, English becomes the language of instruction; it is also, of course, the language of academic testing; and it is essential to commerce in the region. Nevertheless, many Lesotho educators believe that standards of English in school have been declining for several years. 8electing Radio as an Intervention Soon after BANFES began, the English Division at the NCDC, recognizing that a significant number of Lesotho's primary school teachers are underqualified for teaching English as a second language, asked for assistance in investigating the suitability of IRI for supporting English-language instruction in Standards (grades) 1 to 3. Neither the English Division nor the BANMES project was interested in an effort to validate IRI as a method. The model had already proved successful in several subject areas including - 327 - Educational Technology: Chapter 8 English as a second language, and in many countries. In the early 1980s the Radio Language Arts Project in Kenya had demonstrated that IRI provided an effective vehicle for instruction in English as a second language. The issue facing the NCDC, therefore, was adaRtatio . Could English in Action, the IRI series developed for Kenya, be used effectively in Lesotho too? Given fierce budgetary and personnel constraints, conventional solutions -- like an intensive, national teacher-upgrading scheme -- were impossible. Furthermore, the problem of poor achievement in English language skills was urgent: it contributed substantially to low efficiency and low effectiveness in the education system. The NCDC needed a remedy that promised immediate impact. With its demonstrated ability to raise achievement levels quickly and its low cost per pupil, IRI offered an appealing choice. To investigate the potential of interactive radio instruction, the Primary English Panel arranged a small-scale pilot test in February 1986. Lessons 1-20 of the Standard 1 radio series from Kenya were modified slightly and rerecorded at the IMRC on cassette tapes that simulated actual radio broadcasts. Five schools were selected as test sites, and a one-day training session was arranged for cooperating Standard 1 teachers. To observe and evaluate the lessons, panel members and curriculum specialists used the rigorous system of formative evaluation developed in Kenya. After reviewing the pilot-test results, the panel decided, unanimously and enthusiastically, that IRI could be successful in Lesotho. Members identified several areas where changes might be made (for example, to the length of pauses and the type of radio equipment). Then they decided to work toward offering English in Action to every school in the country on a voluntary basis; this would make Lesotho one of the first examples of national implementation of interactive radio. Again the NCDC approached BANFES for assistance. BANFES offered the following resources: technical assistance was to be provided by specialists already on staff (in addition to their other responsibilities); a part-time administrative assistant was hired; and a consultant was engaged to work with a member of the panel on rewriting the scripts. USAID agreed to purchase radios to be sold to schools on a heavily subsidized basis (approximately $24 for a radio-cassette unit costing $80). Adaptation From 1979 to 1985 the Kenya Radio Language Arts Project adapted interactive radio to teaching English as a second language to lower-primary pupils. The series English in Action comprises one 30-minute radio lesson for each school day over a period of three years, with some lessons repeated before and after major school holidays. In all, 525 lessons designed for Kenya were adapted for use in Lesotho schools. The process of adaptation was surprisingly simple. The role of English as a medium of instruction is the same in Kenya as it is in Lesotho, and English language syllabi for the lower primary standards are almost - 328 - Educational Technology: Chapter 8 identical. Structurally and in terms of vocabulary, the two national curricula coincided. At first the lessons focused on school, family, and home. Then scriptwriters widened the focus of the lessons to include the community and the nation. The work of lesson and script adaptation was carried out in the English Division, NCDC. Lessons were recorded at IMRC, also a division of the Ministry of Education. Adaptation was a three-step process: analysis and fit of the syllabi; analysis and fit of the cultural content; rerecording of the lessons. First, a primary teacher and a technical advisor, in frequent consultation with the entire English Division, checked the fit of the two syllabi. Since Kenya and Lesotho both specify vocabulary and structures taught at each standard level, the task was uncomplicated. Approximately 40- 50 vocabulary words from the Lesotho syllabus were incorporated into the radio lessons at each standard level. An equivalent number of vocabulary items from Kenya, many of them particular to the national setting, were dropped. At the next step, each lesson was read carefully and modified appropriately for linguistic and cultural differences. In the earlier lessons, differences were reflected at the vocabulary level (different animals and foods". In the more advanced lessons, the changes often required replacement of whole situations or narratives to reflect environmental or cultural differences (going to the coast in Kenya and going to the mountains in Lesotho). It is important to emphasize, however, that these changes did not affect the teaching of a specific linguistic skill (understanding past tense, reading a narrative, forming questions) but only the cultural settings for those skills. As students' command of English grew, of course, adaptation of the lessons became more complicated and took more time. The third step in the adaptation process was recording the revised lessons. Local actors, Lesotho-speaking but with the best English skills available, recorded the lessons A local wusician provided the music. A technical advisor monitored the studio production on a part-time basis. The adaptation process assured that the radio lesson content was congruent with and supported the objectives and content of the regular English language program; it guaranteed that the interactive radio methodology support-ed the need of teachers for a better way to teach beginning aural-oral second-ianguage skills; and it established clear ownership of the lessons by the English Division, placing them within the normal curriculum development and approval process at NCDC. Pupil Achiefement Evidence from instructional radio projects around the world confirms radio as a successful innovation in the primary classroom. (See Friend, Searle, Suppes, 1980; Imhoof and Christensen, 1986; Eshgh, Hoxeng, Casals, 1988; Edgerton, 1989; Fryer, 1989.) The Kenya Radio Language Arts Project provided convincing data that pupils who received instruction by radio made - 329 - Educational Technology: Chapter 8 greater gains in English language achievement than pupils taught by conventional methods. The radio group showed significant gains in all language skills (listening, speaking, reading, writing), with the mos; striking gains in listening. As reported by Oxford and Spain (1986), the differences in group means between radio and regular pupils consistently favored the radio group. Calculation of effect size clearly demonstrates the superiority of radio instruction in Kenya. If an effect size of .20 is considered a clear success, the Kenya effect size of .47 overall puts interactive radio instruction at considerable advantage over conventional instruction. Interactive radio instruction produces such results almost immediately. Radio, therefore, has a significant advantage over other interventions (like teacher training) that take much longer to implement and are much slower to demonstrate positive results. Data from the Kenya pilot program convinced Lesotho to implement the program on a national scale and suggested that extensive testing of pupils was unnecessary. Two types of evaluation were planned, however. The first was a modification of the pupil achievement testing carried out in the Kenya project. It was assumed, incorrectly as it turned out, that the lapped-year testing design could be reduced to testing one standard only and that test items of vocabulary could be modified without revalidating the entire test. These false assumptions, coupled with test administration problems, the project time-frame, and a prolonged teachers' strike, resulted in the abandonment of pupil achievement testing. The second component of the evaluation was an assessment of teacher use of and attitudes toward the lessons and the methodology. Although problems arose with implementation of that assessment, 90 percent of teachers interviewed expressed their desire to continue using the radio English lessons (Harpring, 1990). The difficulties raised by teachers in this study centered on logistics: delivery of print support materials, radio maintenance and replacement, availability of batteries, and teacher orientation or training. Development and Implementation Implementation of instructional radio for the first time and on a national scale is a complex and intense effort that must be well planned in advance. But it is impossible to anticipate all potential problems. Although it was expected, for example, that the first teacher orientation would be rushed and its logistics difficult and perhaps confused, it was not expected that rains and floods would prevent many trainers from attending the first national radio English workshop. This unanticipated problem resulted in spotty training of teachers in many parts of the country, and its effects were felt over the entire period of implementation. At the time of the Harpring study, only 57% of the teachers using the adapted lessons had received specialized training in teaching by radio. Development and implementation efforts fall into four major areas: adaptation of the lessons, printing and distribution of support materials, - 330 - Educational Technology: Chapter 8 teacher orientation, and broadcast and follow-up. The arasa of effort were anticipated, but the level of effort required was underestimated. Throughout the implementation phase, the level of effort was upgraded again and again, as personnel hours committed to the project increased more or less cortinuously. Adaptation of the print support materials for teachers (in the form of teachers' notes on each of the daily lessons) and for children (in the form of worksheets), necessarily followed the adaptation of the lesson scripts themselves. Printing deadlines were not met in one instal 1, but the major hurdle was distribution of the materials to remote and mou tainous regions. Several methods of distribution were tried before an effective -- but costly In the short term -- system of delivery to numerous regional centers was established. Communication with headteachers and teachers to inform them about the availability of materials in their regions was also difficult. Messages on Radio Lesotho proved most successful. Appropriate mechanisms are now in place to support the instructional strategy. But problems with the national infrastructure remain. It is still difficult for some teachers to pick up materials at their regional centers, to get their radios repaired, or to receive communication. Sustainability Tih* r- to lessons have strong support from teachers. Parents have also indicated * -ir support through school committees. Ministry of Education officials have bee.a convinced that the radio lessons are having a desirable effect on the quality of teaching and perhaps the standards of English. Therefore, a number of steps have been taken to assure sustainability. The complexity of effort and the degree of inter-institutional cooperation necessary for success may seem high, but collaboration is already high and the institutional infrastructure exists. Three dimensions affecting sustainability deserve comment: political, administrative, and technical. Political Factors. By the end of the 1990 school year, all three standards were receiving radio English instruction, interrupted by a teachers' strike in some school3. Given the degree of ownership established by institutions such as NCDC, instructional radio has not been politicized. Administrative Factors. Institutions within two ministries must cooperate in order for the radio lessons to continue in Lesotho. Radio Lesotho, under the Ministry of Information and Broadcasting, broadcasts the lessons and is committed to their continuance; it also provides airtime for other information and education programs and has begun the process of developing a new education channel for increased programming. IMRC, under the Ministry of Education, houses the tape library of the radio lessons, provides Radio Lesotho with the correct tapes for one week's broadcasts at a time, and monitors the quality of the tapes and broadcasts. Normally, IMRC develops information and education programs for the Ministry of Education. Archiving the tapes and monitoring the lessons will be an additional burden, but it is one zhat fits within the duties of the - 331 - Educational Technology: Chapter 8 audiovisual section of IMRC. A repair scheme for school radios, to be administered by IMRC, will be an additional activity. The English Division, NCDC, provides in-service training to teachers using radio lessons, monitors the programs in the schools, and supervises the sale of radios to eligible schools. In addition, it will coordinate and supervise the replenishment of print support materials in collaboration with the School Supply Unit, Ministry of Education. These activities are not unlike the regular curriculum development and implementation tasks carried out by the division. They will require part-time attention of a staff member in addition to other duties. The School Supply Unit normally distributes textbooks and other materials to schools throughout the country. School fees are collected and placed in an SSU account until such time as new texts and materials are purchased commercially and then distributed. The SSU. in collaboration with the English Division, will arrange for the replev'shment of the radio support materials in much the same manner that textbooks are replaced in the school system. Sustaining the Lesotho Radio Language Arts Program will require additional, but modest, administrative effort within the existing administrative infrastructure. Technical Factors. Technical factors are not of great concern, although some of the capacity is not yet in place. Radio Lesotho's technical capabilities are growing. Reception is improving with the implementation of regional transmitters. The creation of an additional broadcast channel will result in better reception in some areas as well as expanded airtime. Implementation of a proposed training program will enhance production and evaluation capabilities, leading perhaps to greater sophistication in use of the meditum in education as well as entertainment. Orn-y minimal technical capabilities are necessary for IMRC to sustain the radio English broadcasts. The institution can sustain but not innovate. It would have d_.fficulty in producing additional, high-quality educational programming due to insufficient and inadequate studio and editing facilities. Can the institutions responsible for an interactive radio effort effectively sustain such a program in Lesotho? Experience with distance education and schools broadcasting in Africa seems to Luggest that without innovation and growth, difficulties with sustainability surface quickly. People get bored, equipment deteriorates, scheduling becomes erratic, support materials are harder to replace, listenership declines. It may not be enough, therefore, to leave in place only those requirements for uncreative repetition of an established structure. Instead it may be necessary to challenge the system and the people who operate it to expand and alter interactive radio instruction so that it becomes an integral and critical factor across subject areas in Lesotho's schools. - 332 - Educational Technology: Chapter 8 Cost Analysts Context. The costs for the English in Action series for grades 1-3 in Lesotho includes both development and incremental costs. The develogment cos&s3 include the expenditures for adapting the radio series from the programs originally created in Kenya to the curriculum and culture of Lesotho. As described earlier, minor changes were made to the curriculum, the scripts were revised to reflect the culture in Lesotho, and all the lessons were recorded again. The incremental costs are those expenses above and beyond the normal expenditures for schools that would be required to sustain the radio programs on a national basis for a long period of time. This section also presents the total annual costs of the series, that is, the sum of the development and incremental costs. In addition, the costs per student are presented. Finally, the allocation of the incremental costs to various groups (Ministry of Education, Ministry of Information and Broadcasting, parents, and teachers) is shown. Development Costs. The development costs for adapting the 525 lessons for grades 1-3 are presented in Table 16. Table 16 Development Costs for Radio English in Lesotho US$ Personnel Host Country 96,201 Technical Assistance 447,931 Furniture and Equipment 33,319 Operational Costs 98,023 Summative Evaluation 45,624 @ 0% Discount Rate 721,098 @ 7.5% Discount Rate 1,225,367 3 Some studies separate start-up costs from development costs. For this study, we have combined all such costs under development costs. In Lesotho the major start-up cost was the pilot project, but this activity actually developed (e.g., adapted) the first 20 lessons. The distinction between start-up costs and development costs does not appear to be useful in this situation. - 333 - Educational Technology: Chapter 8 The total expenditures were $721,098 (the amount in Table 16 showing a 0% discount rate). The discount rate, sometimes referred to as an opportunity cost, reflects the amount of income or benefit that might have accrued if the money was put to another use. For example, the money could have earned interest in a savings account. The discount rate is a value stated as an interest rate that reflects the cost of withdrawing resources now in order to produce greater consumption later. For this analysis, as for the cost studies in this chapter of Bolivia and Honduras, a 7.5% discount rate was used. The full development cost with a 7.5% discount rate was $1,225,367. Approximately three-quarters of the host-country costs were for personnel who worked directly on production of the lessons --actors (38% of the costs), producer (18%), musician (10%), and studio technician (8%). Outside technical assistance was the most expensive component, accounting for 62% of the total development cost. Of the total amount for technical assistance, 47% was devoted to script adaptation and 28% to studio supervision. The summative evaluation accounted for only 6% of the total expenditures on development of the project. Of that 6%, outside technical assistance accounted for 34%; 25% was devoted to test administration, and 27% to test scoring. As mentioned earlier, the resources allocated to the summative evaiuation were inadequate, as the adaptation of the test used in Kenya proved not to result in a good test for Lesotho. Once the total development costs have been determined, it is necessary to average this amount over the useful life of the radio lessons. Assuming a 15-year life for the lessons, as is consistent with the cost studies of Bolivia and Honduras, the average annual development cost is $81,691, including a 7.5% discount rate. Incremental Cost. The development costs are considered sunk in terms of making a decision about whether the lessons should continue to be used in the schools. The decision to continue the series depends on the incremental cost, or the additional expenditure required to sustain the radio lessons in the schools over a long period of time. The incremental cost is composed of both fixed and variable costs, as explained earlier in this chapter. The variable costs depend on the number of students. In Lesotho in 1990, 80,481 students were enrolled in Standard 1, 63,420 were enrolled in Standard 2 and 56,972 were enrolled in Standard 3, for a total of 200,873 students. The total incremental costs (fixed and variable) are shown in Table 17. - 334 - Educational Technology: Chapter 8 Table 17 Total Incremental Costs, Standards 1-3, in Lesotho US$ Percent Fixed Costs Radio Transmission $41,490 87 Administration 6,100 12 Subtotal 47,590 Variable Costs Radio Receivers 34,603 25 Radio Maintenance 2,625 2 Batteries 55,997 40 Teachers' Guides 4,943 4 Student Workbooks 39,127 28 Teacher Training 1,481 1 Delivery of Materials 1,842 1 Subtotal 140,618 Total Incremental Costs: $188,208 The fixed costs account for 25% of the total incremental costs; most of this category of costs is devoted to transmission of the lessons. Of the variable costs, the replacement of batteries is the most significant component (40%); the student workbooks account for 28% (these are depreciated over three years) and the radios account for 25% (depreciated over five years). Given the high cost of batteries, an estimate was made on how much this component might be reduced if Ni-cad rechargeable batteries were used and the recharging was provided by a solar panel. Such an arrangement would reduce the total battery cost by one-half and the total incremental cost by 15%. The incremental cost per student is $0.94. This represents just 1.7% of the total expenditures per primary school pupil. The total cost per primary school student is $55, of which the Ministry of Education budget covers the remaining $24. Allocation of Costs. Although the incremental cost per student is $0.94, the Ministry of Education does not pay this amount. The costs are allocated among the Ministry of Education, the Ministry of Information and Broadcasting, and the parents (with the teachers picking up some of the costs in some schools). The allocation of costs is shown in Table 17. - 335 - Educational Technology: Chapter 8 Table 18 English in Action, Standards 1-3, Allocation of Costs Total Ministry Ministry of Parents Costs of Information & and/or Education Broadcasting Teachers Workbooks $39,127 $39,127 Teacher training $1,481 $1,481 Teacher's guides $4,943 $4,943 Radio receivers $34,603 $24,847 $9,756 Maintenance of radios $2,115 $510 Power, battery $55,997 $55,997 Radio transmission $37,500 $37,500 Duplication of worn tape $3,990 $3,990 Administrative & clerical $3,600 $3,600 Administrative overhead $2,000 $2,000 Administrative travel $500 $500 Delivery of materials $1,842 $1,842 Total Program Cost $188,208 $84,445 $37,500 $66,263 Cost Per Student $0.94 $0.42 $0.19 $0.33 (200,873 students) The Ministry of Information and Broadcasting finances Radio Lesotho, which broadcasts the English in Action series without charge. Although Table 18 shows that the cost to the Ministry of Information and Broadcasting for broadcasting time on Radio Lesotho is $84,445, in reality the budget of the Ministry will be little affected as the English in Action radio programs simply replace other programs. The parents, perhaps with some participation by teachers, buy the radios (at a subsidized price) and the batteries; they also pay for the repair of the radios. On average, they provide $0.33 per student or a total o'; $66,263 per year. The remaining expenditures must be covered by the Ministry of Education. On the average each year (in 1990 U.S. dollars), the Ministry must budget $0.42 per student, or a total of $188,208, to the radio English lessons. This represents 1.79% of the budget for primary education. In the first years, the amount will be less than this as the radios, teachers' - 336 - Educational Technology: Chapter 8 guides, and student workbooks are new and will not need to be replaced immediately. Total Cost. The total cost of the radio series for each year is the sum of the development costs (with a 7.5% discount rate and averaged over 15 years) plus the annual incremental cost. Total Annual Cost - Annualized Development Cost + Annual Incremental Cost The annualized development cost is $81,691. The total incremental cost for one year is $188,208, for a total annual cost of $269,899. It is interesting to compare the costs in Lesotho, where an IRI series was adapted from another country, to the costs in Honduras, where a new series was developed. In Honduras, the incremental cost for sustaining the radio lessons is just 7.4% higher than in Lesotho, but the total costs were twice as high ($2.94), reflecting the considerable additional development costs for creating a new series. IV. The Radio Science Pilot Proiect in Papua New Guinea The Radio Science Project (RSP) is a new application of an existing technology in interactive radio instruction (IRI). It is funded by the USAID Bureau for Science and Technology/Office of Education in Washington, D.C., and is a joint project between the Educational Development Center, Newton, Massachusetts, and the National Department of Education of Papua New Guinea. RSP's goal is to develop science lessons for grades 4, 5, and 6 for use in the Community (Rural) Schools of Papua New Guinea. This is the first application of IRI to an open-ended, inquiry-based curriculum. It brings new challenges and requires new approaches to interactive radio design. For sustainability within the educational infrastructure of a developing country, congruence must exist between the application of a technological intervention and the curricular goals of the national ministry or department of education. The intervention must offer substantial benefits over the existing pedagogy in cost, learner outcomes, and acceptance by teachers. It requires political support as well as technical support, and it must fit into the framework of educational change over time. The Radio Science Project sought to meet all of these conditions. In developing the lessons, the RSP relied on interactive radio's model of formative evaluation. Techniques of production, format, testing, questioning, presentation, activity, and teacher training were continuously modified based on the results of formative evaluation. In addition, input from the National Department of Education, the National Broadcasting - 337 - Educational Technology: Chapter 8 Commission, and other policymakers was solicited on a regular basis to ensure that the final product -- effective radio lessons in science -- would be acceptable and implementable throughout Papua New Guinea. This section covers the significant activities and findings of the project as they developed and concludes by identifying strategies for overcoming barriers to institutionalization. The Radio Science Project Task Encouraged by the success of several previous applications, the Education Development Center proposed a pilot study to determine if IRI could bo applied to an open-ended curriculum like science. The pedagogical challenge was clear. Teachers in both developed and developing countries cite science instruction as among the most difficult to accompli;fh. In Papua New Guinea, as in most developing countries, many different circumstances exacerbate the problem of teaching science: financial resources are limited, rural areas are isolated, there is limited access to primary education, and teachers are scarce and poorly trained. Universal primary education is a national goal, but teacher training institutions are hopelessly overburdened, and supplementary funds for upgrading teachers already in the schools, and for improving their materials and classrooms, are unavailable. In Papua New Guinea it is difficult to achieve an acceptable standard of instruction in any of the major areas of the curriculum for children in the primary grades. With science, the perceived difficulty of the content intensifies the problem. Of all the technologies available to confront this challenge, the one of almost universal application is radio. Radio waves reach nearly everywhere in Papua New Guinea, even the remote and isolated highlands. There is an established school broadcasting system that provides several hours of radio lessons per week to primary schools. Most schools have purchased radio receivers. The Radio Science Project differs from earlie: efforts in interactive radio instruction in two significant ways. First, the lessons are designed for children in the upper primary grades. The reason for this is simple. Although many teachers are quite comfortable with the science content found in the first few grades, they begin to experience difficulty as the content becomes more complicated. Since in developing countries like Papua New Guinea, teachers often have as few as 10 years of formal education, their science background is seldom sufficiently strong to allow them to feel confident about teaching science, especially if they try to incorporate an inquiry approach and the use of even simple science equipment and materials. - 338 - Educational Technology: Chapter 8 In addition, science instruction is typically given more time in the upper primary grades. The second major difference between RSP and other interactive radio projects is that Radio Science incorporates inquiry-based teaching, open- ended questions, and the use of physical materials. This is the most basic challenge of the project. It is relatively simple to generate lively scripts that pose fact/recall questions. Curricular models in primary English and math include much nomenclature and structure, all of which lend themselves to interactions that are short and specific. Here's an example from beginning English: TINA: Safina, is it dark now? SAFINA: No, Tina. It isn't dark now. It's dark at night. RONO: Children, say, "It's dark at night." (Pause 4) RONO: It's dark at ntight. Again. (Pause 3) TINA: Yes ... it's dark at night (Imhoof and Christensen, 1986) In a science curriculum for fourth-, fifth-, and sixth-grade students, however, x.astructional design must accommodate content or questions for which there is more than one logical answer. Given a concept like "Why is coffee grown in the highlands?" or "When does the breeze blow toward the coast?," there is not just one right answer; nor are the various answers likely to be simple or straightforward. With such specific challenges as these, the Radio Science Project and the National Department of Education began joint operations in 1986. Although the project was funded as a pilot study, its goal was a finished curriculum product, instructive and supportive to teachers as well as to students, that improved learning gains significantly and that could be implemented on a national basis at low cost. During the first year in Papua New Guinea, RSP secured facilities, began work on the research design of the project, And designed the radio curriculum for grade 4 based on the official national syllabus. Ten pilot schools in the National Capitol District and the Central Province were selected for testing the lessons and for formative feedback on student response and achievement and on teacher participation. During the 1988 school year 60 30-minute fourth-grade lessons were designed and tested in the 10 - 339 - Educational Technology: Chapter 8 schools. There are two 30-minute lessons each week -- 20 minutes for the radio broadcast and 10 minutes for teacher-led activities. In addition to broadcast tape, each lesson has a student worksheet and notes. A kit of simple materials was developed to be used with the lessons. Thirty schools in the East Sepik Province were selected for the summative evaluation and arrangements were made to broadcast the lessons on a biweekly basis on Radio East Sepik in 1989. The 60 lessons were broadcast and pre- and post-achievement tests were administered. At the same time, grade 5 materials were generated and tested in the schools chosen for formative evaluation. Because of severe budget restrictions in the government and schools, and the resulting difficulties for maintaining a science kit, the grade 5 lessons were designed to be used without a kit. During 1990 Radio East Sepik broadcast the grade 5 lessons to the schools chosen for the summative testing. The project staff refined the grade 6 lessons and prepared for institutionalization. Materials and hands-on activities are still incorporated into the lesson, but the science educators found that over 90% of the concepts could be supported by materials common in the schools or the communities. Most of these were also obtainable without cost. The list consists of bottles, cans, jars, string, wheels, food items, plant material, stones, and other items. Further, it was found that judicious selection among activity types provided children with plenty to do while keeping cost at a minimum and teacher confidence high. For the few activities that required purchased materials (batteries, bulbs, mirrors, and the like), lessons were designed to be demonstrations. Summative Evaluation Results. The summative evaluation of the grade 4 materials resulted in a mean score of 46% overall for the experimental group, compared to 42% for the control group. The effect size is .36, which is quite impressive for a low-cost intervention, with only two 20-minute radio programs per week. Using analysis of variance, we get a value of F-24.45, which makes the difference of the means significant at the pc.001 level. The test can be broken up into two parts: 13 questions on biological science and 17 questions on physical science. For both subject areas, the radio programs produced positive results, with the achievement gain slightly larger for the physical science part of the test. The teachers have been particularly appreciative of the Radio Science Project's efforts to introduce physical science activities into the classroom. The largest gains were registered in the areas of light and electricity/magnetism, which are given more time in the Papua New Guinea grade 4 science curriculum than heat and sound. Of the 30 items on the test, the experimental group performed better on 22, while the control group did better than the experimental group on the remaining eight items. - 340 - Educational Technology: Chapter 8 Table 19 Papua New Guinea, Science Test Results by Treatment Control ExRerimental % Correct SD % Correct SD Total Test 41.5a 14.0 46.5 15.1 (30 items) Biology 47.3 17.2 51.6 19.1 (13 items) Physical Science 37.0 15.4 42.5 15.5 (17 items) Electricity/Magnetism 39.5 21.7 50.1 21.5 (5 items) Light 45.2 22.6 52.3 22.9 (7 items) Heat 32.2 28.0 29.8 26.0 (3 items) Sound 18.0 27.8 13.8 23.9 (2 items) a Effect size - .36; F - 24.45, p<.001 Table 20 shows the test results for students by sex in both control and experimental schools. In both the control and the experimental group, the boys scored significantly higher than the girls on the science test. It is interesting to note, however, that in the experimental group the gender gap has lessened somewhat. In the control group, the difference between boys and girls is 6 percentage points, whereas in the experimental group it is only 4. In other words, although students of both sexes benefited from the radio programs, the girls benefited relatively more than the boys. - 341 - Educational Technology: Chapter 8 Table 20 Papua New Guinea, Science Test Results by Student Gender Control ExRerimental % Correct SD % Correct SD MalTh 44.5 14.2 48.4 15.8 Female 38.4 13.1 44.4 14.2 The conclusions of the evaluation are the following: * The experimental group (science by radio) did better than the control group (traditional science teaching). Teaching science by radio is effective. - The gender gap was narrowed ir the experimental group. The difference between boys and girls was higher in the control group than in the experimental group. e Programs were effective at all ranges of age. The highest scores were obtained by the 12-year-old students. The experimental group's scores were higher than those of the control group at every age between 10 and 14. * Test instructions were understood by virtually all children. Both the experimental and control group had a small number of blank answers. Almost all children finished the test. * The radio programs were effective in both the biological and physical sciences, but slightly more effective in the physical sciences. * The experimental group scored better on 22 of 30 test items. These were spread across the whole curriculum. * The experimental group scored higher than the control group in all types of schools: government, Catholic, and Evangelical. * The radio programs were effective across a wide range of locales--remote rural, rural, and urban. - 342 - Educational Technology: Chapter 8 * Twenty-one of 27 schools in the experimental classes performed better than the control classes from the same school. With every pilot project that develops new materials, the start-up costs are high. USAID funded virtually all the development costs. Technical assistance decreased from a planned four people to three once the project actually began work in Papua New Guinea, to two after the first year, to one during the last year, and to no resident technical assistant during the last several months. The distance of Papua New Guinea from the United States, plus fairly high local cost£, contributed to the high total costs. The resulting reduction in technical assistance required a scaling back of development and research activities. The development cost for the four-year project in Papua New Guinea was approximately US $3 million. There are conditions in Papua New Guinea's educational infrastructure that make RSP implementation on a national basis relatively inexpensive: (1) radio as an educational medium has been in use in Papua New Guinea for 20 years and airtime Is available; (2) each province has a materials officer to distribute materials, and the Curriculum Unit of the NDOE has existing warehouse and distribution facilities, so the added cost for distribution of RSP student and teacher books involves shipping only; and (3) teacher training is done both locally and in the provinces by the In-service College and staff, and travel budgets are in place. An RSP in-service package was developed that included three days of training for inspectors and "trainers of teachers." This cadre of RSP- trained in-service educators will be available to teach their colleagues in each of the provinces how to use the Radio Science lessons. Teachers in Papua New Guinea's Community Schools have one hour of released time each week for training, and this hour will be devoted to the Radio Science Project in the beginning of the 1991 school year. The cost for the printing of student books (60 pages of drawings) is less than a kina (US$ 0.95). Since each book is designed to be shared by two students and to last five years, the cost of the book per student per year comes to less than 10 toea (US$ 0.10). The RSP Teacher's Notes, which consist of five pages of in-service directions and 120 pages of lesson notes, can be produced in the Curriculum Unit's print shop for 1.87 kina (US$ 1.97) each. These books are also expected to last five years. Allowing for teacher turnover, the cost of the Teacher's Notes is estimated at 0.75 kina (US$ 0.79) per teacher per year. The total per student cost for materials is approximately US$ 12. Although these are low costs for sustaining the radio science lessons, there is another striking advantage to implementation of RSP nationwide: it would cost less than the effort to bring the traditional science curriculum up to the standard envisioned for it when it was adopted 20 - 343 - Educational Technology: Chapter 8 years ago. But it was not only because RSP is cost-effective that the Community Schools Board of Studies approved broadcast; it was because students measurably learned so much from the lessons that officials, administrators, and teachers echoed their enthusiasm. Saving money is an added benefit. There were other unanticipated benefits. Because of the emphasis on Community School science and the effort to deliver the curriculum in a different way, many teachers and curriculum officials became critical of the existing science curriculum. It is a third-hand modification (of two imported curriculum projects designed in the 1960s) that contains wonderful activities for children, but requires high scientific competence of teachers. Over the years, all of the higher level obj3ctives have been removed from this curriculum, leaving only recognition and fact/recall objectives. The exisAing curriculum does not address the needs of youngsters who will be leaving formal schooling at the end of grade 6, and who must be equipped to contribute to the development of their communities and their country. RSP focused attention on this problem. Now, through the efforts of leaders of the NDOE's Curriculum Unit, the Community Schools Board of Studies has begun to develop a new science curriculum for grades 1 through 6. Additional attention was also brought to other radio programming for the Community Schools of Papua New Guinea. A comprehensive multidisciplinary broadcast committee established by the First Assistant Secretary of the NDOE has begun meeting to review programs currently serving the schools. Many broadcasts that were developed prior to independence will be replaced; many others will be revised and rerecorded. Most of these older programs are story-oriented and general. Typically, they present characters and situations specific to Papua New Guinea, but the voices are expatriate. The contrast between these programs and the Radio Science broadcasts has given new and heated impetus to the use of indigenous national voices for radio characters and to the appointment of Papua New Guineans as scriptwriters. Two important requirements for the updated radio programs are that they closely support the teaching of the curriculum in the Community Schools and that they be interactive. These specifications are a direct result of t' e discussions that took place during the institutionalization phase of thQ Radio Science Project. Another outcome of the political process of institutionalization is that national expertise was recognized. The project hired and trained over 20 national staff members. 50% of the staff of the Radio Science Project has been hired by the NDOE to continue the process of development of programs after institutionalization. Currently under consideration are instructional broadcasts in math, English, community life, and health. - 344 - Educational Technology: Chapter 8 Institutionalization From the beginning, RSP staff planned to work through the political structure of the National Department of Education toward institutionalization. A broad-based advisory committee guided the effort. It was comprised of members of the NDOE and its Curriculum Development Unit, the National Broadcasting Commission, teachers, administrators, university professors, evaluation specialists, and project staff. The committee established three subcommittees (evaluation, broadcast, and curriculum) that met throughout the life of the project. Their advice was invaluable. Project staff also held regular meetings with key departmental staff members. Informational papers were generated and distributed continuously during the pilot phase, and an informational video made during 1989 was played on the national television station. The video was well received, as were the results of grade 4 evaluation. Several committee members arranged screenings of the video at sites around the country. By now there was a sturdy base of support from teachers and provincial officials who had used the program, and there were key officials in the NDOE who were supportive. At this point the task was to win approval at the Community Schools Board of Studies. Three steps were taken. First, to capitalize on the support from those using Radio Science in the field, additional books were printed (at a cost of US $7,500) to service 10,000 students in three provinces (the evaluation population had numbered only 1,000). The need teachers and administrators felt for help with science teaching was reflected in the fact that the materials were exhausted within 10 days. A staff member traveled to four sites to provide in-service training for the additional teachers. As the term progressed, teachers in these new sites were asked to evaluate the program and to have students write to Radio Science. Hundreds of letters of evaluation arrived from teachers and students and were circulated among officials at the NDOE. Testimonials from those in the field proved eloquent. When the Curriculum Advisory Committee of the NDOE met at the end of the 1989 school year, it mandated the addition of three more schools in the Capital District and the adjacent Central Province to the RSP trial. Project staff met with each committee member who had voiced concerns about institutionalization, transcribed those concerns, and responded to them in writing. The responses provided the basis for presentations at the RSP Committee meeting and at the Community Schools Board of Studies meeting. The national staff of the Radio Science Project are intelligent and capable, and they are now empowered. Typically in Papua New Guinea, NDOE presentations have been made by expatriate members of the Curriculum Unit staff who are deeply involved in departmental decision making. Nationals hired for RSP, however, have gained confidence and expertise during the life - 345 - Educational Technology: Chapter 8 of the project. They prepared, practiced, and delivered presentations about the Radio Science lessons, and Committee members listened to them. Papu.a New Guineans take pride in their accomplishments and in those of their compatriots. During 1989 and 1990 the RSP showcased the experttse of its national staff. The point was made at each opportunity that the project was the result of the work of Papua New Guineans. Each staff member had the opportunity to be heard in public at a meeting or on the radio. Acceptance was overwhelming. The decision to institutionalize was unanimous. In an economy of vEvere need, Radio Science was given priority status. Not only is the project institutionalized in the current curriculum, but the plan for a new science curriculum incluaes IRI as a basic component of instruction. Educational Change Many educational interventions that are both educrltionally effective and cost-effective have failed the test of sustainability in developing countries. Why didn't that happen with the Radio Science Project? There were doubts about long-term implementation at various points during the pilot phase. With production more than half complete, questions arose about whether the grade 6 materials should be developed or whether the time would be better spent on remaking the lessons for grades 4 and 5. In March 1990 serious questions were raised about implementation of 1n= of the lessons on a national scale. In addition, a shortfall in USAID funds cut the budget by over 35%, and some USAID offir'als were not sure that the project could be viable with the reduced level of funding. But the USAID Radio Science Project did continue and has been enthusiastically supported for institutionalization; the NDOE in Papua New Guinea has reorganized its structure to accommodate the project and to plan and implement additional IRI programming on its own. Crucial to institutionalization is that key NDOE administrators were part of the process from the beginning. Politics is a strong motivator among administrative decision makers. They must continually move forward, and they are likely to be biased toward the program that is current, or popular, or both -- in this case the Radio Science Project. To highlight the popularity of the Radio Science Project, additional books were printed and in-service education provided for more teachers. The trial teachers were enthusiastic but were few in number (45). When 300 teachers in three provinces and 4,000 students call for a program, it is difficult for administrators and officials to ignore their united voice. Expanding the base of teachers to a significant number and soliciting their candid response was the single most effective factor in gaining the wide acceptance that led to institutionalization. - 346 - Educational Technology: Chapter 8 The Papua New Guinea project science educator has been hired as a curriculum officer within the Curriculum Unit of the NDOE and assigned the task of administration of the Radio Science Project. With the support of the teachers using the program and the administrators and officials who have served on the RSP Committee, he will be able to sustain Radio Science in Papua New Guinea. Indeed, it is planned that IRI will be incorporated in several other curricular areas. Postscript The end of a project of this magnitude offers opportunities to speculate about the work so far. Interactive radio instruction has great potential for broader use as a teacher-training medium. As RSP tried to improve radio science lessons, the project taught teachers new skills through teachers' notes, through specific directions from radio teachers, and through a step process in which each lesson builds comple: teaching techniques from the simple techniques presented in earlier lessons. Teachers learned vocabulary, how to set up activities, how to integrate science and other curricula, how to ask open-ended questions, how to use the processes of science, how to increase "wait time" while children answer questions, and how to feel comfortable in fielding questions to which they do not know the answers. The question for which there is no empirical data is this: How well does this process of teaching the teachers work? When the cooperating teachers in Papua New Guinea were asked the following question --"After listening to the radio broadcast, could you teach this lesson yourself?" -- the answer from most was a strong, unqualified "yes." Let us hope that, as educators, we are as good at listening to answers as we are at thinking up questions. V. Conclusions Cost-Effectiveness Interactive radio instruction appears to be a highly cost-effective intervention for improving the quality of primary education. Three of the studies in this chapter report a significant positive impact on student achievement with effects sizes of .90 (mathematics in Bolivia), .49 (mental arithmetic in Honduras), and .39 (science in Papua New Guinea). Studies in three of the countries also demonstratecL that annual incremental costs per student are likely to be affordable in most countries: $0.81 in Bolivia, $1.01 in Honduras and $0.94 in Lesotho. In Honduras, this - 347 - Educational Technology: Chapter S represents 1%, and in Lesotho 3%, of the ministry budget per primary school pupil. If one considers only the add-on cost of the IRI lessons that the Ninistry needs to pay, then the cost per pupil in Honduras is $0.40 (or 0.40 of the ministry budget) and $0.42 per pupil in Lesotho (or 1.7% of the Ministry budget). Two different types of cost-effectiveness studles are reported on, one in Bolivia and one in Honduras. The approach for each study was different. In Bolivia, Jamison measured the effect size association witL a year of traditional instruction. He then established the efficlency measure by dividing the total gains by the total costs, wlth a resultlng cost- effectiveness ratio of 0.19. In Honduras, the efficiency was measured by the ratio of the incZemental benefits (in terms of effect sLze) to the incremental costs. The efficiency ratio is .49. Sustainability The four countries represent different approaches to development and implementation. In Honduras and Bolivia, the IRI programs were introduced through non-governmental agencies, one (AVANCE in Honduras) a private-sector organization intended to generate income profitability, the other (Fe y Alegria in Bolivia) a Roman Catholic organization that runs 200 schools under contract to the government. In Papua New Guinea, the Radio Science Project was only an adjunct to the National Department of Education until the completion of pilot activities. In Lesotho, Zngl2h, Ip Actio was from the beginning lodged in the National Curriculum Development Centre of the Ministry of Education. Each institutional situation was, of course, a consequence of particular circumstances in the host country. Although long-term sustainability of the interactive radio series in Bo'ivia, Honduras, Lesotho, and Papua New Guinea cannot be assured, the prospects for sustained use in each country are promising. Those prospects depend on the following .actors: * the development of programs that are highly effective and valued by students and teachers alike * efforts to keep costs low by limLting the need for supplementary materials and tes_her training * the involvement of important political constltuencies, especially in national ministries or departments of education * planning for the inclusion of recurrent costs in the national education budgets - 348 - Educational Technology: Chapter 8 The relative strength of each factor varies from one country to another. Although the initial activities in a country can begin in many different ways, there should be a focus from the beginning on the needs for national implementation. In every country in which interactive radio has been present, long-term sustainability has required the support of the ministry of education. Interactive radio continues to demonstrate its power in the classroom, a pattern established with the Radio Mathematics project in Nicaragua in the mid-1970s. But to have a major impact on education throughout the developing world will necessitate a continuation of the more recent pattern of wide-scale implementation over many years. - 349 ^ Educational Technology: Chapter 8 References CEBIAE (Centro Boliviano de Investigacion y Accion Educativas), (1989), "La Desercion Escolar del Ciclo Basico en Bolivia." Estudios Educativos No.26. La Paz: CEBIAE. Coombs, P.; Hallack, J.; (1987), Cost Analysis in Education: A Tool for Policy and Planning, World Bank: EDI Series in Economic Development. Deutrom, B.; Wilson, M.; (1986), "Science for All in Papua New Guinea", Science Education 70(4): 389-399, John Wiley & Sons, Inc., New York. Edgerton, D., (1989), Honduras Radio Learning Project: Special Interim Report of Status of SEI, AVANCE's Radio Education Unit. Available from Radio Learning Project, EDC, 55 Chapel St., Newton, MA 02160 Eshgh, R.; Hoxeng, J.; and Casals, B.; (Eds.), (1988), Radio-Assisted Community Basic Education (RADECO), Pittsburgh, PA: Duquesne University Press. Friend, J., (1988), "The Development of Instructional Methods Used in Interactive Radio Instruction", Technical Report No. 801, Friend Dialogues, Inc., Shelby, North Carolina. Friend, J.; Searle, B.; and Suppes, P.; (Eds.), (1980), Radio Mathematics in Nicaragua, Stanford, CA: Stanford University. Fryer, M., (1989), First-Year Summative Evaluation: Second Grade Mathematics, November 1988-November 1989, Cochabamba, Trinidad, and Kami, Bolivia. Available from Radio Learning Project, EDC, 55 Chapel St., Newton, MA 02160. Galda, K.; Beach, K.; Friend, J; Gamero, C; Kozlow, S: "The Impact of Honduras Mental Arithmetic Radio Programs on Student Achievement in Grades 1 and 2." Education Development Center, Radio Learning Project, Newton, MA, 1990. Harpring, S., (1990), Evaluation Results for the Radio Language Arts Programme. Available from the BANFES Project, Academy for Educational Development, 1253 23rd St., NW, Washington, DC. Imhoof, M.; Christensen, P.; (1986), Teaching English by Radio: Interactive Radio in Kenya, Academy for Educational Development, Inc., Washington, DC. - 350 - Educational Technology: Chapter 8 Jamison, D.; "The Radio Mathematics Project in Bolivia: Cost-Effectiveness Evaluation," Newton, MA.: Education Development Center, Radio Learning Project, 1990. Jamison, D.; Klees, S.; Wells, S.; The Costs of Educational Media: Guidelines for Planning and Evaluation. Beverly Hills, CA.: Sage, 1978 Jamison, D.; McAnany, E., (1978), Radio for Education and Development, Sage Publications, Beverly Hills, California. Lockheed, M.; Hanushek, E.; (1988), Improving Educational Efficiency in Developing Countries: What Do We Know?, Compare (Vol. 18, No. 1). Oxford, R.; Spain, P.; (1986), Summary of Results. In Imhoof, M.; Christensen, P; (Eds.), Teaching English by Radio: Interactive Radio in Kenya, Washington, DC: Academy for Educational Development. Psacharopoulos, G.; Woodhall, M.; (1985), Education for Development: An Analysis of Investment Choices, Publication of the World Bank. Searle, B.; Friend, J.; Suppes, P.; (Eds.), (1976), The Radio Mathematics Project: Nicaragua 1974-1975, Institute for Mathematical Studies in the Social Sciences, Stanford, California. Suppes, P.; Searle, B.; Friend, J.; (Eds.), (1978), The Radio Mathematics Project: Nicaragua 1976-1977, Institute for Mathematical Studies in the Social Sciences, Stanford, California. Tilson, T.; "The Economics of Radio Education." A paper presented at the African Conference of Radio Education, Harare, Zimbabwe, January, 1990 USAID/AED Honduras Primary Education Project Education Subsector Assessment, (1989), Contract No. LAC-0000-C-00-6074. World Bank (1989), Trends in Developing Countries 1989, Washington, D.C. - 351 -