m B ~~~~~~~~.4 . . . S._ Human Resouroes Development and Operations Policy Tho World Bank January 1994 HROWP 19 ECONOMIC RETURNS FROM INVESTMENTS IN RESEARCH AND TRAINING Edwin Mansfield Papessim t e areifompubcatsftheWodBank Theypsetpriaynp edu respot a f analysis tisciulaedtoa e djsi mid omeat; at and te usseof such a pWer shld take acooun of its prvi onalchaaaL. The findns, iupretts. and oaududos e sd in is paper are eniely those of the auto(s) and shld not be aibd in ay manner to the Wodd Bank, to its affiat organizions, or tb manbers c£ ts Board of Exewtive Deos or the coanres they rpresat Economic Returrs from Investments in Research and Training by Edwin Mansfield University of Pennsylvania Abstract This paper summarizes what is known about the economic returns to investments in advanced scientific research and training. Increasing higher education and national R and D investments have become important componnnts of the economic growth strategies of industrialized countries and also many developing countries. Recent studies by economists indicate that the social rate of return from investments in new industrial technology in developed countries seems to be very high and that technological change in many industries has been based on recent academic research. Without recent academic research, it seems likely that there would have been a substantial reduction in the value of society's economic output. Most of the research concerning the economic returns from R and D in developing countries has focused on agriculture. The estimated social rates of return are very high, often 50 percent or more. This not true, however, for all R and D in developing countries. TIe effective utilization of science and technology has been hampered by low levels of investments, distorted prices and markets, sluggish export growth, lack of competition, short-sighted laws and regulations and other negative aspects cf a country's economic environment. Although estimates have been made of the economic returns from broad categories of education and training, little seems to be known about the social rate of return from a country's investments in highly specialized types of scientific training. Contents 1. Introduction 1............................. I 2. Economic Returns from Investments in Industrial Technology in Developed Countries: Individual Innovations. 1 3. Studies at the Firm or Industry Level. 2 4. The Contribution of Academic Research to Industrial Innovation. 3 5. Locat.on and Characteristics of Academic Research Underlying Industrial Innovations. 5 6. Effects of Faculty Quality, Scale of Research Effort, and Geographical Proximity ... 7 7. The Role of Students as Transfer Agents. 8 8. Technological Change in Developing Countries. 9 9. Economic Returns from R and D in Developing Countries .1C 10. Problems in Using Science and Technology Effectively to Solve Economic and Social Problems .11 11. The Role of Intellectual Property Rights .13 12. Advanced Scientific Training and the Diffusion of New Techniques .15 13. 3ummary and Conclusions .17 REFERENCES ..19 1. Introduction My assignment in this paper is to summarize very briefly what we know about the economic returms to investments in advanced scientiflc research and training, as well as what may be needed to realize benefits from these investments. Increasing higher education and national R and D investments have become inlportant components of the economic growth strategies of industrialized countries and also many developing countries. This paper will discuss the economic rationale for such investments and describe the available evidence regarding the returns countries have derived from them, as well as the time lag between basic research investments and industrial innovations. In addition, attention is devoted to a variety of preconditions that are regarded as important if investments of this kind are to have a substantial economic payoff. Some of these tc :cs have been subjected to econometric analyses, described briefly in sections 24. In sections 5-7, a study of the sources and characteristics of academic research underlying industrial innovations in the United States is summarized. Sections 8-9 are concerned with technological change and the economic returns from R and D in developing countries. Section 10 focuses on some central problems in using science and technology effectively to solve economic and social prublems. Section 11 discusses the role of intellectual property rights, and section 12 treats the relat.onship between advanced scientific training and both R and D and the diffusion process in developing countries. Finally, section 13 provides a summary of the paper's findings. 2. Economic Returns from Investments in Industrial Technology in Developed Countries: Individual Innovations In recent decades, economists have made many studies of the social benefits from technological innovations. Assume that an innovation is a new product used by firms, and that it can lower the supply curve of the industry using the innovation. How far downward it will push this supply curve depends on the pricing policy of the innovator. Assume that the innovator sets a price for its new product that yields a profit to the innovator equivalent to r dollars per unit of output of the industry using the innovation (for example, r ollars per appliance in the case of a new type of metal used by the appliance industry). Also, assume that the industry using the innovation is competitive, that its demand curve is as shown in Figure 1, and that its supply curve is horizontal in the relevant range. In particular, suppose that, before the introduction of the innovation, this supply curve was SI in Figure 1, and the price charged by the industry using the innovation was PI. After the introduction of the innovation, this supply curve is S2 and the price is P2. Figure 1. Social Benefitfrom Product Innovation That Reduces the Cost of the Industry Using It Prioe or Cost P8r Unit PI Ou1tput of Industry Usino Demand the Innov8tion\ pl X _ S p2- Output of Industry Using ths Innovation 0 02 In this situation, the social benefits from the innovation can be measured by the sum of the two shaded areas in Figure 1. The top shaded area is the consumer surplus due to the lower price (P2 rather than 2 PJ) resulting from the application of the innovation. In addition, there is a resource saving, and a corresponding increase in output elsewhere in the economy, because the resource costs of producing the good using the innovation-including the resource costs of producing the innovation-are less than P202. Instead, they are P202 minus the profits of the innovator from the innovation, the latter being merely a transfer from the producers of the good using the innovation to the innovator. Consequently, in addition to the consumer surplus arising from the price reduction, there is a resource saving amounting to the profits of the innovator. In many cases, two adjustments must be made in this estimate corresponding to the lower shaded area in Figure 1. First, if the innovation replaces another product, the resource saving cited in the previous paragraph does not equal the profits of the innovator (from the innovation), but these profits less those that would have been made (by the innova.or or other firms) if the innovation had not occurred and the displaced product had been used instead. This is the correct measure of the resource saving. Second, if other firms imitate the innovator and sell the innovation to the industry that uses it, their profits from the sale of she innovation must be added to those of the innovator to get a full measure of the extent of the resource saving stemming from the innovation.' The social benefits from new products used by individuals rather than firms, and from new processes, can be measured too, but since the principles involved are much the same as those presented above, we will not describe the measurement procedures here. (A description is provided in Mansfield et al. 1977.) The social rate of return from an investuent in new technology is the interest rate received by society as a whole from this investment. To economists, the social rate of return from such investments is important, because it is a measure of the payoff to society from these investmnents. A high social rate of return indicates that society's resources are being used effectively and that more should be devoted to such investments, if the rate of retum remains high. While previous studies measuring the social rates of return from such investments had been made in agriculture (which will be taken up in section 8), it was not until 1977 that the first attempt to measure the social rate of return from investments in industrial innovations was published. The innovations studied came from a variety of .ndustries, occurred in firms of quite different sizes, and generally were of average or routine importance, not major breakthroughs. The results indicated that the median social rate of return from the investment in these innovations was 56 percent, a very high figure (Liansfield et al. 1977). Subsequent studies have obtained similar findings.2 3. Studies at the Firm or Industry Level Besides studying the economic effects of individual innovatiors, as in Figure 1, another technique is to estimate the rate of return from an industry's or firm's R and D expenditures. Ordinarily, it is supposed that the output of a particular industry or firm is a function of the amount of labor and capital used, as well as the amount of R and D done by the industry or firm itself and by its suppliers. Using historical data, economists estimate the relationship, holding other factors constant, between output and the amount of R and D. Given this relationship, one can estimate the rate of return from R and D. This approach has an advantage over that described in the previous section: there is no need to choose samples of individual innovations, which may be biased for one reason or another. (Some studies, but not those cited in the previous section, have looked only at innovations that are known in advance to have been successful; obviously, the estimated rate of return is biased upward.) Also, this approach is relatively I For other adjustments that sometimes must be made, as well as genera! comments on the limitations of this approach, see Mansfield et al. 1977. 2 The National Science Foundation commissioned two studies, one by Poster Associates and one by Robert Nathan Associates, to replicate this study. The median social rates of return in these studies, each including about 20 innovations, were 70 percent and 99 percent. 3 cheap because it relies largely on published data. Unfortunately, however, econometric studies of this sort have many problems of their own. Because they are built around estimated statistical relationships between a firm's or industry's rate of productivity increase and its R and D expenditures, there are many nettlesome problems that must be faced. Output measures, particularly in many science-based industries, are often questionable for these purposes. (For example, how should quality changes be determined, measured, and taken into account?) Data concerning R and D expenditures are often not comparable from firm to firm, or for the same firm over time, because of changes in the composition of R and D expenditures and for other reasons. The mathematical relationships between the dependent and independent variables may be misspecified, and the lag structure may be oversimplified. Despite these and other difficulties, many econometric estimates of the social and private rates of return from aggregate investments in R and D have been made.3 These estimates suggest that the rate of return from R and D has been high. In manufacturing, the estimated rates of return, have often been about 30 percent, and in agriculture, they generally have been about 40 or 50 percent. As would be expected, they differ from one time period to another. For example, Zvi Griliches and Frank Lichtenberg4 estimated that the social rate of return from privately financed industrial R and D was about 10 percent in 1959-63, about 20 percent in 1964-68, and about 35 percent in 1969-73. Further, they seem to differ from one industry to another. For example, Jeffery Bernstein and M.I. Nadiri (1988) have estimated that the social rate of return in 1981 was about 20 percent in the transportation equipment industry, about 30 percent in the chemical and electrical products industries, and over 40 percent in the machinery and instruments industries. 4. The Contribution of Academic Research to Industrial Innovation Recent work by economists has indicated that technological change in some industries has been based to a significant degree on academic research. Richard Nelson (1986), using a survey of R and D managers, and Adam Jaffe (1989), using an econometric study of patent data, found evidence that this was true. My own findings (Mansfield 1991) indicate that a substantial proportio2 of the industrial innovations in some very important industries have been dependent on recent academic research. Based on data obtained from 76 firms in the seven industries listed in Table 1, about 11 percent of their new products and about 9 percent of their new processes could not have been developed (without substantial delay) in the absence of recent academic research (defined as academic research occurring within 15 years of the commercialization of the innovation).S As shown in Table 1, the percentage of new products and processes based in this way on recent academic research seems to be highest in the drug industry and lowest in the petroleum industry.6 3 For a list of publications including such estimates, see Mansfield 1991b. 4 See Griliches 1984. 5 By 'substantial delay,' we mean a delay of a year or more, according to rough estimates made by the firms. 6 New products and processes sometimes could have been developed without the findings of recent academic research, but it would have been much more expensive and time-consuming to do so. In Table 1, such cases are designated as ones where development occurred with 'very substantial aid from recent academic research. Approximately 8 percent of these firms' new products and approximately 6 percent of their new processes during 1975-85 fell into this category. Often while it was technically possible for the firm to have developed them without the findings of recent academic research, it seemed economically undesirable to have tried it. Consequently, in a 4 Table 1. New Products and Processes Based on Recent Academic Research, Seven Industries, United States, 1975-1985 (percent) Could not have been developed Was developed wth very (without substantial delay) in the substantial aid from recent absence of recent research academic research Industry Products Processes Products Processes Information processing 11 11 17 16 Electronics 6 3 3 4 Chemical 4 2 4 4 Instruments 16 2 5 1 Pharmaceutical 27 29 17 8 Metals 13 12 9 9 Petroleum 1 1 1 1 Industry mean 11 9 8 6 Source: Mansfield 1991a. To prevent confusion, it is worthwhile to note that many of the innovations based on recent academic research were not invented at universities. Academic research often provides new theoretical and empirical findings and new types of instrumentation tnat are essential for the development of a new product or process, but does not provide the specific invention itself. As the Government-University- Industry Research Roundtable and Industrial Research Institute (1991) emphasize, industry, not the universities, plays the leading role in the conception of inventions and the embodiment of them in new products and processes. For each firm's new products and processes introduced in 1975-85 that, according to the firm, could not have been developed (without substantial delay) in the absence of recent academic research, I obtained information concerning the mean time interval between the relevant academic research finding and the first commercial introduction of the product or process. (f more than one such research finding was needed for the development of the innovation, this time interval was measured from the year when the last of these findings was obtained.) As shown in Table 2, the mean time lag in these industries was about 7 years. In interpreting this result, note once again that these data pertain only to recent academic research. Particularly in industries like drugs, instruments, and information processing, the contribution of academic research to industrial innovation has been considerable. In the seven industries in Table 1, new products first commercialized in 1982-85 that could not have been developed (without substantial delay) in the absence of academic research accounted for about $24 billion of sales in 1985 alone. And in these industries, new processes first commercialized in 1982-85 that could not have been developed (without substantial delay) in the absence of recent academic research resulted in about $7 billion in savings in 1985 alone. While these figures are rough, they certainly indicate that industrial innovation in these industries has been based to a substantial degree on recent academic research. practical sense, many of these innovations could not have been developed (without substantial delay) in the absence of recent academic research. 5 Table 2. Average lime Lag Between a Recent Academic Research Finding and the Most Commercial Introduction of a New Product or Process Based on this Finding, Seven Industries, 1975-85 Innovations that could not have Innovations that were developed (without substantial developed with very F-""stantial delay) in the absence of recent aidfrom recent aceaemic academic research (mean research (mean number of Industry number of years) years) Infortnation processing 7.0 6.2 Electrical 5.3 4.9 Chemical 6.8 7.3 Instruments 4.2 4.2 Drugs 8.8 10.3 Metals 9.8 5.7 OilP n.a. n.a. Industry mean" 7.0 6.4 a Reiiable data could r - be obtained for a sufficiently large number of innovations to allow us to present figures for this udustry. b Unweighted mean of industry figures. Source: See Section 4. Without recent academic research, it seems likely that there would have been a substantial reduction in the value of society's economic output. If the flow of industrial innovations had been delayed in accord with firms' estimates, a hig'ily simplified model suggests that the social rate of return from the investment in academic research would have been about 20-30 percent (Mansfield 1991). However, crude estimates of this sort vary, depending on a variety of assumptions, and it is relatively easy to criticize them. Clearly, they are by no means a full or satisfactory solution to the longstanding-and extraordinarily difficult-problem of evaluating the payoff to society from academic research. One of the most difficult questions is how to allocate the social returns between academic and industrial research. However, the social rate of return from academic research is substantial (about 28 percent) even if the social rate of return from industrial R and D, plant and equipment, and startup costs is assumed to be over 50 percent, a very generous assumption (Mansfield 1992a). Moreover, many important economic benefits of academic research (for example, its contribution to education) are ignored, so the benefits estimates are likely to be conservative. Certainly, the data indicate that the economic returns from investments in academic research have been considerable. But this does not mean that the bulk of these returns show up quickly. Using data concerning the numbers of scientific papers published and the number of industrial scientists and engineers, Adams (1990) found an average lag of about 20 years between the appearance of research in the scientific community and its effect on productivity in the form of knowledge absorbed by firms. As would be expected, this average lag is much longer than that in Table 2, which pertains only to innovations based on recent academic research. (Given our definition of recent academic research, the latter lag must be less than 15 years.) 5. Location and Characteristics of Academic Research Underlying Industrial Innovations What kinds of academic research have made the greatest contribution to industrial innovation? To help answer this question, I drew a random sample of 70 major firms from the seven industries in Tables 1 6 and 2. Each firm was asked to cite about five academic researchers wiose work in the 1970s and 1980s contributed most importantly to he firm's new products and processes introduced in the 1980c. Eventually, usable data were obtained from 66 of the 70 firms in the sample. These firms, which account for about a third of the R and D expenditures in these industries, cited 321 academic researchers." Table 3 lists the universities and types of departments cited most frequently by the firms in each industry. Generally, the most frequently-cited universities are world leaders in science and technology. For example, MIT, Berkeley, Illinois, Stanford, and CMU are most frequently cited in electronics; and Harvard, UCSF, Stanford, and Yale are most frequently cited in pharmaceuticals. But not all of the most frequently cited universities are world leaders in the relevant fields. Thus, neither Washington Unive,ryW nor the University of Utah are among the top dozen departments of chemistry, according to %. assessments of the National Academy of Sciences (1982). Table 3. Universities and Departments Containing the Largest Percentage of Academic Researchers Cited by 66 Major Firms (in the Electronics, Information Processing, Pharmaceutical, Chemical, Petroleum, Metals, and Insrumental Industries) as Contributing Most Importantly (During the 1970s and 1980s) to the Development of Their New Products and Processes Introduced in the 1980s (percent) ELECIRONICS INFORMATION PROCESSING PHARMACEUTICALS University Delartment Universitv Deartment Universit Peprtment MIT 15 Elect. Eng. 50 MIT 9 Comp. Sci. 38 Harvard 13 Biology 11 Berkeley 13 Mech. Eng. 11 Berkeley 8 Elect. Eng. 10 UCSF 6 Chemistry' 22 Illinois 8 Illinois 6 Mech. Png. 10 Stanford 6 Pharmacolo"y4 Stanford 7 Minnesota 6 Yale 6 CMU 7 Stanford 6 CHEMICALS PETROLEUM METALS University Department Universit Department University Denatment Washington 12 Chemistry 53 Delaware 11 Chem. Eng. 46 Utah 16 Mat. Sci. 32 MIT 8 Chem. Eng. 15 MIT 7 Chemistry 8 MIT 12 Civil Eng. 20 Utah 6 Notre Dame 7 Ohio State 8 Chemistry 12 Princeton 7 Mech. Bug 8 VPI 7 INSTRUMENTS University Department Yale 9 Chemistry 26 Indiana 9 Physics 16 Radiology 16 Includes biochemistry. Source: Mansfield 1993. The bulk of the cited academic research took place in departments closely related to the technology of the industry in question. In the electronics industry, over 60 percent of the cited academic researchers were in electrical engineering or mechanical engineering departments. In the chemical industry, almost 70 percent were in chemistry or chemical engineering departments. However, because the pharmaceutical industry, as defined here, includes some medical products firms, the academic researchers cited by this 7 For details, see Mansfield 1993. 7 Industry seemed to be scattered over a wider varietv of fields and departments than in the electronics or chemical industries. 6. Effects of Faculty Quality, Scale of Research Effort, and Geographical Proximity One factor that would be expected to influence how frequently a particular university is cited in this way is the quality of the university's faculty. There is generally a tendency for the number of citations received by a university to be directly related to the quality of its faculty in the relevant department, as measured by the ratings of the National Academy of Sciences (1982). However, in chemicals no such relationship seems to show up; and in drugs and information processing. The relationship, while direct, is not statistically significant. It is not hard to understand why these statistical *-'ationships frequently are weak. Some universities have very competent researchers in particular specialties, even though their entire faculty in a particular department may not be iated very highly; these universities receive more citations than would be expected based on overall departmental ratings. Also, some universities with fine faculties stress research with a very long-term payoff that would be less likely to show up in citations of this sort. Recall that firms were asked to cite researchers whose work in the 1970s and 1980s contributed most importantly to new products and processes introduced by these firms in the 1980s. Another factor that would be expected to affect how frequently a particular university is cited in this way is the size of the university's annual R and D expenditures in the relevant area. Policy makers and anlYStS often state that a critical mass of researchers and equipment should be present. My results indicate that there is a direct relationship in all the included industries between a university's R and 1D expenditures and its number of citations, but in chemicals and electronics this relationship is not statistically significant. However, this does not mean that the cited academic researchers had very large budgets. Eech of these researchers was asked to estimate the level of his or her average anrnual academic research expenditures during the 1970s and 1980s; responses were obtained from about 90 percent of them. They indicate quite clearly that, except for the pharmaceutical industry, the bulk of these researchers had annual budgets of less than $250,000 (Mansfield 1993). Another factor that is important is geographical proximity. There are many advantages in a firm's working with, and keeping abreast of developments at, local universities.' In many cases, firms that are not close to a leading university find it worthwhile to support research at local universities even though, based on the NAS ratings, their faculties are not among the leaders. Adam Jaffe (1989), using patent data, found "evidence of geographically mediated commercial spillovers from university research. The effect is strongest in drugs, slightly smaller and less significant in chemicals, and smaller but quite significant in electronics... Thus, a state that improves its university research system will increase local innovation both by attracting industrial R and D and augmenting its productivity.9 Audretech, and Feldman (1991), using data regarding the number of innovations, found the same thing. Also, the Government-University-Industry Research Roundtable and Industrial Research Institute (1991) found that "In establishing relationships with universities, geographical proximity is very important to most firms, according to a number of industry interviewees, in order to maximize the potential for communication and interaction."'0 My own findings indicate that the number of citations received by a university increases with the percent of responding firms located in the same state as the university. In the electronics and information processing industries, about 40 percent of the firms' citations were to universities in their own state; in 8 For example, see Peters and Fusfeld 1982. 9 Jaffe 1989, pp. 967-68. 10 Government-University-Industry Research Roundtable and Industrial Research Institute 1991, p.13. 8 the instruments, pharmaceutical, and chemical industries, this was true for about 25 percent of the citations (Table 4). To some extent, of course, this reflects the fact that firts in these industries often locate in areas where there are strong universities. But in cases where firms are not located near strong universities, they still seem to ite local universities. Table 4. Cited Academic Researchers that Worked at Universities in the Same State as the Firm and Percentage that Worked at U.S. Universities, Seven Industries (percent) Percentage of Cited Academic Researchers Located in Same State Located at U.S. Industry Citing the Academic Researcher as Firmn University Electronics 39 95 Information processing 38 91 Pharmaceuticals 23 83 Chemicals 27 80 Petroleum 5 93 Metals 19 80 Instruments 21 100 Mean 25 89 Source: See section 5. In the petroleum industry, the percentage of the firms' citations to universities in their own state seems relatively low (Table 4). Since the sample size is small in this industry, this may be due in part to sampling errors. Also, in some cases, the firms cited universities that were close by, even though they were in another state. Further, they sometin.3s cited universities that were close to some division of their company, even though they were not close to the firms' principal research laboratories. Thus, the figures for these industries may underestimate the extent to which geographical proximity was a factor. The bulk of the cited academic researchers worked in the United States (Table 4). This was particularly true in electronics, information processing, petroleum, and instruments. In pharmaceuticals, chemicals, and metals, about 20 percent of the cited academic researchers worked at foreign universities. The preponderance of U.S. researchers is due partly to the fact that American academic research has been very strong, particularly in many of these areas. Also, the bulk of the cited foreign academic researchers was located in Canada and the United Kingdom, which suggests that geographical proximity, language factors, and cultural similarities may help to determine the extent to which firms draw on academic research outside the United States, as well as within it. 7. The Role of Students as Transfer Agents Having discussed the sources and characteristics of academic research underlying industrial innovations in the United States, we must note the important role of students, both graduate and undergraduate, in permitting and encouraging firms to access the results of recent academic research. For example, the industry participants in the National Science Foundation's Industry/University Cooperative Research Program have found that better personnel recruitment has been one of the principal benefits of their participation in the program Good students can be, and often are, effective agents in the transfer of 9 relatively new academic research findings to industry.'" In the case of the academic researchers cited by the firms as having contributed most importantly to their new products and processes introduced in the 1980s, my results indicate that students have played this role. Indeed, the bulk of these cited academic researchers had students that took jobs with firms that helped to finance the researcher's work. (The percentages were: electronics, 77 percent; information processing, 87 percent; pharmaceuticals, 73 percent; chemicals, 64 percent; and petroleum, 91 percent.) Thus, firms that support academic research often hire students engaged in it.12 Peters and Fusfeld concur, saying that "chemical company representatives look toward new employees that will provide a window on new technologies which will help the company initiate new product or process lines. Hence, they will support university-sponsored Industrial Liaison Programs in biotechnology to gain access to potential employees. In another typical instance, a company has an unforeseen problem with a product. After consulting with a local professor, the company hires one of his students to solve the problem over the summer. Alternatively, the company contracts to solve the problem and later hires the student who worked on it."'3 8. Technological Change in Developing Countries Having reviewed very briefly some of the literature dealing with the economic returns in industrialized countries (particularly the United States) from investments in academic and industrial research, as well as the sources and characteristics of academic research underlying industrial innovations, we turn to a discussion of the economic returns from such investments in the developing countries. To begin with, it is essential to recognize several key points regarding the nature of technological change in developing countries. First, developing countries very rarely produce products or processes that are fundamentally new to the world. Instead, as Jorge Katz (1984) has observed, "specialized technical departments of medium size and large firms ... generate incremental units of technical knowledge in the areas of product design, process engineering and production planning and organization. They adapt foreign technology to the local environment and gradually build up a stock of proprietary technology and know-how highly specific to the firm. It is here where domestic technological capabilities actually appear and develop."1' Second, the design capabilities of even the most sophisticated capital goods producers in Brazil, India, and South Korea are still relatively limited. According to D. Chudnovsky (1985), "the evidence is far from conclusive about the progress made by firms in mastering design and manufacturing technology. All firms rely on licensors for design technology and most of them have not been able as yet to use licensing agreements to learn design methodology. One reason is the reluctance of licensors to providle recent designs ... basic design, and, in some cases, even detailed designs for complex capital goods are not yet mastered by leading producers in the countries studied. Accordingly, they suffer from a major handicap which affects their ability to fulfill their role as eventual generators of technological innovations."'5 " Gray, Gidley, and Koester 1988. 12 Mansfield 1993. 3 Peters and Fusfeld 1982, p.35. 14 J. Katz 1984. Is D. Chudnovsky 1985. 10 Third, some developing countries have nonetheless become efficient producers of industrial goods. Westphal, Rhee, and Pursell (1981) have pointed out that in South Korea technological expertise is much greater in plant operation than in plant and product design. "It thus appears that the know-how to operate production processes efficiently is, to a large degree, independent of the ability to use the underlying engineering principles in investment activity ... That is not to deny that ... Koreans have become increasingly involved in various phases of project implementation. Nonetheless, it is not too great an overstatement to say that Korea has become a significant industrial power simply on the basis of efficiency in production."t6 Fourth, some developing countries are also engaged in the export of technology. Dahlman and Sercovich (1984) have found that most such technology exports are based on a cost advantage (often lower wages) in providing a good or service that is also provided by developed countries, or on advantages rooted in technological experience in developing country conditions. They conclude that "most of the trade takes place in industries where the technology is more easily acquired because it has been around for a long time, and where it is relatively easy to keep up with the world frontier."" International indicators of technology exchange suggest that developing countries do not export significant amounts of technology until they reach a relatively advanced stage of development. 9. Economic Returns from R and D in Developing Countries Most of the research concerning the returns from R and D in developing countries has focused on agriculture. Local agricultural R and D has been a major source of new technology because agricultural technology generally cannot be transferred directly from one region to another; it ordinarily must be adapted to the relevant soil, market and climatic conditions by local agricultural R and D. Pray and Ruttan (1990) have provided a summary of the many studies of the returns from agricultural R and D in Asia. The methods used are essentially the same as those taken up in sections 2 and 3 above. As shown in Table 5, the estimated social rates of return are very high, often 50 percent or more. Based on 159 estimates of the returns to agricultural R and D, most of which pertain to developing countries, Robert Evenson (1989) has concluded that returns to agricultural R and D are bigger than those from other pub!ic-sector investments and generally bigger than those from industrial R and D. Interestingly, the distribution of rates of return for the developing countries does not differ substantially from the distribution for the developed countries. Also, returns to research conducted by international agricultural research centers have been high. Further, five of these studies looked at the social returns from private sector agricultural R and D (agricultural machinery and agricultural chemicals), and found the social returns generally to be high. Very few studies have been made of the economic returns from industrial R and D in developing countries. Howard Pack (1990) has examined the potential returns from productivity-increasing R and D in Philippine textile companies, and has concluded that more than four-fifths of these firms would obtain higher returns from such R and D than from other investments. "Two benefit-cost ratios are presented; the first includes all benefits, the second only half. The smaller estimate is relevant if some of the reduction in cost to the domestic economy could be obtained simply by a decrease in tariffs or removal of import quotas; however, an effort to do so without providing aid to firms that perceive themselves unable to meet international competition will usually encounter significant political opposition The more inclusive measure yields a sector wide benefit-cost ratio of 4.33 and absolute net benefits 16 L. Westphal, Y. Rhee, and G. Pursell 1981. 17 C. Daliman and F. Sercovich 1984. 11 Table 5. &tinaed Social Rates of Returnfrom Agilcultural R and D nAsta (percent) Internal Rate Study Counny Public Program Period Communities of Retur Pray, 1978 Puajab R&D+Ext 1906-56 Crops 34-44 Puajab R&D +Ext 1948-73 Crops 23-37 (Pakistan) Pee, 1977 Malaysia R&D 1932-73 Rubber 24 Pray, 1980 Bangladesh R&D 1961-77 Wheat, rice 30-35 Nagy, 1987 Pakistan R&D 1967-81 Wheat 58 R&D 1967-81 Maize 19 Pray & Ahmed, 1987 Bangladesh R&D 1948-91 Crops 35 Kahlon, Bai, Saxesa & Jha, 1977 India R&D 1961-71 Crops 63 Evenson & Kislev, 1975 India Extension Crops 15 Evenson Philippines R&D 1966-75 Rice 75 Salmon, 1987 Indonesia R&D 1972-77 Rice 100+ Pray & Ahmned, 1987 Bangladesh R&D 1948-81 Crops 100+ Feder, Lau & Northwest Slade India Extension 1983 Wheat (100+) Evenson & Iha, R&D 1953-71 Crops 40 1973 India Extension 1953-71 Crops 14 Evenson & Jha, Asia-Nat. R&D 1950-65 Rice 32-39 1978 Asia-Nat. R&D 1966-75 Rice 73-78 Asia-IARC R&D 1966-75 Rice 74-208 Flores, Evenson & Tropics R&D 1966-75 Rice Haymai, 1987 Nagy, 1987 Pakistan R&D 1959-79 Crops & Livestock 64.5 Evenson, 1978 10 Asian IARC R&D 1972-79 Crops 80+ LDCs Natl. R&D 1972-79 Crops 50 Extension 1972-78 Crops 80+ Evenson, 1988 North India R&D 1956-83 Crops 72 Evenson, 1986 Philippines Natl. R&D 1948-84 Crops 70 Reg. R&D 1948-84 Crops 70 Ext. R&D In their article they report that there was a 90 percent probability that the rate of return was over 15 percent Birkhaemer, Breason and Feder report that if they had used conventional methods to calculate rates of return it would have been over 100 percent. Source: Pray and Ruttan 1990. of $71 million. The narrower calculation, comparing half of the benefits and all of the costs, yields a sector wide ratio of 2.30. Both ratios are quite high when compared with those projected in most industrial project evaluations and are more robust as the activities envisaged are narrower in scope than the major uncertainties encountered in establishing an entirely new plant."'8 10. Problems in Using Science and Technology Effectively to Solve Economic and Social Problems While the results summarized in the previous section suggest that the economic returns from agricultural R and D in developing countries have tended to be high, it does not follow that all R and D in developing 18 H. Pack 1990, 1.225. 12 countries has had large economic benefits. In part, this is because the effective utilization of science and technology can be hampered by low levels of investment, distorted prices and markets, sluggish export growth, lack of competition, short-sighted laws and regulations, and other negative aspects of a country's economic environment. But this is only part of the story. In addition, there can be more subtle, but nonetheless important, problems at the crucial interface between the researchers and the potential users of their research. India is an interesting case in point. About 80 percent of India's R and D is paid for and performed by the government (Table 6). The bulk of this R and D has been carried out in over 200 research institutes, practically all controlled by ministries or departments located in New Delhi. According to Nathan Rosenberg, "the process of producing useful knowledge-the innovation process-seems to have been visualized as a linear process, with the implication that the major problem was seen as commitment of sufficient resources for 'basic' and 'applied' research that would minimize unnecessary duplication of research. The second major implication of this point of view was that the utilization of the product of research activities was not seen as a real issue; rather it seems to have been more or less taken for granted. The functioning of a major Indian public sector research institution-The Council for Scientific and Industrial Research (CSIR)-exemplifies these points. Table 6. Sources of R and D finding and Nature of R and D Performers, Selected Developing and Developed Countries (percent) R and D Funding R and D Performance County Gov't. Industry Foreign Other Industry Univ. Govyt. Argentina 95 0 1 4 41 22 37 Brazil 67 20 5 8 30 17 53 India 87 3 0 0 26 0 74 Mexico 15 1 1 83 30 51 19 South Korea 19 81 0 0 67 11 22 Japan 21 79 0 0 67 20 13 United States 47 50 0 3 73 12 15 Note: For most countries, the data pertain to the mid-1980s. Source: Unesco, Statistical Yearbook, Paris, 1988. "The CSIR used to account for an extremely large share of industrial R and D in India-78 percent in 1968. In the past decade its share in the total industrial R and D expenditure of India has come down to around 20-25 percent as manufacturing firms in both the public and private sectors have increasingly undertaken in-house R and D. Despite the vast amount of resources allocated to the CSIR, it has failed to make any discernible impact upon the productivity and efficiency of the industrial sector in India. One major reason for its failure seems to have been the fact that its research activities were at times effectively insulated from information about the needs of the public and private sector firms that would be the ultimate users of their output. For example, studies showed that most projects tended to be initiated by scientists themselves and that users of technologies generated by CSIR labs tended to be confined to firms situated in close geographical proximity. A related problem was that work on these technologies was terminated at the prototype stage. This left the later development process to the potential users who were completely disassociated from the earlier research activities, and therefore commonly lacked any basis for intelligent decisions with respect to the possibilities for manufacture and subsequent commercialization 13 of the technology."' In contrast, according to Gustav Ranis, the fast-growing East Asian countries like South Korea have forced science and technology institutes to be "useful to the marketplace either by only providing a partial subsidy and/or one that is being reduced over time, as was the case with the Korean Advanced Institute of Science and Technology, or by policies designed to encourage firm level R and D through tax provisions that permit the current costing of R and D. More often, government-financed research shows a bias in favor of export crops in agriculture and large-scale advanced technology in nonagriculture. Scientists in these institutes are all too often subject to the siren calls of the international college of scientists and engineers, with the motivations running in the direction of frontier research and of researchers more interested in preparing papers for international conferences and getting promoted for enhancing advanced techniques than in the comparatively dreary task of rendering existing technologies more appropriate. A really substantial volume of scarce human aWd physical resources is often allocated to such institutes which do not, in fact, yield a measurable rate of return, but represent, in a sense, a closed system setting its own productivity criteria in terms of some basic scientific or high technology output. In contrast, the [East Asian] countries exercised greater self-restraint, spent relatively less human and capital resources on these institutes and were more selective in their allocation mechanisms. Institutes were assisted in this process by budgetary pressures which forced them to rely increasingly on private sector contracts in place of govermnent subsidies."`2 To illustrate another sort of interface problem that can arise, consider Pablo Bifani's (1988) description of academic research on biotechnology in Latin America: "By and large the universities of the region are covering the principal areas of basic research in biotechnology ... However, most of the Latin A.nerican universities are basically teaching institutions and their research is rarely aimed at practical application. ITis is the consequence and cause of little or no tradition of university-industry interaction. Another aspect ... is the evaluation of the infrastructure and equipment of the research centers of Latin American universities in the light of the specific demands of biotechnology. An important point here is the lack of facilities for carrying out experiments at the pilot plant level. On the one hand, the academic approach of Latin American universities generates a certain suspicion towards work in a pilot plant, which is considered the responsibility of technologists and does not entail sufficient prestige. On the other hand, since productive enterprises in Latin America do virtually no R and D, they do not have pilot plants, which tends to reinforce the tradition of buying mature technologies. Thus, one of the links between scientific activity and the practical application of scientific knowledge is particularly weak."21 11. The Role of Intellectual Property Rights Intellectual property consists chiefly of patents, plant breeders' rights, copyrights, trademarks, and trade secrets. Economists have focused more attention on patents than other forms of intellectual property. Patents are widely viewed as an important incentive to get an inventor to put in the work required to produce an invention, to get firms to carry out the further work and make the necessary investment in pilot plants and other items that are needed to bring the invention to commercial use, and to get the invention disclosed to society at large. There are well-known differences between the industrialized countries and the developing countries in their attitudes toward intellectual property rights. Many developing countries like India, Thailand, Brazil, and Nigeria have had relatively weak laws to protect 19 N. Rosenberg 1990, pp.149-50. 20 G. Ranis 1990, pp. 172-73. 21 P. Bifani 1988, pp.266-67. 14 intellectual property and less than diligent enforcement of the laws that exist (see Table 7). Table 7. Major U.S. Chemical and Drug Fsns Reporting that Intellectual Property Rights Protection In Each of 16 Countries Is too Weak to Permit Licensing their Newest and/or Most Effectve Technology to Unrelated Firms There, 1991 (percent) CountWy Percentage Country Percentage Argentina 62 Nigeria 73 Brazil 69 Philippines 47 chile 47 Singapore 25 Hong Kong 33 South Korea 38 India 81 Spain 6 Indonesia 73 Taiwan 44 Japan 12 Thailand 73 Mexico 56 Venezuela 62 Source: Mansfield forthcoming. According to Robert Sherwood (1990), weak intellectual property rights protection has lowered the economic returns from academic research in Brazil. "Interviews with university researchers revealed the problems they face when they attempt to take their technical discoveries from the university to the marketplace. They feel unable to follow the path taken by their counterparts in the developed countries. Rather than disclose their findings to experienced business and technical people under cover of patent applications and nondisclosure agreements, the bad experience of other university researchers warns them to attempt to commercialize their findings on their own ... r[hus,] the inventor/researcher devotes less time to his or her students, ... less time to conducting research, [and probably adds] to the failure rate for start-up businesses in that country. ' Based on recent studies, it seems to be generally agreed that patents are regarded as much more important in the pharmaceutical and chemical industries than in others. Although it is frequently argued that stronger protection of intellectual property rights would help to promote indigenous technological and innovative activities in the developing countries, there is little or no information on which one can base an estimate of how big or small this effect may be. Also, while weak intellectual property rights protection in developing countries seems likely to depress the incentives for technological innovation in the developed countries, no estimates have been made of the magnitude of this effect.' Because developing countries are engaged so heavily in adaptive invention (that is, in modifying the inventions created in the developed countries), some economists argue that they need systems of intellectual property rights protection that promote access to foreign inventions and stimulate domestic adaptive invention. According to Robert Evenson (1990), a major part of such a system should be the utility model (or "petty patent") because it is well suited to stimulating adaptive invention. Ihis point Viis corroborated by two studies of the agricultural implements industry in Brazil (Dahab 1986) and in the Philippines (Mikkelsen 1984) which conclude that the utility model stimulated adaptive inventions in these countries and enabled domestic firms to increase their competitiveness with multinational firms whose inventions they imitated. Another study by Otsuka, Ranis, and Saxonhouse (1988) reports similar I R. Sherwood 1990, p.142. " See E. Mansfield forthcoming. 15 conclusions for textiles in Japan and India. All three studies reported that much of this R and D was of the "informal" or blue-collar type. Ranis (1990) discusses the relevance of informal, blue-collar R and D in improving industrial productivity?' 12. Advanced Scientific Training and the Diffusion of New Techniques Economists like Theodore Schultz and Gary Becker have tried to estimate the profitability both to society and to the person who invests in various levels of education, and Edward Denison, in his studies of the process of economic growth, has estimated the contribution of improved education and training to economic growth. For example, Denison (1962, 1979) found that improved education and training was responsible for about 13 percent of U.S. economic growth in 1909-29, about 27 percent in 1929-57, and about 20 percent in 1969-73. Although evidence of this sort is valuable, it cannot tell us much about the social rate of return from a less developed country's investments in highly specialized types of advanced scientific training in areas like biotechnology. So far as I know, such estimates are not available. Without question, they would be difficult to make because the social rate of return may vary considerably from the private rate of return. To carry out effective R and D, a nation must invest in advanced scientific training. For example, according to Dahlman and Frischtak (1990), 'Brazil's education system is one of the main obstacles to the country's modernization and technological upgrading. Although major deficiencies characterize all components of the system, more fundamental weaknesses are observed in primary education and at the top end in science and engineering... The combination of low-quality undergraduate teaching, few strong graduate programs, a relatively small pool of students in science, mathematics and engineering, and underuse of existing educational capabilities outside formal institutions of higher learning has contributed to Brazil's lagging R and D manpower. "' As shown in Table 8, students in science, mathematics, and engineering constitute a relatively small percentage of Brazil's population.' Particularly in the developing countries, diffusion or imitation may be of much more economic significance than innovation. From an economic point of view, it is much more important for a nation to exploit a new technology successfully than to be the first to introduce it. Advanced scientific training plays a central role in the diffusion of many science based innovations as well as in R and D. Without properly trained human resources, the diffusion process is likely to be stymied. For this and other reasons, economists like Richard Nelson (1990) have concluded that "building up the educational infrastructure, including building in enough research support so that university faculty themselves are able to stay up with their moving fields, probably is the most important thing a govermnent can do to push along the industrialization process these day." 27 Frances Stewart (1990), who emphasizes quite properly that effective technology transfer involves learning based on conscious local effort, points out that "High levels of human capital are essential for this process. Government policy to build up education in general and scientific and technical education in particular has been a vital element in the success of technological development in Japan, South Korea, 24 W. Siebeck with R. Evenson, W. Lesser, and C. Primo Braga 1990, p.42. 25 C. Dahlman and C. Frischtak 1990, pp.20-21. 26 Of course, the figures in Table 8 should be interpreted with caution because they are influenced by the age distribution of a country's population, among other things. 2' R. Nelson 1990, pp.79-80. 16 Table 8. Popldation that are Teriary Students, Selected Countries (percent) Students in Medicine, Students in Architerture, Transport, Science, Communication, Agriculture, Mathematics Tertiary Science, Mathematics and and Students in Country Students Engineering Engineering Engineering S. Korea 3.6 1.4 0.76 0.54 Taiwan 2.1 1.1 0.78 0.68 Hong Kong 1.4 0.7 0.51 0.41 Singapore 1.4 0.9 0.73 0.61 Brazil 1.1 0.4 0.24 0.13 Mexico 1.5 0.7 0.42 0.35 India 0.8 0.2 0.19 0.06 Indonesia 0.6 0.1 0.09 0.07 Japan 2.0 0.6 0.40 0.34 Source: S. Lall forthcoming. and Taiwan, and its absence one of the major missing factors in many African countries."21 Westphal, Kim, and Dahlman (1985), in their comparison of Argentina, Brazil, India, Mexico, and South Korea, conclude that: "What stands out about the educational pattern are the high proportion of postsecondary students abroad, the high secondary enrollment rate and the high percentage of engineering students among postsecondary students ... by the late 1970s South Korea had by far the highest percentage of scientists and engineers." 29 As a further illustration, consider Singapore's strategy to enhance its competence in information technology. In 1980, there were only 850 programmers, systems analysts, and other data-processing professionals in Singapore. A National Computer Board was established to stimulate professional training programs. "Through the crcation of a variety of training and education programs designed for skills at different levels, supplemented by efforts to attract experienced senior talent from abroad, the local computer manpower pool grew to more 4,000, supplemented by expatriates three levels of computer education were established: the public schools, tertiary institutions, and training for managers and professionals in a variety of post-graduate programs. Private schools emerged to meet the demand for access to knowledge about information technology. The total output of computer professionals grew rapidly from 124 in 1982 to more than 500 per year by 1984, levelling at about 600 new entrants annually by 1986... "Overseas consultants and technical trainers were attracted to the market for education, and advanced private-sector enterprises extended internal training capacity to provide information-technology-oriented conference and seminar services to external audiences"'. Whereas the total number of computer professionals was only about 800 in 1980, it had increased to about 5,500 in 1986, and was expected to rise to 10,000 in 1992. Multinational firms like International 28 F. Stewart 1990, pp.319-20. I L. Westphal, L. Kim, and C. Dahlman 1985, as quoted in ibid., p.320. 30 A. Gilbert 1990, pp.325-26. 17 Business Machines, International Computer Limited, and Grumman collaborated with local universities like the National University of Singapore, Ngee Ann Polytechnic, and Nanyang Technological Institute to train manpower and promote technological development. For example, collaboration between Grumman and Nanyang tied to the creation of a center "to promote, disseminate, and develop CAD! CAM technology, and to become a center of excellence in this area of technology and to transfer such know-how and applications to the industries in Singapore."" 13. Summary and Conclusions Recent studies by economists indicate that the social rate of return from investments in new industrial technology in devel- ad countries seems to be very high, perhaps 30 percent or more, and that technological change .n many industries has been based to a significant extent on recent academic research. In the information processing, electronics, chemical, instruments, drugs, metals, and petroleum industries in the United States, about 11 percent of the new products and about 9 percent of the new processes could not have been developed (without substantial delay) in the absence of recent academic research. Academic research often provides new theoretical and empirical findings and new types of instrumentation that are essential for the development of a new product or process, but seldom provides the specific invention itself. Without recent academic research, it seems likely that there would have been a substantial reduction in the value of society's economic output. A highly simplified model suggests that the social rate of return from the investment in academic research may have been about 20-30 percent. In making this estimate, a generous allowance is made for the social returns from the industrial R and D that was needed as well. How much a particular university contributes to industrial innovation seems to be directly related to the quality of its faculty, the size of its R and D expenditures in the relevant area, and the extent to which it is located near firms in the relevant industry. At the project level, the projects credited with making the biggest contributions of this sort tended to have relatively modest budgets. Students often have been effective agents in the transfer of relatively new academic research findings to industry. Most of the research concerning the economic returns from R and D in developing countries has focused on agriculture. The estimated social rates of return are very high, often 50 percent or more. For agricultural R and D, the distribution of rates of return for the developing countries does not differ substantially from the distribution of rates of return for the developed countries. Some studies have looked at the social returns from private-sector agricultural R and D (agricultural machinery and agri- cultural chemicals), and found the social returns generally to be high. The very few studies of industrial R and D in developing countries suggest that its economic returns are sometimes high as well, although the evidence is extremely sparse. While the economic returns from agricultural R and D seem to have been high, this is not true for all R and D in developing countries. In part, the effective utilization of science and technology has been hampered by low levels of investments, distorted prices and markets, sluggish export growth, lack of competition, short-sighted laws and regulations, and other negative aspects of a country's economic environment. In addition, it has been hampered by faulty interfaces between researchers, on the one hand, and the users of research findings, on the other. In some countries, research institutions, including universities, have been cut off from information-or ha, e tended to ignore information-about the needs of the public and private sector firms that would be the ultimate users of their output. The available evidence seems to emphasize the crucial importance of incentives to get researchers to pay proper attention to users' needs. 31 T. Soon and T. Huat 1990, p.340. 18 In recent years, there has been an enormous amount of attention devoted to intellectual property rights. According to some observers, weak intellectual property rights are another factor that lowers the economic returns from academic research in some developing countries. However, there is little or no information on which one can base an estimate of how big or small this effect may be. Some economists also argue that the utility model should be a major part of the system of intellectual property rights protection in developing countries because it would stimulate adaptive invention. Investments in advanced scientific training play a major role in the diffusion of many science-based innovations as well as in R and D. Leading economists concerned with economic development seem to argue that, for a developing country to hasten the industrialization process, investments of this sort are of central importance. However, although estimates have been made of the economic returns from broad categories of education and traiPing, little seems to be known about the social rate of return from a country's investments in highly specialized types of scientific training, particularly at the doctoral level. 19 REFERENCES Acs, Zoltan, D. Aubretsch, and M. Feldman. 1992. "Real Effects of Academic Research: Comment.' American Economic Review March. Adams, J. 1990. "Fundamental Stocks of Knowledge and Productivity Growth." Journal of Political Economy. Bartel, A. and F. Lichtenburg. 1986. "The Comparate Advantage of Educated Workers in Implementing New Technology." Review of Economics and Statistics. Bernstein, J. and M.I. Nadiri. 1988. "Interindustry R and D Spillovers, Rates of Return, and Production in High-Tech Industries." American Economic Review May. Bifani, Pablo. 1988. "Biotechnology: Overview and Developments in Latin America." In Economic and Social Progress in Latin America: 1988 Report. Washington, D.C.: Inter-American Develop-ment Bank. Chudnovsky, D. 1985. "The Entry into the Design and Production of Complex Capital Goods: The Experiences of Brazil, India, and South Korea." In M. Fransman, ed., Capital Goods in Economic Develop. London: Macmillan. Dahab, S. 1986. "Technological Change in the Brazilian Agricultural Implements Industry." Doctoral Dissertation, Yale University. Dahiman, Carl, and Claudio Frischtak. 1990. "National Systems Supporting Technical Advance in Industry: The Brazilian Experience." World Bank, Industry Series Paper No. 32. and F. Sercovich. 1984. "Exports of Technology from Semi-Industrial Economies and Local Technological Development." Journal of Development Economics September-October. Denison, Edward. 1962. The Sources of Economic Growth in the United States. New York: Committee for Economic Development. . 1979. Accounting for Slower Economic Growth. Washington, D.C.: Brookings Institution. Dorfman, Nancy. 1983. "Route 128: The Development of a Regional High-Technology Economy.' Research Policy. Evenson, Robert. 1990. "Intellectual Property Rights, R & D Inventions, Technology Purchase, and Piracy in Economic Development." In R. Evenson and G. Razis, eds., Science and Technology: Lessons for Development Policy. Boulder, Colorado: Westview Press. 1989. "Human Capital and Agricultural Productivity Change." In A. Maunde-r and R. Valdes, eds., Agriculture and Governments in an Independent World. Oxford: University of Oxford Press. Fransman, Marten. 1985. "Conceptualizing Technical Change in the Third World in the 1980s: An Interpretive Survey." Journal of' Development Studies July. Frischtak, Claudia. 1990. The Protection of Intellectual Property Rights and Industrial Technology Development in Brazil." In F. Rushing and C. Brown, eds., Intellectual Propertv Rights in Science, Technology and Economic Performance. Boulder, Colorado: Westview Press. Gibson, D. and R. Smilor, eds. 1992. Technology Transfer in Consortia and Strategic Alliance, Lankam, Md.: Rowman and Littlefield. Gilbert, Arthur. 1990. "Information Technology Transfer: The Singapore Strategy." In M. Chatterji, ed., Technology Transfer in the Developing Countries. London: Macmillan. 20 Goldhor, R. and R. Lund. 1983. 'University-to-lndustry Advanced Technology Transfer." Research Policy. Government-University-Industry-Research Roundtable and Industrial Research Institute. 1991. Industrial Perspectives on Innovation and Interactions With Universities. Washington, D.C.: National Academy Press. Gray, D., T. Gidley, and N. Koester. 1988. "Evaluation of the NSF Industry/University Cooperative Research Centers." North Carolina State University, December. Griliches, Zvi, ed. 1984. R and D. Patents, and Productivity. Chicago: University of Chicago Press. Jaffe, Adam. 1989. "Real Effects of Academic Research." American Economic Review December. Katz, Jorge, ed. 1987. Technology Generation in Latin American Manufacturing Industries New York: St. Martin's Press. Lall, S. Forthcoming. "Explaining Industrial Success in Developing Countries." In S. Lall and V.N. Balasubramanyamf, eds., Current Issues in Development Economics. London: Macmillan. Levin, S. and P. Stephan. 1991. "Research Productivity Over the Life Cycle: Evidence for Academic Scientists." American Economic Review March. Link, Albert and John Rees. 1990. "Firm Size, University Based Research, and Returns to R and D." Small Business Economics March. Mansfield, Edwin, et al. 1977. "Social and Private Rates of Return from Industrial Innovations." Juarterly Journal of Economics May. 1991a. "Academic Research and Industrial Innovation." Research Policy February. 1991b. "Estimates of the Social Returns from Research and Development." In M. Meredith, S. Nelson, and A. Teich, eds., Science and Technology Policy Yearbook. Washington, D.C.: American Association for the Advancement of Science. _=__ .1992a. "Academic Research and Industrial Innovation: A Further Note." Research Poliy. 1992b. "Flexible Manufacturing Systems: Economic Effects in Japan, United States, and Western Europe." Janan and the World Economy. _ 1993. "Academic Research Underlying Industrial Innovations: Sources and Characteristics." Paper presented at the American Economic Association, January. Forthcoming. "The Diffusion of Flexible Manufacturing Systems in Japan, Europe, and the United States." Management Science. . Forthcoming. "Unauthorized Use of Intellectual Property: Effects on Investment, Technology Transfer, and Innovation." National Research Council. Mikkelsen, K.W. 1984. 'Inventive Activity in Philippine Industry." Doctoral Dissertation. Mody, Ashoka. 1990. "New International Environment for Intellectual Property Rights." In F. Rushing and C. Brown, eds., Intellectual Property Rights in Science. Technology, and Economic Performance. Boulder: Westview Press. National Academy of Science. 1982. An Assessment of Research-Doctorate Programs in the United St. Washington, D.C.: National Academy Press. Nelson, Richard. 1986. "Institutions Supporting Technical Advance in Industry." American Economic Review May. 21 1990. "On Technological Capabilities and Their Acquisition." In R. Evenson and G. Ranis, eds., Science and Technology: Lessons for Development Policy. Boulder, Colorado: Westview Press. Otsuka, K., G. Ranis, and G. Saxonhoupe. 1988. Comparative Technology Choice: The India and Japanese Cotton Textile Industries. London: Macmillan, 1988. Pack, Howard. 1990. "Industrial Efficiency and Technology Choice." In R. Evenson and G. Ranis, eds., Science and Technology: Lessons for Developmet,, Policy. Boulder, Colorado: Westview Press. Peters, Lois, and Herbert Fusfeld. 1982. "Current U.S. University-Industry Research Connections." In University-Industry Research Relationships. Washington, D.C.: National Science Board. Pray, Carl and Vernon Ruttan. 1990. "Science and Technology Policy: Lessons from the Agricultural Sector in South and Southeast Asia." In R. Evenson and G. Ranis, eds., Science and Technology: Lessons for Development Policy. Boulder, Colorado: Westview Press. Ranis, 0. 1990. 'Science and Technology Policy: Lessons from Japan and the East Asians NICS." In R. Evenson and G. Ranis, eds., Science and Technology: Lessons for Development Policy. Boulder, Colorado: Westview Press. Rosenberg, Nathan. 1990. "Science and Technology Policy for the Asian NICS: Lessons from Economic History." In R. Ever.son and G. Ranis, eds., Science and Technology: Lessons for Development Policy. Boulder, Colorado: Westview Press. Sherwood, Robert. 1990. Intellectual Property and Economic Development. Boulder, Colorado: Westview Press. Siebeck, W., ed., with R. Evenson, W. Lesser, and C. Primo Braga. 1990. Stren2thenine Protection of Intellectual Property in Developing Countries. World Bank Discussion Paper 112. Washington, D.C. Soon, T.T. and T.C. Huat. 1990. "Role of Transnational Corporations in Transfer of Technology to Singapore." In M. Chatterji, ed., Technology Transfer in the Developing Countries. London: Macmillan. Stewart, Frances. 1990. "Technology Transfer for Development." In R. Evenson and G. Ranis, eds., Science and Technology: Lessons for Development Policy. Boulder, Colorado: Westview Press. Van Dierdonck, K. Debackers, and B. Engelen. 1990. "University-Industry Relationships: R. w Does the Belgian Community Feel About It?" Research Policy December:551-66. Westphal, Larry, L. Kim, and C. Dahlman. 1985. "Reflections on the Republic of Korea's Acquisition of Technological Capability." In N. Rosenberg and C. Frischtak, eds., International Technology Transfer. New York: Praeger. Westphal, Larry, Y. Rhee, and G. Purcell. 1981. "Korean Industrial Competence: Where It Came From." Staff Working Paper No. 469. World Bank. Wozniak, G.. 1987. "Human Capital, Information, and the Early Adoption of New Technology." Journal of Human Resources. Human Resources Development and Operations Policy Worldng Paper Series Contact for Title Author Date paper HROWP1 Social Development is Nancy Birdsall March 1993 L Malca Economic Development 37720 HROWP2 Factors Affecting Eduardo Velez April 1993 8. Washington- Achievement in Primary Ernesto Schiefelbein Diallo Education: A Review of the Jorge Valenzuela 30997 Literature for Latin America and the Caribbean HROWP3 Social Policy and Fertility Thomas W. Merrick May 1993 0. Nadora Transitions 35558 HROWP4 Poverty, Social Sector Norman L. Hicks May 1993 J. Abner Development and the Role of 38875 the World Bank HROWP5 Incorporating Nutrition into F. James Levinson June 1993 0. Nadora Bank-Assisted Social Funds 35558 HROWP6 Global Indicators of Rae Galloway June 1993 0. Nadora Nutritional Risk (II) 35558 HROWP7 Making Nutrition Donald A.P. Bundy July 1993 0. Nadora Improvements at Low Cost Joy Miller Del Rosso 35558 Throigih Parasite Control HROWP8 Municipal and Private Sector Donald R. Winkler August 1993 E. De Castro Response to Decentralization Taryn Rounds 89121 and School Choice: The Case of Chile, 1981-1990 HROWP9 Poverty and Structural lshrat Husain September 1993 M. Youssef Adjustment: The African Case 34614 HROWP10 Protecting Poor Jamaicans Margaret E. Grosh September 1993 M.E. Quintero from Currency Devaluation Judy L. Baker 37792 M. Rodriguez 30407 HROWP1 1 Operational Education George September 1993 L. Malca Indicators Psacharopoulos 37720 HROWP12 The Relationship Between the John Clark October 1993 P. Phillip State and the Voluntary 31779 Sector HROWP13 Obstacles to Women's Joseph Kutzin October 1993 0. Shoffner Access: Issues and Options 37023 for More Effective Interventions to Improve Women's Health HROWP14 Labor Markets and Market- Arvil V. Adams October 1993 S. Khan Oriented Reforms in Socialist 33651 Economies Human Resources Development and Operations Policy Working Paper Series Contact for Tide Author Date paper HROWP15 Reproductive Tract Infections, May T.H. Post October 1993 0. Shoffner HIV/AIDS and Women's 37023 Health HROWP16 Job Security and Labor Ricardo D. Paredes November 1993 S. Khan Market Adjustment in 33651 Developing Countries HROWP17 The Effects of Wage Luis A. Riveros November 1993 S. Khan Indexation on Adjustment, 33651 Inflation and Equity HROWP18 Popular Participation in Philip R. Gerson December 1993 L. Malca Economic Theory and Practice 37720 HROWP19 Economic Retums from Edwin Mansfield January 1994 I. Dione Investments in Research and 31447 Training