Slide 1

1 Making a Grope for an Understanding of Taiwans Scientific Performance through the Use of Quantified Indicators Prof. Dr. Hsien-Chun Meng Science and Technology Information Center (STIC) National Science Council 2002/12/ Slide 2 2 Outline Research Background Research Goals Methodology Results & Discussion Future Works References Slide 3 3 Research Background Insufficient bibliometric indicators to evaluate scientific research performance in Taiwan Limited inquiry function of NSI and JCR Lack of international comparison of the performance with benchmarking countries STIC has aggressively involved in evaluating scientific activities Slide 4 4 Research questions Is Taiwan s research performance world class? Has the performance improved or deteriorated? What are the relatively strong and weak fields? Slide 5 5 Research Goals To assess the position of Taiwan s S&T research system in an international comparative perspective To identify relative strengths and weaknesses of scientific fields To serve as a reference for technology policy formulation and for the future allocation of innovation resources Slide 6 6 Methodology Methodology : Bibliometric Method Study Scope : Time interval : 1995~2001 Benchmarking Country Selection : 7 advanced countries 4 developing countries in Eastern Asia CWTS Classification system : 7 broad areas 43 fields Slide 7 7 Benchmarking Country Selection Advanced Countries United States of America Japan United Kingdom France Germany Netherlands Israel Developing Countries in Eastern Asia South Korea Peoples Republic of China Singapore Taiwan Slide 8 8 CWTS Classification system COMPUTER SCIENCES and MATHEMATICS MATHEMATICS COMPUTER SCIENCE STATISTICS ENGINEERING SCIENCES ELECTRICAL ENGINEERING MATERIALS SCIENCE CIVIL ENGINEERING MECHANICAL ENGINEERING INSTRUMENTS AND INSTRUMENTATION FUELS AND (NUCLEAR) ENERGY GEOLOGICAL ENGINEERING CHEMICAL ENGINEERING AEROSPACE ENGINEERING OTHER ENGINEERING SCIENCES NATURAL SCIENCES CHEMISTRY PHYSICS ASTRONOMY EARTH SCIENCES ENVIRONMENTAL SCIENCES MEDICAL SCIENCES AND LIFE SCIENCES BASIC LIFE SCIENCES BIOLOGY FOOD SCIENCES AND AGRICULTURE BIOMEDICAL SCIENCES CLINICAL MEDICINE HEALTH SCIENCES PHARMACOLOGY DENTISTRY Slide 9 9 CWTS Classification system SOCIAL AND BEHAVIORAL SCIENCES POLITICAL SCIENCE AND PUBLIC ADMINISTRATION INFORMATION AND COMMUNICATION SCIENCES PSYCHOLOGY SOCIOLOGY ANTHROPOLOGY EDUCATIONAL SCIENCES SOCIAL GEOGRAPHY AND DEMOGRAPHY PUBLIC HEALTH AND SOCIAL WELFARE OTHER SOCIAL AND BEHAVIORAL SCIENCE LAW, ARTS, AND THE HUMANITIES LAW LANGUAGE AND LINGUISTICS HISTORY AND ARCHEOLOGY PHILOSOPHY RELIGION CREATIVE ARTS, CULTURE, AND MUSIC MULTIDISCIPLINARY SCIENCE MULTIDISCIPLINARY SCIENCES (Contains a set of broad general journal including Nature, Science, and the Proceedinf of the National Academy of Sciences) Slide 10 10 Bibliometric Indicators (1) P: Publications An indicator of scientific productivity and specialization of a country Cx: Citations excluding self-cited a frequently cited paper has had a greater influence on subsequent research activities A measure of impact of research in the scientific community and recognition of scientific quality Slide 11 11 Bibliometric Indicators (2) CPPx: Average number of citations per publication (excluding self-cited) CPPx/FCSm: normalized citation impact FCSm: World citation average in the subfields CPPx/FCSm score within the range of 0.8 and 1.2 is considered world average; above 1.2 indicates citation impact levels significantly above world average. Slide 12 12 Data sources ISI database SCI (Science Citation Index) SSCI (Social Sciences Citation Index) AHCI (Arts & Humanities Citation Index) Specialty Citation Indexes: CompuMath , Biotechnology , Biochemistry , Chemistry , Materials Science , Neurosciences Slide 13 13 Result and Discussion Slide 14 14 200120001999199819971996 United States1 (27.9%)1 (28.0%)1 (28.6%)1 (29.3%)1 (30.5%)1 (31.3%) United Kingdom2 (7.8%)2 (8.2%)2 (8.1%) 2 (8.5%) Japan3 (7.7%) 3 (7.8%)3 (7.7%)3 (7.5%) Germany4 (7.1%) 4 (7.3%)4 (7.0%)4 (6.8%) France5 (5.1%)5 (5.2%)5 (5.3%)5 (5.4%)5 (5.3%) Peoples Rep. China8 (3.4%)8 (3.1%)9 (2.5%)12 (2.1%)12 (1.9%)14 (1.6%) Netherlands12 (2.1%) 11 (2.2%) 10 (2.2%) South Korea15 (1.6%)16 (1.4%)16 (1.3%)16 (1.2%)18 (1.0%)20 (0.9%) Taiwan ROC18 (1.1%)20 (1.0%) 19 (1.0%)19 (0.9%) Israel21 (1.0%)19 (1.0%)18 (1.0%)18 (1.1%)16 (1.1%)17 (1.1%) Singapore34 (0.4%) 36 (0.3%) 37 (0.3%)40 (0.2%) Overview of publication output Country ranking in worldwide publication output (96 ~01 ) Slide 15 15 Overview of publication output Growth rates in total publication output Slide 16 16 Research profiles of 4 Asian countries TAIWAN CHINASINGAPORE SOUTH KOREA Slide 17 17 Research profiles comparison (Asian 4 countries V.S. Advanced countries) Advanced CountriesEastern Asia(4 countries) Including Taiwan Peoples Republic of China South Korea Singapore Including United States of America Japan United Kingdom, France Germany Netherlands Israel Slide 18 18 Trend of research profiles in Taiwan Slide 19 19 Research profiles in Taiwan (43 fields) Slide 20 20 Research profiles in Taiwan (43 fields) Slide 21 21 Scientific impact of selected countries COMPUTER SCIENCES, MATHEMATICS AND NATURAL SCIENCE Slide 22 22 Scientific impact of selected countries ENGINEERING SCIENCE Slide 23 23 Scientific impact of selected countries MEDICAL AND LIFE SCIENCE Slide 24 24 Scientific impact of selected countries SOCIAL AND BEHAVIORAL SCIENCES Slide 25 25 Scientific impact of selected countries LAW, ARTS, HUMANITIES AND MULTIDISCIPLINARY SCIENCE Slide 26 26 Scientific impact comparison (Within Asian 4 countries) Field\(Rank) Impact FactorTaiwan South KoreaSingaporePRC ANTHROPOLOGY(1) 0.41*(3) 0 (2) 0.3 PHARMACOLOGY(1) 0.52(4) 0.33(2) 0.48(3) 0.36 EARTH SCIENCES(1) 0.55(2) 0.52(4) 0.47(3) 0.48 ELECTRICAL ENGINEERING(1) 0.61(2) 0.6 (4) 0.41 PHYSICS(1) 0.64(3) 0.59(2) 0.6(4) 0.43 CIVIL ENGINEERING(1) 0.71(2) 0.62 PSYCHOLOGY(1) 0.71(3) 0.59(4) 0.58(2) 0.63 STATISTICAL ANALYSIS AND PROBABILITY(1) 0.72(3) 0.65(2) 0.7(4) 0.44 LANGUAGE AND LINGUISTICS(1) 0.74(4) 0.33(3) 0.65(2) 0.69 *(Ranking) CPPx/FCSmx Slide 27 27 Scientific impact comparison (Within Asian 4 countries) Field\(Rank) Impact FactorTaiwan South KoreaSingaporePRC EDUCATIONAL SCIENCE(2) 0.4(3) 0.32(4) 0.31(1) 0.42 ASTRONOMY AND ASTROPHYSICS(2) 0.48(1) 0.7(4) 0(3) 0.35 BASIC LIFE SCIENCES(2) 0.54(3) 0.34(1) 0.99(4) 0.28 COMPUTER SCIENCE(2) 0.61(3) 0.61(1) 0.67(4) 0.59 MATHEMATICS(2) 0.62(3) 0.59(1) 0.77(4) 0.56 ENVIRONMENTAL SCIENCES(2) 0.64(3) 0.63(1) 0.72(4) 0.62 CHEMICAL ENGINEERING(2) 0.74(3) 0.65(1) 0.77(4) 0.65 MATERIALS SCIENCE(2) 0.74(1) 0.83(3) 0.68(4) 0.55 CHEMISTRY(2) 0.76(3) 0.58(1) 1.01(4) 0.43 LAW(2) 0.84(1) 1.02(3) 0.5(4) 0.38 *(Ranking) CPPx/FCSmx Slide 28 28 Scientific impact comparison (Within Asian 4 countries) Field\(Rank) Impact FactorTaiwanSouth KoreaSingaporePRC RELIGION(3) 0 (2) 0.22(1) 0.82 POLITICAL SCIENCE AND PUBLIC ADMINISTRATION(3) 0.36(2) 0.38(1) 0.66(4) 0.33 PHILOSOPHY(3) 0.4(4) 0.27(1) 0.98(2) 0.74 BIOLOGICAL SCIENCES(3) 0.48(2) 0.64(1) 0.72(4) 0.34 INSTRUMENTS AND INSTRUMENTATION(3) 0.48(2) 0.63(1) 0.65(4) 0.42 FUELS AND (NUCLEAR)ENERGY(3) 0.49(2) 0.56(1) 0.79(4) 0.37 AEROSPACE ENGINEERING(3) 0.52(2) 0.65(1) 0.78(4) 0.23 BIOMEDICAL SCIENCES(3) 0.53(1) 0.59(2) 0.54(4) 0.53 CLINICAL MEDICINE(3) 0.55(2) 0.67(1) 0.7(4) 0.39 PUBLIC HEALTH AND SOCIAL WELFARE(3) 0.57(2) 0.81(1) 0.86(4) 0.43 OTHER ENGINEERING SCIENCES(3) 0.59(1) 0.77(2) 0.6(4) 0.47 FOOD SCIENCE AND AGRICULTURE(3) 0.62(4) 0.59(1) 0.95(2) 0.64 MULTIDISCIPLINARY SCIENCES(3) 0.62(2) 0.74(1) 0.96(4) 0.05 CREATIVE ARTS, CULTURE AND MUSIC(3) 0.63(4) 0.41(1) 1.05(2) 0.79 MECHANICAL ENGINEERING(3) 0.68(1) 0.69 (4) 0.54 DENTISTRY(3) 0.71(1) 0.96(4) 0.53(2) 0.8 OTHER SOCIAL AND BEHAVIORAL SCIENCES(3) 0.74(4) 0.47(2) 0.82(1) 0.85 *(Ranking) CPPx/FCSmx Slide 29 29 Scientific impact comparison (Within Asian 4 countries) Field\(Rank) Impact FactorTaiwanSouth KoreaSingaporePRC HISTORY AND ARCHAEOLOGY(4) 0(2) 0.73(1) 1.14(3) 0.61 GEOLOGICAL ENGINEERING(4) 0.3(2) 0.43(1) 0.47(3) 0.4 SOCIAL GEOGRAPHY AND DEMOGRAPHY(4) 0.4(3) 0.66(1) 1.15(2) 0.83 HEALTH SCIENCES(4) 0.53(3) 0.57(1) 0.95(2) 0.61 ECONOMICS, BUSINESS AND MANAGEMENT(4) 0.54(1) 0.86(3) 0.71(2) 0.83 SOCIOLOGY(4) 0.55(2) 0.78(1) 1.18(3) 0.61 INFORMATION AND COMMUNICATION SCIENCES(4) 0.66(2) 0.91(1) 1.2(3) 0.73 *(Ranking) CPPx/FCSmx Slide 30 30 Future Works (1) Micro level bibliometric analyses of Taiwanese science Bibliometric data of 279 subfields in the period of 1995-2001 Other bibliometric indicators are included: CPPx/JCSm: compare the impact factor with the worldwide average in the same journal sets JCSm/FCSm: to evaluate whether the country publishes in top journals Breakdown of output (P) and impact (CPPx/FCSm) into international collaboration and national work Slide 31 31 Example 1 Field (Impact Score= CPPx/FCSmx) Scientific Research Profile Output And Impact in Physics 1991 - 2000 CPPx The average numbers of citations per publication (excluding self-citations) FCSmx The average citation rate of all papers in the fields in which the group is active. Impact Score (CPPx/FCSmx) within 0.8 and 1.2 is considered world average; scores above 1.2 indicates citation impact levels significantly above world average. Impact: Low Average High Slide 32 32 Example 2 Field (Impact Score= JCSm/FCSm) Scientific Research Profile Output And Impact in Physics 1991 - 2000 JCSmx The average citation rate of all papers published in the ISI-covered journals in which a group has published. FCSmx The average citation rate of all papers in the fields in which the group is active. Impact Score (JCSmx/FCSmx) within 0.8 and 1.2 is considered world average; scores above 1.2 indicates citation impact levels significantly above world average. Impact: Low Average High Slide 33 33 Future Works (2) The research performance of Taiwan s universities/institutions 55 universities and 20 institutions are included Breakdown of output and impact into scientific fields International collaboration will be analyzed Slide 34 34 Example 1 Field (Impact Score= CPPx/FCSmx) CPPx The average numbers of citations per publication (excluding self-citations) FCSmx The average citation rate of all papers in the fields in which the group is active. Impact Score (CPPx/FCSmx) within 0.8 and 1.2 is considered world average; scores above 1.2 indicates citation impact levels significantly above world average. Impact: Low Average High Institutional Research Profile Output And Impact Per Institute 1991 - 2000 Taiwan's most active Physics institutes Slide 35 35 Example 2 Impact: Low Average High Field (Impact Score= JCSm/FCSm) Institutional Research Profile Output And Impact Per Institute 1991 - 2000 Taiwan's most active Physics institutes JCSmx The average citation rate of all papers published in the ISI-covered journals in which a group has published. FCSmx The average citation rate of all papers in the fields in which the group is active. Impact Score (JCSmx/FCSmx) within 0.8 and 1.2 is considered world average; scores above 1.2 indicates citation impact levels significantly above world average. Slide 36 36 Collaboration analysis Cooperation Profile:Output And Impact Per Type 1991- 2000 Taiwan s 5 largest Physics institutes CPPx The average numbers of citations per publication (excluding self-citations) FCSmx The average citation rate of all papers in the fields in which the group is active. Impact Score (CPPx/FCSmx) within 0.8 and 1.2 is considered world average; scores above 1.2 indicates citation impact levels significantly above world average. Impact: Low Average High Field (Impact Score= CPPx/FCSm) Slide 37 37 Impact: Low Average High Field (Impact Score= JCSm/FCSm) Cooperation Profile:Output And Impact Per Type 1991- 2000 Taiwan s 5 largest Physics institutes JCSmx The average citation rate of all papers published in the ISI-covered journals in which a group has published. FCSmx The average citation rate of all papers in the fields in which the group is active. Impact Score (JCSmx/FCSmx) within 0.8 and 1.2 is considered world average; scores above 1.2 indicates citation impact levels significantly above world average. Collaboration analysis Slide 38 38 Future Works (3) The links between scientific knowledge and patents To evaluate the contribution of science to technological development in Taiwan academia- industry linkages To identify knowledge flows in the domestic S&T system (e.g., public-private R&D networks; role of patent citations) To develop a model to describe the relationships (e.g., links, flows, users, providers, feedbacks) Slide 39 39 Reference New bibliometric tools for the assessment of national research performance: Database description, overview of indicators and first applications H. F. Moed, R. E. De Bruin, T. N. van Leeuwen, Scientometrics, 5(1995) 381-422 Mapping the scientific performance of German medical research R. J. W. Tijssen, T. N. van Leeuwen, A. F. J. van Raan, 2002 Bibliometric mapping as a science policy and research management tool