A New Measure of Knowledge Diffusion
description
Transcript of A New Measure of Knowledge Diffusion
![Page 1: A New Measure of Knowledge Diffusion](https://reader036.fdocuments.us/reader036/viewer/2022081520/56815f6b550346895dce70e3/html5/thumbnails/1.jpg)
A New Measure of Knowledge Diffusion
Stephen Carley; Alan Porter
Georgia Tech
![Page 2: A New Measure of Knowledge Diffusion](https://reader036.fdocuments.us/reader036/viewer/2022081520/56815f6b550346895dce70e3/html5/thumbnails/2.jpg)
Our Interests
• Interest in research interdisciplinarity Possible key to more creative research Possible means to solve complex scholarly and
societal problems • Interest in research knowledge diffusion
Trans-disciplinary knowledge enrichment Translation of research to clinical practice Fostering science-based innovation
![Page 3: A New Measure of Knowledge Diffusion](https://reader036.fdocuments.us/reader036/viewer/2022081520/56815f6b550346895dce70e3/html5/thumbnails/3.jpg)
Measures• Have devised metrics:
Integration score [how diverse are the research knowledge resources upon which an article (or body of research) draws?]
Specialization score[how diverse are the venues in which a body of research (e.g., the works of one author in one time period) appears?]
• Now adding -- Diffusion score [how diverse are the fields that cite an article (or body of research)?]
![Page 4: A New Measure of Knowledge Diffusion](https://reader036.fdocuments.us/reader036/viewer/2022081520/56815f6b550346895dce70e3/html5/thumbnails/4.jpg)
Heuristics of diversity
(Stirling, 1998; 2007)(Rafols and Meyer,
2009)
Diversity:Attribute of a system whose elements may be apportioned into categories
Characteristics: Variety: Number of distinctive categoriesBalance: Evenness of the distribution Disparity: Degree to which the categories
are different.
Variety
Balance Disparity
Herfindahl (concentration): i pi2
[** Shannon & Herfindahldo not includeDisparity]
Shannon (Entropy): i pi ln pi
Dissimilarity: i di
Generalised Diversity (Stirling) ij(ij) (pipj)a (dij)b
![Page 5: A New Measure of Knowledge Diffusion](https://reader036.fdocuments.us/reader036/viewer/2022081520/56815f6b550346895dce70e3/html5/thumbnails/5.jpg)
Integration• Integration Defined:
equivalently:
• Integration Range: 0 → 1• Integration Benchmarking: ~0.42 for much of modern
scientific research (increasing moderately)
where i = row, j = column, f = frequency, SC = Cited Web of Science Subject Category;cosine similarity of SC cross-citation (based on one year of Web of Science journal cross-citation)
ji
jiij ppsI,
1
![Page 6: A New Measure of Knowledge Diffusion](https://reader036.fdocuments.us/reader036/viewer/2022081520/56815f6b550346895dce70e3/html5/thumbnails/6.jpg)
Changes in Average Integration Scores over Time
Porter, A.L., and Rafols, I. "Is science becoming more interdisciplinary? Measuring and mapping six research fields over time." Scientometrics. 81.3 (2009): 719-745.
![Page 7: A New Measure of Knowledge Diffusion](https://reader036.fdocuments.us/reader036/viewer/2022081520/56815f6b550346895dce70e3/html5/thumbnails/7.jpg)
Diffusion Scoring Example
Paper 1 Paper 2
Cited by: Biology Genetics Math Music Forestry
• In the example above Paper 2 will likely have a higher diffusion score than Paper 1 because there is greater heterogeneity among its citing subject categories [variety & disparity; not clear on balance].
![Page 8: A New Measure of Knowledge Diffusion](https://reader036.fdocuments.us/reader036/viewer/2022081520/56815f6b550346895dce70e3/html5/thumbnails/8.jpg)
Diffusion Scoring• Diffusion Defined:
where:pi = proportion of citing references corresponding to the SC i in a given paper sij = cosine measure of similarity between SCs i & j for cross-citation in WOS
• Diffusion Range: 0 → 1• Diffusion Benchmarking: ~0.42 for much of
modern scientific research [but somewhat higher for papers “out there” longer]
![Page 9: A New Measure of Knowledge Diffusion](https://reader036.fdocuments.us/reader036/viewer/2022081520/56815f6b550346895dce70e3/html5/thumbnails/9.jpg)
Mean Diffusion scores forall benchmark years combined
Modest increase over time since publication
![Page 10: A New Measure of Knowledge Diffusion](https://reader036.fdocuments.us/reader036/viewer/2022081520/56815f6b550346895dce70e3/html5/thumbnails/10.jpg)
Diffusion Score Behavior• Diffusion score behavior is more challenging to study than
Integration:• Calculating Diffusion score requires additional “citation
searching” in Web of Science to retrieve the citing papers• A paper’s Diffusion score is not fixed; over time it will
accrue more citations• Self-citation could affect Diffusion score
• We examined – for 5 of our 6 target Subject Categories -- minimal effect
• For Math – removing self-cites increased scores for 1995 publications by 0.01-.02, until recent years [Since Math 1995 annual Diffusion scores are ~0.22, this is ~5-10% increase]
![Page 11: A New Measure of Knowledge Diffusion](https://reader036.fdocuments.us/reader036/viewer/2022081520/56815f6b550346895dce70e3/html5/thumbnails/11.jpg)
Self-Cites (1)
![Page 12: A New Measure of Knowledge Diffusion](https://reader036.fdocuments.us/reader036/viewer/2022081520/56815f6b550346895dce70e3/html5/thumbnails/12.jpg)
Self-Cites (2)
![Page 13: A New Measure of Knowledge Diffusion](https://reader036.fdocuments.us/reader036/viewer/2022081520/56815f6b550346895dce70e3/html5/thumbnails/13.jpg)
Diffusion Scores: Total vs. Annual
• Annual Diffusion may be of interest• e.g., to pursue Chen’s hypothesis that broad +
rapid diffusion may signal really impactful papers• To characterize research knowledge diffusion
patterns• Calculation issue
• We use a threshold of at least 3 Citing SCs for a paper to get a Diffusion Score[could be 1 cite by a paper whose journal is associated with 3 SCs (rare)]
• For Total score calculation this is less a factor than for Annual scores
![Page 14: A New Measure of Knowledge Diffusion](https://reader036.fdocuments.us/reader036/viewer/2022081520/56815f6b550346895dce70e3/html5/thumbnails/14.jpg)
For most of the 1995 benchmarks, Diffusion scores increase steadily with time. Mathematics is an outlier.
Mean Annual Diffusion Scores for 6 Subject Categories
![Page 15: A New Measure of Knowledge Diffusion](https://reader036.fdocuments.us/reader036/viewer/2022081520/56815f6b550346895dce70e3/html5/thumbnails/15.jpg)
Mean NeuroScience Diffusion scores for all 4 benchmark years
![Page 16: A New Measure of Knowledge Diffusion](https://reader036.fdocuments.us/reader036/viewer/2022081520/56815f6b550346895dce70e3/html5/thumbnails/16.jpg)
Diffusion Patterns
• Can we categorize research knowledge diffusion patterns? A first try here:• Steady growth• Staying power• Late-bloomer• Robust growth
• Next 4 slides illustrate for those 1995 publications in the 6 SCs that fit these patterns
![Page 17: A New Measure of Knowledge Diffusion](https://reader036.fdocuments.us/reader036/viewer/2022081520/56815f6b550346895dce70e3/html5/thumbnails/17.jpg)
Steady Growth Typology
Data Source: 1995 benchmarks
![Page 18: A New Measure of Knowledge Diffusion](https://reader036.fdocuments.us/reader036/viewer/2022081520/56815f6b550346895dce70e3/html5/thumbnails/18.jpg)
Staying Power Typology
Data Source: 1995 benchmarks
![Page 19: A New Measure of Knowledge Diffusion](https://reader036.fdocuments.us/reader036/viewer/2022081520/56815f6b550346895dce70e3/html5/thumbnails/19.jpg)
Late-Bloomer Typology
Powerhouse articles (N’s are small) tend to have high Diffusion scores
Data Source: 1995 benchmarks
![Page 20: A New Measure of Knowledge Diffusion](https://reader036.fdocuments.us/reader036/viewer/2022081520/56815f6b550346895dce70e3/html5/thumbnails/20.jpg)
Robust Growth Typology
Data Source: 1995 benchmarks
![Page 21: A New Measure of Knowledge Diffusion](https://reader036.fdocuments.us/reader036/viewer/2022081520/56815f6b550346895dce70e3/html5/thumbnails/21.jpg)
• Papers are citing more references over time -- does this exert upward pressure on Diffusion scores?
• What happens to the speed of diffusion over time? - Will electronic open source publishing boost Diffusion?
• How differently does research published in various Subject Categories diffuse?
• Chen: papers which are cited quickly and diffusely are more likely to be blockbusters
• Tool for tracking individual researchers or research groups’ influence
Diffusion Issues
![Page 22: A New Measure of Knowledge Diffusion](https://reader036.fdocuments.us/reader036/viewer/2022081520/56815f6b550346895dce70e3/html5/thumbnails/22.jpg)
Taking a Look at the Research of Robert Nerem
• Institute Professor Emeritus and Director, Georgia Tech/Emory Center for Regenerative Medicine
• 240 Web of Science (WOS) publications, in• 76 Journals, over• 47 Years, and more than• 8,000 Citations
![Page 23: A New Measure of Knowledge Diffusion](https://reader036.fdocuments.us/reader036/viewer/2022081520/56815f6b550346895dce70e3/html5/thumbnails/23.jpg)
Diffusion Scores v. Publication Age for Dr. Nerem’s Research
Increasing Diffusion score from his early papers (high Age) to more recent ones
![Page 24: A New Measure of Knowledge Diffusion](https://reader036.fdocuments.us/reader036/viewer/2022081520/56815f6b550346895dce70e3/html5/thumbnails/24.jpg)
For more on Integration see:
www.idr.gatech.edu Porter, Alan, Alex Cohen, David Roessner and Marty
Perreault. "Measuring researcher interdisciplinarity." Scientometrics. 72.1 (2007): 117-147.
Porter, Alan, David Roessner, and Anne Heberger. "How interdisciplinary is a given body of research?" Research Evaluation. 17.4 (2008): 273-282.
Porter, Alan, and Ismael Rafols. "Is science becoming more interdisciplinary? Measuring and mapping six research fields over time." Scientometrics. 81.3 (2009): 719-745.
![Page 25: A New Measure of Knowledge Diffusion](https://reader036.fdocuments.us/reader036/viewer/2022081520/56815f6b550346895dce70e3/html5/thumbnails/25.jpg)
For more on Diffusion see: Chen, Chaomei, and Diana Hicks. "Tracing knowledge
diffusion." Scientometrics. 59.2 (2004): 199-211. Chen, C., Chen, Y., Horowitz, M., Hou, H., Liu, Z., &
Pellegrino, D. (2009). Towards an explanatory and computational theory of scientific discovery. Journal of Informetrics, 3(3), 191-209
Yu, G., Wang, M.Y., Yu, D.R. “Characterizing knowledge diffusion of Nanoscience & Nanotechnology by citation analysis.” Scientometrics, 84.1 (2010): 81-97.
Carley, S. and Porter, A.L. (2011) "A Forward Diversity Index." Scientometrics, forthcoming.
![Page 26: A New Measure of Knowledge Diffusion](https://reader036.fdocuments.us/reader036/viewer/2022081520/56815f6b550346895dce70e3/html5/thumbnails/26.jpg)
Thank you!
![Page 27: A New Measure of Knowledge Diffusion](https://reader036.fdocuments.us/reader036/viewer/2022081520/56815f6b550346895dce70e3/html5/thumbnails/27.jpg)
Mean Biotech Diffusion scores for all 4 benchmark years
![Page 28: A New Measure of Knowledge Diffusion](https://reader036.fdocuments.us/reader036/viewer/2022081520/56815f6b550346895dce70e3/html5/thumbnails/28.jpg)
Mean EE Diffusion scores for all 4 benchmark years
![Page 29: A New Measure of Knowledge Diffusion](https://reader036.fdocuments.us/reader036/viewer/2022081520/56815f6b550346895dce70e3/html5/thumbnails/29.jpg)
Mean Math Diffusion scores for all 4 benchmark years
![Page 30: A New Measure of Knowledge Diffusion](https://reader036.fdocuments.us/reader036/viewer/2022081520/56815f6b550346895dce70e3/html5/thumbnails/30.jpg)
Mean Med-R&E Diffusion scores for all 4 benchmark years
![Page 31: A New Measure of Knowledge Diffusion](https://reader036.fdocuments.us/reader036/viewer/2022081520/56815f6b550346895dce70e3/html5/thumbnails/31.jpg)
Mean Phy-AMC Diffusion scores for all 4 benchmark years