Chinese Traditional Medical Journal Journal Impact Factors ...€¦ · Classifying Entities into...

CTMJ | traditionalmedicinejournals.com Chinese Traditional Medicine Journal | 2019 | Vol 1 | Issue 2 1

Journal Impact Factors can be Predicted by Citations in Pubmed Central on the Topic of Chinese Traditional Medicine

Chien-Yu Huang1, Tung-Hui Jen1,2, Tsair-Wei Chien3, Chia-Chi Yen4*1Department of Chinese Medicine, Chi Mei medical center, Tainan ,Taiwan, 2Bachelor Program of Senior Services, Southern Taiwan Universityof Science and Technology, Tainan ,Taiwan, 3Medical Research Department, Chi-Mei

Foundation Hospital, Jiali, Tainan, Taiwan, 4Department of gynecology and obstetrics, Jiali, Tainan, Taiwan

INTRODUCTION

Many scholars are concerned about journal impact factors (JIF) announced by Clarivate Analytics each year in June. The reason is that the individual academic achievements(AAs) are usually evaluated by journal impact[i.e., the Science Citation Index (SCI) JIF] for reflecting the ability of researchers to convince strict reviewers[1].

Although the impact of the work of a scientist is frequently assessed by JIF, other author-level metrics, such as h-index[2], g/Ag-index[3], and x-index[4], have been proposed for evaluating individual AAs based on article citations in the literature. Those author-level metrics use the core articles for determining author AAs. For instance, h-index is the value at the equivalently equal number of

publications and the responding article citations ranked in descending order. All other articles or citations beyond the core limits are redundant and discarded when computing the metric. As such, when the Kano model[5] is applied to classify the AA features for each entity, all author-based h-indexes will locate on the one-dimension zone. That is the plot along a 45-degree line that runs from the left-bottom to the right-top.

If the x-index [7] ( ( )= ×max i ci ) is alternatively applied to the Kano model, the other two features of the citation-oriented and the production-oriented might be highlighted in the draw when the publications (i) are on the X-axis and citations (ci) on the Y-axis. As a result, the x-index is suitable for Kano model[5] in the assessment of entity characteristics.

In evaluation of single researchers, assessments should be incorporated by both paper and journal impacts in comparison with the AAs. The paper impact (PI) reflects the usefulness of papers for future research and the journal impact (JI) addresses the ability of researchers to convince strict reviewers[1]. However, none were seen using these

ABSTRACT

Numerous scientific scholars are concerned about journal impact factors (JIF) announced by Clarivate Analytics each year in June. Whether the Science Citation Index (SCI) JIFs can be predicted by citations in Pubmed Central(PMC) on the topic of Chinese Traditional Medicine(CTM) is required to verify. We thus extracted 2,417 articles indexed in PMC from 2013 to 2017 and download the respective citations to predict the SCI JIFs on the CTM topic. The impact factors based on PMC citations were calculated to predict the SCI IF in 2018. The xx-index was used for each cities or provinces in China to display the academic achievements(AAs) on dashboards. The Kano model was applied to classify the AA features toward the citation or the production. The correlation coefficient(CC) was used to verify the effect using PMC JIF to predict SCI JIF. Rasch model for continuous variables was applied to detect the most unexpected response using the criterion of Outfit mean square errors(MNSQ) greater than 2.0. We observed that the top three areas with highest x-index on the CTM topic are Beijing(=15.50), Taiwan(=13.12), and Shanghai(=10.59). The CC(=0.15) between PMCHIF and SCIJIF presents a small effect size (t=2.39, df=225, p<0.05). The most unexpected response appears on the journal of Nucleic Acids Res with Outfit MNSQ=59.9 due to the significant deviation between the SCIJIF=11.56 and the PMCJIF=73. Although the PMCJIFs can predict the SCI JIFs on the CTM topic, the Cohen’s effect size is substantially small(=0.15). The approach using PMC JIFs to predict the SCI JIFs is suggested to other disciplines for use in the foreseeable future.

Keywords: Journal impact factor, Pubmed Central, Clarivate Analytics, Chinese Traditional Medicine, Academic achievement

Chinese Traditional Medical Journal

*Correspondence to: Chia-Chi Yen, Chi-Mei Medical Center, 901 Chung Hwa Road, Yung Kung Dist., Tainan 710, Taiwan, Tel.: 88662812811. E-mail: [email protected]

Huang, et al.: Using PMC JIF to Predict SCI JIF in Journals


two PI and JI (called xx-index in this study) to jointly assess individuals and institutes in the past. We thus have the motivation to combine these two to evaluate the city-level AAs on the CTM topic in China.

The xx-index was composed by the squared product of both x-citation and x-SCI. The xx-index is defined by the formula(= x citation x SCI− × −( ) in which both x-citation a n d x - S C I a r e c o m p u t e d b y t h e x - i n d e x [ 7 ] ( ( )= ×max i ci ) where ci is the citations or the journal SCI impact factors, respectively, linked to the article.

In this study, we attempt to (1) presnt the highest AAs based on xx-index among cities/provinces in China, and (2) predict the SCIJIF by the Pubmed Central(PMC) on the CTM topic.

METHODS

Data Source

We obtained 2417 abstracts on journal article from PMC by searching the keywords of “Chinese Traditional medicine”[MeSH Major Topic] from 2013 to 2017. A total number of 4760 citing articles were matched to the citable papers in PMC. All these 1164 articles were cited by at least one publication in PMC. All data were downloaded from PMC, which means the study is not necessary for ethical approval according to the regulation issued by the Taiwan Ministry of Health and Welfare.

The Exponential AWS Used in This Study

We applied the exponential AWS [6,7] as Eq.(1). The sum of all authors in an article byline equals 1.

Wmm

m

mm m

m

m

m= =

=

−

=

−∑ ∑exp

exp

( )

( )

( . )

( . )

γ

γ

γ

γ0

1

0

1

2 27

2 27

(1)

Where the powers (γ m ) as the ordered author name (m) on the article from m-1 to 0, the author number is m-1, more importance is given to the first (=exp(m-1), primary) and the last (=exp(m-1) as the corresponding or supervisory authors. We assume that the others (the middle authors) made smaller contributions to their articles.

The Most Outstanding Cities/Provinces on the CTM Topic in China

A choropleth map was used to highlight the most influential cities/provinces with that authors were affiliated on the CTM topic.

The city/province-based citations were defined below:

Chij

m

h j iW c= ×=

−

∈∑0

1

( ) (2)

Where the author weight came from Eq. (1), the total weights on an article for a city/province(h) are determined by the

city/province-based author order(j) and the related weight on the article(i). The x-index ( ( )= ×max i ci )[7] was applied to represent the AAs for a city or province by (i) sorting the Chi In Eq. (2), and (ii) determine the number of publications at i and the responding ci. The most cited cities/provinces were dispersed with bubbles sized by the x-index using the Kano model[5] to display.

Once Kano model is applied (i.e., publications (i) on the X-axis and citations (ci) on the Y-axis), all those entities (e.g., journals with PMCJIF on the X-axis and SCIJIF on the Y-axis) are located within the one-dimension area along a 45-degree line that runs from the left-bottom to the right-top. Similarly, if two sequential values are not correlated, the journals, for instance, with a big deviation in SCIJIF and PMCJIF will be on either the attractive or the must-be zones defined by the Kano model.

The Pearson’s correlation coefficient(CC) was used to examine the effect using PMC JIF to predict SCI JIF. The effect sizes as small, medium, and lager are determined by the thresholds at 0.10, 0.30 and 0.50 according to Cohen’s definitions[10,11].

The Most Aberrant Journals Using the PMCJIF to Predict the SCIJIF Under Rasch Model

Due to the CC not totally presenting the actual entities happened to the diviations in momentum, Rasch model for continuous variables[8,9] was performed to detect the msot aberrant jouranls when the PMCJIF was used to predict the SCIJIF. The most unexpected responses are observed by the standardized residual Z-score in Eq. (3), where Oni denotes the observed score(i.e., the PMCJIF or the SCIJIF), Eni is the expected score yielded by the modelling, and VARni represents the variance on the response interacted by thejournal(n) and the items (i.e., the PMCJIF or the SCIJIF).

The Z-score for the case on an item=( )O EVarni ni

ni

−, (3)

The Outfit mean square errors (MNSQ) was applied to examine the aberrant pattern in data when the MNSQ ≥2.0. The Outfit MNSQ is defined by Eq. (4).

Outfit MNSQ= ( ) /i

k

iZ i=∑1

2 , (4)

Whereas Zi is the Z-score in each response(or the so-called cell), k is the number of items(=2 in this study). The online program[8] was applied to execute the detection using Rasch model for countiuous varialbes[9].

Creating Dashboards on Google Maps

The x-index and the respecitve citaion at k publication on x-core are yielded by author-made modules in MS-Excel. We created pages of Hyper Text Mark-up Language used for Google Maps. All relevant bibliometric indices can be linked to dashboards on Google Maps.



Classifying Entities into the Category Under the Kano Model

The Kano model [5] is a theory for product development and customer satisfaction developed in the 1980s by Professor Noriaki Kano, which classifies customer preferences into three main categories (named the “attractive,” the “one-dimensional,” and the “must-be” qualities). We applied author-made MS Excel to classify entities(i.e., journals or cities/provinces in this study) under the Kano approach to gain the coordinates for showing them on a dashboard. The confidence interval curves based on the Kano definition were plotted on Google Maps.

RESULTS

Task 1: The Most Outstanding Cities/Provinces on the CTM Topic

The top three areas with the highest xx-index on the CTM topic are Beijing(=15.50), Taiwan(=13.12), and Shanghai(=10.59), see Figure 1.

Due to x-index very hard to distinguish the characteristics for entities (i.e., the cities or provinces in China on the AAs) toward the productive or the attractive, the Kano model helps us to understand the features of the entities in nature, see Figure 2 and 3.

We can see that both Beijing and Taiwan locate in the productive zone and Shanghai is in the one-dimension area when x-citation indexes are applied. In contrast, both Beijing and Shanghai locate in the productive zone and Taiwan in

the one-dimension area when x-SCIJIF indexes are applied. Many other cities/provinces denoted by the green bubbles are located at the attractive zone, indicating their publications are less and the SCIJIFs are relatively huge using x-index to present the AAs. Interested readers are invited to scan the QR-code and clink on the bubble of interest to see the details about the metrics showing on Google Maps.

Task 2: The PMC Citations Used For Prediction SCI JIFs

The 227 journals indexed in PMC have SCI JIFs in comparison to those PMC JIFs. The top 20 journals with higher SCI JIFs are shown in Table 1. We can see that journals of Lancet, Lancet Oncol, and Nat Rev Clin Oncol are ranked at the top places.

The CC(=0.15) between PMCHIF and SCIJIF presents a small effect size (t=2.39, df=225, p<0.05). The most unexpected response appears on the journal of Nucleic Acids Res with Outfit MNSQ=59.9 due to the significant deviation between the SCIJIF=11.56 and the PMCJIF=73. Bubbles in Figure 4 are sized by the Outfit MNSQ and colored by the green, implying that the small effect size denoted by the CC is resulted from those journals with higher journal SCI impact factors.

DISCUSSION

We observed that the top three cities/provinces with highest x-index on the CTM topic are Beijing(=15.50), Taiwan(=13.12), and Shanghai(=10.59). The CC(=0.15) between PMCHIF and SCIJIF presents a small effect size (t=2.39, df=225, p<0.05). The most unexpected response

Figure 1: Academic achievements on the CTM topic using xx-index to display



appears on the journal of Nucleic Acids Res with Outfit MNSQ=59.9 due to the significant deviation between the SCIJIF=11.56 and the PMCJIF=73.

Several features are involved in this study:

First, Kano model has been sophisticatedly designed showing the entity characteristics toward the citation, the publication, or the neutral, which were rarely seen in bibliometric analyses before, particularly using Google Maps to display.

Second, the h-/g-index[2,3] have some drawbacks for use in practice. For instance, those are included as not being adaptive to different scientific disciplines [12], with gender effect [13], being influenced by age and career factors [14], and the assumption of equal credits for all coauthors in an article [15]. The AIF has the similar disadvantage [15-17] without solving the problem of farily quantifying coauthor contributions in an article [15,18]. In this study, we demonstrated the use of the exponential AWS[6,7] that can be applied to the city-level AAs, see Eq.(1). Ohterwise, the AAs either on author or city basis are problematic and unfair to evaluate the AAs in practice.

Third, a dashboard is a novel type of graphical user interface which often provides at-a-glance views of key items or entities relevant to a topic of interest. The “dashboard” is frequently displayed on a web page and linked to a database that allows the report to be constantly updated or ovserved on the user-freinedly basis. In this study, we applied dashboards showing on Google Maps that provide readers to easily understand the contents or information of interest to disclose on the website.

Another feature is the choropleth map[19-21] used in this study. Many examples of disparities in health outcomes across areas, such as dengue outbreaks [22,23], disease hotspots [24], and the Global Health Observatory maps on major health topics [25], have been presented using choropleth maps[19-21] to display. Our representation in Figure 1 on Google Maps is unique and innovative in the literature so far.

It is worth noting that the Rasch model for continuous variables is a tool for detecting the aberrant responses, see Figure 5. If no such a method used for highlighting the most unexpected observed values in data, we would not know the fact that journals with higher SCIJIFs are deviated from the PMCJIF most in the paired data.

Limitations and Suggestions

Despite the findings shown above, several potential limitations require further efforts in the future. First, the sample of this study only comprised articles on the CTM topic. It should not be generalized to other journals on the

Figure 2: Academic achievements on the CTM topic using x-citaion to display

Figure 3: Academic achievements on the CTM topic using x-SCI IF to display\

Figure 4: The most unexpected expectations on journals using PMCJIF to predict



issue of the effect using PMC JIF to predict the SCI JIF. As such, the most outstanding cities/provinces are barely determined by the publications in PMC regarding the CTM topic.

Second, there might be some biases when extracting author’s affiliation with cities/provinces from abstracts because the absence of the city names in databased might affect the result of this study.

Third, we recommend using Kano model to classify entities in characteristics toward different features. They are not limited to the Kano model because many other types of classifications are frequently used in academics. The style of the visual representations might be somewhat different, but the principle and the classification effect for classes in which we are interested are similar.

Fourth, although the exponential AWS was recommended to authors in this study, the assumption of the last as the corresponding author might be not supported always in practice or in the CTM field, other types of AWS can be considered in the future, such as the harmonic, the arithmetic, the geometric, the fractional, etc.[26].

Finally, numerous scientometrics were used for evaluating AAs. We applied x-index in Figure 2 and 3 to illustrate the dominant roles in the discipline. Authors are familiar with indices, such as h-index [2], g-/Ag-index [3], and author factor impact [16], which can be used to measure the AAs on other topics in the future.

CONCLUSSIONS

The PMCJIFs can predict the SCIJIFs on the CTM topic, but with slighter effect size. The approaches we used in this study applying PMC JIFs to predict the SCI IFs are suggested to be applied to other disciplines in the foreseeable future.

List of Abbreviations

AAs: academic achievementsAWS: author-weighted schemeIF: impact factorJCR: journal citation reportsJIF: journal impact factorSCI: science citation indexPMC: Pubmed Central.

DECLARATIONS

Ethics Approval and Consent to Participate

All the data used in this study were extracted from PMD and published article, which means the study does not require ethical approval according to the regulations promulgated by the Taiwan Ministry of Health and Welfare.

Consent to Publish

Not applicable.

Table 1: PMCJIF and SCIJIF for journals in descending order by SCIJIFJournal Citation Publication PMCJIF SCIJIFLancet 0 1 0 53.254Lancet Oncol 2 1 2 36.418Nat Rev Clin Oncol 16 1 16 24.653Gastroenterology 0 1 0 20.773J Hepatol 11 1 11 14.911Trends Pharmacol Sci 15 1 15 12.108Nucleic Acids Res 73 1 73 11.561Nat Prod Rep 16 2 8 11.406Sleep Med Rev 2 1 2 10.602J Thorac Oncol 4 1 4 10.336Curr Biol 17 2 8.5 9.251PLoS Biol 1 1 1 9.163Clin Infect Dis 2 1 2 9.117Am J Kidney Dis 26 4 6.5 7.129Am J Transplant 4 1 4 6.493Cancer Lett 61 2 30.5 6.491J Hazard Mater 0 1 0 6.434Brief Bioinform 10 1 10 6.302Stroke 15 1 15 6.239Allergy 16 1 16 6.048



Availability of Data and Materials

The datasets generated and analyzed during the current study are available on request to the authors.

COMPETING INTERESTS

The authors declare that they have no competing interests.

FUNDING

There are no sources of funding to be declared.

AUTHORS’ CONTRIBUTIONS

CY conceived and designed the study, TH, CY and TW interpreted the data, and CC monitored the process and the manuscript. CY drafted the manuscript. All authors read the manuscript and approved the final manuscript.

ACKNOWLEDGMENTS

We thank Enago (www.enago.tw) for the English language review of this manuscript.

REFERENCES

1. Bornmann L, Haunschild R.Plots for visualizing paper impact and journal impact of single researchers in a single graph.Scientometrics. 2018;115(1):385-394.

2. Hirsch JE. An index to quantify an individual’s scientific research output. Proc. Natl. Acad. Sci. U. S. A. 2005;102: 16569-165728.

3. Egghe L. Theory and practice of the g-index. Scientometrics 2006; 69:131-15.

4. Fenner T, Harris M, Levene M, Bar-Ilan J. A novel bibliometric index with a simple geometric interpretation. PLoS One. 2018;13(7):e0200098.

5. Kano N, Seraku N, Takahashi F, Tsuji S. Attractive Quality and Must-Be Quality. Journal of the Japanese Society for Quality Control 1984; 41: 39-48.

6. Chien, T.-W., Chow, J. C., Chang, Y., & Chou, W. (2018). Applying Gini coefficient to evaluate the author research domains associated with the ordering of author names: A bibliometric study. Medicine, 97(39), e12418-e12418. doi:10.1097/MD.0000000000012418.

7. Chien, T.-W., Wang, H.-Y., Chang, Y., & Kan, W.-C. (2018). Using Google Maps to display the pattern of coauthor collaborations on the topic of schizophrenia: A systematic review between 1937 and 2017. Schizophrenia Research.

doi: https://doi.org/10.1016/j.schres.2018.09.015.8. Chien TW. Rasch model for continuous item responses.

2019/7/14 retrieved at http://www.healthup.org.tw/kpiall/raschcontinous_st.asp.

9. Chien,T.-W, Shao,Y., Kuo,S.-C. Development of a Microsoft Excel tool for one-parameter Rasch model of continuous items: an application to a safety attitude survey. BMC Med Res Methodol. 2017;17(1):4. doi: 10.1186/s12874-016-0276-2.

10. Cohen J. Statistical Power Analysis for the Behavioral Sciences. Routledge, 1988. ISBN 978-1-134-74270-7.

11. Cohen J. A power primer”. Psychological Bulletin 1992; 112 (1): 155–159.

12. Kokko H, Sutherland WJ. What do impact factors tell us? Trends Ecol. Evol. 1999;14: 382–384.

13. Sax LJ, Hagedorn LS, Arredondo M, Dicrisi III FA. Faculty research productivity: exploring the role of gender and family-related factors. Res. Higher Educ. 2002;43: 423-336.

14. Kelly CD, Jennions MD. The h index and career assessment by numbers. Trends Ecol Evol. 2006;21(4):167-70.

15. Sekercioglu CH. Quantifying coauthor contributions. Science 2008; 322 (5900): 371.

16. Pan RK, Fortunato S. Author Impact Factor: tracking the dynamics of individual scientific impact. Sci Rep. 2014; 4:4880.

17. Petersen AM, Fortunato S, Pan RK3, Kaski K, Penner O, Rungi A, Riccaboni M, Stanley HE, Pammolli F. Reputation and impact in academic careers.Proc Natl Acad Sci U S A. 2014 Oct 28;111(43):15316-21.

18. Vavryčuk V. Fair ranking of researchers and research teams. PLoS One. 2018 Apr 5;13(4):e0195509.

19. Cromley RG, Cromley EK. Choropleth map legend design for visualizing community health disparities. Int J Health Geogr. 2009; 8:52.

20. Barcelona Field Studies Centre. Choropleth Map with Proportional Symbols. 2019/4/4 available at https://geographyfieldwork.com/DataPresentationMappingTechniques.htm.

21. Cromley RG, Ye Y. Ogive-based legends for choropleth mapping. Cartogr Geogr Inform Sci.2006;33:257-268. doi: 10.1559/152304006779500650.

22. Chen WJ. Dengue outbreaks and the geographic distribution of dengue vectors in Taiwan: A 20-year epidemiological analysis.Biomed J 2018;41(5):283-289.

23. Lai WT, Chen CH, Hung H, Chen RB, Shete S, Wu CC.Recognizing spatial and temporal clustering patterns of dengue outbreaks in Taiwan.BMC Infect Dis. 2018;18(1):256.

24. Soetens L, Hahné S,Wallinga J. Dot map cartograms for detection of infectious disease outbreaks: an application to Q fever, the Netherlands and pertussis, Germany.Euro Surveill 2017;22(26): pii: 30562.

25. WHO. Global Health Observatory Map Gallery. 2019/4/1 avaiable at https://www.who.int/gho/map_gallery/en.

26. Hagen NT.Harmonic publication and citation counting: sharing authorship credit equitably - not equally, geometrically or arithmetically.Scientometrics. 2010 Sep;84(3):785-793.

Chinese Traditional Medical Journal Journal Impact Factors ...€¦ · Classifying Entities into...

Documents

Transcript of Chinese Traditional Medical Journal Journal Impact Factors ...€¦ · Classifying Entities into...