Journal of Information Processing Systems

  • 제14권4호
  • /
  • Pages.916-925
  • /
  • 2018
  • /
  • 1976-913X(pISSN)
  • /
  • 2092-805X(eISSN)
한국정보처리학회 (Korea Information Processing Society)
권호 표지
DOI QR코드

DOI QR Code

Analysis of Academic Evaluation Indicators Based on Citation Quality

  • Zhang, Mingyue (School of Information Management, Nanjing University) ;
  • Shi, Jin (School of Information Management, Nanjing University) ;
  • Wang, Jin (School of Computer & Communication Engineering, Changsha University of Science & Technology) ;
  • Liu, Chang (Dept. of Logistics and Information Management, Jianghan University)
  • 투고 : 2018.03.15
  • 심사 : 2018.05.20
  • 발행 : 2018.08.31
PDF 다운로드
⟨ 이전 논문 다음 논문 ⟩

초록

The academic research performance is often quantitatively measured by means of using citation frequency. The citation frequency-based indicators, such as h-index and impact factor, are commonly used reflecting the citation quality to some extent. However, these frequency-based indicators are usually carried out based on the assumption that all citations are equal. This may lead to biased evaluations in that, the attributes of the citing objects and cited objects are significant. A high-accuracy evaluation method is needed. In this paper, we review various citation quality-based evaluation indicators, and categorize them considering the algorithms being applied. We discuss the pros and cons of these indicators, and compare them from four dimensions. The outcomes will be useful for our further research on distinguishing citation quality.

키워드

참고문헌

  1. L. Bornmann and R. Haunschild, "Does evaluative scientometrics lose its main focus on scientific quality by the new orientation towards societal impact," Scientometrics, vol. 110, no. 2, pp. 937-943, 2017. https://doi.org/10.1007/s11192-016-2200-2
  2. D. Zhao and A. Strotmann, "Dimensions and uncertainties of author citation rankings: lessons learned from frequency-weighted in-text citation counting," Journal of the Association for Information Science and Technology, vol. 67, no. 3, pp. 671-682, 2016. https://doi.org/10.1002/asi.23418
  3. X. Wan and F. Liu, "Are all literature citations equally important? Automatic citation strength estimation and its applications," Journal of the Association for Information Science and Technology, vol. 65, no. 9, pp. 1929-1938, 2014. https://doi.org/10.1002/asi.23083
  4. Y. Ding, L. Liu, C. Guo, and B. Cronin, "The distribution of references across texts: some implications for citation analysis," Journal of Informetrics, vol. 7, no. 3, pp. 583-592, 2013. https://doi.org/10.1016/j.joi.2013.03.003
  5. Y. Ding, G. Zhang, T. Chambers, M. Song, X. Wang, and C. Zhai, "Content-based citation analysis: the next generation of citation analysis," Journal of the Association for Information Science and Technology, vol. 65, no. 9, pp. 1820-1833, 2014. https://doi.org/10.1002/asi.23256
  6. D. Walker, H. Xie, K. K. Yan, and S. Maslov, "Ranking scientific publications using a model of network traffic," Journal of Statistical Mechanics: Theory and Experiment, vol. 2007, no. 6, article no. P06010, 2007.
  7. K. Jarvelin and O. Persson, "The DCI index: discounted cumulated impact-based research evaluation," Journal of the American Society for Information Science and Technology, vol. 59, no. 9, pp. 1433-1440, 2008. https://doi.org/10.1002/asi.20847
  8. E. Yan and Y. Ding, "Weighted citation: an indicator of an article's prestige," Journal of the American Society for Information Science and Technology, vol. 61, no. 8, pp. 1635-1643, 2010. https://doi.org/10.1002/asi.21349
  9. C. Li, "Research on journal evaluation using the weighted impact factors considering cited quality: taking CSSCI source journals of LIS as an example," Journal of Academic Libraries, vol. 30, no. 1, pp. 29-34, 2012.
  10. M. Zheng and F. Liang, "Impact factor modification based on the journal quality of citations," Acta Editologica, vol. 27, no. 1, pp. 19-21, 2015.
  11. Z. Lin, "Application and discussion of weights of journal quality in the calculation formula of impact factor," Chinese Journal of Scientific and Technical Periodicals, vol. 26, no. 12, pp. 1295-1300, 2015.
  12. D. Fiala, L. Subelj, S. Zitnik, and M. Bajec, "Do PageRank-based author rankings outperform simple citation counts?," Journal of Informetrics, vol. 9, no. 2, pp. 334-348, 2015. https://doi.org/10.1016/j.joi.2015.02.008
  13. G. Pinski and F. Narin, "Citation influence for journal aggregates of scientific publications: theory, with application to the literature of physics," Information Processing & Management, vol. 12, no. 5, pp. 297-312, 1976. https://doi.org/10.1016/0306-4573(76)90048-0
  14. M. Dunaiski, W. Visser, and J. Geldenhuys, "Evaluating paper and author ranking algorithms using impact and contribution awards," Journal of Informetrics, vol. 10, no. 2, pp. 392-407, 2016. https://doi.org/10.1016/j.joi.2016.01.010
  15. J. D. West, T. C. Bergstrom, and C. T. Bergstrom, "The Eigenfactor Metrics: a network approach to assessing scholarly journals," College & Research Libraries, vol. 71, no. 3, pp. 236-244, 2010. https://doi.org/10.5860/0710236
  16. SCImago, (n.d.). SJR-SCImago Journal & Country Rank, 3 July 2018, from http://www.scimagojr.com.
  17. G. Herlach, "Can retrieval of information from citation indexes be simplified? Multiple mention of a reference as a characteristic of the link between cited and citing article," Journal of the American Society for Information Science, vol. 29, no. 6, pp. 308-310, 1978. https://doi.org/10.1002/asi.4630290608
  18. K. W. McCain and K. Turner, "Citation context analysis and aging patterns of journal articles in moleculargenetics," Scientometrics, vol. 17, no. 1-2, pp. 127-163, 1989. https://doi.org/10.1007/BF02017729
  19. B. Finney, "The reference characteristics of scientific texts," City University of London, Centre for Information Science, 1979.
  20. H. Nanba and M. Okumura, "Towards multi-paper summarization using reference information," Journal of Natural Language Processing, vol. 6, no. 82, pp. 79-86, 1999.
  21. B. C. Peritz, "A classification of citation roles for the social sciences and related fields," Scientometrics, vol. 5, no. 5, pp. 303-312, 1983. https://doi.org/10.1007/BF02147226
  22. P. Vinkler, "A quasi-quantitative citation model," Scientometrics, vol. 12, no. 1-2, pp. 47-72, 1987. https://doi.org/10.1007/BF02016689
  23. Q. Zhao, "Research on citation character and citation depth in literature," Journal of Information, vol. 29, no. 10, pp. 46-50, 2010.
  24. J. Ye, "Analyzing the essence of citations and their functions in academic assessment," Journal of Library Science in China, vol. 36, no. 1, pp. 35-39, 2010.
  25. S. Liu, "Research on the citation evaluation based on citation context nature," Information Studies: Theory & Application, vol. 38, no. 3, pp. 77-81, 2015.