Journal of the Korean Society for Library and Information Science (한국문헌정보학회지)

Korean Society For Library And Information Science (한국문헌정보학회)
권호 표지
DOI QR코드

DOI QR Code

A Study on the Effect of Altmetrics about Academic Papers on Citations and Moderating Effect of Open Access

학술논문 알트메트릭스의 피인용 영향과 오픈액세스의 조절효과에 관한 연구

  • 조재인 (인천대학교 문헌정보학과)
  • Received : 2022.04.18
  • Accepted : 2022.05.13
  • Published : 2022.05.31
Download PDF
⟨ Previous Next ⟩

Abstract

As altmetrics has received a lot of attention as an muti-dimensional impact assessment tool, it is necessary to verify whether it can supplement the citation-based research performance evaluation system. This study analyzed and compared the effects of each altmetrics sources on citation by sampling 1,600 high-cited papers published in the last 10 years (Sample A) and non-year-limited papers (Sample B) indexed in Scopus. In addition, it was analyzed whether the OA of the paper had a moderating effect on the numbers of cited-by, and the difference according to the samples was verified. As a result of the analysis, only the number of Mendeley bookmark readers was analyzed to have a positive (+) effect on the numbers of cited-by, and OA status had a significant moderating effect in both groups. However, in sample A, OA showed a reinforcing effect on cited-by, whereas Sample B showed a weakening effect, showing a difference. On the other hand, social mention such as media reports do not have a significant effect on the cited-by regardless of OA conditions, but they can be used to understand the social impact of non-academic mass readers.

연구의 다면적 영향력 평가 도구로 알트메트릭스가 등장하면서 피인용 기반의 연구 성과 평가 체계를 대체 또는 보완할 수 있을지 주목되고 있다. 본 연구는 최근 10년간 Scopus에 색인된 고피인용 논문(Sample A)과 연도를 제한하지 않은 고피인용 논문(Sample B)을 각 1,600건씩 샘플링해 알트메트릭스와 피인용이 어떠한 영향 관계에 있는지 비교 분석해 보았다. 또한 논문의 OA(Open Access) 여부가 피인용에 미치는 영향에 있어 조절효과(Moderating effect)를 수행하는지 분석하고 집단에 따른 차이를 확인하였다. 분석 결과, 두 집단 모두에서 Mendeley 북마크 독자수만이 피인용에 정(+)적 영향을 미치는 것으로 검증되었으며, 이러한 영향관계에서 OA 여부는 유의한 조절 효과를 하는 것으로 확인되었다. 그러나 최근 논문 집단인 Sample A에서는 강화효과가, Sample B에서는 약화효과가 나타나는 차이를 보였다. 한편 언론보도와 같은 사회적 언급은 OA 조건에 무관하게 대부분 피인용에 유의미한 정(+)의 영향력을 발휘하지 못하지만, 학계 밖 대중적 독자들을 대상으로 한 사회적 영향력을 이해하는데 활용될 수 있다.

Keywords

Acknowledgement

이 논문은 2020년 대한민국 교육부와 한국연구재단의 인문사회분야 중견연구자지원사업의 지원을 받아 수행된 연구임(NRF-2020S1A5A2A01040059).

References

  1. Park, J., Ko, Y., & Kim, H., (2019). A study on evaluation model for usability of research data service. Journal of the Korean Society for Information Management, 36(4), 129-159. https://doi.org/10.3743/KOSIM.2019.36.4.129
  2. Lee, H., Li, J., & Shin, S. (2017). Development tendency of altmetrics research: using social network analysis and co-word analysis. Journal of the Korea Institute of Information and Communication Engineering, 21(11), 2089-2094. https://doi.org/10.6109/JKIICE.2017.21.11.2089
  3. Cho, J. (2015). A study about scholarly impact measurement through Altmetrics. Journal of Korean Library and Information Science Society, 46(1), 65-81. https://doi.org/10.16981/kliss.46.1.201503.65
  4. Choi, H., Lee, Y., & Lee, H. (2019). Impact analysis of government subsidy fluctuations on basic research outputs: focused on the IT field. Journal of Social Science, 45(3), 321-344 https://doi.org/10.15820/khjss.2019.45.3.013
  5. Hur, W. (2013). How researchers estimate indirect effect using bootstrapping: the case of simple, multiple, and double mediation. Korea Business Review, 6(3), 43-59.
  6. Hong, S. & Jung, S. (2014). Testing the interaction effects in regression and structural equation models: theories and procedures. The Korean Journal of the Human Development, 21(4), 1-24. https://doi.org/10.15284/kjhd.2014.21.4.1
  7. Adie, E. (2014). Attention! A study of open access vs non-open access articles. figshare. Available: http://dx.doi.org/10.6084/m9.figshare.1213690.
  8. Alhoori, H., Ray Choudhury, S., Kanan, T., Fox, E., Furuta, R., & Giles, C. (2015). On the Relationship between Open Access and Altmetrics. iConference 2015 Proceedings. Available: http://hdl.handle.net/2142/73451
  9. Altmetric (2021). Altmetric database in CSV format. Unpublished raw data.
  10. Archambault, E., Cote, G., Struck, B., & Voorons, M. (2016). Research Impact of Paywalled Versus Open Access Papers. Available: http://digitalcommons.unl.edu/cgi/viewcontent.cgi?article=1028&context=scholcom
  11. Bornmann, L. & Haunschild, R. (2018). Do altmetrics correlate with the quality of papers? A large-scale empirical study based on F1000 Prime data. PLoS ONE, 13(5), e0197133. DOI: 10.1371/journal.pone.0197133
  12. Bornmann, L. (2014). Validity of altmetrics data for measuring societal impact: a study using data from Altmetric and F1000Prime, Journal of Informetrics, 8(4), 935-950. https://doi.org/10.1016/j.joi.2014.09.007
  13. Bornmann, L. (2015). Alternative metrics in scientometrics: a meta-analysis of research into three altmetrics. Scientometrics, 103, 1123-1144 Available: https://doi.org/10.1007/s11192-015-1565-y
  14. Cho, J. (2021). Altmetrics of highly cited research papers in social science. Serials Review, 47(1), 17-27. https://doi.org/10.1080/00987913.2021.1882652
  15. Clements, J. (2017) Open access articles receive more citations in hybrid marine ecology journals. FACETS 2, 1-14. https://doi.org/10.1139/facets-2016-0032
  16. Costas, R., Zahedi, Z., & Wouters, P. (2015). The thematic orientation of publications mentioned on social media Large-scale disciplinary comparison of social media metrics with citations. Aslib Journal of Information Management, 67(3), 260-288 https://doi.org/10.1108/AJIM-12-2014-0173
  17. Costas, R., Zahedi, Z., & Wouters, P. (2015) Do "altmetrics" correlate with citations? Extensive comparison of altmetric indicators with citations from a multidisciplinary perspective. asis&t, 66(10), 2003-2019. Available: https://doi.org/10.1002/asi.23309
  18. DORA (2012) The San Francisco Declaration on Research Assessment. Available: https://sfdora.org/read/
  19. Ebrahimy, S., Mehrad, J., Setareh, F., & Hosseinchari, M. (2016). Path analysis of the relationship between visibility and citation: the mediating roles of save, discussion, and recommendation metrics. Scientometrics, 109, 1497-1510. https://doi.org/10.1007/s11192-016-2130-z
  20. Gorraiz, J., Gumpenberger, C., & Schlogl, C. (2014). Usage versus citation behaviours in four subject areas. Scientometrics, 101, 1077-95. https://doi.org/10.1007/s11192-014-1271-1
  21. Hajjem, C., Harnad, S., & Gingras, Y. (2005). Ten-year cross-disciplinary comparison of the growth of open access and how it increases research citation impact. IEEE Data Engineering Bulletin, 28(4), 39-47.
  22. Haustein, S., Costas, R., & Lariviere, V. (2015). Characterizing social media metrics of scholarly papers: The effect of document properties and collaboration patterns. PLOS ONE, 10(3).
  23. Hayes, A. F. (2017). Introduction to Mediation, Moderation, and Conditional Process Analysis: A Regression-Based Approach (2nd ed.). New York: Guilford Press.
  24. Holmberg, K., Hedman, J., & Bowman, T. D. et al. (2020). Do articles in open access journals have more frequent altmetric activity than articles in subscription-based journals? An investigation of the research output of Finnish universities. Scientometrics, 122, 645-659. https://doi.org/10.1007/s11192-019-03301-x
  25. Kousha, K. & Abdoli, M. (2010). The citation impact of Open Access agricultural research: A comparison between OA and non-OA publications. Online Information Review, 34(5), 772-785. https://doi.org/10.1108/14684521011084618
  26. Lewis, C. (2018). The Open Access Citation Advantage: Does It Exist and What Does It Mean for Libraries?. Information Technology and Libraries, 37(3), 50-65. Available: https://doi.org/10.6017/ital.v37i3.10604
  27. Li, X. & Thelwall, M. (2012). F1000, Mendeley and traditional bibliometric indicators. Proceedings of the 17th International Conference on Science and Technology Indicators, Montreal: 541-551.
  28. Liu, X., Wei, Y., & Zhao, Z. (2020). How researchers view altmetrics: an investigation of ISSI participants, Aslib Journal of Information Management, 72(3), 361-378. https://doi.org/10.1108/ajim-07-2019-0165
  29. McGillivray, B. (2019). The relationship between usage and citations in an open access mega-journal, Scientometrics, Springer; Akademiai Kiado, 121(2), 817-838. https://doi.org/10.1007/s11192-019-03228-3
  30. Mohammadi, E. & Thelwall, M. (2014). Mendeley readership altmetrics for the social sciences and humanities: Research evaluation and knowledge flows. JASIST, 65(8), 1627-1638.
  31. Mohammadi, E., Thelwall, M., Haustein, S., & Lariviere, V. (2015). Who reads research articles? an altmetrics analysis of mendeley user categories. Journal of the American Society for Information Science, 66(9), 1832-1846.
  32. NISO (2016). Output of the NISO Alternative Assessment Metrics Project. Available: https://groups.niso.org/apps/group_public/download.php/17091/
  33. Ortega J. L. (2018). Disciplinary differences of the impact of altmetric. FEMS Microbiol Lett. 365(7):10.1093/femsle/fny049.
  34. Ouchi, A., Saberi, M. K., Ansari, N., Hashempour, L., & Isfandyari-Moghaddam, A. (2019). Do altmetrics correlate with citations? A study based on the 1,000 most-cited articles, Information Discovery and Delivery, 47(4), 192-202. Available: https://doi.org/10.1108/IDD-07-2019-0050
  35. Piwowar, H., Priem, J., Lariviere, V., Alperin, J. P., Matthias, L., Norlander, B., Farley, A., West, J., & Haustein, S. (2018). The state of oa: a large-scale analysis of the prevalence and impact of open access articles. PeerJ 6, e4375. https://doi.org/10.7717/peerj.4375
  36. Priem, J., Taraborelli, D., Groth, P., & Neylon, C. (2010). Altmetrics: a manifesto, Available: http://altmetrics.org/manifesto
  37. Science (2020). Will Trump White House tear down journal paywalls? Many anxiously await a decision. Available: https://www.sciencemag.org/news/2020/05/will-trump-white-house-tear-down-journal-paywalls-many-anxiously-await-decision
  38. Singh, V. K., Singh, P., & Karmakar, M. et al. (2021) The journal coverage of web of science, scopus and dimensions: a comparative analysis. Scientometrics, 126, 5113-5142 https://doi.org/10.1007/s11192-021-03948-5
  39. SPARC Europe (2015). The Open Access Citation Advantage Service (OACA) Available: https://sparceurope.org/what-we-do/open-access/sparc-europe-open-access-resources/open-access-citation-advantage-service-oaca/
  40. Subbaraman, N. (2019). Rumours fly about changes to US government open-access policy, Nature News, 20 DECEMBER 2019. Available: https://www.nature.com/articles/d41586-019-03926-1
  41. Sutton, S., Miles, R., & Konkiel, S. (2018) Awareness of altmetrics among LIS scholars and faculty. Journal of Education for Library and Information Science, 59(1-2), 33-47. https://doi.org/10.3138/jelis.59.1-2.05
  42. Tang, M., Bever, J., & Yu, F. (2017). Open access increases citations of papers in ecology. Ecosphere, 8(7), e01887. Available: https://esajournals.onlinelibrary.wiley.com/doi/full/10.1002/ecs2.1887
  43. Teplitskiy, M., Lu, G., & Duede, E. (2017). Amplifying the impact of open access: wikipedia and the diffusion of science. Journal of the Association for Information Science and Technology, 68, 2116-2127. https://doi.org/10.1002/asi.23687
  44. Thelwall, M. (2017) Are Mendeley reader counts useful impact indicators in all fields? Scientometrics, 113(1), 1721-1731. DOI: 10.1007/s11192-017-2557-x.
  45. Thelwall, M. & Nevill, T. (2018). Could scientists use Altmetric.com scores to predict longer term citation counts? Journal of Informetrics, 12(1), 237-248. DOI: 10.1016/j.joi.2018.01.008
  46. Thelwall, M. (2020). Mendeley reader counts for us computer science conference papers and journal articles. Quantitative Science Studies, 1(1), 347-359. https://doi.org/10.1162/qss_a_00010
  47. Thelwall, M. & Wilson, P. (2016). Mendeley readership altmetrics for medical articles: an analysis of 45 fields, JASIST, 67(8), 1962-1972 Available: https://asistdl.onlinelibrary.wiley.com/doi/10.1002/asi.23501
  48. Tint, T. H. H. & Na, J. (2017). Disciplinary differences in altmetrics for social sciences, Online Information Review, 41(2), 235-251. https://doi.org/10.1108/OIR-12-2015-0386
  49. Wang Z., Chen, Y., & Glanzel, W. (2020). Preprints as accelerator of scholarly communication: an empirical analysis in mathematics. Journal of Informetrics, 14(4), 101097. DOI: 10.1016/j.joi.2020.101097
  50. Wellcome (2020). Sharing research data and findings relevant to the novel coronavirus (nCoV) outbreak (2020.131) Available: https://wellcome.ac.uk/press-release/sharing-research-data-and-findings-relevant-novel-coronavirus-ncov-outbreak
  51. Winter, J. C. F. (2015). The relationship between tweets, citations, and article views for PLOS ONE articles. Scientometrics, 102, 1773-1779 https://doi.org/10.1007/s11192-014-1445-x
  52. Xu, L., Liu, J., & Fang, Q. (2011). Analysis on open access citation advantage: an empirical study based on Oxford open journals, iConference February 2011, 426-432. Available: https://doi.org/10.1145/1940761.1940819
  53. Zahedi, Z. & Haustein, S. (2018). On the relationships between bibliographic characteristics of scientific documents and citation and Mendeley readership counts: a large-scale analysis of Web of Science publications. Journal of Informetrics, 12(1), 191-202. https://doi.org/10.1016/j.joi.2017.12.005
  54. Zahedi, Z., Costas, R., & Wouters, P. (2014). How well developed are altmetrics? a cross-disciplinary analysis of the presence of 'alternative metrics' in scientific publications. Scientometrics, 101(2), 1491-1513. https://doi.org/10.1007/s11192-014-1264-0