Browse > Article
http://dx.doi.org/10.1633/JISTaP.2022.10.4.5

Meta-Analysis of Associations Between Classic Metric and Altmetric Indicators of Selected LIS Articles  

Vysakh, C. (Department of Studies and Research in Library and Information Science, Tumkur University)
Babu, H. Rajendra (Department of Studies and Research in Library and Information Science, Tumkur University)
Publication Information
Journal of Information Science Theory and Practice / v.10, no.4, 2022 , pp. 53-65 More about this Journal
Abstract
Altmetrics or alternative metrics gauge the digital attention received by scientific outputs from the web, which is treated as a supplement to traditional citation metrics. In this study, we performed a meta-analysis of correlations between classic citation metrics and altmetrics indicators of library and information science (LIS) articles. We followed the systematic review method to select the articles and Erasmus Rotterdam Institute of Management Guidelines for reporting the meta-analysis results. To select the articles, keyword searches were conducted on Google Scholar, Scopus, and ResearchGate during the last week of November 2021. Eleven articles were assessed, and eight were subjected to meta-analysis following the inclusion and exclusion criteria. The findings reported negative and positive associations between citations and altmetric indicators among the selected articles, with varying correlation coefficient values from -.189 to 0.93. The result of the meta-analysis reported a pooled correlation coefficient of 0.47 (95% confidence interval, 0.339 to 0.586) for the articles. Sub-group analysis based on the citation source revealed that articles indexed on the Web of Science showed a higher pooled correlation coefficient (0.41) than articles indexed in Google Scholar (0.30). The study concluded that the pooled correlation between citation metrics with altmetric indicators was positive, ranging from low to moderate. The result of the study gives more insights to the scientometrics community to propose and use altmetric indicators as a proxy for traditional citation indicators for quick research impact evaluation of LIS articles.
Keywords
traditional metrics; citation metrics; altmetrics; citations; altmetric indicators; pooled correlation;
Citations & Related Records
Times Cited By KSCI : 1  (Citation Analysis)
연도 인용수 순위
1 Barnes, C. (2015). The use of altmetrics as a tool for measuring research impact. Australian Academic & Research Libraries, 46(2), 121-134. https://doi.org/10.1080/00048623.2014.1003174.   DOI
2 Bornmann, L. (2015). Alternative metrics in scientometrics: A meta-analysis of research into three altmetrics. Scientometrics, 103(3), 1123-1144. https://doi.org/10.1007/s11192-015-1565-y.   DOI
3 Costas, R., Zahedi, Z., & Wouters, P. (2015). Do "altmetrics" correlate with citations? Extensive comparison of altmetric indicators with citations from a multidisciplinary perspective. Journal of the Association for Information Science and Technology, 66(10), 2003-2019. https://doi.org/10.1002/asi.23309.   DOI
4 DerSimonian, R., & Laird, N. (1986). Meta-analysis in clinical trials. Controlled Clinical Trials, 7(3), 177-188. https://doi.org/10.1016/0197-2456(86)90046-2.   DOI
5 Ezema, I. J., & Ugwu, C. I. (2019). Correlating research impact of library and information science journals using citation counts and altmetrics attention. Information Discovery and Delivery, 47(3), 143-153. https://doi.org/10.1108/IDD-08-2018-0029.   DOI
6 Hak, T., Van Rhee, H. J., & Suurmond, R. (2022). How to interpret results of meta-analysis. (Version 1.0). https://www.erim.eur.nl/research-support/meta-essentials/interpret-results.
7 Haustein, S., Bowman, T. D., Holmberg, K., Peters, I., & Lariviere, V. (2014a). Astrophysicists on Twitter: An indepth analysis of tweeting and scientific publication behavior. Aslib Journal of Information Management, 66(3), 279-296. https://doi.org/10.1108/AJIM-09-2013-0081.   DOI
8 Kwok, R. (2013). Research impact: Altmetrics make their mark. Nature, 500(7463), 491-493. https://doi.org/10.1038/nj7463-491a.   DOI
9 Htoo, T. H. H., & Na, J. C. (2017). Disciplinary differences in altmetrics for social sciences. Online Information Review, 41(2), 235-251. https://doi.org/10.1108/OIR-12-2015-0386.   DOI
10 Kolahi, J., Khazaei, S., Iranmanesh, P., Kim, J., Bang, H., & Khademi, A. (2021). Meta-analysis of correlations between altmetric attention score and citations in health sciences. BioMed Research International, 2021, 6680764. https://doi.org/10.1155/2021/6680764.   DOI
11 Meta-Mar. (2021). Meta-analysis: Correlation model. https://meta-mar.shinyapps.io/meta-analysis-calculator/.
12 Mohammadi, E., & Thelwall, M. (2014). Mendeley readership altmetrics for the social sciences and humanities: Research evaluation and knowledge flows. Journal of the Association for Information Science and Technology, 65(8), 1627-1638. https://doi.org/10.1002/asi.23071.   DOI
13 Riley, R. D., Higgins, J. P., & Deeks, J. J. (2011). Interpretation of random effects meta-analyses. BMJ, 342, d549. https://doi.org/10.1136/bmj.d549.   DOI
14 Saberi, M. K., & Ekhtiyari, F. (2019). Usage, captures, mentions, social media and citations of LIS highly cited papers: An altmetrics study. Performance Measurement and Metrics, 20(1), 37-47. https://doi.org/10.1108/PMM-10-2018-0025.   DOI
15 Schlogl, C., Gorraiz, J., Gumpenberger, C., Jack, K., & Kraker, P. (2014). Comparison of downloads, citations and readership data for two information systems journals. Scientometrics, 101(2), 1113-1128. https://doi.org/10.1007/s11192-014-1365-9.   DOI
16 Simmonds, M. (2015). Quantifying the risk of error when interpreting funnel plots. Systematic Reviews, 4, 24. https://doi.org/10.1186/s13643-015-0004-8.   DOI
17 Cohen, J. (1988). Statistical power analysis for the behavioral sciences. 2nd ed. Routledge.
18 Chi, P. S., Gorraiz, J., & Glanzel, W. (2019). Comparing capture, usage and citation indicators: an altmetric analysis of journal papers in chemistry disciplines. Scientometrics, 120(3), 1461-1473. https://doi.org/10.1007/s11192-019-03168-y.   DOI
19 Rangaswamy, B., & Rajendra Babu, H. (2021). A correlation comparative analysis of top articles in library and information science using citation and altmetric attention scores. SRELS Journal of Information Management, 58(3), 157-164. https://doi.org/10.17821/srels/2021/v58i3/158132.   DOI
20 Ryan, R. (2016). Cochrane consumers and communication group reviews: Meta-analysis. http://cccrg.cochrane.org/sites/cccrg.cochrane.org/files/public/uploads/meta-analysis_revised_december_1st_1_2016.pdf.
21 Statistics How To. (2022). What is Fisher Z-Transformation? https://www.statisticshowto.com/fisher-z.
22 Tahamtan, I., & Bornmann, L. (2020). Altmetrics and societal impact measurements: Match or mismatch? A literature review. Profesional De La informacion, 29(1). https://doi.org/10.3145/epi.2020.ene.02.   DOI
23 Thelwall, M. (2021). Measuring societal impacts of research with altmetrics? Common problems and mistakes. Journal of Economic Surveys, 35(5), 1302-1314. https://doi.org/10.1111/joes.12381.   DOI
24 Verma, S., & Madhusudhan, M. (2019a). Altmetric analysis of highly cited publications on digital library in Brazil and India: A comparative study. Library Philosophy and Practice, 2273. https://digitalcommons.unl.edu/libphilprac/2273.
25 Higgins, J. P. T., Thompson, S. G., Deeks, J. J., & Altman, D. G. (2003). Measuring inconsistency in meta-analyses. BMJ, 327(7414), 557-560. https://doi.org/10.1136/bmj.327.7414.557.   DOI
26 Haustein, S., Peters, I., Bar-Ilan, J., Priem, J., Shema, H., & Terliesner, J. (2014b). Coverage and adoption of altmetrics sources in the bibliometric community. Scientometrics, 101(2), 1145-1163. https://doi.org/10.1007/s11192-013-1221-3.   DOI
27 Verma, S., & Madhusudhan, M. (2019b). An altmetric comparison of highly cited digital library publications of India and China. Annals of Library and Information Studies, 66(2), 71-75. https://core.ac.uk/download/pdf/229208328.pdf.
28 Zhang, X., Wang, X., Zhao, H., de Pablos, P. O., Sun, Y., & Xiong, H. (2019). An effectiveness analysis of altmetrics indices for different levels of artificial intelligence publications. Scientometrics, 119(3), 1311-1344. https://doi.org/10.1007/s11192-019-03088-x.   DOI
29 Zhao, Y., & Wolfram, D. (2015). Assessing the popularity of the top-tier journals in the LIS field on Twitter. Proceedings of the Association for Information Science and Technology, 52(1), 1-4. https://doi.org/10.1002/pra2.2015.145052010092.   DOI
30 Sensuse, D. I., Lestari, P. I., & Hakim, S. A. (2021). Exploring factors influencing knowledge sharing mechanisms and technology to support the collaboration ecosystem: A review. DESIDOC Journal of Library & Information Technology, 41(3), 226-234. https://doi.org/10.14429/djlit.41.03.16609.   DOI
31 Ahmad, Z. (2019). Profiles of AMU and JNU researchers on Researchgate: An altmetric analysis. Kelpro Bulletin, 23(1), 101-112. https://www.academia.edu/42459122/Profiles_of_AMU_and_JNU_Researchers_on_Researchgate_An_Altmetric_Analysis.
32 Ali, M. J. (2021). Understanding the altmetrics. Seminars in Ophthalmology, 36(5-6), 351-353. https://doi.org/10.1080/08820538.2021.1930806.   DOI
33 Cho, J. (2021). Altmetrics analysis of highly cited academic papers in the field of library and information science. Scientometrics, 126(9), 7623-7635. https://doi.org/10.1007/s11192-021-04084-w.   DOI
34 Shrivastava, R., & Mahajan, P. (2015). Relationship amongst ResearchGate altmetric indicators and Scopus bibliometric indicators: The case of Panjab University Chandigarh (India). New Library World, 116(9/10), 564-577. https://doi.org/10.1108/NLW-03-2015-0017.   DOI
35 Sugimoto, C. R., Work, S., Lariviere, V., & Haustein, S. (2017). Scholarly use of social media and altmetrics: A review of the literature. Journal of the Association for Information Science and Technology, 68(9), 2037-2062. https://doi.org/10.1002/asi.23833.   DOI
36 Thelwall, M., Haustein, S., Lariviere, V., & Sugimoto, C. R. (2013). Do altmetrics work? Twitter and ten other social web services. PLoS One, 8(5), e64841. https://doi.org/10.1371/journal.pone.0064841.   DOI
37 Ali, M. Y., & Richardson, J. (2017). Pakistani LIS scholars' altmetrics in ResearchGate. Program, 51(2), 152-169. https://doi.org/10.1108/PROG-07-2016-0052.   DOI
38 Borah, R., & Madhusudhan, M. (2022). A literature review of research articles on altmetrics. In K. S. Shivraj, A. A. Suleiman, & P. Gupta (Eds.), Knowledge management in higher education institutions (pp. 174-181). Manipal University Jaipur.
39 Daraio, C. (2021). Altmetrics as an answer to the need for democratization of research and its evaluation. Journal of Altmetrics, 4(1), 5. https://doi.org/10.29024/joa.43.   DOI
40 Gumpenberger, C., Glanzel, W., & Gorraiz, J. (2016). The ecstasy and the agony of the altmetric score. Scientometrics, 108(2), 977-982. https://doi.org/10.1007/s11192-016-1991-5.   DOI
41 Israel, H., & Richter, R. R. (2011). A guide to understanding meta-analysis. The Journal of orthopaedic and sports physical therapy, 41(7), 496-504. https://doi.org/10.2519/jospt.2011.3333.   DOI
42 Mantel, N., & Haenszel, W. (1959). Statistical aspects of the analysis of data from retrospective studies of disease. Journal of the National Cancer Institute, 22(4), 719-748. https://doi.org/10.1093/jnci/22.4.719.   DOI
43 Patthi, B., Prasad, M., Gupta, R., Singla, A., Kumar, J. K., Dhama, K., Ali, I., & Niraj, L. K. (2017). Altmetrics - A collated adjunct beyond citations for scholarly impact: A systematic review. Journal of Clinical and Diagnostic Research, 11(6), ZE16-ZE20. https://doi.org/10.7860/JCDR/2017/26153.10078.   DOI
44 Zuccala, A. A., Verleysen, F. T., Cornacchia, R., & Engels, T. C. E. (2015). Altmetrics for the humanities: Comparing Goodreads reader ratings with citations to history books. Aslib Journal of Information Management, 67(3), 320-336. https://doi.org/10.1108/AJIM-11-2014-0152.   DOI
45 Araujo, A. C., Vanin, A. A., Nascimento, D. P., Gonzalez, G. Z., & Costa, L. O. P. (2021). What are the variables associated with Altmetric scores? Systematic reviews, 10(1), 193. https://doi.org/10.1186/s13643-021-01735-0.   DOI
46 Vysakh, C., & Rajendra Babu, H. (2021). Altmetrics linked scholarly information from Dimensions.ai datasets for the top 100 LIS articles. SRELS Journal of Information Management, 58(3), 197-208. https://doi.org/10.17821/srels/2021/v58i3/158563.   DOI