Browse > Article
http://dx.doi.org/10.1633/JISTaP.2021.9.3.5

On a New Index for Research Assessment  

Farid, Farid O. (Ronin Institute for Independent Scholarship, Montclair)
Publication Information
Journal of Information Science Theory and Practice / v.9, no.3, 2021 , pp. 56-75 More about this Journal
Abstract
We introduce a new research assessment measure, called the research excellence index. The measure, which we denote by RE-index, accurately assesses the research performance of a researcher. The methodology used in deriving the RE-index tackles many of the flaws of popular research performance indicators such as publication counts, citation counts, and the h and g indices. A dataset is introduced, which takes advantage of the wide coverage of Scopus and the Library of Congress, and, at the same time, deals with the Scopus database depth problem. For an academic publication x, a prestige-type and length scores are assigned, and if x is published in an academic periodical publication J, the stature of J is identified through a quartile score. The three scores are used to assign a value score to every academic publication, and cited academic publications are given citation scores that encompass both cases of including and excluding self-citations. The foregoing scores are used to derive another set of scores measuring the combined qualitative and quantitative aspects of the creative work, citations of creative work, informative work and citations of informative work of a researcher. The scores take into consideration co-authorship. From these scores, two versions of the RE-index for a researcher are derived, covering the cases of including and excluding self-citations. The new measure is calculated for two mathematicians.
Keywords
research assessment indicators; academic publication; academic periodical publication; measuring research quality; most relevant quartile score; g, h, and RE indices;
Citations & Related Records
연도 인용수 순위
  • Reference
1 Tramer, M. R., Reynolds, D. J., Moore, R. A., & McQuay, H. J. (1997). Impact of covert duplicate publication on meta-analysis: A case study. BMJ (Clinical research ed.), 315(7109), 635-640. https://doi.org/10.1136/bmj.315.7109.635.   DOI
2 Vavrycuk, V. (2018). Fair ranking of researchers and research teams. PLoS One, 13(4), e0195509. https://doi.org/10.1371/journal.pone.0195509.   DOI
3 Fanelli, D., & Lariviere, V. (2016). Researchers' individual publication rate has not increased in a century. PLoS One, 11(3), e0149504. https://doi.org/10.1371/journal.pone.0149504.   DOI
4 Abelson, P. (1990). Mechanisms for evaluating scientific information and the role of peer review. Journal of the American Society for Information Science, 41(3), 216-222. https://doi.org/10.1002/(SICI)1097-4571(199004)41:3%3C216::AIDASI13%3E3.0.CO;2-6.   DOI
5 Biagioli, M., Kenney, M., Martin, B. R., & Walsh, J. P. (2019). Academic misconduct, misrepresentation and gaming: A reassessment. Research Policy, 48(2), 401-413. https://doi.org/10.1016/j.respol.2018.10.025.   DOI
6 Bornmann, L., Mutz, R., & Daniel, H.-D. (2010). The h index research output measurement: Two approaches to enhance its accuracy. Journal of Informetrics, 4(3), 407-414. https://doi.org/10.1016/j.joi.2010.03.005.   DOI
7 Burnham, J. F. (2006). Scopus database: A review. Biomedical Digital Libraries, 3, 1. https://doi.org/10.1186/1742-5581-3-1.   DOI
8 Butler, D. (2008). Free journal-ranking tool enters citation market. Nature, 451(7174), 6. https://doi.org/10.1038/451006a.   DOI
9 Egghe, L. (2008). Mathematical theory of the h- and g-index in case of fractional counting of authorship. Journal of the American Society for Information Science and Technology, 59(10), 1608-1616. https://doi.org/10.1002/asi.20845.   DOI
10 Stone, R. (2016). In Iran, a shady market for papers flourishes. Science (New York, N.Y.), 353(6305), 1197. https://doi.org/10.1126/science.353.6305.1197.   DOI
11 Jonkers, K., & Zacharewicz, T. (2016). Research performance based funding systems: A comparative assessment. EUR 27837. Publications Office of the European Union.
12 Steen, R. G. (2011). Retractions in the medical literature: How many patients are put at risk by flawed research? Journal of Medical Ethics, 37(11), 688-692. https://doi.org/10.1136/jme.2011.043133.   DOI
13 Nickerson, R. S. (1998). Confirmation bias: A ubiquitous phenomenon in many guises. Review of General Psychology, 2(2), 175-220. https://doi.org/10.1037%2F1089-2680.2.2.175.   DOI
14 Seeber, M., Cattaneo, M., Meoli, M., & Malighetti, P. (2019). Self-citations as strategic response to the use of metrics for career decisions. Research Policy, 48(2), 478-491. https://doi.org/10.1016/j.respol.2017.12.004.   DOI
15 Shibayama, S., & Baba, Y. (2015). Dishonest conformity in peer review. Prometheus, 33(3), 215-233. https://doi.org/10.1080/08109028.2015.1114745.   DOI
16 Westbrook, J. H. (1960). Identifying significant research. Science (New York, N.Y.), 132(3435), 1229-1234. https://doi.org/10.1126/science.132.3435.1229.   DOI
17 Falagas, M. E., Kouranos, V. D., Arencibia-Jorge, R., & Karageorgopoulos, D. E. (2008). Comparison of SCImago journal rank indicator with journal impact factor. FASEB Journal, 22(8), 2623-2628. https://doi.org/10.1096/fj.08-107938.   DOI
18 Honig, B., & Bedi, A. (2012). The fox in the hen house: A critical examination of plagiarism among members of the academy of management. Academy of Management Learning & Education, 11(1), 101-123. https://doi.org/10.5465/amle.2010.0084.   DOI
19 Grossman, J. W. (2005). Patterns of research in mathematics. Notices of the American Mathematical Society, 52(1), 35-41. https://www.ams.org/journals/notices/200501/feagrossman.pdf?adat=January%202005&trk=200501feagrossman&cat=feature&galt=feature.
20 Hirsch, J. E. (2005). An index to quantify an individual's scientific research output. Proceedings of the National Academy of Sciences of the United States of America, 102(46), 16569-16572. https://doi.org/10.1073/pnas.0507655102.   DOI
21 Lopez-Cozar, E. D., Robinson-Garcia, N., & Torres-Salinas, D. (2014). The Google scholar experiment: How to index false papers and manipulate bibliometric indicators. Journal of the Association for Information Science and Technology, 65(3), 446-454. https://doi.org/10.1002/asi.23056.   DOI
22 MacRoberts, M. H., & MacRoberts, B. R. (1986). Quantitative measures of communication in science: A study of the formal level. Social Studies of Science, 16(1), 151-172. https://doi.org/10.1177%2F030631286016001008.   DOI
23 Zhang, C. T. (2009). The e-index, complementing the h-index for excess citations. PLoS One, 4(5), e5429. https://doi.org/10.1371/journal.pone.0005429.   DOI
24 Moed, H. F., Burger, W. J. M., Frankfort, J. G., & Van Raan, A. F. J. (1985). The use of bibliometric data for the measurement of university research performance. Research Policy, 14(3), 131-149. https://doi.org/10.1016/0048-7333(85)90012-5.   DOI
25 MacRoberts, M. H., & MacRoberts, B. R. (2010). Problems of citation analysis: A study of uncited and seldom-cited influences. Journal of the American Society for Information Science and Technology, 61(1), 1-12. https://doi.org/10.1002/asi.21228.   DOI
26 MacRoberts, M. H., & MacRoberts, B. R. (2018). The mismeasure of science: Citation analysis. Journal of the Association for Information Science and Technology, 69(3), 474-482. https://doi.org/10.1002/asi.23970.   DOI
27 Maddox, J. (1995). Plagiarism is worse than mere theft. Nature, 376(6543), 721. https://doi.org/10.1038/376721a0.   DOI
28 Moskovitz, C. (2016). Self-plagiarism, text recycling, and science education. BioScience, 66(1), 5-6. https://doi.org/10.1093/biosci/biv160.   DOI
29 von Elm, E., Poglia, G., Walder, B., & Tramer, M. R. (2004). Different patterns of duplicate publication: An analysis of articles used in systematic reviews. JAMA, 291(8), 974-980. https://doi.org/10.1001/jama.291.8.974.   DOI
30 Waltman, L. (2012). An empirical analysis of the use of alphabetical authorship in scientific publishing. Journal of Informetrics, 6(4), 700-711. https://doi.org/10.1016/j.joi.2012.07.008.   DOI
31 Waltman, L., & van Eck, N. J. (2012). The inconsistency of the h-index. Journal of the American Society for Information Science and Technology, 63(2), 406-415. https://doi.org/10.1002/asi.21678.   DOI
32 Farid, F. O. (1991). Spectral properties of perturbed linear operators and their application to infinite matrices. Proceedings of the American Mathematical Society, 112(4), 1013-1022. https://www.ams.org/journals/proc/1991-112-04/S0002-9939-1991-1057943-2/S0002-9939-1991-1057943-2.pdf.   DOI
33 Callaway, E. (2016). Publishing elite turns against impact factor. Nature, 535(7611), 210-211. https://www.proquest.com/docview/1804902988.   DOI
34 Colussi, T. (2018). Social ties in academia: A friend is a treasure. The Review of Economics and Statistics, 100(1), 45-50. https://doi.org/10.1162/REST_a_00666.   DOI
35 Davis, P. M. (2008). Eigenfactor: Does the principle of repeated improvement result in better estimates than raw citation counts? Journal of the American Society for Information Science and Technology, 59(13), 2186-2188. https://doi.org/10.1002/asi.20943.   DOI
36 Yan, E., & Ding, Y. (2010). Weighted citation: An indicator of an article's prestige. Journal of the American Society for Information Science and Technology, 61(8), 1635-1643. https://doi.org/10.1002/asi.21349.   DOI
37 Falagas, M. E., Zarkali, A., Karageorgopoulos, D. E., Bardakas, V., & Mavros, M. N. (2013). The impact of article length on the number of future citations: A bibliometric analysis of general medicine journals. PLoS One, 8(2), e49476. https://doi.org/10.1371/journal.pone.0049476.   DOI
38 Webster, G. D., Jonason, P. K., & Schember, T. O. (2009). Hot topics and popular papers in evolutionary psychology: Analyses of title words and citation counts in evolution and human behavior, 1979-2008. Evolutionary Psychology, 7(3), 348-362. https://doi.org/10.1177/147470490900700301.   DOI
39 Roig, M. (2010). Plagiarism and self-plagiarism: What every author should know. Biochemia Medica, 20(3), 295-300. https://doi.org/10.11613/BM.2010.037.   DOI
40 Smaldino, P. E., & McElreath, R. (2016). The natural selection of bad science. Royal Society Open Science, 3(9), 160384. https://doi.org/10.1098/rsos.160384.   DOI
41 Egghe, L. (2006). An improvement of the h-index: The gindex. ISSI Newsletter, 2(1), 8-9. https://www.issi-society.org/media/1182/newsletter05.pdf.
42 Abramo, G., D'Angelo, C. A., & Viel, F. (2013). Assessing the accuracy of the h- and g-indexes for measuring researchers' productivity. Journal of the American Society for Information Science and Technology, 64(6), 1224-1234. https://doi.org/10.1002/asi.22828.   DOI
43 Farid, F. O. (1998). Topics on a generalization of Gershgorin's theorem. Linear Algebra and its Applications, 268, 91-116. https://doi.org/10.1016/S0024-3795(97)00030-X.   DOI
44 Fisher, J. C. (1959). Basic research in industry: A count of scientific publications suggests the extent of U.S. industry's effort in basic research. Science, 129(3364), 1653-1657. https://doi.org/10.1126/science.129.3364.1653.   DOI
45 Strange, K. (2008). Authorship: Why not just toss a coin? American Journal of Physiology. Cell Physiology, 295(3), C567-C575. https://doi.org/10.1152/ajpcell.00208.2008.   DOI
46 Szava-Kovats, E. (2004). The false 'Ortega Hypothesis': A literature science case study. Journal of Information Science, 30(6), 496-508. https://doi.org/10.1177%2F0165551504047823.   DOI
47 Rossner, M., Van Epps, H., & Hill, E. (2007). Show me the data. Journal of Cell Biology, 179(6), 1091-1092. https://doi.org/10.1083/jcb.200711140.   DOI
48 Giuffrida, C., Abramo, G., & D'Angelo, C. A. (2019). Are all citations worth the same? Valuing citations by the value of the citing items. Journal of Informetrics, 13(2), 500-514. https://doi.org/10.1016/j.joi.2019.02.008.   DOI
49 Alonso, S., Cabrerizo, F. J., Herrera-Viedma, E., & Herrera, F. (2009). h-Index: A review focused in its variants, computation and standardization for different scientific fields. Journal of Informetrics, 3(4), 273-289. https://doi.org/10.1016/j.joi.2009.04.001.   DOI
50 Garfield, E. (1973). Citation and distinction. Nature, 242, 485. https://doi.org/10.1038/242485a0.   DOI