[KSCI] Korea Science Citation Index Service

http://dx.doi.org/10.1633/JISTaP.2016.4.1.2

Normalization and Valuation of Research Evaluation Indicators in Different Scientific Fields

Chakoli, Abdolreza Noroozi (Department of Knowledge and Information Science and Scientometrics Shahed University)
Ghazavi, Roghayeh (Scientometrics Department Isfahan University of Medical Sciences, Isfahan Knowledge and Information Science Shahid Chamran University of Ahvaz)

Publication Information

Journal of Information Science Theory and Practice / v.4, no.1, 2016 , pp. 21-29 More about this Journal

Abstract

Given the difference in research performance in various scientific fields, this study aims to weight and valuate current indicators used for evaluation of scientific productions (publications), in order to adjust these indicators in comparison to each other and make possible a more precise evaluation of scientific productions. This is a scientometrics study using documentary, evaluative, and survey techniques. The statistical population consisted of 106 top Iranian researchers, scientists, and scientific and research managers. Then their research résumé information was gathered and analyzed based on research questions. In order to compare values, the data gathered from research production performance of the population was weighted using Shannon entropy method. Also, the weights of each scientific production importance according to expert opinions (extracted from other works) was analyzed and after adjustment the final weight of each scientific production was determined. A pairwise matrix was used in order to determine the ratios. According to the results, in the area of engineering sciences, patents (0.142) in the area of science, international articles (0.074) in the area of humanities and social sciences, books (0.174), and in the area of medical sciences, international articles (0.111) had the highest weight compared to other information formats. By dividing the weights for each type of publication, the value of each scientific production compared to other scientific productions in the same field and productions of other fields was calculated. Validation of the results in the studied population resulted in very high credibility for all investigated indicators in all four fields. By using these values and normalized ratios of publication indicators it is possible to achieve precise and adjusted results, making it possible to feasibly use these results in realistic policy making.

Keywords

Valuation; Normalization; Research Evaluation Indicators; Scientific Fields; Scientific Areas; Scientific Production;

Citations & Related Records

Reference

1	Archanbault, E., & Gagne, E. V. (2004). The use of bibliometrics in the social sciences and humanities (Report). Montreal: Social Sciences and Humanities Research Council of Canada (SSHRCC).
2	Chen, K.-H., Tang, M.-C., Wang, C., & Hsiang, J. (2015). Exploring alternative metrics of scholarly performance in the social sciences and humanities in Taiwan. Scientometrics, 102(1), 97-112. DOI
3	Davapanah, M. R. (2010). Scientific Power Indicator: A model to measure and compare the scientific performance of the fields. Journal of Library and Information Science, 13(3), 20-30.
4	Dorta-González, P., Dorta-González, M. I., & Suárez-Vega, R. (2014). An approach to the author citation potential: Measures of scientific performance which are invariant across scientific fields. Scientometrics, 102(2), 1467-1496. DOI
5	Kyvik, S. (1989). Productivity differences, fields of learning, and Lotka’s law. Scientometrics, 15(3), 205-214. DOI
6	Rezaie, M. (2012). Identification and validation of indicators to evaluate the productivity of researchers and universities (Dissertation). Tehran: Shahed University.
7	Waltman, L., & van Eck, N. J. (2015). Field-normalized citation impact indicators and the choice of an appropriate counting method. Journal of Informetrics, 9(4), 872-894. DOI
8	Noroozi Chakoli, A. (2011). Introduction. In Scientometrics (Foundations, concepts, relations & origins). Tehran: Samt; Shahed University.
9	Larivière, V., Archambault, É., Gingras, Y., & Vignola-Gagné, É. (2006). The place of serials in referencing practices: Comparing natural sciences and engineering with social sciences and humanities. Journal of the American Society for Information Science and Technology, 57(8), 997-10. DOI
10	Moed, H. (2005). Citation analysis in research evaluation. Dordrecht, NL: Springer.
11	Rezaie, M., & Noroozi Chakoli, A. (2014). Scientometrics, international special indexes, scientific productivity evaluation. Information Processing and Management, 30(1), 3-39.
12	Schubert, A., & Braun, T. (1996). Cross field normalization of scientometric indicators. Scientometrics, 36(3), 311-324. DOI
13	Torres-Salinas, D., Moreno-Torres, J. G., Delgado- Lo´pez-Co´zar, E., & Herrera, F. (2011). A methodology for Institution-Field ranking based on a bidimensional analysis: The IFQ2A index. Scientometrics, 88(3), 771-786. DOI
14	Waltman, L., van Eck, N. J., van Leeuwen, T. N., Visser, M. S., & van Raan, A. (2010). Towards a new crown indicator: An empirical analysis. arXiv:1004.1632v1.
15	Waltman, L., van Eck, N. J., van Leeuwen, T. N., Visser, M. S., & van Raan, A. (2011). Towards a new crown indicator: Some theoretical considerations. Journal of Informetrics, 5(1), 37-47. DOI
16	Waltman, L., van Eck, N. J., van Leeuwen, T. N., & Visser, M. S. (2013). Some modifications to the SNIP journal impact indicator. Journal of Informetrics, 7(2), 272-285. DOI