Browse > Article
http://dx.doi.org/10.3743/KOSIM.2015.32.4.249

A Review of Declarations on Appropriate Research Evaluation for Exploring Their Applications to Research Evaluation System of Korea  

Yu, So-Young (한남대학교 문헌정보학과)
Lee, Jae Yun (명지대학교 문헌정보학과)
Chung, EunKyung (이화여자대학교 문헌정보학과)
Lee, Boram (이화여자대학교 대학원 문헌정보학과)
Publication Information
Journal of the Korean Society for information Management / v.32, no.4, 2015 , pp. 249-272 More about this Journal
Abstract
Inappropriate applications of bibliometric approach and misinterpretation on the analysis in research evaluation have been found and recognized nationally and internationally as the use of the approach has been rapidly adopted in various sectors in research evaluation systems and research funding agencies. The flood of misuse led to several numbers of declarations and statements on appropriate research evaluation, including Leiden Manifesto, DORA, IEEE Statement, etc. The similar recommendations from five different declarations, Leiden Manifest, IEEE Statement, DORA, Institut de France, and Thomson Reuters White paper were reviewed and meta-analyzed in this study and it is revealed that most of them emphasize evaluation on quality in various aspects with multiple indicators. Research evaluation with assessing multiple aspects of individual research based on the understandings of its purpose and pertinent subject area was revealed as being mostly advised in the declarations, and this recommendation can be regarded as being mostly requested in national research evaluation system. For future study, interviews with relevant stakeholders of national research evaluation system in order to explore its application are needed to confirm the findings of this review.
Keywords
research evaluation; Leiden Manifest; DORA; citation analysis; research policy; science policy;
Citations & Related Records
Times Cited By KSCI : 7  (Citation Analysis)
연도 인용수 순위
1 고영만, 조수련, 박지영 (2013). 학술지의 피인용횟수 순위를 적용한 tapered h-지수의 변형지표 "Kor-hT"에 관한 연구. 정보관리학회지, 30(4), 111-131. http://dx.doi.org/10.3743/KOSIM.2013.30.4.111 Ko, Young Man, Cho, Soo-Ryun, & Park, Ji Young (2013). A study on the "Kor-hT", a modified tapered h-index, by applying the ranking according to the number of citations of journals in evaluating Korean journals. Journal of the Korean Society for Information Management, 30(4), 111-131. http://dx.doi.org/10.3743/KOSIM.2013.30.4.111   DOI
2 김판준 (2011). 연구 성과평가와 연구정보서비스의 연계를 위한 기초 연구: 과학기술 분야 연구개발사업을 중심으로. 정보관리학회지, 28(4), 243-261. http://dx.doi.org/10.3743/KOSIM.2011.28.4.243 Kim, Pan Jun (2011). A study on framework for linkage of research performance evaluation and research information service. Journal of the Korean Society for Information Management, 28(4), 243-261. http://dx.doi.org/10.3743/KOSIM.2011.28.4.243   DOI
3 김판준, 이재윤 (2010). 학술지 영향력 측정을 위한 h-지수의 응용에 관한 연구. 정보관리학회지, 27(1), 269-287. http://dx.doi.org/10.3743/KOSIM.2010.27.1.269 Kim, Pan-Jun, & Lee, Jae Yun (2010). A study on journal impact measurement with Hirsch-type indices. Journal of the Korean Society for Information Management, 27(1), 269-287. http://dx.doi.org/10.3743/KOSIM.2010.27.1.269   DOI
4 설혜심 (2011). 학문의 분화와 통섭. 학림, 32, 91-124. Seol, Hyesim (2011). Differentiation and consilience of disciplines. Hak-Lim, 32, 91-124.
5 이재윤 (2011a). 국내 인용 데이터베이스에서 저널 페이지랭크 측정 방안. 한국비블리아학회지, 22(4), 361-379. Lee, Jae Yun (2011a). Journal PageRank calculation in the Korean Science Citation Database. Journal of the Korean Biblia Society for Library and Information Science, 22(4), 361-379.
6 이재윤 (2011b). 인용 네트워크 분석에 근거한 문헌 인용 지수 연구. 한국문헌정보학회지, 45(2), 119-143. http://dx.doi.org/10.4275/KSLIS.2011.45.2.119 Lee, Jae Yun (2011b). A study on document citation indicators based on citation network analysis. Journal of the Korean Society for Library and Information Science, 45(2), 119-143. http://dx.doi.org/10.4275/KSLIS.2011.45.2.119   DOI
7 이종욱, 양기덕 (2011). 교수연구업적 평가법의 계량적 분석: 국내 문헌정보학과 교수연구업적을 중심으로. 정보관리학회지, 28(4), 119-140. http://dx.doi.org/10.3743/KOSIM.2011.28.4.119 Lee, Jongwook, & Yang, Kiduk (2011). A bibliometric analysis of faculty research performance assessment methods. Journal of the Korean Society for Information Management, 28(4), 119-140. http://dx.doi.org/10.3743/KOSIM.2011.28.4.119   DOI
8 조은성, 송재도 (2011). 국내외 마케팅 학술지의 영향력: Kor-Factor와 Impact Factor의 문제점을 중심으로. 마케팅관리연구, 16(2), 53-82. Cho, Eun Seong, & Song, Jae Do (2011). The influence of Korean and international marketing journals: Focused on the problems of Kor-Factor and Impact Factor. Journal of Marketing Management Research, 16(2), 53-82.
9 중앙일보 대학평가. [2015.11.10]. Retrieved from http://univ.joongang.co.kr/ Joongangilbo University Rank. [2015.11.10]. Retrieved from http://univ.joongang.co.kr/
10 한국정보과학회 (2013). 컴퓨터 분야 성과지표 개선(안). 한국정보과학회. Retrieved from http://an.kaist.ac.kr/-sbmoon/tmp/kiisereport.pdf KIISE (2013). Improvement plan of research performance measures in computer science. KIISE. Retrieved from http://an.kaist.ac.kr/-sbmoon/tmp/kiisereport.pdf
11 Astrom, F., & Hansson, J. (2013). How implementation of bibliometric practice affects the role of academic libraries. Journal of Librarianship and Information Science, 45(4), 316-322. http://dx.doi.org/10.1177/0961000612456867   DOI
12 Ball, R., & Tunger, D. (2006). Bibliometric analysis: A new business area for information professionals in libraries? Scientometrics, 66(3), 561-577. http://dx.doi.org/10.1007/s11192-006-0041-0   DOI
13 Bladek, M. (2014). DORA: San Francisco declaration on research assessment (May 2013). College and Research Libraries News, 75(4), 191-196.   DOI
14 Butler-Adam, J. (2013). DORA: The San Francisco declaration on research assessment. South African Journal of Science, 109(7/8), 1-1. http://dx.doi.org/10.1590/sajs.2013/a0032
15 Grant Steen, R. (2013). Journal impact factor: Baby and bathwater discarded? European Science Editing, 39(3), 64-65.
16 Cagan, R. (2013). The San Francisco declaration on research assessment. Disease Models & Mechanisms, 6(4), 869-870. http://dx.doi.org/10.1242/dmm.012955   DOI
17 Garfield, E. (2009). From information retrieval to scientometrics-is the dog still wagging its tail. Keynote Address at WIS & COLLNET. Retrieved from http://garfield.library.upenn.edu/papers/dalianchina2009.html
18 Gomez Marin, J. E. (2015). Why to disagree with the San Francisco declaration on research assessment. Infectio, 19(3), 99-100. http://dx.doi.org/10.1016/j.infect.2015.04.001   DOI
19 Hagen, N. T. (2008). Harmonic allocation of authorship credit: Source-level correction of bibliometric bias assures accurate publication and citation analysis. PLoS ONE, 3(12), e4021. http://dx.doi.org/10.1371/journal.pone.0004021   DOI
20 Hicks, D., Wouters, P., Waltman, L., de Rijcke, S., & Rafols, I. (2015). Bibliometrics: The Leiden Manifesto for research metrics. Nature, 520(7548), 429-431. Retrieved from http://www.nature.com/news/bibliometrics-the-leiden-manifesto-for-research-metrics-1.17351   DOI
21 Hoppeler, H. (2013). The San Francisco declaration on research assessment. The Journal of Experimental Biology, 216(Pt 12), 2163-4. http://dx.doi.org/10.1242/jeb.090779   DOI
22 IEEE (2013). Appropriate use of bibliometric indicators for the assessment of journals, research proposals, and individuals. Retrieved from https://www.ieee.org/publications_standards/publications/rights/ieee_bibliometric_statement_sept_2013.pdf
23 King, J. (1987). A review of bibliometric and other science indicators and their role in research evaluation. Journal of Information Science, 13(5), 261-271. http://dx.doi.org/10.1177/016555158701300501   DOI
24 Moed, H. F. (2010). Measuring contextual citation impact of scientific journals. Journal of Informetrics, 4(3), 256-277. http://dx.doi.org/10.1016/j.joi.2010.01.002
25 Jimenez-Contreras, E., Lopez-Cozar, E. D., Ruiz-Perez, R., & Fernandez, V. M. (2002). Impactfactor rewards affect Spanish research, Nature, 417 (27 June 2002), 898. http://dx.doi.org/10.1038/417898b
26 Lee, Jae Yun, & Chung, EunKyung (2014). A comparative analysis on multiple authorship counting for author co-citation analysis. Journal of the Korean Society for Information Management, 31(2), 57-77. http://dx.doi.org/10.3743/KOSIM.2014.31.2.057   DOI
27 Leydesdorff, L., & Bornmann, L. (2012). Percentile ranks and the integrated impact indicator (I3). Journal of the American Society for Information Science and Technology, 63(9), 1901-1902. http://dx.doi.org/10.1002/asi.22641   DOI
28 Pourquie, O. (2013). The San Francisco declaration on research assessment. Development (Cambridge, England), 140(13), 2643-4. http://dx.doi.org/10.1242/dev.099234   DOI
29 Pendlebury, D. A. (2009). The use and misuse of journal metrics and other citation indicators. Archivum Immunologiae et Therapiae Experimentalis, 57(1), 1-11. http://dx.doi.org/10.1007/s00005-009-0008-y   DOI
30 Pendlebury, D. A. (2010). Using bibliometrics in evaluating research. Retrieved from https://services.anu.edu.au/files/system/Pendlebury_White_Paper.pdf
31 Pugh, E. N., & Gordon, S. E. (2013). Embracing the principles of the san francisco declaration of research assessment: Robert Balaban's editorial. The Journal of General Physiology, 142(3), 175. http://dx.doi.org/10.1085/jgp.201311077   DOI
32 Schekman, R., & Patterson, M. (2013). Reforming research assessment. eLife, 2(2), e00855. http://dx.doi.org/10.7554/eLife.00855
33 Way, M., & Ahmad, S. A. (2013). The San Francisco declaration on research assessment. Journal of Cell Science, 126(Pt 9), 1903-4. http://dx.doi.org/10.1242/jcs.134460   DOI
34 Schubert, A. (2009). Using the h-index for assessing single publications. Scientometrics, 78(3), 559-565. http://dx.doi.org/10.1007/s11192-008-2208-3   DOI
35 Servaes, J. (2014). On impact factors and research assessment. At the start of volume 31 of telematics and informatics. Telematics and Informatics, 31(1), 1-2. http://dx.doi.org/10.1016/j.tele.2013.09.005   DOI
36 Thomson Reuters (2008). Using bibliometrics: A guide to evaluating research performance with citation data. Retrieved from http://ip-science.thomsonreuters.com/m/pdfs/325133_thomson.pdf