Impacts of the Journal Evaluation Program of the Korean Association of Medical Journal Editors (KAMJE) on the Quality of the Member Journals

Background: In 1997 the Korean Association of Medical Journal Editors (KAMJE) instituted a program to evaluate member journals. Journals that passed the initial evaluation were indexed in the KoreaMed. Here, we report changes in measures of quality of the KAMJE member journals during the last 20 years. Methods: Quality measures used in the study comprised 3 assessment categories; self-assessment by journal editors, assessment of the journals by KAMJE reviewers, and by Korean health science librarians. Each used detailed criteria to score the journals on a scale of 0 to 5 or 6 in multiple dimensions. We compared scores at baseline evaluation and those after 7 years for 129 journals and compared improvements in journals indexed vs. not-indexed by the Web of Science (Science Citation Index Expanded; SCIE). Results: Among 251 KAMJE member journals at the end of 2015, 227 passed evaluation criteria and 129 (56%) had both baseline and 7-year follow-up assessment data. The journals showed improvement overall (increase in median [interquartile range; IQR] score from baseline, 0.47 [0.64]; 95% confidence interval [CI], 0.44-0.61; P < 0.001) and within each category (median [IQR] increase by editor's assessment, 0.17 [0.83]; 95% CI, 0.04-0.26; P = 0.007; by reviewer's, 0.45 [1.00]; 95% CI, 0.29-0.57; P < 0.001; by librarian's, 1.75 [1.08]; 95% CI, 1.77-2.18, P < 0.001). Before the foundation of KAMJE in 1996, there were only 5 Korean medical journals indexed in the MEDLINE and none in SCIE, but 24 journals in the MEDLINE and 34 journals in SCIE were indexed by 2016. Conclusion: The KAMJE journal evaluation program successfully contributes improving the quality of the member journals.