Browse > Article
http://dx.doi.org/10.6116/kjh.2022.37.2.13.

Study on the herbology test items in Korean medicine education using Item Response Theory  

Chae, Han (School of Korean Medicine, Pusan National University)
Han, Sang Yun (College of Korean Medicine, Daejeon University)
Yang, GiYoung (School of Korean Medicine, Pusan National University)
Kim, Hyungwoo (School of Korean Medicine, Pusan National University)
Publication Information
The Korea Journal of Herbology / v.37, no.2, 2022 , pp. 13-21 More about this Journal
Abstract
Objectives : The evaluation of academic achievement is pivotal for establishing accurate direction and adequate level of medical education. The purpose of this study was to firstly establish innovative item analysis technique of Item Response Theory (IRT) for analyzing multiple-choice test of herbology in the traditional Korean medicine education which has not been available for the difficulty of test theory and statistical calculation. Methods : The answers of 390 students (2012-2018) to the 14 item herbology test in college of Korean medicine were used for the item analysis. As for the multidimensional analysis of item characteristics, difficulty, discrimination, and guessing parameters along with item-total correlation and percentage of correct answer were calculated using Classical Test Theory (CTT) and IRT. Results : The validity parameters of strong and weak items were illustrated in multiple perspectives. There were 4 items with six acceptable index scores, and 5 items with only one acceptable index score. The item discrimination of IRT was found to have no significant correlation with difficulty and discrimination indices of CTT emphasizing attention of professionals of medical education as for the test credibility. Conclusion : The critical suggestions for the development, utilization and revision of test items in the e-learning and evidence-based Teaching era were made based on the results of item analysis using IRT. The current study would firstly provide foundation for upgrading the quality of Korean medicine education using test theory.
Keywords
Classical Test Theory; e-Learning; herbology; Evidence-based Teaching; Item Response Theory; medical education;
Citations & Related Records
Times Cited By KSCI : 3  (Citation Analysis)
연도 인용수 순위
1 Schauber SK, Hecht M. How sure can we be that a student really failed? On the measurement precision of individual pass-fail decisions from the perspective of Item Response Theory. Medical Teacher. 2020 ; 42(12) : 1374-84.   DOI
2 De Champlain AF. A primer on classical test theory and item response theory for assessments in medical education. Medical education. 2010 ; 44(1) : 109-17.   DOI
3 Lee SY, Lee Y, Kim MK. Effectiveness of Medical Education Assessment Consortium Clinical Knowledge Mock Examination (2011-2016). Korean Med Educ Rev. 2018 ; 20(1) : 20-31.   DOI
4 Lim EY, Park JH, Kwon I, Song GL, Huh S. Comparison of item analysis results of Korean Medical Licensing Examination according to classical test theory and item response theory. J Educ Eval Health Prof. 2004 ; 1(1) : 67-76.   DOI
5 Huh S. Application of Computerized Adaptive Testing in medical education. Korean J Med Educ. 2009 ; 21(2) : 97-102.   DOI
6 Jeong G-H, Yim MK. Applicability of Item Response Theory to the Korean nurses' licensing examination. J Educ Eval Health Prof. 2005 ; 2(1) : 23-9.   DOI
7 Park JC, Kim KS. A comparison between discrimination indices and Item-Response Theory Using the Rasch model in a clinical course written examination of a medical school. Korean J Med Educ. 2012 ; 24(1) : 15-21.   DOI
8 Park C. Polytomous Item Response Theory Model. Seoul : Educational Science Publishing. 2001 : 11-18.
9 Lim H-S, Lee Y-M, Ahn D-S, Lee J-Y, Im H. Item analysis of Clinical Performance Examination using Item Response Theory and Classical Test Theory. Korean J Med Educ. 2007 ; 19(3) : 185-95.   DOI
10 Han SY, Kim HY, Lim JH, Cheon J, Kwon Y, Kim H, Yang GY, Chae H. The past, present, and future of traditional medicine education in Korea. Integr Med Res. 2016 ; 5(2) : 73-82.   DOI
11 Korea Health Personnel Licensing Examination Institute. Development of multiple choice test item. In. Improvement of item development ability (Workshop). Yangsan campus, Pusan National University: Korea Health Personnel Licensing Examination Institute. 2019 : 29-31.
12 Huh S. Computer-based testing and construction of an item bank database for medical education in Korea. Korean Med Educ Rev. 2014 ; 16(1) : 11-15.   DOI
13 Huh S. Test equating of a medical school lecture examination based on Item Response Theory : a case study. Korean J Med Educ. 2005 ; 17(1) : 15-28.   DOI
14 Park J-W, Jang L-C, Choi J-W, Lee S-J. The experience of web-based test in medical education. Korean J Med Educ. 2006 ; 18(2) : 183-192.   DOI
15 Park JH, Son JY, Kim S. Experiences with establishing and implementing Learning Management System and Computer-Based Test System in medical college. Korean J Med Educ. 2012 ; 24(3) : 213-22.   DOI
16 Seong T. Understanding and application of Item Response Theory. Paju : Educational Science Publishing. 2016: 18-53.
17 Meyer JP. Applied measurement with jMetrik. New York : Routledge. 2014 : 40-52, 82-107.
18 Armstrong R, Belov D, Weissman A. Developing and Assembling the Law School Admission Test. Interfaces. 2005 ; 35(2) : 140-51.   DOI
19 Schauber SK, Hecht M, Nouns ZM. Why assessment in medical education needs a solid foundation in modern test theory. Adv in Health Sci Educ. 2018 ; 23(1) : 217-32.   DOI
20 Choi Y-J, Asiilkalkan A. R package for Item Response Theory Analysis : description and features. Measurement : Interdisciplinary research and perspectives. 2019 ; 17(3) : 168-75.   DOI
21 Lee S, Lee Y, Han SY, Bae N, Hwang M, Lee J, Chae H. Urinary Function of the Sasang Type and Cold-Heat Subgroup Using the Sasang Urination Inventory in Korean Hospital Patients. Evid Based Complement Alternat Med. 2020 ; 2020 : 7313581.
22 Korea Health Personnel Licensing Examination Institute. Computer Based Test. 2021 [cited 2021, Oct 10] Available from https://www.kuksiwon.or.kr/cnt/c_2033/view.do?seq=11
23 Chae H, Cho YI, Lee SJ. The Yin-Yang personality from biopsychological perspective using revised Sasang Personality Questionnaire. Integr Med Res. 2021 ; 10(1) : 100455.   DOI
24 Lee Y-j, Lee S, Kim S-h, Lee J, Chae H. Study on the revision and clinical validation of the Sasang Digestive Function Inventory. J Sasang Constit Med. 2021 ; 33(3) : 54-71.   DOI
25 Ruiz JG, Mintzer MJ, Leipzig RM. The Impact of E-Learning in Medical Education. Academic Medicine. 2006 ; 81(3) : 207-12.   DOI
26 Lee J, Han JH, Kim MS, Lee HS, Han SY, Lee SJ, Chae H. Teaching Yin-Yang biopsychology using the animation, "Pororo the Little Penguin". Eur J Integr Med. 2020 ; 33 : 101037.   DOI
27 Chae Y-m, Park SG, Park I. The relationship between classical item characteristics and item response time on computer-based testing. Korean J Med Educ. 2019 ; 31(1) : 1-9.   DOI
28 Im E-J, Lee W-K, Lee Y-C, Choe B-H, Chung S-K, Lee T-H, Cho H, Sohn J-H, Won D-I, Kong H-H, Chang B-H, Lee J-M. Development of Computer-Based Test (CBT) and student recognition survey on CBT. Korean J Med Educ. 2008 ; 20(2) : 145-54.   DOI
29 Chae H, Hwang S, Kwon Y, Baik Y, Shin S, Yang G, Lee B, Kim JK, Lee B. Study on the Prerequisite Chinese Characters for Education of Traditional Korean Medicine. J Physiol & Pathol Korean Med 2010 ; 24(3) : 373-9.
30 Lee Y, Kwak M-J, Jung H, Ha H-y, Chae H. A study on the statistical methods used in KCI listed journals of traditional Korean medicine from 1999 to 2008. J Korean Orient Med. 2012 ; 18(2) : 55-64.
31 Department of test question development in Korea Health Personnel Licensing Examination Institute. Guide for developing test items in Korea Health Personnel Licensing Examination Institute. Seoul, Korea : Korea Health Personnel Licensing Examination Institute. 2013 : 43-51.
32 Korea Health Personnel Licensing Examination Institute. Item Analysis on the 75th National Licensing Examination (2020) for Korean Medicine Doctor. Seoul: Korea Health Personnel Licensing Examination Institute. 2020 : 2-4, 10-50.
33 Korean Laws Information Center. ACT ON DEVELOPMENT OF E-LEARNING INDUSTRY AND PROMOTION OF UTILIZATION OF E-LEARNING. [cited 2021, Oct 10] Available from https://www.law.go.kr/lsInfoP.do?lsiSeq=177153&lsId=009747&chrClsCd=010202&urlMode=lsInfoP&viewCls=lsInfoP&efYd=20151210&vSct=ACT%20ON%20DEVELOPMENT%20OF%20E-LEARNING%20INDUSTRY%20AND%20PROMOTION%20OF%20UTILIZATION%20OF%20E-LEARNING&ancYnChk=0#0000
34 Kang MJ, Kim MS. Item analysis using Classical Test Theory and Item Response Theory, validity and reliability of the Korean version of a Pressure Ulcer Prevention Knowledge. J Korean Biol Nurs Sci. 2018 ; 20(1) : 11-19.   DOI
35 Yim MK, Huh S. Testing unidimensionality and goodness-of-fitness for the application of Item Response Theory to the Korean medical licensing examination. Korean J Med Educ. 2007 ; 19(2) : 163-9.   DOI