The Korea Journal of Herbology (대한본초학회지)

The Korea Association of Herbology (대한본초학회)
권호 표지
DOI QR코드

DOI QR Code

Study on the herbology test items in Korean medicine education using Item Response Theory

문항반응이론을 활용한 한의학 교육에서 본초학 시험문항에 대한 연구

  • Chae, Han (School of Korean Medicine, Pusan National University) ;
  • Han, Sang Yun (College of Korean Medicine, Daejeon University) ;
  • Yang, GiYoung (School of Korean Medicine, Pusan National University) ;
  • Kim, Hyungwoo (School of Korean Medicine, Pusan National University)
  • 채한 (부산대학교 한의학전문대학원) ;
  • 한상윤 (대전대학교 한의학과) ;
  • 양기영 (부산대학교 한의학전문대학원) ;
  • 김형우 (부산대학교 한의학전문대학원)
  • Received : 2022.01.24
  • Accepted : 2022.03.25
  • Published : 2022.03.30
Download PDF
⟨ Previous Next ⟩

Abstract

Objectives : The evaluation of academic achievement is pivotal for establishing accurate direction and adequate level of medical education. The purpose of this study was to firstly establish innovative item analysis technique of Item Response Theory (IRT) for analyzing multiple-choice test of herbology in the traditional Korean medicine education which has not been available for the difficulty of test theory and statistical calculation. Methods : The answers of 390 students (2012-2018) to the 14 item herbology test in college of Korean medicine were used for the item analysis. As for the multidimensional analysis of item characteristics, difficulty, discrimination, and guessing parameters along with item-total correlation and percentage of correct answer were calculated using Classical Test Theory (CTT) and IRT. Results : The validity parameters of strong and weak items were illustrated in multiple perspectives. There were 4 items with six acceptable index scores, and 5 items with only one acceptable index score. The item discrimination of IRT was found to have no significant correlation with difficulty and discrimination indices of CTT emphasizing attention of professionals of medical education as for the test credibility. Conclusion : The critical suggestions for the development, utilization and revision of test items in the e-learning and evidence-based Teaching era were made based on the results of item analysis using IRT. The current study would firstly provide foundation for upgrading the quality of Korean medicine education using test theory.

Keywords

Acknowledgement

이 과제는 부산대학교 기본연구지원사업(2년)에 의하여 연구되었음.

References

  1. Schauber SK, Hecht M. How sure can we be that a student really failed? On the measurement precision of individual pass-fail decisions from the perspective of Item Response Theory. Medical Teacher. 2020 ; 42(12) : 1374-84. https://doi.org/10.1080/0142159X.2020.1811844
  2. De Champlain AF. A primer on classical test theory and item response theory for assessments in medical education. Medical education. 2010 ; 44(1) : 109-17. https://doi.org/10.1111/j.1365-2923.2009.03425.x
  3. Lee SY, Lee Y, Kim MK. Effectiveness of Medical Education Assessment Consortium Clinical Knowledge Mock Examination (2011-2016). Korean Med Educ Rev. 2018 ; 20(1) : 20-31. https://doi.org/10.17496/KMER.2018.20.1.20
  4. Lim EY, Park JH, Kwon I, Song GL, Huh S. Comparison of item analysis results of Korean Medical Licensing Examination according to classical test theory and item response theory. J Educ Eval Health Prof. 2004 ; 1(1) : 67-76. https://doi.org/10.3352/jeehp.2004.1.1.67
  5. Huh S. Application of Computerized Adaptive Testing in medical education. Korean J Med Educ. 2009 ; 21(2) : 97-102. https://doi.org/10.3946/kjme.2009.21.2.97
  6. Jeong G-H, Yim MK. Applicability of Item Response Theory to the Korean nurses' licensing examination. J Educ Eval Health Prof. 2005 ; 2(1) : 23-9. https://doi.org/10.3352/jeehp.2005.2.1.23
  7. Yim MK, Huh S. Testing unidimensionality and goodness-of-fitness for the application of Item Response Theory to the Korean medical licensing examination. Korean J Med Educ. 2007 ; 19(2) : 163-9. https://doi.org/10.3946/kjme.2007.19.2.163
  8. Lim H-S, Lee Y-M, Ahn D-S, Lee J-Y, Im H. Item analysis of Clinical Performance Examination using Item Response Theory and Classical Test Theory. Korean J Med Educ. 2007 ; 19(3) : 185-95. https://doi.org/10.3946/kjme.2007.19.3.185
  9. Department of test question development in Korea Health Personnel Licensing Examination Institute. Guide for developing test items in Korea Health Personnel Licensing Examination Institute. Seoul, Korea : Korea Health Personnel Licensing Examination Institute. 2013 : 43-51.
  10. Park JC, Kim KS. A comparison between discrimination indices and Item-Response Theory Using the Rasch model in a clinical course written examination of a medical school. Korean J Med Educ. 2012 ; 24(1) : 15-21. https://doi.org/10.3946/kjme.2012.24.1.15
  11. Park C. Polytomous Item Response Theory Model. Seoul : Educational Science Publishing. 2001 : 11-18.
  12. Seong T. Understanding and application of Item Response Theory. Paju : Educational Science Publishing. 2016: 18-53.
  13. Meyer JP. Applied measurement with jMetrik. New York : Routledge. 2014 : 40-52, 82-107.
  14. Park J-W, Jang L-C, Choi J-W, Lee S-J. The experience of web-based test in medical education. Korean J Med Educ. 2006 ; 18(2) : 183-192. https://doi.org/10.3946/kjme.2006.18.2.183
  15. Armstrong R, Belov D, Weissman A. Developing and Assembling the Law School Admission Test. Interfaces. 2005 ; 35(2) : 140-51. https://doi.org/10.1287/inte.1040.0123
  16. Schauber SK, Hecht M, Nouns ZM. Why assessment in medical education needs a solid foundation in modern test theory. Adv in Health Sci Educ. 2018 ; 23(1) : 217-32. https://doi.org/10.1007/s10459-017-9771-4
  17. Choi Y-J, Asiilkalkan A. R package for Item Response Theory Analysis : description and features. Measurement : Interdisciplinary research and perspectives. 2019 ; 17(3) : 168-75. https://doi.org/10.1080/15366367.2019.1586404
  18. Lee S, Lee Y, Han SY, Bae N, Hwang M, Lee J, Chae H. Urinary Function of the Sasang Type and Cold-Heat Subgroup Using the Sasang Urination Inventory in Korean Hospital Patients. Evid Based Complement Alternat Med. 2020 ; 2020 : 7313581.
  19. Chae H, Cho YI, Lee SJ. The Yin-Yang personality from biopsychological perspective using revised Sasang Personality Questionnaire. Integr Med Res. 2021 ; 10(1) : 100455. https://doi.org/10.1016/j.imr.2020.100455
  20. Lee Y-j, Lee S, Kim S-h, Lee J, Chae H. Study on the revision and clinical validation of the Sasang Digestive Function Inventory. J Sasang Constit Med. 2021 ; 33(3) : 54-71. https://doi.org/10.7730/JSCM.2021.33.3.54
  21. Korean Laws Information Center. ACT ON DEVELOPMENT OF E-LEARNING INDUSTRY AND PROMOTION OF UTILIZATION OF E-LEARNING. [cited 2021, Oct 10] Available from https://www.law.go.kr/lsInfoP.do?lsiSeq=177153&lsId=009747&chrClsCd=010202&urlMode=lsInfoP&viewCls=lsInfoP&efYd=20151210&vSct=ACT%20ON%20DEVELOPMENT%20OF%20E-LEARNING%20INDUSTRY%20AND%20PROMOTION%20OF%20UTILIZATION%20OF%20E-LEARNING&ancYnChk=0#0000
  22. Park JH, Son JY, Kim S. Experiences with establishing and implementing Learning Management System and Computer-Based Test System in medical college. Korean J Med Educ. 2012 ; 24(3) : 213-22. https://doi.org/10.3946/kjme.2012.24.3.213
  23. Ruiz JG, Mintzer MJ, Leipzig RM. The Impact of E-Learning in Medical Education. Academic Medicine. 2006 ; 81(3) : 207-12. https://doi.org/10.1097/00001888-200603000-00002
  24. Chae Y-m, Park SG, Park I. The relationship between classical item characteristics and item response time on computer-based testing. Korean J Med Educ. 2019 ; 31(1) : 1-9. https://doi.org/10.3946/kjme.2019.113
  25. Korea Health Personnel Licensing Examination Institute. Computer Based Test. 2021 [cited 2021, Oct 10] Available from https://www.kuksiwon.or.kr/cnt/c_2033/view.do?seq=11
  26. Im E-J, Lee W-K, Lee Y-C, Choe B-H, Chung S-K, Lee T-H, Cho H, Sohn J-H, Won D-I, Kong H-H, Chang B-H, Lee J-M. Development of Computer-Based Test (CBT) and student recognition survey on CBT. Korean J Med Educ. 2008 ; 20(2) : 145-54. https://doi.org/10.3946/kjme.2008.20.2.145
  27. Chae H, Hwang S, Kwon Y, Baik Y, Shin S, Yang G, Lee B, Kim JK, Lee B. Study on the Prerequisite Chinese Characters for Education of Traditional Korean Medicine. J Physiol & Pathol Korean Med 2010 ; 24(3) : 373-9.
  28. Lee Y, Kwak M-J, Jung H, Ha H-y, Chae H. A study on the statistical methods used in KCI listed journals of traditional Korean medicine from 1999 to 2008. J Korean Orient Med. 2012 ; 18(2) : 55-64.
  29. Lee J, Han JH, Kim MS, Lee HS, Han SY, Lee SJ, Chae H. Teaching Yin-Yang biopsychology using the animation, "Pororo the Little Penguin". Eur J Integr Med. 2020 ; 33 : 101037. https://doi.org/10.1016/j.eujim.2019.101037
  30. Han SY, Kim HY, Lim JH, Cheon J, Kwon Y, Kim H, Yang GY, Chae H. The past, present, and future of traditional medicine education in Korea. Integr Med Res. 2016 ; 5(2) : 73-82. https://doi.org/10.1016/j.imr.2016.03.003
  31. Korea Health Personnel Licensing Examination Institute. Development of multiple choice test item. In. Improvement of item development ability (Workshop). Yangsan campus, Pusan National University: Korea Health Personnel Licensing Examination Institute. 2019 : 29-31.
  32. Korea Health Personnel Licensing Examination Institute. Item Analysis on the 75th National Licensing Examination (2020) for Korean Medicine Doctor. Seoul: Korea Health Personnel Licensing Examination Institute. 2020 : 2-4, 10-50.
  33. Huh S. Computer-based testing and construction of an item bank database for medical education in Korea. Korean Med Educ Rev. 2014 ; 16(1) : 11-15. https://doi.org/10.17496/KMER.2014.16.1.011
  34. Huh S. Test equating of a medical school lecture examination based on Item Response Theory : a case study. Korean J Med Educ. 2005 ; 17(1) : 15-28. https://doi.org/10.3946/kjme.2005.17.1.15
  35. Kang MJ, Kim MS. Item analysis using Classical Test Theory and Item Response Theory, validity and reliability of the Korean version of a Pressure Ulcer Prevention Knowledge. J Korean Biol Nurs Sci. 2018 ; 20(1) : 11-19. https://doi.org/10.7586/jkbns.2018.20.1.11