Browse > Article
http://dx.doi.org/10.7739/jkafn.2017.24.3.189

Comparison of Standardized Patient and Faculty Agreement in Evaluating Nursing Students' Assessment and Communication Skills  

Kim, Young Ju (College of Nursing, Sungshin Women's University)
Publication Information
Journal of Korean Academy of Fundamentals of Nursing / v.24, no.3, 2017 , pp. 189-199 More about this Journal
Abstract
Purpose: This study was conducted to examine the level of agreement between a standardized patient (SP) and a faculty member in the evaluation of nursing students' assessment and communication skills. Methods: Participants were 51 third year nursing students in a simulation practice of 'nursing care for a patient admitted with chest pain'. Using a 30-item checklist and a 16-item communication tool, a SP and faculty member evaluated the students' assessment and communication skills during the simulation. Results: The average values for percent agreement and kappa statistic for nursing assessment between the two evaluators were 85.3% and .48 respectively. Twenty of thirty items evaluating assessment skill had above moderate agreement (${\geq}.41$) by kappa between the evaluators. Seven of sixteen items evaluating communication and interpersonal skills showed above fair agreement (${\geq}.40$) between the two evaluators, which was measured by intraclass correlation coefficient. Conclusion: The findings show that the evaluation of the SP was consistent with those of the faculty member to a moderate degree. Clear guidelines for evaluating criteria and optimal time and effort for SP training are necessary to increase the reliability of standardized patients as evaluators in simulation-based nursing education.
Keywords
Patient simulation; Communication; Nursing education; Inter-observer variation;
Citations & Related Records
Times Cited By KSCI : 5  (Citation Analysis)
연도 인용수 순위
1 Choi JY, Jang KS, Choi SH, Hong MS. Validity and reliability of a clinical performance examination using standardized patients. Journal of Korean Academy of Nursing. 2008;38(1):83-91.   DOI
2 Hur HK, Song HY. Effects of simulation-based clinical reasoning education and evaluation of perceived education practices and simulation design characteristics by students nurses. Journal of the Korea Contents Association. 2015;15(3):206-218. https://doi.org/10.5392/JKCA.2015.15.03.206   DOI
3 Lee SO, Eom M, Lee JH. Use of simulation in nursing education. Journal of Korean Academic Society of Nursing Education. 2007;13(1):90-94.
4 Davis AH, Kimble LP. Human patient simulation evaluation rubrics for nursing education: Measuring the essentials of baccalaureate education for professional nursing practice. Journal of nursing education. 2011;50(11):605-611. https://doi.org/10.3928/01484834-20110715-01   DOI
5 Kim JH, Park IH, Shin S. Systematic review of Korean studies on simulation within nursing education. Journal of Korean Academic Society of Nursing Education. 2013;19(3):307-319. https://doi.org/10.5977/jkasne.2013.19.3.307   DOI
6 Lee BG. Recent world trend in performance-based assessments and application of the standardized patient program in Korean medical education. Korean Journal of Medical Education. 2000;12(2):377-392. https://doi.org/10.3946/kjme.2000.12.2.377   DOI
7 Wallace P. Coaching standardized patients for use in the assessment of clinical competence. New York: Springer Publishing Company; 2007. p. 83-108.
8 Kim JJ, Lee KJ, Choi KY, Lee DW. Analysis of the evaluation for clinical performance examination using standardized patients in one medical school. Korean Journal of Medical Education. 2004;16(1):51-61. https://doi.org/10.3946/kjme.2004.16.1.51   DOI
9 Barrows HS. An overview of the uses of standardized patients for teaching and evaluating clinical skills. Academic Medicine. 1993;68(6):443-451.   DOI
10 Yoo HB, Park JH, Ko JK, Yoon TY. A systematic review of training that use an integrated patient simulator. Korean Journal of Medical Education. 2010;22(4):257-268. https://doi.org/10.3946/kjme.2010.22.4.257   DOI
11 Park KI, Jeong YS. Nurse's communication skills as perceived by the parents of inpatients. Korean Parent Child Health Journal. 2011;14(2):69-75.
12 McCarthy B, Trace A, O'Donovan M. Integrating psychology with interpersonal communication skills in undergraduate nursing education: Addressing the challenges. Nurse Education in Practice. 2014;14(3):227-232. https://doi.org/10.1016/j.nepr.2014.01.008   DOI
13 Campbell SH, Pagano MP, O'Shea ER, Connery C, Caron C. Development of the health communication assessment tool: Enhancing relationships, empowerment, and power-sharing skills. Clinical Simulation in Nursing. 2013;9(11):e543-e550. https://doi.org/10.1016/j.ecns.2013.04.016   DOI
14 Yang HM, Hwang SY. Reliability and validity of the assessment tool for measuring communication skills in nursing simulation education. Korean Journal of Adult Nursing. 2016;28(1): 95-105. https://doi.org/10.7475/kjan.2016.28.1.95   DOI
15 Faul F, Erdfelder E, Lang AG, Buchner A. G*power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods. 2007;39(2):175-191. https://doi.org/10.3758/BF03193146   DOI
16 Yudkowsky R, Downing SM, Sandlow LJ. Developing an institution-based assessment of resident communication and interpersonal skills. Academic Medicine. 2006;81(12):1115-1122. https://doi.org/10.1097/01.ACM.0000246752.00689.bf   DOI
17 CPX Consortium Seoul.Gyeonggi. Introduction of consortium [Internet]. Seoul: CPX Consortium; 2004 [cited 2017 February 13]. Available from http://www.cpx.or.kr/CONSORTIUM/intro.jsp
18 Viera AJ, Garrett JM. Understanding interobserver agreement: The kappa statistic. Family Medicine. 2005;37(5):360-363.
19 Pell G, Fuller R, Homer M, Roberts T. How to measure the quality of the OSCE: A review of metrics-AMEE guide no. 49. Medical Teacher. 2010;32(10):802-811. https://doi.org/10.3109/0142159X.2010.507716   DOI
20 Goldstein SD, Lindeman B, Colbert-Getz J, Arbella T, Dudas R, Lidor A, et al. Faculty and resident evaluations of medical students on a surgery clerkship correlate poorly with standardized exam scores. American Journal of Surgery. 2014;207(2):231-235. https://doi.org/10.1016/j.amjsurg.2013.10.008   DOI
21 Park HK. Medical education using standardized patients. Hanyang Medical Reviews. 2012;32(1):35-44. https://doi.org/10.7599/hmr.2012.32.1.35   DOI
22 Kwon I, Kim N, Lee SN, Eo E, Park H, Lee DH, et al. Comparison of the evaluation results of faculty with those of standardized patients in a clinical performance examination experience. Korean Journal of Medical Education. 2005;17(2):173-183.   DOI
23 Park HK, Lee JK, Hwang HS, Lee JU, Choi YY, Kim H, et al. The agreement of checklist recordings between faculties and standardized patients in an objective structured clinical examination (OSCE). Korean Journal of Medical Education. 2003;15(2):141-150.   DOI
24 Armitage P, Berry G. Statistical methods in medical research. 3rd ed. London: Blackwell Science; 1994.
25 Hur J, Lee M, Chung W. Evaluation of checklist and inter-rater agreement in oral case presentation of undergraduate medical students. Ewha Medical Journal. 2007;30(1):47-52.   DOI
26 Lee BG. Current status of the standardized patient programs in Korea and plans to develop and improve the program. Korean Journal of Medical Education. 2001;13(1):27-39. https://doi.org/10.3946/kjme.2001.13.1.27   DOI
27 Kim BJ, Sung JJ, Park HK, Seo DW, Chung CS, Yoon BW. Clinical performance examination utilizing standardized patients in board examination: Based on the board examination of Korean Neurological Association for three years. Korean Journal of Medical Education. 2011;23(2):127-135. https://doi.org/10.3946/kjme.2011.23.2.127   DOI