• Korean Medical Education Review
• /
• v.25 no.2
• /
• pp.126-131
• /
• 2023

The aim of this study was to systematically collect data for evaluating short- and long-term outcomes using Kirkpatrick's four-level evaluation model, Chonnam National Medical School has established plans for developing and managing a database of student and graduate cohorts. The Education Evaluation Committee, with assistance from the Medical Education Office, manages the development and maintenance of cohort data. Data collection began in the 2022 academic year with first- through fourth-year medical students and graduates of the year 2022. The collected data include sociodemographic characteristics, admission information, psychological test results, academic performance data, extracurricular activity data, scholarship records, national medical licensing exam results, and post-graduation career paths. The Education Evaluation Committee and the Medical Education Office analyze the annually updated student and graduate cohort data and report the results to the dean and relevant committees. These results are used for admissions processes, curriculum improvement, and the development of educational programs. Applicants interested in using the student and graduate cohort data to evaluate the curriculum or conduct academic research must undergo review by the Educational Evaluation Committee before being granted access to the data. It is expected that the collected data from student and graduate cohorts will provide a sound and scientific basis for evaluating short- and long-term achievements based on student, school, and other characteristics, thereby supporting medical education policies, innovation, and implementation.

• Korean Medical Education Review
• /
• v.24 no.3
• /
• pp.250-260
• /
• 2022

Evaluating the effectiveness of educational programs involves measuring learning processes as well as outcomes. It is essential to study cohorts of students and graduates to evaluate the long-term effects of educational programs with data generated both during education and after graduation. The purpose of this study was to establish cohorts of students and graduates to evaluate their performance, thereby providing a basis for evaluating the social accountability of medical education. In this study, student and graduate cohorts were built for both students currently enrolled and graduates at Inje University College of Medicine (IUCM). A model involving the process of cohort establishment and an evaluation indicator framework was developed. In the process of cohort establishment, the following steps were conducted: defining the goals and objectives of the student and graduate cohorts, organizing a cohort committee, developing regulations, registering cohorts, acquiring consent, and building a database. A framework of evaluation indicators according to the graduate roles of IUCM was developed by adapting Kirkpatrick's evaluation model. Next, items to be collected in student and graduate cohorts were selected, and the current status of existing data was analyzed. Moreover, a preliminary analysis was conducted, including analyses of the evaluation indicators and graduates' performance. This study suggests that it is necessary to include additional evaluation indicators considering students' learning environment and well-being in student cohorts and to develop strategies or methods for graduates to continue participating in data collection for a long-term study.

• Journal of Korean society of Dental Hygiene
• /
• v.23 no.4
• /
• pp.277-286
• /
• 2023

Objectives: In this study, we aimed to develop a systematic tool that can evaluate the effects of interprofessional education (IPE) by applying four core competencies (values/ethics for interprofessional practice, roles/responsibilities, interprofessional communication, and teams and teamwork) and an educational evaluation model to evaluate the learning, behavior of learners, and results step by step. Methods: Previous studies on IPE evaluation tools were analyzed, and an evaluation tool (draft) was developed by modifying questions suitable for evaluation according to the Kirkpatrick model's stages and core competencies. The evaluation tool was completed by conducting a Delphi survey twice with 4-6 experts. To analyze the Delphi survey, the content validity index (CVI) was calculated, and the reliability coefficient (Cronbach's alpha coefficient) was used to measure reliability. Results: 29 questions on pre-education, consisting of learning and behavior stages, and 54 questions on post-education, consisting of reaction, learning, behavior, and results stages, were developed. The CVI and Cronbach's alpha coefficient values were >0.8 and >0.6, respectively. Conclusions: The IPE evaluation tool developed in this study is expected to contribute to the evaluation of the educational level of IPE and the identification of points for improvement when applied to various educational settings.

• Korean Management Science Review
• /
• v.32 no.4
• /
• pp.155-173
• /
• 2015

Koreatech has adopted a long-term co-op program called IPP (Industry Professional Practice) to address problems in higher education of Korea since 2012, but it was anticipated to face many difficulties in implementing the program due to lack of relevant experiences in Korea. In this regard, a performance evaluation scheme was urgently required to judge the effectiveness of the Co-op program and improve the operational efficiency at the same time. This study aimed to develop comprehensive performance evaluation criteria for the co-op programs on the basis of Kirkpatrick's four-level performance evaluation model for training programs, and apply it to a real Co-op operation to test its feasibility. For this purpose, thorough review on the training program evaluation literature and in-depth analyses of overseas cases of co-op performance evaluation were conducted. Then, a set of Co-op performance evaluation criteria was developed and applied to the Koreatech's Co-op operation in 2013. Two Co-op student questionnaire surveys were administered before and after Co-op terms to measure students' reactions to the Co-op program (Level 1) and changes in participating students' attitudes and competencies (Level 2). In addition, employment rates of Co-op participating and non-participating students were compared (Level 4). The analysis findings showed that participating students were quite satisfied with their jobs and companies, and the Co-op program was effective to a certain degree at enhancing non-major competencies and attitudes to occupation of students. Together, the employment rate of Co-op participants, particularly in small- and medium-sized companies, grew significantly in comparison with that of non-participants. In the last part, several directions for improving the effectiveness of the Co-op programs were discussed.

• Journal of Agricultural Extension & Community Development
• /
• v.4 no.1
• /
• pp.305-315
• /
• 1997

The propose of this study was to analyze the relationships among the levels of training & development evaluation (reaction, learning, transfer). The study has been conducted on 730 trainees who attended in the basic accounting program in L training and development institution through three incidents of tracked research such as reaction survey right after the conclusion of training, learning evaluation through test, and an evaluation of the transferability after 3 months of training. Questionnaires and test papers for analyses were used after their reliability, validity, difficulty, and discrimination have been verified on a pre-test. The research has been conducted for six months from 4 March 1996 to the end of August 1996, and data have been collected through direct research and survey through mail. The collected data have been worked on at SAS program for Windows with a statistical significance level of 5%. Statistical method that had been used was Pearson's correlation coefficient. The result and conclusion acquired from this study were as follows: Between reaction and learning, learning and transfer of training, only a weak positive correlation exists and explanation or prediction variance showing hierarchical relationship was quite weak with 1%. Thus, this research not only does not strongly support Kirkpatrick(1976)'s hierarchical model of $reaction{\rightarrow}learning{\rightarrow}transfer$, but also indicates that the separate measurement on each levels of training evaluation needs to be done. On the other hand, there was a relatively strong positive correlation between reaction and transfer of training. Based on the result, the conclusion, and the restriction perceived through this study, the following suggestions were made. 1. There is a need to empirically analyze and verify the hierarchy of all levels of training evaluation including the evaluation of the fourth level (result) such as organizational productivity, organizational satisfaction, and separation rate. 2. A great deal of efforts will be needed to systematically analyze what the relationships are among the methods measuring the level of evaluation of the training and development, and to apply this result to the training field.

• Korean Journal of Adult Nursing
• /
• v.17 no.5
• /
• pp.697-708
• /
• 2005

Purpose: This study was to develop, implement, and evaluate an e-learning education program for improving practical knowledge and preventing nursing errors and adverse events of nurses working in the operating room (OR). Method: The e-learning program was developed and evaluated according to the following processes: 1) preparation phase 2) implementation phase 3) evaluation phase. In evaluation phase, the effectiveness was analyzed based on the Kirkpatrick's model. Results: The e-learning program consisted of OR basic nursing skills and techniques and nursing activities' manual based on the categories of nursing errors: surgical operation preparation, nursing skills and techniques, environment management, patient safety and comfort, and patient monitoring. The program was provided through on-line, http://cafe.daum.net/pnuhorn, for 4 weeks. The mean score(percent) of participants' satisfaction was $21.24{\pm}1.71$(68.2%). Their total knowledge level was significantly improved(Z=-3.00, p=.003) and specifically in the category of environment management(Z=-3.77, p<.001) and patient monitoring(Z=-2.46, p=.014). The occurrence of nursing errors or adverse events was a little decreased, but not statistically significant(Z=-3.10, p=.756). Conclusion: E-learning for nurses is one way of effective and efficient teaching-learning strategies. For better e-learning, it is important to develop the vital content of the education and objective measures for detecting nursing errors and adverse events.

• The Journal of Korean Academic Society of Nursing Education
• /
• v.5 no.1
• /
• pp.118-132
• /
• 1999

To improve the effectiveness and efficiency of public health personnel training, we evaluated not only how appropriate the students felt the objectives, contents, methods and multimedia used in the train ing courses, but also how much the students accomplished the objectives and applied skill and knowledge to their own works. We selected 5 courses for the study : Tuberculosis control, Radiological technique, Public health information, Immunization, Mental health management courses used by Kirkpatrick's evaluation model. Reaction evaluation was carried out in final day by questionnaire. The results showed that all of them were very satisfied with educational input and curricula, learn Ing environment. Secondly, we measured the degree of learning achievement on pre and post training by questionaire of specific behavioral objectives. The degree of learning achievement was statistically higher just after training than pre training (paired t-test, p<0.01). Thirdly, evaluation of behavioral change to job was conducted to find out how much students applied skill and knowledge to their own job in 3 months after training by questionnaire. The results of behavioral change evaluation showed that 43.5% of the students who were performing job related with the training courses in 3 months after training applied the learned skill and knowledge to their own job quite well and 37.8% of them applied relatively well, therefore total 81.4% of them applied to their own job. And effectiveness of training for the above mentioned students showed that 41.9 % of them had improved or enforced their jobs after training, 35.5% of them had had no remarkable changes, and 15.7% had newly applied the learned skill and knowledge to their jobs. For evaluating the degree of usefulness of material predistribution in two weeks before training, we compared experimental groups with control groups. The results showed that general reactions are helpful but the degree of learning achievement is no discrepancy.