• Journal of Digital Convergence
• /
• v.19 no.12
• /
• pp.51-62
• /
• 2021

This study developed the factors capable of systematic/comprehensive evaluation of the task performance in order to strengthen the performance management of the professional career personnel invitation program (PCPIP). To this end, a performance evaluation framework was developed by analyzing existing project evaluation studies based on boundary theory and Kirkpatrick's four-level evaluation model. Afterwords, through two Delphi surveys, evaluation factors that can measure performance in terms of individual and invitation institutions of PCP were derived and validated. With this procedure, five evaluation factors were finally selected: adaptability, connectivity, clarity, compatibility, and expandability. This study has implications suggesting a performance evaluation factors capable of hybrid quantitative/qualitative evaluation for the performance management of PCPIP operated by National Research Foundation of Korea Research since 1994.

• Journal of Home Health Care Nursing
• /
• v.25 no.3
• /
• pp.240-248
• /
• 2018

Purpose: The purpose of this study was to evaluate the validity of an educational program developed for geriatric visiting nurses in Seoul by assessing their satisfaction level and level of learning goal achievement. Methods: This descriptive research study was conducted to develop, implement, and evaluate the educational program in accordance with the ADDIE Instructional System Model. Participants were 170 nurses hired for the 2016 Seoul Metropolitan Government visiting service for older people. Based on Kirkpatrick's Training Evaluation Model, reaction and learning evaluations were conducted during and after the educational program. Data were analyzed using descriptive statistics. Results: The developed educational program consisted of basic and professional courses. The evaluations showed that participants were highly satisfied with the lectures and field placement. Over 90% of the participants achieved the learning achievement goals. Conclusion: The program developed for geriatric visiting nurses in Seoul is considered a valid educational program because of the participants' high levels of satisfaction and academic achievement.

• Journal of Practical Engineering Education
• /
• v.12 no.1
• /
• pp.101-108
• /
• 2020

KOREATECH has operated a Co-op program called 'Industry Professional Practice (IPP)' that allows students to work full-time in relevant industries for at least four months since 2012, and also developed a systematic performance evaluation model on the basis of Kirkpatrick's four-stage assessment model. This study aims to share KOREATECH's Levels 3 and 4 evaluation results which are clearly what practitioners and academic investigators of cooperative education want to know the most. For this purpose, a questionnaire survey was conducted for KOREATECH graduates and their superiors to examine influence of IPP experiences on employment and job performance. A total of 730 alumni who graduated within last 5 years participated in the survey, with 213 (29.2%) having IPP experiences, and 162 superiors rated their subordinates on seven performance criteria. The analysis results were mixed. On the one hand, employment rate of IPP participants was 89.7%, compared to 86.8% of alumni without field experiences. Participants of the IPP program were more satisfied with their salary and felt less unfair about their career and promotion opportunities than alumni without field experiences. On the other, superiors rated their subordinates without IPP experiences slightly more positively than IPP participants in terms of job performance. These contrasting results are judged to show strengths and weaknesses of the IPP program at the same time. The limitations of the study and future research directions are discussed at the last part.

• Asian Pacific Journal of Cancer Prevention
• /
• v.16 no.2
• /
• pp.501-506
• /
• 2015

Background: To evaluate the effectiveness of the National Train-the-Trainers Program for Hospice and Palliative Care Experts (TTHPC) sponsored by the National Cancer Center of Korea between 2009 and 2012. This program was developed to improve the teaching skills of those in the field of hospice and palliative care (HPC). Materials and Methods: Training was offered in eight 1-day sessions between 2009 and 2012. The effect of the program was measured using Kirkpatrick's model of educational outcomes. First, levels 1 and 2 were evaluated immediately after the 1-day program (n=120). In 2012, the level-3 evaluation test was administered to trainers who offered at least one HPC training (n=78) as well as to their trainees (n=537). Results: The level-1 evaluation addressed participant reactions to and satisfaction with the program. Participants (n=120) were generally satisfied with the content, the method, and the overall course (mean range: 3.94-4.46 on a five-point Likert scale). The level-2 evaluation (learning) showed that participants gained knowledge and confidence related to teaching HPC (4.24 vs. 4.00). The level-3 evaluation (behavioral), which assessed trainers' application of teaching skills to HPC, showed that trainees rated the teaching methods of trainers (mean range: 4.03-4.08) more positively than did trainers (p<0.05). Female trainers were more likely than were male trainers to plan sessions in consideration of their trainees' characteristics (4.11 vs. 3.58; p<0.05), and nurse trainers were more likely than physician trainers to use a variety of instructional methods (4.05 vs. 3.36; p<0.05) Conclusions: We conducted systematic evaluations based on Kirkpatrick's model to assess the effectiveness of our train-the-trainers program. Our educational program was practical, effective, and followed by our HPC experts, who needed guidance to learn and improve their clinical teaching skills.

• Journal of Agricultural Extension & Community Development
• /
• v.8 no.2
• /
• pp.179-190
• /
• 2001

The purpose of this study was to analysis the education effects of web-based on-line cyber program mesaured by Kirkpatrick’s evaluation process. The average score on satisfaction of the program was 4.28(.59), which was designed to evaluate the level 1, reaction. To test level 2, learning, the average score that students achieved was calculated and it was 86.87(std.=7.05) in the term examinations. The level 3, job months. It was reported that most employees who took the course are utilizing the knowledge that they acquired from the course(mean=3.80, std.=.77). To identify the level 4, business results, the mean score of the number of accidents and near misses that happened in their factories for 3 months before and after the course were compared. There was statistically significant difference between the number of accidents that happened 3 months before the course and 3 months after the course, at the significance level of .01, which was tested by Paired t-test.

• Journal of Welding and Joining
• /
• v.33 no.3
• /
• pp.40-46
• /
• 2015

Welding is one of the most fundamental and necessary work in the industry that demand sophistication of skilled workers. This study is to introduce welding simulator as a training tool, to verify its effectiveness and to measure satisfaction of the trainees. A group of freshman students at a Korea Polytechnics College in their twenties with less experience of welding participated in the study. They were divided into two groups and took a traditional training course (comparison group) and a training course with welding simulator applied reality/haptic technology (experimental group) for same hours respectively. To evaluate training effect, a national certificate test and a survey based on Phillips' ROI (Return on Investment) methodology were conducted by the students and the college respectively. And satisfaction survey among the students based on Kirkpatrick's Four-Level Evaluation Model was also carried out. The results showed that all students in the experimental group passed the national certificate test and the ROI of the experimental group for five years were 110% higher than the comparison group. Furthermore, 25% more students in the experimental group replied "very satisfied" about the overall training course and 75% more students in the same group found that the simulation was very similar to the real welding.

• Korean Medical Education Review
• /
• v.25 no.2
• /
• pp.126-131
• /
• 2023

The aim of this study was to systematically collect data for evaluating short- and long-term outcomes using Kirkpatrick's four-level evaluation model, Chonnam National Medical School has established plans for developing and managing a database of student and graduate cohorts. The Education Evaluation Committee, with assistance from the Medical Education Office, manages the development and maintenance of cohort data. Data collection began in the 2022 academic year with first- through fourth-year medical students and graduates of the year 2022. The collected data include sociodemographic characteristics, admission information, psychological test results, academic performance data, extracurricular activity data, scholarship records, national medical licensing exam results, and post-graduation career paths. The Education Evaluation Committee and the Medical Education Office analyze the annually updated student and graduate cohort data and report the results to the dean and relevant committees. These results are used for admissions processes, curriculum improvement, and the development of educational programs. Applicants interested in using the student and graduate cohort data to evaluate the curriculum or conduct academic research must undergo review by the Educational Evaluation Committee before being granted access to the data. It is expected that the collected data from student and graduate cohorts will provide a sound and scientific basis for evaluating short- and long-term achievements based on student, school, and other characteristics, thereby supporting medical education policies, innovation, and implementation.

• Journal of Korean society of Dental Hygiene
• /
• v.23 no.4
• /
• pp.277-286
• /
• 2023

Objectives: In this study, we aimed to develop a systematic tool that can evaluate the effects of interprofessional education (IPE) by applying four core competencies (values/ethics for interprofessional practice, roles/responsibilities, interprofessional communication, and teams and teamwork) and an educational evaluation model to evaluate the learning, behavior of learners, and results step by step. Methods: Previous studies on IPE evaluation tools were analyzed, and an evaluation tool (draft) was developed by modifying questions suitable for evaluation according to the Kirkpatrick model's stages and core competencies. The evaluation tool was completed by conducting a Delphi survey twice with 4-6 experts. To analyze the Delphi survey, the content validity index (CVI) was calculated, and the reliability coefficient (Cronbach's alpha coefficient) was used to measure reliability. Results: 29 questions on pre-education, consisting of learning and behavior stages, and 54 questions on post-education, consisting of reaction, learning, behavior, and results stages, were developed. The CVI and Cronbach's alpha coefficient values were >0.8 and >0.6, respectively. Conclusions: The IPE evaluation tool developed in this study is expected to contribute to the evaluation of the educational level of IPE and the identification of points for improvement when applied to various educational settings.

• Korean Medical Education Review
• /
• v.24 no.3
• /
• pp.250-260
• /
• 2022

Evaluating the effectiveness of educational programs involves measuring learning processes as well as outcomes. It is essential to study cohorts of students and graduates to evaluate the long-term effects of educational programs with data generated both during education and after graduation. The purpose of this study was to establish cohorts of students and graduates to evaluate their performance, thereby providing a basis for evaluating the social accountability of medical education. In this study, student and graduate cohorts were built for both students currently enrolled and graduates at Inje University College of Medicine (IUCM). A model involving the process of cohort establishment and an evaluation indicator framework was developed. In the process of cohort establishment, the following steps were conducted: defining the goals and objectives of the student and graduate cohorts, organizing a cohort committee, developing regulations, registering cohorts, acquiring consent, and building a database. A framework of evaluation indicators according to the graduate roles of IUCM was developed by adapting Kirkpatrick's evaluation model. Next, items to be collected in student and graduate cohorts were selected, and the current status of existing data was analyzed. Moreover, a preliminary analysis was conducted, including analyses of the evaluation indicators and graduates' performance. This study suggests that it is necessary to include additional evaluation indicators considering students' learning environment and well-being in student cohorts and to develop strategies or methods for graduates to continue participating in data collection for a long-term study.

• Korean Management Science Review
• /
• v.32 no.4
• /
• pp.155-173
• /
• 2015

Koreatech has adopted a long-term co-op program called IPP (Industry Professional Practice) to address problems in higher education of Korea since 2012, but it was anticipated to face many difficulties in implementing the program due to lack of relevant experiences in Korea. In this regard, a performance evaluation scheme was urgently required to judge the effectiveness of the Co-op program and improve the operational efficiency at the same time. This study aimed to develop comprehensive performance evaluation criteria for the co-op programs on the basis of Kirkpatrick's four-level performance evaluation model for training programs, and apply it to a real Co-op operation to test its feasibility. For this purpose, thorough review on the training program evaluation literature and in-depth analyses of overseas cases of co-op performance evaluation were conducted. Then, a set of Co-op performance evaluation criteria was developed and applied to the Koreatech's Co-op operation in 2013. Two Co-op student questionnaire surveys were administered before and after Co-op terms to measure students' reactions to the Co-op program (Level 1) and changes in participating students' attitudes and competencies (Level 2). In addition, employment rates of Co-op participating and non-participating students were compared (Level 4). The analysis findings showed that participating students were quite satisfied with their jobs and companies, and the Co-op program was effective to a certain degree at enhancing non-major competencies and attitudes to occupation of students. Together, the employment rate of Co-op participants, particularly in small- and medium-sized companies, grew significantly in comparison with that of non-participants. In the last part, several directions for improving the effectiveness of the Co-op programs were discussed.