• Journal of the Korea Society of Computer and Information
• /
• v.17 no.10
• /
• pp.185-192
• /
• 2012

Call Center requires an ability of agents a lot more than face-to-face contact due to being achieved communication by non face-to face channel for contact with customers. In order to improve the ability of agents, Call Center carries out various educational training according to their work experience and function and with the accomplishment of educational training, Call Center is going to fulfill to develop its quality of counseling and productivity. On the other hand, due to investment of a lot of time and budget to educational training, it is needed to grasp and manage about its effectiveness that how helpful the training is for performance of work-site operations through evaluation of educational training. Having Seen researches about evaluation of educational training until these days, most researches have mainstream to measure satisfaction and a level of learning or degree that how the learning transfers to actions. It is found that a research about an entire evaluation model should be required. This study aims to investigate effectiveness of Call Center educational training from the level of recognition by reflecting Kirkpatrick's the four levels of learning evaluation. By the four levels, reaction, learning, behavior and results, the study found out a connection with standards of evaluation about each levels. In addition, by using structural equation modeling, it was examined goodness of fit about the entire model. Furthermore, by an alternative model, considering a direct relation between a factor of reaction and behavior, it was compared and examined goodness of fit of overall model of the study model and the alternative one.

• Journal of Agricultural Extension & Community Development
• /
• v.8 no.2
• /
• pp.179-190
• /
• 2001

The purpose of this study was to analysis the education effects of web-based on-line cyber program mesaured by Kirkpatrick’s evaluation process. The average score on satisfaction of the program was 4.28(.59), which was designed to evaluate the level 1, reaction. To test level 2, learning, the average score that students achieved was calculated and it was 86.87(std.=7.05) in the term examinations. The level 3, job months. It was reported that most employees who took the course are utilizing the knowledge that they acquired from the course(mean=3.80, std.=.77). To identify the level 4, business results, the mean score of the number of accidents and near misses that happened in their factories for 3 months before and after the course were compared. There was statistically significant difference between the number of accidents that happened 3 months before the course and 3 months after the course, at the significance level of .01, which was tested by Paired t-test.

• Journal of Welding and Joining
• /
• v.33 no.3
• /
• pp.40-46
• /
• 2015

Welding is one of the most fundamental and necessary work in the industry that demand sophistication of skilled workers. This study is to introduce welding simulator as a training tool, to verify its effectiveness and to measure satisfaction of the trainees. A group of freshman students at a Korea Polytechnics College in their twenties with less experience of welding participated in the study. They were divided into two groups and took a traditional training course (comparison group) and a training course with welding simulator applied reality/haptic technology (experimental group) for same hours respectively. To evaluate training effect, a national certificate test and a survey based on Phillips' ROI (Return on Investment) methodology were conducted by the students and the college respectively. And satisfaction survey among the students based on Kirkpatrick's Four-Level Evaluation Model was also carried out. The results showed that all students in the experimental group passed the national certificate test and the ROI of the experimental group for five years were 110% higher than the comparison group. Furthermore, 25% more students in the experimental group replied "very satisfied" about the overall training course and 75% more students in the same group found that the simulation was very similar to the real welding.

• Journal of Practical Engineering Education
• /
• v.12 no.1
• /
• pp.101-108
• /
• 2020

KOREATECH has operated a Co-op program called 'Industry Professional Practice (IPP)' that allows students to work full-time in relevant industries for at least four months since 2012, and also developed a systematic performance evaluation model on the basis of Kirkpatrick's four-stage assessment model. This study aims to share KOREATECH's Levels 3 and 4 evaluation results which are clearly what practitioners and academic investigators of cooperative education want to know the most. For this purpose, a questionnaire survey was conducted for KOREATECH graduates and their superiors to examine influence of IPP experiences on employment and job performance. A total of 730 alumni who graduated within last 5 years participated in the survey, with 213 (29.2%) having IPP experiences, and 162 superiors rated their subordinates on seven performance criteria. The analysis results were mixed. On the one hand, employment rate of IPP participants was 89.7%, compared to 86.8% of alumni without field experiences. Participants of the IPP program were more satisfied with their salary and felt less unfair about their career and promotion opportunities than alumni without field experiences. On the other, superiors rated their subordinates without IPP experiences slightly more positively than IPP participants in terms of job performance. These contrasting results are judged to show strengths and weaknesses of the IPP program at the same time. The limitations of the study and future research directions are discussed at the last part.

• Korean Medical Education Review
• /
• v.25 no.2
• /
• pp.126-131
• /
• 2023

The aim of this study was to systematically collect data for evaluating short- and long-term outcomes using Kirkpatrick's four-level evaluation model, Chonnam National Medical School has established plans for developing and managing a database of student and graduate cohorts. The Education Evaluation Committee, with assistance from the Medical Education Office, manages the development and maintenance of cohort data. Data collection began in the 2022 academic year with first- through fourth-year medical students and graduates of the year 2022. The collected data include sociodemographic characteristics, admission information, psychological test results, academic performance data, extracurricular activity data, scholarship records, national medical licensing exam results, and post-graduation career paths. The Education Evaluation Committee and the Medical Education Office analyze the annually updated student and graduate cohort data and report the results to the dean and relevant committees. These results are used for admissions processes, curriculum improvement, and the development of educational programs. Applicants interested in using the student and graduate cohort data to evaluate the curriculum or conduct academic research must undergo review by the Educational Evaluation Committee before being granted access to the data. It is expected that the collected data from student and graduate cohorts will provide a sound and scientific basis for evaluating short- and long-term achievements based on student, school, and other characteristics, thereby supporting medical education policies, innovation, and implementation.

• Journal of the Ergonomics Society of Korea
• /
• v.35 no.4
• /
• pp.237-245
• /
• 2016

Objective: This study aims to introduce the bridge resource management (BRM) training courses implemented in Korea and to analyze their effectiveness in several views. Background: BRM training will be a mandatory course for crew members of ships from 2017. At this stage, it is needed to check if the BRM training courses implemented until now was effective to the Korean maritime safety and to investigate if there are more effective training methods. Method: The effectiveness of BRM training intervention in Korea was compared with that of the other countries, using Kirkpatrick's (1976) training evaluation framework. Kim (2012)'s data on the BRM training effectiveness were re-analyzed in order to check if the effects of BRM training are dependent on the bridge work experience. Results: Many BRM training courses has been opened in Korea. However, the methods to assess BRM training effects used in Korea focused on the survey of subjective satisfaction level, not investigating trainees' attitude and behavior change. On the other hand, the effectiveness of BRM training was higher to the bridge officers with long work experience than with shorter work experience. Conclusion: The contents of BRM training should be changed to effectively apply to the context of the real-world exercise and be differentiated depending on the work experience. Research on the methods to measure the BRM training effectiveness is also more required. Application: The results of this study will aid to develop the BRM training courses for bridge officers of ships in the BRM training institutions.

• Korean Management Science Review
• /
• v.32 no.4
• /
• pp.155-173
• /
• 2015

Koreatech has adopted a long-term co-op program called IPP (Industry Professional Practice) to address problems in higher education of Korea since 2012, but it was anticipated to face many difficulties in implementing the program due to lack of relevant experiences in Korea. In this regard, a performance evaluation scheme was urgently required to judge the effectiveness of the Co-op program and improve the operational efficiency at the same time. This study aimed to develop comprehensive performance evaluation criteria for the co-op programs on the basis of Kirkpatrick's four-level performance evaluation model for training programs, and apply it to a real Co-op operation to test its feasibility. For this purpose, thorough review on the training program evaluation literature and in-depth analyses of overseas cases of co-op performance evaluation were conducted. Then, a set of Co-op performance evaluation criteria was developed and applied to the Koreatech's Co-op operation in 2013. Two Co-op student questionnaire surveys were administered before and after Co-op terms to measure students' reactions to the Co-op program (Level 1) and changes in participating students' attitudes and competencies (Level 2). In addition, employment rates of Co-op participating and non-participating students were compared (Level 4). The analysis findings showed that participating students were quite satisfied with their jobs and companies, and the Co-op program was effective to a certain degree at enhancing non-major competencies and attitudes to occupation of students. Together, the employment rate of Co-op participants, particularly in small- and medium-sized companies, grew significantly in comparison with that of non-participants. In the last part, several directions for improving the effectiveness of the Co-op programs were discussed.

• Journal of Agricultural Extension & Community Development
• /
• v.4 no.1
• /
• pp.305-315
• /
• 1997

The propose of this study was to analyze the relationships among the levels of training & development evaluation (reaction, learning, transfer). The study has been conducted on 730 trainees who attended in the basic accounting program in L training and development institution through three incidents of tracked research such as reaction survey right after the conclusion of training, learning evaluation through test, and an evaluation of the transferability after 3 months of training. Questionnaires and test papers for analyses were used after their reliability, validity, difficulty, and discrimination have been verified on a pre-test. The research has been conducted for six months from 4 March 1996 to the end of August 1996, and data have been collected through direct research and survey through mail. The collected data have been worked on at SAS program for Windows with a statistical significance level of 5%. Statistical method that had been used was Pearson's correlation coefficient. The result and conclusion acquired from this study were as follows: Between reaction and learning, learning and transfer of training, only a weak positive correlation exists and explanation or prediction variance showing hierarchical relationship was quite weak with 1%. Thus, this research not only does not strongly support Kirkpatrick(1976)'s hierarchical model of $reaction{\rightarrow}learning{\rightarrow}transfer$, but also indicates that the separate measurement on each levels of training evaluation needs to be done. On the other hand, there was a relatively strong positive correlation between reaction and transfer of training. Based on the result, the conclusion, and the restriction perceived through this study, the following suggestions were made. 1. There is a need to empirically analyze and verify the hierarchy of all levels of training evaluation including the evaluation of the fourth level (result) such as organizational productivity, organizational satisfaction, and separation rate. 2. A great deal of efforts will be needed to systematically analyze what the relationships are among the methods measuring the level of evaluation of the training and development, and to apply this result to the training field.

• Korean Journal of Adult Nursing
• /
• v.17 no.5
• /
• pp.697-708
• /
• 2005

Purpose: This study was to develop, implement, and evaluate an e-learning education program for improving practical knowledge and preventing nursing errors and adverse events of nurses working in the operating room (OR). Method: The e-learning program was developed and evaluated according to the following processes: 1) preparation phase 2) implementation phase 3) evaluation phase. In evaluation phase, the effectiveness was analyzed based on the Kirkpatrick's model. Results: The e-learning program consisted of OR basic nursing skills and techniques and nursing activities' manual based on the categories of nursing errors: surgical operation preparation, nursing skills and techniques, environment management, patient safety and comfort, and patient monitoring. The program was provided through on-line, http://cafe.daum.net/pnuhorn, for 4 weeks. The mean score(percent) of participants' satisfaction was $21.24{\pm}1.71$(68.2%). Their total knowledge level was significantly improved(Z=-3.00, p=.003) and specifically in the category of environment management(Z=-3.77, p<.001) and patient monitoring(Z=-2.46, p=.014). The occurrence of nursing errors or adverse events was a little decreased, but not statistically significant(Z=-3.10, p=.756). Conclusion: E-learning for nurses is one way of effective and efficient teaching-learning strategies. For better e-learning, it is important to develop the vital content of the education and objective measures for detecting nursing errors and adverse events.

• Safety and Health at Work
• /
• v.9 no.2
• /
• pp.172-179
• /
• 2018

Background: Insufficient training in infection control and occupational health among healthcare workers (HCWs) in countries with high human immunodeficiency virus (HIV) and tuberculosis (TB) burdens requires attention. We examined the effectiveness of a 1-year Certificate Program in Occupational Health and Infection Control conducted in Free State Province, South Africa in an international partnership to empower HCWs to become change agents to promote workplace-based HIV and TB prevention. Methods: Questionnaires assessing reactions to the program and Knowledge, Attitudes, Skills, and Practices were collected pre-, mid-, and postprogram. Individual interviews, group project evaluations, and participant observation were also conducted. Quantitative data were analyzed using Wilcoxon signed-rank test. Qualitative data were thematically coded and analyzed using the Kirkpatrick framework. Results: Participants recruited (n = 32) were mostly female (81%) and nurses (56%). Pre-to-post-program mean scores improved in knowledge (+12%, p = 0.002) and skills/practices (+14%, p = 0.002). Preprogram attitude scores were high but did not change. Participants felt empowered and demonstrated attitudinal improvements regarding HIV, TB, infection control, and occupational health. Successful projects were indeed implemented. However, participants encountered considerable difficulties in trying to sustain improvement, due largely to lack of pre-existing knowledge and experience, combined with inadequate staffing and insufficient management support. Conclusion: Training is essential to strengthen HCWs' occupational health and infection control knowledge, attitudes, skills, and practices, and workplace-based training programs such as this can yield impressive results. However, the considerable mentorship resources required for such programs and the substantial infrastructural supports needed for implementation and sustainability of improvements in settings without pre-existing experience in such endeavors should not be underestimated.