• Journal of Korean Academic Society of Home Health Care Nursing
• /
• v.25 no.3
• /
• pp.240-248
• /
• 2018

Purpose: The purpose of this study was to evaluate the validity of an educational program developed for geriatric visiting nurses in Seoul by assessing their satisfaction level and level of learning goal achievement. Methods: This descriptive research study was conducted to develop, implement, and evaluate the educational program in accordance with the ADDIE Instructional System Model. Participants were 170 nurses hired for the 2016 Seoul Metropolitan Government visiting service for older people. Based on Kirkpatrick's Training Evaluation Model, reaction and learning evaluations were conducted during and after the educational program. Data were analyzed using descriptive statistics. Results: The developed educational program consisted of basic and professional courses. The evaluations showed that participants were highly satisfied with the lectures and field placement. Over 90% of the participants achieved the learning achievement goals. Conclusion: The program developed for geriatric visiting nurses in Seoul is considered a valid educational program because of the participants' high levels of satisfaction and academic achievement.

• Journal of Gifted/Talented Education
• /
• v.23 no.2
• /
• pp.133-160
• /
• 2013

The purposes of this research are to develop and to evaluate the teacher training program for enhancing scientific creative problem-solving abilities. For this purpose, by considering previous studies, this study suggested 'scientific creative problem solving process'. In the course of elementary teacher program development, the present study followed Instructional Systems Development stages, where Kirkpatrick's four-step evaluation model was applied for a quality of evaluation. As a result of evaluation, it was found that teachers' recognition of competency to teach the scientific creative problem solving process was increased. In addition, teachers' evalution of the program was positive and their willingness to apply it to the field was found high, which indicated that the training program's applicability to schools would be positive.

• Journal of Korean society of Dental Hygiene
• /
• v.23 no.4
• /
• pp.277-286
• /
• 2023

Objectives: In this study, we aimed to develop a systematic tool that can evaluate the effects of interprofessional education (IPE) by applying four core competencies (values/ethics for interprofessional practice, roles/responsibilities, interprofessional communication, and teams and teamwork) and an educational evaluation model to evaluate the learning, behavior of learners, and results step by step. Methods: Previous studies on IPE evaluation tools were analyzed, and an evaluation tool (draft) was developed by modifying questions suitable for evaluation according to the Kirkpatrick model's stages and core competencies. The evaluation tool was completed by conducting a Delphi survey twice with 4-6 experts. To analyze the Delphi survey, the content validity index (CVI) was calculated, and the reliability coefficient (Cronbach's alpha coefficient) was used to measure reliability. Results: 29 questions on pre-education, consisting of learning and behavior stages, and 54 questions on post-education, consisting of reaction, learning, behavior, and results stages, were developed. The CVI and Cronbach's alpha coefficient values were >0.8 and >0.6, respectively. Conclusions: The IPE evaluation tool developed in this study is expected to contribute to the evaluation of the educational level of IPE and the identification of points for improvement when applied to various educational settings.

• Journal of the Korea Society of Computer and Information
• /
• v.17 no.10
• /
• pp.185-192
• /
• 2012

Call Center requires an ability of agents a lot more than face-to-face contact due to being achieved communication by non face-to face channel for contact with customers. In order to improve the ability of agents, Call Center carries out various educational training according to their work experience and function and with the accomplishment of educational training, Call Center is going to fulfill to develop its quality of counseling and productivity. On the other hand, due to investment of a lot of time and budget to educational training, it is needed to grasp and manage about its effectiveness that how helpful the training is for performance of work-site operations through evaluation of educational training. Having Seen researches about evaluation of educational training until these days, most researches have mainstream to measure satisfaction and a level of learning or degree that how the learning transfers to actions. It is found that a research about an entire evaluation model should be required. This study aims to investigate effectiveness of Call Center educational training from the level of recognition by reflecting Kirkpatrick's the four levels of learning evaluation. By the four levels, reaction, learning, behavior and results, the study found out a connection with standards of evaluation about each levels. In addition, by using structural equation modeling, it was examined goodness of fit about the entire model. Furthermore, by an alternative model, considering a direct relation between a factor of reaction and behavior, it was compared and examined goodness of fit of overall model of the study model and the alternative one.

• Journal of Practical Engineering Education
• /
• v.12 no.1
• /
• pp.101-108
• /
• 2020

KOREATECH has operated a Co-op program called 'Industry Professional Practice (IPP)' that allows students to work full-time in relevant industries for at least four months since 2012, and also developed a systematic performance evaluation model on the basis of Kirkpatrick's four-stage assessment model. This study aims to share KOREATECH's Levels 3 and 4 evaluation results which are clearly what practitioners and academic investigators of cooperative education want to know the most. For this purpose, a questionnaire survey was conducted for KOREATECH graduates and their superiors to examine influence of IPP experiences on employment and job performance. A total of 730 alumni who graduated within last 5 years participated in the survey, with 213 (29.2%) having IPP experiences, and 162 superiors rated their subordinates on seven performance criteria. The analysis results were mixed. On the one hand, employment rate of IPP participants was 89.7%, compared to 86.8% of alumni without field experiences. Participants of the IPP program were more satisfied with their salary and felt less unfair about their career and promotion opportunities than alumni without field experiences. On the other, superiors rated their subordinates without IPP experiences slightly more positively than IPP participants in terms of job performance. These contrasting results are judged to show strengths and weaknesses of the IPP program at the same time. The limitations of the study and future research directions are discussed at the last part.

• Korean Medical Education Review
• /
• v.24 no.3
• /
• pp.250-260
• /
• 2022

Evaluating the effectiveness of educational programs involves measuring learning processes as well as outcomes. It is essential to study cohorts of students and graduates to evaluate the long-term effects of educational programs with data generated both during education and after graduation. The purpose of this study was to establish cohorts of students and graduates to evaluate their performance, thereby providing a basis for evaluating the social accountability of medical education. In this study, student and graduate cohorts were built for both students currently enrolled and graduates at Inje University College of Medicine (IUCM). A model involving the process of cohort establishment and an evaluation indicator framework was developed. In the process of cohort establishment, the following steps were conducted: defining the goals and objectives of the student and graduate cohorts, organizing a cohort committee, developing regulations, registering cohorts, acquiring consent, and building a database. A framework of evaluation indicators according to the graduate roles of IUCM was developed by adapting Kirkpatrick's evaluation model. Next, items to be collected in student and graduate cohorts were selected, and the current status of existing data was analyzed. Moreover, a preliminary analysis was conducted, including analyses of the evaluation indicators and graduates' performance. This study suggests that it is necessary to include additional evaluation indicators considering students' learning environment and well-being in student cohorts and to develop strategies or methods for graduates to continue participating in data collection for a long-term study.

• Journal of Agricultural Extension & Community Development
• /
• v.8 no.2
• /
• pp.179-190
• /
• 2001

The purpose of this study was to analysis the education effects of web-based on-line cyber program mesaured by Kirkpatrick’s evaluation process. The average score on satisfaction of the program was 4.28(.59), which was designed to evaluate the level 1, reaction. To test level 2, learning, the average score that students achieved was calculated and it was 86.87(std.=7.05) in the term examinations. The level 3, job months. It was reported that most employees who took the course are utilizing the knowledge that they acquired from the course(mean=3.80, std.=.77). To identify the level 4, business results, the mean score of the number of accidents and near misses that happened in their factories for 3 months before and after the course were compared. There was statistically significant difference between the number of accidents that happened 3 months before the course and 3 months after the course, at the significance level of .01, which was tested by Paired t-test.

• Asian Pacific Journal of Cancer Prevention
• /
• v.16 no.2
• /
• pp.501-506
• /
• 2015

Background: To evaluate the effectiveness of the National Train-the-Trainers Program for Hospice and Palliative Care Experts (TTHPC) sponsored by the National Cancer Center of Korea between 2009 and 2012. This program was developed to improve the teaching skills of those in the field of hospice and palliative care (HPC). Materials and Methods: Training was offered in eight 1-day sessions between 2009 and 2012. The effect of the program was measured using Kirkpatrick's model of educational outcomes. First, levels 1 and 2 were evaluated immediately after the 1-day program (n=120). In 2012, the level-3 evaluation test was administered to trainers who offered at least one HPC training (n=78) as well as to their trainees (n=537). Results: The level-1 evaluation addressed participant reactions to and satisfaction with the program. Participants (n=120) were generally satisfied with the content, the method, and the overall course (mean range: 3.94-4.46 on a five-point Likert scale). The level-2 evaluation (learning) showed that participants gained knowledge and confidence related to teaching HPC (4.24 vs. 4.00). The level-3 evaluation (behavioral), which assessed trainers' application of teaching skills to HPC, showed that trainees rated the teaching methods of trainers (mean range: 4.03-4.08) more positively than did trainers (p<0.05). Female trainers were more likely than were male trainers to plan sessions in consideration of their trainees' characteristics (4.11 vs. 3.58; p<0.05), and nurse trainers were more likely than physician trainers to use a variety of instructional methods (4.05 vs. 3.36; p<0.05) Conclusions: We conducted systematic evaluations based on Kirkpatrick's model to assess the effectiveness of our train-the-trainers program. Our educational program was practical, effective, and followed by our HPC experts, who needed guidance to learn and improve their clinical teaching skills.

• Korean Journal of Adult Nursing
• /
• v.17 no.5
• /
• pp.697-708
• /
• 2005

Purpose: This study was to develop, implement, and evaluate an e-learning education program for improving practical knowledge and preventing nursing errors and adverse events of nurses working in the operating room (OR). Method: The e-learning program was developed and evaluated according to the following processes: 1) preparation phase 2) implementation phase 3) evaluation phase. In evaluation phase, the effectiveness was analyzed based on the Kirkpatrick's model. Results: The e-learning program consisted of OR basic nursing skills and techniques and nursing activities' manual based on the categories of nursing errors: surgical operation preparation, nursing skills and techniques, environment management, patient safety and comfort, and patient monitoring. The program was provided through on-line, http://cafe.daum.net/pnuhorn, for 4 weeks. The mean score(percent) of participants' satisfaction was $21.24{\pm}1.71$(68.2%). Their total knowledge level was significantly improved(Z=-3.00, p=.003) and specifically in the category of environment management(Z=-3.77, p<.001) and patient monitoring(Z=-2.46, p=.014). The occurrence of nursing errors or adverse events was a little decreased, but not statistically significant(Z=-3.10, p=.756). Conclusion: E-learning for nurses is one way of effective and efficient teaching-learning strategies. For better e-learning, it is important to develop the vital content of the education and objective measures for detecting nursing errors and adverse events.

• Korean Management Science Review
• /
• v.32 no.4
• /
• pp.155-173
• /
• 2015

Koreatech has adopted a long-term co-op program called IPP (Industry Professional Practice) to address problems in higher education of Korea since 2012, but it was anticipated to face many difficulties in implementing the program due to lack of relevant experiences in Korea. In this regard, a performance evaluation scheme was urgently required to judge the effectiveness of the Co-op program and improve the operational efficiency at the same time. This study aimed to develop comprehensive performance evaluation criteria for the co-op programs on the basis of Kirkpatrick's four-level performance evaluation model for training programs, and apply it to a real Co-op operation to test its feasibility. For this purpose, thorough review on the training program evaluation literature and in-depth analyses of overseas cases of co-op performance evaluation were conducted. Then, a set of Co-op performance evaluation criteria was developed and applied to the Koreatech's Co-op operation in 2013. Two Co-op student questionnaire surveys were administered before and after Co-op terms to measure students' reactions to the Co-op program (Level 1) and changes in participating students' attitudes and competencies (Level 2). In addition, employment rates of Co-op participating and non-participating students were compared (Level 4). The analysis findings showed that participating students were quite satisfied with their jobs and companies, and the Co-op program was effective to a certain degree at enhancing non-major competencies and attitudes to occupation of students. Together, the employment rate of Co-op participants, particularly in small- and medium-sized companies, grew significantly in comparison with that of non-participants. In the last part, several directions for improving the effectiveness of the Co-op programs were discussed.