• 대한한의학회지
• /
• 제45권1호
• /
• pp.1-16
• /
• 2024

Introduction: The importance of clinical skills training in traditional Korean medicine education is increasingly emphasized. Since the clinical skills tests are high-stakes tests that determine success in national licensing exams, it is essential to develop reliable multifaceted analysis methods for clinical skills tests in actual education settings. In this study, we applied the multifaceted validity evaluation methods to the evaluation results of the cardiopulmonary resuscitation module to confirm the applicability and effectiveness of the methods. Methods: In this study, we used internal consistency, factor analysis, generalizability theory G-study and D-study, ANOVA, Kendall's tau, descriptive statistics, and other statistical methods to analyze the multidimensional validity of a cardiopulmonary resuscitation test in clinical education settings over the past three years. Results: The factor analysis and internal consistency analysis showed that the evaluation rubric had an unstable structure and low concordance. The G-study showed that the error of the clinical skills assessment was large due to the evaluator and unexpected errors. The D-study showed that the variance error of the evaluator should be significantly reduced to validate the evaluation. The ANOVA and Kendall's tau confirmed that evaluator heterogeneity was a problem. Discussion and Conclusion: Clinical skills tests should be continuously evaluated and managed for validity in two steps of pre-production and actual implementation. This study has presented specific methods for analyzing the validity of clinical skills training and testing in actual education settings. This study would contribute to the foundation for competency-based evidence-based education in practical clinical training.

• 한국직업건강간호학회지
• /
• 제26권1호
• /
• pp.1-9
• /
• 2017

Purpose: This study was conducted to evaluate the validity of the 'Enterprise Health Promotion Index', a tool for assessing health promotion activities of worksite developed by the Ministry of Employment and Labor and the Korea Occupational Safety and Health Agency. Methods: This is designed a methodological study that evaluates the validity of the Enterprise Health Promotion Index. For this study, a questionnaire survey was conducted for the workplace health professional and safety professional, and a total of 297 data were extracted as final data. We developed the informal health promotion needs and activity measurement tools. The correlation coefficient between the score of the enterprise health promotion index and the score of the informal health promotion tool was analyzed for the criterion validity evaluation. Results: The criterion validity of the Enterprise Health Promotion Index and activity index were supported by finding moderate (r=.597) and high (r=.783) correlation coefficient. However, the requirement index has low levels of criterion validity (r=.364). Conclusion: Based on these results, we suggest to improve the utility of the corporate health promotion index by developing user manual, active public relations, and providing briefing sessions.

• 공학교육연구
• /
• 제15권4호
• /
• pp.76-84
• /
• 2012

The purpose of this study is to develop an evaluation criteria for students' portfolios in an annual intramural contest. To accomplish the purpose, evaluation categories and their questions were developed after previous literature, ABEEK's evaluation guidelines, and eight Universities' evaluation forms were analyzed. Furthermore, experts' validity tests were completed by showing that the evaluation criteria form was validated with a mean score of 4.24. Also, the inter-rater reliability was turned out with a high score of .86. The interview results by evaluators of students' portfolios made the final version of evaluation criteria form include five evaluation criteria categories and their allotted score portions: overall organization(40), personal information(15), certificates and awards(5), curriculum(15), and extra-curriculum(15). The importance of overall organization was discussed from the perspectives of teachers as well as students.

• 수산해양교육연구
• /
• 제23권4호
• /
• pp.673-683
• /
• 2011

The purpose of this study was to analyze the validity of teacher evaluation policy for professional development and to suggest the ways to improve the policy in school. The teacher evaluation policy for professional development is designed to improve teacher professionalism and quality of school education. For this purpose, A survey was conducted and 361 elementary and middle school teachers participated in Busan. Collected data were analyzed using SPSS WIN 12.0. The conclusions of this study are as follows: First, although teachers are highly interested in the teacher evaluation policy, they have negative attitudes for practicing the new policy. Second, assessors must be composed of reliable, competent experts because teacher evaluation has a great influence on teaching activities. The assessors should be composed of those who have the specialization and reliability on the basis. Also the assessors must be trained to enhance the fairness, objectivity and reliability by rigid discipline. Finally, teacher evaluation policy should be closely linked to increasing teacher's expertise such as training opportunities, teacher consultants, and senior teacher systems and should be widely utilized for the evaluation to be effective.

• Journal of Nutrition and Health
• /
• 제44권6호
• /
• pp.562-576
• /
• 2011

The purpose of this study was to develop a reliable questionnaire to assess the effectiveness of the healthy restaurant program, which was implemented by Seocho-gu District in Seoul. Evaluation indices were deduced from the logic model of the healthy restaurant program. The questionnaires consisted of three sections (process evaluation, outcome evaluation, and general characteristics) for restaurateurs and customers who were participating in the program or not. To validate the questionnaire, 133 restaurateurs and 246 customers were sampled using convenient methods. Data were collected by interviews. Reliability was tested by Cronbach's alpha coefficient. The alpha coefficient for restaurateurs was 0.75-0.99 for the process and 0.79-0.97 for the outcome evaluation questions. The alpha coefficients for customers were 0.76-0.92 for the outcome evaluation questions. Face and content validity were examined for all questions, and construct validity was tested for latent variables. We are confident that more accurate and significant information will be collected using this questionnaire.

• 한국건축시공학회지
• /
• 제8권6호
• /
• pp.155-160
• /
• 2008

The BTL project, 4 years since its operation. has benchmarked the PFI Project in Japan and has been introduced. Given the evaluation step to select a preferred bidder, in a technological factor, the basic plans are corrected and complemented, whereas in a price factor, the low price bidding system is being enforced. There is concern that how to select preferred bidders and how to operate project costs during operation and management period may be problematic. Thus, in this study, using the Analytic Hierarchy Process, the method of deciding the priority to select preferred bidders in an early stage of the project and the evaluation model to evaluate the validity of BTL project expenses in process of project enforcement are established. Targeting the group composed of experts who have experiences in the BTL project. Then, the levelling of evaluation factors and grouping have been carried out as following : LCC analysis and disbursement for government including two more detailed factors.

• 대한인간공학회지
• /
• 제35권2호
• /
• pp.75-84
• /
• 2016

Objective: The aim of this study is to suggest some improvement ideas based on the validity and the reliability analyses of the current safety culture measurement method applied to the Korean nuclear power industry. Background: Wrong safety culture is known as one of the major causes of the disasters such as the space shuttle Columbia disaster or the Fukushima Nuclear Power Plant accident. Assessment of safety culture of an organization is important to build a safer organizational environment as well as to identify the risks hidden in the organization. Method: A face validity of the current safety culture measurement method was analyzed by comparison of the key factors of safety culture in the Korean nuclear power industry with those factors reviewed in the previous studies. The current interview method was analyzed to identify the problems which degrade the consistency of evaluation. Results: Most safety culture factors reviewed in the literatures are covered in the list of the Korean nuclear power industry safety culture factors. However the unstructured questions used in the interview may result in inconsistency of safety culture evaluation among interviewers. Conclusion: This study suggests some examples which might improve the consistency of interviewers' evaluation on safety culture such as a post interview evaluation form. Application: An extended post interview evaluation form might help to increase the accuracy of the interviewing method for Korean nuclear industry safety culture evaluation.

• 간호행정학회지
• /
• 제1권2호
• /
• pp.338-348
• /
• 1995

We need to continuously evaluate for nursing quality and the outcome to reflect nursing, to repair and improve of nursing. The first problem for nursing quality improvement is development of evaluation tool for the quality of nursing care. We need a system for quality assurance evaluation that is, development and framework of evaluation tool for nursing care quality. The purpose of this study was attempted to develop an evaluation tool for the quality of nursing care in musculoskeletal disorder patients. The approach method of nursing quality for the development of the tool in this study was process evaluation framework. The study were devided dvelopment process and reliability, validity verification process. The subjects of development process of this tool are three nursing export groups, and the subject of reliability, validity verification process are 20 samples of two university hospital in seoul, who were within discharge 3 months after admission treatment in musculo-skeletal disorder. Data for this study was collected from March 10 to April 13, 1995. The development process of the tool were as follows : 1. Make preliminary list of the tool by focal group were constituted 12 clinical nurses. 2. Modify and add preliminary list by 4 nursing expert panel. 3. Calculate content validity of the tool by 25 nursing expert panel of judge. 4. Verify reliability and validity of the tool. 5. Finalize an evaluation tool for the quality of the nursing care in musculo-skeletal disorder. The results of this study were as follows : 1) Development an evaluation tool for the quality of nursing care in musculo-skeletal disorder. (1) The evalution tool of this study was developed 5 standards, 33 criterias and 133 indicators. (2) 5 standards were divided according to Nursing Process. from standard 1 to standard 5, involved criterias were each 6(18%), 3(9%), 3(9%), 15(46%), 6(18%). 2) Verify reliability and validity of the tool. (1) Score of adequate degree for content validity of 33 criterias and 133 indicators were every average 2.82. (2) Inter-rater reliabilities(consentaneity score) of the tool by pearson correlation coefficient between three raters were : r=.7506, r=.8934, r=.6695. and Inter-rater reliabilities by single-facet crossed design were : r=.7464. (3) The alpha coefficient relating to internal consistency was .8524 over all 30 items of 33 criterias of developed tool. (4) Score of the quality of nursing care following to generaal characteristics of this study subjects were stastically significant differences according to educational level (F=2.93, p=.029)and diagnosing classification (F=2.50, p=.042). Through this study, I'm sure that the developed tool for the quality of patient care in musculo-skeletal disorder will show the way of more improvement of the quality of nursing care and effective nursing interventions.

• 동의신경정신과학회지
• /
• 제25권4호
• /
• pp.351-358
• /
• 2014

Objectives: This study was performed to evaluate the reliability and validity of the instrument of oriental medical evaluation for Hwa-byung. Methods: 97 participants diagnosed with Hwa-byung divided into 5 pattern identifications by the instrument on pattern identifications for Hwa-byung were tested by the instrument of oriental medical evaluation for Hwa-byung twice a week. Results: The intraclass correlation coefficient of the test-retest analysis of oriental medical evaluation for Hwa-byung had over moderate agreement. The intraclass correlation coefficient of test-retest analysis of evaluation index of symptoms had moderate agreement for the total symptoms, and fair agreement for the physical and mental symptoms. The validity analysis of the instrument represented a positive correlation in the pattern identification of stagnation of liver Gi, malfunction of gallbladder due to phlegm stagnation, and disharmony between heart and kidneys. Conclusions: The results showed that the reliability analysis of the instrument of oriental medical evaluation for Hwa-byung showed over moderate agreement, and validity analysis represented a positive correlation except for the pattern identification of flare-up of liver fire and deficiency of both Gi and blood.

• 교육시설 논문지
• /
• 제25권5호
• /
• pp.3-12
• /
• 2018

The purpose of this study is to examine the degree of appropriateness of using design quality indicators for the educational building, and complements for design quality indicators. For doing this, each items of the Korean educational building design indicator was validated by teachers, school staff and an education officer. First, School Personnel and Teachers considered indicators for the classroom designed for general subjects as being valid. However, in terms of indicators for the outside space, they thought of them as not being valid. In particular, it seemed that school staff would have a passive attitude toward the open of the school facility for the local society. It is judged that there happens an adverse effect caused by the open of the school facility for the local society in the educational environment of the school. Second, Education Office Supervisors thought that eco-friendly indicators, LED lighting, an outer wall, and widows and doors had a high validity. On the contrary, he viewed the validity of a rainwater retention basin as being low, which was likely because he rarely had expertise on the construction. When it comes to the indicator of technology capability, ultrahigh-speed information network was seen as being valid, and the indicator of economic feasibility had high validity on maintenance and durability. Third, compared to groups of Teachers and School Personnel, the group of education officers presented high validity of indicators. Validity of indicators might be differently showed because different items on indicators were measured. However, it is speculated that Education Office Supervisors mostly acknowledged the validity of indicators. Fourth, a majority of Teachers, School staff, and Education Office Supervisors were favor of the system of the evaluation accreditation for school facilities since it played a guide role in improving the quality of school. In order to settle the system of the evaluation accreditation for school facilities, it is most important that institutions in the local society which are publicly reliable should participate in the phase of designing the system of the evaluation accreditation for school facilities. In sum, overall respondents agreed with the system of the evaluation accreditation for school facilities but were aware that reliable organizations in public would need to take part in planning the system of the evaluation accreditation for school facilities from the beginning.