• Journal of Applied Reliability
• /
• v.18 no.2
• /
• pp.161-172
• /
• 2018

Purpose: Test & Evaluation items are main contents which verify performance of the development system and they are deduced going through project management system stages. However, there is no systematic or standardized framework for Test & Evaluation items, so a lot of differences occur between projects and people are questioning whether verification items are selected for the optimization of the development system. In this regard, this research proposes the process of deducing Test & Evaluation items in a systematic way during a development stage by applying QFD (Quality Function Deployment) technique which reflects customers' requirements. Methods: Test & Evaluation items are recognized as customers' requirements, and QFD technique which refines and gathers various requirements of customers effectively during general products' development stage is applied to complete HOQ (House of Quality) Conclusion: There is no specific methodology for Test & Evaluation items, so currently, they are selected by including items which are given by similar projects or higher authorities. However, by utilizing QFD technique, selection of evaluation items which goes through more systematic process is expected to be possible.

• Child Health Nursing Research
• /
• v.8 no.3
• /
• pp.334-343
• /
• 2002

This study was aimed at primarily to develop a tool to evaluate sexuality education and secondarily to test effectiveness through application of developed evaluation tools in elementary school. The results from this study were summarized below: 1. On the basis of targeting the lower grades (1st- 3th year) and the higher grades (4th-6th year) elementary school students' sexuality education guidebooks published by Korea Ministry of Education & Human Resources Development, 71 preliminary items targeting the lower grades, 90 preliminary items targeting the higher grades were developed. 2. Through the validity test about the contents of the preliminary items three times, the items were regulated to 65 items targeting the lower grads and 57 items targeting the higher grades. And then, the preliminary items were re-regulated to 40 items separately. Then, pre-test which targeting each 30 students was enforced. 3.Finally, the evaluation tools for sexuality education that consisted of 40 items targeting the lower grades and the higher grades were developed. 4. eliability test of the developed tools, sexuality education evaluation tools showed alpha coefficient of internal consistency were 0.8355 (for the use of the lower grades) and 0.8881 (for the use of the higher grades). 5. o apply the developed sexuality education evaluation tools, 10-times sexuality education were carried out class unit and pre-post test were done using same questionnaire, which contains developed tool, there were significant difference in low grade (t=16.548, p=0.000), high grade (t=14.773, p=0.000). The results of this study suggest that the evaluation tool for sexuality education in elementary school may be a useful tool with a high degree of reliability and validity. In this sense, the evaluation tool for sexuality education developed by this study can be effectively utilized in Korea elementary schools.

• Journal of Korean Dental Science
• /
• v.12 no.2
• /
• pp.58-65
• /
• 2019

Purpose: This study was aimed to investigate the possibility of utilizing smart device-based test (SBT) for competency evaluation in dental education and to analyze the student responses on overall competency evaluation using SBT method, in comparison to ubiquitous-based test (UBT). Materials and Methods: Questionnaire surveys have been conducted at Yonsei University College of Dentistry from 2015 to 2018 to obtain students' feedback on the application of SBT to competency evaluation. In addition, in order to supplement the competency evaluation procedure, considerations were explored by comparing the expected and actual difficulty of each item when preparing items for competency evaluation with SBT. Result: According to the survey results, student responses between the initial two years (2015 and 2016) differed from those in next two years (2017 and 2018). Students in 2017 and 2018 had more positive responses on competency evaluation with SBT. To determine the test validity, criterion-referenced evaluation was adopted to compare the data in 2017 and 2018 and slight differences in test difficulty in 2018 between the expected and actual difficulty of items were found. Conclusion: The results indicated that SBT was more appropriate for competency evaluation than UBT, based on four-year period of competency evaluation. The SBT was not affected by either the file size or the number of test-takers. Interestingly, students were not sensitive to test version of competency evaluation (paper-based test and SBT). This study suggests that the quality of the test items should be measured by continuous monitoring of the expected and actual difficulty of items for determining test validity. More detailed results and discussions of the findings are given for the development of test procedure and further potential research directions in dental education.

• Journal of The Korean Association For Science Education
• /
• v.22 no.2
• /
• pp.378-386
• /
• 2002

In this study, the students' responses on the chemistry items of in the general science of College Scholastics Ability Test (CSAT) implemented for the past 3 years since 1999 were investigated. The number of items by content and inquiry process, the average percent correct by content and inquiry process, the distribution of items by the level of percent correct, and the items with high and/or low percent correct were analysed. There were the fewest items in 'environment' area, especially in 'ozon layer', no test item had been made. The most difficult content area was 'acid rain' in 'environment'. By inquiry process, the most number of items belonged to 'analyzing & interpreting data', and 'identifying problems & formulating hypothesis' was the most difficult process. No test item came under the level of 'very difficult', and many items under the 'easy' or 'very easy' level. Students were generally poor at solving test items demanding several concepts, and very good at simply requiring basic concept treated in lower grade. Educational implications are discussed.

• Journal of The Korean Association For Science Education
• /
• v.30 no.4
• /
• pp.452-471
• /
• 2010

As the Korean College scholastic Ability Test (CSAT) has been implemented for 17 years since 1994, it is becoming more and more difficult to make new items that haven't been previously used to measure students' thinking ability. Therefore, it is necessary to keep conducting research on making new test items that can measure students' scholastic ability reliably. For this reason, multiple choice items on the Japanese university entrance exam, which is a Japanese National Center Test for University Admissions (NCTUA) equivalent of CSAT, were analyzed in order to draw implications for CSAT item development. In this study, we analyzed the Japanese NCTUA administered in January 2009 to investigate the structure of its science test. We also analyzed the NCTUA items by the domains of contents and behaviors, and tried to predict item difficulty from the perspective of Korean applicants. Major findings are as follows: Most NCTUA items measure understanding knowledge or low level thinking ability. Also the alloted time for each item is longer than CSAT. The number of test items, and the number of choice and alloted points for each item are diverse, unlike CSAT. The number of items using real-life materials are much more, but the items are not rigorous in sentence expression compared to CSAT. And the difference of difficulty level among science tests were larger with reference to CSAT. Also science score is required for most applicants regardless whether they are taking liberal arts or going onto the science track.

• The Korean Journal of Emergency Medical Services
• /
• v.24 no.3
• /
• pp.41-53
• /
• 2020

Purpose: This study tests the validity of a standard evaluation tool of ambulance ride practices with new evaluation items and supports its application by 119 preceptors. Methods: We tested the validity of standard evaluation items collected from 19 on-site professionals assessing ambulance ride practices. New evaluation items, 'understanding of emergency rescue equipment' and 'ability to communicate', were added as additional criteria. The modified Delphi technique was used to test the three evaluation areas and ten evaluation items. We used the analytic hierarchy process to analyze the weighting value of the reconstructed evaluation tool model. Results: All three evaluation areas and 10 evaluation items within the standard evaluation tools used for ambulance ride practices corresponded with the consistency index, degree of convergence, and agreement in the modified Delphi panel. Conclusion: These results provide evidence of the consistency and usefulness behind preceptors' use of this standard evaluation tool in ambulance ride practice.

• Journal for History of Mathematics
• /
• v.25 no.3
• /
• pp.119-140
• /
• 2012

This study analyzed the questions related to the content knowledge of mathematics subject for the middle and high school teacher employment examination conducted from the school year of 2009 to 2012 based on the evaluation scope of the mathematics subject content knowledge and evaluation contents which were presented by the Korea Institute for Curriculum and Evaluation(KICE). To achieve the objectives of this study, a total of 105 questions were collected with respect to the questions that appeared on the test over the last 4 years which aimed to evaluate the level of applicants' knowledge related to the contents of mathematics subject. The ratio of the contents covered in the test was assessed based on the scope of evaluation and the items for evaluation among the 9 subjects. Based on the results, suggestions were presented in relation to the operation of the mathematics curriculum for the department of mathematics education at the college of education or the restructuring of the evaluation scope. There was a significant difference in the ratio of items that appeared on the test among the 9 subjects related to the content knowledge of mathematics. Also, there was a remarkable difference in the ratio of items covered in the test among the evaluation scope by subject. The results of analysis on the evaluation content items suggested that 149 items out of 256 items did not appear in the teacher employment examination for 4 years. Based on such results of analysis, this study discussed the need for readjusting the ratio of items covered in the test of content knowledge related to mathematics or the evaluation scope and evaluation content items.

• Journal of The Korean Association For Science Education
• /
• v.33 no.6
• /
• pp.1202-1218
• /
• 2013

This study analyzed questions related to the content knowledge on the subject of chemistry for middle and high school teacher employment examinations conducted from the school years 2009 to 2013 based on the evaluation scope of the chemistry subject content knowledge and evaluation contents that were presented by the Korea Institute for Curriculum and Evaluation (KICE). To achieve the objectives of this study, a total of 140 questions were collected with respect to the questions that appeared on the test over the last 5 years, which aimed to evaluate the level of applicants' knowledge related to the contents of chemistry subject. The ratio of the contents covered in the test was assessed based on the scope of evaluation and the items for evaluation among the 4 subjects. Based on the results, suggestions were presented in relation to the operation of the chemistry curriculum for the Department of Chemistry Education at the college of education or the restructuring of the evaluation scope. There was a significant difference in the ratio of items that appeared on the test among the 4 subjects related to the content knowledge on chemistry. Also, there was a remarkable difference in the ratio of items covered in the test among the evaluation scope by subject. The results of the analysis on the evaluation content items suggested that 41 items out of 122 did not appear in the teacher employment examination for 5 years. Based on such results of analysis, this study discussed the need for readjusting the ratio of items covered in the test on content knowledge related to chemistry or the evaluation scope and evaluation content items.

• Journal of the Korean Operations Research and Management Science Society
• /
• v.28 no.4
• /
• pp.155-189
• /
• 2003

In this research, the comprehensive evaluation model of IS(information systems) for public sectors is theoretically constructed through reviewing and integrating previous IS evaluation-related research. This model is hypothesized to consist of five sectors(System Sector, User Sector, Organization and Management Sector, Resource Management Sector and Strategic Contribution Sector), ten evaluation items belonging to these five evaluation sectors, and sixty two evaluation indices for these ten evaluation items. The results of empirical analyses to test the validity of the research model show that the evaluation sectors turn out to be the same ones as proposed in the hypothetical model. The evaluation items(System Quality, Information Quality, Service Quality) in System Sector proves to be consistent with the hypothetical model. However, in case of User Sector, two evaluation items(Perceived Usefulness and User Satisfaction) which are included in the hypothetical model are found to be merged into one item(User Satisfaction). And, it is shown that four evaluation items such as the Efficiency of Internal Process, Customer Satisfaction, Organizational Innovation, and the Degree of Informatization competence improvement are included in Organization and Management Sector, differently from the research model in which the items, Organizational innovation and the Degree of Informatization competence improvement, are not divided. As a final result, sixty two evaluation indices that comprise all the evaluation items belonging to five sectors are found to be reduced to fifty seven ones through factor analyses, criterion-related validity tests and reliability analyses.

• Quality Improvement in Health Care
• /
• v.4 no.2
• /
• pp.260-278
• /
• 1997

Background : Quality of professional nursing care is the most essential factor for survival and growth of nursing profession. Then, nursing professionals have responsibility for the evaluation of quality of professional nursing care. The purpose of this study was to develope an evaluation tool of nursing care for patients received gastrointestinal surgery with general anesthesia. This study was a primary work for the developement of a computer program for the evaluation of nursing care. Methods : This study was done through some consecutive steps. They were (1) Developement of items for the tool (2) Developement of an evaluation tool of nursing care quality for the G-I surgery patient (3) Test of reliability and validity of the tool. Two groups of experts and expert pannels who had much experience of the QA and the care of G-I surgery patients participated for developement of the items. 85 nursing records were used for the test of reliability and validity of the developed tool. The evaluation tools were developed with two types of scoring, norm-referenced tool and criterion-referenced tool. Results The system of items for tool was evaluation area evaluation item-indicator. There were 7evaluation areas which contained 32evaluation items which contained 7lindicators. Evaluation areas 1, 2, 3, 4 were for the evaluation of process and 5, 6, 7 were for the evaluation of outcome of nursing care for G-I surgery patient. For the test of interrator reliability, correlation coefficients of each scores of items and intragroup correlation coefficients were calculated. The average correlation coefficients between two rators were 0.65, 0.54 and the intragroup correlation coefficient were 0.99 and 1.00 by the types of scoring. The Cronbach alpha coefficients of the tools were 0.54 and 0.46 by the types of scoring. The average content validity index of the items was 0.95 from 4 pairs of experts. Because there were significant differences between some scores of quality of nursing care of 3 general hospitals regardless of the types of scoring, the tools could be thought to have some construct validity. And also, there were significant correlations between some scores of quality of nursing care and admission days and admission days after surgery regardless of the types of scoring, the tools could be thought to have predictive validity. Conclusion In this study, the evaluation tool of nursing care was developed for the very specified group of patient, G-I surgery patient. And the items were developed and tested by the experts of nursing practice. Because of these reasons, it was supposed that the tool could be used effectively in nursing pratice. And the procedures for the development and the test of the evaluation tool of nursing care in this study were supposed to be used for the developement of other tools.