• 수산해양교육연구
• /
• 제27권6호
• /
• pp.1546-1555
• /
• 2015

This study was performed to analyze the item characteristics of competency inventory for designer (CID), which Gil (2011) developed for measurement of design competency. To accomplish the purpose of this study, general partial credit (GPC) model based on polytomous item response theory was applied. The findings were as follows: First, CID is a reliable instrument for measuring design competency. Second, most items of CID have low item discrimination and average item difficulty according to the GPC model. Especially, there are some problems to have low item discrimination in view of validation. To improve the goodness of CID, we will need to examine why CID has low item discrimination.

• Journal of Korean Biological Nursing Science
• /
• 제20권1호
• /
• pp.11-19
• /
• 2018

Purpose: The purposes of this study were to perform items analysis using the classical test theory (CTT) and the item response theory (IRT), and to establish the validity and reliability of the Korean version of pressure ulcer prevention knowledge. Methods: The 26-item pressure ulcer prevention knowledge instrument was translated into Korean, and the item analysis of the 22 items having an adequate content validity index (CVI), was conducted. A total of 240 registered nurses in 2 university hospitals completed the questionnaire. Each item was analyzed applying CTT and IRT according to 2-parameter logistic model. Response alternatives quality, item difficulty and item discrimination were evaluated. For testing validity and reliability, Pearson correlation coefficient and Kuder Richardson-20 (KR-20) were used. Results: Scale CVI was .90 (Item-CVI range= .75-1.00). The total correct answer rate for this study population was relatively low as 52.5%. The quality of response alternatives was found to be relatively good (range= .02-.83). The item difficulty of the questions ranged form .10 to .86 according to CTT and -12.19 to 29.92 according to the IRT. This instrument had 12-low, 2-medium and 8-high item difficulty applying IRT. The values for the item discrimination ranged .04-.57 applying CTT and .00-1.47 applying IRT. And overall internal consistency (KR-20) was .62 and stability (test-retest) was .82. Conclusion: The instrument had relatively weak construct validity, item discrimination according to the IRT. Therefore, the cautious usage of a Korean version of this instrument would be recommended for discrimination because there are so many attractive response alternatives and low internal consistency.

• 한국보육지원학회지
• /
• 제19권6호
• /
• pp.39-54
• /
• 2023

Objective: This study aimed to develop a parent play interaction observation scale (PPIOS-Toddler) and analyze it in terms of item discrimination, reliability, and validity. Methods: The subjects of the study were 97 toddlers and mothers. This scale consisted of three categories, six domains and 22 items on a 5-point scale. For the item discrimination of the observation scale, an independent standard t-test was conducted to analyze the significant difference in average between the upper and lower groups for each item. The reliability of the observation scale was calculated by Cronbach's α, the intra-item agreement, and the validity was examined through content validity, the correlation between subdomains and total scores, and official validity using PICCOLO. Results: In item discrimination analysis, all items exhibited differences between upper and lower groups. The overall internal agreement for the observation scale was 0.95, with factor-specific internal agreement ranging from 0.83 to 0.95. The observation scale demonstrated notable correlations between total scores and sub-factors (0.45 to 0.93) and significant correlations with PICCOLO total scores (0.66 to 0.86). Conclusion/Implications: The study successfully verified the item discrimination, reliability, and validity of the Parent Play Interaction Observation Scale (PPIOS-Toddler).

• 영어어문교육
• /
• 제18권2호
• /
• pp.189-211
• /
• 2012

The purpose of the study was to explore ways to improve the test items on the Korean scholastic ability test. More specifically, the researchers investigated whether use of the target language in test items would make a difference in total scores, discriminatory power, and item difficulty. A total of 288 high school seniors participated in the study. The subjects were divided into the experimental group (N=145) and the control group (N=143). A 25-item test resembling the Korean scholastic ability test was administered to both groups. The experimental group was given items whose questions and alternatives were all presented in English, whereas the control group was given items whose questions and alternatives were presented in Korean only. Statistical analyses revealed that use of English vs. Korean in the questions and alternatives made a significant difference in total scores, item discrimination, and item difficulty level. The findings strongly suggest that use of English is one way to improve the quality of the Korean scholastic ability test by enhancing item discrimination and face validity. Considering that the test in question is a high-stakes exam in Korea, further research on how to improve the Korean scholastic ability test is urgently called for.

• 한국응급구조학회지
• /
• 제28권2호
• /
• pp.7-19
• /
• 2024

Purpose: To adjust item numbers in a national test, this study used item response theory to examine changes in average scores, reliability, difficulty, and discrimination according to the adjustment of item numbers. Methods: We analyzed the dichotomous coding of correct and incorrect answers of 473 examinees in a mock test conducted in 2023. Additionally, as an explanatory pilot study, we used an online questionnaire to survey experts on their perceptions of the appropriate item numbers for each test subject from January 18, 2024, to February 15, 2024. Results: Regarding the item numbers on the national exam, experts preferred to reduce the number of management of emergency patients (33.14±6.09, p<.05) and advanced emergency medical care: subtopics (104.49±11.55, p<.05), and the total number of questions (217.82±20.95, p<.05). In a simulation set in which items with low item fit were removed after fitting a two-parameter item response theory model, reliability was maintained at .910 until the 5th test consisting of 185 questions with little loss of difficulty, discrimination, and average score, and there was no correlation between item numbers and average score. Conclusion: Experts responded that reducing the number of items on the national exam was appropriate. As a result of the item reduction simulation, there was no significant loss in the average score, difficulty, discrimination, or reliability. More reliable results could be obtained if the results were based on a validity analysis and analyzed using actual national exams.

• 대한본초학회지
• /
• 제37권2호
• /
• pp.13-21
• /
• 2022

Objectives : The evaluation of academic achievement is pivotal for establishing accurate direction and adequate level of medical education. The purpose of this study was to firstly establish innovative item analysis technique of Item Response Theory (IRT) for analyzing multiple-choice test of herbology in the traditional Korean medicine education which has not been available for the difficulty of test theory and statistical calculation. Methods : The answers of 390 students (2012-2018) to the 14 item herbology test in college of Korean medicine were used for the item analysis. As for the multidimensional analysis of item characteristics, difficulty, discrimination, and guessing parameters along with item-total correlation and percentage of correct answer were calculated using Classical Test Theory (CTT) and IRT. Results : The validity parameters of strong and weak items were illustrated in multiple perspectives. There were 4 items with six acceptable index scores, and 5 items with only one acceptable index score. The item discrimination of IRT was found to have no significant correlation with difficulty and discrimination indices of CTT emphasizing attention of professionals of medical education as for the test credibility. Conclusion : The critical suggestions for the development, utilization and revision of test items in the e-learning and evidence-based Teaching era were made based on the results of item analysis using IRT. The current study would firstly provide foundation for upgrading the quality of Korean medicine education using test theory.

• 아동학회지
• /
• 제37권6호
• /
• pp.35-56
• /
• 2016

Objective: This study aimed to develop and validate the Childcare Adaptation Scale for Infants and Toddlers (CASIT), which is rated by teachers of Korean children. Methods: The participants consisted of 326 childcare teachers working with infants (ages 0-2 years). Content validity, discriminant validity, convergent validity, concurrent validity, internal consistency, inter-rater reliability, and item discrimination were examined using PASW 18.0 and AMOS 19.0. Results: The results of an exploratory factor analysis identified the 29-item scale and six dimensions of the scale, including group life adaptation, negative behaviors, positive affect, regular routines, activity/interest, and peer interaction. Convergent validity was examined via confirmatory factor analysis, average variation extracted (AVE), and construct reliability, and acceptable evidences of convergent validity was established. The scales were shown to be highly consistent internally and among raters. Also, the mean between the upper group and lower group of each item regarding item discrimination showed a significant difference. Conclusion: It was concluded that the CASIT, which is a quick and convenient tool for teachers to use, is a valid and reliable instrument.

• 음성과학
• /
• 제13권2호
• /
• pp.7-18
• /
• 2006

An aphasia screening test can serve a main purpose of differentiating aphasics from non-aphasic patients in a quick as well as efficient manner. As a preliminary study for developing a standardized aphasia screening test for Korean patients, we constructed an aphasia screening test constituting items from the Paradise' Korean version-the Western Aphasia Battery(P K-WAB). All test items were analyzed in order to extract items with optimal item discrimination and adequate item difficulty indices. From the results, we were able to select some items from each subtest with optimal results of discriminant function analysis for aphasic and normal control groups. It is expected, thus, that information on the item analysis could be utilized in developing a Korean aphasia screening test.

• 대한치과기공학회지
• /
• 제44권3호
• /
• pp.104-110
• /
• 2022

Purpose: This study analyzes the difficulty and discrimination of the paper-based test (PBT) and the computer-based test (CBT) according to item response theory, focusing on the National Dental Technician Examination. Methods: A mock test was conducted from September 15 to 23, 2020, and the final 179 (1 out of 180 absentees)people were the subjects of this study. Both frequency analysis and factor analysis were performed. The collected data were analyzed using IBM SPSS Statistics ver. 18.0 (IBM) and jMetrik programs. The significance level was set to 0.05. Results: The difficulty of the mock test was more easily responded to in CBT. It was also predicted that the CBT could better measure the ability of test takers than the PBT could. Conclusion: The difficulty, discrimination, and reliability of the questions were not affected by the examination method through the mock test. The feasibility of a future change to the CBT was confirmed by the National Dental Technician Examination.

• 컴퓨터교육학회논문지
• /
• 제9권6호
• /
• pp.11-18
• /
• 2006

본 연구는 2005년, 2006년 대학수학능력시험 직업탐구영역의 "컴퓨터 일반" 문항을 질적 연구와 양적연구로 분석하였다. 질적 연구로 내용 타당도 검사를 실시하고, 양적연구로 2-모수 문항 반응 모형에 근거한 베이지안(Bayesian) 1.0을 이용하여 문항의 난이도 및 변별도를 측정하였으며 고전검사이론 프로그램인 테스트안(Testan) 1.0을 이용하여 문항의 신뢰도 및 오답지를 분석하였다. "컴퓨터 일반" 영역의 문항들을 분석하여 문항의 질을 높이고 난이도의 정확한 예측을 하여 보다 신뢰도와 변별력 있는 문항을 개발하기 위한 자료로 제시하고자 한다.