• 영어어문교육
• /
• 제12권1호
• /
• pp.221-238
• /
• 2006

This study investigates the reliability of plagiarism measurement in the ESL essay writing context. The current study aims to address the answers to the following research questions: 1) How does plagiarism measurement affect test reliability in a psychometric view? and 2) how do raters conceive the plagiarism in their analytic scoring? This study uses the mixed-methodology that crosses quantitative-qualitative techniques. Thirty eight international students took an ESL placement writing test offered by the University of Illinois. Two native expert raters rated students' essays in terms of 5 analytic features (organization, content, language use, source use, plagiarism) and made a holistic score using a scoring benchmark. For research question 1, the current study, using G-theory and Multi-facet Rasch model, found that plagiarism measurement threatened test reliability. For research question 2, two native raters and one non-native rater in their email correspondences responded that plagiarism was not a valid analytic area to be measured in a large-scale writing test. They viewed the plagiarism as a difficult measurement are. In conclusion, this study proposes that a systematic training program for avoiding plagiarism should be given to students. In addition, this study suggested that plagiarism is measured reliably in the small-scale classroom test.

• 한국수학교육학회지시리즈D:수학교육연구
• /
• 제10권1호
• /
• pp.55-70
• /
• 2006

The purpose of this study was to develop math creative problem solving test in order to identify the mathematically gifted on the basis of their math creative problem solving ability and evaluate the goodness of the test in terms of its reliability and validity of measuring creativity in math problem solving on the basis of fluency in producing valid solutions. Ten open math problems were developed requiring math thinking abilities such as intuitive insight, organization of information, inductive and deductive reasoning, generalization and application, and reflective thinking. The 10 open math test items were administered to 2,029 Grade 5 students who were recommended by their teachers as candidates for gifted education programs. Fluency, the number of valid solutions, in each problem was scored by math teachers. Their responses were analyzed by BIGSTEPTS based on Rasch's 1-parameter item-response model. The item analyses revealed that the problems were good in reliability, validity, difficulty, and discrimination power even when creativity was scored with the single criteria of fluency. This also confirmed that the open problems which are less-defined, less-structured and non-entrenched were good in measuring math creativity of the candidates for math gifted education programs. In addition, it discriminated applicants for two different gifted educational institutions and between male and female students as well.

• 대한간호학회지
• /
• 제46권1호
• /
• pp.109-117
• /
• 2016

Purpose: The aim of this study was to develop a Korean version of Night Eating Questionnaire (KNEQ) and test its psychometric properties and evaluate items according to item response theory. Methods: The 14-item NEQ as a measure of severity of the night eating syndrome was translated into Korean, and then this KNEQ was evaluated. A total of 1171 participants aged 20 to 50 completed the KNEQ on the Internet. To test reliability and validity, Cronbach's alpha, correlation, simple regression, and factor analysis were used. Each item was analyzed according to Rasch-Andrich rating scale model and item difficulty, discrimination, infit/outfit, and point measure correlation were evaluated. Results: Construct validity was evident. Cronbach's alpha was .78. The items of evening hyperphagia and nocturnal ingestion showed high ability in discriminating people with night eating syndrome, while items of morning anorexia and mood/sleep provided relatively little information. The results of item analysis showed that item2 and item7 needed to be revised to improve the reliability of KNEQ. Conclusion: KNEQ is an appropriate instrument to measure severity of night eating syndrome with good validity and reliability. However, further studies are needed to find cut-off scores to screen persons with night eating syndrome.

• 한국어교육
• /
• 제28권1호
• /
• pp.173-196
• /
• 2017

Writing is a subjective and performative activity. Writing ability has multi-facets and compoundness. To understand the examinees's writing ability accurately and provide effective writing scores, raters first ought to have the competency regarding assessment. Therefore, this study is significant as a fundamental research about rater's characteristics on the TOPIK writing assessment. 150 scripts of the 47th TOPIK examinees were selected randomly, and were further rated independently by 20 raters. The many-facet Rasch model was used to generate individualized feedback reports on each rater's relative severity and consistency with respect to particular categories of the rating scale. This study was analyzed using the FACETS ver 3.71.4 program. Overfit and misfit raters showed many difficulties for noticing the difference between assessment factors and interpreting the criteria. Writing raters appear to have much confusion when interpreting the assessment criteria, and especially, overfit and misfit teachers interpret the criteria arbitrarily. The main reason of overfit and misfit is the confusion about assessment factors and criteria in finding basis for scoring. Therefore, there needs to be more training and research is needed for raters based on this type of writing assessment characteristics. This study is recognized significantly in that it collectively examined writing assessment characteristics of writing raters, and visually confirmed the assessment error aspects of writing assessment.

• 한국전문물리치료학회지
• /
• 제19권3호
• /
• pp.91-97
• /
• 2012

The primary aim of this study was to compare responsiveness of self-report by worker and therapist-scored functional capacity instrument. Self-report and therapist-scored interval-level person measures and item difficulties were compared at admission and discharge. Therapist and worker ratings were collected on 230 clients from 27 rehabilitation sites using the newly developed Occupational Rehabilitation Data Base (ORDB) functional capacity instrument. ORDB comprises several subscales measuring relevant variables of "a return-to-work model" in work-related rehabilitation clinics. The functional capacity scale deals with 10 DOT job factors. The rating scale categories were 1-severely impaired, 2-moderately impaired, 3-mildly impaired, and 4-not impaired. Only data from clients with low back pain (n=98) with complete data (both admission and discharge scores) were used for the present study. Therapists and workers completed the functional capacity instrument at admission and discharge. Rasch analysis [1-parameter item response theory model (IRT)] was applied to calibrate item difficulty and person ability measure of therapist and workers ratings. Effect sizes for therapist and self-report ratings were slightly different, .69 and .30, respectively. Therapist and worker ratings were more consistent at discharge (r=.54) than at admission (r=.32). Workers have a tendency to be more severe in their ratings (show higher item difficulties) than therapists at admission and discharge. Therapists and workers report similar magnitudes of improvement following treatment program. These findings challenge the belief that injured workers may unreliable source for monitoring therapeutic outcomes. Self-report measures have the advantage of conserving therapist time for treatment (versus evaluation). While the therapist and self-report ratings are comparable at discharge, there is less consistency at admission. Comparable therapist-worker ratings may be achieved by controlling for rating severity using IRT methodologies.

• 대한불안의학회지
• /
• 제13권2호
• /
• pp.115-122
• /
• 2017

Objective : The experiences in close relationships questionnaire-revised (ECR-R) (Fraley, Waller & Brennan, 2000) is a valuable tool for measuring adult attachment, and its Korean version, the ECRR-K (Kim, 2004), is widely used in Korea. However, given its substantial length, this study was aimed to develop and validate a short version of the ECRR-K called the ECRR-K14. Methods : Two hundred and ninety-four medical students participated in this study in 2016. They completed the ECRR-K, the Perceived Stress Scale (PSS), the Rosenberg Self-Esteem Scale (RSES), and the UCLA Loneliness Scale (UCLA-LS). The study authors applied the Rasch rating scale to check each item's model fit and then performed confirmatory factor analyses (CFAs) to test the new scale's validity. Results : The authors selected seven items each for the anxiety and avoidance subscales, and the ECRR-K14 showed fair to good internal consistency (Cronbach's ${\alpha}=0.93$ and 0.92 for anxiety and avoidance, respectively). The anxiety subscale showed concurrent validity with the PSS and the RSES while the avoidance subscale showed concurrent validity with the UCLA-LS. The CFAs also demonstrated the validity of the model with a goodness-of-fit index of 0.916. Conclusion : The ECRR-K14 showed excellent reliability and validity and appears to be a promising instrument for measuring the two attachment dimensions in adults.

• 대한감각통합치료학회지
• /
• 제21권2호
• /
• pp.45-57
• /
• 2023

목적 : 이 연구의 목적은 학령기아동의 바른 자세유지를 위한 학교기반 웰니스 프로그램 적용 후, 아동의 자기보고식(self-report) 및 보호자의 대리보고식(proxy-report) 삶의 질 평가도구의 동시타당도를 검증하고자 하였다. 연구방법 : 학령기 아동 및 아동의 보호자 각 9명씩 총 18명을 표적집단으로 선정하여 연구대상으로 하였다. 초등학교 아동의 바른자세 유지하기 위한 웰니스 프로그램을 실시한 후 변화된 아동의 삶의 질에 대한 평가를 위해 한글판 KIDSCREEN-10 평가도구(아동용 및 보호자용)를 적용하였다. 라쉬 평정척도 모형을 적용하여, 문항의 적합도 및 난이도, 문항-대상자 도표 비교를 통하여 아동의 자기보고식 평가와 보호자 대리보고식 평가의 동시타당도를 검증하였다. 결과 : 아동의 자기보고식 평가에서는 자율성, 가정생활, 집중/배움, 또래집단/사회적 지지 4개문항, 보호자의 대리보고식 평가에서는 자아 인지적, 기분/정서적인 2개 문항이 적합도 기준을 벗어났다. 아동의 자기보고식 평가는 20점부터 50점 후반대에 분포하였고, 보호자의 대리보고식 평가는 30점 중반부터 50점 후반 영역에 주로 분포하여 비슷한 난이도 분포를 보였다. 아동과 보호자 평가의 상관관계분석결과, 스페어만 상관계수 p=.533(p>.05)으로 중간정도의 관련성을 보였으나 통계학적으로 유의하지 않았다. 아동은 자아인지적 문항을 비교적 쉬운 난이도로 인지하였으나(난이도 13.01), 보호자는 비교적 어려운 난이도 문항으로 인지하였다(난이도 46.21). 아동은 심리적, 신체적인 문항을 보호자보다 어렵게 인지하였고(난이도 각각 50.78, 50.78), 보호자는 아동보다 보다 쉽게 인지하는 반응을 보였다(난이도 각각 38.25, 34.88). 결론 : 향후 아동을 대상으로 하는 삶의 질 연구에서 신체적, 심리적, 자아인지 문항에서 아동과 보호자 평가의 차이점을 고려하여 아동의 삶의 질 평가가 이루어 져야 하겠다.

• 한국학교수학회논문집
• /
• 제9권3호
• /
• pp.405-424
• /
• 2006

본 연구는 수학 영재의 행동특성 검사 도구를 개발하는데 목적이 있다. 연구는 문헌 연구를 통해 수학 영재의 행동 특성을 추출하여 유목화한 후 측정변인으로 규정하였으며, 각각의 측정변인별로 문항을 개발하여 예비검사 과정을 통하여 최종적으로 51개 문항으로 구성된 검사 도구를 개발하였다. 기존에 영재교육을 받은 학생과 표준화된 수학 창의적 문제해결력에서 상의 10%와 교사 지명 학생 포함하여 424명을 연구 대상으로 본 검사를 실시하였다. 검사도구의 신뢰도와 타당도를 검증한 결과 높은 신뢰도(.95)를 확보하였으며 Rash 1모수 모형을 이용하여 내적 타당도를 검증하였다. 주성분 요인추출방법으로 요인을 추출하여 Varimax 방법으로 직각회전을 한 구인타당도 검증에서 일반적인 수학정신 능력 요인, 수학적 능력 요인, 정보 수집과 처리 능력 요인과 수학적 성향 요인이 추출되었다 따라서 본 연구에서 개발한 수학 영재 행동 특성 검사 도구는 신뢰도와 타당도가 양호하게 검증되었다고 볼 수 있다.

• 대한간호학회지
• /
• 제42권5호
• /
• pp.632-643
• /
• 2012

Purpose: This study was done to develop tool to evaluate the core competencies regarding nursing cancer patients on chemotherapy, and to verify the reliability and efficacy of the developed tool. Methods: A tool to evaluate the core competencies was developed from a preliminary tool consisting of 112 items verified by expert groups. The adequacy of the preliminary tool was analyzed and refined to the final evaluation tool containing 76 items in 8 core competencies and 18 specific competencies. The evaluation tool is in the form of a self-report, and each item is evaluated according to a 3-point scale. From September 22 to October 14, 2011, 349 survey responses were analyzed using SPSS 20.0 and the WINSTEPS program that employs the Rasch model. Results: Results indicated that there were no inappropriate items and the items had low levels of difficulty in comparison with the knowledge levels of the study participants. The results of factor analysis yielded 18 factors, and the reliability of the tools was very high with Cronbach's ${\alpha}$=.97. Conclusion: The results of this study can be used for training and evaluation of core competencies for nursing cancer patients, and for standardizing nursing practices associated with chemotherapy.

• 한국전문물리치료학회지
• /
• 제27권4호
• /
• pp.233-240
• /
• 2020

Background: Cross-culturally adapted questionnaires may not be comparable to their original version. Objects: To examine concurrent validity of two health-related quality of life (HRQOL) instruments for the Korean versions of EuroQOL-5 Dimension (EQ-5D) and the abbreviated version of the World Health Organization Quality of Life (WHOQOL-BREF) instrument. Methods: A total of 139 cancer survivors from two rehabilitation institutes was recruited. All participants were registered for palliative rehabilitation care. Both instruments were concurrently administered by health care providers following the second bout of the rehabilitation cares. Rasch partial credit model and Spearman's correlation analysis were used to investigate: 1) dimensionality, 2) hierarchical item difficulty, and 3) concurrent validity using correlations between two instruments. Results: For the WHOQOL-BREF, all items except negative feeling, pain, dependence of medical aid, were found to be acceptable, while all items of EQ-5D were acceptable. There was an evidence of negative correlations between EQ-5D and 4 domains of WHOQOL-BREF. Two correlations were strong (EQ-5D vs. physical health domain, ρ = -0.610, 95% CI = -0.716 to -0.475) and moderate (EQ-5D vs. psychosocial domain, ρ = -0.402, 95% CI = -0.546 to -0.236). Other two correlations were weak (EQ-5D vs. social relationship and environmental domains, ρ = -0.242, 95% CI = -0.401 to -0.075 and ρ = -0.364, 95% CI = -0.514 to -0.207, respectively). Item difficulty calibrations of the two measurements were ranged from -0.84 to 0.86 for the EQ-5D and -1.07 to 1.06 for the WHOQOL-BREF. Conclusion: The study provides some supports for the concurrent validity of the two Korean versions of HRQOL instrument, with evidences of weak to strong correlations between the EQ-5D and four domains of the WHOQOL-BREF applied to various cancer survivors. Additionally, the cancer survivors appeared to have more of a tendency to view the EQ-5D items as being slightly more challenging than the WHOQOL-BREF items.