• 제목/요약/키워드: Rater Reliability

검색결과 228건 처리시간 0.027초

Reliability and Validity of the Measurement of Pelvic Movement in Low Back Pain Patients using Cushion Sensor in Sitting Position (앉은 자세에서 방석센서를 이용한 요통환자 골반가동성 측정의 신뢰도와 타당도)

  • Jung, Seung-Hwa;Park, Dae-Sung
    • Journal of the Korean Society of Physical Medicine
    • /
    • 제15권2호
    • /
    • pp.83-91
    • /
    • 2020
  • PURPOSE: Postural and structural asymmetry due to muscle imbalances around the lower back and pelvis are the causes of back pain. Muscle imbalances in patients with chronic low back pain affect the pelvic tilt and movement, and it is necessary to assess the pelvic movement ability using the appropriate tools to determine the mediating effects of lower back pain. This paper reports the reliability and validity of the Sensbalance Therapy Cushion (STC) for pelvic movement and proprioception. METHODS: In this study, the Wii balance board (WBB) was used as a golden standard for pelvic movement measurements. FABQ, KODI, Myovision, and Pelvic movement were measured in 50 patients with chronic low back pain. The correlation between the lower-back muscle activity and pelvic movement was checked. The pelvic movement parameter was measured twice to determine the intra-rater reliability. RESULTS: The STC showed high test-retest reliability in the pelvic tilt measurements (ICC = .672 - .809). The test-retest reliability of proprioception measurements (ICC = .588 - .859) and reaction time measurements (ICC = .542 - .836) were also high. The relationship between the WBB and STC showed a significant positive correlation with the pelvic tilt test (p < .01). The posterior pelvic tilt and lower-back muscle activity showed a significant negative correlation (p < .01). The pelvic left tilt and lower-back muscle activity showed a significant negative correlation (p < .05). CONCLUSION: The results revealed the high reliability and validity of the STC. Therefore, the STC can be used as an objective measuring device for evaluating pelvic tilt, proprioception, and reaction time in low back pain patients.

The Development of Assessment Tool on Student's Character Competence based on Collaborative Problem-Solving Instruction Model (협력적 문제해결 중심 교수모델에 기반 한 학생 인성 역량 평가 도구 개발)

  • Jeon, RanYeong;Kim, HeeHwa;Nam, Jeonghee;Kang, EuGene;Son, Jeongwoo;Park, Jongseok
    • Journal of The Korean Association For Science Education
    • /
    • 제38권3호
    • /
    • pp.419-430
    • /
    • 2018
  • The purpose of this study is to develop an assessment tool to evaluate student character competence in applying a Collaborative Problem-Solving Instruction Model in science education. Through the literature analysis, nine elements of character are extracted. They are: openness, empathy, tolerance, caring, integrity, self-regulation, honesty, responsibility, and cooperation. Based on these existing measures of character competence, experts discussed and developed items for evaluation of a student's character competence based on the Collaborative Problem-Solving Instruction Model. The first 88 preliminary items were investigated, corrected, and supplemented based on the results of the first survey. A second validity survey was conducted on 71 middle- and high school science teachers to determine the content validity of the items. Inter-rater reliability was calculated for the assessors to verify the reliability of the items. Overall, the inter-rater reliability and content validity of the assessment items are good with 53 items ultimately being selected based on the analysis results. The assessment tool developed in this study could be used to explore changes in student character competence through a Collaborative Problem-Solving Instruction Model, as well as to evaluate student character competence in science education.

Rating criteria to evaluate student performance in digital wax-up training using multi-purpose software

  • Mino, Takuya;Kurosaki, Yoko;Tokumoto, Kana;Higuchi, Takaharu;Nakanoda, Shinichi;Numoto, Ken;Tosa, Ikue;Kimura-Ono, Aya;Maekawa, Kenji;Kim, Tae Hyung;Kuboki, Takuo
    • The Journal of Advanced Prosthodontics
    • /
    • 제14권4호
    • /
    • pp.203-211
    • /
    • 2022
  • PURPOSE. The aim of this study was to introduce rating criteria to evaluate student performance in a newly developed, digital wax-up preclinical program for computer-aided design (CAD) of full-coverage crowns and preliminarily investigate the reliability and internal consistency of the rating system. MATERIALS AND METHODS. This study, conducted in 2017, enrolled 47 fifth-year dental students of Okayama University Dental School. Digital wax-up training included a fundamental practice using computer graphics (CG), multipurpose CAD software programs, and an advanced practice to execute a digital wax-up of the right mandibular second molar (#47). Each student's digital wax-up work (stereolithography data) was evaluated by two instructors using seven qualitative criteria. The total qualitative score (0-90) of the criteria was calculated. The total volumetric discrepancy between each student's digital wax-up work and a reference prepared by an instructor was automatically measured by the CAD software. The inter-rater reliability of each criterion was analyzed using a weighted kappa index. The relationship between the total volume discrepancy and the total qualitative score was analyzed using Spearman's correlation. RESULTS. The weighted kappa values for the seven qualitative criteria ranged from 0.62 - 0.93. The total qualitative score and the total volumetric discrepancy were negatively correlated (ρ = -0.27, P = .09, respectively); however, this was not statistically significant. CONCLUSION. The established qualitative criteria to evaluate students' work showed sufficiently high inter-rater reliability; however, the digitally measured volumetric discrepancy could not sufficiently predict the total qualitative score.

The Reliability and Validity of Patient-Generated Subjective Global Assessment (PG-SGA) in Stroke Patients (뇌졸중 환자에서 '환자 주도적 총체적 영양사정' 도구의 신뢰도 및 타당도 평가)

  • Yoo, Sung-Hee;Oh, Eui-Guem;Youn, Mi-Jung
    • Korean Journal of Adult Nursing
    • /
    • 제21권6호
    • /
    • pp.559-569
    • /
    • 2009
  • Purpose: This study was to examine the reliability and validity of Patient-Generated Subjective Global Assessment (PG-SGA) as a nutritional measurement for stroke patients. Methods: This was a methodological study performed from May 6 to June 10, 2009 at a tertiary university hospital in Seoul. For reliability of PG-SGA, inter-rater reliability was used for statistics. For concurrent validity, BMI and biomarkers were compared between PG-SGA 0 ~ 8 and ${\geq}$ 9. In addition, sensitivity, specificity, and predictive value of PG-SGA compared with SGA were calculated using a contingency table. For predictive validity, hospital day, complications, and readmission within 1-month after discharge were compared between PG-SGA 0 ~ 8 and ${\geq}$ 9. Results: Correlation of PG-SGA score between two observers was 0.83, and kappa value for the agreement of severe malnutrition was 0.78(all $p_s$ < .001). The scored PG-SGA showed high sensitivity and specificity (100% and 96.7%, respectively). Severe undernourished patients (PG-SGA ${\geq}$ 9) had significantly low TLC, protein, albumin, and prealbumin (all $p_s$ < .01) compared with non-undernourished patients (PG-SGA 0 ~ 8). Also, in severe undernourished patients, complications and readmission (all $p_s$ = 0.01) were more often represented, and hospital days (p = .013) were significantly delayed. Conclusion: PG-SGA is a reliable and valid measurement to assess nutritional status for stroke patients.

  • PDF

Development of the Korean Version of the Social Functioning Scale in the Schizophrenics : A Study on the Reliability and Validity (정신분열병 환자의 한국어판 사회기능척도 개발 : 신뢰도 및 타당도 연구)

  • Kim, Chul-Kwon;Lee, Ji-Ah
    • Korean Journal of Biological Psychiatry
    • /
    • 제16권2호
    • /
    • pp.76-111
    • /
    • 2009
  • Objectives : The purpose of this study was to develop the Korean version of the Social Functioning Scale (KSFS) in the patients with schizophrenia. Methods : KSFS was administered to 90 schizophrenic patients and 80 their parents and 90 normal controls for examining the reliability and validity. Results : Data analysis showed statistically significant reliabilities and validities of KSFS. The test-retest reliability, rater vs. self-report reliability, and internal consistency for total scores of KSFS were 0.93, 0.44 and 0.94 respectively. Evidence for discriminant validity of KSFS comes from the results that the mean scores of schizophrenic patients were significantly higher than those of normal controls. Construct validity was assessed by calculating the 7 inter-areas correlations of the KSFS, and all areas were statistically significant. Significant correlations between the total scores of KSFS and those of SOFAS lend support for the concurrent validity of this instrument. Factor analyses were performed and two factors were extracted accounting for 63.7% of the variance. Sensitivity was assessed indirectly via the distribution and range of scores on the SFS. The normal control group showed a distribution around a higher mean with a moderate positive skew. Conclusion : KSFS was found to be a valid, reliable, and sensitive instrument which can be used to evaluate the degree of social functioning in the patients with schizophrenia.

  • PDF

INTERRATER RELIABILITY OF CARIES DIAGNOSIS BY LASER FLUORESCENCE (레이저형광 우식진단법의 검사자간 신뢰도에 관한 연구)

  • Lee, Kwang-Hee
    • Journal of the korean academy of Pediatric Dentistry
    • /
    • 제28권4호
    • /
    • pp.668-672
    • /
    • 2001
  • The purpose of study was to measure the interrater reliability of the caries diagnostic instrument, Diagnodent(KaVo, Germany), using a new laser fluorescence technique. Occlusal surfaces and smooth surfaces of one hundred extracted human premolars and molars were examined by four dentists. Pearson correlation coeffcients were 0703 to 0.870 for occlusal surfaces and 0.764 to 0.932 for smooth surfaces (P<0.01, all). Reliability coefficients which were calculated by intraclass correlation coefficients were 0.9450 for occlusal surfaces and 0.9605 for smooth surfaces. Therefore, the interrater reliability of caries diagnosis by Diagnodent is very high, and the rater training would be necessary more for occlusal surfaces than for smooth surfaces.

  • PDF

Korean Cultural Adaptation of WHODAS 2.0 (36-Item Version): Reliability and Linking to ICF

  • Song, Ju Min;Lee, Hae Jung
    • The Journal of Korean Physical Therapy
    • /
    • 제30권6호
    • /
    • pp.246-255
    • /
    • 2018
  • Purpose: This study was to conduct a Korean cultural adaptation of the WHO disability assessment schedule (WHODAS) 2.0: 36-item version. Methods: An internationally standardized process of translation and cultural adaptation of an instrument was used to develop a Korean version of WHODAS 2.0: 36-item version. Linking each item into the International Classification of Functioning, Disability and Health (ICF) was also conducted in order to ensure the concepts in the translated instrument were compatible with ICF. All translated versions of the instrument, linking results and feedback from participants were used for the final adaptation of the Korean version of the instrument. The Korean version of the instrument was assessed twice on different occasions to examine Inter- and intra-rater reliability, and the intra-class correlation coefficient (ICC) was calculated. Results: Twelve participants were involved in the translation and linking process. Ninety-five volunteers were invited to participate to examine the reliability of the instrument. Fifty participants completed the self-rated version of the instrument and 45 finished the interviewer version. The Korean WHODAS 2.0: 36-item version was found to have excellent reliability: self-rated version and interviewer version reliability coefficients were ICC=0.92 and ICC=0.94, respectively. Thirty-four items of the translated instrument were to be linked to ICF categories. Some adaptation was made; details and a familiar example were added to help respondents answer the questions. Conclusion: The study results show that the adaptation of the 36-item version of WHODAS 2.0 to Korean was successful and the instrument is ready for use in testing its psychometric properties.

A Method to Evaluate Rate of 'Soft-Hard' In a Drawing (그림의 '부드러운-딱딱한' 정도의 평가 방법)

  • Yoon, Seok-Hoon
    • Journal of the Korea Academia-Industrial cooperation Society
    • /
    • 제10권12호
    • /
    • pp.3963-3970
    • /
    • 2009
  • This study proposes a method to evaluate the level of 'soft-hard' of color quantitatively by evaluating the shape with edge sharpness automatically and by evaluating color in the color image scale in a drawing in art therapy using a computer. The dependent variable is the rank for the color experts to rate the level of 'soft-hard'. The mean and standard deviation of Value(V), and Chroma(C), colors, main color, clusters, length of edge, and sharp line rate of edge are considered as the independent variable. The appropriate independent variables to explain the dependent variable are selected through the step wise regression analysis. The inter-rater reliability of two raters is checked and the validity of developed system is verified by the rank correlations coefficient between the ranks of rater's and system's. This system can be used to evaluate of the shape or color in a drawing objectively and quantitatively for art therapy assessment, and to give the useful information to the fashion, textile, interior industry as well as color psychology and art therapy.

A Study of Evaluating Eye Movement Based on Ocular Motor Score for Children with Spastic Cerebral Palsy

  • Jeong, Seo-Young;Oh, Tae-Young
    • The Journal of Korean Physical Therapy
    • /
    • 제32권6호
    • /
    • pp.341-347
    • /
    • 2020
  • Purpose: The purpose of this study is to introduce a tool for evaluating eye movement and analyze the reliability of measurement based on 10 cases of evaluating the oculomotor function of children with spastic cerebral palsy Methods: The participants were selected by 6 medical and welfare institutions in Busan as GMFCS grade 1-3 among spastic diplegia and hemiplegia. Seven examiners evaluated 3 children for the evaluation of inter-rater agreement of Ocular Motor Score (OMS) and evaluated the condition of the ocular motor of 10 children using OMS, a re-examination was performed at six weeks after the initial examination. SPSS ver.25.0 was used to calculate the interclass correlation coefficient and Wilcoxon's signed-rank test, Mann Whitney-U test. Results: The inter-rater agreement of OMS was 0.89. Second mean values were decreased compared first examination in motility/ductions/version, saccades and smooth pursuit, but there was no significant difference. Children under 6 years old had a high mean value of saccades in first examination and the motility/ductions/version, fixation, saccades in second examination, but there was no significant difference. Spastic diplegia children's mean values were higher in head posture, fixation in 8 gaze directions than hemiplegia children in both first and second examination, but there were no significant differences. Conclusion: Ocular motor function in 10 children of spastic children who participated in the study and could see that the scores was differed depending on age, type, grade of cerebral palsy. OMS may be available for this purpose.

A Review of Published Instruments for Children and their Families in the Korean Journal of Child Studies (아동학회지에 게재된 측정도구 개발 및 타당화 연구 고찰: 1984-2016년)

  • Kim, Kyoung-Eun;Park, Ju Hee;Lim, Ji Young
    • Korean Journal of Child Studies
    • /
    • 제38권3호
    • /
    • pp.91-111
    • /
    • 2017
  • Objective: This study aimed to review scales and tests developed for or validated with children and their families that were published in the Korean Journal of Child Studies from 1984 to 2016. Specifically, the purpose of the present study was to analyze the contents and format of the selected instruments, as well as to evaluate their psychometric properties. Methods: Using several databases and journal archives from the Korean Journal of Child Studies, searches were implemented using the key terms: instrument, scale, development, and validation. Instruments from 76 selected studies were reviewed based on several characteristics, such as assessment areas, contents, respondents, responding types, and psychometric properties. Results and Conclusion: First, a majority of the reviewed instruments were developed for infants and children, whereas only one was developed for adolescents. With regard to their specific measurement domains, many instruments focused on social emotional development among children. Second, with a few exceptions, the selected studies provided appropriate evidence for the reliability of the instrument, including its internal consistency, inter-rater reliability, and spilt-half reliability. Many studies also reported on the criterion-related or construct validity of the instrument to establish its validity. Future studies need to develop instruments across diverse developmental areas that collect information from multiple sources and raters. In addition, more evidence on the reliability and validity of the reviewed instruments should be provided to demonstrate their psychometric qualities.