• Title/Summary/Keyword: Rater Reliability

Search Result 230, Processing Time 0.024 seconds

A Review of Published Instruments for Children and their Families in the Korean Journal of Child Studies (아동학회지에 게재된 측정도구 개발 및 타당화 연구 고찰: 1984-2016년)

  • Kim, Kyoung-Eun;Park, Ju Hee;Lim, Ji Young
    • Korean Journal of Child Studies
    • /
    • v.38 no.3
    • /
    • pp.91-111
    • /
    • 2017
  • Objective: This study aimed to review scales and tests developed for or validated with children and their families that were published in the Korean Journal of Child Studies from 1984 to 2016. Specifically, the purpose of the present study was to analyze the contents and format of the selected instruments, as well as to evaluate their psychometric properties. Methods: Using several databases and journal archives from the Korean Journal of Child Studies, searches were implemented using the key terms: instrument, scale, development, and validation. Instruments from 76 selected studies were reviewed based on several characteristics, such as assessment areas, contents, respondents, responding types, and psychometric properties. Results and Conclusion: First, a majority of the reviewed instruments were developed for infants and children, whereas only one was developed for adolescents. With regard to their specific measurement domains, many instruments focused on social emotional development among children. Second, with a few exceptions, the selected studies provided appropriate evidence for the reliability of the instrument, including its internal consistency, inter-rater reliability, and spilt-half reliability. Many studies also reported on the criterion-related or construct validity of the instrument to establish its validity. Future studies need to develop instruments across diverse developmental areas that collect information from multiple sources and raters. In addition, more evidence on the reliability and validity of the reviewed instruments should be provided to demonstrate their psychometric qualities.

Clinical utility of auditory perceptual assessments in the discrimination of a diplophonic voice (이중음성 판별에 있어 청지각적 평가의 임상적 유용성)

  • Bae, Inho;Kwon, Soonbok
    • Phonetics and Speech Sciences
    • /
    • v.10 no.1
    • /
    • pp.75-81
    • /
    • 2018
  • Diplophonia is generally defined as the perception of more than one fundamental frequency component in a voice. Its perceptual aspect has traditionally been used to evaluate diplophonia because the perceptions can be easily evaluated, but there are limitations in the validity of the reliability of the intra- and inter-raters, examination situation, and variation of voice sample. Therefore, the purpose of this study is to confirm the reliability and accuracy of auditory perceptual evaluation by comparing non-invasive indirect assessment methods (sound waveform and EGG analysis), and to identify their usefulness with diplophonia. A total of 28 diplophonic voices and 39 non-periodic voices were assessed. Three raters assessed the diplophonia by performing an auditory perception evaluation and identifying the quasi-periodic perturbations of the acoustic waveform and EGG. Among the three discrimination methods, intra- and inter-rater reliability, sensitivity, specificity, accuracy, positive likelihood ratio, and negative likelihood ratio were examined, and the McNemar test was performed to compare the discriminant agreement. The accuracy of the auditory perceptual evaluation (86.57%) was not significantly different from that of sound waveform acoustic (88.06%), but it was significantly different from that of EGG (83.33%). The reading time (6.02 s) for the auditory perceptual evaluation was significantly different from that for sound waveform analysis (30.15 s) and EGG analysis (16.41 s). In the discrimination of diplophonia, auditory perceptual evaluation has sufficient reliability and accuracy as compared to sound waveform and EGG. Since immediate feedback is possible, auditory perceptual evaluation is more convenient. Therefore, it can continue to be used as a tool to discriminate diplophonia in clinical practice.

The Clinical Application of modified Emory Functional Ambulation Profile for Chronic Stroke Patients (만성 편마비 환자의 modified Emory Functional Ambulation Profile의 임상 적용)

  • Kim, Seong-Yeol;Lee, Je-Hoon;An, Seung-Heon
    • Journal of the Korean Society of Physical Medicine
    • /
    • v.5 no.4
    • /
    • pp.655-666
    • /
    • 2010
  • Purpose : The examine the reliability and validity of the modified Emory Functional Ambulation Profile(mEFAP) for assessing gait function in chronic stroke patients. Methods : A total of 45 stroke patients, who had a stroke more than 6 months, participated in the study. Reliability was determined by Intra-class Correlation Coefficient($ICC_{3,1}$), including Bland and Altman method (Standard Error of Measurement: SEM, Small Real Differences: SRD). Validity was examined by correlating results to the gait ability(mEFAP, Modified Motor Assessment Scale-Gait(MMAS-G), Scandinavian Stroke Scale-Gait(SSS-G), Functional Ambulation Category(FAC), 10m Waking Test(10m WT)), and Fugl Meyer-Lower/Extremity(FM-L/E), Berg Balance Scale(BBS). Results : Inter-rater reliability for the total mEFAP was High($ICC_{2,1}$=.998), and absolute reliability were excellent (SEM: 1.75, SRD: 4.85). Subjects without assistance factor performed better on all tests than did subjects who had stroke. There were significant correlations between the mEFAP and MMAS-G, SSS-G, FAC(r=-.66~-.79), 10 m WT(r=-.86), and FM-L/E, BBS(r=-.72~-.78), indicating good validity. Increased times on the mEFAP correlated with poor performance on the gait ability, motor function of lower extremity, BBS and slow gait speeds on the 10 m WT in stroke patients. Conclusion : The mEFAP can be administered easily and comprehensively. It is a reliable gait assessment tool for patients with stroke and correlated with known of function, the mEFAP may be a clinically useful measure of ambulation.

A Preliminary Study for Standardizing the Fine Motor Scale of the Korean-Bayley Scales of Infant and Toddler Development, Third Edition (한국형 베일리 영유아 발달검사 3판 소근육 운동척도 예비 연구)

  • Lee, Soon Hang;Ahn, So Hyun;Lee, Eun Ji;Bang, Hee Jeong
    • Korean Journal of Child Studies
    • /
    • v.35 no.5
    • /
    • pp.37-61
    • /
    • 2014
  • The purpose of the present study was to conduct a preliminary study for developing a Korean version of the Fine Motor Scale of the Bayley Scales of Infant and Toddler Development, $3^{rd}$ Edition(Bayley-III). The subjects for this study included 313 Korean infants and toddlers aged from 16 days to 42 months and 15 days. The results of this study were as follows. Firstly, the analysis of the degree of item difficulty indicated that the arrangement of the items corresponded with child development by age. Secondly, the analysis of the inter-item consistency, the test-retest reliability and the inter-rater reliability revealed a high degree of reliability of the scale. Thirdly, the Fine Motor Scale of K-Bayley-III score showed a significant high correlation with the Motor composite score but a significant moderate correlation with the Language composite score, providing evidence of its convergent and discriminant validity. This study demonstrated that the preliminary version of the K-Bayley-III Fine Motor Scales can be applied to assess the development of fine motor skills of Korean infants and toddlers.

Validity and Reliability of Cognitive Performance Scale in Long Term Care Hospital in Korea (인지수행척도(Cognitive Performance Scale)의 타당도와 신뢰도)

  • Lee, Ji Yun;Kim, Sun Min;Kim, A Reum
    • 한국노년학
    • /
    • v.30 no.1
    • /
    • pp.81-91
    • /
    • 2010
  • The purpose of this study was to test a validity and reliability of Cognitive Performance Scale(CPS), a cognitive measure generated from 5 items(comatose status, decision making, short-term memory, making self understood, and eating). Method: 393 patients in 2 hospitals for the elderly with dementia were measured with CPS by two nurses independently. The inter-rater agreement was tested by comparing two scores. The CPS score was compared with GDS, which was measured by doctors and nurses, and MMSE score which was drawn from the claim data of Health Insurance Review & Assessment Service. Result: The correlation coefficient between CPS and GDS was 0.742(p<0.0001), CPS and MMSE was -0.794(p<0.0001). The Cronbach's coefficient alpha of CPS was 0.742, Kappa value was 0.772~1.000. The CPS showed high validity and reliability in long term care hospitals of Korea.

A Research to Evaluate the Reliability and Validity of Pattern Identifications Tool for Cognitive Disorder: A Clinical Study Protocol (인지장애 변증평가도구의 신뢰도와 타당도 평가: 임상연구 프로토콜)

  • Lee, Ji-yoon;Kim, Hwan;Seo, Young Kyung;Kang, Hyung Won;Kang, Wee-Chang;Jung, In Chul
    • Journal of Oriental Neuropsychiatry
    • /
    • v.29 no.4
    • /
    • pp.255-266
    • /
    • 2018
  • Objectives: The objective of this study was to evaluate the reliability and validity of Pattern Identifications Tool for Cognitive Disorders (PIT-C) and verify the correlation with other related scales. Methods: The study in this protocol is a single group, prospective, observational one. The subjects of the study were men and women between the ages of 45 and 85, diagnosed with neurocognitive disorders by Diagnostic and Statistical Manual of Mental Disorder (fifth Edition) criteria (n=60, Clinical Dementia Rating (CDR)=0.5, Korean Version of Montreal Cognitive Assessment $(MoCA-K){\leq}22$). The reliability of PIT-C was evaluated as test-retest and inter-rater reliability. And correlation between PIT-C and other related scales was also assessed. Results: This study was approved by the Institutional Review Board (IRB) of Dunsan Korean Medicine Hospital of Daejeon University and registered in the Clinical Research Information Service (CRIS), and was made public in advance to ensure transparency of the research process and conduct ethical clinical trials. Conclusions: The results of this study can be used to classify neurocognitive disorders as Korean medicine and PIT-C will be helpful tool for primary health care.

Reliability and Validity Inquiry for Tele-assessment Based on Video Conferencing

  • Suha Lee;Changho Song
    • Physical Therapy Rehabilitation Science
    • /
    • v.12 no.2
    • /
    • pp.105-114
    • /
    • 2023
  • Objective: The study examined the validity and reliability of tele-assessment of cervical range of motion (ROM) through video conferencing. Design: A cross-sectional study Methods: The study included 22 healthy adults as subjects and 10 physical therapists as evaluators. The subjects underwent both face-to-face and tele-assessment of cervical ROM. The CROM (Cervical Range of Motion) device was used for reference measurements in the face-to-face assessment. The evaluators assessed the subjects' ROM through visual inspection (VI) and using a Universal Goniometer (UG) in the tele-assessment. Results: The results showed a significant correlation between the face-to-face and tele-assessment measurements for all ROM contents. The correlation coefficients ranged from r=0.54 to 0.71 (p<0.05), indicating concurrent validity. In terms of inter-rater reliability, the ICC values for both VI and UG assessments in the tele-assessment were high, with ICCs of 0.93 and 0.92, respectively. Additionally, the study investigated the usability of the tele-assessment method and environment. The satisfaction of the participants and evaluators was assessed using the Telehealth Usability Questionnaire (TUQ), which evaluates perceptions, usability, and satisfaction with telehealth services. Conclusions: In conclusion, tele-assessment of cervical range of motion using video conferencing was valid and reliable. Also, investigated participant and evaluator satisfaction and opinions, providing insights into the clinical value of tele-assessment. It is important to note that the study was conducted only healthy adults, and further research may be needed to validate the tele-assessment method in populations with cervical dysfunction or neck pain. These findings support the potential effectiveness of telerehabilitation services in assessing and managing musculoskeletal disorders.

Multiple Average Ratings of Auditory Perceptual Analysis for Dysphonia

  • Choi, Seong-Hee;Choi, Hong-Shik
    • Phonetics and Speech Sciences
    • /
    • v.1 no.4
    • /
    • pp.165-170
    • /
    • 2009
  • This study was to investigate for comparison between single rating and average ratings from multiple presentations of the same stimulus for measuring the voice quality of dysphonia using 7-point equal-appearing interval (EAI) rating scale. Overall severity of voice quality for 46 /a/ vowel stimuli (23 stimuli from dysphonia, 23 stimuli from control) was rated by 3 experienced speech-language pathologists (averaged 19 years; range = 7 to 40 years). For average ratings, each stimulus was rated five times in random order and averaged from two to five times. Although higher inter-rater reliability was found in average ratings than in single rating, there were no significant differences in rating scores between single and multiple average ratings judged by experienced listeners, suggesting that auditory perceptual ratings judged by well-trained listeners have relatively good agreement with the same stimulus across the judgment. Larger variations in perceptual ratings were observed for moderate voices than for mild or severe voices, even in the average ratings.

  • PDF

The Image of Nursing projected in Newspapers (신문에 나타난 간호의 이미지에 관한 연구)

  • 정면숙;강영실
    • Journal of Korean Academy of Nursing
    • /
    • v.23 no.1
    • /
    • pp.16-28
    • /
    • 1993
  • The purpose of this study was to identify the im-. age of nursing, that is, to see how nursing is viewed in newspapers. Articles about nursing from two Korean daily newspapers from Jan. 1, 1987 to Dec.31, 1991 were examined for subject, type, attitude and author-ship. The inter-rater reliability was 0.89(by The Holsti method). The major findings were as follows : 1. The total number of articles were 110. 2. As for the subjests matter, articles related to professional nursing activities appeared most frequently(29.6%) , there about labor issues and activity to promote nurses's job climate 19.4%, and about official activities of nursing 11.2%. 3. Commentary articles appeared most frequently(41.2%) , Other article forms were straight news(27. 1%), contribution(17.6%) and inter-views (10.6%). 4. Feature stories acounted for 62.4% and news articles for 37.6%. Most of the articles were of national interests(96.5%), the rest(3.5%) of news from abroad. 5. Articles favorable toward nursing accounted for 54.1%, neutral 28.2%, negative 17.6%. 6. Many articles were written by the reporters (66.3%).

  • PDF

Inter-Rater Reliability of the Gross Motor Performance Measure (대동작 운동 수행능력 측정 도구의 측정자간 신뢰도)

  • Yi, Chung-Hwi;Park, So-Yeon;Ko, Myung-Suk
    • Physical Therapy Korea
    • /
    • v.10 no.4
    • /
    • pp.17-22
    • /
    • 2003
  • 대동작 운동 수행능력 측정도구(GMPM)는 뇌성마비 아동의 움직임을 질적인 면에서 평가하기 위해 개발된 도구이다. 이 연구의 목적은 대동작 운동 수행능력 측정도구의 측정자간 신뢰도를 알아보는 것이다. 뇌성마비 아동 10명(평균 5.6세, 범위 4~8세)에게 GMPM 평가를 실시하였다. 평가 과정을 비디오로 녹화하여 각 속성 항목별로 3명의 평가자간의 급간내 상관계수로 일치도를 보았다. 전반적으로 측정자간 신뢰도는 '불량~보통'범주에 속했다. 이 연구의 결과는 충분한 교육을 받지 않고 평가하면 그 결과를 신뢰하기 어렵다는 것을 말해준다. 향후 임상에서 GMPM을 이용하여 평가할 때 측정자간 신뢰도에 어떤 변화가 있는지 알아보는 연구가 필요하다.

  • PDF