• Title/Summary/Keyword: Rater Reliability

Search Result 228, Processing Time 0.02 seconds

Parasellar Extension Grades and Surgical Extent in Endoscopic Endonasal Transsphenoidal Surgery for Pituitary Adenomas : A Single Surgeon's Consecutive Series with the Aspects of Reliability and Clinical Validity

  • Lee, Sang-Hyo;Park, Jae-Sung;Lee, Song;Kim, Sung-Won;Hong, Yong-Kil
    • Journal of Korean Neurosurgical Society
    • /
    • v.59 no.6
    • /
    • pp.577-583
    • /
    • 2016
  • Objective : The inter-rater reliability of the modified Knosp's classification was measured before the analysis. The clinical validity of the parasellar extension grading system was evaluated by investigating the extents of resection and complication rates among the grades in the endoscopic endonasal transsphenoidal surgery (EETS) for pituitary adenomas. Methods : From November 2008 to August 2015, of the 286 patients who underwent EETS by the senior author, 208 were pituitary adenoma cases (146 non-functioning pituitary adenomas, 10 adrenocorticotropic hormone-secreting adenomas, 31 growth hormone-secreting adenomas, 17 prolactin-secreting adenomas, and 4 thyroid-stimulating hormone-secreting adenomas; 23 microadenomas, 174 macroadenomas, and 11 giant adenomas). Two neurosurgeons and a neuroradiologist independently measured the degree of parasellar extension on the preoperative sellar MRI according to the modified Knosp's classification. Inter-rater reliability was statistically assessed by measuring the intraclass correlation coefficient. The extents of resection were evaluated by comparison of the pre- and post-operative MR images; the neurovascular complications were assessed by reviewing the patients' medical records. The extent of resection was measured in each parasellar extension grade; thereafter, their statistical differences were calculated. Results : The intraclass correlation coefficient value of reliability across the three raters amounted to 0.862. The gross total removal (GTR) rates achieved in each grade were 70.0, 69.8, 62.9, 21.4, 37.5, and 4.3% in Grades 0, 1, 2, 3A, 3B, and 4, respectively. A significant difference in the extent of resection was observed only between Grades 2 and 3A. In addition, significantly higher complication rates were observed in the groups above Grade 3A. Conclusion : Although the modified Knosp's classification system appears to be complex, its inter-rater reliability proves to be excellent. Regarding the clinical validity of the parasellar extension grading system, Grades 3A, 3B, and 4 have a negative predictive value for the GTR rate, with higher complication rates.

Study of Validity and Interrater Reliability of Korean Version of the Peabody Developmental Motor Scale 2 (한글판 Peabody Developmental Motor Scale 2의 타당도와 검사자간 신뢰도 연구)

  • Lee, Ji-Ho;Kim, Kyeong-Mi;Chang, Moon-Young;Hong, Eunkyoung
    • The Journal of Korean Academy of Sensory Integration
    • /
    • v.17 no.3
    • /
    • pp.14-25
    • /
    • 2019
  • Objective : This study aims to verify the content validity and inter-rater reliability of the Korean version of the Peabody Developmental Motor Scale 2 (PDMS-2) and to identify the concurrent validity by comparing it with the Korean version of the Bruininks-Oseretsky Test of Motor Proficiency-2 (BOT-2). Methods : PDMS-2 was translated by the researcher and an eighth-year clinical occupational therapist. The content consistency of the Korean version of the PDMS-2 was verified by three professors with experience using it. After the verification of the content consistency of the PDMS-2 by the five clinical occupational therapists and the additional revision, the Korean version of the PDMS-2 was completed. The researcher and another occupational therapist evaluated the Korean version of PDMS-2 in 50 children and measured the inter-rater reliability. Concurrent validity was measured by comparing the results of the Korean version of PDMS-2 and Korean version of BOT-2. Results : The content consistency test showed overall agreement of mean 3.45, and the content understanding test showed a high level of understanding of mean 3.69. The inter-rater reliability and concurrent validity of the Korean version of the PDMS-2 showed a statistically significant correlation. Conclusion : The Korean version of the PDMS-2 showed high content understanding, reliability, and validity. It can assist clinicians and researchers who work in fields related to child treatment or development.

Surface Topographic Measurement Method for Assessing Lower Extremity Alignment: Examination on a novel clinical and research Tool (하지 정렬 평가를 위한 체표면 토포그래피를 이용한 측정법: 새로운 임상 및 연구 도구에 대한 검토)

  • Yim, Ji-Young;Yim, Hyun-Seung;Park, Dae-Sung;Lee, Na-Kyung;Lee, Na-Kyung
    • Journal of rehabilitation welfare engineering & assistive technology
    • /
    • v.10 no.1
    • /
    • pp.37-45
    • /
    • 2016
  • The purpose of this study was to assess the intra-rater, inter-rater and test-retest reliability and validity of frontal plane lower extremity alignment estimated from a rasterstereographic method using ABW-Mapper. Eighteen subjects participated in this study. The S angle (stereographic angle-frontal plane lower extremity alignment estimated from a rasterstereographic method) in standing was measured throughout the two sessions with one week interval by two different readers. In the first session, a reader measured S angle twice per subject with a short break in-between. The Q-angle (quadriceps angle) was measured using a standard goniometer from a photography taken through digital camera with the participant standing in the same position as in the S angle measurement. The HKA(hip-knee-ankle) angle was measured from a computer based digital radiograph with the computerized measurement software. Reliability was tested using intraclass correlation coefficients(ICC). Validity was tested using a Pearson's correlation coefficient. Excellent intra-rater(ICC=0.956~0.974), inter-rater(ICC=0.962), test-retest reliability (ICC=0.945) were demonstrated. There were strong negative correlations between S angle and Q-angle (r=-0.739), and between S angle and HKA angle (r=-0.702). Therefore, the S angle measured using a rasterstereographic mapper may be used to as a preliminary or supplementary tool to evaluate and study LE alignment in the frontal plane in relation to HKA angle or Q-angle.

Comparison of the Reliability and Validity of Fall Risk Assessment Tools in Patients with Acute Neurological Disorders (급성기 신경계 환자에서 낙상 위험 사정 도구의 신뢰도 및 타당도 비교)

  • Kim, Sung Reul;Yoo, Sung-Hee;Shin, Young Sun;Jeon, Ji Yoon;Kim, Jun Yoo;Kang, Su Jung;Choi, Hea Sook;Lee, Hea Lim;An, Young Hee
    • Korean Journal of Adult Nursing
    • /
    • v.25 no.1
    • /
    • pp.24-32
    • /
    • 2013
  • Purpose: The aim of the study was to identify the most appropriate fall-risk assessment tool for neurological patients in an acute care setting. Methods: This descriptive study compared the reliability and validity of three fall-risk assessment tools (Morse Fall Scale, MFS; St Thomas's Risk Assessment Tool in Falling Elderly Inpatients, STRATIFY; Hendrich II Fall Risk Model, HFRM II). We assessed patients who were admitted to the Department of Neurology, Neurosurgery, and Rehabilitation at Asan Medical Center between July 1 and October 31, 2011, using a constructive questionnaire including general and clinical characteristics, and each item from the three tools. We analyzed inter-rater reliability with the kappa value, and the sensitivity, specificity, predictive value, and the area under the curve (AUC) of the three tools. Results: The analysis included 1,026 patients, and 32 falls occurred during this study. Inter-rater reliability was above 80% in all three tools. and the sensitivity was 50.0% (MFS), 84.4%(STRATIFY), and 59.4%(HFRM II). The AUC of the STRATIFY was 82.8. However, when the cutoff point was regulated as not 50 but 40 points, the AUC of the MFS was higher at 83.7. Conclusion: These results suggest that the STRATIFY may be the best tool for predicting falls for acute neurological patients.

Preliminary Study to Develop the Korean Medical Pathologic Aging Scale and Korean Medical Pattern Identification for Dementia (한의학 병리적 노화 척도와 치매 한의학적 변증진단 개발 및 신뢰도 평가)

  • Lee, Go eun;Moon, Kwang Su;Kim, Nam Kwen;Chung, sun yong;Jung, In Chul;Kang, Hyung Won
    • The Journal of Korean Medicine
    • /
    • v.38 no.3
    • /
    • pp.111-123
    • /
    • 2017
  • Objectives: To develop and investigate the reliability of the pathologic aging scale based on korean medical theory and korean medical pattern identification for dementia. Methods: We searched the textbook of korean neurophychiatry and Donguibogam and selected items through professional consensus. We compared between dementia(n=40) and normal elderly(n=38) and tested the reliability of two scales. Results: After professional consensus, we drafted the Korean Medical Pathologic Aging Scale(12 items, Likert 3 scale) and Korean Medical Pattern Identification for Dementia(4 patterns, 28 items, Likert 5 scale). On Korean Medical Pathologic Aging Scale, There is no significant difference between two groups. We had good internal consistency(Cronbach's alpha = 0.6) and test-retest reliability(r=0.631) but low inter-rater reliability(r=0.430). On Korean Medical Pattern Identification for Dementia, dementia patients diagnosed with Qi deficiency are significantly more than those in normal group. We had fairly good internal consistency(Cronbach's alpha = 0.574) and excellent test-retest(kappa= .800) and inter-rater reliability(kappa = .733). Conclusions: Korean Medical Pattern Identification for Dementia is appropriate for diagnosing korean medical pattern. But Korean Medical Pathologic Aging Scale isn't appropriate to discriminate dementia from normal elderly because of many subjective items. Therefore objective measurement of sensory dysfunction would be needed to measure pathologic aging based on korean medical theory.

A Study on Standardization of K-WPPSI-IV: Analyses of Reliability and Validity (한국 Wechsler 유아지능검사 4판(K-WPPSI-IV)의 표준화연구: 신뢰도와 타당도분석)

  • Park, Hyewon;Lee, KyungOk;Lee, Sang-Hee;Park, Minjung
    • Korean Journal of Childcare and Education
    • /
    • v.12 no.4
    • /
    • pp.111-130
    • /
    • 2016
  • This study examined the reliability and validity evidence of the Korean version of Wechsler Primary & Preschool Scale for Intelligence(K-WPPSI-IV), which was standardized recently. Following the stratified sampling for Korean population in terms of region and maternal education level, 1,700 children aged 2.6~7.7 years were recruited for Korean norm development. The split-half reliability (N=1,700) was high(Fisher's z(.75-.89)) and test-retest reliability(N=100) was satisfactory ranging from .95 to .68. The inter-rater reliability(N=110) was very high(.96-1.0). Discriminant and convergent validities were revealed by correlational patterns among subtests and indices. Future studies are needed to extend both reliability and validity of K-WPPSI-IV by collecting data with a various sample group and a variety of instruments.

Development and Psychometric Testing of the Clinical Nursing Competency Scale for Clinical Preceptor Use (CNCS-CP) (임상간호실습 현장지도자용 임상간호역량 평가도구 개발)

  • Kwak, Eunmi;Oh, Heeyoung
    • Journal of Korean Academy of Nursing
    • /
    • v.48 no.4
    • /
    • pp.419-431
    • /
    • 2018
  • Purpose: The purpose of this study was to develop and establish the psychometric properties of a clinical nursing competency evaluation tool to be utilized by clinical preceptors. Methods: The initial items were identified through in-depth literature review and field interviews based on a hybrid model. Content validation of the items was evaluated through three rounds of content validity testing. Participants were 34 clinical preceptors and 443 nursing students participating in clinical practice. Data were analyzed using exploratory and confirmatory factor analysis, convergence and discriminant validity, internal consistency and inter-rater reliability. Results: The final scale consisted of 23 items and four factors, fundamental nursing skills performance, critical thinking skills based on the nursing process, basic nursing knowledge, and professional attitude; these factor explained 69.7% of the total variance. The analysis with multi-trait/multi-item matrix correlation coefficients yielded 100.0% and 95.7 % convergence and discriminant validity, respectively. Cronbach's alpha for the total items was .95. The four subscale model tested by confirmatory factor analysis was satisfactory. Inter-rater reliability ranged from .912 to .967. Conclusion: This scale was found to be a reliable and valid instrument that clinical preceptors can apply for evaluating the clinical nursing competency of nursing students in clinical settings.

Development of Korean Infant Developmental Assessment Scale (한국형 영아발달사정도구의 개발)

  • Han Kyung-Ja;Bang Kyung-Sook
    • Child Health Nursing Research
    • /
    • v.10 no.3
    • /
    • pp.321-331
    • /
    • 2004
  • Purpose: To develop an Infant Developmental Assesment Scale appropriate for Korean infants that includes cultural context. Method: The total sample included 990 infants aged birth to 24months. The developmental tests were conducted at hospital well baby clinic, daycare centers, public health centers and homes located in city, urban and rural areas. Item analysis was done to examine content validity of the scale. Test responses were analyzed by SPSS to examine the inter-rater reliability, and construct validity. Result: Based on the naturalistic observation and analysis of other developmenta l instruments, 260 Korean infant developmental items were created. The mean score of the developmental quotient of 966 infants was 101.63±10.89. Correlation coefficient of inter-rater reliability was .99. In addition, 84.6% of total items showed Kappa .70 and above. The development quotient was significantly different depending upon the residential area and father's occupation, therefore, construct validity using known group technique was supported. Conclusion: The developed instrument can be used to assess the development of handicapped infants, as well as normal infant's general abilities and to study individual differences within the normal range for Korean.

  • PDF

Influence of the Condition with and without External Support on the Strength of Hip Flexor in Supine in Subjects without Core Stabilization

  • Jeon, In-Cheol
    • The Journal of Korean Physical Therapy
    • /
    • v.32 no.6
    • /
    • pp.335-340
    • /
    • 2020
  • Purpose: This study was performed to investigate the influence of the condition with and without external support on the strength of hip flexor in supine position in subjects without core stabilization. Hip flexor muscles are very functional in the hip joint structures. Therefore, it is essential to evaluate the strength of hip flexor in a clinical and precise way. Methods: Twenty subjects participated in this study. The double bent leg-lower test was used to evaluate subjects without core stabilization. The strength of hip flexor muscles was evaluated in supine position, both with and without external support condition. The paired t-test was used to compare the strength of hip flexor muscles according to external support. The level of statistical significance was at α=0.05. The intra-rater reliability of the repeated measures of hip flexor strength was estimated by calculating the intra-class correlation coefficients (ICC). Results: In subjects without core stabilization, the strength of hip flexor in supine was greater with external support than that without external support (p<0.05). In addition, the intra-rater reliability with an ICC (3, 1) of the strength measurement of hip flexor with external support was higher than that without external support. Conclusion: In subjects without core stabilization, the condition with external support can contribute to the strength of hip flexor in supine position and the strength measurement of hip flexor should be considered with the condition with and without external support.