• The Journal of the Korea Contents Association
• /
• v.13 no.5
• /
• pp.354-365
• /
• 2013

The purpose of the study was to develop the home fall prevention checklist for community-dwelling older adults. And the validity and reliability of the checklist were tested. The preliminary questions were developed through content validity by twenty experts using the CVI(Content Validity Index). Following the establishment of content validity, 52 items of the checklist were developed. Responses of 299 community-dwelling older adults were analyzed to further establish both reliability and validity of the checklist. Reliability using cohen's kappa coefficient and test-retest reliability(rate of concordance(%)), and construct validity using known-group comparison technique were tested. 51 items were over 0.80 in the cohen's kappa coefficient of the checklist, 45 items were over 80.0% in test-retest reliability. Construct validity was established by known-group comparison(t=3.50, p=.001). Validity and reliability of the checklist were confirmed. This checklist will help further studies to develop more safe environment to prevent falls.

• Journal of the Korean Data and Information Science Society
• /
• v.15 no.4
• /
• pp.957-963
• /
• 2004

A new agreement measure for multivariate interval data by different sets of raters is proposed. The proposed approach builds on Um's multivariate extension of Cohen's kappa. The proposed measure is compared with corresponding earlier measures based on Berry and Mielke's approach and Janson and Olsson approach, respectively. Application of the proposed measure is exemplified using hypothetical data set.

• Journal of the Korean Society of Physical Medicine
• /
• v.16 no.1
• /
• pp.1-7
• /
• 2021

PURPOSE: The purpose of this study is to conduct inter-rater and intra-rater reliability tests in patients with low back pain (LBP) using the prone instability test (PIT) and side-lying instability test (SIT). We have analyzed the Korean version Oswestry disability index (K-ODI) correlations and radiograph finding (RF) for validity. METHODS: Individuals (n = 51) (mean age of 40.27 ± 13.28) with LBP for at least over a week were recruited, together with two participating physical therapist examiners. The measurement consisted of PIT, PST, K-ODI, and RF. Sensitivity (Sn), specificity (Sp), positive predictive value, negative predictive value, prevalence index, agreement %, Cohen's kappa, and prevalence-adjusted bias-adjusted kappa (PABAK) were calculated. The PIT and SIT were compared with RF for validity analysis, while PIT, SIT, K-ODI, and RF were calculated for the correlation analysis. RESULTS: The intra-rater reliability test measured for the PIT (kappa = .79, PABAK = .88) and SIT (kappa = .73, PABAK = .84), and inter-rater reliability test measured for the SIT (kappa = .80, PABAK = .88) showed good agreements. The PIT (Sn = .65, Sp = .63) and SIT validities (Sn = .68, Sp = .70) were compared with RF, showing a significant correlation in PIT and RF (r = .69), SIT and RF (r = .73), and PIT and K-ODI (r = .53). CONCLUSION: The SIT is a more comfortable position test than the PIT in patients. Both PIT and SIT have acceptable reliability and validity.

• Korean Journal of Radiology
• /
• v.24 no.4
• /
• pp.284-293
• /
• 2023

Objective: To validate a simplified ordinal scoring method, referred to as modified length-based grading, for assessing coronary artery calcium (CAC) severity on non-electrocardiogram (ECG)-gated chest computed tomography (CT). Materials and Methods: This retrospective study enrolled 120 patients (mean age ± standard deviation [SD], 63.1 ± 14.5 years; male, 64) who underwent both non-ECG-gated chest CT and ECG-gated cardiac CT between January 2011 and December 2021. Six radiologists independently assessed CAC severity on chest CT using two scoring methods (visual assessment and modified length-based grading) and categorized the results as none, mild, moderate, or severe. The CAC category on cardiac CT assessed using the Agatston score was used as the reference standard. Agreement among the six observers for CAC category classification was assessed using Fleiss kappa statistics. Agreement between CAC categories on chest CT obtained using either method and the Agatston score categories on cardiac CT was assessed using Cohen's kappa. The time taken to evaluate CAC grading was compared between the observers and two grading methods. Results: For differentiation of the four CAC categories, interobserver agreement was moderate for visual assessment (Fleiss kappa, 0.553 [95% confidence interval {CI}: 0.496-0.610]) and good for modified length-based grading (Fleiss kappa, 0.695 [95% CI: 0.636-0.754]). The modified length-based grading demonstrated better agreement with the reference standard categorization with cardiac CT than visual assessment (Cohen's kappa, 0.565 [95% CI: 0.511-0.619 for visual assessment vs. 0.695 [95% CI: 0.638-0.752] for modified length-based grading). The overall time for evaluating CAC grading was slightly shorter in visual assessment (mean ± SD, 41.8 ± 38.9 s) than in modified length-based grading (43.5 ± 33.2 s) (P < 0.001). Conclusion: The modified length-based grading worked well for evaluating CAC on non-ECG-gated chest CT with better interobserver agreement and agreement with cardiac CT than visual assessment.

• Journal of the Korea Convergence Society
• /
• v.11 no.11
• /
• pp.443-450
• /
• 2020

The purpose of this study was to examine the inter-rater and intra-rater reliability of the oral mucosa pressure ulcer classification system based on the photographs. The study consisted of two stages; development and evaluation. In the developmental stage, 9 photographs of 82 were selected. In the evaluation stage, a total of 49 participants were invited web-based survey by e-mail. Cohen's weighted kappa and Krippendorff's alpha were used to define the inter-rater reliability. Nine photographs consisted of two, three, three, and one in normal, stage 1, stage 2, and stomatitis, respectively. The inter-rater reliabilities of wound care nurse specialist, intensive care nurse specialist, and dentist groups were 0.75, 0.70, and 0.78, respectively. The intra-rater reliability was 0.73. The inter-rater and intra-rater reliabilities of the oral mucosa pressure ulcer classification system showed substantially good agreement.

• Journal of Sasang Constitutional Medicine
• /
• v.26 no.1
• /
• pp.64-74
• /
• 2014

Objectives This study aimed to evaluate reliability of questionnaire when self reporting questionnaire created by Korea Institute of Oriental Medicine was applied to Vietnamese. Methods This study began to collect 135 Vietnamese patients questionnaires collaborated with National Hospital of Traditional Medicine located in Hanoi, Vietnam from March to August 2013. All participants for this study filled out the questionnaires respectively. After initial survey finished, additional survey was performed on the same questionnaires used at the beginning eight weeks later. In order to evaluate internal coherence in terms of questionnaires of classification, Cronbach's alpha and Cohen's kappa was measured. Results After analysis of 78 questions collected, less than 0.4 in Kappa was achieved by 21(26.9%) out of 78 questions, 0.4 to 0.75 Kappa by 49(62.8%) and 0.75 over in 5(6.5%) questions, respectively. More than 0.6 Cronbach's alpha was defined from 41 out of 78 questions connected with internal coherence of character, digestion, perspiration, excrement, urine, cold and heat. Conclusions The questionnaire has credibility according to values of Kappa and Cronbach's alpha. Therefore, Sasang Constitution questionnaire can be applied to Sasang Diagnosis. In order to increase usefulness, questions in questionnaire should be revised and validity study must be performed afterwards.

• The Journal of the Korean dental association
• /
• v.54 no.11
• /
• pp.880-896
• /
• 2016

The comparison of the repeated measurements is often needed to see whether they agree sufficiently, when a measurement is repeated under identical conditions by different raters. Such investigations are often analyzed inappropriately, by using correlation coefficient. The purpose of this study is to introduce statistical methods for accessing the agreement of the repeated measurements, which include Bland-Altman plot, intra class correlation, Passing-Bablok regression and Cohen's kappa coefficient, and to show how to execute them using examples.

• Nutrition Research and Practice
• /
• v.7 no.4
• /
• pp.326-329
• /
• 2013

Height and weight are important indicators to calculate Body Mass Index (BMI); measuring height and weight directly is the most exact method to get this information. However, it is ineffective in terms of cost and time on large population samples. The aim of our study was to investigate the validity of self-reported height and weight data compared to our measured data in Korean children to predict obese status. Four hundred twenty-two fifth-grade (mean age $10.5{\pm}0.5$ years) children who had self-reported and measured height and weight data were final subjects for this study. Overweight/obese was defined as a BMI of or above the 85th percentile of the gender-specific BMI for age in the 2007 Korean National Growth Charts or a BMI of 25 or higher (underweight : < 5th, normal : ${\geq}5th$ to < 85th, overweight : ${\geq}85th$ to < 95th). The differences between self-reported and measured data were tested using paired t-test. Differences based on overweight/obese status were tested using analysis of variance (ANOVA) and linear trends. Pearson's correlation and Cohen's kappa were tested to examine agreements between the self-reported and measured data. Although measured and self-reported height, weight and BMI were significantly different and children tended to overreport their height and underreport their weight, the correlation between the two methods of height, weight and BMI were high (r = 0.956, 0.969, 0.932, respectively; all P < 0.001), and both genders reported their overweight/non-overweight status accurately (Cohen's kappa = 0.792, P < 0.001). Although there were differences between the self-reported and our measured methods, the self-reported weight and height was valid enough to classify overweight/obesity status correctly, especially in non-overweight/obese children. Due to bigger underestimation of weight and overestimation of height in obese children, however, we need to be aware that the self-reported anthropometric data were less accurate in overweight/obese children than in non-overweight/obese children.

• Journal of Preventive Medicine and Public Health
• /
• v.30 no.4 s.59
• /
• pp.852-869
• /
• 1997

This study was conducted to develop and evaluate the reliability and the validity of a questionnaire in order to determine the applicability as a screening tool for estimating environmental exposure and health effects related to air pollution. The questionnaire was developed with adopting some items of others such as ISAAC or ATS-DLD. And then we performed test-retest to 89 middle school students and their mothers at interval of three months. Cohen's Kappa values, weighted Kappa values, Spearman's correlation coefficients, and Pearson's correlation coefficients for each item were computed as reliability coefficients. The validity coefficients and validity coefficient bounds were also obtained by simply using these reliability coefficients. As results, Kappa ranged broadly from 0.10 to 0.61 of the items 'diet', $0.52\sim0.79$ of the environmental tobacco smoke, $0.39\sim0.44$ of the functional categories of surrounding environment, and $0.39\sim0.44$ of the using transportation systems; these items were regarded as confounding factors. For items related to health outcomes, Kappa ranged from -0.02 to 0.37 in the respiratory system of past medical history, and from 0.11 to 0.55 in the current health status. But Kappa of the others were over 0.60. In conclusion, if some items can be corrected or modified, the questionnaire developed in this study can be used as a tool for evaluating environmental exposure and health effects associated with air pollution.

• The Journal of Korean Medicine Ophthalmology and Otolaryngology and Dermatology
• /
• v.30 no.4
• /
• pp.97-103
• /
• 2017

Objectives : To identify whether pattern identification using nasal endoscopy for rhinitis can be applied as a tool for evaluating rhinitis in routine care setting, we performed a inter-rater reliability study on this pattern identification. Methods : Two Korean medicine doctors assessed 290 left/right nasal endoscopy photograph cases of rhinitis patients with pattern identification using nasal endoscopy. This pattern identification consist of four assessment items, nasal membrane color(pale/hyperemia), nasal membrane humidity(dryness/dampness), rhinorrhea(watery/yellow), and turbinate membrane edema(atrophic/edematous). Cohen's kappa statistic and Percentage agreement were used to evaluate the inter-rater reliability. Results : Inter-rater percentage agreement and Kappa coefficient for left nasal endoscopy photograph cases was from 'slight' to 'moderate'(% agreement: 40.00-67.59%/Kappa: 0.06-0.407). Only the agreement of 'rhinorrhea (watery/yellow)' item was moderate(% agreement: 67.59%/Kappa: 0.407). Inter-rater percentage agreement and Kappa coefficient for right nasal endoscopy photograph cases was also from 'slight' to 'moderate'(% agreement: 42.41-68.97%/Kappa: 0.109-0.465). Only the agreement of 'rhinorrhea(watery/yellow)' item was moderate(% agreement: 68.97%/Kappa: 0.465). Conclusions : It is necessary to resolve problems such as cut-off value setting, bipolar evaluation values(pale/hyperemia, dryness/dampness, watery/yellow, atrophic/edematous) and weighting items. Further rigorous studies that overcome the limitations of the current research are warranted.