• Title/Summary/Keyword: kappa coefficient

Search Result 243, Processing Time 0.027 seconds

Reliability and Validity of the Alcohol Use Disorders Identification Test - Consumption in Screening for Adults with Alcohol Use Disorders and Risky Drinking In Japan

  • Osaki, Yoneatsu;Ino, Aro;Matsushita, Sachio;Higuchi, Susumu;Kondo, Yoko;Kinjo, Aya
    • Asian Pacific Journal of Cancer Prevention
    • /
    • v.15 no.16
    • /
    • pp.6571-6574
    • /
    • 2014
  • Background: Alcohol is well established as a risk factor for cancer development in many organ sites. To assess the reliability and validity of the Alcohol Use Disorders Identification Test - Consumption (AUDIT-C) for detecting alcohol use disorders or risky drinking in Japanese adults the present study was conducted. Materials and Methods: A test-retest method was applied with a 2-week interval with 113 health care employees. The k coefficient, Cronbach's coefficient alpha, Spearman's correlation coefficient, and intraclass correlation coefficient (ICC) were determined and the validity of the AUDIT-C was analyzed using the data from a nationwide survey on adult alcohol use conducted in 2008 (n=4,123). Results: The reliability of the AUDIT-C score was high (${\kappa}$ coefficient=0.63, Cronbach's alpha=0.98, correlation coefficient=0.95, and ICC=0.95). According to the likelihood ratio and Youden index, appropriate cutoffs for the AUDIT-C were ${\geq}5points$ in men and ${\geq}4$ points in women. The sensitivity and specificity of these cutoffs for identifying ${\geq}8$ points on the AUDIT were 0.88 and 0.80, respectively, for men (positive likelihood ratio [LR+]=4.5) and 0.96 and 0.87, respectively, for women (LR+=7.7). The sensitivity and specificity of the cutoffs for identifying ${\geq}12$ points on the AUDIT were 0.90 and 0.84, respectively, for men (LR+=5.8) and 0.93 and 0.94, respectively, for women (LR+=15.8). The sensitivity and specificity of the cutoffs for identifying ${\geq}16$ points on the AUDIT were 0.93 and 0.80, respectively, for men (LR+=4.7) and 0.92 and 0.98, respectively, for women (LR+=55.6). With higher scores on the AUDIT, the specificity decreased and false-positives increased. The appropriate cutoffs for identifying risky drinking were the same for both genders. Conclusions: The reliability and validity of the AUDIT-C are high, indicating that it is useful for identifying alcohol use disorders or risky drinking among the general population in Japan, a group at high risk of cancer development.

Agreement of Label Information of Antihistamine, Anti-allergy Medications in Pregnancy among Korea, the USA, the UK, and Japan (임신부에서 항히스타민제와 알레르기용약의 국가별 안전정보 일치도 분석 : 한국, 미국, 영국, 일본 허가사항을 중심으로)

  • Park, Mi-Ju;Shin, Ju-Young;Kim, Hong-Ah;Park, Hyo-Ju;Kim, Mi-Hee;Shin, Sun-Mi;Park, Byung-Joo
    • Korean Journal of Clinical Pharmacy
    • /
    • v.23 no.4
    • /
    • pp.327-333
    • /
    • 2013
  • Background: Antihistamine and anti-allergy medications are widely used during pregnancy. Reading label information is one of the easiest ways to get safety information. But there are content gaps among countries. Objective: To compare the risk level and the recommendation level of antihistamine/anti-allergy drug's label information in pregnant women among Korea, the USA, the UK, and Japan. Method: Study drugs of antihistamine/anti-allergy medications were selected according to Korea drug classification codes. Based on the label information of selected product, risk level was classified into 5 categories as follows: 'Definite', 'Probable', 'Possible', and 'Unlikely', 'Unclassified' according to the level of evidence. Recommendation level was classified into 4 categories as follows: 'Contraindicated', 'Cautious', 'Compatible', and 'Unclassified'. Frequency and proportion were presented according to the each category. To estimate agreement of each category among 4 countries, percent agreement and kappa (k) coefficient were calculated. Results: Total 13 drug ingredients were selected for antihistamine/anti-allergy medications. In risk level, Korea (46%) and Japan (69%) were mostly classified in the category of 'Unclassified', but 'Unlikely' category was more frequent in the UK (62%) and the USA (46%). In recommendation level, the proportion of 'Contraindicated' was highest in Korea (46%) compared to other countries. In contrast, the category of 'Cautious' was 77%-85% in the USA, the UK, and Japan. The percent agreement for risk level was highest in the USA-UK (54%). The recommendation level of Korea-USA showed lowest agreement for percent agreement (46%) and kappa coefficient (k=0.02). Conclusion: We confirmed the differences among safety information provided by four different countries. 'Contraindicated' was more likely in Korea compared with other countries.

Analysis of Burn Severity in Large-fire Area Using SPOT5 Images and Field Survey Data (SPOT5영상과 현장조사자료를 융합한 대형산불지역의 피해강도 분석)

  • Won, Myoungsoo;Kim, Kyongha;Lee, Sangwoo
    • Korean Journal of Agricultural and Forest Meteorology
    • /
    • v.16 no.2
    • /
    • pp.114-124
    • /
    • 2014
  • For classifying fire damaged areas and analyzing burn severity of two large-fire areas damaged over 100 ha in 2011, three methods were employed utilized supervised classification, unsupervised classification and Normalized Difference Vegetation Index (NDVI). In this paper, the post-fire imageries of SPOT were used to compute the Maximum Likelihood (MLC), Minimum Distance (MIN), ISODATA, K-means, NDVI and to evaluate large-scale patterns of burn severity from 1 m to 5 m spatial resolutions. The result of the accuracy verification on burn severity from satellite images showed that average overall accuracy was 88.38 % and the Kappa coefficient was 0.8147. To compare the accuracy between burn severity and field survey at Uljin and Youngduk, two large fire sites were selected as study areas, and forty-four sampling plots were assigned in each study area for field survey. The burn severities of the study areas were estimated by analyzing burn severity (BS) classes from SPOT images taken one month after the occurrence of the fire. The applicability of composite burn index (CBI) was validated with a correlation analysis between field survey data and burn severity classified by SPOT5, and by their confusion matrix. The result showed that correlation between field survey data and BS by SPOT5 were closely correlated in both Uljin (r = -0.544 and p<0.01) and Youngduk (r = -0.616 and p<0.01). Thus, this result supported that the proposed burn severity analysis is an adequate method to measure burn severity of large fire areas in Korea.

Water body extraction using block-based image partitioning and extension of water body boundaries (블록 기반의 영상 분할과 수계 경계의 확장을 이용한 수계 검출)

  • Ye, Chul-Soo
    • Korean Journal of Remote Sensing
    • /
    • v.32 no.5
    • /
    • pp.471-482
    • /
    • 2016
  • This paper presents an extraction method for water body which uses block-based image partitioning and extension of water body boundaries to improve the performance of supervised classification for water body extraction. The Mahalanobis distance image is created by computing the spectral information of Normalized Difference Water Index (NDWI) and Near Infrared (NIR) band images over a training site within the water body in order to extract an initial water body area. To reduce the effect of noise contained in the Mahalanobis distance image, we apply mean curvature diffusion to the image, which controls diffusion coefficients based on connectivity strength between adjacent pixels and then extract the initial water body area. After partitioning the extracted water body image into the non-overlapping blocks of same size, we update the water body area using the information of water body belonging to water body boundaries. The update is performed repeatedly under the condition that the statistical distance between water body area belonging to water body boundaries and the training site is not greater than a threshold value. The accuracy assessment of the proposed algorithm was tested using KOMPSAT-2 images for the various block sizes between $11{\times}11$ and $19{\times}19$. The overall accuracy and Kappa coefficient of the algorithm varied from 99.47% to 99.53% and from 95.07% to 95.80%, respectively.

Variation of Seasonal Groundwater Recharge Analyzed Using Landsat-8 OLI Data and a CART Algorithm (CART알고리즘과 Landsat-8 위성영상 분석을 통한 계절별 지하수함양량 변화)

  • Park, Seunghyuk;Jeong, Gyo-Cheol
    • The Journal of Engineering Geology
    • /
    • v.31 no.3
    • /
    • pp.395-432
    • /
    • 2021
  • Groundwater recharge rates vary widely by location and with time. They are difficult to measure directly and are thus often estimated using simulations. This study employed frequency and regression analysis and a classification and regression tree (CART) algorithm in a machine learning method to estimate groundwater recharge. CART algorithms are considered for the distribution of precipitation by subbasin (PCP), geomorphological data, indices of the relationship between vegetation and landuse, and soil type. The considered geomorphological data were digital elevaion model (DEM), surface slope (SLOP), surface aspect (ASPT), and indices were the perpendicular vegetation index (PVI), normalized difference vegetation index (NDVI), normalized difference tillage index (NDTI), normalized difference residue index (NDRI). The spatio-temperal distribution of groundwater recharge in the SWAT-MOD-FLOW program, was classified as group 4, run in R, sampled for random and a model trained its groundwater recharge was predicted by CART condidering modified PVI, NDVI, NDTI, NDRI, PCP, and geomorphological data. To assess inter-rater reliability for group 4 groundwater recharge, the Kappa coefficient and overall accuracy and confusion matrix using K-fold cross-validation were calculated. The model obtained a Kappa coefficient of 0.3-0.6 and an overall accuracy of 0.5-0.7, indicating that the proposed model for estimating groundwater recharge with respect to soil type and vegetation cover is quite reliable.

Analysis of Clinical Indicators related to Pattern-Identification in Acute Cerebral Infarction Patient (급성기 뇌경색 환자에 있어 변증형별 유의한 임상지표의 분석)

  • Lee, Eun-chan;Hyun, Sang-ho;Kwak, Seung-hyuk;Woo, Su-kyung;Park, Ju-young;Jung, Woo-sang;Moon, Sang-kwan;Cho, Ki-ho;Park, Sung-wook;Ko, Chang-nam
    • The Journal of the Society of Stroke on Korean Medicine
    • /
    • v.13 no.1
    • /
    • pp.33-42
    • /
    • 2012
  • Object : The aim of this study was to assess the clinical indicators related to Pattern-Identification(PI) in acute cerebral infarction patients. Methods : We studied hospitalized patients within 30days after ictus, who admitted at Korean Medicine Center of Kyung-Hee University from January 2010 to October 2012.(n=290) Two Traditional Korean Medicine(TKM) physicians evaluated the patients independently and diagnosed PI. Inter-rater reliability was measured using simple percentage agreement and the Cohen's kappa(κ) coefficient. To assess the clinical indicators closely related to each PI, we analysed average score of each indicator in each group. Results : Simple percentage agreement of PI between raters was 64.83% and Cohen's kappa(κ) coefficient was 0.526(95% CI: 0.451-0.600). Inter-rater reliability level was fair to good. We analysed the clinical indicators in each group. Significant indicators for Fire-Heat Pattern(FHP) were reddened complexion and strong pulse power, and meaningful indicators for FHP were halitosis and thick tongue fur. Significant indicator for Dampness-Phlegm Pattern(DPP) was overweight and there was no meaningful indicator. Significant indicator for Yin-Deficiency Pattern(YDP) was dry tongue fur and meaningful indicator for YDP was thirst. There was no significant indicator for Qi-Deficiency Pattern(QDP) and pale complexion and faint low voice were meaningful indicators for QDP. Conclusions : This study reveals the significant and meaningful clinical indicators related to each Pattern-Identification in acute cerebral infarction patients. It will contribute to standardization of Korean Medical Diagnosis and Treatment in acute cerebral infarction patients.

  • PDF

The Automated Scoring of Kinematics Graph Answers through the Design and Application of a Convolutional Neural Network-Based Scoring Model (합성곱 신경망 기반 채점 모델 설계 및 적용을 통한 운동학 그래프 답안 자동 채점)

  • Jae-Sang Han;Hyun-Joo Kim
    • Journal of The Korean Association For Science Education
    • /
    • v.43 no.3
    • /
    • pp.237-251
    • /
    • 2023
  • This study explores the possibility of automated scoring for scientific graph answers by designing an automated scoring model using convolutional neural networks and applying it to students' kinematics graph answers. The researchers prepared 2,200 answers, which were divided into 2,000 training data and 200 validation data. Additionally, 202 student answers were divided into 100 training data and 102 test data. First, in the process of designing an automated scoring model and validating its performance, the automated scoring model was optimized for graph image classification using the answer dataset prepared by the researchers. Next, the automated scoring model was trained using various types of training datasets, and it was used to score the student test dataset. The performance of the automated scoring model has been improved as the amount of training data increased in amount and diversity. Finally, compared to human scoring, the accuracy was 97.06%, the kappa coefficient was 0.957, and the weighted kappa coefficient was 0.968. On the other hand, in the case of answer types that were not included in the training data, the s coring was almos t identical among human s corers however, the automated scoring model performed inaccurately.

Interpretation of Complete Tumor Response on MRI Following Chemoradiotherapy of Rectal Cancer: Inter-Reader Agreement and Associated Factors in Multi-Center Clinical Practice

  • Hae Young Kim;Seung Hyun Cho;Jong Keon Jang;Bohyun Kim;Chul-min Lee;Joon Seok Lim;Sung Kyoung Moon;Soon Nam Oh;Nieun Seo;Seong Ho Park
    • Korean Journal of Radiology
    • /
    • v.25 no.4
    • /
    • pp.351-362
    • /
    • 2024
  • Objective: To measure inter-reader agreement and identify associated factors in interpreting complete response (CR) on magnetic resonance imaging (MRI) following chemoradiotherapy (CRT) for rectal cancer. Materials and Methods: This retrospective study involved 10 readers from seven hospitals with experience of 80-10210 cases, and 149 patients who underwent surgery after CRT for rectal cancer. Using MRI-based tumor regression grading (mrTRG) and methods employed in daily practice, the readers independently assessed mrTRG, CR on T2-weighted images (T2WI) denoted as mrCRT2W, and CR on all images including diffusion-weighted images (DWI) denoted as mrCRoverall. The readers described their interpretation patterns and how they utilized DWI. Inter-reader agreement was measured using multi-rater kappa, and associated factors were analyzed using multivariable regression. Correlation between sensitivity and specificity of each reader was analyzed using Spearman coefficient. Results: The mrCRT2W and mrCRoverall rates varied widely among the readers, ranging 18.8%-40.3% and 18.1%-34.9%, respectively. Nine readers used DWI as a supplement sequence, which modified interpretations on T2WI in 2.7% of cases (36/1341 [149 patients × 9 readers]) and mostly (33/36) changed mrCRT2W to non-mrCRoverall. The kappa values for mrTRG, mrCRT2W, and mrCRoverall were 0.56 (95% confidence interval: 0.49, 0.62), 0.55 (0.52, 0.57), and 0.54 (0.51, 0.57), respectively. No use of rectal gel, larger initial tumor size, and higher initial cT stage exhibited significant association with a higher interreader agreement for assessing mrCRoverall (P ≤ 0.042). Strong negative correlations were observed between the sensitivity and specificity of individual readers (coefficient, -0.718 to -0.963; P ≤ 0.019). Conclusion: Inter-reader agreement was moderate for assessing CR on post-CRT MRI. Readers' varying standards on MRI interpretation (i.e., threshold effect), along with the use of rectal gel, initial tumor size, and initial cT stage, were significant factors associated with inter-reader agreement.

Agreement of Manual Muscle Testing and Test-Retest Reliability of Hand Held Dynamometer for the Posterior Gluteus Medius Muscle for Patients With Low Back Pain (요통 환자를 대상으로 후중둔근 도수근력검사의 일치도 및 휴대용 근력계 측정 방법의 신뢰도 검사)

  • Park, Kyue-Nam;Kim, Hyun-Sook;Choi, Houng-Sik;Lee, Won-Hwee;Ha, Sung-Min;Kim, Su-Jung
    • Physical Therapy Korea
    • /
    • v.18 no.3
    • /
    • pp.67-75
    • /
    • 2011
  • The purpose of this study was to assess the agreement of manual muscle testing (MMT) and test-retest reliability of a hand held dynamometer for the posterior gluteus medius muscle, with and without lumbar stabilization, using a pressure biofeedback unit for patients with low back pain. The pressure biofeedback unit was used to minimize the substitute motion of the lumbopelvic region during hip abduction in patients lying on their side. Fifteen patients with low back pain participated in this study. A tester determined the MMT grades of the posterior gluteus medius with and without the pressure biofeedback unit. Active hip abduction range of motion with an inclinometer and the strength of their posterior gluteus medius using a hand held dynamometer were measured with and without the pressure biofeedback unit in the MMT position. The agreement of the grade of muscle strength in the MMT, and intra-rater reliability of both the active hip abduction range of motion and the strength of posterior gluteus medius were analyzed using the weighted kappa and intraclass correlation coefficient (ICC), respectively. The agreement of MMT with the pressure biofeedback unit (weighted kappa=.92) was higher than the MMT (weighted kappa=.34)(p<.05). The inclinometer with pressure biofeedback unit measurement of the active hip abduction range of motion had an excellent intra-rater reliability (ICC=.90). Also, the hand held dynamometer with pressure biofeedback unit measure of strength of the posterior gluteus medius had a good intra-rater reliability (ICC=.85). Therefore, the test for muscle strength with pressure biofeedback unit will be a reliable method for the determination of the MMT grades or amount of posterior gluteus medius muscle strength and the measurement of the range of motion for hip abduction in patients with low back pain.

Reliability of Self-Reported Information by Farmers on Pesticide Use (일부 농업인에서 자기 기입식 농약 노출 설문에 대한 신뢰도 연구)

  • Lee, Yo-Han;Cha, Eun-Shil;Moon, Eun-Kyeong;Kong, Kyoung-Ae;Koh, Sang-Baek;Lee, Yun-Keun;Lee, Won-Jin
    • Journal of Preventive Medicine and Public Health
    • /
    • v.43 no.6
    • /
    • pp.535-542
    • /
    • 2010
  • Objectives: Exposure assessment is a major challenge faced by studies that evaluate the association between pesticide exposure and adverse health outcomes. The objective of this study was to investigate the reliability of information that farmers self-report regarding their pesticide use. Methods: Twenty five items based upon existing questionnaires were designed to focus on pesticide exposure. In 2009, a selfadministrated survey was conducted on two occasions four weeks apart among 205 farmers residing in Gyeonggi and Gangwon provinces. For a reliability measure, we calculated the percentage agreement, the kappa statistics and the intraclass correlation coefficient (ICC) between the two reports according to the characteristics of the subjects. Results: Agreement for ever-never use of any pesticide was 96.4% (kappa 0.61). For both 'years used' and 'age at the first use' of overall pesticides, high agreement was obtained (ICC: 0.88 and, 0.78, respectively), whereas those of 'days used' and 'hours used' were relatively low (ICC: 0.42 and, 0.66, respectively). The kappa value for the use of personal protective equipment ranged from 0.46 to 0.59, and hygiene activities came out at 0.19 to 0.37. The agreement for individual pesticide use ranged widely and there was relatively low agreement due to the low response rates. The reliability scores did not significantly vary according to gender, age, the education level, the types of crop or the years of farming. Conclusions: Our results support that carefully designed, self-reported information on ever-never pesticide use among farmers is reliable. However, the reliability of data on individual pesticide exposure may be unstable due to low response rates and needs to be refined.