• Title/Summary/Keyword: Area Under the Receiver Operating Characteristic Curve (AUC)

Search Result 162, Processing Time 0.028 seconds

Effects of 1 year of training on the performance of ultrasonographic image interpretation: A preliminary evaluation using images of Sjogren syndrome patients

  • Kise, Yoshitaka;Moystad, Anne;Bjornland, Tore;Shimizu, Mayumi;Ariji, Yoshiko;Kuwada, Chiaki;Nishiyama, Masako;Funakoshi, Takuma;Yoshiura, Kazunori;Ariji, Eiichiro
    • Imaging Science in Dentistry
    • /
    • v.51 no.2
    • /
    • pp.129-136
    • /
    • 2021
  • Purpose: This study investigated the effects of 1 year of training on imaging diagnosis, using static ultrasonography (US) salivary gland images of Sjögren syndrome patients. Materials and Methods: This study involved 3 inexperienced radiologists with different levels of experience, who received training 1 or 2 days a week under the supervision of experienced radiologists. The training program included collecting patient histories and performing physical and imaging examinations for various maxillofacial diseases. The 3 radiologists (observers A, B, and C) evaluated 400 static US images of salivary glands twice at a 1-year interval. To compare their performance, 2 experienced radiologists evaluated the same images. Diagnostic performance was compared between the 2 evaluations using the area under the receiver operating characteristic curve (AUC). Results: Observer A, who was participating in the training program for the second year, exhibited no significant difference in AUC between the first and second evaluations, with results consistently comparable to those of experienced radiologists. After 1 year of training, observer B showed significantly higher AUCs than before training. The diagnostic performance of observer B reached the level of experienced radiologists for parotid gland assessment, but differed for submandibular gland assessment. For observer C, who did not complete the training, there was no significant difference in the AUC between the first and second evaluations, both of which showed significant differences from those of the experienced radiologists. Conclusion: These preliminary results suggest that the training program effectively helped inexperienced radiologists reach the level of experienced radiologists for US examinations.

Combination of Quantitative Parameters of Shear Wave Elastography and Superb Microvascular Imaging to Evaluate Breast Masses

  • Eun Ji Lee;Yun-Woo Chang
    • Korean Journal of Radiology
    • /
    • v.21 no.9
    • /
    • pp.1045-1054
    • /
    • 2020
  • Objective: This study aimed to evaluate the diagnostic value of combining the quantitative parameters of shear wave elastography (SWE) and superb microvascular imaging (SMI) to breast ultrasound (US) to differentiate between benign and malignant breast masses. Materials and Methods: A total of 200 pathologically confirmed breast lesions in 192 patients were retrospectively reviewed using breast US with B-mode imaging, SWE, and SMI. Breast masses were assessed based on the breast imaging reporting and data system (BI-RADS) and quantitative parameters using the maximum elasticity (Emax) and ratio (Eratio) in SWE and the vascular index in SMI (SMIVI). The area under the receiver operating characteristic curve (AUC) value, sensitivity, specificity, accuracy, negative predictive value, and positive predictive value of B-mode alone versus the combination of B-mode US with SWE or SMI of both parameters in differentiating between benign and malignant breast masses was compared, respectively. Hypothetical performances of selective downgrading of BI-RADS category 4a (set 1) and both upgrading of category 3 and downgrading of category 4a (set 2) were calculated. Results: Emax with a cutoff value of 86.45 kPa had the highest AUC value compared to Eratio of 3.57 or SMIVI of 3.35%. In set 1, the combination of B-mode with Emax or SMIVI had a significantly higher AUC value (0.829 and 0.778, respectively) than B-mode alone (0.719) (p < 0.001 and p = 0.047, respectively). B-mode US with the addition of Emax, Eratio, and SMIVI had the best diagnostic performance of AUC value (0.849). The accuracy and specificity increased significantly from 68.0% to 84.0% (p < 0.001) and from 46.1% to 79.1% (p < 0.001), respectively, and the sensitivity decreased from 97.6% to 90.6% without statistical loss (p = 0.199). Conclusion: Combining all quantitative values of SWE and SMI with B-mode US improved the diagnostic performance in differentiating between benign and malignant breast lesions.

Landslide Risk Assessment of Cropland and Man-made Infrastructures using Bayesian Predictive Model (베이지안 예측모델을 활용한 농업 및 인공 인프라의 산사태 재해 위험 평가)

  • Al, Mamun;Jang, Dong-Ho
    • Journal of The Geomorphological Association of Korea
    • /
    • v.27 no.3
    • /
    • pp.87-103
    • /
    • 2020
  • The purpose of this study is to evaluate the risk of cropland and man-made infrastructures in a landslide-prone area using a GIS-based method. To achieve this goal, a landslide inventory map was prepared based on aerial photograph analysis as well as field observations. A total of 550 landslides have been counted in the entire study area. For model analysis and validation, extracted landslides were randomly selected and divided into two groups. The landslide causative factors such as slope, aspect, curvature, topographic wetness index, elevation, forest type, forest crown density, geology, land-use, soil drainage, and soil texture were used in the analysis. Moreover, to identify the correlation between landslides and causative factors, pixels were divided into several classes and frequency ratio was also extracted. A landslide susceptibility map was constructed using a bayesian predictive model (BPM) based on the entire events. In the cross validation process, the landslide susceptibility map as well as observation data were plotted with a receiver operating characteristic (ROC) curve then the area under the curve (AUC) was calculated and tried to extract a success rate curve. The results showed that, the BPM produced 85.8% accuracy. We believed that the model was acceptable for the landslide susceptibility analysis of the study area. In addition, for risk assessment, monetary value (local) and vulnerability scale were added for each social thematic data layers, which were then converted into US dollar considering landslide occurrence time. Moreover, the total number of the study area pixels and predictive landslide affected pixels were considered for making a probability table. Matching with the affected number, 5,000 landslide pixels were assumed to run for final calculation. Based on the result, cropland showed the estimated total risk as US $ 35.4 million and man-made infrastructure risk amounted to US $ 39.3 million.

Effects of various cone-beam computed tomography settings on the detection of recurrent caries under restorations in extracted primary teeth

  • Kamburoglu, Kivanc;Sonmez, Gul;Berktas, Zeynep Serap;Kurt, Hakan;Ozen, Dogukan
    • Imaging Science in Dentistry
    • /
    • v.47 no.2
    • /
    • pp.109-115
    • /
    • 2017
  • Purpose: The aim of this study was to assess the ex vivo diagnostic ability of 9 different cone-beam computed tomography (CBCT) settings in the detection of recurrent caries under amalgam restorations in primary teeth. Materials and Methods: Fifty-two primary teeth were used. Twenty-six teeth had dentine caries and 26 teeth did not have dentine caries. Black class II cavities were prepared and restored with amalgam. In the 26 carious teeth, recurrent caries were left under restorations. The other 26 intact teeth that did not have caries served as controls. Teeth were imaged using a $100{\times}90-mm$ field of view and a 0.2-mm voxel size with 9 different CBCT settings. Four observers assessed the images using a 5-point scale. Kappa values were calculated to assess observer agreement. CBCT settings were compared with the gold standard using a receiver operating characteristic analysis. The area under the curve (AUC) values for each setting were compared using the chi-square test, with a significance level of ${\alpha}=.05$. Results: Intraobserver kappa values ranged from 0.366 to 0.664 for observer 1, from 0.311 to 0.447 for observer 2, from 0.597 to 1.000 for observer 3, and from 0.869 to 1 for observer 4. Furthermore, interobserver kappa values among the observers ranged from 0.133 to 0.814 for the first reading and from 0.197 to 0.805 for the second reading. The highest AUC values were found for setting 5 (0.5916) and setting 3 (0.5886), and were not found to be statistically significant(P>.05). Conclusion: Variations in tube voltage and tube current did not affect the detection of recurrent caries under amalgam restorations in primary teeth.

Diagnostic Performance Using a Combination of MRI Findings for Evaluating Cognitive Decline (인지기능 저하평가를 위한 MR 영상 소견 조합의 진단능)

  • Jin Young Byun;Min Kyoung Lee;So Lyung Jung
    • Journal of the Korean Society of Radiology
    • /
    • v.85 no.1
    • /
    • pp.184-196
    • /
    • 2024
  • Purpose We investigated potentially promising imaging findings and their combinations in the evaluation of cognitive decline. Materials and Methods This retrospective study included 138 patients with subjective cognitive impairments, who underwent brain MRI. We classified the same group of patients into Alzheimer's disease (AD) and non-AD groups, based on the neuropsychiatric evaluation. We analyzed imaging findings, including white matter hyperintensity (WMH) and cerebral microbleeds (CMBs), using the Kruskal-Wallis test for group comparison, and receiver operating characteristic (ROC) curve analysis for assessing the diagnostic performance of imaging findings. Results CMBs in the lobar or deep locations demonstrated higher prevalence in the patients with AD compared to those in the non-AD group. The presence of lobar CMBs combined with periventricular WMH (area under the ROC curve [AUC] = 0.702 [95% confidence interval: 0.599-0.806], p < 0.001) showed the highest performance in differentiation of AD from non-AD group. Conclusion Combinations of imaging findings can serve as useful additive diagnostic tools in the assessment of cognitive decline.

Prediction Model for unfavorable Outcome in Spontaneous Intracerebral Hemorrhage Based on Machine Learning

  • Shengli Li;Jianan Zhang;Xiaoqun Hou;Yongyi Wang;Tong Li;Zhiming Xu;Feng Chen;Yong Zhou;Weimin Wang;Mingxing Liu
    • Journal of Korean Neurosurgical Society
    • /
    • v.67 no.1
    • /
    • pp.94-102
    • /
    • 2024
  • Objective : The spontaneous intracerebral hemorrhage (ICH) remains a significant cause of mortality and morbidity throughout the world. The purpose of this retrospective study is to develop multiple models for predicting ICH outcomes using machine learning (ML). Methods : Between January 2014 and October 2021, we included ICH patients identified by computed tomography or magnetic resonance imaging and treated with surgery. At the 6-month check-up, outcomes were assessed using the modified Rankin Scale. In this study, four ML models, including Support Vector Machine (SVM), Decision Tree C5.0, Artificial Neural Network, Logistic Regression were used to build ICH prediction models. In order to evaluate the reliability and the ML models, we calculated the area under the receiver operating characteristic curve (AUC), specificity, sensitivity, accuracy, positive likelihood ratio (PLR), negative likelihood ratio (NLR), diagnostic odds ratio (DOR). Results : We identified 71 patients who had favorable outcomes and 156 who had unfavorable outcomes. The results showed that the SVM model achieved the best comprehensive prediction efficiency. For the SVM model, the AUC, accuracy, specificity, sensitivity, PLR, NLR, and DOR were 0.91, 0.92, 0.92, 0.93, 11.63, 0.076, and 153.03, respectively. For the SVM model, we found the importance value of time to operating room (TOR) was higher significantly than other variables. Conclusion : The analysis of clinical reliability showed that the SVM model achieved the best comprehensive prediction efficiency and the importance value of TOR was higher significantly than other variables.

Mapping Landslide Susceptibility Based on Spatial Prediction Modeling Approach and Quality Assessment (공간예측모형에 기반한 산사태 취약성 지도 작성과 품질 평가)

  • Al, Mamun;Park, Hyun-Su;JANG, Dong-Ho
    • Journal of The Geomorphological Association of Korea
    • /
    • v.26 no.3
    • /
    • pp.53-67
    • /
    • 2019
  • The purpose of this study is to identify the quality of landslide susceptibility in a landslide-prone area (Jinbu-myeon, Gangwon-do, South Korea) by spatial prediction modeling approach and compare the results obtained. For this goal, a landslide inventory map was prepared mainly based on past historical information and aerial photographs analysis (Daum Map, 2008), as well as some field observation. Altogether, 550 landslides were counted at the whole study area. Among them, 182 landslides are debris flow and each group of landslides was constructed in the inventory map separately. Then, the landslide inventory was randomly selected through Excel; 50% landslide was used for model analysis and the remaining 50% was used for validation purpose. Total 12 contributing factors, such as slope, aspect, curvature, topographic wetness index (TWI), elevation, forest type, forest timber diameter, forest crown density, geology, landuse, soil depth, and soil drainage were used in the analysis. Moreover, to find out the co-relation between landslide causative factors and incidents landslide, pixels were divided into several classes and frequency ratio for individual class was extracted. Eventually, six landslide susceptibility maps were constructed using the Bayesian Predictive Discriminant (BPD), Empirical Likelihood Ratio (ELR), and Linear Regression Method (LRM) models based on different category dada. Finally, in the cross validation process, landslide susceptibility map was plotted with a receiver operating characteristic (ROC) curve and calculated the area under the curve (AUC) and tried to extract success rate curve. The result showed that Bayesian, likelihood and linear models were of 85.52%, 85.23%, and 83.49% accuracy respectively for total data. Subsequently, in the category of debris flow landslide, results are little better compare with total data and its contained 86.33%, 85.53% and 84.17% accuracy. It means all three models were reasonable methods for landslide susceptibility analysis. The models have proved to produce reliable predictions for regional spatial planning or land-use planning.

Development and validation of the Korean Nursing Delirium Scale (한국어판 간호 섬망 선별 도구 개발 및 검증)

  • Kim, Kyoung-Nam;Kim, Cheol-Ho;Kim, Kwang-Il;Yoo, Hyun-Jung;Park, Si-Young;Park, Yeon-Hwan
    • Journal of Korean Academy of Nursing
    • /
    • v.42 no.3
    • /
    • pp.414-423
    • /
    • 2012
  • Purpose: The aims of this study were to develop and test the validity of the Korean Nursing Delirium Scale (Nu-DESC) for older patients in hospital. Methods: The Korean Nu-DESC was developed based on the Nu-DESC (Gaudreau, 2005), and revised according to nursing records related to signs and symptoms of older patients with delirium (n=361) and the results of a pilot study (n=42) in one general hospital. To test the validity of the Korean Nu-DESC, 75 older patients whom nurses suspected of delirium from 731 older patients from 12 nursing units were assessed by bedside nurses using the Korean Nu-DESC. A Receiver Operating Characteristic Curve of the Korean Nu-DESC was constructed with an accompanying Area Under the Curve (AUC). Results: Specific examples such as irritable, kidding, sleeping tendency, which were observed by bedside nurses in Korea, were identified in the five features of signs and symptoms of delirium in the instrument. The Korean Nu-DESC was psycho-metrically valid and had a sensitivity and specificity of .81-.76 and .97-.73, respectively. The AUC were .89, .74. Conclusion: Results of this study indicate that the Korean Nu-DESC is well-suited for widespread clinical use in busy inpatients settings and shows promise as a research instrument.

Detection of MicroRNA-21 Expression as a Potential Screening Biomarker for Colorectal Cancer: a Meta-analysis

  • Jiang, Jian-Xin;Zhang, Na;Liu, Zhong-Min;Wang, Yan-Ying
    • Asian Pacific Journal of Cancer Prevention
    • /
    • v.15 no.18
    • /
    • pp.7583-7588
    • /
    • 2014
  • Background: Colorectal cancer (CRC) is a major cause of cancer-related death and cancer-related incidence worldwide. The potential of microRNA-21 (miR-21) as a biomarker for CRC detection has been studied in several studies. However, the results were inconsistent. Therefore, we conducted the present meta-analysis to systematically assess the diagnostic value of miR-21 for CRC. Materials and Methods: Using a random-effect model, the pooled sensitivity (SEN), specificity (SPE), positive likelihood ratio (PLR), negative likelihood ratio (NLR), and diagnostic odds ratio (DOR) were calculated to evaluate the diagnostic performance of miR-21 for CRC. A summary receiver operating characteristic (SROC) curve and an area under the curve (AUC) were also generated to assess the diagnosis accuracy of miR-21 for CRC. Q test and I2 statistics were used to assess between-study heterogeneity. Publication bias was evaluated by the Deeks' funnel plot asymmetry test. Results: A total of 986 CRC patients and 702 matched healthy controls from 8 studies were involved in the meta-analysis. The pooled results for SEN, SPE, PLR, NLR, DOR, and AUC were 57% (95%CI: 39%-74%), 87% (95%CI: 78%-93%), 4.4 (95%CI: 2.4-8.0), 0.49 (95%CI: 0.32-0.74), 9 (95%CI: 4-22), and 0.83 (95%CI: 0.79-0.86), respectively. Subgroup analyses further suggested that blood-based studies showed a better diagnostic accuracy compared with feces-based studies, indicating that blood may be a better matrix for miR-21 assay and CRC detection. Conclusions: Our findings suggest that miR-21 has a potential diagnostic value for CRC with a moderate level of overall diagnostic accuracy. Hence, it could be used as auxiliary means for the initial screening of CRC and avoid unnecessary colonoscopy, which is an invasive and expensive procedure.

Use of positron emission tomography-computed tomography to predict axillary metastasis in patients with triple-negative breast cancer

  • Youm, Jung Hyun;Chung, Yoona;Yang, You Jung;Han, Sang Ah;Song, Jeong Yoon
    • Korean Journal of Clinical Oncology
    • /
    • v.14 no.2
    • /
    • pp.135-141
    • /
    • 2018
  • Purpose: Axillary lymph node dissection (ALND) and sentinel lymph node biopsy (SLNB) are important for staging of patients with node-positive breast cancer. However, these can be avoided in select micrometastatic diseases, preventing postoperative complications. The present study evaluated the ability of axillary lymph node maximum standardized uptake value (SUVmax) on positron emission tomography-computed tomography (PET-CT) to predict axillary metastasis of breast cancer. Methods: The records of invasive breast cancer patients who underwent pretreatment (surgery and/or chemotherapy) PET-CT between January 2006 and December 2014 were reviewed. ALNs were preoperatively evaluated by PET-CT. Lymph nodes were dissected by SLNB or ALND. SUVmax was measured in both the axillary lymph node and primary tumor. Student t-test and chi-square test were used to analyze sensitivity and specificity. Receiver operating characteristic (ROC) and area under the ROC curve (AUC) analyses were performed. Results: SUV-tumor (SUV-T) and SUV-lymph node (SUV-LN) were significantly higher in the triple-negative breast cancer (TNBC) group than in other groups (SUV-T: 5.99, P<0.01; SUV-LN: 1.29, P=0.014). The sensitivity (0.881) and accuracy (0.804) for initial ALN staging were higher in fine needle aspiration+PET-CT than in other methods. For PET-CT alone, the subtype with the highest sensitivity (0.870) and negative predictive value (0.917) was TNBC. The AUC for SUV-LN was greatest in TNBC (0.797). Conclusion: The characteristics of SUV-T and SUV-LN differed according to immunohistochemistry subtype. Compared to other subtypes, the true positivity of axillary metastasis on PET-CT was highest in TNBC. These findings could help tailor management for therapeutic and diagnostic purposes.