• Title/Summary/Keyword: Area Under the Receiver Operating Characteristic Curve (AUC)

Search Result 163, Processing Time 0.036 seconds

Use of an Artificial Neural Network to Construct a Model of Predicting Deep Fungal Infection in Lung Cancer Patients

  • Chen, Jian;Chen, Jie;Ding, Hong-Yan;Pan, Qin-Shi;Hong, Wan-Dong;Xu, Gang;Yu, Fang-You;Wang, Yu-Min
    • Asian Pacific Journal of Cancer Prevention
    • /
    • v.16 no.12
    • /
    • pp.5095-5099
    • /
    • 2015
  • Background: The statistical methods to analyze and predict the related dangerous factors of deep fungal infection in lung cancer patients were several, such as logic regression analysis, meta-analysis, multivariate Cox proportional hazards model analysis, retrospective analysis, and so on, but the results are inconsistent. Materials and Methods: A total of 696 patients with lung cancer were enrolled. The factors were compared employing Student's t-test or the Mann-Whitney test or the Chi-square test and variables that were significantly related to the presence of deep fungal infection selected as candidates for input into the final artificial neural network analysis (ANN) model. The receiver operating characteristic (ROC) and area under curve (AUC) were used to evaluate the performance of the artificial neural network (ANN) model and logistic regression (LR) model. Results: The prevalence of deep fungal infection from lung cancer in this entire study population was 32.04%(223/696), deep fungal infections occur in sputum specimens 44.05%(200/454). The ratio of candida albicans was 86.99% (194/223) in the total fungi. It was demonstrated that older (${\geq}65$ years), use of antibiotics, low serum albumin concentrations (${\leq}37.18g/L$), radiotherapy, surgery, low hemoglobin hyperlipidemia (${\leq}93.67g/L$), long time of hospitalization (${\geq}14$days) were apt to deep fungal infection and the ANN model consisted of the seven factors. The AUC of ANN model($0.829{\pm}0.019$)was higher than that of LR model ($0.756{\pm}0.021$). Conclusions: The artificial neural network model with variables consisting of age, use of antibiotics, serum albumin concentrations, received radiotherapy, received surgery, hemoglobin, time of hospitalization should be useful for predicting the deep fungal infection in lung cancer.

Texture Analysis of Three-Dimensional MRI Images May Differentiate Borderline and Malignant Epithelial Ovarian Tumors

  • Rongping Ye;Shuping Weng;Yueming Li;Chuan Yan;Jianwei Chen;Yuemin Zhu;Liting Wen
    • Korean Journal of Radiology
    • /
    • v.22 no.1
    • /
    • pp.106-117
    • /
    • 2021
  • Objective: To explore the value of magnetic resonance imaging (MRI)-based whole tumor texture analysis in differentiating borderline epithelial ovarian tumors (BEOTs) from FIGO stage I/II malignant epithelial ovarian tumors (MEOTs). Materials and Methods: A total of 88 patients with histopathologically confirmed ovarian epithelial tumors after surgical resection, including 30 BEOT and 58 MEOT patients, were divided into a training group (n = 62) and a test group (n = 26). The clinical and conventional MRI features were retrospectively reviewed. The texture features of tumors, based on T2-weighted imaging, diffusion-weighted imaging, and contrast-enhanced T1-weighted imaging, were extracted using MaZda software and the three top weighted texture features were selected by using the Random Forest algorithm. A non-texture logistic regression model in the training group was built to include those clinical and conventional MRI variables with p value < 0.10. Subsequently, a combined model integrating non-texture information and texture features was built for the training group. The model, evaluated using patients in the training group, was then applied to patients in the test group. Finally, receiver operating characteristic (ROC) curves were used to assess the diagnostic performance of the models. Results: The combined model showed superior performance in categorizing BEOTs and MEOTs (sensitivity, 92.5%; specificity, 86.4%; accuracy, 90.3%; area under the ROC curve [AUC], 0.962) than the non-texture model (sensitivity, 78.3%; specificity, 84.6%; accuracy, 82.3%; AUC, 0.818). The AUCs were statistically different (p value = 0.038). In the test group, the AUCs, sensitivity, specificity, and accuracy were 0.840, 73.3%, 90.1%, and 80.8% when the non-texture model was used and 0.896, 75.0%, 94.0%, and 88.5% when the combined model was used. Conclusion: MRI-based texture features combined with clinical and conventional MRI features may assist in differentitating between BEOT and FIGO stage I/II MEOT patients.

Deep Learning-Enabled Detection of Pneumoperitoneum in Supine and Erect Abdominal Radiography: Modeling Using Transfer Learning and Semi-Supervised Learning

  • Sangjoon Park;Jong Chul Ye;Eun Sun Lee;Gyeongme Cho;Jin Woo Yoon;Joo Hyeok Choi;Ijin Joo;Yoon Jin Lee
    • Korean Journal of Radiology
    • /
    • v.24 no.6
    • /
    • pp.541-552
    • /
    • 2023
  • Objective: Detection of pneumoperitoneum using abdominal radiography, particularly in the supine position, is often challenging. This study aimed to develop and externally validate a deep learning model for the detection of pneumoperitoneum using supine and erect abdominal radiography. Materials and Methods: A model that can utilize "pneumoperitoneum" and "non-pneumoperitoneum" classes was developed through knowledge distillation. To train the proposed model with limited training data and weak labels, it was trained using a recently proposed semi-supervised learning method called distillation for self-supervised and self-train learning (DISTL), which leverages the Vision Transformer. The proposed model was first pre-trained with chest radiographs to utilize common knowledge between modalities, fine-tuned, and self-trained on labeled and unlabeled abdominal radiographs. The proposed model was trained using data from supine and erect abdominal radiographs. In total, 191212 chest radiographs (CheXpert data) were used for pre-training, and 5518 labeled and 16671 unlabeled abdominal radiographs were used for fine-tuning and self-supervised learning, respectively. The proposed model was internally validated on 389 abdominal radiographs and externally validated on 475 and 798 abdominal radiographs from the two institutions. We evaluated the performance in diagnosing pneumoperitoneum using the area under the receiver operating characteristic curve (AUC) and compared it with that of radiologists. Results: In the internal validation, the proposed model had an AUC, sensitivity, and specificity of 0.881, 85.4%, and 73.3% and 0.968, 91.1, and 95.0 for supine and erect positions, respectively. In the external validation at the two institutions, the AUCs were 0.835 and 0.852 for the supine position and 0.909 and 0.944 for the erect position. In the reader study, the readers' performances improved with the assistance of the proposed model. Conclusion: The proposed model trained with the DISTL method can accurately detect pneumoperitoneum on abdominal radiography in both the supine and erect positions.

Performance of the R-way Colposcopic Evaluation System in Cervical Cancer Screening

  • Zhao, Jian;Zhang, Xi;Chen, Rui;Zhao, Yu-Qian;Wang, Ting-Ting;He, Shan;Qiao, You-Lin
    • Asian Pacific Journal of Cancer Prevention
    • /
    • v.16 no.10
    • /
    • pp.4223-4228
    • /
    • 2015
  • Objective: To investigate the diagnostic value of the R-way colposcopic evaluation system (R-way system) in cervical cancer screening. Materials and Methods: Between August 2013 and August 2014, a total of 1,059 cases referred to colposcopy in Peking University First Hospital were studied using both the R-way system and conventional colposcopy. Our study evaluated and compared the diagnostic ability of the two methods in detecting high-grade lesions and cervical cancer (hereinafter called CIN2+). Evaluation indicators including sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), Youden index and the area under the curve (AUC) of the receiver operating characteristic (ROC) were calculated. Results: The R-way system had a slightly lower specificity (94.5%) than conventional colposcopy (96.0%) for CIN2+ detection (P=0.181). However, the sensitivity (77.8%) was significantly higher than with the conventional colposcopic method (46.6%) (${\chi}^2=64.351$, P<0.001). In addition, the AUC of the ROC for CIN2+ detection using the R-way system (0.839) was larger than that with conventional colposcopy (0.731) (Z=4.348, P<0.001). If preliminary result had been drawn from cervical exfoliated cytology before colposcopy referral, combination of the R-way system with cytology could increase the sensitivity to 93.9% for CIN2+ detection (excluding ASCUS\LSIL), confirmed by multipoint biopsy or ECC. Conclusions: The diagnostic value of the R-way evaluation system is higher than that of conventional colposcopic evaluation in cervical cancer screening. Moreover, taking the ease of use and standardized quality control management into account, the R-way system is highly preferable.

Network Anomaly Detection Technologies Using Unsupervised Learning AutoEncoders (비지도학습 오토 엔코더를 활용한 네트워크 이상 검출 기술)

  • Kang, Koohong
    • Journal of the Korea Institute of Information Security & Cryptology
    • /
    • v.30 no.4
    • /
    • pp.617-629
    • /
    • 2020
  • In order to overcome the limitations of the rule-based intrusion detection system due to changes in Internet computing environments, the emergence of new services, and creativity of attackers, network anomaly detection (NAD) using machine learning and deep learning technologies has received much attention. Most of these existing machine learning and deep learning technologies for NAD use supervised learning methods to learn a set of training data set labeled 'normal' and 'attack'. This paper presents the feasibility of the unsupervised learning AutoEncoder(AE) to NAD from data sets collecting of secured network traffic without labeled responses. To verify the performance of the proposed AE mode, we present the experimental results in terms of accuracy, precision, recall, f1-score, and ROC AUC value on the NSL-KDD training and test data sets. In particular, we model a reference AE through the deep analysis of diverse AEs varying hyper-parameters such as the number of layers as well as considering the regularization and denoising effects. The reference model shows the f1-scores 90.4% and 89% of binary classification on the KDDTest+ and KDDTest-21 test data sets based on the threshold of the 82-th percentile of the AE reconstruction error of the training data set.

99mTc-3PRGD2 SPECT/CT Imaging for Diagnosing Lymph Node Metastasis of Primary Malignant Lung Tumors

  • Liming Xiao;Shupeng Yu;Weina Xu;Yishan Sun;Jun Xin
    • Korean Journal of Radiology
    • /
    • v.24 no.11
    • /
    • pp.1142-1150
    • /
    • 2023
  • Objective: To evaluate 99mtechnetium-three polyethylene glycol spacers-arginine-glycine-aspartic acid (99mTc-3PRGD2) single-photon emission computed tomography (SPECT)/computed tomography (CT) imaging for diagnosing lymph node metastasis of primary malignant lung neoplasms. Materials and Methods: We prospectively enrolled 26 patients with primary malignant lung tumors who underwent 99mTc-3PRGD2 SPECT/CT and 18F-fluorodeoxyglucose (18F-FDG) positron emission tomography (PET)/CT imaging. Both imaging methods were analyzed in qualitative (visual dichotomous and 5-point grades for lymph nodes and lung tumors, respectively) and semiquantitative (maximum tissue-to-background radioactive count) manners for the lymph nodes and lung tumors. The performance of the differentiation of lymph nodes with and without metastasis was determined at the per-lymph node station and per-patient levels using histopathological results as the reference standard. Results: Total 42 stations had metastatic lymph nodes and 136 stations had benign lymph nodes. The differences between metastatic and benign lymph nodes in the visual qualitative and semiquantitative analyses of 99mTc-3PRGD2 SPECT/CT and 18F-FDG PET/CT were statistically significant (all P < 0.001). The area under the receiver operating characteristic curve (AUC) in the semi-quantitative analysis of 99mTc-3PRGD2 SPECT/CT was 0.908 (95% confidence interval [CI], 0.851-0.966), and the sensitivity, specificity, positive predictive value, and negative predictive value were 0.86 (36/42), 0.88 (120/136), 0.69 (36/52), and 0.95 (120/126), respectively. Among the 26 patients (including two patients each with two lung tumors), 15 had pathologically confirmed lymph node metastasis. The difference between primary lung lesions in patients with and without lymph node metastasis was statistically significant only in the semi-quantitative analysis of 99mTc-3PRGD2 SPECT/CT (P = 0.007), with an AUC of 0.807 (95% CI, 0.641-0.974). Conclusion: 99mTc-3PRGD2 SPECT/CT imaging may notably perform in the direct diagnosis of lymph node metastasis of primary malignant lung tumors and indirectly predict the presence of lymph node metastasis through uptake in the primary lesions.

Role of Myocardial Extracellular Volume Fraction Measured with Magnetic Resonance Imaging in the Prediction of Left Ventricular Functional Outcome after Revascularization of Chronic Total Occlusion of Coronary Arteries

  • Yinyin Chen;Xinde Zheng;Hang Jin;Shengming Deng;Daoyuan Ren;Andreas Greiser;Caixia Fu;Hongxiang Gao;Mengsu Zeng
    • Korean Journal of Radiology
    • /
    • v.20 no.1
    • /
    • pp.83-93
    • /
    • 2019
  • Objective: The purpose of this study was to prospectively investigate the value of the myocardial extracellular volume fraction (ECV) in predicting myocardial functional outcome after revascularization of coronary chronic total occlusion (CTO). Materials and Methods: Thirty patients with CTO underwent cardiovascular magnetic resonance (CMR) before and 6 months after revascularization. Three baseline markers of functional outcome were evaluated in the dysfunctional segments assigned to the CTO vessels: ECV, transmural extent of infarction (TEI), and unenhanced rim thickness (RIM). At the global level, the ECV values of the whole myocardium with and without a hyperenhanced region (global and remote ECV) were respectively measured. Results: In per-segment analysis, ECV was superior to TEI and RIM in predicting functional recovery (area under receiver operating characteristic curve [AUC]: 0.86 vs. 0.75 and 0.73, all p values < 0.010), and it emerged as the only independent predictor of regional functional outcome (odds ratio [OR] = 0.83, 95% confidence interval [CI]: 0.77-0.89; p < 0.001) independent of collateral circulation. In per-patient analysis, global baseline ECV was indicative of ejection fraction (EF) at the follow-up examination (β = -0.61, p < 0.001) and changes in EF (β = -0.57, p = 0.001) in multivariate regression analysis. A patient with global baseline ECV less than 30.0% (AUC, 0.93; sensitivity 94%, specificity 80%) was more likely to demonstrate significant EF improvement (OR: 0.38; 95% CI: 0.17-0.85; p = 0.019). Conclusion: Extracellular volume fraction obtained by CMR may provide incremental value for the prediction of functional recovery both at the segmental and global levels in CTO patients, and may facilitate the identification of patients who can benefit from revascularization.

Performance Comparison of Machine Learning based Prediction Models for University Students Dropout (머신러닝 기반 대학생 중도 탈락 예측 모델의 성능 비교)

  • Seok-Bong Jeong;Du-Yon Kim
    • Journal of the Korea Society for Simulation
    • /
    • v.32 no.4
    • /
    • pp.19-26
    • /
    • 2023
  • The increase in the dropout rate of college students nationwide has a serious negative impact on universities and society as well as individual students. In order to proactive identify students at risk of dropout, this study built a decision tree, random forest, logistic regression, and deep learning-based dropout prediction model using academic data that can be easily obtained from each university's academic management system. Their performances were subsequently analyzed and compared. The analysis revealed that while the logistic regression-based prediction model exhibited the highest recall rate, its f-1 value and ROC-AUC (Receiver Operating Characteristic - Area Under the Curve) value were comparatively lower. On the other hand, the random forest-based prediction model demonstrated superior performance across all other metrics except recall value. In addition, in order to assess model performance over distinct prediction periods, we divided these periods into short-term (within one semester), medium-term (within two semesters), and long-term (within three semesters). The results underscored that the long-term prediction yielded the highest predictive efficacy. Through this study, each university is expected to be able to identify students who are expected to be dropped out early, reduce the dropout rate through intensive management, and further contribute to the stabilization of university finances.

Use of Artificial Intelligence for Reducing Unnecessary Recalls at Screening Mammography: A Simulation Study

  • Yeon Soo Kim;Myoung-jin Jang;Su Hyun Lee;Soo-Yeon Kim;Su Min Ha;Bo Ra Kwon;Woo Kyung Moon;Jung Min Chang
    • Korean Journal of Radiology
    • /
    • v.23 no.12
    • /
    • pp.1241-1250
    • /
    • 2022
  • Objective: To conduct a simulation study to determine whether artificial intelligence (AI)-aided mammography reading can reduce unnecessary recalls while maintaining cancer detection ability in women recalled after mammography screening. Materials and Methods: A retrospective reader study was performed by screening mammographies of 793 women (mean age ± standard deviation, 50 ± 9 years) recalled to obtain supplemental mammographic views regarding screening mammography-detected abnormalities between January 2016 and December 2019 at two screening centers. Initial screening mammography examinations were interpreted by three dedicated breast radiologists sequentially, case by case, with and without AI aid, in a single session. The area under the receiver operating characteristic curve (AUC), sensitivity, specificity, and recall rate for breast cancer diagnosis were obtained and compared between the two reading modes. Results: Fifty-four mammograms with cancer (35 invasive cancers and 19 ductal carcinomas in situ) and 739 mammograms with benign or negative findings were included. The reader-averaged AUC improved after AI aid, from 0.79 (95% confidence interval [CI], 0.74-0.85) to 0.89 (95% CI, 0.85-0.94) (p < 0.001). The reader-averaged specificities before and after AI aid were 41.9% (95% CI, 39.3%-44.5%) and 53.9% (95% CI, 50.9%-56.9%), respectively (p < 0.001). The reader-averaged sensitivity was not statistically different between AI-unaided and AI-aided readings: 89.5% (95% CI, 83.1%-95.9%) vs. 92.6% (95% CI, 86.2%-99.0%) (p = 0.053), although the sensitivities of the least experienced radiologists before and after AI aid were 79.6% (43 of 54 [95% CI, 66.5%-89.4%]) and 90.7% (49 of 54 [95% CI, 79.7%-96.9%]), respectively (p = 0.031). With AI aid, the reader-averaged recall rate decreased by from 60.4% (95% CI, 57.8%-62.9%) to 49.5% (95% CI, 46.5%-52.4%) (p < 0.001). Conclusion: AI-aided reading reduced the number of recalls and improved the diagnostic performance in our simulation using women initially recalled for supplemental mammographic views after mammography screening.

Prediction of Axillary Lymph Node Metastasis in Early Breast Cancer Using Dynamic Contrast-Enhanced Magnetic Resonance Imaging and Diffusion-Weighted Imaging

  • Jeong, Eun Ha;Choi, Eun Jung;Choi, Hyemi;Park, Eun Hae;Song, Ji Soo
    • Investigative Magnetic Resonance Imaging
    • /
    • v.23 no.2
    • /
    • pp.125-135
    • /
    • 2019
  • Purpose: The purpose of this study was to evaluate dynamic contrast-enhanced breast magnetic resonance imaging (DCE-MRI), and diffusion-weighted imaging (DWI) variables, for axillary lymph node (ALN) metastasis in the early stage of breast cancer. Materials and Methods: January 2011-April 2015, 787 patients with early stage of breast cancer were retrospectively reviewed. Only cases of invasive ductal carcinoma, were included in the patient population. Among them, 240 patients who underwent 3.0-T DCE-MRI, including DWI with b value 0 and $800s/mm^2$ were enrolled. MRI variables (adjacent vessel sign, whole-breast vascularity, initial enhancement pattern, quantitative kinetic parameters, signal enhancement ratio (SER), tumor apparent diffusion coefficient (ADC), peritumoral ADC, and peritumor-tumor ADC ratio) clinico-pathologic variables (age, T stage, multifocality, extensive intraductal carcinoma component (EIC), estrogen receptor, progesterone receptor, HER-2 status, Ki-67, molecular subtype, histologic grade, and nuclear grade) were compared between patients with axillary lymph node metastasis and those with no lymph node metastasis. Multivariate regression analysis was performed, to determine independent variables associated with ALN metastasis, and the area under the receiver operating characteristic curve (AUC), for predicting ALN metastasis was analyzed, for those variables. Results: On breast MRI, moderate or prominent ipsilateral whole-breast vascularity (moderate, odds ratio [OR] 3.45, 95% confidence interval [CI] 1.28-9.51 vs. prominent, OR = 15.59, 95% CI 2.52-96.46), SER (OR = 1.68, 95% CI 1.09-2.59), and peritumor-tumor ADC ratio (OR = 6.77, 95% CI 2.41-18.99), were independently associated with ALN metastasis. Among clinico-pathologic variables, HER-2 positivity was independently associated, with ALN metastasis (OR = 23.71, 95% CI 10.50-53.54). The AUC for combining selected MRI variables and clinico-pathologic variables, was higher than that of clinico-pathologic variables (P < 0.05). Conclusion: SER, moderate or prominent increased whole breast vascularity, and peritumor-tumor ADC ratio on breast MRI, are valuable in predicting ALN metastasis, in patients with early stage of breast cancer.