• Title/Summary/Keyword: ROC AUC

Search Result 292, Processing Time 0.024 seconds

Effect of Entrepreneurial Ecosystem Quality on Entrepreneurship Performance (창업 생태계 품질이 창업 성과에 미치는 영향)

  • Lee, Eun-Ji;Cho, Young-Ju
    • Journal of Korean Society for Quality Management
    • /
    • v.50 no.3
    • /
    • pp.305-332
    • /
    • 2022
  • Purpose: As the public interest in entrepreneurship has been highlighted and entrepreneurship policies have been generated, this study is to construct Entrepreneurship Ecosystem (EE) models which have a significant relationship to national entrepreneurship with quantitative analysis. It aims to provide implications to EE policymakers that which national components are effective in cultivating innovative entrepreneurship and validate its EE quality based on quantitative performance goals. Methods: This study utilizes secondary data, categorized under the PESTLE factor from credible international organizations (WB, UNDP, GEM, GEDI, and OECD) to determine significant factors in the quality of the entrepreneurial ecosystem. This paper uses the Multiple Linear Regression (MLR) analysis to select the significant variables contributing to entrepreneurship performance. Using the AUC-ROC performance evaluation method for machine learning MLR results, this paper evaluates the performance of EE models so that it can allow approving EE quality by predicting potential performance. Results: Among nine hypothesis models, MLR analysis examines that the number of the Unicorn company, Unicorn companies' economic value, and entrepreneurship measured as GEI can be reasonable dependent variables to indicate the performance derived from EE quality. Rather than government policies and regulations, the social, finance, technology, and economic variables are significant factors of EE quality determining its performance. By having high Area Under Curve values under AUC-ROC analysis, accepted MLR models are regarded as having high prediction accuracy. Conclusion: Superior EE contributes to the outstanding Unicorn companies, and improvement in macro-environmental components can enhance EE quality.

Siamese Neural Networks to Overcome the Insufficient Data Problems in Product Defect Detection (제품 결함 탐지에서 데이터 부족 문제를 극복하기 위한 샴 신경망의 활용)

  • Shin, Kang-hyeon;Jin, Kyo-hong
    • Proceedings of the Korean Institute of Information and Commucation Sciences Conference
    • /
    • 2022.05a
    • /
    • pp.108-111
    • /
    • 2022
  • Applying deep learning to machine vision systems for defect detection of products requires vast amounts of training data about various defect cases. However, since data imbalance occurs according to the type of defect in the actual manufacturing industry, it takes a lot of time to collect product images enough to generalize defect cases. In this paper, we apply a Siamese neural network that can be learned with even a small amount of data to product defect detection, and modify the image pairing method and contrastive loss function by properties the situation of product defect image data. We indirectly evaluated the embedding performance of Siamese neural networks using AUC-ROC, and it showed good performance when the images only paired among same products, not paired among defective products, and learned with exponential contrastive loss.

  • PDF

Disease Diagnosis on Fundus Images: A Cross-Dataset Study (망막 이미지에서의 질병 진단: 교차 데이터셋 연구)

  • Van-Nguyen Pham;Sun Xiaoying;Hyunseung Choo
    • Annual Conference of KIPS
    • /
    • 2024.10a
    • /
    • pp.754-755
    • /
    • 2024
  • This paper presents a comparative study of five deep learning models-ResNet50, DenseNet121, Vision Transformer (ViT), Swin Transformer (SwinT), and CoatNet-on the task of multi-label classification of fundus images for ocular diseases. The models were trained on the Ocular Disease Recognition (ODIR) dataset and validated on the Retinal Fundus Multi-disease Image Dataset (RFMiD), with a focus on five disease classes: diabetic retinopathy, glaucoma, cataract, age-related macular degeneration, and myopia. The performance was evaluated using the area under the receiver operating characteristic curve (AUC-ROC) score for each class. CoatNet achieved the best AUC-ROC scores for diabetic retinopathy, glaucoma, cataract, and myopia, while ViT outperformed CoatNet for age-related macular degeneration. Overall, CoatNet exhibited the highest average performance across all classes, highlighting the effectiveness of hybrid architectures in medical image classification. These findings suggest that CoatNet may be a promising model for multi-label classification of fundus images in cross-dataset scenarios.

A Comparative Study on the Performance of Machine Learning Algorithms and Key Feature Analysis for Predicting Heart Attack (심장마비 예측을 위한 머신러닝 알고리즘의 성능 비교 및 주요 변수 분석)

  • JunSu KOH;Min Soo KANG;Dong Hoon HAN
    • Journal of Korea Artificial Intelligence Association
    • /
    • v.2 no.2
    • /
    • pp.31-37
    • /
    • 2024
  • In this study, we compare the performance of various machine learning algorithms for predicting heart attacks, a major cause of mortality globally, with a focus on identifying key predictive features. Using a dataset of 918 records, the research evaluates models such as Random Forest, Logistic Regression, XGBoost, SVM, KNN, and Decision Tree to enhance prediction accuracy for heart attack risks. The methodology emphasizes robust preprocessing techniques, including feature scaling and handling class imbalances through Stratified K-Fold cross-validation, to improve model reliability. Results reveal that ensemble models, particularly Random Forest, achieve the highest ROC AUC score of 0.9301, significantly outperforming traditional algorithms. Key predictors, such as ST_Slope, were identified as critical variables in determining heart attack risks, while less influential features, such as RestingECG, had minimal impact. The findings underscore the efficacy of ensemble learning in predicting heart attacks and highlight the importance of feature importance analysis in enhancing model interpretability. This study provides valuable insights into the integration of machine learning in personalized healthcare, offering a foundation for future research to refine predictive models and improve early detection and prevention strategies for cardiovascular diseases.

Machine Learning-Based Prediction of COVID-19 Severity and Progression to Critical Illness Using CT Imaging and Clinical Data

  • Subhanik Purkayastha;Yanhe Xiao;Zhicheng Jiao;Rujapa Thepumnoeysuk;Kasey Halsey;Jing Wu;Thi My Linh Tran;Ben Hsieh;Ji Whae Choi;Dongcui Wang;Martin Vallieres;Robin Wang;Scott Collins;Xue Feng;Michael Feldman;Paul J. Zhang;Michael Atalay;Ronnie Sebro;Li Yang;Yong Fan;Wei-hua Liao;Harrison X. Bai
    • Korean Journal of Radiology
    • /
    • v.22 no.7
    • /
    • pp.1213-1224
    • /
    • 2021
  • Objective: To develop a machine learning (ML) pipeline based on radiomics to predict Coronavirus Disease 2019 (COVID-19) severity and the future deterioration to critical illness using CT and clinical variables. Materials and Methods: Clinical data were collected from 981 patients from a multi-institutional international cohort with real-time polymerase chain reaction-confirmed COVID-19. Radiomics features were extracted from chest CT of the patients. The data of the cohort were randomly divided into training, validation, and test sets using a 7:1:2 ratio. A ML pipeline consisting of a model to predict severity and time-to-event model to predict progression to critical illness were trained on radiomics features and clinical variables. The receiver operating characteristic area under the curve (ROC-AUC), concordance index (C-index), and time-dependent ROC-AUC were calculated to determine model performance, which was compared with consensus CT severity scores obtained by visual interpretation by radiologists. Results: Among 981 patients with confirmed COVID-19, 274 patients developed critical illness. Radiomics features and clinical variables resulted in the best performance for the prediction of disease severity with a highest test ROC-AUC of 0.76 compared with 0.70 (0.76 vs. 0.70, p = 0.023) for visual CT severity score and clinical variables. The progression prediction model achieved a test C-index of 0.868 when it was based on the combination of CT radiomics and clinical variables compared with 0.767 when based on CT radiomics features alone (p < 0.001), 0.847 when based on clinical variables alone (p = 0.110), and 0.860 when based on the combination of visual CT severity scores and clinical variables (p = 0.549). Furthermore, the model based on the combination of CT radiomics and clinical variables achieved time-dependent ROC-AUCs of 0.897, 0.933, and 0.927 for the prediction of progression risks at 3, 5 and 7 days, respectively. Conclusion: CT radiomics features combined with clinical variables were predictive of COVID-19 severity and progression to critical illness with fairly high accuracy.

Detection of Proximal Caries Lesions with Deep Learning Algorithm (심층학습 알고리즘을 활용한 인접면 우식 탐지)

  • Hyuntae, Kim;Ji-Soo, Song;Teo Jeon, Shin;Hong-Keun, Hyun;Jung-Wook, Kim;Ki-Taeg, Jang;Young-Jae, Kim
    • Journal of the korean academy of Pediatric Dentistry
    • /
    • v.49 no.2
    • /
    • pp.131-139
    • /
    • 2022
  • This study aimed to evaluate the effectiveness of deep convolutional neural networks (CNNs) for diagnosis of interproximal caries in pediatric intraoral radiographs. A total of 500 intraoral radiographic images of first and second primary molars were used for the study. A CNN model (Resnet 50) was applied for the detection of proximal caries. The diagnostic accuracy, sensitivity, specificity, receiver operating characteristic (ROC) curve, and area under ROC curve (AUC) were calculated on the test dataset. The diagnostic accuracy was 0.84, sensitivity was 0.74, and specificity was 0.94. The trained CNN algorithm achieved AUC of 0.86. The diagnostic CNN model for pediatric intraoral radiographs showed good performance with high accuracy. Deep learning can assist dentists in diagnosis of proximal caries lesions in pediatric intraoral radiographs.

Determination of cut-off value by receiver operating characteristic curve of norquetiapine and 9-hydroxyrisperidone concentrations in urine measured by LC-MS/MS

  • Kim, Seon Yeong;Shin, Dong Won;Kim, Jin Young
    • Analytical Science and Technology
    • /
    • v.34 no.2
    • /
    • pp.78-86
    • /
    • 2021
  • The objective of this study was to investigate urinary cut-off concentrations of quetiapine and risperidone for distinction between normal and abnormal/non-takers who were being placed on probation. Liquid chromatography-tandem mass spectrometric (LC-MS/MS) method was employed for determination of antipsychotic drugs in urine from mentally disordered probationers. The optimal cut-off values of antipsychotic drugs were calculated using receiver operating characteristic (ROC) curve analysis. The sensitivity and specificity of the method for the detection of antipsychotic drugs in urine were subsequently evaluated. The area under the ROC curve (AUC) was 0.927 for norquetiapine and 0.791 for 9-hydroxyrisperidone, respectively. These antipsychotic drugs are classified readily in the ROC curve analysis. The cut-off values for distinguishing regular and irregular/non-takers were 39.1 ng/mL for norquetiapine and 67.9 ng/mL for 9-hydroxyrisperidone, respectively. The results of this study suggest the cut-off values of quetiapine and risperidone were highly useful to distinguish regular takers from irregular/non-takers.

Assessment of Gait as a Diagnostic Tool for Patients with Dementia (치매 진단도구로서 치매노인의 보행능력 평가에 대한 연구)

  • Lee, Han-Suk;Park, Sun-Wook
    • Journal of the Korean Society of Physical Medicine
    • /
    • v.12 no.2
    • /
    • pp.129-136
    • /
    • 2017
  • PURPOSE: The purpose of this study was to compare the gait of elderly patients with and without dementia to investigate the possibility of an ambulation assessment test as a diagnostic tool for dementia. METHODS: A total of 96 subjects were included with 60 participants without dementia (control group) and 36 patients with dementia (dementia group). To compare the walking ability of the two groups, a 4-m walking test (4MWT) and Groningen Meander Walking Test (GMWT) were conducted. The GMWT is graded by amount of time in seconds and by number of oversteps outside the track. Mann-Whitney U test was used to compare the gait between the groups and the area under the curve (AUC) with Received Operating Characteristic (ROC) curve was analyzed. Statistical significance was considered at a p<.05, with a 95% confidence interval. RESULTS: There were statistically significant differences (p<.05) between the dementia group and the control group for the 4MWT, GMWTSEC, and GMWTSTEP scores. The AUC was .95 for 4MWT, .92 for GMWTSEC, and .96 for GMWTSTEP with the 95% confidence interval. The cut-off values of the ROC curve were 1.03m/s for 4MWT, 10.8 second for GMWTSEC, and 3.75 steps for GMSTEP. CONCLUSION: In our study, we investigated the utility of ambulatory assessment tools to predict dementia. The results of this study suggest that the 4MWT and the GMWT used in this study are appropriate assessment tools for dementia prediction.

Diagnostic performance of enzyme-linked immnosorbent assays for diagnosing paratuberculosis in cattle: a meta-analysis

  • Pak, Son-Il
    • Korean Journal of Veterinary Research
    • /
    • v.44 no.4
    • /
    • pp.669-676
    • /
    • 2004
  • To evaluate the diagnostic accuracy of two commercial ELISA tests (Allied- and CSL-ELISA) for the diagnosis of Mycobacterium paratuberculosis in cattle, Meta-analysis using English language papers published during 1990-2001 was performed. Diagnostic odds ratios (DOR) were analyzed using regression analysis together with summary receiver operating characteristic (ROC) curves. The difference in diagnostic performance between the two ELISA systems was evaluated by using linear regression. Publication bias was assessed by funnel plot and linear regression. The pooled sensitivity and specificity were 44% (95% CI, 38 to 51) and 98% (95% CI, 96 to 99) for the random-effect model. The DOR between studies was heterogeneous. The area under the fitted ROC curve (AUC) was 0.72 for the unweighted and 0.77 for the weighted model. Maximum joint sensitivity and specificity for the unweighted and weighted model from their summary ROC curve were 70% and 75%, respectively. Based on the fitted model, at a specificity of 95%, sensitivity was estimated to be 52% for the unweighted and 57% for the weighted model. From the final multivariable model study characteristic, the country was the only significant variable with an explained component variance of 13.3%. There were no significant differences in discriminatory power, sensitivity, and specificity between the two ELISA tests. The overall diagnostic accuracy of two commercial ELISA tests was moderate, as judged by the AUC, maximum joint sensitivity and specificity, and estimates from the fitted model and clinical usefulness of the tests for screening program is limited because of low sensitivity and heterogeneous of DOR. It is, therefore, recommended to use ELISA tests as a parallel testing with other diagnostic tests together to increase test sensitivity in the screening program.

Use of positron emission tomography-computed tomography to predict axillary metastasis in patients with triple-negative breast cancer

  • Youm, Jung Hyun;Chung, Yoona;Yang, You Jung;Han, Sang Ah;Song, Jeong Yoon
    • Korean Journal of Clinical Oncology
    • /
    • v.14 no.2
    • /
    • pp.135-141
    • /
    • 2018
  • Purpose: Axillary lymph node dissection (ALND) and sentinel lymph node biopsy (SLNB) are important for staging of patients with node-positive breast cancer. However, these can be avoided in select micrometastatic diseases, preventing postoperative complications. The present study evaluated the ability of axillary lymph node maximum standardized uptake value (SUVmax) on positron emission tomography-computed tomography (PET-CT) to predict axillary metastasis of breast cancer. Methods: The records of invasive breast cancer patients who underwent pretreatment (surgery and/or chemotherapy) PET-CT between January 2006 and December 2014 were reviewed. ALNs were preoperatively evaluated by PET-CT. Lymph nodes were dissected by SLNB or ALND. SUVmax was measured in both the axillary lymph node and primary tumor. Student t-test and chi-square test were used to analyze sensitivity and specificity. Receiver operating characteristic (ROC) and area under the ROC curve (AUC) analyses were performed. Results: SUV-tumor (SUV-T) and SUV-lymph node (SUV-LN) were significantly higher in the triple-negative breast cancer (TNBC) group than in other groups (SUV-T: 5.99, P<0.01; SUV-LN: 1.29, P=0.014). The sensitivity (0.881) and accuracy (0.804) for initial ALN staging were higher in fine needle aspiration+PET-CT than in other methods. For PET-CT alone, the subtype with the highest sensitivity (0.870) and negative predictive value (0.917) was TNBC. The AUC for SUV-LN was greatest in TNBC (0.797). Conclusion: The characteristics of SUV-T and SUV-LN differed according to immunohistochemistry subtype. Compared to other subtypes, the true positivity of axillary metastasis on PET-CT was highest in TNBC. These findings could help tailor management for therapeutic and diagnostic purposes.