• Title/Summary/Keyword: Area Under the Receiver Operating Characteristic Curve (AUC)

Search Result 163, Processing Time 0.025 seconds

A Prediction Triage System for Emergency Department During Hajj Period using Machine Learning Models

  • Huda N. Alhazmi
    • International Journal of Computer Science & Network Security
    • /
    • v.24 no.7
    • /
    • pp.11-23
    • /
    • 2024
  • Triage is a practice of accurately prioritizing patients in emergency department (ED) based on their medical condition to provide them with proper treatment service. The variation in triage assessment among medical staff can cause mis-triage which affect the patients negatively. Developing ED triage system based on machine learning (ML) techniques can lead to accurate and efficient triage outcomes. This study aspires to develop a triage system using machine learning techniques to predict ED triage levels using patients' information. We conducted a retrospective study using Security Forces Hospital ED data, from 2021 through 2023 during Hajj period in Saudia Arabi. Using demographics, vital signs, and chief complaints as predictors, two machine learning models were investigated, naming gradient boosted decision tree (XGB) and deep neural network (DNN). The models were trained to predict ED triage levels and their predictive performance was evaluated using area under the receiver operating characteristic curve (AUC) and confusion matrix. A total of 11,584 ED visits were collected and used in this study. XGB and DNN models exhibit high abilities in the predicting performance with AUC-ROC scores 0.85 and 0.82, respectively. Compared to the traditional approach, our proposed system demonstrated better performance and can be implemented in real-world clinical settings. Utilizing ML applications can power the triage decision-making, clinical care, and resource utilization.

Potential Impact of Climate Change on Distribution of Hedera rhombea in the Korean Peninsula (기후변화에 따른 송악의 잠재서식지 분포 변화 예측)

  • Park, Seon Uk;Koo, Kyung Ah;Seo, Changwan;Kong, Woo-Seok
    • Journal of Climate Change Research
    • /
    • v.7 no.3
    • /
    • pp.325-334
    • /
    • 2016
  • We projected the distribution of Hedera rhombea, an evergreen broad-leaved climbing plant, under current climate conditions and predicted its future distributions under global warming. Inaddition, weexplained model uncertainty by employing 9 single Species Distribution model (SDM)s to model the distribution of Hedera rhombea. 9 single SDMs were constructed with 736 presence/absence data and 3 temperature and 3 precipitation data. Uncertainty of each SDM was assessed with TSS (Ture Skill Statistics) and AUC (the Area under the curve) value of ROC (receiver operating characteristic) analyses. To reduce model uncertainty, we combined 9 single SDMs weighted by TSS and resulted in an ensemble forecast, a TSS weighted ensemble. We predicted future distributions of Hedera rhombea under future climate conditions for the period of 2050 (2040~2060), which were estimated with HadGEM2-AO. RF (Random Forest), GBM (Generalized Boosted Model) and TSS weighted ensemble model showed higher prediction accuracies (AUC > 0.95, TSS > 0.80) than other SDMs. Based on the projections of TSS weighted ensemble, potential habitats under current climate conditions showed a discrepancy with actual habitats, especially in the northern distribution limit. The observed northern boundary of Hedera rhombea is Ulsan in the eastern Korean Peninsula, but the projected limit was eastern coast of Gangwon province. Geomorphological conditions and the dispersal limitations mediated by birds, the lack of bird habitats at eastern coast of Gangwon Province, account for such discrepancy. In general, potential habitats of Hedera rhombea expanded under future climate conditions, but the extent of expansions depend on RCP scenarios. Potential Habitat of Hedera rhombea expanded into Jeolla-inland area under RCP 4.5, and into Chungnam and Wonsan under RCP 8.5. Our results would be fundamental information for understanding the potential effects of climate change on the distribution of Hedera rhombea.

Comparison of Abbreviated MRI and Full Diagnostic MRI in Distinguishing between Benign and Malignant Lesions Detected by Breast MRI: A Multireader Study

  • Eun Sil Kim;Nariya Cho;Soo-Yeon Kim;Bo Ra Kwon;Ann Yi;Su Min Ha;Su Hyun Lee;Jung Min Chang;Woo Kyung Moon
    • Korean Journal of Radiology
    • /
    • v.22 no.3
    • /
    • pp.297-307
    • /
    • 2021
  • Objective: To compare the performance of simulated abbreviated breast MRI (AB-MRI) and full diagnostic (FD)-MRI in distinguishing between benign and malignant lesions detected by MRI and investigate the features of discrepant lesions of the two protocols. Materials and Methods: An AB-MRI set with single first postcontrast images was retrospectively obtained from an FD-MRI cohort of 111 lesions (34 malignant, 77 benign) detected by contralateral breast MRI in 111 women (mean age, 49.8. ± 9.8; range, 28-75 years) with recently diagnosed breast cancer. Five blinded readers independently classified the likelihood of malignancy using Breast Imaging Reporting and Data System assessments. McNemar tests and area under the receiver operating characteristic curve (AUC) analyses were performed. The imaging and pathologic features of the discrepant lesions of the two protocols were analyzed. Results: The sensitivity of AB-MRI for lesion characterization tended to be lower than that of FD-MRI for all readers (58.8-82.4% vs. 79.4-100%), although the findings of only two readers were significantly different (p < 0.05). The specificity of AB-MRI for lesion characterization was higher than that of FD-MRI for 80% of readers (39.0-74.0% vs. 19.5-45.5%, p ≤ 0.001). The AUC of AB-MRI was comparable to that of FD-MRI for all readers (p > 0.05). Fifteen percent (5/34) of the cancers were false-negatives on AB-MRI. More suspicious margins or internal enhancement on the delayed phase images were related to the discrepancies. Conclusion: The overall performance of AB-MRI was similar to that of FD-MRI in distinguishing between benign and malignant lesions. AB-MRI showed lower sensitivity and higher specificity than FD-MRI, as 15% of the cancers were misclassified compared to FD-MRI.

Forensic Decision of Median Filtering Image Using a Coefficient of Variation of Fourier Transform (Fourier 변환 변이계수를 이용한 미디언 필터링 영상의 포렌식 판정)

  • RHEE, Kang Hyeon
    • Journal of the Institute of Electronics and Information Engineers
    • /
    • v.52 no.8
    • /
    • pp.67-73
    • /
    • 2015
  • In a distribution of digital image, there is a serious problem that is the image alteration by a forger. For the problem solution, this paper proposes the forensic decision algorithm of a median filtering (MF) image using the feature vector based on a coefficient of variation (c.v.) of Fourier transform. In the proposed algorithm, we compute Fourier transform (FT) coefficients of row and column line respectively of an image first, then c.v. between neighboring lines is computed. Subsquently, 10 Dim. feature vector is defined for the MF detection. On the experiment of MF detection, the proposed scheme is compared to MFR (Median Filter Residual) and Rhee's MF detection schemes that have the same 10 Dim. feature vector both. As a result, the performance is excellent at Unaltered, JPEG (QF=90), Down scaling (0.9) and Up scaling (1.1) images, and it showed good performance at Gaussian filtering ($3{\times}3$) image. However, in the performance evaluation of all measured items of the proposed scheme, AUC (Area Under ROC (Receiver Operating Characteristic) Curve) by the sensitivity and 1-specificity approached to 1 thus, it is confirmed that the grade of the performance evaluation is rated as 'Excellent (A)'.

Life Risk Assessment of Landslide Disaster in Jinbu Area Using Logistic Regression Model (로지스틱 회귀분석모델을 활용한 평창군 진부 지역의 산사태 재해의 인명 위험 평가)

  • Rahnuma, Bintae Rashid Urmi;Al, Mamun;Jang, Dong-Ho
    • Journal of The Geomorphological Association of Korea
    • /
    • v.27 no.2
    • /
    • pp.65-80
    • /
    • 2020
  • This paper deals with risk assessment of life in a landslide-prone area by a GIS-based modeling method. Landslide susceptibility maps can provide a probability of landslide prone areas to mitigate or proper control this problems and to take any development plan and disaster management. A landslide inventory map of the study area was prepared based on past historical information and aerial photography analysis. A total of 550 landslides have been counted at the whole study area. The extracted landslides were randomly selected and divided into two different groups, 50% of the landslides were used for model calibration and the other were used for validation purpose. Eleven causative factors (continuous and thematic) such as slope, aspect, curvature, topographic wetness index, elevation, forest type, forest crown density, geology, land-use, soil drainage, and soil texture were used in hazard analysis. The correlation between landslides and these factors, pixels were divided into several classes and frequency ratio was also extracted. Eventually, a landslide susceptibility map was constructed using a logistic regression model based on entire events. Moreover, the landslide susceptibility map was plotted with a receiver operating characteristic (ROC) curve and calculated the area under the curve (AUC) and tried to extract a success rate curve. Based on the results, logistic regression produced an 85.18% accuracy, so we believed that the model was reliable and acceptable for the landslide susceptibility analysis on the study area. In addition, for risk assessment, vulnerability scale were added for social thematic data layer. The study area predictive landslide affected pixels 2,000 and 5,000 were also calculated for making a probability table. In final calculation, the 2,000 predictive landslide affected pixels were assumed to run. The total population causalities were estimated as 7.75 person that was relatively close to the actual number published in Korean Annual Disaster Report, 2006.

Pediatric Dehydration Assessment at Triage: Prospective Study on Refilling Time

  • Caruggi, Samuele;Rossi, Martina;De Giacomo, Costantino;Luini, Chiara;Ruggiero, Nicola;Salvatoni, Alessandro;Salvatore, Silvia
    • Pediatric Gastroenterology, Hepatology & Nutrition
    • /
    • v.21 no.4
    • /
    • pp.278-288
    • /
    • 2018
  • Purpose: Dehydration is a paediatric medical emergency but there is no single standard parameter to evaluate it at the emergency department. Our aim was to evaluate the reliability and validity of capillary refilling time as a triage parameter to assess dehydration in children. Methods: This was a prospective pilot cohort study of children who presented to two paediatric emergency departments in Italy, with symptoms of dehydration. Reliability was assessed by comparing the triage nurse's measurements with those obtained by the physician. Validity was demonstrated by using 6 parameters suggestive of dehydration. Comparison between refilling time (RT) and a validated Clinical Dehydration Score (CDS) was also considered. The scale's discriminative ability was evaluated for the outcome of starting intravenous rehydration therapy by using a receiver operating characteristic (ROC) curve. Results: Participants were 242 children. All nurses found easy to elicit the RT after being trained. Interobserver reliability was fair, with a Cohen's kappa of 0.56 (95% confidence interval [CI], 0.41 to 0.70). There was a significant correlation between RT and weight loss percentage (r-squared=-0.27; 95% CI, -0.47 to -0.04). The scale's discriminative ability yielded an area under the ROC curve (AUC) of 0.65 (95% CI, 0.57 to 0.73). We found a similarity between RT AUC and CDS-scale AUC matching the two ROC curves. Conclusion: The study showed that RT represents a fast and handy tool to recognize dehydrated children who need a prompt rehydration and may be introduced in the triage line-up.

Diagnostic performance of enzyme-linked immnosorbent assays for diagnosing paratuberculosis in cattle: a meta-analysis

  • Pak, Son-Il
    • Korean Journal of Veterinary Research
    • /
    • v.44 no.4
    • /
    • pp.669-676
    • /
    • 2004
  • To evaluate the diagnostic accuracy of two commercial ELISA tests (Allied- and CSL-ELISA) for the diagnosis of Mycobacterium paratuberculosis in cattle, Meta-analysis using English language papers published during 1990-2001 was performed. Diagnostic odds ratios (DOR) were analyzed using regression analysis together with summary receiver operating characteristic (ROC) curves. The difference in diagnostic performance between the two ELISA systems was evaluated by using linear regression. Publication bias was assessed by funnel plot and linear regression. The pooled sensitivity and specificity were 44% (95% CI, 38 to 51) and 98% (95% CI, 96 to 99) for the random-effect model. The DOR between studies was heterogeneous. The area under the fitted ROC curve (AUC) was 0.72 for the unweighted and 0.77 for the weighted model. Maximum joint sensitivity and specificity for the unweighted and weighted model from their summary ROC curve were 70% and 75%, respectively. Based on the fitted model, at a specificity of 95%, sensitivity was estimated to be 52% for the unweighted and 57% for the weighted model. From the final multivariable model study characteristic, the country was the only significant variable with an explained component variance of 13.3%. There were no significant differences in discriminatory power, sensitivity, and specificity between the two ELISA tests. The overall diagnostic accuracy of two commercial ELISA tests was moderate, as judged by the AUC, maximum joint sensitivity and specificity, and estimates from the fitted model and clinical usefulness of the tests for screening program is limited because of low sensitivity and heterogeneous of DOR. It is, therefore, recommended to use ELISA tests as a parallel testing with other diagnostic tests together to increase test sensitivity in the screening program.

Linear prediction analysis-based method for detecting snapping shrimp noise (선형 예측 분석 기반의 딱총 새우 잡음 검출 기법)

  • Jinuk Park;Jungpyo Hong
    • The Journal of the Acoustical Society of Korea
    • /
    • v.42 no.3
    • /
    • pp.262-269
    • /
    • 2023
  • In this paper, we propose a Linear Prediction (LP) analysis-based feature for detecting Snapping Shrimp (SS) Noise (SSN) in underwater acoustic data. SS is a species that creates high amplitude signals in shallow, warm waters, and its frequent and loud sound is a major source of noise. The proposed feature takes advantage of the characteristic of SSN, which is sudden and rapidly disappearing, by using LP analysis to detect the exact noise interval and reduce the effects of SSN. The error between the predicted and measured value is large and results in effective SSN detection. To further improve performance, a constant false alarm rate detector is incorporated into the proposed feature. Our evaluation shows that the proposed methods outperform the state-of-the-art MultiLayer-Wavelet Packet Decomposition (ML-WPD) in terms of receiver operating characteristic curve and Area Under the Curve (AUC), with the LP analysis-based feature achieving a higher AUC by 0.12 on average and lower computational complexity.

Cut-Off Values of the Post-Intensive Care Syndrome Questionnaire for the Screening of Unplanned Hospital Readmission within One Year

  • Kang, Jiyeon;Jeong, Yeon Jin;Hong, Jiwon
    • Journal of Korean Academy of Nursing
    • /
    • v.50 no.6
    • /
    • pp.787-798
    • /
    • 2020
  • Purpose: This study aimed to assign weights for subscales and items of the Post-Intensive Care Syndrome questionnaire and suggest optimal cut-off values for screening unplanned hospital readmissions of critical care survivors. Methods: Seventeen experts participated in an analytic hierarchy process for weight assignment. Participants for cut-off analysis were 240 survivors who had been admitted to intensive care units for more than 48 hours in three cities in Korea. We assessed participants using the 18-item Post-Intensive Care Syndrome questionnaire, generated receiver operating characteristic curves, and analysed cut-off values for unplanned readmission based on sensitivity, specificity, and positive likelihood ratios. Results: Cognitive, physical, and mental subscale weights were 1.13, 0.95, and 0.92, respectively. Incidence of unplanned readmission was 25.4%. Optimal cut-off values were 23.00 for raw scores and 23.73 for weighted scores (total score 54.00), with an area of under the curve (AUC) of .933 and .929, respectively. There was no significant difference in accuracy for original and weighted scores. Conclusion: The optimal cut-off value accuracy is excellent for screening of unplanned readmissions. We recommend that nurses use the Post-Intensive Care Syndrome Questionnaire to screen for readmission risk or evaluating relevant interventions for critical care survivors.

Evaluation of Galactomannan Enzyme Immunoassay and Quantitative Real-Time PCR for the Diagnosis of Invasive Pulmonary Aspergillosis in a Rat Model

  • Lin, Jian-Cong;Xing, Yan-Li;Xu, Wen-Ming;Li, Ming;Bo, Pang;Niu, Yuan-Yuan;Zhang, Chang-Ran
    • Journal of Microbiology and Biotechnology
    • /
    • v.24 no.8
    • /
    • pp.1044-1050
    • /
    • 2014
  • Since there is no consensus about the most reliable assays to detect invasive aspergillosis from samples obtained by minimally invasive or noninvasive methods, we compared the efficacy of an enzyme-linked immunosorbent assay (ELISA) for galactomannan (GM) detection and quantitative real-time PCR assay (qRT-PCR) for the diagnosis of invasive pulmonary aspergillosis. Neutropenic, male Sprague-Dawley rats (specific pathogen free; 8 weeks old; weight, $200{\pm}20g$) were immunosuppressed with cyclophosphamide and infected with Aspergillus fumigatus intratracheally. Tissue and whole blood samples were harvested on days 1, 3, 5, and 7 post-infection and examined with GM ELISA and qRT-PCR. The A. fumigatus DNA detection sequence was detected in the following number of samples from 12 immunosuppressed, infected rats examined on the scheduled days: day 1 (0/12), day 3 (0/12), day 5 (6/12), and day 7 (8/12) post-infection. The sensitivity and specificity of the qRT-PCR assay was 29.2% and 100%, respectively. Receiver operating characteristic curve (ROC) analysis indicated a Ct (cycle threshold) cut-off value of 15.35, and the area under the curve (AUC) was 0.627. The GM assay detected antigen in sera obtained on day 1 (5/12), day 3 (9/12), day 5 (12/12), and day 7 (12/12) post-infection, and thus had a sensitivity of 79.2% and a specificity of 100%. The ROC of the GM assay indicated that the optimal Ct cut-off value was 1.40 (AUC, 0.919). The GM assay was more sensitive than the qRT-PCR assay in diagnosing invasive pulmonary aspergillosis in rats.