• Title/Summary/Keyword: Precision-recall curve

Search Result 38, Processing Time 0.026 seconds

Assessment of the Object Detection Ability of Interproximal Caries on Primary Teeth in Periapical Radiographs Using Deep Learning Algorithms (유치의 치근단 방사선 사진에서 딥 러닝 알고리즘을 이용한 모델의 인접면 우식증 객체 탐지 능력의 평가)

  • Hongju Jeon;Seonmi Kim;Namki Choi
    • Journal of the korean academy of Pediatric Dentistry
    • /
    • v.50 no.3
    • /
    • pp.263-276
    • /
    • 2023
  • The purpose of this study was to evaluate the performance of a model using You Only Look Once (YOLO) for object detection of proximal caries in periapical radiographs of children. A total of 2016 periapical radiographs in primary dentition were selected from the M6 database as a learning material group, of which 1143 were labeled as proximal caries by an experienced dentist using an annotation tool. After converting the annotations into a training dataset, YOLO was trained on the dataset using a single convolutional neural network (CNN) model. Accuracy, recall, specificity, precision, negative predictive value (NPV), F1-score, Precision-Recall curve, and AP (area under curve) were calculated for evaluation of the object detection model's performance in the 187 test datasets. The results showed that the CNN-based object detection model performed well in detecting proximal caries, with a diagnostic accuracy of 0.95, a recall of 0.94, a specificity of 0.97, a precision of 0.82, a NPV of 0.96, and an F1-score of 0.81. The AP was 0.83. This model could be a valuable tool for dentists in detecting carious lesions in periapical radiographs.

The application of convolutional neural networks for automatic detection of underwater object in side scan sonar images (사이드 스캔 소나 영상에서 수중물체 자동 탐지를 위한 컨볼루션 신경망 기법 적용)

  • Kim, Jungmoon;Choi, Jee Woong;Kwon, Hyuckjong;Oh, Raegeun;Son, Su-Uk
    • The Journal of the Acoustical Society of Korea
    • /
    • v.37 no.2
    • /
    • pp.118-128
    • /
    • 2018
  • In this paper, we have studied how to search an underwater object by learning the image generated by the side scan sonar in the convolution neural network. In the method of human side analysis of the side scan image or the image, the convolution neural network algorithm can enhance the efficiency of the analysis. The image data of the side scan sonar used in the experiment is the public data of NSWC (Naval Surface Warfare Center) and consists of four kinds of synthetic underwater objects. The convolutional neural network algorithm is based on Faster R-CNN (Region based Convolutional Neural Networks) learning based on region of interest and the details of the neural network are self-organized to fit the data we have. The results of the study were compared with a precision-recall curve, and we investigated the applicability of underwater object detection in convolution neural networks by examining the effect of change of region of interest assigned to sonar image data on detection performance.

Deep Learning-based Spine Segmentation Technique Using the Center Point of the Spine and Modified U-Net (척추의 중심점과 Modified U-Net을 활용한 딥러닝 기반 척추 자동 분할)

  • Sungjoo Lim;Hwiyoung Kim
    • Journal of Biomedical Engineering Research
    • /
    • v.44 no.2
    • /
    • pp.139-146
    • /
    • 2023
  • Osteoporosis is a disease in which the risk of bone fractures increases due to a decrease in bone density caused by aging. Osteoporosis is diagnosed by measuring bone density in the total hip, femoral neck, and lumbar spine. To accurately measure bone density in the lumbar spine, the vertebral region must be segmented from the lumbar X-ray image. Deep learning-based automatic spinal segmentation methods can provide fast and precise information about the vertebral region. In this study, we used 695 lumbar spine images as training and test datasets for a deep learning segmentation model. We proposed a lumbar automatic segmentation model, CM-Net, which combines the center point of the spine and the modified U-Net network. As a result, the average Dice Similarity Coefficient(DSC) was 0.974, precision was 0.916, recall was 0.906, accuracy was 0.998, and Area under the Precision-Recall Curve (AUPRC) was 0.912. This study demonstrates a high-performance automatic segmentation model for lumbar X-ray images, which overcomes noise such as spinal fractures and implants. Furthermore, we can perform accurate measurement of bone density on lumbar X-ray images using an automatic segmentation methodology for the spine, which can prevent the risk of compression fractures at an early stage and improve the accuracy and efficiency of osteoporosis diagnosis.

Human Detection using Real-virtual Augmented Dataset

  • Jongmin, Lee;Yongwan, Kim;Jinsung, Choi;Ki-Hong, Kim;Daehwan, Kim
    • Journal of information and communication convergence engineering
    • /
    • v.21 no.1
    • /
    • pp.98-102
    • /
    • 2023
  • This paper presents a study on how augmenting semi-synthetic image data improves the performance of human detection algorithms. In the field of object detection, securing a high-quality data set plays the most important role in training deep learning algorithms. Recently, the acquisition of real image data has become time consuming and expensive; therefore, research using synthesized data has been conducted. Synthetic data haves the advantage of being able to generate a vast amount of data and accurately label it. However, the utility of synthetic data in human detection has not yet been demonstrated. Therefore, we use You Only Look Once (YOLO), the object detection algorithm most commonly used, to experimentally analyze the effect of synthetic data augmentation on human detection performance. As a result of training YOLO using the Penn-Fudan dataset, it was shown that the YOLO network model trained on a dataset augmented with synthetic data provided high-performance results in terms of the Precision-Recall Curve and F1-Confidence Curve.

Machine Learning for Flood Prediction in Indonesia: Providing Online Access for Disaster Management Control

  • Reta L. Puspasari;Daeung Yoon;Hyun Kim;Kyoung-Woong Kim
    • Economic and Environmental Geology
    • /
    • v.56 no.1
    • /
    • pp.65-73
    • /
    • 2023
  • As one of the most vulnerable countries to floods, there should be an increased necessity for accurate and reliable flood forecasting in Indonesia. Therefore, a new prediction model using a machine learning algorithm is proposed to provide daily flood prediction in Indonesia. Data crawling was conducted to obtain daily rainfall, streamflow, land cover, and flood data from 2008 to 2021. The model was built using a Random Forest (RF) algorithm for classification to predict future floods by inputting three days of rainfall rate, forest ratio, and stream flow. The accuracy, specificity, precision, recall, and F1-score on the test dataset using the RF algorithm are approximately 94.93%, 68.24%, 94.34%, 99.97%, and 97.08%, respectively. Moreover, the AUC (Area Under the Curve) of the ROC (Receiver Operating Characteristics) curve results in 71%. The objective of this research is providing a model that predicts flood events accurately in Indonesian regions 3 months prior the day of flood. As a trial, we used the month of June 2022 and the model predicted the flood events accurately. The result of prediction is then published to the website as a warning system as a form of flood mitigation.

Deep Interpretable Learning for a Rapid Response System (긴급대응 시스템을 위한 심층 해석 가능 학습)

  • Nguyen, Trong-Nghia;Vo, Thanh-Hung;Kho, Bo-Gun;Lee, Guee-Sang;Yang, Hyung-Jeong;Kim, Soo-Hyung
    • Annual Conference of KIPS
    • /
    • 2021.11a
    • /
    • pp.805-807
    • /
    • 2021
  • In-hospital cardiac arrest is a significant problem for medical systems. Although the traditional early warning systems have been widely applied, they still contain many drawbacks, such as the high false warning rate and low sensitivity. This paper proposed a strategy that involves a deep learning approach based on a novel interpretable deep tabular data learning architecture, named TabNet, for the Rapid Response System. This study has been processed and validated on a dataset collected from two hospitals of Chonnam National University, Korea, in over 10 years. The learning metrics used for the experiment are the area under the receiver operating characteristic curve score (AUROC) and the area under the precision-recall curve score (AUPRC). The experiment on a large real-time dataset shows that our method improves compared to other machine learning-based approaches.

Identifying the Optimal Machine Learning Algorithm for Breast Cancer Prediction

  • ByungJoo Kim
    • International journal of advanced smart convergence
    • /
    • v.13 no.3
    • /
    • pp.80-88
    • /
    • 2024
  • Breast cancer remains a significant global health burden, necessitating accurate and timely detection for improved patient outcomes. Machine learning techniques have demonstrated remarkable potential in assisting breast cancer diagnosis by learning complex patterns from multi-modal patient data. This study comprehensively evaluates several popular machine learning models, including logistic regression, decision trees, random forests, support vector machines (SVMs), naive Bayes, k-nearest neighbors (KNN), XGBoost, and ensemble methods for breast cancer prediction using the Wisconsin Breast Cancer Dataset (WBCD). Through rigorous benchmarking across metrics like accuracy, precision, recall, F1-score, and area under the ROC curve (AUC), we identify the naive Bayes classifier as the top-performing model, achieving an accuracy of 0.974, F1-score of 0.979, and highest AUC of 0.988. Other strong performers include logistic regression, random forests, and XGBoost, with AUC values exceeding 0.95. Our findings showcase the significant potential of machine learning, particularly the robust naive Bayes algorithm, to provide highly accurate and reliable breast cancer screening from fine needle aspirate (FNA) samples, ultimately enabling earlier intervention and optimized treatment strategies.

Automated Areal Feature Matching in Different Spatial Data-sets (이종의 공간 데이터 셋의 면 객체 자동 매칭 방법)

  • Kim, Ji Young;Lee, Jae Bin
    • Journal of Korean Society for Geospatial Information Science
    • /
    • v.24 no.1
    • /
    • pp.89-98
    • /
    • 2016
  • In this paper, we proposed an automated areal feature matching method based on geometric similarity without user intervention and is applied into areal features of many-to-many relation, for confusion of spatial data-sets of different scale and updating cycle. Firstly, areal feature(node) that a value of inclusion function is more than 0.4 was connected as an edge in adjacency matrix and candidate corresponding areal features included many-to-many relation was identified by multiplication of adjacency matrix. For geometrical matching, these multiple candidates corresponding areal features were transformed into an aggregated polygon as a convex hull generated by a curve-fitting algorithm. Secondly, we defined matching criteria to measure geometrical quality, and these criteria were changed into normalized values, similarity, by similarity function. Next, shape similarity is defined as a weighted linear combination of these similarities and weights which are calculated by Criteria Importance Through Intercriteria Correlation(CRITIC) method. Finally, in training data, we identified Equal Error Rate(EER) which is trade-off value in a plot of precision versus recall for all threshold values(PR curve) as a threshold and decided if these candidate pairs are corresponding pairs or not. To the result of applying the proposed method in a digital topographic map and a base map of address system(KAIS), we confirmed that some many-to-many areal features were mis-detected in visual evaluation and precision, recall and F-Measure was highly 0.951, 0.906, 0.928, respectively in statistical evaluation. These means that accuracy of the automated matching between different spatial data-sets by the proposed method is highly. However, we should do a research on an inclusion function and a detail matching criterion to exactly quantify many-to-many areal features in future.

Contrast Media Side Effects Prediction Study using Artificial Intelligence Technique (인공지능 기법을 이용한 조영제 부작용 예측 연구)

  • Sang-Hyun Kim
    • Journal of the Korean Society of Radiology
    • /
    • v.17 no.3
    • /
    • pp.423-431
    • /
    • 2023
  • The purpose of this study is to analyze the factors affecting the classification of the severity of contrast media side effects based on the patient's body information using artificial intelligence techniques to be used as basic data to reduce the degree of contrast medium side effects. The data used in this study were 606 examiners who had no contrast medium side effects in the past history survey among 1,235 cases of contrast medium side effects among 58,000 CT scans performed at a general hospital in Seoul. The total data is 606, of which 70% was used as a training set and the remaining 30% was used as a test set for validation. Age, BMI(Body Mass Index), GFR(Glomerular Filtration Rate), BUN(Blood Urea Nitrogen), GGT(Gamma Glutamyl Transgerase), AST(Aspartate Amino Transferase,), and ALT(Alanine Amiono Transferase) features were used as independent variables, and contrast media severity was used as a target variable. AUC(Area under curve), CA(Classification Accuracy), F1, Precision, and Recall were identified through AdaBoost, Tree, Neural network, SVM, and Random foest algorithm. AdaBoost and Random Forest show the highest evaluation index in the classification prediction algorithm. The largest factors in the predictions of all models were GFR, BMI, and GGT. It was found that the difference in the amount of contrast media injected according to renal filtration function and obesity, and the presence or absence of metabolic syndrome affected the severity of contrast medium side effects.

Analysis Study on the Detection and Classification of COVID-19 in Chest X-ray Images using Artificial Intelligence (인공지능을 활용한 흉부 엑스선 영상의 코로나19 검출 및 분류에 대한 분석 연구)

  • Yoon, Myeong-Seong;Kwon, Chae-Rim;Kim, Sung-Min;Kim, Su-In;Jo, Sung-Jun;Choi, Yu-Chan;Kim, Sang-Hyun
    • Journal of the Korean Society of Radiology
    • /
    • v.16 no.5
    • /
    • pp.661-672
    • /
    • 2022
  • After the outbreak of the SARS-CoV2 virus that causes COVID-19, it spreads around the world with the number of infections and deaths rising rapidly caused a shortage of medical resources. As a way to solve this problem, chest X-ray diagnosis using Artificial Intelligence(AI) received attention as a primary diagnostic method. The purpose of this study is to comprehensively analyze the detection of COVID-19 via AI. To achieve this purpose, 292 studies were collected through a series of Classification methods. Based on these data, performance measurement information including Accuracy, Precision, Area Under Cover(AUC), Sensitivity, Specificity, F1-score, Recall, K-fold, Architecture and Class were analyzed. As a result, the average Accuracy, Precision, AUC, Sensitivity and Specificity were achieved as 95.2%, 94.81%, 94.01%, 93.5%, and 93.92%, respectively. Although the performance measurement information on a year-on-year basis gradually increased, furthermore, we conducted a study on the rate of change according to the number of Class and image data, the ratio of use of Architecture and about the K-fold. Currently, diagnosis of COVID-19 using AI has several problems to be used independently, however, it is expected that it will be sufficient to be used as a doctor's assistant.