• Title/Summary/Keyword: Precision-recall

Search Result 728, Processing Time 0.021 seconds

Detection and Prediction of Subway Failure using Machine Learning (머신러닝을 이용한 지하철 고장 탐지 및 예측)

  • Kuk-Kyung Sung
    • Advanced Industrial SCIence
    • /
    • v.2 no.4
    • /
    • pp.11-16
    • /
    • 2023
  • The subway is a means of public transportation that plays an important role in the transportation system of modern cities. However, congestion often occurs due to sudden breakdowns and system outages, causing inconvenience. Therefore, in this paper, we conducted a study on failure prediction and prevention using machine learning to efficiently operate the subway system. Using UC Irvine's MetroPT-3 dataset, we built a subway breakdown prediction model using logistic regression. The model predicted the non-failure state with a high accuracy of 0.991. However, precision and recall are relatively low, suggesting the possibility of error in failure prediction. The ROC_AUC value is 0.901, indicating that the model can classify better than random guessing. The constructed model is useful for stable operation of the subway system, but additional research is needed to improve performance. Therefore, in the future, if there is a lot of learning data and the data is well purified, failure can be prevented by pre-inspection through prediction.

Diagnostic Performance of a New Convolutional Neural Network Algorithm for Detecting Developmental Dysplasia of the Hip on Anteroposterior Radiographs

  • Hyoung Suk Park;Kiwan Jeon;Yeon Jin Cho;Se Woo Kim;Seul Bi Lee;Gayoung Choi;Seunghyun Lee;Young Hun Choi;Jung-Eun Cheon;Woo Sun Kim;Young Jin Ryu;Jae-Yeon Hwang
    • Korean Journal of Radiology
    • /
    • v.22 no.4
    • /
    • pp.612-623
    • /
    • 2021
  • Objective: To evaluate the diagnostic performance of a deep learning algorithm for the automated detection of developmental dysplasia of the hip (DDH) on anteroposterior (AP) radiographs. Materials and Methods: Of 2601 hip AP radiographs, 5076 cropped unilateral hip joint images were used to construct a dataset that was further divided into training (80%), validation (10%), or test sets (10%). Three radiologists were asked to label the hip images as normal or DDH. To investigate the diagnostic performance of the deep learning algorithm, we calculated the receiver operating characteristics (ROC), precision-recall curve (PRC) plots, sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) and compared them with the performance of radiologists with different levels of experience. Results: The area under the ROC plot generated by the deep learning algorithm and radiologists was 0.988 and 0.988-0.919, respectively. The area under the PRC plot generated by the deep learning algorithm and radiologists was 0.973 and 0.618-0.958, respectively. The sensitivity, specificity, PPV, and NPV of the proposed deep learning algorithm were 98.0, 98.1, 84.5, and 99.8%, respectively. There was no significant difference in the diagnosis of DDH by the algorithm and the radiologist with experience in pediatric radiology (p = 0.180). However, the proposed model showed higher sensitivity, specificity, and PPV, compared to the radiologist without experience in pediatric radiology (p < 0.001). Conclusion: The proposed deep learning algorithm provided an accurate diagnosis of DDH on hip radiographs, which was comparable to the diagnosis by an experienced radiologist.

Segmentation of Natural Fine Aggregates in Micro-CT Microstructures of Recycled Aggregates Using Unet-VGG16 (Unet-VGG16 모델을 활용한 순환골재 마이크로-CT 미세구조의 천연골재 분할)

  • Sung-Wook Hong;Deokgi Mun;Se-Yun Kim;Tong-Seok Han
    • Journal of the Computational Structural Engineering Institute of Korea
    • /
    • v.37 no.2
    • /
    • pp.143-149
    • /
    • 2024
  • Segmentation of material phases through image analysis is essential for analyzing the microstructure of materials. Micro-CT images exhibit variations in grayscale values depending on the phases constituting the material. Phase segmentation is generally achieved by comparing the grayscale values in the images. In the case of waste concrete used as a recycled aggregate, it is challenging to distinguish between hydrated cement paste and natural aggregates, as these components exhibit similar grayscale values in micro-CT images. In this study, we propose a method for automatically separating the aggregates in concrete, in micro-CT images. Utilizing the Unet-VGG16 deep-learning network, we introduce a technique for segmenting the 2D aggregate images and stacking them to obtain 3D aggregate images. Image filtering is employed to separate aggregate particles from the selected 3D aggregate images. The performance of aggregate segmentation is validated through accuracy, precision, recall, and F1-score assessments.

Classifying Social Media Users' Stance: Exploring Diverse Feature Sets Using Machine Learning Algorithms

  • Kashif Ayyub;Muhammad Wasif Nisar;Ehsan Ullah Munir;Muhammad Ramzan
    • International Journal of Computer Science & Network Security
    • /
    • v.24 no.2
    • /
    • pp.79-88
    • /
    • 2024
  • The use of the social media has become part of our daily life activities. The social web channels provide the content generation facility to its users who can share their views, opinions and experiences towards certain topics. The researchers are using the social media content for various research areas. Sentiment analysis, one of the most active research areas in last decade, is the process to extract reviews, opinions and sentiments of people. Sentiment analysis is applied in diverse sub-areas such as subjectivity analysis, polarity detection, and emotion detection. Stance classification has emerged as a new and interesting research area as it aims to determine whether the content writer is in favor, against or neutral towards the target topic or issue. Stance classification is significant as it has many research applications like rumor stance classifications, stance classification towards public forums, claim stance classification, neural attention stance classification, online debate stance classification, dialogic properties stance classification etc. This research study explores different feature sets such as lexical, sentiment-specific, dialog-based which have been extracted using the standard datasets in the relevant area. Supervised learning approaches of generative algorithms such as Naïve Bayes and discriminative machine learning algorithms such as Support Vector Machine, Naïve Bayes, Decision Tree and k-Nearest Neighbor have been applied and then ensemble-based algorithms like Random Forest and AdaBoost have been applied. The empirical based results have been evaluated using the standard performance measures of Accuracy, Precision, Recall, and F-measures.

Research on Improving the Performance of YOLO-Based Object Detection Models for Smoke and Flames from Different Materials (다양한 재료에서 발생되는 연기 및 불꽃에 대한 YOLO 기반 객체 탐지 모델 성능 개선에 관한 연구 )

  • Heejun Kwon;Bohee Lee;Haiyoung Jung
    • Journal of the Korean Institute of Electrical and Electronic Material Engineers
    • /
    • v.37 no.3
    • /
    • pp.261-273
    • /
    • 2024
  • This paper is an experimental study on the improvement of smoke and flame detection from different materials with YOLO. For the study, images of fires occurring in various materials were collected through an open dataset, and experiments were conducted by changing the main factors affecting the performance of the fire object detection model, such as the bounding box, polygon, and data augmentation of the collected image open dataset during data preprocessing. To evaluate the model performance, we calculated the values of precision, recall, F1Score, mAP, and FPS for each condition, and compared the performance of each model based on these values. We also analyzed the changes in model performance due to the data preprocessing method to derive the conditions that have the greatest impact on improving the performance of the fire object detection model. The experimental results showed that for the fire object detection model using the YOLOv5s6.0 model, data augmentation that can change the color of the flame, such as saturation, brightness, and exposure, is most effective in improving the performance of the fire object detection model. The real-time fire object detection model developed in this study can be applied to equipment such as existing CCTV, and it is believed that it can contribute to minimizing fire damage by enabling early detection of fires occurring in various materials.

Fashion Category Oversampling Automation System

  • Minsun Yeu;Do Hyeok Yoo;SuJin Bak
    • Journal of the Korea Society of Computer and Information
    • /
    • v.29 no.1
    • /
    • pp.31-40
    • /
    • 2024
  • In the realm of domestic online fashion platform industry the manual registration of product information by individual business owners leads to inconvenience and reliability issues, especially when dealing with simultaneous registrations of numerous product groups. Moreover, bias is significantly heightened due to the low quality of product images and an imbalance in data quantity. Therefore, this study proposes a ResNet50 model aimed at minimizing data bias through oversampling techniques and conducting multiple classifications for 13 fashion categories. Transfer learning is employed to optimize resource utilization and reduce prolonged learning times. The results indicate improved discrimination of up to 33.4% for data augmentation in classes with insufficient data compared to the basic convolution neural network (CNN) model. The reliability of all outcomes is underscored by precision and affirmed by the recall curve. This study is suggested to advance the development of the domestic online fashion platform industry to a higher echelon.

An Ensemble Classification of Mental Health in Malaysia related to the Covid-19 Pandemic using Social Media Sentiment Analysis

  • Nur 'Aisyah Binti Zakaria Adli;Muneer Ahmad;Norjihan Abdul Ghani;Sri Devi Ravana;Azah Anir Norman
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.18 no.2
    • /
    • pp.370-396
    • /
    • 2024
  • COVID-19 was declared a pandemic by the World Health Organization (WHO) on 30 January 2020. The lifestyle of people all over the world has changed since. In most cases, the pandemic has appeared to create severe mental disorders, anxieties, and depression among people. Mostly, the researchers have been conducting surveys to identify the impacts of the pandemic on the mental health of people. Despite the better quality, tailored, and more specific data that can be generated by surveys,social media offers great insights into revealing the impact of the pandemic on mental health. Since people feel connected on social media, thus, this study aims to get the people's sentiments about the pandemic related to mental issues. Word Cloud was used to visualize and identify the most frequent keywords related to COVID-19 and mental health disorders. This study employs Majority Voting Ensemble (MVE) classification and individual classifiers such as Naïve Bayes (NB), Support Vector Machine (SVM), and Logistic Regression (LR) to classify the sentiment through tweets. The tweets were classified into either positive, neutral, or negative using the Valence Aware Dictionary or sEntiment Reasoner (VADER). Confusion matrix and classification reports bestow the precision, recall, and F1-score in identifying the best algorithm for classifying the sentiments.

Factors Influencing Sexual Experiences in Adolescents Using a Random Forest Model: Secondary Data Analysis of the 2019~2021 Korea Youth Risk Behavior Web-based Survey Data (랜덤 포레스트 모델을 활용한 국내 청소년 성경험 영향요인 분석 연구: 2019~2021년 청소년건강행태조사 데이터)

  • Yang, Yoonseok;Kwon, Ju Won;Yang, Youngran
    • Journal of Korean Academy of Nursing
    • /
    • v.54 no.2
    • /
    • pp.193-210
    • /
    • 2024
  • Purpose: The objective of this study was to develop a predictive model for the sexual experiences of adolescents using the random forest method and to identify the "variable importance." Methods: The study utilized data from the 2019 to 2021 Korea Youth Risk Behavior Web-based Survey, which included 86,595 man and 80,504 woman participants. The number of independent variables stood at 44. SPSS was used to conduct Rao-Scott χ2 tests and complex sample t-tests. Modeling was performed using the random forest algorithm in Python. Performance evaluation of each model included assessments of precision, recall, F1-score, receiver operating characteristics curve, and area under the curve calculations derived from the confusion matrix. Results: The prevalence of sexual experiences initially decreased during the COVID-19 pandemic, but later increased. "Variable importance" for predicting sexual experiences, ranked in the top six, included week and weekday sedentary time and internet usage time, followed by ease of cigarette purchase, age at first alcohol consumption, smoking initiation, breakfast consumption, and difficulty purchasing alcohol. Conclusion: Education and support programs for promoting adolescent sexual health, based on the top-ranking important variables, should be integrated with health behavior intervention programs addressing internet usage, smoking, and alcohol consumption. We recommend active utilization of the random forest analysis method to develop high-performance predictive models for effective disease prevention, treatment, and nursing care.

IoT-Based Automatic Water Quality Monitoring System with Optimized Neural Network

  • Anusha Bamini A M;Chitra R;Saurabh Agarwal;Hyunsung Kim;Punitha Stephan;Thompson Stephan
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.18 no.1
    • /
    • pp.46-63
    • /
    • 2024
  • One of the biggest dangers in the globe is water contamination. Water is a necessity for human survival. In most cities, the digging of borewells is restricted. In some cities, the borewell is allowed for only drinking water. Hence, the scarcity of drinking water is a vital issue for industries and villas. Most of the water sources in and around the cities are also polluted, and it will cause significant health issues. Real-time quality observation is necessary to guarantee a secure supply of drinking water. We offer a model of a low-cost system of monitoring real-time water quality using IoT to address this issue. The potential for supporting the real world has expanded with the introduction of IoT and other sensors. Multiple sensors make up the suggested system, which is utilized to identify the physical and chemical features of the water. Various sensors can measure the parameters such as temperature, pH, and turbidity. The core controller can process the values measured by sensors. An Arduino model is implemented in the core controller. The sensor data is forwarded to the cloud database using a WI-FI setup. The observed data will be transferred and stored in a cloud-based database for further processing. It wasn't easy to analyze the water quality every time. Hence, an Optimized Neural Network-based automation system identifies water quality from remote locations. The performance of the feed-forward neural network classifier is further enhanced with a hybrid GA- PSO algorithm. The optimized neural network outperforms water quality prediction applications and yields 91% accuracy. The accuracy of the developed model is increased by 20% because of optimizing network parameters compared to the traditional feed-forward neural network. Significant improvement in precision and recall is also evidenced in the proposed work.

Analysis of Ammunition Inspection Record Data and Development of Ammunition Condition Code Classification Model (탄약검사기록 데이터 분석 및 탄약상태기호 분류 모델 개발)

  • Young-Jin Jung;Ji-Soo Hong;Sol-Ip Kim;Sung-Woo Kang
    • Journal of the Korea Safety Management & Science
    • /
    • v.26 no.2
    • /
    • pp.23-31
    • /
    • 2024
  • In the military, ammunition and explosives stored and managed can cause serious damage if mishandled, thus securing safety through the utilization of ammunition reliability data is necessary. In this study, exploratory data analysis of ammunition inspection records data is conducted to extract reliability information of stored ammunition and to predict the ammunition condition code, which represents the lifespan information of the ammunition. This study consists of three stages: ammunition inspection record data collection and preprocessing, exploratory data analysis, and classification of ammunition condition codes. For the classification of ammunition condition codes, five models based on boosting algorithms are employed (AdaBoost, GBM, XGBoost, LightGBM, CatBoost). The most superior model is selected based on the performance metrics of the model, including Accuracy, Precision, Recall, and F1-score. The ammunition in this study was primarily produced from the 1980s to the 1990s, with a trend of increased inspection volume in the early stages of production and around 30 years after production. Pre-issue inspections (PII) were predominantly conducted, and there was a tendency for the grade of ammunition condition codes to decrease as the storage period increased. The classification of ammunition condition codes showed that the CatBoost model exhibited the most superior performance, with an Accuracy of 93% and an F1-score of 93%. This study emphasizes the safety and reliability of ammunition and proposes a model for classifying ammunition condition codes by analyzing ammunition inspection record data. This model can serve as a tool to assist ammunition inspectors and is expected to enhance not only the safety of ammunition but also the efficiency of ammunition storage management.