• Title/Summary/Keyword: Data Imbalance

Search Result 468, Processing Time 0.025 seconds

A rubber o-ring defect detection system using data augmentation based on the SinGAN and random forest algorithm (SinGAN기반 데이터 증강과 random forest알고리즘을 이용한 고무 오링 결함 검출 시스템)

  • Lee, Yong Eun;Lee, Han Sung;Kim, Dae Won;Kim, Kyung Chun
    • Journal of the Korean Society of Visualization
    • /
    • v.19 no.3
    • /
    • pp.63-68
    • /
    • 2021
  • In this study, data was augmentation through the SinGAN algorithm using small image data, and defects in rubber O-rings were detected using the random forest algorithm. Unlike the commonly used data augmentation image rotation method to solve the data imbalance problem, the data imbalance problem was solved by using the SinGAN algorithm. A study was conducted to distinguish between normal products and defective products of rubber o-ring by using the random forest algorithm. A total of 20,000 image date were divided into transit and testing datasets, and an accuracy result was obtained to distinguish 97.43% defects as a result of the test.

A Study on Gait Imbalance Estimation System using 3-axis Accelerometer (3축 가속도 센서를 이용한 보행 불균형 평가 시스템에 관한 연구)

  • Choi, C.H.;Park, Y.D.;Sim, H.M.;Lee, S.M.
    • Journal of rehabilitation welfare engineering & assistive technology
    • /
    • v.9 no.1
    • /
    • pp.37-43
    • /
    • 2015
  • In this paper, an efficient system using 3-axis accelerometer is proposed to diagnose the gait imbalance. The proposed hardware system consists of two 3-axis accelerometers to measure 3 directional acceleration of ankles and an embedded system to transfer the data. The acquired data were normalized and then compared to analyze the symmetry between normal and abnormal gait with ROCC (ratio of correlation coefficient). 10 healthy subjects were participated and each subject repeated the experiment 5 times. To make unbalanced ambulation, the height of the heel of one foot was changed during experiments. From the results, it is verified that ROCC index grew apart from the reference according to growing imbalance and the proposed system could be available for estimation of gait imbalance.

  • PDF

Establishing a Green Space Management Zone for an Environmental City - Focusing on Changwon City - (환경도시 건설을 위한 도시녹지의 관리권역 설정 - 창원시를 대상으로 -)

  • Jung, Sung-Gwan;Lee, Woo-Sung
    • Journal of the Korean Institute of Landscape Architecture
    • /
    • v.35 no.6
    • /
    • pp.64-73
    • /
    • 2008
  • The purpose of this study is to classify urban green space, to assess an imbalance by an administrative district (Dong), and to establish the management zone of urban green spaces for the construction of an environmental city in Changwon. The spatial data of 1:5,000 digital maps, park data in Changwon, land cover by the Ministry of Environment, and IKONOS satellite images from 2003 were used for this analysis. The assessment of the imbalance of urban green spaces was analyzed with the Lorenz curve and Gini's coefficient. The establishment of the management zone was performed by network analysis of GIS. The results of this study are as follows: the urban green spaces were classified as a park green space, a natural green space, and a riparian green space. According to the results of assessment of the imbalance of green spaces, Gini's coefficient was analyzed at higher than 0.4. Thus, the spatial imbalance of urban green spaces in Changwon was evident. The management zones to solve the imbalance were established: "rich zone", "fair zone", "poor zone" and "broken zone". Therefore, the rich and fair zones which have rich green spaces must maintain the good conditions through analysis of the green network and a survey of civic attitudes. The poor and broken zones which have poor green spaces must improve quality and quantity through creation of additional green spaces, construction of an eco-industrial park, and utilization of children's parks and pocket parks.

Application and Comparison of Data Mining Technique to Prevent Metal-Bush Omission (메탈부쉬 누락예방을 위한 데이터마이닝 기법의 적용 및 비교)

  • Sang-Hyun Ko;Dongju Lee
    • Journal of Korean Society of Industrial and Systems Engineering
    • /
    • v.46 no.3
    • /
    • pp.139-147
    • /
    • 2023
  • The metal bush assembling process is a process of inserting and compressing a metal bush that serves to reduce the occurrence of noise and stable compression in the rotating section. In the metal bush assembly process, the head diameter defect and placement defect of the metal bush occur due to metal bush omission, non-pressing, and poor press-fitting. Among these causes of defects, it is intended to prevent defects due to omission of the metal bush by using signals from sensors attached to the facility. In particular, a metal bush omission is predicted through various data mining techniques using left load cell value, right load cell value, current, and voltage as independent variables. In the case of metal bush omission defect, it is difficult to get defect data, resulting in data imbalance. Data imbalance refers to a case where there is a large difference in the number of data belonging to each class, which can be a problem when performing classification prediction. In order to solve the problem caused by data imbalance, oversampling and composite sampling techniques were applied in this study. In addition, simulated annealing was applied for optimization of parameters related to sampling and hyper-parameters of data mining techniques used for bush omission prediction. In this study, the metal bush omission was predicted using the actual data of M manufacturing company, and the classification performance was examined. All applied techniques showed excellent results, and in particular, the proposed methods, the method of mixing Random Forest and SA, and the method of mixing MLP and SA, showed better results.

F_MixBERT: Sentiment Analysis Model using Focal Loss for Imbalanced E-commerce Reviews

  • Fengqian Pang;Xi Chen;Letong Li;Xin Xu;Zhiqiang Xing
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.18 no.2
    • /
    • pp.263-283
    • /
    • 2024
  • Users' comments after online shopping are critical to product reputation and business improvement. These comments, sometimes known as e-commerce reviews, influence other customers' purchasing decisions. To confront large amounts of e-commerce reviews, automatic analysis based on machine learning and deep learning draws more and more attention. A core task therein is sentiment analysis. However, the e-commerce reviews exhibit the following characteristics: (1) inconsistency between comment content and the star rating; (2) a large number of unlabeled data, i.e., comments without a star rating, and (3) the data imbalance caused by the sparse negative comments. This paper employs Bidirectional Encoder Representation from Transformers (BERT), one of the best natural language processing models, as the base model. According to the above data characteristics, we propose the F_MixBERT framework, to more effectively use inconsistently low-quality and unlabeled data and resolve the problem of data imbalance. In the framework, the proposed MixBERT incorporates the MixMatch approach into BERT's high-dimensional vectors to train the unlabeled and low-quality data with generated pseudo labels. Meanwhile, data imbalance is resolved by Focal loss, which penalizes the contribution of large-scale data and easily-identifiable data to total loss. Comparative experiments demonstrate that the proposed framework outperforms BERT and MixBERT for sentiment analysis of e-commerce comments.

Study on Lifelog Anomaly Detection using VAE-based Machine Learning Model (VAE(Variational AutoEncoder) 기반 머신러닝 모델을 활용한 체중 라이프로그 이상탐지에 관한 연구)

  • Kim, Jiyong;Park, Minseo
    • The Journal of the Convergence on Culture Technology
    • /
    • v.8 no.4
    • /
    • pp.91-98
    • /
    • 2022
  • Lifelog data continuously collected through a wearable device may contain many outliers, so in order to improve data quality, it is necessary to find and remove outliers. In general, since the number of outliers is less than the number of normal data, a class imbalance problem occurs. To solve this imbalance problem, we propose a method that applies Variational AutoEncoder to outliers. After preprocessing the outlier data with proposed method, it is verified through a number of machine learning models(classification). As a result of verification using body weight data, it was confirmed that the performance was improved in all classification models. Based on the experimental results, when analyzing lifelog body weight data, we propose to apply the LightGBM model with the best performance after preprocessing the data using the outlier processing method proposed in this study.

Development of machine learning model for reefer container failure determination and cause analysis with unbalanced data (불균형 데이터를 갖는 냉동 컨테이너 고장 판별 및 원인 분석을 위한 기계학습 모형 개발)

  • Lee, Huiwon;Park, Sungho;Lee, Seunghyun;Lee, Seungjae;Lee, Kangbae
    • Journal of the Korea Convergence Society
    • /
    • v.13 no.1
    • /
    • pp.23-30
    • /
    • 2022
  • The failure of the reefer container causes a great loss of cost, but the current reefer container alarm system is inefficient. Existing studies using simulation data of refrigeration systems exist, but studies using actual operation data of refrigeration containers are lacking. Therefore, this study classified the causes of failure using actual refrigerated container operation data. Data imbalance occurred in the actual data, and the data imbalance problem was solved by comparing the logistic regression analysis with ENN-SMOTE and class weight with the 2-stage algorithm developed in this study. The 2-stage algorithm uses XGboost, LGBoost, and DNN to classify faults and normalities in the first step, and to classify the causes of faults in the second step. The model using LGBoost in the 2-stage algorithm was the best with 99.16% accuracy. This study proposes a final model using a two-stage algorithm to solve data imbalance, which is thought to be applicable to other industries.

Performance Characteristics of an Ensemble Machine Learning Model for Turbidity Prediction With Improved Data Imbalance (데이터 불균형 개선에 따른 탁도 예측 앙상블 머신러닝 모형의 성능 특성)

  • HyunSeok Yang;Jungsu Park
    • Ecology and Resilient Infrastructure
    • /
    • v.10 no.4
    • /
    • pp.107-115
    • /
    • 2023
  • High turbidity in source water can have adverse effects on water treatment plant operations and aquatic ecosystems, necessitating turbidity management. Consequently, research aimed at predicting river turbidity continues. This study developed a multi-class classification model for prediction of turbidity using LightGBM (Light Gradient Boosting Machine), a representative ensemble machine learning algorithm. The model utilized data that was classified into four classes ranging from 1 to 4 based on turbidity, from low to high. The number of input data points used for analysis varied among classes, with 945, 763, 95, and 25 data points for classes 1 to 4, respectively. The developed model exhibited precisions of 0.85, 0.71, 0.26, and 0.30, as well as recalls of 0.82, 0.76, 0.19, and 0.60 for classes 1 to 4, respectively. The model tended to perform less effectively in the minority classes due to the limited data available for these classes. To address data imbalance, the SMOTE (Synthetic Minority Over-sampling Technique) algorithm was applied, resulting in improved model performance. For classes 1 to 4, the Precision and Recall of the improved model were 0.88, 0.71, 0.26, 0.25 and 0.79, 0.76, 0.38, 0.60, respectively. This demonstrated that alleviating data imbalance led to a significant enhancement in Recall of the model. Furthermore, to analyze the impact of differences in input data composition addressing the input data imbalance, input data was constructed with various ratios for each class, and the model performances were compared. The results indicate that an appropriate composition ratio for model input data improves the performance of the machine learning model.

Generation of optical fringe patterns using deep learning (딥러닝을 이용한 광학적 프린지 패턴의 생성)

  • Kang, Ji-Won;Kim, Dong-Wook;Seo, Young-Ho
    • Journal of the Korea Institute of Information and Communication Engineering
    • /
    • v.24 no.12
    • /
    • pp.1588-1594
    • /
    • 2020
  • In this paper, we discuss a data balancing method for learning a neural network that generates digital holograms using a deep neural network (DNN). Deep neural networks are based on deep learning (DL) technology and use a generative adversarial network (GAN) series. The fringe pattern, which is the basic unit of a hologram to be created through a deep neural network, has very different data types depending on the hologram plane and the position of the object. However, because the criteria for classifying the data are not clear, an imbalance in the training data may occur. The imbalance of learning data acts as a factor of instability in learning. Therefore, it presents a method for classifying and balancing data for which the classification criteria are not clear. And it shows that learning is stabilized through this.

An Experimental Study on Micro-vibration Measurement Methods of a Reaction Wheel (반작용휠의 미소진동 측정법에 관한 실험적 연구)

  • Kim, Dae-Kwan;Oh, Shi-Hwan;Lee, Seon-Ho;Yong, Ki-Lyuk
    • Transactions of the Korean Society for Noise and Vibration Engineering
    • /
    • v.21 no.9
    • /
    • pp.828-833
    • /
    • 2011
  • A reaction wheel assembly(RWA) is the largest disturbance source that can induce high frequency micro-vibration on an optical payload of satellites. To ensure a tight pointing-stability budget of satellites, the RWA disturbance effect on spacecraft should be accurately analyzed and evaluated for whole design phases. For this purpose, the micro-vibration disturbance of RWA should be precisely measured. In the present study, two measurement methods on RWA micro-vibration disturbances are compared and investigated. One is a free run-down speed test and the other is a constant speed test. The micro-vibration data measured by the two methods are analyzed in terms of spectrum characteristics, static and dynamic imbalance values, and root sum square(RSS) values. The analysis results show that both methods can measure very similar results in time and frequency domains and that the free run-down speed method is more adequate in respects to wheel friction modeling, noise rejection of imbalance and RSS peak evaluation.