KIPS Transactions on Computer and Communication Systems (정보처리학회논문지:컴퓨터 및 통신 시스템)

Korea Information Processing Society (한국정보처리학회)
권호 표지
DOI QR코드

DOI QR Code

Imbalanced Data Improvement Techniques Based on SMOTE and Light GBM

SMOTE와 Light GBM 기반의 불균형 데이터 개선 기법

  • 한영진 (이화여자대학교 소프트웨어학부) ;
  • 조인휘 (한양대학교 컴퓨터소프트웨어학부)
  • Received : 2022.07.05
  • Accepted : 2022.08.03
  • Published : 2022.12.31
Download PDF
⟨ Previous Next ⟩

Abstract

Class distribution of unbalanced data is an important part of the digital world and is a significant part of cybersecurity. Abnormal activity of unbalanced data should be found and problems solved. Although a system capable of tracking patterns in all transactions is needed, machine learning with disproportionate data, which typically has abnormal patterns, can ignore and degrade performance for minority layers, and predictive models can be inaccurately biased. In this paper, we predict target variables and improve accuracy by combining estimates using Synthetic Minority Oversampling Technique (SMOTE) and Light GBM algorithms as an approach to address unbalanced datasets. Experimental results were compared with logistic regression, decision tree, KNN, Random Forest, and XGBoost algorithms. The performance was similar in accuracy and reproduction rate, but in precision, two algorithms performed at Random Forest 80.76% and Light GBM 97.16%, and in F1-score, Random Forest 84.67% and Light GBM 91.96%. As a result of this experiment, it was confirmed that Light GBM's performance was similar without deviation or improved by up to 16% compared to five algorithms.

디지털 세상에서 불균형 데이터에 대한 클래스 분포는 중요한 부분이며 사이버 보안에 큰 의미를 차지한다. 불균형 데이터의 비정상적인 활동을 찾고 문제를 해결해야 한다. 모든 트랜잭션의 패턴을 추적할 수 있는 시스템이 필요하지만, 일반적으로 패턴이 비정상인 불균형 데이터로 기계학습을 하면 소수 계층에 대한 성능은 무시되고 저하되며 예측 모델은 부정확하게 편향될 수 있다. 본 논문에서는 불균형 데이터 세트를 해결하기 위한 접근 방식으로 Synthetic Minority Oversampling Technique(SMOTE)와 Light GBM 알고리즘을 이용하여 추정치를 결합하여 대상 변수를 예측하고 정확도를 향상시켰다. 실험 결과는 Logistic Regression, Decision Tree, KNN, Random Forest, XGBoost 알고리즘과 비교하였다. 정확도, 재현율에서는 성능이 모두 비슷했으나 정밀도에서는 2개의 알고리즘 Random Forest 80.76%, Light GBM 97.16% 성능이 나왔고, F1-score에서는 Random Forest 84.67%, Light GBM 91.96% 성능이 나왔다. 이 실험 결과로 Light GBM은 성능이 5개의 알고리즘과 비교하여 편차없이 비슷하거나 최대 16% 향상됨을 접근 방식으로 확인할 수 있었다.

Keywords

References

  1. R. Leuning, E. Van Gorsel, W. J. Massman, and P. R. Isaac, "Reflections on the surface energy imbalance problem," Agricultural and Forest Meteorology, Vol.156, pp.65-74, 2012. https://doi.org/10.1016/j.agrformet.2011.12.002
  2. M. Artis, M. Ayuso, and M. Guillen, "Detection of automobile insurance fraud with discrete choice models and misclassified claims," Journal of Risk and Insurance, Vol.69, No.3, pp.325-340, 2002. https://doi.org/10.1111/1539-6975.00022
  3. D. F. Stroup, G. D. Williamson, J. L. Herndon, and J. M. Karon, "Detection of aberrations in the occurrence of notifiable diseases surveillance data. Statistics in medicine," Vol.8, No.3, pp.323-329, 1989. https://doi.org/10.1002/sim.4780080312
  4. T. Y. Liu, "Easyensemble and feature selection for imbalance data sets," In 2009 international Joint Conference on Bioinformatics, Systems Biology and Intelligent Computing, IEEE, pp.517-520, 2009.
  5. N. V. Chawla, K. W. Bowyer, L. O. Hall, and W. P. Kegelmeyer, "SMOTE: synthetic minority over-sampling technique," Journal of Artificial Intelligence Research, Vol.16, pp.321-357, 2002. https://doi.org/10.1613/jair.953
  6. S. Hasmita, F. Nhita, D. Saepudin, and A. Aditsania, "Chili commodity price forecasting in bandung regency using the adaptive synthetic sampling (ADASYN) and k-nearest neighbor (KNN) algorithms," In 2019 International Conference on Information and Communications Technology (ICOIACT), IEEE, pp.434-438, 2019.
  7. C. XGBoost, S. N. L. P. LightGBM, and B. Quinto, "NextGeneration Machine Learning with Spark," 2020.
  8. C. Sammut and G. I. Webb, (Eds.). "Encyclopedia of machine learning," Springer Science and Business Media, 2011.
  9. V. Lalchand, "Extracting more from boosted decision trees: A high energy physics case study," arXiv preprint arXiv: 2001.06033, 2020.
  10. D. Zwillinger, "CRC standard mathematical tables and formulas," Chapman and Hall/CRC, 2018.
  11. AP Statistics Review - "Density Curves and the Normal Distributions," Archived from the original on 2 April 2015. Retrieved 16 March 2015.
  12. L. Torgo, R. P. Ribeiro, B. Pfahringer, and P. Branco, Smote for regression, In Portuguese Conference on Artificial Intelligence, Springer, Berlin, Heidelberg, pp.378-389, 2013.
  13. N. Kozlovskaia and A. Zaytsev, "Deep ensembles for imbalanced classification," In 2017 16th IEEE International Conference on Machine Learning and Applications (ICMLA), IEEE, pp.908-913, 2017.
  14. D. Wang, Y. Zhang, and Y. Zhao, "LightGBM: An effective miRNA classification method in breast cancer patients," In Proceedings of the 2017 International Conference on Computational Biology and Bioinformatics, pp.7-11, 2017.
  15. D. W. Hosmer and S. Lemeshow, "Applied Logistic Regression," John Wiley & Sons. New York, 2000.
  16. R. Kohavi and R. Quinlan, "Decision tree discovery handbook of data mining and knowledge discovery," 2002.
  17. E. Mirkes, "KNN and Potential Energy (Applet)," University of Leicester, 2011.
  18. R. Zhu, D. Zeng, and M. R. Kosorok, "Reinforcement learning trees," Journal of the American Statistical Association, Vol.110, No.512, pp.1770-1784, 2015. https://doi.org/10.1080/01621459.2015.1036994
  19. T. Chen and C. Guestrin, "Xgboost: A scalable tree boosting system," In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp.785-794, 2016.