Journal of the Korea Convergence Society (한국융합학회논문지)

Korea Convergence Society (한국융합학회)
권호 표지
DOI QR코드

DOI QR Code

Performance Evaluation of One Class Classification to detect anomalies of NIDS

NIDS의 비정상 행위 탐지를 위한 단일 클래스 분류성능 평가

  • Seo, Jae-Hyun (Division of Computer Science & Engineering, WonKwang University)
  • 서재현 (원광대학교 컴퓨터.소프트웨어공학과)
  • Received : 2018.09.10
  • Accepted : 2018.11.20
  • Published : 2018.11.28
Download PDF
⟨ Previous Next ⟩

Abstract

In this study, we try to detect anomalies on the network intrusion detection system by learning only one class. We use KDD CUP 1999 dataset, an intrusion detection dataset, which is used to evaluate classification performance. One class classification is one of unsupervised learning methods that classifies attack class by learning only normal class. When using unsupervised learning, it difficult to achieve relatively high classification efficiency because it does not use negative instances for learning. However, unsupervised learning has the advantage for classifying unlabeled data. In this study, we use one class classifiers based on support vector machines and density estimation to detect new unknown attacks. The test using the classifier based on density estimation has shown relatively better performance and has a detection rate of about 96% while maintaining a low FPR for the new attacks.

본 논문에서는 단일 클래스만을 학습하여 네트워크 침입탐지 시스템 상에서 새로운 비정상 행위를 탐지하는 것을 목표로 한다. 분류 성능 평가를 위해 KDD CUP 1999 데이터셋을 사용한다. 단일 클래스 분류는 정상 클래스만을 학습하여 공격 클래스를 분류해내는 비지도 학습 방법 중 하나이다. 비지도 학습의 경우에는 학습에 네거티브 인스턴스를 사용하지 않기 때문에 상대적으로 높은 분류 효율을 내는 것이 어렵다. 하지만, 비지도 학습은 라벨이 없는 데이터를 분류하는데 적합한 장점이 있다. 본 연구에서는 서포트벡터머신 기반의 단일 클래스 분류기와 밀도 추정 기반의 단일 클래스 분류기를 사용한 실험을 통해 기존에 없던 새로운 공격에 대한 탐지를 한다. 밀도 추정 기반의 분류기를 사용한 실험이 상대적으로 더 좋은 성능을 보였고, 신규 공격에 대해 낮은 FPR을 유지하면서도 약 96%의 탐지율을 보인다.

Keywords

OHHGBW_2018_v9n11_15_f0001.png 이미지

Fig. 1. A flowchart of the proposed method

OHHGBW_2018_v9n11_15_f0002.png 이미지

Fig. 2. A graph of TPR and FPR for known attack types

OHHGBW_2018_v9n11_15_f0003.png 이미지

Fig. 3. A graph of FP and TN for new attack types

OHHGBW_2018_v9n11_15_f0004.png 이미지

Fig. 4. Performance comparison of one-class SVM (LibSVM) and Hempstalk’s one-class classifier

Table 1. Attack types of KDD 1999 dataset

OHHGBW_2018_v9n11_15_t0001.png 이미지

Table 2. Attack types of training dataset

OHHGBW_2018_v9n11_15_t0002.png 이미지

Table 3. Attack types of test dataset

OHHGBW_2018_v9n11_15_t0003.png 이미지

Table 4. Confusion matrix [20]

OHHGBW_2018_v9n11_15_t0004.png 이미지

Table 5. The measures used in the proposed method

OHHGBW_2018_v9n11_15_t0005.png 이미지

Table 6. TPR, FPR, accuracy, F1-score of one-class SVM according to nu parameter.

OHHGBW_2018_v9n11_15_t0006.png 이미지

Table 7. Results of one-class SVM according to nu parameter.

OHHGBW_2018_v9n11_15_t0007.png 이미지

Table 8. Results of Hempstalk’s one-class classifier

OHHGBW_2018_v9n11_15_t0008.png 이미지

References

  1. B. Mukherjee, L. T. Heberlein, & K. N. Levitt. (1994). Network intrusion detection. IEEE network, 8(3), 26-41. https://doi.org/10.1109/65.283931
  2. P. Garcia-Teodoro, J. Diaz-Verdejo, G. Macia-Fernandez, & E. Vazquez. (2009). Anomaly-based network intrusion detection: Techniques, systems and challenges. computers & security, 28(1-2), 18-28. https://doi.org/10.1016/j.cose.2008.08.003
  3. S. S. Khan & M. G. Madden. (2009). A survey of recent trends in one class classification. In Irish Conference on Artificial Intelligence and Cognitive Science, 188-197. Springer, Berlin, Heidelberg.
  4. G. Ratsch, S., Mika, B., Scholkopf, & K. R. Muller. (2002). Constructing boosting algorithms from SVMs: an application to one-class classification. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(9), 1184-1199. https://doi.org/10.1109/TPAMI.2002.1033211
  5. K. L. Li, H. K. Huang, S. F. Tian, & W. Xu. (2003, November). Improving one-class SVM for anomaly detection. In Machine Learning and Cybernetics, 2003 International Conference on, 5, 3077-3081. IEEE.
  6. G. Giacinto, R. Perdisci, M. Del Rio, & F. Roli. (2008). Intrusion detection in computer networks by a modular ensemble of one-class classifiers. Information Fusion, 9(1), 69-82. https://doi.org/10.1016/j.inffus.2006.10.002
  7. I. Kang, M. K. Jeong, & D. Kong. (2012). A differentiated one-class classification method with applications to intrusion detection. Expert Systems with Applications, 39(4), 3899-3905. https://doi.org/10.1016/j.eswa.2011.06.033
  8. J. H. Seo. (2018). Detection of Car Hacking Using One Class Classifier. Journal of the Korea Convergence Society, 9(6), 33-38. https://doi.org/10.15207/JKCS.2018.9.6.033
  9. L. Portnoy, E. Eskin, & S. Stolfo. (2001). Intrusion detection with unlabeled data using clustering. In Proceedings of ACM CSS Workshop on Data Mining Applied to Security (DMSA-2001).
  10. L. M. Manevitz & M. Yousef. (2001). One-class SVMs for document classification. Journal of machine Learning research, 2, 139-154.
  11. J. H. Seo. (2018). Feature Selection for Anomaly Detection Based on Genetic Algorithm, Journal of the Korea Convergence Society, 9(7), 1-7. https://doi.org/10.15207/JKCS.2018.9.7.001
  12. J. G. Kang, J. Y. Lee, & Y. Y. You. (2017). A Study on Implementation of Fraud Detection System (FDS) Applying BigData Platform, Journal of the Korea Convergence Society, 8(4), 19-24. https://doi.org/10.15207/JKCS.2017.8.4.019
  13. M. M. Moya & D. R. Hush. (1996). Network constraints and multi-objective optimization for one-class classification. Neural Networks, 9(3), 463-474. https://doi.org/10.1016/0893-6080(95)00120-4
  14. D. M. J. Tax. (2001). One-class classification: concept-learning in the absence of counter-examples [Ph. D. thesis]. Delft University of Technology, Stevinweg, The Netherlands.
  15. K. Hempstalk, E. Frank, & I. H. Witten. (2008, September). One-class classification by combining density and class probability estimation. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, 505-519. Springer, Berlin, Heidelberg.
  16. S. S. Khan & M. G. Madden. (2014). One-class classification: taxonomy of study and review of techniques. The Knowledge Engineering Review, 29(3), 345-374. https://doi.org/10.1017/S026988891300043X
  17. P. Nader, P. Honeine, & P. Beauseroy. (2014). lp-norms in One-Class Classification for Intrusion Detection in SCADA Systems. IEEE Transactions on Industrial Informatics, 10(4), 2308-2317. https://doi.org/10.1109/TII.2014.2330796
  18. KDD Cup 1999 Data, http://kdd.ics.uci.edu/databases/kddcup99/kddcup99.html
  19. WEKA, https://www.cs.waikato.ac.nz/ml/weka/
  20. Confusion matrix, https://en.wikipedia.org/wiki/Confusion_matrix
  21. C. Zhou & R. C. Paffenroth. (2017). Anomaly detection with robust deep autoencoders. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 665-674.
  22. H. Moeini & F. M. Torab. (2017). Comparing compositional multivariate outliers with autoencoder networks in anomaly detection at Hamich exploration area, east of Iran. Journal of Geochemical Exploration, 180, 15-23. https://doi.org/10.1016/j.gexplo.2017.05.008
  23. Y. T. K. Lai, J. S. Hu, Y. H. Tsai, & W. Y. Chiu. (2018). Industrial Anomaly Detection and One-class Classification using Generative Adversarial Networks. In 2018 IEEE/ASME International Conference on Advanced Intelligent Mechatronics (AIM), 1444-1449.