Browse > Article
http://dx.doi.org/10.5392/JKCA.2018.18.04.048

Spark-based Network Log Analysis Aystem for Detecting Network Attack Pattern Using Snort  

Baek, Na-Eun (전북대학교 컴퓨터공학과)
Shin, Jae-Hwan (전북대학교 컴퓨터공학과)
Chang, Jin-Su (전북대학교 컴퓨터공학과)
Chang, Jae-Woo (전북대학교 IT정보공학과)
Publication Information
The Journal of the Korea Contents Association / v.18, no.4, 2018 , pp. 48-59 More about this Journal
Abstract
Recently, network technology has been used in various fields due to development of network technology. However, there has been an increase in the number of attacks targeting public institutions and companies by exploiting the evolving network technology. Meanwhile, the existing network intrusion detection system takes much time to process logs as the amount of network log increases. Therefore, in this paper, we propose a Spark-based network log analysis system that detects unstructured network attack pattern. by using Snort. The proposed system extracts and analyzes the elements required for network attack pattern detection from large amount of network log data. For the analysis, we propose a rule to detect network attack patterns for Port Scanning, Host Scanning, DDoS, and worm activity, and can detect real attack pattern well by applying it to real log data. Finally, we show from our performance evaluation that the proposed Spark-based log analysis system is more than two times better on log data processing performance than the Hadoop-based system.
Keywords
Network Log; Network Attack Detecting; Log Analysis; Snort; Spark;
Citations & Related Records
Times Cited By KSCI : 1  (Citation Analysis)
연도 인용수 순위
1 Khamphakdee, Nattawat, Nunnapus Benjamas, and Saiyan Saiyod, "Improving intrusion detection system based on snort rules for network probe attack detection," Information and Communication Technology (ICoICT), 2014 2nd International Conference On. IEEE, 2014.
2 Matei Zaharia, Mosharaf Chowdhury, Tathagata Das, Ankur Dave, Justin Ma, Murphy McCauley, Michael J. Franklin, Scott Shenker, and Ion Stoica, "Resilient distributed datasets: A fault-tolerant abstractVion for in-memory cluster computing," Proceedings of the 9th USENIX conference on Networked Systems Design and Implementation, USENIX Association, 2012.
3 Matei Zaharia, Mosharaf Chowdhury, Michael J. Franklin, Scott Shenker, and Ion Stoica, "Spark: Cluster computing with working sets," HotCloud, Vol.10, No.10, p.95, 2010.
4 최대수, 문길종, 김용민, 노봉남, "MapReduce 를 이용한 대용량 보안 로그 분석," 한국정보기술학회논문지, 제9권, 제8호, pp.125-132, 2011.
5 Jian Zhang, Pin Liu, Jianbiao He, and Yawei Zhang, "A Hadoop Based Analysis and Detection Model for IP Spoofing Typed DDoS Attack," Trustcom/BigDataSE/I​ SPA, 2016 IEEE. IEEE, 2016.
6 https://www.r-project.org/, 2018.1.5.
7 http://www.vacommunity.org/VASTChallenge 2012, 2018.1.5
8 Cook, Kristin, et al. "VAST Challenge 2012: Visual analytics for big data." Visual Analytics Science and Technology (VAST), 2012 IEEE Conference on. IEEE, 2012.
9 http://www.hping.org/, 2018.1.5
10 J. J. Cheon and T. Y. Choe, "Distributed processing of snort alert log using hadoop," International Journal of Engineering and Technology, Vol.5, No.3, pp.2685-2690, 2013.
11 Anna Sperotto, Gregor Schaffrath, Ramin Sadre, Cristian Morariu, Aiko Pras, and Burkhard Stiller, "An overview of ip flow-based intrusion detection," IEEE Communications Surveys and Tutorials, Vol.12, No.3, pp.343-356, 2010.   DOI
12 이동건, 김휘강, 김은진, "RGB Palette를 이용한 보안 로그 시각화 및 보안 위협 인식," 정보보호학회논문지, 제25권, 제1호, pp.61-73, 2015.   DOI
13 장진수, 신재환, 장재우, "MapReduce 환경에서 네트워크 공격 탐지를 위한 실시간 로그 분석 시스템 개발," 한국정보처리학회 추계학술발표대회, 2017.
14 https://hadoop.apache.org/, 2018.1.5.
15 https://www.snort.org/, 2018.1.5
16 http://spark.apache.org/, 2018.1.5
17 https://oisf.net/suricata/, 2018.1.5
18 https://www.bro.org/, 2018.1.5
19 P. G. Prathibha and E. D. Dileesh, "Design of a hybrid intrusion detection system using snort and hadoop," International Journal of Computer Applications, Vol.73, No.10, 2013.