Browse > Article
http://dx.doi.org/10.6109/jkiice.2015.19.8.1797

Design and Implementation of a Hadoop-based Efficient Security Log Analysis System  

Ahn, Kwang-Min (Department of Information and Communications Engineering, Daejeon University)
Lee, Jong-Yoon (Department of Information and Communications Engineering, Daejeon University)
Yang, Dong-Min (Department of Information and Communications Engineering, Daejeon University)
Lee, Bong-Hwan (Department of Information and Communications Engineering, Daejeon University)
Publication Information
Journal of the Korea Institute of Information and Communication Engineering / v.19, no.8, 2015 , pp. 1797-1804 More about this Journal
Abstract
Integrated log management system can help to predict the risk of security and contributes to improve the security level of the organization, and leads to prepare an appropriate security policy. In this paper, we have designed and implemented a Hadoop-based log analysis system by using distributed database model which can store large amount of data and reduce analysis time by automating log collecting procedure. In the proposed system, we use the HBase in order to store a large amount of data efficiently in the scale-out fashion and propose an easy data storing scheme for analysing data using a Hadoop-based normal expression, which results in improving data processing speed compared to the existing system.
Keywords
Log Analysis; Non-relational Database; Enterprise Security Management System; Cloud Computing; Hadoop;
Citations & Related Records
Times Cited By KSCI : 2  (Citation Analysis)
연도 인용수 순위
1 D. H. Kim, "SIEM Trend Evolving into Intelligent Log Management Platform in Bigdata Environment", NIPA, ITFIND, 2013. 8.
2 B. M. Choi, J. H. Gong, S. S. Hong, and M. M. Han, “The Method of Analyzing Firewall Log Data using MapReduce based on NoSQL”, Journal of Korea Institute of Information Security & Cryptology, Vol.23, No.4, pp. 667-677, 2013.   DOI
3 M. J. Kim, S. H, Han, W. Choi, and H. G. Lee, “Design and Implementation of MongoDB-based Unstructured Log Processing System over Cloud Computing Environment”, KSII Transactions on Internet and Information Systems, Vol.14, No.6, pp.71-84, 2013.12.
4 D. S. Choi, J. J. Moon, Y. M. Kim, and B. N. Noh, “An Analysis of Large-Scale Security Log using MapReduce”, Journal of KIIT, Vol.9, No.8, pp. 125-132, 2011.8.
5 Fengying Yang, "Research on Cloud-Based Mass Log Data Management Mechanism", Journal of Computers, Vol. 9, No. 6, June 2014.   DOI
6 H. J. Jeong, "Integration of Large-scale Security Log based on NoSQL in Cloud Computing Environment", Chosun University Master's Thesis, 2014.
7 W. J. Kim and H. Y Yeum “Integrated Management and IT Compliance for Heterogeneous Log”, Journal of Korea Institute of Information Security & Cryptology, Vol.20, No.5, pp.73-86, 2010.10.
8 H. W. Lee "Design and Implementation of Web Attack Detection Based on Integrated Web Audit Data", KSII Transactions on Internet and Information Systems, Vol.11, No.6, pp.73-86, 2010.12.