Browse > Article
http://dx.doi.org/10.13089/JKIISC.2015.25.6.1399

Real time predictive analytic system design and implementation using Bigdata-log  

Lee, Sang-jun (Graduate School of Information Security, Korea University)
Lee, Dong-hoon (Graduate School of Information Security, Korea University)
Publication Information
Journal of the Korea Institute of Information Security & Cryptology / v.25, no.6, 2015 , pp. 1399-1410 More about this Journal
Abstract
Gartner is requiring companies to considerably change their survival paradigms insisting that companies need to understand and provide again the upcoming era of data competition. With the revealing of successful business cases through statistic algorithm-based predictive analytics, also, the conversion into preemptive countermeasure through predictive analysis from follow-up action through data analysis in the past is becoming a necessity of leading enterprises. This trend is influencing security analysis and log analysis and in reality, the cases regarding the application of the big data analysis framework to large-scale log analysis and intelligent and long-term security analysis are being reported file by file. But all the functions and techniques required for a big data log analysis system cannot be accommodated in a Hadoop-based big data platform, so independent platform-based big data log analysis products are still being provided to the market. This paper aims to suggest a framework, which is equipped with a real-time and non-real-time predictive analysis engine for these independent big data log analysis systems and can cope with cyber attack preemptively.
Keywords
Bigdata; Advanced bigdata analytics; Predictive analytics; Preemptive Countermeasure; Log Management;
Citations & Related Records
연도 인용수 순위
  • Reference
1 WIKIPEDIA, "Predictive analytics," Retrieved May. 27. 2015 from https://en.wikipedia.org/wiki/Predictive_analytics
2 Quinton Anderson, "Storm Real-Time Processing Cookbook," acorn publishing, 2014
3 Jerry Shenk, "SANS Seventh Annual Log Management Survey Report," pp. 5-6, April. 2011.
4 Sang-Jun Lee et al. (Unetsystem), "Unusual action decision system," Patent 10-2013-0134805, 2013.
5 Gregory Piatetsky, "R leads Rapid Miner, Python catches up, Big Data tools grow, Spark ignites," Retrieved May. 17. 2015 from http://www.kdnuggets.com/2015/05/poll-r-rapidminer-python-big-dataspark.html
6 Sung-Min Hong, "Open Language, R Language! Beyond that age," Cheil Communications(Magazine of Cheil Worldwide Inc.), 2004.
7 Choong-Hyun Yoo, "Technology Trends in Big Data Analytics and Introduction to R," NexR, 2012.
8 Kwang-Man KO, Beom-Chul Kwon, Sung-Chul Kim, Sang-Jun Lee, "Development of Statistical Prediction Engine for Integrated Log Analysis Systems," Journal of The 2013 Fall Conference of the KIPS, Vol. 20, No. 2, 2013.
9 Sang-Jun Lee et al.(Unetsystem), "Integrated log analysis system," Patent 10-1484290, 2015.