• Title/Summary/Keyword: Apache Storm

Search Result 12, Processing Time 0.016 seconds

Real-time and Parallel Semantic Translation Technique for Large-Scale Streaming Sensor Data in an IoT Environment (사물인터넷 환경에서 대용량 스트리밍 센서데이터의 실시간·병렬 시맨틱 변환 기법)

  • Kwon, SoonHyun;Park, Dongwan;Bang, Hyochan;Park, Youngtack
    • Journal of KIISE
    • /
    • v.42 no.1
    • /
    • pp.54-67
    • /
    • 2015
  • Nowadays, studies on the fusion of Semantic Web technologies are being carried out to promote the interoperability and value of sensor data in an IoT environment. To accomplish this, the semantic translation of sensor data is essential for convergence with service domain knowledge. The existing semantic translation technique, however, involves translating from static metadata into semantic data(RDF), and cannot properly process real-time and large-scale features in an IoT environment. Therefore, in this paper, we propose a technique for translating large-scale streaming sensor data generated in an IoT environment into semantic data, using real-time and parallel processing. In this technique, we define rules for semantic translation and store them in the semantic repository. The sensor data is translated in real-time with parallel processing using these pre-defined rules and an ontology-based semantic model. To improve the performance, we use the Apache Storm, a real-time big data analysis framework for parallel processing. The proposed technique was subjected to performance testing with the AWS observation data of the Meteorological Administration, which are large-scale streaming sensor data for demonstration purposes.

Real-time Abnormal Behavior Detection System based on Fast Data (패스트 데이터 기반 실시간 비정상 행위 탐지 시스템)

  • Lee, Myungcheol;Moon, Daesung;Kim, Ikkyun
    • Journal of the Korea Institute of Information Security & Cryptology
    • /
    • v.25 no.5
    • /
    • pp.1027-1041
    • /
    • 2015
  • Recently, there are rapidly increasing cases of APT (Advanced Persistent Threat) attacks such as Verizon(2010), Nonghyup(2011), SK Communications(2011), and 3.20 Cyber Terror(2013), which cause leak of confidential information and tremendous damage to valuable assets without being noticed. Several anomaly detection technologies were studied to defend the APT attacks, mostly focusing on detection of obvious anomalies based on known malicious codes' signature. However, they are limited in detecting APT attacks and suffering from high false-negative detection accuracy because APT attacks consistently use zero-day vulnerabilities and have long latent period. Detecting APT attacks requires long-term analysis of data from a diverse set of sources collected over the long time, real-time analysis of the ingested data, and correlation analysis of individual attacks. However, traditional security systems lack sophisticated analytic capabilities, compute power, and agility. In this paper, we propose a Fast Data based real-time abnormal behavior detection system to overcome the traditional systems' real-time processing and analysis limitation.