• Title/Summary/Keyword: Hadoop security

Search Result 35, Processing Time 0.025 seconds

High Rate Denial-of-Service Attack Detection System for Cloud Environment Using Flume and Spark

  • Gutierrez, Janitza Punto;Lee, Kilhung
    • Journal of Information Processing Systems
    • /
    • v.17 no.4
    • /
    • pp.675-689
    • /
    • 2021
  • Nowadays, cloud computing is being adopted for more organizations. However, since cloud computing has a virtualized, volatile, scalable and multi-tenancy distributed nature, it is challenging task to perform attack detection in the cloud following conventional processes. This work proposes a solution which aims to collect web server logs by using Flume and filter them through Spark Streaming in order to only consider suspicious data or data related to denial-of-service attacks and reduce the data that will be stored in Hadoop Distributed File System for posterior analysis with the frequent pattern (FP)-Growth algorithm. With the proposed system, we can address some of the difficulties in security for cloud environment, facilitating the data collection, reducing detection time and consequently enabling an almost real-time attack detection.

Algorithm Design to Judge Fake News based on Bigdata and Artificial Intelligence

  • Kang, Jangmook;Lee, Sangwon
    • International Journal of Internet, Broadcasting and Communication
    • /
    • v.11 no.2
    • /
    • pp.50-58
    • /
    • 2019
  • The clear and specific objective of this study is to design a false news discriminator algorithm for news articles transmitted on a text-based basis and an architecture that builds it into a system (H/W configuration with Hadoop-based in-memory technology, Deep Learning S/W design for bigdata and SNS linkage). Based on learning data on actual news, the government will submit advanced "fake news" test data as a result and complete theoretical research based on it. The need for research proposed by this study is social cost paid by rumors (including malicious comments) and rumors (written false news) due to the flood of fake news, false reports, rumors and stabbings, among other social challenges. In addition, fake news can distort normal communication channels, undermine human mutual trust, and reduce social capital at the same time. The final purpose of the study is to upgrade the study to a topic that is difficult to distinguish between false and exaggerated, fake and hypocrisy, sincere and false, fraud and error, truth and false.

Design of Splunk Platform based Big Data Analysis System for Objectionable Information Detection (Splunk 플랫폼을 활용한 유해 정보 탐지를 위한 빅데이터 분석 시스템 설계)

  • Lee, Hyeop-Geon;Kim, Young-Woon;Kim, Ki-Young;Choi, Jong-Seok
    • The Journal of Korea Institute of Information, Electronics, and Communication Technology
    • /
    • v.11 no.1
    • /
    • pp.76-81
    • /
    • 2018
  • The Internet of Things (IoT), which is emerging as a future economic growth engine, has been actively introduced in areas close to our daily lives. However, there are still IoT security threats that need to be resolved. In particular, with the spread of smart homes and smart cities, an explosive amount of closed-circuit televisions (CCTVs) have been installed. The Internet protocol (IP) information and even port numbers assigned to CCTVs are open to the public via search engines of web portals or on social media platforms, such as Facebook and Twitter; even with simple tools these pieces of information can be easily hacked. For this reason, a big-data analytics system is needed, capable of supporting quick responses against data, that can potentially contain risk factors to security or illegal websites that may cause social problems, by assisting in analyzing data collected by search engines and social media platforms, frequently utilized by Internet users, as well as data on illegal websites.

A Study on the Enhancement Process of the Telecommunication Network Management using Big Data Analysis (Big Data 분석을 활용한 통신망 관리 시스템의 개선방안에 관한 연구)

  • Koo, Sung-Hwan;Shin, Min-Soo
    • Journal of the Korea Academia-Industrial cooperation Society
    • /
    • v.13 no.12
    • /
    • pp.6060-6070
    • /
    • 2012
  • Real-Time Enterprise (RTE)'s key requirement is that it should respond and adapt fast to the change of the firms' internal and external situations including the change of market and customers' needs. Recently, the big data processing technology to support the speedy change of the firms is spotlighted. Under the circumstances that wire and wireless communication networks are evolving with an accelerated rate, it is especially critical to provide a strong security monitoring function and stable services through a real-time processing of massive communication data traffic. By applying the big data processing technology based on a cloud computing architecture, this paper solves the managerial problems of telecommunication service providers and discusses how to operate the network management system effectively.

An Analysis of Impact on the Quality of Life for Chronic Patients based Big Data (빅데이터 기반 만성질환자의 삶의 질에 미치는 영향분석)

  • Kim, Min-kyoung;Cho, Young-bok
    • Journal of the Korea Institute of Information and Communication Engineering
    • /
    • v.23 no.11
    • /
    • pp.1351-1356
    • /
    • 2019
  • The purpose of this study is to investigate the effect of personal factors and community factors on the quality of life based on the presence of chronic patients based on the Big Data Platform. As a method of study, second data of 2017 community health survey and Statistics Korea by City·Gun·Gu public office were used and a multi-level analysis was conducted after separating EQ-5D index, individual factor and community factor. As a result, men, age, education level, monthly household income, having economic activity, the number of sports infrastructure were positively associated with the quality of life, and subjective health not good, extremely perceived stress were negatively associated with the quality of life. Research will continue to provide a platform independent of hardware that can utilize the cloud and open source for medical big data analysis in the future.