• Title/Summary/Keyword: Log 분산처리

Search Result 41, Processing Time 0.028 seconds

Log Management System of Web Server Based on Blockchain in Cloud Environment (클라우드 환경에서 블록체인 기반의 웹서버 로그 관리 시스템)

  • Son, Yong-Bum;Kim, Young-Hak
    • KIPS Transactions on Computer and Communication Systems
    • /
    • v.9 no.7
    • /
    • pp.143-148
    • /
    • 2020
  • Recently, web services have been expanded to various areas with the evolution of cloud environment. Whenever a user accesses a web service, the user's log information is stored in the web server. This log information is used as data to analyze the user's web service tendencies and is also used as important data to track the user's system access when a security problem in the system occurs. Currently, most web servers manage user log information in a centralized manner. When user log information is managed in a centralized manner, it is simple in the side of operation, but has a disadvantage of being very vulnerable to external malicious attacks. In the case of centralized management, user log information stored in the web server can be arbitrarily manipulated by external attacks, and in severe cases, the manipulated information can be leaked. In this case, it not only decreases the trust of the web service, but also makes it difficult to trace the source and cause of the attack on the web server. In order to solve these problems, this paper proposes a new method of managing user log information in a cloud environment by applying blockchain technology as an alternative to the existing centralized log management method. The proposed method can manage log information safely from external attacks because user log information is distributed and stored in blockchain on a private network with cloud environment.

Real-time log analysis system for detecting network attacks in a MapReduce environment (MapReduce 환경에서 네트워크 공격 탐지를 위한 실시간 로그 분석 시스템 개발)

  • Chang, Jin-Su;Shin, Jae-Hwan;Chang, Jae-Woo
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2017.11a
    • /
    • pp.37-40
    • /
    • 2017
  • 네트워크 기술의 발전으로 인터넷의 보급률이 증가함에 따라, 네트워크 사용량 또한 증가하고 있다. 그러나 네트워크 사용량이 증가함에 따라 악의적인 네트워크 접근 또한 증가하고 있다. 이러한 악의적인 접근은 네트워크에서 발생하는 보안 로그를 분석함으로써 탐지가 가능하다. 그러나 대규모의 네트워크 트래픽이 발생함에 따라, 보안 로그의 처리 및 분석에 많은 시간이 소요된다. 본 논문에서는 MapReduce 환경에서 네트워크 공격 탐지를 위한 실시간 로그 분석 시스템을 개발한다. 이를 위해, Hadoop의 MapReduce를 통해 보안 로그의 속성을 추출하고 대용량의 보안 로그를 분산 처리한다. 아울러 처리된 보안 로그를 분석함으로써 실시간으로 발생하는 네트워크 공격 패턴을 탐지하고, 이를 시각적으로 표현함으로써 사용자가 네트워크 상태를 보다 쉽게 파악할 수 있도록 한다.

Distributed IoT Sensor based Laboratory Safety Management System (분산 IoT센서 기반 실험실 안전관리 시스템)

  • Jeong, Daejin;Kim, Jaeyoon;Bae, Sangjung;Jung, Hoekyung
    • Journal of the Korea Institute of Information and Communication Engineering
    • /
    • v.23 no.1
    • /
    • pp.90-96
    • /
    • 2019
  • Storage cabinet in a lab in these days measures various environmental factors in real-time with IoT sensors. Preexisting system collects sensor data, analyze a risk and then command other equipment. Such centralized control system tends to have an issue with of speed slowing down. It's because when there are more storage cabinets, there are more data to process. In order to solve this issue, this report addresses decentralized IoT sensor based lab safety control system. It can analyze internal state of storage cabinet to identify any hazardous situations and effectively control them. Such decentralized control system using sensor modules for internal environment of the cabinet storage and automated control algorithm based on administrator's log history can manage any hazardous situations by automated control of environment factors of inside a lab. It would allow users to deal with a hazard if it happens. Even better, it can prevent it to happen from the beginning.

NoSQL-based User Behavior Detection System in Cloud Computing Environment (NoSQL 기반 클라우드 사용자 행동 탐지 시스템 설계)

  • Ahn, Kwang-Min;Lee, Bong-Hwan
    • Proceedings of the Korean Institute of Information and Commucation Sciences Conference
    • /
    • 2012.10a
    • /
    • pp.804-807
    • /
    • 2012
  • Cloud service provider has to protect client's information securely since all the resources are offered by the service provider, and a large number of users share the resources. In this paper, a NoSQL-based anomaly detection system is proposed in order to enhance the security of mobile cloud services. The existing integrated security management system that uses a relational database can not be used for real-time processing of data since security log from a variety of security equipment and data from cloud node have different data format with unstructured features. The proposed system can resolve the emerging security problem because it provides real time processing and scalability in distributed processing environment.

  • PDF

Real-time Network Attack Pattern Analysis System using Snort Log on MapReduce Environment (MapReduce 환경에서 Snort 로그를 이용한 실시간 네트워크 공격패턴 분석 시스템)

  • Kang, Moon-Hwan;Jang, Jin-Su;Shin, Young-Sung;Chang, Jae-Woo
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2017.04a
    • /
    • pp.75-77
    • /
    • 2017
  • 최근 급격히 증가하고 있는 네트워크 로그 상에서 보안위협에 신속히 대응하기 위해 기업들은 방화벽, IDS 등의 네트워크 보안 로그를 분석하여 보안 위협을 파악한다. Snort는 이러한 보안 위협에 대응하기 위해 네트워크 로그를 수집하는 도구 중 하나이다. 그러나 보안 관제 담당자는 방대한 양의 보안 관련 로그를 분석하기 위해 많은 시간이 필요하기 때문에, 관제 결과를 보고하고 대응하기까지 시간이 지체되는 문제가 존재한다. 이러한 문제를 해결하기 위해, 본 논문에서는 Snort 로그를 이용한 실시간 네트워크 공격패턴 분석 시스템을 제안한다. 제안하는 시스템은 대용량 데이터 처리에 효과적인 MapReduce 분산 처리를 활용하여 방대한 네트워크 로그를 추출 및 분석하기 때문에 보안 위협 상황 발생 여부를 실시간으로 빠르게 인지할 수 있다.

Process-level integration method for performance improvement of large scaled batch data processing in EAI environment (EAI에서 대용량 배치 데이터의 통합 성능 향상을 위한 Process-level 방식)

  • Kim Yonghee;Kwon Juhum
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2004.11a
    • /
    • pp.19-22
    • /
    • 2004
  • 분산 시스템 환경에서 여러 시스템에 정보를 전송하기 위한 방법으로 최근 EAI 의 DB Trigger 및 Redo Log 등을 이용한 실시간 데이터 통합 방식을 적용해 왔다. 그러나 기업에서 순차적인 배치 프로세스들을 통해 처리하는 대량의 데이터에 대해 기존의 EAI 의 데이터 통합 방식을 사용할 경우 모든 변경 건수에 대해 이벤트가 발생하여 Source 시스템의 부하 및 통합 성능상의 문제점이 있다. 본 논문에서는 순차적인 배치 프로세스들을 EAI 의 프로세스 레벨 통합을 적용하여 최종 변경된 데이터에 대해서만 통합하도록 하여 통합 처리 시간을 단축할 수 있는 방법을 제시하고자 한다.

  • PDF

User Access Patterns Discovery based on Apriori Algorithm under Web Logs (웹 로그에서의 Apriori 알고리즘 기반 사용자 액세스 패턴 발견)

  • Ran, Cong-Lin;Joung, Suck-Tae
    • The Journal of Korea Institute of Information, Electronics, and Communication Technology
    • /
    • v.12 no.6
    • /
    • pp.681-689
    • /
    • 2019
  • Web usage pattern discovery is an advanced means by using web log data, and it's also a specific application of data mining technology in Web log data mining. In education Data Mining (DM) is the application of Data Mining techniques to educational data (such as Web logs of University, e-learning, adaptive hypermedia and intelligent tutoring systems, etc.), and so, its objective is to analyze these types of data in order to resolve educational research issues. In this paper, the Web log data of a university are used as the research object of data mining. With using the database OLAP technology the Web log data are preprocessed into the data format that can be used for data mining, and the processing results are stored into the MSSQL. At the same time the basic data statistics and analysis are completed based on the processed Web log records. In addition, we introduced the Apriori Algorithm of Web usage pattern mining and its implementation process, developed the Apriori Algorithm program in Python development environment, then gave the performance of the Apriori Algorithm and realized the mining of Web user access pattern. The results have important theoretical significance for the application of the patterns in the development of teaching systems. The next research is to explore the improvement of the Apriori Algorithm in the distributed computing environment.

A Study on Distributed Processing of Big Data and User Authentication for Human-friendly Robot Service on Smartphone (인간 친화적 로봇 서비스를 위한 대용량 분산 처리 기술 및 사용자 인증에 관한 연구)

  • Choi, Okkyung;Jung, Wooyeol;Lee, Bong Gyou;Moon, Seungbin
    • Journal of Internet Computing and Services
    • /
    • v.15 no.1
    • /
    • pp.55-61
    • /
    • 2014
  • Various human-friendly robot services have been developed and mobile cloud computing is a real time computing service that allows users to rent IT resources what they want over the internet and has become the new-generation computing paradigm of information society. The enterprises and nations are actively underway of the business process using mobile cloud computing and they are aware of need for implementing mobile cloud computing to their business practice, but it has some week points such as authentication services and distributed processing technologies of big data. Sometimes it is difficult to clarify the objective of cloud computing service. In this study, the vulnerability of authentication services on mobile cloud computing is analyzed and mobile cloud computing model is constructed for efficient and safe business process. We will also be able to study how to process and analyze unstructured data in parallel to this model, so that in the future, providing customized information for individuals may be possible using unstructured data.

Blockchain Based Log Management Service for Non-Modifiable (블록체인을 활용한 위·변조가 불가능한 로그관리시스템)

  • Kim, Jin-Ju;Han, Young-Geun;Byun, Jae-Young
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2019.10a
    • /
    • pp.31-33
    • /
    • 2019
  • 시스템을 안정적으로 운용하기 위해서는 신뢰성 기반의 로그관리시스템이 필요하다. 모든 이력이 기록되는 로그의 위·변조를 방지하기 위해 로그 정보를 블록체인 기술로 관리하여 어떠한 상황에서도 시스템 이력을 신뢰할 수 있는 서비스를 제안한다. Hyplerledger Fabric을 사용하여 인증 관리 시스템에 의해 허가된 사용자만이 접근할 수 있다. 또한 분산원장에 한 번 기록된 로그 파일은 더 이상 수정하거나 삭제될 수 없다. 이 시스템을 활용하면 로그 파일의 위·변조 여부를 판단하는데 발생하는 시간, 비용, 불확실성을 크게 줄일 수 있을 것으로 기대된다.

A MapReduce-Based Workflow BIG-Log Clustering Technique (맵리듀스기반 워크플로우 빅-로그 클러스터링 기법)

  • Jin, Min-Hyuck;Kim, Kwanghoon Pio
    • Journal of Internet Computing and Services
    • /
    • v.20 no.1
    • /
    • pp.87-96
    • /
    • 2019
  • In this paper, we propose a MapReduce-supported clustering technique for collecting and classifying distributed workflow enactment event logs as a preprocessing tool. Especially, we would call the distributed workflow enactment event logs as Workflow BIG-Logs, because they are satisfied with as well as well-fitted to the 5V properties of BIG-Data like Volume, Velocity, Variety, Veracity and Value. The clustering technique we develop in this paper is intentionally devised for the preprocessing phase of a specific workflow process mining and analysis algorithm based upon the workflow BIG-Logs. In other words, It uses the Map-Reduce framework as a Workflow BIG-Logs processing platform, it supports the IEEE XES standard data format, and it is eventually dedicated for the preprocessing phase of the ${\rho}$-Algorithm that is a typical workflow process mining algorithm based on the structured information control nets. More precisely, The Workflow BIG-Logs can be classified into two types: of activity-based clustering patterns and performer-based clustering patterns, and we try to implement an activity-based clustering pattern algorithm based upon the Map-Reduce framework. Finally, we try to verify the proposed clustering technique by carrying out an experimental study on the workflow enactment event log dataset released by the BPI Challenges.