• Title/Summary/Keyword: SQL analysis

Search Result 142, Processing Time 0.022 seconds

Digital Forensic Investigation of HBase (HBase에 대한 디지털 포렌식 조사 기법 연구)

  • Park, Aran;Jeong, Doowon;Lee, Sang Jin
    • KIPS Transactions on Computer and Communication Systems
    • /
    • v.6 no.2
    • /
    • pp.95-104
    • /
    • 2017
  • As the technology in smart device is growing and Social Network Services(SNS) are becoming more common, the data which is difficult to be processed by existing RDBMS are increasing. As a result of this, NoSQL databases are getting popular as an alternative for processing massive and unstructured data generated in real time. The demand for the technique of digital investigation of NoSQL databases is increasing as the businesses introducing NoSQL database in their system are increasing, although the technique of digital investigation of databases has been researched centered on RDMBS. New techniques of digital forensic investigation are needed as NoSQL Database has no schema to normalize and the storage method differs depending on the type of database and operation environment. Research on document-based database of NoSQL has been done but it is not applicable as itself to other types of NoSQL Database. Therefore, the way of operation and data model, grasp of operation environment, collection and analysis of artifacts and recovery technique of deleted data in HBase which is a NoSQL column-based database are presented in this paper. Also the proposed technique of digital forensic investigation to HBase is verified by an experimental scenario.

Performance Comparison of DW System Tajo Based on Hadoop and Relational DBMS (하둡 기반 DW시스템 타조와 관계형 DBMS의 성능 비교)

  • Liu, Chen;Ko, Junghyun;Yeo, Jeongmo
    • KIPS Transactions on Software and Data Engineering
    • /
    • v.3 no.9
    • /
    • pp.349-354
    • /
    • 2014
  • Since Hadoop which is the Big-data processing platform was announced, SQL-on-Hadoop is the spotlight as the technique to analyze data using SQL on Hadoop. Tajo created by Korean programmers has recently been promoted to Top-Level-Project status by the Apache in April and has been paid attention all around world. Despite a sensible change caused by Hadoop's appearance in DW market, researches of those performance is insufficient. Thus, this study has been conducted to help choose a DW solution based on SQL-on-Hadoop as progressing the test on comparison analysis of RDBMS and Tajo. It has shown that Tajo based on Hadoop is more superior than RDBMS if it is used with accurate strategy. In addition, open-source project Tajo is expected not only to achieve improvements in technique due to active participation of many developers but also to be in charge of an important role of DW in the filed of data analysis.

The Method of Analyzing Firewall Log Data using MapReduce based on NoSQL (NoSQL기반의 MapReduce를 이용한 방화벽 로그 분석 기법)

  • Choi, Bomin;Kong, Jong-Hwan;Hong, Sung-Sam;Han, Myung-Mook
    • Journal of the Korea Institute of Information Security & Cryptology
    • /
    • v.23 no.4
    • /
    • pp.667-677
    • /
    • 2013
  • As the firewall is a typical network security equipment, it is usually installed at most of internal/external networks and makes many packet data in/out. So analyzing the its logs stored in it can provide important and fundamental data on the network security research. However, along with development of communications technology, the speed of internet network is improved and then the amount of log data is becoming 'Massive Data' or 'BigData'. In this trend, there are limits to analyze log data using the traditional database model RDBMS. In this paper, through our Method of Analyzing Firewall log data using MapReduce based on NoSQL, we have discovered that the introducing NoSQL data base model can more effectively analyze the massive log data than the traditional one. We have demonstrated execellent performance of the NoSQL by comparing the performance of data processing with existing RDBMS. Also the proposed method is evaluated by experiments that detect the three attack patterns and shown that it is highly effective.

The Application of SQL in Terrain Information Analysis for Route Design (도로 설계를 위한 지형정보 해석에 있어서 SQL의 응용)

  • Kang, Joon-Mook;Yoon, Hee-Cheon;Lee, Hyung-Seok;Lee, Sung-Soong
    • Journal of Korean Society for Geospatial Information Science
    • /
    • v.3 no.2 s.6
    • /
    • pp.29-42
    • /
    • 1995
  • Route design in topographical plane map brings many problems in efficiency and requires much time and labor by hand Recently, the active studies of efficient route design method using 3-D terrain information are being developed according to increasing concentration on GSIS. In order to analyze terrain information for route design efficiently, this study presents objective and overall datum by applying SQL in construction and analysis of database and the possibility of three-dimensional terrain information analysis, This study generates 3-D base map on topographical map of scale 1:5,000 and acquires terrain information that have various thematic map data; contour, land use, roadway, and drange. This is a study on the application of SQL in route design and construction of the terrain information that linked by graphic datum of completed topographical map and attributed datum of database. As the result of this study, we can produce promptly and efficiently design datum of profile annotation, cross section, and volume computations to the preliminary route for route design and apply this efficient method to route design by understanding visual DTM which is composed of the roadway and the natural scene after design.

  • PDF

The Method of Deleted Record Recovery for MySQL MyISAM Database (MySQL MyISAM 데이터베이스의 삭제 레코드에 대한 복구 기법)

  • Noh, Woo-seon;Jang, Sung-min;Kang, Chul-hoon;Lee, Kyung-min;Lee, Sang-jin
    • Journal of the Korea Institute of Information Security & Cryptology
    • /
    • v.26 no.1
    • /
    • pp.125-134
    • /
    • 2016
  • MySQL database is currently used by many users and It has gained a big market share in the database market. Even though MyISAM storage engine was used as a default storage engine before, but records recovery method does not existed. Deleted records have a high possibility for important evidence and it is almost impossible to determine that investigators manually examine large amounts of database directly. This paper suggests the universal recovery method for deleted records and presents the experimental results.

Query optimizing Efficient Extracting Warehouse use PL/SQL for Analysis Data in data (데이터웨어하우스에서 효율적인 분석데이터 추출시 PL/SQL을 이용한 질의 최적화)

  • Jeong, Seung-Kyung
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2006.11a
    • /
    • pp.313-316
    • /
    • 2006
  • 기존에 기업의 의사결정을 위해 사용되는 데이터 웨어하우스나 데이터 마트에서 성능향상을 위한 많은 연구들이 있었다. 하지만 복잡한 업무요건이 추가되고 기업의 요구가 다양해짐에 따라 RDBMS의 성능은 점점 낮아지고 이를 해결하기 위한 필요성이 요구되었다. 따라서 본 논문에서는 이를 해결하기 위해 버퍼기능을 하는 PL/SQL Package를 구현하여 효율적인 질의 최적화 방법을 제안하고자 한다. 그리고 본 논문에서 제안한 방법이 기존의 방법보다 성능이 좋다는 것을 실험을 통해 증명해 보였다.

  • PDF

Design and Implementation of MongoDB-based Unstructured Log Processing System over Cloud Computing Environment (클라우드 환경에서 MongoDB 기반의 비정형 로그 처리 시스템 설계 및 구현)

  • Kim, Myoungjin;Han, Seungho;Cui, Yun;Lee, Hanku
    • Journal of Internet Computing and Services
    • /
    • v.14 no.6
    • /
    • pp.71-84
    • /
    • 2013
  • Log data, which record the multitude of information created when operating computer systems, are utilized in many processes, from carrying out computer system inspection and process optimization to providing customized user optimization. In this paper, we propose a MongoDB-based unstructured log processing system in a cloud environment for processing the massive amount of log data of banks. Most of the log data generated during banking operations come from handling a client's business. Therefore, in order to gather, store, categorize, and analyze the log data generated while processing the client's business, a separate log data processing system needs to be established. However, the realization of flexible storage expansion functions for processing a massive amount of unstructured log data and executing a considerable number of functions to categorize and analyze the stored unstructured log data is difficult in existing computer environments. Thus, in this study, we use cloud computing technology to realize a cloud-based log data processing system for processing unstructured log data that are difficult to process using the existing computing infrastructure's analysis tools and management system. The proposed system uses the IaaS (Infrastructure as a Service) cloud environment to provide a flexible expansion of computing resources and includes the ability to flexibly expand resources such as storage space and memory under conditions such as extended storage or rapid increase in log data. Moreover, to overcome the processing limits of the existing analysis tool when a real-time analysis of the aggregated unstructured log data is required, the proposed system includes a Hadoop-based analysis module for quick and reliable parallel-distributed processing of the massive amount of log data. Furthermore, because the HDFS (Hadoop Distributed File System) stores data by generating copies of the block units of the aggregated log data, the proposed system offers automatic restore functions for the system to continually operate after it recovers from a malfunction. Finally, by establishing a distributed database using the NoSQL-based Mongo DB, the proposed system provides methods of effectively processing unstructured log data. Relational databases such as the MySQL databases have complex schemas that are inappropriate for processing unstructured log data. Further, strict schemas like those of relational databases cannot expand nodes in the case wherein the stored data are distributed to various nodes when the amount of data rapidly increases. NoSQL does not provide the complex computations that relational databases may provide but can easily expand the database through node dispersion when the amount of data increases rapidly; it is a non-relational database with an appropriate structure for processing unstructured data. The data models of the NoSQL are usually classified as Key-Value, column-oriented, and document-oriented types. Of these, the representative document-oriented data model, MongoDB, which has a free schema structure, is used in the proposed system. MongoDB is introduced to the proposed system because it makes it easy to process unstructured log data through a flexible schema structure, facilitates flexible node expansion when the amount of data is rapidly increasing, and provides an Auto-Sharding function that automatically expands storage. The proposed system is composed of a log collector module, a log graph generator module, a MongoDB module, a Hadoop-based analysis module, and a MySQL module. When the log data generated over the entire client business process of each bank are sent to the cloud server, the log collector module collects and classifies data according to the type of log data and distributes it to the MongoDB module and the MySQL module. The log graph generator module generates the results of the log analysis of the MongoDB module, Hadoop-based analysis module, and the MySQL module per analysis time and type of the aggregated log data, and provides them to the user through a web interface. Log data that require a real-time log data analysis are stored in the MySQL module and provided real-time by the log graph generator module. The aggregated log data per unit time are stored in the MongoDB module and plotted in a graph according to the user's various analysis conditions. The aggregated log data in the MongoDB module are parallel-distributed and processed by the Hadoop-based analysis module. A comparative evaluation is carried out against a log data processing system that uses only MySQL for inserting log data and estimating query performance; this evaluation proves the proposed system's superiority. Moreover, an optimal chunk size is confirmed through the log data insert performance evaluation of MongoDB for various chunk sizes.

Selection Analysis of Databases to Manage Big Data (빅데이터 관리를 위한 데이터베이스 선정분석)

  • Park, Sungbum;Lee, Sangwon;Ahn, Hyunsup;Jung, In-Hwan
    • Proceedings of the Korean Institute of Information and Commucation Sciences Conference
    • /
    • 2013.10a
    • /
    • pp.258-260
    • /
    • 2013
  • There are two major factors to use NoSQL in order to manage Big Data; to increase productivity of an application programmer and to increase data access performance. But, in many business fields, this hopeful plan lacks careful consideration. For efficient and effective management and analysis of Big Data, it is necessary to perform a test with the expectation for productivity and performance of the application programmer before deciding whether NoSQL technique is used or not. In this paper, we research on programmer productivity, data access performance, risk distribution, and so forth.

  • PDF

A Study of Web Site Hacking Through Vulnerability Analysis (취약점 분석을 통한 Web Site 해킹 연구)

  • Song, Jin-Young;Park, Dea-Woo
    • Proceedings of the Korean Institute of Information and Commucation Sciences Conference
    • /
    • 2010.05a
    • /
    • pp.303-306
    • /
    • 2010
  • Personal information being leaked, and personal assets that through a malicious web site for hackers to exploit. Other confidential information via the web site of the country, and your personal information by illegally accessing the data has been obtained who Hacker forces are operating in some countries. Due to the problem of web site management has many vulnerabilities that web sites, as well as programs. In this paper, in the trend world, as well as domestic XSS, SQL Injection, Web Shell analysis of the vulnerability to attacks and XSS, SQL Injection, Web Shell is a direct attack to attack. Security measures are presented what after the attack demonstrated the hack to data collection, analysis. In this study, web site management, web site security and safety can be improved and research will contribute.

  • PDF

NoSQL-based Sensor Web System for Fine Particles Analysis Services (미세먼지 분석 서비스를 위한 NoSQL 기반 센서 웹 시스템)

  • Kim, Jeong-Joon;Kwak, Kwang-Jin;Park, Jeong-Min
    • The Journal of the Institute of Internet, Broadcasting and Communication
    • /
    • v.19 no.2
    • /
    • pp.119-125
    • /
    • 2019
  • Recently, it has become a social problem due to fine particles. There are more people wearing masks, weather alerts and disaster notices. Research and policy are actively underway. Meteorologically, the biggest damage caused by fine particles is the inversion layer phenomenon. In this study, we designed a system to warn fine Particles by analyzing inversion layer and wind direction. This weather information system proposes a system that can efficiently perform scalability and parallel processing by using OGC sensor web enablement system and NoSQL storage for sensor control and data exchange.