• Title/Summary/Keyword: 관계형 데이터베이스 관리 시스템

Search Result 196, Processing Time 0.029 seconds

Design and Implementation of MongoDB-based Unstructured Log Processing System over Cloud Computing Environment (클라우드 환경에서 MongoDB 기반의 비정형 로그 처리 시스템 설계 및 구현)

  • Kim, Myoungjin;Han, Seungho;Cui, Yun;Lee, Hanku
    • Journal of Internet Computing and Services
    • /
    • v.14 no.6
    • /
    • pp.71-84
    • /
    • 2013
  • Log data, which record the multitude of information created when operating computer systems, are utilized in many processes, from carrying out computer system inspection and process optimization to providing customized user optimization. In this paper, we propose a MongoDB-based unstructured log processing system in a cloud environment for processing the massive amount of log data of banks. Most of the log data generated during banking operations come from handling a client's business. Therefore, in order to gather, store, categorize, and analyze the log data generated while processing the client's business, a separate log data processing system needs to be established. However, the realization of flexible storage expansion functions for processing a massive amount of unstructured log data and executing a considerable number of functions to categorize and analyze the stored unstructured log data is difficult in existing computer environments. Thus, in this study, we use cloud computing technology to realize a cloud-based log data processing system for processing unstructured log data that are difficult to process using the existing computing infrastructure's analysis tools and management system. The proposed system uses the IaaS (Infrastructure as a Service) cloud environment to provide a flexible expansion of computing resources and includes the ability to flexibly expand resources such as storage space and memory under conditions such as extended storage or rapid increase in log data. Moreover, to overcome the processing limits of the existing analysis tool when a real-time analysis of the aggregated unstructured log data is required, the proposed system includes a Hadoop-based analysis module for quick and reliable parallel-distributed processing of the massive amount of log data. Furthermore, because the HDFS (Hadoop Distributed File System) stores data by generating copies of the block units of the aggregated log data, the proposed system offers automatic restore functions for the system to continually operate after it recovers from a malfunction. Finally, by establishing a distributed database using the NoSQL-based Mongo DB, the proposed system provides methods of effectively processing unstructured log data. Relational databases such as the MySQL databases have complex schemas that are inappropriate for processing unstructured log data. Further, strict schemas like those of relational databases cannot expand nodes in the case wherein the stored data are distributed to various nodes when the amount of data rapidly increases. NoSQL does not provide the complex computations that relational databases may provide but can easily expand the database through node dispersion when the amount of data increases rapidly; it is a non-relational database with an appropriate structure for processing unstructured data. The data models of the NoSQL are usually classified as Key-Value, column-oriented, and document-oriented types. Of these, the representative document-oriented data model, MongoDB, which has a free schema structure, is used in the proposed system. MongoDB is introduced to the proposed system because it makes it easy to process unstructured log data through a flexible schema structure, facilitates flexible node expansion when the amount of data is rapidly increasing, and provides an Auto-Sharding function that automatically expands storage. The proposed system is composed of a log collector module, a log graph generator module, a MongoDB module, a Hadoop-based analysis module, and a MySQL module. When the log data generated over the entire client business process of each bank are sent to the cloud server, the log collector module collects and classifies data according to the type of log data and distributes it to the MongoDB module and the MySQL module. The log graph generator module generates the results of the log analysis of the MongoDB module, Hadoop-based analysis module, and the MySQL module per analysis time and type of the aggregated log data, and provides them to the user through a web interface. Log data that require a real-time log data analysis are stored in the MySQL module and provided real-time by the log graph generator module. The aggregated log data per unit time are stored in the MongoDB module and plotted in a graph according to the user's various analysis conditions. The aggregated log data in the MongoDB module are parallel-distributed and processed by the Hadoop-based analysis module. A comparative evaluation is carried out against a log data processing system that uses only MySQL for inserting log data and estimating query performance; this evaluation proves the proposed system's superiority. Moreover, an optimal chunk size is confirmed through the log data insert performance evaluation of MongoDB for various chunk sizes.

A Measuring Technique for Hydraulic Information in Small and Medium Sized Streams by an Image Analysis (영상분석기법을 이용한 중소하천의 수리량 계측 및 정보화 기술)

  • Lee, Nam-joo;Yu, Kwonkou;Kang, Taeuk
    • Proceedings of the Korea Water Resources Association Conference
    • /
    • 2017.05a
    • /
    • pp.15-15
    • /
    • 2017
  • 하천의 수위와 유량 등의 수리량 자료는 치수, 이수, 환경 등에 관한 하천 계획과 관리의 기초자료로 활용된다. 따라서 체계적인 하천 관리를 위해서는 신뢰성 있는 하천 수리량 자료의 확보가 요구된다. 이에 따라 우리나라에서는 많은 하천에 수위 관측소를 설치하여 실시간 수위를 측정하고 있다. 하지만 유량의 측정은 국가하천 등 주요 대규모 하천에 국한되어 있는 실정이다. 반면에 우리나라 대부분의 홍수 피해는 중 소하천에서 발생하고 있지만, 중 소하천에 대한 수문조사는 상대적으로 매우 미진한 실정이다. 이러한 중 소하천에 대한 수리 정보 조사의 가장 큰 어려움은 대부분의 중 소하천이 지방자치단체에서 관리하고 있어 예산과 인력 지원이 어렵기 때문이다. 이에 본 연구에서는 중 소하천에 산재한 보와 같은 하천 횡단 구조물의 유량 측정 기능을 이용하고, 급성장하고 있는 영상 분석 기술과 IT를 하천공학 분야에 접목하여 중 소하천의 수리 정보를 저비용, 실시간으로 원격 측정하는 시스템을 개발하고자 하였다. 이를 위해 연구에서는 저가형 영상 장비를 이용하여 하천 횡단구조물 주변의 흐름 영상을 원격으로 확보하고, 해당 영상을 데이터베이스로 송신하는 장치인 WIA 시스템(wireless image acquisition system)을 개발하였다. 그리고 촬영된 영상을 분석하여 수위를 도출하는 영상 분석 프로그램, 수위-유량 관계 곡선(rating curve), 미계측 지점에 대한 하천 수리 정보를 계산하기 위한 HEC-RAS를 포함하는 River-HQ 시스템을 개발하였다. 개발된 WIA & River-HQ 시스템은 한국건설기술연구원 하천실험센터 내 직선 수로(실규모 하천) 구간에 대하여 정확성이 검증되었다. 이러한 WIA & River-HQ 시스템은 실제 하천에 대한 적용성과 안정성 검토를 통해 향후 중 소하천의 환경 관리와 홍수예경보 업무 등에 활용될 수 있을 것으로 판단된다.

  • PDF

The traffic performance evaluation between remote server and mobile for applying to encryption protocol in the Wellness environment (웰니스 환경에서 암호화 프로토콜 적용을 위한 모바일과 원격 서버간 트래픽 성능 평가)

  • Lee, Jae-Pil;Kim, Young-Hyuk;Lee, Jae-Kwang
    • Journal of Digital Convergence
    • /
    • v.11 no.11
    • /
    • pp.415-420
    • /
    • 2013
  • U-WHS refers to a means of remote health monitoring service to combine fitness with wellbing. U-WHS is a system which can measure and manage biometric information of patients without any limitation on time and space. In this paper, we performed in order to look into the influence that the encryption module influences on the communication evaluation in the biometric information transmission gone to the smart mobile device and Hospital Information System.In the case of the U-WHS model, the client used the Objective-c programming language for software development of iOS Xcode environment and SEED and HIGHT encryption module was applied. In the case of HIS, the MySQL which is the Websocket API of the HTML5 and relational database management system for the client and inter-server communication was applied. Therefore, in WIFI communication environment, by using wireshark, data transfer rate of the biometric information, delay and loss rate was checked for the evaluation.

The Establishment of an Activity-Based EVM - PMIS Integration Model (액티비티 기반의 EVM - PMIS 통합모델 구축)

  • Na, Kwang-Tae;Kang, Byeung-Hee
    • Journal of the Korea Institute of Building Construction
    • /
    • v.10 no.1
    • /
    • pp.199-212
    • /
    • 2010
  • To establish an infrastructure for technology and information in the domestic construction industry, several construction regulations pertaining to construction information have been institutionalized. However, there are major problems with the domestic information classification system, earned value management (EVM) and project management information system (PMIS). In particular, the functions of the current PMIS have consisted of a builder-oriented system, and as EVM is not applied to PMIS, the functions of reporting, analysis and forecast for owners are lacking. Moreover, owners cannot confirm information on construction schedule and cost in real time due to the differences between the EVM and PMIS operation systems. The purpose of this study is to provide a framework that is capable of operating PMIS efficiently under an e-business environment, by providing a proposal on how to establish a work breakdown structure (WBS) and an EVM - PMIS integration model, so that PMIS may provide the function of EVM, and stakeholders may have all information in common. At the core of EVM - PMIS integration is the idea that EVM and PMIS have the same operation system, in order to be an activity-based system. The principle of the integration is data integration, in which the information field of an activity is connected with the field of a relational database table consisting of sub-modules for the schedule and cost management function of PMIS using a relational database management system. Therefore, the planned value (PV), cost value (CV), actual cost (AC), schedule variance (SV), schedule performance index (SPI), cost variance (CV) and cost performance index (CPI) of an activity are connected with the field of the relational database table for the schedule and cost sub-modules of PMIS.

Design and Implementation of SRS Data Model for IoT Environment (IoT 환경을 위한 SRS 데이터 모델의 설계 및 구현)

  • Lee, Sukhoon;Jeong, Dongwon;Jung, Hyunjun;Baik, Doo-Kwon
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2015.10a
    • /
    • pp.1235-1238
    • /
    • 2015
  • 센서 레지스트리 시스템(Sensor Registry System, CRS)은 이기종 센서 네트워크 환경에서 센서 데이터의 일관성 있는 의미 해석을 위하여 센서 메타데이터를 등록하고 관리하는 시스템이다. 최근 사물인터넷(Internet of Things, IoT) 패러다임이 대두됨에 따라 센서 네트워크의 개념 및 이용 목적 등이 변화되고 있으며, SRS 역시 이를 반영하여 센서와 연관된 데이터 모델의 개선 및 확장이 요구된다. 따라서 이 논문은 IoT 환경에서 기존 SRS를 개선하기 위하여 Semantic Sensor Network Ontology(SSNO) 기반의 SRS 데이터 모델을 제안한다. 이를 위하여 IoT 환경에서 SRS의 목적 및 요구사항을 분석하고 SSNO의 개념들 중 필요 요소와 불필요 요소를 반영하여 제안 모델을 설계한다. 또한 생성된 SRS 데이터 모델을 이용하여 관계형 데이터베이스로 구축하고 SRS를 웹 어플리케이션으로 구현한다. 제안하는 SRS 데이터 모델은 기존 모델들에 비해 SSNO 온톨로지를 가장 적합하게 표현하므로 풍부한 의미 처리가 가능하다.

A Digital Library Prototype for Access to Diverse Collections (다양한 장서 접근을 위한 디지털 도서관의 프로토타입 구축)

  • Choi Won-Tae
    • Journal of the Korean Society for Library and Information Science
    • /
    • v.32 no.2
    • /
    • pp.295-307
    • /
    • 1998
  • This article is an overview of the digital library project, indicating what roles Koreas diverse digital collections may play. Our digital library prototype has simple architecture, consisting of digital repositories, filters, indexing and searching, and clients. Digital repositories include various types of materials and databases. The role of filters is to recognize a format of a document collection and mark the structural components of each of its documents. We are using a database management system (ORACLE and ConText) supporting user-defined functions and access methods that allows us to easily incorporate new object analysis, structuring, and indexing technology into a repository. Clients can be considered browsers or viewers designed for different document data types, such as image, audio, video, SGML, PDF, and KORMARC. The combination of navigational tools supports a variety of approaches to identifying collections and browsing or searching for individual items. The search interface was implemented using HTML forms and the World Wide Web's CGI mechanism.

  • PDF

GWB: An integrated software system for Managing and Analyzing Genomic Sequences (GWB: 유전자 서열 데이터의 관리와 분석을 위한 통합 소프트웨어 시스템)

  • Kim In-Cheol;Jin Hoon
    • Journal of Internet Computing and Services
    • /
    • v.5 no.5
    • /
    • pp.1-15
    • /
    • 2004
  • In this paper, we explain the design and implementation of GWB(Gene WorkBench), which is a web-based, integrated system for efficiently managing and analyzing genomic sequences, Most existing software systems handling genomic sequences rarely provide both managing facilities and analyzing facilities. The analysis programs also tend to be unit programs that include just single or some part of the required functions. Moreover, these programs are widely distributed over Internet and require different execution environments. As lots of manual and conversion works are required for using these programs together, many life science researchers suffer great inconveniences. in order to overcome the problems of existing systems and provide a more convenient one for helping genomic researches in effective ways, this paper integrates both managing facilities and analyzing facilities into a single system called GWB. Most important issues regarding the design of GWB are how to integrate many different analysis programs into a single software system, and how to provide data or databases of different formats required to run these programs. In order to address these issues, GWB integrates different analysis programs byusing common input/output interfaces called wrappers, suggests a common format of genomic sequence data, organizes local databases consisting of a relational database and an indexed sequential file, and provides facilities for converting data among several well-known different formats and exporting local databases into XML files.

  • PDF

Design and Implementation of Sensor Registry Data Model for IoT Environment (IoT 환경을 위한 센서 레지스트리 데이터 모델의 설계 및 구현)

  • Lee, Sukhoon;Jeong, Dongwon;Jung, Hyunjun;Baik, Doo-Kwon
    • KIPS Transactions on Software and Data Engineering
    • /
    • v.5 no.5
    • /
    • pp.221-230
    • /
    • 2016
  • With emerging the Internet of Things (IoT) paradigm, the sensor network and sensor platform technologies have been changed according to exploding amount of sensors. Sensor Registry System (SRS) as a sensor platform is a system that registers and manages sensor metadata for consistent semantic interpretation in heterogeneous sensor networks. However, the SRS is unsuitable for the IoT environment. Therefore, this paper proposes sensor registry data model to register and manager sensor information in the IoT environment. We analyze Semantic Sensor Network Ontology (SSNO) for improving the existed SRS, and design metamodel based on the analysis result. We also build tables in a relational database using the designed metamodel, then implement SRS as a web application. This paper applies the SSNO and sensor ontology examples with translating into the proposed model in order to verify the suitability of the proposed sensor registry data model. As the evaluation result, the proposed model shows abundant expression of semantics by comparison with existed models.

Experiment and Simulation for Evaluation of Jena Storage Plug-in Considering Hierarchical Structure (계층 구조를 고려한 Jena Plug-in 저장소의 평가를 위한 실험 및 시뮬레이션)

  • Shin, Hee-Young;Jeong, Dong-Won;Baik, Doo-Kwon
    • Journal of the Korea Society for Simulation
    • /
    • v.17 no.2
    • /
    • pp.31-47
    • /
    • 2008
  • As OWL(Web Ontology Language) has been selected as a standard ontology description language by W3C, many ontologies have been building and developing in OWL. The lena developed by HP as an Application Programming Interface(API) provides various APIs to develop inference engines as well as storages, and it is widely used for system development. However, the storage model of Jena2 stores most owl documents not acceptable into a single table and it shows low processing performance for a large ontology data set. Most of all, Jena2 storage model does not consider hierarchical structures of classes and properties. In addition, it shows low query processing performance using the hierarchical structure because of many join operations. To solve these issues, this paper proposes an OWL ontology relational database model. The proposed model semantically classifies and stores information such as classes, properties, and instances. It improves the query processing performance by managing hierarchical information in a separate table. This paper also describes the implementation and evaluation results. This paper also shows the experiment and evaluation result and the comparative analysis on both results. The experiment and evaluation show our proposal provides a prominent performance as against Jena2.

  • PDF

Design and Implementation of Middleware supporting translation of EDI using XML (XML기반의 EDI 문서교환을 위한 미들웨어 설계 및 구현)

  • Choi, Gwang-Mi;Park, Su-Young;Jung, Chai-Yeoung
    • The KIPS Transactions:PartB
    • /
    • v.9B no.6
    • /
    • pp.845-852
    • /
    • 2002
  • Electronic document processing using EDl (Electronic Data Interchange) must exchange documents using VAN (Value Added Network). However. the use of exclusive software needs alteration of a new document and the use of VAN(Value Added Network) demands an exchange of document and high cost for maintenance. Due to these problems, the existing EDI (Electronic Data Interchange) is turning into Web-based EDI (Electronic Data Interchange). This paper suggests techniques that change EDI (Electronic Data Interchange) messages which exist in two relational databases into XML (extensible Markeup Language) using the JDBC bridge. Also this paper proposes a method that recovers schema using converted XML (extensible Markeup Language) file, and a method which inserts an original record into a declared table. This solves the limitation of an original method that have to use sane database management system and also overcomes the problem in certain circumstances where the EDI (Electronic Data Interchange) exchange does not work.