• Journal of Internet Computing and Services
• /
• v.14 no.6
• /
• pp.71-84
• /
• 2013

Log data, which record the multitude of information created when operating computer systems, are utilized in many processes, from carrying out computer system inspection and process optimization to providing customized user optimization. In this paper, we propose a MongoDB-based unstructured log processing system in a cloud environment for processing the massive amount of log data of banks. Most of the log data generated during banking operations come from handling a client's business. Therefore, in order to gather, store, categorize, and analyze the log data generated while processing the client's business, a separate log data processing system needs to be established. However, the realization of flexible storage expansion functions for processing a massive amount of unstructured log data and executing a considerable number of functions to categorize and analyze the stored unstructured log data is difficult in existing computer environments. Thus, in this study, we use cloud computing technology to realize a cloud-based log data processing system for processing unstructured log data that are difficult to process using the existing computing infrastructure's analysis tools and management system. The proposed system uses the IaaS (Infrastructure as a Service) cloud environment to provide a flexible expansion of computing resources and includes the ability to flexibly expand resources such as storage space and memory under conditions such as extended storage or rapid increase in log data. Moreover, to overcome the processing limits of the existing analysis tool when a real-time analysis of the aggregated unstructured log data is required, the proposed system includes a Hadoop-based analysis module for quick and reliable parallel-distributed processing of the massive amount of log data. Furthermore, because the HDFS (Hadoop Distributed File System) stores data by generating copies of the block units of the aggregated log data, the proposed system offers automatic restore functions for the system to continually operate after it recovers from a malfunction. Finally, by establishing a distributed database using the NoSQL-based Mongo DB, the proposed system provides methods of effectively processing unstructured log data. Relational databases such as the MySQL databases have complex schemas that are inappropriate for processing unstructured log data. Further, strict schemas like those of relational databases cannot expand nodes in the case wherein the stored data are distributed to various nodes when the amount of data rapidly increases. NoSQL does not provide the complex computations that relational databases may provide but can easily expand the database through node dispersion when the amount of data increases rapidly; it is a non-relational database with an appropriate structure for processing unstructured data. The data models of the NoSQL are usually classified as Key-Value, column-oriented, and document-oriented types. Of these, the representative document-oriented data model, MongoDB, which has a free schema structure, is used in the proposed system. MongoDB is introduced to the proposed system because it makes it easy to process unstructured log data through a flexible schema structure, facilitates flexible node expansion when the amount of data is rapidly increasing, and provides an Auto-Sharding function that automatically expands storage. The proposed system is composed of a log collector module, a log graph generator module, a MongoDB module, a Hadoop-based analysis module, and a MySQL module. When the log data generated over the entire client business process of each bank are sent to the cloud server, the log collector module collects and classifies data according to the type of log data and distributes it to the MongoDB module and the MySQL module. The log graph generator module generates the results of the log analysis of the MongoDB module, Hadoop-based analysis module, and the MySQL module per analysis time and type of the aggregated log data, and provides them to the user through a web interface. Log data that require a real-time log data analysis are stored in the MySQL module and provided real-time by the log graph generator module. The aggregated log data per unit time are stored in the MongoDB module and plotted in a graph according to the user's various analysis conditions. The aggregated log data in the MongoDB module are parallel-distributed and processed by the Hadoop-based analysis module. A comparative evaluation is carried out against a log data processing system that uses only MySQL for inserting log data and estimating query performance; this evaluation proves the proposed system's superiority. Moreover, an optimal chunk size is confirmed through the log data insert performance evaluation of MongoDB for various chunk sizes.

• KIPS Transactions on Software and Data Engineering
• /
• v.5 no.5
• /
• pp.221-230
• /
• 2016

With emerging the Internet of Things (IoT) paradigm, the sensor network and sensor platform technologies have been changed according to exploding amount of sensors. Sensor Registry System (SRS) as a sensor platform is a system that registers and manages sensor metadata for consistent semantic interpretation in heterogeneous sensor networks. However, the SRS is unsuitable for the IoT environment. Therefore, this paper proposes sensor registry data model to register and manager sensor information in the IoT environment. We analyze Semantic Sensor Network Ontology (SSNO) for improving the existed SRS, and design metamodel based on the analysis result. We also build tables in a relational database using the designed metamodel, then implement SRS as a web application. This paper applies the SSNO and sensor ontology examples with translating into the proposed model in order to verify the suitability of the proposed sensor registry data model. As the evaluation result, the proposed model shows abundant expression of semantics by comparison with existed models.

• The Journal of the Korea Contents Association
• /
• v.5 no.4
• /
• pp.11-19
• /
• 2005

With the development of IT technologies, IT environment is making great change over life whole and is displacing business and business achievement systems of industry at the fast speed. The system which is proposed in this paper, supports environments to share codes, execution files and objects between project teams to maximize productivity and efficiency in web environments. Through impact assessment which is achieved at correction and development of existing programming codes by maintenance requirements, it minimizes obstacle occurrence factors and supports tracing obstacle sources. Through conversion the correction and development histories of existing sources and executable files between project teams to XML documents suitable to share in web, it stores them to relational database. In this paper, through impact assessment and change management for the correction and development of applications in the existing system development environments, we support efficient IT developments.

• KIISE Transactions on Computing Practices
• /
• v.24 no.1
• /
• pp.24-32
• /
• 2018

This paper proposes a Big Data system for energy Big Data which is aggregated in real-time from industrial and public sources. The constructed Big Data system is based on Hadoop and the Spark framework is simultaneously applied on Big Data processing, which supports in-memory distributed computing. In the paper, we focus on Big Data, in the form of heat energy for district heating, and deal with methodologies for storing, managing, processing and analyzing aggregated Big Data in real-time while considering properties of energy input and output. At present, the Big Data influx is stored and managed in accordance with the designed relational database schema inside the system and the stored Big Data is processed and analyzed as to set objectives. The paper exemplifies a number of heat demand plants, concerned with district heating, as industrial sources of heat energy Big Data gathered in real-time as well as the proposed system.

• Journal of the Korean Institute of Intelligent Systems
• /
• v.17 no.6
• /
• pp.766-772
• /
• 2007

As XML is widely used, many researches on the XML storage and query processing have been done. But, previous works on path query processing have mainly focused on the storage and retrieval methods for a large XML document or XML documents had a same DTD. Those researches did not efficiently process partial match queries on the differently-structured document set. To resolve the problem, we suggested a new index structure using relational table. The method constructs the $B^+$-tree index using backward label paths instead of forward label paths used in previous researches for storing path information and allows for finding the label paths that match the partial match queries efficiently using it when process the queries.

• Journal of KIISE:Computer Systems and Theory
• /
• v.27 no.7
• /
• pp.639-649
• /
• 2000

In the relational database systems, the join operation is one of the most time-consuming query operations. Many parallel join algorithms have been developed to reduce the execution time. Multiple hash join algorithm using allocation tree is one of most efficient ones. However, it may have some delay on the processing each node of allocation tree, which is occurred in tuple-probing phase by the difference between one page reading time of outer relation and the processing time of already read one. In this paper, to solve the performance degrading problem by the delay, we develop a join algorithm using the concept of 'synchronization of page execution time' for multiple hash joins. We reduce the processing time of each nodes in the allocation tree and improve the total system performance. In addition, we analyze the performance by building the analytical cost model and verify the validity of it by various performance comparison with previous method.

• Journal of Korea Spatial Information System Society
• /
• v.11 no.1
• /
• pp.25-34
• /
• 2009

As the trend of the Web is changing toward 'Web 2.0', OpenAPIs, Web 2.0's core technology, are used in many web sites. In the past, services in websites are used in its own, but recently it is possible to use services in other websites by using OpenAPI. In u-GIS many vendors also can provide combined service by using OpenAPI. There are already lots of OpenAPIs and the numer of OpenAPI increases very fast. So it is difficult to find a service that we want to use, and also difficult to find services for mashup. In this paper, we developed retrieval methods for OpenAPIs and mashup capable services based on similarity. First we define the integrated service information model to cover various protocols of OpenAPI, then developed a retrieval methods based on it. By implementing system according these methods by using relational database and JSP, we prove that the system can provide an ranked result sets based on similarity, OpenAPI's integration retrieval results and mashup capable service retrieval results.

• Journal of the Korean Society of Safety
• /
• v.21 no.3 s.75
• /
• pp.16-21
• /
• 2006

This paper presents a knowledge-based electrical fire cause diagnosis system using the fuzzy reasoning. The cause diagnosis of electrical fires may be approached either by studying electric facilities or by investigating cause using precision instruments at the fire site. However, cause diagnosis methods for electrical fires haven't been systematized yet. The system focused on database(DB) construction and cause diagnosis can diagnose the causes of electrical fires easily and efficiently. The cause diagnosis system for the electrical fire was implemented with entity-relational DB systems using Access 2000, one of DB development tools. Visual Basic is used as a DB building tool. The inference to confirm fire causes is conducted on the knowledge-based by combined approach of a case-based and a rule-based reasoning. A case-based cause diagnosis is designed to match the newly occurred fire case with the past fire cases stored in a DB by a kind of pattern recognition. The rule-based cause diagnosis includes intelligent objects having fuzzy attributes and rules, and is used for handling knowledge about cause reasoning. A rule-based using a fuzzy reasoning has been adopted. To infer the results from fire signs, a fuzzy operation of Yager sum was adopted. The reasoning is conducted on the rule-based reasoning that a rule-based DB system built with many rules derived from the existing diagnosis methods and the expertise in fire investigation. The cause diagnosis system proposes the causes obtained from the diagnosis process and showed possibility of electrical fire causes.

• Journal of the Korea Institute of Information and Communication Engineering
• /
• v.13 no.8
• /
• pp.1637-1646
• /
• 2009

Recently many researchers have been studying various methods for data integration. Among the integration methods that the researchers have studied, there are a method using metadata repository, and Topic Map which identifies the relationships between the data. This study suggests Meta-Topic Map to create Topic Map about search keyword by applying metadata and Topic Map, and the XMDR as a way to connect Meta-Topic Map with metadata in the legacy system. Considering the semantic relationship of user's keyword in the legacy system, the Meta-Topic Map provides the Topic Map format and generates the Topic Map about user's keyword. The XMDR performs structural integration through solving the problem of heterogeneity among metadata in the legacy system. The suggested svides isproves the interoperability among existing Relational Database constructed in the legacy system and the search efficiency and is efficient in expanding the system.

• The KIPS Transactions:PartB
• /
• v.9B no.6
• /
• pp.845-852
• /
• 2002

Electronic document processing using EDl (Electronic Data Interchange) must exchange documents using VAN (Value Added Network). However. the use of exclusive software needs alteration of a new document and the use of VAN(Value Added Network) demands an exchange of document and high cost for maintenance. Due to these problems, the existing EDI (Electronic Data Interchange) is turning into Web-based EDI (Electronic Data Interchange). This paper suggests techniques that change EDI (Electronic Data Interchange) messages which exist in two relational databases into XML (extensible Markeup Language) using the JDBC bridge. Also this paper proposes a method that recovers schema using converted XML (extensible Markeup Language) file, and a method which inserts an original record into a declared table. This solves the limitation of an original method that have to use sane database management system and also overcomes the problem in certain circumstances where the EDI (Electronic Data Interchange) exchange does not work.