• Journal of Internet Computing and Services
• /
• v.14 no.6
• /
• pp.71-84
• /
• 2013

Log data, which record the multitude of information created when operating computer systems, are utilized in many processes, from carrying out computer system inspection and process optimization to providing customized user optimization. In this paper, we propose a MongoDB-based unstructured log processing system in a cloud environment for processing the massive amount of log data of banks. Most of the log data generated during banking operations come from handling a client's business. Therefore, in order to gather, store, categorize, and analyze the log data generated while processing the client's business, a separate log data processing system needs to be established. However, the realization of flexible storage expansion functions for processing a massive amount of unstructured log data and executing a considerable number of functions to categorize and analyze the stored unstructured log data is difficult in existing computer environments. Thus, in this study, we use cloud computing technology to realize a cloud-based log data processing system for processing unstructured log data that are difficult to process using the existing computing infrastructure's analysis tools and management system. The proposed system uses the IaaS (Infrastructure as a Service) cloud environment to provide a flexible expansion of computing resources and includes the ability to flexibly expand resources such as storage space and memory under conditions such as extended storage or rapid increase in log data. Moreover, to overcome the processing limits of the existing analysis tool when a real-time analysis of the aggregated unstructured log data is required, the proposed system includes a Hadoop-based analysis module for quick and reliable parallel-distributed processing of the massive amount of log data. Furthermore, because the HDFS (Hadoop Distributed File System) stores data by generating copies of the block units of the aggregated log data, the proposed system offers automatic restore functions for the system to continually operate after it recovers from a malfunction. Finally, by establishing a distributed database using the NoSQL-based Mongo DB, the proposed system provides methods of effectively processing unstructured log data. Relational databases such as the MySQL databases have complex schemas that are inappropriate for processing unstructured log data. Further, strict schemas like those of relational databases cannot expand nodes in the case wherein the stored data are distributed to various nodes when the amount of data rapidly increases. NoSQL does not provide the complex computations that relational databases may provide but can easily expand the database through node dispersion when the amount of data increases rapidly; it is a non-relational database with an appropriate structure for processing unstructured data. The data models of the NoSQL are usually classified as Key-Value, column-oriented, and document-oriented types. Of these, the representative document-oriented data model, MongoDB, which has a free schema structure, is used in the proposed system. MongoDB is introduced to the proposed system because it makes it easy to process unstructured log data through a flexible schema structure, facilitates flexible node expansion when the amount of data is rapidly increasing, and provides an Auto-Sharding function that automatically expands storage. The proposed system is composed of a log collector module, a log graph generator module, a MongoDB module, a Hadoop-based analysis module, and a MySQL module. When the log data generated over the entire client business process of each bank are sent to the cloud server, the log collector module collects and classifies data according to the type of log data and distributes it to the MongoDB module and the MySQL module. The log graph generator module generates the results of the log analysis of the MongoDB module, Hadoop-based analysis module, and the MySQL module per analysis time and type of the aggregated log data, and provides them to the user through a web interface. Log data that require a real-time log data analysis are stored in the MySQL module and provided real-time by the log graph generator module. The aggregated log data per unit time are stored in the MongoDB module and plotted in a graph according to the user's various analysis conditions. The aggregated log data in the MongoDB module are parallel-distributed and processed by the Hadoop-based analysis module. A comparative evaluation is carried out against a log data processing system that uses only MySQL for inserting log data and estimating query performance; this evaluation proves the proposed system's superiority. Moreover, an optimal chunk size is confirmed through the log data insert performance evaluation of MongoDB for various chunk sizes.

• Journal of Korean Society of Forest Science
• /
• v.16 no.1
• /
• pp.1-32
• /
• 1972

By means of the interspecific hybridization in the Sub-genus Diploxylon of the Genus Pinus, $F_1$ hybrids of Pinus rigida${\times}$elliottii, Pinus rigida${\times}$radiata, P. rigida${\times}$serotina and P. densiflora${\times}$thunbergii had been produced. And on the basis of the crossabilities of these hybrids the taxonomic affinities of these pines were examined. And the needle characteristics of these hybrid and the occurence of phenolic substances in these $F_1$ hybrid were also investigated to see the potential usefulness of these characteristics for the diagnosis of the taxonomic affinity. And, the growth performances of the $F_1$ hybrids have also been compared with those of parental species. In order to contribute to the establishment of the hybrid seed orchard the introgression phenomena between P. densiflora and P. thunbergii in the eastern coastal area have also been investigated along with the investigation of the heterozygosity of plus trees of P. densiflora growing in the clone bank in Suwon. And the results were summarized as follows. 1. On the basis of crossabilities as well as on the taxonomic affinities according to the systems of Shaw, Pilger and Duffield, it has been proven that the parental species of those hybrids are of close affinities and range of the fertile hybrid seed production rate was as high as 28-58% in the best hybrid combination (Table 13). 2. Among those hybrids, the ${\times}$ Pinus, rigiserotina hybrid seemed to be most promising in the growth performance exhibiting 109-155% more volume growth compared to the seed parent with the statistic significance of 1% level (Tables 16 and 17). 3. Notwithstanding the fact that the all of the pollen parents are cold tender, all hybrids exhibit cold hardiness as much as their seed parent and it seems to suggest that the characteristics of cold hardiness were transmitted from the seed parent. 4. Though a striking difference in needle length was observed between the parental species of each hybrid, it was difficult to distinguish each hybrid from their seed parent by the needle length except ${\times}$P. rigiserotina which is characterized by long needle which is 65% more longer than the needle of the seed parent (Table 21). 5. With regard to the anatomical characteristics of needle, the hypoderm is apparently thicker in most of the $F_1$ hybrid pines and the characteristics of resin canals are dominated by medial in most $F_1$ hybrid. And, the fibrovascular bundles were apart as were in their seed parent. Therefore it was found to be possible to distinguish the hybrids pines from their parents by the needle characteristics. And, it is to be noticed that the ${\times}$P. densithunbergii was more close to the pollen parent having RDI value of 0.73 (Fig.l, Table 22). 6. It has been demonstrated that ${\times}$P. rigielliottii, ${\times}$P. rigiradiata and ${\times}$P. rigitaeda have a phenolic substance (No.7) of light yellow at Rf-0.46, same as their seed parent, but no trace of phenolic substance was observed in their pollen parent. This fact will serve as an important criteria for early identification of hybridity in progeny testing. However, the fact that both of ${\times}$P. rigiserotina and ${\times}$P. densithunbergii exhibit the same reactions of phenolic substances as well their parental species seems to indicate the close affinities between the parental species of the respective hybrid (Fig.2, Table 23). 7. The separation and the reaction of phenolic substance developed on TLC were found to be same in the same species showing no variations between the individuals, and no variations due to tree part of sampling, tree age or pollen sources. And the reaction was also observed regardless of the not varied by the kind of developing solvent whether it is Aceton-Chloroform (3:7 v/v) or Benzene-Methanol-Acetic acid (90:16:8 v/v). 8. The introgression phenomena of natural Pinus densifiora stand in both east and west coastal area indicates that the major part of the red pines investigated are all heterozygous and the heterozygosity of pines are higher in the west coast than in the east coast(Tables 24 and 25). 9. Based on the RDI, among the plus trees of Pinus densiflora selected in Korea and Japan as well, no pure P. densiflora has been found. Since all of the sample trees of Pinus densiflora were found to be as heterozygous bearing part of the characteristics of P. thunbergii, those red pines were considered to be natural heterotic hybrid pines(Figs. 3 and 4. Tables 26 and 27).