• Title/Summary/Keyword: 비정형 환경

Search Result 247, Processing Time 0.026 seconds

An Effective Control Scheme for Unstructued Dataset in the Communication Environments (통신 환경에서 비정형적 구조를 갖는 데이터세트의 효과적인 제어 방법)

  • Bae, Myung-Nam;Choi, Wan;Lee, Dong-Chun
    • The KIPS Transactions:PartC
    • /
    • v.9C no.1
    • /
    • pp.31-38
    • /
    • 2002
  • Communication systems, such as Switching System, are operated in the restricted conditions that the suggested events must finish in the time-constraints. Therefore, the data in the systems requires not only rapid access time, but also completion in the restricted time. Many existing data systems have been developed and used in the communication environments. But, the system construct a structural scheme and provide users with basic data services only. In recent, as the complexity of data in the communication area is rapidly increasing, it requires the data system which can represent the unstructured dataset and complete the data access in this dataset on the restricted condition. In this paper, we propose the data model which is suitable to the unstructured multi-dataset environment. The data model supports the rapid data access for unstructured dataset and enables users to easily retrieve data needed at the execution. In addition to, we define the several algorithms to clarify the structure of our model.

Structural Performance Assessment of Buildings Considering Beam Discontinuity and Horizontal Irregularity under Wind and Earthquake Loads (보부재 불연속성과 수평비정형성을 고려한 건물의 풍하중과 지진하중에 의한 응답해석)

  • Chakraborty, Sudipta;Islam, Md. Rajibul;Kim, Dookie
    • Journal of the Korea institute for structural maintenance and inspection
    • /
    • v.26 no.5
    • /
    • pp.10-19
    • /
    • 2022
  • Irregularity in structural shape is a ubiquitous phenomenon. Structural hazards evoked from irregularity need to be checked against extreme lateral loadings. Structures containing four distinct types of irregularities in terms of continuity and discontinuity in upper half-length and all story levels along with O-shape are investigated. The structures were analyzed numerically and different seismic responses such as displacements, bending moment, axial forces, torsions, story drift, etc. were scrutinized. The seismic and wind load analysis was conducted for ACI 318-11 conditions. Results show that buildings having discontinuous beams on the upper half exhibit better resilience. It is also concluded that O-shaped building structures provide better resistance to overturning, making this shape relatively safe.

Design and Implementation of MongoDB-based Unstructured Log Processing System over Cloud Computing Environment (클라우드 환경에서 MongoDB 기반의 비정형 로그 처리 시스템 설계 및 구현)

  • Kim, Myoungjin;Han, Seungho;Cui, Yun;Lee, Hanku
    • Journal of Internet Computing and Services
    • /
    • v.14 no.6
    • /
    • pp.71-84
    • /
    • 2013
  • Log data, which record the multitude of information created when operating computer systems, are utilized in many processes, from carrying out computer system inspection and process optimization to providing customized user optimization. In this paper, we propose a MongoDB-based unstructured log processing system in a cloud environment for processing the massive amount of log data of banks. Most of the log data generated during banking operations come from handling a client's business. Therefore, in order to gather, store, categorize, and analyze the log data generated while processing the client's business, a separate log data processing system needs to be established. However, the realization of flexible storage expansion functions for processing a massive amount of unstructured log data and executing a considerable number of functions to categorize and analyze the stored unstructured log data is difficult in existing computer environments. Thus, in this study, we use cloud computing technology to realize a cloud-based log data processing system for processing unstructured log data that are difficult to process using the existing computing infrastructure's analysis tools and management system. The proposed system uses the IaaS (Infrastructure as a Service) cloud environment to provide a flexible expansion of computing resources and includes the ability to flexibly expand resources such as storage space and memory under conditions such as extended storage or rapid increase in log data. Moreover, to overcome the processing limits of the existing analysis tool when a real-time analysis of the aggregated unstructured log data is required, the proposed system includes a Hadoop-based analysis module for quick and reliable parallel-distributed processing of the massive amount of log data. Furthermore, because the HDFS (Hadoop Distributed File System) stores data by generating copies of the block units of the aggregated log data, the proposed system offers automatic restore functions for the system to continually operate after it recovers from a malfunction. Finally, by establishing a distributed database using the NoSQL-based Mongo DB, the proposed system provides methods of effectively processing unstructured log data. Relational databases such as the MySQL databases have complex schemas that are inappropriate for processing unstructured log data. Further, strict schemas like those of relational databases cannot expand nodes in the case wherein the stored data are distributed to various nodes when the amount of data rapidly increases. NoSQL does not provide the complex computations that relational databases may provide but can easily expand the database through node dispersion when the amount of data increases rapidly; it is a non-relational database with an appropriate structure for processing unstructured data. The data models of the NoSQL are usually classified as Key-Value, column-oriented, and document-oriented types. Of these, the representative document-oriented data model, MongoDB, which has a free schema structure, is used in the proposed system. MongoDB is introduced to the proposed system because it makes it easy to process unstructured log data through a flexible schema structure, facilitates flexible node expansion when the amount of data is rapidly increasing, and provides an Auto-Sharding function that automatically expands storage. The proposed system is composed of a log collector module, a log graph generator module, a MongoDB module, a Hadoop-based analysis module, and a MySQL module. When the log data generated over the entire client business process of each bank are sent to the cloud server, the log collector module collects and classifies data according to the type of log data and distributes it to the MongoDB module and the MySQL module. The log graph generator module generates the results of the log analysis of the MongoDB module, Hadoop-based analysis module, and the MySQL module per analysis time and type of the aggregated log data, and provides them to the user through a web interface. Log data that require a real-time log data analysis are stored in the MySQL module and provided real-time by the log graph generator module. The aggregated log data per unit time are stored in the MongoDB module and plotted in a graph according to the user's various analysis conditions. The aggregated log data in the MongoDB module are parallel-distributed and processed by the Hadoop-based analysis module. A comparative evaluation is carried out against a log data processing system that uses only MySQL for inserting log data and estimating query performance; this evaluation proves the proposed system's superiority. Moreover, an optimal chunk size is confirmed through the log data insert performance evaluation of MongoDB for various chunk sizes.

Development of integrated management solution through log analysis based on Big Data (빅데이터기반의 로그분석을 통한 통합 관리 솔루션 개발)

  • Kang, Sun-Kyoung;Lee, Hyun-Chang;Shin, Seong-Yoon
    • Proceedings of the Korean Institute of Information and Commucation Sciences Conference
    • /
    • 2017.10a
    • /
    • pp.541-542
    • /
    • 2017
  • In this paper, we intend to develop an integrated management solution that can be easily operated by integrating complex and various cloud environments. This has the advantage that users and administrators can conveniently solve problems by collecting and analyzing fixed log data and unstructured log data based on big data and realizing integrated monitoring in real time. Hypervisor log pattern analysis technology will be able to manage existing complex and various cloud environment more efficiently.

  • PDF

A Meta Analysis of Innovation Diffusion Theory based e-Commerce Environment in Korea (국내 전자상거래 환경에서 혁신확산이론 선행연구에 관한 메타분석)

  • Nam, Soo-Tai;Jin, Chan-Yong
    • Proceedings of the Korean Institute of Information and Commucation Sciences Conference
    • /
    • 2017.05a
    • /
    • pp.147-148
    • /
    • 2017
  • 빅데이터 분석은 데이터베이스에 잘 정리된 정형 데이터뿐 아니라 인터넷, 소셜 네트워크 서비스, 모바일 환경에서 생성되는 웹 문서, 이메일, 소셜 데이터 등 비정형 데이터를 효과적으로 분석하는 기술을 말한다. 메타분석은 여러 실증연구의 정량적인 결과를 통합과 분석을 통해 전체 결과를 조망할 수 있는 기회를 제공하는 통계적 통합 방법이다. 전자상거래 연구에서 혁신확산에 영향을 미치는 요인으로 상대적 이점, 적합성, 복잡성, 시험 가능성, 관찰 가능성, 편리성 그리고 커뮤니케이션 채널을 외부 요인으로 설정된 연구를 대상으로 하고자 한다. 다음으로 국내 주요 학회지에 게재된 혁신확산이론 관련연구에서 어떠한 요인들을 사용하고 있고 또한 이러한 외부요인들이 종속변수에 어느 정도의 설명력을 가지는지를 메타분석을 통해 알아보고자 한다. 이러한 연구모델을 바탕으로 학문적 실무적 의의를 논의하고자 한다.

  • PDF

Document Embedding for Entity Linking in Social Media (문서 임베딩을 이용한 소셜 미디어 문장의 개체 연결)

  • Park, Youngmin;Jeong, Soyun;Lee, Jeong-Eom;Shin, Dongsoo;Kim, Seona;Seo, Junyun
    • Annual Conference on Human and Language Technology
    • /
    • 2017.10a
    • /
    • pp.194-196
    • /
    • 2017
  • 기존의 단어 기반 접근법을 이용한 개체 연결은 단어의 변형, 신조어 등이 빈번하게 나타나는 비정형 문장에 대해서는 좋은 성능을 기대하기 어렵다. 본 논문에서는 문서 임베딩과 선형 변환을 이용하여 단어 기반 접근법의 단점을 해소하는 개체 연결을 제안한다. 문서 임베딩은 하나의 문서 전체를 벡터 공간에 표현하여 문서 간 의미적 유사도를 계산할 수 있다. 본 논문에서는 또한 비교적 정형 문장인 위키백과 문장과 비정형 문장인 소셜 미디어 문장 사이에 선형 변환을 수행하여 두 문형 사이의 표현 격차를 해소하였다. 제안하는 개체 연결 방법은 대표적인 소셜 미디어인 트위터 환경 문장에서 단어 기반 접근법과 비교하여 높은 성능 향상을 보였다.

  • PDF

NoSQL-based User Behavior Detection System in Cloud Computing Environment (NoSQL 기반 클라우드 사용자 행동 탐지 시스템 설계)

  • Ahn, Kwang-Min;Lee, Bong-Hwan
    • Proceedings of the Korean Institute of Information and Commucation Sciences Conference
    • /
    • 2012.10a
    • /
    • pp.804-807
    • /
    • 2012
  • Cloud service provider has to protect client's information securely since all the resources are offered by the service provider, and a large number of users share the resources. In this paper, a NoSQL-based anomaly detection system is proposed in order to enhance the security of mobile cloud services. The existing integrated security management system that uses a relational database can not be used for real-time processing of data since security log from a variety of security equipment and data from cloud node have different data format with unstructured features. The proposed system can resolve the emerging security problem because it provides real time processing and scalability in distributed processing environment.

  • PDF

A study on unstructured text mining algorithm through R programming based on data dictionary (Data Dictionary 기반의 R Programming을 통한 비정형 Text Mining Algorithm 연구)

  • Lee, Jong Hwa;Lee, Hyun-Kyu
    • Journal of Korea Society of Industrial Information Systems
    • /
    • v.20 no.2
    • /
    • pp.113-124
    • /
    • 2015
  • Unlike structured data which are gathered and saved in a predefined structure, unstructured text data which are mostly written in natural language have larger applications recently due to the emergence of web 2.0. Text mining is one of the most important big data analysis techniques that extracts meaningful information in the text because it has not only increased in the amount of text data but also human being's emotion is expressed directly. In this study, we used R program, an open source software for statistical analysis, and studied algorithm implementation to conduct analyses (such as Frequency Analysis, Cluster Analysis, Word Cloud, Social Network Analysis). Especially, to focus on our research scope, we used keyword extract method based on a Data Dictionary. By applying in real cases, we could find that R is very useful as a statistical analysis software working on variety of OS and with other languages interface.

A Mobile Agent System for Meaningful Information Filtering for XML Documents (비정형 XML 문서에서의 의미정보 검색을 위한 이동에이전트 시스템)

  • Kong, Yong-Hae;Choi, In-Seok;Lee, Kyeung-Soo
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2002.04b
    • /
    • pp.1021-1024
    • /
    • 2002
  • 본 연구에서는 인터넷에 존재하는 다양한 XML 문서에서 의미정보를 검색하기 위한 의미정보수집 이동에이전트 시스템을 개발하였다. 이동에이전트는 실행 프로그램이 이기종 분산 환경의 네트웍에서 자율적으로 이동 및 반응하며 실제로 데이터가 존재하고 있는 장소로 이동하여 목적을 수행한다. 의미 정보수집 이동에이전트 시스템의 연구를 위하여, 정보를 개념화하고 포괄적 DTD를 자동으로 생성할 수 있는 DTD생성기를 개발하였으며 의미정보를 추론할 수 있는 추론알고리즘을 연구하였다. 개발된 의미정보수집 이동에이전트 시스템은 정보가 존재하는 원격지 사이트에 파견되어 비정형 XML 문서를 대상으로 포괄적 DTD와 추론엔진을 이용하여, 의미정보를 추출하고 전송하는 임무를 수행한다. 따라서, 의미정보수집 이동에이전트 시스템을 이용한 정보수집은 정보의 질을 향상시키고 네트워크의 부하를 감소시킨다.

  • PDF