• Title/Summary/Keyword: unstructured data

Search Result 723, Processing Time 0.027 seconds

Development of an Unstructured Parallel Overset Mesh Technique for Unsteady Flow Simulations around bodies with Relative Motion (상대운동이 있는 물체주위의 비정상 유동해석을 위한 병렬화된 비정렬 중첩격자기법 개발)

  • Jung, Mun-Seung;Kwon, Oh-Joon
    • Journal of the Korean Society for Aeronautical & Space Sciences
    • /
    • v.33 no.2
    • /
    • pp.1-10
    • /
    • 2005
  • An unstructured parallel overset mesh method has been developed for the simulation of unsteady flows around multiple bodies in relative motion. For this purpose, an efficient and robust search method is proposed for the unstructured grid system. A new data-structure is also proposed to handle the variable number of data on parallel sub-domain boundary. The interpolation boundary is defined for data communication between grid systems. An interpolation method to retain second-order spatial accuracy and to treat the points inside the neighboring solid bodies are also suggested. A single store separating from the Eglin/Pylon configuration is calculated and the result is compared with experimental data for validation. Simulation of unsteady flows around multiple bodies in relative motion is also performed.

A Study on Information Linkage Service for Disaster Situation Management : Focusing on Earthquake (재난 상황관리를 위한 재난안전정보 연계 서비스 방안 연구 : 지진을 중심으로)

  • Yu, Eun-Ji;Shim, Hyoung Seop
    • Journal of Digital Contents Society
    • /
    • v.19 no.1
    • /
    • pp.67-73
    • /
    • 2018
  • Researchers have increased their interest in effectively managing the disaster that appear in large scale and complex form. There are two types of disaster information, which are unstructured text data and structured data. Unstructured text data usually refers to text documents that have been referenced by disaster management personnel such as disaster manuals and related regulations, while structured data refers to various disaster information build in the disaster related organization system. This paper proposes a methodology of constructing a disaster information sharing system that enables joint use of disaster related organizations through the establishment of a mutual linkage system by utilizing both unstructured and structured form of disaster information. Especially, Based on the linkage information between structured earthquake information in earthquake related system and earthquake manuals and countermeasures against earthquake disaster, we propose a service that provides the necessary information for earthquake management. It is expected that the task manager will perform effective earthquake state management by acquiring the integrated structured and unstructured earthquake information of the ministries and related organizations.

Fat Client-Based Abstraction Model of Unstructured Data for Context-Aware Service in Edge Computing Environment (에지 컴퓨팅 환경에서의 상황인지 서비스를 위한 팻 클라이언트 기반 비정형 데이터 추상화 방법)

  • Kim, Do Hyung;Mun, Jong Hyeok;Park, Yoo Sang;Choi, Jong Sun;Choi, Jae Young
    • KIPS Transactions on Computer and Communication Systems
    • /
    • v.10 no.3
    • /
    • pp.59-70
    • /
    • 2021
  • With the recent advancements in the Internet of Things, context-aware system that provides customized services become important to consider. The existing context-aware systems analyze data generated around the user and abstract the context information that expresses the state of situations. However, these datasets is mostly unstructured and have difficulty in processing with simple approaches. Therefore, providing context-aware services using the datasets should be managed in simplified method. One of examples that should be considered as the unstructured datasets is a deep learning application. Processes in deep learning applications have a strong coupling in a way of abstracting dataset from the acquisition to analysis phases, it has less flexible when the target analysis model or applications are modified in functional scalability. Therefore, an abstraction model that separates the phases and process the unstructured dataset for analysis is proposed. The proposed abstraction utilizes a description name Analysis Model Description Language(AMDL) to deploy the analysis phases by each fat client is a specifically designed instance for resource-oriented tasks in edge computing environments how to handle different analysis applications and its factors using the AMDL and Fat client profiles. The experiment shows functional scalability through examples of AMDL and Fat client profiles targeting a vehicle image recognition model for vehicle access control notification service, and conducts process-by-process monitoring for collection-preprocessing-analysis of unstructured data.

A Study on the Characteristics of Formal Expression of Atypical Buildings (비정형 건축물의 형태 표현특징에 관한 연구)

  • Jiang, Bo;Hong, Kwan-Seon
    • The Journal of the Korea Contents Association
    • /
    • v.21 no.4
    • /
    • pp.795-814
    • /
    • 2021
  • With the development of science and technology in recent years, various types of unstructured buildings have begun to be implemented by combining traditional architectural styles with digital tools, which are significantly different from conventional formal buildings. Designers will use various methods or digital tools to complete the unstructured and freer architectural forms when building an unstructured building. Based on this background, the need for research on the criteria for evaluating the characteristics of unstructured architectural forms is raised. First, the text was reconstructed by considering and integrating the elements of the unstructured exterior form based on prior research, with the external form of the unstructured building as the main research object. Second, the purpose of this study was to classify various types of unstructured forms and to provide important basic data for designing digital processes of unstructured architectural forms.Third, from 2000 to 2020, the text focused on the study of unstructured buildings and conducted in-depth analysis of the characteristics of their form expressions. While providing a case basis for the study of related fields, distribution laws and presence values related to the characteristics of unstructured buildings were sought. In addition, the analysis was conducted in combination with the distribution of functional use of buildings, and this study is differentiated from the existing research in that the atypical form is applied to the buildings by use, and the application trend of this type is understood to enhance understanding of the atypical form of buildings.

Design and Implementation of MongoDB-based Unstructured Log Processing System over Cloud Computing Environment (클라우드 환경에서 MongoDB 기반의 비정형 로그 처리 시스템 설계 및 구현)

  • Kim, Myoungjin;Han, Seungho;Cui, Yun;Lee, Hanku
    • Journal of Internet Computing and Services
    • /
    • v.14 no.6
    • /
    • pp.71-84
    • /
    • 2013
  • Log data, which record the multitude of information created when operating computer systems, are utilized in many processes, from carrying out computer system inspection and process optimization to providing customized user optimization. In this paper, we propose a MongoDB-based unstructured log processing system in a cloud environment for processing the massive amount of log data of banks. Most of the log data generated during banking operations come from handling a client's business. Therefore, in order to gather, store, categorize, and analyze the log data generated while processing the client's business, a separate log data processing system needs to be established. However, the realization of flexible storage expansion functions for processing a massive amount of unstructured log data and executing a considerable number of functions to categorize and analyze the stored unstructured log data is difficult in existing computer environments. Thus, in this study, we use cloud computing technology to realize a cloud-based log data processing system for processing unstructured log data that are difficult to process using the existing computing infrastructure's analysis tools and management system. The proposed system uses the IaaS (Infrastructure as a Service) cloud environment to provide a flexible expansion of computing resources and includes the ability to flexibly expand resources such as storage space and memory under conditions such as extended storage or rapid increase in log data. Moreover, to overcome the processing limits of the existing analysis tool when a real-time analysis of the aggregated unstructured log data is required, the proposed system includes a Hadoop-based analysis module for quick and reliable parallel-distributed processing of the massive amount of log data. Furthermore, because the HDFS (Hadoop Distributed File System) stores data by generating copies of the block units of the aggregated log data, the proposed system offers automatic restore functions for the system to continually operate after it recovers from a malfunction. Finally, by establishing a distributed database using the NoSQL-based Mongo DB, the proposed system provides methods of effectively processing unstructured log data. Relational databases such as the MySQL databases have complex schemas that are inappropriate for processing unstructured log data. Further, strict schemas like those of relational databases cannot expand nodes in the case wherein the stored data are distributed to various nodes when the amount of data rapidly increases. NoSQL does not provide the complex computations that relational databases may provide but can easily expand the database through node dispersion when the amount of data increases rapidly; it is a non-relational database with an appropriate structure for processing unstructured data. The data models of the NoSQL are usually classified as Key-Value, column-oriented, and document-oriented types. Of these, the representative document-oriented data model, MongoDB, which has a free schema structure, is used in the proposed system. MongoDB is introduced to the proposed system because it makes it easy to process unstructured log data through a flexible schema structure, facilitates flexible node expansion when the amount of data is rapidly increasing, and provides an Auto-Sharding function that automatically expands storage. The proposed system is composed of a log collector module, a log graph generator module, a MongoDB module, a Hadoop-based analysis module, and a MySQL module. When the log data generated over the entire client business process of each bank are sent to the cloud server, the log collector module collects and classifies data according to the type of log data and distributes it to the MongoDB module and the MySQL module. The log graph generator module generates the results of the log analysis of the MongoDB module, Hadoop-based analysis module, and the MySQL module per analysis time and type of the aggregated log data, and provides them to the user through a web interface. Log data that require a real-time log data analysis are stored in the MySQL module and provided real-time by the log graph generator module. The aggregated log data per unit time are stored in the MongoDB module and plotted in a graph according to the user's various analysis conditions. The aggregated log data in the MongoDB module are parallel-distributed and processed by the Hadoop-based analysis module. A comparative evaluation is carried out against a log data processing system that uses only MySQL for inserting log data and estimating query performance; this evaluation proves the proposed system's superiority. Moreover, an optimal chunk size is confirmed through the log data insert performance evaluation of MongoDB for various chunk sizes.

Conflict Analysis in Construction Project with Unstructured Data: A Case Study of Jeju Naval Base Project in South Korea

  • Baek, Seungwon;Han, Seung Heon;Lee, Changjun;Jang, Woosik;Ock, Jong Ho
    • International conference on construction engineering and project management
    • /
    • 2017.10a
    • /
    • pp.291-296
    • /
    • 2017
  • Infrastructure development as national project suffers from social conflict which is one of main risk to be managed. Social conflicts have a negative impact on not only the social integration but also the national economy as they require enormous social costs to be solved. Against this backdrop, this study analyzes social conflict using articles published by online news media based on web-crawling and natural language processing (NLP) techniques. As an illustrative case, the Jeju Naval Base (JNB) project which is one of representative conflict case in South Korea is analyzed. Total of 21,788 articles and representative keywords are identified annually. Additionally, comparative analysis is conducted between the extracted keywords and actual events occurred during the project. The authors explain actual events in the JNB project based on the extracted words by the year. This study contributes to analyze social conflict and to extract meaningful information from unstructured data.

  • PDF

News Article Based Industry Risk Index Prediction for Industry-Specific Evaluation

  • Kyungwon Kim;Kyoungro Yoon
    • Journal of Web Engineering
    • /
    • v.20 no.3
    • /
    • pp.795-816
    • /
    • 2021
  • The existing industry evaluation method utilizes the method of collecting the structured information such as the financial information of the companies included in the relevant industry and deriving the industrial evaluation index through the statistical analysis model. This method takes a long time to calculate the structured data and cause the time delay problem. In this paper, to solve this time delay problem, we derive monthly industry-specific interest and likability as a time series data type, which is a new industry evaluation indicator based on unstructured data. In addition, we propose a method to predict the industrial risk index, which is used as an important factor in industrial evaluation, based on derived industry-specific interest and likability time series data.

A Study on Improvement of Pension Operation and Management using Big Data Analysis Techniques (빅데이터 분석기법을 활용한 숙박업체 운영 개선 방안에 대한 연구)

  • Yoon, Sunhee
    • The Journal of the Convergence on Culture Technology
    • /
    • v.7 no.4
    • /
    • pp.815-821
    • /
    • 2021
  • The advantage of big data is to collect a large amount of data on the Internet and refine and use valuable data. That is, the unstructured data is processed so that the user can analyze and utilize it from a necessary point of view. This paper is a relatively small project and is based on unstructured data that can be closely applied to real life and used for marketing. The subjects of the experiment were modeled on lodging companies in the Seoul metropolitan area an hour away from Seoul, and analyzed for the increase in lodging rates before and after marketing using big data. As an experiment that shows the effects of increasing sales, reducing costs, and increasing returns by users, we propose a system to determine and filter whether data input in the process of analyzing big data such as social networks can be used as accommodation-related information.

Big Data Platform for Public Library Users: Focusing on the Cultural Programs and Community Service (이용자를 위한 공공도서관 빅데이터 플랫폼 구축 방안 연구 - 문화프로그램 및 커뮤니티 서비스 정보를 중심으로 -)

  • Yoon, SoYoung
    • Journal of the Korean BIBLIA Society for library and Information Science
    • /
    • v.33 no.3
    • /
    • pp.347-370
    • /
    • 2022
  • Most public library websites provide unstructured cultural program data, which cannot be produced and utilized systematically as bibliographic information. It is not sufficiently used in existing library big data research or cases, and there is a risk of disappearing when the website is reorganized or the person in charge is changed. This study developed a data schema that can be used in conjunction with bibliographic data by collecting and analyzing cultural programs and community service data produced in an unstructured manner and proposed to share and utilize public library cultural programs and community service data, and establish a library big data platform that can serve as an information channel between librarians who are cultural program planners. Library program data posted on the library website can be integrated and managed through the platform, securing continuity of work, and systematically managing and preserving the specialized service history of individual libraries.

A Study on the Method for Extracting the Purpose-Specific Customized Information from Online Product Reviews based on Text Mining (텍스트 마이닝 기반의 온라인 상품 리뷰 추출을 통한 목적별 맞춤화 정보 도출 방법론 연구)

  • Kim, Joo Young;Kim, Dong soo
    • The Journal of Society for e-Business Studies
    • /
    • v.21 no.2
    • /
    • pp.151-161
    • /
    • 2016
  • In the era of the Web 2.0, characterized by the openness, sharing and participation, it is easy for internet users to produce and share the data. The amount of the unstructured data which occupies most of the digital world's data has increased exponentially. One of the kinds of the unstructured data called personal online product reviews is necessary for both the company that produces those products and the potential customers who are interested in those products. In order to extract useful information from lots of scattered review data, the process of collecting data, storing, preprocessing, analyzing, and drawing a conclusion is needed. Therefore we introduce the text-mining methodology for applying the natural language process technology to the text format data like product review in order to carry out extracting structured data by using R programming. Also, we introduce the data-mining to derive the purpose-specific customized information from the structured review information drawn by the text-mining.