• Title/Summary/Keyword: post data processing

Search Result 554, Processing Time 0.029 seconds

A Study on the Use of Stopword Corpus for Cleansing Unstructured Text Data (비정형 텍스트 데이터 정제를 위한 불용어 코퍼스의 활용에 관한 연구)

  • Lee, Won-Jo
    • The Journal of the Convergence on Culture Technology
    • /
    • v.8 no.6
    • /
    • pp.891-897
    • /
    • 2022
  • In big data analysis, raw text data mostly exists in various unstructured data forms, so it becomes a structured data form that can be analyzed only after undergoing heuristic pre-processing and computer post-processing cleansing. Therefore, in this study, unnecessary elements are purified through pre-processing of the collected raw data in order to apply the wordcloud of R program, which is one of the text data analysis techniques, and stopwords are removed in the post-processing process. Then, a case study of wordcloud analysis was conducted, which calculates the frequency of occurrence of words and expresses words with high frequency as key issues. In this study, to improve the problems of the "nested stopword source code" method, which is the existing stopword processing method, using the word cloud technique of R, we propose the use of "general stopword corpus" and "user-defined stopword corpus" and conduct case analysis. The advantages and disadvantages of the proposed "unstructured data cleansing process model" are comparatively verified and presented, and the practical application of word cloud visualization analysis using the "proposed external corpus cleansing technique" is presented.

Developing A Pre-and Post-Procellor for Building Analysis (건축구조해석을 위한 선후처리 프로그램의 개발)

  • 이정재
    • Magazine of the Korean Society of Agricultural Engineers
    • /
    • v.36 no.2
    • /
    • pp.31-43
    • /
    • 1994
  • General concepts and overall procedures of interactive graphical user interface, a preand post- processor, for building analysis are introduced. Attention is forcused on the data structures and the modeling operators which can ensure the intergrity of its database should have. An example of model building process is presented to illustrate its capability, its facilities for modifying, and for processing.

  • PDF

Reinforcement Post-Processing and Feedback Algorithm for Optimal Combination in Bottom-Up Hierarchical Classification (상향식 계층분류의 최적화 된 병합을 위한 후처리분석과 피드백 알고리즘)

  • Choi, Yun-Jeong;Park, Seung-Soo
    • The KIPS Transactions:PartB
    • /
    • v.17B no.2
    • /
    • pp.139-148
    • /
    • 2010
  • This paper shows a reinforcement post-processing method and feedback algorithm for improvement of assigning method in classification. Especially, we focused on complex documents that are generally considered to be hard to classify. A basis factors in traditional classification system are training methodology, classification models and features of documents. The classification problem of the documents containing shared features and multiple meanings, should be deeply mined or analyzed than general formatted data. To address the problems of these document, we proposed a method to expand classification scheme using decision boundary detected automatically in our previous studies. The assigning method that a document simply decides to the top ranked category, is a main factor that we focus on. In this paper, we propose a post-processing method and feedback algorithm to analyze the relevance of ranked list. In experiments, we applied our post-processing method and one time feedback algorithm to complex documents. The experimental results show that our system does not need to change the classification algorithm itself to improve the accuracy and flexibility.

Pre- and Post Processing System on Prediction Analysis of Thermal Stress in Mass Concrete Structure (매스콘크리트의 온도균열 예측해석에서의 전후처리 시스템 개발에 관한 연구)

  • 김유석;강석화;박칠림
    • Proceedings of the Korea Concrete Institute Conference
    • /
    • 1996.04a
    • /
    • pp.270-274
    • /
    • 1996
  • Until recently pre & post-processing of finite element model has been heavily relied on expensive graphic peripheral devices. But today, with the aid of inexpensive microcomputers, very effective pre & postprocessor graphics has been developed. In this study, Pre & Post processor(MASSPRE, MASSPOST) of prediction analysis of thermal stress in mass concrete structure is developed. The developed pre & post processors are raise to the efficiency in making input data for the main program and analysis of the results produced by the main program. This MASSPOST presents a stress contour graph, volume slice, time-temperature history graph, time-stress history graph, etc.

  • PDF

Development of a CAE Middleware and a Visualization System for Supporting Interoperability of Continuous CAE Analysis Data (연속해석 데이터의 상호운용성을 지원하는 CAE 미들웨어와 가시화 시스템의 개발)

  • Song, In-Ho;Yang, Jeong-Sam;Jo, Hyun-Jei;Choi, Sang-Su
    • Korean Journal of Computational Design and Engineering
    • /
    • v.15 no.2
    • /
    • pp.85-93
    • /
    • 2010
  • This paper proposes a CAE data translation and visualization technique that can verify time-varying continuous analysis simulation in a virtual reality (VR) environment. In previous research, the use of CAE analysis data has been problematic because of the lack of any interactive simulation controls for visualizing continuous simulation data. Moreover, the research on post-processing methods for real-time verification of CAE analysis data has not been sufficient. We therefore propose a scene graph based visualization method and a post-processing method for supporting interoperability of continuous CAE analysis data. These methods can continuously visualize static analysis data independently of any timeline; it can also continuously visualize dynamic analysis data that varies in relation to the timeline. The visualization system for continuous simulation data, which includes a CAE middleware that interfaces with various formats of CAE analysis data as well as functions for visualizing continuous simulation data and operational functions, enables users to verify simulation results with more realistic scenes. We also use the system to do a performance evaluation with regard to the visualization of continuous simulation data.

Improvement of WRF forecast meteorological data by Model Output Statistics using linear, polynomial and scaling regression methods

  • Jabbari, Aida;Bae, Deg-Hyo
    • Proceedings of the Korea Water Resources Association Conference
    • /
    • 2019.05a
    • /
    • pp.147-147
    • /
    • 2019
  • The Numerical Weather Prediction (NWP) models determine the future state of the weather by forcing current weather conditions into the atmospheric models. The NWP models approximate mathematically the physical dynamics by nonlinear differential equations; however these approximations include uncertainties. The errors of the NWP estimations can be related to the initial and boundary conditions and model parameterization. Development in the meteorological forecast models did not solve the issues related to the inevitable biases. In spite of the efforts to incorporate all sources of uncertainty into the forecast, and regardless of the methodologies applied to generate the forecast ensembles, they are still subject to errors and systematic biases. The statistical post-processing increases the accuracy of the forecast data by decreasing the errors. Error prediction of the NWP models which is updating the NWP model outputs or model output statistics is one of the ways to improve the model forecast. The regression methods (including linear, polynomial and scaling regression) are applied to the present study to improve the real time forecast skill. Such post-processing consists of two main steps. Firstly, regression is built between forecast and measurement, available during a certain training period, and secondly, the regression is applied to new forecasts. In this study, the WRF real-time forecast data, in comparison with the observed data, had systematic biases; the errors related to the NWP model forecasts were reflected in the underestimation of the meteorological data forecast by the WRF model. The promising results will indicate that the post-processing techniques applied in this study improved the meteorological forecast data provided by WRF model. A comparison between various bias correction methods will show the strength and weakness of the each methods.

  • PDF

Development of PC Based Signal Postprocessing System in MR Spectroscopy: Normal Brain Spectrum in 1.5T MR Spectroscopy (PC를 이용한 자기공명분광 신호처리분석 시스템 개발: 1.5T MR Spectroscopy에서의 정상인 뇌 분광 신호)

  • 백문영;강원석;이현용;신운재;은충기
    • Investigative Magnetic Resonance Imaging
    • /
    • v.4 no.2
    • /
    • pp.128-135
    • /
    • 2000
  • Purpose : The aim of this study is to develope the Magnetic Resonance Spectroscopy(MRS) data processing S/W which plays an important role as a diagnostic tool in clinical field. Materials and methods : Post-processing software of MRS based on graphical user interface(GUI) under windows operating system of personal computer(PC) was developed using MATLAB(Mathwork, U.S.A.). This tool contains many functions to increase the quality of spectrum data such as DC correction, zero filling, line broadening, Gauss-Lorentzian filtering, phase correction, etc. And we obtained the normal human brain $^1H$ MRS data from parietal white matter, basal ganglia and occipital grey matter region using 1.5T Gyroscan ACS-NT R6 (philips, Amsterdam, Netherland) MRS package. The analysis of the MRS peaks were performed by obtaining the ratio of peak area. Results : The peak ratios of NAA/Cr, Cho/Cr, MI/Cr for the different MRS machines have a little different values. But these peak ratios were not significantly different between different echo time MRS peak ratios in the same machine (p<0.05). Conclusion : MRS post-processing S/W based on GUI using PC was developed and applied to the analysis of normal human brain $^1H$ MRS. This independent MRS processing job increases the performance and throughput of patient scan of main console. Finally, we suggest that the database for normal in-yivo human MRS data should be obtained before clinical applications.

  • PDF

Post-processing of Direct Teaching Trajectory in Industrial Robots

  • Choi, Tae-Yong;Park, Chan-Hun;Do, Hyun-Min;Chung, Kwang-Cho;Park, Dong-Il;Kyung, Jin-Ho
    • International Journal of Fuzzy Logic and Intelligent Systems
    • /
    • v.12 no.3
    • /
    • pp.256-262
    • /
    • 2012
  • Direct teaching of the industrial robot is a novel technique to easily teach manipulators. However, teaching data by human hand cannot help having large noise errors ranging from low to high frequency. To use teaching data, post-processing to correct the teaching trajectory is required. Here, a novel shape-based trajectory correction method to rebuild teaching data with the feature information of curvature and velocity is proposed. The proposed method is tested on square and circular objects.

HIGH-SPEED SOFTWARE FRAME SYNCHRONIZER USING SSE2 TECHNOLOGY

  • Koo, In-Hoi;Ahn, Sang-Il;Kim, Tae-Hoon;Sakong, Young-Ho
    • Proceedings of the KSRS Conference
    • /
    • 2007.10a
    • /
    • pp.522-525
    • /
    • 2007
  • Frame Synchronization is applied to not only digital data transmission for data synchronization between transmitter and receiver but also data communication with satellite. When satellite image data with high resolution and mass storage is transmitted, hardware frame synchronizer for real-time processing or software frame synchronizer for post-processing is used. In case of hardware, processing with high speed is available but data loss may happen for Search of Frame Synchronization. In case of software, data loss does not happen but speed is relatively slow. In this paper, Pending Buffer concept was proposed to cope with data loss according to processing status of Frame Synchronization. Algorithm to process Frame synchronization with high speed using bit threshold search algorithm with pattern search technique and SIMD is also proposed.

  • PDF

Impact of Administrative Post and Gender on Lecturers' Research Motivation in Vietnam

  • TRAN, Thi Kim Nhung;NGUYEN, Thanh Do;PHAM, Thanh Van;NGUYEN, Thi Lien Huong;GIANG, Minh Duc;HA, Ngoc Thang
    • The Journal of Asian Finance, Economics and Business
    • /
    • v.8 no.3
    • /
    • pp.705-715
    • /
    • 2021
  • This study examines the simultaneous impact of gender and administrative post on the research motivation of university lecturers. There are 475 selected questionnaires that are qualified for data processing and the collected data are analyzed by two-way analysis of variance (ANOVA). The results show the relationship between administration involvement and scientific research of lecturers. In general, lecturers with administrative post have higher research motivation than lecturers without administrative post. More specifically, the level of administrative post has positive correlation with research motivation of female lecturers, but has negative correlation of male lecturers. It means that a female lecturer with higher administrative post also has more motivation to conduct research. Conversely, the higher administrative post a male lecturer has, the less research motivation he is. The cause of this difference is the pressure of earning money on Asian men, including Vietnamese men, who are considered to be main breadwinners of their family. When having an administration post, the more chances male lecturers have to earn money, the less time they can spend for scientific research. This suggests that the solution for managers is to perform annual assessment, fulfill promotion process, build up motivation and reward system, and establish recruitment and promotion policies.