• Title/Summary/Keyword: data partitioning

Search Result 387, Processing Time 0.027 seconds

Performance Analysis on Declustering High-Dimensional Data by GRID Partitioning (그리드 분할에 의한 다차원 데이터 디클러스터링 성능 분석)

  • Kim, Hak-Cheol;Kim, Tae-Wan;Li, Ki-Joune
    • The KIPS Transactions:PartD
    • /
    • v.11D no.5
    • /
    • pp.1011-1020
    • /
    • 2004
  • A lot of work has been done to improve the I/O performance of such a system that store and manage a massive amount of data by distributing them across multiple disks and access them in parallel. Most of the previous work has focused on an efficient mapping from a grid ceil, which is determined bY the interval number of each dimension, to a disk number on the assumption that each dimension is split into disjoint intervals such that entire data space is GRID-like partitioned. However, they have ignored the effects of a GRID partitioning scheme on declustering performance. In this paper, we enhance the performance of mapping function based declustering algorithms by applying a good GRID par-titioning method. For this, we propose an estimation model to count the number of grid cells intersected by a range query and apply a GRID partitioning scheme which minimizes query result size among the possible schemes. While it is common to do binary partition for high-dimensional data, we choose less number of dimensions than needed for binary partition and split several times along that dimensions so that we can reduce the number of grid cells touched by a query. Several experimental results show that the proposed estimation model gives accuracy within 0.5% error ratio regardless of query size and dimension. We can also improve the performance of declustering algorithm based on mapping function, called Kronecker Sequence, which has been known to be the best among the mapping functions for high-dimensional data, up to 23 times by applying an efficient GRID partitioning scheme.

Stochastic Simple Hydrologic Partitioning Model Associated with Markov Chain Monte Carlo and Ensemble Kalman Filter (마코프 체인 몬테카를로 및 앙상블 칼만필터와 연계된 추계학적 단순 수문분할모형)

  • Choi, Jeonghyeon;Lee, Okjeong;Won, Jeongeun;Kim, Sangdan
    • Journal of Korean Society on Water Environment
    • /
    • v.36 no.5
    • /
    • pp.353-363
    • /
    • 2020
  • Hydrologic models can be classified into two types: those for understanding physical processes and those for predicting hydrologic quantities. This study deals with how to use the model to predict today's stream flow based on the system's knowledge of yesterday's state and the model parameters. In this regard, for the model to generate accurate predictions, the uncertainty of the parameters and appropriate estimates of the state variables are required. In this study, a relatively simple hydrologic partitioning model is proposed that can explicitly implement the hydrologic partitioning process, and the posterior distribution of the parameters of the proposed model is estimated using the Markov chain Monte Carlo approach. Further, the application method of the ensemble Kalman filter is proposed for updating the normalized soil moisture, which is the state variable of the model, by linking the information on the posterior distribution of the parameters and by assimilating the observed steam flow data. The stochastically and recursively estimated stream flows using the data assimilation technique revealed better representation of the observed data than the stream flows predicted using the deterministic model. Therefore, the ensemble Kalman filter in conjunction with the Markov chain Monte Carlo approach could be a reliable and effective method for forecasting daily stream flow, and it could also be a suitable method for routinely updating and monitoring the watershed-averaged soil moisture.

Study on Program Partitioning and Data Protection in Computation Offloading (코드 오프로딩 환경에서 프로그램 분할과 데이터 보호에 대한 연구)

  • Lee, Eunyoung;Pak, Suehee
    • KIPS Transactions on Software and Data Engineering
    • /
    • v.9 no.11
    • /
    • pp.377-386
    • /
    • 2020
  • Mobile cloud computing involves mobile or embedded devices as clients, and features small devices with constrained resource and low availability. Due to the fast expansion of smart phones and smart peripheral devices, researches on mobile cloud computing attract academia's interest more than ever. Computation offloading, or code offloading, enhances the performance of computation by migrating a part of computation of a mobile system to nearby cloud servers with more computational resources through wired or wireless networks. Code offloading is considered as one of the best approaches overcoming the limited resources of mobile systems. In this paper, we analyze the factors and the performance of code offloading, especially focusing on static program partitioning and data protection. We survey state-of-the-art researches on analyzed topics. We also describe directions for future research.

Plastic Displacement Estimates in Creep Crack Growth Testing (크리프 균열 성장 실험을 위한 소성 변위 결정법)

  • Huh Nam-Su;Yoon Kee-Bong;Kim Yun-Jae
    • Transactions of the Korean Society of Mechanical Engineers A
    • /
    • v.30 no.10 s.253
    • /
    • pp.1219-1226
    • /
    • 2006
  • The ASTM test standard recommends the use of the compact tension specimen for creep crack growth rates measurement. In the creep crack growth rate test, the displacement rate due to creep is obtained by subtracting the contribution of elastic and plastic components from the total load line displacement rate based on displacement partitioning method fur determining $C^*-integral$, which involves Ramberg-Osgood (R-O) fitting procedures. This paper investigates the effect of the R-O fitting procedures on plastic displacement rate estimates in creep crack growth testing, via detailed two-dimensional and three-dimensional finite element analyses of the standard compact tension specimen. Four different R-O fitting procedures are considered; (i) fitting the entire true stress-strain data up to the ultimate tensile strength, (ii) fitting the true stress-strain data from 0.1% strain to 0.8 of the true ultimate strain, (iii) fitting the true stress-strain data only up to 5% strain, and (iv) fitting the engineering stress-strain data. It is found that the last two procedures provide reasonably accurate plastic displacement rates and thus should be recommended in creep crack growth testing. Moreover, several advantages of fitting the engineering stress-strain data over fitting the true stress-strain data only up to 5% strain are discussed.

Spatio-Temporal Analysis of Trajectory for Pedestrian Activity Recognition

  • Kim, Young-Nam;Park, Jin-Hee;Kim, Moon-Hyun
    • Journal of Electrical Engineering and Technology
    • /
    • v.13 no.2
    • /
    • pp.961-968
    • /
    • 2018
  • Recently, researches on automatic recognition of human activities have been actively carried out with the emergence of various intelligent systems. Since a large amount of visual data can be secured through Closed Circuit Television, it is required to recognize human behavior in a dynamic situation rather than a static situation. In this paper, we propose new intelligent human activity recognition model using the trajectory information extracted from the video sequence. The proposed model consists of three steps: segmentation and partitioning of trajectory step, feature extraction step, and behavioral learning step. First, the entire trajectory is fuzzy partitioned according to the motion characteristics, and then temporal features and spatial features are extracted. Using the extracted features, four pedestrian behaviors were modeled by decision tree learning algorithm and performance evaluation was performed. The experiments in this paper were conducted using Caviar data sets. Experimental results show that trajectory provides good activity recognition accuracy by extracting instantaneous property and distinctive regional property.

Tree-structured Clustering for Continuous Data (연속형 자료에 대한 나무형 군집화)

  • Huh Myung-Hoe;Yang Kyung-Sook
    • The Korean Journal of Applied Statistics
    • /
    • v.18 no.3
    • /
    • pp.661-671
    • /
    • 2005
  • The aim of this study is to propose a clustering method, called tree-structured clustering, by recursively partitioning continuous multivariate dat a based on overall $R^2$ criterion with a practical node-splitting decision rule. The clustering method produces easily interpretable clustering rules of tree types with the variable selection function. In numerical examples (Fisher's iris data and a Telecom case), we note several differences between tree-structured clustering and K-means clustering.

The design of Stream producer for MPEG-4 encoder (MPEG-­4 부호화기를 위한 스트림 생성기 설계)

  • 송인근;서기범
    • Journal of the Korea Institute of Information and Communication Engineering
    • /
    • v.7 no.8
    • /
    • pp.1776-1784
    • /
    • 2003
  • In this paper, we propose the architecture of stream producer for MPEG­4 Video encoding. This module receives the quantized coefficient from DCT and Quantization module in macroblock unit and performs the VLC coding according to the encoding mode, and supports the error concealment mode of MPEG­4 and data partitioning mode. Using the VHDL, we designed the module using this architecture and performed the evaluations of this module by performing the post­-simulation.

Data Sampling-based Angular Space Partitioning for Parallel Skyline Query Processing (데이터 샘플링을 통한 각 기반 공간 분할 병렬 스카이라인 질의처리 기법)

  • Chung, Jaehwa
    • The Journal of Korean Association of Computer Education
    • /
    • v.18 no.5
    • /
    • pp.63-70
    • /
    • 2015
  • In the environment that the complex conditions need to be satisfied, skyline query have been applied to various field. To processing a skyline query in centralized scheme, several techniques have been suggested and recently map/reduce platform based approaches has been proposed which divides data space into multiple partitions for the vast volume of multidimensional data. However, the performances of these approaches are fluctuated due to the uneven data loading between servers and redundant tasks. Motivated by these issues, this paper suggests a novel technique called MR-DEAP which solves the uneven data loading using the random sampling. The experimental result gains the proposed MR-DEAP outperforms MR-Angular and MR-BNL scheme.

Effects of Hydro-Climate Conditions on Calibrating Conceptual Hydrologic Partitioning Model (개념적 수문분할모형의 보정에 미치는 수문기후학적 조건의 영향)

  • Choi, Jeonghyeon;Seo, Jiyu;Won, Jeongeun;Lee, Okjeong;Kim, Sangdan
    • Journal of Korean Society on Water Environment
    • /
    • v.36 no.6
    • /
    • pp.568-580
    • /
    • 2020
  • Calibrating a conceptual hydrologic model necessitates selection of a calibration period that produces the most reliable prediction. This often must be chosen randomly, however, since there is no objective guidance. Observation plays the most important role in the calibration or uncertainty evaluation of hydrologic models, in which the key factors are the length of the data and the hydro-climate conditions in which they were collected. In this study, we investigated the effect of the calibration period selected on the predictive performance and uncertainty of a model. After classifying the inflows of the Hapcheon Dam from 1991 to 2019 into four hydro-climate conditions (dry, wet, normal, and mixed), a conceptual hydrologic partitioning model was calibrated using data from the same hydro-climate condition. Then, predictive performance and post-parameter statistics were analyzed during the verification period under various hydro-climate conditions. The results of the study were as follows: 1) Hydro-climate conditions during the calibration period have a significant effect on model performance and uncertainty, 2) calibration of a hydrologic model using data in dry hydro-climate conditions is most advantageous in securing model performance for arbitrary hydro-climate conditions, and 3) the dry calibration can lead to more reliable model results.

Bit-Vector-Based Space Partitioning Indexing Scheme for Improving Node Utilization and Information Retrieval (노드 이용률과 검색 속도 개선을 위한 비트 벡터 기반 공간 분할 색인 기법)

  • Yeo, Myung-Ho;Seong, Dong-Ook;Yoo, Jae-Soo
    • Journal of KIISE:Computing Practices and Letters
    • /
    • v.16 no.7
    • /
    • pp.799-803
    • /
    • 2010
  • The KDB-tree is a traditional indexing scheme for retrieving multidimensional data. Much research for KDB-tree family frequently addresses the low storage utilization and insufficient retrieval performance as their two bottlenecks. The bottlenecks occur due to a number of unnecessary splits caused by data insertion orders and data skewness. In this paper, we propose a novel index structure, called as $KDB_{CS}^+$-tree, to process skewed data efficiently and improve the retrieval performance. The $KDB_{CS}^+$-tree increases the number of fan-outs by exploiting bit-vectors for representing splitting information and pointer elimination. It also improves the storage utilization by representing entries as a hierarchical structure in each internal node.