• Title/Summary/Keyword: data partitioning

Search Result 390, Processing Time 0.026 seconds

A Path Partitioning Technique for Indexing XML Data (XML 데이타 색인을 위한 경로 분할 기법)

  • 김종익;김형주
    • Journal of KIISE:Databases
    • /
    • v.31 no.3
    • /
    • pp.320-330
    • /
    • 2004
  • Query languages for XML use paths in a data graph to represent queries. Actually, paths in a data graph are used as a basic constructor of an XML query. User can write more expressive Queries by using Patterns (e.g. regular expressions) for paths. There are many identical paths in a data graph because of the feature of semi-structured data. Current researches for indexing XML utilize identical paths in a data graph, but such an index can grow larger than source data graph and cannot guarantee efficient access path. In this paper we propose a partitioning technique that can partition all the paths in a data graph. We develop an index graph that can find appropriate partitions for a path query efficiently. The size of our index graph can be adjusted regardless of the source data. So, we can significantly improve the cost for index graph traversals. In the performance study, we show our index much faster than other graph based indexes.

An Efficient Angular Space Partitioning Based Skyline Query Processing Using Sampling-Based Pruning (데이터 샘플링 기반 프루닝 기법을 도입한 효율적인 각도 기반 공간 분할 병렬 스카이라인 질의 처리 기법)

  • Choi, Woosung;Kim, Minseok;Diana, Gromyko;Chung, Jaehwa;Jung, Soonyong
    • KIPS Transactions on Software and Data Engineering
    • /
    • v.6 no.1
    • /
    • pp.1-8
    • /
    • 2017
  • Given a multi-dimensional dataset of tuples, a skyline query returns a subset of tuples which are not 'dominated' by any other tuples. Skyline query is very useful in Big data analysis since it filters out uninteresting items. Much interest was devoted to the MapReduce-based parallel processing of skyline queries in large-scale distributed environment. There are three requirements to improve parallelism in MapReduced-based algorithms: (1) workload should be well balanced (2) avoid redundant computations (3) Optimize network communication cost. In this paper, we introduce MR-SEAP (MapReduce sample Skyline object Equality Angular Partitioning), an efficient angular space partitioning based skyline query processing using sampling-based pruning, which satisfies requirements above. We conduct an extensive experiment to evaluate MR-SEAP.

Design and Evaluation of Flexible Thread Partitioning System (융통성 있는 스레드 분할 시스템 설계와 평가)

  • Jo, Sun-Moon
    • Journal of Internet Computing and Services
    • /
    • v.8 no.3
    • /
    • pp.75-83
    • /
    • 2007
  • Multithreaded model is an effective parallel system in that it can reduce the long memory reference latency time and solve the synchronization problems. When compiling the non-strict functional programs for the multithreaded parallel machine, the most important thing is to find an set of sequentially executable instructions and to partitions them into threads. The existing partitioning algorithm partitions the condition of conditional expression, true expression and false expression into the basic blocks and apply local partitioning to these basic blocks. We can do the better partitioning if we modify the definition of the thread and allow the branching within the thread. The branching within the thread do not reduce the parallelism, do not increase the number of synchronization and do not violate the basic rule of the thread partitioning. On the contrary, it can lengthen the thread and reduce the number of synchronization. In the paper, we enhance the method of the partition of threads by combining the three basic blocks into one of two blocks.

  • PDF

UEP Turbo Encoder for H.264/AVC (H.264/AVC를 위한 UEP Turbo Encoder)

  • Kim, June;Kim, Youngseop;Park, In-Ho
    • Journal of the Semiconductor & Display Technology
    • /
    • v.14 no.1
    • /
    • pp.51-53
    • /
    • 2015
  • H.264/AVC is international video coding standard, which shows improved code and efficiency than the existing video standards. H.264/AVC proposes data partitioning method that considerably to be an effective layering technique which separates important addressing data from the residual data. UEP(Unequal Error Protection) turbo code of H.264/AVC uses retransmission system to get the UEP effectively. However, Data partitioning system of H.264/AVC is inefficient method in turbo code of H.264/AVC. Based on this observation, we propose the new UEP turbo code algorithm that reconstructs input sequence of turbo code without retransmission system.

Design of a Block Data Flow Architecture for 2-D DWT/IDWT (2차원 DWT/IDWT의 블록 데이터 플로우 구조 설계)

  • 정갑천;강준우
    • Proceedings of the IEEK Conference
    • /
    • 1998.10a
    • /
    • pp.1157-1160
    • /
    • 1998
  • This paper describes the design of a block data flow architecture(BDFA) which implements 2-D discrete wavelet transform(DWT)/inverse discrete wavelet transform(IDWT) for real time image processing applications. The BDFA uses 2-D product separable filters for DWT/IDWT. It consists of an input module, a processor array, and an output module. It use both data partitioning and algorithm partitioning to achieve high efficiency and high throughput. The 2-D DWT/IDWT algorithm for 256$\times$256 lenna image has been simulated using IDL(Interactive Data Language). The 2-D array structured BDFA for the 2-D filter has been modeled and simulated using VHDL.

  • PDF

Data Partitioning for Error Resilience and Incremental Rendering of 3D Model (삼차원 모델의 점진적인 렌더링과 오류 강인을 위한 효율적인 데이터 분할 방법 (CODAP))

  • 송문섭;안정환;김성진;한만진;호요성
    • Proceedings of the IEEK Conference
    • /
    • 1999.11a
    • /
    • pp.1089-1092
    • /
    • 1999
  • Applications using 3D models are increasing recently. Since 3D polygonal models are structured by a triangular mesh, the coding of polygonal models in strips of triangles is an efficient way of representing the data. These strips may be very long, and may take a long time to render or transmit. If the triangle strips are partitioned, it may be possible to perform more efficient data transmission in an error-prone environment and to display the 3D model progressively. In this paper, we devised the Component Based Data Partitioning (CODAP) which is based on Topological Surgery (TS). In order to support the error resilience and the progressively build-up rendering, we partition the connectivity, geometry, and properties of a 3D polygonal model. Each partitioned component is independently encoded and resynchronization between partitioned components is done.

  • PDF

Clustering Data with Categorical Attributes Using Inter-dimensional Association Rules and Hypergraph Partitioning (차원간 연관관계와 하이퍼그래프 분할법을 이용한 범주형 속성을 가진 데이터의 클러스터링)

  • 이성기;윤덕균
    • Journal of Korean Society of Industrial and Systems Engineering
    • /
    • v.24 no.65
    • /
    • pp.41-50
    • /
    • 2001
  • Clustering in data mining is a discovery process that groups a set of data such that the intracluster similarity is maximized and intercluster similarity is minimized. The discovered clusters from clustering process are used to explain the characteristics of the data distribution. In this paper we propose a new methodology for clustering related transactions with categorical attributes. Our approach starts with transforming general relational databases into a transactional databases. We make use of inter-dimensional association rules for composing hypergraph edges, and a hypergraph partitioning algorithm for clustering the values of attributes. The clusters of the values of attributes are used to find the clusters of transactions. The suggested procedure can enhance the interpretation of resulting clusters with allocated attribute values.

  • PDF

Deriving ratings from a private P2P collaborative scheme

  • Okkalioglu, Murat;Kaleli, Cihan
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.13 no.9
    • /
    • pp.4463-4483
    • /
    • 2019
  • Privacy-preserving collaborative filtering schemes take privacy concerns into its primary consideration without neglecting the prediction accuracy. Different schemes are proposed that are built upon different data partitioning scenarios such as a central server, two-, multi-party or peer-to-peer network. These data partitioning scenarios have been investigated in terms of claimed privacy promises, recently. However, to the best of our knowledge, any peer-to-peer privacy-preserving scheme lacks such study that scrutinizes privacy promises. In this paper, we apply three different attack techniques by utilizing auxiliary information to derive private ratings of peers and conduct experiments by varying privacy protection parameters to evaluate to what extent peers' data can be reconstructed.

Reviews About food Safety on Packaging Materials and Printing Ink Solvent (식품용 포장제와 인쇄 용제의 안정성에 관한 연구)

  • An, Duek-Jun;Kim, Youn-Uck;Park, Hoon
    • KOREAN JOURNAL OF PACKAGING SCIENCE & TECHNOLOGY
    • /
    • v.12 no.2
    • /
    • pp.91-96
    • /
    • 2006
  • The widespread use of packaging materials on food packages sometimes causes off-flavor and deterioration in the food by migration of packaging materials which consist of mainly plastic materials and printing ink solvents. Even though migration of the residual packaging materials does not generally cause safety problems in the contained food, it certainly can generate off-flavor and finally deteriorate quality of the finished product. In highly consumer-oriented markets, quality of the finished product is gaining more importance economically, so profound and thorough study about migration into food to maintain the fine quality of the end product becomes an importance issue. Studies have been conducted about migration of various packaging materials and the chemical reactions between the food components and the materials used for food packaging (Brody, 1989; Mcneal and Breder, 1987; Lawson, G and Lawson, C, 1996). Several of these studies have measured partition coefficient values (Kp) between packaging materials and various food samples and involved finding factors that affected the partitioning behavior of packaging materials into the contained food. However, to enable prediction of partitioning, data are still needed on the relation of packaging material chemical structure and properties to partitioning behavior, and on the partitioning behavior of various food ingredients and the total food compositions.

  • PDF

Cockpit Crew Scheduling using Set Partitioning Problem (집합분할모형을 이용한 운항승무원의 승무경로 일정계획)

  • 김국연;이영훈
    • Korean Management Science Review
    • /
    • v.21 no.1
    • /
    • pp.39-55
    • /
    • 2004
  • Efficient crew scheduling for cockpit crew is important in airline industry due to operational safety and cost associated with the flight duty time. Because of complexity of regulations imposed to the cockpit crew. it is complicated to generate an efficient schedule. Schedule of cockpit crew can be generated through two steps; selecting of flight patterns. and scheduling of them to the specific time horizon. Heuristic method is developed and applied with massive data in a limited time of computation. A set of flight patterns is selected from all possible flight patterns. which are generated by composing the flight leg based on regulations. by using the set partitioning problem with objective function of oversea stay cost. The selected set of flight patterns found at the first step is allocated to 4 week crew schedule to minimize the variance of total fight time assigned to each crew. The crew schedules obtained are evaluated and compared with the ones currently used in one of major airline company.