• Title/Summary/Keyword: parallel join

Search Result 48, Processing Time 0.025 seconds

Performance Study of the Index-based Parallel Join

  • Jeong, Byeong-Soo;Edward Omiecinski
    • The Journal of Information Technology and Database
    • /
    • v.2 no.2
    • /
    • pp.87-109
    • /
    • 1995
  • The index file has been used a access database records effectively. The join operation in a relational database system requires a large execution time, especially in the case of handling large size tables. If the indexes are available on the joining attributes for both relations involved in the join and the join selectivity is relatively small, we can improve the execution time of the join operation. In this paper. we investigate the performance trade-offs of parallel index-based join algorithms where different indexing schemes are used. We also present a comparison of our index-based parallel join algorithms with the hash-based parallel join algorithm.

  • PDF

An Advanced Parallel Join Algorithm for Managing Data Skew on Hypercube Systems (하이퍼큐브 시스템에서 데이타 비대칭성을 고려한 향상된 병렬 결합 알고리즘)

  • 원영선;홍만표
    • Journal of KIISE:Computer Systems and Theory
    • /
    • v.30 no.3_4
    • /
    • pp.117-129
    • /
    • 2003
  • In this paper, we propose advanced parallel join algorithm to efficiently process join operation on hypercube systems. This algorithm uses a broadcasting method in processing relation R which is compatible with hypercube structure. Hence, we can present optimized parallel join algorithm for that hypercube structure. The proposed algorithm has a complete solution of two essential problems - load balancing problem and data skew problem - in parallelization of join operation. In order to solve these problems, we made good use of the characteristics of clustering effect in the algorithm. As a result of this, performance is improved on the whole system than existing algorithms. Moreover. new algorithm has an advantage that can implement non-equijoin operation easily which is difficult to be implemented in hash based algorithm. Finally, according to the cost model analysis. this algorithm showed better performance than existing parallel join algorithms.

Parallel Processing of Multi-Way Spatial Join (다중 공간 조인의 병렬 처리)

  • Ryu, Woo-Seok;Hong, Bong-Hee
    • Journal of KIISE:Databases
    • /
    • v.27 no.2
    • /
    • pp.256-268
    • /
    • 2000
  • Multi-way spatial join is a nested expression of two or more spatial joins. It costs much to process multi-way spatial join, but there have not still reported the scheme of parallel processing of multi-way spatial join. In this paper, parallel processing of multi-way spatial join consists of parallel multi-way spatial filter and parallel spatial refinement. Parallel spatial refinement is executed by the following two steps. The first is the generation of a graph used for reducing duplication of both spatial objects and spatial operations from pairs candidate object table that are the results of multi-way spatial filter. The second is the parallel spatial refinement using that graph. Refinement using the graph is proved to be more efficient than the others. In task creation for parallel refinement, minimum duplication partitioning of the Spatial_Obicct_On_Node graph shows best performance.

  • PDF

Task Creation and Assignment based on Object Caching for Parallel Spatial Join (병렬공간 조인을 위한 객체 캐쉬 기반 태스크 생성 및 할당)

  • 서영덕;김진덕;홍봉희
    • Journal of KIISE:Software and Applications
    • /
    • v.26 no.10
    • /
    • pp.1178-1178
    • /
    • 1999
  • A spatial join has the property that its execution time exponentially increases in proportion to the number of spatial objects. Recently, there have been many attempts for improving the performance of the spatial join by using parallel processing schemes, In the case of executing parallel spatial join using the parallel machine with shared disk architecture, the disk bottleneck of parallel processing of spatial join worsens in comparison with sequential spatial join. This paper presents the algorithms of task creation and assignment to reduce the disk bottleneck caused by accessing the shared disk at the same time, and to minimize message passing between processors, This paper proposes object caching which is a higher level of abstraction than page caching, and uses it to do creation and assignment of tasks according to temporal and spatial localities for minimizing disk access time. The object caching shows the performance improvement of 50%. The task creation and assignment using localities gives the gain of 30% and 20%. Overall performance evaluation of the proposed algorithms shows 7.2 times speed up than those of sequential execution of spatial joins.

Performance Comparison of Join Operations Parallelization by using GPGPU (GPGPU 기반 조인 연산 병렬화 성능 비교)

  • Lee, Jong-Sub;Lee, Sang-Back;Lee, Kyu-Chul
    • Database Research
    • /
    • v.34 no.3
    • /
    • pp.28-44
    • /
    • 2018
  • In a database system, the most expensive operation among relational operations is a join operation. Generally, CPU-based join operations uses parallel processing with either 1 core or 16 cores at most, which does not significantly improve the function. On the other hand, GPGPU(General-Purpose computing on Graphics Processing Units) allows parallel processing through thousands of processing units, greatly reducing the time required to perform join operations. Parallelization of the operation using GPGPU uses NVIDIA's CUDA SDK. In this paper, we implement parallelization of the join operation using GPGPU and compare the performances. The used join operations are Nested Loop Join (NLJ), Sort Merge Join (SMJ) and Hash Join (HJ), and GPGPU equipment uses TITAN Xp, GTX 1080 Ti and GTX 1080. We measure and compare the performance of join operations based on CPU and GPGPU. We compare this performance with the performance of the previous study on the join operation based on GPGPU. The results of experiment show that the performance based on GPGPU is 6~328 times faster than the one based on CPU.

Uniform Load Distribution Using Sampling-Based Cost Estimation in Parallel Join (병렬 조인에서 샘플링 기반 비용 예측 기법을 이용한 균등 부하 분산)

  • Park, Ung-Gyu
    • The Transactions of the Korea Information Processing Society
    • /
    • v.6 no.6
    • /
    • pp.1468-1480
    • /
    • 1999
  • In database systems, join operations are the most complex and time consuming ones which limit performance of such system. Many parallel join algorithms have been proposed for the systems. However, they did not consider data skew, such as attribute value skew (AVS) and join product skew (JPS). In the skewness environments, performance of framework for a uniform load distribution and an efficient parallel join algorithm using the framework to handle AVS and JPS. In our algorithm, we estimate data distributions of input and output relations of join operations using the sampling methodology and evaluate join cost for the estimated data distributions. Finally, using the histogram equalization method we distribute data among nodes to achieve good load balancing among nodes in the local joining phase. For performance comparison, we present simulation model of our algorithm and other join algorithms and present the result of some simulation experiments. The results indicate that our algorithm outperforms other algorithms in the skewed case.

  • PDF

Efficient Parallel Spatial Join Processing Method in a Shared-Nothing Database Cluster System (비공유 공간 클러스터 환경에서 효율적인 병렬 공간 조인 처리 기법)

  • Chung, Warn-Ill;Lee, Chung-Ho;Bae, Hae-Young
    • The KIPS Transactions:PartD
    • /
    • v.10D no.4
    • /
    • pp.591-602
    • /
    • 2003
  • Delay and discontinuance phenomenon of service are cause by sudden increase of the network communication amount and the quantity consumed of resources when Internet users are driven excessively to a conventional single large database sewer. To solve these problems, spatial database cluster consisted of several single nodes on high-speed network to offer high-performance is risen. But, research about spatial join operation that can reduce the performance of whole system in case process at single node is not achieved. So, in this paper, we propose efficient parallel spatial join processing method in a spatial database cluster system that uses data partitions and replications method that considers the characteristics of space data. Since proposed method does not need the creation step and the assignment step of tasks, and does not occur additional message transmission between cluster nodes that appear in existent parallel spatial join method, it shows performance improvement of 23% than the conventional parallel R-tree spatial join for a shared-nothing architecture about expensive spatial join queries. Also, It can minimize the response time to user because it removes redundant refinement operation at each cluster node.

Design of Multiprocess Models for Parallel Protocol Implementation (병렬 프로토콜 구현을 위한 다중 프로세스 모델의 설계)

  • Choi, Sun-Wan;Chung, Kwang-Sue
    • The Transactions of the Korea Information Processing Society
    • /
    • v.4 no.10
    • /
    • pp.2544-2552
    • /
    • 1997
  • This paper presents three multiprocess models for parallel protocol implementation, that is, (1)channel communication model, (2)fork-join model, and (3)event polling model. For the specification of parallelism for each model, a parallel programming language, Par. C System, is used. to measure the performance of multiprocess models, we implemented the Internet Protocol Suite(IPS) Internet Protocol (IP) for each model by writing the parallel language on the Transputer. After decomposing the IP functions into two parts, that is, the sending side and the receiving side, the parallelism in both sides is exploited in the form of Multiple Instruction Single Data (MISD). Three models are evaluated and compared on the basis of various run-time overheads, such as an event sending via channels in the parallel channel communication model, process creating in the fork-join model and context switching in the event polling model, at the sending side and the receiving side. The event polling model has lower processing delays as about 77% and 9% in comparison with the channel communication model and the fork-join model at the sending side, respectively. At the receiving side, the fork-join model has lower processing delays as about 55% and 107% in comparison with the channel communication model and the event polling model, respectively.

  • PDF

k-NN Join Based on LSH in Big Data Environment

  • Ji, Jiaqi;Chung, Yeongjee
    • Journal of information and communication convergence engineering
    • /
    • v.16 no.2
    • /
    • pp.99-105
    • /
    • 2018
  • k-Nearest neighbor join (k-NN Join) is a computationally intensive algorithm that is designed to find k-nearest neighbors from a dataset S for every object in another dataset R. Most related studies on k-NN Join are based on single-computer operations. As the data dimensions and data volume increase, running the k-NN Join algorithm on a single computer cannot generate results quickly. To solve this scalability problem, we introduce the locality-sensitive hashing (LSH) k-NN Join algorithm implemented in Spark, an approach for high-dimensional big data. LSH is used to map similar data onto the same bucket, which can reduce the data search scope. In order to achieve parallel implementation of the algorithm on multiple computers, the Spark framework is used to accelerate the computation of distances between objects in a cluster. Results show that our proposed approach is fast and accurate for high-dimensional and big data.

Parallel Spatial Join Method Using Efficient Spatial Relation Partition In Distributed Spatial Database Systems (분산 공간 DBMS에서의 효율적인 공간 릴레이션 분할 기법을 이용한 병렬 공간 죠인 기법)

  • Ko, Ju-Il;Lee, Hwan-Jae;Bae, Hae-Young
    • Journal of Korea Spatial Information System Society
    • /
    • v.4 no.1 s.7
    • /
    • pp.39-46
    • /
    • 2002
  • In distributed spatial database systems, users nay issue a query that joins two relations stored at different sites. The sheer volume and complexity of spatial data bring out expensive CPU and I/O costs during the spatial join processing. This paper shows a new spatial join method which joins two spatial relation in a parallel way. Firstly, the initial join operation is divided into two distinct ones by partitioning one of two participating relations based on the region. This two join operations are assigned to each sites and executed simultaneously. Finally, each intermediate result sets from the two join operations are merged to an ultimate result set. This method reduces the number of spatial objects participating in the spatial operations. It also reduces the scope and the number of scanning spatial indices. And it does not materialize the temporary results by implementing the join algebra operators using the iterator. The performance test shows that this join method can lead to efficient use in terms of buffer and disk by narrowing down the joining region and decreasing the number of spatial objects.

  • PDF