• Title/Summary/Keyword: graph processing

Search Result 687, Processing Time 0.024 seconds

Dynamic Block Reassignment for Load Balancing of Block Centric Graph Processing Systems (블록 중심 그래프 처리 시스템의 부하 분산을 위한 동적 블록 재배치 기법)

  • Kim, Yewon;Bae, Minho;Oh, Sangyoon
    • KIPS Transactions on Software and Data Engineering
    • /
    • v.7 no.5
    • /
    • pp.177-188
    • /
    • 2018
  • The scale of graph data has been increased rapidly because of the growth of mobile Internet applications and the proliferation of social network services. This brings upon the imminent necessity of efficient distributed and parallel graph processing approach since the size of these large-scale graphs are easily over a capacity of a single machine. Currently, there are two popular parallel graph processing approaches, vertex-centric graph processing and block centric processing. While a vertex-centric graph processing approach can easily be applied to the parallel processing system, a block-centric graph processing approach is proposed to compensate the drawbacks of the vertex-centric approach. In these systems, the initial quality of graph partition affects to the overall performance significantly. However, it is a very difficult problem to divide the graph into optimal states at the initial phase. Thus, several dynamic load balancing techniques have been studied that suggest the progressive partitioning during the graph processing time. In this paper, we present a load balancing algorithms for the block-centric graph processing approach where most of dynamic load balancing techniques are focused on vertex-centric systems. Our proposed algorithm focus on an improvement of the graph partition quality by dynamically reassigning blocks in runtime, and suggests block split strategy for escaping local optimum solution.

Embedding between a Macro-Star Graph and a Matrix Star Graph (매크로-스타 그래프와 행렬 스타 그래프 사이의 임베딩)

  • Lee, Hyeong-Ok
    • The Transactions of the Korea Information Processing Society
    • /
    • v.6 no.3
    • /
    • pp.571-579
    • /
    • 1999
  • A Macro-Star graph which has a star graph as a basic module has node symmetry, maximum fault tolerance, and hierarchical decomposition property. And, it is an interconnection network which improves a network cost against a star graph. A matrix star graph also has such good properties of a Macro-Star graph and is an interconnection network which has a lower network cost than a Maco-Star graph. In this paper, we propose a method to embed between a Macro-Star graph and a matrix star graph. We show that a Macro-Star graph MS(k, n) can be embedded into a matrix star graph MS\ulcorner with dilation 2. In addition, we show that a matrix star graph MS\ulcorner can be embedded into a Macro-Star graph MS(k,n+1) with dilation 4 and average dilation 3 or less as well. This result means that several algorithms developed in a star graph can be simulated in a matrix star graph with constant cost.

  • PDF

Improving the I/O Performance of Disk-Based Graph Engine by Graph Ordering (디스크 기반 그래프 엔진의 입출력 성능 향상을 위한 그래프 오더링)

  • Lim, Keunhak;Kim, Junghyun;Lee, Eunjae;Seo, Jiwon
    • KIISE Transactions on Computing Practices
    • /
    • v.24 no.1
    • /
    • pp.40-45
    • /
    • 2018
  • With the advent of big data and social networks, large-scale graph processing becomes popular research topic. Recently, an optimization technique called Gorder has been proposed to improve the performance of in-memory graph processing. This technique improves performance by optimizing the graph layout on memory to have better cache locality. However, since it is designed for in-memory graph processing systems, the technique is not suitable for disk-based graph engines; also the cost for applying the technique is significantly high. To solve the problem, we propose a new graph ordering called I/O Order. I/O Order considers the characteristics of I/O accesses for SSDs and HDDs to improve the performance of disk-based graph engine. In addition, the algorithmic complexity of I/O Order is simple compared to Gorder, hence it is cheaper to apply I/O Ordering. I/O order reduces the cost of pre-processing up to 9.6 times compared to that of Gorder's, still its performance is 2 times higher compared to the Random in low-locality graph algorithms.

Incremental Processing Scheme for Graph Streams Considering Data Reuse (데이터 재사용을 고려한 그래프 스트림의 점진적 처리 기법)

  • Cho, Jungkweon;Han, Jinsu;Kim, Minsoo;Choi, Dojin;Bok, Kyoungsoo;Yoo, Jaesoo
    • The Journal of the Korea Contents Association
    • /
    • v.18 no.1
    • /
    • pp.465-475
    • /
    • 2018
  • Recently, as the use of social media and IoT has increased, large graph streams has been generating and studies on real-time processing for them have been actively carrying out. In this paper we propose a incremental graph stream processing scheme that reuses previous result data when the graph changes continuously. We also propose a cost model to selectively perform incremental processing and static processing. The proposed cost model computes the predicted value of the detection cost and the processing cost of the recalculation area based on the actually processed history and performs the incremental processing when the incremental processing is more profit than the static processing. The proposed incremental processing increases the efficiency by processing only the part that changes when the graph update occurs. Also, by collecting only the previous result data of the changed part and performing the incremental processing, the disk I/O costs are reduced. It is shown through various performance evaluations that the proposed scheme outperforms the existing schemes.

Graph Compression by Identifying Recurring Subgraphs

  • Ahmed, Muhammad Ejaz;Lee, JeongHoon;Na, Inhyuk;Son, Sam;Han, Wook-Shin
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2017.04a
    • /
    • pp.816-819
    • /
    • 2017
  • Current graph mining algorithms suffers from performance issues when querying patterns are in increasingly massive network graphs. However, from our observation most data graphs inherently contains recurring semantic subgraphs/substructures. Most graph mining algorithms treat them as independent subgraphs and perform computations on them redundantly, which result in performance degradation when processing massive graphs. In this paper, we propose an algorithm which exploits these inherent recurring subgraphs/substructures to reduce graph sizes so that redundant computations performed by the traditional graph mining algorithms are reduced. Experimental results show that our graph compression approach achieve up to 69% reduction in graph sizes over the real datasets. Moreover, required time to construct the compressed graphs is also reasonably reduced.

A Graph Embedding Technique for Weighted Graphs Based on LSTM Autoencoders

  • Seo, Minji;Lee, Ki Yong
    • Journal of Information Processing Systems
    • /
    • v.16 no.6
    • /
    • pp.1407-1423
    • /
    • 2020
  • A graph is a data structure consisting of nodes and edges between these nodes. Graph embedding is to generate a low dimensional vector for a given graph that best represents the characteristics of the graph. Recently, there have been studies on graph embedding, especially using deep learning techniques. However, until now, most deep learning-based graph embedding techniques have focused on unweighted graphs. Therefore, in this paper, we propose a graph embedding technique for weighted graphs based on long short-term memory (LSTM) autoencoders. Given weighted graphs, we traverse each graph to extract node-weight sequences from the graph. Each node-weight sequence represents a path in the graph consisting of nodes and the weights between these nodes. We then train an LSTM autoencoder on the extracted node-weight sequences and encode each nodeweight sequence into a fixed-length vector using the trained LSTM autoencoder. Finally, for each graph, we collect the encoding vectors obtained from the graph and combine them to generate the final embedding vector for the graph. These embedding vectors can be used to classify weighted graphs or to search for similar weighted graphs. The experiments on synthetic and real datasets show that the proposed method is effective in measuring the similarity between weighted graphs.

Optimization of Graph Processing based on In-Storage Processing (스토리지 내 프로세싱 방식을 사용한 그래프 프로세싱의 최적화 방법)

  • Song, Nae Young;Han, Hyuck;Yeom, Heon Young
    • KIISE Transactions on Computing Practices
    • /
    • v.23 no.8
    • /
    • pp.473-480
    • /
    • 2017
  • In recent years, semiconductor-based storage devices such as flash memory (SSDs) have been developed to high performance. In addition, a trend has been observed of optimally utilizing resources such as the central processing unit (CPU) and memory of the internal controller in the storage device according to the needs of the application. This concept is called In-Storage Processing (ISP). In a storage device equipped with the ISP function, it is possible to process part of the operation executed on the host system, thus reducing the load on the host. Moreover, since the data is processed in the storage device, the data transferred to the host are reduced. In this paper, we propose a method to optimize graph query processing by utilizing these ISP functions, and show that the optimized graph processing method improves the performance of the graph 500 benchmark by up to 20%.

A Study on Digital Image Processing Algorithm for Area Measurement of an Object Image by the Hierarchical Angle-Distance Graphs (계층적 각-거리 그래프를 이용한 물체 면적 측정을 위한 디지털 영상처리 알고리즘에 관한 연구)

  • Kim Woong-Ki;Ra Sung-Woong;Lee Jung-Won
    • The KIPS Transactions:PartB
    • /
    • v.13B no.2 s.105
    • /
    • pp.83-88
    • /
    • 2006
  • Digital image processing algorithm was proposed to measure the area inside of an object image using angle-distance graph used to analyze the pattern of an object in the digital image processing techniques. The first angle-distance graph is generated from a point inside of an object area. The second angle-distance graphs are generated for the areas missed in the first graph by extracting the positions with large gradient in the first angle-distance graph. The order of the graph increases according to the complexity of an object pattern. Size of the area inside of an object boundary is measured by integrating square of distance multiplied by angle for each area from the hierarchical angie-distance graphs.

The Construction of Universal Mulitple Processing Unit based on De Bruijn Graph

  • Park, Chun-Myoung;Song, Hong-Bok
    • Proceedings of the IEEK Conference
    • /
    • 2002.07b
    • /
    • pp.959-962
    • /
    • 2002
  • This paper presents a method of constructing the universal multiple processing element unit(UMPEU) based on De Bruijn Graph. The proposed method is as following. Firstly we propose transformation operators in order to construct the De Bruijn graph using properties of graph. Secondly we construct the transformation table of De Bruijn graph using above transformation operators. Finally we construct the De Bruijn graph using transformation table. The proposed UMPEU is capable of constructing the De Bruijn geraph for any prime number and integer value of finite fields. Also the UMPEU is applied to fault-tolerant computing system, pipeline class, parallel processing network, switching function and its circuits.

  • PDF

An Efficient Graph Algorithm Processing Scheme using GPUs with Limited Memory (제한된 메모리를 가진 GPU를 이용한 효율적인 그래프 알고리즘 처리 기법)

  • Song, Sang-ho;Lee, Hyeon-byeong;Choi, Do-jin;Lim, Jong-tae;Bok, Kyoung-soo;Yoo, Jae-soo
    • The Journal of the Korea Contents Association
    • /
    • v.22 no.8
    • /
    • pp.81-93
    • /
    • 2022
  • Recently, research on processing a large-capacity graph using GPUs has been conducting. In order to process a large-capacity graph in a GPU with limited memory, the graph must be divided into subgraphs and then processed by scheduling subgraphs. In this paper, we propose an efficient graph algorithm processing scheme in GPU environments with limited memory and performance evaluation. The proposed scheme consists of a graph differential subgraph scheduling method and a graph segmentation method. The bulk graph segmentation method determines how a large-capacity graph can be segmented into subgraphs so that it can be processed efficiently by the GPU. The differential subgraph scheduling method schedule subgraphs processed by GPUs to reduce redundant transmission of the repeatedly used data between HOST-GPUs. It shows the superiority of the proposed scheme by performing various performance evaluations.