Browse > Article
http://dx.doi.org/10.5392/JKCA.2018.18.01.465

Incremental Processing Scheme for Graph Streams Considering Data Reuse  

Cho, Jungkweon (충북대학교 빅데이터학과)
Han, Jinsu (충북대학교 정보통신학과)
Kim, Minsoo (충북대학교 정보통신학과)
Choi, Dojin (충북대학교 정보통신학과)
Bok, Kyoungsoo (충북대학교 정보통신학과)
Yoo, Jaesoo (충북대학교 정보통신학과)
Publication Information
The Journal of the Korea Contents Association / v.18, no.1, 2018 , pp. 465-475 More about this Journal
Abstract
Recently, as the use of social media and IoT has increased, large graph streams has been generating and studies on real-time processing for them have been actively carrying out. In this paper we propose a incremental graph stream processing scheme that reuses previous result data when the graph changes continuously. We also propose a cost model to selectively perform incremental processing and static processing. The proposed cost model computes the predicted value of the detection cost and the processing cost of the recalculation area based on the actually processed history and performs the incremental processing when the incremental processing is more profit than the static processing. The proposed incremental processing increases the efficiency by processing only the part that changes when the graph update occurs. Also, by collecting only the previous result data of the changed part and performing the incremental processing, the disk I/O costs are reduced. It is shown through various performance evaluations that the proposed scheme outperforms the existing schemes.
Keywords
Stream Processing; Graph Processing; Incremental Processing; Data Reuse; Cost Model;
Citations & Related Records
Times Cited By KSCI : 1  (Citation Analysis)
연도 인용수 순위
1 B. Pramod, W. Alexander, A. E. Istemi, R. Rodrigo, and A. A. Umut, "Large-scale incremental data processing with change propagation," Proc. USENIX Workshop on Hot Topics in Cloud Computing, 2011.
2 J. Wuyang, L. Jianxin, Y. Weiren, and Z. Richong, "iGraph: an incremental data processing system for dynamic graph," Frontiers of Computer Science, Vol.10, No.3, pp.462-476, 2016.   DOI
3 https://snap.stanford.edu/data/
4 O. Salem, L. Yaning, and M. Ahmed, "Anomaly detection in medical wireless sensor networks," Computing Science and Engineering, Vol.7, No.4, pp.272-284, 2013.   DOI
5 F. Elijorde, K. Sungho, and L. Jaewan, "A wind turbine fault detection approach based on cluster analysis and frequent pattern mining," KSII Transactions on Internet and Information Systems, Vol.8, No.2, pp.664-677, 2014.   DOI
6 G. Malewicz, H. M. Austern, J. A. Bik, J. Dehnert, I. Horn, N. Leiser, and G. M. Czajkowski, "Pregel: a system for large-scale graph processing," Proc. ACM SIGMOD International Conference on Management of data, pp.135-146, 2010.
7 Y. A. Kim and G. W. Park, "Topic-Driven SocialRank: Personalized search result ranking by identifying similar, credible users in a social network," Knowledge-Based Systems, Vol.54, pp.230-242, 2013.   DOI
8 서복일, 김재인, 황부현, "스트림 데이터 환경에서 배치 가중치를 이용하여 사용자 특성을 반영한 빈발항목 집합 탐사," 한국콘텐츠학회논문지, 제11권, 제1호, pp.56-64, 2011.   DOI
9 https://carestruck.org/happens-internet-minute/
10 Y. Low, J. Gonzalez, A. Kyrola, D. Bickson, C. Guestrin, and J. Hellerstein, "Distributed GraphLab: A Framework for Machine Learning in the Cloud," Proceedings of the VLDB Endowment, Vol.5, No.8, pp.716-727, 2012.   DOI
11 J. Gonzalez, Y. Low, H. Gu, D. Bickson, and C. Guestrin, "PowerGraph: Distributed graphparallel computation on natural graphs," Proc. USENIX Symposium on Operating Systems Design and Implementation, pp.17-30, 2012.
12 R. S. Xin, J. Gonzalez, F. J. Michael, and S. Ion, "Graphx: A resilient distributed graph system on spark," Proc. International Workshop on Graph Data Management Experiences and Systems, p.2, 2013.
13 U. Gupta and L. Fegaras, "Distributed Incremental Graph Analysis," Proc. IEEE International Congress on Big Data, pp.75-82, 2016.