International Journal of Contents

  • 제14권1호
  • /
  • Pages.28-33
  • /
  • 2018
  • /
  • 1738-6764(pISSN)
  • /
  • 2093-7504(eISSN)
한국콘텐츠학회 (The Korea Contents Association)
권호 표지
DOI QR코드

DOI QR Code

Distributed Moving Objects Management System for a Smart Black Box

  • Lee, Hyunbyung (Dept. of Computer Engineering Korea National University of Transportation) ;
  • Song, Seokil (Dept. of Computer Engineering Korea National University of Transportation)
  • 투고 : 2017.03.07
  • 심사 : 2017.05.23
  • 발행 : 2018.03.28
PDF 다운로드
⟨ 이전 논문 다음 논문 ⟩

초록

In this paper, we design and implement a distributed, moving objects management system for processing locations and sensor data from smart black boxes. The proposed system is designed and implemented based on Apache Kafka, Apache Spark & Spark Streaming, Hbase, HDFS. Apache Kafka is used to collect the data from smart black boxes and queries from users. Received location data from smart black boxes and queries from users becomes input of Apache Spark Streaming. Apache Spark Streaming preprocesses the input data for indexing. Recent location data and indexes are stored in-memory managed by Apache Spark. Old data and indexes are flushed into HBase later. We perform experiments to show the throughput of the index manager. Finally, we describe the implementation detail in Scala function level.

키워드

E1CTBR_2018_v14n1_28_f0001.png 이미지

Fig. 1. In-memory Grid Index Structure based on Apache SparkStreaming

E1CTBR_2018_v14n1_28_f0002.png 이미지

Fig. 1. Architecture of the proposed moving object management system

E1CTBR_2018_v14n1_28_f0003.png 이미지

Fig. 3. Index Manager of the proposed distributed in-memory moving object management system

E1CTBR_2018_v14n1_28_f0004.png 이미지

Fig. 4. Indexing throughput with varying the number ofexecutors (the number of nodes is 4)

E1CTBR_2018_v14n1_28_f0005.png 이미지

Fig. 5. Indexing throughput with varying the number of nodes(the number of executors is 4)

E1CTBR_2018_v14n1_28_f0006.png 이미지

Fig. 6. Lineage of transformations for the implemented movingobject management system

E1CTBR_2018_v14n1_28_f0007.png 이미지

Fig. 7. Block diagram of the implemented moving objectmanagement system

Table 1. Parameters of Performance Evaluation

E1CTBR_2018_v14n1_28_t0001.png 이미지

Table 2. Function list for the implemented moving object management system

E1CTBR_2018_v14n1_28_t0002.png 이미지

참고문헌

  1. H. Li, Y. Lee, and S. Song, "Grid based Distributed Inmemory Indexing for Moving Objects," Proceedings of International Symposium on Information Technology Convergence, Jeonju, Republic of Korea, Oct. 30-31 2014.
  2. K. Kim, S. K. Cha, and K. Kwon, "Optimizing Multidimensional Index Trees for Main Memory Access," SIGMOD Record, vol. 30, no. 2, 2001, pp.139-150. https://doi.org/10.1145/376284.375679
  3. L. Biveinis, S. Saltenis, and C. S. Jensen, "Main-memory Operation Buffering for Efficient R-tree Update," Proceedings of the 33rd 41st International Conference on Very Large Data Bases, Vienna, Austria, Sep. 23-28 2007, pp. 591-602.
  4. D. Sidlauskas, S. Saltenis, and C. S. Jensen, "Parallel Mainmemory Indexing for Moving-object Query and Update Workloads," Proceedings of the 2012 ACM SIGMOD International Conference on Management of Data, Arizona, USA, May. 20-24 2012, pp. 37-48.
  5. S. Nishimura, S. Das, D. Agrawal, and A. E. Abbadi, "MD-HBase: A Scalable Multi-dimensional Data Infrastructure for Location Aware Services," Proceedings of the 2011 12th IEEE International Conference on Mobile Data Management, Lulea, Sweden, Jun. 6-9 2011, pp.7-16.
  6. A. Aji, F. Wang, H. Vo, R. Lee, Q. Liu, X. Zhang, and J. H. Saltz, "Hadoop-GIS: A High Performance Spatial Data Warehousing System over Mapreduce," Proceedings of the 39rd 41st International Conference on Very Large Data Bases, Trento, Italy, Aug. 26-30 2013, pp. 1009-1020.
  7. A. Eldawyand and M. F. Mokbel, "Spatial Hadoop: A Mapreduce Framework for Spatial Data," Proceedings of the 2015 IEEE 31st International Conference on Data Engineering, Seoul, Republic of Korea, Apr. 3-17 2015, pp. 1352-1363.
  8. J. Lu and R. H. Guting, "Parallel Secondo: A Practical System for Largescale Processing of Moving Objects," Proceedings of the 2014 IEEE 30st International Conference on Data Engineering, Chicago, USA, Mar. 31-Apr. 4 2014, pp. 1190-1193.
  9. A. R. Mahmood, A. M. Aly, T. Qadah, E. K. Rezig, A. Daghistani, A. Madkour, A. S. Abdelhamid, M. S. Hassan, S. B. Walid, and G. Aref, "Tornado: A Distributed Spatiotextual Stream Processing System," Proceedings of the 41st International Conference on Very Large Data Bases, Hawaii, USA, Aug. 31- Sep. 4 2015, pp. 2020-2023.
  10. Apache Spark, http://spark.apache.org/
  11. Apache Hadoop, http://hadoop.apache.org/
  12. Apache Storm, http://storm.apache.org/
  13. H. Lee, Y. Kwak, and S. Song, "Implementation of Distributed In-Memory Moving Objects Management System," Advanced Science Letters, vol. 23, no.10, 2017, pp. 10361-10365. https://doi.org/10.1166/asl.2017.10453