[KSCI] Korea Science Citation Index Service

http://dx.doi.org/10.5392/IJoC.2018.14.1.028

Distributed Moving Objects Management System for a Smart Black Box

Lee, Hyunbyung (Dept. of Computer Engineering Korea National University of Transportation)
Song, Seokil (Dept. of Computer Engineering Korea National University of Transportation)

Publication Information

International Journal of Contents / v.14, no.1, 2018 , pp. 28-33 More about this Journal

Abstract

In this paper, we design and implement a distributed, moving objects management system for processing locations and sensor data from smart black boxes. The proposed system is designed and implemented based on Apache Kafka, Apache Spark & Spark Streaming, Hbase, HDFS. Apache Kafka is used to collect the data from smart black boxes and queries from users. Received location data from smart black boxes and queries from users becomes input of Apache Spark Streaming. Apache Spark Streaming preprocesses the input data for indexing. Recent location data and indexes are stored in-memory managed by Apache Spark. Old data and indexes are flushed into HBase later. We perform experiments to show the throughput of the index manager. Finally, we describe the implementation detail in Scala function level.

Keywords

Moving Objects; Distributed System; Index; Spatio-temporal;

Citations & Related Records

Reference

1	K. Kim, S. K. Cha, and K. Kwon, "Optimizing Multidimensional Index Trees for Main Memory Access," SIGMOD Record, vol. 30, no. 2, 2001, pp.139-150. DOI
2	L. Biveinis, S. Saltenis, and C. S. Jensen, "Main-memory Operation Buffering for Efficient R-tree Update," Proceedings of the 33rd 41st International Conference on Very Large Data Bases, Vienna, Austria, Sep. 23-28 2007, pp. 591-602.
3	D. Sidlauskas, S. Saltenis, and C. S. Jensen, "Parallel Mainmemory Indexing for Moving-object Query and Update Workloads," Proceedings of the 2012 ACM SIGMOD International Conference on Management of Data, Arizona, USA, May. 20-24 2012, pp. 37-48.
4	S. Nishimura, S. Das, D. Agrawal, and A. E. Abbadi, "MD-HBase: A Scalable Multi-dimensional Data Infrastructure for Location Aware Services," Proceedings of the 2011 12th IEEE International Conference on Mobile Data Management, Lulea, Sweden, Jun. 6-9 2011, pp.7-16.
5	A. Aji, F. Wang, H. Vo, R. Lee, Q. Liu, X. Zhang, and J. H. Saltz, "Hadoop-GIS: A High Performance Spatial Data Warehousing System over Mapreduce," Proceedings of the 39rd 41st International Conference on Very Large Data Bases, Trento, Italy, Aug. 26-30 2013, pp. 1009-1020.
6	A. Eldawyand and M. F. Mokbel, "Spatial Hadoop: A Mapreduce Framework for Spatial Data," Proceedings of the 2015 IEEE 31st International Conference on Data Engineering, Seoul, Republic of Korea, Apr. 3-17 2015, pp. 1352-1363.
7	J. Lu and R. H. Guting, "Parallel Secondo: A Practical System for Largescale Processing of Moving Objects," Proceedings of the 2014 IEEE 30st International Conference on Data Engineering, Chicago, USA, Mar. 31-Apr. 4 2014, pp. 1190-1193.
8	A. R. Mahmood, A. M. Aly, T. Qadah, E. K. Rezig, A. Daghistani, A. Madkour, A. S. Abdelhamid, M. S. Hassan, S. B. Walid, and G. Aref, "Tornado: A Distributed Spatiotextual Stream Processing System," Proceedings of the 41st International Conference on Very Large Data Bases, Hawaii, USA, Aug. 31- Sep. 4 2015, pp. 2020-2023.
9	Apache Hadoop, http://hadoop.apache.org/
10	Apache Spark, http://spark.apache.org/
11	Apache Storm, http://storm.apache.org/
12	H. Lee, Y. Kwak, and S. Song, "Implementation of Distributed In-Memory Moving Objects Management System," Advanced Science Letters, vol. 23, no.10, 2017, pp. 10361-10365. DOI
13	H. Li, Y. Lee, and S. Song, "Grid based Distributed Inmemory Indexing for Moving Objects," Proceedings of International Symposium on Information Technology Convergence, Jeonju, Republic of Korea, Oct. 30-31 2014.