ETRI Journal

Electronics and Telecommunications Research Institute (한국전자통신연구원)
권호 표지
DOI QR코드

DOI QR Code

DART: Fast and Efficient Distributed Stream Processing Framework for Internet of Things

  • Received : 2016.08.31
  • Accepted : 2017.02.09
  • Published : 2017.04.01
Download PDF
⟨ Previous Next ⟩

Abstract

With the advent of the Internet-of-Things paradigm, the amount of data production has grown exponentially and the user demand for responsive consumption of data has increased significantly. Herein, we present DART, a fast and lightweight stream processing framework for the IoT environment. Because the DART framework targets a geospatially distributed environment of heterogeneous devices, the framework provides (1) an end-user tool for device registration and application authoring, (2) automatic worker node monitoring and task allocations, and (3) runtime management of user applications with fault tolerance. To maximize performance, the DART framework adopts an actor model in which applications are segmented into microtasks and assigned to an actor following a single responsibility. To prove the feasibility of the proposed framework, we implemented the DART system. We also conducted experiments to show that the system can significantly reduce computing burdens and alleviate network load by utilizing the idle resources of intermediate edge devices.

Keywords

References

  1. D. Evans, The Internet of Things: How the Next Evolution of the Internet is Changing Everything, Cisco Internet Business Solution Group (IBSG) White Paper, Apr. 2011.
  2. S. Sagiroglu and D. Sinanc, "Big Data: a Review," Int. Conf. Collaboration Technol. Syst., San Diego, CA, USA, May 20-24, 2013, pp. 42-27.
  3. G. Eleftherakis et al., "Architecting the IoT Paradigm: a Middleware for Autonomous Distributed Sensor Networks," Int. J. Distrib. Sensor Netw., vol. 2015, Dec. 2015, pp. 1-17.
  4. M. Cherniack et al., "Scalable Distributed Stream Processing," Conf. Innovative Data Syst. Res., Asilomar, CA, USA, Jan. 5-8, 2003, pp. 1-13.
  5. J. Soldatos et al., "OpenIot: Open Source Internet-of-Things in the Cloud," Interoperability and Open-Source Solutions for the Internet of Things, New York, USA: Springer International Publishing, 2015, pp. 13-25.
  6. H. Hromic et al., "Real Time Analysis of Sensor Data for the Internet of Things by Means of Clustering and Event Processing," IEEE Int. Conf. Commun., London, UK, June 8-12, 2015, pp. 685-691.
  7. B.B.P. Rao et al., "Cloud Computing for Internet of Things & Sensing Based Applications," Int. Conf. Sensing Technol., Kolkata, India, Dec. 18-21, 2012, pp. 347-380.
  8. T. Arampatzis, J. Lygeros, and S. Manesis., "A Survey of Applications of Wireless Sensors and Wireless Sensor Networks," Proc. IEEE Int. Symp., Mediterrean Conf. Contr. Autom. Intell. Contr., Limassol, Cyprus, June 27-29, 2005, pp. 719-724.
  9. S. Zhong et al., GearPump - Real-Time Streaming Engine Using Akka, Intel Big Data Technology Team, Dec. 2014.
  10. Typesafe Inc., (a). Akka Documentation: Release 2.0.2, Accessed Oct. 2012. http://doc.akka.io/docs/akka/2.0.2/Akka.pdf
  11. M. Zaharia et al., "Resilient Distributed Datasets: a Fault-Tolerant abstraction for in-Memory Cluster Computing," Proc. USENIX Conf. Netw. Syst. Des. Implementation, San Jose, CA, USA, Apr. 25-27, 2012, pp. 1-14.
  12. M. Zaharia et al., "Spark: Cluster Computing with Working Sets," Proc. USENIX Conf. Hot Topics Cloud Comput., Boston, MA, USA, June 22-25, 2010, p. 10.
  13. J.F. Goncalves, J.J.M. Mendes, and M.G.C. Resende, "A Genetic Algorithm for the Resource Constrained Multi-project Scheduling Problem," Eur. J. Operational Res., vol. 189, no. 3, Sept. 2008, pp. 1171-1190. https://doi.org/10.1016/j.ejor.2006.06.074
  14. J. Heinemann and D. Shah, "Location-Aware Scheduling with Minimal Infrastructure," Proc. Annu. Conf. USENIX Annu. Tech. Conf., San Diego, CA, USA, June 18-23, 2000, p. 11.
  15. F. Bonomi et al., "Fog Computing and Its Role in the Internet of Things," Proc. Edition MCC Workshop Mobile Cloud Comput., Hwlainki, Finland, Aug. 17, 2012, pp. 13-16.
  16. S. De Vito et al., "On Field Calibration of an Electronic Nose for Benzene Estimation in an Urban Pollution Monitoring Scenario," Sensors Actuators B: Chemical, vol. 129, no. 2, Feb. 2008, pp. 750-757. https://doi.org/10.1016/j.snb.2007.09.060
  17. J.L. Reyes-Ortiz et al., "Transition-Aware Human Activity Recognition Using Smartphones," Neurocomput., vol. 171, Jan. 2016, pp. 754-767. https://doi.org/10.1016/j.neucom.2015.07.085
  18. S. Shalev-Shwartz et al., "Pegasos: Primal Estimated Sub-gradient Solver for SVM," Math. Programming, vol. 127, no. 1, Mar. 2011, pp. 3-30. https://doi.org/10.1007/s10107-010-0420-4

Cited by

  1. 계층 구조에 기반을 둔 스마트 홈 시스템를 위한 스마트 센서 프레임워크의 설계 vol.17, pp.4, 2017, https://doi.org/10.7236/jiibc.2017.17.4.49
  2. Field microclimate monitoring system based on wireless sensor network vol.35, pp.2, 2017, https://doi.org/10.3233/jifs-169676
  3. Spatial Region Estimation for Autonomous CoT Clustering Using Hidden Markov Model vol.40, pp.1, 2017, https://doi.org/10.4218/etrij.18.0117.0142
  4. Design and Fabrication of a Thermoelectric Generator Based on BiTe Legs to power Wearable Device vol.73, pp.11, 2018, https://doi.org/10.3938/jkps.73.1760
  5. Predicting required licensed spectrum for the future considering big data growth vol.41, pp.2, 2017, https://doi.org/10.4218/etrij.2017-0273
  6. A probabilistic model for assigning queries at the edge vol.102, pp.4, 2020, https://doi.org/10.1007/s00607-019-00767-8
  7. Probabilistic Hesitant Fuzzy Methods for Prioritizing Distributed Stream Processing Frameworks for IoT Applications vol.2021, pp.None, 2017, https://doi.org/10.1155/2021/6655477