KSII Transactions on Internet and Information Systems (TIIS)

Korean Society for Internet Information (한국인터넷정보학회)
권호 표지
DOI QR코드

DOI QR Code

Auto Regulated Data Provisioning Scheme with Adaptive Buffer Resilience Control on Federated Clouds

  • Received : 2016.08.09
  • Accepted : 2016.10.17
  • Published : 2016.11.30
Download PDF
⟨ Previous Next ⟩

Abstract

On large-scale data analysis platforms deployed on cloud infrastructures over the Internet, the instability of the data transfer time and the dynamics of the processing rate require a more sophisticated data distribution scheme which maximizes parallel efficiency by achieving the balanced load among participated computing elements and by eliminating the idle time of each computing element. In particular, under the constraints that have the real-time and limited data buffer (in-memory storage) are given, it needs more controllable mechanism to prevent both the overflow and the underflow of the finite buffer. In this paper, we propose an auto regulated data provisioning model based on receiver-driven data pull model. On this model, we provide a synchronized data replenishment mechanism that implicitly avoids the data buffer overflow as well as explicitly regulates the data buffer underflow by adequately adjusting the buffer resilience. To estimate the optimal size of buffer resilience, we exploits an adaptive buffer resilience control scheme that minimizes both data buffer space and idle time of the processing elements based on directly measured sample path analysis. The simulation results show that the proposed scheme provides allowable approximation compared to the numerical results. Also, it is suitably efficient to apply for such a dynamic environment that cannot postulate the stochastic characteristic for the data transfer time, the data processing rate, or even an environment where the fluctuation of the both is presented.

Keywords

References

  1. A. Jacobs, The pathologies of big data, Commun. ACM, vol. 52, pp. 36-44, 2009.
  2. J. Dean, S. Ghemawat, "Mapreduce: simplified data processing on large clusters," Commun. ACM 51, pp. 107-113, 2008.
  3. J. Andreeva, S. Campana, F. Fanzago, J. Herrala, "High-energy physics on the grid: the atlas and cms experience," Journal of Grid Computing, vol. 6, no. 1, pp. 3-13, 2008. https://doi.org/10.1007/s10723-007-9087-3
  4. Erlich, Yaniv, "A vision for ubiquitous sequencing," Genome Research, vol. 25, no. 10, pp 1411-1416, 2015. https://doi.org/10.1101/gr.191692.115
  5. J. Holler, V. Tsiatsis, C. Mulligan, S. Karnouskos, S. Avesand, "From Machine-to-Machine to the Internet of Things: Introduction to a New Age of Intelligence," Elsevier, 2014.
  6. H. Qi and A. Gani, "Research on mobile cloud computing: Review, trend and perspectives," Second International Conference on Digital Information and Communication Technology and it's Applications, pp. 195-202, 2012.
  7. Khan, A.N., Kiah, M.L.M., Ali, M. et al., "BSS: block-based sharing scheme for secure data storage services in mobile cloud environment," Journal of Supercomputing, vol. 70, no 2, pp 946-976, 2014. https://doi.org/10.1007/s11227-014-1269-8
  8. J. Gascon-Samson, F. P. Garcia, B. Kemme and J. Kienzle, "Dynamoth: A Scalable Pub/Sub Middleware for Latency-Constrained Applications in the Cloud," in Proc. of IEEE 35th International Conference on Distributed Computing Systems (ICDCS), pp. 486-496, 2015. doi: 10.1109/ICDCS.2015.56.
  9. D. Huang, A. Jaikar, G. Kim, Y. Kim, and S. Noh, "A Self Synchronization Mechanism in a Federated Cloud," International Journal of Software Engineering and Its Applications, vol. 10, no. 1, pp. 233-240, 2016.
  10. D. G. O. Veeravalli Bharadwaj, Thomas G. Robertazzi, "Scheduling Divisible Loads in Parallel and Distributed Systems, "IEEE Computer Society Press, 1996.
  11. Y.Wang, H. Chen, B.Wang, J. M. Xu, H. Lei, "A scalable queuing service based on an in-memory data grid," in Proc. of 2010 IEEE 7th International Conference on e-Business Engineering (ICEBE), pp. 236-243, 2010.
  12. B. Veeravalli, J. Yao, "Divisible load scheduling strategies on distributed multi-level tree networks with communication delays and buffer constraints," Computer Communications, vol. 27, no. 1, 93-110, 2004. https://doi.org/10.1016/S0140-3664(03)00181-6
  13. A. Shokripour, M. Othman, "Categorizing Researches about DLT in Ten Groups," International Association of Computer Science and Information Technology, pp. 45-49, 2009.
  14. Y. Yang, H. Casanova, M. Drozdowski, M. Lawenda, A. Legrand, "On the Complexity of Multi-Round Divisible Load Scheduling," Research Report RR-6096, 2007.
  15. A. R, J. Agarkhed, "Evaluation of Auto Scaling and Load Balancing Features in Cloud," International Journal of Computer Applications, vol.117 no.6, pp. 30-33, 2015. https://doi.org/10.5120/20561-2949
  16. B. Javadi, R. K. Thulasiram, R. Buyya, "Characterizing spot price dynamics in public cloud environments," Future Generation Computer Systems, vol 29, no 4, pp. 988-999, 2013. https://doi.org/10.1016/j.future.2012.06.012
  17. J. Hwang, and J. Yoo, "FaST: Fine-grained and Scalable TCP for Cloud Data Center Networks," KSII Transactions on Internet and Information Systems(TIIS), vol. 8, no. 3, pp.762-777, 2014. https://doi.org/10.3837/tiis.2014.03.003
  18. W. Allcock, J. Bresnahan, R. Kettimuthu, M. Link, C. Dumitrescu, I. Raicu, I. Foster, "The globus striped ridftp framework and server," in Proc. of the 2005 ACM/IEEE conference on Supercomputing, SC '05, IEEE Computer Society, pp. 54-65, 2005.
  19. L. Ramakrishnan, C. Guok, K. Jackson, E. Kissel, D. M. Swany, D. Agarwal, "On-demand overlay networks for large scientific data transfers," in Proc. of 2010 10th IEEE/ACM International Conference on Cluster, Cloud and Grid Computing (CCGrid), pp. 359-367, 2010.
  20. D. Yin, E. Yildirim, S. Kulasekaran, B. Ross, T. Kosar, "A data throughput prediction and optimization service for widely distributed many task computing," IEEE Transactions on Parallel and Distributed Systems, vol.22, no.6, pp. 899-909, 2011. https://doi.org/10.1109/TPDS.2010.187
  21. V. Garonne, A. Tsaregorodtsev, E. Caron, "A study of meta-scheduling architectures for high throughput computing: Pull versus push," International Symposium on Parallel and Distributed Computing, pp. 226-233, 2005.
  22. J.T. Moscicki, "Diane - distributed analysis environment for grid-enabled simulation and analysis of physics data," in Proc. of IEEE Nuclear Science Symposium Conference Record vod. 3, pp. 1617-1620, 2003.
  23. A. Tsaregorodtsev, V. Garonne, I. Stokes-Rees, "Dirac: A scalable lightweight architecture for high throughput computing," in Proc. of GRID '04 Proceedings of the 5th IEEE/ACM InternationalWorkshop on Grid Computing, pp. 19-25, 2004.
  24. J. Diaz-Montes; M. Diaz-Granados; M. Zou; S. Tao; M. Parashar, "Supporting Data-intensive Workflows in Software-defined Federated Multi-Clouds," in Proc. of IEEE Transactions on Cloud Computing, vol.PP, no.99, pp.1-1 Sep, 2015.
  25. Dandan Wang, Yang Yang and Zhenqiang Mi, "QoS-Based and Network-Aware Web Service Composition across Cloud Datacenters," KSII Transactions on Internet and Information Systems(TIIS ), vol. 9, no 3, pp.971-989, Mar. 2015. https://doi.org/10.3837/tiis.2015.03.008
  26. Haoran Ji, Weidong Bao, Xiaomin Zhu and Wenhua Xiao, "Topology-based Workflow Scheduling in Commercial Clouds," KSII Transactions on Internet and Information Systems(TIIS), vol.9, No. 11 pp.4311-4330, Nov. 2015. https://doi.org/10.3837/tiis.2015.11.003
  27. B. Kim, C.-H. Youn, "A performance evaluation of the synchronized provisioning with adaptive buffer resilience scheme over grid networks," IEEE Communications Letters, vol. 16, no. 4, Apr, 2012.
  28. C. Cassandras, Y. Wardi, B. Melamed, G. Sun, C. Panayiotou, "Perturbation analysis for online control and optimization of stochastic fluid models," IEEE Transactions on Automatic Control, vol. 47, no. 8, pp. 1234 - 1248, 2002. https://doi.org/10.1109/TAC.2002.800739
  29. Y. Zhao, B. Melamed, "Ipa derivatives for make-to-stock production inventory systems with backorders," Methodology and Computing in Applied Probability vol.8, pp. 191-222, 2006. https://doi.org/10.1007/s11009-006-8548-7
  30. F. Howell, R. Mcnab, simjava: a discrete event simulation library for java, pp. 51-56, 1998.
  31. J. A. Buzacott, J. G. Shanthikumar, Stochastic Models of Manufacturing Systems, Prentice Hall, 1993.