KSII Transactions on Internet and Information Systems (TIIS)

Korean Society for Internet Information (한국인터넷정보학회)
권호 표지
DOI QR코드

DOI QR Code

A Novel Compressed Sensing Technique for Traffic Matrix Estimation of Software Defined Cloud Networks

  • Qazi, Sameer (Graduate School of Science and Engineering (GSSE), PAF Karachi Institute of Economics and Technology (PAF-KIET)) ;
  • Atif, Syed Muhammad (Graduate School of Science and Engineering (GSSE), PAF Karachi Institute of Economics and Technology (PAF-KIET)) ;
  • Kadri, Muhammad Bilal (Graduate School of Science and Engineering (GSSE), PAF Karachi Institute of Economics and Technology (PAF-KIET))
  • Received : 2017.08.09
  • Accepted : 2018.05.07
  • Published : 2018.10.31
Download PDF
⟨ Previous Next ⟩

Abstract

Traffic Matrix estimation has always caught attention from researchers for better network management and future planning. With the advent of high traffic loads due to Cloud Computing platforms and Software Defined Networking based tunable routing and traffic management algorithms on the Internet, it is more necessary as ever to be able to predict current and future traffic volumes on the network. For large networks such origin-destination traffic prediction problem takes the form of a large under- constrained and under-determined system of equations with a dynamic measurement matrix. Previously, the researchers had relied on the assumption that the measurement (routing) matrix is stationary due to which the schemes are not suitable for modern software defined networks. In this work, we present our Compressed Sensing with Dynamic Model Estimation (CS-DME) architecture suitable for modern software defined networks. Our main contributions are: (1) we formulate an approach in which measurement matrix in the compressed sensing scheme can be accurately and dynamically estimated through a reformulation of the problem based on traffic demands. (2) We show that the problem formulation using a dynamic measurement matrix based on instantaneous traffic demands may be used instead of a stationary binary routing matrix which is more suitable to modern Software Defined Networks that are constantly evolving in terms of routing by inspection of its Eigen Spectrum using two real world datasets. (3) We also show that linking this compressed measurement matrix dynamically with the measured parameters can lead to acceptable estimation of Origin Destination (OD) Traffic flows with marginally poor results with other state-of-art schemes relying on fixed measurement matrices. (4) Furthermore, using this compressed reformulated problem, a new strategy for selection of vantage points for most efficient traffic matrix estimation is also presented through a secondary compression technique based on subset of link measurements. Experimental evaluation of proposed technique using real world datasets Abilene and GEANT shows that the technique is practical to be used in modern software defined networks. Further, the performance of the scheme is compared with recent state of the art techniques proposed in research literature.

Keywords

References

  1. Morteza Mardani and Georgios B. Giannakis, "Estimating Traffic and Anomaly Maps viaNetworkTomography", in IEEE/ACM Transactions on Networking ,Volume: 24, Issue:3, pp:1533-1547, 2016. https://doi.org/10.1109/TNET.2015.2417809
  2. LaisenNie, "A Novel Network Tomography Approach for Traffic Matrix Estimation Problem in Large-Scale IP BackboneNetworks", in Proc. of Proceedings 2015 International Conference on Computer Science and Mechanical Automation (CSMA),Wuhan, Hubei, China, pp:97-101, October 2015.
  3. Linbo He, Lili Yan and Jun Zhao, "Research of Traffic Matrix Estimation in IP Networks," in Proc. of Proceedings 2011 International Conference on Internet Technology and Applications (iTAP), Wuhan, China, August 2011.
  4. Mathew Roughan, "Simplifying the synthesis of Internet traffic matrices," ACM Computer Communication. Review, Volume: 35, Issue: 5, pp: 93-96, 2005.
  5. Yin Zhang, Zihui Ge, Albert Greenberg, and Mathew Roughan. "Network anomography," in Proc. of ACM SIGCOMM Internet Measurement Conference (IMC), Berkeley, California, USA, October 2005.
  6. Augustin Soule, Antonio Nucci, Rene Cruz, Emilio Leonardi, and Nina Taft. "How to identify and estimate the largest traffic matrix elements in a dynamic environment," SIGMETRICS Performance Evaluation Review, 32(1), 73-84, June 2004. https://doi.org/10.1145/1012888.1005698
  7. Neshat Etemadi Rad; Yariv Ephraim; Brian L. Mark, "Delay Network Tomography Using a Partially Observable Bivariate Markov Chain," IEEE/ACM Transactions on Networking, 2016.
  8. Nick Johnson, John Thompson, Steve Laughlin and Francisco J. Garcia, "A comparison of some bottleneck-link detection methods for network tomography," in Proc. of 2009 17th European Signal Processing Conference, pp: 1660 - 1664, 2009.
  9. Yan Chen, David Bindel and Randy H. Katz, "Tomography-based overlay network monitoring," in Proc. of Proceedings of the 3rd ACM SIGCOMM Conference on Internet Measurement (IMC'03:), Miami Beach, Florida, USA, pp: 216-231, October 27-29, 2003.
  10. Konstantinos (Kostas) Christodoulopoulos, NocolaSambo and Emmanuel Varvarigos, "Exploiting network kriging for fault localization," in Proc. of Proceedings of 2016 Optical Fiber Communications Conference and Exhibition (OFC), Anaheim, California, USA, March 2016
  11. David .B. Chua, Eric D. Kolaczyk and Mark Crovella, "Network Kriging," IEEE Journal of Selected Areas in Communications, Vol. 24, Issue 12, pp: 2263-2272, 2006. https://doi.org/10.1109/JSAC.2006.884025
  12. Ketan Rajawat, Emiliano Dall'Anese and Georgios B. Giannakis, "Dynamic network kriging," in Proc. of Proceedings of IEEE Statistical Signal Processing Workshop (SSP), Ann Arbor, Michigan, USA, pp: 121-124, August 2012.
  13. KetanRajawat, Emiliano Dall'Anese and Georgios B. Giannakis, "Dynamic Network Delay Cartography," IEEE Transactions on Information Theory, Volume: 60, Issue:5, pp:2910-2920, 2014. https://doi.org/10.1109/TIT.2014.2311802
  14. Mark Coates, Yvan Pointurier and Michael Rabbat, "Compressed network monitoring for ip and all optical networks," in Proc. of IMC'07: Proceedings of the 7th ACM SIGCOMM Conference on Internet Measurement, San Diego, California, USA, pp. 241-252, October, 2007.
  15. Vidarshana W. Bandara; Anura P. Jayasumana; Rick Whitner, "An adaptive compressive sensing scheme for network tomography based fault localization", IEEE International Conference on Communications (ICC), Sydney, Australia, pp:1290-1295, June 2014.
  16. L. Nie and D. Jiang, "A compressive sensing-based network tomography approach to estimating origin-destination flow traffic in large-scale backbone networks," International Journal of Communication Systems, Volume 28, Issue 5, pp: 889-900, 2015 https://doi.org/10.1002/dac.2713
  17. Awnish Kumar, Vijaya.V. Saradhi and T. Venkatesh, "Compressive Sensing of Internet Traffic Matrices using CUR Decomposition," in Proc. of Proceedings of the ACM, 19th International Conference on Distributed Computing and Networking (ICDCN '18), Varanasi, India, January 4-7, 2018.
  18. Augustin Soule, AnukoolLakhina,NinaTaft and KonstantinaPapagiannaki, "Traffic Matrices: Balancing Measurements, Inference and Modeling," in Proc. of Proceedings of SIGMETRICS'05, Banff,Alberta,Canada June 6-10, 2005.
  19. Paul Tune and Matthew Roughan, "Spatiotemporal Traffic Matrix Synthesis," in Proc. of Proceedings of the 2015 ACM Conference on Special Interest Group on Data Communication (ACM SIGCOMM), London, UK, pp: 579-592, August 2015.
  20. Alberto Medina, Nina Taft, Kave Salamatian, Supratik Bhattacharyya and Christophe Diot, "Traffic Matrix Estimation: Existing Techniques and New Directions," in Proc. of Proceedings of 2002 ACM Conference on Special Interest Group on Data Communication (SIGCOMM'02), Pittsburgh, Pennsylvania, USA, pp: 161-174, August 19-23, 2002.
  21. Claudia Tebaldi and Mike West, "Bayesian Inference of Network Traffic Using Link Count Data," Journal of the American Statistical Association, Volume: 93, Issue: 442, pp: 557-573, June 1998. https://doi.org/10.1080/01621459.1998.10473707
  22. Y. Vardi, "Network Tomography: Estimating Source-Destination Traffic Intensities from Link Data," Journal of the American Statistical Association., Vol. 91, No. 433, pp: 365-377, 1996. https://doi.org/10.1080/01621459.1996.10476697
  23. Laisen Nie, Dingde Jiang and .Zhihan Lv, "Modeling Network Traffic for Traffic Matrix Estimation and Anomaly Detection based on Bayesian Network in Cloud Computing," Annals Telecommunications, Issue 5-6, 2017.
  24. XiaoBo Fan and Xingming Li, "Network Tomography via Sparse Bayesian Learning," IEEE Communications Letters, Vol 21, Issue 4, pp: 781-184, 2017. https://doi.org/10.1109/LCOMM.2017.2649494
  25. Ting He, Athanasios Gkelias, Liang Ma, Kin K. Leung, Ananthram Swami and Don Towsley, "Robust and Efficient Monitor Placement for Network Tomography in Dynamic Networks," IEEE/ACM Transactions on Networking, Vol: 25, Issue 3, pp: 1732-1745, 2017. https://doi.org/10.1109/TNET.2016.2642185
  26. Wei Ren and Wei Dong, "Robust network tomography: K-Identifiability and monitor assignment," in Proc. of IEEE INFOCOM 2016 - The 35th Annual IEEE International Conference on Computer Communications, San Francisco, CA, April, 2016.
  27. Ting He; Liang Ma; Athanasios Gkelias; Kin K. Leung; Ananthram Swami; Don Towsley, "Robust monitor placement for network tomography in dynamic networks," in Proc. of IEEE INFOCOM 2016 - The 35th Annual IEEE International Conference on Computer Communications, San Francisco, CA, April, 2016.
  28. Wei Dong; Yi Gao; Wenbin Wu; Jiajun Bu; Chun Chen; Xiang-Yang Li, "Optimal Monitor Assignment for Preferential Link Tomography in Communication Networks," IEEE/ACM Transactions on Networking, Vol: 25, Issue 1, pp: 210-223, 2016. https://doi.org/10.1109/TNET.2016.2581176
  29. Nagao Ogino; Takeshi Kitahara; Shin'ichi Arakawa; Go Hasegawa; Masayuki Murata, "Decentralized boolean network tomography based in network partitioning," in Proc. of NOMS 2016 - 2016 IEEE/IFIP Network Operations and Management Symposium, Istanbul , Turkey, pp: 162 - 170, April 2016.
  30. Masaki Mukamoto; Takahiro Matsuda; Shinsuke Hara; Kenichi Takizawa; Fumie Ono; RyuMiura, "Adaptive boolean network tomography for link failure detection," in Proc. of 2015 IFIP/IEEE International Symposium on Integrated Network Management (IM), Ottawa, Canada, Pages: 646-651, May 2015.
  31. Sameer Qazi and Tim Moors, "On the Impact of Routing Matrix Inconsistencies on Statistical Path Monitoring in Overlay Networks," Computer Networks, Vol: 54, Issue: 10, pp: 1554-1572, 2010. https://doi.org/10.1016/j.comnet.2009.11.006
  32. Sameer Qazi, Rashida Memon and Adnan A. Farooqui, "Evaluation of Performance of Genetic Algorithms for Network Tomography," Journal of Internet Technology, Volume: 16, Issue: 1, pp: 75-85, 2015.
  33. H. Tian, Y. Sang, H. Shen, C. Zhou, "Probability Model Based Network Traffic Matrix Estimation," Computer Science and Information Systems, Vol. 11, Issue 1, pp: 309-320, 2014. https://doi.org/10.2298/CSIS130212010T
  34. G.H. Golub, C.F.V. Loan, "Matrix Computations," third ed., John Hopkins, 1996.

Cited by

  1. Design and Implementation of a Robust Convolutional Neural Network-Based Traffic Matrix Estimator for Cloud Networks vol.2021, pp.None, 2018, https://doi.org/10.1155/2021/1039613