Browse > Article
http://dx.doi.org/10.1109/JCN.2016.000110

A Probabilistic Sampling Method for Efficient Flow-based Analysis  

Jadidi, Zahra (School of Information and Communication Technology, Griffith University, QLD)
Muthukkumarasamy, Vallipuram (School of Information and Communication Technology, Griffith University, QLD)
Sithirasenan, Elankayer (School of Information and Communication Technology, Griffith University, QLD)
Singh, Kalvinder (School of Information and Communication Technology, Griffith University, QLD)
Publication Information
Journal of Communications and Networks / v.18, no.5, 2016 , pp. 818-825 More about this Journal
Abstract
Network management and anomaly detection are challenges in high-speed networks due to the high volume of packets that has to be analysed. Flow-based analysis is a scalable method which reduces the high volume of network traffic by dividing it into flows. As sampling methods are extensively used in flow generators such as NetFlow, the impact of sampling on the performance of flow-based analysis needs to be investigated. Monitoring using sampled traffic is a well-studied research area, however, the impact of sampling on flow-based anomaly detection is a poorly researched area. This paper investigates flow sampling methods and shows that these methods have negative impact on flow-based anomaly detection. Therefore, we propose an efficient probabilistic flow sampling method that can preserve flow traffic distribution. The proposed sampling method takes into account two flow features: Destination IP address and octet. The destination IP addresses are sampled based on the number of received bytes. Our method provides efficient sampled traffic which has the required traffic features for both flow-based anomaly detection and monitoring. The proposed sampling method is evaluated using a number of generated flow-based datasets. The results show improvement in preserved malicious flows.
Keywords
Anomaly detection; destination IP address; flow-based analysis; monitoring; octet; probabilistic sampling;
Citations & Related Records
연도 인용수 순위
  • Reference
1 The CAIDA UCSD Anonymized Internet Traces 2013, [Online]. Available: http://www.caida.org/data/passive/passive_2013_dataset.xml
2 The CAIDA UCSD Anonymized Internet Traces 2012, [Online]. Available: http://www.caida.org/data/passive/passive_2012_dataset.xml
3 A. Sperotto et al., "An overview of IP flow-based intrusion detection," IEEE Commun. Surveys Tuts., vol. 12, pp. 343-356, 2010.   DOI
4 Z. Jadidi, V. Muthukkumarasamy, and E. Sithirasenan, "Metaheuristic algorithms based flow anomaly detector," in Proc. APCC, 2013, pp. 717-722.
5 Z. Jadidi, V. Muthukkumarasamy, E. Sithirasenan, and M. Sheikhan, "Flow-based anomaly detection using neural network optimized with GSA algorithm," in Proc. IEEE NFSP, 2013, pp.76-81.
6 M. Sheikhan and Z. Jadidi, "Flow-based anomaly detection in high-speed links using modified GSA-optimized neural network," Neural Comput. Appl., vol. 24, pp. 599-611, 2014.   DOI
7 P. Gogoi, D. Bhattacharyya, B. Borah, and J. K. Kalita, "MLH-IDS: A multi-level hybrid intrusion detection method," The Computer Journal, vol. 57, pp. 602-623, 2014.   DOI
8 N. Hohn and D. Veitch, "Inverting sampled traffic," IEEE/ACM Trans. Netw., vol. 14, pp. 68-80, 2006.   DOI
9 N. Duffield, C. Lund, and M. Thorup, "Estimating flow distributions from sampled flow statistics," IEEE/ACM Trans. Netw., vol. 13, pp. 933-946, 2005.   DOI
10 B.-Y. Choi, J. Park, and Z.-L. Zhang, "Adaptive packet sampling for accurate and scalable flow measurement," in Proc. IEEE GLOBECOM, 2004, pp. 1448-1452.
11 N. Duffield, C. Lund, and M. Thorup, "Properties and prediction of flow statistics from sampled packet streams," in Proc. ACM SIGCOMM, 2002, pp. 159-171.
12 B. Li, J. Springer, G. Bebis, and M. Hadi Gunes, "A survey of network flow applications," J. Netw. Comput. Appl., vol. 36, pp. 567-581, 2013.   DOI
13 J. Mai, C.-N. Chuah, A. Sridharan, T. Ye, and H. Zang, "Is sampled data sufficient for anomaly detection?," in Proc. ACM SIGCOMM, 2006, pp. 165-176.
14 P. Winter, E. Hermann, and M. Zeilinger, "Inductive intrusion detection in flow-based network data using one-class support vector machines," in Proc. IFIP NTMS, 2011, pp. 1-5.
15 A. Sperotto and A. Pras, "Flow-based intrusion detection," in Proc. IFIP/IEEE IM, 2011, pp. 958-963.
16 K. Bartos and M. Rehak, "Towards efficient flow sampling technique for anomaly detection," in Proc. TMA, 2012, pp. 93-106.
17 J. Mai, A. Sridharan, C.-N. Chuah, H. Zang, and T. Ye, "Impact of packet sampling on portscan detection," J. Sel. Areas Commun., vol. 24, pp. 2285-2298, 2006.   DOI
18 The CAIDA UCSD "DDoS Attack 2007" Dataset, [Online]. Available: http://www.caida.org/data/passive/ddos-200708nct04_dataset.xml
19 G. Androulidakis, V. Chatzigiannakis, and S. Papavassiliou, "Network anomaly detection and classification via opportunistic sampling," IEEE Netw., vol. 23, pp. 6-12, 2009.
20 C. Estan and G. Varghese, "New directions in traffic measurement and accounting," in Proc. ACM SIGCOMM, vol. 32, 2002.
21 Z. Jadidi, V. Muthukkumarasamy, E. Sithirasenan, and K. Singh, "Intelligent sampling using an optimized neural network," J. Netw., vol. 11, pp. 16-27, 2016.
22 G. Androulidakis and S. Papavassiliou, "Improving network anomaly detection via selective flow-based sampling," IET Commun., vol. 2, pp. 399-409, 2008.   DOI
23 V. Carela-Espanol, P. Barlet-Ros, A. Cabellos-Aparicio, and J. Sole-Pareta, "Analysis of the impact of sampling on NetFlow traffic classification," Computer Netw., vol. 55, pp. 1083-1099, 2011.   DOI
24 Z. Jadidi, V. Muthukkumarasamy, E. Sithirasenan, and K. Singh, "Performance of flow-based anomaly detection in sampled traffic," J. Netw., vol. 10, pp. 512-520, 2016.
25 Q. A. Tran, F. Jiang, and J. Hu, "A real-time netflow-based intrusion detection system with improved BBNN and high-frequency field programmable gate arrays," in Proc. IEEE TrustCom, 2012, pp. 201-208.
26 [Online]. Available: http://www.mindrot.org/projects/softflowd/, as of June 2014.
27 [Online]. Available: http://www.mindrot.org/projects/flowd/, as of June 2014.
28 T. Qin, X. Guan, W. Li, P. Wang, and M. Zhu, "A new connection degree calculation and measurement method for large scale network monitoring," J. Netw. Comput. Appl., vol. 41, pp. 15-26, 2014.   DOI
29 I. Paredes-Oliva, P. Barlet-Ros, and J. Sole-Pareta, "Scan detection under sampling: A new perspective," Computer, vol. 46, pp. 38-44, 2013.
30 G. Androulidakis and S. Papavassiliou, "Intelligent flow-based sampling for effective network anomaly detection," in Proc. IEEE GLOBECOM, 2007, pp. 1948-1953.