Browse > Article
http://dx.doi.org/10.3745/JIPS.03.0164

High Rate Denial-of-Service Attack Detection System for Cloud Environment Using Flume and Spark  

Gutierrez, Janitza Punto (Dept of Computer Science and Engineering, Seoul National University of Science and Technology)
Lee, Kilhung (Dept of Computer Science and Engineering, Seoul National University of Science and Technology)
Publication Information
Journal of Information Processing Systems / v.17, no.4, 2021 , pp. 675-689 More about this Journal
Abstract
Nowadays, cloud computing is being adopted for more organizations. However, since cloud computing has a virtualized, volatile, scalable and multi-tenancy distributed nature, it is challenging task to perform attack detection in the cloud following conventional processes. This work proposes a solution which aims to collect web server logs by using Flume and filter them through Spark Streaming in order to only consider suspicious data or data related to denial-of-service attacks and reduce the data that will be stored in Hadoop Distributed File System for posterior analysis with the frequent pattern (FP)-Growth algorithm. With the proposed system, we can address some of the difficulties in security for cloud environment, facilitating the data collection, reducing detection time and consequently enabling an almost real-time attack detection.
Keywords
Denial-of-Service; FP-Growth Pre-filtering; HDFS Spark Streaming; Web Log;
Citations & Related Records
연도 인용수 순위
  • Reference
1 M. T. Manavi, "Defense mechanisms against distributed denial of service attacks: a survey," Computers & Electrical Engineering, vol. 72, pp. 26-38, 2018.   DOI
2 T. C. Vance, N. Merati, C. Yang, and M. Yuan, "Cloud computing for ocean and atmospheric science," in Proceedings of 2016 MTS/IEEE Conference in Monterey (OCEAN), Monterey, CA, 2016, pp. 1-4.
3 O. Osanaiye, K. K. R. Choo, and M. Dlodlo, "Distributed denial of service (DDoS) resilience in cloud: review and conceptual cloud DDoS mitigation framework," Journal of Network and Computer Applications, vol. 67, pp. 147-165, 2016.   DOI
4 M. A. Khan, "A survey of security issues for cloud computing," Journal of Network and Computer Applications, vol. 71, pp. 11-29, 2016.   DOI
5 Apache Software Foundation, "Flume User Guide," 2021 [Online]. Available: https://flume.apache.org/FlumeUserGuide.html.
6 Apache Software Foundation, "Apache Hadoop Project," 2021 [Online]. Available: https://hadoop.apache.org/.
7 Apache Software Foundation, "HDFS Architecture Guide," 2018 [Online]. Available: https://hadoop.apache.org/docs/r1.2.1/hdfs_design.html.
8 Apache Software Foundation, "Frequent Pattern Mining: RDD-based API," 2018 [Online]. Available: https://spark.apache.org/docs/2.3.0/mllib-frequent-pattern-mining.html.
9 K. Sornalakshmi, "Detection of DoS attack and zero day threat with SIEM," in Proceedings of 2017 International Conference on Intelligent Computing and Control Systems (ICICCS), Madurai, India, 2017, pp. 1-7.
10 J. Han, J. Pei, and Y. Yin, "Mining frequent patterns without candidate generation," ACM SIGMOD Record, vol. 29, no. 2, pp. 1-12, 2000.   DOI
11 Fortinet, "FortiDDoS: Protection Profile Settings," 2019 [Online]. Available: https://help.fortinet.com/fddos/4-3-0/FortiDDoS/Managing_thresholds.htm.
12 J. Gera and B. P. Battula, "Detection of spoofed and non-spoofed DDoS attacks and discriminating them from flash crowds," EURASIP Journal on Information Security, vol. 2018, article no. 9, 2018. https://doi.org/10.1186/s13635-018-0079-6   DOI
13 R. Zhang and X. Xiao, "Study of danger-theory-based intrusion detection technology in virtual machines of cloud computing environment," Journal of Information Processing Systems, vol. 14, no. 1, pp. 239-251, 2018.   DOI
14 S. T. Zargar, J. Joshi, and D. Tipper, "A survey of defense mechanisms against distributed denial of service (DDoS) flooding attacks," IEEE Communications Surveys & Tutorials, vol. 15, no. 4, pp. 2046-2069, 2013.   DOI
15 Apache Software Foundation, "Spark Streaming Programming Guide," 2021 [Online]. Available: https://spark.apache.org/docs/latest/streaming-programming-guide.html.
16 M. H. Bhuyan, D. K. Bhattacharyya, and J. K. Kalita, "An empirical evaluation of information metrics for low-rate and high-rate DDoS attack detection," Pattern Recognition Letters, vol. 51, pp. 1-7, 2015.   DOI
17 H. Arshad, A. B. Jantan, and O. I. Abiodun, "Digital forensics: review of issues in scientific validation of digital evidence," Journal of Information Processing Systems, vol. 14, no. 2, pp. 346-376, 2018.   DOI
18 P. Dahiya and D. K. Srivastava, "Network intrusion detection in big dataset using spark," Procedia Computer Science, vol. 132, pp. 253-262, 2018.   DOI
19 M. Idhammad, K. Afdel, and M. Belouch, "Distributed intrusion detection system for cloud environments based on data mining techniques," Procedia Computer Science, vol. 127, pp. 35-41, 2018.   DOI
20 A. Abusitta, M. Bellaiche, and M. Dagenais, "An SVM-based framework for detecting DoS attacks in virtualized clouds under changing environment," Journal of Cloud Computing, vol. 7, article no. 9, 2018.
21 M. R. Mesbahi, A. M. Rahmani, and M. Hosseinzadeh, "Reliability and high availability in cloud computing environments: a reference roadmap," Human-centric Computing and Information Sciences, vol. 8, article no. 20, 2018. https://doi.org/10.1186/s13673-018-0143-8   DOI
22 A. Souri and R. Hosseini, "A state-of-the-art survey of malware detection approaches using data mining techniques," Human-centric Computing and Information Sciences, vol. 8, article no. 3, 2018. https://doi.org/10.1186/s13673-018-0125-x   DOI
23 J. H. Park, S. H. Na, J. Y. Park, E. N. Huh, C. W. Lee, and H. C. Kim, "A study on cloud forensics and challenges in SaaS application environment," in Proceedings of 2016 IEEE 18th International Conference on High Performance Computing and Communications; IEEE 14th International Conference on Smart City; IEEE 2nd International Conference on Data Science and Systems (HPCC/SmartCity/DSS), Sydney, Australia, 2016, pp. 734-740.
24 E. Morioka and M. S. Sharbaf, "Digital forensics research on cloud computing: an investigation of cloud forensics solutions," in Proceedings of 2016 IEEE Symposium on Technologies for Homeland Security (HST), Waltham, MA, 2016, pp. 1-6.
25 G. Donkal and G. K, Verma, "multimodal fusion based framework to reinforce IDS for securing big data environment using spark," Journal of Information Security and Applications, vol. 43, 1-11, 2018.   DOI
26 K. K. R. Choo, C. Esposito, and A. Castiglione, "Evidence and forensics in the cloud: challenges and future research directions," IEEE Cloud Computing, vol. 4, no. 3, pp. 14-19, 2017.   DOI
27 S. A. Hussain, M. Fatima, A. Saeed, I. Raza, and R. K. Shahzad, "Multilevel classification of security concerns in cloud computing," Applied Computing and Informatics, vol. 13, no. 1, pp. 57-65, 2017.   DOI
28 S. Nanda and R. A. Hansen, "Forensics as a service: three-tier architecture for cloud based forensic analysis," in Proceedings of 2016 15th International Symposium on Parallel and Distributed Computing (ISPDC), Fuzhou, China, 2016, pp. 178-183.
29 S. Zawoad and R. Hasan, "Trustworthy digital forensics in the cloud," Computer, vol. 49, no. 3, pp. 78-81, 2016.   DOI
30 G. Sibiya, H. S. Venter, and T. Fogwill, "Digital forensics in the cloud: the state of the art," in Proceedings of 2015 IST-Africa Conference, Lilongwe, Malawi, 2015, pp. 1-9.
31 A. Odebade, T. Welsh, S. Mthunzi, and E. Benkhelifa, "Mitigating anti-forensics in the cloud via resourcebased privacy preserving activity attribution," in Proceedings of 2017 Fourth International Conference on Software Defined Systems (SDS), Valencia, Spain, 2017, pp. 143-149.