Journal of Information Processing Systems

  • 제17권4호
  • /
  • Pages.675-689
  • /
  • 2021
  • /
  • 1976-913X(pISSN)
  • /
  • 2092-805X(eISSN)
한국정보처리학회 (Korea Information Processing Society)
권호 표지
DOI QR코드

DOI QR Code

High Rate Denial-of-Service Attack Detection System for Cloud Environment Using Flume and Spark

  • Gutierrez, Janitza Punto (Dept of Computer Science and Engineering, Seoul National University of Science and Technology) ;
  • Lee, Kilhung (Dept of Computer Science and Engineering, Seoul National University of Science and Technology)
  • 투고 : 2019.03.27
  • 심사 : 2020.04.14
  • 발행 : 2021.08.31
PDF 다운로드
⟨ 이전 논문 다음 논문 ⟩

초록

Nowadays, cloud computing is being adopted for more organizations. However, since cloud computing has a virtualized, volatile, scalable and multi-tenancy distributed nature, it is challenging task to perform attack detection in the cloud following conventional processes. This work proposes a solution which aims to collect web server logs by using Flume and filter them through Spark Streaming in order to only consider suspicious data or data related to denial-of-service attacks and reduce the data that will be stored in Hadoop Distributed File System for posterior analysis with the frequent pattern (FP)-Growth algorithm. With the proposed system, we can address some of the difficulties in security for cloud environment, facilitating the data collection, reducing detection time and consequently enabling an almost real-time attack detection.

키워드

과제정보

This study was supported by the Research Program funded by the Seoul National University of Science and Technology (SeoulTech).

참고문헌

  1. H. Arshad, A. B. Jantan, and O. I. Abiodun, "Digital forensics: review of issues in scientific validation of digital evidence," Journal of Information Processing Systems, vol. 14, no. 2, pp. 346-376, 2018. https://doi.org/10.3745/JIPS.03.0095
  2. P. Dahiya and D. K. Srivastava, "Network intrusion detection in big dataset using spark," Procedia Computer Science, vol. 132, pp. 253-262, 2018. https://doi.org/10.1016/j.procs.2018.05.169
  3. M. Idhammad, K. Afdel, and M. Belouch, "Distributed intrusion detection system for cloud environments based on data mining techniques," Procedia Computer Science, vol. 127, pp. 35-41, 2018. https://doi.org/10.1016/j.procs.2018.01.095
  4. M. R. Mesbahi, A. M. Rahmani, and M. Hosseinzadeh, "Reliability and high availability in cloud computing environments: a reference roadmap," Human-centric Computing and Information Sciences, vol. 8, article no. 20, 2018. https://doi.org/10.1186/s13673-018-0143-8
  5. A. Abusitta, M. Bellaiche, and M. Dagenais, "An SVM-based framework for detecting DoS attacks in virtualized clouds under changing environment," Journal of Cloud Computing, vol. 7, article no. 9, 2018.
  6. J. Gera and B. P. Battula, "Detection of spoofed and non-spoofed DDoS attacks and discriminating them from flash crowds," EURASIP Journal on Information Security, vol. 2018, article no. 9, 2018. https://doi.org/10.1186/s13635-018-0079-6
  7. R. Zhang and X. Xiao, "Study of danger-theory-based intrusion detection technology in virtual machines of cloud computing environment," Journal of Information Processing Systems, vol. 14, no. 1, pp. 239-251, 2018. https://doi.org/10.3745/JIPS.03.0089
  8. A. Souri and R. Hosseini, "A state-of-the-art survey of malware detection approaches using data mining techniques," Human-centric Computing and Information Sciences, vol. 8, article no. 3, 2018. https://doi.org/10.1186/s13673-018-0125-x
  9. G. Donkal and G. K, Verma, "multimodal fusion based framework to reinforce IDS for securing big data environment using spark," Journal of Information Security and Applications, vol. 43, 1-11, 2018. https://doi.org/10.1016/j.jisa.2018.10.001
  10. K. K. R. Choo, C. Esposito, and A. Castiglione, "Evidence and forensics in the cloud: challenges and future research directions," IEEE Cloud Computing, vol. 4, no. 3, pp. 14-19, 2017. https://doi.org/10.1109/MCC.2017.39
  11. S. A. Hussain, M. Fatima, A. Saeed, I. Raza, and R. K. Shahzad, "Multilevel classification of security concerns in cloud computing," Applied Computing and Informatics, vol. 13, no. 1, pp. 57-65, 2017. https://doi.org/10.1016/j.aci.2016.03.001
  12. E. Morioka and M. S. Sharbaf, "Digital forensics research on cloud computing: an investigation of cloud forensics solutions," in Proceedings of 2016 IEEE Symposium on Technologies for Homeland Security (HST), Waltham, MA, 2016, pp. 1-6.
  13. S. Nanda and R. A. Hansen, "Forensics as a service: three-tier architecture for cloud based forensic analysis," in Proceedings of 2016 15th International Symposium on Parallel and Distributed Computing (ISPDC), Fuzhou, China, 2016, pp. 178-183.
  14. J. H. Park, S. H. Na, J. Y. Park, E. N. Huh, C. W. Lee, and H. C. Kim, "A study on cloud forensics and challenges in SaaS application environment," in Proceedings of 2016 IEEE 18th International Conference on High Performance Computing and Communications; IEEE 14th International Conference on Smart City; IEEE 2nd International Conference on Data Science and Systems (HPCC/SmartCity/DSS), Sydney, Australia, 2016, pp. 734-740.
  15. G. Sibiya, H. S. Venter, and T. Fogwill, "Digital forensics in the cloud: the state of the art," in Proceedings of 2015 IST-Africa Conference, Lilongwe, Malawi, 2015, pp. 1-9.
  16. S. Zawoad and R. Hasan, "Trustworthy digital forensics in the cloud," Computer, vol. 49, no. 3, pp. 78-81, 2016. https://doi.org/10.1109/MC.2016.89
  17. A. Odebade, T. Welsh, S. Mthunzi, and E. Benkhelifa, "Mitigating anti-forensics in the cloud via resourcebased privacy preserving activity attribution," in Proceedings of 2017 Fourth International Conference on Software Defined Systems (SDS), Valencia, Spain, 2017, pp. 143-149.
  18. S. T. Zargar, J. Joshi, and D. Tipper, "A survey of defense mechanisms against distributed denial of service (DDoS) flooding attacks," IEEE Communications Surveys & Tutorials, vol. 15, no. 4, pp. 2046-2069, 2013. https://doi.org/10.1109/SURV.2013.031413.00127
  19. M. H. Bhuyan, D. K. Bhattacharyya, and J. K. Kalita, "An empirical evaluation of information metrics for low-rate and high-rate DDoS attack detection," Pattern Recognition Letters, vol. 51, pp. 1-7, 2015. https://doi.org/10.1016/j.patrec.2014.07.019
  20. O. Osanaiye, K. K. R. Choo, and M. Dlodlo, "Distributed denial of service (DDoS) resilience in cloud: review and conceptual cloud DDoS mitigation framework," Journal of Network and Computer Applications, vol. 67, pp. 147-165, 2016. https://doi.org/10.1016/j.jnca.2016.01.001
  21. M. T. Manavi, "Defense mechanisms against distributed denial of service attacks: a survey," Computers & Electrical Engineering, vol. 72, pp. 26-38, 2018. https://doi.org/10.1016/j.compeleceng.2018.09.001
  22. T. C. Vance, N. Merati, C. Yang, and M. Yuan, "Cloud computing for ocean and atmospheric science," in Proceedings of 2016 MTS/IEEE Conference in Monterey (OCEAN), Monterey, CA, 2016, pp. 1-4.
  23. M. A. Khan, "A survey of security issues for cloud computing," Journal of Network and Computer Applications, vol. 71, pp. 11-29, 2016. https://doi.org/10.1016/j.jnca.2016.05.010
  24. Apache Software Foundation, "Flume User Guide," 2021 [Online]. Available: https://flume.apache.org/FlumeUserGuide.html.
  25. Apache Software Foundation, "Apache Hadoop Project," 2021 [Online]. Available: https://hadoop.apache.org/.
  26. Apache Software Foundation, "HDFS Architecture Guide," 2018 [Online]. Available: https://hadoop.apache.org/docs/r1.2.1/hdfs_design.html.
  27. Apache Software Foundation, "Spark Streaming Programming Guide," 2021 [Online]. Available: https://spark.apache.org/docs/latest/streaming-programming-guide.html.
  28. Apache Software Foundation, "Frequent Pattern Mining: RDD-based API," 2018 [Online]. Available: https://spark.apache.org/docs/2.3.0/mllib-frequent-pattern-mining.html.
  29. J. Han, J. Pei, and Y. Yin, "Mining frequent patterns without candidate generation," ACM SIGMOD Record, vol. 29, no. 2, pp. 1-12, 2000. https://doi.org/10.1145/335191.335372
  30. K. Sornalakshmi, "Detection of DoS attack and zero day threat with SIEM," in Proceedings of 2017 International Conference on Intelligent Computing and Control Systems (ICICCS), Madurai, India, 2017, pp. 1-7.
  31. Fortinet, "FortiDDoS: Protection Profile Settings," 2019 [Online]. Available: https://help.fortinet.com/fddos/4-3-0/FortiDDoS/Managing_thresholds.htm.