Browse > Article
http://dx.doi.org/10.13089/JKIISC.2021.31.2.175

A Study Protocol Reverse Engineering Research Trend and Construction of Optimal Environment for Malware Analysis  

Shin, Kangsik (KAIST Cyber Security Research Center)
Jung, Dong-Jae (KAIST Cyber Security Research Center)
Choe, Min-Ji (KAIST Cyber Security Research Center)
Cho, Ho-Mook (KAIST Cyber Security Research Center)
Publication Information
Journal of the Korea Institute of Information Security & Cryptology / v.31, no.2, 2021 , pp. 175-186 More about this Journal
Abstract
With the advent of a variety of complex and intelligent malicious code these days, the number of C&C (command and control) servers exchanging commands with malicious code is also increasing. Analysis of communication protocols between botnets and C & C servers is essential for a deeper understanding and defense of botnets. The communication protocol reverse engineering analysis method for analyzing such a closed communication protocol can be divided into a network-based analysis method and an execution-based analysis method. In this paper, we have developed a protocol tracer, a dynamic analysis tool based on Fakenet-ng and Pintool, to understand the research trends of each method and further improve its performance. It acts as an Anti-VM responsive environment and virtual. C&C server. We constructed a hybrid analysis environment that combines analysis methods and execution, and improved the results quantitatively and qualitatively from the analysis environment obtained by the experiment compared to the data of the existing environment.
Keywords
malware; command & control; dynamic analysis; reverse engineering; pintool;
Citations & Related Records
연도 인용수 순위
  • Reference
1 Lin, Zhiqiang, et al. "Automatic Protocol Format Reverse Engineering through Context-Aware Monitored Execution," NDSS, Vol. 8, Feb. 2008.
2 Goo, Young-Hoon, et al. "Framework for precise protocol reverse engineering based on network traces," NOMS 2018-2018 IEEE/IFIP Network Operations and Management Symposium. IEEE, pp. 1-4, Apr. 2018.
3 virustotal, "90b309d0616391af7732ef3eb70ad4a39c61dd9163774a17f7df69094e95745e" https://www.virustotal.com/gui/file/, May. 2021.
4 virustotal, "8924332e99cdc1cea5fb5a1a61c1633dc4fa7d40765072f2177ee8235093b8ef" https://www.virustotal.com/gui/file/, May. 2021.
5 CHEN, Xu, et al., "Towards an understanding of anti-virtualization and anti-debugging behavior in modern malware," 2008 IEEE International Conference on Dependable Systems and Networks With FTCS and DCC (DSN), IEEE, pp.177-186, Jun, 2008.
6 Moser, Andreas, Christopher Kruegel, and Engin Kirda. "Exploring multiple execution paths for malware analysis," 2007 IEEE Symposium on Security and Privacy (SP'07). IEEE, pp.231-245, May, 2007.
7 Jiang, Xuxian, Xinyuan Wang, and Dongyan Xu. "Stealthy malware detection and monitoring through VMM-based "out-of-the-box" semantic view reconstruction," ACM Transactions on Information and System Security (TISSEC), vol.13, no.2, pp1-28, Mar, 2010.
8 Xu, Zhaoyan, et al. "Autoprobe: Towards automatic active malicious server pro bing using dynamic binary analysis," Proceedings of the 2014 ACM SIGSAC Conference on Computer and Communications Security, pp. 179-190, Nov. 2014.
9 Ho-Mook Cho, Kyeong-Seok Lee, Yong-Min Kim, "Intelligent Malware Distributing Web Page Detection based on Real Web Browser," The Korean Institute of Information Scientists and Engineers, pp. 1075-1077, Jun. 2017.
10 virustotal, "9002a6aa9685d0d41a98142c4d0699a6d6df827553bf750e73ae5875e8bc88b4" https://www.virustotal.com/gui/file/, May. 2021.
11 J. W. Kim, J. W. Bang and M. J. Choi. "Anti-Anti-Debugging Study to Understand and Disable Anti-De bugging for Malware Analysis," The Journal of Korean Institute of Communications and Information Sciences 45(1), pp.105-116, Jan. 2020.   DOI
12 A. Moser, C. Krugel, and E. Kirda, "Exploring multiple execution paths for malware analysis," IEEE Security and Privacy, pp.231-245, May. 2007.
13 virustotal, "3a8735434cfa5b86bde96f88d7594976e8d5cef4e553c282079a5cbc54831029" https://www.virustotal.com/gui/file/, May. 2021.
14 Park, Juhyun, et al. "Automatic Detection and Bypassing of Anti-Debugging Techniques for Microsoft Windows Environments," Advances in Electrical and Computer Engineering, 19(2), pp23-29, May, 2019.   DOI
15 Bossert, Georges, Frederic Guihery, and Guillaume Hiet. "Towards automat ed protocol reverse engineering using semantic information," Proceedings of the 9th ACM symposium on Information, computer and communications security, pp. 51-62, Jun. 2014.
16 Graziano, Mariano, Corrado Leita, and Davide Balzarotti. "Towards network containment in malware analysis systems," Proceedings of the 28th Annual Computer Security Applications Conference, pp. 339-348, Dec. 2012.
17 Luo, Jian-Zhen, and Shun-Zheng Yu. "Position-based automatic reverse engineering of network protocols," Journal of Network and Computer Applications, Vol. 36, No. 3, pp. 1070-1077, May. 2013.   DOI
18 Caballero, Juan, and Dawn Song. "Automatic protocol reverse-engineering: Message format extraction and field semantics inference," Computer Networks, Vol. 57, No. 2, pp. 451-474, Feb. 2013.   DOI
19 Bermudez, Ignacio, et al. "Automatic protocol field inference for deeper protocol under standing," 2015 IFIP Networking Conference (IFIP Networking), pp. 1-9, May. 2015.
20 Qadeer, Mohammed Abdul, et al. "Network traffic analysis and intrusion detection using packet sniffer," 2010 Second International Conference on Communication Software and Networks. IEEE, pp. 313-317, Feb. 2010.
21 Wikipedia, "Promiscuous mode" https://en.wikipedia.org/wiki/Promiscuous_mode, May. 2021.
22 Google, "Bintext" https://www.aldeid.com/wiki/BinText, May. 2021.
23 Gitgub, "Fakenet" https://github.com/fireeye/flare-fakenet-ng/releases, May. 2021.
24 Crandall, Jedidiah R., et al. "Temporal search: Detecting hidden malware timebombs with virtual machines," ACM SIGOPS Operating Systems Review, Vol. 40, No. 5, pp. 25-36, Oct. 2006.   DOI
25 Graziano, Mariano, et al., "Needles in a haystack: mining information from public dynamic analysis sandboxes for malware intelligence," Proc of the 24th USENIX Conference on Security Symposium, USENIX Association, pp.1057-1072, Aug, 2015.
26 Talukder, Sajedul, and Zahidur Talukder. "A survey on malware detection and analysis tools," International Journal of Network Security & Its Applications vol. 12, no.2, pp.37-57, Mar, 2020.   DOI
27 Choi, Suk-June, Deuk-Hun Kim, and Jin Kwak. "A study on the Prevention of Malware Anti-VM Technique," Proceedings of the Korea Information Processing Society Conference. Korea Information Processing Society, pp.246-249, Apr. 2017.
28 Chakkaravarthy, S. Sibi, D. Sangeetha, and V. Vaidehi. "A survey on mal ware analysis and mitigation techniques," Computer Science Review, vol.32, pp.1-23, May, 2019.   DOI
29 Lin, Jie, et al. "VMRe: A Reverse Framework of Virtual Machine Protection Packed Binaries," 2019 IEEE Fourth International Conference on Data Science in Cyberspace (DSC). IEEE, pp.528-535, Jun, 2019.
30 Sun, Li, Tim Ebringer, and Serdar Boztas. "An automatic anti-anti-VM ware technique applicable for multistage packed malware," 2008 3rd International Conference on Malicious and Unwanted Software (MALWARE). IEEE, pp17-23, Oct, 2008.
31 Raffetseder, Thomas, Christopher Kruegel, and Engin Kirda. "Detecting system emulators," International Conference on Information Security. Springer, Berlin, Heidelberg, pp.1-18, Oct, 2007.
32 Homook Cho, et al. "Automatic Binary Execution Environment based on Real-machines for Intelligent Malware Analysis," KIISE Transactions on Computing Practices, 22(3), pp. 139-144, Mar. 2016.   DOI
33 J. W. Kim, J. W. Bang, and M. J. Choi, "A study on automatic disabling of anti-debugging in manual unpacking," KNOM 2019 Conference, pp. 58-61, May. 2019.
34 Sun-Kyun Kim, Hajin Kim And Mi-Jung Choi. "Design and Implementation of Malware Automatic Unpacking System in Anti-VM/Debugging Environment," The Journal of Korean Institute of Communications and Information Sciences 43(11), pp.1929-1940, Nov, 2018.   DOI
35 Or-Meir, Ori, et al. "Dynamic malware analysis in the modern era-A state of the art survey," ACM Computing Surveys (CSUR) 52.5, vol.52, no.5, pp.1-48, Sep, 2019.   DOI
36 ASLAN, Omer; SAMET, Refik. "Investigation of possibilities to detect malware using existing tools," IEEE/ACS 14th International Conference on Computer Syste ms and Applications (AICCSA). IEEE, pp. 1277-1284, Oct. 2017.
37 Afianian, Amir, et al. "Malware dynamic analysis evasion techniques: Asurvey," ACMComputing Surveys (CSUR), Vol. 52, No. 6, Nov. 2019.
38 CheckPoint, "Cyber Security Report 2020", CheckPoint, 5 Ha'Solelim Street, Tel Aviv 67897, Israel, 80, 2020.
39 securelist, "DDoS attacks" https://securelist.com/ddos-report-in-q1-2018/85373/, May. 2021
40 github, "unipacker" https://github.com/unipacker/unipacker, May. 2021.
41 Shahzad, Raja Khurram, and Niklas Lavesson. "Detecting scareware by mining variable length instruction sequences," IEEE, pp.1-8, Aug. 2011.
42 Wang, Yipeng, et al. "A semantics aware approach to automated reverse engi neering unknown protocols," 20th IEEE International Conference on Network Protocols (ICNP), pp. 1-10, Oct. 2012.