Browse > Article
http://dx.doi.org/10.3745/KTCCS.2021.10.11.297

Analysis of Anti-Reversing Functionalities of VMProtect and Bypass Method Using Pin  

Park, Seongwoo (한양대학교 컴퓨터소프트웨어학과)
Park, Yongsu (한양대학교 컴퓨터소프트웨어학부)
Publication Information
KIPS Transactions on Computer and Communication Systems / v.10, no.11, 2021 , pp. 297-304 More about this Journal
Abstract
Commercial obfuscation tools (protectors) aim to create difficulties in analyzing the operation process of software by applying obfuscation techniques and Anti-reversing techniques that delay and interrupt the analysis of programs in software reverse engineering process. In particular, in case of virtualization detection and anti-debugging functions, the analysis tool exits the normal execution flow and terminates the program. In this paper, we analyze Anti-reversing techniques of executables with Debugger Detection and Viralization Tools Detection options through VMProtect 3.5.0, one of the commercial obfuscation tools (protector), and address bypass methods using Pin. In addition, we predicted the location of the applied obfuscation technique by finding out a specific program termination routine through API analysis since there is a problem that the program is terminated by the Anti-VM technology and the Anti-DBI technology and drew up the algorithm flowchart for bypassing the Anti-reversing techniques. Considering compatibility problems and changes in techniques from differences in versions of the software used in experiment, it was confirmed that the bypass was successful by writing the pin automation bypass code in the latest version of the software (VMProtect, Windows, Pin) and conducting the experiment. By improving the proposed analysis method, it is possible to analyze the Anti-reversing method of the obfuscation tool for which the method is not presented so far and find a bypass method.
Keywords
Reverse Engineering; Dynamic Analysis; Protector; Pin;
Citations & Related Records
연도 인용수 순위
  • Reference
1 Peter Ferrie: The "Ultimate" Anti-Debugging Reference, 2011. http://pferrie.host22.com/papers/antidebug.pdf
2 D. C. D'Elia, E. Coppa, S. Nicchi, F. Palmaro, and L. Cavallaro, "SoK: Using dynamic binary instrumentation for security (And how you may get caught red Handed)," Proceedings of the 2019 ACM Asia Conference on Computer and Communications Security, pp.15-27, 2019.
3 S. Kim. "Code Automatic Analysis Technique for Virtualizationbased Obfuscation and Deobfuscation," Journal of Korea Institute of Information, Electronics, and Communication Technology, pp.724-731, 2018.   DOI
4 R. R. Branco, G. N. Barbosa, and P. D. Neto, "Scientific but Not Academical Overview of Malware Anti-Debugging, Anti-Disassembly and Anti-VM Technologies," black hat USA, 2012. https://media.blackhat.com/bh-us-12/Briefings/Branco/BH_US_12_Branco_Scientific_Academic_Slides.pdf
5 Y. Kang, M. Park, and D. Lee. "Implementation of the Automated De-Obfuscation Tool to Restore Working Executable." Journals of the Korea Institute of Information Security And Cryptology, Vol.27, No.4, pp.785-802, 2017.
6 J. Kirsch, Z. Zhechev, B. Bierbaumer, and T. Kittel, "PwIN - Pwning Intel piN: Why DBI is Unsuitable for Security pplications," In: J. Lopez, J. Zhou, and M. Soriano (eds), Computer Security. ESORICS 2018. Lecture Notes in Computer Science, Vol.11098. Springer, Cham., 2018.
7 Y. B. Lee, J. H. Suk, and D. H. Lee, "Bypassing Anti-Analysis of Commercial Protector Methods Using DBI Tools," IEEE Access, Vol.9, pp.7655-7673, 2021.   DOI
8 VMSoft. "VMProtect software: VMProtect virtualizes code," 2018. [Internet], http://vmpsoft.com/products/vmprotect/,
9 Chi-Keung Luk, et al., "Pin: building customized program analysis tools with dynamic instrumentation," In Proceedings of the 2005 ACM SIGPLAN Conference on Programming Language Design and Implementation, Vol.40, No.6, pp.190-200, 2015.
10 P. Chen C. Huygens L. Desmet, and W. Joosen, "Advanced or not? A comparative study of the use of anti-debugging and anti-vm techniques in generic and targeted malware," IFIP International Conference on ICT Systems Security and Privacy Protection, pp.323-336, 2016.
11 C. Bang, J. H. Suk, and S. Lee, "VMProtect Operation Principle Analysis and Automatic Deobfuscation Implementation," Journal of the Korea Institute of Information Security & Cryptology, Vol.30, No.4, pp.605-616, Aug. 2020.   DOI
12 J. Lee, B. Lee, and S. Cho, "A Study on the Analysis Method to API Wrapping that Difficult to Normalize in the Latest Version of Themida," Journal of the Korea Institute of Information Security & Cryptology, Vol.29, No.6, pp.1375-1382, Dec. 2019.   DOI
13 J. Park Y. Jang S. Hong, and Y. Park, "Automatic detection and bypassing of anti-debugging techniques for microsoft windows environments," Advances in Electrical and Computer Engineering, Vol.19, No.2 pp.23-29, 2019.   DOI