KIPS Transactions on Computer and Communication Systems (정보처리학회논문지:컴퓨터 및 통신 시스템)

Korea Information Processing Society (한국정보처리학회)
권호 표지
DOI QR코드

DOI QR Code

Analysis of Anti-Reversing Functionalities of VMProtect and Bypass Method Using Pin

VMProtect의 역공학 방해 기능 분석 및 Pin을 이용한 우회 방안

  • 박성우 (한양대학교 컴퓨터소프트웨어학과) ;
  • 박용수 (한양대학교 컴퓨터소프트웨어학부)
  • Received : 2021.04.01
  • Accepted : 2021.09.16
  • Published : 2021.11.30
Download PDF
⟨ Previous Next ⟩

Abstract

Commercial obfuscation tools (protectors) aim to create difficulties in analyzing the operation process of software by applying obfuscation techniques and Anti-reversing techniques that delay and interrupt the analysis of programs in software reverse engineering process. In particular, in case of virtualization detection and anti-debugging functions, the analysis tool exits the normal execution flow and terminates the program. In this paper, we analyze Anti-reversing techniques of executables with Debugger Detection and Viralization Tools Detection options through VMProtect 3.5.0, one of the commercial obfuscation tools (protector), and address bypass methods using Pin. In addition, we predicted the location of the applied obfuscation technique by finding out a specific program termination routine through API analysis since there is a problem that the program is terminated by the Anti-VM technology and the Anti-DBI technology and drew up the algorithm flowchart for bypassing the Anti-reversing techniques. Considering compatibility problems and changes in techniques from differences in versions of the software used in experiment, it was confirmed that the bypass was successful by writing the pin automation bypass code in the latest version of the software (VMProtect, Windows, Pin) and conducting the experiment. By improving the proposed analysis method, it is possible to analyze the Anti-reversing method of the obfuscation tool for which the method is not presented so far and find a bypass method.

상용 난독화 도구(프로텍터)들은 소프트웨어 역공학 과정에서 프로그램의 분석을 지연시키고 방해하는 난독화 기술 및 역공학 방해(안티리버싱) 기법을 적용시킴으로써 소프트웨어의 동작 과정을 분석하는데 어려움을 발생시키는데 목적이 있다. 특히, 가상화 탐지와 안티디버깅 기능 같은 경우 분석 도구가 발견되면 정상적인 실행 흐름을 벗어나 프로그램을 종료시킨다. 본 논문에서는 상용 난독화 도구(프로텍터) 중 하나인 VMProtect 3.5.0을 통해 Debugger Detection, Virualization Tools Detection 옵션을 적용시킨 실행 파일의 안티리버싱 기법을 분석하고 Pin을 이용한 우회 방안을 제안한다. 또한, 적용된 안티리버싱 기법을 분석하는 과정에서 Amti-VM 기술과 Anti-DBI 기술에 의해 프로그램이 종료되는 문제가 발생하기 때문에 API 분석을 통해 특정 프로그램 종료 루틴을 알아내어 적용된 안티리버싱 기법의 위치를 예상하고 위치를 바탕으로 안티리버싱 기법 우회 방안 알고리즘 순서도를 작성하였다. 실험에 사용된 소프트웨어들의 버전의 차이로부터 발생하는 호환성 문제, 기법의 변화 등을 고려하여 최신 버전의 소프트웨어(VMProtect, Windows, Pin)에서 Pin 자동화 우회 코드를 작성하고 실험을 진행하여 성공적으로 우회됨을 확인하였다. 제안된 분석 방안을 개선하여 기법이 제시되지 않은 난독화 도구의 안티리버싱 기법을 분석하고 우회 방안을 찾아낼 수 있다.

Keywords

Acknowledgement

본 연구는 한국연구재단 연구과제(2020R1F1A1048443) 지원으로 수행하였습니다.

References

  1. J. Kirsch, Z. Zhechev, B. Bierbaumer, and T. Kittel, "PwIN - Pwning Intel piN: Why DBI is Unsuitable for Security pplications," In: J. Lopez, J. Zhou, and M. Soriano (eds), Computer Security. ESORICS 2018. Lecture Notes in Computer Science, Vol.11098. Springer, Cham., 2018.
  2. D. C. D'Elia, E. Coppa, S. Nicchi, F. Palmaro, and L. Cavallaro, "SoK: Using dynamic binary instrumentation for security (And how you may get caught red Handed)," Proceedings of the 2019 ACM Asia Conference on Computer and Communications Security, pp.15-27, 2019.
  3. Y. B. Lee, J. H. Suk, and D. H. Lee, "Bypassing Anti-Analysis of Commercial Protector Methods Using DBI Tools," IEEE Access, Vol.9, pp.7655-7673, 2021. https://doi.org/10.1109/ACCESS.2020.3048848
  4. VMSoft. "VMProtect software: VMProtect virtualizes code," 2018. [Internet], http://vmpsoft.com/products/vmprotect/,
  5. S. Kim. "Code Automatic Analysis Technique for Virtualizationbased Obfuscation and Deobfuscation," Journal of Korea Institute of Information, Electronics, and Communication Technology, pp.724-731, 2018. https://doi.org/10.17661/JKIIECT.2018.11.6.724
  6. Chi-Keung Luk, et al., "Pin: building customized program analysis tools with dynamic instrumentation," In Proceedings of the 2005 ACM SIGPLAN Conference on Programming Language Design and Implementation, Vol.40, No.6, pp.190-200, 2015.
  7. P. Chen C. Huygens L. Desmet, and W. Joosen, "Advanced or not? A comparative study of the use of anti-debugging and anti-vm techniques in generic and targeted malware," IFIP International Conference on ICT Systems Security and Privacy Protection, pp.323-336, 2016.
  8. R. R. Branco, G. N. Barbosa, and P. D. Neto, "Scientific but Not Academical Overview of Malware Anti-Debugging, Anti-Disassembly and Anti-VM Technologies," black hat USA, 2012. https://media.blackhat.com/bh-us-12/Briefings/Branco/BH_US_12_Branco_Scientific_Academic_Slides.pdf
  9. C. Bang, J. H. Suk, and S. Lee, "VMProtect Operation Principle Analysis and Automatic Deobfuscation Implementation," Journal of the Korea Institute of Information Security & Cryptology, Vol.30, No.4, pp.605-616, Aug. 2020. https://doi.org/10.13089/JKIISC.2020.30.4.605
  10. J. Lee, B. Lee, and S. Cho, "A Study on the Analysis Method to API Wrapping that Difficult to Normalize in the Latest Version of Themida," Journal of the Korea Institute of Information Security & Cryptology, Vol.29, No.6, pp.1375-1382, Dec. 2019. https://doi.org/10.13089/JKIISC.2019.29.6.1375
  11. Y. Kang, M. Park, and D. Lee. "Implementation of the Automated De-Obfuscation Tool to Restore Working Executable." Journals of the Korea Institute of Information Security And Cryptology, Vol.27, No.4, pp.785-802, 2017.
  12. J. Park Y. Jang S. Hong, and Y. Park, "Automatic detection and bypassing of anti-debugging techniques for microsoft windows environments," Advances in Electrical and Computer Engineering, Vol.19, No.2 pp.23-29, 2019. https://doi.org/10.4316/AECE.2019.02003
  13. Peter Ferrie: The "Ultimate" Anti-Debugging Reference, 2011. http://pferrie.host22.com/papers/antidebug.pdf