Browse > Article
http://dx.doi.org/10.9708/jksci.2017.22.03.017

Evaluation of Static Analyzers for Weakness in C/C++ Programs using Juliet and STONESOUP Test Suites  

Seo, Hyunji (Dept. of Computer Engineering, Hongik University)
Park, Young-gwan (Dept. of Computer Engineering, Hongik University)
Kim, Taehwan (Dept. of Computer Engineering, Hongik University)
Han, Kyungsook (Dept. of Computer Engineering, Korea Polytechnic University)
Pyo, Changwoo (Dept. of Computer Engineering, Hongik University)
Publication Information
Journal of the Korea Society of Computer and Information / v.22, no.3, 2017 , pp. 17-25 More about this Journal
Abstract
In this paper, we compared four analyzers Clang, CppCheck, Compass, and a commercial one from a domestic startup using the NIST's Juliet test suit and STONESOUP that is introduced recently. Tools showed detection efficacy in the order of Clang, CppCheck, the domestic one, and Compass under Juliet tests; and Clang, the domestic one, Compass, and CppCheck under STONESOUP tests. We expect it would be desirable to utilize symbolic execution for vulnerability analysis in the future. On the other hand, the results of tool evaluation also testifies that Juliet and STONESOUP as a benchmark for static analysis tools can reveal differences among tools. Finally, each analyzer has different CWEs that it can detect all given test programs. This result can be used for selection of proper tools with respect to specific CWEs.
Keywords
Static Analyzer; Software Weakness; C/C++ Program; JULIET Test Suite; STONESOUP;
Citations & Related Records
연도 인용수 순위
  • Reference
1 C. Joo, and H. Na, "A Study of Research Trend about Internet of Things," NIA(National Information society Agency), Vol. 22, No. 3, pp.3-15, Autumn 2015
2 CWE, Common Weakness Enumeration, http://cwe.mitre.org
3 CVE, Common Vulnerabilities and Exposures, http://cve.mitre.org
4 C. Cadar, and K. Sen, "Symbolic execution for software testing: three decades later," Communications of the ACM, 56.2, pp.82-90, July 2013.   DOI
5 P. Cousot, and R. Cousot, "Abstract interpretation: A unified lattice model for static analysis of programs by construction or approximation of fixpoints," Proceedings of the 4th ACM SIGACT-SIGPLAN symposium on Principles of programming languages, pp.238-252, ACM, January 1977.
6 Compass User Manual, http://rosecompiler.org/Compass.pdf
7 S. Hendrik, and S. Kowalewski, "Static analysis of Sequential Function Charts using abstract interpretation," Emerging Technologies and Factory Automation (ETFA), pp.1-4, 2016 IEEE 21st International Conference on, September 2016.
8 Clang, http://Clang-analyzer.llvm.org
9 CppCheck, http://CppCheck.sourceforge.net
10 NIST, http://samate.nist.gov/SRD/testsuite.php
11 IARPA, STONESOUP(Securely Taking On New Executable Software of Uncertain Provenance)
12 SAMATE, Juliet Test Suite v1.2 for C/C++ User Guide, National Security Agency
13 IARPA, http://www.iarpa.gov
14 MINESTRONE, http://nsl.cs.columbia.edu/projects/m-inestrone
15 PEASOUP, http://www.grammatech.com/software-hardening/research
16 VIBRANCE, http://stonesoup.kestrel.edu
17 NIST, Report on the Static Analysis Tool Exposition (SATE) IV
18 LDRA Testbed, http://www.ldra.com/en/testbedtbvision
19 INFER, http://fbinfer.com
20 Parasoft C++ test, http://www.parasoft.com/product/static-analysis-cc
21 Red Lizard Software Goanna, http://redlizards.com
22 ROSE compiler infrastructure, http://rosecompiler.org
23 C. Lattner, and V. Adve, "LLVM: A compilation framework for lifelong program analysis & transformation," Proceedings of the international symposium on Code generation and optimization: feedback-directed and runtime optimization, pp.75, IEEE Computer Society, March 2004.
24 B. C. Lopes, and R. Auler, "Getting started with LLVM core libraries," Packt Publishing Ltd, pp.73-104, 2014.
25 K. Cooper, and L. Torczon, "Engineering a compiler," Elsevier, pp.231-232, 2011.