Browse > Article
http://dx.doi.org/10.3837/tiis.2020.06.003

A GQM Approach to Evaluation of the Quality of SmartThings Applications Using Static Analysis  

Chang, Byeong-Mo (Sookmyung Women's University)
Son, Janine Cassandra (Sookmyung Women's University)
Choi, Kwanghoon (Chonnam National University)
Publication Information
KSII Transactions on Internet and Information Systems (TIIS) / v.14, no.6, 2020 , pp. 2354-2376 More about this Journal
Abstract
SmartThings is one of the most popular open platforms for home automation IoT solutions that allows users to create their own applications called SmartApps for personal use or for public distribution. The nature of openness demands high standards on the quality of SmartApps, but there have been few studies that have evaluated this thoroughly yet. As part of software quality practice, code reviews are responsible for detecting violations of coding standards and ensuring that best practices are followed. The purpose of this research is to propose systematically designed quality metrics under the well-known Goal/Question/Metric methodology and to evaluate the quality of SmartApps through automatic code reviews using a static analysis. We first organize our static analysis rules by following the GQM methodology, and then we apply the rules to real-world SmartApps to analyze and evaluate them. A study of 105 officially published and 74 community-created real-world SmartApps found a high ratio of violations in both types of SmartApps, and of all violations, security violations were most common. Our static analysis tool can effectively inspect reliability, maintainability, and security violations. The results of the automatic code review indicate the common violations among SmartApps.
Keywords
Quality; Evaluation; Static analysis; SmartThings; IoT applications;
Citations & Related Records
연도 인용수 순위
  • Reference
1 Kashif Iqbal, Muhammad Adnan Khan, Sagheer Abbas, Zahid Hasan, Areej Fatima, "Intelligent transportation system (ITS) for smart-cities using mamdani fuzzy inference system," International Journal of Advanced Computer Science and Applications (IJACSA), vol.9, No.2, pp.94-105, 2018.
2 Ayesha Atta, Sagheer Abbas, M. Adnan Khan, Gulzar Ahmed, Umber Farooq, "An adaptive approach: smart traffic congestion control system," Journal of King Saud University-Computer and Information Sciences, 2018.
3 Y. Yang, L. Wu, G. Yin, L. Li, H. Zhao, "A survey on security and privacy issues in Internet-of-Things," IEEE Internet of Things Journal, vol. 4, no. 5, pp. 1250-1258, October, 2017.   DOI
4 E. Fernandes, J. Jung, A. Prakash, "Security analysis of emerging smart home applications," in Proc. of IEEE Symposium on Security and Privacy, pp. 636-654, May 22-26, 2016.
5 "Smartthings developer documentation", 2017. [Online]. Available: http://docs.smartthings.com
6 Z. B. Celik, L. Babun, A. K. Sikder, H. Aksu, G. Tan, P. D. McDaniel, A. S. Uluagac, "Sensitive information tracking in commodity iot," in Proc. of 27th USENIX Security Symposium, Baltimore, MD, USA, pp. 1687-1704, August 15-17, 2018.
7 R. van Solingen, E. Berghout, "Integrating goal-oriented measurement in industrial software engineering: industrial experiences with and additions to the goal/question/metric method (gqm)," in Proc. of 7th Int'l Software Metrics Symposium, pp. 246-258, April 4-6, 2001.
8 R. M. Hartog, "Octopull: Integrating static analysis with code reviews," Master's thesis, Delft University of Technology, the Netherlands, December 16, 2015.
9 D. Singh, V. R. Sekar, K. T. Stolee, B. Johnson, "Evaluating how static analysis tools can reduce code review effort," in Proc. of IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC), pp. 101-105, October 11-14, 2017.
10 V. R. Basili, "Software modeling and measurement: the goal/question/metric paradigm," Tech. report, College Park, MD, USA, September, 1992.
11 CodeNarc, 2018. [Online]. Available: http://codenarc.sourceforge.net
12 Groovy, 2003. [Online]. Available: http://groovy-lang.org
13 Janine Cassandra Son, "Automatic code review for SmartThings applications using static analysis," Master thesis, Sookmyung Women's University, June, 2018.
14 Janine Casandra Son, Byeong-Mo Chang, Kwanghoon Choi, "Automatic code review for SmartThings application using static analysis," in Proc. of Korea Software Congress (KSC2017), Bexco, Busan, pp.513-515, December 20-22, 2017.
15 Checkstyle, 2001. [Online] Available: http://checkstyle.sourceforge.net
16 J.-S. Oh, H.-J. Choi, "A reflective practice of automated and manual code reviews for a studio project," in Proc. of Fourth Annual ACIS International Conference on Computer and Information Science (ICIS'05), pp. 37-42, July 14-16, 2005.
17 PMD, 2017. [Online] Available: https://pmd.github.io
18 FindBugs, 2015. [Online] Available: http://findbugs.sourceforge.net
19 S. Panichella, V. Arnaoudova, M. D. Penta, G. Antoniol, "Would static analysis tools help developers with code reviews?," in Proc. of IEEE 22nd International Conference on Software Analysis, Evolution, and Reengineering (SANER), pp. 161-170, March 2-6, 2015.
20 I. Gomes, P. Morgado, T. Gomes, R. Moreira, "An overview on the static code analysis approach in software development," Faculdade de Engenharia da Universidade do Porto, Portugal, 2009.
21 A. Costin, "Lua-code: security overview and practical approaches to static analysis," in Proc. of IEEE Security and Privacy Workshops (SPW), May 25, 2017.
22 Sunil Manandhar, Kevin Moran, Kaushal Kafle, Ruhao Tang, Denys Poshyvanyk, Adwait Nadkarni, "Towards a natural perspective of smart homes for practical security and safety analyses," in Proc. of 41st IEEE Symposium on Security and Privacy, San Francisco, CA, USA, pp.1-18, May 18-20, 2020.
23 M. Kim, J. H. Park. N. Y. Lee, "A quality model for IoT service," in Proc. of Advances in Computer Science and Ubiquitous Computing, J. H. Park, Y. Pan, G. Yi, V. Loia (Eds.), Springer Singapore, Singapore, pp. 497-504, 2016.
24 ISO/IEC 25010 - Systems and software engineering - systems and software quality requirements and evaluation (SQuaRE) - systems and software quality models, Technical report, 2010.
25 B. Chess, G. McGraw, "Static analysis for security," IEEE Security & Privacy, vol. 2, no. 6, pp.76-79, November, 2004.
26 D. Insa, J. Silva, "Automatic assessment of Java code," Computer Languages, Systems & Structures, vol.53, pp.59-72, September, 2018.   DOI
27 JetBrains, IntelliJ IDEA. [Online]. Available: https://www.jetbrains.com/idea
28 H. Washizaki, R. Namiki, T. Fukuoka, Y. Harda, H. Watanabe, "A framework for measuring and evaluating program source code quality," in Proc. of Product-Focused Software Process Improvement, J. Munch, P. Abrahamsson (Eds.), pp.284-299, July 2-4, 2007.
29 Z. B. Celik, P. D. McDaniel, G. Tan, "Soteria: automated iot safety and security analysis," in Proc. of 2018 USENIX Annual Technical Conference, Baltimore, MD, USA, July, pp. 147-158, 2018.
30 Z. Berkay Celik, Earlence Fernandes, Eric Pauley, Gang Tan, Patrick McDaniel, "Program analysis of commodity iot applications for security and privacy: challenges and opportunities," ACM Computing Survey, vol.52, no.4, article 74, August, 2019.
31 Byeong-Mo Chang, Janine Cassandra Son, Kwanghoon Choi, "An evaluation of the quality of IoT applications on the SmartThings platform using static analysis (extended version)," Preprint submitted to KSII Trans. on Internet and Information Systems(The Extended Version), pp. 1-30, 2019.
32 B. Chess, J. West, Secure programming with static analysis, 1st Edition, Addison-Wesley Professional, 2007.
33 S. McConnell, Code complete: a practical handbook of software construction, Code Series, Microsoft Press, 1993.
34 SmartThings community, Smartthigns open-source DeviceTypeHandlers and SmartApps code, 2015. [Online]. Available: https://github.com/SmartThingsCommunity/SmartThingsPublic
35 Y. Tian, N. Zhang, Y.-H. Lin, X. Wang, B. Ur, X. Guo, P. Tague, "Smartauth: user-centered authorization for the internet of things," in Proc. of 26th USENIX Security Symposium (USENIX Security 17), USENIX Association, Vancouver, BC, pp.361-378, August 16-18, 2017.
36 S. Wagner, J. Jürjens, C. Koller, P. Trischberger, "Comparing bug finding tools with reviews and tests," in Proc. of Testing of Communicating Systems, F. Khendek, R. Dssouli (Eds.), Springer Berlin Heidelberg, Berlin, Heidelberg, pp. 40-55, May 31-June 2, 2005.
37 D. Evans, D. Larochelle, "Improving security using extensible lightweight static analysis," IEEE Software, vol. 19, no. 1, pp.42-51, August 7, 2002.   DOI