Browse > Article
http://dx.doi.org/10.3837/tiis.2021.03.007

A Dissimilarity with Dice-Jaro-Winkler Test Case Prioritization Approach for Model-Based Testing in Software Product Line  

Sulaiman, R. Aduni (Software Engineering Department, School of Computing Faculty of Engineering Universiti Teknologi Malaysia)
Jawawi, Dayang N.A. (Software Engineering Department, School of Computing Faculty of Engineering Universiti Teknologi Malaysia)
Halim, Shahliza Abdul (Faculty of Computer Science and Information System Universiti Tun Hussein Onn Malaysia)
Publication Information
KSII Transactions on Internet and Information Systems (TIIS) / v.15, no.3, 2021 , pp. 932-951 More about this Journal
Abstract
The effectiveness of testing in Model-based Testing (MBT) for Software Product Line (SPL) can be achieved by considering fault detection in test case. The lack of fault consideration caused test case in test suite to be listed randomly. Test Case Prioritization (TCP) is one of regression techniques that is adaptively capable to detect faults as early as possible by reordering test cases based on fault detection rate. However, there is a lack of studies that measured faults in MBT for SPL. This paper proposes a Test Case Prioritization (TCP) approach based on dissimilarity and string based distance called Last Minimal for Local Maximal Distance (LM-LMD) with Dice-Jaro-Winkler Dissimilarity. LM-LMD with Dice-Jaro-Winkler Dissimilarity adopts Local Maximum Distance as the prioritization algorithm and Dice-Jaro-Winkler similarity measure to evaluate distance among test cases. This work is based on the test case generated from statechart in Software Product Line (SPL) domain context. Our results are promising as LM-LMD with Dice-Jaro-Winkler Dissimilarity outperformed the original Local Maximum Distance, Global Maximum Distance and Enhanced All-yes Configuration algorithm in terms of Average Fault Detection Rate (APFD) and average prioritization time.
Keywords
Model-based Testing; Software Product Line; Software Testing; Test Case Prioritization;
Citations & Related Records
연도 인용수 순위
  • Reference
1 C. Henard, M. Papadakis, G. Perrouin, J. Klein, P. Heymans, and Y. Le Traon, "Bypassing the Combinatorial Explosion : Using Similarity to Generate and Prioritize T-wise Test Configurations for Software Product Lines," IEEE Transactions on Software Engineering, vol. 40, no. 7, pp. 650-670, 2016.   DOI
2 H. Hemmati, A. Arcuri, and L. Briand, "Reducing the Cost of Model-Based Testing through Test Case Diversity," in Proc. of IFIP International Confernece Federation for Information Processing, vol. 6435, no. 2, pp. 63-78, 2010.
3 C. Henard, M. Papadakis, G. Perrouin, J. Klein, and Y. L. Traon, "Multi-objective Test Generation for Software Product Lines," in Proc. of the 17th International Software Product Line Conference, pp. 62-71, 2013.
4 K. Czarnecki and M. Antkiewicz, "Mapping Features to Models : A Template Approach Based on Superimposed Variants Background : Feature Modeling," in Proc. of the 4th International Confernce on Generative Programming and Component Engineering, pp. 422-437, 2005.
5 M. S. H. Lackner and M. Schmidet, "Towards the Assessment of Software Product Line Tests : A Mutation System for Variable Systems," in Proc. of the 18th International Software Product Line Conference, pp. 62-69, 2014.
6 X. Devroey, G. Perrouin, and P. Y. Schobbens, "Abstract test case generation for behavioural testing of software product lines," in Proc. of the 18th International Software Product Line Conference: Companion Volume for Workshops, vol 2, pp. 86-93, 2014.
7 A. Reuys, E. Kamsties, K. Pohl, and S. Reis, "Model-Based System Testing of Software Product Families," in Proc. of International Conference Advanced Information Systems Engineering, pp. 519-534, 2010.
8 S. Wang, S. Ali, and A. Gotlieb, "Minimizing Test Suites in Software Product Lines Using Weight-based Genetic Algorithms," in Proc. of the 15th Annual Conference on Genetic and Evolutionary Computation, pp. 1493-1500, 2013.
9 R. A. Tumeng, "Test case prioritization with requirements change using string metrics," Universiti Teknologi Malaysia, 2017.
10 G. Perrouin, Y. Traon, B. Baudry, S. Sen, J. Klein, and S. Oster, "Pairwise testing for software product lines: comparison of two approaches," Software Quality Journal, vol. 20, pp. 605-643, 2012.   DOI
11 I. Machado, "Fault Model-Based Variability Testing," Ph. D. Thesis, Universidade Salvador, Brazil, 2014.
12 S. Weissleder, F. Wartenberg, and H. Lackner, "Testing Software and Systems," in Proc. of IFIP International Confernece on Testing Software and Systems, vol. 9447, pp. 86-101, 2015.
13 F. Ensan, E. Bagheri, and D. Gasevic, "Evolutionary Search-based Test Generation for Software Product Line Feature Models," in Proc. of International Conference on Advanced Information Systems Engineering, pp. 613-628, 2012.
14 L. Shen, X. Peng, and W. Zhao, "A Comprehensive Feature-Oriented Traceability Model for Software Product Line Development," in Proc. of ECMDA Traceability Workshop Proceedings, pp. 77-86, 2008.
15 C. Henard, "Enabling Testing of Large Scale Highly Configurable Systems with Search-based Software Engineering : The Case of Model-based Software Product Lines" Ph. D. Dissertation, Universite Du Luxemborg, 2015.
16 S. Yoo and M. Harman, "Regression testing minimization, selection and prioritization : a survey," Software Testing Minimization, Selection and Prioritization: a survey, vol. 22.2, pp. 67-120, 2012.
17 B. Utting, M. Pretschner, and A. Legeard, "A taxonomy of model-based testing approaches," Software: Testing, Verification and Reliablity, vol. 22, no. 5, pp. 291-312, 2012.
18 C. Henard, M. Papadakis, G. Perrouin, J. Klein, and Y. Le Traon, "Assessing Software Product Line Testing via Model-based Mutation: An Application to Similarity Testing," in Proc. of the 6th International Conference on Software Testing, Verification and Validation Workshops, pp. 188-197, 2013.
19 S. Lity, M. Al-Hajjaji, T. Thum, and I. Schaefer, "Optimizing product orders using graph algorithms for improving incremental product-line analysis," in Proc. of the 11th International Workshop on Variability Modelling of Software-intensive Systems, pp. 60-67, 2017.
20 L. Inozemtseva and R. Holmes, "Coverage Is Not Strongly Correlated with Test Suite Effectiveness," in Proc. of the 36th International Conference on Software Engineering, pp. 435-445, 2014.
21 S. Wang, D. Buchmann, S. Ali, A. Gotlieb, D. Pradhan, and M. Liaaen, "Multi-Objective Test Prioritization in Software Product Line Testing : An Industrial Case Study," in Proc. of the 18th International Software Product Line Confernece, vol. 1, pp. 32-41, 2014.
22 I. Schaefer, M. Al-Hajjaji, R. Lachmann, F. Furchtegott, and S. Lity, "Fine-grained test case prioritization for integration testing of delta-oriented software product lines," in Proc. of the 7th International workshop on Feature-Oriented Software, vol. 1, no. 212, pp. 1-10, 2016.
23 X. Devroey, "Behavioural Model Based Testing of Software Product Lines," Ph. D. Thesis, University of Namur, Belgium, 2014.
24 M. Sahak, "Effective similarity based test case prioritization technique for software product lines," Universiti Teknologi Malaysia, 2018.
25 X. Devroey, G. Perrouin, M. Cordy, H. Samih, A. Legay, and P. S. Patrick, "Statistical prioritization for software product line testing : an experience report," Software and Systems Modeling, vol. 16, no. 1, pp. 153-171, 2017.   DOI
26 M. Sahak, S. A. Halim, D. N. A. Jawawi, and M. A. Isa, "Evaluation of Software Product Line Test Case Prioritization Technique," International Journal on Advanced Science, vol. 7, no. 4-2, pp. 1601-1608, 2017.
27 X. Devroey, G. Perrouin, A. Legay, P. Heymans, and P. Heymans, "Dissimilar Test Case Selection for Behavioural Software Product Line Testing," 2017.
28 M. Al-Hajjaji, S. Lity, R. Lachmann, T. Thum, I. Schaefer, and G. Saake, "Delta-Oriented Product Prioritization for Similarity-Based Product-Line Testing," in Proc. of IEEE/ACM 2nd International Workshop on Variablity and Complexity in Software Design, pp. 34-40, 2017.
29 R. Kazmi, D. N. A. Jawawi, R. Mohamad, and I. Ghani, "Effective Regression Test Case Selection," ACM Computing Surveys, vol. 50, no. 2, pp. 1-32, 2017.
30 S. B. Lity, "Model-Based Product-Line Regression Testing of Variants and Versions of Variants," Technische Universitat Braunschweig, 2019.
31 R. Kazmi, D. N. A. Jawawi, R. Mohamad, and I. Ghani, "Effective Regression Test Case Selection Technique using Weighted Average Scoring," ACM Computing Survery, vol. 50, no. 2, 2017.
32 F. Heidenreich, J. Kopcsek, and C. Wende, "FeatureMapper: mapping features to models," in Proc. of Companion of the 30th International Conference on Software Engineering, vol. 30, pp. 943-944, 2008.
33 A. Emilia and V. Barbosa, "Similarity-based test suite reduction in the context of Model-Based Testing," Universidade Federal de Campina Grande, 2015.
34 S. Halim, D. N. Jawawi, and M. Sahak, "Similarity Distance Measure and Prioritization," Journal of Informaion and Communication Technology, vol. 18, no. 1, pp. 57-75, 2019.   DOI
35 A. Egyed, S. Segura, R. E. Lopez-Herrejon, A. Ruiz-Cortes, J. A. Parejo, and A. B. Sanchez, "Multi-objective test case prioritization in highly configurable systems: A case study," Journal of Systmes and Software, vol. 122, pp. 287-310, 2016.   DOI
36 R. A. Sulaiman and D. N. A. Jawawi, "Derivation of Test Cases for Model-based Testing of Software Product Line with Hybrid Heuristic Approach," in Proc. of International Conference of Reliable Information and Communication Technology, vol. 1073, pp. 199-208.
37 A. Abbas, I. F. Siddiqui, and S. U. Lee, "Multi-Objective Optimization of Feature Model in Software Product Line : Perspectives and Challenges," Indian Journal of Science and Technology, vol. 9, no. 45, pp. 1-7, Dec. 2016.
38 A. Arrieta, G. Sagardui, and L. Etxeberria, "A model-based testing methodology for the systematic validation of highly configurable cyber-physical systems," in Proc. of the 6 th International Conference on Advances in System Testing and Validation Lifecycle, pp. 66-72, 2014.
39 S. A. Ajila and A. B. Kaba, "Using Traceability Mechanisms to Support Software Product Line Evolution," in Proc. of International Conference on Information Reuse and Integration, pp. 157-162, 2004.