Browse > Article
http://dx.doi.org/10.5762/KAIS.2012.13.2.562

Validation Technique of Simulation Model using Weighted F-measure with Hierarchical X-means (WF-HX) Method  

Yang, Dae-Gil (Dept. of Industrial and Management Engineering, Korea University)
HwangBo, Hun (Industrial and Systems Engineering Division, Texas A&M University)
Cheon, Hyun-Jae (The Information Security Institute, Korea University)
Lee, Hong-Chul (Dept. of Information and Management Engineering, Korea University)
Publication Information
Journal of the Korea Academia-Industrial cooperation Society / v.13, no.2, 2012 , pp. 562-574 More about this Journal
Abstract
Simulation validation techniques which have been employed in most studies are statistical analysis, which validate a model with mean or variance of throughput and resource utilization as an evaluation object. However, these methods have not been able to ensure the reliability of individual elements of the model well. To overcome the problem, the weighted F-measure method was proposed, but this technique also had some limitations. First, it is difficult to apply the technique to complex system environment with numerous values of interarrival time because it assigns a class to an individual value of interarrival time. In addition, due to unbounded weights, the value of weighted F-measure has no lower bound, so it is difficult to determine its threshold. Therefore, this paper propose weighted F-measure technique with cluster analysis to solve these problems. The classes for the technique are defined by each cluster, which reduces considerable number of classes and enables to apply the technique to various systems. Moreover, we improved the validation technique in the way of assigning minimum bounded weights without any lack of objectivity.
Keywords
Discrete Event Simulation; Validation technique; Cluster Analysis; X-means; Weighted F-measure;
Citations & Related Records
연도 인용수 순위
  • Reference
1 Balci, o., "Validation, verification, and testing techniques throughout the life cycle of a simulation study", Annals of Operations Research, Vol.53, pp.121-173, 1994.   DOI
2 David Arthur, Sergei Vassilvitskii, "k-means++: the advantages of careful seeding", Proceeding SODA '07 Proceedings of the eighteenth annual ACM-SIAM symposium on Discrete algorithms, pp.1027-1035, 2007.
3 Dan Pelleg and Andrew Moore, "X-means : Extending K-means with Efficient Estimation of the number of clusters", proceedings of the 17th International Conference on Machine Learning, pp.727-734, 2000.
4 Harrison, S. R., "Regression of a model on real-system output: An invalid test of model validity", Agricultural systems, Vol.34, No.3, pp.183-190, 1990.   DOI
5 Hwangbo hun , Cheon hyeon-jae , Lee hong-cheol, "Using the weighted F-measure of Trace-Driven Simulation model verification methods," Journal of Simulation, Vol.18, No.4, pp.185-195, 2009.
6 Jack P.C. Kleijnen, "Verification and Validation of Simulation Models", European J. of Operational Research, pp.82, 145-162, 1995.   DOI
7 Jack P.C. Kleijnen, Bert Bettonvil, Willem Van Groenendahl, "Validation of Trace -Driven Simulation Models : Regression Analysis Revised", Proceedings of the WSC, pp.352-359, 1994.
8 Lee young-hae, Gwak seong-geun, Kim suk-han, "Under a distributed environment, research on Web-based simulation", Journal of Korea Society for Simulation, No. 7 No. 2, pp.79-90, 1998.
9 Box G. E. P, Jenkins G. M, and Reinsel G. C., "Time Series Analysis ; Forecasting and Control", pp.463-479, Prentice Hall 3rd, 1994.
10 Kang seok-bok, "Statistical estimation and hypothesis testing", pp.156-318, Gyeongmunsa, 2002.
11 L. Excoffier and M. Slatkin, "Maximum - Likelihood Estimation of Molecular Haplotype Frequencies in a Diploid Population", Oxford Journals Life Sciences & Medicine Molecular Biologyand Evolution, Volume12, Issue5, pp. 921-927, 1995.
12 Kim dae-hak, "Utilizing spss, analysis of variance", pp.39-65, Gyowoosa, 2004.
13 Ross, S. M., Simulation 4th Ed., pp.294-306, Elsevier Academic Press, 2006.
14 Law, A. M. and Kelton, W. D., "Simulation modeling and analysis 3rd Ed.", pp.283-290, McGraw-Hill, 2000.
15 S. Lloyd, "Least Squares Quantization in PCM", IEEE Transactions on Information Theory, Vol.28, pp.129-137, 1982.   DOI
16 L. Yen, D. Vanvyve, F. Wouters, F. Fouss, M. Verleysen, M. Saerens, "Clustering using a random walk based distance measure", proceedings - European Symposium on Artificial Neural Networks Bruges April, pp.317-324, 2005.
17 Lee young-hae, Choe gwan-grim, "Intervention Analysis techniques, utilizing research on Validation of simulation models," Journal of Korea Society for Simulation, pp.175, 1997.
18 Murray-Smith, D. J., "Methods for the external validation of continuous system simulation models", Mathematical and computer modelling of dynamic systems, Vol.4, No.1, pp.5-31, 1998.   DOI
19 S. N. Crozier, D. D. Falconer, S. A. Mahmoud, "Least sum of squared errors (LSSE) channel estimation", Radar and Signal Processing, IEEE Proceedings F, Volume 138, Issue 4, pp.371-378, 1991.   DOI
20 Sargent, R. G., "Verification and Validation of Simulation Models", Proceedings of the 2010 Winter Simulation Conference, pp.166-183, 2010.
21 Smith, E. P. and Rose, K. A., "Model goodness- of-fit analysis using regression and related techniques", Ecological modelling, Vol.77, No.1, pp.49-64, 1995.   DOI
22 Wei, X. C. and Li, E. P., "Reflection of transmitting antenna in reverberation chamber and its effect on chi-square validation", Antennas and propagation society international symposium 2006, pp.3573-3576, 2006.
23 A. Kumar, Y. Sabharwal, and S. Sen, "A simple linear time (1+${\varepsilon}$)-approximation algorithm for k-means clustering in any dimensions", Proc. 45th IEEE Symposium on Foundations of Computer Science, pp.454-462, 2004.