Browse > Article
http://dx.doi.org/10.5351/KJAS.2021.34.5.761

Analysis of simulation results using statistical models  

Kim, Ji-Hyun (Department of Statistics and Actuarial Science, Soongsil University)
Kim, Bongseong (Department of Statistics and Actuarial Science, Soongsil University)
Publication Information
The Korean Journal of Applied Statistics / v.34, no.5, 2021 , pp. 761-772 More about this Journal
Abstract
Simulation results for the comparison of estimators of interest are usually reported in tables or plots. However, if the simulations are conducted under various conditions for many estimators, the comparison can be difficult to be made with tables or plots. Furthermore, for algorithms that take a long time to run, the number of iterations of the simulation is costly to to be increased. The analysis of simulation results using regression models allows us to compare the estimators more systematically and effectively. Since variances in performance measures may vary depending on the simulation conditions and estimators, the heteroscedasticity of the error term should be allowed in the regression model. And multiple comparisons should be made because multiple estimators should be compared simultaneously. We introduce background theories of heteroscedasticity and multiple comparisons in the context of analyzing simulation results. We also present a concrete example.
Keywords
heteroscedasiticity-consistent estimator; covariance matrix; simultaneous confidence intervals; conditional plots;
Citations & Related Records
연도 인용수 순위
  • Reference
1 Breusch TS and Pagan AR (1979). A simple test for heteroscedasticity and random coefficient variation, Econometrica, 47, 1287-1294.   DOI
2 Hothorn T, Bretz F, and Westfall P (2008). Simultaneous inference in general parametric models, Biometrical Journal, 50, 346-363.   DOI
3 Levene H (1960). Robust tests for equality of variances. In Ingram Olkin; Harold Hotelling; et al. (eds.), Contributions to Probability and Statistics: Essays in Honor of Harold Hotelling, Stanford University Press, 278-292.
4 Lu M, Sadiq S, Feaster DJ, and Ishwaran H (2018). Estimating individual treatment effect in observational data using random forest methods, Journal of Computational and Graphical Statistics, 27, 209-219.   DOI
5 MacKinnon JG and White H (1985). Some heteroskedasticity-consistent covariance matrix estimators with improved finite sample properties, Journal of Econometrics, 29, 305-325.   DOI
6 White H (1980). A heteroskedasticity-consistent covariance matrix estimator and a direct test for heteroskedasticity, Econometrica, 48, 817-838.   DOI
7 Zeileis A (2004). Econometric computing with HC and HAC covariance matrix estimators, Journal of Statistical Software, 11, 1-17.   DOI
8 Hu L, Ji J, and Li F (2020). Estimating heterogeneous survival treatment effect in observational data using machine learning, https://arxiv.org/abs/2008.07044, under review.