The Korean Journal of Applied Statistics (응용통계연구)

The Korean Statistical Society (한국통계학회)
권호 표지
DOI QR코드

DOI QR Code

Analysis of simulation results using statistical models

통계모형을 이용하여 모의실험 결과 분석하기

  • Kim, Ji-Hyun (Department of Statistics and Actuarial Science, Soongsil University) ;
  • Kim, Bongseong (Department of Statistics and Actuarial Science, Soongsil University)
  • 김지현 (숭실대학교 정보통계보험수리학과) ;
  • 김봉성 (숭실대학교 정보통계보험수리학과)
  • Received : 2021.06.21
  • Accepted : 2021.08.02
  • Published : 2021.10.31
Download PDF
⟨ Previous Next ⟩

Abstract

Simulation results for the comparison of estimators of interest are usually reported in tables or plots. However, if the simulations are conducted under various conditions for many estimators, the comparison can be difficult to be made with tables or plots. Furthermore, for algorithms that take a long time to run, the number of iterations of the simulation is costly to to be increased. The analysis of simulation results using regression models allows us to compare the estimators more systematically and effectively. Since variances in performance measures may vary depending on the simulation conditions and estimators, the heteroscedasticity of the error term should be allowed in the regression model. And multiple comparisons should be made because multiple estimators should be compared simultaneously. We introduce background theories of heteroscedasticity and multiple comparisons in the context of analyzing simulation results. We also present a concrete example.

모의실험 결과를 보통 표나 그림으로 보고한다. 하지만 모의실험이 다양한 조건에서 실시되었고 실험조건마다 추정량의 성능 우위에 대한 결과가 다르면 표와 그림만으로 추정량의 성능을 비교하기가 어렵다. 또한 실행시간이 오래 걸리는 모의실험의 경우 모의실험 반복횟수를 크게 하기 어렵다. 회귀모형을 이용하여 모의실험 결과를 분석하면 보다 체계적이고 효과적으로 추정량의 성능을 비교할 수 있다. 이때 실험조건과 추정량에 따라 성능의 변동이 다를 수 있으므로 회귀모형에서 오차항의 이분산성을 허용해야 하며, 여러 개의 추정량을 동시에 비교해야 하므로 다중비교를 실시해야 한다. 모의실험 결과에 대한 분석이라는 맥락에서 이분산성과 다중비교에 대한 배경이론을 소개하고 예시를 통해 구체적 분석방법도 제시한다.

Keywords

Acknowledgement

귀중한 시간을 할애해서 심사를 진행해주신 편집위원님과 정확하고 건설적인 심사 의견을 주신 두 분 심사위원님께 감사드립니다.

References

  1. Breusch TS and Pagan AR (1979). A simple test for heteroscedasticity and random coefficient variation, Econometrica, 47, 1287-1294. https://doi.org/10.2307/1911963
  2. Hothorn T, Bretz F, and Westfall P (2008). Simultaneous inference in general parametric models, Biometrical Journal, 50, 346-363. https://doi.org/10.1002/bimj.200810425
  3. Hu L, Ji J, and Li F (2020). Estimating heterogeneous survival treatment effect in observational data using machine learning, https://arxiv.org/abs/2008.07044, under review.
  4. Levene H (1960). Robust tests for equality of variances. In Ingram Olkin; Harold Hotelling; et al. (eds.), Contributions to Probability and Statistics: Essays in Honor of Harold Hotelling, Stanford University Press, 278-292.
  5. Lu M, Sadiq S, Feaster DJ, and Ishwaran H (2018). Estimating individual treatment effect in observational data using random forest methods, Journal of Computational and Graphical Statistics, 27, 209-219. https://doi.org/10.1080/10618600.2017.1356325
  6. MacKinnon JG and White H (1985). Some heteroskedasticity-consistent covariance matrix estimators with improved finite sample properties, Journal of Econometrics, 29, 305-325. https://doi.org/10.1016/0304-4076(85)90158-7
  7. White H (1980). A heteroskedasticity-consistent covariance matrix estimator and a direct test for heteroskedasticity, Econometrica, 48, 817-838. https://doi.org/10.2307/1912934
  8. Zeileis A (2004). Econometric computing with HC and HAC covariance matrix estimators, Journal of Statistical Software, 11, 1-17. https://doi.org/10.18637/jss.v011.i10