The Korean Journal of Applied Statistics (응용통계연구)

The Korean Statistical Society (한국통계학회)
권호 표지
DOI QR코드

DOI QR Code

On Information Criteria in Linear Regression Model

  • Published : 2009.02.28
Download PDF
⟨ Previous Next ⟩

Abstract

In the model selection problem, the main objective is to choose the true model from a manageable set of candidate models. An information criterion gauges the validity of a statistical model and judges the balance between goodness-of-fit and parsimony; "how well observed values ran approximate to the true values" and "how much information can be explained by the lower dimensional model" In this study, we introduce some information criteria modified from the Akaike Information Criterion (AIC) and the Bayesian Information Criterion(BIC). The information criteria considered in this study are compared via simulation studies and real application.

Keywords

References

  1. Akaike, H. (1973). Information theory and an extension of the maximum likelihood principle, Second International Symposium on Information Theory, 267-281
  2. Akaike, H. (1980). Likelihood and the Bayes procedure, Bayesian Statistics, University Press, Valencia Spain
  3. Belsley, D. A., Kuh, E. and Welsch, R. E. (1980). Regression Diagnostics: Identifying Influential Data and Sources of Collinearity, John Wiley & Sons, New York
  4. Breirnan, L., Friedman, J. H., Olshen, R. A. and Stone, C. (1984). Classification and Regression Trees, Chapman & Hall/CRC, New York
  5. Cavanaugh, J. E. (1999). A large-sample model selection criterion based on Kullback's symmetric divergence, Statistics and Probability Letters, 42, 333-343 https://doi.org/10.1016/S0167-7152(98)00200-4
  6. Foster, D. P. and George, E. I. (1994), The risk information criterion for multiple regression, The Annals of Statistics, 22, 1947-1975 https://doi.org/10.1214/aos/1176325766
  7. George, E. I. and Foster, D. P. (2000). Calibration and empirical Bayes variable selection, Biometrika, 87, 731-747 https://doi.org/10.1093/biomet/87.4.731
  8. Hurvich, C. M. and Tsai, C. L. (1989). Regression and time series model selection in small samples, Biometrika, 76, 297-307 https://doi.org/10.1093/biomet/76.2.297
  9. Kullback, S. and Liebler, R. A. (1951). On information and sufficiency, The Annals of Mathematical Statistics, 22, 79-86 https://doi.org/10.1214/aoms/1177729694
  10. Rahman, M. S, and King, M. L. (1997). Probability of correct identification in model selection, In Proceedings of the Econometric Society Australasian Meeting, 631-640, University of Melbourne, Melbourne
  11. Rahman, M. S. and King, M. L. (1999). Improved model selection criterion, Communications in Statistics-Simulation and Computation, 28, 51-71 https://doi.org/10.1080/03610919908813535
  12. Schwarz, G. (1978). Estimating the dimension of a model, The Annals of Statistics, 6, 461-464 https://doi.org/10.1214/aos/1176344136
  13. Theil, H. (1961). Economic Forecasts and Policy, (Second Ed.), North-Holland, Amsterdam
  14. Tibshirani, R. and Knight, K. (1999). The covariance inflation criterion for adaptive model selection, Journal of the Royal Statistical Society, Series B, 61, 529-546 https://doi.org/10.1111/1467-9868.00191
  15. Wei, C. Z. (1992). On predictive least squares principles, The Annals of Statistics, 20, 1-42 https://doi.org/10.1214/aos/1176348511
  16. Wu, T. J. and Sepulveda, A. (1998). The weighted average information criterion for order selection in time series and regression models, Statistics and Probability Letters, 39, 1-10 https://doi.org/10.1016/S0167-7152(98)00003-0