Communications for Statistical Applications and Methods

The Korean Statistical Society (한국통계학회)
권호 표지
DOI QR코드

DOI QR Code

Penalized maximum likelihood estimation with symmetric log-concave errors and LASSO penalty

  • Received : 2022.03.22
  • Accepted : 2022.05.16
  • Published : 2022.11.30
Download PDF
⟨ Previous Next ⟩

Abstract

Penalized least squares methods are important tools to simultaneously select variables and estimate parameters in linear regression. The penalized maximum likelihood can also be used for the same purpose assuming that the error distribution falls in a certain parametric family of distributions. However, the use of a certain parametric family can suffer a misspecification problem which undermines the estimation accuracy. To give sufficient flexibility to the error distribution, we propose to use the symmetric log-concave error distribution with LASSO penalty. A feasible algorithm to estimate both nonparametric and parametric components in the proposed model is provided. Some numerical studies are also presented showing that the proposed method produces more efficient estimators than some existing methods with similar variable selection performance.

Keywords

Acknowledgement

The work of Byungtae Seo was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (No. NRF-2022R1A2C1006462).

References

  1. Bagnoli M and Bergstrom T (2005). Log-concave probability and its applications, Economic Theory, 26, 445-469. https://doi.org/10.1007/s00199-004-0514-4
  2. Balabdaoui F, Rufibach K, and Wellner JA (2009). Limit distribution theory for maximum likelihood estimation of a log-concave density, The Annals of Statistics, 37, 1299.
  3. Breiman L and Friedman JH (1985). Estimating optimal transformations for multiple regression and correlation, Journal of the American Statistical Association, 80, 580-598. https://doi.org/10.1080/01621459.1985.10478157
  4. Chen Y and Samworth RJ (2013). Smoothed log-concave maximum likelihood estimation with applications, Statistica Sinica, 23, 1373-1398.
  5. Dharmadhikari S and Joag-Dev K (1988). Unimodality, Convexity, and Applications, Academic Press Boston, Boston.
  6. Dumbgen L, Husler A, and Rufibach K (2007). Active set and EM algorithms for log-concave densities based on complete and censored data. Available from: arXiv preprint arXiv:0707.4643
  7. Dumbgen L and Rufibach K (2009). Maximum likelihood estimation of a log-concave density and its istribution function: Basic properties and uniform consistency, Bernoulli, 15, 40-68.
  8. Dumbgen L and Rufibach K (2011). Logcondens: Computations related to univariate log-concave density estimation, Journal of Statistical Software, 39, 1-28.
  9. Dumbgen L, Samworth R, and Schuhmacher D (2011). Approximation by log-concave distributions, with applications to regression, The Annals of Statistics, 39, 702-730. https://doi.org/10.1214/10-AOS853
  10. Fan J and Li R (2001). Variable selection via nonconcave penalized likelihood and its oracle properties, Journal of the American Statistical Association, 96, 1348-1360. https://doi.org/10.1198/016214501753382273
  11. Friedman J, Hastie T, and Tibshirani R (2010). Regularization paths for generalized linear models via coordinate descent, Journal of Statistical Software, 33, 1.
  12. Hu H, Yao W, and Wu Y (2017). The robust EM-type algorithms for log-concave mixtures of regression models, Computational Statistics & Data Analysis, 111, 14-26. https://doi.org/10.1016/j.csda.2017.01.004
  13. Kim S and Seo B (2018). Linear regression under log-concave and Gaussian scale mixture errors: comparative study, Communications for Statistical Applications and Methods, 25, 633-645. https://doi.org/10.29220/CSAM.2018.25.6.633
  14. Kim S and Seo B (2021). Modal linear regression using log-concave distributions, Journal of the Korean Statistical Society, 50, 479-494. https://doi.org/10.1007/s42952-020-00089-y
  15. Konis K and Konis MK (2020). lpSolveAPI: R Interface to 'lp solve' Version 5.5.2.0, R package version 5.5.2.0-17.7.
  16. Lange KL, Little RJ, and Taylor JM (1989). Robust statistical modeling using the t-distribution, Journal of the American Statistical Association, 84, 881-896. https://doi.org/10.2307/2290063
  17. Liu Y and Wang Y (2018). A fast algorithm for univariate log-concave density estimation, Australian & New Zealand Journal of Statistics, 60, 258-275. https://doi.org/10.1111/anzs.12232
  18. Nierenberg DW, Stukel TA, Baron JA, Dain BJ, Greenberg ER, and Skin Cancer Prevention Study Group (1989). Determinants of plasma levels of beta-carotene and retinol, American Journal of Epidemiology, 130, 511-521. https://doi.org/10.1093/oxfordjournals.aje.a115365
  19. Prekopa and Andras (1971). Logarithmic concave measures with application to stochastic programming, Acta Scientiarum Mathematicarum, 32, 301-316.
  20. Rousseeuw PJ and Leroy AM (1987). Robust Regression and Outlier Detection, Wiley, New York.
  21. Schwarz G (1978). Estimating the dimension of a model, The Annals of Statistics, 6, 461-464. https://doi.org/10.1214/aos/1176344136
  22. Sherwood B and Maidman A (2020). rqPen: Penalized Quantile Regression, R package version 2.2.2.
  23. Silverman BW (1982). On the estimation of a probability density function by the maximum penalized likelihood method, The Annals of Statistics, 10, 795-810. https://doi.org/10.1214/aos/1176345872
  24. Tibshirani R (1996). Regression shrinkage and selection via the Lasso, Journal of the Royal Statistical Society, Series B, 58, 267-288.
  25. Wang H, Li G, and Jiang G (2007). Robust regression shrinkage and consistent variable selection through the LAD-Lasso, Journal of Business & Economic Statistics, 25, 347-355. https://doi.org/10.1198/073500106000000251
  26. Wang Y, Lawson CL, and Hanson RJ (2020). lsei: Solving Least Squares or Quadratic Programming Problems under Equality/Inequality Constraints, R package version 1.3-0.
  27. Zhang CH (2010). Nearly unbiased variable selection under minimax concave penalty, The Annals of Statistics, 38, 894-942.
  28. Zhang HH and Lin CY (2013). Cosso: Fit Regularized Nonparametric Regression Models Using COSSO Penalty, R package version 2.1-1.
  29. Zou H (2006). The adaptive lasso and its oracle properties, Journal of the American Statistical Association, 101, 1418-1429. https://doi.org/10.1198/016214506000000735