• Communications for Statistical Applications and Methods
• /
• v.22 no.2
• /
• pp.147-157
• /
• 2015

We consider a sparse high-dimensional linear regression model. Penalized methods using LASSO or non-convex penalties have been widely used for variable selection and estimation in high-dimensional regression models. In penalized regression, the selection and prediction performances depend on which penalty function is used. For example, it is known that LASSO has a good prediction performance but tends to select more variables than necessary. In this paper, we propose an additive sparse penalty for variable selection using a combination of LASSO and minimax concave penalties (MCP). The proposed penalty is designed for good properties of both LASSO and MCP.We develop an efficient algorithm to compute the proposed estimator by combining a concave convex procedure and coordinate descent algorithm. Numerical studies show that the proposed method has better selection and prediction performances compared to other penalized methods.

• The Transactions of the Korean Institute of Electrical Engineers D
• /
• v.53 no.11
• /
• pp.747-752
• /
• 2004

This paper presents an iterative linear matrix inequality (LMI) approach to the design of a static output feedback (SOF) stabilization controller. A linear penalty function is incorporated into the objective function for the non-convex rank constraint so that minimizing the penalized objective function subject to LMIs amounts to a convex optimization problem. Hence, the overall procedure results in solving a series of semidefinite programs (SDPs). With an increasing sequence of the penalty parameter, the solution of the penalized optimization problem moves towards the feasible region of the original non-convex problem. The proposed algorithm is, therefore, convergent. Extensive numerical experiments are Deformed to illustrate the proposed algorithm.

• The Transactions of the Korean Institute of Electrical Engineers A
• /
• v.53 no.12
• /
• pp.655-660
• /
• 2004

This paper deals with the design of a fixed-structure $H_\infty$ power system stabilizer (PSS) by using an iterative linear matrix inequality (LMI) method. The fixed-structure $H_\infty$ controller is represented in terms of LMIs with a rank condition. To solve the non-convex rank-constrained LMI problem, a linear penalty function is incorporated into the objective function so that minimizing the penalized objective function subject to LMIs amounts to a convex optimization problem. With an increasing sequence of the penalty parameter, the solution of the penalized optimization problem moves towards the feasible region of the original non-convex problem. The proposed algorithm is, therefore, convergent. Numerical experiments show the practical applicability of the proposed algorithm.

• Communications for Statistical Applications and Methods
• /
• v.27 no.1
• /
• pp.129-140
• /
• 2020

In this paper, we introduce an efficient algorithm for the non-convex penalized multinomial logistic regression that can be uniformly applied to a class of non-convex penalties. The class includes most non-convex penalties such as the smoothly clipped absolute deviation, minimax concave and bridge penalties. The algorithm is developed based on the concave-convex procedure and modified local quadratic approximation algorithm. However, usual quadratic approximation may slow down computational speed since the dimension of the Hessian matrix depends on the number of categories of the output variable. For this issue, we use a uniform bound of the Hessian matrix in the quadratic approximation. The algorithm is available from the R package ncpen developed by the authors. Numerical studies via simulations and real data sets are provided for illustration.

• Proceedings of the KIEE Conference
• /
• 2004.11c
• /
• pp.729-731
• /
• 2004

This paper deals with the design of a low order $H_{\infty}$ controller by using an iterative linear matrix inequality (LMI) method. The low order $H_{\infty}$ controller is represented in terms of LMIs with a rank condition. To solve the non-convex rank-constrained LMI problem, a linear penalty function is incorporated into the objective function so that minimizing the penalized objective function subject to LMIs amounts to a convex optimization problem. With an increasing sequence of the penalty parameter, the solution of the penalized optimization problem moves towards the feasible region of the original non-convex problem. The proposed algorithm is, therefore, convergent. Numerical experiments show the effectiveness of the proposed algorithm.

• Communications for Statistical Applications and Methods
• /
• v.25 no.5
• /
• pp.453-470
• /
• 2018

We study how to distinguish the parameters of the sparse autoregressive (AR) process from zero using a non-convex penalized estimation. A class of non-convex penalties are considered that include the smoothly clipped absolute deviation and minimax concave penalties as special examples. We prove that the penalized estimators achieve some standard theoretical properties such as weak and strong oracle properties which have been proved in sparse linear regression framework. The results hold when the maximal order of the AR process increases to infinity and the minimal size of true non-zero parameters decreases toward zero as the sample size increases. Further, we construct a practical method to select tuning parameters using generalized information criterion, of which the minimizer asymptotically recovers the best theoretical non-penalized estimator of the sparse AR process. Simulation studies are given to confirm the theoretical results.

• Communications for Statistical Applications and Methods
• /
• v.19 no.5
• /
• pp.655-662
• /
• 2012

Classification is an important research field in pattern recognition with high-dimensional predictors. The support vector machine(SVM) is a penalized feature selector and classifier. It is based on the hinge loss function, the non-convex penalty function, and the smoothly clipped absolute deviation(SCAD) suggested by Fan and Li (2001). We developed the algorithm for the multiclass SVM with the SCAD penalty function using the local quadratic approximation. For multiclass problems we compared the performance of the SVM with the $L_1$, $L_2$ penalty functions and the developed method.

• Journal of Institute of Control, Robotics and Systems
• /
• v.11 no.4
• /
• pp.279-283
• /
• 2005

This paper deals with the design of a low-order H/sub ∞/ controller by using an iterative linear matrix inequality (LMI) method. The low-order H/sub ∞/ controller is represented in terms of LMIs with a rank condition. To solve the non-convex rank-constrained LMI problem, the recently developed penalty function method is applied. With an increasing sequence of the penalty parameter, the solution of the penalized optimization problem moves towards the feasible region of the original non-convex problem. Numerical experiments showed the effectiveness of the proposed algorithm.

• Journal of Biomedical Engineering Research
• /
• v.36 no.5
• /
• pp.211-220
• /
• 2015

Recently, model-based iterative reconstruction (MBIR) has played an important role in transmission tomography by significantly improving the quality of reconstructed images for low-dose scans. MBIR is based on the penalized-likelihood (PL) approach, where the penalty term (also known as the regularizer) stabilizes the unstable likelihood term, thereby suppressing the noise. In this work we further improve MBIR by using a more expressive regularizer which can restore the underlying image more accurately. Here we used a spline regularizer derived from a linear combination of the two-dimensional splines with first- and second-order spatial derivatives and applied it to a non-quadratic convex penalty function. To derive a PL algorithm with the spline regularizer, we used a separable paraboloidal surrogates algorithm for convex optimization. The experimental results demonstrate that our regularization method improves reconstruction accuracy in terms of both regional percentage error and contrast recovery coefficient by restoring smooth edges as well as sharp edges more accurately.

• The Journal of Korean Institute of Communications and Information Sciences
• /
• v.29 no.10C
• /
• pp.1402-1413
• /
• 2004

In non-rigid image registration, the Jacobian determinant of the estimated deformation should be positive everywhere since physical deformations are always invertible. We propose a constrained optimization technique at ensures the positiveness of Jacobian determinant for cubic B-spline based deformation. We derived sufficient conditions for positive Jacobian determinant by bounding the differences of consecutive coefficients. The parameter set that satisfies the conditions is convex; it is the intersection of simple half spaces. We solve the optimization problem using a gradient projection method with Dykstra's cyclic projection algorithm. Analytical results, simulations and experimental results with inhale/exhale CT images with comparison to other methods are presented.