• Bulletin of the Korean Mathematical Society
• /
• v.55 no.4
• /
• pp.1303-1315
• /
• 2018

For principal component analysis (PCA) to efficiently analyze large scale matrices, it is crucial to find a few singular vectors in cheaper computational cost and under lower memory requirement. To compute those in a fast and robust way, we propose a new stochastic method. Especially, we adopt the stochastic variance reduced gradient (SVRG) method [11] to avoid asymptotically slow convergence in stochastic gradient descent methods. For that purpose, we reformulate the PCA problem as a unconstrained optimization problem using a quadratic penalty. In general, increasing the penalty parameter to infinity is needed for the equivalence of the two problems. However, in this case, exact penalization is guaranteed by applying the analysis in [24]. We establish the convergence rate of the proposed method to a stationary point and numerical experiments illustrate the validity and efficiency of the proposed method.

• Journal of the Korean Society for Industrial and Applied Mathematics
• /
• v.25 no.4
• /
• pp.162-172
• /
• 2021

This paper proposes stochastic methods to find an approximate solution for the L²-Wasserstein least squares problem of Gaussian measures. The variable for the problem is in a set of positive definite matrices. The first proposed stochastic method is a type of classical stochastic gradient methods combined with projection and the second one is a type of variance reduced methods with projection. Their global convergence are analyzed by using the framework of proximal stochastic gradient methods. The convergence of the classical stochastic gradient method combined with projection is established by using diminishing learning rate rule in which the learning rate decreases as the epoch increases but that of the variance reduced method with projection can be established by using constant learning rate. The numerical results show that the present algorithms with a proper learning rate outperforms a gradient projection method.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• v.14 no.11
• /
• pp.4355-4371
• /
• 2020

Phase retrieval, recovering a signal from phaseless measurements, is generally considered to be an NP-hard problem. This paper adopts an amplitude-based nonconvex optimization cost function to develop a new stochastic gradient algorithm, named stochastic reweighted phase retrieval (SRPR). SRPR is a stochastic gradient iteration algorithm, which runs in two stages: First, we use a truncated sample stochastic variance reduction algorithm to initialize the objective function. The second stage is the gradient refinement stage, which uses continuous updating of the amplitude-based stochastic weighted gradient algorithm to improve the initial estimate. Because of the stochastic method, each iteration of the two stages of SRPR involves only one equation. Therefore, SRPR is simple, scalable, and fast. Compared with the state-of-the-art phase retrieval algorithm, simulation results show that SRPR has a faster convergence speed and fewer magnitude-only measurements required to reconstruct the signal, under the real- or complex- cases.

• Structural Engineering and Mechanics
• /
• v.32 no.1
• /
• pp.21-36
• /
• 2009

An importance sampling method is presented for computing the first passage probability of elasto-plastic structures under stochastic excitations. The importance sampling distribution corresponds to shifting the mean of the excitation to an 'adapted' stochastic process whose future is determined based on information only up to the present. A stochastic control approach is adopted for designing the adapted process. The optimal control law is determined by a control potential, which satisfies the Bellman's equation, a nonlinear partial differential equation on the response state-space. Numerical results for a single-degree-of freedom elasto-plastic structure shows that the proposed method leads to significant improvement in variance reduction over importance sampling using design points reported recently.

• Journal of the Korea Society for Simulation
• /
• v.9 no.4
• /
• pp.67-75
• /
• 2000

Often system analysts are interested in the estimation of percentile for system performance. For instance, in PERT network system, the percentile that the project. Typically the control variate method is used to reduce the variability of mean response using the correlation between the response and the control variates with a little additional cost during the course of simulation. In the same spirit, we apply this method to estimate the percentile of project completion time in PERT system, and evaluate the efficiency of the controlled estimator for its percentile.1 Simulation results indicate that the controlled estimators are more effective in reducing the variances of estimators than the simple estimators, however those tend to a little underestimate the percentiles for some critical values. We need more simulation experiments to examine such a kind of bias problem. We expect this research presents a step forward in the area of variance reduction techniques of stochastic simulation.

• Journal of the Korea Computer Graphics Society
• /
• v.16 no.1
• /
• pp.9-20
• /
• 2010

In order to visualize participating media in 3D space, they usually calculate the incoming radiance by subdividing the ray path into small subintervals, and accumulating their respective light energy due to direct illumination, scattering, absorption, and emission. Among these light phenomena, scattering behaves in very complicated manner in 3D space, often requiring a great deal of simulation efforts. To effectively simulate the light scattering effect, several approximation techniques have been proposed. Volume photon mapping takes a simple approach where the light scattering phenomenon is represented in volume photon map through a stochastic simulation, and the stored information is explored in the rendering stage. While effective, this method has a problem that the number of necessary photons increases very fast when a higher variance reduction is needed. In an attempt to resolve such problem, we propose a different approach for rendering particle-based volume data where kernel smoothing, one of several density estimation methods, is explored to represent and reconstruct the light in-scattering effect. The effectiveness of the presented technique is demonstrated with several examples of volume data.