• 전기학회논문지P
• /
• 제58권2호
• /
• pp.164-171
• /
• 2009

This paper presents new learning algorithm of dynamic Bayesian networks (DBN) by means of constrained least square (LS) estimation algorithm and gradient descent method. First, we propose constrained LS based parameter estimation for a Markov chain (MC) model given observation data sets. Next, a gradient descent optimization is utilized for online estimation of a hidden Markov model (HMM), which is bi-linearly constructed by adding an observation variable to a MC model. We achieve numerical simulations to prove its reliability and superiority in which a series of non stationary random signal is applied for the DBN models respectively.

• 한국정보처리학회논문지
• /
• 제4권11호
• /
• pp.2769-2779
• /
• 1997

본 논문에서는 공액기울기법과 터널링 시스템을 조합사용하여 신경망의 학습성능을 향상시킬 수 있는 효율적인 방법을 제안하였다. 빠른 수렴속도의 학습을 위하여 공액 기울기법에 기초한 후향전파 알고리즘을 사용하였고, 국소최적해를 만났을 때 이를 벗어난 다른 연결가중치의 설정을 위해 동적터널링 시스템에 기초한 후향전파 알고리즘을 조합한 학습 알고리즘을 적용하였다. 제안된 방법을 패리티 검사 및 패턴분류 문제에 각각 적용하여 기존의 기울기 하강법에 기초한 후향전파 알고리즘 및 기울기 하강법과 동적터널링 시스템을 조합한 후향전파 알고리즘방법의 결과와 비교 고찰하여 제안된 방법이 다른 방법들 보다 학습성능에서 우수함을 나타내었다．

• 대한전자공학회:학술대회논문집
• /
• 대한전자공학회 1999년도 하계종합학술대회 논문집
• /
• pp.1033-1036
• /
• 1999

This paper proposes an efficient fault diagnosis for digital circuits using multilayer neural networks. The efficient learning algorithm is also proposed for the multilayer neural network, which is combined the steepest descent for high-speed optimization and the dynamic tunneling for global optimization. The fault-diagnosis system using the multilayer neural network of the proposed algorithm has been applied to the parity generator circuit. The simulation results shows that the proposed system is higher convergence speed and rate, in comparision with system using the backpropagation algorithm based on the gradient descent.

• 한국유체기계학회 논문집
• /
• 제2권1호
• /
• pp.29-34
• /
• 1999

To develop the efficient numerical optimization method for the design of an airfoil, an evaluation of various methods coupled with two-dimensional Naviev-Stokes analysis is presented. Simplex method and Hook-Jeeves method we used as direct search methods, and steepest descent method, conjugate gradient method and DFP method are used as indirect search methods and are tested to determine the search direction. To determine the moving distance, the golden section method and cubic interpolation method are tested. The finite volume method is used to discretize two-dimensional Navier-Stokes equations, and SIMPLEC algorithm is used for a velocity-pressure correction method. For the optimal design of two-dimensional airfoil, maximum thickness, maximum ordinate of camber line and chordwise position of maximum ordinate are chosen as design variables, and the ratio of drag coefficient to lift coefficient is selected as an objective function. From the results, it is found that conjugate gradient method and cubic interpolation method are the most efficient for the determination of search direction and the moving distance, respectively.

• 제어로봇시스템학회논문지
• /
• 제7권12호
• /
• pp.985-992
• /
• 2001

This paper proposes a self-organizing fuzzy modeling(SOFUM)which an create a new hyperplane shaped cluster and adjust parameters of the fuzzy model in repetition. The suggested algorithm SOFUM is composed of four steps: coarse tuning. fine tuning cluster creation and optimization of learning rates. In the coarse tuning fuzzy C-regression model(FCRM) clustering and weighted recursive least squared (WRLS) algorithm are used and in the fine tuning gradient descent algorithm is used to adjust parameters of the fuzzy model precisely. In the cluster creation, a new hyperplane shaped cluster is created by applying multiple regression to input/output data with relatively large fuzzy entropy based on parameter tunings of fuzzy model. And learning rates are optimized by utilizing meiosis-genetic algorithm in the optimization of learning rates To check the effectiveness of the suggested algorithm two examples are examined and the performance of the identified fuzzy model is demonstrated via computer simulation.

• Journal of information and communication convergence engineering
• /
• 제13권2호
• /
• pp.123-131
• /
• 2015

Training neural networks is a complex task with great importance in the field of supervised learning. In the training process, a set of input-output patterns is repeated to an artificial neural network (ANN). From those patterns weights of all the interconnections between neurons are adjusted until the specified input yields the desired output. In this paper, a new hybrid algorithm is proposed for global optimization of connection weights in an ANN. Dynamic swarms are shown to converge rapidly during the initial stages of a global search, but around the global optimum, the search process becomes very slow. In contrast, the gradient descent method can achieve faster convergence speed around the global optimum, and at the same time, the convergence accuracy can be relatively high. Therefore, the proposed hybrid algorithm combines the dynamic particle swarm optimization (DPSO) algorithm with the backpropagation (BP) algorithm, also referred to as the DPSO-BP algorithm, to train the weights of an ANN. In this paper, we intend to show the superiority (time performance and quality of solution) of the proposed hybrid algorithm (DPSO-BP) over other more standard algorithms in neural network training. The algorithms are compared using two different datasets, and the results are simulated.

• 한국수자원학회:학술대회논문집
• /
• 한국수자원학회 2018년도 학술발표회
• /
• pp.448-448
• /
• 2018

In this study, the regression models (Load ESTimator and eight-parameter model) were evaluated to estimate instantaneous pollutant loads under various criteria and optimization methods. As shown in the results, LOADEST commonly used in interpolating pollutant loads could not necessarily provide the best results with the automatic selected regression model. It is inferred that the various regression models in LOADEST need to be considered to find the best solution based on the characteristics of watersheds applied. The recently developed eight-parameter model integrated with Genetic Algorithm (GA) and Gradient Descent Method (GDM) were also compared with LOADEST indicating that the eight-parameter model performed better than LOADEST, but it showed different behaviors in calibration and validation. The eight-parameter model with GDM could reproduce the nitrogen loads properly outside of calibration period (validation). Furthermore, the accuracy and precision of model estimations were evaluated using various criteria (e.g., $R^2$ and gradient and constant of linear regression line). The results showed higher precisions with the $R^2$ values closed to 1.0 in LOADEST and better accuracy with the constants (in linear regression line) closed to 0.0 in the eight-parameter model with GDM. In hence, based on these finding we recommend that users need to evaluate the regression models under various criteria and calibration methods to provide the more accurate and precise results for pollutant load estimations.

• Journal of applied mathematics & informatics
• /
• 제34권1_2호
• /
• pp.157-165
• /
• 2016

In this paper, we develop a new hybridization conjugate gradient method for solving the unconstrained optimization problem. Under mild assumptions, we get the sufficient descent property of the given method. The global convergence of the given method is also presented under the Wolfe-type line search and the general Wolfe line search. The numerical results show that the method is also efficient.

• 한국지능시스템학회논문지
• /
• 제8권5호
• /
• pp.98-106
• /
• 1998

본 논문에서는 신경망의 학습성능을 개선하기 위해 확룰적 근사법과 공액기울기법에 기초를 둔 새로운 학습방법을 제안하였다. 제안된 방법에서는 확률적 근사법과 공액기울기법을 조합 사용한 전역 최적화 기법의 역전파 알고리즘을 적용함으로써 학습성능을 최대한 개선할 수 있도록 하였다. 확률적 근사법은 국소최소점을 벗어나 전역최적점에 치우친 근사점을 결정해 주는 기능을 하도록 하며, 이점을 초기값으로 하여 결정론적 기법의 공액기울기법을 적용함으로써 빠른 수렴속도로 전역최적점으로의 수렴확률을 놓였다. 제안된 방법을 패리티 검사와 패턴 분류에 각각 적용하여 그 타당성과 성능을 확인한 결과 제안된 방법은 초기값을 무작위로 설정하는 기울기하강법에 기초를 둔 기존의 역전파 알고리즘이나 확률적 근사법과 기울기하강법에 기초를 둔 역전파 알고리즘에 비해 최적해로의 수렴 확률과 그 수렴속도가 우수함을 확인할 수 있었다.

• Journal of Electrical Engineering and information Science
• /
• 제2권6호
• /
• pp.191-196
• /
• 1997

Fuzzy clustering has been playing an important role in solving many problems. Fuzzy c-Means(FCM) algorithm is most frequently used for fuzzy clustering. But some fixed point of FCM algorithm, know as Tucker's counter example, is not a reasonable solution. Moreover, FCM algorithm is impossible to perform the on-line learning since it is basically a batch learning scheme. This paper presents unsupervised learning networks as an attempt to improve shortcomings of the conventional clustering algorithm. This model integrates optimization function of FCM algorithm into unsupervised learning networks. The learning rule of the proposed scheme is a result of formal derivation based on the gradient descent procedure of a fuzzy objective function. Using the result of formal derivation, two algorithms of fuzzy cluster analysis, the batch learning version and on-line learning version, are devised. They are tested on several data sets and compared with FCM. The experimental results show that the proposed algorithms find out the reasonable solution on Tucker's counter example.