• 제목/요약/키워드: gradient descent optimization

검색결과 82건 처리시간 0.026초

TYPE-2 퍼지 추론 구동형 RBF 신경 회로망 설계 및 최적화 (Design of Radial Basis Function Neural Network Driven to TYPE-2 Fuzzy Inference and Its Optimization)

  • 백진열;김웅기;오성권;김현기
    • 대한전기학회:학술대회논문집
    • /
    • 대한전기학회 2008년도 학술대회 논문집 정보 및 제어부문
    • /
    • pp.247-248
    • /
    • 2008
  • 본 논문에서는 TYPE-2 퍼지 추론 기반의 RBF 뉴럴 네트워크(TYPE-2 Radial Basis Function Neural Network, T2RBFNN)를 설계하고 PSO(Particle Swarm Optimization) 알고리즘을 이용하여 모델의 파라미터를 동정한다. 제안된 모델의 은닉층은 TYPE-2 가우시안 활성 함수로 구성되며, 출력층은 Interval set 형태의 연결가중치를 갖는다. 여기에서 규칙 전반부 활성함수의 중심 선택은 C-means 클러스터링 알고리즘을 이용하고, 규칙 후반부 Interval set 형태의 연결가중치 결정에는 경사 하강법(Gradient descent method)을 이용한 오류 역전파 알고리즘을 사용하여 학습한다. 또한, 최적의 모델을 설계하기 위한 학습율 및 활성함수의 활성화 영역 결정에는 입자 군집 최적화(PSO; Particle Swarm Optimization) 알고리즘으로 동조한다. 마지막으로, 제안된 모델의 평가를 위하여 모의 데이터 집합(Synthetic dadaset)을 적용하고 근사화 및 일반화 능력에 대하여 토의한다.

  • PDF

확률적 근사법과 후형질과 알고리즘을 이용한 다층 신경망의 학습성능 개선 (Improving the Training Performance of Multilayer Neural Network by Using Stochastic Approximation and Backpropagation Algorithm)

  • 조용현;최흥문
    • 전자공학회논문지B
    • /
    • 제31B권4호
    • /
    • pp.145-154
    • /
    • 1994
  • This paper proposes an efficient method for improving the training performance of the neural network by using a hybrid of a stochastic approximation and a backpropagation algorithm. The proposed method improves the performance of the training by appliying a global optimization method which is a hybrid of a stochastic approximation and a backpropagation algorithm. The approximate initial point for a stochastic approximation and a backpropagation algorihtm. The approximate initial point for fast global optimization is estimated first by applying the stochastic approximation, and then the backpropagation algorithm, which is the fast gradient descent method, is applied for a high speed global optimization. And further speed-up of training is made possible by adjusting the training parameters of each of the output and the hidden layer adaptively to the standard deviation of the neuron output of each layer. The proposed method has been applied to the parity checking and the pattern classification, and the simulation results show that the performance of the proposed method is superior to that of the backpropagation, the Baba's MROM, and the Sun's method with randomized initial point settings. The results of adaptive adjusting of the training parameters show that the proposed method further improves the convergence speed about 20% in training.

  • PDF

C-Means 클러스터링 기반의 Type-2 퍼지 논리 시스템을 이용한 비선형 모델 설계 (Design of Nonlinear Model Using Type-2 Fuzzy Logic System by Means of C-Means Clustering)

  • 백진열;이영일;오성권
    • 한국지능시스템학회논문지
    • /
    • 제18권6호
    • /
    • pp.842-848
    • /
    • 2008
  • 본 논문에서는 비선형 모델의 설계를 위해 Type-2 퍼지 논리 집합을 이용하여 불확실성 문제를 다룬다. 제안된 모델은 규칙의 전 후반부가 Type-2 퍼지 집합으로 주어진 Type-2 퍼지 논리 시스템을 설계하고 불확실성의 변화에 대한 비선형 모델의 성능을 해석한다 여기서 규칙 전반부 멤버쉽 함수의 정점 선택은 C-means 클러스터링 알고리즘을 이용하고, 규칙 무반부 퍼지 집합의 정점 결정에는 경사 하강법(Gradient descent method)을 이용한 오류 역전파 알고리즘을 사용하여 학습한다. 또한, 제안된 모델에 관련된 파라미터는 입자 군집 최적화(Particle Swarm Optimization; PSO) 알고리즘으로 동조한다. 제안된 모델은 모의 데이터집합(Synthetic dadaset), Mackey-Glass 시계열 공정 데이터를 적용하여 논증되고, 기존 Type-1 퍼지 논리 시스템과의 근사화 및 일반화 능력에 대하여 비교 토의한다.

퍼지 모델을 이용한 신경망의 학습률 조정 (Tuning Learning Rate in Neural Network Using Fuzzy Model)

  • 라혁주;서재용;김성주;전홍태
    • 대한전자공학회:학술대회논문집
    • /
    • 대한전자공학회 2003년도 하계종합학술대회 논문집 Ⅲ
    • /
    • pp.1239-1242
    • /
    • 2003
  • The neural networks are a famous model to learn the nonlinear function or nonlinear system. The main point of neural network is that the difference actual output from desired output is used to update weights. Usually, the gradient descent method is used for the learning process. On training process, if learning rate is too large, neural networks hardly guarantee convergence of neural networks. On the other hand, if learning rate is too small, the training spends much time. Therefore, one major problem in use of neural networks are to decrease the teaming time while neural networks are guaranteed convergence. In this paper, we suggest the model of fuzzy logic to neural networks to calibrate learning rate. This method is to tune learning rate dynamically according to error and demonstrates the optimization of training.

  • PDF

Crack Identification Using Neuro-Fuzzy-Evolutionary Technique

  • Shim, Mun-Bo;Suh, Myung-Won
    • Journal of Mechanical Science and Technology
    • /
    • 제16권4호
    • /
    • pp.454-467
    • /
    • 2002
  • It has been established that a crack has an important effect on the dynamic behavior of a structure. This effect depends mainly on the location and depth of the crack. Toidentifythelocation and depth of a crack in a structure, a method is presented in this paper which uses neuro-fuzzy-evolutionary technique, that is, Adaptive-Network-based Fuzzy Inference System (ANFIS) solved via hybrid learning algorithm (the back-propagation gradient descent and the least-squares method) and Continuous Evolutionary Algorithms (CEAs) solving sir ale objective optimization problems with a continuous function and continuous search space efficiently are unified. With this ANFIS and CEAs, it is possible to formulate the inverse problem. ANFIS is used to obtain the input(the location and depth of a crack) - output(the structural Eigenfrequencies) relation of the structural system. CEAs are used to identify the crack location and depth by minimizing the difference from the measured frequencies. We have tried this new idea on beam structures and the results are promising.

적응 퍼지-뉴럴 네트워크를 이용한 비선형 공정의 On-line 모델링 (On-line Modeling for Nonlinear Process Systems using the Adaptive Fuzzy-Neural Network)

  • 박춘성;오성권;김현기
    • 대한전기학회:학술대회논문집
    • /
    • 대한전기학회 1998년도 추계학술대회 논문집 학회본부 B
    • /
    • pp.537-539
    • /
    • 1998
  • In this paper, we construct the on-line model structure for the nonlinear process systems using the adaptive fuzzy-neural network. Adaptive fuzzy-neural network usually consists of two distinct modifiable structure, with both, the premise and the consequent part. These two parts can be adapted by different optimization methods, which are the hybrid learning procedure combining gradient descent method and least square method. To achieve the on-line model structure, we use the recursive least square method for the consequent parameter identification of nonlinear process. We design the interface between PLC and main computer, and construct the monitoring and control simulator for the nonlinear process. The proposed on-line modeling to real process is carried out to obtain the effective and accurate results.

  • PDF

웨이블릿 신경 회로망을 이용한 이동 로봇의 경로 추종 제어 (Path Tracking Control Using a Wavelet Neural Network for Mobile Robots)

  • 오준섭;박진배;최윤호
    • 대한전기학회:학술대회논문집
    • /
    • 대한전기학회 2003년도 하계학술대회 논문집 D
    • /
    • pp.2414-2416
    • /
    • 2003
  • In this raper, we present a Wavelet Neural Network(WNN) approach to the solution of the tracking problem for mobile robots that possess complexity, nonlinearity and uncertainty. The neural network is constructed by the wavelet orthogonal decomposition to form a wavelet neural network that can overcome the problems caused by local minima of optimization and various uncertainties. This network structure is helpful to determine the number of the hidden nodes and the initial value of weights with compact structure. In our control method, the control signals are directly obtained by minimizing the difference between the reference track and the pose of a mobile robot that is controlled through a wavelet neural network. The control process is a dynamic on-line process that uses the wavelet neural network trained by the gradient-descent method. Through computer simulations, we demonstrate the effectiveness and feasibility of the proposed control method.

  • PDF

비선형 전기부하구성 예측을 위한 최적화 기법 비교 평가 (Numerical Optimization Applied to Estimate the Composition of Nonlinear Electric Loads)

  • 이순;박정욱
    • 대한전기학회:학술대회논문집
    • /
    • 대한전기학회 2007년도 제38회 하계학술대회
    • /
    • pp.576-577
    • /
    • 2007
  • 본 논문은 전력시스템 수용가를 구성하는 비선형 부하에 흐르는 왜곡된 파형을 가진 전류의 상대적 비율을 결정하여 비선형 전기부하구성을 예측하기 위한 연구이다. 본 논문에서는 수용가 전기부하 구성 예측을 위한 해결 절차로써 수용가의 수리적 모델링을 통한 시스템 방정식을 도출하였고 최적화 이론을 적용하였다. 또한, 본 시스템에 적용한 최적화 알고리즘으로 steepest descent, conjugate gradient, Broydon-Fletcher-Goldfarb-Shanno (BFGS) 기법을 사용하였고, 예측된 결과들의 성능을 나타내는 지표인 수렴 속도와 정확도 비교를 통하여 분산 전력시스템의 전기부하구성 예측을 위해 BFGS 기법을 적용하는 것이 가장 효율적인 방안임을 보였다.

  • PDF

이동 물체 포착을 위한 비젼 서보 제어 시스템 개발 (Development of Visual Servo Control System for the Tracking and Grabbing of Moving Object)

  • 최규종;조월상;안두성
    • 동력기계공학회지
    • /
    • 제6권1호
    • /
    • pp.96-101
    • /
    • 2002
  • In this paper, we address the problem of controlling an end-effector to track and grab a moving target using the visual servoing technique. A visual servo mechanism based on the image-based servoing principle, is proposed by using visual feedback to control an end-effector without calibrated robot and camera models. Firstly, we consider the control problem as a nonlinear least squares optimization and update the joint angles through the Taylor Series Expansion. And to track a moving target in real time, the Jacobian estimation scheme(Dynamic Broyden's Method) is used to estimate the combined robot and image Jacobian. Using this algorithm, we can drive the objective function value to a neighborhood of zero. To show the effectiveness of the proposed algorithm, simulation results for a six degree of freedom robot are presented.

  • PDF

Trust-Region ICA 알고리듬 (A Trust-Region ICA algorithm)

  • Park, Heeyoul;Kim, Sookjeong;Park, Seungjin
    • 한국정보과학회:학술대회논문집
    • /
    • 한국정보과학회 2004년도 봄 학술발표논문집 Vol.31 No.1 (B)
    • /
    • pp.721-723
    • /
    • 2004
  • A trust-region method is a quite attractive optimization technique. It is, in general, faster than the steepest descent method and is free of a learning rate unlike the gradient-based methods. In addition to its convergence property (between linear and quadratic convergence), ifs stability is always guaranteed, in contrast to the Newton's method. In this paper, we present an efficient implementation of the maximum likelihood independent component analysis (ICA) using the trust-region method, which leads to trust-region-based ICA (TR-ICA) algorithms. The useful behavior of our TR-ICA algorithms is confimed through numerical experimental results.

  • PDF