• Title/Summary/Keyword: forgetting

Search Result 166, Processing Time 0.022 seconds

A study on sequential iterative learning for overcoming catastrophic forgetting phenomenon of artificial neural network (인공 신경망의 Catastrophic forgetting 현상 극복을 위한 순차적 반복 학습에 대한 연구)

  • Choi, Dong-bin;Park, Young-beom
    • Journal of Platform Technology
    • /
    • v.6 no.4
    • /
    • pp.34-40
    • /
    • 2018
  • Currently, artificial neural networks perform well for a single task, but NN have the problem of forgetting previous learning by learning other kinds of tasks. This is called catastrophic forgetting. To use of artificial neural networks in general purpose this should be solved. There are many efforts to overcome catastrophic forgetting. However, even though there was a lot of effort, it did not completely overcome the catastrophic forgetting. In this paper, we propose sequential iterative learning using core concepts used in elastic weight consolidation (EWC). The experiment was performed to reproduce catastrophic forgetting phenomenon using EMNIST data set which extended MNIST, which is widely used for artificial neural network learning, and overcome it through sequential iterative learning.

The Effect of Worker Heterogeneity in Learning and Forgetting on System Productivity (학습과 망각에 대한 작업자들의 이질성 정도가 시스템 생산성에 미치는 영향)

  • Kim, Sungsu
    • Journal of the Korean Operations Research and Management Science Society
    • /
    • v.40 no.4
    • /
    • pp.145-156
    • /
    • 2015
  • Incorporation of individual learning and forgetting behaviors within worker-task assignment models produces a mixed integer nonlinear program (MINLP) problem, which is difficult to solve as a NP hard due to its nonlinearity in the objective function. Previous studies commonly assume homogeneity among workers in workforce scheduling that takes account of learning and forgetting characteristics. This paper expands previous researches by considering heterogeneous individual learning/forgetting, and investigates the impact of worker heterogeneity in initial expertise, steady-state productivity, learning and forgetting on system performance to assist manager's decision-making in worker-task assignments without tackling complex MINLP models. In order to understand the performance implications of workforce heterogeneity, this paper examines analytically how heterogeneity in each of the four parameters of the exponential learning and forgetting (L/F) model affects system performance in three cases : consecutive assignments with no break, n breaks of s-length each, and total b break-periods occurred over T periods. The study presents the direction of change in worker performance under different assignment schedules as the variance in initial expertise, steady-state productivity, learning or forgetting increases. Thus, it implies whether having more heterogenous workforce in terms of each of four parameters in the L/F model is desired or not in different schedules from the perspective of system productivity measurement.

Time Variant Parameter Estimation using RLS Algorithm with Adaptive Forgetting Factor Based on Newton-Raphson Method (Newton-Raphson법 기반의 적응 망각율을 갖는 RLS 알고리즘에 의한 원격센서시스템의 시변파라메타 추정)

  • Kim, Kyung-Yup;Lee, Joon-Tark
    • Proceedings of the Korean Institute of Intelligent Systems Conference
    • /
    • 2007.04a
    • /
    • pp.435-439
    • /
    • 2007
  • This paper deals with RLS algorithm using Newton-Raphson method based adaptive forgetting factor for a passive telemetry RF sensor system in order to estimate the time variant parameter to be included in RF sensor model. For this estimation with RLS algorithm, phasor typed RF sensor system modelled with inductive coupling principle is used. Instead of applying constant forgetting factor to estimate time variant parameter, the adaptive forgetting factor based on Newton-Raphson method is applied to RLS algorithm without constant forgetting factor to be determined intuitively. Finally, we provide numerical examples to evaluate the feasibility and generality of the proposed method in this paper.

  • PDF

Kernel RLS Algorithm Using Variable Forgetting Factor (가변 망각인자를 사용한 커널 RLS 알고리즘)

  • Lim, Jun-Seok;Pyeon, Yong-Guk
    • The Journal of Korean Institute of Communications and Information Sciences
    • /
    • v.40 no.9
    • /
    • pp.1793-1801
    • /
    • 2015
  • In a recent work, kernel recursive least-squares tracker (KRLS-T) algorithm has been proposed. It is capable of tracking in non-stationary environments using a forgetting mechanism built on a Bayesian framework. The forgetting mechanism in KRLS-T is implemented by a fixed forgetting factor. In practice, however, we frequently meet that the fixed forgetting factor cannot handle time-varying system effectively. In this paper we propose a new KRLS-T with a variable forgetting factor. Experimental results show that proposed algorithm can handle time-varying system more effectively than the KRLS-T.

An Improved New RLS Algorithm with Forgetting Factor of Erlang Function for System Identification (시스템 식별을 위한 Erlang 함수의 망각 인자를 가진 개선된 RLS 알고리즘)

  • Seok, Jin-Wuk;Choi, Kyung-Sam;Lee, Jong-Soo;Cho, Seong-Won
    • Journal of Institute of Control, Robotics and Systems
    • /
    • v.5 no.4
    • /
    • pp.394-402
    • /
    • 1999
  • In this paper, we present an effective RLS algorithm with forgetting factor of Erlang function for the system identification. In the proposed algorithm, the forgetting factor decreases monotonically in the first stage, and then it increases monotonically in the second stage in contrary to the conventional forgetting factor RLS algorithms. In addition, annealing effect and an asymptotically stability of the proposed algorithm is discussed based on the analysis of convergency property on. Simulation results for the system identification problem indicate the superiority of the proposed algorithm in comparison to the RLS algorithm such as NLMS and Kalman filter based algorithm.

  • PDF

Low Complexity Gauss Newton Variable Forgetting Factor RLS for Time Varying System Estimation (시변 시스템 추정을 위한 연산량이 적은 가우스 뉴턴 가변 망각인자를 사용하는 RLS 알고리즘)

  • Lim, Jun-Seok;Pyeon, Yong-Guk
    • The Journal of Korean Institute of Communications and Information Sciences
    • /
    • v.41 no.9
    • /
    • pp.1141-1145
    • /
    • 2016
  • In general, a variable forgetting factor is applied to the RLS algorithm for the time-varying parameter estimation in the non-stationary environments. The introduction of a variable forgetting factor to RLS needs heavy additional calculation complexity. We propose a new Gauss Newton variable forgetting factor RLS algorithm which needs small amount of calculation as well as estimates the better parameters in time-varying nonstationary environment. The algorithm performs as good as the conventional Gauss Newton variable forgetting factor RLS and the required additional calculation complexity reduces from $O(N^2)$ to O(N).

Online Estimation of Rotational Inertia of an Excavator Based on Recursive Least Squares with Multiple Forgetting

  • Oh, Kwangseok;Yi, Kyong Su;Seo, Jaho;Kim, Yongrae;Lee, Geunho
    • Journal of Drive and Control
    • /
    • v.14 no.3
    • /
    • pp.40-49
    • /
    • 2017
  • This study presents an online estimation of an excavator's rotational inertia by using recursive least square with forgetting. It is difficult to measure rotational inertia in real systems. Against this background, online estimation of rotational inertia is essential for improving safety and automation of construction equipment such as excavators because changes in inertial parameter impact dynamic characteristics. Regarding an excavator, rotational inertia for swing motion may change significantly according to working posture and digging conditions. Hence, rotational inertia estimation by predicting swing motion is critical for enhancing working safety and automation. Swing velocity and damping coefficient were used for rotational inertia estimation in this study. Updating rules are proposed for enhancing convergence performance by using the damping coefficient and forgetting factors. The proposed estimation algorithm uses three forgetting factors to estimate time-varying rotational inertia, damping coefficient, and torque with different variation rates. Rotational inertia in a typical working scenario was considered for reasonable performance evaluation. Three simulations were conducted by considering several digging conditions. Presented estimation results reveal the proposed estimation scheme is effective for estimating varying rotational inertia of the excavator.

Robust Sequential Estimation based on t-distribution with forgetting factor for time-varying speech (망각소자를 갖는 t-분포 강인 연속 추정을 이용한 음성 신호 추정에 관한 연구)

  • 이주헌
    • Proceedings of the Acoustical Society of Korea Conference
    • /
    • 1998.08a
    • /
    • pp.470-474
    • /
    • 1998
  • In this paper, to estimate the time-varying parameters of speech signal, we use the robust sequential estimator based on t-distribution and, for time-varying signal, introduce the forgetting factor. By using the RSE based on t-distribution with small degree of freedom, we can alleviate efficiently the effects of outliers to obtain the better performance of parameter estimation. Moreover, by the forgetting factor, the proposed algorithm can estimate the accurate parameters under the rapid variation of speech signal.

  • PDF

A Study on Reduced Variance Self-Tuning Algorithm Using a Variable Forgetting Factor (시변 망각 인자를 사용하는 최소 자승 추정의 극점 -배치 자기동조 알고리즘에 관한 연구)

  • Park, Chan-Young;Do, Mi-Sun;Park, Mi-Gnon;Lee, Sang-Bae
    • Proceedings of the KIEE Conference
    • /
    • 1988.07a
    • /
    • pp.305-308
    • /
    • 1988
  • Pole assignment controller with variable forgetting factor is generalizaed to allow the output and/or input variance to be reduced. The algorithm can give significant reductions in variance for little extra computational effort and is presented for servo-tracking using leat-squares estimation. Moreover, the use of a variable forgetting factor with correct choice of information bound can avoid 'blowing-up' of the covariance matrix of the estimates and subsequent unstable control.

  • PDF

Continual Learning using Data Similarity (데이터 유사도를 이용한 지속적 학습방법)

  • Park, Seong-Hyeon;Kang, Seok-Hoon
    • Journal of IKEEE
    • /
    • v.24 no.2
    • /
    • pp.514-522
    • /
    • 2020
  • In Continuous Learning environment, we identify that the Catastrophic Forgetting phenomenon, which forgets the information of previously learned data, occurs easily between data having different domains. To control this phenomenon, we introduce how to measure the relationship between previously learned data and newly learned data through the distribution of the neural network's output, and how to use these measurements to mitigate the Catastrophic Forcing phenomenon. MNIST and EMNIST data were used for evaluation, and experiments showed an average 22.37% improvement in accuracy for previous data.