• Title/Summary/Keyword: Catastrophic forgetting

Search Result 12, Processing Time 0.017 seconds

Regularization Strength Control for Continuous Learning based on Attention Transfer (어텐션 기반의 지속학습에서 정규화값 제어 방법)

  • Kang, Seok-Hoon;Park, Seong-Hyeon
    • Journal of IKEEE
    • /
    • v.26 no.1
    • /
    • pp.19-26
    • /
    • 2022
  • In this paper, we propose an algorithm that applies a different variable lambda to each loss value to solve the performance degradation caused by domain differences in LwF, and show that the retention of past knowledge is improved. The lambda value could be variably adjusted so that the current task to be learned could be well learned, by the variable lambda method of this paper. As a result of learning by this paper, the data accuracy improved by an average of 5% regardless of the scenario. And in particular, the performance of maintaining past knowledge, the goal of this paper, was improved by up to 70%, and the accuracy of past learning data increased by an average of 22% compared to the existing LwF.

Anchor Free Object Detection Continual Learning According to Knowledge Distillation Layer Changes (Knowledge Distillation 계층 변화에 따른 Anchor Free 물체 검출 Continual Learning)

  • Gang, Sumyung;Chung, Daewon;Lee, Joon Jae
    • Journal of Korea Multimedia Society
    • /
    • v.25 no.4
    • /
    • pp.600-609
    • /
    • 2022
  • In supervised learning, labeling of all data is essential, and in particular, in the case of object detection, all objects belonging to the image and to be learned have to be labeled. Due to this problem, continual learning has recently attracted attention, which is a way to accumulate previous learned knowledge and minimize catastrophic forgetting. In this study, a continaul learning model is proposed that accumulates previously learned knowledge and enables learning about new objects. The proposed method is applied to CenterNet, which is a object detection model of anchor-free manner. In our study, the model is applied the knowledge distillation algorithm to be enabled continual learning. In particular, it is assumed that all output layers of the model have to be distilled in order to be most effective. Compared to LWF, the proposed method is increased by 23.3%p mAP in 19+1 scenarios, and also rised by 28.8%p in 15+5 scenarios.