International journal of advanced smart convergence

The Institute of Internet, Broadcasting and Communication (한국인터넷방송통신학회)
권호 표지
DOI QR코드

DOI QR Code

Learning Deep Representation by Increasing ConvNets Depth for Few Shot Learning

  • Fabian, H.S. Tan (Department of Computer Engineering, Dongseo University) ;
  • Kang, Dae-Ki (Department of Computer Engineering, Dongseo University)
  • Received : 2019.10.02
  • Accepted : 2019.10.16
  • Published : 2019.12.31
Download PDF
⟨ Previous Next ⟩

Abstract

Though recent advancement of deep learning methods have provided satisfactory results from large data domain, somehow yield poor performance on few-shot classification tasks. In order to train a model with strong performance, i.e. deep convolutional neural network, it depends heavily on huge dataset and the labeled classes of the dataset can be extremely humongous. The cost of human annotation and scarcity of the data among the classes have drastically limited the capability of current image classification model. On the contrary, humans are excellent in terms of learning or recognizing new unseen classes with merely small set of labeled examples. Few-shot learning aims to train a classification model with limited labeled samples to recognize new classes that have neverseen during training process. In this paper, we increase the backbone depth of the embedding network in orderto learn the variation between the intra-class. By increasing the network depth of the embedding module, we are able to achieve competitive performance due to the minimized intra-class variation.

Keywords

References

  1. A. Krizhevsky, I. Sutskever, and G. E. Hinton, "Imagenet Classification with Deep Convolutional Neural Networks," in Proc. NIPS, pp. 1106-1114, 2012.
  2. H. Edwards and A. Storkey, "Towards a Neural Statistician," in Proc. International Conference on Learning Representation (ICLR), Poster, 2017.
  3. C. Finn, P. Abbeel, and S. Levine, "Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks," in Proc. International Conference on Learning Representation (ICML), 2017.
  4. G. Koch, R. Zemel, and R. Salakhutdinov, "Siamese Neural Networks for One-Shot Image Recognition," in Proc. ICML, 2015.
  5. O. Vinyals, C. Blundell, T. Lillicrap, K. Kavukcuoglu, and D. Wierstra, "Matching Networks for One Shot Learning," in Proc. NIPS, 2016.
  6. J. Snell, K. Swersky, and R. S. Zemel, "Prototypical Networks for Few-Shot Learning," in Proc. NIPS, 2017.
  7. S. Ravi and H. Larochelle, "Optimization as a Model for Few-Shot Learning," in Proc. ICLR, 2017.
  8. H. Qi, M. Brown, and D. G. Lowe, "Low-Shot Learning with Imprinted Weights," in In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018.
  9. Z. Li, F. Zhou, F. Chen, and H. Li, "Meta-SGD: Learning to Learn Quickly for Few-Shot Learning," in Proc. CoRR, 2017.
  10. A. Santoro, S. Bartunov, M. Botvinick, D. Wierstra, and T. Lillicrap, "Meta-Learning with Memory Augmented Neural Network," in ICML, 2016.
  11. F. Sung, Y. Yang, L. Zhang, T. Xiang, P. H.S. Torr, and T. M. Hospedales, "Learning to Compare: Relation Network for Few-Shot Learning," in Proc. CVPR, 2018.
  12. L. Kaiser, O. Nachum, A. Roy, and S. Bengio, "Learning to Remember Rare Events," in Proc. ICLR, 2017.
  13. A. Graves, G. Wayne, I. Daniehelka, "Neural Turing Machines," arXiv preprint arXiv:1410.5401, 2014.
  14. S. Ioffe and C. Szegedy, "Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift," arXiv preprint arXiv:1502.03167, 2015.
  15. B. Lake, R. M. Salakhutdinov, J. Gross and J. B. Tenenbaum, "One Shot Learning of Simple Visual Concept," in Proc. 33rd Annual Conference of the Cognitive Science Society, Vol. 172, 2011.
  16. D. P. Kingma and J. L. Ba, "Adam: A Method for Stochastic Optimization," in Proc. ICLR, 2015.
  17. K. Li and D.-K. Kang, "FAST-ADAM in Semi-Supervised Generative Adversarial Networks," International Journal of Internet, Broadcasting and Communication (IJIBC), Vol. 11, No. 4, pp. 31-36, Nov. 2019. DOI: http://dx.doi.org/10.7236/IJIBC.2019.11.4.31.
  18. Z.-Y. Wang and D.-K. Kang, "Experimental Analysis of Equilibrization in Binary Classification for Non-Image Imbalanced Data Using Wasserstein GAN," International Journal of Internet, Broadcasting and Communication (IJIBC), Vol. 11, No. 4, pp. 37-42, Nov. 2019. DOI: http://dx.doi.org/10.7236/IJIBC.2019.11.4.37.