Journal of IKEEE (전기전자학회논문지)

Institute of Korean Electrical and Electronics Engineers (한국전기전자학회)
권호 표지
DOI QR코드

DOI QR Code

Recent advances in few-shot learning for image domain: a survey

이미지 분석을 위한 퓨샷 학습의 최신 연구동향

  • Ho-Sik Seok (Dept. of Artificial Intelligence and Data Science, Korea Military Academy)
  • Received : 2023.09.25
  • Accepted : 2023.11.14
  • Published : 2023.12.31
Download PDF
⟨ Previous Next ⟩

Abstract

In many domains, lack of data inhibits adoption of advanced machine learning models. Recently, Few-Shot Learning (FSL) has been actively studied to tackle this problem. Utilizing prior knowledge obtained through observations on related domains, FSL achieved significant performance with only a few samples. In this paper, we present a survey on FSL in terms of data augmentation, embedding and metric learning, and meta-learning. In addition to interesting researches, we also introduce major benchmark datasets. FSL is widely adopted in various domains, but we focus on image analysis in this paper.

퓨삿학습(few-shot learning)은 사전에 확보한 관련 지식과 소규모의 학습데이터를 이용하여 학습데이터의 부족으로 인한 어려움을 해결할 수 있는 가능성을 제시해주어 최근 많은 주목을 받고 있다. 본 논문에서는 퓨삿학습의 개념과 주요 접근방법을 빠르게 파악할 수 있도록 데이터 증강, 임베딩과 측도학습, 메타학습의 세 관점에서 최신연구동향을 설명한다. 또한 퓨샷학습을 적용하려는 연구자들에게 도움을 제공할 수 있도록 주요 벤치마크 데이터셋에 대하여 간략하게 소개하였다. 퓨삿학습은 이미지 분석과 자연어 처리 등 다양한 분야에서 활용되고 있으나, 본 논문은 이미지 처리를 위한 퓨삿학습의 접근법에 집중하였다.

Keywords

Acknowledgement

This study was supported by research fund of Korea Military Academy, (Future Strategy and Technology Research Institute). (RN: 23-AI-03).

References

  1. Y. Wang et al., "Generalizing from a Few Examples: A Survey on Few-Shot Learning," ACM Computing Survey, Vol.53, No.3, pp.1-34, 2020. DOI: 10.1145/3386252 
  2. S. Laenen and L. Bertinetto, "On Episodes, Prototypical Networks, and Few-Shot Learning," In Proc. of 35th Conferenceon Neural Information Processing Systems (NeurIPS2021), 2021. DOI: 10.48550/arXiv.2012.09831 
  3. A. Rajeswaran et al., "Meta-Learning with Implicit Gradients," In Proc. of 33rd Conference on Neural Information Processing Systems (NeurIPS2019), 2019. DOI: 10.48550/arXiv.1909.04630 
  4. C. Shorten and T. M. Khoshgoftaar, "A Survey on Image Data Augmentation for Deep Learning," J Big Data, vol.6, Article Number:60, 2019. DOI:10.1186/s40537-019-0197-0 
  5. R. Zhang et al., "Multi-Task Few-Shot Learning with Composed Data Augmentation for Image Classification,"IET Computer Vision, vol.17, no.2, pp.211-221, 2023. DOI: 10.1049/cvi2.12150 
  6. V. Verma et al., "Manifold Mixup: Better Representations by Interpolating Hidden States," In Proc. of the Thirty-sixth International Conference on Machine Learning (ICML2019), 2019 DOI: 10.48550/arXiv.1806.05236. 
  7. H. Zhang et al., "mixup: Beyond Empirical Risk Minimization," In Proc. of International Conference Learning Representation (ICLR 2018), 2018. DOI: 10.48550/arXiv.1710.09412 
  8. L. Zhao et al., "Maximum-Entropy Adversarial Data Augmentation for Improved Generalization and Robustness,"In Proc. of the 34th Conference on Neural Information Processing Systems (Neur-IPS2020), 2020. DOI: 10.48550/arXiv.2010.08001 
  9. C. Gong et al., "MaxUp: Lightweight Adversarial Training with Data Augmentation Improves Neural Network Training,", In Proc. 2021 IEEE Conference on Computer Vision and Pattern Recognition (CVPR2021), pp.2474-2483, 2021. DOI: 10.1109/CVPR46437.2021.00250 
  10. P. T. Jackson et al., "Style Augmentation: Data Augmentation via Style Randomization," In CVPR Workshop 2019, pp.83-92, 2019. DOI: 10.48550/arXiv.1809.05375 
  11. Y. Jiang, B. Zhu, and B. Xie, "Remote Sensing Images Data Augmentation Based on Style Transfer under the Condition of Few Samples," J. Phys.: Conf. Ser., 1653 012039, 2020. DOI: 10.1088/1742-6596/1653/1/012039 
  12. J. Hemmerich, E. Asilar, and G. F. Ecker, "COVER: Conformational Oversampling as Data augmentation for Molecules," J. Cheminformatics, vol.12, article number 18, 2020. DOI: 10.1186/s13321-020-00420-z 
  13. L. Wu et al., "Data Augmentation based on Multiple Oversampling Fusion for Medical Image Segmentation," PLos One, vol.17, no.10, e0274522, 2022. DOI: 10.1371/journal.pone.0274522 
  14. A. Moreo, A. Esuli, and F. Sebastiani, "Distributional Random Oversampling for Imbalanced Text Classification'" In Proc. of the 39th International ACM SIGIR conference on Research and Development in Information (SIGIR2016), pp. 805-808, 2016. DOI: 10.1145/2911451.2914722 
  15. A. Anand et al., "Phishing URL Detection with Oversampling based on Text Generative Adversarial Networks," In Proc. IEEE International Conference on Big Data, 2018. DOI: 10.1109/BigData.2018.8622547 
  16. Ni. V. Chawla et al., "SMOTE: Synthetic Minority Over-sampling Technique," J. Artif. Intell. Res., vol.16, no.1, pp.321-357, 2002. DOI: 10.1613/jair.953 
  17. M. Ochal et al.. Wang, "Few-Shot Learning with Class Imbalance," IEEE Trans. Artif., Early access, 2023. DOI: 10.1109/TAI.2023.3298303 
  18. D. Wertheimer and B. Hariharan, "Few-Shot Learning with Localization in Realistic Settings," In Proc. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR2019), pp 6558-6567, 2019. DOI: 10.48550/arXiv.1904.08502 
  19. B. Kulis, "Metric Learning: A Survey," Now Foundations and Trends, 2013. DOI: 10.1561/2200000019 
  20. A. Bellet, A. Habrard, and M. Sebban, "A Survey on Metric Learning for Feature Vectors and Structured Data," Technical Report, arXiv: 1306.6709, 2013. DOI: 10.48550/arXiv.1306.6709 
  21. D. Kedem et al., "Non-Linear Metric Learning," In Proc. of Advances in Neural Information Processing Systems 25 (NIPS2012), 2012. 
  22. M. Kaya and H. S. Bilge, "Deep Metric Learning: A Survey," Symmetry, vol.11, no.9, 1066, 2019. DOI: 10.3390/sym11091066 
  23. O. Vinyals et al., "Matching Networks for One Shot Learning," In Proc. of the 30th Conference on Neural Information Processing Systems (NIPS2016), 2016. DOI: 10.48550/arXiv.1606.04080 
  24. F. Sung et al., "Learning to Compare: Relation Network for Few-Shot Learning," In Proc. of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR2018), pp.1199-1208, 2018. DOI: 10.1109/CVPR.2018.00131 
  25. H.-J. Ye et al., "Few-Shot Learning via Embedding Adaptation with Set-to-Set Functions," In Proc. of IEEE/CVF International Conference on Computer Vision (ICCV-20), pp.8808-8817, 2020. DOI: 10.1109/CVPR42600.2020.00883 
  26. C. Liu et al., "Learning a Few-shot Embedding Model with Contrastive Learning," In Proc. of The Thirty-Fifth AAAI Conference on Artificial Intelligence (AAAI-21), pp.8635-8643, 2021. DOI: 10.1609/aaai.v35i10.17047 
  27. Y. Gao et al., "Contrastive Prototype Learning with Augmented Embeddings for Few-Shot Learning," In Proc. of the Thirty-Seventh Conference on Uncertainty in Artificial Intelligence (UAI 2021), pp.140-150, 2021. DOI: 10.48550/arXiv.2101.09499 
  28. P. Rodriguez et al., "Embedding Propagation: Smoother Manifold for Few-Shot Classication," In Proc. of European Conference on Computer Vision (ECCV2020), pp.121-138, 2020. DOI: 10.48550/arXiv.2003.04151 
  29. S. Xiang, F. Nie, and C. Zhang, "Learning a Mahalanobis Distance Metric for Data Clustering and Classification," Pattern Recogn., vol.41, no.12, pp.3600-3612, 2008. DOI: 10.1016/j.patcog.2008.05.018 
  30. H.-J. Ye et al., "Learning Mahalanobis Distance Metric: Considering Instance Disturbance Helps," In Proc. of the 2017 International Joint Conference on Artificial Intelligence (IJCAI-17), 2017, pp. 3315-3321. DOI: 10.24963/ijcai.2017/463 
  31. J. Goldberger et al., "Neighbourhood Components Analysis," In Proc. of Advances in Neural Information Processing Systems 17 (NIPS 2004), 2004. 
  32. J. Snell, K. Swersky, and R. Zemel. "Prototypical Networks for Few-Shot Learning," In Proc. of the 31st Conference on Neural Information Processing Systems (NIPS2017), 2017.
  33. P. Bateni et al., "Improved Few-Shot Visual Classification," In Proc. of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR2020), pp.14481-14490, 2020. DOI: 10.1109/CVPR42600.2020.01450 
  34. W. Li et al., "Distribution Consistency based Covariance Metric Networks for Few-Shot Learning," In Proc. of the Thirty-Third AAAI Conference on Artificial Intelligence (AAAI-19), pp.8642-8649, 2019. DOI: 10.1609/aaai.v33i01.33018642 
  35. L. Qiao et al., "Transductive Episodic-wise Adaptive Metric for Few-Shot Learning," In Proc. of 2019 IEEE/CVF International Conference on Computer Vision (ICCV-19), pp.3603-3612, 2019. DOI: 10.1109/ICCV.2019.00370 
  36. T. Hospedales et al., "Meta-Learning in Neural Networks: A Survey," IEEE Trans. Pattern Anal. Mach., vol.44, pp.5149-5169, 2022. DOI: 10.1109/TPAMI.2021.3079209 
  37. C. Finn, P. Abbeel, and S. Levine, "Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks," In Proc. of the 34th International Conference on Machine Learning (ICML2017), pp.1126-1135, 2017. DOI: 10.48550/arXiv.1703.03400 
  38. A. A. Rusu et al., "Meta-Learning with Latent Embedding Optimization,"In Proc. of International Conference Learning Representation (ICLR 2019), 2019. DOI: 10.48550/arXiv.1807.05960 
  39. H. Yao et al., "Hierarchically Structured Meta-Learning," In Proc. of the 36th International Conference on Machine Learning (ICML2019), pp.7045-7054, 2019. DOI: 10.48550/arXiv.1905.05301 
  40. C. Fifty et al., "Efficiently Identifying Task Groupings for Multi-Task Learning," In Proc. of the Thirty-fifth Conference on Neural Information Processing Systems (NeurIPS2021), 2021.
  41. A. Zhou, T. Knowles, and C. Finn, "Meta-Learning Symmetries by Reparameterization," In Proc. of International Conference Learning Representation (ICLR 2021), 2021. DOI: 10.48550/arXiv.2007.02933 
  42. S.-O. Kaba et al., "Equivariance with Learned Canonicalization Functions,"In Proc. of the 40th International Conference on Machine Learning (ICML2023), 2023. DOI: 10.48550/arXiv.2211.06489 
  43. S. Basu et al., "Equivariant Few-Shot Learning from Pretrained Models," arXiv:2305.09900, 2023. DOI: 10.48550/arXiv.2305.09900 
  44. E. Triantafillou et al., "Learning a Universal Template for Few-shot Dataset Generalization," In Proc. of the 38th International Conference on Machine Learning (ICML2021), pp.10424-10433, 2021. DOI: 10.48550/arXiv.2105.07029 
  45. V. Dumoulin et al., "Feature-wise Transformations," Distill, 2018. DOI: 10.23915/distill.00011 
  46. M. Andrychowicz et al., "Learning to Learn by Gradient Descent by Gradien Descent," In Proc. of the 30th Conference on Neural Information Processing Systems (NIPS2016), 2016. DOI: 10.48550/arXiv.1606.04474 
  47. K. Li and J. Malik, "Learning to Optimize," In Proc. of International Conference Learning Representation (ICLR 2017), 2017. DOI: 10.48550/arXiv.1703.00441 
  48. S. Ravi and H. Larochelle, "Optimization as a Model for Few-Shot Learning," In Proc. of International Conference Learning Representation (ICLR 2017), 2017.
  49. T. Chen et al., "Learning to Optimize: a Primer and a Benchmark," J Mach Learn Res, Vol.23, no.1, pp.8562-8620, 2023. 
  50. Z. Yue et al., "Interventional Few-Shot Learning," In Proc. of the Thirty-fourth Conference on Neural Information Processing Systems (Neur IPS2020), 2020. DOI: 10.48550/arXiv.2009.13000 
  51. C. Finn et al., "Online Meta-Learning," In Proc. of the 36th International Conference on Machine Learning (ICML2019), pp.1920-1930, 2019. DOI: 10.48550/arXiv.1902.08438 
  52. C. Wah et al., "The Caltech-UCSD Birds-200-2011 Dataset," Technical Report CNS-TR-2011-001, California Institute of Technology, 2011. 
  53. B. M. Lake, R. Salakhutdinov, and J. B. Tenenbaum, "Human-level Concept Learning through Probabilistic Program Induction," Science, vol.350, no.6266, pp.1332-1338, 2015.  https://doi.org/10.1126/science.aab3050
  54. A. Shaban et al., "One-Shot Learning for Semantic Segmentation,"In Proc. of British Machine Vision Conference (BMVC2017), 2017. DOI: 10.48550/arXiv.1709.03410 
  55. E. Triantafillou et al., "Meta-Dataset: A Dataset of Datasets for Learning to Learn from Few Examples,"In Proc. of International Conference Learning Representation (ICLR 2020), 2020. DOI: 10.48550/arXiv.1903.03096