Browse > Article
http://dx.doi.org/10.5909/JBE.2022.27.4.561

Efficient Memory Update Module for Video Object Segmentation  

Jo, Junho (Department of ECE, INMC, Seoul National University)
Cho, Nam Ik (Department of ECE, INMC, Seoul National University)
Publication Information
Journal of Broadcast Engineering / v.27, no.4, 2022 , pp. 561-568 More about this Journal
Abstract
Most deep learning-based video object segmentation methods perform the segmentation with past prediction information stored in external memory. In general, the more past information is stored in the memory, the better results can be obtained by accumulating evidence for various changes in the objects of interest. However, all information cannot be stored in the memory due to hardware limitations, resulting in performance degradation. In this paper, we propose a method of storing new information in the external memory without additional memory allocation. Specifically, after calculating the attention score between the existing memory and the information to be newly stored, new information is added to the corresponding memory according to each score. In this way, the method works robustly because the attention mechanism reflects the object changes well without using additional memory. In addition, the update rate is adaptively determined according to the accumulated number of matches in the memory so that the frequently updated samples store more information to maintain reliable information.
Keywords
Video Object Segmentation; Memory; Update; Adaptive;
Citations & Related Records
연도 인용수 순위
  • Reference
1 Oh, S. W., Lee, J. Y., Xu, N., & Kim, S. J., "Video object segmentation using space-time memory networks." Proceedings of the IEEE/CVF International Conference on Computer Vision. 2019. doi: https://doi.org/10.1109/ICCV.2019.00932   DOI
2 Li, Yu, Zhuoran Shen, and Ying Shan., "Fast video object segmentation using the global context module." European Conference on Computer Vision. Springer, Cham, 2020. doi: https://doi.org/10.1007/978-3-030-58607-2_43   DOI
3 Liang, Y., Li, X., Jafari, N., & Chen, J., "Video object segmentation with adaptive feature bank and uncertain-region refinement." Advances in Neural Information Processing Systems 33: 3430-3441., 2020.
4 Yao, R., Lin, G., Xia, S., Zhao, J., & Zhou, Y., "Video object segmentation and tracking: A survey." ACM Transactions on Intelligent Systems and Technology (TIST) 11.4 ,1-47p, 2020. doi: http://dx.doi.org/10.1145/3391743   DOI
5 T. -Y. Lin, P. Dollar, R. Girshick, K. He, B. Hariharan and S. Belongie, "Feature Pyramid Networks for Object Detection.", IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 936-944, 2017. doi: https://doi.org/10.1109/CVPR.2017.106   DOI
6 He, K., Zhang, X., Ren, S., & Sun, J., "Deep residual learning for image recognition.", Proceedings of the IEEE conference on computer vision and pattern recognition, p. 770-778, 2016. doi: https://doi.org/10.1109/CVPR.2016.90   DOI
7 Pont-Tuset, J., Perazzi, F., Caelles, S., Arbelaez, P., Sorkine-Hornung, A., & Van Gool, L, "The 2017 davis challenge on video object segmentation." arXiv preprint arXiv:1704.00675, 2017.
8 Ning Xu, Linjie Yang, Dingcheng Yue, Jianchao Yang, Brian Price, Jimei Yang, Scott Cohen, Yuchen Fan, Yuchen Liang, and Thomas Huang., "Youtube-vos: Sequence-to-sequence video object segmentation." In European Conference on Computer Vision (ECCV), 2018. doi: https://doi.org/10.1007/978-3-030-01228-1_36   DOI
9 Wang, H., Jiang, X., Ren, H., Hu, Y., & Bai, S., "Swiftnet: Real-time video object segmentation." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021. doi: https://doi.org/10.1109/CVPR46437.2021.00135   DOI
10 Hu, Yuan-Ting, Jia-Bin Huang, and Alexander G. Schwing. "Video-match: Matching based video object segmentation." Proceedings of the European conference on computer vision (ECCV). 2018. doi: https://doi.org/10.1007/978-3-030-01237-3_4   DOI