Browse > Article
http://dx.doi.org/10.5909/JBE.2015.20.4.557

Enhancement of Saliency Map Using Motion and Affinity Model  

Gil, Jong In (Dept. of Computer and Communications Engineering, Kangwon National University)
Choi, Changyeol (Dept. of Computer and Communications Engineering, Kangwon National University)
Kim, Manbae (Dept. of Computer and Communications Engineering, Kangwon National University)
Publication Information
Journal of Broadcast Engineering / v.20, no.4, 2015 , pp. 557-567 More about this Journal
Abstract
Over the past decades, a variety of spatial saliency methods have been introduced. Recently, motion saliency has gained much interests, where motion data estimated from an image sequence are utilized. In general, motion saliency requires reliable motion data as well as image segmentation for producing satisfactory saliency map which poses difficulty in most natural images. To overcome this, we propose a motion-based saliency generation that enhances the spatial saliency based on the combination of spatial and motion saliencies as well as motion complexity without the consideration of complex motion classification and image segmentation. Further, an affinity model is integrated for the purpose of connecting close-by pixels with different colors and obtaining a similar saliency. In experiment, we performed the proposed method on eleven test sets. From the objective performance evaluation, we validated that the proposed method produces better result than spatial saliency based on objective evaluation as well as ROC test.
Keywords
Saliency Map; Affinity Model; Motion; Saliency Enhancement;
Citations & Related Records
연도 인용수 순위
  • Reference
1 Y. Boykov and G. Funka-Lea, "Graph Cuts and Efficient N-D Image Segmentation," Int J. Computer Vision, vol. 70, no. 2, pp. 109-131, 2006.   DOI
2 D. Tsai, M. Flagg, and J. M. Rehg, "Motion coherent tracking with multi-label MRF optimization," in Proc. Brit. Mach. Vis. Conf., 2010.
3 K. Fukuchi, K. Miyazato, A. Kimura, S. Takagi, and J. Yamato, "Saliency-based video segmentation with graph cuts and sequentially updated priors," in Proc. IEEE Int. Conf. Multimedia Expo, pp. 638-641, June-July, 2009.
4 Y. Zhai and M. Shah, "Visual attention detection in video sequences using spatiotemporal cues," Proceedings of the 14th annual ACM Int' Conf. on Multimedia, pp. 815-824, 2006.
5 M. Cheng, G. Zhang, N. Mitra, X. Huang and S. Hu, "Global Contrast based Salient Region Detection," IEEE Conf. Computer Vision and Pattern Recognition, pp. 409-416, 2011.
6 R. Achanta and S. Susstrunk, "Saliency detection for content-aware image resizing," IEEE Int' Conf. Image Processing, pp.1005-1008, 2009.
7 V. Gopalakrishnan, Y. Hu and D, Rajan, "Salient Region Detection by Modeling Distributions of Color and Orientation", IEEE Trans. on Multimedia, Vol. 11, No. 5, Aug. 2009.   DOI   ScienceOn
8 J. Li, Y. Tian, T. Huang and W. Gao "A dataset and evaluation methodology for visual saliency in video," IEEE Int' Conf. on Multimedia and Expo, pp. 442-445, June, 2009.
9 L. Itti, C. Koch, and E. Niebur, "A model of saliency-based visual attention for rapid scene analysis," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 20, no. 11, pp. 1254-1259, Nov. 1998.   DOI   ScienceOn
10 Y. Xia, R. Hu, Z. Huang and Y. Su, "A novel method for generation of motion saliency," Proc. of IEEE 17th Int' Conf. on Image Processing, Sep. 2010.
11 D. Baltieri, R. Vezzani and R. Cucchiara, "3DPes: 3D People Dataset for Surveillance and Forensics," in Proceedings of the 1st International ACM Workshop on Multimedia access to 3D Human Objects, Scottsdale, Arizona, USA, pp. 59-64, Nov-Dec, 2011. (http://imagelab.ing. unimore.it/visor/3dpes.asp)
12 H. Seo and P. Milanfar, "Static and space-time visual saliency detection by self-resemblance," Journal of Vision, 9(12): 15, 2009.   DOI   ScienceOn
13 C. Huang, Y. Chang, Z. Yang and Y. Lin, "Video saliency map detection by dominant camera motion removal," IEEE Tran. on Circuits and Systems for Video Technology, 2012.
14 Y. Zhu, N. Jacobson, H. Pan, and T. Nguyen, "Motion-decision based spatiotemporal saliency for video sequences," IEEE Int' Conf. on Acoustics, Speech and Signal Processing, pp. 1333-1336, 2011.
15 X. Yang, R. Hu, Z. Huang and Y. Su, "A novel method for generation of motion saliency," IEEE Int' Conf. on Image Processing, pp. 4685-4688, Sep. 2010.
16 W. Li, H. Chang, K. Lien, H. Chang, and Y. F. Wang. "Exploring Visual and Motion Saliency for Automatic Video Object Extraction," IEEE Trans. on Image Processing, Vol. 22, No. 7, July 2013.
17 A. Mumtaz, W. Zhang and A. Chan, "Joint motion segmentation and background estimation in dynamic scenes," IEEE Conf. Computer Vision and Pattern Recognition, 2014.
18 J. Shi and J. Malik, "Normalized Cuts and Image Segmentation," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 8, pp. 888-905, Aug. 2000.   DOI   ScienceOn