Journal of the Institute of Electronics and Information Engineers (전자공학회논문지)

The Institute of Electronics and Information Engineers (대한전자공학회)
권호 표지
DOI QR코드

DOI QR Code

Automatic Detection of Objects-of-Interest using Visual Attention and Image Segmentation

시각 주의와 영상 분할을 이용한 관심 객체 자동 검출 기법

  • Shi, Do Kyung (Dept. of Computer Science and Engineering, Hanyang University) ;
  • Moon, Young Shik (Dept. of Computer Science and Engineering, Hanyang University)
  • 신도경 (한양대학교 컴퓨터공학과) ;
  • 문영식 (한양대학교 컴퓨터공학과)
  • Received : 2013.12.10
  • Accepted : 2014.04.29
  • Published : 2014.05.25
Download PDF
⟨ Previous Next ⟩

Abstract

This paper proposes a method of detecting object of interest(OOI) in general natural images. OOI is subjectively estimated by human in images. The vision of human, in general, might focus on OOI. As the first step for automatic detection of OOI, candidate regions of OOI are detected by using a saliency map based on the human visual perception. A saliency map locates an approximate OOI, but there is a problem that they are not accurately segmented. In order to address this problem, in the second step, an exact object region is automatically detected by combining graph-based image segmentation and skeletonization. In this paper, we calculate the precision, recall and accuracy to compare the performance of the proposed method to existing methods. In experimental results, the proposed method has achieved better performance than existing methods by reducing the problems such as under detection and over detection.

본 논문에서는 일반적인 자연 영상에서 관심 객체를 자동으로 검출하기 위한 방법을 제안한다. 영상에서의 관심 객체는 사람에 따라서 주관적으로 판단되며, 일반적으로 사람의 시각은 관심 객체에 초점이 맞춰지게 된다. 관심 객체의 자동 검출을 위한 첫 번째 단계로서 사람의 시각 인지기반의 돌출 맵을 이용하여 관심 객체의 후보 영역을 검출한다. 검출된 후보영역은 객체에 대한 대략적인 위치 정보를 가지고 있지만 관심 객체를 정확하게 분할하지 못하는 문제점이 존재한다. 따라서 두 번째 단계에서 영상의 색상과 에지를 고려한 그래프 기반의 영상 분할 기법과 객체 영역의 세선화(skeletonization)를 결합함으로써 정확한 객체 영역을 자동으로 검출한다. 본 논문에서는 제안하는 방법과 기존 방법들의 성능을 비교하기 위해서 정확률(precision), 재현율(recall) 그리고 정밀도(accuracy)를 계산하였다. 그 결과, 제안하는 방법은 미 검출(under detection) 및 과검출(over detection)에 대한 문제점을 줄임으로써 기존 방법보다 더 향상된 결과를 보인다.

Keywords

References

  1. S. C. Zhu and A. Yuille, "Taratorin, Magnetic Information Storage Technology, Region competition: Unifying snakes, region growing, and bayes/mdl for multiband image segmentation," IEEE Transaction on Pattern Recognition and Machine Intelligence, vol. 18, no 9, pp. 884-900, August 1996. https://doi.org/10.1109/34.537343
  2. M. Rousson, T. Brox, and R. Deriche, "Active unsupervised texture segmentation on a diffusion based feature space." In Proc. of IEEE Conference on Computer Vision and Pattern Recognition, vol 2, pp. 699-704, Madison, Wisconsin, June 2003.
  3. C. Rother, V. Kolmogorov, and A. Blake. "grabcut-interactive foreground extraction using iterated graph cuts," ACM Transactions on Graphics (SIG-GRAPH'04), pp. 309-314, 2004.
  4. Y. Boykov and V. Kolmogorov, "An Experimental Comparison of Min-Cut/Max-Flow Algorithms for Energy Minimization in Vision," IEEE Transaction Pattern Analysis and Machine Intelligence, vol. 26, no. 9, Sept. 2004
  5. N. Paragios and R. Deriche. "Geodesic active regions and level set methods for supervised texture segmentation," International Journal of Computer Vision, pp. 223-247, 2002.
  6. A. Blake, C. Rother, M. Brown, P. P'erez, and P. Torr, "Interactive image segmentation using an adaptive gaussian mixture mrf model," In Proc. of the 8th European Conference on Computer Vision, pp. 428-441, 2004.
  7. M. Rousson, T. Brox, and R. Deriche, "Active unsupervised texture segmentation on a diffusion based feature space." In Proc. of IEEE Conference on Computer Vision and Pattern Recognition, vol. 2, pp. 699-704, June 2003.
  8. G. Hua, Z. Liu, Z. Zhang, and Y Wu. Automatic segmentation of objects of interest from an image. Technical Report 2006-10, Microsoft Research, Redmond, WA, January 2006.
  9. Y. Boykov and M. Jolly, "Interactive graph cuts for optimal boundary & region segmentation of objects in ND images," in Proc. of IEEE Conference on Computer Vision and Pattern Recognition, vol. I, pp. 105-112, July 2001.
  10. Y. Boykov and G. Funka-Lea, "Graph cuts and efficient nd image segmentation," International Journal of Computer Vision, vol. 70, no. 2, pp. 109-131, November 2006. https://doi.org/10.1007/s11263-006-7934-5
  11. C. Jung, B. Kim and C Kim, "Automatic segmentation of salient objects using iterative reversible graph cut," In Proc. IEEE International Conference Multimedia Expo, pp. 590-595, July, 2010.
  12. A. Yarbus, "Eye movements and vision," Plenum press, 1967
  13. U. Neisser, "Cognitive psychology. Appleton-Century-Crofts," New York, 1967.
  14. Y. Hu, X. Xie, W. Y. Ma, L. T. Chia and D. Rajan, "Salienct region detection using weighted feature maps based on the human visual attention model," In Proc. Pacific Conference Multimedia, vol. 2, pp. 993-1000, November, 2004.
  15. N. Bruce and J. Tsotsos, "Saliency based on information maximization," In Proc. of Advances in Neural Information Processing Systems, pp. 155-162, December 2005.
  16. J. Harel, C. Koch, and P. Perona, "Graph-based visual saliency," in Advances in Neural Information Processing Systems, 2007.
  17. T. Liu, J. Sun, N.-N. Zheng, X. Tang, and H.-Y. Shum, "Learning to detect a salient object," In Proc. of IEEE Conference on Computer Vision and Pattern Recognition, 2007.
  18. D. Gao, V. Mahadevan, and N. Vasconcelos, "On the plausibility of the discriminant centerurround hypothesis for visual saliency," Journal of Vision, vol. 8, no. 7, 2008.
  19. R. Achanta, S. Hemami, F. Estrada, and S. Ssstrunk, "Frequency-tuned Salient Region Detection," In Proc. of IEEE Conference on Computer Vision and Pattern Recognition, vol. 33, 2009.
  20. C. Guo and L. Zhang, "A novel multiresolution spatiotemporal saliency detection model and its applications in image and video compression," IEEE Transaction on Image Processing, vol. 19, no. 1, pp. 185-198, 2010. https://doi.org/10.1109/TIP.2009.2030969
  21. J. Li, "Visual Saliency Based on Scale-Space Analysis in the Frequency Domain," In: IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 35, pp. 996-1010, 2012
  22. Y. L. Xie, H. C. Lu, and M. H. Yang, "Bayesian saliency via low and mid level cues," IEEE Transaction on Image Processing, vol. 22, no. 5, May 2013.
  23. Y. Fu, J. Cheng, Z. Li and H. Lu, "Saliency cuts: An automatic approach to object segmentation," in Proc. IEEE Internatioanl Conference on Pattern Recognition, pp. 1-4, December, 2008.
  24. C. Jung and C Kim, "A Unified spectral-domain approach for saliency detection and its application to automatic object segmentation," IEEE Transaction on Image Processing, vol. 21, no. 3, pp. 1272-1283, 2012. https://doi.org/10.1109/TIP.2011.2164420
  25. Z. Liu, R Shi, L. Shen, Y. Xue, K. N. Ngan, Z. Zang, "Unsupervised salient object segmentation based on kernel density estimation and two-phase graph cut," IEEE Transaction on Multimedia, vol. 14, no. 4, pp. 1275-1289, 2012. https://doi.org/10.1109/TMM.2012.2190385
  26. S. Han, G. Jung, S. Lee, Y. Hong and S. Lee, "Automatic salient object segmentation using saliency map and color segmentation," Journal of Central South Univaersity, vol. 20, pp. 2407-2413, 2013. https://doi.org/10.1007/s11771-013-1750-1
  27. R. P. Rao, G. Zelinsky, M. Hayhoe, and D. H. Ballard, "Eye movements in iconic visual search," Vision Research, vol. 42, no. 11, pp. 1447-1463, Nov 2002. https://doi.org/10.1016/S0042-6989(02)00040-8
  28. J. M. Wolfe, T. S. Horowitz, N. Kenner, M. Hyle, and N. Vasan, "How fast can you change your mind? The speed of top-down guidance in visual search," Vision Research, vol. 44, no. 12, pp. 1411-1426, Jun 2004. https://doi.org/10.1016/j.visres.2003.11.024
  29. A. Oliva, A. Torralba, M. S. Castelhano,, and J. M. Henderson, "Top-down control of visual attention in object detection," In Proc. of IEEE International Conference on Image Processing, pp. 14-17, September 2003.
  30. A. Santella, M. Agrawala, D. Decarlo, D. Salesin, and M. Cohen, "Gaze-based interaction for semi-automatic photo cropping," In Proc. of the SIGCHI Conference on Human Factors in Computing Systems, pp. 771-780, 2006.
  31. L. Chen, X. Xie, X. Fan, W. Ma, H. Shang, and H. Zhou, "A visual attention mode for adapting images on small displays. Technical report," Microsoft Research, Redmond, WA, 2002.
  32. L. Itti. "Models of Bottom-Up and Top-Down Visual Attention," PhD thesis, California Institute of Technology Pasadena, 2000.
  33. V. Navalpakkam and L. Itti, "An integrated model of top-down and bottom-up attention for optimizing detection speed," In Proc. of IEEE Conference on Computer Vision and Pattern Recognition, vol. 2, pp. 2049-2056, 2006.
  34. C. Jeabong, H. Seok-Wun, "Object Tracking by Extracting Region of Interesting," Proc. of the Korea Multimedia Society Conference, pp. 299-302, 2006.
  35. B. Young Hyun, M. Sung Ryong, "Object-based Image Retrieval for Color Query Image Detection," Journal of the Institute of Electronics Engineers of Korea, Vol. 45, CI, No. 3, pp. 97-102, 2008.
  36. K. Sooyeong, K. Byoungchul, B. Hyeran, "Automatic Salient-object Extraction using the Contrast map and Salient point," Proc. of the KIISE Conference, Vol. 31, No. 1(B), pp. 808-810, 2004.
  37. J. Chanho, K. Beonjoon, K. Changick, "Automatic Segmentation of Salient Object using Iterative Reversible Graph Cuts," Proc. of the Institute of Electronics Engineers of Korea Conference, pp. 339-340, 2009.
  38. P. Felzenszwalb and D. Huttenlocher, "Efficient Graph-Based Image Segmentation," International Journal of Computer Vision, vol. 59, no. 2, pp. 167-181, 2004. https://doi.org/10.1023/B:VISI.0000022288.19776.77
  39. T. Ell, "Quaternion-fourier transforms for analysis of twodimensional linear time-invariant partial differential systems," In Proc. of the 32nd IEEE Conference on Decision and Control, vol. 2, pp. 1830-1841, Dec 1993.

Cited by

  1. Using a Method Based on a Modified K-Means Clustering and Mean Shift Segmentation to Reduce File Sizes and Detect Brain Tumors from Magnetic Resonance (MRI) Images vol.89, pp.3, 2016, https://doi.org/10.1007/s11277-016-3420-8