Browse > Article
http://dx.doi.org/10.3837/tiis.2018.05.021

Salient Region Extraction based on Global Contrast Enhancement and Saliency Cut for Image Information Recognition of the Visually Impaired  

Yoon, Hongchan (Software Development Division, PCT Inc.)
Kim, Baek-Hyun (KRRI(Korea Railroad Research Institute))
Mukhriddin, Mukhiddinov (Department of IT Convergence Engineering, Gachon University)
Cho, Jinsoo (Department of Computer Engineering, College of IT, Gachon University)
Publication Information
KSII Transactions on Internet and Information Systems (TIIS) / v.12, no.5, 2018 , pp. 2287-2312 More about this Journal
Abstract
Extracting key visual information from images containing natural scene is a challenging task and an important step for the visually impaired to recognize information based on tactile graphics. In this study, a novel method is proposed for extracting salient regions based on global contrast enhancement and saliency cuts in order to improve the process of recognizing images for the visually impaired. To accomplish this, an image enhancement technique is applied to natural scene images, and a saliency map is acquired to measure the color contrast of homogeneous regions against other areas of the image. The saliency maps also help automatic salient region extraction, referred to as saliency cuts, and assist in obtaining a binary mask of high quality. Finally, outer boundaries and inner edges are detected in images with natural scene to identify edges that are visually significant. Experimental results indicate that the method we propose in this paper extracts salient objects effectively and achieves remarkable performance compared to conventional methods. Our method offers benefits in extracting salient objects and generating simple but important edges from images containing natural scene and for providing information to the visually impaired.
Keywords
Salient region extraction; Visual attention; The visually impaired; Saliency map; Saliency cut; Image enhancement;
Citations & Related Records
Times Cited By KSCI : 1  (Citation Analysis)
연도 인용수 순위
1 Q. Jiang, F. Shao and G. Jiang, "Efficient saliency detection using color contrast and similarity distribution information," in Proc. of Asia-Pacific Signal and Information Processing Association Annual Summit and Conference, December 9-12, 2014.
2 Q. Zhou, J. Chen, Sh. Ren, Y. Zhou, J. Chen and W. Liu, "On Contrast Combinations for Visual Saliency Detection," in Proc. of IEEE Int. Conf. on Image Processing, pp. 2665-2669, September 15-18, 2013.
3 J. Zhou, J. Zhai and Y. Ren, "Salient Object Detection via Adaptive Region Merging," KSII Transactions on Internet and Information Systems, vol. 10, no. 9, pp. 4386-4404, September, 2016.   DOI
4 X. Qian, J. Han, G. Cheng and L. Guo, "Optimal contrast based saliency detection," Pattern Recognition Letters, vol. 34, no. 11, pp. 1270-1278, August 1, 2013.   DOI
5 X. Hou and L. Zhang, "Saliency detection: A spectral residual approach," in Proc. of IEEE Conf. Computer Vision and Pattern Recognition, pp. 1-8, June 17-22, 2007.
6 C. Guo and L. Zhang, "A novel multiresolution spatiotemporal saliency detection model and its applications in image and video compression," IEEE Transactions on Image Processing, vol. 19, no. 1, pp. 185-198, January, 2010.   DOI
7 C. Rother, V. Kolmogorov, and A. Blake, "GrabCut: Interactive foreground extraction using iterated graph cuts," Association for Computing Machinery Transactions on Graphics, vol. 23, no. 3, pp. 309-314, August, 2004.
8 S. Li, R. Ju, T. Ren and G. Wu, "Saliency cuts based on adaptive triple thresholding," in Proc. of IEEE Int. Conf. on Image Processing, pp. 4609-4613, September 27-30, 2015.
9 P. Mehrani and O. Veksler, "Saliency Segmentation based on Learning and Graph Cut Refinement," in Proc. of British Machine Vision Conf., pp. 1-12. August 31-September 3, 2010.
10 Y. Fu, J. Cheng, Z. Li, and H. Lu, "Saliency Cuts: An automatic approach to object segmentation," in Proc. of 19th Int. Conf. Pattern Recognition, pp. 1-4, December 8-11, 2008.
11 C. Aytekin, E. C. Ozan, S. Kiranyaz and M. Gabbouj, "Visual saliency by extended quantum cuts," in Proc. of Int. Conf. on Image Processing, pp. 1692-1696, September 27-30, 2015.
12 J. Peng, J. Shen, Y. Jia and X. Li, "Saliency Cut in Stereo Images," in Proc. of IEEE Int. Conf. on Computer Vision Workshops, pp. 22-28, December 2-8, 2013.
13 T. Zhao, L. Li, X. Ding, Y. Huang and D. Zeng, "Saliency Detection with Spaces of Background-Based Distribution," IEEE Signal Processing Letters, vol. 23, no. 5, pp. 683-687, May, 2016.   DOI
14 Z. Liu, O. Le Meur and Sh. Luo, "Superpixel-based saliency detection," in Proc. of 14th Int. Workshop on Image and Audio Analysis for Multimedia Interactive Services, pp. 1-4, July 3-5, 2013.
15 M. M. Cheng, G. X. Zhang, N. J. Mitra, X. Huang and Sh. M. Hu, "Global contrast based salient region detection," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 37, no. 3, pp. 569-582, March, 2015.   DOI
16 Y. F. Ma and H. J. Zhang, "Contrast-based Image Attention Analysis by Using Fuzzy Growing," in Proc. of the 11th ACM Int. Conf. on Multimedia, pp. 374-381, November 2-8, 2003.
17 A. Borji and L. Itti, "State-of-the-art in visual attention modeling," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 35, no. 1, pp. 185-207, January, 2012.   DOI
18 S. Wang, X. Yang and Y. Tian, "Detecting signage and doors for blind navigation and wayfinding," Network Modeling Analysis in Health Informatics and Bioinformatics, vol. 2, no. 2, pp. 81-93, July, 2013.   DOI
19 L. Chen, B. Guo and W. Sun, "Obstacle Detection System for Visually Impaired People Based on Stereo Vision," in Proc. of IEEE 4th Int. Conf. on Genetic and Evolutionary Computing, pp. 723-726, December 13-15, 2010.
20 L. Muthulakshmi and A. B. Ganesh, "Bimodal Based Environmental Awareness System for Visually Impaired People," in Proc. of Int. Conf. on modeling, optimization and computing, vol. 38, pp. 1132-1137, 2012.
21 N. Tagaki and J. Chen, "Character string extraction from scene images by eliminating non-character elements," in Proc. of IEEE Int. Conf. on Systems, pp. 3685-3690, October 5-8, 2014.
22 N. Takagi, J. Chen, "A Broken Line Classification Method of Mathematical Graphs for Automating Translation into Scalable Vector Graphic," in Proc. of 43rd IEEE Int. Symposium on Multiple-Valued Logic, pp. 71-76, May 22-24, 2013.
23 J. Chen and N. Takagi, "A Pattern Recognition Method for Automating Tactile Graphics Translation from Hand-Drawn Maps," in Proc. of IEEE Int. Conf. on Systems, pp. 4173-4178, October 13-16, 2013.
24 X. Wei, Z. Tao, Ch. Zhang and X. Cao, "Structured Saliency Fusion Based on Dempster-Shafer Theory," IEEE Signal Processing Letters, vol. 22, no. 9, pp. 1345-1349, September, 2015.   DOI
25 M. Donoser, M. Urschler, M. Hirzer and H. Bischof, "Saliency Driven Total Variation Segmentation," in Proc. of IEEE 12th Int. Conf. on Computer Vision, pp. 814-824, September 29-October 2, 2009.
26 G. Li, Y. Xie, L. Lin and Y. Yu, "Instance-Level salient object segmentation," arXiv preprint, April 12, 2017.
27 Y. Gao, M. Shi, D. Tao and Ch. Xu, "Database Saliency for Fast Image Retrieval," IEEE Transactions on Multimedia, vol. 17, no. 3, pp. 359-369, March, 2015.   DOI
28 M. Wang, R. Hong, X. T. Yuan, Sh. Yan and T. S. Chua, "Movie2Comics: Towards a Lively Video Content Presentation," IEEE Transactions on Multimedia, vol. 14, no. 3, pp. 858-870, June, 2012.   DOI
29 S. S. Pathak, P. Dahiwale and G. Padole, "A combined effect of local and global method for contrast image enhancement," in Proc. of IEEE Int. Conf. on Engineering and Technology, pp. 1-5, March 20-25, 2015.
30 N. Otsu, "A threshold selection method from gray-level histograms," IEEE Transactions on Systems, Man, and Cybernetics, vol. 9, no. 1, pp. 62-66, January, 1979.   DOI
31 M. Rabbani and R. Joshi, "An overview of the JPEG2000 still image compression standard," Signal Processing: Image Communication, vol. 17, no. 1, pp. 3-48, January, 2002.   DOI
32 R. Gallea, E. Ardizzone and R. Pirrone, "Physical Metaphor for Streaming Media Retargeting," IEEE Transactions on Multimedia, vol. 16, no. 4, pp. 971-979, June, 2014.   DOI
33 U. Rutishauser, D. Walther, Ch. Koch and P. Perona, "Is bottom-up attention useful for object recognition?," in Proc. of IEEE Computer Society Conf. on Computer Vision and Pattern Recognition, vol. 2, pp. II-37-II-44, June 27-July 2, 2004.
34 M. M. Cheng, J. Warrell, W. Y. Lin, S. Zheng, V. Vineet and N. Crook, "Efficient salient region detection with soft image abstraction" in Proc. of IEEE Int. Conf. on Computer Vision, pp. 1529-1536, December 1-8, 2013.
35 S. Paris, P. Kornprobst, J. Tumblin and F. Durand, "Bilateral Filtering: Theory and Applications," Foundations and Trends in Computer Graphics and Vision, vol. 4, no. 1, pp. 1-73, August, 2009.   DOI
36 G. Jie and L. Ning, "An improved adaptive threshold canny edge detection algorithm," in Proc. of Int. Conf. on Computer Science and Electronics Engineerin, pp. 164-168, March 23-25, 2012.
37 S. Goferman, L. Z. Manor and A. Tal, "Context-aware saliency detection," IEEE Transactions on Pattern Analysis Machine Intelligence, vol. 34, no. 10, pp. 1915-1926, October, 2012.   DOI
38 X. Li, Y. Li, C. Shen, A. R. Dick and A. V. D. Hengel, "Contextual hypergraph modeling for salient object detection" in Proc. of IEEE Int. Conf. on Computer Vision, pp.3328-3335, December 1-8, 2013.
39 B. Jiang, L. Zhang, H. Lu, C. Yang and M. H. Yang, "Saliency detection via absorbing Markov chain," in Proc. of IEEE Int. Conf. on Computer Vision, pp. 1665-1672, December 1-8, 2013.
40 L. Duan, C. Wu, J. Miao, L. Qing and Y. Fu, "Visual saliency detection by spatially weighted dissimilarity," in Proc. of IEEE Conf. Computer Vision and Pattern Recognition, pp. 473-480, June 20-25, 2011.
41 R. Margolin, L. Z. Manor and A. Tal, "Saliency for image manipulation," The Visual Computer, vol. 29. no. 5, pp. 381-392, May 2013.   DOI
42 R. Hong, M. Wang, G. Li and X. T. Yuan, "iComics. Automatic conversion of movie into comics," in Proc. of 18th ACM Int. Conf. on Multimedia, pp. 1599-1602, October 25-29, 2010.
43 J. Lei, C. Zhang, Y. Fang, Z. Gu, N. Ling and Ch. Hou, "Depth Sensation Enhancement for Multiple Virtual View Rendering," IEEE Transactions on Multimedia, vol. 17, no. 4, pp. 457-469, April, 2015.   DOI
44 G. X. Zhang, M. M. Cheng, Sh. M. Hu and R. Martin, "A Shape-Preserving Approach to Image Resizing," Journal compilation:The Eurographics Association and Blackwell Publishing Ltd, vol. 28, no. 7, pp. 1897-1906, Octobor,2009.
45 M. M. Cheng, F. L. Zhang, N. J. Mitra, X. Huang and Sh.-M. Hu, "RepFinder: Finding Approximately Repeated Scene Elements for Image Editing," ACM Transactions on Graphics, vol. 29, no. 4, pp. 83, July, 2010.   DOI
46 R. Hong, J. He, H. Zhang and T. S. Chua, "Mental visual indexing: Towards Fast Video Browsing," in Proc. of the 2016 ACM on Multimedia Conf., pp. 621-625, October 15-19, 2016.
47 J. Jungil, Y. Hongchan, L. Hyelim and C. Jinsoo, "Graphic haptic electronic board-based education assistive technology system for blind people," in Proc. of IEEE Int. Conf. on Consumer Electronics, pp. 364-365, January 9-12, 2015.
48 L. Nie, R. Hong, L. Zhang, Y. Xia, D. Tao and N. Sebe, "Perceptual attributes optimization for multivideo summarization," IEEE Transactions on Cybernetics, vol. 46, no. 12, pp. 2991-3003, December, 2016.   DOI
49 Ch. Cheng, Sh. Li, Y. Wang, H. Qin and A. Hao, "Video saliency detection via spatial-temporal fusion and low-rank coherency diffusion," IEEE Transactions on Image Processing, February, 2017.
50 Q. Yan, L. Xu, J. Shi and J. Jia, "Hierarchical Saliency Detection," in Proc. of IEEE Conf. on Computer Vision and Pattern Recognition, pp. 1155-1162, June 23-28, 2013.
51 R. S. Srivatsa and R. V. Babu, "Salient object detection via objectness measure," in Proc. of IEEE Int. Conf. on Image Processing, pp. 4481-4485, September 27-30, 2015.
52 J. Shi, Q. Yan, L. Xu and J. Jia, "Hierarchical Image Saliency Detection on Extended CSSD," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 38, no. 4, pp. 717-729, April, 2016.   DOI
53 C. Koch and S. Ullman, "Shifts in selective visual attention: towards the underlying neural circuitry," Human Neurbiol, vol. 4, no. 4, pp. 219-227, 1985.
54 P. Siva, C. Russell, T. Xiang and L. Agapito, "Looking beyond the image: Unsupervised learning for object saliency and detection," in Proc. of IEEE Conf. on Computer Vision and Pattern Recognition, pp. 3238-3245, June 23-28, 2013.
55 R. Margolin, A. Tal and L. Z. Manor, "What makes a patch distinct?," in Proc. of IEEE Conf. on Computer Vision and Pattern Recognition, pp. 1139-1146, June 23-28, 2013.
56 E. Erdem and A. Erdem, "Visual saliency estimation by nonlinearly integrating features using region covariances," Journal Vision, vol. 13, no. 4, pp. 1-20, March, 2013.   DOI
57 X. Hou, J. Harel and C. Koch, "Image signature: Highlighting sparse salient regions," IEEE transactions on pattern analysis and machine intelligence, vol. 34, no. 1, pp. 194-201, January, 2012.   DOI
58 R. Achanta, S. Hemami, F. Estrada and S. Susstrunk, "Frequency-tuned salient region detection," in Proc. of IEEE Conf. on Computer Vision and Pattern Recognition, pp. 1597-1604, June 20-25, 2009.
59 L. Itti, C. Koch and E. Niebur, "A model of saliency-based visual attention for rapid scene analysis," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 20, no. 11, pp. 1254-1259, November, 1998.   DOI
60 J. Harel, C. Koch and P. Perona, "Graph-based visual saliency," in Proc. of 19th Int. Conf. on Neural Information Processing Systems, pp. 545-552, December 4-7, 2006.
61 M. Du, X. Wu, W. Chen and J. Wang, "Fusing Region Contrast and Graph Regularization for Saliency Detection," in Proc. of 2015 27th Chinese Control and Decision Conf., pp. 5789-5794, May 23-25, 2015.
62 T. Liu, J. Sun, N. N. Zheng, X. Tang and H. Y. Shum, "Learning to Detect A Salient Object," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 33, no. 2, pp. 353-367, February, 2011.   DOI
63 P. Felzenszwalb and D. Huttenlocher, "Efficient graph-based image segmentation," Int. Journal of Computer Vision, vol. 59, no. 2, pp. 167- 181, September, 2004.   DOI
64 D. Comaniciu and P. Meer, "Mean shift: A robust approach toward feature space analysis," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 24, no. 5, pp. 603-619, May, 2002.   DOI
65 R. Achanta, A. Shaji, K. Smith, A. Lucchi, P. Fua and S. Susstrunk, "SLIC Superpixels," EPFL, no. Technical Report 149300, June, 2010.
66 J. Lou, M. Ren and H. Wang, "Regional principal color based saliency detection," PLoS ONE, vol. 9, no. 11, pp. 1-13, November, 2014.
67 J. Han, D. Zhang, X. Hu, L. Guo, J. Ren and F. Wu, "Background Prior-Based Salient Object Detection via Deep Reconstruction Residual," IEEE Transactions on Circuits and Systems for Video Technology, vol. 25, no. 8, August, 2015.
68 F. Perazzi, P. Krahenbuhl, Y. Pritch and A. Hornung, "Saliency filters: Contrast based filtering for salient region detection," in Proc. of IEEE Conf. on Computer Vision and Pattern Recognition, pp. 733-740, June 16-21, 2012.
69 M. Hua, X. Bie and W. Wang, "Contrast based saliency detection using representative background priors," in Proc. IEEE Int. Conf. on Image Processing, pp. 3422-3425, September 15-18, 2013.