Browse > Article

Spatiotemporal Removal of Text in Image Sequences  

Lee, Chang-Woo (Dept. of Computer Eng. At Kyungpook National Univ)
Kang, Hyun (Dept. of Computer Eng. At Kyungpook National Univ)
Jung, Kee-Chul (College of Information Science at Soongsil Univ)
Kim, Hang-Joon (Dept. of Computer Eng. At Kyungpook National Univ)
Publication Information
Journal of the Institute of Electronics Engineers of Korea CI / v.41, no.2, 2004 , pp. 113-130 More about this Journal
Abstract
Most multimedia data contain text to emphasize the meaning of the data, to present additional explanations about the situation, or to translate different languages. But, the left makes it difficult to reuse the images, and distorts not only the original images but also their meanings. Accordingly, this paper proposes a support vector machines (SVMs) and spatiotemporal restoration-based approach for automatic text detection and removal in video sequences. Given two consecutive frames, first, text regions in the current frame are detected by an SVM-based texture classifier Second, two stages are performed for the restoration of the regions occluded by the detected text regions: temporal restoration in consecutive frames and spatial restoration in the current frame. Utilizing text motion and background difference, an input video sequence is classified and a different temporal restoration scheme is applied to the sequence. Such a combination of temporal restoration and spatial restoration shows great potential for automatic detection and removal of objects of interest in various kinds of video sequences, and is applicable to many applications such as translation of captions and replacement of indirect advertisements in videos.
Keywords
Text detection; Text removal; Motion estimation; Spatiotemporal restoration; SVM;
Citations & Related Records
연도 인용수 순위
  • Reference
1 P. Perona, J. Malik, 'Scale-space And Edge Detection using Anisotropic Diffusion,' IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 12, no. 7, pp. 629-639, 1990   DOI   ScienceOn
2 C. W. Lee, Spatiotemporal Approach to Removal of Superimposed Text in Image Sequences, PhD thesis, Kyungpook National University, 2003. 12
3 Haykin S. Neural Network- A Comprehensive Foundation, 2nd edition. Prentice Hall, NJ. 1999
4 M. M. Oliveira, B. Bowen, R. McKenna, and Y. S. Chang, 'Fast Digital Image Inpainting,' in Proc, of the International Conferenee on Visualization, Imaging and Image Processing, pp. 261-266, 200
5 N. Habili, A. Moini, N. Burgess, 'Automatic Thresholding for Change Detection in Digital Video,' Proc. Visual Com. And Image Processing, 4067, pp. 133-142, 2000   DOI
6 R. Nakagaki, and A. K Katsaggelos, 'A VQ-Based Blind Image Restoration Algorithm,' IEEE Transactions on Image Processing, vol. 12, no. 9, pp. 1044-1053, 2003   DOI   ScienceOn
7 B. T. Chun, Y. Bae, 'A Method for Recover ing Original Image for Video Caption Area and Replacing Caption Text,' in Proceeding on International Workshop of Content-based Multimedia Indexing'2001 (CBMI'01), Brescia, Italy, September, 2001
8 M. Bertalrnio, G. Sapiro, V. Caselles, C. Ballester, 'Image Inpainting,' Siggraph 2000 Conference Proceedings, pp. 417 - 424, 2000   DOI
9 L. Y. Wei, M. Levoy, 'Fast Texture Syn thesis using Tree-structured Vector Quantization,' Siggraph 2000 Conference Proceedings, pp. 479-488, 2000   DOI
10 A. C. Kokaram, R. D. Morris, W. J. Fitzgerald, P. J. W. Rayner, 'Interpolation of Miss ing Data in Image Sequences,' IEEE Transaction on Image Processing, vol. 4, no. 11, pp. 1509-1519, 1995   DOI   ScienceOn
11 M. D. Richard, R. P. Lippmann, 'Neural Net work Classifiers Estimates Bayesian a Posteriori Probabilities,' Neural Computation, vol. 3, pp. 461-483, 1991   DOI
12 T. Chan, J. Shen, 'Inpainting, zooming, and edge coding,' Special Session on Inverse Problems and Image Analysis at the AMS Annual Conference, January 2001
13 A. K Jain, B. Yu, 'Automatic Text Location in Images and Video Frames,' Pattern Re cognition, vol. 31, no. 12, pp. 2055-2076, 1998   DOI   ScienceOn
14 M. Irani, S. Peleg, 'Motion Analysis for Image Enhancement: Resolution, Occlusion, and Transparency,' Journal on Visual Communications and Image Representation, vol. 4, no. 4, pp. 324 - 335, 1993   DOI   ScienceOn
15 R. Lienhart, F. Stuber, 'Automatic Text Recognition in Digital Videos,' SPIE - The Inter national Society for Optical Engineering, pp. 180-188, 1996   DOI
16 K Y. Jeong, K Jung, E. Y. Kim, H. J. Kim, 'Neural Network-based Text Location for News Video Indexing,' Proc. of International Conference of Image Processing, vol. 3, pp, 319-323, 1999   DOI
17 E. Y. Kim, K. Jung, K Y. Jeong, H. J. Kim, 'Automatic Text Region Extraction Using Cluster-based Templates,' International Conference on Application and Pattern Recognition and Digital Techniques, pp. 418-421, 2000   DOI
18 Y. Zhong, K. Karu, A. K. Jain, 'Locating Text in Complex Color Images,' Pattern Recognition, vol. 28, no. 10, pp. 1523 - 1535, 1995   DOI   ScienceOn
19 A. K. Jain, K. Karu, 'Learning Texture Discrimination Masks,' IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 18, no. 2, pp, 195-205, 1996   DOI   ScienceOn
20 H. Li, D. Doerman, O. Kia, 'Automatic Text Detection and Tracking in Digital Video,' IEEE Transactions on Image Processing, vol. 9, no. 1, pp. 147-156, 2000   DOI   ScienceOn
21 Scholkopf B., Sung K. K., Burges C. J. C, Girosi F., Niyogi P., Poggio T., and Vapnik V., 'Comparing Support Vector Machines with Gaussian Kernels to Radial Basis Function Classifiers,' IEEE Transactions on Signal Processing 45, pp. 2753-2765, 1997   DOI   ScienceOn
22 Y. Zhong, H. Zhang, A. K Jain, 'Automatic Caption Localization in Compressed Video' IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 22, no. 4, pp. 385-392, 2000   DOI   ScienceOn
23 K. Jung, 'Neural network-based text location in color images,' Pattern Recognition Letters, vol. 22, pp. 1503 - 1515, 2001   DOI   ScienceOn
24 T. Hastie, R. Tibshirani, J. Friedman, Elements of Statistical Learning: Data Mining, Inference and Prediction, Springer-Verlag, New York. 2001
25 Y. M. Y. Hasan, L. J. Karam, 'Morpholo gical Text Extraction from Images,' IEEE Transactions on Image Processing, vol. 9, no. 11, pp. 1978-1983, 2000   DOI   ScienceOn
26 V. Vapnik, Statistical Learning Theory, John Wiley & Sons, New York, 1998
27 B. Schokopf, A. J. Smola, Learning with Kernels, The MIT Press, 2002
28 K. I. Kim, K. Jung, S. H. Park, H. J. Kim, 'Support Vector Machines for Texture Classification,' IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 24, no. 11, pp. 1542 - 1550, 2002   DOI   ScienceOn
29 S. Scholkopf, C. J. C. Burges, V. Vapnik, 'Extracting Support Data for a Given Task,' Proc. Int. Conf. on Knowledge Discovery & Data Mining, PP. 252-257, 1995
30 A. Bovik, Hand Book of Image and Video Processing, Academic Press, 2000
31 T. M. Cover, 'Geometrical and Statistical Properties of Systems of Linear Inequalities with Applications in Pattern Recognition,' IEEE Transactions on Electronic Computers, vol. 14, pp. 326-334, 1965   DOI   ScienceOn