Journal of the Institute of Electronics Engineers of Korea SP (대한전자공학회논문지SP)

The Institute of Electronics and Information Engineers (대한전자공학회)
권호 표지

Video retrieval method using non-parametric based motion classification

비-파라미터 기반의 움직임 분류를 통한 비디오 검색 기법

  • 김낙우 (한국전자통신연구원 광대역통합망연구단) ;
  • 최종수 (중앙대학교 첨단영상대학원 영상공학과)
  • Published : 2006.03.01
Download PDF
⟨ Previous Next ⟩

Abstract

In this paper, we propose the novel video retrieval algorithm using non-parametric based motion classification in the shot-based video indexing structure. The proposed system firstly gets the key frame and motion information from each shot segmented by scene change detection method, and then extracts visual features and non-parametric based motion information from them. Finally, we construct real-time retrieval system supporting similarity comparison of these spatio-temporal features. After the normalized motion vector fields is created from MPEG compressed stream, the extraction of non-parametric based motion feature is effectively achieved by discretizing each normalized motion vectors into various angle bins, and considering a mean, a variance, and a direction of these bins. We use the edge-based spatial descriptor to extract the visual feature in key frames. Experimental evidence shows that our algorithm outperforms other video retrieval methods for image indexing and retrieval. To index the feature vectors, we use R*-tree structures.

본 논문에서는 샷(shot) 기반 비디오 색인 구조에서 비-파라미터(non-parametric) 기반의 움직임 분류를 통한 비디오 영상 검색 기법을 제안한다. 본 논문에서 제안하는 비디오 검색 시스템은 장면 전환 기법을 통해 얻은 샷 단위의 짧은 비디오로부터 대표 프레임과 움직임 정보를 취득한 후, 이를 통해 시각적 특징과 움직임 특징을 추출하여 유사도를 비교함으로써 시-공간적 특징을 이용한 실시간 검색이 가능하도록 구현되었다. 비-파라미터 기반의 움직임 특징의 추출은 MPEG 압축 스트림으로부터 정규화된 움직임 벡터계(界)를 추출한 후, 각각의 정규화된 움직임 벡터를 여러 개의 각도 빈(bin)으로 양자화하고 이의 평균과 분산, 방향 등을 고려함으로써 효과적으로 이루어진다. 대표 프레임에서의 시각 특징 검출을 위해서는 에지 기반의 공간 기술자를 이용하였다. 실험 결과는 영상 색인 및 검색에 있어서 제안된 시스템이 매우 효과적임을 잘 나타내고 있다. 데이터베이스 내 영상의 색인을 위해서는 R*-tree 구조를 이용한다.

Keywords

References

  1. M. Flickner et al., 'Query by image and video content: The QBIC system,' IEEE computer, vol. 28, no. 9, pp. 23-32, 1995 https://doi.org/10.1109/2.410146
  2. V. Ogle and M. Stonebraker, 'Chabot: Retrieval from a relational database of images,' IEEE computer, vol. 28, no. 9, pp. 40-48, 1995 https://doi.org/10.1109/2.410150
  3. J. R. Smith and S.-F. Chang, 'VisualSEEK: A fully automated content-based image query system,' in ACM Multimedia Conf., 1996 https://doi.org/10.1145/244130.244151
  4. A. Pentland, R. Picard, and S. Sclaroff, 'Photobook: Content-based manipulation of image databases,' IJCV, vol. 18, no. 3, pp. 233-254, 1996 https://doi.org/10.1007/BF00123143
  5. R. Brunelli, O. Mich, C.M. Modena, 'A survey on video indexing,' IRST-Technical Report 9612-06, 1996
  6. 이미숙, 황본우, 이성환, '내용 기반 영상 및 비디오 검색 기술의 연구 현황,' 정보과학회지, 제15권, 제9호, pp, 10-19, 1997
  7. N. Beckmann, H.-P. Kriegel, R. Schneider, and B. Seeger, 'The R*-tree; An efficient and robust access method for points and rectangles', Proc. ACM SIGMOD, pp. 322-331, 1990 https://doi.org/10.1145/93597.98741
  8. H.J. Zhang, C.Y. Low, S.W. Smoliar, and J.H Wu, 'Video parsing retrieval. and browsing: An integrated and content-based solution,' in Proc. ACM Multimedia, pp. 15-24, 1995 https://doi.org/10.1145/217279.215068
  9. M. Rautiainen, M. Hosio, I. Hanski, M. Varanka, J. Kortelainen, T. Ojala, and T. Seppnen, 'TRECVID 2004 experiments at MediaTeam Oulu,' Proc. TRECVID Workshop at Text Retrieval Conference TREC 2004, in press, 2004
  10. B. Adams et al., 'IBM Research TREC-2002 video retrieval system,' Proc. Text Retrieval Conference TREC 2002 Video Track, 2002
  11. J. Huang, S. R. Kumar, M. Mitra, W. J. Zhu, and R. Zabih, 'Image indexing using color correlograms,' CVPR, pp. 762-768, 1997 https://doi.org/10.1109/CVPR.1997.609412
  12. . Mann and R.W. Picard, 'Video orbits of the projective group: A simple approach to featureless estimation of parameters,' IEEE Trans. Image Processing, vol. 6, pp, 1281-1295, 1997 https://doi.org/10.1109/83.623191
  13. T. Yu and Y. Zhang, 'Motion feature extraction for content-based video sequence retrieval,' Internet Image Ⅱ, SPIE-4311, pp. 378-388, 2001 https://doi.org/10.1117/12.411912
  14. T. Yu and Y. Zhang, 'Retrieval of video clips using global motion information,' Electron. Lett., vol. 37, no. 14, pp. 893-895, 2001 https://doi.org/10.1049/el:20010597
  15. W. Chen and S.-F. Chang, 'VISMap: An interactive image/video retrieval system using visualization and concept maps,' in Proc. IEEE int, Conf. Image Processing, vol. 3, pp. 588-591 2001 https://doi.org/10.1109/ICIP.2001.958187
  16. B.L. Yeo, B. Liu, 'Rapid scene analysis on compressed video,' IEEE Trans. on Circuits and Systems for Video Technology, vol. 5, no. 6, pp. 533-544, 1995 https://doi.org/10.1109/76.475896
  17. Y. Nakajima, K Ujihara, A. Yoneyama, 'Universal scene change detection on MPEG-coded data domain,' in Proc. SPIE Visual Comm. and Image Proc., pp. 992-1003, 1997 https://doi.org/10.1117/12.263179
  18. R. Zabih, J. Miller, K. Mai, 'A feature-based algorithm for detecting and classifying scene breaks,' ACM International Conference on Multimedia, pp. 189-200, 1995 https://doi.org/10.1145/217279.215266
  19. J. Meug, Y. Juan, S.F. Chang, 'Scene change detection in a MPEG compressed video sequence,' Digital Video Compression: Algorithms and Technologies, SPIE-2419, pp. 14-25, Feb. 1995 https://doi.org/10.1117/12.206359
  20. E. Izquierdo, J. Xia, and R. Mech, 'A generic video analysis and segmentation system,' in Proc. IEEE Int., Conf. Acoustics, Speech, and Signal Processing, vol. 4, pp. 3592-3595, 2002 https://doi.org/10.1109/ICASSP.2002.1004693
  21. N.W. Kim, E.K Kang, et al., 'Scene change detection and classification algorithm on compressed video streams,' Proc. of the ITC-CSCC 2001, vol. 1, pp. 279-282, 2001
  22. R. Wang R., T Huang, 'Fast camera motion analysis in .MPEG domain,' International Conference on Image Processing, vol. 3, pp. 691-694, 1999 https://doi.org/10.1109/ICIP.1999.817204
  23. E. Ardizzone, M.L. Cascia, A. Avanzato, and A. Bruna, 'Video indexing using MPEG motion compensation vectors,' IEEE Int. conf. on multimedia computing and systems, vol. 2, pp. 725-729, 1999 https://doi.org/10.1109/MMCS.1999.778574
  24. N.W. Kim, T.Y. Kim, and J,S. Choi, 'Probability-based motion analysis using bi-directional prediction-independent framework in compressed domain,' Optical engineering, vol. 44, no. 6, 2005 https://doi.org/10.1117/1.1926027
  25. Y. Deng, C. Kenney, M.S. Moore, and B.S. Manjunath, 'Peer group filtering and perceptual color image quantization,' Proc. of IEEE Intl. Symposium on Circuits and Systems, vol. 4, pp. 21-24 1999 https://doi.org/10.1109/ISCAS.1999.779933
  26. W. Wolf, 'Key frame selection by motion analysis,' in Proc. IEEE Int. Conf. Acoust., Speech, and Signal Proc., 1996 https://doi.org/10.1109/ICASSP.1996.543588
  27. N.W. Kim, T.Y. Kim, and J,S. Choi, 'Edge-based spatial descriptor using color vector angle for effective image retrieval,' LNAI, vol. 3558, 2005 https://doi.org/10.1007/11526018_36
  28. J. Huang, S. R Kumar, M. Mitra, W. J, Zhu, and R Zabih, 'Image indexing using color correlograrns,' CVPR, pp. 762-768, 1997 https://doi.org/10.1109/CVPR.1997.609412
  29. G. Pass and R Zabih, 'Histogram refinement for content-based image retrieval,' IEEE WACV, pp. 96-102, 1996 https://doi.org/10.1109/ACV.1996.572008