ETRI Journal

Electronics and Telecommunications Research Institute (한국전자통신연구원)
권호 표지

Multimodal Approach for Summarizing and Indexing News Video

  • Kim, Jae-Gon (Broadcasting Media Technology Department, Electronics and Telecommunications Research Institute (ETRI)) ;
  • Chang, Hyun-Sung (Broadcasting Media Technology Department, Electronics and Telecommunications Research Institute (ETRI)) ;
  • Kim, Young-Tae (Broadcasting Media Technology Department, Electronics and Telecommunications Research Institute (ETRI)) ;
  • Kang, Kyeong-Ok (Broadcasting Media Technology Department, Electronics and Telecommunications Research Institute (ETRI)) ;
  • Kim, Mun-Churl (School of Engineering, Information and Communications University (ICU)) ;
  • Kim, Jin-Woong (Broadcasting Media Technology Department, Electronics and Telecommunications Research Institute (ETRI)) ;
  • Kim, Hyung-Myung (Department of Electrical Engineering and Computer Science, Korea Advanced Institute of Science and Technology (KAIST))
  • Received : 2001.06.27
  • Published : 2002.02.28
Download PDF
⟨ Previous Next ⟩

Abstract

A video summary abstracts the gist from an entire video and also enables efficient access to the desired content. In this paper, we propose a novel method for summarizing news video based on multimodal analysis of the content. The proposed method exploits the closed caption data to locate semantically meaningful highlights in a news video and speech signals in an audio stream to align the closed caption data with the video in a time-line. Then, the detected highlights are described using MPEG-7 Summarization Description Scheme, which allows efficient browsing of the content through such functionalities as multi-level abstracts and navigation guidance. Multimodal search and retrieval are also within the proposed framework. By indexing synchronized closed caption data, the video clips are searchable by inputting a text query. Intensive experiments with prototypical systems are presented to demonstrate the validity and reliability of the proposed method in real applications.

Keywords

References

  1. Proc. IEEE ICMCS 98 Exploring Video Structure Beyond the Sohots Rui, Y.;Huang, T.S.;Mehrotra, S.
  2. Proc. IS&T/SPIE Storage and Retrieval for Still Image and Video Database IV v.2670 Clustering Methods for Video Browsing and Annotation Zhong, D.;Zhang, H.J.;Chang, S.F.
  3. IEEE Trans. Circuits Syst. Video Technol. v.9 no.8 An Integrated Scheme for Automated Video Abstraction Based on Unsupervised Cluster-Validity Analysis Hanjalic, A.;Zhang, H.J.
  4. Proc. IEEE CVPR’97 Video Skimming and Characterization through the Combination of Image and Language Understanding Techniques Smith, M.A.;Kanade, T.
  5. J. Vis. Comm. Image Represent. v.7 no.4 Abstracting Digital Movies Automatically Pfeiffer, S.;Lienhart, R.;Fischer, S.;Effelsberg, W.
  6. Proc. IS&T/SPIE Visual Comm. and Image Processing v.4067 Summary Description Schemes for Efficient Video Navigation and Browsing Kim, J.G.;Chang, H.S.;Kim, M.;Kim, J.;Kim, H.M.
  7. ETRI J. v.23 no.2 MPEG-7 Homogeneous Texture Descriptor Ro, Y.M.;Kim, M.;Kang, H.K.;Manjunath, B.S.;Kim, J.
  8. IEEE Trans. Circuits and Syst. for Video Techno. v.11 no.6 MPEG-7 Multimedia Description Schemes Salembier, P.;Smith, J.R.
  9. Text of ISO/IEC FDIS 15938-5 Information Technology ？ Multimedia Content Description Interface ？ Part 5 Multimedia Description Schemes;ISO/IEC JTC1/SC29/WG11 N4205 MPEG MDS Group
  10. ACM Multimedia Systems v.2 no.6 Automatic Parsing and Indexing of News Video Zhang, H.J.;Tan, S.Y.;Smoliar, S.W.;Yihong, G.
  11. Proc. IS&T/SPIE Storage and Retrieval for Image and Video Databases VII v.3656 Semi-Automatic News Analysis, Indexing and Classification System Based on Topics Preselection Hanjalic, A.;Lagendijk, R.L.;Biemond, J.
  12. Proc. ACM Multimedia’97 Broadcast News Navigation Using Story Segmentation Merlino, A.;Morey, D.;Maybury, M.
  13. Proc. IEEE ICASSP'99 Automated Generation of News Content Hierarchy by Integrating Audio, Video, and Text Information Huang, Q.;Liu, Z.;Rosenberg, A.;Gibbon, D.;Shahraray, B.
  14. Proc. IEEE DSP Workshop Application of Speech Recognition with Closed Caption for Content-Based Video Segmentation Son, J.;Kim, J.;Kang, K.;Bae, K.
  15. Proc. IS&T/SPIE Visual Comm. and Image Processing v.4067 A Statistical Approach to Shot Boundary Detection in an MPGE-2 Compressed Video Sequence Shin, T.;Kim, J.G.;Kim, J.;Ahn, B.H.
  16. MPEG-7 Description Schemes (v0.5);ISO/IEC JTC1/SC29/WG11 N2844 MPEG MDS Group
  17. Improved Structure of Hierarchical Summary Description Scheme;ISO/IEC JTC1/SC29/WG11 M6057 Chang, H.S.;Kim, J.G.;Kim, M.;Kim, J.