Browse > Article
http://dx.doi.org/10.5909/JBE.2020.25.4.620

Subdivision Ensemble Model for Highlight Detection  

Lee, Hansol (Dept. of Media IT Engineering, Graduate School, Seoul National University of Science and Technology)
Lee, Gyemin (Dept. of Media IT Engineering, Graduate School, Seoul National University of Science and Technology)
Publication Information
Journal of Broadcast Engineering / v.25, no.4, 2020 , pp. 620-628 More about this Journal
Abstract
Automatically predicting video highlight is an important task for media industry and streaming platform providers to save time and cost of manual video editing process. We propose a new ensemble model that combines multiple highlight detectors with each focusing on different parts of highlight events. Therefore, our model can capture more information-rich sections of events. Furthermore, the proposed model can extract improved features for highlight detection particularly when the train video set is small. We evaluate our model on e-sports and baseball videos.
Keywords
Video highlight; Ensemble model; BiLSTM; Event subsection; Event subdivision;
Citations & Related Records
Times Cited By KSCI : 5  (Citation Analysis)
연도 인용수 순위
1 K. Zhang, WL. Chao, F. Sha, and K. Grauman, "Video Summarization with Long Short-term Memory," European Conference on Computer Vision, Amsterdam, Netherlands, pp. 766-782, 2016, https://doi.org/10.1007/978-3-319-46478-7_47
2 K. Zhou, Y. Qiao, and Tao Xiang, "Deep Reinforcement Learning for Unsupervised Video Summarization with Diversity-Representativeness Reward," In Thirty-Second AAAI Conference on Artificial Intelligence, pp. 7582-7589, 2018.
3 B. Zhao, X. Li, and X. Lu, "HSA-RNN: Hierarchical Structure-Adaptive RNN for Video Summarization," The IEEE Conference on Computer Vision and Pattern Recognition(CVPR), pp. 7405-7414, 2018, https://doi.org/10.1109/cvpr.2018.00773
4 B. Mahasseni, M. Lam, and S. Todorovic, "Unsupervised Video Summarization with Adversarial LSTM Networks," In CVPR, pp. 2982-2991, 2017, https://doi.org/10.1109/cvpr.2017.318.
5 I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, "Generative Adversarial Nets," In NIPS, pp. 2672-2680, 2014.
6 K. Zhang, K. Grauman, and F. Sha, "Retrospective Encoders for Video Summarization," In ECCV, pp. 383-399, 2018, https://doi.org/10.1007/978-3-030-01237-3_24.
7 H. Lee, G. Lee, "Summarizing Long-Length Videos with GAN-Enhanced Audio/Visual Features," In ICCV workshop, 2019, https://doi.org/10.1109/iccvw.2019.00462
8 H. Lee, G. Lee, "Video Highlight Prediction Using GAN and Multiple Time-Interval Information of Audio and Image," Journal of Broadcast Engineering, Vol. 25, No. 2, pp. 143-150, 2020, https://doi.org/10.5909/JBE.2020.25.2.143   DOI
9 E. Kim, G. Lee, "Highlight Detection in Personal Broadcasting by Analysing Chat Traffic : Game Contests as a Test Case," Journal of Broadcast Engineering, Vol. 23, No. 2, pp. 218-226, 2018, http://dx.doi.org/10.5909/JBE.2018.23.2.218.   DOI
10 E. Kim, G. Lee, "Video Highlight Prediction Using Multiple Time-Interval Information of Chat and Audio," Journal of Broadcast Engineering, Vol. 24, No. 4, pp. 553-563, 2019, https://doi.org/10.5909/JBE.2019.24.4.1.   DOI
11 Twitch, https://www.twitch.tv/ (accessed May. 20, 2020).
12 Kakao TV, https://tv.kakao.com/ (accessed May. 20, 2020).
13 A. Krizhevsky, I. Sutskever, and G. Hinton, "Imagenet Classification with Deep Convolutional Neural Networks," In NIPS, 2012, https://doi.org/10.1145/3065386.
14 K. He, X. Zhang, S. Ren, and J. Sun, "Deep Residual Learning for Image Recognition," In CVPR, pp. 770-778, 2016, https://doi.org/10.1109/cvpr.2016.90.
15 Naver-sports, https://sports.news.naver.com/ (accepted May. 20, 2020).
16 OGN, http://ogn.tving.com/ (accepted May. 20, 2020).