DOI QR코드

DOI QR Code

Automatic Video Editing Technology based on Matching System using Genre Characteristic Patterns

장르 특성 패턴을 활용한 매칭시스템 기반의 자동영상편집 기술

  • Mun, Hyejun (Department of IT Media Engineering, Duksung Women's University) ;
  • Lim, Yangmi (Department of IT Media Engineering, Duksung Women's University)
  • 문혜준 (덕성여자대학교 IT 미디어공학과) ;
  • 임양미 (덕성여자대학교 IT 미디어공학과)
  • Received : 2020.09.14
  • Accepted : 2020.10.21
  • Published : 2020.11.30

Abstract

We introduce the application that automatically makes several images stored in user's device into one video by using the different climax patterns appearing for each film genre. For the classification of the genre characteristics of movies, a climax pattern model style was created by analyzing the genre of domestic movie drama, action, horror and foreign movie drama, action, and horror. The climax pattern was characterized by the change in shot size, the length of the shot, and the frequency of insert use in a specific scene part of the movie, and the result was visualized. The model visualized by genre developed as a template using Firebase DB. Images stored in the user's device were selected and matched with the climax pattern model developed as a template for each genre. Although it is a short video, it is a feature of the proposed application that it can create an emotional story video that reflects the characteristics of the genre. Recently, platform operators such as YouTube and Naver are upgrading applications that automatically generate video using a picture or video taken by the user directly with a smartphone. However, applications that have genre characteristics like movies or include video-generation technology to show stories are still insufficient. It is predicted that the proposed automatic video editing has the potential to develop into a video editing application capable of transmitting emotions.

본 논문은 영화 장르마다 나타나는 클라이맥스 패턴이 다름을 활용하여 사용자의 디바이스 내에 저장되어 있는 이미지들을 하나의 영상으로 자동생성해주는 애플리케이션 개발을 소개한다. 영화의 장르 특성 분류는 국내 영화 드라마, 액션, 공포와 국외 영화 드라마, 액션, 공포 장르를 분석하여 클라맥스 패턴 모델형을 만들었다. 클라이맥스 패턴은 영화의 특정 씬 부분에서 샷사이즈의 변화, 샷의 길이, 인서트샷 사용의 빈도를 특성 요소로 하였고, 결과를 시각화하였다. 장르별 시각화된 모델을 Firebase DB를 활용하는 템플릿으로 개발하였다. 사용자의 디바이스에 저장된 이미지를 선택하여 장르별 템플릿으로 개발된 클라이맥스 패턴 모델과 매칭하였다. 짧은 영상이지만 장르의 특성이 반영되어 감성스토리 영상을 자동생성할 수 있는 것이 본 애플리케이션의 특징이다. 최근 유튜브, 네이버와 같은 플랫폼 사업자들은 사용자가 스마트폰으로 직접 촬영한 사진이나 영상을 활용하여 자동으로 영상을 생성해주는 애플리케이션들을 매년 업그래이드하고 있으나, 영화와 같이 장르 특성을 갖는다거나, 스토리가 보이는 영상생성 기술을 포함한 애플리케이션은 아직 미흡하다. 제안한 자동영상편집은 감성전달이 가능한 영상편집 애플리케이션으로써의 발전 가능성이 있다고 예측한다.

Keywords

References

  1. IBM Research, https://www-903.ibm.com/kr/research/?lnk=fdi
  2. S.R. Park and Y.M. Lim, "The Implementing a Color, Edge, Optical Flow based on Mixed Algorithm for Shot Boundary Improvemen," Journal of Korea Multimedia Society, Vol. 21, No. 8, pp. 829-836, August 2018. https://doi.org/10.9717/KMMS.2018.21.8.829
  3. S.R. Park, J.E. Eum and Y.M. Lim, "A Study of Shot Boundary Detection for Film and TV Drama Images," Proceeding of the 14th International Conference on Multimedia Information Technology and Applications, Shanghai, China, pp. 111-113, 2018.
  4. D.W. Kang and Y.M. Lim, "Generating Extreme Close-up Shot Dataset Based On ROI Detection for Classifying Shots Using Artificial Neural Network," Journal of the Korean Society Of Broad Engineers, Vol. 24, No. 6, November 2019, pp.983-991, https://doi.org/10.5909/JBE.2019.24.6.983
  5. J.E. Eum, S.R. Park and Y.M. Lim, "The System Design and Implementation for Detecting the Types of Shot Siz," Proceeding of Multimedia Information Technology and Applications. Vol. 21, No. 1, pp. 968-969, spring 2018.
  6. J.E. Eum, S.R. Park and Y.M. Lim, "Design of Deep Learning for Movie Shot Size Recognition," Proceeding of Multimedia Information Technology and Applications. Vol. 20, No. 2, pp. 555-556, autumn 2017.
  7. KCC, Research on the method of promoting the platform and application indust, Broadcasting communication policy research, 2011-2, 2011.
  8. Google LLC., https://play.google.com/store/apps/details?id=com.darinsoft.vimo&hl=ko
  9. Facebook Inc., https://about.fb.com/ko/news/2018/07/watch-party/
  10. BylineNetwork Inc., https://byline.network/2020/01/20-76/
  11. J. Long, E. Shelhamer and T. Darrell, "Fully Convolutional Networks for Semantic Segmentation," 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 3431-3440, June 2015, DOI: 10.1109/CVPR.2015.7298965
  12. V. Gopalakrishnan, Y. Hu and D. Rajan, "Salient region detection by modeling distributions of color and orientation," IEEE Trans. Multimedia, Vol. 11, pp. 892-905, 2009, https://doi.org/10.1109/TMM.2009.2021726
  13. N. Liu, J. Han, and M. H. Yang, "PiCANet: Learning pixel-wise contextual attention for saliency detection," In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3089-3098, 2018, https://doi.org/10.1109/CVPR.2018.00326
  14. K. Simonyan, A. Zisserman, "Very Deep Convolutional Networks for Large-Scale Image Recognition," Published International Conference on Learning Representations, at ICLR 2015
  15. Google Pixabay., https://pixabay.com/ko/