Journal of Broadcast Engineering (방송공학회논문지)

The Korean Institute of Broadcast and Media Engineers (한국방송∙미디어공학회)
권호 표지
DOI QR코드

DOI QR Code

Implementation of Integrated Player System based on Free-Viewpoint Video Service according to User Selection

사용자 선택에 따른 자유 시점 비디오 서비스 기반의 통합 플레이어 시스템 구현

  • Yang, Ji-hee (Dept. of Information Technology and Media Engineering, The graduate School of Nano IT Design Fusion, Seoul National University of Science and Technology) ;
  • Song, Min-ki (Dept. of Media IT Engineering, The Graduate School, Seoul National University of Science and Technology) ;
  • Park, Gooman (Dept. of Electronic IT Media Engineering, Seoul National University of Science and Technology)
  • 양지희 (서울과학기술대학교 나노IT디자인융합대학원 정보통신미디어공학전공) ;
  • 송민기 (서울과학기술대학교 일반대학원 미디어IT공학과) ;
  • 박구만 (서울과학기술대학교 전자IT미디어공학과)
  • Received : 2020.02.25
  • Accepted : 2020.03.24
  • Published : 2020.03.30
Download PDF
⟨ Previous Next ⟩

Abstract

Free-viewpoint video service is a technology that allows users to watch at any angle, location and distance through interaction. In this paper, the free-viewpoint video services are defined in four viewing modes: Inward view, outward view, 3D object view and first person view. And we developed and implemented a new integrated program that plays all the suggested views. In the contents of girl band performances and basketball games, multi-view cameras suitable for each viewing mode are installed to acquire media, and data stored on the server is streamed over the network, making it available for viewing. Users can freely choose four viewing modes, space location, angle and so on, and the media data such as images and sounds are provided to them by rendering appropriately for the selected the viewpoint. Our system is expected to be a scalable free-viewpoint video service player as well as provide users with immersion and presence by combining various viewing modes.

자유 시점 비디오 서비스는 사용자와의 상호작용을 통해 원하는 임의의 각도나 위치, 거리에서 시청할 수 있게 하는 기술이다. 본 논문에서는 사용자가 선택하여 시청할 수 있는 자유 시점 비디오 서비스를 Inward view, Outward view, 3D object view, First person view의 네 가지 시청 모드로 정의하였으며 하나의 플레이어에서 시청이 가능한 새로운 통합 프로그램을 개발 및 구현하였다. 아이돌 공연과 농구 경기 콘텐츠에 대해 각 시청 모드에 적합한 다시점 카메라를 설치하여 데이터를 확보하였으며, 서버에 저장된 데이터가 네트워크를 통해 스트리밍 됨으로써 시청이 가능하도록 하였다. 사용자는 자유롭게 네 가지의 시청 모드와 공간상의 위치, 각도 등을 선택할 수 있으며, 선택된 시점에 맞는 영상과 음향이 렌더링 되어 통합 플레이어에 표출된다. 이는 기존의 자유 시점 비디오 서비스를 포함한 다양한 시청 형태를 결합함으로써 사용자에게 몰입감과 현장감을 제공할 수 있을 뿐만 아니라 확장성이 있는 자유 시점 비디오 서비스 플레이어가 될 것으로 기대된다.

Keywords

References

  1. M. Domanski, O. Stankiewicz, K, Wegner and T, Grajek, "Immersive Visual Media-MPEG-I:360 Video, Virtual Navigation and Beyond", Proceeding of International Conference on Systems, Signals and Image Processing(IWSSIP), Poznan, Poland, pp.1-9, May, 2017, doi: 10.1109/IWSSIP.2017.7965623
  2. X. Zhang, L. Toni, P. Frossard, Y. Zhao and C. Lin, "Adaptive Streaming in Interactive Multiview Video Systems", Transactions on Circuits and Systems for Video Technology, Vol.29, No.4, pp.1130-1144, April, 2019, doi: 10.1109/TCSVT.2018.2819804
  3. G. Park, "Free-Viewpoint Content Composition Technology Development Case", Broadcasting and Media Magazine, Vol.24, No.3, pp.24-34. July, 2019.
  4. G. Park, "Free-Viewpoint Video Technology Trends and Free Selection View Development Case", The Journal of The Korean Institute of Communication Sciences, Vol.36, No.12, pp.3-9, Nov, 2019.
  5. G. Lafruit, M. Domanski, K. Wegner and T. Grajek, et al, "New Visual Coding Exploration in MPEG: Super-MultiView and Free Navigation in Free Viewpoint TV", Electronics Imaging, Stereoscopic Displays and Applications XXVII, pp.1-9, Feb, 2016, doi: 10.2352/ISSN. 2470-1173.2016.5.SDA-426
  6. S. Song, Y. Park and J. Wee, "Streaming Technology for Free-Viewpoint Service", The Journal of The Korean Institute of Communication Sciences, Vol.36, No.12, pp.32-37, Nov, 2019.
  7. S. Yairi, Y. Iwaya, M. Kobayashi, M. Otani, Y. Suzuki and T. Chiba, "The Effects of Ambient Sounds on The Quality of 3D Virtual Sound Space", Proceeding of International Conference on Intelligent Information Hiding and Multimedia Signal Processing, Kyoto, Japan, pp.1122-1125, Sept, 2009, doi: 10.1109/IIH-MSP.2009.267
  8. Fmod API Documentation, https://www.fmod.com/resources/documentation-api.
  9. S. Jeon, C. Kim and G. Park, "Implementation of Omni-directional Image Viewer Program for Effective Monitoring", Journal of Broadcasting Engineering, Vol.23, No.6, pp. 939-946, Nov, 2018, doi: 10.5909/JBE.2018.23.6.939
  10. J. Jeong, M. Yoon, S. Kim and G. Park, "Design and Production of Realtime 3D Animation Viewer based on Motion Capture", Proceeding of The Institute of Electronics and Information Engineers Conference, Jeju, Korea, pp.531-535, July, 2019.
  11. Z. Cao, G. Hidalgo, T. Simon, S. E. Wei and Y. Sheikh, "OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields", Transactions on Pattern Analysis and Machine Intelligence, pp.1-14, June, 2018, doi: 10.1109/TPAMI.2019.2929257
  12. J. Jeon, H. Lim and C. Kim, "A Study of Dynamic 3D Modeling based on Photogrammetry Technology for Object Observation View", The Journal of The Korean Institute of Communication Sciences, Vol.36, No.12, pp.38-44, Nov, 2019.