The Journal of the Korea institute of electronic communication sciences (한국전자통신학회논문지)

Korea Institute of Electronic Communication Science (한국전자통신학회)
권호 표지
DOI QR코드

DOI QR Code

A Study for Change of Audio Data according to Rotation Degree of VR Video

VR 영상의 회전각도에 따른 오디오 데이터 변화에 관한 연구

  • 고은지 (서울과학기술대학교 NID융합기술대학원) ;
  • 양지희 (서울과학기술대학교 NID융합기술대학원) ;
  • 김영애 (서울과학기술대학교 일반대학원) ;
  • 박구만 (서울과학기술대학교 NID융합기술대학원) ;
  • 김성권 (서울과학기술대학교 NID융합기술대학원)
  • Received : 2016.12.15
  • Accepted : 2017.12.15
  • Published : 2017.12.31
Download PDF
⟨ Previous Next ⟩

Abstract

In this paper, we propose an algorithm that can automatically mix the screen and sound by tracking the change of the sound data according to the screen change so that the real sound can be implemented in the personal broadcasting service. Since the personal broadcasting service is often broadcasted lively, it should be convenient to have a real-time mixing. Through experiments, it was confirmed that the sound pressure changes in a wide range in the high frequency band related to the clarity for understanding according to the rotation angle change of the screen. Regression analysis of the sound pressure changes at 2kHz, 4kHz, and 8kHz, The attenuation change of sound pressure was observed at the slope of -1.17, the slope of -2.0, and the slope of -2.44 for each frequency. Therefore, these experiment results can be applied to the VR service. This study is expected to be useful data in the implementation of personal broadcasting service.

본 논문에서는 개인방송서비스에서 실감 음향을 구현할 수 있도록, 화면 변화에 따른 음향 데이터의 변화를 추적하여, 자동적으로 화면과 음향을 믹싱하도록 도와주는 알고리즘을 제안하고자 한다. 실험을 통하여 화면의 회전각도 변화에 따른 음원에서의 고주파 대역에서의 음압 변화를 확인하였고, 소리의 명료도와 관련되는 주파수 대역인 2kHz, 4kHz, 8kHz에서의 음압 변화를 회귀분석을 통하여 각각 -1.17 dB/degree, -2.0 dB/degree, -2.44 dB/degree로 음압의 감쇄 변화 추세를 파악하였다. 이 실험결과는 음압열화를 보완하여 실감음향을 향상시키는 믹싱 알고리즘에 사용되어, 실감 개인방송 서비스 보급에 유용할 것으로 기대된다.

Keywords

References

  1. H. Bae and K. Yoon, "Standardization trend of immersive media based on MPEG-V," J. of the Korea Institute of Communication and Information Sciences, vol. 32, no. 3, 2015, pp. 11-17.
  2. W. Seo, K. Kim, and J. Jeong, "The trend of immersive miedia technology," J. of the Korea Institute of Communication and Information Sciences, vol. 30, no. 5, 4.2013, pp. 79-87.
  3. D. Jang, J. Seo, T. Lee, and K. Kang, "Present and Future of UHD Sound Technology," Korea Society Broadcast Engineers Magazine, vol. 17, no. 5, 10.2012, pp. 47-59.
  4. J. Chun and W. Kim, "3D face pose estimation by a robust real time tracking of facial features," Multimedia Tools and Applications, vol. 75, no. 23, 12. 2016, pp. 15693-15708. https://doi.org/10.1007/s11042-014-2356-9
  5. K. Kim and H. Wei, "Development of a Face Detection and Recognition System Using a RaspberryPi," J. of the Korea Institute of Electronic Communication Sciences, vol. 12, no. 5, 2017, pp. 859-864. https://doi.org/10.13067/JKIECS.2017.12.5.859
  6. S. K. Thompson, K. von Kriegstein, A. Deane-Pratt, T. Marquardt1, R. Deichmann, T. D. Griffiths, and D. McAlpine, "Representation of interaural time delay in the human auditory midbrain," Nature neuroscience, vol. 9, no. 9, 9.2006, pp. 1096-1098. https://doi.org/10.1038/nn1755
  7. S. Sargunam, K. Moghadam, M. Suhail, and E. D. Ragan, "Guided head rotation and amplified head rotation: Evaluating semi-natural travel and viewing techniques in virtual reality," Virtual Reality(VR) 2017 IEEE, 3.2017.
  8. H Wittek and G Theile, "Development and Application of a Stereophonic Multichannel Recording Technique for 3D Audio and VR," Audio Engineering Society Convention 143, 10.2017.
  9. J. Park, "Audio Localization Method for VR Application," Audio Engineering Society Convention 143, 10.2017.
  10. M. Raspaud, H. Viste, and G. Evangelista, "Binaural Source Localization by Joint Estimation of ILD and ITD," IEEE Transactions on Audio, Speech, and Language Processing, vol. 18, no. 1, 1.2010, pp. 68-77. https://doi.org/10.1109/TASL.2009.2023644
  11. Y. Kim, S. Park, and D. Kim, "Research on Robust Face Recognition against Lighting Variation using CNN," J. of the Korea Institute of Information and Communication Engineering, vol. 12, no. 2, 2017, pp. 325-330.