Journal of Broadcast Engineering (방송공학회논문지)

The Korean Institute of Broadcast and Media Engineers (한국방송∙미디어공학회)
권호 표지
DOI QR코드

DOI QR Code

A Real Time 6 DoF Spatial Audio Rendering System based on MPEG-I AEP

MPEG-I AEP 기반 실시간 6 자유도 공간음향 렌더링 시스템

  • Kyeongok Kang (Media Research Division, Hyper-Reality Metaverse Research Laboratory, ETRI) ;
  • Jae-hyoun Yoo (Media Research Division, Hyper-Reality Metaverse Research Laboratory, ETRI) ;
  • Daeyoung Jang (Media Research Division, Hyper-Reality Metaverse Research Laboratory, ETRI) ;
  • Yong Ju Lee (Media Research Division, Hyper-Reality Metaverse Research Laboratory, ETRI) ;
  • Taejin Lee (Media Research Division, Hyper-Reality Metaverse Research Laboratory, ETRI)
  • 강경옥 (한국전자통신연구원 초실감메타버스연구소 미디어연구본부) ;
  • 유재현 (한국전자통신연구원 초실감메타버스연구소 미디어연구본부) ;
  • 장대영 (한국전자통신연구원 초실감메타버스연구소 미디어연구본부) ;
  • 이용주 (한국전자통신연구원 초실감메타버스연구소 미디어연구본부) ;
  • 이태진 (한국전자통신연구원 초실감메타버스연구소 미디어연구본부)
  • Received : 2022.12.23
  • Accepted : 2023.02.14
  • Published : 2023.03.30
Download PDF
⟨ Previous Next ⟩

Abstract

In this paper, we introduce a spatial sound rendering system that provides 6DoF spatial sound in real time in response to the movement of a listener located in a virtual environment. This system was implemented using MPEG-I AEP as a development environment for the CfP response of MPEG-I Immersive Audio and consists of an encoder and a renderer including a decoder. The encoder serves to offline encode metadata such as the spatial audio parameters of the virtual space scene included in EIF and the directivity information of the sound source provided in the SOFA file and deliver them to the bitstream. The renderer receives the transmitted bitstream and performs 6DoF spatial sound rendering in real time according to the position of the listener. The main spatial sound processing technologies applied to the rendering system include sound source effect and obstacle effect, and other ones for the system processing include Doppler effect, sound field effect and etc. The results of self-subjective evaluation of the developed system are introduced.

본 논문에서는 가상환경에 위치한 청취자의 움직임에 대응하여 실시간으로 6DoF 공간음향을 제공하는 공간음향 렌더링 시스템에 대해 소개한다. 본 시스템은 MPEG-I Immersive Audio CfP 대응을 위하여 MPEG-I AEP를 개발환경으로 사용하여 구현되었으며 인코더와, 디코더를 포함하는 렌더러로 구성된다. 인코더는 인코더 입력 포맷(EIF) 파일에 포함된 가상공간 장면의 공간적 오디오 파라미터와, SOFA 파일로 제공되는 음원의 지향성 정보 등의 메타데이터를 오프라인으로 부호화하여 비트스트림으로 전달하는 역할을 하며, 렌더러는 전달된 비트스트림을 수신하여 청취자의 위치에 따라 실시간으로 6DoF 공간음향 렌더링을 수행한다. 개발된 렌더링 시스템에 적용한 주요 공간음향 처리 기술로는 음원 효과 및 장애물 효과 처리 기술이 있으며, 그 외 시스템 동작에 필요한 기술로는 도플러 효과 및 음장효과 처리 기술 등이 있다. 개발된 시스템에 대한 성능평가 결과로서 자체 주관평가 결과를 소개한다.

Keywords

Acknowledgement

This work was supported by Electronics and Telecommunications Research Institute (ETRI) grant funded by the Korean government. [22ZH1200, The research of the basic media·contents technologies]

References

  1. Lauri Savioja et al., "Creating Interactive Virtual Acoustic Environments," Journal of Audio Engineering Society, Vol.47, No.9, pp.675-705, September 1999. http://www.aes.org/e-lib/browse.cfm?elib=12095
  2. Lauri Savioja et al., "Overview of geometrical room acoustic modeling techniques," Journal of Acoustic Society of America, Vol.138, No.2, pp.708-730, August 2015. doi: https://doi.org/10.1121/1.4926438
  3. Room Acoustics Modeling with Interactive Visualizations (by Lauri Savioja), https://interactiveacoustics.info/ (accessed Feb. 9, 2023)
  4. Resonance Audio, https://resonance-audio.github.io/resonance-audio/ (accessed Feb. 9, 2023)
  5. EVERTims, https://evertims.github.io/ (accessed Feb. 9, 2023)
  6. RAVEN, https://www.akustik.rwth-aachen.de/go/id/dwoc/lidx/1/file/183613 (accessed Feb. 9, 2023) RAVEN: A real-time framework for the auralization of interactive virtual environments
  7. ISO/IEC JTC1/SC29/WG6 N0056, "MPEG-I Immersive Audio Call for proposals," Virtual, April 2021.
  8. ISO/IEC JTC1/SC29/WG6 N100, "MPEG-I Immersive Audio Documentation for the Audio Evaluation Platform, Version 2" Virtual, October 2021.
  9. ISO/IEC JTC1/SC29/WG6 N0028, "MPEG-I Immersive Audio Architecture and Requirements," Virtual, January 2021.
  10. ISO/IEC JTC1/SC29/WG6 N0054, "MPEG-I Immersive Audio Encoder Input Format," Virtual, April 2021.
  11. ETSI TS 126 260 V15.0.0 (2018-10), "5G; Objective test methodologies for the evaluation of immersive audio systems (3GPP TS 26.260 version 15.0.0 Release 15), 2018
  12. N0101 (N20921), "MPEG-I Immersive Audio Subjective Test Logistics document", 5th WG6 meeting (136th MPEG meeting), October 2021, Virtual.
  13. ISO/IEC JTC1/SC29/WG6 N0105, "MPEG-I Immersive Audio additional instructions for Test Supervisors and Test Subjects," Virtual, October 2021, Virtual.
  14. ISO/IEC JTC1/SC29/WG6 m55106, "Thoughts on evaluation procedure for MPEG-I CfP," Virtual, October 2020..
  15. ISO/IEC JTC1/SC29/WG6 N0084, "MPEG-I Immersive Audio Test and Evaluation Procedures," Virtual, July 2021.