• 한국HCI학회:학술대회논문집
• /
• 2009.02a
• /
• pp.150-157
• /
• 2009

In this paper, we propose a person tracking based multi-channel audio panning system for multi-view broadcasting services. Multi-view broadcasting is to render the video sequences that are captured from a set of cameras based on different viewpoints, and multi-channel audio panning techniques are necessary for audio rendering in these services. In order to apply such a realistic audio technique to this multi-view broadcasting service, person tracking techniques which are to estimate the position of users are also necessary. For these reasons, proposed methods are composed of two parts. The first part is a person tracking method by using ultrasonic satellites and receiver. We could obtain user's coordinates of high resolution and short duration about 10 mm and 150 ms. The second part is MPEG Surround parameter-based multi-channel audio panning method. It is a method to obtain panned multi-channel audio by controlling the MPEG Surround spatial parameters. A MUSHRA test is conducted to objectively evaluate the perceptual quality and measure localization performance using a dummy head. From the experiments, it is shown that the proposed method provides better perceptual quality and localization performance than the conventional parameter-based audio panning method. In addition, we implement the prototype of person tracking based multi-view broadcasting system by integrating proposed methods with multi-view display system.

• The Journal of the Acoustical Society of Korea
• /
• v.43 no.5
• /
• pp.529-535
• /
• 2024

This paper presents our research on automatic audio panning for media content production. Previously, tracking an audio was done manually. With the advent of the immersive audio era, the need for an automatic audio panning system has increased, yet no substantial research has been progressed to date. Therefore, we propose a computer vision-based human tracking and depth feature processing system which processes depth feature through using 2-dimensional coordinates and models 3-dimensional view transformation for automatic audio panning to ensure audiovisual congruence. Also, this system applies stage size estimation model which gets input as an image and extrapolates stage width and depth as meter unit. Since our system estimates stage sizes and directly applies them to view transformation, no additional depth data training is required. To validate the proposed system, we also conducted a pilot test with Unity based sample video. Our team expects that our system will enable automated audio panning, assisting many audio engineers.

• Proceedings of the Korean Society of Broadcast Engineers Conference
• /
• 2003.11a
• /
• pp.237-240
• /
• 2003

실제공간에서 음원이 존재하는 것처럼 음향을 재현하려 연 스테레오만으로는 완벽한 재현이 어렵다. 본 논문에서는 4채널 오디오 출력을 이용해서 가상환경의 음원이 실제 공간에 존재하는 것처럼 재생하는 방법을 기술한다. 잔향 임펄스 응답을 이용해서 공간감을 구현하였고, 음원과 청자의 거리에 따라 감을 조절하였으며, 4채널에 균일 파워 패닝 기법을 사용해서 방향감을 구현하였다. PC에서 입체음향이 되도록 구현하였으며, 실시간으로 음원과 청자의 위치를 변화시킬 수 있도록 하였다. 본 논문에서는 공간감, 음량감과 방향감을 고려하는 4채널 오디오 출력을 이용한 실시간 입체음향 구현에 대해 기술한다.

• Journal of Broadcast Engineering
• /
• v.17 no.6
• /
• pp.924-932
• /
• 2012

Decomposition of a stereo signal into the primary and ambient components is a key step to the stereo upmix and it is often based on the principal component analysis (PCA). However, major shortcoming of the PCA-based method is that accuracy of the decomposed components is dependent on both the primary-to-ambient power ratio (PAR) and the panning angle. Previously, a modified PCA was suggested to solve the PAR-dependent problem. However, its performance is still dependent on the panning angle of the primary signal. In this paper, we proposed a new PCA-based primary-ambient decomposition algorithm whose performance is not affected by the PAR as well as the panning angle. The proposed algorithm finds scale factors based on a criterion that is set to preserve the powers of the mixed components, so that the original primary and ambient powers are correctly retrieved. Simulation results are presented to show the effectiveness of the proposed algorithm.

• Proceedings of the Korean Society of Broadcast Engineers Conference
• /
• 2012.07a
• /
• pp.160-163
• /
• 2012

업믹스(Upmix) 기술은 홈시어터와 같은 다채널 스피커 재생 환경에서 콘텐츠의 대부분을 차지하는 스테레오 음원을 다채널 환경에 재생하기 위한 채널 포맷 변환 기술을 말한다. 업믹스를 위한 전처리 단계로서 특정 방향으로 패닝된 주(primary)성분과 잔향 및 배경음과 같은 Ambient 성분을 분리하는 과정이 필요하다. Primary와 Ambient를 분리하기 위한 방법으로 채널 간의 상관도, 적응 필터 및 주성분 분석법(principal component analysis, PCA)이 널리 이용되고 있다. 이에 본 논문에서는 비교적 정확하게 Primary와 Ambient를 분리한다고 알려진 주성분 분석법을 이용하여 신호를 분리해 내고 이 때 주성분 분석법이 가지는 문제점을 해결한 향상된 Primary-Ambient 분리 알고리즘을 제안하였다. 제안된 알고리즘은 분리 성능이 Primary 성분이 패닝된 각도에 영향을 받지 않으며 또한 Primary 성분에 섞인 잔여 Ambient를 제거함으로써 기존의 주성분 분석법 보다 더 정확하게 Primary와 Ambient를 분리 할 수 있고 상관성이 없는 Ambient 특성을 좀 더 정확하게 반영한다.

• The Journal of the Acoustical Society of Korea
• /
• v.36 no.5
• /
• pp.338-344
• /
• 2017

In this paper, an interpolation method of HRTF (Head-Related Transfer Function) is proposed for small-sized measurement data set, especially. The proposed algorithm is based on acoustic modeling of HRTFs, and the algorithm tries to interpolate the HRTFs via estimation the model coefficients. However, the estimation of the model coefficients is hard if there is lack of measurement points, so the algorithm solves the problem by a data augmentation using the VBAP (Vector Based Amplitude Panning). Therefore, the proposed algorithm consists of two steps, which are data augmentation step based on VBAP and model coefficients estimation step by least squares method. The proposed algorithm was evaluated by a simulation with a measured data from CIPIC (Center for Image Processing and Integrated Computing) HRTF database, and the simulation results show that the proposed algorithm reduces mean-squared error by 1.5 dB ~ 4 dB than the conventional algorithms.

• Journal of the Institute of Electronics Engineers of Korea SP
• /
• v.47 no.5
• /
• pp.34-44
• /
• 2010

In this study, the algorithms for multichannel format conversion and robust representation of spatial sound information are proposed. In the spatial analysis, the directional information of sound source is estimated and sound sources are separated from stereo signal. In the spatial resynthesis, the multichannel matrixing with spatial repanning and post-scaling method are applied to represent a spatial sound. The conventional method about channel format conversion has the problem that the energy of sound source and the spatial information are not preserved in the desired channel format. Because the proposed method is designed in consideration of the target multichannel format and its resynthesized signal, the robust representation of spatial sound can be achieved in the multichannel format conversion.