• The KIPS Transactions:PartB
• /
• v.12B no.4 s.100
• /
• pp.379-386
• /
• 2005

This paper provides a rigorous theory for analysis of quantization effects and optimum filter bank design in quantized multidimensional subband filter banks. Even though subband filter design has been a hot topic for last decades, a few results have been reported on the subband filter with a quantizer. Each pdf-optimized quantizer is modeled by a nonlinear gain-plus-additive uncorrelated noise and embedded into the subband structure. Using polyphase decomposition of the analysis/synthesis filter banks, we derive the exact expression for the output mean square quantization error. Based on the minimization of the output mean square error, the technique for optimal filter design methodology is developed. Numerical design examples for optimum nonseparable paraunitary and biorthogonal filter banks are presented with a quincunx subsampling lattice. Through the simulation, $10\~20\;\%$ decreases in MSE have been observed compared with subband filter with no quantizers especially for low bit rate cases.