A Study on the VLSI Design of Efficient Color Interpolation Technique Using Spatial Correlation for CCD/CMOS Image Sensor

화소 간 상관관계를 이용한 CCD/CMOS 이미지 센서용 색 보간 기법 및 VLSI 설계에 관한 연구

  • Lee, Won-Jae (Department of Electrical and Electronic Engineering, Yonsei University) ;
  • Lee, Seong-Joo (Department of Information and Communication Engineering, Sejong University) ;
  • Kim, Jae-Seok (Department of Electrical and Electronic Engineering, Yonsei University)
  • 이원재 (연세대학교 전기전자공학과) ;
  • 이성주 (세종대학교 정보통신공학과) ;
  • 김재석 (연세대학교 전기전자공학과)
  • Published : 2006.11.25

Abstract

In this paper, we propose a cost-effective color filter may (CFA) demosaicing method for digital still cameras in which a single CCD or CMOS image sensor is used. Since a CFA is adopted, we must interpolate missing color values in the red, green and blue channels at each pixel location. While most state-of-the-art algorithms invest a great deal of computational effort in the enhancement of the reconstructed image to overcome the color artifacts, we focus on eliminating the color artifacts with low computational complexity. Using spatial correlation of the adjacent pixels, the edge-directional information of the neighbor pixels is used for determining the edge direction of the current pixel. We apply our method to the state-of-the-art algorithms which use edge-directed methods to interpolate the missing color channels. The experiment results show that the proposed method enhances the demosaiced image qualify from $0.09{\sim}0.47dB$ in PSNR depending on the basis algorithm by removing most of the color artifacts. The proposed method was implemented and verified successfully using verilog HDL and FPGA. It was synthesized to gate-level circuits using 0.25um CMOS standard cell library. The total logic gate count is 12K, and five line memories are used.

본 논문에서는 화소간의 상관관계를 이용한 CCD/CMOS 이미지 센서용 효율적인 색 보간 기법을 제안한다. 최근 각광받고 있는 CCD/CMOS 이미지 센서는 컬러 필터 배열(Color Filter Array)을 사용하기 때문에, 각 화소는 컬러 영상을 만들기 위한 3가지 색 채널 중 한 가지 채널만 갖고 있게 된다. 따라서 컬러 영상을 만들기 위해서는 색 보간 구조가 필요하다. 최근 제안되는 색 보간 기법은 보간된 영상의 품질 향상에만 주력하고 있는데 반해, 본 논문에서는 낮은 복잡도를 갖으면서 잘못된 색을 최소화하기 위한 방법을 제안한다. 제안된 색 보간 기법에서는 인접한 화소간의 상관관계를 이용하여, 현재 화소의 방향성을 결정할 때 이웃 화소의 방향성 정보를 이용하였다. 기존의 방향성을 고려한 색 보간 기법에 제안된 기법을 적용한 결과, 알고리즘의 종류에 따라 PSNR이 $0.09{\sim}0.47dB$ 향상되었고, 대부분의 잘못된 색(False color)을 최소화함으로써 색 보간된 컬러영상의 품질이 향상되었다. 제안된 색 보간 기법은 Verilog HDL 및 FPGA를 이용하여 실시간으로 구현 검증되었다. 0.25um CMOS 표준 셀 라이브러리를 이용하여 합성하였을 때, 총 게이트 수는 12K개였으며 5개의 라인 메모리가 사용되었다.

Keywords

References

  1. Bayer, Bryce E. 'Color imaging array' U.S. Patent 3,971,065
  2. Keys, Robert.G. et.al., 'Cubic Convolution Interpolation for Digital Image Processing' IEEE Transactions on Acoustic, Speech and Signal Processing, Vol. ASSP-29, P1153-1160, 1981 https://doi.org/10.1109/TASSP.1981.1163711
  3. D.R. Cok, 'Signal processing method and apparatus for producing interpolated chrominance values in a sampled color image,' U.S. patent 4,642,768
  4. J. A. Weldy, 'Optimized design for a single-sensor color electronic camera system,' Proc. SPIE, vol. 1071, pp. 300-307, 1988
  5. Adams, James E. 'Interactions between color plane interpolation and other image processing functions in electronic photography,' Proceedings of SPIE Vol. 2416 P.144-151 https://doi.org/10.1117/12.204825
  6. S.-C. Pei and I.-K. Tam, 'Effective color interpolation in CCD color filter arrays using signal correlation,' IEEE Trans. Circuits and Systems for Video Technology, vol. 13, no. 6, pp. 503-513, June 2003 https://doi.org/10.1109/TCSVT.2003.813422
  7. R. Kakarala and Z. Baharav, 'Adaptive demosaicing with the principle vector method,' IEEE Trans. Consumer Electron., vol. 48, pp. 932-937, Nov. 2002 https://doi.org/10.1109/TCE.2003.1196423
  8. C. A. Laroche and M. A. Prescott, 'Apparatus and method for adaptively interpolating a full color image utilizing chrominance gradients,' U.S. Patent 5,373,322, 1994
  9. R. H. Hibbard, 'Apparatus and method for adaptively interpolating a full color image utilizing luminance gradients,' U.S. Patent 5,382,976, 1995
  10. J. E. Adams and J. F. Hamilton Jr., 'Adaptive color plane interpolation in single color electronic camera,' U.S. Patent 5 506 619, Apr. 1996
  11. J. E. Adams and J. F. Hamilton Jr., 'Adaptive color plane interpolation in single sensor color electronic camera' U.S. Patent 5,629,734
  12. X. Li and M. T. Orchard, 'New edge directed interpolation,' IEEE Transactions on Image Processing, vol. 10, no. 10, 2001 https://doi.org/10.1109/83.951537
  13. R. Kimmel, 'Demosaicing: image reconstruction from CCD samples,' IEEE Trans. Image Processing, vol. 8, pp. 1221-1228, 1999 https://doi.org/10.1109/83.784434
  14. S. S. D. Alleysson and J. Herault, 'Color demosaicing by estimating luminance and opponent chromatic signals in the fourier domain,' Proc. IS&T/SID 10th Color Imaging Conference, pp. 331-336, 2002
  15. B. S. Hur and M. G. Kang, 'High definition color interpolation scheme for progressive scan CCD image sensor,' IEEE Transactions on Consumer Electronics, vol. 47, no. 1, pp. 179-186, Feb. 2001 https://doi.org/10.1109/30.920437
  16. B. K. Gunturk, Y. Altunbasak and R. M. Mersereau, 'Color plane interpolation using alternating projections,' IEEE Transactions on Image Processing, vol. 11, no. 9, 2002 https://doi.org/10.1109/TIP.2002.801121
  17. W. Lu, and Y. Tan, 'Color filter array demosaicing: New method and performance measeure,' IEEE Trans. Image Processing, vol. 12, no. 10, pp. 1194-1210, Oct. 2003 https://doi.org/10.1109/TIP.2003.816004
  18. X. Li, 'Demosaicing by Successive Approximation, IEEE Transactions on Image Processing, vol. 14, no. 3, March 2005 https://doi.org/10.1109/TIP.2004.840683