Journal of the Korea Institute of Information and Communication Engineering (한국정보통신학회논문지)

The Korea Institute of Information and Commucation Engineering (한국정보통신학회)
권호 표지
DOI QR코드

DOI QR Code

Real-Time Hardware Design of Image Quality Enhancement Algorithm using Multiple Exposure Images

다중 노출 영상을 이용한 영상의 화질 개선 알고리즘의 실시간 하드웨어 설계

  • Lee, Seungmin (Department of Electronic Engineering, Dong-A University) ;
  • Kang, Bongsoon (Department of Electronic Engineering, Dong-A University)
  • Received : 2018.07.13
  • Accepted : 2018.08.14
  • Published : 2018.11.30
Download PDF
⟨ Previous Next ⟩

Abstract

A number of algorithms for improving the image quality of low light images have been studied using a single image or multiple exposure images. The low light image is low in contrast and has a large amount of noise, which limits the identification of information of the subject. This paper proposes the hardware design of algorithms that improve the quality of low light image using 2 multiple exposure images taken with a dual camera. The proposed hardware structure is designed in real time processing in a way that does not use frame memory and line memory using transfer function. The proposed hardware design has been designed using Verilog and validated in Modelsim. Finally, when the proposed algorithm is implemented on FPGA using xc7z045-2ffg900 as the target board, the maximum operating frequency is 167.617MHz. When the image size is 1920x1080, the total clock cycle time is 2,076,601 and can be processed in real time at 80.7fps.

단일 노출 영상, 또는 다중 노출 영상을 사용하여 저조도 영상의 화질 개선 알고리즘이 수많이 연구되고 있다. 저조도 영상은 명암이 낮고, 잡음이 많아 피사체의 정보를 식별하기에 한계가 있다. 본 논문에서는 듀얼카메라로 촬영한 다중 노출 영상 2개를 이용하여 저조도 영상의 화질 개선하는 알고리즘의 하드웨어 설계를 제안한다. 제안하는 하드웨어 구조는 전달함수를 사용하여 프레임 메모리와 라인 메모리를 쓰지 않는 방식으로 실시간 처리로 설계되었다. 그리고 제안하는 하드웨어 설계는 Verilog로 설계했고, Modelsim을 사용하여 검증했다. 마지막으로 Xilinx사의 xc7z045-2ffg900을 목표 보드로 이용하여 FPGA를 구현했을 때 최대 동작 주파수 167.617MHz로 확인하였고, 영상 크기가 $1920{\times}1080$ 일 때, 소요된 총 클럭 사이클은 2,076,601이며 80.7fps로 실시간 처리가 가능하다.

Keywords

HOJBC0_2018_v22n11_1462_f0001.png 이미지

Fig. 1 Flow chart of the proposed algorithm

HOJBC0_2018_v22n11_1462_f0002.png 이미지

Fig. 2 Input images (a) LET image (b) SET image

HOJBC0_2018_v22n11_1462_f0003.png 이미지

Fig. 3 Transfer function using LET image characteristics

HOJBC0_2018_v22n11_1462_f0004.png 이미지

Fig. 4 Transfer function using SET image characteristics

HOJBC0_2018_v22n11_1462_f0005.png 이미지

Fig. 5 Histogram of LET image

HOJBC0_2018_v22n11_1462_f0006.png 이미지

Fig. 6 Histogram of SET image

HOJBC0_2018_v22n11_1462_f0007.png 이미지

Fig. 7 Histogram of blended image

HOJBC0_2018_v22n11_1462_f0008.png 이미지

Fig. 8 Result images (a) blended image (b) proposed result image

HOJBC0_2018_v22n11_1462_f0009.png 이미지

Fig. 9 Histogram of proposed result image

HOJBC0_2018_v22n11_1462_f0010.png 이미지

Fig. 10 Hardware block diagram

Table. 1 Summary of Xilinx Synthetic Result

HOJBC0_2018_v22n11_1462_t0001.png 이미지

References

  1. W. R. Lee, D. G. Hwang, and B. M. Jun, "Color Restoration of Low Light Level Images Using a Tone Mapping and Retinex," Journal of Koreanstudies Information Service System: Software and Applications, vol. 40, no. 6, pp. 342-349, Jun. 2013.
  2. W. R. Lee, W. G. Jun, and B. M. Jun, "Performance Evaluation of Color Constancy Methods using Low Light Level synthesized images," in Proceedings of the Korea Entertainment Industry Association, pp. 214-217, 2013.
  3. K. G. Kim, J. W. Chong, "Contrast Enhancement of Image in Low-Light Condition Based on the Ratio of Illumination," in proceedings of the Institute of Electronics and Information Engineers, pp. 565-568, 2013.
  4. G. J. Kim, and B. S. Kang, "Hardware Design of Real-Time Wide Dynamic Range Algorithm Based on Tone Mapping Method for Image Quality Enhancement," Journal of the Korea Institute of Information and Communication Engineering, vol. 22, no. 2, pp. 270-275, Feb. 2018. https://doi.org/10.6109/JKIICE.2018.22.2.270
  5. T. U. Kang, W. J. Lee, and J. C. Jeong, "HDR Tone Mapping using Separated Histogram Equalization," in proceedings of the Korean Institute of Broadcast and Media Engineers, pp. 189-192, 2015.
  6. D. H. Kim, "SVD-based Image Enhancement Method using Weighted Average of Histogram Stretching and Equalization," Asia-pacific Journal of Multimedia Services Convergent with Art, Humanities, and Sociology, vol. 5, no. 5, pp. 77-85, Oct. 2015.
  7. H. J. Kwon, S. H. Lee, G. Y. Lee, and K. I. Song, "HDR Reproduction Algorithm Using the Radiance Map Considering Weighted Intensities of Multi Exposure Images," The Journal of Korean Institute of Information Technology, vol. 13, no. 10, pp. 137-144, Oct. 2015. https://doi.org/10.14801/jkiit.2015.13.10.137
  8. T. Jinno, and M. Okuda, "Multiple Exposure Fusion for High Dynamic Range Image Acquisition," The Journal of Institute of Electrical and Electronics Engineers Transactions on Image Processing, vol. 21, no. 1, pp. 358-365, Jun. 2011.
  9. H. S. Cho, G. J. Kim, K. H. Jang, S. M. Lee, and B. S. Kang, "Color Image Enhancement Based on Adaptive Nonlinear Curves of Luminance Feature," The Journal of Semiconductor Technology and Science, vol. 15, no. 1, pp. 60-67, Feb. 2015. https://doi.org/10.5573/JSTS.2015.15.1.060