Browse > Article
http://dx.doi.org/10.5909/JBE.2018.23.3.369

HVS-Aware Single-Shot HDR Imaging Using Deep Convolutional Neural Network  

Vien, An Gia (Pukyong National University, Department of Computer Engineering)
Lee, Chul (Pukyong National University, Department of Computer Engineering)
Publication Information
Journal of Broadcast Engineering / v.23, no.3, 2018 , pp. 369-382 More about this Journal
Abstract
We propose a single-shot high dynamic range (HDR) imaging algorithm using a deep convolutional neural network (CNN) for row-wise varying exposures in a single image. The proposed algorithm restores missing information resulting from under- and/or over-exposed pixels in an input image and reconstructs the raw radiance map. The main contribution of this work is the development of a loss function for the CNN employing the human visual system (HVS) properties. Then, the HDR image is obtained by applying a demosaicing algorithm. Experimental results demonstrate that the proposed algorithm provides higher-quality HDR images than conventional algorithms.
Keywords
High dynamic range (HDR) imaging; spatially varying exposure (SVE) imaging; convolutional neural network (CNN);
Citations & Related Records
연도 인용수 순위
  • Reference
1 R. Mantiuk, K. Myszkowski, and H. P. Seidel, "Lossy Compression of High Dynamic Range Images and Video," Proceeding of SPIE, Vol. 6057, Feb. 2006.
2 O. Blackwell and H. Blackwell, “Visual Performance Data for 156 Normal Observers of Various Ages,” Journal of the Illuminating Engineering Society, Vol. 1, No. 1, pp. 3-13, 1971.   DOI
3 J. E. Adams, "Design of Practical Color Filter Array Interpolation Algorithms for Digital Cameras," Proceeding of SPIE, Vol. 3028, pp. 117-125, Feb. 1997.
4 T. O. Aydin, R. Mantiuk, and H. P. Seidel, "Extending Quality Metrics to Full Luminance Range Images," Proceeding of SPIE, Vol. 6806, p. 68060B, Mar. 2008.
5 M. Narwaria, R. K. Mantiuk, M. P. D. Silva, and P. L. Callet, "HDR-VDP-2.2: A Calibrated Method for Objective Quality Prediction of High Dynamic Range and Standard Images," Journal of Electronic Imaging, Vol. 24, No. 1, Jan. 2015.
6 C. Lee and E. Y. Lam, “Computationally Efficient Truncated Nuclear Norm Minimization for High Dynamic Range Imaging,” IEEE Transactions on Image Processing, Vol. 25, No. 9, pp. 4145-4157, Sep. 2016.   DOI
7 S. K. Nayar and T. Mitsunaga, "High Dynamic Range Imaging: Spatially Varying Pixel Exposures," Proceeding of IEEE Conference on Computer Vision and Pattern Recognition, Vol. 1, pp. 472-479, Jun. 2000.
8 J. Gu, Y. Hitomi, T. Mitsunaga, and S. Nayar, "Coded Rolling Shutter Photography: Flexible Space-Time Sampling," Proceeding of IEEE International Conference on Computational Photography, pp. 1-8, Mar. 2010.
9 K. Hirakawa and P. M. Simon, "Single-Shot High Dynamic Range Imaging with Conventional Camera Hardware," Proceeding of IEEE International Conference on Computer Vision, pp. 1339-1346, Nov. 2011.
10 P. Sen and C. Aguerrebere, "Practical High Dynamic Range Imaging of Everyday Scenes: Photographing The World as We Can See It with Our Own Eyes," IEEE Signal Processing Magazine, Vol. 33, pp. 36-44, Sep. 2016.
11 P. E. Debevec and J. Malik, "Recovering High Dynamic Range Radiance Maps from Photographs," Proceeding of ACM SIGGRAPH, Aug. pp. 369-378, 1997.
12 T. Mitsunaga and S. K. Nayar, "Radiometric Self-Calibration," Proceeding of IEEE Conference on Computer Vision and Pattern Recognition, pp.374-380, Jun. 1999.
13 M. Aggarwal and N. Ahuja, "Split Aperture Imaging for High Dynamic Range," Proceeding of IEEE International Conference on Computer Vision, Vol. 2, pp. 10-17, Jul. 2001.
14 H. Zimmer, A. Bruhn, and J. Weickert, "Freehand HDR Imaging of Moving Scenes with Simultaneous Resolution Enhancement," Computer Graphics Forum, Vol. 30, pp. 405-414, Apr. 2011.   DOI
15 P. Sen, N. K. Kalantari, M. Yaesoubi, S. Darabi, D. B. Goldman, and E. Shechtman, "Robust Patch-Based HDR Reconstruction of Dynamic Scenes," ACM Transactions on Graphics, Vol. 31, No. 6, pp. 203:1-203:11, Nov. 2012.
16 C. Lee, Y. Li, and V. Monga, "Ghost-free High Dynamic Range Imaging via Rank Minimization," IEEE Signal Processing Letters, Vol. 21, No.9, pp. 1045-1049, Sep. 2014.   DOI
17 V. G. An and C. Lee, "Single-Shot High Dynamic Range Imaging via Deep Convolutional Neural Network," Proceeding of APSIPA ASC, pp. 1768-1772, Dec. 2017.
18 H. Cho, S. J. Kim, and S. Lee, “Single-Shot High Dynamic Range Imaging Using Coded Electronic Shutter,” Computer Graphics Forum, Vol. 33, No. 7, pp. 329-338, Oct. 2014.   DOI
19 A. Serrano, F. Heide, D. Gutierrez, G. Wetzstein, and B. Masia, “Convolutional Sparse Coding for High Dynamic Range Imaging,” Computer Graphics Forum, Vol. 35, No. 2, pp. 153-163, May 2016.   DOI
20 J. Li, C. Bai, Z. Lin, and J. Yun, "Penrose High-Dynamic-Range Imaging," Journal of Electronic Imaging, Vol. 25, No. 3, pp. 033 024:1-13, Jun. 2016.
21 S. Ioffe and C. Szegedy, "Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift," Proceeding of International Conference on Machine Learning, pp. 448-456, Jul. 2015.
22 V. Nair and G. E. Hilton, "Rectified Linear Units Improve Restricted Boltzman Machines," Proceeding of International Conference on Machine Learning, pp. 807-814, Jun. 2010.
23 J. Johson, A. Alahi, and F. F. Li, "Perceptual Losses for Real-Time Style Transfer and Super-Resolution," Proceeding of European Conference on Computer Vision, pp. 694-711, Oct. 2016.
24 N. K. Kalantari and R. Ramamoorthi, "Deep High Dynamic Range Imaging of Dynamic Scenes," ACM Transactions on Graphics, Vol. 36, No. 4, Jul. 2017.
25 G. Eilertsen, J. Kronander, G. Denes, R. K. Mantiuk, and J. Unger, "HDR Image Reconstruction from a Single Exposure Using Deep CNNs," ACM Transactions on Graphics, Vol. 36, No. 6, Nov. 2017.
26 C. Aguerrebere, A. Almansa, Y. Gousseau, J. Delon, and P. Muse, "Single Shot High Dynamic Range Imaging Using Piecewise Linear Estimators," Proceeding of IEEE International Conference on Computational Photography, pp. 1-10, May 2014.
27 Y. Li, C. Lee, and V. Monga, “A Maximum a Posteriori Estimation Framework for Robust High Dynamic Range Video Synthesis,” IEEE Transactions on Image Processing, Vol. 26, No. 3, pp. 1143-1157, Mar. 2017.   DOI