Browse > Article
http://dx.doi.org/10.3745/JIPS.02.0162

Evaluation of Artificial Intelligence-Based Denoising Methods for Global Illumination  

Faradounbeh, Soroor Malekmohammadi (Dept. of Computer Engineering, Keimyung University)
Kim, SeongKi (Division of SW Convergence, Sangmyung University)
Publication Information
Journal of Information Processing Systems / v.17, no.4, 2021 , pp. 737-753 More about this Journal
Abstract
As the demand for high-quality rendering for mixed reality, videogame, and simulation has increased, global illumination has been actively researched. Monte Carlo path tracing can realize global illumination and produce photorealistic scenes that include critical effects such as color bleeding, caustics, multiple light, and shadows. If the sampling rate is insufficient, however, the rendered results have a large amount of noise. The most successful approach to eliminating or reducing Monte Carlo noise uses a feature-based filter. It exploits the scene characteristics such as a position within a world coordinate and a shading normal. In general, the techniques are based on the denoised pixel or sample and are computationally expensive. However, the main challenge for all of them is to find the appropriate weights for every feature while preserving the details of the scene. In this paper, we compare the recent algorithms for removing Monte Carlo noise in terms of their performance and quality. We also describe their advantages and disadvantages. As far as we know, this study is the first in the world to compare the artificial intelligence-based denoising methods for Monte Carlo rendering.
Keywords
Denoising; Filtering; Global Illumination; Monte Carlo Noise; Noise Removal;
Citations & Related Records
연도 인용수 순위
  • Reference
1 P. Sen and S. Darabi, "On filtering the noise from the random parameters in Monte Carlo rendering," ACM Transactions on Graphics, vol. 31, no. 3, article no. 18, 2012. https://doi.org/10.1145/2167076.2167083   DOI
2 K. Dabov, A. Foi, V. Katkovnik, and K. Egiazarian, "Image denoising by sparse 3-D transform-domain collaborative filtering," IEEE Transactions on Image Processing, vol. 16, no. 8, pp. 2080-2095, 2007.   DOI
3 L. Fan, F. Zhang, H. Fan, and C. Zhang, "Brief review of image denoising techniques," Visual Computing for Industry, Biomedicine, and Art, vol. 2, article no. 7, 2019. https://doi.org/10.1186/s42492-019-0016-7   DOI
4 N. K. Kalantari, S. Bako, and P. Sen, "A machine learning approach for filtering Monte Carlo noise," ACM Transactions on Graphics, vo. 34, no. 4, article no. 122, 2015. https://doi.org/10.1145/2766977   DOI
5 N. K. Kalantari, "Utilizing machine learning for filtering general Monte Carlo noise," Ph.D. dissertation, University of California, Santa Barbara, CA, 2015.
6 S. Bako, T. Vogels, B. McWilliams, M. Meyer, J. Novak, A. Harvill, P. Sen, T. Derose, and F. Rousselle, "Kernel-predicting convolutional networks for denoising Monte Carlo renderings," ACM Transactions on Graphics, vol. 36, no. 4, article no. 97, 2017. https://doi.org/10.1145/3072959.3073708   DOI
7 C. Schied, A. Kaplanyan, C. Wyman, A. Patney, C. R. A. Chaitanya, J. Burgess, S. Liu, C. Dachsbacher, A. Lefohn, and M. Salvi, "Spatiotemporal variance-guided filtering: real-time reconstruction for path-traced global illumination," in Proceedings of High Performance Graphics, Los Angeles, CA, 2017, pp. 1-12.
8 Y. Liu, C. Zheng, Q. Zheng, and H. Yuan, "Removing Monte Carlo noise using a Sobel operator and a guided image filter," The Visual Computer, vol. 34, no. 4, pp. 589-601, 2018.   DOI
9 K. P. Kumar, "Fuzzy-based machine learning algorithm for intelligent systems," in Data Management, Analytics and Innovation. Singapore: Springer, 2019, pp. 321-339
10 N. Chauhan and B. J. Choi, "Denoising approaches using fuzzy logic and convolutional autoencoders for human brain MRI image," International Journal of Fuzzy Logic and Intelligent Systems, vol. 19, no. 3, pp. 135-139, 2019.   DOI
11 B. Costa and J. Jain, "Fuzzy deep stack of autoencoders for dealing with data uncertainty," in Proceedings of 2019 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE), New Orleans, LA, 2019, pp. 1-6.
12 J. Lehtinen, T. Aila, J. Chen, S. Laine, and F. Durand, "Temporal light field reconstruction for rendering distribution effects," in Proceedings of ACM SIGGRAPH 2011 papers, Vancouver, Canada, 2011, pp. 1-12.
13 J. Lehtinen, T. Aila, S. Laine, and F. Durand, "Reconstructing the indirect light field for global illumination," ACM Transactions on Graphics, vol. 31, no. 4, article no. 51, 2012. https://doi.org/10.1145/2185520.2185547   DOI
14 T, Hachisuka, W. Jarosz, R. P. Weistroffer, K. Dale, G. Humphreys, M. Zwicker, and H. W. Jensen, "Multidimensional adaptive sampling and reconstruction for ray tracing," in Proceedings of ACM SIGGRAPH 2008 papers, Los Angeles, CA, 2008, pp. 1-10.
15 Q. Zhang, Y. Li, F. Al-Turjman, X. Zhou, and X. Yang, "Transient ischemic attack analysis through noncontact approaches," Human-centric Computing and Information Sciences, vol. 10, article no. 16, 2020. https://doi.org/10.1186/s13673-020-00223-z   DOI
16 P. Bauszat, M. Eisemann, S. John, and M. Magnor, "Sample-based manifold filtering for interactive global illumination and depth of field," Computer Graphics Forum, vol. 34, no. 1, pp. 265-276, 2015.
17 B. Bitterli, F. Rousselle, B. Moon, J. A. Iglesias-Guitian, D. Adler, K. Mitchell, W. Jarosz, and J. Novak, "Nonlinearly weighted first-order regression for denoising Monte Carlo renderings," Computer Graphics Forum, vol. 35, no. 4, pp. 107-117, 2016.
18 M. Gharbi, T. M. Li, M. Aittala, J. Lehtinen, and F. Durand, "Sample-based Monte Carlo denoising using a kernel-splatting network," ACM Transactions on Graphics, vol. 38, no. 4, article no. 125, 2019. https://doi.org/10.1145/3306346.3322954   DOI
19 J. T. Kajiya, "The rendering equation," in Proceedings of the 13th Annual Conference on Computer Graphics And Interactive Techniques, Dallas, TX, 1986, pp. 143-150.
20 E. Veach, "Robust Monte Carlo methods for light transport simulation," Ph.D. dissertation, Stanford University, Stanford, CA, 1997.
21 B. Cantrell and N. Yates, Modeling the Environment: Techniques and Tools for the 3D Illustration of Dynamic Landscapes. Hoboken, NJ: John Wiley & Sons, 2012.
22 C. Schied, C. Peters, and C. Dachsbacher, "Gradient estimation for real-time adaptive temporal filtering," Proceedings of the ACM on Computer Graphics and Interactive Techniques, vol. 1, no. 2, article no. 24, 2018. https://doi.org/10.1145/3233301   DOI
23 X. Meng, Q. Zheng, V. Varshney, G. Singh, and M. Zwicker, "Real-time Monte Carlo denoising with the neural bilateral grid," in Proceedings of the 31st Eurographics Symposium on Rendering (EGSR): Digital Library Only Track, London, UK, 2020, pp. 13-24.
24 Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, "Image quality assessment: from error visibility to structural similarity," IEEE Transactions on Image Processing, vol. 13, no. 4, pp. 600-612, 2004.   DOI
25 F. Rousselle, C. Knaus, and M. Zwicker, "Adaptive sampling and reconstruction using greedy error minimization," ACM Transactions on Graphics, vol. 30, no. 6, pp. 1-12, 2011.
26 N. K. Kalantari and P. Sen, "Removing the noise in Monte Carlo rendering with general image denoising algorithms," Computer Graphics Forum, vol. 32, no. 2pt1, pp. 93-102, 2013.
27 C. R. A. Chaitanya, A. S. Kaplanyan, C. Schied, M. Salvi, A. Lefohn, D. Nowrouzezahrai, and T. Aila, "Interactive reconstruction of Monte Carlo image sequences using a recurrent denoising autoencoder," ACM Transactions on Graphics, vol. 36, no. 4, article no. 98, 2017. https://doi.org/10.1145/3072959.3073601   DOI
28 P. Gallinari, Y. Lecun, S. Thiria, and F. F. Soulie, "Distributed associative memories: a comparison," in Proceedings of COGNITIVA, Paris, La Villette, 1987.
29 D. G. Mixon and S. Villar, "Sunlayer: stable denoising with generative networks," 2018 [Online]. Available: https://arxiv.org/abs/1803.09319.
30 T. Vogels, F. Rousselle, B. McWilliams, G. Rothlin, A. Harvill, D. Adler, M. Meyer, and J. Novak, "Denoising with kernel prediction and asymmetric loss functions," ACM Transactions on Graphics, vol. 37, no. 4, article no. 124, 2018. https://doi.org/10.1145/3197517.3201388   DOI
31 H. W. Jensen, "Global illumination using photon maps," in Rendering Techniques '96. Vienna, Austria: Springer, 1996, pp. 21-30.
32 H. Dahlberg, D. Adler, and J. Newlin, "Machine-learning denoising in feature film production," in Proceedings of ACM SIGGRAPH 2019 Talks, Los Angeles, CA, 2019.
33 S. Laine, T. Karras, J. Lehtinen, and T. Aila, "High-quality self-supervised deep image denoising," Advances in Neural Information Processing Systems, vol. 32, pp. 6970-6980, 2019.
34 A. Keller, L. Fascione, M. Fajardo, I. Georgiev, P. Christensen, J. Hanika, C. Eisenacher, and G. Nichols, "The path tracing revolution in the movie industry," in Proceedings of ACM SIGGRAPH 2015 Courses, Los Angeles, CA, 2015.
35 A. Alsaiari, R. Rustagi, M. M. Thomas, and A. G. Forbes, "Image denoising using a generative adversarial network," in Proceedings of 2019 IEEE 2nd International Conference on Information and Computer Technologies (ICICT), Kahului, HI, 2019, pp. 126-132.
36 S. Agarwal, A. Agarwal, and M. Deshmukh, "Denoising images with varying noises using autoencoders," in Computer Vision and Image Processing. Singapore: Springer, 2019, pp. 3-14.
37 H. Dai and L. Shao, "Pointae: point auto-encoder for 3d statistical shape and texture modelling," in Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, Korea, 2019, pp. 5410-5419.
38 T. Chen, J. Shi, J. Yang, and G. Li, "Enhancing network cluster synchronization capability based on artificial immune algorithm," Human-centric Computing and Information Sciences, vol. 9, article no. 3, 2019. https://doi.org/10.1186/s13673-019-0164-y   DOI
39 Z. Zeng, L. Wang, B. B. Wang, C. M. Kang, and Y. N. Xu, "Denoising stochastic progressive photon mapping renderings using a multi-residual network," Journal of Computer Science and Technology, vol. 35, pp. 506-521, 2020.   DOI
40 K. Vardis, "Efficient illumination algorithms for global illumination in interactive and real-time rendering," Ph.D. dissertation, Athens University of Economics & Business, Greece, 2016.
41 P. Dutre, K. Bala, and P. Bekaert, Advanced Global Illumination. Boca Raton, FL: AK Peters/CRC Press, 2006.
42 M. Mara, M. McGuire, B. Bitterli, and W. Jarosz, "An efficient denoising algorithm for global illumination," in Proceedings of High Performance Graphics, Los Angeles, CA, 2017.
43 Y. Zhang, H. Wang, and X. Fan, "Algorithm for detection of fire smoke in a video based on wavelet energy slope fitting," Journal of Information Processing Systems, vol. 16, no. 3, pp. 557-571, 2020.   DOI
44 A. Buades, B. Coll, and J. M. Morel, "A non-local algorithm for image denoising," in Proceedings of 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), San Diego, CA, 2005, pp. 60-65.
45 J. Lv and X. Luo, "Image denoising via fast and fuzzy non-local means algorithm," Journal of Information Processing Systems, vol. 15, no. 5, pp. 1108-1118, 2019.   DOI
46 Z. Wang, X. Huang, and F. Huang, "A new image enhancement algorithm based on bidirectional diffusion," Journal of Information Processing Systems, vol. 16, no. 1, pp. 49-60, 2020.   DOI
47 R. Wan, L. Ding, N. Xiong, W. Shu, and L. Yang, "Dynamic dual threshold cooperative spectrum sensing for cognitive radio under noise power uncertainty," Human-centric Computing and Information Sciences, vol. 9, article no. 22, 2019. https://doi.org/10.1186/s13673-019-0181-x   DOI
48 D. Vicini, D. Adler, J. Novak, F. Rousselle, and B. Burley, "Denoising deep Monte Carlo renderings," Computer Graphics Forum, vol. 38, no. 1, pp. 316-327, 2019.   DOI
49 H. Park, B. Moon, S. Kim, and S., E. Yoon, "P-RPF: pixel-based random parameter filtering for Monte Carlo rendering," in Proceedings of 2013 International Conference on Computer-Aided Design and Computer Graphics, Guangzhou, China, 2013, pp. 123-130.
50 J. Xie, L. Xu, and E. Chen, "Image denoising and inpainting with deep neural networks," Advances in Neural Information Processing Systems, vol. 25, pp. 341-349, 2012.
51 B. Bayar and M. C. Stamm, "A deep learning approach to universal image manipulation detection using a new convolutional layer," in Proceedings of the 4th ACM Workshop on Information Hiding and Multimedia Security, Vigo Galicia, Spain, 2016, pp. 5-10.
52 M. Kettunen, E. Harkonen, and J. Lehtinen, "Deep convolutional reconstruction for gradient-domain rendering," ACM Transactions on Graphics, vol. 38, no. 4, pp. 126, 2019. https://doi.org/10.1145/3306346.3323038   DOI
53 K. M. Wong and T. T. Wong, "Deep residual learning for denoising Monte Carlo renderings," Computational Visual Media, vol. 5, no. 3, pp. 239-255, 2019.   DOI
54 B. Bitterli, C. Wyman, M. Pharr, P. Shirley, A. Lefohn, and W. Jarosz, "Spatiotemporal reservoir resampling for real-time ray tracing with dynamic direct lighting," ACM Transactions on Graphics, vol. 39, no. 4, article no. 148, 2020. https://doi.org/10.1145/3386569.3392481   DOI
55 K. Zhang, W. Zuo, Y. Chen, D. Meng, and L. Zhang, "Beyond a Gaussian denoiser: residual learning of deep CNN for image denoising," IEEE Transactions on Image Processing, vol. 26, no. 7, pp. 3142-3155, 2017.   DOI
56 J. Talbot, D. Cline, and P. K. Egbert, "Importance resampling for global illumination," in Proceedings of the Eurographics Symposium on Rendering Techniques, Konstanz, Germany, 2005, pp. 139-146.
57 X. Yang, D. Wang, W. Hu, L. J. Zhao, B. C. Yin, Q. Zhang, X. P. Wei, and H. Fu, "DEMC: a deep dual-encoder network for denoising Monte Carlo rendering," Journal of Computer Science and Technology, vol. 34, no. 5, pp. 1123-1135, 2019.   DOI
58 M. Pharr, W. Jakob, and G. Humphreys, Physically Based Rendering: From Theory to Implementation, 3rd ed. Cambridge, MA: Morgan Kaufmann, 2014.