Browse > Article
http://dx.doi.org/10.9708/jksci.2021.26.03.051

Artificial Neural Network Method Based on Convolution to Efficiently Extract the DoF Embodied in Images  

Kim, Jong-Hyun (School of Software Application, Kangnam University)
Publication Information
Journal of the Korea Society of Computer and Information / v.26, no.3, 2021 , pp. 51-57 More about this Journal
Abstract
In this paper, we propose a method to find the DoF(Depth of field) that is blurred in an image by focusing and out-focusing the camera through a efficient convolutional neural network. Our approach uses the RGB channel-based cross-correlation filter to efficiently classify the DoF region from the image and build data for learning in the convolutional neural network. A data pair of the training data is established between the image and the DoF weighted map. Data used for learning uses DoF weight maps extracted by cross-correlation filters, and uses the result of applying the smoothing process to increase the convergence rate in the network learning stage. The DoF weighted image obtained as the test result stably finds the DoF region in the input image. As a result, the proposed method can be used in various places such as NPR(Non-photorealistic rendering) rendering and object detection by using the DoF area as the user's ROI(Region of interest).
Keywords
Artificial neural network; Convolutional neural network; Depth of field; Image processing; Region of interest; Object detection;
Citations & Related Records
연도 인용수 순위
  • Reference
1 Hane, Christian, Lionel Heng, Gim Hee Lee, Friedrich Fraundorfer, Paul Furgale, Torsten Sattler, and Marc Pollefeys. "3D visual perception for self-driving cars using a multi-camera system: Calibration, mapping, localization, and obstacle detection." Image and Vision Computing, pp. 14-27, 2017.
2 Muller, K., Merkle, P. and Wiegand, T., "3-D video representation using depth maps". Proceedings of the IEEE, 99(4), pp.643-656, 2010.
3 Hane, Christian, Christopher Zach, Jongwoo Lim, Ananth Ranganathan, and Marc Pollefeys. "Stereo depth map fusion for robot navigation." IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 1618-1625, 2011.
4 Hadsell, Raia and Sermanet, Pierre and Ben, Jan and Erkan, Ayse and Scoffier, Marco and Kavukcuoglu, Koray and Muller, Urs and LeCun, Yann, Learning long-range vision for autonomous off-road driving. Journal of Field Robotics, pp. 120-144, 2009.   DOI
5 Rafiee, Gholamreza and Dlay, Satnam Singh and Woo, Wai Lok, Region-of-interest extraction in low depth of field images using ensemble clustering and difference of Gaussian approaches. Pattern Recognition, pp. 2685-2699, 2013.
6 Park, Jungwoo and Kim, Changick, Extracting focused object from low depth-of-field image sequences. Visual Communications and Image Processing, pp. 60771O, 2006.
7 Simonyan, Karen and Zisserman, Andrew, Data-driven estimation of cloth simulation models. arXiv preprint arXiv:1409.1556, 2014.
8 Eigen, David, Christian Puhrsch, and Rob Fergus. "Depth map prediction from a single image using a multi-scale deep network." In Advances in neural information processing systems, pp. 2366-2374. 2014.
9 Laina, Iro, Christian Rupprecht, Vasileios Belagiannis, Federico Tombari, and Nassir Navab. "Deeper depth prediction with fully convolutional residual networks." In 2016 Fourth international conference on 3D vision (3DV), pp. 239-248. IEEE, 2016.
10 Liu, Fayao, Chunhua Shen, and Guosheng Lin. "Deep convolutional neural fields for depth estimation from a single image." In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 5162-5170. 2015.
11 Geiger, Andreas, Philip Lenz, and Raquel Urtasun. "Are we ready for autonomous driving? the kitti vision benchmark suite." In 2012 IEEE Conference on Computer Vision and Pattern Recognition, pp. 3354-3361, 2012.
12 Gargallo, Pau, and Peter Sturm. "Bayesian 3D modeling from images using multiple depth maps." In 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05), vol. 2, pp. 885-891. IEEE, 2005.
13 Silberman, Nathan, Derek Hoiem, Pushmeet Kohli, and Rob Fergus. "Indoor segmentation and support inference from rgbd images." In European conference on computer vision, pp. 746-760. Springer, Berlin, Heidelberg, 2012.
14 Li, Zhengqi, and Noah Snavely. "Megadepth: Learning single-view depth prediction from internet photos." In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2041-2050. 2018.
15 Srinivasan, Pratul P., Rahul Garg, Neal Wadhwa, Ren Ng, and Jonathan T. Barron. "Aperture supervision for monocular depth estimation." In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6393-6401. 2018.
16 Suwajanakorn, Supasorn, Carlos Hernandez, and Steven M. Seitz. "Depth from focus with your mobile phone." In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3497-3506. 2015.
17 Barron, Jonathan T., Andrew Adams, YiChang Shih, and Carlos Hernandez. "Fast bilateral-space stereo for synthetic defocus." In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4466-4474. 2015.
18 Ha, Hyowon, Sunghoon Im, Jaesik Park, Hae-Gon Jeon, and In So Kweon. "High-quality depth from uncalibrated small motion clip." In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5413-5421. 2016.
19 Joshi, Neel, and C. Lawrence Zitnick. "Micro-baseline stereo." Technical Report MSR-TR-2014-73, p. 8, 2014.
20 Klose, Felix, Oliver Wang, Jean-Charles Bazin, Marcus Magnor, and Alexander Sorkine-Hornung. "Sampling based scene-space video processing." ACM Transactions on Graphics (TOG) vol, 34, no. 4, pp. 67, 2015.
21 Haeberli, Paul, and Kurt Akeley. "The accumulation buffer: hardware support for high-quality rendering." ACM SIGGRAPH computer graphics vol. 24, no. 4, pp. 309-318, 1990.   DOI
22 Yang, Yang, Haiting Lin, Zhan Yu, Sylvain Paris, and Jingyi Yu. "Virtual dslr: High quality dynamic depth-of-field synthesis on mobile platforms." Electronic Imaging, no. 18, pp. 1-9, 2016.   DOI
23 Lee, Sungkil, Elmar Eisemann, and Hans-Peter Seidel. "Real-time lens blur effects and focus control." ACM Transactions on Graphics (TOG) vol. 29, no. 4, pp. 1-7, 2010.
24 Kraus, Martin, and Magnus Strengert. "Depth-of-field rendering by pyramidal image processing." In Computer graphics forum, vol. 26, no. 3, pp. 645-654. Oxford, UK: Blackwell Publishing Ltd, 2007.   DOI
25 Lee, Sungkil, Gerard Jounghyun Kim, and Seungmoon Choi. "Real-time depth-of-field rendering using anisotropically filtered mipmap interpolation." IEEE Transactions on Visualization and Computer Graphics vol. 15, no. 3, pp. 453-464, 2009.   DOI
26 Dong, Chao and Loy, Chen Change and He, Kaiming and Tang, Xiaoou, Image super-resolution using deep convolutional networks. IEEE transactions on pattern analysis and machine intelligence, pp. 295-307, 2015.