Browse > Article
http://dx.doi.org/10.5909/JBE.2021.26.4.419

Shadow Removal based on the Deep Neural Network Using Self Attention Distillation  

Kim, Jinhee (Department of Electrical and Electronics Engineering, Konkuk University)
Kim, Wonjun (Department of Electrical and Electronics Engineering, Konkuk University)
Publication Information
Journal of Broadcast Engineering / v.26, no.4, 2021 , pp. 419-428 More about this Journal
Abstract
Shadow removal plays a key role for the pre-processing of image processing techniques such as object tracking and detection. With the advances of image recognition based on deep convolution neural networks, researches for shadow removal have been actively conducted. In this paper, we propose a novel method for shadow removal, which utilizes self attention distillation to extract semantic features. The proposed method gradually refines results of shadow detection, which are extracted from each layer of the proposed network, via top-down distillation. Specifically, the training procedure can be efficiently performed by learning the contextual information for shadow removal without shadow masks. Experimental results on various datasets show the effectiveness of the proposed method for shadow removal under real world environments.
Keywords
shadow removal; shadow detection; convolutional neural networks; self attention distillation;
Citations & Related Records
연도 인용수 순위
  • Reference
1 X. Hu, C. Fu, L. Zhu, J. Qin, and P. Heng, "Direction-aware spatial context features for shadow detection and removal," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 42, no. 11, pp. 2795-2808, May. 2019.   DOI
2 A. Krizhevsky, I. Sutskever, and G. E. Hinton, "ImageNet classification with deep convolutional neural networks," in Proc. Advances in Neural Information Processing Systems, Dec. 2017, pp. 1097-1105.
3 S. Xie, R. Girshick, P. Dollar, Z. Tu, and K. He, "Aggregated residual transformations for deep neural networks," in Proc. IEEE Conference on Computer Vision and Pattern Recognition, Jun. 2017, pp. 1493-1500.
4 K. Simonyan and A. Zisserman, "Very deep convolutional networks for large-scale image recognition," in Proc. Int. Conf. Learn. Representation, May 15, 2015, pp. 1-14.
5 Y. Zhang, Y. Tian, Y. Kong, B. Zhong, and Y. Fu, "Residual dense network for image super-resolution," in Proc. IEEE Conference on Computer Vision and Pattern Recognition, Jun. 2018, pp. 2472-2481.
6 L. Qu, J. Tian, S. He, Y. Tang, and R. W. Lau, "DeshadowNet: A multi-context embedding deep network for shadow removal," in Proc. IEEE Conference on Computer Vision and Pattern Recognition, Jun. 2017, pp. 4067-4075.
7 H. Gong and D. P. Cosker, "Interactive shadow removal and ground truth for variable scene categories," in Proc. British Machine Vision Conference, Sep. 2014, pp. 1-11.
8 A. Paszke, S. Gross, S. Chintala, G. Chanan, E. Yang, Z. Devito, Lin, A. Desmaison, L. Antiga, A. Lerer, "Automatic differentiation in pytorch". in Proc. Conference and Workshop on Neural Information Processing Systems (NIPS), pp. 1-4, 2017.
9 G. Hinton, O. Vinyals, and J. Dean, "Distilling the knowledge in a neural network," in Proc. Conference and Workshop on Neural Information Processing Systems (NIPS), Dec. 2015, pp. 1-9.
10 F. Wang, M. Jiang, C. Qian, S. Yang, C. Li, H. Zhang, X. Wang, "Residual attention network for image classification," in Proc. IEEE Conference on Computer Vision and Pattern Recognition, Jun. 2017, pp. 3156-3164.
11 S. Zagoruyko and N. Komodakis, "Paying more attention to attention: improving the performance of convolutional neural networks via attention transfer," in Proc. International Conference on Learning Representations, Sep. 2017, pp. 1-13.
12 J. Wang, X. Li, and J. Yang, "Stacked conditional generative adversarial networks for jointly learning shadow detection and shadow removal," in Proc. IEEE Conference on Computer Vision and Pattern Recognition, Jun. 2018, pp. 1788-1797.
13 G. Finlayson, S. D. Hordley, C. Lu, and M. S. Drew, "On the removal of shadows from images," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 28, no. 1, pp. 59-68, Nov. 2006.   DOI
14 R. Guo, Q. Dai, and D. Hoiem, "Paired regions for shadow detection and removal," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 35, no. 12, pp. 2956-2967, Oct. 2013.   DOI