Journal of Broadcast Engineering (방송공학회논문지)

The Korean Institute of Broadcast and Media Engineers (한국방송∙미디어공학회)
권호 표지
DOI QR코드

DOI QR Code

Shadow Removal based on the Deep Neural Network Using Self Attention Distillation

자기 주의 증류를 이용한 심층 신경망 기반의 그림자 제거

  • Kim, Jinhee (Department of Electrical and Electronics Engineering, Konkuk University) ;
  • Kim, Wonjun (Department of Electrical and Electronics Engineering, Konkuk University)
  • 김진희 (건국대학교 전기전자공학부) ;
  • 김원준 (건국대학교 전기전자공학부)
  • Received : 2021.04.28
  • Accepted : 2021.07.09
  • Published : 2021.07.30
Download PDF
⟨ Previous Next ⟩

Abstract

Shadow removal plays a key role for the pre-processing of image processing techniques such as object tracking and detection. With the advances of image recognition based on deep convolution neural networks, researches for shadow removal have been actively conducted. In this paper, we propose a novel method for shadow removal, which utilizes self attention distillation to extract semantic features. The proposed method gradually refines results of shadow detection, which are extracted from each layer of the proposed network, via top-down distillation. Specifically, the training procedure can be efficiently performed by learning the contextual information for shadow removal without shadow masks. Experimental results on various datasets show the effectiveness of the proposed method for shadow removal under real world environments.

그림자 제거는 객체 추적 및 검출 등 영상처리 기술의 핵심 전처리 요소이다. 최근 심층 합성곱 신경망 (Deep Convolutional Neural Network) 기반의 영상 인식 기술이 발전함에 따라 심층 학습을 이용한 그림자 제거 연구들이 활발히 진행되고 있다. 본 논문에서는 자기 주의 증류(Self Attention Distillation)를 이용하여 심층 특징을 추출하는 새로운 그림자 제거 방법을 제안한다. 제안된 방법은 각 층에서 추출된 그림자 검출 결과를 하향식 증류를 통해 점진적으로 정제한다. 특히, 그림자 검출 결과에 대한 정답을 이용하지 않고 그림자 제거를 위한 문맥적 정보를 형성함으로써 효율적인 심층 신경망 학습을 수행한다. 그림자 제거를 위한 다양한 데이터 셋에 대한 실험 결과를 통해 제안하는 방법이 실제 환경에서 발생한 그림자 제거에 효과적임을 보인다.

Keywords

Acknowledgement

이 논문은 정부(과학기술정보통신부)의 재원으로 한국연구재단의 지원을 받아 수행된 연구임 (No. 2020R1F1A1068080).

References

  1. G. Finlayson, S. D. Hordley, C. Lu, and M. S. Drew, "On the removal of shadows from images," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 28, no. 1, pp. 59-68, Nov. 2006. https://doi.org/10.1109/TPAMI.2006.18
  2. R. Guo, Q. Dai, and D. Hoiem, "Paired regions for shadow detection and removal," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 35, no. 12, pp. 2956-2967, Oct. 2013. https://doi.org/10.1109/TPAMI.2012.214
  3. J. Wang, X. Li, and J. Yang, "Stacked conditional generative adversarial networks for jointly learning shadow detection and shadow removal," in Proc. IEEE Conference on Computer Vision and Pattern Recognition, Jun. 2018, pp. 1788-1797.
  4. X. Hu, C. Fu, L. Zhu, J. Qin, and P. Heng, "Direction-aware spatial context features for shadow detection and removal," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 42, no. 11, pp. 2795-2808, May. 2019. https://doi.org/10.1109/tpami.2019.2919616
  5. A. Krizhevsky, I. Sutskever, and G. E. Hinton, "ImageNet classification with deep convolutional neural networks," in Proc. Advances in Neural Information Processing Systems, Dec. 2017, pp. 1097-1105.
  6. S. Xie, R. Girshick, P. Dollar, Z. Tu, and K. He, "Aggregated residual transformations for deep neural networks," in Proc. IEEE Conference on Computer Vision and Pattern Recognition, Jun. 2017, pp. 1493-1500.
  7. K. Simonyan and A. Zisserman, "Very deep convolutional networks for large-scale image recognition," in Proc. Int. Conf. Learn. Representation, May 15, 2015, pp. 1-14.
  8. Y. Zhang, Y. Tian, Y. Kong, B. Zhong, and Y. Fu, "Residual dense network for image super-resolution," in Proc. IEEE Conference on Computer Vision and Pattern Recognition, Jun. 2018, pp. 2472-2481.
  9. L. Qu, J. Tian, S. He, Y. Tang, and R. W. Lau, "DeshadowNet: A multi-context embedding deep network for shadow removal," in Proc. IEEE Conference on Computer Vision and Pattern Recognition, Jun. 2017, pp. 4067-4075.
  10. A. Paszke, S. Gross, S. Chintala, G. Chanan, E. Yang, Z. Devito, Lin, A. Desmaison, L. Antiga, A. Lerer, "Automatic differentiation in pytorch". in Proc. Conference and Workshop on Neural Information Processing Systems (NIPS), pp. 1-4, 2017.
  11. H. Gong and D. P. Cosker, "Interactive shadow removal and ground truth for variable scene categories," in Proc. British Machine Vision Conference, Sep. 2014, pp. 1-11.
  12. G. Hinton, O. Vinyals, and J. Dean, "Distilling the knowledge in a neural network," in Proc. Conference and Workshop on Neural Information Processing Systems (NIPS), Dec. 2015, pp. 1-9.
  13. F. Wang, M. Jiang, C. Qian, S. Yang, C. Li, H. Zhang, X. Wang, "Residual attention network for image classification," in Proc. IEEE Conference on Computer Vision and Pattern Recognition, Jun. 2017, pp. 3156-3164.
  14. S. Zagoruyko and N. Komodakis, "Paying more attention to attention: improving the performance of convolutional neural networks via attention transfer," in Proc. International Conference on Learning Representations, Sep. 2017, pp. 1-13.