Journal of KIISE (정보과학회 논문지)

Korean Institute of Information Scientists and Engineers (한국정보과학회)
권호 표지
DOI QR코드

DOI QR Code

(Searching Effective Network Parameters to Construct Convolutional Neural Networks for Object Detection)

물체 검출 컨벌루션 신경망 설계를 위한 효과적인 네트워크 파라미터 추출

  • 김누리 (서울대학교 전기정보공학부) ;
  • 이동훈 (서울대학교 전기정보공학부) ;
  • 오성회 (서울대학교 전기정보공학부)
  • Received : 2017.01.17
  • Accepted : 2017.05.18
  • Published : 2017.07.15
⟨ Previous Next ⟩

Abstract

Deep neural networks have shown remarkable performance in various fields of pattern recognition such as voice recognition, image recognition and object detection. However, underlying mechanisms of the network have not been fully revealed. In this paper, we focused on empirical analysis of the network parameters. The Faster R-CNN(region-based convolutional neural network) was used as a baseline network of our work and three important parameters were analyzed: the dropout ratio which prevents the overfitting of the neural network, the size of the anchor boxes and the activation function. We also compared the performance of dropout and batch normalization. The network performed favorably when the dropout ratio was 0.3 and the size of the anchor box had not shown notable relation to the performance of the network. The result showed that batch normalization can't entirely substitute the dropout method. The used leaky ReLU(rectified linear unit) with a negative domain slope of 0.02 showed comparably good performance.

최근 몇 년간 딥러닝(deep learning)은 음성 인식, 영상 인식, 물체 검출을 비롯한 다양한 패턴인식 분야에서 혁신적인 성능 발전을 거듭해왔다. 그에 비해 네트워크가 어떻게 작동하는지에 대한 깊은 이해는 잘 이루어지지 않고 있다. 본 논문은 효과적인 신경망 네트워크를 구성하기 위해 네트워크 파라미터들이 신경망 내부에서 어떻게 작동하고, 어떤 역할을 하고 있는지 분석하였다. Faster R-CNN 네트워크를 기반으로 하여 신경망의 과적합(overfitting)을 막는 드랍아웃(dropout) 확률과 앵커 박스 크기, 그리고 활성 함수를 변화시켜 학습한 후 그 결과를 분석하였다. 또한 드랍아웃과 배치 정규화(batch normalization) 방식을 비교해보았다. 드랍아웃 확률은 0.3일 때 가장 좋은 성능을 보였으며 앵커 박스의 크기는 최종 물체 검출 성능과 큰 관련이 없다는 것을 알 수 있었다. 드랍아웃과 배치 정규화 방식은 서로를 완전히 대체할 수는 없는 것을 확인할 수 있었다. 활성화 함수는 음수 도메인의 기울기가 0.02인 leaky ReLU가 비교적 좋은 성능을 보였다.

Keywords

Acknowledgement

Supported by : 한국연구재단

References

  1. G. E. Hinton and R. R. Salakhutdinov, "Reducing the dimensionality of data with neural networks," Science, pp. 504-507, 2006.
  2. Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton, "Imagenet classification with deep convolutional neural networks," Advances in neural information processing systems, 2012.
  3. Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun, "Faster R-CNN: Towards real-time object detection with region proposal networks," Advances in neural information processing systems, 2015.
  4. Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian sun, "Deep residual learning for image recognition," Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2016.
  5. Jonathan Long, Evan Shelhamer, and Trevor Darrell, "Fully Convolutional Networks for Semantic Segmentation," Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2015.
  6. Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun, "Spatial pyramid pooling in deep convolutional networks for visual recognition," European Conference on Computer Vision, 2014.
  7. Ross Girshick, "Fast r-cnn," Proc. of the IEEE International Conference on Computer Vision, 2015.
  8. Heeyoul Choi, and Yoonhong Min, "Understanding Dropout Algorithms," Journal of KIISE, Vol. 33, No. 8, pp. 32-38, 2015. (in Korean)
  9. Pierre Baldi and Peter Sadowski, "Understanding dropout," Advances in Neural Information Processing Systems, 2013.
  10. Nitish Srivastava, Geoffrey Hinton, Al.ex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov, "Dropout: a simple way to prevent neural networks from overfitting," Journal of Machine Learning Research, Vol. 15, No. 1, pp. 1929-1958, 2014.
  11. Vinod Nair and Geoffrey E. Hinton, "Rectified linear units improve restricted boltzmann machines," Proc. of the 27th International Conference on Machine Learning, 2010.
  12. Sergey Ioffe, Christrian Szegedy, "Batch normalization: Accelerating deep network training by reducing internal covariate shift," arXiv, 2015.
  13. Karen Simonyan and Andrew Zisserman, "Very deep convolutional networks for large-scale image recognition," arXiv, 2014.