Browse > Article
http://dx.doi.org/10.7471/ikeee.2018.22.4.1115

Accuracy Improvement Method for 1-Bit Convolutional Neural Network  

Im, Sung-Hoon (Dept. of Computer Engineering, Hanbat National University)
Lee, Jae-Heung (Dept. of Computer Engineering, Hanbat National University)
Publication Information
Journal of IKEEE / v.22, no.4, 2018 , pp. 1115-1122 More about this Journal
Abstract
In this paper, we analyze the performance degradation of previous 1-Bit convolutional neural network method and introduce ways to mitigate it. Previous work applies 32-Bit operation to first and last layers. But our method applies 32-Bit operation to second layer too. We also show that nonlinear activation function can be removed after binarizing inputs and weights. In order to verify the method proposed in this paper, we experiment the object detection neural network for korean license plate detection. Our method results in 96.1% accuracy, but the existing method results in 74% accuracy.
Keywords
XNOR Neural Network; 1-Bit Neural Network; Quantized Neural Network; Convolutional Neural Network; Neural Network;
Citations & Related Records
연도 인용수 순위
  • Reference
1 M. Rastegari, V. Ordonez, J. Redmon and A. Farhadi., "Xnor-net: Imagenet classification using binary convolutional neural networks," European Conference on Computer Vision, pp.525-542, 2016.
2 B. Jacob, S. Kligys, B. Chen, M. Zhu, M. Tang, A. Howard, H. Adam and D. Kalenichenko, "Quantization and training of neural networks for efficient integer-arithmetic-only inference," Computer Vision and Pattern Recognition pp.2704-2713, 2018.
3 C. Zhu, S. Han, H. Mao and W. J. Dally, "Trained ternary quantization," International Conference on Learning Representations, 2017.
4 Michael Mathieu, Mikael Henaff and Yann LeCun, "Fast training of convolutional networks through ffts," International Conference on Learning Representations, 2013.
5 A. Lavin and S. Gray, "Fast algorithms for convolutional neural networks," Computer Vision and Pattern Recognition, pp.4013-4021, 2016.
6 K. He, X. Zhang, S. Ren and J. Sun, "Delving deep into rectifiers: Surpassing human-level performance on imagenet classification," International Conference on Computer Vision, pp.1026-1034, 2015.
7 S. Ioffe and C. Szegedy, "Batch normalization: Accelerating deep network training by reducing internal covariate shift", International Conference on International Conference on Machine Learning, pp.448-456, 2015.
8 Y. Bengio, N. Leonard and A. Courville, "Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation," https://arxiv.org/abs/1308.3432.
9 J. Redmon and A. Farhadi, "YOLO9000: Better, faster, stronger," Computer Vision and Pattern Recognition, pp.6517-6525, 2017.