Browse > Article
http://dx.doi.org/10.5573/IEIESPC.2015.4.1.035

Deep Convolution Neural Networks in Computer Vision: a Review  

Yoo, Hyeon-Joong (Department of IT Engineering, Sangmyung University)
Publication Information
IEIE Transactions on Smart Processing and Computing / v.4, no.1, 2015 , pp. 35-43 More about this Journal
Abstract
Over the past couple of years, tremendous progress has been made in applying deep learning (DL) techniques to computer vision. Especially, deep convolutional neural networks (DCNNs) have achieved state-of-the-art performance on standard recognition datasets and tasks such as ImageNet Large-Scale Visual Recognition Challenge (ILSVRC). Among them, GoogLeNet network which is a radically redesigned DCNN based on the Hebbian principle and scale invariance set the new state of the art for classification and detection in the ILSVRC 2014. Since there exist various deep learning techniques, this review paper is focusing on techniques directly related to DCNNs, especially those needed to understand the architecture and techniques employed in GoogLeNet network.
Keywords
Deep learning; Convolutional neural network; ImageNet; Computer vision; GoogLeNet;
Citations & Related Records
연도 인용수 순위
  • Reference
1 L. Deng and D. Yu, Deep Learning Methods and Applications, now Publishers Inc., 2014.
2 W. McCulloch and W. Pitts, "A Logical calculus of the ideas immanent in nervous activity", Bulletin of Mathematical Biophysics, vol. 5, pp. 115-133, 1943.   DOI
3 Donald O. Hebb, The Organization of Behavior: A Neuropsychological Theory, Wiley, June 1949.
4 F. Rosenblatt, "The perceptron: A probabilistic model for information storage and organization in the brain," Psychological Review, vol. 65, pp. 386-408, 1958.   DOI   ScienceOn
5 B. Widrow and M.E. Hoff, Jr., "Adaptive Switching Circuits," IRE WESCON Convention Record, Part 4, pp. 96-104, August 1960.
6 M. Minsky and Seymour Papert, Percetrons, Cambridge, MIT Press, 1969.
7 T. Kohonen, "Correlation Matrix Memories," IEEE Transactions on Computers, vol. 21, pp. 353-359, April 1972.
8 James A. Anderson, "A simple neural network generating an interactive memory," Mathematical Biosciences, vol. 14, pp. 197-220, 1972.   DOI   ScienceOn
9 S. Grossberg, "Adaptive pattern classification and universal recoding: I. Parallel development and coding of neural feature detectors," Biological Cybernetics, vol. 23, pp. 121-134, 1976.   DOI
10 J. J. Hopfield, "Neural Networks and Physical Systems with Emergent Collective Computational Abilities," Proceedings of the National Academy of Sciences of the United States of America (PNAS), vol. 79, pp. 2554-2558, 1982.   DOI
11 P. J. Werbos, Beyond Regression: New Tools for Prediction and Analysis in the Behavioral Sciences, PhD thesis, Harvard University, 1974.
12 D. E. Rumelhart, G. E. Hinton, and R. J. Williams, "Learning internal representations by error propagation," Parallel Distributed Processing: Explorations in the Microstructure of Cognition, D. E. Rumelhart and J. L McClelland, eds, vol. I, pp 318-362, MIT, Cambridge, 1986.
13 Y. LeCun, B. Boser, J. S. Denker, D. Henderson, R. E. Howard, W. Hubbard, and L. D. Jackel, "Backpropagation applied to handwritten zip code recognition," Neural Computation, vol. 1, pp. 541-551, 1989.   DOI
14 Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, "Gradient based learning applied to document recognition," Proceedings of the IEEE, vol. 86, pp. 2278-2324, 1998.   DOI   ScienceOn
15 Geoffrey E. Hinton and Simon Osindero, "A fast learning algorithm for deep belief nets," Neural Computation, vol. 18, pp. 1527-1554, 2006.   DOI   ScienceOn
16 Alex Krizhevsky, Ilya Sutskever, and Geoff Hinton, "Imagenet classification with deep convolutional neural networks," Advances in Neural Information Processing Systems 25, pp. 1106-1114, 2012.
17 Ross B. Girshick, Jeff Donahue, Trevor Darrell, and Jitendra Malik, "Rich feature hierarchies for accurate object detection and semantic segmentation," CVPR 2014, IEEE Conference on, 2014.
18 Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, Andrew Rabinovich, "Going Deeper with Convolutions," CoRR, 2014.
19 K. Fukushima, "Neocognitron: A Self-organizing Neural Network Model for a Mechanism of Pattern Recognition Unaffected by Shift in Position," Biol. Cybernetics, vol. 36, pp. 193-202, 1980.   DOI
20 D. H. Hubel and T. N. Wiesel, "Receptive fields, binocular interaction, and functional architecture in the cats visual cortex," Journal of Physiology (London), vol. 160, pp. 106-154, 1962.   DOI
21 M.D. Zeiler, R. Fergus, "Visualizing and Understanding Convolutional Networks," ECCV 2014 (Honorable Mention for Best Paper Award), Arxiv 1311.2901, Nov 28, 2013.
22 Min Lin, Qiang Chen, and Shuicheng Yan, "Network in network," CoRR, abs/1312.4400, 2013.
23 Pierre Sermanet, David Eigen, Xiang Zhang, Michael Mathieu, Rob Fergus, and Yann LeCun, "Overfeat: Integrated recognition, localization and detection using convolutional networks," CoRR, abs/1312.6229, 2013.
24 Matthew D. Zeiler, Hierarchical Convolutional Deep Learning in Computer Vision, Ph.D. Thesis, Nov. 8, 2013.
25 Geoffrey E. Hinton, Nitish Srivastava, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov, "Improving neural networks by preventing co-adaptation of feature detectors," CoRR, abs/1207.0580, 2012.
26 Pierre Sermanet and Yann LeCun, "Traffic Sign Recognition with Multi-Scale Convolutional Networks," IJCNN, 2011.
27 J. Fan, W. Xu, Y. Wu, and Y. Gong, "Human tracking using convolutional neural networks," Neural Networks, IEEE Transactions on, vol. 21, pp. 1610-1623, 2010.   DOI   ScienceOn
28 B. A. Olshausen and D. J. Field, Sparse coding with an overcomplete basis set: a strategy employed by V1?, Vision Research, vol. 37, pp. 3311-3325, 1997.   DOI   ScienceOn
29 P. C. Bush and T. J. Sejnowski, The cortical neuron, Oxford University Press, 1995.
30 R. Douglas and K. Martin, "Recurrent excitation in neocortical circuits," Science, vol. 269, pp. 981-985, 1995.   DOI   ScienceOn
31 Xavier Glorot, Antoine Bordes, and Yoshua Bengio, "Deep Sparse Rectifier Neural Networks", Proceedings of the 14th International Conference on Artificial Intelligence and Statistics (AISTATS) 2011, Fort Lauderdale, FL, USA, vol. 15 of JMLR:W&CP 15, pp. 315-323, 2011.
32 V. Nair and G. E. Hinton, "Rectified linear units improve restricted boltzmann machines," Proc. 27th International Conference on Machine Learning, 2010.
33 M. Zeiler, G. Taylor, and R. Fergus, "Adaptive deconvolutional networks for mid and high level feature learning," ICCV, 2011.
34 Sanjeev Arora, Aditya Bhaskara, Rong Ge, and Tengyu Ma, "Provable bounds for learning some deep representations," CoRR, abs/1310.6343, 2013.
35 Andrew G. Howard, "Some improvements on deep convolutional neural network based image classification," CoRR, abs/1312.5402, 2013.