Journal of Electrical Engineering and Technology

  • 제13권5호
  • /
  • Pages.2099-2106
  • /
  • 2018
  • /
  • 1975-0102(pISSN)
  • /
  • 2093-7423(eISSN)
대한전기학회 (The Korean Institute of Electrical Engineers)
권호 표지
DOI QR코드

DOI QR Code

Discriminative Manifold Learning Network using Adversarial Examples for Image Classification

  • Zhang, Yuan (Safety Technology and Engineering Specialty, Anhui University of Science and Technology) ;
  • Shi, Biming (Safety Technology and Engineering Specialty, Anhui University of Science and Technology)
  • 투고 : 2017.06.17
  • 심사 : 2018.05.23
  • 발행 : 2018.09.01
PDF 다운로드
⟨ 이전 논문 다음 논문 ⟩

초록

This study presents a novel approach of discriminative feature vectors based on manifold learning using nonlinear dimension reduction (DR) technique to improve loss function, and combine with the Adversarial examples to regularize the object function for image classification. The traditional convolutional neural networks (CNN) with many new regularization approach has been successfully used for image classification tasks, and it achieved good results, hence it costs a lot of Calculated spacing and timing. Significantly, distrinct from traditional CNN, we discriminate the feature vectors for objects without empirically-tuned parameter, these Discriminative features intend to remain the lower-dimensional relationship corresponding high-dimension manifold after projecting the image feature vectors from high-dimension to lower-dimension, and we optimize the constrains of the preserving local features based on manifold, which narrow the mapped feature information from the same class and push different class away. Using Adversarial examples, improved loss function with additional regularization term intends to boost the Robustness and generalization of neural network. experimental results indicate that the approach based on discriminative feature of manifold learning is not only valid, but also more efficient in image classification tasks. Furthermore, the proposed approach achieves competitive classification performances for three benchmark datasets : MNIST, CIFAR-10, SVHN.

키워드

참고문헌

  1. Tomar, V. S. and Rose, R. C., "Manifold Regularized Deep Neural Networks," INTER-SPEECH., 2014.
  2. Feng, Z., Jin, L., Tao, D. and Huang, S., "Dlanet: a manifold-learning-based discriminative feature learning network for scene classification,". Neurocomputing, 157, pp. 11-21, 2015. https://doi.org/10.1016/j.neucom.2015.01.043
  3. Lee, T., Choi, M. and Yoon, S., "Manifold regularized deep neural networks using adversarial examples," Computer Science, 2015.
  4. Masci, J., Boscaini, D., Bronstein, M. M. and Vandergheynst, P., "Shapenet: convolutional neural networks on non-euclidean manifolds," Epfl, pp. 832-840, 2015.
  5. Hinton, G. E., Srivastava, N., Krizhevsky, A., Sutskever, I. and Salakhutdinov, R. R., "Improving neural networks by preventing co-adaptation of feature detectors," Computer Science, vol. 3, no. 4, pp. 212-223, 2012.
  6. Wan, L., Zeiler, M., Zhang, S., Cun, Y. L. and Fergus, R., "Regularization of neural networks using dropconnect," International Conference on Machine Learning, pp. 1058-1066, 2013.
  7. Goodfellow, I. J., Wardefarley, D., Mirza, M., Courville, A. and Bengio, Y., "Maxout net-works," Computer Science, pp. 1319-1327, 2013.
  8. Zeiler, M. D. and Fergus, R., "Stochastic pooling for regularization of deep convolutional neural networks," Eprint Arxiv, 2013.
  9. Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S. and Anguelov, D., et al., "Going deeper with convolutions," P. Computer Vision and Pattern Recognition. IEEE, pp. 1-9, 2015.
  10. Yang, W., Sun, C. and Lei, Z., "A multi-manifold discriminant analysis method for image feature extraction," Pattern Recognition, vol. 44, no. 8, pp. 1649-1657, 2011. https://doi.org/10.1016/j.patcog.2011.01.019
  11. Goodfellow, I. J., Shlens, J. and Szegedy, C., "Explaining and harnessing adversarial examples," Computer Science, 2014.
  12. Lecun, Y., Bottou, L., Bengio, Y. and Haffner, P., "Gradient-based learning applied to document recognition," Proceedings of the IEEE, vol. 86, no. 11, pp. 2278-2324, 1998. https://doi.org/10.1109/5.726791
  13. Tabacof, P. and Valle, E., "Exploring the space of adversarial images," Computer Science, 2015.
  14. Gisbrecht, A., Schulz, A. and Hammer, B., "Parametric nonlinear dimensionality reduction using kernel t-sne," Neurocomputing, vol. 147, no. 1, pp. 71-82, 2015. https://doi.org/10.1016/j.neucom.2013.11.045
  15. Hinton, G. E., "Visualizing high-dimensional data using t-sne," Vigiliae Christianae, vol. 9, no. 2, pp. 2579-2605, 2008.
  16. Han, Y., Xu, Z., Ma, Z. and Huang, Z., "Image classification with manifold learning for out-of-sample data," Signal Processing, vol. 93, no. 8, pp. 2169-2177, 2013. https://doi.org/10.1016/j.sigpro.2012.05.036
  17. Vural E, Guillemot C., "Out-of-Sample Generalizations for Supervised Manifold Learning for Classification," IEEE Transactions on Image Processing A Publication of the IEEE Signal Processing Society, vol. 25, no. 3, p. 1410, 2016. https://doi.org/10.1109/TIP.2016.2520368
  18. Papernot N, Mcdaniel P, Goodfellow I, et al. "Practical Black-Box Attacks against Deep Learning Systems using Adversarial Examples", 2016.
  19. TensorFlow. http://www.tensorflow.org/