Browse > Article
http://dx.doi.org/10.3837/tiis.2019.05.018

A Deep Approach for Classifying Artistic Media from Artworks  

Yang, Heekyung (Dept. of Computer Science, Graduate School, Sangmyung Univ.)
Min, Kyungha (Dept. of Computer Science, Sangmyung Univ.)
Publication Information
KSII Transactions on Internet and Information Systems (TIIS) / v.13, no.5, 2019 , pp. 2558-2573 More about this Journal
Abstract
We present a deep CNN-based approach for classifying artistic media from artwork images. We aim to classify most frequently used artistic media including oilpaint brush, watercolor brush, pencil and pastel, etc. For this purpose, we extend VGGNet, one of the most widely used CNN structure, by substituting its last layer with a fully convolutional layer, which reveals class activation map (CAM), the region of classification. We build two artwork image datasets: YMSet that collects more than 4K artwork images for four most frequently used artistic media from various internet websites and WikiSet that collects almost 9K artwork images for ten most frequently used media from WikiArt. We execute a human baseline experiment to compare the classification performance. Through our experiments, we conclude that our classifier is superior in classifying artistic media to human.
Keywords
CNN; VGGNet; classification; artistic media; artwork;
Citations & Related Records
연도 인용수 순위
  • Reference
1 Gatys, L., Ecker, A., Bethge, M., "Image style transfer using convolutional neural networks," in Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2414-2423, 2016.
2 Gygli, M., Grabner, H., Riemenschneider, H., Nater, F., and Van Gool, L., "The interestingness of images," in Proc. of the IEEE International Conference on Computer Vision, pp. 1633-1640, 2013.
3 Isola, P., Xiao, J., Torralba, A., and Oliva, A., "What makes an image memorable?," in Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 145-152, 2011.
4 Karayev, S., Trentacoste, M., Han, H., Agarwala, A., Darrell, T., Hertzmann, A., and Winnemoeller, H., "Recognizing image style," in Proc. of the British Machine Vision Conference 2014, pp. 1-20, 2014.
5 Keren, D., "Painter identification using local reatures and naive Bayes," in Proc. of ICPR 2002, pp. 474-477, 2002.
6 Lecoutre, A., Negrevergne, B. and Yger, F., "Recognizing art styles automatically in painting with deep learning," in Proc. of Asian Conference on Machine Learning, pp. 327-342, 2017.
7 Long, J., Shelhamer, E. and Darrell, T., "Fully convolutional networks for semantic segmentation," in Proc. of the IEEE conference on Computer Vision and Pattern Recognition, pp. 3431-3440, 2015.
8 Matsuo, S. and Yanai, K., "CNN-based style vector for style image retrieval," in Proc. of International Conference on Multimedia Retrieval, pp. 309-312, 2016.
9 Mensink, T. and van Gemert, J., "The Rijksmuseum Challenge: Museum-centered visual recognition," in Proc. of ICMR, pp. 451, 2014.
10 Shamir, L., Macura, T., Orlov, N., Eckley, D., and Goldberg, I., "Impressionism, Expressionism, Surrealism: Automated recognition of painters and schools of art," ACM Trans. Applied Perc., vol. 7(2), pp. 8, 2010.
11 Srivastava, N., Hinton, G. E., Krizhevsky, A., Sutskever, I., and Salakhutdinov, R., "Dropout: a simple way to prevent neural networks from overfitting," Journal of Machine Learning Research, vol. 15(1), pp. 1929-1958, 2014.
12 Shamir, L., Nissel, J. and Winner, E., "Distinguishing between abstract art by artists vs. children and animals: Comparison between human and machine perception," ACM Transactions on Applied Perception, Vol. 13, No. 3, pp. 17, 2016.
13 Simonyan, K., and Zisserman, A., "Very deep convolutional networks for large-scale image recognition," arXiv preprint arXiv:1409.1556, 2014.
14 Selim, A., Mohamed E., and Linda D., "Painting style transfer for head portraits using convolutional neural networks," ACM Transactions on Graphics, vol. 35(4), pp. 129, 2016.
15 Sun, T., Wang, Y., Yang, J. and Hu, X., "Convolution neural networks with two pathways for image style recognition," IEEE Transactions on Image Processing, vol. 26(9), pp. 4102-4113, 2017.   DOI
16 Dai, J., Li, Y., He, K., and Sun, J., "R-fcn: Object detection via region-based fully convolutional networks," In Advances in neural information processing systems, pp. 379-387, 2016.
17 Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., and Torralba, A., "Learning deep features for discriminative localization," in Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2921-2929, 2016.
18 Borth, D., Ji, R., Chen, T., Breuel, T., and Chang, S. F., "Large-scale visual sentiment ontology and detectors using adjective noun pairs," in Proc. of the 21st ACM international conference on Multimedia, pp. 223-232, 2013.
19 Chu, W.-T. and Wu, Y.-L., "Deep correlation features for image style classification," in Proc. of ACM Multimedia, pp. 402-406, 2016.
20 Condorovici, R., Florea, C. and Vertan, C., "Automatically classifying paintings with perceptual inspired descriptors," Journal of Visual Communication and Image Representation, vol. 26, pp. 222-230, 2015.   DOI
21 Datta, R., Joshi, D., Li, J., and Wang, J., "Studying aesthetics in photographic images using a computational approach," in Proc. of European Conference on Computer Vision, pp. 288-301, 2006.
22 Dhar, S., Ordonez, V. and Berg, T., "High level describable attributes for predicting aesthetics and interestingness," in Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1657-1664, 2011.