Browse > Article
http://dx.doi.org/10.15207/JKCS.2019.10.9.023

Image Mood Classification Using Deep CNN and Its Application to Automatic Video Generation  

Cho, Dong-Hee (Dept. Computer Science, Kwangwoon University)
Nam, Yong-Wook (Dept. Computer Science, Kwangwoon University)
Lee, Hyun-Chang (Dept. Computer Science, Kwangwoon University)
Kim, Yong-Hyuk (Dept. Computer Science, Kwangwoon University)
Publication Information
Journal of the Korea Convergence Society / v.10, no.9, 2019 , pp. 23-29 More about this Journal
Abstract
In this paper, the mood of images was classified into eight categories through a deep convolutional neural network and video was automatically generated using proper background music. Based on the collected image data, the classification model is learned using a multilayer perceptron (MLP). Using the MLP, a video is generated by using multi-class classification to predict image mood to be used for video generation, and by matching pre-classified music. As a result of 10-fold cross-validation and result of experiments on actual images, each 72.4% of accuracy and 64% of confusion matrix accuracy was achieved. In the case of misclassification, by classifying video into a similar mood, it was confirmed that the music from the video had no great mismatch with images.
Keywords
Convergence; Machine Learning; Multi-class Classification; Mood Classification; Convolutional Neural Network; Multilayer Perceptron;
Citations & Related Records
Times Cited By KSCI : 3  (Citation Analysis)
연도 인용수 순위
1 Y. Yan, M. Chen, M. L. Shyu & S. C. Chen (2015, December). Deep learning for imbalanced multimedia data classification. 2015 IEEE International Symposium on Multimedia. (pp. 483-488). DOI : 10.1109/ISM.2015.126
2 M. K. Lee, D. H. Kim, D. Y. Choi, and B. C. Song. (2017). Emotion recognition system based deep learning. Journal of the Korean Society Of Broad Engineers, 16-18.
3 S. H. Kim. (2016). Sentiment classification for videos using deep learning algorithms. Master dissertation. Seoul University, Seoul.
4 J. A. Russell. (1980). A circumplex model of affect. Journal of personality and social psychology, 39(6), 1161-1178 DOI : 10.1037/h0077714   DOI
5 D. H. Ko, H. K. Moon, J. W. Jun, J. M. Yu & M. G. Jeon. (2017). Face Verification based on DeepConvolutional Nerual Network. Journal of The Korean Institute of Information Scientists and Engineers
6 D. G. Lee. (2018). Classification of Trucks using Convolutional Neural Network. Journal of Convergence for Information Technology, 8(6), 375-380 DOI : 10.22156/CS4SMB.2018.8.6.375   DOI
7 A. M. Ramadhani, N. R. Kim & H. R. Choi. (2018). Predicting Employment Earning using Deep Convolutional Neural Networks. Journal of Digital Convergence, 16(6), 151-161. DOI : 10.14400/JDC.2018.16.6.151   DOI
8 J. Y. Lee, C. B. Moon and B. M. Kim. (2018). Music crawler for mood-based music classification and retrieval systems. Journal of Korea Information Science Society, 699-701
9 C. W. Lee. (2005). Development of automatic synchronization tool for scene and background music. Chungcheongbuk-do : INET.
10 C. Olston & M. Najork. (2010). Web crawling. Foundations and $Trends^{(R)}$ in Information Retrieval, 4(3), 175-246. DOI : 10.1561/1500000017   DOI
11 N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, & R. Salakhutdinov. (2014). Dropout: a simple way to prevent neural networks from overfitting. The journal of machine learning research, 15(1), 1929-1958. DOI : 10.1214/12-AOS1000
12 G. B. Huang, H. Zhou, X. Ding & R. Zhang. (2011). Extreme learning machine for regression and multiclass classification. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 42(2), 513-529. DOI : 10.1109/TSMCB.2011.2168604   DOI
13 M. Riedmiller. (1994). Advanced supervised learning in multi-layer perceptrons-from backpropagation to adaptive learning algorithms. Computer Standards & Interfaces, 16(3), 265-278. DOI : 10.1016/0920-5489(94)90017-5   DOI
14 A. Krizhevsky, I. Sutskever, & G. E. Hinton. (2012). Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems, 1097-1105 DOI : 10.1145/3065386
15 T. Kincl, M. Novak & J. Pribil. (2013, November). Getting inside the minds of the customers: automated sentiment analysis. ECMLG2013-Proceedings For the 9th European Conference on Management Leadership and Governance: ECMLG 2013. (pp. 122-128). Klagenfurt : Academic Conferences Limited
16 V. Gajarla & A. Gupta. (2015). Emotion detection and sentiment analysis of images. Atlanta : Georgia Institute of Technology.
17 S. Hochreiter. (1998). The vanishing gradient problem during learning recurrent neural nets and problem solutions. International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems, 6(02), 107-116. DOI : 10.1142/S0218488598000094   DOI
18 R. A. Dunne & N. A. Campbell. (1997, June). On the pairing of the softmax activation and cross-entropy penalty functions and the derivation of the softmax activation function. Proc. 8th Aust. Conf. on the Neural Networks, Melbourne. DOI : 10.1.1.49.6403
19 A. Krogh & J. Vedelsby. (1995). Neural network ensembles, cross validation, and active learning. Advances in neural information processing systems. (pp. 231-238). Cambridge,MA:MITPress.
20 M. Sokolova & G. Lapalme. (2009). A systematic analysis of performance measures for classification tasks. Information processing & management, 45(4), 427-437. DOI : 10.1016/j.ipm.2009.03.002   DOI