Journal of Integrative Natural Science (통합자연과학논문집)

The Basic Science Institute Chosun University (조선대학교 기초과학연구원)
권호 표지
DOI QR코드

DOI QR Code

A Study on Image Recommendation System based on Speech Emotion Information

  • Kim, Tae Yeun (SW Convergence Education Institute, Chosun University) ;
  • Bae, Sang Hyun (Department of Computer Science & Statistics, Chosun University)
  • Received : 2018.08.30
  • Accepted : 2018.09.17
  • Published : 2018.09.30
Download PDF
⟨ Previous Next ⟩

Abstract

In this paper, we have implemented speeches that utilized the emotion information of the user's speech and image matching and recommendation system. To classify the user's emotional information of speech, the emotional information of speech about the user's speech is extracted and classified using the PLP algorithm. After classification, an emotional DB of speech is constructed. Moreover, emotional color and emotional vocabulary through factor analysis are matched to one space in order to classify emotional information of image. And a standardized image recommendation system based on the matching of each keyword with the BM-GA algorithm for the data of the emotional information of speech and emotional information of image according to the more appropriate emotional information of speech of the user. As a result of the performance evaluation, recognition rate of standardized vocabulary in four stages according to speech was 80.48% on average and system user satisfaction was 82.4%. Therefore, it is expected that the classification of images according to the user's speech information will be helpful for the study of emotional exchange between the user and the computer.

Keywords

References

  1. H.-T. Choi and S.-B. Cho, "A Collaborative Filtering Recommendation System using ConceptNetbased Mood Classification by Genre," in Proceeding of the 38th Annual Symposium on Korea Information Science Society, Korea, pp. 216-219, Jun. 2011.
  2. B.-H. Oh, J.-H. Yang, and H.-J. Lee "A Hybrid Recommender System based on Collaborative Filtering with Selective Utilization of Content-based Predicted Ratings," Journal of KIISE, Vol. 41, No. 4, pp. 289-294, Apr. 2014.
  3. S.-Z. Lee, Y.-H. Seong, and H.-J. Kim, “Modeling and Measuring User Sensitivity for Customized Service of Music Contents,” Journal of Korean Society For Computer Game, Vol. 26, No. 1, pp. 163-171, March 2013.
  4. J. Byun and D.-K. Kim, “Design and Implementation of Location Recommending Services using Personal Emotional Information based on Collaborative Filtering,” Journal of the Korea Institute of Information and Communication Engineering, Vol. 20, No. 8, pp. 1407-1414, Aug. 2016. https://doi.org/10.6109/jkiice.2016.20.8.1407
  5. T.-Y. Kim, B.-H. Song, and S.-H. Bae, "A Design and Implementation of Music & Image Retrieval Recommendation System based on Emotion," Journal of The Institute of Electronics Engineers of Korea, Vol. 47, No. 1, pp. 73-79. 2010.
  6. J.-H. Bang and S.-Y. Lee, "Call Speech Emotion Recognition for Emotion based Services," Journal of KIISSE, Vol. 41, No. 3, pp. 208-213. 2014.
  7. A. B. Kandali, A. Routray, and T. K. Basu, "Emotion recognition from Assamese speeches using MFCC features and GMM classifier," TENCON 2008-2008 IEEE Region 10 Conference, pp.1-5, Nov, 2008.
  8. Z. Xiao, Dellandrea, L. Chen, and W. Dou, "Recognition of emotions in speech by a hierarchical approach," ACII 2009. 3rd International Conference, pp.401-408, 2009.
  9. Y.-h. Cho and K.-S. Park, "A Study on The Improvement of Emotion Recognition by Gender Discrimination," Journal of IEEK, Vol. 45, pp.401-408, 2008.
  10. J. Li and J. Z. Wang, “Automatic linguistic indexing of pictures by a statistical modeling approach,” IEEE Trans. Pattern Analysis and Machine Intelligence, Vol. 25, No. 9, pp. 1075-1088, 2003. https://doi.org/10.1109/TPAMI.2003.1227984
  11. S.-K. Baek, "Kansei Distribution Space Creation by Visual Information Elements for Kansei-based Image Retrieval," Master's thesis, Chosun University, 2007.
  12. M. Zulkifly and N. Yahya, "Relative spectral-perceptual linear prediction (RASTA-PLP) speech signals analysis using singular value decomposition (SVD)," IEEE 3rd International Symposium in Robotics and Manufacturing Automation (ROMA), pp. 1-5, 2017.
  13. B.-W. Jung, "Personalized Emotion Recognition using PLP Analysis of Speech Signal," Master thesis Pusan National University, 2008.
  14. S. Hwangbo, S.-Y. Chun, S.-Y Gang, and C.-S. Lee, “Lighting Control using Frequency Analysis of Music,” Journal of Korea Multimedia Society, Vol. 16, No. 11, pp. 1325-1337, 2013. https://doi.org/10.9717/kmms.2013.16.11.1325
  15. S.-C. Hur, S.-H. Cho, and J.-S. Sim, “Compressed pattern matching for dictionary-based compressed text,” Journal of Korean Institute of Next Generation Computing, Vol. 12, No. 1, pp. 67-74, 2016.
  16. H.-Y. Noh, S.-B. Choi, and H.-C. Ahn, “Social Network-based Hybrid Collaborative Filtering using Genetic Algorithms,” Journal of KIISS, Vol. 23, No. 2, pp. 19-38, 2017.