Journal of KIISE:Software and Applications (한국정보과학회논문지:소프트웨어및응용)

Korean Institute of Information Scientists and Engineers (한국정보과학회)
권호 표지

Emotion Image Retrieval through Query Emotion Descriptor and Relevance Feedback

질의 감성 표시자와 유사도 피드백을 이용한 감성 영상 검색

  • 유헌우 (연세대학교 인지과학연구소)
  • Published : 2005.03.01
Download PDF
⟨ Previous Next ⟩

Abstract

A new emotion-based image retrieval method is proposed in this paper. Query emotion descriptors called query color code and query gray code are designed based on the human evaluation on 13 emotions('like', 'beautiful', 'natural', 'dynamic', 'warm', 'gay', 'cheerful', 'unstable', 'light' 'strong', 'gaudy' 'hard', 'heavy') when 30 random patterns with different color, intensity, and dot sizes are presented. For emotion image retrieval, once a query emotion is selected, associated query color code and query gray code are selected. Next, DB color code and DB gray code that capture color and, intensify and dot size are extracted in each database image and a matching process between two color codes and between two gray codes are peformed to retrieve relevant emotion images. Also, a new relevance feedback method is proposed. The method incorporates human intention in the retrieval process by dynamically updating weights of the query and DB color codes and weights of an intra query color code. For the experiments over 450 images, the number of positive images was higher than that of negative images at the initial query and increased according to the relevance feedback.

본 논문에서는 새로운 감성기반 영상검색방법을 제안한다. 서로 다른 색상, 명도, 도트크기를 나타내는 30개의 랜덤 패턴이 제시될 때 인간이 느끼는13가지 감성("like", "beautiful", "natural", "dynamic", "warm", "gay", "cheerful", "unstable", "light", "strong", "gaudy", "hard", "heavy") 평가 데이타로부터 질의 칼라코드와 질의 그레이코드로 명명한 질의 감성 표시자를 설계한다. 감성영상검색을 위해서 질의 감성을 선택하면 질의를 표현하는 칼라코드와 그레이코드가 선택되고 데이타베이스의 영상의 색상 정보를 나타내는 DB 칼라코드와 명도와 도트크기 정보를 나타내는 DB그레이코드값을 추출하여, 칼라코드간의 매칭과 그레이 코드간의 매칭을 통해 유사도를 판단한다. 또한 검색과정에 사용자의 의도를 반영하여 질의 칼라코드와 질의 그레이코드사이의 가중치와 칼라코드내의 가중치를 자동적으로 갱신하는 새로운 유사도 피드백 방법을 제안한다. 430개의 영상에 대해 실험한 결과 최초 질의에 대해 적합한 영상이 부적합한 영상보다 많았으며 유사도 피드백을 사용함에 따라 적합한 영상의 개수가 증가하였다.

Keywords

References

  1. T. Joseph and A. Cardenas, 'PicQuery: A High-level query language for pictorial database management,' IEEE Trans. on Software Engineering, vol. 14, no. 5, pp. 630-638, 1988 https://doi.org/10.1109/32.6140
  2. N. Roussopolous, C. Faloutsos, and T. Sellis, 'An efficient pictorial database system for pictorial structured query language (PSQL),' IEEE Trans. on Software Engineering, vol. 14, no. 5, pp. 639-650, 1988 https://doi.org/10.1109/32.6141
  3. M. Flickner et al., 'Query by image and video content: The QBIC system,' IEEE computer, vol. 28, no. 9, pp. 23-32, 1995 https://doi.org/10.1109/2.410146
  4. A. Pentland, R. Picard, and S. Sclaroff, 'Photobook: Content-based manipulation of image databases,' IJCV, vol. 18, no. 3, pp. 233-254, 1996 https://doi.org/10.1007/BF00123143
  5. J.R. Bach, C. Fuller, A. Gupta, A. Hampapur, B. Horowitz, R. Humphrey, R.C. Jain, and C. Shu, 'The Virage Image Search Engine: An Open Framework for Image Management,' In Proc. SPIE Vol. 2670: Storage and Retrieval for Images and Video Databases IV, pp. 76-86, 1996 https://doi.org/10.1117/12.234785
  6. J.R. Smith and S.-E. Chang, 'VisualSEEK: A Fully Automated Content-Based Image Query System,' in Proc. ACM Multimedia, pp.87-98, 1996 https://doi.org/10.1145/244130.244151
  7. W.Y. Ma and B.S. Manjunath, 'Netra: A toolbox for navigating large image databases,' Multimedia Systems, vol. 7, no. 3, pp. 184-198, 1999 https://doi.org/10.1007/s005300050121
  8. C. Carson, S. Belongie, H. Greenspan, and J. Malick, 'Blobworld: Image segmentation using Expectation-Maximization and its application to image querying,' IEEE Trans. on Pattern Analysis and Machine Intelligence, vol. 24, no. 8, pp. 1026-1038, 2002 https://doi.org/10.1109/TPAMI.2002.1023800
  9. H.-W. Yoo, D.-S. Jang, S.-H. Jung, J.-H. Park, and K.-S. Song, 'Visual Information Retrieval System via Content-Based Approach,' Pattern Recognition, vol. 35, no. 3, pp. 749-769, 2002 https://doi.org/10.1016/S0031-3203(01)00072-3
  10. H.-W. Yoo, S.-H. Jung, D.-S. Jang, and Y.-K. Na, 'Extraction of Major Object Features Using VQ Clustering for Content-Based Image Retrieval,' Pattern Recognition, vol. 35, no. 5, pp. 1115-1126, 2002 https://doi.org/10.1016/S0031-3203(01)00105-4
  11. T. P. Minka and R. W. Picard, 'Interactive Learning Using a Society of Models,' Pattern Recognition, vol. 30, no.3, pp. 565-581, 1997 https://doi.org/10.1016/S0031-3203(96)00113-6
  12. A. Vailaya, A. K. Jain, and H.J Zhang, 'On Image Classification: City Images vs. Landscapes,' Pattern Recognition, vol. 31, no. 12, pp. 1921-1936, 1998 https://doi.org/10.1016/S0031-3203(98)00079-X
  13. A. Vailaya, M. A. T. Figueiredo, A. K. Jain, and H.J Zhang, 'Image Classification for Content-based Indexing,' IEEE Trans. on Image Processing, vol. 10, no. 1, pp. 117-130, 2001 https://doi.org/10.1109/83.892448
  14. Y. Rui, T.S. Huang, M. Ortega, and S. Mehrota, 'Relevance Feedback: A Power Tool in Interactive Content-Based Image Retrieval,' IEEE Trans. on Circuits and Systems Video Technology, vol. 8, no. 5, pp. 644-655, 1998 https://doi.org/10.1109/76.718510
  15. I.J. Cox, M.L. Miller, T.P. Minka, T.V. Papathomas, and P.N. Yianilos, 'The Bayesian Image Retrieval System, PicHunter : Theory, Implementation and Psycophysical Experiments,' IEEE Trans. on Image Processing, vol. 9, no 1, pp. 20-37, 2000 https://doi.org/10.1109/83.817596
  16. S.-B. Cho, 'Towards Creative Evolutionary Systems with Interactive Genetic Algorithm,' Applied Intelligence, vol. 16, no. 2, pp. 129-138, 2002 https://doi.org/10.1023/A:1013614519179
  17. H. Takagi, T. Noda, and S-B. Cho, 'Psychological Space to Hold Impression among Media in Common for Media Database Retrieval System,' in Proc. IEEE Int. Conf. on System, Man, and Cybernetics, pp.263-268, 1999
  18. J.-S. Um, K.-B. Eum, and J.-W. Lee, 'A Study of the Emotional Evaluation Models of Color Patterns Based on the Adaptive Fuzzy System and the Neural Network,' Color Research and Application, vol. 27, no. 3, pp. 208-216, 2002 https://doi.org/10.1002/col.10052
  19. C. Colombo, A. Del Bimbo, and P. Pala, 'Seman-tics in Visual Information Retrieval,' IEEE Multimedia, vol. 6, no. 3, pp.38-53, 1999 https://doi.org/10.1109/93.790610
  20. T. Soen, T. Shimada, and M. Akita, 'Objective evaluation of color design,' Color Research and Application, 1987, vol. 12, no. 4, pp.184-194 https://doi.org/10.1002/col.5080120406
  21. J. Itten, Art of Color (Kunst der Farbe), Otto Maier Verlag, Ravensburg, Germany, 1961 (in German)
  22. H. Tamura, S. Mori, and T. Yamawaki, 'Texture features corresponding to visual perception,' IEEE Trans on Sys, Man, and Cyb, vol. SMC-8, no. 6, pp. 460-473, 1978