Browse > Article

Emotion-based Video Scene Retrieval using Interactive Genetic Algorithm  

Yoo Hun-Woo (연세대학교 인지과학연구소)
Cho Sung-Bae (연세대학교 컴퓨터산업공학부)
Publication Information
Journal of KIISE:Computing Practices and Letters / v.10, no.6, 2004 , pp. 514-528 More about this Journal
Abstract
An emotion-based video scene retrieval algorithm is proposed in this paper. First, abrupt/gradual shot boundaries are detected in the video clip representing a specific story Then, five video features such as 'average color histogram' 'average brightness', 'average edge histogram', 'average shot duration', and 'gradual change rate' are extracted from each of the videos and mapping between these features and the emotional space that user has in mind is achieved by an interactive genetic algorithm. Once the proposed algorithm has selected videos that contain the corresponding emotion from initial population of videos, feature vectors from the selected videos are regarded as chromosomes and a genetic crossover is applied over them. Next, new chromosomes after crossover and feature vectors in the database videos are compared based on the similarity function to obtain the most similar videos as solutions of the next generation. By iterating above procedures, new population of videos that user has in mind are retrieved. In order to show the validity of the proposed method, six example categories such as 'action', 'excitement', 'suspense', 'quietness', 'relaxation', 'happiness' are used as emotions for experiments. Over 300 commercial videos, retrieval results show 70% effectiveness in average.
Keywords
Emotion-based Retrieval; Video Scene Retrieval; Interactive Genetic Algorithm (IGA);
Citations & Related Records
연도 인용수 순위
  • Reference
1 J.-Y. Lee and S.-B. Cho, 'Interactive Genetic Algorithm for Content-Based Image Retrieval,' in Proc. Asia Fuzzy Systems Symposium, pp. 479-484, 1998   과학기술학회마을
2 H. Takagi, 'Interactive Evolutionary Computation: Fusion of the Capabilities of EC Optimization and Human Evaluation,' Proc. of the IEEE, vol. 89, no. 9, pp. 1275-1296, 2001   DOI   ScienceOn
3 J. A. Biles, 'GenJam: A Genetic Algorithm for Generating Jazz Solos,' in Proc. Int. Computer Music Conf., pp. 131-137, 1994
4 C. Caldwell and V. S. Johnston, 'Tracking a Criminal Suspect through Face-Space with a Genetic Algorithm,' in Proc. Int. Conf. Genetic Algorithm, pp. 416-421, 1991
5 W. Banzhaf, 'Interactive Evolution,' Handbook of Evolutionary Computation, 1997
6 C. Colombo, A. Del Bimbo, and P. Pala, 'Retrieval of Commercials by Semantic Content: The Semiotic Perspective,' Multimedia Tools and Applications, vol. 13, no. 1, pp. 93-118, 2001   DOI   ScienceOn
7 J. Itten, Art of Color (Kunst der Farbe), Otto Maier Verlag, Ravensburg, Germany, 1961 (in German)
8 H.-W. Yoo and D-S. Jang, 'Automated Video Segmentation Using Computer Vision Technique,' International Journal of Information Technology and Decision Making, vol. 2, no. 4, 2003 (To appear)
9 D. E. Goldberg, Genetic Algorithms in Search, Optimization, and Machine Learning, Addison-Wesley, 1989
10 S.-B. Cho, 'Towards Creative Evolutionary Systems with Interactive Genetic Algorithm,' Applied Intelligence, vol. 16, no. 2, pp. 129-138, 2002   DOI
11 H. Takagi, T. Noda, and S-B. Cho, 'Psychological Space to Hold Impression among Media in Common for Media Database Retrieval System,' in Proc. IEEE Int. Conf. on System, Man, and Cybernetics, pp.263-268, 1999
12 J.-S. Um, K.-B. Eum, and J.-W. Lee, 'A Study of the Emotional Evaluation Models of Color Patterns Based on the Adaptive Fuzzy System and the Neural Network,' Color Research and Application, vol. 27, no. 3, pp. 208-216, 2002   DOI   ScienceOn
13 I.J. Cox, M.L. Miller, T.P. Minka, T.V. Papathomas, and P.N. Yianilos, 'The Bayesian Image Retrieval System, PicHunter : Theory, Implementation and Psycophysical Experiments,' IEEE Trans, on Image Processing, vol. 9, no 1, pp. 20-37, 2000   DOI   ScienceOn
14 C. Colombo, A. Del Bimbo, and P. Pala, 'Seman-tics in Visual Information Retrieval,' IEEE Multimedia, vol. 6, no. 3, pp.38-53, 1999   DOI   ScienceOn
15 A. Vailaya, M. A. T. Figueiredo, A. K. Jain, and H.J Zhang, 'Image Classification for Contentbased Indexing,' IEEE Trans, on Image Processing, vol. 10, no. 1, pp. 117-130, 2001   DOI   ScienceOn
16 Y. Rui, T.S. Huang, M. Ortega, and S. Mehrota, 'Relevance Feedback: A Power Tool in Interactive Content-Based Image Retrieval,' IEEE Trans, on Circuits and Systems Video Technology, vol. 8, no. 5, pp. 644-655, 1998   DOI   ScienceOn
17 T. Soen, T. Shimada, and M. Akita, 'Objective Evaluation of Color Design,' Color Research and Application, vol. 12, no. 4, pp. 184-194, 1987
18 Ullas Gargi, Tangachar Kasturi, and Susan H. Srayer, 'Performance Characterization of Videl-Shot-Change Detection Methods', IEEE Trans. Circuits and Systems for Video Technology, pp.1-13, Vol. 10, No.1, 2000   DOI   ScienceOn
19 T. P. Minka and R. W. Picard, 'Interactive Learning Using a Society of Models,' Pattern Recognition, vol. 30, no.3, pp. 565-581, 1997   DOI   ScienceOn
20 A. Vailaya, A. K. Jain, and H.J Zhang, 'On Image Classification: City Images vs. Landscapes,' Pattern Recognition, vol. 31, no. 12, pp. 1921-1936, 1998   DOI   ScienceOn
21 B. T. Truong, C. Dorai, and S. Venkatesh, 'New Enhancements to Cut, Fade, and Dissolve Detection Processes in Video Segmentation,' in Proc. ACM Int. Conf. on Multimedia, pp.219-227, 2000   DOI
22 C. Carson, S. Belongie, H. Greenspan, and J. Malick, 'Blobworld: Image Segmentation Using Expectation-Maximization and Its Application to Image Querying,' IEEE Trans, on Pattern Ana-lysis and Machine Intelligence, vol. 24, no. 8, pp. 1026-1038, 2002   DOI   ScienceOn
23 H.-W. Yoo, D.-S. Jang, S.-H. Jung, J.-H. Park, and K.-S. Song, 'Visual Information Retrieval System via Content-Based Approach,' Pattern Recognition, vol. 35, no. 3, pp. 749-769, 2002   DOI   ScienceOn
24 H.-W. Yoo, S.-H. Jung, D.-S. Jang, and Y.-K. Na, 'Extraction of Major Object Features Using VQ Clustering for Content-Based Image Retrieval,' Pattern Recognition, vol. 35, no. 5, pp. 1115-1126, 2002   DOI   ScienceOn
25 J.R. Bach, C. Fuller, A. Gupta, A. Hampapur, B. Horowitz, R. Humphrey, R.C. Jain, and C. Shu, 'The Virage Image Search Engine: An Open Framework for Image Management,' In Proc. SPIE Vol. 2670: Storage and Retrieval for Images and Video Databases IV, pp. 76-86, 1996   DOI
26 J.R. Smith and S.-E. Chang, 'VisualSEEK: A Fully Automated Content-Based Image Query System,' in Proc. ACM Multimedia, pp.87-98, 1996   DOI
27 W.Y. Ma and B.S. Manjunath, 'Netra: A Toolbox for Navigating Large Image Databases,' Multimedia Systems, vol. 7, no. 3, pp. 184-198, 1999   DOI
28 M. Flickner et al., 'Query by image and video content: The QBIC system,' IEEE computer, vol. 28, no. 9, pp. 23-32, 1995   DOI   ScienceOn
29 A. Pentland, R.W. Picard, and S. Sclaroff, 'Photobook: Content-Based Manipulation of Image Data-bases,' International Journal of Computer Vision, vol. 18, no. 3, pp. 233-254, 1996   DOI   ScienceOn