Journal of Information Processing Systems

Korea Information Processing Society (한국정보처리학회)
권호 표지
DOI QR코드

DOI QR Code

Viewer's Affective Feedback for Video Summarization

  • Dammak, Majdi (Research Groups in Intelligent Machines (REGIM), University of Sfax) ;
  • Wali, Ali (Research Groups in Intelligent Machines (REGIM), University of Sfax) ;
  • Alimi, Adel M. (Research Groups in Intelligent Machines (REGIM), University of Sfax)
  • Received : 2013.12.27
  • Accepted : 2014.05.21
  • Published : 2015.03.31
Download PDF
⟨ Previous Next ⟩

Abstract

For different reasons, many viewers like to watch a summary of films without having to waste their time. Traditionally, video film was analyzed manually to provide a summary of it, but this costs an important amount of work time. Therefore, it has become urgent to propose a tool for the automatic video summarization job. The automatic video summarization aims at extracting all of the important moments in which viewers might be interested. All summarization criteria can differ from one video to another. This paper presents how the emotional dimensions issued from real viewers can be used as an important input for computing which part is the most interesting in the total time of a film. Our results, which are based on lab experiments that were carried out, are significant and promising.

Keywords

References

  1. M. Dammak, M. Ben Ammar, and A. M. Alimi, "Real-time analysis of non-verbal upper-body expressive gestures," in Proceedings of International Conference on Multimedia Computing and Systems (ICMCS2012), Tangier, 2012, pp. 334-339.
  2. H. Joho, J. Staiano, N. Sebe, and J. M. Jose, "Looking at the viewer: analysing facial activity to detect personal highlights of multimedia contents," Multimedia Tools and Applications, vol. 51, no. 2, pp. 505-523, 2011. https://doi.org/10.1007/s11042-010-0632-x
  3. H. M. Zawbaa, N. El-Bendary, A. E. Hassanien, and T. H. Kim, "Event detection based approach for soccer video summarization using machine learning," International Journal of Multimedia and Ubiquitous Engineering, vol. 7, no. 2, pp. 63-80, 2012.
  4. D. Tjondronegoro, Y. P. P. Chen, and B. Pham, "Sports video summarization using highlights and play-breaks," in Proceedings of the 5th ACM SIGMM International Workshop on Multimedia Information Retrieval (MIR2003), Berkeley, CA, 2003, pp. 201-208.
  5. Y. Qi, A. Hauptmann, and T. Liu, "Supervised classification for video shot segmentation," in Proceedings of the International Conference on Multimedia and Expo (ICME2003), Baltimore, MD, 2003, pp. 689-692.
  6. D. Tjondronegoro, Y. P. P. Chen, and Pham, "Integrating highlights for more complete sports video summarization," IEEE Multimedia, vol. 11, no. 4, pp. 22-37, 2004. https://doi.org/10.1109/MMUL.2004.28
  7. M. G. Christel, M. A. Smith, C. R. Taylor, and D. B. Winkler, "Evolving video skims into useful multimedia abstractions," in Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Los Angeles, CA, 1998, pp. 171-178.
  8. Y. Takahashi, N. Nitta, and N. Babaguchi, "Video summarization for large sports video archives," in Proceedings of the IEEE International Conference on Multimedia and Expo (ICME2005), Amsterdam, 2005, pp. 1170-1173.
  9. A. Hanjalic, "Generic approach to highlights extraction from a sport video," in Proceedings of the International Conference on Image Processing (ICIP 2003), Barcelona, Spain, 2003, pp. 1-4.
  10. M. A. Refaey, W. Abd-Almageed, and L. S. Davis, "A logic framework for sports video summarization using text-based semantic annotation," in Proceedings of the 3rd International Workshop on Semantic Media Adaptation and Personalization (SMAP2008), Prague, 2008, pp. 69-75.
  11. F. Coldefy and P. Bouthemy, "Unsupervised soccer video abstraction based on pitch, dominant color and camera motion analysis," in Proceedings of the 12th Annual ACM International Conference on Multimedia, New York, NY, 2004, pp. 268-271.
  12. G. H. Bower, "How might emotions affect learning," in The Handbook of Emotion and Memory: Research and Theory, S. Christianson, Ed. Hillsdale, NJ: Erlbaum, 1992, pp. 3-31.
  13. T. D. Bui, "Creating emotions and facial expressions for embodied agents," Ph.D. dissertation, University of Twente, Enschede, The Netherlands, 2004.
  14. E. A. R. Tanguy, "Emotions: the art of communication applied to virtual actors," Ph.D. dissertation, University of Bath, England, 2006.
  15. M. Dammak, M. Ben Ammar, and A. M. Alimi, "A new approach to emotion recognition," in Proceedings of the International Conference on Innovations in Information Technology (IIT), Abu Dhabi, 2011, pp. 110-113.
  16. P. Ekman and W. W. Friesen, Unmasking the Face: A Guide to Recognizing Emotions from Facial Clues. Englewood Cliffs, NJ: Prentice-Hall, 1975.
  17. M. Coulson, "Attributing emotion to static body postures: recognition accuracy, confusions, and viewpoint dependence," Journal of Nonverbal Behavior, vol. 28, no. 2, pp. 117-139, 2004. https://doi.org/10.1023/B:JONB.0000023655.25550.be
  18. T. Balomenos, A. Raouzaiou, S. Ioannou, A. Drosopoulos, K. Karpouzis, and S. Kollias, "Emotion analysis in man-machine interaction systems," in Machine Learning for Multimodal Interaction. Heidelberg: Springer, 2005, pp. 318-328.
  19. G. Littlewort, M. S. Bartlett, I. Fasel, J. Susskind, and J. Movellan, "Dynamics of facial expression extracted automatically from video," Image and Vision Computing, vol. 24, no. 6, pp. 615-625, 2006. https://doi.org/10.1016/j.imavis.2005.09.011
  20. H. Gunes and M. Piccardi, "A bimodal face and body gesture database for automatic analysis of human nonverbal affective behavior," in Proceedings of the 18th International Conference on Pattern Recognition (ICPR 2006), Hong Kong, 2006, pp. 1148-1153.
  21. M. Ellouze, N. Boujemaa, and A. M. Alimi, "$IM(S)^2$: Interactive movie summarization system," Journal of Visual Communication and Image Representation, vol. 21, no. 4, pp. 283-294, 2010. https://doi.org/10.1016/j.jvcir.2010.01.007
  22. M. A. Turk, and A. P. Pentland, "Eigenfaces for recognition," Journal of Cognitive Neuroscience, vol. 3, no. 1, pp. 71-86, 1991. https://doi.org/10.1162/jocn.1991.3.1.71
  23. R. Larsen, "Estimation of motion vector fields," in Proceedings of the 2nd Danish Conference on Pattern Recognition and Image Analysis (DANKOMB, DSAGM yearly meeting), 1993, pp. 37-42.
  24. V. Parshin and L. Chen, "Video summarization based on user-defined constraints and preferences," in Computer-Assisted Information Retrieval (Recherched'Informationetses Applications [RIAO]), Avignon, France, 2004, pp. 18-24.