Browse > Article
http://dx.doi.org/10.9717/kmms.2019.22.8.874

The Influence of Topic Exploration and Topic Relevance On Amplitudes of Endogenous ERP Components in Real-Time Video Watching  

Kim, Yong Ho (Dept. of Mass Communications, College of Humanities and Social Sciences, Pukyong National University)
Kim, Hyun Hee (Dept. of Library and Information Science, College of Humanities, Myongji University)
Publication Information
Journal of Korea Multimedia Society / v.22, no.8, 2019 , pp. 874-886 More about this Journal
Abstract
To delve into the semantic gap problem of the automatic video summarization, we focused on an endogenous ERP responses at around 400ms and 600ms after the on-set of audio-visual stimulus. Our experiment included two factors: the topic exploration of experimental conditions (Topic Given vs. Topic Exploring) as a between-subject factor and the topic relevance of the shots (Topic-Relevant vs. Topic-Irrelevant) as a within-subject factor. For the Topic Given condition of 22 subjects, 6 short historical documentaries were shown with their video titles and written summaries, while in the Topic Exploring condition of 25 subjects, they were asked instead to explore topics of the same videos with no given information. EEG data were gathered while they were watching videos in real time. It was hypothesized that the cognitive activities to explore topics of videos while watching individual shots increase the amplitude of endogenous ERP at around 600 ms after the onset of topic relevant shots. The amplitude of endogenous ERP at around 400ms after the onset of topic-irrelevant shots was hypothesized to be lower in the Topic Given condition than that in the Topic Exploring condition. The repeated measure MANOVA test revealed that two hypotheses were acceptable.
Keywords
Automatic Video Summarization; Semantic Gap Problem; Endogenous ERP Component; Topic-Relevance; Shot Boundary Detection;
Citations & Related Records
Times Cited By KSCI : 1  (Citation Analysis)
연도 인용수 순위
1 A.G. Money and H. Agius, "Video Summarisation: A Conceptual Framework and Survey," Journal of Visual Communication and Image Representation, Vol. 19, No. 2, pp. 121-143, 2008.   DOI
2 A.W.M. Smeulders, M. Worring, S. Satini, A. Gupta, and R. Jain, "Content-based Image Retrieval at the End of the Early Years," IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 22, No. 12, pp. 1349-1380, 2000.   DOI
3 A.G. Money and H. Agius, "Analysing User Physiological Responses for Affecive Video Summarisation," Displays, Vol. 30, No. 2, pp. 59-70, 2009.   DOI
4 S. Koelstra, C. Muhl, and I. Patras, "EEG Analysis for Implicit Tagging of Video Data," Proceeding of the 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops, pp. 1-6, 2009.
5 M.J.A. Eugster, T. Ruotsalo, M.M. Spape, I. Kosunen, S. Barral, and N. Ravaja, et al., "Predicting Term-Relevance from Brain Signals," Proceedings of the 37th International Association for Computing Machinery Special Interest Group on Information Retrieval Conference on Research and Development in Information Retrieval, pp. 425-434, 2014.
6 X. Zhu, A.B. Goldberg, M. Eldawy, C.R. Dyer, and B. Strock, "A Text-to-Picture Synthesis System for Augmenting Communication," Association for the Advancement of Artificial Intelligence, Vol. 7, pp. 1590-1595, 2007.
7 R.E. Mayer, Cognitive Theory of Multimedia Learning, The Cambridge Handbook of Multimedia Learning, Cambridge University Press, New York, 2005.
8 M. Allegretti, Y. Moshfeghi, J. Hadjigeorgieva, F.E. Pollick, J.M. Jose, and G. Pasi, "When Relevance Judgement is Happening? An EEGbased Study," Proceedings of the 38th International Association for Computing Machinery Special Interest Group on Information Restrieval Conference on Research and Development in Information Retrieval, pp. 719-722, 2015.
9 Y.H. Kim and H.H. Kim, "A Video Summarization Study On Selecting-Out Topic-Irrelevant Shots Using N400 ERP Components in the Real-Time Video Watching," Journal of Korea Multimedia Society, Vol. 20, No. 8, pp. 1258-1270, 2017.   DOI
10 P. Burkhardt, "Inferential Bridging Relations Reveal Distinct Neural Mechanisms: Evidene from Event-related Brain Potentials," Brain and Language, Vol. 98, No. 2, pp. 159-168, 2006.   DOI
11 L. Wang and P.B. Schumacher, "New Is Not Always Costly: Evidence from Online Processing of Topic and Contrast in Japanese," Frontiers in Psychology, Vol. 4, No. 363, pp. 1-20, 2013
12 E. Kaan and T.Y. Swaab, "Repair, Revision, and Complexity in Syntactic Analysis: An Electrophysiological Differentiation," Journal of Cognitive Neuroscience, Vol. 15, No. 1, pp. 59-201, 2003.
13 C. Hendahewa and C. Shah, "Evaluating User Search Trails in Exploratory Search Tasks," Information Processing and Management, Vol. 53, No. 4, pp. 905-922, 2017.   DOI
14 X., Xu and X. Zhou, "Topic Shift Impairs Pronoun Resolution during Sentence Comprehension: Evidence from Event Related Potentials," Psychophysiology, Vol. 53, No. 2, pp. 129-142, 2016.   DOI
15 J. Burmester, K. Spalek, and I. Wartenburger, "Context Updating during Sentence Comprehension: The Effect of Aboutness Topic," Brain and Language, Vol. 137, pp. 62-76, 2014.   DOI
16 M.J. Eugster, T. Ruotsalo, M.M. Spape, I. Kosunen, O. Barral, N. Ravaja, et al., "Predicting Term-Relevance from Brain Signals," Proceedings of the 37th International ACM SIGIR Conference on Research & Development in Information Retrieval, pp. 425-434, 2014.
17 H. Nakano, M.A.M. Rosario, Y. Oshima-Takane, L. Pierce, and S.G. Tate, "Electrophysiological Response to Omitted Stimulus in Sentence Processing," NeuroReport, Vol. 25, No. 14, pp. 1169-1174, 2014.   DOI
18 H.H. Kim and Y.H. Kim, "Video Summarization Using Event-Relate Potential Responses to Shot Boundaries in Real-Time Video Wathing," Journal of the Association for Information Science and Technology, Vol. 70, No. 2, pp. 164-175, 2019.   DOI
19 H.H. Kim and Y.H. Kim, "ERP/MMR Algorithm for Classifying Topic-Relevant and Topic-Irrelevant Visual Shots of Documentary Videos," Journal of the Association for Information Science and Technology, Vol. 70, No. 9, pp. 931-941, 2019.   DOI
20 S.J. Luck, Ten Simple Rules for Designing and Interpreting ERP Experiments, Event-Related Potentials: A Methods Handbook, Massachusetts Institute of Technology Press, Cambridge, Massachusetts, 2004.