Browse > Article
http://dx.doi.org/10.3743/KOSIM.2015.32.3.377

Toward a Key-frame Automatic Extraction Method for Video Storyboard Surrogates Based on Users' EEG Signals and Discriminant Analysis  

Kim, Hyun-Hee (명지대학교 문헌정보학과)
Publication Information
Journal of the Korean Society for information Management / v.32, no.3, 2015 , pp. 377-396 More about this Journal
Abstract
This study proposed a key-frame automatic extraction method for video storyboard surrogates based on users' cognitive responses, EEG signals and discriminant analysis. Using twenty participants, we examined which ERP pattern is suitable for each step, assuming that there are five image recognition and process steps (stimuli attention, stimuli perception, memory retrieval, stimuli/memory comparison, relevance judgement). As a result, we found that each step has a suitable ERP pattern, such as N100, P200, N400, P3b, and P600. Moreover, we also found that the peak amplitude of left parietal lobe (P7) and the latency of FP2 are important variables in distinguishing among relevant, partial, and non-relevant frames. Using these variables, we conducted a discriminant analysis to classify between relevant and non-relevant frames.
Keywords
EEG; ERP; key-frame; discriminant analysis; relevance;
Citations & Related Records
Times Cited By KSCI : 3  (Citation Analysis)
연도 인용수 순위
1 권준수 (2000). 인지기능연구에서의 사건관련전위의 이용. 인지과학작업, 1(1), 79-98. (Kwon, J. (2000). The use of event-related potentials in the study of cognitive functions. Journal of Cognitive Science, 1(1), 79-98.)
2 김명선, 조상수, 권준수 (2000). 즉각적 재인 기억과 지연 재인 기억이 사건관련전위에 미치는 영향. 한국인지과학회논문지, 11(3-4), 83-93. (Kim, M., Cho, S., & Kwon, J. (2000). The effect of immediate and delayed recognition memory on event-related potential (ERP). Korean Journal of Cognitive Science, 11(3-4), 83-93.)
3 김현희, 김용호 (2015). 이용자 기반의 비디오 키프레임 자동 추출을 위한 뇌파측정기술(EEG) 적용. 한국문헌정보학회지, 49(1), 443-464. http://dx.doi.org/10.4275/KSLIS.2015.49.1.443 (Kim, H., & Kim, Y. (2015). Toward a key-frame extraction framework for video storyboard surrogates based on users' EEG signals. Journal of the Korean Society for Library and Information Science, 49(1), 443-464. http://dx.doi.org/10.4275/KSLIS.2015.49.1.443)   DOI
4 이지영 (2006). 뇌 연구방법론을 통해 살펴본 음악 처리과정 연구: 음악과 언어, 음악과 정서를 중심으로. 낭만음악, 18(3), 69-146. (Lee, J. (2006). Neurophysiology and brain-imaging study of music-music & language, music & emotion-. Nang Man Music, 18(3), 69-146.)
5 이충연 외 (2011). EEG 기반 뇌기능 분석을 이용한 영화 장면- 대사 기억 게임에서의 인지 학습 특성. 한국정보과학회, 38(1), 210-213. (Lee, C. et al. (2011). Properties of human cognitive learning in a movie scene-dialogue memory game using EEG-based brain function analysis. Journal of Computing Science and Engineering, 38(1), 210-213.)
6 임용수, 이승환, 홍석인 (2010). 정신분열병 환자에서 생물학적 지표로서 N100, P300과 정량화뇌파의 적용. 대한정신약물학회지, 21(2), 78-86. (Lim, Y., Lee, S., & Hong, S. (2010). Application of N100, P300 and QEEG as a biological marker in patients with schizophrenia. The Korean Journal of Psychopharmacology, 21(2), 78-86.)
7 장윤석, 한재웅 (2014). 시각자극 과제에 의한 집중 시의 뇌파분석. 한국전자통신학회논문지, 9(5), 589-594. http://dx.doi.org/10.13067/JKIECS.201.9.5.589 (Jang, Y., & Han, J. (2014). Analysis of EEG generated from concentration by visual stimulus task. The Journal of the Korea institute of electronic communication sciences, 9(5), 589-594. http://dx.doi.org/10.13067/JKIECS.201.9.5.589)   DOI
8 Andreassi, J. L. (2006). Psychophysiology: Human behavior and physiological response. Psychology Press.
9 Baddeley, A. (1997). Human memory: Theory and practice (revised edition). Psychology Press.
10 Banich, M. T., & Compton, R. J. (2011). Cognitive neuroscience, 3rd Edition. Wadsworth.
11 Banich M. T., & Compton, R. J. (2011). Cognitive neuroscience. 김명선 등역 (2014). 인지 신경과학. 서울: 박학사.
12 Davidson, R. J., & Irwin, W. (1999). The functional neuroanatomy of emotion and affective style. Trends in Cognitive Science, 3, 11-21.   DOI
13 DeFrance, J. F. (1997). Age-related changes in cognitive ERPs of attenuation. Brain Topography, 9(4), 283-293.   DOI
14 Donchin, E., & Coles, M. (1988). Is the P300 component a manifestation of context updating? Behavioral and Brain Science, 11, 357-374.   DOI
15 Elias, L. & Saucier, D. (2005). Neuropsychology: Clinical and experimental foundations. 김명선 역 (2009). 임상 및 실험 신경심리학. 서울: 시그마프레스.
16 Hamann, S. B., & Squire L. R. (1997). Intact perceptual memory in the absence of conscious memory. Behavioral Neuroscience, 111, 850-54.   DOI
17 Hillyard, S., Hink, R., Schwent, V., & Picton, T. (1973). Electrical signs of selective attention in the human brain. Science, 182(4108), 177-179.   DOI
18 Hillyard, S., & Woods, D. (1979). Electrophysiological analysis of human brain function. In M. S. Gazzaniga (Ed.), Handbook of behavioral neurobiology: Vol 2. Neuropsychology (pp. 345-378). New York: Plenum Press.
19 Iyer, H., & Lewis, C. (2007). Prioritization strategies for video storyboard keyframes. Journal of the American Society for Information Science and Technology, 58(5), 629-644.   DOI
20 Jung, H. et al. (2012). Reduced source activity of event-related potentials for affective facial pictures in schizophrenia patients. Schizophrenia Research, 136, 150-159.   DOI
21 Kapoor, A., Pradeep Shenoy, P., & Tan, D. (2008). Combining brain computer interfaces with vision for object categorization. IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), 1-8.
22 Kim, H., & Kim, Y. (2010). Toward a conceptual framework of key-frame extraction and storyboard display for video summarization. Journal of the American Society for Information Science and Technology, 61(5), 927-939.   DOI
23 Koelstra, S., Muehl, C., & Patras, I. (2009). EEG analysis for implicit tagging of video data. Proceeding of the 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops. IEEE Computer Society Press.
24 Kutas, M., & Hillyard, S. A. (1980). Reading senseless sentences: Brain potentials reflect semantic incongruity. Science, 207, 203-208.   DOI
25 Moscovitch, M. (1992). Memory and working-with-memory: A component process model based on modules and central systems. Journal of Cognitive Neuroscience, 4, 257-267.   DOI
26 Naatanen, R., Simpson, M., & Loveless, N. (1982). Stimulus deviance and evoked potentials. Biological Psycology, 14, 53-98.   DOI
27 Naghavi, H., & Nyberg, L. (2005). Common fronto-parietal activity in attention, memory, and consciousness: Shared demands on integration? Consciousness and Cognition, 14(2), 390-425.   DOI
28 Picton, T. (1988). The endogenous evoked potentials. In: E. Basar (Ed.) Dynamics of sensory and cognitive processing by the brain. Springer-Verlag, Berlin; 258-265.
29 Smith, M. E. (1993) Neurophysiological manifestations of recollective experience during recognition memory judgments. Journal of Cognitive Neuroscience, 5, 1-13.   DOI
30 Picton, T., Campbell, K., Barlbeau-Beaun, J., & Proulx, G. (1978). The neurophysiology of human attention: A tutorial review. In J. Requin (Ed). Attention and performance: VII. (pp. 429-467). New York: Wiley.
31 Smith, E. E., & Jonides, J. (1999) Storage and executive processes in the frontal lobes. Science, 283, 1657-1661.   DOI
32 Song, Y., Marchionini, G., & Oh, C. (2010). What are the most eye-catching and ear-catching features in the video? Implications for video summarization. WWW 2010, April 26-30, 2010, Raleigh, North Carolina.
33 Tulving, E. et al. (1994). Hemispheric encoding / retrieval asymmetry in episodic memory: Position emission tomography findings. Proceedings of the National Academy of Sciences of the United States of America, 91(6), 2016-2020.   DOI
34 Yuji Hakoda et al. (2010). Cognitive psychology: Brain, modeling and evidence. 강윤봉 역 (2014). 인지심리학. 서울: 한국뇌기반교육연구소.
35 Wang, Y. (2009). A cognitive informatics theory for visual information processing, Proc. 7th IEEE International Conference on Cognitive Informatics (ICCI' '08), Stanford University, CA.