Browse > Article
http://dx.doi.org/10.4275/KSLIS.2015.49.1.443

Toward a Key-frame Extraction Framework for Video Storyboard Surrogates Based on Users' EEG Signals  

Kim, Hyun-Hee (명지대학교 문헌정보학과)
Kim, Yong-Ho (부경대학교 신문방송학과)
Publication Information
Journal of the Korean Society for Library and Information Science / v.49, no.1, 2015 , pp. 443-464 More about this Journal
Abstract
This study examined the feasibility of using EEG signals and ERP P3b for extracting video key-frames based on users' cognitive responses. Twenty participants were used to collect EEG signals. This research found that the average amplitude of right parietal lobe is higher than that of left parietal lobe when relevant images were shown to participants; there is a significant difference between the average amplitudes of both parietal lobes. On the other hand, the average amplitude of left parietal lobe in the case of non-relevant images is lower than that in the case of relevant images. Moreover, there is no significant difference between the average amplitudes of both parietal lobes in the case of non-relevant images. Additionally, the latency of MGFP1 and channel coherence can be also used as criteria to extract key-frames.
Keywords
EEG; ERP; P3b; Key-frame; Video; Relevance;
Citations & Related Records
Times Cited By KSCI : 6  (Citation Analysis)
연도 인용수 순위
1 고유빈 외. 2012. 뇌 전도를 이용한 집중력 정량화 연구. HCI 2012 학술대회, 23-25.(Koh, Y. et al. 2012. "Study of quantifying concentration index using electroencephalography (EEG)." HCI Society of Korea, 23-25.)
2 권준수. 2000. 인지기능연구에서의 사건관련전위의 이용. 인지과학작업, 1(1): 79-98.(Kwon, J. 2000. "The Use of event-related potentials in the study of cognitive functions." Journal of Cognitive Science, 1(1): 79-98.)
3 권형규. 2011. 컴퓨터 덧셈학습의 인지적 개인차에 대한 암묵적 연합검사를 적용한 뇌파 분석. 정보교육학회논문지, 15(4): 635-644.(Kwon, H. 2011. "Brainwave activities of the cognitive individual differences in computerized arithmetic addition by implicit association test." Journal of the Korean Association of Information Education, 15(4): 635-644.)
4 김종화. 2013. 사회감성 인식을 위한 EEG 코히런스를 사용한 뇌 기능 연결성 분석. 박사학위논문, 상명대학교 감성공학과.(Kim, J. 2013. Recognition of social emotion using EEG coherence analysis. Ph.D Dissertation, Department of Emotion Engineering, Sangmyung University.)
5 김용호. 2006. 뇌파측정기술(EEG)을 이용한 TV 영상 감성반응의 실험 연구. 한국방송학보, 20(1): 7-49.(Kim, Y. 2006. "A study on measuring the lateral specification of brain activity using brainwave measurement(EEG)." Korean Journal of Broadcasting and Telecommunication Studies, 20(1): 7-49.)
6 김현희. 2008. 영상 초록 구현을 위한 키프레임 추출 알고리즘의 설계와 성능 평가. 정보관리학회지, 25(4): 131-148.(Kim, H. 2008. "Design and evaluation of the key-frame extraction algorithm for constructing the Virtual Storyboard Surrogates." Journal of the Korean Society for Information Management, 25(4): 131-148.)   DOI
7 박두흠, 권준수, 정도언. 2001. 기면병에서 전산화 뇌파 지도화 기법으로 측정한 뇌파 동시성 변화. 수면.정신생리, 8(2): 121-128.(Park, D., Kwon, J. and Jeong, D. 2001. "Changes of EEG coherence in narcolepsy measured with computerized EEG mapping technique." Sleep medicine and psychophysiology, 8(2): 121-128.)
8 이성은. 2010. 간접화행의 인지적 이해모델. 독일어 문화권 연구, 19: 133-159.(Lee, S. E. 2010. "Das kognitive Verstehensmodell indirekter Sprechakte." Zeitschrift fur Deutschsprachige Kultur & Literaturen, 19: 133-159.)
9 이지영. 2006. 뇌 연구방법론을 통해 살펴본 음악 처리과정 연구: 음악과 언어, 음악과 정서를 중심으로. 낭만음악, 18(3): 69-146.(Lee, J. 2006. "Neurophysiology and brain-imaging study of music -music & language, music & emotion-." Nang Man Music, 18(3): 69-146.)
10 이충연, 장병탁. 2014. 뇌의 기억 인출에 대한 유효 EEG 연결성 분석. 컴퓨팅의 실제 및 레터, 20(4): 257-261.(Lee, C. and Zhang, B. 2014. "Analysis on effective EEG connectivity of memory retrieval in the brain." Journal of KIISE: Computing Practices and Letters, 20(4): 257-261.)
11 이충연 외. 2011. EEG 기반 뇌기능 분석을 이용한 영화 장면-대사 기억 게임에서의 인지 학습 특성. 한국정보과학회, 38(1): 210-213.(Lee, C. et al. 2011. "Properties of human cognitive learning in a movie scene-dialogue memory game using EEG-based brain function analysis." Journal of Computing Science and Engineering, 38(1): 210-213.)
12 임용수, 이승환, 홍석인. 2010. 정신분열병 환자에서 생물학적 지표로서 N100, P300과 정량화뇌파의 적용. 대한정신약물학회지, 21(2): 78-86.(Lim, Y., Lee, S. and Hong, S. 2010. "Application of N100, P300 and QEEG as a biological marker in patients with schizophrenia." The Korean Journal of Psychopharmacology, 21(2): 78-86.)
13 장윤석, 한재웅. 2014. 시각자극 과제에 의한 집중 시의 뇌파분석. 한국전자통신학회 논문지, 9(5): 589-594.(Jang, Y. and Han, J. 2014. "Analysis of EEG generated from concentration by visual stimulus task." The Journal of the Korea institute of electronic communication sciences, 9(5): 589-594.)
14 Behneman, A. et al. 2009. Enhancing text-based analysis using neurophysiological measures. In D. D. Schmorrow et al. (Eds.), Foundations of Augmented Cognition. Neuroergonomics and Operational Neuroscience. Vol. 5638, 449-458. Berlin, Heidelberg: Springer.
15 Browne, P. and Smeaton, A. F. 2005. "Video retrieval using dialogue, keyframe similarity and video objects." ICIP 2005 - International Conference on Image Processing, Genova, Italy, 11-14.
16 Davidson, R. J. and Irwin, W. 1999. "The functional neuroanatomy of emotion and affective style." Trends in Cognitive Science, 3: 11-21.   DOI
17 DeFrance, J. F. 1997. "Age-related changes in cognitive ERPs of attenuation." Brain Topography, 9(4): 283-293.   DOI
18 Hillyard, S. A. and Woods, D. L. 1979. "Electrophysiological analysis of human brain function." In M. S. Gazzaniga (Ed.), Handbook of behavioral neurobiology: Vol 2. Neuropsychology (pp. 345-378). New York: Plenum Press.
19 Galletti, C. et al. 2003. "Role of the medial parieto-occipital cortex in the control of reaching and grasping movements." Exp Brain Res, 153: 158-170.   DOI
20 Gwizdka, J. et al. 2013. "Applications of neuroimaging in information science: Challenges and opportunities." Proceedings of the American Society for Information Science and Technology, 50(1): 1-4.
21 Iyer, H. and Lewis, C.D. 2007. "Prioritization strategies for video storyboard keyframes." Journal of the American Society for Information Science and Technology, 58(5): 629-644.   DOI
22 Jung, H. et al. 2012. "Reduced source activity of event-related potentials for affective facial pictures in schizophrenia patients." Schizophrenia Research, 136: 150-159.   DOI
23 Kim, H. and Kim, Y. 2010. "Toward a conceptual framework of key-frame extraction and storyboard display for video summarization." Journal of the American Society for Information Science and Technology, 61(5): 927-939.   DOI
24 Koelstra, S., Muehl, C. and Patras, I. 2009. "EEG analysis for implicit tagging of video data." Proceeding of the 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops, ACII 2009, 27-32, Los Alamitos, 2009. IEEE Computer Society Press.
25 Moshfeghi, Y., Pinto, L. R., Pollick, F. R. and Jose, J. M. 2013. "Understanding relevance: An fMRI study." In P. Serdyukov et al., eds. Advances in Information Retrieval. Springer Berlin Heidelberg.
26 Tulving, E. et al. 1994. "Hemispheric encoding / retrieval asymmetry in episodic memory: position emission tomography findings." Proceedings of the National Academy of Sciences of the United States of America, 91(6): 2016-2020.   DOI
27 Naghavi, H. and Nyberg, L. 2005. "Common fronto-parietal activity in attention, memory, and consciousness: Shared demands on integration?" Consciousness and Cognition, 14(2): 390-425.   DOI
28 Song, Y., Marchionini, G. and Oh, C. 2010. "What are the most eye-catching and ear-catching features in the video?" Implications for video summarization. WWW 2010, April 26-30, 2010, Raleigh, North Carolina.
29 Thatcher, R. W., Krause, P. J. and Hrybyk, M. 1986. "Cortico-cortical associations and EEG coherence: a two-compartmental model." Electroencephalogr Clin Neurophysiol, 64: 123-143.   DOI
30 Wang, S., Zhu, Y., Wu, G. and Ji, Q. 2013. "Hybrid video emotional tagging using users' EEG and video content." Multimedia Tools and Applications, 72(2): 1257-1283.   DOI
31 Yang, M. 2005. An exploration of users' video relevance criteria. Unpublished doctoral dissertation, University of North Carolina at Chapel Hill.
32 Yang, M. and Marchionini, G. 2004. Exploring users' video relevance criteria: A pilot study. Proceedings of the ASIST Annual Meeting, 41: 229-238. Medford, NJ: Information Today.