Browse > Article
http://dx.doi.org/10.4275/KSLIS.2018.52.4.091

Design and Evaluation of Video Summarization Algorithm based on EEG Information  

Kim, Hyun-Hee (명지대학교 문헌정보학과)
Kim, Yong-Ho (부경대학교 신문방송학과)
Publication Information
Journal of the Korean Society for Library and Information Science / v.52, no.4, 2018 , pp. 91-110 More about this Journal
Abstract
We proposed a video summarization algorithm based on an ERP (Event Related Potentials)-based topic relevance model, a MMR (Maximal Marginal Relevance), and discriminant analysis to generate a semantically meaningful video skim. We then conducted implicit and explicit evaluations to evaluate our proposed ERP/MMR-based method. The results showed that in the implicit and explicit evaluations, the average scores of the ERP / MMR methods were statistically higher than the average score of the SBD (Shot Boundary Detection) method used as a competitive baseline, respectively. However, there was no statistically significant difference between the average score of ERP/MMR (${\lambda}=0.6$) method and that of ERP/MMR (${\lambda}=1.0$) method in both assessments.
Keywords
ERP; ERP/MMR Model; Topic Relevance Model; Video Skim; Implicit Evaluation; Explicit Evaluation;
Citations & Related Records
Times Cited By KSCI : 1  (Citation Analysis)
연도 인용수 순위
1 Kwon, Jun Soo. 2000. "The Use of Event-related Potentials in the Study of Cognitive Functions." Journal of Cognitive Science, 1(1): 79-98.
2 Kim, Hyun-Hee and Kim, Yong-Ho. 2016. "Automatic Extraction Techniques of Topic-Relevant Visual Shots using Real Time Brainwave Responses: ERP N400 and P600 Hypotheses Test." Journal of Korea Multimedia Society, 19(8): 1260-1274.   DOI
3 Chung, Young-Mee. 2012. Research in Information Retrieval. Seoul: Yonsei University Press.
4 Allegretti, M. et al. 2015. "When Relevance Judgement is Happening?: An EEG-based study." In Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval (pp. 719-722). New York: ACM.
5 Burmester, J., Spalek, K. and Wartenburger, I. 2014. "Context Updating during Sentence Comprehension: The Effect of Aboutness Topic." Brain and Language, 137: 62-76.   DOI
6 Carbonell, J. and Goldstein, J. 1998. "The Use of MMR, Diversity-based Reranking for Reordering Documents and Producing Summaries." In Proceedings of the 21st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (pp. 335-336). New York: ACM.
7 Wang, L. and Schumacher, P. B. 2013. "New is not always Costly: Evidence from Online Processing of Topic and Contrast in Japanese." Frontiers in Psychology, 4: 363.
8 Xu, R. et al. 2014. "Enhanced low-latency Detection of Motor Intention from EEG for Closed-loop Brain-computer Interface Applications." IEEE Transactions on Biomedical Engineering, 61(2): 288-296.   DOI
9 Yang, J. et al. 2012. "Channel Selection and Classification of Electroencephalogram Signals: An Artificial Neural Network and Genetic Algorithm-based Approach." Artificial Intelligence in Medicine, 55(2): 117-126.   DOI
10 Zhu, X. et al. 2007. "A Text-to-picture Synthesis System for Augmenting Communication." In Proceedings of the 22nd National Conference on Artificial Intelligence-Volume 2 (pp. 1590-1595). Menlo Park, CA: AAAI Press.
11 Over, P., Smeaton, A. F. and Awad, G. 2008. "The TRECVid 2008 BBC Rushes Summarization Evaluation." In Proceedings of the 2nd ACM TRECVid Video Summarization Workshop (pp. 1-20). New York: ACM.
12 Mostafa, J. and Gwizdka, J. 2016. "Deepening the Role of the User: Neuro-Physiological Evidence as a Basis for Studying and Improving Search." In Proceedings of the 2016 ACM on Conference on Human Information Interaction and Retrieval (pp. 63-70). New York: ACM.
13 Mudrik, L. et al. 2014. "Synchronous Contextual Irregularities Affect Early Scene Processing: Replication and Extension." Neuropsychologia, 56: 447-458.   DOI
14 Nakano, H. et al. 2014. "Electrophysiological Response to Omitted Stimulus in Sentence Processing." NeuroReport, 25(14): 1169-1174.   DOI
15 Schumacher, P. B. and Hung, Y. C. 2012. "Positional Influences on Information Packaging: Insights from Topological Fields in German." Journal of Memory and Language, 67(2): 295-310.   DOI
16 Seoane, L. F., Gabler, S. and Blankertz, B. 2015. "Images from the Mind: BCI Image Evolution based on Rapid Serial Visual Presentation of Polygon Primitives." Brain-Computer Interfaces, 2(1): 40-56.   DOI
17 Sitnikova, T. et al. 2008. "Two Neurocognitive Mechanisms of Semantic Integration during the Comprehension of Visual Real-world Events." Journal of Cognitive Neuroscience, 20(11): 2037-2057.   DOI
18 Tavassolipour, M., Karimian, M. and Kasaei, S. 2014. "Event Detection and Summarization in Soccer Videos using Bayesian Network and Copula." IEEE Transactions on Circuits and Systems for Video technology, 24(2): 291-304.   DOI
19 Mishra, R. et al. 2015. "Real time and Non Real time Video Shot Boundary Detection using Dual Tree Complex Wavelet Transform." 2015 International Conference on Industrial Instrumentation and Control (ICIC, pp. 1495-1500). New York: IEEE.
20 Chen, F., Delannay, D. and De Vleeschouwer, C. 2011. "An Autonomous Framework to Produce and Distribute Personalized Team-sport Video Summaries: A Basketball Case Study." IEEE Transactions on Multimedia, 13(6): 1381-1394.   DOI
21 Eugster, M. J. et al. 2016. "Natural Brain-information Interfaces: Recommending Information by Relevance Inferred from Human Brain Signals." Scientific Reports, 6(38580), 1-10.   DOI
22 Evans, W. J., Cui, L. and Starr, A. 1995. "Olfactory Event-related Potentials in Normal Human Subjects: Effects of Age and Gender." Electroencephalography and Clinical Neurophysiology, 95(4): 293-301.   DOI
23 Hu, W. et al. 2011. "A Survey on Visual Content-based Video Indexing and Retrieval." IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), 41(6): 797-819.   DOI
24 IBM Research Corp. 2016. Morgan | IBM Creates First Movie Trailer by AI. [online] [cited 2018. 8. 3.]
25 Kim, H. H. and Kim, Y. H. 2010. "Toward a Conceptual Framework of Key-frame Extraction and Storyboard Display for Video Summarization." Journal of the Association for Information Science and Technology, 61(5): 927-939.
26 Lin, C. 2004. "ROUGE: A Package for Automatic Evaluation of Summaries." In Proceedings of the Workshop on Text Summarization Branches Out (WAS 2004), Barcelona, Spain.
27 Luck, S. J. 2014. An Introduction to the Event-related Potential Technique. Cambridge: MIT press.
28 Mayer, R. E. 2009. Multimedia Learning. New York: Cambridge University Press.
29 Mehmood, I. et al. 2016. "Divide-and-conquer based Summarization Framework for Extracting Affective Video Content." Neurocomputing, 174: 393-403.   DOI