[KSCI] Korea Science Citation Index Service

http://dx.doi.org/10.3837/tiis.2020.04.008

Decoding Brain Patterns for Colored and Grayscale Images using Multivariate Pattern Analysis

Zafar, Raheel (Department of Engineering, National University of Modern Languages)
Malik, Muhammad Noman (Department of Computer Science, National University of Modern Languages)
Hayat, Huma (Department of Engineering, National University of Modern Languages)
Malik, Aamir Saeed (Centre for Intelligent Signal & Imaging Research (CISIR), Universiti Teknologi PETRONAS)

Publication Information

KSII Transactions on Internet and Information Systems (TIIS) / v.14, no.4, 2020 , pp. 1543-1561 More about this Journal

Abstract

Taxonomy of human brain activity is a complicated rather challenging procedure. Due to its multifaceted aspects, including experiment design, stimuli selection and presentation of images other than feature extraction and selection techniques, foster its challenging nature. Although, researchers have focused various methods to create taxonomy of human brain activity, however use of multivariate pattern analysis (MVPA) for image recognition to catalog the human brain activities is scarce. Moreover, experiment design is a complex procedure and selection of image type, color and order is challenging too. Thus, this research bridge the gap by using MVPA to create taxonomy of human brain activity for different categories of images, both colored and gray scale. In this regard, experiment is conducted through EEG testing technique, with feature extraction, selection and classification approaches to collect data from prequalified criteria of 25 graduates of University Technology PETRONAS (UTP). These participants are shown both colored and gray scale images to record accuracy and reaction time. The results showed that colored images produces better end result in terms of accuracy and response time using wavelet transform, t-test and support vector machine. This research resulted that MVPA is a better approach for the analysis of EEG data as more useful information can be extracted from the brain using colored images. This research discusses a detail behavior of human brain based on the color and gray scale images for the specific and unique task. This research contributes to further improve the decoding of human brain with increased accuracy. Besides, such experiment settings can be implemented and contribute to other areas of medical, military, business, lie detection and many others.

Keywords

Multivariate pattern analysis (MVPA); Human Brain; Taxonomy; EEG;

Citations & Related Records

Reference

1	D. Bai, T. Qiu, and X. Li, "sample entropy and its application in EEG based epilepsy detection," Sheng wu yi xue gong cheng xue za zhi= Journal of biomedical engineering= Shengwu yixue gongchengxue zazhi, vol. 24, pp. 200-205, 2007.
2	M. Weippert, M. Behrens, A. Rieger, and K. Behrens, "Sample entropy and traditional measures of heart rate dynamics reveal different modes of cardiovascular control during low intensity exercise," Entropy, vol. 16, pp. 5698-5711, 2014. DOI
3	R. E. Yohanes, W. Ser, and G.-B. Huang, "Discrete Wavelet Transform coefficients for emotion recognition from EEG signals," in Proc. of Engineering in Medicine and Biology Society (EMBC), 2012 Annual International Conference of the IEEE, pp. 2251-2254, 2012.
4	I. Simanova, M. Van Gerven, R. Oostenveld, and P. Hagoort, "Identifying object categories from event-related EEG: toward decoding of conceptual representations," PloS one, vol. 5, p. e14465, 2010. DOI
5	R. Merry, "Wavelet theory and applications: a literature study," 2005.
6	M. Cauchoix, G. Barragan-Jason, T. Serre, and E. J. Barbeau, "The neural dynamics of face detection in the wild revealed by MVPA," The Journal of Neuroscience, vol. 34, pp. 846-854, 2014. DOI
7	R. Zafar, N. Kamel, M. Naufal, A. S. Malik, S. C. Dass, R. F. Ahmad, J. M. Abdullah, and F. Reza, "Decoding of visual activity patterns from fMRI responses using multivariate pattern analyses and convolutional neural network," Journal of integrative neuroscience, vol. 16, pp. 275-289, 2017. DOI
8	A. Gardumi, D. Ivanov, L. Hausfeld, G. Valente, E. Formisano, and K. Uludag, "The effect of spatial resolution on decoding accuracy in fMRI multivariate pattern analysis," NeuroImage, vol. 132, pp. 32-42, 2016. DOI
9	M. Taghizadeh-Sarabi, M. R. Daliri, and K. S. Niksirat, "Decoding objects of basic categories from electroencephalographic signals using wavelet transform and support vector machines," Brain topography, vol. 28, pp. 33-46, 2015. DOI
10	P. Agrawal, D. Stansbury, J. Malik, and J. L. Gallant, "Pixels to voxels: modeling visual representation in the human brain," arXiv preprint arXiv:1407.5104, 2014.
11	R. Zafar, S. C. Dass, A. S. Malik, N. Kamel, M. J. U. Rehman, R. F. Ahmad, J. M. Abdullah, and F. Reza, "Prediction of human brain activity using likelihood ratio based score fusion," IEEE access, vol. 5, pp. 13010-13019, 2017. DOI
12	R. Zafar, N. Kamel, M. Naufal, A. S. Malik, S. C. Dass, R. F. Ahmad, J. M. Abdullah, and F. Reza, "A study of decoding human brain activities from simultaneous data of EEG and fMRI using MVPA," Australasian physical & engineering sciences in medicine, vol. 41, pp. 633-645, 2018. DOI
13	R. Zafar, A. S. Malik, N. Kamel, S. C. Dass, J. M. Abdullah, F. Reza, and A. H. Abdul Karim, "Decoding of visual information from human brain activity: A review of fMRI and EEG studies," Journal of integrative neuroscience, vol. 14, no. 2, pp. 155-168, 2015. DOI
14	J.-D. Haynes and G. Rees, "Decoding mental states from brain activity in humans," Nature Reviews Neuroscience, vol. 7, pp. 523-534, 2006. DOI
15	M. Lewis and A. Edmonds, "Searching for faces in scrambled scenes," Visual Cognition, vol. 12, pp. 1309-1336, 2005. DOI
16	S. M. McClure, J. Li, D. Tomlin, K. S. Cypert, L. M. Montague, and P. R. Montague, "Neural correlates of behavioral preference for culturally familiar drinks," Neuron, vol. 44, pp. 379-387, 2004. DOI
17	G. Van Doorn and B. Miloyan, "The Pepsi Paradox: A review," Food quality and preference, vol. 65, pp. 194-197, 2018. DOI
18	P. Baldi and A. D. Long, "A Bayesian framework for the analysis of microarray expression data: regularized t-test and statistical inferences of gene changes," Bioinformatics, vol. 17, pp. 509-519, 2001. DOI
19	T. N. Tran, R. Wehrens, and L. M. Buydens, "Clustering multispectral images: a tutorial," Chemometrics and Intelligent Laboratory Systems, vol. 77, pp. 3-17, 2005. DOI
20	S. M. Crouzet, N. A. Busch, and K. Ohla, "Taste quality decoding parallels taste sensations," Current Biology, vol. 25, pp. 890-896, 2015. DOI
21	K. N. Kay, J. Winawer, A. Mezer, and B. A. Wandell, "Compressive spatial summation in human visual cortex," Journal of Neurophysiology, vol. 110, no. 2, pp. 481-494, 2013. DOI
22	A. Subasi and M. I. Gursoy, "EEG signal classification using PCA, ICA, LDA and support vector machines," Expert Systems with Applications, vol. 37, no. 12, pp. 8659-8666, 2010. DOI
23	"Deceiving the law," Nat Neuroscience, vol. 11, pp. 1231-1231, 2008. DOI
24	M. Akin, "Comparison of wavelet transform and FFT methods in the analysis of EEG signals," Journal of Medical Systems, vol. 26, pp. 241-247, 2002. DOI
25	M. Murugappan, N. Ramachandran, and Y. Sazali, "Classification of human emotion from EEG using discrete wavelet transform," Journal of Biomedical Science and Engineering, vol. 3, p. 390, 2010. DOI
26	R. Zafar, S. C. Dass, and A. S. Malik, "Electroencephalogram-based decoding cognitive states using convolutional neural network and likelihood ratio based score fusion," PloS one, vol. 12, p. e0178410, 2017. DOI
27	M. Bindemann and A. M. Burton, "The role of color in human face detection," Cognitive Science, vol. 33, no. 6, pp. 1144-1156, 2009. DOI
28	Z. Lu, B. Guo, A. Boguslavsky, M. Cappiello, W. Zhang, and M. Meng, "Distinct effects of contrast and color on subjective rating of fearfulness," Frontiers in psychology, vol. 6, p. 1521, 2015. DOI
29	M. A. Rahman, W. Ma, D. Tran, and J. Campbell, "A comprehensive survey of the feature extraction methods in the EEG research," in Proc. of International Conference on Algorithms and Architectures for Parallel Processing, pp. 274-283, 2012.
30	K. N. Kay, T. Naselaris, R. J. Prenger, and J. L. Gallant, "Identifying natural images from human brain activity," Nature, vol. 452, pp. 352-355, 2008. DOI
31	S. Freiders, S. Lee, D. Statz, and T. Kim, "The Influence of Color on Physiological Response,".
32	S. Sanei and J. A. Chambers, EEG signal processing, John Wiley & Sons, 2013.
33	M. W. Self and S. Zeki, "The integration of colour and motion by the human visual brain," Cerebral Cortex, vol. 15, pp. 1270-1279, 2004. DOI
34	A. S. Al-Fahoum and A. A. Al-Fraihat, "Methods of EEG Signal Features Extraction Using Linear Analysis in Frequency and Time-Frequency Domains," International Scholarly Research Notices, vol. 2014, 2014.
35	M. Hajinoroozi, T.-P. Jung, C.-T. Lin, and Y. Huang, "Feature extraction with deep belief networks for driver's cognitive states prediction from EEG data," in Proc. of Signal and Information Processing (ChinaSIP), 2015 IEEE China Summit and International Conference on, pp. 812-815, 2015.
36	W. Zhu, J. Drewes, and K. R. Gegenfurtner, "Animal detection in natural images: effects of color and image database," PLoS One, vol. 8, p. e75816, 2013. DOI
37	EGI, "Net Station 5 Software," 2016. Available at: https://www.egi.com/research-division/geodesic-eeg-system-components/eeg-software.
38	T. Naselaris, R. J. Prenger, K. N. Kay, M. Oliver, and J. L. Gallant, "Bayesian reconstruction of natural images from human brain activity," Neuron, vol. 63, no. 6, pp. 902-915, 2009. DOI
39	J. A. Mumford, B. O. Turner, F. G. Ashby, and R. A. Poldrack, "Deconvolving BOLD activation in event-related designs for multivoxel pattern classification analyses," NeuroImage, vol. 59, no. 3, pp. 2636-2643, 2012. DOI
40	C. Kanan and G. W. Cottrell, "Color-to-grayscale: does the method matter in image recognition?," PloS one, vol. 7, p. e29740, 2012. DOI
41	R. Haddadi, E. Abdelmounim, M. E. Hanine, and A. Belaguid, "Discrete Wavelet Transform Based Algorithm for Recognition of QRS Complexes," in Proc. of 2014 International Conference on Multimedia Computing and Systems (ICMCS), 2014.
42	G. J. Jiang, S.-Z. Fan, M. F. Abbod, H.-H. Huang, J.-Y. Lan, F.-F. Tsai, H.-C. Chang, Y.-W. Yang, F.-L. Chuang, and Y.-F. Chiu, "Sample entropy analysis of EEG signals via artificial neural networks to model patients' consciousness level based on anesthesiologists experience," BioMed research international, vol. 2015, 2015.