Browse > Article
http://dx.doi.org/10.5391/JKIIS.2015.25.6.592

Big Data Analysis Using Principal Component Analysis  

Lee, Seung-Joo (Department of Statistics, Cheongju University)
Publication Information
Journal of the Korean Institute of Intelligent Systems / v.25, no.6, 2015 , pp. 592-599 More about this Journal
Abstract
In big data environment, we need new approach for big data analysis, because the characteristics of big data, such as volume, variety, and velocity, can analyze entire data for inferring population. But traditional methods of statistics were focused on small data called random sample extracted from population. So, the classical analyses based on statistics are not suitable to big data analysis. To solve this problem, we propose an approach to efficient big data analysis. In this paper, we consider a big data analysis using principal component analysis, which is popular method in multivariate statistics. To verify the performance of our research, we carry out diverse simulation studies.
Keywords
Big Data; Principal Component Analysis; Eigenvalue; Big Data Analysis; Statistical Analysis;
Citations & Related Records
Times Cited By KSCI : 6  (Citation Analysis)
연도 인용수 순위
1 K. Pearson, "On lines and planes of closest fit to systems of points in space", Phil Mag, vol. 2, pp. 559-572, 1901.   DOI
2 J. Gower, "Some distance properties of latent root and vector methods used in multivariate analysis", Biometrika, vol. 53, pp. 325-338, 1966.   DOI
3 G. Arnold and A. Collins, "Interpretation of transformed axes in multivariate analysis", Applied Statistics, vol. 42, pp. 381-400, 1993.   DOI
4 I. Jolliffe, Principal component analysis, Springer, 2002.
5 M. Oleksiak, J. Roach, and D. Crawford, "Natural variation in cardiac metabolism and gene expression in fundulus heteroclitus", Nature Genetics, vol. 37, pp. 62-72, 2005.
6 Johnson, R. A. and Wichern, D. W., Applied multivariate statistical analysis, Prentice-Hall, NJ, 1982.
7 W. R. Zwick and W. F. Velicer, "Comparison of five rules for determining the number of components to retain", Psychological Bulletin, vol. 99, pp. 432-442, 1986.   DOI
8 M. S. Bartlett, "Tests of significance in factor analysis", British Journal of Psychology, vol. 3, pp. 77-85, 1950.
9 M. S. Bartlett, "A further note on tests of significance in factor analysis", British Journal of Psychology, vol. 4, pp. 1-2, 1951.
10 H. F. Kaiser, "The application of electronic computers to factor analysis", Educational and Psychological Measurement, vol. 20, pp. 141-151, 1960.   DOI
11 R. B. Cattle, "The scree test for the number of factors", Multivariate Behavioral Research, vol. 1, pp. 245-276, 1966.   DOI
12 J. L. Horn, "A rationale and test for the number of factors in factor analysis", Psychometrika, vol. 30, pp. 179-185, 1965.   DOI
13 W. F. Velicer, "Determining the number of components from the matrix of partial correlations", Psychometrika, vol. 41, pp. 321-327, 1976.   DOI
14 J. Han and M. Kamber, Data mining: concepts & techniques, 2nd ed., Elsevier Inc., New York, 2006.
15 S. Jun, "A Big Data Learning for Patent Analysis", Journal of Korean Institute of Intelligent Systems, Vol. 23, No. 5, pp. 406-411, 2013.   DOI
16 B. Choi, J. Kong, and M. Han, "The Model of Network Packet Analysis based on Big Data", Journal of Korean Institute of Intelligent Systems, Vol. 23, No. 5, pp. 392-399, 2013.   DOI
17 K. Kim, J. Jeong, and G. Park, "Assessment of External Force Acting on Ship Using Big Data in Maritime Traffic", Journal of Korean Institute of Intelligent Systems, Vol. 23, No. 5, pp. 379-384, 2013.   DOI
18 S. Hong, and M. Han, "The Efficient Method of Parallel Genetic Algorithm using MapReduce of Big Data", Journal of Korean Institute of Intelligent Systems, Vol. 23, No. 5, pp. 385-391, 2013.   DOI
19 H. C. Cho, and Y. J. Jung, "Probabilistic Modeling of Photovoltaic Power Systems with Big Learning Data Sets", Journal of Korean Institute of Intelligent Systems, Vol. 23, No. 5, pp. 412-417, 2013.   DOI
20 J. H. Cho, D. J. Lee, J. I. Park and M. G. Chun, "Feature Extraction and Classification of High Dimensional Biomedical Spectral Data", Journal of Korean Institute of Intelligent Systems, Vol. 19, No. 3, pp. 297-303, 2009.   DOI
21 W. G. Cochran, Sampling techniques, 3rd ed., New York, Wiley, 1977.
22 W. R. Zwick and W. F. Velicer, "Factors influencing four rules for determining the number of components to retain", Multivariate Behavioral Research, vol. 17, pp. 253-269, 1982.   DOI
23 L. W. Glorfeld, "An improvement on Horn's parallel analysis methodology for selecting the correct number of factors to rertain", Educational and Psychological Measurement, vol. 55, pp. 377-393, 1995.   DOI
24 N. Cliff, "The eigen value greater than one rule and the reliability of components", Psychological Bulletin, vol. 103, pp. 276-279, 1988.   DOI
25 R. L. Gorsuch, Factor analysis, 2nd ed., Lawrence Erlbaum Associates, Inc., 1983.
26 B. P. O'Connor, "SPSS and SAS programs for determining the number og components using parallel analysis and Velicer's MAP test", Behavioral Research Methods Instruments & Computers, vol. 32, pp. 396-402, 2000.   DOI
27 R Development Core Team, R: A language and environment for statistical computing, R Foundation for statistical computing, http://www.R-project.org, 2011.