Journal of KIISE (정보과학회 논문지)

Korean Institute of Information Scientists and Engineers (한국정보과학회)
권호 표지
DOI QR코드

DOI QR Code

Incremental Linear Discriminant Analysis for Streaming Data Using the Minimum Squared Error Solution

스트리밍 데이터에 대한 최소제곱오차해를 통한 점층적 선형 판별 분석 기법

  • 이경훈 (충남대학교 컴퓨터공학과) ;
  • 박정희 (충남대학교 컴퓨터공학과)
  • Received : 2017.07.25
  • Accepted : 2017.11.14
  • Published : 2018.01.15
⟨ Previous Next ⟩

Abstract

In the streaming data where data samples arrive sequentially in time, it is difficult to apply the dimension reduction method based on batch learning. Therefore an incremental dimension reduction method for the application to streaming data has been studied. In this paper, we propose an incremental linear discriminant analysis method using the least squared error solution. Instead of computing scatter matrices directly, the proposed method incrementally updates the projective direction for dimension reduction by using the information of a new incoming sample. The experimental results demonstrate that the proposed method is more efficient compared with previously proposed incremental dimension reduction methods.

시간에 따라 순차적으로 들어오는 스트리밍 데이터에서는 전체 데이터 셋을 한꺼번에 모두 이용하는 배치 학습에 기반한 차원축소 기법을 적용하기 어렵다. 따라서 스트리밍 데이터에 적용하기 위한 점층적 차원 감소 방법이 연구되어왔다. 이 논문에서는 최소제곱오차해를 통한 점층적 선형 판별 분석법을 제안한다. 제안 방법은 분산행렬을 직접 구하지 않고 새로 들어오는 샘플의 정보를 이용하여 차원 축소를 위한 사영 방향을 점층적으로 업데이트한다. 실험 결과는 이전에 제안된 점층적 차원축소 알고리즘과 비교하여 이 논문에서 제안한 방법이 더 효과적인 방법임을 입증한다.

Keywords

Acknowledgement

Supported by : 한국연구재단

References

  1. R. O. duda, P. E. Hart, and D. G. Stork, Pattern Classification, 2nd Ed., Wiley-interscience, New York, 2001.
  2. J. Weng, Y. Zhang, and W. S. Hwang, "Candid covariance-free incremental principal component analysis," IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 25, No. 8, pp. 1034-1040, 2003. https://doi.org/10.1109/TPAMI.2003.1217609
  3. Y. Zhang, and J. Weing, "Convergence analysis of complementary candid incremental principal component analysis," Michigan State University, 2001.
  4. S. Pang, S. Ozawa, and N. Kasabov, "Incremental linear discriminant analysis for classification of data streams," IEEE transactions on Systems, Man, and Cybernetics, part B(Cybernetics), Vol. 35, No. 5, pp. 905-914, 2005. https://doi.org/10.1109/TSMCB.2005.847744
  5. R. K. Agrawal, and A. Chaudhary, "Modified incremental linear discriminant analysis for face recognition," Bharati Vidyapeeth's Institute of Computer Applications and Management, pp. 37, 2009.
  6. H. Zhao, and P. C. Yuen, "Incremental linear discriminant analysis for face recognition," IEEE Transactions on Systems, Man, and Cybernetics, Part B(Cybernetics), Vol. 38, No. 1, pp. 210-221, 2008. https://doi.org/10.1109/TSMCB.2007.908870
  7. L. P. Liu, Y. jiang, and Z. H. Zhou, "Least square incremental linear discriminant analysis," Data Mining, 2009. ICDM'09. Ninth IEEE International Conference on, pp. 298-306, 2009.
  8. Y. R. Yeh, and Y. C. F Wang, "A rank-one update method for least squares linear discriminant analysis with concept drift," Pattern recognition, Vol. 46, No. 5, pp. 1267-1276, 2013. https://doi.org/10.1016/j.patcog.2012.11.008
  9. J. Yan, B. Zhang, S. Yan, Q. Yang, H. Li, Z. Chen, W. Xi, W. Fan, W. Y. Ma, and Q. Cheng, "IMMC: incremental maximum margin criterion," proc. of the tenth ACM SIGKDD international conference on Knowledge discovery, and data minig, pp. 725-730, 2004.
  10. J. Yan, B. Zhang, N. Liu, S. Yan, Q. Cheng, W. Fan, Q. Yang, W. Xi, and Z. Chen, "Effective and efficient dimensionality reduction for large-scale and streaming data preprocessing," IEEE transactions on Knowledge and Data Engineering, Vol. 18, No. 3, pp. 320-333, 2006. https://doi.org/10.1109/TKDE.2006.45
  11. Q. Zeng, and G. Z. Li, "Incremental partial least squares analysis of big streaming data," Pattern Recognition, Vol. 47, No. 11, pp. 3726-3735, 2014. https://doi.org/10.1016/j.patcog.2014.05.022
  12. C. H. Park, and H. Park, "A relationship between linear discriminant analysis and the generalized minimum sqaured error solution," SIAM Journal on Matrix Analysis and Applications, Vol. 27, No. 2, pp. 474-492, 2005. https://doi.org/10.1137/040607599
  13. C. H. Park, "Application of adaptive incremental classifier for streaming data," Journal of KIISE, Vol. 43, No. 12, pp. 1396-1403, 2016. https://doi.org/10.5626/JOK.2016.43.12.1396
  14. W. Hager, "Updating the inverse of a matrix," SIAM review, Vol. 31, No. 2, pp. 221-239, 1989. https://doi.org/10.1137/1031049
  15. http://www.cs.waikato.ac.nz/ml/weka
  16. UCI machine learning repository
  17. http://people.csail.mit.edu/jrennie/20Newsgroups