Journal of applied mathematics & informatics

The Korean Society for Computational and Applied Mathematics (한국전산응용수학회)
권호 표지
DOI QR코드

DOI QR Code

CONVERGENCE ANALYSIS OF THE EAPG ALGORITHM FOR NON-NEGATIVE MATRIX FACTORIZATION

  • Yang, Chenxue (Department of Computer Science and Engineering, University of Electronic Science and Technology of China, P.R. China and State Key Lab. for Novel Software Technology Nanjing University) ;
  • Ye, Mao (Department of Computer Science and Engineering, University of Electronic Science and Technology of China, P.R. China and State Key Lab. for Novel Software Technology Nanjing University)
  • Received : 2011.04.10
  • Accepted : 2011.09.10
  • Published : 2012.05.30
Download PDF
⟨ Previous Next ⟩

Abstract

Non-negative matrix factorization (NMF) is a very efficient method to explain the relationship between functions for finding basis information of multivariate nonnegative data. The multiplicative update (MU) algorithm is a popular approach to solve the NMF problem, but it fails to approach a stationary point and has inner iteration and zero divisor. So the elementwisely alternating projected gradient (eAPG) algorithm was proposed to overcome the defects. In this paper, we use the fact that the equilibrium point is stable to prove the convergence of the eAPG algorithm. By using a classic model, the equilibrium point is obtained and the invariant sets are constructed to guarantee the integrity of the stability. Finally, the convergence conditions of the eAPG algorithm are obtained, which can accelerate the convergence. In addition, the conditions, which satisfy that the non-zero equilibrium point exists and is stable, can cause that the algorithm converges to different values. Both of them are confirmed in the experiments. And we give the mathematical proof that the eAPG algorithm can reach the appointed precision at the least iterations compared to the MU algorithm. Thus, we theoretically illustrate the advantages of the eAPG algorithm.

Keywords

References

  1. D.D. Lee, and H.S. Seung, Learning the parts of objects by non-negative matrix factorization, Nature 401 (2002), 788-791.
  2. P. Paatero, and U. Tapper, Positive matrix factorization: A non-negative factor model with optimal utilization of error, Environmetrics 5 (1994), 111-126. https://doi.org/10.1002/env.3170050203
  3. D.D. Lee, and H.S. Seung, Algorithms for Non-negative Matrix Factorization, Advances in Neural Information Processing, MIT Press 13 (2001), 556-562.
  4. M. Berry, and M. Browne, Algorithms and applications for approximate nonnegative matrix factorization, Computational Statistics and Data Analysis 52 (2006), 155-173.
  5. M. Chu, and F. Diele, Optimality, computation, and interpretation of nonnegative matrix factorizations, Siam Journal on Matrix Analysis 4 (2004), 8030-8048.
  6. P.M. Kim, and B. Tidor, Subsystem identification through dimensionality reduction of largescale gene expression data, Genome Research 13 (2003), 1706-1718. https://doi.org/10.1101/gr.903503
  7. L.K. Saul, F. Sha and D.D. Lee, Statistical signal processing with nonnegativity constraints, Proceedings of EuroSpeech 2 (2003), 1001-1004.
  8. F. Shahnaz, M.W. Berry, V.P. Pauca, and R. Plemmons, Document clusting using nonnegative matrix factorization, Information Processing and Management 42 (2006), 373-386. https://doi.org/10.1016/j.ipm.2004.11.005
  9. E.F. Gonzales, and Y. Zhang, Accelerating the Lee-Seung algorithm for non-negative matrix factorization, Dept. Comput. Appl. Math., Rice Univ., Houston, TX, Tech. Rep ,2005.
  10. V.P. Pauca, J. Piper, and R.J. Plemmons, Nonnegative matrix factorization for spectral data analysis, Linear Algebra and its Applications 416 (2006), 29-47. https://doi.org/10.1016/j.laa.2005.06.025
  11. F. Sha, and L.K. Saul, Real-time pitch determination of one or more voices by nonnegative matrix factorization, Advances in Neural Information Processing Systems, online papers (2005).
  12. P. Smaragdis, and J.C. Brown, Non-negative matrix factorization for polyphonic music transcription, Proceedings of IEEEWorkshop on Applications of Signal Processing to Audio and Acoustics (2003), 177-180.
  13. L. Lin, Alternative gradient algorithms with applications to nonnegative matrix factorizations, Applied Mathematics and Computation (2010), 1763-1770.
  14. A. Cichocki, R. Zdunek, and S. Amari, New algorithms for non-negative matrix factorization in applications to blind source separation, ICASSP, Toulouse, France (2006), 621-625.
  15. C.J. Lin, Projected gradient methods for non-negative matrix factorization, Tech. Report Information and Support Service ISSTECH (2005), 95-013.
  16. S.M. Yang, and Z. Yi, Convergence analysis of non-negative matrix factorization for BSS algorithm, Neural Process Lett (2010), 45-64.
  17. A. Cichocki, and R. Zdunek, Multilayer non-negative matrix factorization using project gradient approaches, International Journal of Neural Systems (2007), 431-446.
  18. R. Salakhutdinov, S.T. Roweis, and Z. Ghahramani, The convergence of bound optimization algorithms, International Conference on Uncertainty in Artificial Intelligence 19 (2003).
  19. C.J. Lin, On the convergence of multiplicative update algorithms for nonnegative matrix factorization,Transactions on Neural Networks 18 (2007), 1589-1596. https://doi.org/10.1109/TNN.2007.895831
  20. V.L. Kocic, and G. Ladas, Global behavior of nonlinear difference equations of higher order, Kluwer Academic Publishers, 1953.
  21. J.K. Hale, Theory of functional differential equations, Lecture Notes in Mathematics (1971), 183-1971.
  22. J. Dai, and S. Meyn, Stabilitity and convergence of moments for multiclass queueing networks via fluid limit models, Translation on Automatic Control (1995), 1889-1904.
  23. Y.B. Liu, A novel fast multi-objective evolutionary algorithm for QoS multicast routing in manet, International Journal of Computational Intelligence Systems 3 (2009),