Journal of KIISE:Software and Applications (한국정보과학회논문지:소프트웨어및응용)

Korean Institute of Information Scientists and Engineers (한국정보과학회)
권호 표지

Evolutionary Algorithms with Distribution Estimation by Variational Bayesian Mixtures of Factor Analyzers

변분 베이지안 혼합 인자 분석에 의한 분포 추정을 이용하는 진화 알고리즘

  • 조동연 (서울대학교 컴퓨터공학부) ;
  • 장병탁 (서울대학교 컴퓨터공학부)
  • Published : 2005.11.01
Download PDF
⟨ Previous Next ⟩

Abstract

By estimating probability distributions of the good solutions in the current population, some researchers try to find the optimal solution more efficiently. Particularly, finite mixtures of distributions have a very useful role in dealing with complex problems. However, it is difficult to choose the number of components in the mixture models and merge superior partial solutions represented by each component. In this paper, we propose a new continuous evolutionary optimization algorithm with distribution estimation by variational Bayesian mixtures of factor analyzers. This technique can estimate the number of mixtures automatically and combine good sub-solutions by sampling new individuals with the latent variables. In a comparison with two probabilistic model-based evolutionary algorithms, the proposed scheme achieves superior performance on the traditional benchmark function optimization. We also successfully estimate the parameters of S-system for the dynamic modeling of biochemical networks.

최근 들어 확률 분포를 개체군으로부터 추정하여 보다 효율적으로 최적화를 해결하려는 연구가 진행되고 있다. 특히 복잡한 문제의 해결을 위해서 혼합 분포가 사용되고 있다. 그러나 이 경우 몇 개의 성분으로 혼합 분포를 나타낼 것인가를 결정하기 어려운 문제가 있으며, 각 분포에 의하여 표현되는 이전 세대의 우수한 부분 해들을 잘 결합하지 못하는 단점이 있다. 본 논문에서는 변분 베이지안 혼합 인자 분석(variational Bayesian mixtures of factor analyzers) 기법을 사용한 개체군의 분포 추정을 통해 실수 공간에서의 최적화 문제를 해결하는 방법을 제안한다. 이 기법은 혼합 분포의 개수 추정을 자동화하며, 잠재 변수(latent variable)를 사용하여 각 분포가 표현하는 세부 개체군 내에 포함된 부분 해들의 혼합을 효율적으로 수행할 수 있다. 잘 알려진 함수 최적화 문제들에 대해 다른 분포 추정 진화 알고리즘과 비교하여 제안하는 방법의 우수성을 검증하였다. 또한 시스템 생물학에서 다루고 있는 생화학 네트워크의 동적 모델링을 위한 매개변수 추정도 성공적으로 수행하였다.

Keywords

References

  1. Schwefel, H.-P., Evolution and Optimum Seeking, John Wiley & Sons, New York, 1995
  2. Mulenbein, H. and Paab, G., 'From recombination of genes to the estimation of distribution I. Binary parameters,' Lecture Notes in Computer Science, Vol. 1141, pp. 178-187, 1996 https://doi.org/10.1007/3-540-61723-X_982
  3. Larranaga, P., 'A review on estimation of distribution algorithms,' Estimation of Distribution Algorithms: A New Tools for Evolutionary Computation, pp. 57-100, Kluwer Academic Publisher, 2001
  4. Pelikan, M., Goldberg, D. E., and Lobo, F. G., 'A survey of optimization by building and using probabilistic models,' Computational Optimization and Applications, Vol. 21, No. 1, pp. 5-20, 2002 https://doi.org/10.1023/A:1013500812258
  5. Tsutsui, S., Pelikan, M., and Goldberg, D. E., 'Evolutionary algorithm using marginal histogram in continuous domain,' Proceedings of the GECCO-2001 Workshop Program, pp. 230-233, 2001
  6. Gallagher, M., Frean, M., and Downs, T., 'Realvalued evolutionary optimization using a flexible probability density estimator,' Proceedings of 1999 Genetic and Evolutionary Computation Conference, Vol. 1, pp. 840-846, 1999
  7. Bosman, P. and Thierens, D., 'Advancing continuous IDEAs with mixture distributions and factorization selection metrics,' Proceedings of the GECCO-2001 Workshop Program, pp. 208-212, 2001
  8. Cho, D.-Y. and Zhang, B.-T., 'Evolutionary optimization by distribution estimation with mixtures of factor analyzers,' Proceedings of the 2002 Congress on Evolutionary Computation, Vol. 2, pp. 1396-1401, 2002 https://doi.org/10.1109/CEC.2002.1004447
  9. Mahfoud, S. W., Niching Methods for Genetic Algorithms, Ph.D. thesis, University of Illinois at Urbana-Champaign, 1995
  10. Deb, K. and Spears, W. M., 'Speciation methods,' Evolutionary Computation 2: Advanced Algorithms and Operators, pp. 93-100, Institute of Physics Publishing, 2000
  11. Martin, W. N., Lienig, J., and Cohoon, J. P., 'Island (migration) models: evolutionary algorithms based on punctuated equilibria,' Evolutionary Computation 2: Advanced Algorithms and Operators, pp. 101-124, Institute of Physics Publishing, 2000
  12. Ghahramani, Z. and Beal, M. J., 'Variational inference for Bayesian mixtures of factor analysers,' Advances in Neural Information Processing Systems 12, pp. 449-455, MIT Press, 2000
  13. Duda, R. O., Hart, P. E., and Stork, D. G., 'Unsupervised learning and clustering,' Pattern Classification, pp. 517-599, 2nd Ed., John Wiley & Sons, New York, 2001
  14. Bosman, P. and Thierens, D., 'IDEAs based on the normal kernels probability density function,' Utrecht University, Technical Report UU-CS-2000-11, Mar. 2000
  15. Ocenasek, J. and Schwarz, J., 'Estimation of distribution algorithm for mixed continuousdiscrete optimization,' Proceedings of the 2nd Euro-International Symposium on Computational Intelligence, pp. 227-232, 2002
  16. Pelikan, M. and Goldberg, D. E., 'Escaping hierarchical traps with competent genetic algorithms,' Proceedings of 2001 Genetic and Evolutionary Computation Conference, pp. 511-518, 2001
  17. Kern, S., Muller, S., Hansen, N., Buche, D., Ocenasek, J., and Koumoutsakos, P., 'Learning probability distributions in continuous evolutionary algorithms - a complete review,' Natural Computing, Vol. 3, No. 1, pp. 77-112, 2004 https://doi.org/10.1023/B:NACO.0000023416.59689.4e
  18. Bishop, C. M., 'Latent variable models,' Learning in Graphical Models, pp. 371-403, MIT Press, 1999
  19. Bartholomew, D. J. and Knott, M., Latent Variable Models and Factor Analysis, 2nd Ed., Arnold, London, 1999
  20. Rubin, D. B. and Thayer, D. T., 'EM algorithm for ML factor analysis,' Psychometrika, Vol. 47, No. 1, pp. 69-76, 1982 https://doi.org/10.1007/BF02293851
  21. Ghahramani, Z. and Hinton, G. E., 'The EM algorithm for mixtures of factor analyzers,' Department of Computer Science, University of Toronto, Technical Report CGG-TR-96-1, Feb. 1997
  22. Beal, M. J., Variational Algorithms for Approximate Bayesian Inference, Ph.D. thesis, The Gatsby Computational Neuroscience Unit, University College London, 2003
  23. Green, P. J., 'Reversible jump Markov chain Monte Carlo computation and Bayesian model determination,' Biometrika, Vol. 82, No. 4, pp. 711-732, 1995 https://doi.org/10.1093/biomet/82.4.711
  24. Zhang, B.-T.,
  25. Zhang, Q., Sun, J., and Tsan, E., 'An evolutionary algorithm with guided mutation for the maximum clique problem,' IEEE Transactions on Evolutionary Computation, Vol. 9, No. 2, pp. 192-200, 2005 https://doi.org/10.1109/TEVC.2004.840835
  26. 조동연, 장병탁, '생화학 시스템의 동적 모델링을 위한 S-tree 기반의 진화 연산', 정보과학회 가을학술발표 논문집 (II), 제30권, 제2호, pp. 823-825, 2003
  27. Spieth, C, Streichert, F., Speer, N., and Zell, A., 'Optimizing topology and parameters of gene regulatory network models from time-series experiments,' Lecture Notes in Computer Science, Vol. 3102, pp. 461-470, 2004