Journal of KIISE:Software and Applications (한국정보과학회논문지:소프트웨어및응용)

Korean Institute of Information Scientists and Engineers (한국정보과학회)
권호 표지

Construction of Robust Bayesian Network Ensemble using a Speciated Evolutionary Algorithm

종 분화 진화 알고리즘을 이용한 안정된 베이지안 네트워크 앙상블 구축

  • 유지오 (연세대학교 컴퓨터과학과) ;
  • 김경중 (연세대학교 컴퓨터과학과) ;
  • 조성배 (연세대학교 컴퓨터과학과)
  • Published : 2004.12.01
Download PDF
⟨ Previous Next ⟩

Abstract

One commonly used approach to deal with uncertainty is Bayesian network which represents joint probability distributions of domain. There are some attempts to team the structure of Bayesian networks automatically and recently many researchers design structures of Bayesian network using evolutionary algorithm. However, most of them use the only one fittest solution in the last generation. Because it is difficult to combine all the important factors into a single evaluation function, the best solution is often biased and less adaptive. In this paper, we present a method of generating diverse Bayesian network structures through fitness sharing and combining them by Bayesian method for adaptive inference. In order to evaluate performance, we conduct experiments on learning Bayesian networks with artificially generated data from ASIA and ALARM networks. According to the experiments with diverse conditions, the proposed method provides with better robustness and adaptation for handling uncertainty.

베이지안 네트워크는 불확실한 상황을 모델링하기 위한 확률 기반의 모델로서 확실한 수학적 토대를 가지고 있다. 베이지안 네트워크의 구조론 자동 학습하기 위한 연구가 많이 있었고, 최근에는 진화 알고리즘을 이용한 연구가 많이 진행되고 있다. 그러나 대부분은 마지막 세대의 가장 좋은 개체만을 이용하고 있다. 시스템이 요구하는 다양한 요구 조건을 하나의 적합도 평가 수식으로 나타내기 어렵기 때문에, 마지막 세대의 가장 좋은 개체는 종종 편향되거나 변화하는 환경에 덜 적응적일 수 있다. 본 논문에서는 적합도 공유 방법으로 다양한 베이지안 네트워크를 생성하고, 이를 베이즈 규칙을 통해 결합하여 변화하는 환경에 적응적인 추론 모델을 구축할 수 있는 방법을 제안한다. 성능 평가를 위해 ASIA와 ALARM 네트워크에서 인공적으로 생성한 데이타를 이용한 구조 학습 및 추론 실험을 수행하였다. 다양한 조건에서 학습된 네트워크를 실험한 결과, 제안한 방법이 변화하는 환경에서 더욱 강건하고 적응적인 모델을 생성할 수 있음을 알 수 있었다.

Keywords

References

  1. P. Larranaga, M. Poza, Y. Yurramendi, R. H. Murga, and C.M.H. Kuijpers, 'Structure learning of Bayesian networks by genetic algorithms: A performance analysis of control parameters,' IEEE Trans. on Pattern Analysis and Machine Intelligence, vol. 18, no. 9, pp. 912-926, 1996 https://doi.org/10.1109/34.537345
  2. P. Larranaga, C.M.H. Kuijpers, R. H. Murga, and Y. Yurramendi, 'Learning Bayesian network structures by searching for the best ordering with genetic algorithm,' IEEE Trans. on Systems, Man and Cybernetics-Part A, vol. 26, no. 4, pp. 487-493, 1996 https://doi.org/10.1109/3468.508827
  3. M. L. Wong, W. Lam, and K. S. Leung, 'Using evolutionary programming and minimum description length principle for data mining of Bayesian networks,' IEEE Trans. on Pattern Analysis and Machine Intelligence, vol. 21, no. 2, pp. 174-178, 1999 https://doi.org/10.1109/34.748825
  4. W. Lam, 'Bayesian network refinement via machine learning approach,' IEEE Trans. on Pattern Analysis and Machine Intelligence, vol. 20, no. 3, pp. 240-251, 1998 https://doi.org/10.1109/34.667882
  5. S. Yang, and K.-C. Chang, 'Comparison of score metrics for Bayesian network learning,' IEEE Trans. on Systems, Man and Cybernetics-Part A, vol. 32, no. 3, pp. 419-428, 2002 https://doi.org/10.1109/TSMCA.2002.803772
  6. D. M. Chickering, D. Geiger, and D. Heckerman, 'Learning Bayesian networks is NP-complete,' Learning from Data: Artificial Intelligence and Statistics V (D. Fisher, H. Lenz (Eds.)), Springer-Verlag, pp. 121-130, 1996
  7. R. W. Robinson, 'Counting unlabeled acyclic digraphs,' Lecture Notes in Mathematics 622: Combinatorial Mathematics V, pp. 28-43, 1977
  8. Z. Kim, and R. Nevatia, 'Expandable Bayesian networks for 3D object description from multiple views and multiple mode inputs,' IEEE Trans. on Pattern Analysis and Machine Intelligence, vol. 25, no. 6, pp. 769-774, 2003 https://doi.org/10.1109/TPAMI.2003.1201825
  9. R. Etxeberria, P. Larranaga and J. M. Picaza, 'Analysis of the behaviour of genetic algorithms when learning Bayesian network structure from data,' Pattern Recognition Letters, vol. 18, no. 11-13, pp. 1269-1273, Nov 1997 https://doi.org/10.1016/S0167-8655(97)00106-2
  10. G. Li, F. Tong, and H. Dai, 'Evolutionary structure learning algorithm for Bayesian network and penalized mutual information metric,' Proc. of IEEE Int. Conf. on Data Mining, pp. 615-616, 2001 https://doi.org/10.1109/ICDM.2001.989580
  11. C. Cotta, and J. Muruzabal, 'Towards a more efficient evolutionary induction of Bayesian networks,' Lecture Notes in Computer Science 2439, pp. 730-739, 2002
  12. H. C. Andersen, An Investigation into Genetic Algorithms, and the Relationship Between Speciation and the Tracking of Optima in Dynamic Functions, Honours thesis, Queensland University of Technology, Brisbane, Australia, November 1991
  13. W. Cedeno, and V. R. Vemuri, 'On the use of niching for dynamic landscapes,' IEEE Int. Conf. on Evolutionary Computation, pp. 361-366, 1997 https://doi.org/10.1109/ICEC.1997.592336
  14. A. Ghosh, S. Tstutsui, and H. Tanaka, 'Function optimization in non-stationary environment using steady state genetic algorithms with aging of individuals,' IEEE Int. Conf. on Evolutionary Computation, pp. 666-671, 1998 https://doi.org/10.1109/ICEC.1998.700119
  15. A. Garg, V. Pavlovic and J. M. Rehg, 'Boosted learning in dynamic Bayesian networks for multimodal speaker detection,' Proc. of the IEEE, vol. 91, no. 9, pp. 1355-1369, 2003 https://doi.org/10.1109/JPROC.2003.817119
  16. V. Robles, P. Larranaga, J. M. Pena, E. Menasalvas, M. S. Perez, and V. Herves, 'Bayesian networks as consensed voting system in the construction of a multi-classifier for protein secondary structure prediction,' Artificial Intelligence in Medicine, 2004. (in press)
  17. G. F. Cooper and E. Herskovits, 'A Bayesian method for the induction of probabilistic networks from data,' Machine Learning, vol. 7, pp. 299-347, 1992
  18. S. L. Lauritzen and D. J. Spiegelhalter, 'Local computations with probabilities on graphical structures and their applications on expert systems,' Journal Royal Statistical Society B, vol. 50, no. 2, pp. 157-224, 1988
  19. I. A. Beinlinch, H. J. Suermondt, R. M. Chavez and G. F. Cooper, 'The ALARM monitoring system: A case study with two probabilistic inference techniques for belief networks,' Proc. of the Second European Conf. on Artificial Intelligence in Medicine, pp. 247-256, 1989
  20. I. Good, 'Rational decisions,' Journal of the Royal Statistical Society B, vol. 14, pp. 107-114, 1952