Journal of the Institute of Electronics Engineers of Korea CI (전자공학회논문지CI)

The Institute of Electronics and Information Engineers (대한전자공학회)
권호 표지

Approximation of Polynomials and Step function for cosine modulated Gaussian Function in Neural Network Architecture

뉴로 네트워크에서 코사인 모듈화 된 가우스함수의 다항식과 계단함수의 근사

  • Lee, Sang-Wha (Department of Information and Communication, Seowon University)
  • 이상화 (서원대학교 정보통신공학과)
  • Received : 2011.10.27
  • Accepted : 2012.03.05
  • Published : 2012.03.25
Download PDF
⟨ Previous Next ⟩

Abstract

We present here a new class of activation functions for neural networks, which herein will be called CosGauss function. This function is a cosine-modulated gaussian function. In contrast to the sigmoidal-, hyperbolic tangent- and gaussian activation functions, more ridges can be obtained by the CosGauss function. It will be proven that this function can be used to aproximate polynomials and step functions. The CosGauss function was tested with a Cascade-Correlation-Network of the multilayer structure on the Tic-Tac-Toe game and iris plants problems, and results are compared with those obtained with other activation functions.

본 논문에서는 CosGauss라고 하는 코사인함수로 모듈화 된 가우시안 활성화함수가 뉴로 네트워크에서 다항식과 계단함수의 근사에 사용될 수 있음을 증명한다. CosGauss 함수는 시그모이드, 하이퍼볼릭 탄젠트, 가우시안 활성화 함수보다 더 많은 범프(bump)를 구성 할 수 있다. 이 함수를 캐스케이드 코릴레이션 뉴로 네트워크 학습에 사용하여 벤치마크 문제인 Tic-Tac-Toe 게임과 아이리스(iris) 식물 문제와 실험하고 여기에서 얻어진 결과를 다른 활성화 함수를 사용한 결과와 비교 분석한다.

Keywords

References

  1. B. DasGupta und G. Schnitger, "The Power of Approximating: a Comparison of Activation Functions", Advances in Neural Information Processing Systems5, Edited by R. P. Lippmann, J. E. Moody and D. S. Touretzky, Morgan Kaufmann, pp. 615-622, 1993.
  2. B. DasGupta und G. Schnitger, "Efficient approximation with neural networks: A comparison of gate functions", Pennsylvania State University, 1993.
  3. S. E. Fahlman and C. Lebiere, "The cascadecorrelation learning architecture," Advances in Neural Information Processing Systems 2, Morgan Kaufmann, 1990.
  4. G. W. Flake, "Nonmonotonic activation functions in multilayer perceptrons", Dissertation, Institute for Advance Computer Studies department of Computer Science University of Maryland, 1993.
  5. B. Hammer, A. Micheli and A. Sperduti, "Universal approximation capability of cascade correlation for structures," Neural Computation, Vol.17, No.5, pp.1109-1159, 2005. https://doi.org/10.1162/0899766053491878
  6. L. Prechelt, "PROBEN1-A Set of Neural Network Benchmark Problems and Benchmarking Rules," Technical Report 21/94, Department of Computer Science, University of Karlsruhe, 1999.
  7. W. Walter, "AnalysisI", zweite Auflage, Springer- Verlag, pp.103, 1990.