Journal of the Korean Operations Research and Management Science Society (한국경영과학회지)

The Korean Operations Research and Management Science Society (한국경영과학회)
권호 표지

The Size Reduction of Artificial Neural Network by Destroying the Connections

연결선 파괴에 의한 인공 신경망의 크기 축소

  • Published : 2002.03.01
Download PDF
⟨ Previous Next ⟩

Abstract

A fully connected Artificial Neural Network (ANN) contains many connections. Compared to the pruned ANN with fewer connections, the fully connected ANN takes longer time to produce solutions end may not provide appropriate solutions to new unseen date. Therefore, by reducing the sloe of ANN, we can overcome the overfitting problem and increase the computing speed. In this research, we reduced the size of ANN by destroying the connections. In other words, we investigated the performance change of the reduced ANN by systematically destroying the connections. Then we found the acceptable level of connection-destruction on which the resulting ANN Performs as well as the original fully connected ANN. In the previous researches on the sloe reduction of ANN, the reduced ANN had to be retrained every time some connections were eliminated. Therefore, It tool lolly time to obtain the reduced ANN. In this research, however, we provide the acceptable level of connection-destruction according to the size of the fully connected ANN. Therefore, by applying the acceptable level of connection-destruction to the fully connected ANN without any retraining, the reduced ANN can be obtained efficiently.

Keywords

References

  1. 한국경영학회지 v.24 no.3 유전적 알고리즘을 이용한 인공신경망의 구조 설계 이재식, 차봉근
  2. CA : Univ. of CA, Dept. of Information and Computer Science UCI Repository of Machine Learning Databases Blake, C., E. Keogh;C.J. Merz,
  3. Neurocomputing Pruning of Basis Functions in Nonlinear Approximators Chandrasekaran, H., H.H. Chen;M.T. Manry
  4. Expert Systems with Applications v.3 Aritificial Neural Networks : Foundations and Application to Decision Problem Hansen, J.V., F. William;J.R. Messier https://doi.org/10.1016/0957-4174(91)90094-U
  5. Proc. of the 11th Int'l Conf Irrelevant Features and the Subset Selection Problem John, G.H., R. Kohavi;K. Pfeger
  6. Advances in Neural Information Processing no.2 Optimal Brain Damage Le Cun, Y., J.S. Denker;S.A. Solla, D.S. Touretzky(ed)
  7. Connection Science v.1 Using Relevance to Reduce Network Size Automatically Mozer, M.C.;P.Smolensky https://doi.org/10.1080/09540098908915626
  8. Expert Systems with Applications v.19 Using Change-point Detection to Support Artificial Neural Networks for Interest Rates Forecasting Oh, K.J.;I.Han https://doi.org/10.1016/S0957-4174(00)00025-7
  9. Applied Statistics v.28 no.2 A Non-parametric Approach to the Change-point Problem Pettitt, A.N. https://doi.org/10.2307/2346729
  10. IEEE Trans. on Neural Networks v.10 no.4 A Formal Selection and Pruning Algorithm for Feedforward Artificial Neural Network Optimization Ponnapalli, P.V.S., K.C. Ho;M. Thomson https://doi.org/10.1109/72.774273
  11. IEEE Trans. on Neural Networks v.8 no.3 Neural-network Feature Selector Setiono, R.;H. Liu https://doi.org/10.1109/72.572104
  12. Neural Networks v.4 Creating Artificial Neural Networks that Generalize Sietsma, J.;R.J.F. Dow https://doi.org/10.1016/0893-6080(91)90033-2
  13. Int'l Journal of Neural Systems v.1 Improving Generalization of Neural Networks through Pruning Thodberg, H.H. https://doi.org/10.1142/S0129065791000352