The KIPS Transactions:PartB (정보처리학회논문지B)

Korea Information Processing Society (한국정보처리학회)
권호 표지
DOI QR코드

DOI QR Code

MLP Design Method Optimized for Hidden Neurons on FPGA

FPGA 상에서 은닉층 뉴런에 최적화된 MLP의 설계 방법

  • 경동욱 (숭실대학교 IT대학 미디어학부 미디어공학) ;
  • 정기철 (숭실대학교 IT대학 미디어학부)
  • Published : 2006.08.01
Download PDF
⟨ Previous Next ⟩

Abstract

Neural Networks(NNs) are applied for solving a wide variety of nonlinear problems in several areas, such as image processing, pattern recognition etc. Although NN can be simulated by using software, many potential NN applications required real-time processing. Thus they need to be implemented as hardware. The hardware implementation of multi-layer perceptrons(MLPs) in several kind of NNs usually uses a fixed-point arithmetic due to a simple logic operation and a shorter processing time compared to the floating-point arithmetic. However, the fixed-point arithmetic-based MLP has a drawback which is not able to apply the MLP software that use floating-point arithmetic. We propose a design method for MLPs which has the floating-point arithmetic-based fully-pipelining architecture. It has a processing speed that is proportional to the number of the hidden nodes. The number of input and output nodes of MLPs are generally constrained by given problems, but the number of hidden nodes can be optimized by user experiences. Thus our design method is using optimized number of hidden nodes in order to improve the processing speed, especially in field of a repeated processing such as image processing, pattern recognition, etc.

일반적으로 신경망은 비선형성 문제를 해결하기 위해서 소프트웨어로 많이 구현되었지만, 영상처리 및 패턴인식과 같은 실시간 처리가 요구되는 응용에서는 빠른 처리가 가능한 하드웨어로 구현되고 있다. 다양한 종류의 신경망 중에서 다층 신경망(MLP: multi-layer perceptron)의 하드웨어 설계는 빠른 처리속도와 적은 면적 그리고 구현의 용이성으로 고정소수점 연산을 많이 사용하였다. 하지만 고정소수점 연산을 사용하는 하드웨어 설계는 높은 정확도의 부동소수점 연산을 많이 사용하는 소프트웨어 MLP를 쉽게 적용할 수 없는 문제점을 가진다. 본 논문에서는 높은 정확도와 높은 유연성을 가지는 부동소수점 연산을 사용하면서도 은닉층 뉴런수를 주기(cycle)로 빠르게 수행하는 MLP의 완전 파이프라이닝(fully-pipelining) 설계방법을 제안한다. MLP는 주어진 문제에 의해서 자연스럽게 입력층과 출력층의 구조가 결정되지만, 은닉층 구조는 사용자에 의해서 결정된다. 그러므로 제안된 설계방법은 많은 반복수행이 요구되는 영상처리 및 패턴인식 등의 분야에서 은닉층 뉴런수를 최적화 하여 쉽게 성능 향상을 이룰 수 있다.

Keywords

References

  1. S. Coric, I. Latinovic and A. Pavasovic, 'A Neural Network FPGA Implementation,' In Proceedings of the 5th Seminar on Neural Network Applications in Electrical Engineering, Belgrade, pp.117-120, Sept. 2000
  2. Nan Bu, Taiji Hamamoto, Toshio Tsuji and Osamu Fukuda, 'FPGA Implementation of a Probabilistic Neural Network for a Bioeletctric Human Interface,' In Proceedings of the 47th IEEE Midwest Symposium on Circuits and Systems, Vol.3, pp.29-32, 2004 https://doi.org/10.1109/MWSCAS.2004.1354283
  3. Ossoinig, H., Reisinger, E., Steger, C., Weiss, R., 'Design and FPGA-Implementation of a Neural Network,' In Proceedings of the 7th International Conference on Signal Processing Applications & Technology, pp.939-942, 1996
  4. Ma Xiaobin, Jm Lianwen, Shen Dongsheng, Ym junxun, 'A Mixed Parallel Neural Networks Computing Unit Implemented in FPGA,' In Proceedings of the 2003 International Conference on Neural Networks and Signal Processing Vol.1, pp.324-327, 2003 https://doi.org/10.1109/ICNNSP.2003.1279275
  5. Sameh A,El Kader, M.SAA, 'Generic Floating Point Library for Neuro-Fuzzy Controllers based on FPGA Technology,' In Proceedings of the 4th IEEE International Symposium on Signal Processing and Information Technology, pp.369-372 Dec. 2004 https://doi.org/10.1109/ISSPIT.2004.1433796
  6. Hikawa H, 'Pulse Mode Multilayer Neural Network based on Floating Point Number Representation,' In Proceedings of the IEEE International Symposium on Circuits and Systems, Geneva, pp.145-148, May, 2000 https://doi.org/10.1109/ISCAS.2000.856017
  7. Watkins S.S, Chau PM, 'Reduced-complexity Circuit for Neural Networks,' Electronics Letters, Vol.31, No.19, pp.1644-1646, 1995 https://doi.org/10.1049/el:19951113
  8. Wust H, Kasper K, Reininger H, 'Hybrid Number Representation for the FPGA-realization of a versatile neuro-processor,' In Proceedings of the 24th Euromicro Conference, Vol.2, pp.694-701, 1998 https://doi.org/10.1109/EURMIC.1998.708090
  9. Keechul Jung, 'Neural Network-based Text Localization in Color Images,' Pattern Recognition Letters, Vol.22, No.14, pp.1503-1515, 2001 https://doi.org/10.1016/S0167-8655(01)00096-4
  10. Keechul Jung and JungHyun Han, 'Hybrid Approach to Efficient Text Extraction in Complex Color Images,' Pattern Recognition Letters, Vol.25, No.6, pp.679-699, Apr., 2004 https://doi.org/10.1016/j.patrec.2004.01.017
  11. ANSI and IEEE, 'IEEE Standard for Binary Floating-Point Arithmetic,' ANSI/IEEE standard, 754-1985, New York, 1985
  12. Attila Hidvegi, 'Implementation of Neural Networks in FPGA,' http:// www.sysf.physto.se/~attila/ANN.pdf, 2002
  13. K Basterretxea, J.M Tarela and I. del Campo, 'Approximation of Sigmoid Function and the Derivative for Hardware Implementation of Artificial Neurons,' In Proceedings of the IEE Circuits, Devices and Systems, Vol.151, No.1, pp.18-24, Feb., 2004 https://doi.org/10.1049/ip-cds:20030607