The Journal of Korean Institute of Next Generation Computing (한국차세대컴퓨팅학회논문지)

Korean Institute of Next Generation Computing (한국차세대컴퓨팅학회)
권호 표지

Implementation of back propagation algorithm for wearable devices using FPGA

FPGA를 이용한 웨어러블 디바이스를 위한 역전파 알고리즘 구현

  • 최현식 (조선대학교 IT융합대학 전자공학부)
  • Received : 2019.04.08
  • Accepted : 2019.04.25
  • Published : 2019.04.30

⟨ Previous Next ⟩

Abstract

Neural networks can be implemented in variety of ways, and specialized chips is being developed for hardware improvement. In order to apply such neural networks to wearable devices, the compactness and the low power operation are essential. In this point of view, a suitable implementation method is a digital circuit design using field programmable gate array (FPGA). To implement this system, the learning algorithm which takes up a large part in neural networks must be implemented within FPGA for better performance. In this paper, a back propagation algorithm among various learning algorithms is implemented using FPGA, and this neural network is verified by OR gate operation. In addition, it is confirmed that this neural network can be used to analyze various users' bio signal measurement results by learning algorithm.

신경 회로망을 구현하기 위해 다양한 시도들이 이루어지고 있으며, 하드웨어적인 개선을 위해 전용 칩 개발이 이루어지고 있다. 이러한 신경 회로망을 웨어러블 디바이스에 적용하기 위해서는 소형화와 저전력 동작이 필수적이다. 이러한 관점에서 적합한 구현 방법은 FPGA (field programmable gate array)를 사용한 디지털 회로 설계이다. 이 시스템을 구현하기 위해서는 성능 향상을 위해 신경 회로망의 많은 부분을 차지하는 학습 알고리즘을 FPGA 내에 구현하여야 한다. 본 논문에서는 FPGA를 이용하여 다양한 학습 알고리즘 중 역전파 알고리즘을 구현하였으며, 구현 된 신경 회로망은 OR 게이트 연산을 통해 검증되었다. 또한 이러한 신경 회로망을 활용하여 다양한 사용자의 생체 신호 측정 결과를 분석할 수 있음을 확인하였다.

Keywords

References

  1. J. Schmidhuber, "Deep learning in neural networks: An overview," Neural Networks, vol. 61, pp. 85-117, Nov. 2015. https://doi.org/10.1016/j.neunet.2014.09.003
  2. S. Gupta, A. Agrawal, K. Gopalakrishnan, and P. Narayanan, "Deep learning with limited numerical precision," in International Conference on Machine Learning, 2015, pp. 1737-1746.
  3. M. Oquab, L. Bottou, I. Laptev, and J. Sivic, "Learning and transferring mid-level image representations using convolutional neural networks," in Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, 2014, pp. 1717-1724.
  4. F. Akopyan, "Design and tool flow of IBM's TrueNorth: An ultra-low power programmable neurosynaptic chip with 1 million neurons," in Proceedings of the 2016 on International Symposium on Physical Design, 2016, pp. 59-60.
  5. A. G. Andreou, A. A. Dykman, K. D. Fischl, G. Garreau, D. R. Mendat, G. Orchard, A. S. Cassidy, P. Merolla, J. Arthur, R. A.-Icaza, B. L. Jackson, and D. S. Modha, "Real-time sensory information processing using the TrueNorth neurosynaptic system," in IEEE International Symposium on Circuits and Systems, 2016, pp. 2911-2920.
  6. R. Wang, C. S. Thakur, G. Cohen, T. J. Hamilton, J. Tapson, and A. Schaik, "Neuromorphic Hardware Architecture Using the Neural Engineering Framework for Pattern Recognition," IEEE Trans. on Biomedical Circuits and Systems, vol. 11, no. 3, pp. 574-584, Jun. 2017. https://doi.org/10.1109/TBCAS.2017.2666883
  7. J. Park, T. Yu, S. Joshi, C. Maier, and G. Cauwenberghs, "Hierarchical Address Event Routing for Reconfigurable Large-Scale Neuromorphic Systems," IEEE Trans. Neural Networks and Learning Systems, vol. 28, no. 10, pp. 2408-2422, Oct. 2017. https://doi.org/10.1109/TNNLS.2016.2572164
  8. G. Indiveri and S.-C. Liu, "Memory and Information Processing in Neuromorphic Systems," Proceedings of the IEEE, vol. 10, no. 8, pp. 1379-1397, Aug. 2015.
  9. S. Furber, "Large-scale neuromorphic computing systems," Journal of Neural Engineering, vol. 13, p.051001, Aug. 2016. https://doi.org/10.1088/1741-2560/13/5/051001
  10. A. Yousefzadeh, T. Masquelier, T. SerranoGotarredona, and B. Linares-Barranco, "Hardware implementation of convolution STDP for on-line visual feature learning," in IEEE International Symposium on Circuits and Systems, 2017, pp. 1-4.
  11. S. R. Kheradphsheh, M. Ganjtabesh, S. J. Thorpe, and T. Masquelier, "STDP-based spiking deep convolutional neural networks for object recognition," Neural Networks, vol. 99, pp. 56-67, Mar. 2018. https://doi.org/10.1016/j.neunet.2017.12.005
  12. G. Srinivasan and A. Sengupta, "Magnetic Tunnel Junction Based Long-Term Short-Term Stochastic Synapse for a Spiking Neural Network with On-Chip STDP Learning," Scientific Reports, vol. 6, p. 29545, Jul. 2016. https://doi.org/10.1038/srep29545
  13. G. K. Chen, R. Kumar, H. E. Sumbul, P. C. Knag, and R. K. Krishnamurthy, "A 4096-Neuron 1M-Synapse 3.8-pJ/SOP Spiking Neural Network With On-Chip STDP Learning and Sparse Weights in 10-nm FinFET CMOS," IEEE Journal of Solid-State Circuits, vol. 54, no. 4, pp. 992-1002, Apr. 2019. https://doi.org/10.1109/JSSC.2018.2884901
  14. A. Nagabandi, G. Kahn, R. S. Fearing, and S. Levine, "Neural Network Dynamics for Model-Based Deep Reinforcement Learning with Model-Free Fine-Tuning," in IEEE International Conference on Robotics and Automation, 2018, pp. 7579-7586, Aug. 2016.
  15. I. Bello, B. Zoph, V. Vasudevan, and Q. V. Le, "Neural optimizer search with reinforcement learning," in Internation Conference on Machine Learning, 2017, pp. 459-468.