Browse > Article
http://dx.doi.org/10.9723/jksiis.2017.22.5.017

A Study on Compression of Connections in Deep Artificial Neural Networks  

Ahn, Heejune (서울과학기술대학교 전기정보공학과)
Publication Information
Journal of Korea Society of Industrial Information Systems / v.22, no.5, 2017 , pp. 17-24 More about this Journal
Abstract
Recently Deep-learning, Technologies using Large or Deep Artificial Neural Networks, have Shown Remarkable Performance, and the Increasing Size of the Network Contributes to its Performance Improvement. However, the Increase in the Size of the Neural Network Leads to an Increase in the Calculation Amount, which Causes Problems Such as Circuit Complexity, Price, Heat Generation, and Real-time Restriction. In This Paper, We Propose and Test a Method to Reduce the Number of Network Connections by Effectively Pruning the Redundancy in the Connection and Showing the Difference between the Performance and the Desired Range of the Original Neural Network. In Particular, we Proposed a Simple Method to Improve the Performance by Re-learning and to Guarantee the Desired Performance by Allocating the Error Rate per Layer in Order to Consider the Difference of each Layer. Experiments have been Performed on a Typical Neural Network Structure such as FCN (full connection network) and CNN (convolution neural network) Structure and Confirmed that the Performance Similar to that of the Original Neural Network can be Obtained by Only about 1/10 Connection.
Keywords
Deep Learning; Artificial Neural Network; CNN; MLP; Computation Efficiency;
Citations & Related Records
연도 인용수 순위
  • Reference
1 LeCuns, Y., Bengio, Y., and Hinton, G., "Deep learning," Nature, Vol. 521, No. 7553, pp. 436-444, 2015.   DOI
2 Lee, K., H, lee, S.-Y., and Chung, N., Koo, C., "The Effects of IT Usage of Exhibition Onsite and Overall Effectiveness Toward Attendee's Satisfaction", The Journal of Internet Electronic Commerce Research, Vol. 16, No. 6, pp. 77-94, 2016.
3 Hong, T. H. and Kim, J. W., "Intelligent Intrusion Detection Systems Using the Asymmetric costs of Errors in Data Mining," The Journal of Information Systems, Vol. 15, No. 4, pp. 211-224, 2016
4 LeCun, Y., Bottou, L., Bengio, Y., and Haffner, P., "Gradient-based Learning Applied to Document Recognition," Proceedings of the IEEE, Vol. 86, No. 11, pp.2278-2324, 1998.   DOI
5 Han, S., Pool, J., Tran, J., and Dally, W., "Learning both Weights and Connections for Efficient Neural Network." In Advances in Neural Information Processing Systems, pp. 1135-1143, 2015.
6 Naumov, M., Chien, L. S., Vandermersch, P., and Kapasi, U. "Cusparse Library," In GPU Technology Conference, Spetmber, 2010.
7 Chen, T., Du, Z., Sun, N., Wang, J., Wu, C., Chen, Y., and Temam, O. "Diannao: A Small-Footprint High-throughput Accelerator for Ubiquitous Machine-learning. In ACM Sigplan Notices, Vol. 49, No. 4, pp. 269-284. 2014.
8 Zhang, S., Du, Z., Zhang, L., Lan, H., Liu, S., Li, L., and Chen, Y., "Cambricon-X: An Accelerator for Sparse Neural Networks," 49th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO), pp. 1-12, 2016.