ETRI Journal

Electronics and Telecommunications Research Institute (한국전자통신연구원)
권호 표지
DOI QR코드

DOI QR Code

Performance analysis of local exit for distributed deep neural networks over cloud and edge computing

  • Lee, Changsik (Hyper-connected Communication, Research Laboratory, Electronics and Telecommunications Research Institute) ;
  • Hong, Seungwoo (Hyper-connected Communication, Research Laboratory, Electronics and Telecommunications Research Institute) ;
  • Hong, Sungback (Hyper-connected Communication, Research Laboratory, Electronics and Telecommunications Research Institute) ;
  • Kim, Taeyeon (Hyper-connected Communication, Research Laboratory, Electronics and Telecommunications Research Institute)
  • Received : 2020.03.24
  • Accepted : 2020.08.24
  • Published : 2020.11.16
Download PDF
⟨ Previous Next ⟩

Abstract

In edge computing, most procedures, including data collection, data processing, and service provision, are handled at edge nodes and not in the central cloud. This decreases the processing burden on the central cloud, enabling fast responses to end-device service requests in addition to reducing bandwidth consumption. However, edge nodes have restricted computing, storage, and energy resources to support computation-intensive tasks such as processing deep neural network (DNN) inference. In this study, we analyze the effect of models with single and multiple local exits on DNN inference in an edge-computing environment. Our test results show that a single-exit model performs better with respect to the number of local exited samples, inference accuracy, and inference latency than a multi-exit model at all exit points. These results signify that higher accuracy can be achieved with less computation when a single-exit model is adopted. In edge computing infrastructure, it is therefore more efficient to adopt a DNN model with only one or a few exit points to provide a fast and reliable inference service.

Keywords

References

  1. C. Szegedy et al., Going deeper with convolutions, in Proc. IEEE Conf. Comput. Vision Pattern. Recogn. (Boston, MA, USA), June 2015, pp. 1-9.
  2. A. van den Oord et al., WaveNet: A generative model for raw audio, in Proc. ISCA Speech Synthesis Workshop (Sunnyvale, CA USA), Sept. 2016.
  3. D. Wang and E. Nyberg, A long short-term memory model for answer sentence selection in question answering, in Proc. Annu. Meeting Ass. Comput. Linguistics Int. Joint Conf. Natural Language Process (Beijing, China), July 2015, pp. 707-712.
  4. M. Kim, Supervised learning-based DDoS attacks detection: tuning hyperparameters, ETRI J. 41 (2019), 560-573. https://doi.org/10.4218/etrij.2019-0156
  5. D. Reinsel, J. Gantz, and J. Rydning, The digitization of the world from edge to core, White Paper US44413318, IDC (Framingham, MA, USA), Nov. 2018, pp. 1-28.
  6. M. Murshed et al., Machine learning at the network edge: A survey, arXiv preprint, 20192019, arXiv:1908.00080.
  7. Z. Zhou et al., Edge intelligence: paving the last mile of artificial intelligence with edge computing, Proc. IEEE 107 (2019), 1738-1762. https://doi.org/10.1109/JPROC.2019.2918951
  8. S. Han, H. Mao, and W. Dally, Deep compression: compressing deep neural networks with pruning, trained quantization and huffman coding, in Proc. Int. Conf. Learn. Representations (San Juan, Puerto Rico), May 2016.
  9. Y. Kim et al., Compression of deep convolutional neural networks for fast and low power mobile applications, in Proc. Int. Conf. Learn. Representations (San Juan, Puerto Rico), May 2016.
  10. J. Wu et al., Quantized convolutional neural networks for mobile devices, in Proc. IEEE Conf. Comput. Vis. Pattern Recogn. (Las Vegas, NV, USA), 2016, pp. 4820-4828.
  11. N. Lane et al., Dxtk: Enabling resource-efficient deep learning on mobile and embedded devices with the deepx toolkit, in Proc. MobiCASE (Cambridge, UK), Dec. 2016, pp. 98-107.
  12. F. N. Iandola et al., Squeezenet: Alexnet-level accuracy with 50x fewer parameters and <0.5mb model size, arXiv preprint, 2016, arXiv:1602.07360.
  13. Y. Kang et al., Neurosurgeon: Collaborative intelligence between the cloud and mobile edge, in Proc. Int. Conf. Archit. Support Program. Lang. Oper. Syst. (Xian, China), Apr. 2017, pp. 615-629.
  14. S. Teerapittayanon, B. McDanel, and H. T. Kung, Distributed deep neural networks over the cloud, the edge and end devices, in Proc. IEEE Int. Conf. Distrib. Comput. Syst. (Atlanta, GA, USA), June 2017, pp. 328-339.
  15. S. Teerapittayanon, B. McDanel, and H. Kung, Branchynet: Fast inference via early exiting from deep neural networks, in Proc. Int. Conf. Pattern Recogn. (Cancun, Mexico), Dec. 2016, pp. 2464-2469.
  16. E. Li, Z. Zhou, and X. Chen, Edge intelligence: On-demand deep learning model co-inference with device-edge synergy, in Proc. Workshop Mobile Edge Commun. (Budapest, Hungary), Aug. 2018, pp. 31-36.
  17. ETSI, Executive Briefing-Mobile Edge Computing (MEC) Initiative, Sept. 2014.
  18. ETSI Gs MEC-IEG 004, Mobile Edge Computing (MEC) Service Scenarios V1.1.1, 2015.
  19. ETSI Gs MEC 003, Mobile Edge Computing (MEC) Framework and Reference Architecture V1.1.1, 2016.
  20. ITU-T Rec. Y.3172, Architectural framework for machine learning in future networks including IMT-2020, 2019.
  21. OpenFog Consortium Architecture Working Group, OpenFog architecture overview, Open fog consortium (Tokyo, Japan), White Paper OPFWP001.0216, Feb. 2016.
  22. OpenFog Consortium Architecture Working Group, Openfog reference architecture for fog computing, Open fog consortium (Tokyo, Japan), Feb. 2017.
  23. M. Satyanarayanan, The emergence of edge computing, Comput. 50 (2017), 30-39. https://doi.org/10.1109/MC.2017.9
  24. Z. Chen et al., An empirical study of latency in an emerging class of edge computing applications for wearable cognitive assistance, in Proc. ACM/IEEE Symp. Edge Comput. (San Jose, CA, USA), Oct. 2017, pp. 1-14.
  25. J. Wang et al., Towards scalable edge-native applications, in Proc. ACM/IEEE Symp. Edge Comput. (Rlington, VA, USA), 2019, pp. 152-165.
  26. Edge Computing Consortium, White paper of edge computing consortium, ECC (Beijing, China), White Paper, Nov. 2016.
  27. S.-W. Lin et al., Industrial internet reference architecture, Ind. Internet Consortium (IIC) (Needham, MA, USA) Tech. Rep., June. 2015.
  28. Y. LeCun et al., Gradient-based learning applied to document recognition, Proc. IEEE 86 (1998), 2278-2324.
  29. A. Krizhevsky, I. Sutskever, and G. E. Hinton, ImageNet classification with deep convolutional neural networks, in Proc. Conf. Neural Inf. Process Syst. (Stateline, NV, USA), 2012, pp. 1097-1105.
  30. K. Simonyan and A. Zisserman, Very deep convolutional networks for large-scale image recognition, arXiv preprint, 2014, arXiv:1409.1556.
  31. C. Szegedy et al., Going deeper with convolutions, in Proc. IEEE Conf. Comput. Vision Pattern Recogn. (Boston, MA, USA), June 2015, pp. 1-9.
  32. K. He et al., Deep residual learning for image recognition, arXiv preprint, 2015, arXiv:1512.03385, 2015.
  33. R. Girshick et al., Rich feature hierarchies for accurate object detection and semantic segmentation, in Proc. IEEE Conf. Comput. Vision Pattern Recogn. (Columbus, OH, USA), June 2014, pp. 580-587.
  34. R. Girshick, Fast R-CNN, in Proc. IEEE Int. Conf. Comput. Vision (Santiago, Chile), Dec. 2015, pp. 1440-1448.
  35. S. Ren, Faster R-CNN: Towards real time object detection with region proposal networks, in Proc. Int. Conf. Neural Inf. Process. Syst. (Montreal, Canada), 2015, pp. 91-99.
  36. K. He et al., Spatial pyramid pooling in deep convolutional networks for visual recognition, in Proc. Eur. Conf. Comput. Vis. (Zurich, Switzerland), Sept. (2014), 346-361.
  37. J. Redmon, et al., You only look once: Unified, real time object detection, in Proc. IEEE Conf. Comput. Vision Pattern Recogn. (Las Vegas, NV, USA), June 2016, pp. 779-788.

Cited by

  1. Research on Business English Translation Architecture Based on Artificial Intelligence Speech Recognition and Edge Computing vol.2021, 2020, https://doi.org/10.1155/2021/5518868
  2. Research on Digital Economy and Human Resources Based on Fuzzy Clustering and Edge Computing vol.2021, 2021, https://doi.org/10.1155/2021/5583967
  3. Finite Element Structure Analysis of Automobile Suspension Control Arm Based on Neural Network Control vol.2021, 2020, https://doi.org/10.1155/2021/9978701
  4. Research on Mining of Applied Mathematics Educational Resources Based on Edge Computing and Data Stream Classification vol.2021, 2020, https://doi.org/10.1155/2021/5542718