Browse > Article
http://dx.doi.org/10.3745/KTCCS.2020.9.12.291

Distributed Edge Computing for DNA-Based Intelligent Services and Applications: A Review  

Alemayehu, Temesgen Seyoum (아주대학교 라이프케어사이언스랩)
Cho, We-Duke (아주대학교 전자공학부)
Publication Information
KIPS Transactions on Computer and Communication Systems / v.9, no.12, 2020 , pp. 291-306 More about this Journal
Abstract
Nowadays, Data-Network-AI (DNA)-based intelligent services and applications have become a reality to provide a new dimension of services that improve the quality of life and productivity of businesses. Artificial intelligence (AI) can enhance the value of IoT data (data collected by IoT devices). The internet of things (IoT) promotes the learning and intelligence capability of AI. To extract insights from massive volume IoT data in real-time using deep learning, processing capability needs to happen in the IoT end devices where data is generated. However, deep learning requires a significant number of computational resources that may not be available at the IoT end devices. Such problems have been addressed by transporting bulks of data from the IoT end devices to the cloud datacenters for processing. But transferring IoT big data to the cloud incurs prohibitively high transmission delay and privacy issues which are a major concern. Edge computing, where distributed computing nodes are placed close to the IoT end devices, is a viable solution to meet the high computation and low-latency requirements and to preserve the privacy of users. This paper provides a comprehensive review of the current state of leveraging deep learning within edge computing to unleash the potential of IoT big data generated from IoT end devices. We believe that the revision will have a contribution to the development of DNA-based intelligent services and applications. It describes the different distributed training and inference architectures of deep learning models across multiple nodes of the edge computing platform. It also provides the different privacy-preserving approaches of deep learning on the edge computing environment and the various application domains where deep learning on the network edge can be useful. Finally, it discusses open issues and challenges leveraging deep learning within edge computing.
Keywords
IoT; Deep Learning; Edge Computing; Distributed Training; DNA;
Citations & Related Records
연도 인용수 순위
  • Reference
1 A. L. Caterini and D. E. Chang, "Recurrent neural networks," Design and Applications, Vol.5, 2018.
2 P. Agarwal, and M. A. Alam, "Lightweight Deep Learning Model for Human Activity Recognition on Edge Devices," Procedia Computer Science, Vol.167, 2020
3 E. Li, L. Zeng, Z. Zhou, and X. Chen, "Edge AI: On-Demand Accelerating Deep Neural Network Inference," IEEE Transactions on Wireless Communications, Vol.19, Issue 1, 2019
4 B. Ravandi and I. Papapanagiotou, "A self-learning scheduling in cloud software defined block storage," in 2017 IEEE 10th International Conference on Cloud Computing (CLOUD), pp.415-422, Jun. 2017.
5 Q. Pu, G. Ananthanarayanan, P. Bodik, S. Kandula, A. Akella, P. Bahl, and I. Stoica, "Low latency geo-distributed data analytics," in Proc. Of ACM SIGCOMM, 2015.
6 Y. Sahni, J. Cao, and L. Yang, "Data-aware task allocation for achieving low latency in collaborative edge computing," IEEE Internet of Things Journal, Vol.6, Issue 2, pp.3512- 3524, 2019.   DOI
7 J. Ren, Y. He, G. Huang, G. Yu, Y. Cai, and Z. Zhang, "An edge-computing based architecture for mobile augmented reality," IEEE Network, Vol.33, Issue 4, pp.162-169, 2019.   DOI
8 M. Satyanarayanan, "The emergence of edge computing," Computer (Long. Beach. Calif)., Vol.50, Issue 1, pp.30-39, 2017.   DOI
9 Y. Mao, C. You, J. Zhang, K. Huang, and K. B. Letaief, "A survey on mobile edge computing: The communication perspective," IEEE Communications Surveys & Tutorials, Vol.19, Issue 4, pp.2322-2358, 2017.   DOI
10 S. Weisong, J. Cao, Q. Zhang, Y. Li, and L. Xu, "Edge computing: Vision and challenges," IEEE Internet of Things Journal, Vol.3, Issue 5, pp.637-646, 2016.   DOI
11 Y. LeCun, Y. Bengio, and G. Hinton. "Deep learning," Nature 521, Vol.7553, pp.436-444, 2015
12 I. Goodfellow, Y. Bengio, A. Courville, and Y. Bengio, "Deep Learning," Vol.1. Cambridge, MA, USA: MIT Press, 2016.
13 S. Verma, Y. Kawamoto, Z. M. Fadlullah, H. Nishiyama, and N. Kato, "A Survey on Network Methodologies for Real-Time Analytics of Massive IoT Data and Open Research Issues," IEEE Communications Surveys and Tutorials, Vol.19, Issue 3, pp.1457-1477, 2017.   DOI
14 R. Collobert, J. Weston, L. Bottou, M. Karlen, K. Kavukcuoglu, and P. Kuksa, "Natural language processing (almost) from scratch," Journal of Machine Learning Research, 12 ARTICLE, pp.2493-2537, 2011.
15 A. Krizhevsky, I. Sutskever, and G. E. Hinton, "ImageNet classification with deep convolutional neural networks," Communications of the ACM, Vol.60, Issue 6, pp.84-90, 2017.   DOI
16 M. McClellan, C. Cervello-Pastor, and S. Sallent, "Deep Learning at the Mobile Edge: Opportunities for 5G Networks," Applied Sciences, Vol.10, Issue 4, pp.4735, 2020.   DOI
17 C. Yang, Z. Yang, A. M. Khattak, L. Yang, W. Zhang, W. Gao, and M. Wang, "Structured Pruning of Convolutional Neural Networks via L1 Regularization," IEEE Access, Vol.7, pp.106385-106394, 2019.   DOI
18 F. Tung and G. Mori, "Deep Neural Network Compression by In-Parallel Pruning-Quantization," IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020.
19 H. Hu, R. Peng, Y.-W. Tai, and C.-K. Tang, "Network trimming: A data-driven neuron pruning approach towards efficient deep architectures," arXiv preprint arXiv: 1607.03250, 2016.
20 J.-H. Luo, J. Wu, and W. Lin, "Thinet: a filter level pruning method for deep neural net- work compression," Proceedings of the IEEE International Conference on Computer Vision, 2017.
21 S. Yao, Y, Zhao, A. Zhang, L. Su and T. Abdelzaher, "DeepIoT: Compressing Deep Neural Network Structures for Sensing Systems with a Compressor-Critic Framework," Proceedings of the 15th ACM Conference on Embedded Network Sensor Systems, pp.1-14, 2017.
22 T. J. Yang, Y. H. Chen, and V. Sze, "Designing energyefficient convolutional neural networks using energyaware pruning," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp.5687-5695, 2017.
23 G. Hinton, O. Vinyals, and J. Dean, "Distilling the Knowledge in a Neural Network," arXiv preprint arXiv:1503.02531, 2015.
24 M.-C. Wu, C.-T. Chiu, and K.-H. Wu, "Multi-teacher knowledge distillation for compressed video action recognition on deep neural networks," ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, pp.2202-2206, 2019.
25 M. Gao, Y. Shen, Q. Li, J. Yan, L. Wan, D. Lin, C. Change Loy, and X. Tang, "An embarrassingly simple approach for knowledge distillation," arXiv preprint arXiv:1812.01819, 2018.
26 Y. Lin, S. W. Han, H. Mao, Y. Wang, and W. J. Dally, "Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Training," arXiv preprint arXiv:1712.01887, 2017.
27 S. Teerapittayanon, B. McDanel and H.T. Kung, "Distributed Deep Neural Networks Over the Cloud, the Edge and End Devices," 2017 IEEE 37th International Conference on Distributed Computing Systems (ICDCS), IEEE, pp.328-339, 2017.
28 B. Arshad, R. Ogie, J. Barthelemy, B. Pradhan, N. Verstaevel, and P. Perez, "Computer Vision and IoT-Based Sensors in Flood Monitoring and Mapping: A Systematic Review," Sensors, Vol.19, Issue 22, pp.5012, 2019.   DOI
29 M. Abadi, P. Barham, J. Chen, Z. Chen, A. Davis, J. Dean, M. Devin, S. Ghemawat, G. Irving, M. Isard, M. Kudlur, J. Levenberg, R. Monga, S. Moore, D. G. Murray, B. Steiner, P. Tucker, V. Vasudevan, P. Warden, M. Wicke, Y. Yu, and X. Zheng. Tensorflow, "A system for large-scale machine learning," The 12th USENIX Conference on Operating Systems Design and Implementation({OSDI} 16), 2016.
30 J. Dean, G. S. Corrado, R. Monga, K. Chen, M. Devin, Q. V. Le, M. Z. Mao, M. Ranzato, A. Senior, P. Tucker, K. Yang, and A. Y. Ng, "Large scale distributed deep networks," Advances in Neural Information Processing Systems, Vol.25, pp1223-1231, 2012.
31 R. Mayer and H.-A. Jacobsen, "Scalable Deep Learning on Distributed Infrastructures: Challenges, Techniques, and Tools," ACM Computing Surveys (CSUR), Vol.53, Issue 1, pp.1-37, 2020.
32 Z. Jia, M. Zaharia, and A. Aiken, "Beyond data and model parallelism for deep neural networks," arXiv preprint arXiv:1807.05358, 2018.
33 J. Dean, G. S. Corrado, R. Monga, K. Chen, M. Devin, Q. V. Le, M. Z. Mao, M. Ranzato, A. Senior, P. Tucker, K. Yang, and A. Y. Ng, "Large scale distributed deep networks," Advances in Neural Information Processing Systems, Vol.25, pp.1223-1231, 2012.
34 J. Konecny, H. B. McMahan, F. X. Yu, P. Richtarik, A. T. Suresh, and D. Bacon, "Federated learning: Strategies for improving communication efficiency," arXiv preprint arXiv:1610.05492, 2016.
35 W. Meng et al., "Two-bit networks for deep learning on resource-constrained embedded devices," arXiv preprint arXiv:1701.00485, 2017.
36 J. Konecny, B. McMahan, and D. Ramage, "Federated optimization: Distributed optimization beyond the datacenter," arXiv preprint arXiv:1511.03575, 2015.
37 C. Gentry, "Fully homomorphic encryption using ideal lattices," Proceedings of the Forty-first Annual ACM Symposium on Theory of Computing, pp.169-178, 2009.
38 B. Jacob et al., "Quantization and training of neural networks for efficient integer-arithmetic-only inference," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp.2204-2713, 2018.
39 P. Gysel, M. Motamedi, and S. Ghiasi, "Hardware-oriented Approximation of Convolutional Neural Networks," arXiv preprint arXiv:1604.03168, 2016.
40 Intel Developer Zone [Internet], https://software.intel.com
41 H. Khelifi, S. Luo, B. Nour et al., "Bringing deep learning at the edge of information-centric internet of things," IEEE Communications Letters, Vol.23, Issue 1, pp.52-55, 2019.   DOI
42 Y. Liu, C. Yang, L. Jiang, and Y. Zhang, "Intelligent Edge Computing for IoT-Based Energy Management in Smart Cities," IEEE Network, Vol.33, Issue 2, pp.111-117, 2019.   DOI
43 M. Merenda, F. G. Praticò, R. Fedele, R. Carotenuto, and F. G. D. "A Real-Time Decision Platform for the Management of Structures and Infrastructures," Electronics, Vol.8, Issue 10, pp.1180, 2019.   DOI
44 S. E. Bibri, "The IoT for smart sustainable cities of the future: An analytical framework for sensor-based big data applications for environmental sustainability," Sustainable Cities and Society, Vol.38, pp.230-253, April, 2018.   DOI
45 C. Ieracitano, N. Mammone, A. Hussain and F. C. Morabito, "A novel multi-modal machine learning based approach for automatic classification of EEG recordings in dementia," Neural Networks, Vol.123, pp.176-190, March, 2020.   DOI
46 A. Rajkomar, J. Dean, and I. Kohane, "Machine learning in medicine," The New England Journal of Medicine, Vol.380, pp.1347-1358, 2019.   DOI
47 K. Y. Ngiam and I. W. Khor, "Big data and machine learning algorithms for health-care delivery," The Lancet Oncology, Vol.20, Issue 5, pp.e262-e273, 2019.   DOI
48 J. Schmidhuber, "Deep learning in neural networks: An overview," Neural Networks, Vol.61, pp.85-117, Jan. 2015.   DOI
49 International Data Corporation (IDC) [Internet], https://www.idc.com/getdoc.jsp?containerId=prUS45213219#:-:text=A%20new%20forecast%20from%20International,these%20devices%20will%20also%20grow.
50 L. R. Zheng, H. Tenhunen, and Z. Zou, "Smart Electronic Systems: Heterogeneous Integration of Silicon and Printed Electronics," John Wiley & Sons, 2018.
51 M. Anandhalli and V. P Baligar, "A novel approach in real-time vehicle detection and tracking using Raspberry Pi," Alexandria Engineering Journal, Vol.57, Issue 3, pp.1597-1607, 2018.   DOI
52 M. Syafrudin, G. Alfian, N. L. Fitriyani, and J. Rhee, "Performance Analysis of IoT-Based Sensor, Big Data Processing, and Machine Learning Model for Real-Time Monitoring System in Automotive Manufacturing," Sensors, Vol.18, Issue 9, pp.2946, 2018.   DOI
53 S. Mittal, N. Negi, and R. Chauhan, "Integration of edge computing with cloud computing," 2017 International Conference on Emerging Trends in Computing and Communication Technologies (ICETCCT), pp.1-6, 2017.
54 C. S. Lai, Y. Jia, Z. Dong, D. Wang, Y. Tao, Q. H. Lai, T. K. W. Richard, F. Z. Ahmed, R. Wu, and L. L Lai, "A Review of Technical Standards for Smart Cities," Clean Technologies, Vol.2, Issue 3, pp.290-310, 2020.   DOI
55 J. Souza, A. Francisco, C. Piekarski, and G. Prado, "Data Mining and Machine Learning to Promote Smart Cities: A Systematic Review from 2000 to 2018," Sustainability, Vol.11, Issue 4, pp.1077, 2019.   DOI
56 F. Cicirelli, A. Guerrieri, G. Spezzano, and A. Vinci, "An edge-based platform for dynamic smart city applications," Future Generation Computer Systems, Vol.76, pp.106-118, 2017.   DOI
57 A. Essien, I. Petrounias, P. Sampaio et al., "A deep-learning model for urban traffic flow prediction with traffic events mined from twitter," World Wide Web, pp.1-24, 2020
58 Z. Liu, Z. Li, K. Wu, and M. Li., "Urban traffic prediction from mobility data using deep learning," IEEE Network, Vol.32, Issue 4, pp.40-46, 2018.   DOI
59 L. Greco, G. Percannella, P. Ritrovato, and F. Tortorella, M. Vento, "Trends in IoT based solutions for health care: Moving AI to the edge," Pattern Recognition Letters, 2020.
60 Y. Liu, C. Yang, L. Jiang, S. Xie, and Y. Zhang, "Intelligent edge computing for IoT-based energy management in smart cities," IEEE Network, Vol.33, Issue 2, pp.111-117, 2019   DOI
61 S. Tuli, N. Basumatary, S. S. Gill, M. Kahani, R. C. Arya, G. S.Wander, and R. Buyya, "Healthfog: An ensemble deep learning based smart healthcare system for automatic diagnosis of heartdiseases in integrated iot and fog computing environments," Future Generation Computer Systems, Vol.104, pp.187-200, 2020   DOI
62 M. Chen, W. Li, Y. Hao, Y. Qian, and I. Humar, "Edge cognitive computing based smart healthcare system," Future Generation Computer Systems, Vol.86, pp.403-411, 2018.   DOI
63 A. Esteva, B. Kuprel, R. Novoa, J. Ko, S. Swetter, H. Blau and S. Thrun, "Dermatologist-level classification of skin cancer with deep neural networks," nature, Vol.542, Issue 7639, pp.115-118, 2017.   DOI
64 L. Tsochatzidis, L. Costaridou, and I. Pratikakis, "Deep Learning for Breast Cancer Diagnosis from Mammograms-A Comparative Study," Journal of Imaging, Vol.5, Issue 3, pp.37, 2019.   DOI
65 T. Ben-Nun and T. Hoefler, "Demystifying parallel and distributed deep learning: An in-depth concurrency analysis," ACM Computing Surveys (CSUR), Vol.52, Issue 4, 2018.
66 R. Xu, J. B. Joshi, and C. Li, "CryptoNN: Training neural networks over encrypted data," 2019 IEEE 39th International Conference on Distributed Computing Systems (ICDCS). IEEE, pp.1199-1209, 2019.
67 P. Li, J. Li, Z. Huang, T. Li, C.-Z Gao, S.-M. You, and K. Chen, "Multi-key privacy-preserving deep learning in cloud computing," Future Generation Computer Systems, Vol.74, pp.76-85, 2017.   DOI
68 T. Zhang, Z. He, and R. B. Lee, "Privacy-preserving machine learning through data obfuscation," arXiv preprint arXiv: 1807.01860, 2018.
69 E. Bresson, D. Catalano, and D. Pointcheval, "A simple public-key cryptosystem with a double trapdoor decryption mechanism and its applications," International Conference on the Theory and Application of Cryptology and Information Security, Springer, Berlin, Heidelberg, pp.37-54, 2003.
70 C. Gentry, "Fully homomorphic encryption using ideal lattices," Proceedings of the Forty-first Annual ACM Symposium on Theory of Computing, pp.169-178, 2009.
71 R. Shokri and V. Shmatikov, "Privacy-preserving deep learning," Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security, pp.1310 -1321, 2015.
72 M. Abadi, A. Chu, I. Goodfellow, H. B. McMahan, I. Mironov, K. Talwar, and L. Zhang, "Deep Learning with Differential Privacy," Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, pp.308-318, 2016.
73 J. Wang, J. Zhang, W. Bao, X. Zhu, B. Cao, and P. S. Yu, "Not Just Privacy: Improving Performance of Private Deep Learning in Mobile Cloud," Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp.2407-2416, 2018.
74 Y. Mao, S. Yi, Q. Li, J. Feng, F. Xu, and S. Zhong, "Learning from differentially private neural activations with edge computing," 2018 IEEE/ACM Symposium on Edge Computing (SEC), pp.90-102, 2018.
75 D. Li and Y. Liu, "Deep Learning in Natural Language Processing," Springer, 2018.
76 Amazon [Internet], "Alexa Voice Service," 2016.
77 Team Siri, "Hey Siri: An On-Device DNN-Powered Voice Trigger for Apple's Personal Assistant," Apple Machine Learning Journal, Vol.1, Issue 6, 2017
78 X. Jiang, A. Hadid, Y. Pang, E. Granger, and X. Feng, "Deep Learning in Object Detection and Recognition," Springer:, 2019.
79 M. S. Mehrjardi, A. Trablesi, and O. R. Zaiane, "Self-Attentional Models Application in Task-Oriented Dialogue Generation Systems," arXiv preprint arXiv:1909.05246, 2019.
80 S. P. Singh, A. Kumar, H. Darbari, L. Singh, A. Rastogi, and S. Jain, "Machine translation using deep learning: An overview," 2017 International Conference on Computer, Communications and Electronics (Comptelix), pp.162-167, 2017.
81 V. Aggarwal and G. Kaur, "A review:deep learning technique for image classification," ACCENTS Transactions on Image Processing and Computer Vision, Vol.4, Issue 11, pp.21-25, 2018.   DOI
82 C. C. Hung, G. Ananthanarayanan, P. Bodik, L. Golubchik, M. Yu, V. Bahl, and M. Philipose, "VideoEdge: Processing Camera Streams using Hierarchical Clusters," 2018 IEEE/ACM Symposium on Edge Computing (SEC), IEEE, pp.115-131, 2018.
83 X. Wang, Y. Han, C. Wang, Q. Zhao, X. Chen, and M. Chen, "In-edge AI: Intelligentizing mobile edge computing, caching and communication by federated learning," IEEE Network, Vol.33, Issue 5, pp.156-165, 2019.   DOI
84 S. B. Carlo, M. Touna, D. C. Verma, and A. Cullen, "Edge Computing Architecture for applying AI to IoT," IEEE International Conference on Big Data (Big Data), IEEE, pp.3012-3016, 2017.
85 S. Sureddy, K. Rashmi, R. Gayathri, and A. S. Nadhan, "Flexible Deep Learning in Edge Computing for Internet of Thinkgs," International Journal of Pure and Applied Mathematics, Vol.119, Issue 10, pp.531-543, 2018
86 H. Li, K. Ota, and M. Dong, "Learning IoT in Edge: Deep Learning for the Internet of Things with Edge Computing," IEEE Network, Vol.32, Issue 1, pp.96-101, 2018.   DOI
87 M. Merenda, C. Porcaro, and D. Iero, "Edge Machine Learning for AI-Enabled IoT Devices: A Review," Sensors. Vol.20, Issue 9, pp.2533, 2020.   DOI
88 Z. Zhou, X. Chen, E. Li, L. Zeng, K. Luo, and J. Zhang, "Edge intelligence: Paving the last mile of artificial intelligence with edge computing," Proceedings of the IEEE, Vol.107, Issue 8, pp.1738-1762, 2019.   DOI
89 J. Chen and X. Ran, "Deep Learning with Edge Computing: A Review," Proceedings of the IEEE, Vol.107, Issue 8, pp. 1655-1674, 2019.   DOI