Browse > Article
http://dx.doi.org/10.30693/SMJ.2022.11.11.25

Road Image Recognition Technology based on Deep Learning Using TIDL NPU in SoC Enviroment  

Yunseon Shin (한동대학교 전산전자공학부)
Juhyun Seo (한동대학교 전산전자공학부)
Minyoung Lee (한동대학교 일반대학원 전산전자공학과)
Injung Kim (한동대학교 전산전자공학부)
Publication Information
Smart Media Journal / v.11, no.11, 2022 , pp. 25-31 More about this Journal
Abstract
Deep learning-based image processing is essential for autonomous vehicles. To process road images in real-time in a System-on-Chip (SoC) environment, we need to execute deep learning models on a NPU (Neural Procesing Units) specialized for deep learning operations. In this study, we imported seven open-source image processing deep learning models, that were developed on GPU servers, to Texas Instrument Deep Learning (TIDL) NPU environment. We confirmed that the models imported in this study operate normally in the SoC virtual environment through performance evaluation and visualization. This paper introduces the problems that occurred during the migration process due to the limitations of NPU environment and how to solve them, and thereby, presents a reference case worth referring to for developers and researchers who want to port deep learning models to SoC environments.
Keywords
Deep Learning; NPU; SoC; Image Classification; Object Detection; Semantic Segmentation;
Citations & Related Records
Times Cited By KSCI : 1  (Citation Analysis)
연도 인용수 순위
1 T. Lin, P. Goyal, R. Girshick, K. He, and P. Dollar, "Focal Loss for Dense Object Detection," Proc. of the IEEE international conference on computer vision (ICCV), pp. 2980-2988, 2017. 
2 N. Dvornik, K. Shmelkov, J. Mairal, and C. Schmid, "BlitzNet: A Real-Time Deep Network for Scene Understanding," Proc. of the IEEE international conference on computer vision (ICCV), pp. 4154-4162, 2017. 
3 J. Borrego-Carazo, D. Castells-Rufas, E. Biempica, and J. Carrabina, "Resource-Constrained Machine Learning for ADAS: A Systematic Review," in IEEE Access, vol. 8, pp. 40573-40598, 2020.   DOI
4 P. Chao, C. Kao, Y. Ruan, C. Huang, and Y. Lin, "HarDNet: A Low Memory Traffic Network," Proc. of the IEEE/CVF international conference on computer vision (ICCV), pp. 3552-3561, 2019. 
5 W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C. Fu, and A. C. Berg, "SSD: Single Shot MultiBox Detector," European Conference on Computer Vision (ECCV), pp. 21-37. Cham. 2016. 
6 A. Bochkovskiy, C. Wang, and H. M. Liao, "YOLOv4: Optimal Speed and Accuracy of Object Detection," arXiv preprint, arXiv:2004.10934, Apr. 2020. 
7 K. He, X. Zhang. S. Ren, and J. Sun, "Deep Residual Learning for Image Recognition," Proc. of the IEEE conference on computer vision and pattern recognition CVPR, pp. 770-778, 2016. 
8 T. Tan and G. Cao, "FastVA: Deep Learning Video Analytics Through Edge Processing and NPU in Mobile," IEEE INFOCOM 2020-IEEE Conference on Computer Communications, pp. 1947-1956, IEEE, 2020. 
9 M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, and L. Chen, "MobileNetV2: Inverted Residuals and Linear Bottlenecks," Proc. of the IEEE conference on computer vision and pattern recognition (CVPR), pp. 4510-4520, 2018. 
10 L. Tabelini, R. Berriel, T. M. Paixao, C. Badue, A. F. D. Souza, and T. Oliveira-Santos, "Keep your Eyes on the Lane: Real-time Attention-guided Lane Detection," Proc. of the IEEE/CVF conference on computer vision and pattern recognition (CVPR), pp. 294-302, 2021. 
11 A. Ignatov, R. Timofte, A. Kulik, S. Yang, K. Wang, F. Baum, M. Wu, L. Xu, and L. V. Gool, "AI Benchmark: All About Deep Learning on Smartphones in 2019," 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW), pp. 3617-3635, IEEE, 2019. 
12 M. Mathew, K. Desappan, P. K. Swami, S. Nagori, and B. M. Gopinath, "Embedded low-power deep learning with tidl," Texas Instrum., Dallas, TX, USA, Tech. Rep. SPRY314, 2018. 
13 K. He, X. Zhang. S. Ren, and J. Sun, "Spatial pyramid pooling in deep convolutional networks for visual recognition," Proc. of the IEEE transactions on pattern analysis and machine intelligence (TPAMI), vol. 37, no. 9, pp. 1904-1916, 2015.    DOI
14 T.-Y. Lin, M. Maire, S. Belongie, L. Bourdev, R. Girshick, J. Hays, P. Perona, D. Ramanan, C. L. Zitnick, and P. Dollar, "Microsoft COCO: Common Objects in Context," European Conference on Computer Vision, (ECCV), pp. 740-755. 2014. 
15 M. Cordts, M. Omran, S. Ramos, T. Rehfeld, M. Enzweiler, R. Benenson, U. Franke, S. Roth, and B. Schiele, "The Cityscapes Dataset for Semantic Urban Scene Understanding," Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3213-3223, 2016.