Browse > Article
http://dx.doi.org/10.14372/IEMEK.2021.16.5.215

Survey on Deep Learning Methods for Irregular 3D Data Using Geometric Information  

Cho, Sung In (Dongguk University)
Park, Haeju (Dongguk University)
Publication Information
IEMEK Journal of Embedded Systems and Applications / v.16, no.5, 2021 , pp. 215-223 More about this Journal
Abstract
3D data can be categorized into two parts : Euclidean data and non-Euclidean data. In general, 3D data exists in the form of non-Euclidean data. Due to irregularities in non-Euclidean data such as mesh and point cloud, early 3D deep learning studies transformed these data into regular forms of Euclidean data to utilize them. This approach, however, cannot use memory efficiently and causes loses of essential information on objects. Thus, various approaches that can directly apply deep learning architecture to non-Euclidean 3D data have emerged. In this survey, we introduce various deep learning methods for mesh and point cloud data. After analyzing the operating principles of these methods designed for irregular data, we compare the performance of existing methods for shape classification and segmentation tasks.
Keywords
3D deep learning; Irregular data; Mesh; Point cloud; Classification; Segmentation;
Citations & Related Records
연도 인용수 순위
  • Reference
1 C.R. Qi, L. Yi, H. su, L.J. Guibas, "PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space," In NeurIPS, pp. 5099-5108, 2017.
2 Y. Wang, S. Asafi, O.V. Kaick, H. Zhang, D. CohenOr, B Chen, "Active Co-analysis of a set of Shapes," ACM Transactions on Graphics, Vol. 31, No. 6, 2012.
3 L. Yi, V.G. Kim, D. Ceylan, I.-C. Shen, M. Yan, H. Su, C. Lu, Q. Huang, A. Sheffer, L. Guibas, "A Scalable Active Framework for Region Annotation in 3D Shape Collections," ACM Transactions on Graphics, Vol. 35, No. 6, 2016.
4 A. Krizhevsky, I. Sutskever, G.E. Hinton, "Imagenet Classification with Deep Convolutional Neural Networks," In Advances in neural information processing systems, pp. 1097-1105, 2012.
5 J. Long, E. Shelhamer, T. Darrell, "Fully Convolutional Networks for Semantic Segmentation," In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 3431-3440, 2015.
6 J.W.H. Tangelder, R.C. Veltkamp, "A Survey of Content Based 3D Shape Retrieval Methods," Multimedia tools and applications, Vol. 39, No. 3, pp. 441-471, 2008.   DOI
7 K. Michael, T. Funkhouser, S. Rusinkiewicz, "Rotation Invariant Spherical Harmonic Representation of 3D Shape Descriptors," Symposium on geometry processing, Vol. 6, pp. 156-164, 2003.
8 Z. Wu, S. Song, A. Khosla, F. Yu, L. Zhang, X. Tang, J. Xiao, "3D ShapeNets: A Deep Representation for Volumetric Shapes," In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1912-1920, 2015.
9 E. Ahmed, A. Saint, A. Shabayek, K. Cherenkova, R. Das, G. Gusev, D. Aouada, "A Survey on Deep Learning Advances on Different 3D Data Representations," arXiv preprint arXiv:1808.01462, 2018.
10 H. Su, S. Maji, E. Kalogerakis, E. Learned-Miller, "Multi-View Convolutional Neural Networks for 3D Shape Recognition," In Proceedings of the IEEE International Conference on Computer Vision, pp. 945-953, 2015.
11 Y. Liu, B. Fan, S. Xiang, C. Pan, "Relation-shape Convolutional Neural Network for Point Cloud Analysis," Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019.
12 D. Maturana, S. Scherer, "Voxnet: A 3D Convolutional Neural Network for Real-time Object Recognition," IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 922-928, 2015.
13 Z.B. Liu, S.H. Bu, K. Zhou, S.M. Gao, J.W. Han, J. Wu, "A Survey on Partial Retrieval of 3D Shapes," Journal of Computer Science and Technology, Vol. 28, No. 5, pp. 836-851, 2013.   DOI
14 P. Sermanet, D. Eigen, X. Zhang, M. Mathieu, R. Fergus, Y. LeCun, "Overfeat: Integrated Recognition, Localization and Detection Using Convolutional networks," arXiv preprint arXiv:1312.6229, 2013.
15 M. Jiang, Y. Wu, T. Zhao, Z. Zhao, "PointSIFT: A SIFT-like Network Module for 3D Point Cloud Semantic Segmentation," arXiv preprint arXiv:1807.00652, 2018.
16 L. Landrieu, M. Simonovsky, "Large-scale Point Cloud Semantic Segmentation with Superpoint Graphs," In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4558-4567, 2018.
17 Y. Xu, T. Fan, M. Xu, L. Zeng, Y. Qiao, "Spidercnn: Deep Learning on Point Sets with Parameterized Convolutional Filters," Proceedings of the European Conference on Computer Vision (ECCV), pp. 87-102, 2018.
18 R. Klokov, V. Lempitsky, "Escape from Cells: Deep Kd-Networks for the Recognition of 3D Point Cloud Models," In Proceedings of the IEEE International Conference on Computer Vision, pp. 863-872, 2017.
19 M. Gadelha, R. Wang, S. Maji, "Multiresolution Tree Networks for 3D Point Cloud Processing," In Proceedings of the European Conference on Computer Vision, pp. 103-118, 2018.
20 Y. Feng, Yi. Feng, H. You, X. Zhao, Y. Gao, "Meshnet: Mesh Neural Network for 3D Shape Representation," Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33, No. 1, 2019.
21 X. Liu, Z. Han, Y.S. Liu, M. Zwicker, "Point2sequence: Learning the Shape Representation of 3D Point Clouds with an Attention-based Sequence to Sequence Network," Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33, No. 1, 2019.
22 I. Kostrikov, Z. Jiang, D. Panozzo, D. Zorin, J. Bruna, "Surface Networks," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2540 - 2548, 2018.
23 M. Jaderberg, K. Simonyan, A. Zisserman, "Spatial Transformer Networks," Advances in neural information processing systems, Vol. 28, pp. 2017-2025, 2015.
24 Y. Qang, Y. SUN, "Dynamic Graph cnn for Learning on Point Clouds," ACM Transactions On Graphics, Vol. 38, No. 5, 2019.
25 Y. Feng, Z. Zhang, X. Zhao, R. Ji, Y. Gao, "GVCNN: Group-view Convolutional Neural Networks for 3D Shape Recognition," In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 264-272, 2018.
26 R. Hanocka, A. Hertz, N. Fish, R. Giryes, S. Fleishman, D. Cohenor, "MeshCNN: A Network with an Edge," ACM Transactions on Graphics, Vol. 38, pp. 1-12, 2019.
27 C.R. Qi, H. Su, K. Mo, L.J. Guibas, "Pointnet: Deep Learning on Point Sets for 3D Classification and Segmentation," Proceedings of the IEEE conference on computer vision and pattern recognition, 2017.
28 J. Li, B.M. Chen, G.H. Lee, "SO-Net: Self-Organizing Network for Point Cloud Analysis," In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 9397-9406, 2018.
29 H. Hoppe, "View-dependent Refinement of Progressive Meshes," In Proceedings of the 24th annual conference on Computer graphics and interactive techniques, pp. 189-198, 1997.