Browse > Article
http://dx.doi.org/10.3745/JIPS.04.0058

Convolutional Neural Network Based Multi-feature Fusion for Non-rigid 3D Model Retrieval  

Zeng, Hui (Beijing Engineering Research Center of Industrial Spectrum Imaging, School of Automation and Electrical Engineering, University of Science and Technology Beijing)
Liu, Yanrong (Beijing Engineering Research Center of Industrial Spectrum Imaging, School of Automation and Electrical Engineering, University of Science and Technology Beijing)
Li, Siqi (Beijing Engineering Research Center of Industrial Spectrum Imaging, School of Automation and Electrical Engineering, University of Science and Technology Beijing)
Che, JianYong (The Tiantan Park Management Office)
Wang, Xiuqing (Vocational & Technical Institute, Hebei Normal University)
Publication Information
Journal of Information Processing Systems / v.14, no.1, 2018 , pp. 176-190 More about this Journal
Abstract
This paper presents a novel convolutional neural network based multi-feature fusion learning method for non-rigid 3D model retrieval, which can investigate the useful discriminative information of the heat kernel signature (HKS) descriptor and the wave kernel signature (WKS) descriptor. At first, we compute the 2D shape distributions of the two kinds of descriptors to represent the 3D model and use them as the input to the networks. Then we construct two convolutional neural networks for the HKS distribution and the WKS distribution separately, and use the multi-feature fusion layer to connect them. The fusion layer not only can exploit more discriminative characteristics of the two descriptors, but also can complement the correlated information between the two kinds of descriptors. Furthermore, to further improve the performance of the description ability, the cross-connected layer is built to combine the low-level features with high-level features. Extensive experiments have validated the effectiveness of the designed multi-feature fusion learning method.
Keywords
Convolutional Neural Network; HKS; Multi-Feature Fusion; Non-rigid 3D Model; WKS;
Citations & Related Records
연도 인용수 순위
  • Reference
1 Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, "Gradient-based learning applied to document recognition," Proceedings of the IEEE, vol. 86, no. 11, pp. 2278-2324, 1998.
2 J. Xie, G. Dai, F. Zhu, E. K. Wong, and Y. Fang, "DeepShape: deep-learned shape descriptor for 3D shape retrieval," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 39, no. 7, pp. 1335-1345, 2017.   DOI
3 R. Osada, T. Funkhouser, B. Chazelle, and D. Dokin, "Shape distributions," ACM Transactions on Graphics, vol. 21, no. 4, pp. 807-832, 2002.   DOI
4 A. Wang, J. Lu, J. Cai, T. J. Cham, and G. Wang "Large-margin multi-modal deep learning for RGB-D object recognition," IEEE Transactions on Multimedia, vol. 17, no. 11, pp. 1887-1898, 2015.   DOI
5 H. Tabia, H. Laga, D. Picard, and P. H. Gosselin, "Covariance descriptors for 3D shape matching and retrieval," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, 2014, pp. 4185-4192.
6 A. Agathos, I. Pratikakis, P. Papadakis, S. J. Perantonis, P. N. Azariadis, and N. S. Sapidis, "Retrieval of 3D articulated objects using a graph-based representation," in Proceedings of the Eurographics Workshop on 3D Object Retrieval, Munich, Germany, 2009, pp. 29-36.
7 H. Tabia, D. Picard, H. Laga, and P. H. Gosselin, "Compact vectors of locally aggregated tensors for 3D shape retrieval," in Proceedings of the Eurographics Workshop on 3D Object Retrieval, Girona, Spain, 2013, pp. 17-24.
8 P. Papadakis, I. Pratikakis, T. Theoharis, G. Passalis, and S. Perantonis, "3D object retrieval using an efficient and compact hybrid shape descriptor," in Proceedings of the Eurographics Workshop on 3D Object Retrieval, Crete, Greece, 2008, pp. 9-16.
9 G. Lavoue, "Combination of bag-of-words descriptors for robust partial shape retrieval," The Visual Computer, vol. 28, no. 9, pp. 931-942, 2012.   DOI
10 Z. Lian, A. Godil, T. Fabry, T. Furuya, J. Hermans, R. Ohbuchi, et al., "SHREC'10 Track: non-rigid 3D shape retrieval," Proceedings of the Eurographics Workshop on 3D Object Retrieval, Zurich, Switzerland, 2015, pp. 107-120.
11 Y. Matsuda, N. Miura, A. Nagasaka, H. Kiyomizu, and T. Miyatake, "Finger-vein authentication based on deformation-tolerant feature-point matching," Machine Vision and Applications, vol. 27, no. 2, pp. 237-250, 2016.   DOI
12 A. Frome, D. Huber, R. Kolluri, T. Bulow, and J. Malik, "Recognizing objects in range data using regional point descriptors," in Proceedings of the European Conference on Computer Vision, Prague, Czech Republic, 2004, pp. 224-237.
13 A. E. Johnson and M. Hebert, "Using spin images for efficient object recognition in cluttered 3D scenes," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 21, no. 5, pp. 443-449, 1999.
14 H. Chen and B. Bhanu, "3D free-form object recognition in range images using local surface patches," Pattern Recognition Letters, vol. 28, no. 10, pp. 1252-1262, 2007.   DOI
15 A. Flint, A. Dick, and A. Van den Hengel, "Local 3D structure recognition in range images," IET Computer Vision, vol. 2, no. 4, pp. 208-217, 2008   DOI
16 A. E. Johnson and M. Hebert, "Surface matching for object recognition in complex three-dimensional scenes," Image and Vision Computing, vol. 16, no. 9-10, pp. 635-651, 1998.   DOI
17 G. Hetzel, B. Leibe, P. Levi, and B. Schiele, "3D object recognition from range images using local feature histograms," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Kauai, HI, 2001, pp. 394-399.
18 F. Tombari, S. Salti, and L. Di Stefano, "Unique signatures of histograms for local surface description," in Proceedings of the European Conference on Computer Vision, Crete, Greece, 2010, pp. 356-369.
19 Y. Guo, F. Sohel, M. Bennamoun, M. Lu, and J. Wan, "Rotational projection statistics for 3D local surface description and object recognition," International Journal of Computer Vision, vol. 105, no. 1, pp. 63-86, 2013.   DOI
20 R. B. Rusu, N. Blodow, and M. Beetz, "Fast point feature histograms (FPFH) for 3D registration," in Proceedings of the IEEE International Conference on Robotics and Automation, Kobe, Japan, 2009, pp. 3212- 3217.
21 Z. Lian, A. Godil, B. Bustos, M. Daoudi, J. Hermans, S. Kawamura, et al., "A comparison of methods for non-rigid 3D shape retrieval," Pattern Recognition, vol. 46, no. 1, pp. 449-461, 2013.   DOI
22 S. Salti, F. Tombari, and L. Di Stefano, "SHOT: unique signatures of histograms for surface and texture description," Computer Vision & Image Understanding, vol. 125, no. 8, pp. 251-264, 2014.   DOI
23 J. Sun, M. Ovsjanikov, and L. Guibas, "A concise and provably informative multi‐scale signature based on heat diffusion," Computer Graphics Forum, vol. 28, no. 5, pp. 1383-1392, 2009.   DOI
24 A. M. Bronstein, M. M. Bronstein, R. Kimmel, M. Mohmoudi, and G. Sapiro, "A Gromov-Hausdorff framework with diffusion geometry for topologically-robust non-rigid shape matching," International Journal of Computer Vision, vol. 89, no. 2-3, pp. 266-286, 2010.   DOI
25 A. M. Bronstein, M. M. Bronstein, L. J. Guibas, and M. Ovsjanikow, "Shape Google: geometric words and expressions for invariant shape retrieval," ACM Transactions on Graphics, vol. 30, no. 1, pp. 623-636, 2011.
26 M. Aubry, U. Schlickewei, and D. Cremers, "The wave kernel signature: a quantum mechanical approach to shape analysis," in Proceedings of the IEEE International Conference on Computer Vision Workshops, Barcelona, Spain, 2011, pp. 1626-1633.