Browse > Article
http://dx.doi.org/10.7472/jksii.2018.19.6.91

Sketch-based 3D object retrieval using Wasserstein Center Loss  

Ji, Myunggeun (Department of Computer Science, Kyonggi University)
Chun, Junchul (Department of Computer Science, Kyonggi University)
Kim, Namgi (Department of Computer Science, Kyonggi University)
Publication Information
Journal of Internet Computing and Services / v.19, no.6, 2018 , pp. 91-99 More about this Journal
Abstract
Sketch-based 3D object retrieval is a convenient way to search for various 3D data using human-drawn sketches as query. In this paper, we propose a new method of using Sketch CNN, Wasserstein CNN and Wasserstein center loss for sketch-based 3D object search. Specifically, Wasserstein center loss is a method of learning the center of each object category and reducing the Wasserstein distance between center and features of the same category. To do this, the proposed 3D object retrieval is performed as follows. Firstly, Wasserstein CNN extracts 2D images taken from various directions of 3D object using CNN, and extracts features of 3D data by computing the Wasserstein barycenters of features of each image. Secondly, the features of the sketch are extracted using a separate Sketch CNN. Finally, we learn the features of the extracted 3D object and the features of the sketch using the proposed Wasserstein center loss. In order to demonstrate the superiority of the proposed method, we evaluated two sets of benchmark data sets, SHREC 13 and SHREC 14, and the proposed method shows better performance in all conventional metrics compared to the state of the art methods.
Keywords
Convolutional Neural Network; Image retrieval; Deep Learning; Sketch-based 3D object retrieval;
Citations & Related Records
연도 인용수 순위
  • Reference
1 M. Eitz, R. Richter, T. Boubekeur, K. Hildebrand, and M. Alexa, "Sketch-based shape retrieval," ACM Transactions on Graphics, vol. 31, no. 4, pp. 1-10, 2012. https://doi.org/10.1145/2185520.2335382
2 B. Li, Y. Lu, A. Godil, T. Schreck, B. Bustos, A. Ferreira, T. Furuya, M. J. Fonseca, H. Johan, T. Matsuda, R. Ohbuchi, P. B. Pascoal, and J. M. Saavedra, "A comparison of methods for sketch-based 3D shape retrieval," Computer Vision and Image Understanding, vol. 119, pp. 57-80, 2014. https://doi.org/10.1016/j.cviu.2013.11.008   DOI
3 Fang Wang, Le Kang, and Yi Li, "Sketch-based 3D shape retrieval using Convolutional Neural Networks," 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015. https://doi.org/10.1109/cvpr.2015.7298797
4 R. Hadsell, S. Chopra, and Y. LeCun, "Dimensionality Reduction by Learning an Invariant Mapping," 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 2 (CVPR'06), 2006 https://doi.org/10.1109/cvpr.2006.100
5 F. Schroff, D. Kalenichenko, and J. Philbin, "FaceNet: A unified embedding for face recognition and clustering," 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015. https://doi.org/10.1109/cvpr.2015.7298682
6 Y. Wen, K. Zhang, Z. Li, and Y. Qiao, "A Discriminative Feature Learning Approach for Deep Face Recognition," Lecture Notes in Computer Science, pp. 499-515, 2016. https://doi.org/10.1007/978-3-319-46478-7_31
7 J. Xie, G. Dai, F. Zhu, and Y. Fang, "Learning Barycentric Representations of 3D Shapes for Sketch-Based 3D Shape Retrieval," 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017. https://doi.org/10.1109/cvpr.2017.385
8 A. Rolet, M. Cuturi, and G. Peyr'e. "Fast dictionary learning with a smoothed wasserstein loss," International Conference on Artificial Intelligence and Statistics, Cadiz, Spain, pp. 630-638, 2016. http://www.jmlr.org/proceedings/papers/v51/rolet16.pdf
9 B. Li, Y. Lu, A. Godil, T. Schreck, M. Aono, H. Johan, J. M. Saavedra, and S. Tashiro. "Shrec'13 track: Large scale sketchbased 3D shape retrieval," Eurographics Workshop on 3D Object Retrieval, Girona, Spain, pp. 89-96, 2013. https://dx.doi.org/10.2312/3DOR/3DOR13/089-096
10 T. Furuya and R. Ohbuchi. "Ranking on cross-domain manifold for sketch-based 3D model retrieval," International Conference on Cyberworlds, Yokohama, Japan, pp. 274- 281, 2013. https://doi.org/10.1109/cw.2013.60
11 M. Cuturi. "Sinkhorn distances: Lightspeed computation of optimal transport," Advances in Neural Information Processing Systems, Lake Tahoe, Nevada, USA, pp. 2292-2300, 2013. https://papers.nips.cc/paper/4927-sinkhorn-distances-light speed-computation-of-optimal-transport.pdf
12 He, Xinwei, et al. "Triplet-Center Loss for Multi-View 3D Object Retrieval," arXiv preprint arXiv:1803.06189, 2018. http://openaccess.thecvf.com/content_cvpr_2018/CameraReady/1632.pdf
13 V. I. Bogachev and A. V. Kolesnikov, "The Monge-Kantorovich problem: achievements, connections, and perspectives," Russian Mathematical Surveys, vol. 67, no. 5, pp. 785-890, 2012. https://doi.org/10.1070/rm2012v067n05abeh004808   DOI
14 Y. Rubner, C. Tomasi, and L. J. Guibas. "The Earth Mover's Distance as a metric for image retrieval," International Journal of Computer Vision, vol. 40, no. 2 pp. 99-121, 2000. https://doi.org/10.1023/a:1026543900054   DOI
15 P.-T. de Boer, D. P. Kroese, S. Mannor, and R. Y. Rubinstein, "A Tutorial on the Cross-Entropy Method," Annals of Operations Research, vol. 134, no. 1, pp. 19-67, 2005. https://doi.org/10.1007/s10479-005-5724-z   DOI
16 R. Sinkhorn, "Diagonal Equivalence to Matrices with Prescribed Row and Column Sums," The American Mathematical Monthly, vol. 74, no. 4, p. 402, 1967. https://doi.org/10.2307/2314570   DOI
17 J.-D. Benamou, G. Carlier, M. Cuturi, L. Nenna, and G. Peyre, "Iterative Bregman Projections for Regularized Transportation Problems," SIAM Journal on Scientific Computing, vol. 37, no. 2, pp. A1111-A1138, 2015. https://doi.org/10.1137/141000439   DOI
18 N. Bonneel, G. Peyre, and M. Cuturi, "Wasserstein barycentric coordinates," ACM Transactions on Graphics, vol. 35, no. 4, pp. 1-10, 2016. https://doi.org/10.1145/2897824.2925918
19 B. Li, Y. Lu, C. Li, A. Godil, T. Schreck, M. Aono, M. Burtscher, H. Fu, T. Furuya, H. Johan, J. Liu, R. Ohbuchi, A. Tatsuma, and C. Zou. "Extended large scale sketch-based 3D shape retrieval," Eurographics Workshop on 3D Object Retrieval, Strasbourg, France, pp. 121-130, 2014. http://dx.doi.org/10.2312/3dor.20141058
20 L. van der Maaten and G. Hinton. "Visualizing highdimensional data using t-SNE.," Journal of Machine Learning Research, vol. 9, pp. 2579-2605, 2008. http://www.jmlr.org/papers/volume9/vandermaaten08a/vandermaaten08a.pdf
21 K. He, X. Zhang, S. Ren, and J. Sun, "Deep Residual Learning for Image Recognition," 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016. https://doi.org/10.1109/cvpr.2016.90
22 S. Ferradans, G.-S. Xia, G. Peyre, and J.-F. Aujol, "Static and Dynamic Texture Mixing Using Optimal Transport," Scale Space and Variational Methods in Computer Vision, pp. 137-148, 2013. https://doi.org/10.1007/978-3-642-38267-3_12