Browse > Article
http://dx.doi.org/10.3837/tiis.2017.06.018

Deformable Surface 3D Reconstruction from a Single Image by Linear Programming  

Ma, Wenjuan (Department of Digital Media Technology,Zhejiang Sci-Tech University)
Sun, Shusen (Department of Digital Media Technology,Zhejiang Sci-Tech University)
Publication Information
KSII Transactions on Internet and Information Systems (TIIS) / v.11, no.6, 2017 , pp. 3121-3142 More about this Journal
Abstract
We present a method for 3D shape reconstruction of inextensible deformable surfaces from a single image. The key of our approach is to represent the surface as a 3D triangulated mesh and formulate the reconstruction problem as a sequence of Linear Programming (LP) problems. The LP problem consists of data constraints which are 3D-to-2D keypoint correspondences and shape constraints which are designed to retain original lengths of mesh edges. We use a closed-form method to generate an initial structure, then refine this structure by solving the LP problem iteratively. Compared with previous methods, ours neither involves smoothness constraints nor temporal consistency, which enables us to recover shapes of surfaces with various deformations from a single image. The robustness and accuracy of our approach are evaluated quantitatively on synthetic data and qualitatively on real data.
Keywords
deformable 3D reconstruction; single image reconstruction; linear programming; convex optimization;
Citations & Related Records
연도 인용수 순위
  • Reference
1 T. D. Ngo, J. Östlund and P. Fua, "Template-based Monocular 3D Shape Recovery using Laplacian Meshes," IEEE Transactions on Pattern Analysis and Machine Intelligence, 38(1): 172-187, 2016.   DOI
2 AllaSheffer, Emil Praun, and Kenneth Rose, "Mesh parameterization methods and their applications," Foundations and Trends in Computer Graphics and Vision, 2(2):105-171, 2006.   DOI
3 V. Gay-Bellile, A. Bartoli, and P. Sayd, "Direct estimation of non-rigid registrations with image-based self-occlusion reasoning," in Proc. of IEEE International Conference on Computer Vision, 2007.
4 Tim McInerney and DemetriTerzopoulos, "A finite element model for 3d shape reconstruction and nonrigid motion tracking," in Proc. of the International Conference on Computer Vision, Berlin, Germany, 1993.
5 Mathieu Salzmann, Francesc Moreno-Noguer, Vincent Lepetit, and Pascal Fua, "Closed-form solution to non-rigid 3d surface registration," in Proc. of the 10th European Conference on ComputerVision, 2008.
6 Mathieu Perriollat, Richard Hartley, and AdrienBartoli, "Monocular template-based reconstruction of inextensible surfaces," in Proc. of the 19th British Machine Vision Conference, 2008.
7 Vincent Lepetit and Pascal Fua, "Monocular model-based 3d tracking of rigid objects: A survey," Foundations and Trends in Computer Graphics and Vision, 1(1):1-89, 2005.   DOI
8 Rene Vidal, Yi Ma, Stefano Soatto, and Shankar Sastry, "Two-view multibody structure from motion," International Journal of Computer Vision, 68(1):7-25, Jun. 2006.   DOI
9 David Forsyth, OkanArikan, Leslie Ikemoto, James Brien, and Deva Ramanan, "Computational studies of human motion: Part 1, tracking and motion synthesis," Foundations and Trends in ComputerGraphics and Vision, 1(2):77-254, 2006.
10 Carlo Tomasi and Takeo Kanade, "Shape and motion from image streams under orthography: a factorization method," International Journal of Computer Vision, 9(2):137-154, Nov. 1992.   DOI
11 ChristophBregler, Aaron Hertzmann, and Henning Biermann, "Recovering non-rigid 3d shape from image streams," in Proc. of IEEE Conference on Computer Vision and Pattern Recognition, 2000.
12 S. Vicente and L. Agapito, "Soft inextensibility constraints for template-free non-rigid reconstruction," in Proc. of the 12th European Conference on ComputerVision, 2012.
13 Florent Brunet, Adrien Bartoli, Richard I. Hartley, "Monocular template-based 3D surface reconstruction: Convex inextensible and nonconvex isometric methods," Computer Vision and Image Understanding 125: 138-154, 2014.   DOI
14 E. Serradell, M. Pinheiro, R. Sznitman, J. Kybic and F. Moreno-Noguer et al., "Non-Rigid Graph Registration using Active Testing Search," IEEE Transactions on Pattern Analysis and Machine Intelligence, 37(3): 625-638, 2015.   DOI
15 Matthew Brand, "A direct method for 3d factorization of nonrigid motion observed in 2d," in Proc. of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2005.
16 Lorenzo Torresani, Aaron Hertzmann, and ChristophBregler, "Non-rigid structure-from-motion: Estimating shape and motion with hierarchical priors," IEEE Transactions on Pattern Analysis and MachineIntelligence, 30(5):878-892, 2008.   DOI
17 A. Bartoli Y. Gerard, F. Chadebecq, and T. Collins, "On template-based reconstruction from a single view: analytical solutions and proofs of well-posedness for developable, isometric and conformal surfaces," in Proc. of IEEE Conference on Computer Vision and Pattern Recognition, 2012.
18 A. Bartoli, D. Pizarro, and T. Collins, "A robust analytical solution to isometric shape-from-template with focal length calibration," in Proc. of International Conference on Computer Vision, 2013.
19 A. Chhatkuuli, D. Pizarro and A. Bartoli, Stable "Template-based isometric 3D reconstruction in all imaging conditions by linear least-squares," in Proc. of IEEE Conference on Computer Vision and Pattern Recognition, 2014.
20 Fredrik Kahl, "Multiple view geometry and the $L_{\infty}$ norm," in Proc. of International Conference on Computer Vision, 2005.
21 QifaKe and Takeo Kanade, "Quasiconvex optimization for robust geometric reconstruction," in Proc. of International Conference on Computer Vision, 2005.
22 Stephen Boyd and LievenVandenberghe, Convex optimization, Cambridge University Press, 2004.
23 YongduekSeo and Richard Hartley, "A fast method to minimize $L_{\infty}$ error norm for geometric vision problems," in Proc. of International Conference on Computer Vision, 2007.
24 Kristy Sim and Richard Hartley, "Removing outliers using the linfty norm," in Proc. of IEEE Conference on Computer Vision and Pattern Recognition, 2006.
25 Jos Sturm, "Using sedumi 1.02, a matlab toolbox for optimization over symmetric cones," Optimization Methods and Software, 11-12:625-653, 1999.
26 A. Agudo and F. Moreno-Noguer, "Simultaneous pose and non-rigid shape with particle dynamics," in Proc. of IEEE Conference on Computer Vision and Pattern Recognition, 2015.
27 David Lowe, "Distinctive image features from scale-invariant keypoints," International Journal of Computer Vision, 60(2):91-110, 2004.   DOI
28 Alessio Del Bue and Lourdes Agapito, "Non-rigid stereo factorization," International Journal of Computer Vision, 66(3):193-207, 2006.   DOI
29 A. Bartoli, V. Gay-Bellile, U. Castellani, J. Peyras, S. Olsen, and P. Sayd, "Coarse-to-fine low-rank structure-from-motion, in Proc. of IEEE Computer Society Conference on Computer Visionand Pattern Recognition, 2008.
30 A. Agudo, L. Agapito, B. Calvo, J. M. M. Montiel, "Good vibrations: A modal analysis approach for sequential non-rigid structure from motion," in Proc. of IEEE Conference on Computer Vision and Pattern Recognition, 2014.
31 A. Agudo, F. Moreno-Noguer, B. Calvo and J.M.M. Montiel, "Sequential Non-Rigid Structure from Motion using Physical Priors," IEEE Transactions on Pattern Analysis and MachineIntelligence, 38(5):979-994, 2016.   DOI
32 Mathieu Salzmann, Richard Hartley, and Pascal Fua, "Convex optimization for deformable surface 3-d tracking," in Proc. of International Conference on Computer Vision, 2007.
33 Timothy F. Cootes, Gareth J. Edwards, and Christopher J. Taylor, "Active appearance models," IEEE Transactions on Pattern Analysis and Machine Intelligence, 23(6):681-685, Jun. 2001.   DOI
34 Jing Xiao and Takeo Kanade, "Uncalibrated perspective reconstruction of deformable structures," in Proc. of International Conference on Computer Vision, 2005.
35 Iain Matthews and Simon Baker, "Active appearance models revisited," International Journal of Computer Vision, 60(2):135-164, 2004.   DOI
36 Mathieu Salzmann, Raquel Urtasun, and Pascal Fua, "Local deformation models for monocular 3d shape recovery," in Proc. of IEEE Conference on Computer Vision and Pattern Recognition, 2008.