KSII Transactions on Internet and Information Systems (TIIS)

Korean Society for Internet Information (한국인터넷정보학회)
권호 표지
DOI QR코드

DOI QR Code

Deformable Surface 3D Reconstruction from a Single Image by Linear Programming

  • Ma, Wenjuan (Department of Digital Media Technology,Zhejiang Sci-Tech University) ;
  • Sun, Shusen (Department of Digital Media Technology,Zhejiang Sci-Tech University)
  • Received : 2016.05.26
  • Accepted : 2017.02.07
  • Published : 2017.06.30
Download PDF
⟨ Previous Next ⟩

Abstract

We present a method for 3D shape reconstruction of inextensible deformable surfaces from a single image. The key of our approach is to represent the surface as a 3D triangulated mesh and formulate the reconstruction problem as a sequence of Linear Programming (LP) problems. The LP problem consists of data constraints which are 3D-to-2D keypoint correspondences and shape constraints which are designed to retain original lengths of mesh edges. We use a closed-form method to generate an initial structure, then refine this structure by solving the LP problem iteratively. Compared with previous methods, ours neither involves smoothness constraints nor temporal consistency, which enables us to recover shapes of surfaces with various deformations from a single image. The robustness and accuracy of our approach are evaluated quantitatively on synthetic data and qualitatively on real data.

Keywords

References

  1. Vincent Lepetit and Pascal Fua, "Monocular model-based 3d tracking of rigid objects: A survey," Foundations and Trends in Computer Graphics and Vision, 1(1):1-89, 2005. https://doi.org/10.1561/0600000001
  2. Rene Vidal, Yi Ma, Stefano Soatto, and Shankar Sastry, "Two-view multibody structure from motion," International Journal of Computer Vision, 68(1):7-25, Jun. 2006. https://doi.org/10.1007/s11263-005-4839-7
  3. David Forsyth, OkanArikan, Leslie Ikemoto, James Brien, and Deva Ramanan, "Computational studies of human motion: Part 1, tracking and motion synthesis," Foundations and Trends in ComputerGraphics and Vision, 1(2):77-254, 2006.
  4. Carlo Tomasi and Takeo Kanade, "Shape and motion from image streams under orthography: a factorization method," International Journal of Computer Vision, 9(2):137-154, Nov. 1992. https://doi.org/10.1007/BF00129684
  5. ChristophBregler, Aaron Hertzmann, and Henning Biermann, "Recovering non-rigid 3d shape from image streams," in Proc. of IEEE Conference on Computer Vision and Pattern Recognition, 2000.
  6. Matthew Brand, "A direct method for 3d factorization of nonrigid motion observed in 2d," in Proc. of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2005.
  7. Lorenzo Torresani, Aaron Hertzmann, and ChristophBregler, "Non-rigid structure-from-motion: Estimating shape and motion with hierarchical priors," IEEE Transactions on Pattern Analysis and MachineIntelligence, 30(5):878-892, 2008. https://doi.org/10.1109/TPAMI.2007.70752
  8. Jing Xiao and Takeo Kanade, "Uncalibrated perspective reconstruction of deformable structures," in Proc. of International Conference on Computer Vision, 2005.
  9. Alessio Del Bue and Lourdes Agapito, "Non-rigid stereo factorization," International Journal of Computer Vision, 66(3):193-207, 2006. https://doi.org/10.1007/s11263-005-3958-5
  10. A. Bartoli, V. Gay-Bellile, U. Castellani, J. Peyras, S. Olsen, and P. Sayd, "Coarse-to-fine low-rank structure-from-motion, in Proc. of IEEE Computer Society Conference on Computer Visionand Pattern Recognition, 2008.
  11. A. Agudo, L. Agapito, B. Calvo, J. M. M. Montiel, "Good vibrations: A modal analysis approach for sequential non-rigid structure from motion," in Proc. of IEEE Conference on Computer Vision and Pattern Recognition, 2014.
  12. A. Agudo and F. Moreno-Noguer, "Simultaneous pose and non-rigid shape with particle dynamics," in Proc. of IEEE Conference on Computer Vision and Pattern Recognition, 2015.
  13. A. Agudo, F. Moreno-Noguer, B. Calvo and J.M.M. Montiel, "Sequential Non-Rigid Structure from Motion using Physical Priors," IEEE Transactions on Pattern Analysis and MachineIntelligence, 38(5):979-994, 2016. https://doi.org/10.1109/TPAMI.2015.2469293
  14. Mathieu Salzmann, Richard Hartley, and Pascal Fua, "Convex optimization for deformable surface 3-d tracking," in Proc. of International Conference on Computer Vision, 2007.
  15. Timothy F. Cootes, Gareth J. Edwards, and Christopher J. Taylor, "Active appearance models," IEEE Transactions on Pattern Analysis and Machine Intelligence, 23(6):681-685, Jun. 2001. https://doi.org/10.1109/34.927467
  16. Iain Matthews and Simon Baker, "Active appearance models revisited," International Journal of Computer Vision, 60(2):135-164, 2004. https://doi.org/10.1023/B:VISI.0000029666.37597.d3
  17. Mathieu Salzmann, Raquel Urtasun, and Pascal Fua, "Local deformation models for monocular 3d shape recovery," in Proc. of IEEE Conference on Computer Vision and Pattern Recognition, 2008.
  18. T. D. Ngo, J. Östlund and P. Fua, "Template-based Monocular 3D Shape Recovery using Laplacian Meshes," IEEE Transactions on Pattern Analysis and Machine Intelligence, 38(1): 172-187, 2016. https://doi.org/10.1109/TPAMI.2015.2435739
  19. AllaSheffer, Emil Praun, and Kenneth Rose, "Mesh parameterization methods and their applications," Foundations and Trends in Computer Graphics and Vision, 2(2):105-171, 2006. https://doi.org/10.1561/0600000011
  20. V. Gay-Bellile, A. Bartoli, and P. Sayd, "Direct estimation of non-rigid registrations with image-based self-occlusion reasoning," in Proc. of IEEE International Conference on Computer Vision, 2007.
  21. Tim McInerney and DemetriTerzopoulos, "A finite element model for 3d shape reconstruction and nonrigid motion tracking," in Proc. of the International Conference on Computer Vision, Berlin, Germany, 1993.
  22. Mathieu Salzmann, Francesc Moreno-Noguer, Vincent Lepetit, and Pascal Fua, "Closed-form solution to non-rigid 3d surface registration," in Proc. of the 10th European Conference on ComputerVision, 2008.
  23. Mathieu Perriollat, Richard Hartley, and AdrienBartoli, "Monocular template-based reconstruction of inextensible surfaces," in Proc. of the 19th British Machine Vision Conference, 2008.
  24. Florent Brunet, Adrien Bartoli, Richard I. Hartley, "Monocular template-based 3D surface reconstruction: Convex inextensible and nonconvex isometric methods," Computer Vision and Image Understanding 125: 138-154, 2014. https://doi.org/10.1016/j.cviu.2014.04.003
  25. E. Serradell, M. Pinheiro, R. Sznitman, J. Kybic and F. Moreno-Noguer et al., "Non-Rigid Graph Registration using Active Testing Search," IEEE Transactions on Pattern Analysis and Machine Intelligence, 37(3): 625-638, 2015. https://doi.org/10.1109/TPAMI.2014.2343235
  26. A. Bartoli Y. Gerard, F. Chadebecq, and T. Collins, "On template-based reconstruction from a single view: analytical solutions and proofs of well-posedness for developable, isometric and conformal surfaces," in Proc. of IEEE Conference on Computer Vision and Pattern Recognition, 2012.
  27. A. Bartoli, D. Pizarro, and T. Collins, "A robust analytical solution to isometric shape-from-template with focal length calibration," in Proc. of International Conference on Computer Vision, 2013.
  28. A. Chhatkuuli, D. Pizarro and A. Bartoli, Stable "Template-based isometric 3D reconstruction in all imaging conditions by linear least-squares," in Proc. of IEEE Conference on Computer Vision and Pattern Recognition, 2014.
  29. S. Vicente and L. Agapito, "Soft inextensibility constraints for template-free non-rigid reconstruction," in Proc. of the 12th European Conference on ComputerVision, 2012.
  30. Fredrik Kahl, "Multiple view geometry and the $L_{\infty}$ norm," in Proc. of International Conference on Computer Vision, 2005.
  31. QifaKe and Takeo Kanade, "Quasiconvex optimization for robust geometric reconstruction," in Proc. of International Conference on Computer Vision, 2005.
  32. Stephen Boyd and LievenVandenberghe, Convex optimization, Cambridge University Press, 2004.
  33. YongduekSeo and Richard Hartley, "A fast method to minimize $L_{\infty}$ error norm for geometric vision problems," in Proc. of International Conference on Computer Vision, 2007.
  34. Kristy Sim and Richard Hartley, "Removing outliers using the linfty norm," in Proc. of IEEE Conference on Computer Vision and Pattern Recognition, 2006.
  35. Jos Sturm, "Using sedumi 1.02, a matlab toolbox for optimization over symmetric cones," Optimization Methods and Software, 11-12:625-653, 1999.
  36. David Lowe, "Distinctive image features from scale-invariant keypoints," International Journal of Computer Vision, 60(2):91-110, 2004. https://doi.org/10.1023/B:VISI.0000029664.99615.94