The Journal of Korea Robotics Society (로봇학회논문지)

Korea Robotics Society (한국로봇학회)
권호 표지
DOI QR코드

DOI QR Code

2D-3D Pose Estimation using Multi-view Object Co-segmentation

다시점 객체 공분할을 이용한 2D-3D 물체 자세 추정

  • Kim, Seong-heum (School of Electrical Engineering, KAIST) ;
  • Bok, Yunsu (Research Assistant Professor, School of Electrical Engineering, KAIST) ;
  • Kweon, In So (Electrical Engineering, KAIST)
  • Received : 2016.07.26
  • Accepted : 2017.01.31
  • Published : 2017.02.28
Download PDF
⟨ Previous Next ⟩

Abstract

We present a region-based approach for accurate pose estimation of small mechanical components. Our algorithm consists of two key phases: Multi-view object co-segmentation and pose estimation. In the first phase, we explain an automatic method to extract binary masks of a target object captured from multiple viewpoints. For initialization, we assume the target object is bounded by the convex volume of interest defined by a few user inputs. The co-segmented target object shares the same geometric representation in space, and has distinctive color models from those of the backgrounds. In the second phase, we retrieve a 3D model instance with correct upright orientation, and estimate a relative pose of the object observed from images. Our energy function, combining region and boundary terms for the proposed measures, maximizes the overlapping regions and boundaries between the multi-view co-segmentations and projected masks of the reference model. Based on high-quality co-segmentations consistent across all different viewpoints, our final results are accurate model indices and pose parameters of the extracted object. We demonstrate the effectiveness of the proposed method using various examples.

Keywords

References

  1. D.I. Park, C.H. Park, D.H. Kim, J.h. Kyung, "Analysis and Design of the Dual Arm Manipulator for Rescue Robot," The Journal of Korea Robotics Society (KROS), vol. 11, no. 4, pp. 235-241, Dec. 2016. https://doi.org/10.7746/jkros.2016.11.4.235
  2. C.H. Park, D.I. Park, D.H. Kim, "The Analysis of Trajectory Tracking Error Caused by the Tolerance of the Design Parameters of a Parallel Kinematic Manipulator", The Journal of Korea Robotics Society (KROS), vol. 11, no. 4, pp. 248-255, Dec. 2016. https://doi.org/10.7746/jkros.2016.11.4.248
  3. H.M. Do, T.-Y. Choi, D.I. Park, D.H. Kim, Youngsu Son, "Design of Self-Reconfigurable Kinematics and Control Engine for Modular Robot," The Journal of Korea Robotics Society (KROS), vol. 11, no. 4, pp. 270-276, Dec. 2016. https://doi.org/10.7746/jkros.2016.11.4.270
  4. T.Y. Choi, H.M. Do, D.I. Park, C.H. Park, J.H. Kyung, D.H. Kim, "Examples of Art Performing with Industrial Dual-arm Robots" The Journal of Korea Robotics Society, vol. 11, no. 4, pp. 293-299, Dec. 2016. https://doi.org/10.7746/jkros.2016.11.4.293
  5. S.-H. Kim, I.-K. Jung, "UGR Detection and Tracking in Aerial Images from UFR for Remote Control," The Journal of Korea Robotics Society (KROS), vol. 10, no. 2, pp. 104-111, May 2015. https://doi.org/10.7746/jkros.2015.10.2.104
  6. H.H, Song J.-B., "Artificial Landmark based Pose-Graph SLAM for AGVs in Factory Environments," The Journal of Korea Robotics Society (KROS), vol. 10, no. 2, pp. 112-118, May 2015. https://doi.org/10.7746/jkros.2015.10.2.112
  7. H.-S. Choi, D.-I. Kim, J.-B. Song, "Simultaneous path tracking and orientation control for three-wheeled omni-directional robots," The Journal of Korea Robotics Society (KROS), vol. 10, no. 3, pp. 154-161, Aug. 2015. https://doi.org/10.7746/jkros.2015.10.3.154
  8. S.W. Song, J.B. Song, "Development of a 5 DOF Manipulator for Weight Handling based on Counterbalance Mechanism," The Journal of Korea Robotics Society (KROS), vol. 11, no. 4, pp. 242-247, Dec. 2016. https://doi.org/10.7746/jkros.2016.11.4.242
  9. B.T. Ahn, E.-H. Kim, J.-H. Sohn, I.S. Kweon, "Robust Real-time Tracking of Facial Features with Application to Emotion Recognition," The Journal of Korea Robotics Society, vol. 8, no. 4, pp. 266-272, Dec. 2013. https://doi.org/10.7746/jkros.2013.8.4.266
  10. J.-C. Bazin, S.-h. Kim, D.-G. Choi, J.-Y. Lee, I.S. Kweon, "Mixing Collaborative and Hybrid Vision Devices for Robotic Applications," The Journal of Korea Robotics Society (KROS), vol. 6, no. 3, pp. 266-272, Sep. 2011.
  11. S.-H. Kim, Y.-W. Tai, Y. Bok, H. Kim, and I.-S. Kweon, "Two phase approach for multi-view object extraction," Proceedings of International Conference on Image Processing (ICIP), 2011.
  12. S.-H. Kim, Y.-W. Tai, J. Park, and I.-S. Kweon, "Multi-view object extraction with fractional boundaries," IEEE Trans. on Image Processing (TIP), 25(8): 3639-3654, Aug. 2016. https://doi.org/10.1109/TIP.2016.2555698
  13. W. Lee, W. Woo, and E. Boyer, "Silhouette segmentation in multiple views," IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI), 33(7): 1429-1441, 2011. https://doi.org/10.1109/TPAMI.2010.196
  14. A. Kowdle, S.N. Sinha, and R. Szeliski, "Multiple view object cosegmentation using appearance and stereo cues," European Conference on Computer Vision (ECCV), 2012.
  15. A. Djelouah, J.-S. Franco, E. Boyer, F.L. Clerc, and P. Perez, "Multi-view object segmentation in space and time," International Conference on Computer Vision (ICCV), 2013.
  16. A. Djelouah, J.-S. Franco, E. Boyer, F.L. Clerc, and P. Perez, "Sparse multi-view consistency for object segmentation," IEEE Trans. on Pattern Analysis and Machine Intelligence (PAMI), 37(9): 1890-1903, 2015. https://doi.org/10.1109/TPAMI.2014.2385704
  17. J.-Y. Guillemaut and A. Hilton, "Joint multi-layer segmentation and reconstruction for free-viewpoint video applications," International Journal on Computer Vision (IJCV), 93(1):73-100, 2011. https://doi.org/10.1007/s11263-010-0413-z
  18. Y. Boykov and M.-P. Jolly, "Interactive graph cuts for optimal boundary and region segmentation of objects in n-d images," ICCV, 2001.
  19. C. Rother, V. Kolmogorov, and A. Blake, "Grabcut: Interactive foreground extraction using iterated graph cuts," Proceedings of ACM SIGGRAPH, 23(3):309-314, 2004. https://doi.org/10.1145/1015706.1015720
  20. Y.-W. Tai, J. Jia, and C.-K. Tang, "Soft color segmentation and its applications," IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI), 29(9):1520-1537, 2007. https://doi.org/10.1109/TPAMI.2007.1168
  21. C. Wu, 'VisualSFM: A visual structure from motion system,' Available: http://ccwu.me/vsfm/, 2011.
  22. Y. Boykov, O. Veksler, and R.Zabih, "Efficient Approximate Energy Minimization via Graph Cuts," IEEE TPAMI, 20(12):1222-1239, Nov. 2001.
  23. V. Kolmogorov and R.Zabih, "What Energy Functions can be Minimized via Graph Cuts?," Pattern Analysis and Machine Intelligence (PAMI), 26(2):147-159, 2004. https://doi.org/10.1109/TPAMI.2004.1262177
  24. Y. Boykov and V. Kolmogorov, "An Experimental Comparison of Min-Cut/Max-Flow Algorithms for Energy Minimization in Vision," PAMI, 26(9):1124-1137, 2004. https://doi.org/10.1109/TPAMI.2004.60
  25. V. Prisacariu, and I. Reid, "Pwp3d: Real-time segmentation and tracking of 3d objects," International Journal of Computer Vision (IJCV), 98(3):335-354, 2012. https://doi.org/10.1007/s11263-011-0514-3
  26. A. Vedaldi and B. Fulkerson, "VLFeat: An open and portable library of computer vision algorithms," Proceedings of the 18th ACM International Conf. on Multimedia, 2010.
  27. H. Fu et al. "Upright orientation of man-made objects," ACM transactions on graphics (TOG). vol. 27, no. 3, 2008.
  28. Autodesk, '123D Design,' Available: http://www.123dapp.com/. [Accessed: Jul. 26, 2016]
  29. M. Pharr and G. Humphreys, 'Physically Based Rendering Techniques,' Available: http://www.pbrt.org/. [Accessed: Jul. 26, 2016]