Browse > Article
http://dx.doi.org/10.5909/JBE.2021.26.3.247

A New Calibration of 3D Point Cloud using 3D Skeleton  

Park, Byung-Seo (Kwangwoon university Electronic Materials Engineering)
Kang, Ji-Won (Kwangwoon university Electronic Materials Engineering)
Lee, Sol (Kwangwoon university Electronic Materials Engineering)
Park, Jung-Tak (Kwangwoon university Electronic Materials Engineering)
Choi, Jang-Hwan (Kwangwoon university Electronic Materials Engineering)
Kim, Dong-Wook (Kwangwoon university Electronic Materials Engineering)
Seo, Young-Ho (Kwangwoon university Electronic Materials Engineering)
Publication Information
Journal of Broadcast Engineering / v.26, no.3, 2021 , pp. 247-257 More about this Journal
Abstract
This paper proposes a new technique for calibrating a multi-view RGB-D camera using a 3D (dimensional) skeleton. In order to calibrate a multi-view camera, consistent feature points are required. In addition, it is necessary to acquire accurate feature points in order to obtain a high-accuracy calibration result. We use the human skeleton as a feature point to calibrate a multi-view camera. The human skeleton can be easily obtained using state-of-the-art pose estimation algorithms. We propose an RGB-D-based calibration algorithm that uses the joint coordinates of the 3D skeleton obtained through the posture estimation algorithm as a feature point. Since the human body information captured by the multi-view camera may be incomplete, the skeleton predicted based on the image information acquired through it may be incomplete. After efficiently integrating a large number of incomplete skeletons into one skeleton, multi-view cameras can be calibrated by using the integrated skeleton to obtain a camera transformation matrix. In order to increase the accuracy of the calibration, multiple skeletons are used for optimization through temporal iterations. We demonstrate through experiments that a multi-view camera can be calibrated using a large number of incomplete skeletons.
Keywords
point cloud; 3d reconstruction; rgb-d; skeleton; pose estimation;
Citations & Related Records
연도 인용수 순위
  • Reference
1 F. Endres, J. Hess, J. Sturm, D. Cremers, and W. Burgard, "3D mapping with an RGB-D camera," IEEE Transactions on Robotics (T-RO), vol. 30, no. 1, pp. 177-187, 2013.   DOI
2 M. Munaro and E. Menegatti, "Fast RGB-D People Tracking for Service Robots," Autonomous Robots, vol 37, pp. 227-242, 2014.   DOI
3 C. Choi and H. Christensen, "RGB-D object tracking: A particle filter approach on GPU," in Proc. of IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Tokyo, Japan, pp. 1084-1091, 2013.
4 J. Tang, S. Miller, A. Singh, and P. Abbeel, "A textured object recognition pipeline for color and depth image data," in Proc. of the IEEE International Conference on Robotics and Automation, Saint Paul, USA, pp. 3467-3474, 2012.
5 T. Munea, Y. Jembre, H. Weldegebriel, L. Chen, C. Huang and C. Yang, "The Progress of Human Pose Estimation: A Survey and Taxonomy of Models Applied in 2D Human Pose Estimation," IEEE Access, vol. 8, pp. 133330-133348, 2020.   DOI
6 M. Labb and F. Michaud, "Online global loop closure detection for large-scale multi-session graph-based SLAM," in Proc. of IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2014, pp. 2661-2666.
7 M. Zollhofer, P. Stotko, A. Gorlitz, C. Theobalt, M. Niessner, R. Klein and A. Kolb, "State of the Art on 3D Reconstruction with RGB-D Cameras," Computer Graphics Forum, vol. 37, pp. 625-652, 2018.
8 N. Fukushima, "ICP with Depth Compensation for Calibration of Multiple ToF Sensors," 2018 - 3DTV-Conference: The True Vision - Capture, Transmission and Display of 3D Video (3DTV-CON), Helsinki, Finland, pp. 1-4, 2018.
9 K. Zheng, Y. Chen, F. Wu, and X. Chen, "A general batch-calibration framework of service robots," in Proc. Int. Conf. Intell. Robot. Appl. Springer, pp. 275-286, 2017.
10 Giancola S., Valenti M., Sala R. "State-of-the-Art Devices Comparison. In: A Survey on 3D Cameras: Metrological Comparison of Time-of-Fli ght, Structured-Light and Active Stereoscopy Technologies," Springer Briefs in Computer Science, pp. 29-39, 2018.
11 M. Lindner, I. Schiller, A. Kolb, and R. Koch, "Time-of-flight sensor calibration for accurate range sensing," Computer Vision and Image Understanding, vol. 114, no. 12, pp. 1318-1328, 2010.   DOI
12 A. Kuznetsova and B. Rosenhahn, "On calibration of a low-cost time-offlight camera," Computer Vision - ECCV 2014 Workshops, Springer International Publishing, pp. 415-427, 2015.
13 D. Ferstl, C. Reinbacher, G. Riegler, M. Rther, and H. Bischof, "Learning depth calibration of time-of-flight cameras," Proceedings of the British Machine Vision Conference (BMVC), pp. 102.1-102.12, September, 2015.
14 A. Perez-Yus, E. Fernandez-Moral, G. Lopez-Nicolas, J. Guerrero, and P. Rives, "Extrinsic calibration of multiple RGB-D cameras from line observations," IEEE Robot. Automat. Lett, vol. 3, no. 1, pp. 273-280, 2018.   DOI
15 K. Desai, B. Prabhakaran and S. Raghuraman, "Skeleton-based Continuous Extrinsic Calibration of Multiple RGB-D Kinect Cameras," Proceedings of the 9th ACM Multimedia Systems Conference, pp. 250-257, June 2018.
16 S. Yeh, Y. Chiou, H. Chang, W. Hsu, S. Liu and F. Tsai, "Performance improvement of offline phase for indoor positioning systems using Asus Xtion and smartphone sensors," Journal of Communications and Networks, vol. 18, no. 5, pp. 837-845, October 2016.   DOI
17 K. Kim, B. Park, D. Kim, S. Kim, and Y. Seo, "Real-time 3D Volumetr ic Model Generation using Multiview RGB-D Camera," Journal of Bro adcast Engineering,Vol. 25, No. 3, pp. 439-448, May 2020.
18 I. Mikhelson, P. Lee, A. Sahakian, Y. Wu, and A. Katsaggelos, "Automatic, fast, online calibration between depth and color cameras," Journal of Visual Communication and Image Representation, vol. 25, 2014.
19 G. Chen, G. Cui, Z. Jin, F. Wu and X. Chen, "Accurate Intrinsic and Extrinsic Calibration of RGB-D Cameras With GP-Based Depth Correction," IEEE Sensors Journal, vol. 19, no. 7, pp. 2685-2694, 1 2019.   DOI
20 W. Yun and J. Kim, "3D Modeling and WebVR Implementation using Azure Kinect, Open3D, and Three.js," 2020 International Conference on Information and Communication Technology Convergence (ICTC), Jeju Island, pp. 240-243, 2020.
21 G. Unal, A. Yezzi, S. Soatto and G. Slabaugh, "A Variational Approach to Problems in Calibration of Multiple Cameras," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 29, no. 8, pp. 1322-1338, Aug. 2007, https://doi.org/10.1109/TPAMI.2007.1035.   DOI
22 F. Basso, E. Menegatti and A. Pretto, "Robust Intrinsic and Extrinsic Calibration of RGB-D Cameras," IEEE Transactions on Robotics, vol. 34, no. 5, pp. 1315-1332, Oct. 2018.   DOI
23 K. Khoshelham and S. O. Elberink, "Accuracy and resolution of kinect depth data for indoor mapping applications," Sensors, vol. 12, no. 2, pp. 1437-1454, 2012.   DOI
24 I. V. Mikhelson, P. G. Lee, A. V. Sahakian, Y. Wu, and A. K. Katsaggelos, "Automatic, fast, online calibration between depth and color cameras," Journal of Visual Communication and Image Representation, vol. 25, 2014.
25 A. N. Staranowicz, G. R. Brown, F. Morbidi, and G. L. Mariottini, "Practical and accurate calibration of RGB-D cameras using spheres," Computer Vision and Image Understanding, vol. 137, pp. 102-114, 2015.   DOI
26 A. Zabatani et al., "Intel® RealSense™ SR300 Coded Light Depth Camera," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 42, no. 10, pp. 2333-2345, 1 Oct. 2020.   DOI