Browse > Article
http://dx.doi.org/10.5909/JBE.2019.24.2.264

3D Human Reconstruction from Video using Quantile Regression  

Han, Jisoo (Inha University, Department of Information and Communication Engineering)
Park, In Kyu (Inha University, Department of Information and Communication Engineering)
Publication Information
Journal of Broadcast Engineering / v.24, no.2, 2019 , pp. 264-272 More about this Journal
Abstract
In this paper, we propose a 3D human body reconstruction and refinement method from the frames extracted from a video to obtain natural and smooth motion in temporal domain. Individual frames extracted from the video are fed into convolutional neural network to estimate the location of the joint and the silhouette of the human body. This is done by projecting the parameter-based 3D deformable model to 2D image and by estimating the value of the optimal parameters. If the reconstruction process for each frame is performed independently, temporal consistency of human pose and shape cannot be guaranteed, yielding an inaccurate result. To alleviate this problem, the proposed method analyzes and interpolates the principal component parameters of the 3D morphable model reconstructed from each individual frame. Experimental result shows that the erroneous frames are corrected and refined by utilizing the relation between the previous and the next frames to obtain the improved 3D human reconstruction result.
Keywords
3D human body reconstruction; human pose and shape; quantile regression; temporal consistency;
Citations & Related Records
연도 인용수 순위
  • Reference
1 M. Loper, N. Mahmood, J. Romero, G. Pons-Moll, and M. J. Black, "SMPL: A skinned multi-person linear model," ACM Trans. on Graphics, vol. 34, no. 6, pp. 248, November 2015.
2 L. Pishchulin, E. Insafutdinov, S. Tang, and B. Andress, "DeepCut: Joint subset partition and labeling for multi person pose estimation," Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 4929-4937, June 2016.
3 K. Grauman and T. Darrell, "Fast contour matching using approximate earth mover's distance," Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, June 2004.
4 J. S. Han, M. R. Cho, and I. K. Park, "Moving human shape and pose reconstruction from video," Proceedings of Korean Society of Broadcast and Media Engineers Fall Conference, pp. 66-68, November 2018.
5 L. Sigal, A. Balan, and M. J. Black, "HumanEva: Synchronized video and motion capture dataset and baseline algorithm for evaluation of articulated human motion," International Journal of Computer Vision, vol. 87, no. 1-2, pp. 4-27, March 2010.   DOI
6 C. Ionescu, D. Papva, V. Olaru, and C. Sminchisescu," Human3.6M: Large scale datasets and predictive methods for 3D human sensing in natural environments," IEEE Trans. on Pattern Analysis and Machine Intelligence, vol. 36, no. 7, pp. 1325-1339, July 2014.   DOI
7 K. He, G. Gkioxari, P. Dollar, and R. Girshick, "Mask R-CNN," Proceedings of IEEE International Conference on Computer Vision, pp.2980-2988, October 2017.
8 P. Guan, A. Weiss, A. O. Balan, and M. J. Black, "Estimating human shape and pose from a single image," Proceedings of IEEE International Conference on Computer Vision, pp. 1381-1388, September 2009.
9 F. Bogo, A. Kanazawa, C. Lassner, and P. Gehler, "Keep it SMPL: Automatic estimation of 3D human pose and shape from a single image," Proceedings of European Conference on Computer Vision, pp. 561-578, October 2016.
10 N. Meinshausen, "Quantile regression forests," Journal of Machine Learning Research, vol. 7, no. 6, pp. 983-999, June 2006.
11 M. Lopez-Quintero, M. Marin-jimenez, et al. "Stereo pictorial structure for 2D articulated human pose estimation," Machine Vision and Applications, vol. 27, no. 2, pp. 157-174, February 2016.   DOI
12 C. Zhang, S. Pujades, M. J. Black, and G. Pons-Moll, "Detailed, accurate, human shape estimation from clothed 3D scan sequences," Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, July 2017.