Browse > Article
http://dx.doi.org/10.3837/tiis.2022.07.011

Multi-resolution Fusion Network for Human Pose Estimation in Low-resolution Images  

Kim, Boeun (Artificial Intelligence Research Center, Korea Electronics Technology Institute)
Choo, YeonSeung (Artificial Intelligence Research Center, Korea Electronics Technology Institute)
Jeong, Hea In (Artificial Intelligence Research Center, Korea Electronics Technology Institute)
Kim, Chung-Il (Artificial Intelligence Research Center, Korea Electronics Technology Institute)
Shin, Saim (Artificial Intelligence Research Center, Korea Electronics Technology Institute)
Kim, Jungho (Artificial Intelligence Research Center, Korea Electronics Technology Institute)
Publication Information
KSII Transactions on Internet and Information Systems (TIIS) / v.16, no.7, 2022 , pp. 2328-2344 More about this Journal
Abstract
2D human pose estimation still faces difficulty in low-resolution images. Most existing top-down approaches scale up the target human bonding box images to the large size and insert the scaled image into the network. Due to up-sampling, artifacts occur in the low-resolution target images, and the degraded images adversely affect the accurate estimation of the joint positions. To address this issue, we propose a multi-resolution input feature fusion network for human pose estimation. Specifically, the bounding box image of the target human is rescaled to multiple input images of various sizes, and the features extracted from the multiple images are fused in the network. Moreover, we introduce a guiding channel which induces the multi-resolution input features to alternatively affect the network according to the resolution of the target image. We conduct experiments on MS COCO dataset which is a representative dataset for 2D human pose estimation, where our method achieves superior performance compared to the strong baseline HRNet and the previous state-of-the-art methods.
Keywords
Human keypoint detection; Human pose estimation; Low-resolution image; Small person pose estimation; 2D pose estimation;
Citations & Related Records
Times Cited By KSCI : 3  (Citation Analysis)
연도 인용수 순위
1 Dong, Xiang, et al, "Dual Attention Based Image Pyramid Network for Object Detection," KSII TRANSACTIONS ON INTERNET AND INFORMATION SYSTEMS, 15(12), 4439-4455, 2021.
2 Z. Cao, T. Simon, S. Wei, and Y. Sheikh, "Realtime Multi-person 2D Pose Estimation Using Part Affinity Fields," in Proc. of 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1302-1310, 2017.
3 A. Newell, K. Yang, and J. Deng, "Stacked Hourglass Networks for Human Pose Estimation," in Proc. of European Conference on Computer Vision (ECCV), pp. 483-499, 2016.
4 X. Bin, H. Wu, and Y. Wei, "Simple baselines for human pose estimation and tracking," in Proc. of the European conference on computer vision (ECCV), 2018.
5 K. Sun, B. Xiao, D. Liu, and J. Wang, "Deep High-Resolution Representation Learning for Human Pose Estimation," in Proc. of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5686-5696, 2019.
6 J. Huang, Z. Zhu, F. Guo, and G. Huang, "The devil is in the details: Delving into unbiased data processing for human pose estimation," in Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5700-5709, 2020.
7 Y. Bai, Y. Zhang, M. Ding, and B. Ghanem, "SOD-MTGAN: Small Object Detection via Multi-Task Generative Adversarial Network," in Proc. of European Conference on Computer Vision (ECCV), pp. 210-226, 2018.
8 L. Neumann, A. Vedaldi, "Tiny people pose," in Proc. of Asian Conference on Computer Vision (ACCV), vol. 11363, pp. 558-574, May. 2018.
9 Z. Zhang, L. Wan, W. Xu, and S. Wang, "Estimating a 2D pose from a tiny person image with super-resolution reconstruction," Elsevier Computers & Electrical Engineering, vol. 93, July, 2021.
10 N. Yali, J. Lee, S. Yoon, and D. S. Park, "A Multi-Stage Convolution Machine with Scaling and Dilation for Human Pose Estimation," KSII Transactions on Internet and Information Systems (TIIS), 13(6), 3182-3198, 2019.   DOI
11 Li, Zhigang, et al, "Temporal and Spatial Traffic Analysis Based on Human Mobility for Energy Efficient Cellular Network," KSII Transactions on Internet and Information Systems (TIIS), 15(1), 114-130, 2021.
12 S. Park, M. Ji, and J. Chun, "2D human pose estimation based on object detection using RGB-D information," KSII Transactions on Internet and Information Systems (TIIS), 12(2), 800-816, 2018.   DOI
13 A. Newell, A. Huang, and J. Deng, "Associative Embedding: End-to-End Learning for Joint Detection and Grouping," Advances in Neural Information Processing Systems (NeurIPS), pp. 2274-2284, 2017.
14 S. Liu, G. Hua, and Y. Li, "2.5D human pose estimation for shadow puppet animation," KSII Transactions on Internet and Information Systems (TIIS), 13(4), 2042-2059, 2019.   DOI
15 T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollar, and C. L. Zitnick, "Microsoft coco: Common objects in context," in Proc. of European conference on computer vision (ECCV), pp. 740-755, 2014.
16 A. Mykhaylo, "2d human pose estimation: New benchmark and state of the art analysis," in Proc. of the IEEE Conference on computer Vision and Pattern Recognition (CVPR), 2014.
17 Ma, Ruoxin, Shengjie Zhao, and Samuel Cheng, "Self-Supervised Rigid Registration for Small Images," KSII Transactions on Internet and Information Systems (TIIS), 15(1), 180-194, 2021.
18 Zhao, Liquan, and Yupeng Zhang, "Generative Adversarial Networks for single image with high quality image," KSII TRANSACTIONS ON INTERNET AND INFORMATION SYSTEMS, 15(12), 4326-4344, 2021.
19 D. C. He, and L. Wang, "Texture Unit, Texture Spectrum, And Texture Analysis," IEEE Transactions on Geoscience and Remote Sensing, vol. 28, no. 4, pp. 509-512, July 1990.   DOI
20 N. Dalal, and B. Triggs, "Histograms of oriented gradients for human detection," in Proc. of 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognitions (CVPR), pp. 886-893, 2005.
21 M. Li, Z. Zhou, J. Li, and X. Liu, "Bottom-up Pose Estimation of Multiple Person with Bounding Box Constraint," in Proc. of Interational Conference on Pattern Recognition (ICPR), pp. 115-120, 2018.
22 X. Xu, H. Chen, F. Moreno-Noguer, L. A. Jeni, and F. De la Torre, "3D Human Shape and Pose from a Single Low-Resolution Image with Self-Supervised Learning," in Proc. of European Conference on Computer Vision (ECCV), pp. 284-300, 2020.
23 X. Wang, T. X. Han, S. Yan, "An HOG-LBP human detector with partial occlusion handling," in Proc. of International Conference on Computer Vision (ICCV), pp. 32-39, 2009.
24 M. Kocabas, S. Karagoz, and E. Akbas, "MultiPoseNet: Fast Multi-Person Pose Estimation using Pose Residual Network," in Proc. of European Conference on Computer Vision (ECCV), pp. 437-453, 2018.
25 Y. Chen, Z. Wang, Y. Peng, Z. Zhang, G. Yu, and J. Sun, "Cascade Pyramid Network for Multi-Person Pose Estimation," in Proc. of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 7103-7112, 2018.
26 Y. Bai, Y. Zhang, M. Ding, and B. Ghanem, "Finding Tiny Faces in the wild with Generative Adversarial Network," in Proc. of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 21-30, 2018.