Journal of Digital Contents Society (디지털콘텐츠학회 논문지)

Digital Contents Society (한국디지털콘텐츠학회)
권호 표지
DOI QR코드

DOI QR Code

Automatic Arm Region Segmentation and Background Image Composition

자동 팔 영역 분할과 배경 이미지 합성

  • Kim, Dong Hyun (Dept. of Computer Science and CCBM, Graduate School, Gyeongsang Nat'l University) ;
  • Park, Se Hun (Dept. of Computer Science and CCBM, Graduate School, Gyeongsang Nat'l University) ;
  • Seo, Yeong Geon (Dept. of Computer Science and CCBM, Graduate School, Gyeongsang Nat'l University)
  • 김동현 (경상대학교 컴퓨터과학과, 대학원 문화융복합학과) ;
  • 박세훈 (경상대학교 컴퓨터과학과, 대학원 문화융복합학과) ;
  • 서영건 (경상대학교 컴퓨터과학과, 대학원 문화융복합학과)
  • Received : 2017.11.01
  • Accepted : 2017.12.25
  • Published : 2017.12.31
Download PDF
⟨ Previous Next ⟩

Abstract

In first-person perspective training system, the users needs realistic experience. For providing this experience, the system should offer the users virtual and real images at the same time. We propose an automatic a persons's arm segmentation and image composition method. It consists of arm segmentation part and image composition part. Arm segmentation uses an arbitrary image as input and outputs arm segment or alpha matte. It enables end-to-end learning because we make use of FCN in this part. Image composition part conducts image combination between the result of arm segmentation and other image like road, building, etc. To train the network in arm segmentation, we used arm images through dividing the videos that we took ourselves for the training data.

일인칭 관점의 훈련 시스템에서, 사용자는 실제적인 경험을 필요로 하는데, 이런 실제적인 경험을 제공하기 위하여 가상의 이미지 또는 실제의 이미지를 동시에 제공해야 한다. 이를 위해 본 논문에서는 자동적으로 사람의 팔을 분할하는 것과 이미지 합성 방법을 제안한다. 제안 방법은 팔 분할 부분과 이미지 합성 부분으로 구성된다. 팔 분할은 임의의 이미지들을 입력으로 받아서 팔을 분할하고 알파 매트(alpha matte)를 출력한다. 이는 종단 간 학습이 가능한데 이 부분에서 우리는 FCN(Fully Convolutional Network)을 활용했기 때문이다. 이미지 합성부분은 팔 분할의 결과와 길과 건물 같은 다른 이미지와의 이미지 조합을 만들어 낸다. 팔 분할 부분에서 네트워크를 훈련시키기 위하여, 훈련 데이터는 전체 비디오 중에서 팔의 이미지를 잘라내어 사용하였다.

Keywords

References

  1. M. Cha et al, "A virtual reality based fire training simulator integrated with fire dynamics data", Fire Safety Journal, Vol. 50, pp. 12-24, 2012. https://doi.org/10.1016/j.firesaf.2012.01.004
  2. Q. Kennedy et al, "Age and Expertise Effects in Aviation Decision Making and Flight Control in a Flight Simulator", Aviation, Space, and Environmental Medicine, Vol. 81, No. 5, pp. 489-497, 2010. https://doi.org/10.3357/ASEM.2684.2010
  3. P. Backlund et al, "Games for traffic education: An experimental study of a game-based driving simulator", Simulation & Gaming, Vol. 41, No. 2, pp. 145-169, 2010. https://doi.org/10.1177/1046878107311455
  4. P. Salamin et al, "Quantifying effects of exposure to the third and first-person perspectives in virtual-reality-based training", IEEE Transactions on Learning Technologies, Vol. 3, No. 3, pp. 272-276, 2010. https://doi.org/10.1109/TLT.2010.13
  5. F. S. Dean, P. Garrity and C. B. Stapleton, "Mixed reality: A tool for integrating live, virtual and constructive domains to support training transformation", The Interservice/Industry Training, Simulation and Education Conference (I/ITSEC), 2004.
  6. A. Levin, D. Lischinski and Y. Weiss, "A closed-form solution to natural image matting", IEEE Trans. Pattern Anal. Mach. Intell., Vol. 30, No. 2, pp. 228-242, 2008. https://doi.org/10.1109/TPAMI.2007.1177
  7. Q. Chen, D. Li and C. Tang, "KNN matting", IEEE Trans. Pattern Anal. Mach. Intell., Vol. 35, No. 9, pp. 2175-2188, 2013. https://doi.org/10.1109/TPAMI.2013.18
  8. D. Cho, Y. Tai and I. Kweon, "Natural image matting using deep convolutional neural networks", European Conference on Computer Vision, pp. 626-643, 2016.
  9. N. Xu et al, "Deep Image Matting", Available: http://arxiv.org/abs/1703.03872, 2017.
  10. X. Shen et al, "Automatic Portrait Segmentation for Image Stylization", Computer Graphics Forum, Vol. 35, No. 2, pp. 93-102, 2016. https://doi.org/10.1111/cgf.12814
  11. X. Shen et al, "Deep automatic portrait matting," European Conference on Computer Vision, pp. 92-107, 2016.
  12. A. Krizhevsky, I. Sutskever and G. E. Hinton, "Imagenet classification with deep convolutional neural networks", Advances in Neural Information Processing Systems, pp. 1097-1105, 2012.
  13. K. Simonyan and A. Zisserman, "Very Deep Convolutional Networks for Large-Scale Image Recognition", 2014.
  14. J. Long, E. Shelhamer and T. Darrell, "Fully convolutional networks for semantic segmentation", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3431-3440, 2015.
  15. H. Noh, S. Hong and B. Han, "Learning deconvolution network for semantic segmentation", Proceedings of the IEEE International Conference on Computer Vision, pp. 1520-1528, 2015.
  16. J. Redmon et al, "You only look once: Unified, real-time object detection", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 779-788, 2016.
  17. Y. Lecun et al, "Gradient-based learning applied to document recognition", Jproc, Vol. 86, No. 11, pp. 2278-2324, 1998.
  18. S. Jang and H. Jang, "Training Artificial Neural Networks and Convolutional Neural Networks Using WFSO Algorithm", Journal of Digital Contents Society, Vol. 18, No. 5, pp. 969-976, 2017. https://doi.org/10.9728/DCS.2017.18.5.969
  19. D. P. Kingma and J. Ba, "Adam: A Method for Stochastic Optimization", 2014.
  20. M. Abadi et al, "TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems", 2016.