Browse > Article
http://dx.doi.org/10.3745/KTSDE.2020.9.3.109

Design and Implementation of Immersive Media System Based on Dynamic Projection Mapping and Gesture Recognition  

Kim, Sang Joon (서울과학기술대학교 정보통신미디어공학)
Koh, You Jon (서울미디어대학원대학교 뉴미디어학부 미디어공학전공)
Choi, Yoo-Joo (서울미디어대학원대학교 뉴미디어학부)
Publication Information
KIPS Transactions on Software and Data Engineering / v.9, no.3, 2020 , pp. 109-122 More about this Journal
Abstract
In recent, projection mapping, which has attracted high attention in the field of realistic media, is regarded as a technology to increase the users' immersion. However, most existing methods perform projection mapping on static objects. In this paper, we developed a technology to track the movements of users and dynamically map the media contents to the users' bodies. The projected media content is built by predefined gestures just using the user's bare hands without the special devices. An interactive immersive media system has been implemented by integrating these dynamic projection mapping technologies and gesture-based drawing technologies. The proposed realistic media system recognizes the movements and open / closed states of the user 's hands, selects the functions necessary to draw a picture. The users can freely draw the picture by changing the color of the brush using the colors of any real objects. In addition, the user's drawing is dynamically projected on the user's body, allowing the user to design and wear his t-shirt in real-time.
Keywords
Dynamic Projection Mapping; Realistic Media; Augmented Reality; Drawing System;
Citations & Related Records
Times Cited By KSCI : 2  (Citation Analysis)
연도 인용수 순위
1 Yo-sung Ho and Eun-kyung Lee, "Multi-view Content Creation and Reproduction Technology for 3D Realistic Broadcasting Service," Broadcasting and Media Magazine, Vol.16, No.3, pp.51-67, 2011.
2 Hee-Jun Ahn, "Image Matching Technology for Panorama Realistic Image Production," The Journal of the Korean Institute of Communication Sciences, Vol.35, No.9, pp.12-18, 2018.
3 Hyun-cheol Lee, Ki-Chang Park, Eun-seok Kim, and Ji-Taek Heo, "A Study on the Development of Multi-Sensory Virtual Experience System based on Realistic Media," Journal of Korea Multimedia Society, Vol.20, No.9, pp.1574-1583, 2017.   DOI
4 Realistic Media Type [Internet] http://realmc.real4dhub.or.kr /introduce/media
5 Room-Alive [Internet] https://github.com/Microsoft/RoomAliveToolkit
6 Su-ran Park, Han-gkee Kim, Daeh-wan Kim, Hea-mi Kim, Ki-suk Lee, and Kee-hong Kim, "Projection-based Virtual Experience System Using Punching Gestures," The HCI Society of Korea, pp.847-849, 2017.
7 Jun-ho Choi, Yong-Yi Lee, Kyu-Je Lim, Ahmed Bilal, and Kwan -H. Lee, "Chameleon: Participant Display System Using Space Augmented Reality," The HCI Society of Korea, pp.67-69, 2014.
8 Kinect Driven Projection Mapping [Internet] https://www.youtube.com/watch?v=uQi9ZwOTGf0
9 Interactive Projection Mapping [Internet] https://www.youtube.com/watch?v=oCwE5ayHgjM
10 Ronald T. Azuma, "A Survey of augmented Reality," Presence: Teleoperators and Virtual Environments, Vol.6. Iss.4, pp.355-385, 1997.   DOI
11 R. Raskar, G. Welch, K.-L. Low, and D. Bandyopadhyay, "Shader lamps:Animating real objects with image-based illumination," in Rendering Techniques (Eurographics Series), S. Gortler and K. Myszkowski, Eds. Berlin, Germany: Springer, pp.89-102, 2001.
12 Box [Internet] https://www.youtube.com/watch?v=lX6JcybgDFo
13 Sang-Joon Kim, "Design and Implementation of Authoring Tool for Dynamic Projection Mapping Content," Degree thesis (Bachelor's degree), Seoul Media Graduate School: New Media Studies Department 2019.2
14 Kint v1 vs. Kint v2 Thoroughly Comparable [Internet] https://www.buildinsider.net/small/kinectv2cpp/01
15 Kienct Learning Record [Internet] https://www.twblogs.net/a/5b7f8ade2b717767c6b00d19/zh-cn/
16 Camera Calibration [Internet] http://darkpgmr.tistory.com/32.
17 Understanding the Kint V2 Joint and Coordinate System [Internet] https://medium.com/@lisajamhoury/understanding-kinect-v2-joints-and-coordinate-system-4f4b90b9df16
18 Understanding Kint v2 Coordinate Mapping [Internet] https://pterneas.com/2014/05/06/understanding-kinect-coordinate-mapping/
19 Jae-young Lee, "Camera Calibration and Distortion Calibration," Korea Robotics Society Review, Vol.10, No.1, pp.23-29, 2013.
20 Ui-Tae Jeong and In-Soo Lee, "Simplified 3D Camera Calib ration Algorithm for 3D Vision Inspection System," The Journal of Korean Institute of Information Technology, Vol.16, No.7, pp.43-50, 2018.
21 Camera Parameters [Internet] http://physbam.s tanford.edu/cs448x/old/CameraParameters.html
22 Z. Zhang, "A flexible new technique for camera calibration," IEEE Transactionson Pattern Analysis and Machine Intelligence, Vol.22, No.11, pp.1330-1334, 2000.   DOI
23 Wekipedia Camera Resectioning [Internet] https://en.wikipedia.org/wiki/Camera_resectioning
24 Understanding Augmented Reality 1 [Internet] http://secmem.tistory.com/101
25 Projector Calibration [Internet] http://www.sanko-shoko.net/note.php?id=t1bk
26 OpenFrameworks Introduction [Internet] https://openframeworks.cc/ko/about/