Browse > Article
http://dx.doi.org/10.9708/jksci/2012.17.12.219

Implementation of Paper Keyboard Piano with a Kinect  

Lee, Jung-Chul (School of Electrical Engineering, University of Ulsan)
Kim, Min-Seong (School of Electrical Engineering, University of Ulsan)
Publication Information
Journal of the Korea Society of Computer and Information / v.17, no.12, 2012 , pp. 219-228 More about this Journal
Abstract
In this paper, we propose a paper keyboard piano implementation using the finger movement detection with the 3D image data from a kinect. Keyboard pattern and keyboard depth information are extracted from the color image and depth image to detect the touch event on the paper keyboard and to identify the touched key. Hand region detection error is unavoidable when using the simple comparison method between input depth image and background depth image, and this error is critical in key touch detection. Skin color is used to minimize the error. And finger tips are detected using contour detection with area limit and convex hull. Finally decision of key touch is carried out with the keyboard pattern information at the finger tip position. The experimental results showed that the proposed method can detect key touch with high accuracy. Paper keyboard piano can be utilized for the easy and convenient interface for the beginner to learn playing piano with the PC-based learning software.
Keywords
Keyboard Instrument; Kinect; Image Processing;
Citations & Related Records
Times Cited By KSCI : 4  (Citation Analysis)
연도 인용수 순위
1 WX5 YAMAHAMidi Controller, http://www.yamaha. com/
2 Finale&SmartMusic, http://www.makemusic.com/
3 P. Messic, "Maximum MIDI: Music Applications in C++," Manning Publications, 1998.
4 K.M. Cho, J.H. Jang, and K.S. Hong, "Adaptive Skin-Color Filter," Pattern Recognition, vol.34, no.5, pp.1067-1073, May 2001.   DOI   ScienceOn
5 P. Viola , M. Jones, "Rapid Object Detection using a Boosted Cascade of Simple Features", IEEE Conference on Computer Vision and Pattern Recognition, vol.1, pp.511- 518, Dec. 2001.
6 B.M. Kim, J.W. Kim, and K.H. Lee, "An Application of AdaBoost learning Algorithm and Kalman Filter to Hand Detection and Tracking," Journal of The Korea Society of Computer and Information, vol.10, no.4, pp.47-56, Sep. 2005.   과학기술학회마을
7 S.Y. Cho, H.R. Byun, H.K. Lee, and J.H. Cha, "Hand Gesture Recognition from Kinect Sensor Data," Journal of Broadcast Engineering, vol.17, no.3, pp.447-458, May 2012.   DOI   ScienceOn
8 J.M. Jeong, J.R. Jang, Y.I. Kim, J.W. Park, and W.J. Lee, "Development of the Hand Recognition System for the Mouse Control," Proceeding of Winter Conference of KSCI, vol.19, no.1, pp.173-174, Jan. 2011.
9 S.W. Jang, Y.J. Park, and G.Y. Kim, "Human Skin Region Detection Utilizing Depth Information," Journal of The Korea Society of Computer and Information, vol.17, no.6, pp.29-36, June 2012.   과학기술학회마을   DOI   ScienceOn
10 Microsoft Kinect, http://www.microsoft.com/en-us/ kinectforwindows/
11 OpenNI, http://www.openni.org/documentation/
12 S.H. Lee, H.S. Hahn, and Y.J. Han, "The Estimation of Hand Pose Based on Mean-Shift Tracking Using the Fusion of Color and Depth Information for Marker-less Augmented Reality," Journal of The Korea Society of Computer and Information, vol.17, no.7, pp.155-166, July 2012.   과학기술학회마을   DOI   ScienceOn
13 Y. Cheng, "Mean shift, mode seeking, and clustering," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol.17, no.8, pp.790-799, Aug. 1995.   DOI   ScienceOn
14 kinect piano download, http://www.google.co.kr/#q =kinect+piano+download&hl=ko&newwindow= 1&sa=N&tbo=u&source=univ&tbm=vid&ei=R CPQUNz8OOqaiAe1qoHAAg&ved=0CEsQqwQ4C g&bav=on.2,or.r_gc.r_pw.r_qf.&bvm=bv.135553 4169,d.aGc&fp=7605525a9eab9576&bpcl=40096 503&biw=1090&bih=1036
15 Akai EWI4000S, http://www.akaipro.com/
16 OpenNI, http://www.openni.org/Downloads/ OpenNI-Modules.aspx
17 Xianquan Zhang and Zhenjun Tang, "A Fast Convex Hull Algorithm for Binary Image," Informatica vol.34, no.3, pp.369-376, Mar. 2010.