International Journal of Fuzzy Logic and Intelligent Systems

Korean Institute of Intelligent Systems (한국지능시스템학회)
권호 표지
DOI QR코드

DOI QR Code

Gesture based Input Device: An All Inertial Approach

  • Chang Wook (Interaction LAB, Samsung Advanced Institute of Technology) ;
  • Bang Won-Chul (Interaction LAB, Samsung Advanced Institute of Technology) ;
  • Choi Eun-Seok (Interaction LAB, Samsung Advanced Institute of Technology) ;
  • Yang Jing (Interaction LAB, Samsung Advanced Institute of Technology) ;
  • Cho Sung-Jung (Interaction LAB, Samsung Advanced Institute of Technology) ;
  • Cho Joon-Kee (Interaction LAB, Samsung Advanced Institute of Technology) ;
  • Oh Jong-Koo (Interaction LAB, Samsung Advanced Institute of Technology) ;
  • Kim Dong-Yoon (Interaction LAB, Samsung Advanced Institute of Technology)
  • Published : 2005.09.01
Download PDF
⟨ Previous Next ⟩

Abstract

In this paper, we develop a gesture-based input device equipped with accelerometers and gyroscopes. The sensors measure the inertial measurements, i.e., accelerations and angular velocities produced by the movement of the system when a user is inputting gestures on a plane surface or in a 3D space. The gyroscope measurements are integrated to give orientation of the device and consequently used to compensate the accelerations. The compensated accelerations are doubly integrated to yield the position of the device. With this approach, a user's gesture input trajectories can be recovered without any external sensors. Three versions of motion tracking algorithms are provided to cope with wide spectrum of applications. Then, a Bayesian network based recognition system processes the recovered trajectories to identify the gesture class. Experimental results convincingly show the feasibility and effectiveness of the proposed gesture input device. In order to show practical use of the proposed input method, we implemented a prototype system, which is a gesture-based remote controller (Magic Wand).

Keywords

References

  1. S. Keates and P. Robinson, 'The use of gestures in multimodal input,' in Proceedings of the third international ACM conference on Assistive technologies, Marina del Rey, California, United States, 1998, pp. 35-42
  2. R. B. Dannenberg and D. Amon, 'A gesture based user interface prototyping system,' in Proceedings of the 2nd annual ACM SIGGRAPH symposium on User interface software and technology, Nov. 1989, pp. 127-132
  3. C. G. Wolf, 'Can people use gesture commands?' IBM, Tech. Rep. 86A002284, 1986
  4. K. R. Wheeler and C. C. Jorgensen, 'Gestures as input: neuroelectric joysticks and keyboards,' IEEE Pervasive Computing, no. 2, pp. 56- 61, April-June 2003
  5. G. Welch and E. Foxlin, 'Motion tracking: No silver bullet, but a respectable arsenal,' IEEE Computer Graphics and Applications, pp. 24-38, November / December 2002
  6. J. Morris, 'Accelerometry - a technique for the measurement of human body movement,' Biomechanics, pp. 729-736, 1973
  7. N. Herbst and C. Liu, 'Automatic signature verification,' IBM Thomas J.Watson Research Center, Yorktown Heights, New York 10598, Tech. Rep. RC5810, Nov. 1975
  8. A. D. Cheok, K. G. Kumar, and S. Prince, 'Micro-accelerometer based hardware interfaces for wearable computer mixed reality applications,' in Proceedings of ISWC2002, 2002
  9. T. Miyagawa, Y. Yonezawa, K. Itoh, and M. Hashimoto, 'Handwritten pattern reproduction using 3d inertial measurement of handwriting movement,' Transactions of the Society of Instrument and Control Engineers, vol. 38, Jan. 2002
  10. E. Kobayashi, K. Masamune, S. Tadokoro, M. Suzuki, T. Dohi, and D. Hashimoto, 'Head mouse system using a gyro sensor - the man-machine interface for laparoscopic manipulator,' in Proceedings of the 12th International Symposium and Exhibition on CAR'98. Computer Assisted Radiology and Surgery. Elsevier Science, 1998, pp. 742-746
  11. H. Ishikawa, Y. Yonezawa, K. Itoh, and M. Hashimoto, 'Extraction of character information from acceleration of handwriting pen,' Transactions on the Institute of Electronics, Information and Communication Engineers D-I, vol. J76D-1, pp. 541-543, Oct. 1993
  12. H. Nonaka and T. Da-Te, 'The development of a pointing device with a cursor in inertial motion,' Transactions of the Information Processing Society of Japan, vol. 31, no. 2, pp. 268-274, 1990
  13. R. Gjone and J. Miraglia, 'Pen for signature verification by acceleration and pressure,' IBM Technical Disclosure Bulletin, vol. 24, p. 902, July 1981
  14. N. M. Herbst and C. J. Liu, 'Signature verification based on complete accelerometry,' IBM Technical Disclosure Bulletin, pp. 4827-4829, May 1977
  15. J. H. Morrissey, 'Electronic calculator based on character recognition of input from stylus acceleration dynamics,' IBM Technical Disclosure Bulletin, vol. 19, pp. 2816-2817, Dec. 1976
  16. P. E. Jones, 'Three-dimensional input device with six degree of freedom,' Mechatronics, vol. 9, pp. 717-729, Oct. 1999 https://doi.org/10.1016/S0957-4158(99)00032-X
  17. D. Strickland, A. Patel, C. Stovall, J. Palmer, and D. McAllister, 'Self tracking of human motion for virtual reality systems,' in Proceedings of SPIE - the International Society for Optical Engineering, vol. 2177, 1994, pp. 278-287
  18. S. Lee, G. J. Nam, J. Chae, H. Kim, and A. J. Drake, 'Two-dimensional position detection systems with MEMS accelerometer for mouse applications,' in Proceedings of the 38th Design Automation Conference, June 2001, pp. 852-857
  19. M. Haukijarvi and J. Yrjanajnen, 'Accelerometer vibration analysis in finding out velocities,' in Proceedings of EUSIPCO 2000, vol. 1, Sept. 2000, pp. 585-588
  20. T. N. nd R. B. Yates, 'A novel accelerometer for sensing pen tip motion,' in Proceedings of the 4th European Conference on Smart Structures and Materials in conjunction with the 2nd International Conference on Micromechanics, Intelligent Materials and Robotics, Bristol, UK, 1998, pp. 709-714
  21. B. Milner, 'Handwriting recognition using acceleration- based motion detection,' in IEE colloquium on document image processing and multimedia, London, UK, 25 March 1999, pp. 5/1-5/6
  22. J. K. Perng, B. Fisher, S. Hollar, and K. S. J. Pister, 'Acceleration sensing glove (ASG),' in Digest of paper. Third International Symposium on Wearable Computers. Los Alamos, CA, USA: IEEE Computing Society, Oct. 1999, pp. 178-180 https://doi.org/10.1109/ISWC.1999.806717
  23. N. M. Herbst and J. H. Morrissey, 'Signature verification method and apparatus,' U.S. Patent 3 983 535, Sept. 28, 1976
  24. M. Epperson, 'Autonomous computer input device and marking instrument,' U.S. Patent 5 247 137, Sept. 21, 1993
  25. Y. Sato, T. Inoue, E. Fujisawa, T. Kitaguchi, T. Furuta, N. Murata, and M. Shingyouchi, 'Pen-shaped handwriting input apparatus using accelerometers and gyroscopes and an associated operational device for determining pen movement,' U.S. Patent 5 981 884, Nov. 9, 1999
  26. Y. Sato, T. Inoue, E. Fujisawa, T. Kitaguchi, T. Furuta, N. Murata, and M. Shingyouchi, 'Pen-shaped handwriting input apparatus using accelerometers and gyroscopes and an associated operational device for determining pen movement,' U.S. Patent 5 902 968, May 11, 1999
  27. T. J. Chainer, R. A. Scranton, and T. K. Worthington, 'Data input pen for signature verification,' U.S. Patent 4 513 437, Apr. 23, 1985
  28. Seiko, 'Seiko,' Japan Patent JP H6-67 799, 1994
  29. E. Baron, A. Prishvin, Z. Bar-Itzhak, and V. Korsensky, 'Handwriting input apparatus for handwriting recognition using more than one sensing technique,' U.S. Patent 5 517 579, May 14, 1996
  30. S. Hidenori, 'Pen type input device,' Japan Patent Application 2000-330 717, Nov. 30, 2000
  31. 'Method and device for tracking an object,' U.S. European Patent 02 071 324, Sept. 12, 2002
  32. 'Method for utilization of a gyroscopic or inertial device as a user interface mechanism for computer,' U.S. European Patent 0 188 896, Nov. 22, 2002
  33. G. J. Grimes, 'Digital data entry glove interface device,' U.S. Patent US 4 414 537, Nov. 8, 1983
  34. C. Verplaetse, 'Inertial proprioceptive devices: self-motion- sensing toys and tools,' IBM Systems Journal, vol. 34, no. 3 & 4, pp. 639-650, 1996
  35. K. Matsushima, 'Coordinate detecting apparatus having acceleration detectors,' U.S. Patent US 5 587 558, Dec. 24, 1996
  36. B. J. Glynn, 'Computer apparatus input device for three-dimensional information,' U.S. Patent US 5 181 181, 1993 19, Jan
  37. M. Tokioka, A. Tanaka, Y. Yoshimura, R. Yanagisawa, K. Kobayshi, and H. Sato, 'Gesture input method and apparatus,' U.S. Patent US 5 714 698, Feb. 3, 1998
  38. T. Miyagawa, Y. Yonezawa, K. Itoh, and M. Hashimoto, 'Handwriting pattern reproduction using 3D inertial measurement of handwriting movement,' Transactions of the Society of Instrument and Control Engineers, vol. 38, pp. 1-8, Jan. 2002 https://doi.org/10.9746/sicetr1965.38.1
  39. M. LaBiche and A. Derouen, 'Spatial input apparatus,' U.S. Patent US 4 839 838, June 13, 1989
  40. T. Kirabayashi, 'Multidimensional signal input device,' U.S. Patent US 5 329 276, July 12, 1994
  41. C. W. C. Jr., T. Hovden, G. C. Smith, S. P. Hotelling, and M. K. Sabina, 'Electronic pointing apparatus and method,' U.S. Patent US 5 825 350, Oct. 20, 1998
  42. A. Noureldin, D. Irvine-Halliday, and M. M. P. Mintchev, 'Accuracy limitations of fog-based continuous measurement-while-drilling surveying instruments for horizontal wells,' IEEE Transactions on Instrumentation and Measurement, vol. 51, no. 6, pp. 1177-1191, Dec. 2002 https://doi.org/10.1109/TIM.2002.808025
  43. J. R. Huddle, 'Trends in inertial systems technology for high accuracy AUV navigation,' in Proceedings of the 1998 workshop on autonomous underwater vehicles, 1998. AUV'98, Aug. 1998, pp. 63-73 https://doi.org/10.1109/AUV.1998.744442
  44. W. Chang, D. Y. Kim, K. H. Kang, E.-S. Choi, and W.-C. Bang, 'Method and apparatus for inputting information spatially,' Korea A 2004-0 107 144, Dec. 20, 2004
  45. W. Chang, D. Y. Kim, K. H. K. and Eun-Seok Choi, and W.-C. Bang, 'Apparatus and method for inputting information spatially,' Korea A 2004-0 108 218, Dec. 23, 2004
  46. E.-S. Choi, D. Y. Kim, W.-C. Bang, W. Chang, and K. H. Kang, 'a input system based on three dimensional inertial navigation system and method for trajectory estimation thereof,' Korea A 2004-0 082 128, Sept. 24, 2004
  47. W. Chang, D. Y. Kim, K. H. Kang, E.-S. Choi, and W.-C. Bang, 'Pen-shaped input device using inertial measurement units and method thereof,' Korea A 2004-0 082 558, Sept. 30, 2004
  48. J. S. A. Hepburn, J. Z. Zywiel, and D. B. Reid, 'INS reset removal for enhanced post-flight performance analysis,' in Position location and navigation symposium, 1990. Record. The 1990's - A decade of Excellence in the navigation sciences. IEEE PLANS'90., Las Vegas, NV, USA, Mar. 1990, pp. 226-230 https://doi.org/10.1109/PLANS.1990.66181
  49. W. Sohne, O. Heinze, and E. Groten, 'Integrated INS/GPS system for high precision navigation applications,' in IEEE 1994 Position Location and Navigation Symposium, Apr. 1994, pp. 310-313 https://doi.org/10.1109/PLANS.1994.303329
  50. B. Stieler, 'Gyroscopic goniometry during quasistatic and kinematic conditions using a new and conventional methods for aiding,' Optical gyros and their application, p. 13/1, 1999
  51. W. C. Bang, D. Y. Kim, W. Chang, K. H. Kang, and E.-S. Choi, 'Method and apparatus for compensating the acceleration error and inertial navigation system using thereof,' Korea A 2004-0 107 968, Dec. 23, 2004
  52. W.-C. Bang, D.-Y. Kim, W. Chang, K. H. Kang, and E.-S. Choi, 'Motion recognition system capable of distinguishment a stroke for writing motion and method thereof,' Korea A 2004-0 081 855, Sept. 23, 2004
  53. W.-C. Bang, W. Chang, E.-S. Choi, K.-H. Kang, and D.-Y. Kim, 'Design of self-contained gesture-based input device for human robot interface,' Oct. 2004, to be published
  54. W.-C. Bang, D.-Y. Kim, W. Chang, K. H. Kang, and E.-S. Choi, 'Motion recognition system using a imaginary writing plane and method thereof,' Korea A 2004-0 081 854, Sept. 23, 2004
  55. E. S. Choi, W. Chang, W. C. Bang, J. Yang, S. J. Cho, J. K. Oh, J. K. Cho, and D. Y. Kim, 'Development of the gyro-free handwriting input device based on inertial navigation system (ins) theory,' in SICE Annual Conference in Sapporo, Hokkaido Institute of Technology, Sapporo, Japan, Aug. 2004, pp. 1176-1181
  56. T. Sakaguchi, T. Kanamori, H. Katayose, K. Saot, and S. Inokuchi, 'Human motion capture by integrating gyroscopes and accelerometers,' in Proceedings of IEEE/SICE/RSJ International Conference on Multisensor Fusion and Integration for Intelligent Systems, Dec. 1996, pp. 470-475 https://doi.org/10.1109/MFI.1996.572219
  57. S. J. Cho and J. H. Kim, 'Bayesian network modeling of hangul characters for on-line handwritten recognition,' in Proc. Seventh International Conference on Document Analysis and Recognition, Edinburgh, Scotland, Aug. 2003, pp. 207-211 https://doi.org/10.1109/ICDAR.2003.1227660
  58. S. J. Cho and J. H. Kim, 'Bayesian network modeling of strokes and their relationships for on-line handwriting recognition,' Pattern Recognition, vol. 37, no. 2, pp. 253-264, Feb. 2004 https://doi.org/10.1016/j.patcog.2003.01.001
  59. S. J. Cho, J. K. Oh, W. C. Bang, W. Chang, E. S. Choi, Y. Jing, J. K. Cho, and D. Y. Kim, 'Magic wand: A hand-drawn gesture input device in 3-d space with inertial sensors,' in Proc. Samsung Tech. Conference 2004. Giheung, Kyonggi, Korea: SAIT, Nov. 2004, p. 35
  60. F. Jensen, An Introduction to Bayesian Networks. New York: Springer, 1996
  61. K. H. Kang, D. Y. Kim, W.-C. Bang, E.-S. Choi, and W. Chang, 'Control device of electronic devices based on motion,' Korea A 2004-0 081 270, Sept. 21, 2004
  62. S. Park, 'Apparatus and method for move recognizing of mobile phone,' Korea Patent 2001-0 017 203, Mar. 5, 2001
  63. D. H. Kang, J. H. Baek, W. B. Park, and K. H. Lee, 'Method and device for executing pedometer in wireless terminal,' Korea A 2004-0 093 811, Nov. 19, 2004