Journal of the HCI Society of Korea (한국HCI학회논문지)

The HCI Society of Korea (한국HCI학회)
권호 표지

Research Trends and Case Study on Keypoint Recognition and Tracking for Augmented Reality in Mobile Devices

모바일 증강현실을 위한 특징점 인식, 추적 기술 및 사례 연구

  • 최희승 (한국과학기술연구원 영상미디어연구단) ;
  • 안상철 (한국과학기술연구원 영상미디어연구단) ;
  • 김익재 (한국과학기술연구원 영상미디어연구단)
  • Received : 2015.07.13
  • Accepted : 2015.08.27
  • Published : 2015.11.30
Download PDF
⟨ Previous Next ⟩

Abstract

In recent years, keypoint recognition and tracking technologies are considered as crucial task in many practical systems for markerless augmented reality. The keypoint recognition and technologies are widely studied in many research areas, including computer vision, robot navigation, human computer interaction, and etc. Moreover, due to the rapid growth of mobile market related to augmented reality applications, several effective keypoint-based matching and tracking methods have been introduced by considering mobile embedded systems. Therefore, in this paper, we extensively analyze the recent research trends on keypoint-based recognition and tracking with several core components: keypoint detection, description, matching, and tracking. Then, we also present one of our research related to mobile augmented reality, named mobile tour guide system, by real-time recognition and tracking of tour maps on mobile devices.

최근 증강현실 분야에서 특징점 인식 및 추적 기술은 비마커 기반의 증강 현실 서비스 구현에 중요한 역할을 담당하고 있다. 특징점 인식 및 추적 기술은 오래 전부터 컴퓨터 비전 등 여러 분야의 많은 연구자들에 의해 심도 있게 연구되어 왔으며, 특히 최근 급성장하고 있는 모바일 관련 시스템에 적용하기 위해 모바일 임베디드 환경에 접목 가능한 특징점 기반의 다양한 인식 및 추적 기술들이 소개되고 있다. 따라서 본 논문에서는 널리 활용되고 있는 특징점 기반의 매칭 및 추적의 다양한 핵심 요소 기술 (특징점 추출, 특징점 기술, 특징점 매칭 및 추적)에 대한 최신 동향을 분석하고, 본 한국과학기술연구원 연구팀이 수행한 모바일 증강현실 서비스 관련 사례 연구인 관광 지도 인식 및 추적 연구를 소개하고자 한다.

Keywords

References

  1. Tuytelaars, T. and Mikolajczyk, K. Local Invariant Feature Detectors: A Survey. Foundations and Trends in Computer Graphics and Vision. Vol. 3. No. 3. pp. 177-280. 2007. https://doi.org/10.1561/0600000017
  2. Harris, C. and Stephens, M. A combined corner and edge detector. Proceedings of the 4th Alvey Vision Conference. pp. 147-151. 1988.
  3. Shi, J. and Tomasi, C. Good Features to Track. Computer Vision and Pattern Recognition. 1994.
  4. Lindeberg, T. Feature Detection with Automatic Scale Selection. International Journal of Computer Vision. Vol. 30. No. 2. pp. 79-116. 1998. https://doi.org/10.1023/A:1008045108935
  5. Mikolajczyk, K. and Schmid, C. Indexing based on Scale Invariant Interest Points. Proceedings of the International Conference on Computer Vision. pp. 525-531. 2001.
  6. Lowe, D. G. Distinctive Image Features from Scale Invariant Keypoints. International Journal of Computer Vision. Vol. 60. No. 2. pp. 91-110. 2004. https://doi.org/10.1023/B:VISI.0000029664.99615.94
  7. Mikolajczyk, K. and Schmid, C. Scale & Affine Invariant Interest Point Detectors. International Journal of Computer Vision. Vol. 60. No. 1. pp. 63-86. 2004. https://doi.org/10.1023/B:VISI.0000027790.02288.f2
  8. Smith, S. M. and Brady, J. M. SUSAN- A New Approach to Low Level Image Processing. International Journal of Computer Vision. Vol. 23. No. 1. pp. 45-78. 1997. https://doi.org/10.1023/A:1007963824710
  9. Rosten, E. and Drummond, T. Fusing points and lines for high performance tracking. Proceedings of the International Conference on Computer Vision. pp. 1508-1511. 2005.
  10. Klein, G. and Murray, D. Parallel Tracking and Mapping on a Camera Phone. Proceedings of International Symposium on Mixed and Augmented Reality. pp. 83-86. 2009.
  11. Mair, E., Hager, G. D., Burschka, D., Suppa, M. and Hirzinger, D. Adaptive and Generic Corner Detection Based on the Accelerated Segment Test. European Conference on Computer Vision. 2010.
  12. Leutenegger, S., Chli, M. and Siegwart, R. Y. BRISK: Binary robust invariant scalable keypoints. International Conference on Computer Vision. 2011.
  13. Matas, J., Chum, O., Urban, M. and Pajdla, T. Robust wide-baseline stereo from maximally stable extremal regions. Proceedings of the British Machine Vision Conference, pp. 384-393, 2002.
  14. Ren X. and Malik, J. Learning a classification model for segmentation. Proceedings of the International Conference on Computer Vision. pp. 10-17, 2003.
  15. Bay, H., Tuytelaars, T., Van Gool, and L. SURF: Speeded Up Robust Features. European Conference on Computer Vision. 2006.
  16. Cornelis, N. and Van Gool, L. Fast scale invariant feature detection and matching on programmable graphics hardware. Computer Vision and Pattern Recognition Workshops. 2008.
  17. Ke, Y. and Sukthankar, R. PCA-SIFT: A More Distinctive Representation for Local Image Descriptors. Computer Vision and Pattern Recognition. 2004.
  18. Mikolajczyk, K. and Schmid, C. A performance evaluation of local descriptors. IEEE Transactions on Pattern Analysis and Machine Intelligence. Vol. 10, No. 27, pp. 1615-1630. 2005.
  19. Tola, E., Lepetit, V. and Fua, P. DAISY: An Efficient Dense Descriptor Applied to Wide Baseline Stereo. IEEE Transactions on Pattern Analysis and Machine Intelligence. Vol. 32, No. 5, pp. 815-830. 2010. https://doi.org/10.1109/TPAMI.2009.77
  20. Calonder, M., Lepetit, V., Strecha, C. and Fua, P. Brief: Binary robust independent elementary features. European Conference on Computer Vision. 2010.
  21. Rublee, E., Rabaud, V., Konolige, K. and Bradski, G. ORB: an efficient alternative to SIFT or SURF. Proceedings of the International Conference on Computer Vision. pp. 2564-2571. 2011.
  22. Alahi, A., Ortiz, R. and Vandergheynst, P. FREAK: Fast retina keypoints. Computer Vision and Pattern Recognition. 2012.
  23. Grauman, K. and Leibe, B. Visual Object Recognition. Morgan & Claypool Publisher. 2010.
  24. Freidman, J., Bentley, J. and Finkel, A. An Algorithm for Finding Best Matches in Logarithmic Expected Time. ACM Transactions on Mathematical Software. Vol. 3, No. 3, pp. 209-226. 1997.
  25. Muja, M. and Lowe, D. Fast approximate nearest neighbors with automatic algorithm configuration. International Conference on Computer Vision Theory and Application. 2009.
  26. Silpa-Anan, C. and Hartley, R. Optimised kd-trees for fast image descriptor matching, Computer Vision and Pattern Recognition. 2008.
  27. Lecture on Kd-tree. www.cise.ufl.edu/class/cot5520fa09. July 10. 2015.
  28. Lepetit, V. and Fua, P. Keypoint Recognition using Randomized Trees. IEEE Transactions on Pattern Analysis and Machine Intelligence. Vol. 28, No. 9, pp. 1465-1479. 2006. https://doi.org/10.1109/TPAMI.2006.188
  29. Ozuysal, M., Calonder, M., Lepetit, V. and Fua, P. Fast Keypoint Recognition Using Random Ferns. IEEE Transactions on Pattern Analysis and Machine Intelligence. Vol. 32, No. 3, pp. 448-461. 2010. https://doi.org/10.1109/TPAMI.2009.23
  30. Shakhnarovich, G., Darrell, T. and Indyk, P. Nearest-Neighbor Methods in Learning and Vision: Theory and Practice. MIT Press. 2006.
  31. Locality sensitive hashing. https://micvog.files.wordpress.com/2013/08/lsh1.png. July 10. 2015.
  32. Fischler, M. and Bolles, R. Random Sampling Consensus: A Paradigm for Model Fitting with Application to Image Analysis and Automated Cartography. Communications of the ACM. Vol. 24. No. 6. pp. 381-395. 1981. https://doi.org/10.1145/358669.358692
  33. Ballard, D. H. Generalizing the Hough Transform to Detect Arbitrary Shapes. Pattern Recognition, Vol.13. No.2. pp.111-122. 1981. https://doi.org/10.1016/0031-3203(81)90009-1
  34. Kalal, Z., Mikolajczyk, K. and Matas, J. Trackinglearning-detection. IEEE Transactions on Pattern Analysis and Machine Intelligence. Vol. 34. No. 7 pp. 1409-1422. 2012. https://doi.org/10.1109/TPAMI.2011.239
  35. Hare, S., Saffari, A. and Torr, P. Efficient online structured output learning for keypoint-based object tracking. Computer Vision and Pattern Recognition. 2012.
  36. Zhang, K., Zhang, L. and Yang, M. Real-time compressive tracking. European Conference on Computer Vision. 2012.
  37. Bouachir, W. and Bilodeau, G. Structure-aware keypoint tracking for partial occlusion handling. IEEE Winter Conference on Applications of Computer Vision. 2014.
  38. Lucas, B. D. and Kanade, T. An iterative image registration technique with an application in stereo vision. International Joint Conferences on Artificial Intelligence. 1981.
  39. Yu, Q., Dinh, T. B. and Medioni, G. Online tracking and reacquisition using co-trained generative and discriminative trackers. European Conference on Computer Vision. 2008.
  40. Zhou, H., Yuan, Y. and Shi, C. Object tracking using sift features and mean shift. Computer Vision and Image Understand. Vol. 113. No. 3. pp. 345-352. 2009. https://doi.org/10.1016/j.cviu.2008.08.006
  41. He, W., Yamashita, T., Lu, H. and Lao, S. Surf tracking. International Conference on Computer Vision. 1586-1592. 2009.
  42. Comaniciu, D., Ramesh, V. and Meer, P. Kernelbased object tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence. Vol. 25. No. 5. pp. 564-577. 2003. https://doi.org/10.1109/TPAMI.2003.1195991
  43. Li, M., Zhang, Z., Huang, K. and Tan, T. Robust visual tracking based on simplified biologically inspired features. International Conference on Image Processing. 2009.
  44. Ronfard, R. Region-based strategies for active contour models, International Journal of Computer Vision, Vol. 13 No. 2. pp. 229-251. 1994. https://doi.org/10.1007/BF01427153
  45. Kalman, R. A new approach to linear filtering and prediction problems. Journal of Fluids Engineering. Vol. 82 No. 1 pp. 35-45. 1960.
  46. Welch, G. and Bishop, G. An introduction to the Kalman filter. University of North Carolina at Chapel Hill Chapel Hill. Technical Report TR95041. 2000.
  47. Arulampalam, M., Maskell, S., Gordon, N. and Clapp, T. A tutorial on particle filters for online nonlinear/non-Gaussian Bayesian tracking. IEEE Transactions on Signal Processing. Vol. 50. No. 2 pp. 174-188. 2002. https://doi.org/10.1109/78.978374
  48. Julier, S. J. and Uhlmann, J. K. A new extension of the Kalman filter to nonlinear systems. International Symposium on Aerospace Defense Sensing, Simulations and Controls, 1997.
  49. Wan, E. and Merwe, R. The unscented Kalman filter for nonlinear estimation. Adaptive Systems for Signal Processing, Communications, and Control Symposium. 2000.
  50. Tomasi, C. and Kanade, T. Detection and tracking of point features. Carnegie Mellon University. 1991.
  51. Heinly, J., Dunn, E. and Frahm, J. Comparative evaluation of binary features. European Conference on Computer Vision. 2012.