Browse > Article
http://dx.doi.org/10.4218/etrij.2017-0047

Robust Multithreaded Object Tracker through Occlusions for Spatial Augmented Reality  

Lee, Ahyun (Hyper-connected Communication Research Laboratory, ETRI)
Jang, Insung (Hyper-connected Communication Research Laboratory, ETRI)
Publication Information
ETRI Journal / v.40, no.2, 2018 , pp. 246-256 More about this Journal
Abstract
A spatial augmented reality (SAR) system enables a virtual image to be projected onto the surface of a real-world object and the user to intuitively control the image using a tangible interface. However, occlusions frequently occur, such as a sudden change in the lighting environment or the generation of obstacles. We propose a robust object tracker based on a multithreaded system, which can track an object robustly through occlusions. Our multithreaded tracker is divided into two threads: the detection thread detects distinctive features in a frame-to-frame manner, and the tracking thread tracks features periodically using an optical-flow-based tracking method. Consequently, although the speed of the detection thread is considerably slow, we achieve real-time performance owing to the multithreaded configuration. Moreover, the proposed outlier filtering automatically updates a random sample consensus distance threshold for eliminating outliers according to environmental changes. Experimental results show that our approach tracks an object robustly in real-time in an SAR environment where there are frequent occlusions occurring from augmented projection images.
Keywords
Augmented reality; RANSAC; Spatial augmented reality; Stitching; Tracking;
Citations & Related Records
Times Cited By KSCI : 3  (Citation Analysis)
연도 인용수 순위
1 L. Cheng, M. Li, Y. Liu, W. Cai, Y. Chen, and K. Yang, "Remote Sensing Image Matching by Integrating Affine Invariant Feature Extraction and RANSAC," Comput. Electr. Eng., vol. 38, no. 4, 2012, pp. 1023-1032.   DOI
2 A. Lee and J.-H. Lee, "Multi-threaded Tracker with Outlier Filtering for Spatial Augmented Reality," in Int. Tech. Conf. Circuits Syst., Comput. Commun. (ITC-CSCC), Seoul, Rep. of Korea, July 2015, pp. 494-495.
3 J.-H. Lee et al., "Calibration Issues in FRC: Camera, Projector, Kinematics Based Hybrid Approach," in Proc. Ubiquitous Robots Ambient Intell., Daejeon, Rep. of Korea, Nov. 2012, pp. 218-219.
4 J.-H. Lee, "An Analytic Solution to Projector Pose Estimation Problem," ETRI J., vol. 34, no. 6, Dec. 2012, pp. 978-981.   DOI
5 Z. Zhang, "A Flexible New Technique for Camera Calibration," IEEE Trans. Pattern Anal. Mach. Intell., vol. 22, no. 11, Nov. 2000, pp. 1330-1334.   DOI
6 J. Weng, P. Cohen, and M. Herniou, "Camera Calibration with Distortion Models and Accuracy Evaluation," IEEE Trans. Pattern Anal. Mach. Intell., vol. 14, no. 10, 1992, pp. 965-980.   DOI
7 R. Tsai, "A Versatile Camera Calibration Technique for High-Accuracy 3D Machine Vision Metrology Using Off- the-Shelf TV Cameras and Lenses," IEEE J. Robot. Autom., vol. 3, no. 4, 1987, pp. 323-344.   DOI
8 J.-Y. Bouguet, "Pyramidal Implementation of the Affine Lucas Kanade Feature Tracker Description of the Algorithm," Intel Corp., vol. 5, 2001, pp. 1-10.
9 J.M. Prewitt and M.L. Mendelsohn, "The Analysis of Cell Images," Annu. NY Acad. Sci, vol. 128, no. 3, 1966, pp. 1035-1053.
10 J.-W. Choi, D. Moon, and H.H. Yoo, "Robust Multi-person Tracking for Real-Time Intelligent Video Surveillance," ETRI J., vol. 37, no. 3, June 2015, pp. 551-561.   DOI
11 M. Fornasier and H. Rauhut, "Iterative Thresholding Algorithms," Appl. Comput. Harmon. Anal., vol. 25, no. 2, 2008, pp. 187-208.   DOI
12 W.H. Tsai, "Moment-Preserving Thresholding: A New Approach," Comput. Vis. Graph. Image Process., vol. 29, no. 3, 1985, pp. 377-393.   DOI
13 W. Doyle, "Operations Useful for Similarity-Invariant Pattern Recognition," J. ACM, vol. 9, no. 2, 1962, pp. 259- 267.   DOI
14 N. Otsu, "A Threshold Selection Method from Gray-Level Histograms," IEEE Trans. Sys. Man. Cybern., vol. 9, 1975, pp. 62-66.
15 J.H. Cha, Y.S. Jeon, Y.S. Moon, and S.H. Lee, "Seamless and Fast Panoramic Image Stitching," in IEEE Int. Conf. Consumer Electron. (ICCE), Las Vegas, NV, USA, Jan. 2012, pp. 29-30.
16 J.H. Lee et al., "FRC Based Augment Reality for Aiding Cooperative Activities," in 2013 IEEE RO-MAN, Gyeongju, Rep. of Korea, Aug. 2013, pp. 294-295.
17 F. Zhang and F. Liu, "Parallax-Tolerant Image Stitching," in Proc. IEEE Conf. Comput. Vis. Pattern Recogn., Columbus, OH, USA, June 2014, pp. 3262-3269.
18 Y. Xiong and K. Pulli, "Sequential Image Stitching for Nobile Panoramas," in IEEE Int. Conf. Inf., Commun. Signal Process. (ICICS), Macau, China, Dec. 2009.
19 M. Brown and D.G. Lowe, "Automatic Panoramic Image Stitching Using Invariant Features," Int. J. Comput. Vis., vol. 74, no. 1, 2007, pp. 59-73.   DOI
20 H. Benko, R. Jota, and A. Wilson, "MirageTable: Freehand Interaction on a Projected Augmented Reality Tabletop," in Proc. SIGCHI Conf. Human Factors Comput. Syst., Austin, TX, USA, May 2012, pp. 199-208.
21 R. Ziola, S. Grampurohit, N. Landes, J. Fogarty, and B. Harrison, "Examining Interaction with General-Purpose Object Recognition in LEGO OASIS," in 2011 IEEE Symp. Vis. Lang. Human-Centric Comput. (VL/HCC), Pittsburgh, PA, USA, Sept. 2011, pp. 65-68.
22 A. Lee, J.D. Suh, and J. Lee, "Interactive Design of Planar Curves Based on Spatial Augmented Reality," in Proc. Companion Publication Int. Conf. Intell. User Interfaces Companion, Santa Monica, CA, USA, Mar. 2013, pp. 53-54.
23 A. Lee, J.H. Lee, and J. Kim, "Data-Driven Kinematic Control for Robotic Spatial Augmented Reality System with Loose Kinematic Specifications," ETRI J., vol. 38, no. 2, Apr. 2016, pp. 337-346.   DOI
24 D. Lowe, "Distinctive Image Features from Scale-Invariant Keypoints," Int. J. Comput. Vis., vol. 60, no. 2, 2004, pp. 91-110.   DOI
25 H. Bay, T. Tuytelaars, and L. Van Gool, "Speeded-Up Robust Features (SURF)," Eur. Conf. Comput. Vis., vol. 3951, 2006, pp. 404-417.
26 E. Rublee, T. Tuytelaars, and L. Van Gool, "ORB: An Efficient Alternative to SIFT or SURF," 2011 IEEE Int. Conf. Comput. Vis. (ICCV), Barcelona, Spain, Nov. 2011, pp. 2564-2571.
27 T. Lee and T. Hollerer, "Multithreaded Hybrid Feature Tracking for Markerless Augmented Reality," IEEE Trans. Vis. Comput. Graph., vol. 15, no. 3, 2009, pp. 355-368.   DOI
28 A. Davison, I.D. Reid, N.D. Molton, and O. Stasse, "MonoSLAM: Real-Time Single Camera SLAM," IEEE Trans. Pattern Anal. Mach. Intell., vol. 29, no. 6, 2007, pp. 1052-1067.   DOI
29 X. Wang, Z. Yao, and Z. Yang, "The Use of Object Tracking in Visual SLAM," in IEEE Int. Conf. Appl. Syst. Innovation (ICASI), Sapporo, Japan, May 2017, pp. 850- 853.
30 M.A. Fischler and C.B. Robert, "Random Sample Consensus: A Paradigm for Model Fitting with Applications to Image Analysis and Automated Cartography," Commun. ACM, vol. 24, no. 6, 1981, pp. 381-395.   DOI
31 R. Mur-Artal, J.M.M. Montiel, and J.D. Tardos, "ORB- SLAM: A Versatile and Accurate Monocular SLAM System," IEEE Trans. Robot., vol. 31, no. 5, 2015, pp. 1147-1163.   DOI