Browse > Article
http://dx.doi.org/10.3837/tiis.2021.08.017

Pose Tracking of Moving Sensor using Monocular Camera and IMU Sensor  

Jung, Sukwoo (Contents Convergence Research Center, Korea Electronics Techonology Institute)
Park, Seho (Contents Convergence Research Center, Korea Electronics Techonology Institute)
Lee, KyungTaek (Contents Convergence Research Center, Korea Electronics Techonology Institute)
Publication Information
KSII Transactions on Internet and Information Systems (TIIS) / v.15, no.8, 2021 , pp. 3011-3024 More about this Journal
Abstract
Pose estimation of the sensor is important issue in many applications such as robotics, navigation, tracking, and Augmented Reality. This paper proposes visual-inertial integration system appropriate for dynamically moving condition of the sensor. The orientation estimated from Inertial Measurement Unit (IMU) sensor is used to calculate the essential matrix based on the intrinsic parameters of the camera. Using the epipolar geometry, the outliers of the feature point matching are eliminated in the image sequences. The pose of the sensor can be obtained from the feature point matching. The use of IMU sensor can help initially eliminate erroneous point matches in the image of dynamic scene. After the outliers are removed from the feature points, these selected feature points matching relations are used to calculate the precise fundamental matrix. Finally, with the feature point matching relation, the pose of the sensor is estimated. The proposed procedure was implemented and tested, comparing with the existing methods. Experimental results have shown the effectiveness of the technique proposed in this paper.
Keywords
IMU; pose estimation; pose tracking; sensor fusion; visual-inertial fusion;
Citations & Related Records
Times Cited By KSCI : 1  (Citation Analysis)
연도 인용수 순위
1 A. R. Jimenez, F. Seco, J. C. Prieto, J. Guevara, "Indoor pedestrian navigation using an INS/EKF framework for yaw drift reduction and a foot-mounted IMU," in Proc. of 2010 7th Workshop on Positioning, Navigation and Communication, pp. 135-143, 2010.
2 F. Caron, E. Duflos, D. Pomorski, P. Vanheeghe, "GPS/IMU data fusion using multisensor Kalman filtering: introduction of contextual aspects," Information Fusion, vol. 7, no. 2, pp. 221-230, 2006.   DOI
3 F. Subhan, S. Ahmed, S. Haider, S. Saleem, A. Khan, S. Ahmed, M. Numan, "Hybrid Indoor Position Estimation using K-NN and MinMax," KSII Transactions on Internet and Information Systems, vol. 13, no. 9, pp. 4408-4428, 2019.   DOI
4 H. Bay, A. Ess, T. Tuytelaars, L. V. Gool, "SURF: Speeded Up Robust Features," Computer Vision and Image Understanding, pp. 404-417, 2006.
5 D. G. Lowe, "Object recognition from local scale-invariant features," in Proc. of the Seventh IEEE International Conference on Computer Vision, vol. 2, pp. 1150-1157, 1999.
6 M. Calonder, V. Lepetit, C. Strecha, P. Fua, "BRIEF: Binary Robust Independent Elementary Features," in Proc. of ECCV 2010: Proc. of the 11th European Conference on Computer Vision, pp.778-792, 2010.
7 K. Kumar, A. Varghese, P. K. Reddy, N. Narendra, P. Swamy, M. G. Chandra, P. Balamuralidhar, "An Improved Tracking using IMU and Vision Fusion for Mobile Augmented Reality Applications," International Journal of Multimedia and its Applications, vol. 6, no. 5, 2014.
8 S. Jung, S. Song, M. Chang, S. Park, "Range Image Registration based on 2D Synthetic Images," Computer-Aided Design, vol. 94, pp. 16-27, 2018.   DOI
9 S. Jung, Y. Cho, K. Lee, M. Chang, "Moving Object Detection with Single Moving Camera and IMU Sensor using Mask R-CNN Instance Image Segmentation," International Journal of Precision Engineering and Manufacturing, vol. 22, pp. 1049-1059, 2021.   DOI
10 S. Jung, Y. Cho, D. Kim, M. Chang, "Moving Object Detection from Moving Camera Image Sequences Using an Inertial Measurement Unit Sensor," Applied Sciences, vol. 10, pp. 268, 2020.   DOI
11 G. Bradski, "The OpenCV Library," Dr. Dobb's Journal of Software Tools, 2000.
12 J. L., J. A. Besada, A. M. Bernardos, P. Tarrio, J. R. Casar, "A novel system for object pose estimation using fused vision and inertial data," Information Fusion, vol. 33, pp. 15-28, 2017.   DOI
13 E.D. Marti, D. Martin, J. Garcia, A. De la Escalera, J.M. Molina, J.M. Armingol, "Context-Aided Sensor Fusion for Enhanced Urban Navigation," Sensors, vol. 12, no. 12, pp. 16802-16837, 2012.   DOI
14 Y. Wu, X. Niu, J. Du, L. Chang, H. Tang, H. Zhang, "Artificial Marker and MEMS IMU-based Pose Estimation Method to Meet Multirotor UAV Landing Requirements," Sensors, vol. 19, no. 24, pp. 5428, 2019.   DOI
15 M. Blosch, S. Weiss, D. Scaramuzza, R. Siegwart, "Vision based MAV navigation in unknown and unstructured environments," in Proc. of 2010 IEEE International Conference on Robotics and Automation, pp. 21-28, 2010.
16 M. E. Ragab, G. F. Elkabbany, "A Parallel Implementation of Multiple Non-overlapping Cameras for Robot Pose Estimation," KSII Transactions on Internet and Information Systems, vol. 8, no. 11, pp. 4103-4117, 2014.   DOI
17 E. Foxlin, L. Naimak, "VIS-Tracker: A Wearable Vision-inertial Self-Tracker," in Proc. of IEEE VR2003, vol. 1, pp. 199, 2003.
18 H. Rehbinder, B. K. Ghosh, "Pose estimation using line-based dynamic vision and inertial sensors," IEEE Transactions on Automatic Control, vol. 48, no. 2, pp. 186-199, 2003.   DOI
19 E. Rublee, V. Rabaud, K. Konolige, G. Bradski, "ORB: An efficient alternative to SIFT or SURF," in Proc. of 2011 International Conference on Computer Vision, pp. 2564-2571, 2011.
20 P. Alcantarilla, J. Nuevo, A. Bartoli, "Fast Explicit Diffusion for Accelerated Features in Nonlinear Scale Spaces," in Proc. of British Machine Vision Conference 2013, vol. 13, pp. 1-11, 2013.
21 Y. Tian, Jie Zhang, J. Tan, "Adaptive-frame-rate monocular vision and IMU fusion for robust indoor positioning," in Proc. of 2013 IEEE International Conference on Robotics and Automation, pp. 2257-2262, 2013.
22 T. Kim, T.H. Park, "Extended Kalman Filter (EKF) Design for Vehicle Position Tracking Using Reliability Function of Radar and Lidar," Sensors, vol. 20, pp. 4126, 2020.   DOI
23 G. Qian, R. Chellappa, Qinfen Zheng, "Bayesian structure from motion using inertial information," in Proc. of International Conference on Image Processing, 2002.
24 F. M. Mirzaei, S. I. Roumeliotis, "A Kalman Filter-Based Algorithm for IMU-Camera Calibration: Observability Analysis and Performance Evaluation," IEEE Transactions on Robotics, vol. 24, no. 5, pp. 1143-1156, 2008.   DOI
25 L. Li, Y. Liu, T. Jiang, K. Wang, M. Fang, "Adaptive Trajectory Tracking of Nonholonomic Mobile Robots Using Vision-Based Position and Velocity Estimation," IEEE Transactions on Cybernetics, vol. 48, no. 2, pp. 571-582, 2018.   DOI
26 H. Deilamsalehy, T. C. Havens, "Sensor fused three-dimensional localization using IMU, camera and LiDAR," IEEE Sensors, pp. 1-3, 2016.
27 C. Yu, H. Lan, F. Gu, F. Yu, N. El-Sheimy, "A Map/INS/Wi-Fi Integrated System for Indoor Location-based Service Applications," Sensors, vol. 17, pp. 1272, 2017.   DOI
28 X. Li, Y. Wang, K. Khoshelham, "A Robust and Adaptive Complementary Kalman Filter based on Mahalanobis Distance for Ultra Wideband/Inertial Measurement Unit Fusion Positioning," Sensors, vol. 18, pp. 3435, 2018.   DOI