Browse > Article

Towards 3D Modeling of Buildings using Mobile Augmented Reality and Aerial Photographs  

Kim, Se-Hwan (Computer Science Department, University of California, Santa Barbara)
Ventura, Jonathan (Computer Science Department, University of California)
Chang, Jae-Sik (Telecommunication R&D Center, Samsung Electronics)
Lee, Tae-Hee (Computer Science Department, University of California, Los Angeles)
Hollerer, Tobias (Computer Science Department, University of California, Santa Barbara)
Publication Information
Journal of the Institute of Electronics Engineers of Korea CI / v.46, no.2, 2009 , pp. 84-91 More about this Journal
Abstract
This paper presents an online partial 3D modeling methodology that uses a mobile augmented reality system and aerial photographs, and a tracking methodology that compares the 3D model with a video image. Instead of relying on models which are created in advance, the system generates a 3D model for a real building on the fly by combining frontal and aerial views. A user's initial pose is estimated using an aerial photograph, which is retrieved from a database according to the user's GPS coordinates, and an inertial sensor which measures pitch. We detect edges of the rooftop based on Graph cut, and find edges and a corner of the bottom by minimizing the proposed cost function. To track the user's position and orientation in real-time, feature-based tracking is carried out based on salient points on the edges and the sides of a building the user is keeping in view. We implemented camera pose estimators using both a least squares estimator and an unscented Kalman filter (UKF). We evaluated the speed and accuracy of both approaches, and we demonstrated the usefulness of our computations as important building blocks for an Anywhere Augmentation scenario.
Keywords
Outdoor augmented reality; online modeling; feature-based tracking; UKF; camera pose estimation;
Citations & Related Records
연도 인용수 순위
  • Reference
1 G. Reitmayr, and T. Drummond, 'Going out: Robust model-based tracking for outdoor augmented reality,' Proc. of IEEE ISMAR'06, pp. 109-118, 2006
2 E. Rosten, and T. Drummond, T., 'Fusing points and lines for high performance tracking,' Proc. of ICCV'05, pp. 1508-1511, 2005
3 A. Davison, W. Mayol, and D. Murray, 'Real-time localisation and mapping with wearable active vision,' Proc. of ISMAR'03, 2003
4 J. Wither, S. DiVerdi, and T. Hollerer, 'Using aerial photographs for improved mobile ar annotation,' Proc. of IEEE ISMAR'06, pp. 159-162, 2006
5 Y. Boykov and M.-P. Jolly, 'Interactive graph cuts for optimal boundary & region segmentation of objects in n-d images,' IEEE ICCV'01, pp. 105-112, 2001
6 S. Haykin, Kalman Filtering and Neural Networks, John Wiley & Sons, Inc., 2001
7 T. Lee and T. Hollerer, 'Hybrid Feature Tracking and User Interaction for Markerless Augmented Reality,' Proc. of IEEE VR'08, pp. 145-152, 2008
8 G. Simon, A. Fitzgibbon, and A. Zisserman, 'Markerless tracking using planar structures in the scene,' Proc. of ISAR'00, pp. 120-128, 2000
9 Z. Zhang, 'A flexible new technique for camera calibration,' IEEE Transactions on PAMI, 22, 11, pp. 1330-1334, 2008   ScienceOn
10 T. Hollerer, J. Wither, and S. DiVerdi, Anywhere Augmentation: Towards Mobile Augmented Reality in Unprepared Environments, G. Gartner, M.P. Peterson, and W. Cartwright (Eds.), Location Based Services and TeleCartography, Springer Verlag, 2007
11 G. King, W. Piekarski, and B. Thomas, 'Arvino - outdoor augmented reality visualisation of viticulture gis data,' Proc. of IEEE ISMAR'05, pp. 52-55, 2005
12 J. Shi, and C. Tomasi, 'Good features to track,' Proc. of CVPR'94, pp. 593-600, 1994
13 Intel, OpenCV Library, http://www.intel.com/technology/computing/opencv/, 2007
14 D. G. Lowe, 'Distinctive Image Features from Scale-Invariant Keypoints,' IJCV, vol. 60, no. 2, pp.91–110, 2004   DOI   ScienceOn
15 J. Hershberger and J. Snoeyink, 'Speeding up the douglas-peucker line simplification algorithm,' Spatial Data Handling. IGU Commission on GIS, pp. 134-143, 1992
16 S. Kim, S. DiVerdi, J. S. Chang, T. Kang, R. Iltis, and T. Hollerer, 'Implicit 3D Modeling and Tracking for Anywhere Augmentation,' VRST'07, pp. 19-27, 2007
17 S. Julier, and J. Uhlmann, 'Unscented filtering and nonlinear estimation,' Proc. of the IEEE, vol. 92, pp. 401-422, 2004   DOI   ScienceOn
18 C. Rother, V. Kolmogorov, and A. Blake, 'grabcut: interactive foreground extraction using iterated graph cuts,' ACM Trans. Graph, 23(3), pp. 309-314, 2004   DOI   ScienceOn
19 B. Lucas, and T. Kanade, 'An iterative image registration technique with an application to stereo vision,' Proc. of the International Joint Conference on AI, pp. 674-679, 1981