Journal of the Korean Society of Surveying, Geodesy, Photogrammetry and Cartography (한국측량학회지)

Korean Society of Surveying, Geodesy, Photogrammetry and Cartography (한국측량학회)
권호 표지
DOI QR코드

DOI QR Code

Seamline Determination from Images and Digital Maps for Image Mosaicking

모자이크 영상 생성을 위한 영상과 수치지도로부터 접합선 결정

  • Kim, Dong Han (Dept. of Environment, Energy & Geoinformatics, Sejong University) ;
  • Oh, Chae-Young (Technology & Support Dept., ESRI) ;
  • Lee, Dae Geon (Dept. of Environment, Energy & Geoinformatics, Sejong University) ;
  • Lee, Dong-Cheon (Dept. of Environment, Energy & Geoinformatics, Sejong University)
  • Received : 2018.11.05
  • Accepted : 2018.12.14
  • Published : 2018.12.31
Download PDF
⟨ Previous Next ⟩

Abstract

Image mosaicking, which combines several images into one image, is effective for analyzing images and important in various fields of spatial information such as a continuous image map. The crucial processes of the image mosaicking are optimal seamline determination and color correction of mosaicked images. In this study, the overlap regions were determined by SURF (Speeded Up Robust Features) for image matching. Based on the characteristics of the edges extracted by Canny filter, seamline candidates were selected from classified edges with their characteristics, and the edges were connected by using Dijkstra algorithm. In particular, anisotropic filter and image pyramid were applied to extract reliable seamlines. In addition, it was possible to determine seamlines effectively and efficiently by utilizing building and road layers from digital maps. Finally, histogram matching and seamline feathering were performed to improve visual quality of the mosaicked images.

여러 장의 영상을 조합하여 한 장의 영상으로 제작하는 영상 모자이크는 넓은 지역의 영상을 판독하고 분석하는데 효과적이며, 연속 영상지도 등 다양한 공간정보 분야의 활용에 중요하다. 영상 모자이크의 중요한 과정은 인접 영상의 중복지역에서 최적의 접합선 추출과 모자이크된 영상의 색조보정이다. 이를 위해 본 연구에서는 SURF(Speeded Up Robust Features)에 의한 영상정합을 수행하여 중복지역을 결정하였다. Canny 필터로 추출한 윤곽선의 특성에 따라 등급을 정하여 접합선이 될 가능성이 높은 윤곽선을 선별하고, Dijkstra 알고리즘을 사용하여 윤곽선들을 연결하여 접합선을 결정하였다. 특히 비등방성 필터와 영상 피라미드를 적용하여 신뢰성 있는 접합선을 추출할 수 있었다. 또한 수치지도의 건물과 도로 레이어를 이용하여 효과적이고 효율적인 접합선을 결정할 수 있었다. 최종적으로 인접 영상들의 색조를 조절하여 품질을 향상시키기 위하여 히스토그램 정합과 접합선 feathering을 수행하였다.

Keywords

GCRHBD_2018_v36n6_483_f0001.png 이미지

Fig. 1. Work flow of seamless image mosaicking

GCRHBD_2018_v36n6_483_f0002.png 이미지

Fig. 2. RANSAC algorithm to improve matching between adjacent images

GCRHBD_2018_v36n6_483_f0003.png 이미지

Fig. 3. Determination of overlap region with EOP information

GCRHBD_2018_v36n6_483_f0004.png 이미지

Fig. 4. Determination of overlap region using image matching

GCRHBD_2018_v36n6_483_f0005.png 이미지

Fig. 5. Canny edge detection

GCRHBD_2018_v36n6_483_f0006.png 이미지

Fig. 6. Edges after anisotropic diffusion with different coefficient and iteration

GCRHBD_2018_v36n6_483_f0007.png 이미지

Fig. 7. Edges in image pyramid after applying anisotropic diffusion

GCRHBD_2018_v36n6_483_f0008.png 이미지

Fig. 8. Seamline search scheme

GCRHBD_2018_v36n6_483_f0009.png 이미지

Fig. 9. Incorrectly determined seamlines

GCRHBD_2018_v36n6_483_f0010.png 이미지

Fig. 10. Mosaicking of image block

GCRHBD_2018_v36n6_483_f0011.png 이미지

Fig. 11. Components of an edge segment

GCRHBD_2018_v36n6_483_f0012.png 이미지

Fig. 12. Classification of edges based on proposed method

GCRHBD_2018_v36n6_483_f0013.png 이미지

Fig. 13. Proposed scheme for seamline determination

GCRHBD_2018_v36n6_483_f0014.png 이미지

Fig. 14. Cumulative histogram matching

GCRHBD_2018_v36n6_483_f0015.png 이미지

Fig. 15. Adjacent images before histogram matching

GCRHBD_2018_v36n6_483_f0016.png 이미지

Fig. 16. Histogram matching results from different schemes

GCRHBD_2018_v36n6_483_f0017.png 이미지

Fig. 17. Seamline feathering region

GCRHBD_2018_v36n6_483_f0018.png 이미지

Fig. 18. Seamline feathering region in overlap area

GCRHBD_2018_v36n6_483_f0019.png 이미지

Fig. 19. Demonstration of feathering by Laplacian pyramid blending algorithm

GCRHBD_2018_v36n6_483_f0020.png 이미지

Fig. 20. Aerial image: Case A

GCRHBD_2018_v36n6_483_f0021.png 이미지

Fig. 21. Aerial image: Case B

GCRHBD_2018_v36n6_483_f0022.png 이미지

Fig. 22. Terrestrial image: Case C

GCRHBD_2018_v36n6_483_f0023.png 이미지

Fig. 23. Terrestrial image: Case D

GCRHBD_2018_v36n6_483_f0024.png 이미지

Fig. 24. Building and road layers from digital map in overlap region of Case A

GCRHBD_2018_v36n6_483_f0025.png 이미지

Fig. 25. Seamline from image and mosaicked image of Case A

GCRHBD_2018_v36n6_483_f0026.png 이미지

Fig. 26. Seamline from digital map and mosaicked image of Case A

GCRHBD_2018_v36n6_483_f0027.png 이미지

Fig. 27. Seamline from image and mosaicked image of Case B

GCRHBD_2018_v36n6_483_f0028.png 이미지

Fig. 28. Seamline from image and mosaicked image of Case C

GCRHBD_2018_v36n6_483_f0029.png 이미지

Fig. 29. Seamline from images in strip 1 and mosaicked image of Case D

GCRHBD_2018_v36n6_483_f0030.png 이미지

Fig. 30. Seamline from images in strip 2 and mosaicked image of Case D

GCRHBD_2018_v36n6_483_f0031.png 이미지

Fig. 31. Seamline from image of strip 1 and 2, and mosaicked image of Case D

GCRHBD_2018_v36n6_483_f0032.png 이미지

Fig. 32. Seamline comparison and difference of mosaicked images

GCRHBD_2018_v36n6_483_f0033.png 이미지

Fig. 33. Comparison of seamlines

GCRHBD_2018_v36n6_483_f0034.png 이미지

Fig. 34. Seamline characteristics

GCRHBD_2018_v36n6_483_f0035.png 이미지

Fig. 35. Close-range terrestrial image mosaicking

Table 1. Test images

GCRHBD_2018_v36n6_483_t0001.png 이미지

References

  1. Anitha, J. and Deepa, S. (2014), Tracking and recognition of objects using SURF descriptor and Harris corner detection, International Journal of Current Engineering and Technology, Vol. 4, No. 2, pp. 775-778.
  2. Awrangjeb, M. and Lu, G. (2008), An improved curvature scale-space corner detector and a robust corner matching technique for transformed image identification, IEEE Transactions on Image Processing, Vol. 17, No. 12, pp. 2425-2441. https://doi.org/10.1109/TIP.2008.2006441
  3. Bay, H., Tuytelaars, T., and Van Gool, L. (2006), SURF: Speeded up robust features, In: Leonardis, A, Bischof, H., and Pinz, A. (eds.), Computer Vision: Proceedings 9th European Conference on Computer Vision Part I, Lecture Notes in Computer Science (LNCS 3951), Springer-Verlag, Berlin, Heidelberg, pp. 404-417.
  4. Brown, M., Szeliski, R., and Winder, S. (2004), Multi-Image Matching Using Multi-scale Oriented Patches, Technical Report MSR-TR-2004-133, Microsoft Corporation, Redmond, WA, 46p.
  5. Brown, M. and Lowe, D. (2007), Automatic panoramic image stitching using invariant features, International Journal of Computer Vision, Vol. 74, No. 1, pp. 59-73. https://doi.org/10.1007/s11263-006-0002-3
  6. Burt, P. and Adelson E. (1983), The Laplacian pyramid as a compact image code, IEEE transactions on Communications, Vol. 31, No. 4, pp. 532-540. https://doi.org/10.1109/TCOM.1983.1095851
  7. Canny, J. (1986), A computational approach to edge detection, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 8, pp. 679-698.
  8. Chen, Q., Sun, M., Hu, X., and Zhang, Z. (2014), Automatic seamline network generation for urban orthophoto mosaicking with the use of a digital surface model, Remote Sensing, Vol. 6, pp. 12334-12359. https://doi.org/10.3390/rs61212334
  9. Choi, J., Jung, H., and Yun, S. (2015), An efficient mosaic algorithm considering seasonal variation: Application to KOMPSAT-2 satellite images, Sensors, Vol. 15, pp. 5650-5665.
  10. Chon, J., Kim, H., and Lin, C. (2010), Seam-line determination for image mosaicking: A technique minimizing the maximum local mismatch and the global cost, ISPRS Journal of Photogrammetry and Remote Sensing, Vol. 65, pp. 86-92. https://doi.org/10.1016/j.isprsjprs.2009.09.001
  11. Dijkstra, E. (1959), A note on two problems in connection with graphs, Numerische Mathematik, Vol. 1, No.1, pp. 269-271. https://doi.org/10.1007/BF01386390
  12. Fischler, M. and Bolles, R. (1981), Random sample consensus: A paradigm for model fitting with application to image analysis and automated cartography, Communications of the ACM, Vol. 24, pp. 381-395. https://doi.org/10.1145/358669.358692
  13. Gerig, G., Kubler, O., Kikinis, R., and Jolesz, F. (1992), Nonlinear anisotropic filtering of MRI data, IEEE Transactions on Medical Imaging, Vol. 11, No. 2, pp. 221-232. https://doi.org/10.1109/42.141646
  14. Horn, B. (1986), Robot Vision, MIT Press, Cambridge, MA, 509p.
  15. Jung, H., Lim, S., and Lee, D.C. (2008), Utilizing airborne LiDAR Data for building extraction and superstructure analysis for modeling, Journal of the Korean Society of Surveying, Geodesy, Photogrammetry and Cartography, Vol. 26, No. 3, pp. 227-239.
  16. Kim, D., Yoon, U., and Jo, G. (2016), Feature point filtering method based on CS-RANSAC for efficient planar homography estimating, Korea Information Processing Society Transactions on Software and Data Engineering, Vol. 5, No. 6 pp. 307-312.
  17. Kim, J. and Kim, D. (2014), Matching points filtering applied panorama image processing using SURF and RANSAC Algorithm, Journal of The Institute of Electronics and Information Engineers, Vol. 51, No. 4, pp. 820-834.
  18. Lee, D. (2018), Determination of High Level-of-Detail Model Key Points of Buildings Using Airborne LiDAR data, Master's thesis, Sejong University, Seoul, Korea, 138p.
  19. Lee, D. and Lee, D.C. (2016), Satellite image geocoding using image and map matching, Proceedings of Korean Society of Surveying, Geodesy, Photogrammetry, and Cartography, 28-29 April, Suwon, Korea, unpagenated CD-ROM. (in Korean with English abstract)
  20. Li, L., Yao, J., Liu, Y., Yuan, W., Shi, S., and Yuan, S. (2017), Optimal seamline detection for orthoimage mosaicking by combining deep convolutional neural network and graph cuts, Remote Sensing, Vol. 9, No. 7, pp. 1-22.
  21. Litvinov, A. and Schechner, Y. (2005), Radiometric framework for image mosaicking, Journal of the Optical Society of America, Vol. 22, No. 5, pp. 839-848. https://doi.org/10.1364/JOSAA.22.000839
  22. Lowe, D. (2004), Distinctive image features from scale-invariant key points, Computer Vision and Image Understanding, Vol. 20, pp. 91-110.
  23. Mills, S. and McLeod, P. (2013), Global seamline networks for orthomosaic generation via local search, ISPRS Journal of Photogrammetry and Remote Sensing, Vol. 75, No. 1, pp. 101-111. https://doi.org/10.1016/j.isprsjprs.2012.11.003
  24. Oshel, E., Stevenson, P., and Heipke, C. (1995), Experiments with anisotropic diffusion for adaptive image smoothing, In: Heipke, C. (ed.), From Digital Images to Spatial Objects, Report No. 433, Dept. of Geodetic Science and Surveying, The Ohio State University, Columbus, OH, pp. 67-85.
  25. Pan, J., Zhou, Q., and Wang, M. (2014), Seamline determination based on segmentation for urban image mosaicking, IEEE Geoscience and Remote Sensing Letters, Vol. 11, No. 8, pp. 1335-1339. https://doi.org/10.1109/LGRS.2013.2293197
  26. Perona, P. and Malik, J. (1990), Scale-space and edge detection using anisotropic diffusion, IEEE Transactions on Pattern Analysis And Machine Intelligence, Vol. 12. No. 7, pp. 629-639. https://doi.org/10.1109/34.56205
  27. Rosten, E. and Drummond, T. (2006), Machine learning for high speed corner detection, In: Leonardis, A, Bischof, H., and Pinz, A. (eds.), Computer Vision: Proceedings 9th European Conference on Computer Vision Part I, Lecture Notes in Computer Science (LNCS 3951), Springer-Verlag, Berlin, Heidelberg, pp. 430-443.
  28. Saito, S., Arai, R., and Aoki, Y. (2015), Seamline determination based on semantic segmentation for aerial image mosaicking, IEEE Access, Vol. 3, pp. 2847-2856. https://doi.org/10.1109/ACCESS.2015.2508921
  29. Sapiro, G. (2006), Geometric Partial Differential Equations and Image Analysis, Cambridge University Press, New York, NY, 385p.
  30. Song, N. (2005), Methodology of Seamline Extraction for Automatic Image Mosaic Generation, Master's thesis, Inha University, Incheon, Korea, 63p.
  31. Woo, H,, Noh, M., Park, J., Cho, W., and Kim, B. (2009), Color assessment for mosaic imagery using HSI model, Journal of the Korean Society of Surveying, Geodesy, Photogrammetry and Cartography, Vol. 27, No. 4, pp. 429-435.