Browse > Article
http://dx.doi.org/10.3837/tiis.2019.10.015

Manhole Cover Detection from Natural Scene Based on Imaging Environment Perception  

Liu, Haoting (Beijing Engineering Research Center of Industrial Spectrum Imaging, School of Automation and Electrical Engineering, University of Science and Technology Beijing)
Yan, Beibei (Department of R&D, Beijing Institute of Aerospace Control Device)
Wang, Wei (Department of R&D, Beijing Institute of Aerospace Control Device)
Li, Xin (Jiuquan Satellite Launch Center)
Guo, Zhenhui (Jiuquan Satellite Launch Center)
Publication Information
KSII Transactions on Internet and Information Systems (TIIS) / v.13, no.10, 2019 , pp. 5095-5111 More about this Journal
Abstract
A multi-rotor Unmanned Aerial Vehicle (UAV) system is developed to solve the manhole cover detection problem for the infrastructure maintenance in the suburbs of big city. The visible light sensor is employed to collect the ground image data and a series of image processing and machine learning methods are used to detect the manhole cover. First, the image enhancement technique is employed to improve the imaging effect of visible light camera. An imaging environment perception method is used to increase the computation robustness: the blind Image Quality Evaluation Metrics (IQEMs) are used to percept the imaging environment and select the images which have a high imaging definition for the following computation. Because of its excellent processing effect the adaptive Multiple Scale Retinex (MSR) is used to enhance the imaging quality. Second, the Single Shot multi-box Detector (SSD) method is utilized to identify the manhole cover for its stable processing effect. Third, the spatial coordinate of manhole cover is also estimated from the ground image. The practical applications have verified the outdoor environment adaptability of proposed algorithm and the target detection correctness of proposed system. The detection accuracy can reach 99% and the positioning accuracy is about 0.7 meters.
Keywords
Manhole cover detection; UAV; environment perception; image enhancement; deep learning;
Citations & Related Records
Times Cited By KSCI : 3  (Citation Analysis)
연도 인용수 순위
1 H. Liu, H. Lu, and Y. Zhang, "Image enhancement for outdoor long-range surveillance using IQ-learning multiscale Retinex," IET Image Processing, vol. 11, no. 9, pp. 786-795, September, 2017.   DOI
2 L. Wang, X. Yao, Z. Meng, T. Liu, Z. Li, B. Shi, Y. Su, R. Zhang, and W. Liu, "An optical coherence tomography attenuation compensation algorithm based on adaptive multi-scale Retinex," Chinese Journal of Lasers, vol. 40, no. 12, pp. 1204001-1 - 1204001-6, December, 2013.   DOI
3 A. Carrio, C. Sampedro, A. Rodriguez-Ramos, and P. Campoy, "A review of deep learning methods and applications for unmanned aerial vehicles," Journal of Sensors, vol. 2017, pp. 3296874-1 - 3296874-13, August, 2017.
4 W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C.-Y. Fu, and A. C. Berg, "SSD: Single Shot MultiBox Detector," in Proc. of European Conference on Computer Vision, pp. 21-37, December, 2016.
5 Z. Wu, L. Huang, D. Hu, and C. Ding, "Ground resolution analysis based on gradient method in geosynchronous SAR," in Proc. of IEEE International Conference on Signal Processing, Communication and Computing, pp. 1-4, August 5-8, 2013.
6 H. Liu, W.Wang, F. Gao, Z. Liu, Y. Sun, and Z. Liu, "Development of space photographic robotic arm based on binocular vision servo," in Proc. of International Conference on Advanced Computational Intelligence, pp. 345-349, October 19-21, 2013.
7 C. Ramirez-Atencia, V. Rodriguez-Fernandez, A. Gonzalez-Pardo, D. Camacho, "New artificial intelligence approaches for future UAV ground control stations," in Proc. of IEEE Congress on Evolutionary Computation, pp. 2775-2782, June 5-8, 2017.
8 T. Jiang, J. Li, B. Li, K. Huang, C. Yang, and Y. Jiang, "Trajectory optimization for a cruising unmanned aerial vehicle attacking a target at back slope while subjected to a wind gradient," Mathematical Problems in Engineering, vol. 2015, pp. 635395-1 - 635395-14, June, 2015.
9 Z. Zhang, and K. Li, "Study on algorithm for panoramic image basing on high sensitivity and high resolution panoramic surveillance camera," in Proc. of IEEE International Conference on Advanced Video and Signal Based Surveillance, pp. 359-364, August 27-30, 2013.
10 A. K. Moorthy, and A. C. Bovik, "Blind image quality assessment: from natural scene statistics to perceptual quality," IEEE Transactions on Image Processing, vol. 20, no. 12, pp. 3350-3364, December, 2011.   DOI
11 H. Liu, W. Wang, Z. He, Q. Tong, X. Wang, W. Yu, and M. Lv, "Blind image quality evaluation metrics design for UAV photographic application," in Proc. of IEEE International Conference on Cyber Technology in Automation, Control, and Intelligent Systems, pp. 293-297, June 8-12, 2015.
12 S. Jaiswal, and M. Valstar, "Deep learning the dynamic appearance and shape of facial action units," in Proc. of IEEE Winter Conference on Applications of Computer Vision, pp. 1-8, March 7-10, 2016.
13 S. Bianco, M. Buzzelli, D. Mzaaini, and R. Schettini, "Deep learning for logo recognition," Neurocomputing, vol. 245, pp. 23-30, July, 2017.   DOI
14 J. Chang, H. Jiang, Z. Weng, X. Cong, and Y. Jin, "Design of wide angle space optical systems of long focal length," Acta Armamentarii, vol. 24, no. 1, pp. 42-44, January, 2003.   DOI
15 H. Liu, C. Wang, H. Lu, and W. Yang, "Outdoor camera calibration method for a GPS & PTZ camera based surveillance system," in Proc. of IEEE International Conference on Industrial Technology, pp.263-267, March 14-17, 2010.
16 D. Mery, E. Svec, M. Arias, V. Riffo, J. M. Saavedra, and S. Banerjee, "Modern computer vision techniques for X-ray testing in baggage inspection," IEEE Transactions on Systems, Man, and Cybernetics: Systems, vol. 47, no. 4, pp. 682-692, April, 2017.   DOI
17 K. Shi, H. Bao, and N. Ma, "Forward vehicle detection based on incremental learning and fast R-CNN," in Proc. of International Conference on Computational Intelligence and Security, pp. 73-76, December 15-18, 2017.
18 Y. Wang, and J. Zheng, "Real-time face detection based on YOLO," in Proc. of IEEE International Conference on Knowledge Innovation and Invention, pp. 221-224, July 23-27, 2018.
19 P. Dong, and W. Wang, "Better region proposals for pedestrian detection with R-CNN," in Proc. of Visual Communications and Image Processing, pp. 1-4, November 27-30, 2016.
20 R. U. Khan, X. Zhang, R. Kumar, and H. A. Tariq, "Analysis of resnet model for malicious code detection," in Proc. of International Computer Conference on Wavelet Active Media Technology and Information Processing, pp. 239-242, December 15-17, 2017.
21 B. Liu, W. Zhao, and Q. Sun, "Study of object detection based on faster R-CNN," in Proc. of Chinese Automation Congress, pp. 6233-6236, October 20-22, 2017.
22 F. Ye, C. Chen, Y. Lai, and J. Chen, "Fast circle detection algorithm using sequenced Hough transform," Optics and Precision Engineering, vol. 22, no. 4, pp. 1104-1111, April, 2014.   DOI
23 W. Sultani, S. Mokhtari, and H.-B. Yun, "Automatic pavement object detection using superpixel segmentation combined with conditional random field," IEEE Transactions on Intelligent Transaction Systems, vol. 19, no. 7, pp. 2076-2085, July, 2018.   DOI
24 Y. Yu, J. Li, H. Guan, C. Wang, and J. Yu, "Automated detection of road manhole and sewer well covers from mobile LiDAR point clouds," IEEE Geoscience and Remote Sensing Letters, vol. 11, no. 9, pp. 1549-1553, September, 2014.   DOI
25 V. Ghase, A. A. Pouyan, and M. Sharifi, "Human acitivity recognition in smart homes based on a difference of convex programming problem," KSII Transactions on Internet and Information Systems, vol. 11, no. 1, pp. 321-344, January, 2017.   DOI
26 G. Abdelbacet, Z. Ghada, S.Mounir, and K. Abdennaceur, "A real time environmental monitoring for smart city surveillance based GUI on Android platform," in Proc. of IEEE International Multi-Conference on Systems, Signals & Devices, pp. 1-6, March 16-19, 2015.
27 L. Yin, C. Liu, X. Lu, J. Chen, and C. Liu, "Efficient compression algorithm with limited resource for continuous surveillance," KSII Transactions on Internet and Information Systems, vol. 10, no. 11, pp. 5476-5496, November, 2016.   DOI
28 H. Wang, N. Huo, J. Li, K. Wang, and Z. Wang, "A road quality detection method based on the Mahalanobis-Taguchi system," IEEE Access, vol. 6, pp. 29078-29087, May, 2018.   DOI
29 X. Chen, J. Xu, and W. Guo, "The research about video surveillance platform based on cloud computing," in Proc. of International Conference on Machine Learning and Cybernetics, pp. 979-983, July 14-17, 2013.
30 N. D. Hoang, "Detection of surface crack in building structures using image processing technique with an improve Otsu method for image thresholding," Advanced in Civil Engineering, vol. 2018, pp. 3924120-1 - 3924120-10, April, 2018.
31 H. Cui, J. Liu, and G. Su, "Combined static and dynamic platform calibration for an aerial multi-camera system," KSII Transactions on Internet and Information Systems, vol. 10, no. 6, pp. 2689-2708, June, 2016.   DOI
32 Y. Yu, H. Guan, and Z. Ji, "Automated detection of urban road manhole covers using mobile laser scanning data," IEEE Transactions on Intelligent Transportation Systems, vol. 16, no. 6, pp. 3258-3269, December, 2015.   DOI
33 G. Jia, G. Han, H. Rao, and L. Shu, "Edge computing-based intelligent manhole cover management system for smart cities," IEEE Internet of Things Journal, vol. 5, no. 3, pp. 1648-1656, June, 2018.   DOI
34 A. Giyenko, and Y. I. Cho, "Intelligent UAV in smart cities using IoT," in Proc. of International Conference on Control, Automation and Systems, pp. 207-210, October 16-19, 2016.
35 G. Zhang, L. Wang, Z. Zheng, Y. Chen, Z. Zhou, and K. Zhao, "No-reference aerial image quality assessment based on natural scene statisticsand color correlation blur metric," in Proc. of IEEE PES International Conference on Transmission & Distribution Construction, Operation & Live-Line Maintenance, pp. 1-4, September 12-15, 2016.
36 X. Zhao, Q. Fei, and Q. Geng, "Vision based ground target tracking for rotor UAV," in Proc. of IEEE International Conference on Control and Automation, pp. 1907-1911, June 12-14, 2013.
37 C. Kim, D. S. Han, J. K. Kim, and B. I. Kim, "Automatic detection of defective welding electrode tips using color segmentation and Hough circle detection," in Proc. of IEEE Region 10 Conference, pp. 1371-1374, November 22-25, 2016.