Journal of Broadcast Engineering (방송공학회논문지)

The Korean Institute of Broadcast and Media Engineers (한국방송∙미디어공학회)
권호 표지
DOI QR코드

DOI QR Code

The Power Line Deflection Monitoring System using Panoramic Video Stitching and Deep Learning

딥 러닝과 파노라마 영상 스티칭 기법을 이용한 송전선 늘어짐 모니터링 시스템

  • Park, Eun-Soo (Department of Computer Education, Sungkyunkwan University) ;
  • Kim, Seunghwan (Department of Computer Education, Sungkyunkwan University) ;
  • Lee, Sangsoon (Department of Computer Engineering, Gachon University) ;
  • Ryu, Eun-Seok (Department of Computer Education, Sungkyunkwan University)
  • 박은수 (성균관대학교 컴퓨터교육과) ;
  • 김승환 (성균관대학교 컴퓨터교육과) ;
  • 이상순 (가천대학교 컴퓨터공학과) ;
  • 류은석 (성균관대학교 컴퓨터교육과)
  • Received : 2019.07.03
  • Accepted : 2020.01.20
  • Published : 2020.01.30
Download PDF
⟨ Previous Next ⟩

Abstract

There are about nine million power line poles and 1.3 million kilometers of the power line for electric power distribution in Korea. Maintenance of such a large number of electric power facilities requires a lot of manpower and time. Recently, various fault diagnosis techniques using artificial intelligence have been studied. Therefore, in this paper, proposes a power line deflection detect system using artificial intelligence and computer vision technology in images taken by vision system. The proposed system proceeds as follows. (i) Detection of transmission tower using object detection system (ii) Histogram equalization technique to solve the degradation in image quality problem of video data (iii) In general, since the distance between two transmission towers is long, a panoramic video stitching process is performed to grasp the entire power line (iv) Detecting deflection using computer vision technology after applying power line detection algorithm This paper explain and experiment about each process.

한국에는 전력 분배를 위하여 약 9백만 개의 전신주와 1.3백만 킬로미터의 송전선이 있다. 이러한 많은 전력 설비의 유지보수를 위해서는 많은 인력과 시간이 소요된다. 최근 인공지능을 사용한 여러 고장진단 기술들이 연구되어 오고 있기 때문에 본 논문에서는 송전선의 여러 요인으로 인한 늘어짐을 감지하기 위해 기존의 현장에서의 검증 방법이 아닌 카메라 시스템으로 촬영한 영상에서의 인공 지능 기술을 활용한 송전선 늘어짐 감지 시스템을 제안한다. 제안하는 시스템은 (i) 객체 탐지 시스템을 이용한 송전탑 감지 (ii) 동영상 촬영 데이터의 화질 저하 문제를 해결하기 위한 히스토그램 평활화 기법 (iii) 송전선 전체를 파악하기 위한 파노라마 영상 스티칭(iv) 송전선 탐지 알고리즘 적용 후 파노라마 영상 스티칭 기술을 이용한 늘어짐 판단 과정으로 진행된다. 본 논문에서는 각각의 과정들에 대한 설명 및 실험 결과를 보인다.

Keywords

References

  1. H. T. Lim and S. Kim, "The Socio-technical Constituency behind New & Renewable Energy Technology Development in a Latecomer: The Case study of New & Renewable Technology Program of Korea," Journal of Energy Engineering, Vol.20, No.4, pp.267-277, 2011 https://doi.org/10.5855/ENERGY.2011.20.4.267
  2. S. Y. Hyun, M. H. Choi, S. H. Bae, and J. S. Ryoo, "A study on the Interconnect Protection of Distributed Generators between Distribution System," The Korean Institute of Electrical Engineers (KIEE) pp.551-552, 2015.
  3. No. 86 (2016) Korea Electric Power Statistics, Korea Electric Power Corporation, 100-101, 2017
  4. E. S. Cho and K. Kim, "The power line tracking system using image processing algorithm," Institute of Control, Robotics and Systems, pp.739-744, 2011.
  5. A. K. Jardine, D. Lin and D. Banjevic, "A review on machinery diagnostics and prognostics implementing condition-based maintenance," Mechanical Systems and Signal Processing , Vol.20, No.7, pp.1483-1510, 2006 https://doi.org/10.1016/j.ymssp.2005.09.012
  6. S. Kim and S. Lee, "Deep Learning in Mechanical Engineering," The Korean Society of Mechanical Engineers, pp.103-104, 2017.
  7. B. A. Paya, I. I. Esat, and M. N. M. Badi, "ARTIFICIAL NEURAL NETWORK BASED FAULT DIAGNOSTICS OF ROTATING MACHINERY USING WAVELET TRANSFORMS AS A PREPROCESSOR," Mechanical Systems and Signal Processing, Vol.11, No.5, pp. 751-765, 1997. https://doi.org/10.1006/mssp.1997.0090
  8. T. M. Khoshgoftaar and D. L. Lanning, "A Neural Network Approach for Early Detection of Program Modules Having High Risk in the Maintenance Phase," Journal of Systems and Software, Vol.29, pp.85-91, 1995. https://doi.org/10.1016/0164-1212(94)00130-F
  9. E. Park, S. Kim, J. Jeong and E-S Ryu, "Overview of AI-based Fault Detection and Diagnostics," The Korean Institute of Broadcast and Media Engineers, pp.235-237, 2018
  10. J. Redmon, S. Divvala, R. Girshick and A. Farhadi " You Only Look Once: Unified, Real-Time Object Detection," The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp.779-788, 2016
  11. D. Kim and J. Choi, "View Interpolation Algorithm, for Continuously Changing Viewpoints in the Multi-panorama Based Navigation," IEIE Journal (SP), Vol.40, pp.141-148, 2003
  12. S. Kim, K. Kim and W. Woo, "Multiple Camera Calibration for Panoramic 3D Virtual Environment," IEIE Journal (CI), Vol.41, pp.137-148, 2004
  13. C. Xie, X. Zhang, H. Yang, and Z. Gao, "Video Stitching Based on Optical Flow," IEEE International Symposium on Broadband Multimedia Systems and Broadcasting (BMSB), pp.1-5, 2018
  14. M. T. Ibrahim, R. Hafiz, M. M. Khan, Y. Cho and J. Cha, "Automatic Reference Selection for Parametric Color Correction Schemes for Panoramic Video Stitching," Advances in Visual Computing. ISVC, Vol.7431, pp.492-501, 2012
  15. H. Guo, S. Liu, T. He, S. Zhu, B. Zeng and M. Gabbouj, "Joint Video Stitching and Stabilization From Moving Cameras," IEEE Transactions on Image Processing, Vol.25, 5491-5503, 2016 https://doi.org/10.1109/TIP.2016.2607419
  16. E. Saban, I. Mostafa, K. Ayman and R. Mahmoud, "Improved optimal seam selection blending for fast video stitching of videos captured from freely moving devices," Proceedings - International Conference on Image Processing, ICIP, pp.1481-1484, 2011
  17. W. Xu, "Panoramic Video Stitching," PhD thesis, University of Colorado, Boulder Boulder, 1-1-2012.
  18. D. G. Lowe, "Distinctive Image Features from Scale-Invariant Keypoints," International Journal of Computer Vision, Vol.60, pp.91-110, 2004 https://doi.org/10.1023/B:VISI.0000029664.99615.94
  19. H. Bay, T. Tuytelaars and L. V. Gool, "SURF: Speeded Up Robust Features. Computer Vision and Image Understanding,", Computer Vision and Image Understanding, Vol.110, pp.346-359, 2008 https://doi.org/10.1016/j.cviu.2007.09.014
  20. Y-T. Kim, "Contrast enhancement using brightness preserving bi-histogram equalization," IEEE Transactions on Consumer Electronics, Vol.43, pp.1-8, 1997 https://doi.org/10.1109/30.580378
  21. S. Chen and A. R. Ramli, "Contrast Enhancement using Recursive Mean-Separate Histogram Equalization for Scalable Brightness Preservation," IEEE Transactions on Consumer Electronics, Vol.49, pp.1301-1309, 2003 https://doi.org/10.1109/TCE.2003.1261233
  22. Y. Wang, Q. Chen and B. M. Zhang, "Image Enhancement based on Equal Area Dualistic sub-Image Histogram Equalization Method," IEEE Transaction on Consumer Electronics, Vol.45, pp.68-75, 1999 https://doi.org/10.1109/30.754419
  23. K. S. Sim, C. P. Tso and Y. Y. Tan, "Recursive sub-image histogram equalization applied to gray scale images," Pattern Recognition Letters, Vol.28, pp.1209-1221, 2007 https://doi.org/10.1016/j.patrec.2007.02.003
  24. R. C. Gonzalez and P. Wintz, "Digital Image Processing," Addison-Wesley Publishing Company 2009.
  25. J. Y. Kim, L. S. Kim and S. H. Hwang, "An Advanced Contrast Enhancement using Partially Overlapped Sub-Block Histogram Equalization," IEEE Circuits and Systems for Video Technology, Vol.11, pp.475-484, 2001 https://doi.org/10.1109/76.915354
  26. K. Zuiderveld, "Contrast Limited Adaptive Histogram Equalization. Graphics Gems IV," Academic Press Professional, Inc., pp.474-485, 1994
  27. C. H. Park, K. Choi and I. Lee, "Lane Extraction through UAV Mapping and Its Accuracy Assessment," Journal of the Korean Society of Surveying, Geodesy, Photogrammetry and Cartography, Vol.34, pp.11-19, 2016 https://doi.org/10.7848/ksgpc.2016.34.1.11
  28. J. S. Lee, H. S. Kim and J. B. Park, "Multi-lane Detection and Driving Lane Information Extraction Algorithm Using Inverse Perspective Mapping," The Korean Institute of Electrical Engineers (KIEE), pp.257-258, 2016
  29. S. H. Park and Y. G. Kim, "A Study of Detecting Curved Lane by Hough Transform for Autonomous Driving," Korean Institute of Information Scientists and Engineers, pp.2104-2106, 2017
  30. J. M. Choi and C. Kim, "Interval Hough Transform for Prominent Line Detection" Journal of Korea Multimedia Society, Vol.16, pp.1288-1296, 2013 https://doi.org/10.9717/kmms.2013.16.11.1288
  31. W. G. Jeon and B. G. Choi, "A Study on the Automatic Detection of Railroad Power Lines Using LiDAR Data and RANSAC Algorithm," Journal of the Korean Society of Surveying, Geodesy, Photogrammetry and Cartography, Vol.31, pp.331-339, 2013 https://doi.org/10.7848/ksgpc.2013.31.4.331
  32. YOLO: Real-Time Object Detection. https://pjreddie.com/darknet/yolo/ (accessed Mar. 31, 2019).
  33. Y. H. Lee, J. H. Park and Y. Kim, "Comparative Analysis of the Performance of SIFT and SURF," Journal of the Semiconductor & Display Technology, Vol.12, pp.59-63, 2013
  34. M. Brown and D. G. Lowe, "Automatic Panoramic Image Stitching using Invariant Features," International Journal of Computer Vision, Vol.74, pp.59-73, 2007 https://doi.org/10.1007/s11263-006-0002-3
  35. R. Shaoging, H. Kaiming, G. Ross and S. Jian, "Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks", Advances in Neural Information Processing Systems (NIPS), Vol. 28, 2015
  36. VaFRIC (Variable Frame-Rate Imperial College) Dataset, https://www.doc.ic.ac.uk/-ahanda/VaFRIC/index.html (accessed Jul. 1, 2019)